Skip to content

Do we need --specified-platforms-only in stubtest_third_party? #13641

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
srittau opened this issue Mar 17, 2025 · 2 comments · May be fixed by #13643
Open

Do we need --specified-platforms-only in stubtest_third_party? #13641

srittau opened this issue Mar 17, 2025 · 2 comments · May be fixed by #13643
Labels
project: infrastructure typeshed build, test, documentation, or distribution related

Comments

@srittau
Copy link
Collaborator

srittau commented Mar 17, 2025

When this option is specified (which it is when running in CI), stubtest won't run when a platform is not specified in METADATA.toml. The test is marked as "skipped". When it's not specified (the default), stubtest will run and will most likely case an error.

I noticed this when running stubtest_third_party.py locally on my Linux system and it errored on pywin32.

I suggest we either:

  1. Remove that option altogether and always skip stubtest for stubs not specified in METADATA.toml; or
  2. reverse the logic so that those tests are skipped by default and only generate errors when an explicit option is given.
@srittau srittau added the project: infrastructure typeshed build, test, documentation, or distribution related label Mar 17, 2025
@AlexWaygood
Copy link
Member

Added in #9173; it may be useful to review the discussion in that PR thread

@srittau
Copy link
Collaborator Author

srittau commented Mar 17, 2025

The original comment by @Avasam was in #8923 (comment):

@sobolevn & @AlexWaygood There's an unforseen issue: Now I can't run stubtest locally on any stub that doesn't explicitely specify my non-linux OS!

If no platform is specified in METADATA.toml, it should be allowed to be run on any OS, but the CI should only run linux.

Which is of course true. I think I slightly misunderstood the purpose of the flag. So another solution:

  1. Always skip running stubtest when the current platform is not specified in tool.stubtest.platforms, if it exists.

srittau added a commit to srittau/typeshed that referenced this issue Mar 17, 2025
@srittau
Fix stubtest_third_party.py on linux
b7e1f7a 
Previously, when running it on Linux without the `--specified-platforms-only`
flag, it printed spurious warnings and ran stubtest anyway for distributions
that didn't include `linux` as a supported platform.

Rearrange things:

* The `--specified-platforms-only` now works consistently on all platforms.
  Previously, it was ignored on Linux.
* Instead, in CI the flag is added only for non-Linux platforms.
* The note about stubtest not running for a certain platform in CI is now
  printed only after stubtest tested a distribution successfully to avoid
  breaking the output.

Closes: python#13641
@srittau srittau linked a pull request Mar 17, 2025 that will close this issue
Draft
srittau added a commit to srittau/typeshed that referenced this issue Mar 31, 2025
@srittau
Fix stubtest_third_party.py on linux
e7c5593 
Previously, when running it on Linux without the `--specified-platforms-only`
flag, it printed spurious warnings and ran stubtest anyway for distributions
that didn't include `linux` as a supported platform.

Rearrange things:

* The `--specified-platforms-only` now works consistently on all platforms.
  Previously, it was ignored on Linux.
* Instead, in CI the flag is added only for non-Linux platforms.
* The note about stubtest not running for a certain platform in CI is now
  printed only after stubtest tested a distribution successfully to avoid
  breaking the output.

Closes: python#13641
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
project: infrastructure typeshed build, test, documentation, or distribution related
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix stubtest_third_party.py on linux srittau/typeshed
2 participants
@srittau @AlexWaygood