-
-
Notifications
You must be signed in to change notification settings - Fork 2.9k
Use checkmember.py to check protocol subtyping #18943
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
This comment has been minimized.
This comment has been minimized.
OK, again as expected, it looks like the biggest problem here is performance. But also it looks very heterogeneous. It looks like unit tests, as well as most of the mypy_primer batches are roughly the same performance. But couple batches are much slower. I remember mypy_primer used to show the performance difference, @hauntsaninja is there a way to put that feature back? Even if it is noisy, it would be helpful for this PR. In the meantime, @sterliakov could you please run your magic "found fixed issues" tool on this PR? |
Sure, there you go! Results are available at sterliakov/mypy-issues#11 (won't copy here to avoid triggering a bunch of PR-issue links) Ignore one FP with topic-incremental, that's a caching issue I haven't resolved yet, but the rest should be representative. Quite a few good changes! BTW, you can also just create an issue with PR number in title and wait for results:) |
This comment has been minimized.
This comment has been minimized.
IMO your current implementation is reasonable - consistent behavior of typevars during specialization is more important. Given that we accept
If (in other words, |
In addition to pushing a commit re-enabling the mypy_primer timings, I ran some benchmarking on mypy_primer corpus myself. I think this PR does make mypy single digit percentage slower for the median project, with some projects seeing 10-20% slowdown. I think part of what you're seeing with the batching is unrelated to this PR though. Before yesterday, for dumb reasons, we weren't really checking https://github.com/colour-science/colour properly in primer, and that project is exceptionally slow for us now (possible there was some other regression). That said, colour-science also happens to be the project most adversely affected by this PR on performance (and most positively impacted in type checking) — the 30% slowdown in Github Actions matches what I measure. |
Diff from mypy_primer, showing the effect of this PR on open source code: pyinstrument (https://github.com/joerick/pyinstrument)
- pyinstrument/vendor/decorator.py:295: error: Incompatible types in assignment (expression has type "Callable[[Any, Any, VarArg(Any), KwArg(Any)], Any]", variable has type "Callable[[_GeneratorContextManagerBase[_G_co], Callable[..., _G_co], tuple[Any, ...], dict[str, Any]], None]") [assignment]
+ pyinstrument/vendor/decorator.py:295: error: Incompatible types in assignment (expression has type "Callable[[Any, Any, VarArg(Any), KwArg(Any)], Any]", variable has type "Callable[[_GeneratorContextManagerBase[Generator[Any, None, None]], Callable[..., Generator[Any, None, None]], tuple[Any, ...], dict[str, Any]], None]") [assignment]
- pyinstrument/vendor/decorator.py:301: error: Incompatible types in assignment (expression has type "Callable[[Any, Any, VarArg(Any), KwArg(Any)], Any]", variable has type "Callable[[_GeneratorContextManagerBase[_G_co], Callable[..., _G_co], tuple[Any, ...], dict[str, Any]], None]") [assignment]
+ pyinstrument/vendor/decorator.py:301: error: Incompatible types in assignment (expression has type "Callable[[Any, Any, VarArg(Any), KwArg(Any)], Any]", variable has type "Callable[[_GeneratorContextManagerBase[Generator[Any, None, None]], Callable[..., Generator[Any, None, None]], tuple[Any, ...], dict[str, Any]], None]") [assignment]
static-frame (https://github.com/static-frame/static-frame): 1.11x slower (283.3s -> 313.2s in single noisy sample)
optuna (https://github.com/optuna/optuna): 1.09x slower (246.6s -> 269.7s in single noisy sample)
hydpy (https://github.com/hydpy-dev/hydpy)
- hydpy/core/itemtools.py:951: error: Argument 2 to "update_variable" of "ChangeItem" has incompatible type "float64"; expected "ndarray[tuple[int, ...], dtype[float64]]" [arg-type]
- hydpy/core/itemtools.py:954: error: Argument 2 to "update_variable" of "ChangeItem" has incompatible type "float64"; expected "ndarray[tuple[int, ...], dtype[float64]]" [arg-type]
- hydpy/auxs/ppolytools.py:253: error: Value of type variable "_AnyShapeType" of "__call__" of "_ConstructorEmpty" cannot be "tuple[int, signedinteger[_64Bit]]" [type-var]
- hydpy/auxs/ppolytools.py:253: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "signedinteger[_64Bit]" [type-var]
- hydpy/auxs/ppolytools.py:256: error: Incompatible types in assignment (expression has type "ndarray[tuple[int, signedinteger[_64Bit]], dtype[float64]]", variable has type "Sequence[Sequence[float] | ndarray[tuple[int, ...], dtype[float64]]] | ndarray[tuple[int, ...], dtype[float64]]") [assignment]
- hydpy/auxs/ppolytools.py:661: error: Value of type "float64" is not indexable [index]
steam.py (https://github.com/Gobot1234/steam.py)
- steam/user.py:506: error: Argument "type" to "__init__" of "ID" has incompatible type "int"; expected "TypeT | None" [arg-type]
- steam/message.py:74: error: Argument 1 to "__init__" of "Message" has incompatible type "Self"; expected "Message[UserT, ChannelT]" [arg-type]
scipy-stubs (https://github.com/scipy/scipy-stubs): 1.09x slower (280.2s -> 305.3s in single noisy sample)
cwltool (https://github.com/common-workflow-language/cwltool): 1.07x slower (165.1s -> 176.0s in single noisy sample)
pandas (https://github.com/pandas-dev/pandas): 1.07x slower (866.4s -> 927.6s in single noisy sample)
+ pandas/core/array_algos/quantile.py:199: error: Unused "type: ignore" comment [unused-ignore]
+ pandas/core/sorting.py:479: error: Unused "type: ignore" comment [unused-ignore]
+ pandas/io/parsers/python_parser.py:1471: error: Unused "type: ignore" comment [unused-ignore]
- pandas/core/internals/managers.py:972: error: Incompatible types in assignment (expression has type "signedinteger[_32Bit | _64Bit]", variable has type "int") [assignment]
+ pandas/core/internals/construction.py:637: error: Unused "type: ignore" comment [unused-ignore]
+ pandas/core/internals/blocks.py:2101: error: Unused "type: ignore" comment [unused-ignore]
rclip (https://github.com/yurijmikhalevich/rclip)
- rclip/model.py:218: error: Argument "key" to "sorted" has incompatible type "Callable[[tuple[float64, int]], float64]"; expected "Callable[[tuple[float64, int]], SupportsDunderLT[Any] | SupportsDunderGT[Any]]" [arg-type]
- rclip/model.py:218: error: Incompatible return value type (got "float64", expected "SupportsDunderLT[Any] | SupportsDunderGT[Any]") [return-value]
- rclip/model.py:220: error: Incompatible return value type (got "list[tuple[float64, int]]", expected "list[tuple[float, int]]") [return-value]
- rclip/model.py:220: note: "list" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance
- rclip/model.py:220: note: Consider using "Sequence" instead, which is covariant
- rclip/model.py:220: note: Perhaps you need a type annotation for "sorted_similarities"? Suggestion: "list[tuple[float, int]]"
xarray (https://github.com/pydata/xarray): 1.11x slower (461.5s -> 513.2s in single noisy sample)
colour (https://github.com/colour-science/colour): 1.30x slower (1352.9s -> 1752.9s in single noisy sample)
- colour/algebra/interpolation.py:692: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/algebra/interpolation.py:693: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/algebra/interpolation.py:700: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/io/luts/lut.py:1544: error: Value of type "floating[_16Bit] | floating[_32Bit] | float64" is not indexable [index]
- colour/colorimetry/spectrum.py:1611: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/colorimetry/generation.py:762: error: Argument 1 to "sd_single_led_Ohno2005" has incompatible type "floating[_16Bit] | floating[_32Bit] | float64"; expected "float" [arg-type]
- colour/plotting/volume.py:261: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/plotting/volume.py:268: error: Incompatible types in assignment (expression has type "int | floating[_16Bit] | floating[_32Bit] | float64", variable has type "floating[_16Bit] | floating[_32Bit] | float64") [assignment]
- colour/plotting/colorimetry.py:195: error: No overload variant of "max" matches argument types "floating[_16Bit] | floating[_32Bit] | float64", "floating[_16Bit] | floating[_32Bit] | float64" [call-overload]
- colour/plotting/colorimetry.py:195: note: Possible overload variants:
- colour/plotting/colorimetry.py:195: note: def [SupportsRichComparisonT: SupportsDunderLT[Any] | SupportsDunderGT[Any]] max(SupportsRichComparisonT, SupportsRichComparisonT, /, *_args: SupportsRichComparisonT, key: None = ...) -> SupportsRichComparisonT
- colour/plotting/colorimetry.py:195: note: def [_T] max(_T, _T, /, *_args: _T, key: Callable[[_T], SupportsDunderLT[Any] | SupportsDunderGT[Any]]) -> _T
- colour/plotting/colorimetry.py:195: note: def [SupportsRichComparisonT: SupportsDunderLT[Any] | SupportsDunderGT[Any]] max(Iterable[SupportsRichComparisonT], /, *, key: None = ...) -> SupportsRichComparisonT
- colour/plotting/colorimetry.py:195: note: def [_T] max(Iterable[_T], /, *, key: Callable[[_T], SupportsDunderLT[Any] | SupportsDunderGT[Any]]) -> _T
- colour/plotting/colorimetry.py:195: note: def [SupportsRichComparisonT: SupportsDunderLT[Any] | SupportsDunderGT[Any], _T] max(Iterable[SupportsRichComparisonT], /, *, key: None = ..., default: _T) -> SupportsRichComparisonT | _T
- colour/plotting/colorimetry.py:195: note: def [_T1, _T2] max(Iterable[_T1], /, *, key: Callable[[_T1], SupportsDunderLT[Any] | SupportsDunderGT[Any]], default: _T2) -> _T1 | _T2
- colour/plotting/colorimetry.py:195: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/plotting/colorimetry.py:196: error: No overload variant of "min" matches argument types "floating[_16Bit] | floating[_32Bit] | float64", "floating[_16Bit] | floating[_32Bit] | float64" [call-overload]
- colour/plotting/colorimetry.py:196: note: Possible overload variants:
- colour/plotting/colorimetry.py:196: note: def [SupportsRichComparisonT: SupportsDunderLT[Any] | SupportsDunderGT[Any]] min(SupportsRichComparisonT, SupportsRichComparisonT, /, *_args: SupportsRichComparisonT, key: None = ...) -> SupportsRichComparisonT
- colour/plotting/colorimetry.py:196: note: def [_T] min(_T, _T, /, *_args: _T, key: Callable[[_T], SupportsDunderLT[Any] | SupportsDunderGT[Any]]) -> _T
- colour/plotting/colorimetry.py:196: note: def [SupportsRichComparisonT: SupportsDunderLT[Any] | SupportsDunderGT[Any]] min(Iterable[SupportsRichComparisonT], /, *, key: None = ...) -> SupportsRichComparisonT
- colour/plotting/colorimetry.py:196: note: def [_T] min(Iterable[_T], /, *, key: Callable[[_T], SupportsDunderLT[Any] | SupportsDunderGT[Any]]) -> _T
- colour/plotting/colorimetry.py:196: note: def [SupportsRichComparisonT: SupportsDunderLT[Any] | SupportsDunderGT[Any], _T] min(Iterable[SupportsRichComparisonT], /, *, key: None = ..., default: _T) -> SupportsRichComparisonT | _T
- colour/plotting/colorimetry.py:196: note: def [_T1, _T2] min(Iterable[_T1], /, *, key: Callable[[_T1], SupportsDunderLT[Any] | SupportsDunderGT[Any]], default: _T2) -> _T1 | _T2
- colour/plotting/colorimetry.py:196: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/plotting/colorimetry.py:223: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/plotting/colorimetry.py:223: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/plotting/colorimetry.py:224: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/plotting/colorimetry.py:229: error: List item 0 has incompatible type "tuple[floating[_16Bit] | floating[_32Bit] | float64, int]"; expected "_SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]]" [list-item]
- colour/plotting/colorimetry.py:231: error: List item 2 has incompatible type "tuple[floating[_16Bit] | floating[_32Bit] | float64, int]"; expected "_SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]]" [list-item]
- colour/plotting/colorimetry.py:243: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/plotting/colorimetry.py:406: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/plotting/colorimetry.py:407: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/plotting/colorimetry.py:423: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/plotting/colorimetry.py:424: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/plotting/colorimetry.py:543: error: Incompatible types in assignment (expression has type "list[Any]", variable has type "floating[_16Bit] | floating[_32Bit] | float64") [assignment]
- colour/plotting/colorimetry.py:544: error: Item "floating[_16Bit]" of "floating[_16Bit] | floating[_32Bit] | float64" has no attribute "__iter__" (not iterable) [union-attr]
- colour/plotting/colorimetry.py:544: error: Item "floating[_32Bit]" of "floating[_16Bit] | floating[_32Bit] | float64" has no attribute "__iter__" (not iterable) [union-attr]
- colour/plotting/colorimetry.py:544: error: Item "float64" of "floating[_16Bit] | floating[_32Bit] | float64" has no attribute "__iter__" (not iterable) [union-attr]
- colour/plotting/colorimetry.py:768: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/plotting/colorimetry.py:768: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/plotting/tm3018/components.py:553: error: Argument "xy" to "annotate" of "Axes" has incompatible type "tuple[int, floating[_16Bit] | float64 | floating[_32Bit]]"; expected "tuple[float, float]" [arg-type]
- colour/plotting/tm3018/components.py:568: error: Argument "xy" to "annotate" of "Axes" has incompatible type "tuple[int, floating[_16Bit] | float64 | floating[_32Bit]]"; expected "tuple[float, float]" [arg-type]
- colour/notation/tests/test_munsell.py:558: error: Argument 1 to "as_array" has incompatible type "list[tuple[floating[Any], list[float]]]"; expected "Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes] | KeysView[Any] | ValuesView[Any]" [arg-type]
- colour/notation/tests/test_munsell.py:2045: error: List item 0 has incompatible type "list[floating[_16Bit] | floating[_32Bit] | float64]"; expected "float" [list-item]
- colour/notation/tests/test_munsell.py:2301: error: "floating[_16Bit]" object is not iterable [misc]
- colour/notation/tests/test_munsell.py:2301: error: "floating[_32Bit]" object is not iterable [misc]
- colour/notation/tests/test_munsell.py:2301: error: "float64" object is not iterable [misc]
- colour/notation/tests/test_munsell.py:2360: error: "floating[_16Bit]" object is not iterable [misc]
- colour/notation/tests/test_munsell.py:2360: error: "floating[_32Bit]" object is not iterable [misc]
- colour/notation/tests/test_munsell.py:2360: error: "float64" object is not iterable [misc]
- colour/notation/tests/test_munsell.py:2391: error: "floating[_16Bit]" object is not iterable [misc]
- colour/notation/tests/test_munsell.py:2391: error: "floating[_32Bit]" object is not iterable [misc]
- colour/notation/tests/test_munsell.py:2391: error: "float64" object is not iterable [misc]
- colour/examples/plotting/examples_section_plots.py:130: error: Argument "color" to "Line2D" has incompatible type "floating[_16Bit] | floating[_32Bit] | float64"; expected "tuple[float, float, float] | str | str | tuple[float, float, float, float] | tuple[tuple[float, float, float] | str, float] | tuple[tuple[float, float, float, float], float] | None" [arg-type]
- colour/examples/algebra/examples_interpolation.py:135: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/examples/algebra/examples_interpolation.py:136: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/examples/algebra/examples_interpolation.py:148: error: Value of type variable "SupportsRichComparisonT" of "min" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/examples/algebra/examples_interpolation.py:149: error: Value of type variable "SupportsRichComparisonT" of "max" cannot be "floating[_16Bit] | floating[_32Bit] | float64" [type-var]
- colour/algebra/tests/test_extrapolation.py:140: note: x: expected setter type "Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | builtins.bool | int | float | complex | str | bytes | _NestedSequence[builtins.bool | int | float | complex | str | bytes]", got "ndarray[tuple[int], dtype[floating[Any] | integer[Any] | numpy.bool[builtins.bool]]]"
+ colour/algebra/tests/test_extrapolation.py:140: note: x: expected setter type "Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | int | float | complex | str | _NestedSequence[builtins.bool | int | float | complex | str | bytes]", got "ndarray[tuple[int], dtype[floating[Any] | integer[Any] | numpy.bool[builtins.bool]]]"
- colour/algebra/tests/test_extrapolation.py:150: note: x: expected setter type "Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes]", got "ndarray[tuple[int], dtype[float64]]"
+ colour/algebra/tests/test_extrapolation.py:150: note: x: expected setter type "Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | int | float | complex | str | _NestedSequence[bool | int | float | complex | str | bytes]", got "ndarray[tuple[int], dtype[float64]]"
operator (https://github.com/canonical/operator): 1.12x faster (46.5s -> 41.6s in single noisy sample)
hydra-zen (https://github.com/mit-ll-responsible-ai/hydra-zen): 1.14x slower (126.5s -> 144.7s in single noisy sample)
pandas-stubs (https://github.com/pandas-dev/pandas-stubs): 1.06x slower (496.3s -> 527.5s in single noisy sample)
jax (https://github.com/google/jax)
+ jax/_src/pallas/hlo_interpreter.py:93: error: Unused "type: ignore" comment [unused-ignore]
scrapy (https://github.com/scrapy/scrapy)
+ scrapy/utils/datatypes.py:85: error: Unused "type: ignore" comment [unused-ignore]
discord.py (https://github.com/Rapptz/discord.py): 1.12x slower (226.3s -> 253.2s in single noisy sample)
- discord/enums.py:90: error: "tuple[_T_co, ...]" has no attribute "value" [attr-defined]
+ discord/enums.py:90: error: "tuple[Never, ...]" has no attribute "value" [attr-defined]
- discord/enums.py:91: error: "tuple[_T_co, ...]" has no attribute "value" [attr-defined]
+ discord/enums.py:91: error: "tuple[Never, ...]" has no attribute "value" [attr-defined]
- discord/enums.py:92: error: "tuple[_T_co, ...]" has no attribute "value" [attr-defined]
+ discord/enums.py:92: error: "tuple[Never, ...]" has no attribute "value" [attr-defined]
- discord/enums.py:93: error: "tuple[_T_co, ...]" has no attribute "value" [attr-defined]
+ discord/enums.py:93: error: "tuple[Never, ...]" has no attribute "value" [attr-defined]
- discord/ext/commands/_types.py:62: error: Unused "type: ignore" comment [unused-ignore]
werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/datastructures/structures.py:711: error: Unused "type: ignore[arg-type, attr-defined]" comment [unused-ignore]
+ src/werkzeug/datastructures/structures.py:711: error: Cannot infer type argument 2 of "setdefault" of "MutableMapping" [misc]
+ src/werkzeug/datastructures/structures.py:711: note: Error code "misc" not covered by "type: ignore" comment
+ src/werkzeug/datastructures/structures.py:792: error: Unused "type: ignore[assignment]" comment [unused-ignore]
+ src/werkzeug/datastructures/structures.py:806: error: Unused "type: ignore[assignment]" comment [unused-ignore]
- tests/test_local.py:146: error: No overload variant of "list" matches argument type "LocalProxy[Any]" [call-overload]
- tests/test_local.py:146: note: Possible overload variants:
- tests/test_local.py:146: note: def [_T] __init__(self) -> list[_T]
- tests/test_local.py:146: note: def [_T] __init__(self, Iterable[_T], /) -> list[_T]
scikit-learn (https://github.com/scikit-learn/scikit-learn): 1.10x slower (249.8s -> 274.6s in single noisy sample)
scipy (https://github.com/scipy/scipy): 1.13x slower (232.0s -> 261.8s in single noisy sample)
- scipy/spatial/tests/test_spherical_voronoi.py:194: error: Value of type variable "SupportsRichComparisonT" of "sorted" cannot be "floating[Any]" [type-var]
|
@sterliakov OK, yes, this is a good argument. Over next few days I will spot-check |
This is a fourth "major" PR toward #7724. This is one is watershed/crux of the whole series (but to set correct expectations, there are almost a dozen smaller follow-up/clean-up PRs in the pipeline).
The core of the idea is to set current type-checker as part of the global state. There are however some details:
is_subtype()
before type-checking. For now, I fall back to old logic in this cases. In follow up PRs we may switch to using type-checker instances before type checking phase (this requires some care).__call__
, and one temporary hack forfreshen_all_functions_type_vars
(to reduce performance impact).find_member()
. Now we will expand type by instance, so we have something like this:def (x: T) -> T
).