Skip to content

Bug with comparing ManifestArrays using __eq__ which have fill_value=NaN #501

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
TomNicholas opened this issue Mar 24, 2025 · 0 comments · Fixed by #502
Closed

Bug with comparing ManifestArrays using __eq__ which have fill_value=NaN #501

TomNicholas opened this issue Mar 24, 2025 · 0 comments · Fixed by #502
Labels
bug Something isn't working

Comments

@TomNicholas
Copy link
Member

I'm stuck on why this assert_identical on the datasets doesn't pass. It complains that all the coordinates are different, but every attribute I can think to check looks identical 😖

Originally posted by @TomNicholas in #349 (comment)

I found out what the problem is - two ManifestArrays with identical .metadata will compare not equal if they both have NaN for a fill_value. This is because the __eq__ check introspects deeper until it finds e.g. the np.float32(nan) type, but

In [4]: bool(np.float32('nan') == np.float32('nan'))
Out[4]: False

(See https://stackoverflow.com/a/10059796 for why numpy NaNs behave like this.)

The solution needs to be to actually check two Metadata classes are __eq__ with dedicated code, not just trusting the python dataclasses' automatically-generated __eq__ method to fi it.

I would ideally like to move that fix upstream into zarr-python.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant