Skip to content

Why doesn't the API follow that of concurrent.futures? #1427

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
TomNicholas opened this issue Mar 25, 2025 · 3 comments
Open

Why doesn't the API follow that of concurrent.futures? #1427

TomNicholas opened this issue Mar 25, 2025 · 3 comments

Comments

@TomNicholas
Copy link

TomNicholas commented Mar 25, 2025

Why doesn't the API follow that of concurrent.futures? The lithops executors are clearly modelled after the executors in that standard library module, but they don't actually follow the API, nor do they inherit from the concurrent.futures.Executor ABC. The lithops future object doesn't inherit from concurrent.futures.Future either.

This matters - in particular the lithops .map() method returns a future whereas the concurrent.futures .map executes eagerly. This means downstream code cannot treat other executors such as concurrent.futures.ThreadPoolExecutor and lithops executors interchangably.

I can understand why you extended the API to add new methods (e.g. .map_reduce()), but why not follow the convention concurrent.futures set for the methods it does define? That standard library module predates this package by many years.

If this difference is intended, it would be nice to note why on the docs page on lithops futures, to avoid other people going down this rabbit hole too.

@TomNicholas
Copy link
Author

There's also no .submit() method on lithops executors.

@TomNicholas
Copy link
Author

Another difference: lithops wait() is not a top-level function, it's a staticmethod on the lithops executor class, unlike concurrent.futures.wait.

@TomNicholas
Copy link
Author

To be more constructive, could we imagine adding a compliant executor interface to lithops, like I did here?

zarr-developers/VirtualiZarr#349 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant