Skip to content

.Net: Change UpsertAsync return type to IReadOnlyList<TKey> #11402

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 10, 2025

Conversation

roji
Copy link
Member

@roji roji commented Apr 7, 2025

Note: this PR is based on top of #11392, review 2nd commit only

Closes #10692

@markwallace-microsoft markwallace-microsoft added .NET Issue or Pull requests regarding .NET code kernel Issues or pull requests impacting the core kernel memory labels Apr 7, 2025
@github-actions github-actions bot changed the title Change UpsertAsync return type to IReadOnlyList<TKey> .Net: Change UpsertAsync return type to IReadOnlyList<TKey> Apr 7, 2025
@roji roji temporarily deployed to integration April 7, 2025 10:49 — with GitHub Actions Inactive
@roji roji self-assigned this Apr 7, 2025
@roji roji force-pushed the UpsertReturnsList branch from 68a8ade to fafbfbf Compare April 7, 2025 10:57
@roji roji marked this pull request as ready for review April 7, 2025 10:57
@roji roji requested a review from a team as a code owner April 7, 2025 10:57
@roji roji temporarily deployed to integration April 7, 2025 10:58 — with GitHub Actions Inactive
@adamsitnik
Copy link
Member

@roji are you sure that we should return IReadOnlyList<TKey> rather than Task (void). I just want to make sure we want to support the key generation.

@roji
Copy link
Member Author

roji commented Apr 7, 2025

are you sure that we should return IReadOnlyList rather than Task (void). I just want to make sure we want to support the key generation.

I'm honestly not completely sure. From our last design discussion on this, I got the impression that @westey-m and @dmytrostruk felt strongly that this is something we should keep. I'm on the fence - it is a capability that's shared across at least our relational providers (PG, SQL Server, SQLite), and I think @westey-m said at least one other non-relational database supports it (we should confirm, though even if not, that might not be a sufficient reason not to support it).

I do think that once we remove the IAsyncEnumerable in this PR, there's much less objection to supporting this - it no longer really gets in the way, since users can just ignore the return value. So I think there's at least consensus that this is the minimum we want to do.

But I'm totally fine with discussing this again and seeing how we feel about removing the feature altogether. It would be good to have the definitive list of database which do support this, though.

@adamsitnik
Copy link
Member

My understanding was that we do the research first to find out whether any non-relational DB supports it. If not, we make it return Task, if yes, we make it return a list.

@roji
Copy link
Member Author

roji commented Apr 7, 2025

OK, no problem - let's wait with this PR then until the research is done and we rediscuss (@westey-m I think this one's on you - I'll assign the issue back to you). I'll turn this into a draft in the meantime.

@roji roji marked this pull request as draft April 7, 2025 12:06
@roji roji force-pushed the UpsertReturnsList branch from fafbfbf to 4a57c6d Compare April 7, 2025 12:07
@roji roji temporarily deployed to integration April 7, 2025 12:08 — with GitHub Actions Inactive
@roji roji removed their assignment Apr 8, 2025
@roji roji force-pushed the UpsertReturnsList branch from 4a57c6d to e296ea7 Compare April 10, 2025 15:02
@roji roji marked this pull request as ready for review April 10, 2025 15:03
@roji
Copy link
Member Author

roji commented Apr 10, 2025

@westey-m @adamsitnik I've rebased this PR and updated the docs to make it extra clear that batching UpsertAsync isn't idempotent when store-generated keys are in use, and atomicity isn't guaranteed.

Copy link
Member

@adamsitnik adamsitnik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you @roji !

{
yield return (TKey)keyProperty.GetValueAsObject(record!)!;
}
return records.Select(r => (TKey)keyProperty.GetValueAsObject(r!)!).ToList();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the bang still required?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The one on r isn't (because we now have a notnull constraint on TRecord), the other unfortunately still is... We need to replace at least some of the calls to GetValueAsObject (and Set) with generic getters/setters - that would allow #11183, but would also obviate the bang (since TKey is constrained to notnull. Later :)

@@ -47,11 +47,7 @@ public override async Task InitializeAsync()

protected virtual async Task SeedAsync()
{
// TODO: UpsertBatchAsync returns IAsyncEnumerable<TKey> (to support server-generated keys?), but this makes it quite hard to use:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks much better now 💯

@roji roji force-pushed the UpsertReturnsList branch from e296ea7 to 69ecf92 Compare April 10, 2025 19:49
@roji
Copy link
Member Author

roji commented Apr 10, 2025

I actually now seem to remember that a .NET design guideline is to not return collection interfaces, but rather prefer concrete types (so List<T>), because users may end up downcasting them anyway, and a change of the concrete type would end up breaking people in any case. I'm not sure if that's still a guideline - I'll go ahead and merge as-is with IReadOnlyList for now, and raise in API review (it's trivial to change later anyway).

@roji roji merged commit ae0fa62 into microsoft:feature-vector-data-preb2 Apr 10, 2025
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kernel Issues or pull requests impacting the core kernel memory .NET Issue or Pull requests regarding .NET code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants