Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added API functions to get information about the fragments written by a WRITE query. #1396

Merged
merged 1 commit into from
Oct 9, 2019

Conversation

stavrospapadopoulos
Copy link
Member

Closes #1090

@ihnorton
Copy link
Member

ihnorton commented Oct 8, 2019

@stavrospapadopoulos could you clarify the timeframe when these calls are valid? Is it from return of tiledb_query_submit tiledb_query_finalize until any operation modifying the query object, or array close?

@stavrospapadopoulos
Copy link
Member Author

Those calls are valid any time throughout the lifetime of tiledb_query_t. Before tiledb_query_submit, there will be 0 fragments written, whereas after tiledb_query_submit and tiledb_query_finalize (for global order writes) there will be >0 fragments written.

Please note that there may even be >1 fragments written after a change I am planning on pushing over the next couple of weeks. Based on a config, we will allow the user to cap the fragment size and, therefore, one write operation may result in producing >1 fragments.

Copy link
Contributor

@tdenniston tdenniston left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. One note is that returning the const char* pointer into the written fragments vector is a little tricky -- if the user performs another write, the vector and its elements may move in memory due to reallocations, invalidating the previously fetched string pointer.

@tdenniston tdenniston added this to the 1.7.0 milestone Oct 8, 2019
@stavrospapadopoulos
Copy link
Member Author

LGTM. One note is that returning the const char* pointer into the written fragments vector is a little tricky -- if the user performs another write, the vector and its elements may move in memory due to reallocations, invalidating the previously fetched string pointer.

Good point, I will think about how to revise. It is cumbersome to have the user pre-allocate buffers for strings, mainly because handling overflows will be tedious.

@tdenniston
Copy link
Contributor

Most high-level APIs (including the one you already added for C++) will immediately make a copy, so it's probably not a big deal. I think it would be ok if you just added a note to the doc string that says "make a copy of this, as the pointer may be invalidated on subsequent writes" or similar.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature Request: Return fragment uuid when writing Dense Array
3 participants