Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not slice ListVector #22041

Merged
merged 1 commit into from
May 22, 2024
Merged

Do not slice ListVector #22041

merged 1 commit into from
May 22, 2024

Conversation

wendigo
Copy link
Contributor

@wendigo wendigo commented May 20, 2024

Description

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text:

# Section
* Fix some things. ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label May 20, 2024
@github-actions github-actions bot added the bigquery BigQuery connector label May 20, 2024
@wendigo wendigo force-pushed the serafin/do-not-transfer-v2 branch from b1f45be to d35d036 Compare May 21, 2024 07:48
@wendigo wendigo requested review from ebyhr and vlad-lyutenko May 21, 2024 07:49
@wendigo wendigo changed the title Wip: Do not slice ListVector Do not slice ListVector May 21, 2024
@wendigo
Copy link
Contributor Author

wendigo commented May 21, 2024

I don't know why are we copying memory in the first place to just read a vector.

@hashhar
Copy link
Member

hashhar commented May 21, 2024

I don't know why are we copying memory in the first place to just read a vector.

Probably for explicit ownership of the underlying buffers otherwise maybe the contents change between when it's sent back vs when it's actually read?

@wendigo
Copy link
Contributor Author

wendigo commented May 21, 2024

@hashhar this is read only memory. I don't think this is the reason

@wendigo
Copy link
Contributor Author

wendigo commented May 21, 2024

And we seems to do it only for lists, but not for other types and it doesn't make sense because they are encoded in the similiar way

@wendigo
Copy link
Contributor Author

wendigo commented May 21, 2024

I think that we were using much more memory than needed

@wendigo
Copy link
Contributor Author

wendigo commented May 21, 2024

@ebyhr @hashhar please review

@wendigo wendigo merged commit 24a6da9 into master May 22, 2024
18 checks passed
@wendigo wendigo deleted the serafin/do-not-transfer-v2 branch May 22, 2024 01:51
@github-actions github-actions bot added this to the 449 milestone May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bigquery BigQuery connector cla-signed
Development

Successfully merging this pull request may close these issues.

3 participants