Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-217: Fix Travis w.r.t conda 4.1.0 changes #90

Closed
wants to merge 1 commit into from

Conversation

xhochy
Copy link
Member

@xhochy xhochy commented Jun 15, 2016

Travis is happy, fixes the problems we see with Travis in #85

@xhochy xhochy force-pushed the fix-conda-show-channel-urls branch from bded63d to 0137ce2 Compare June 15, 2016 07:07
@xhochy
Copy link
Member Author

xhochy commented Jun 15, 2016

For reference, after the conda config --add channels calls.

conda info -a some days ago:

         channel URLs : https://conda.anaconda.org/apache/osx-64/
                        https://conda.anaconda.org/apache/noarch/
                        https://conda.anaconda.org/conda-forge/osx-64/
                        https://conda.anaconda.org/conda-forge/noarch/
                        https://repo.continuum.io/pkgs/free/osx-64/
                        https://repo.continuum.io/pkgs/free/noarch/
                        https://repo.continuum.io/pkgs/pro/osx-64/
                        https://repo.continuum.io/pkgs/pro/noarch/

now:

         channel URLs : https://conda.anaconda.org/apache/osx-64/
                        https://conda.anaconda.org/apache/noarch/
                        https://conda.anaconda.org/conda-forge/osx-64/
                        https://conda.anaconda.org/conda-forge/noarch/

i.e. continuum's channels are missing. Those were listed before we added the channels.

@xhochy xhochy force-pushed the fix-conda-show-channel-urls branch 2 times, most recently from 65db19b to a1d6af6 Compare June 15, 2016 09:08
@xhochy xhochy changed the title Fix conda show_channel_urls ARROW-217: Fix Travis w.r.t conda 4.1.0 changes Jun 15, 2016
@xhochy xhochy force-pushed the fix-conda-show-channel-urls branch from a1d6af6 to 82e9840 Compare June 15, 2016 09:42
@wesm
Copy link
Member

wesm commented Jun 15, 2016

+1, thank you

@asfgit asfgit closed this in b4e0e93 Jun 15, 2016
@wesm
Copy link
Member

wesm commented Jun 15, 2016

@kalefranz can you or someone on the conda team comment on what may have changed with the channel URLs? thank you =)

@kalefranz
Copy link

In 4.1.0 we introduced a bug where defaults wasn't automatically added to channels in ~/.condarc if the file already existed.

We've fixed it here: conda/conda#2670
4.1.1 will be released with that fix and others in a few hours

In general, as of 4.1.0, channel order now matters. Before, all channels were thrown together and the package with the highest version comparison won. Now, if a package exists in a higher priority channel (i.e. the channel comes first in the channel list), then that package wins.

We'll continue to refine and improve on channel/package priority. This is just a first step.

@wesm
Copy link
Member

wesm commented Jun 15, 2016

@kalefranz thank you for the quick reply. Per the channel priority, am I correct that the last channel added has the highest priority? (so in our case, apache channel would supercede conda-forge?)

@kalefranz
Copy link

kalefranz commented Jun 15, 2016

Yes, the last channel added with conda config ends up being the first channel in the channels list in ~/.condarc, and thus has the highest priority.

@kalefranz
Copy link

FYI we're also adding a auto_update_conda option that defaults to true, but that can be set to false to disable conda's aggressive self-update. Definitely useful in some CI situations. That'll be in both 4.1.1 and 4.0.9 coming out later today.

@xhochy xhochy deleted the fix-conda-show-channel-urls branch March 7, 2017 16:16
wesm pushed a commit to wesm/arrow that referenced this pull request Sep 2, 2018
Basic write support with only DataPage as page type, no compression and fixed page sizes. Also allocates and consumes more memory than the optimal solution. But I hope that it is in a state where we can review & merge, so that afterwards the work can be split up.

One outstanding design question for me is `WriteDataPage`. We could pass here a `DataPage` instance but would also need to take care of `num_rows` and `num_values` correctly for the thrift metadata, hoping for good suggestions here.

Also there are numerous TODOs in here for which we need to decide if they should be done as part of the PR or if we should open JIRAs for them.

Author: Uwe L. Korn <[email protected]>

Closes apache#90 from xhochy/parquet-436 and squashes the following commits:

ebfc3e0 [Uwe L. Korn] Address review comments
4a81afe [Uwe L. Korn] Address review comments
80af8ca [Uwe L. Korn] Address review comments
0cf2aa1 [Uwe L. Korn] Address code review comments
77da3c7 [Uwe L. Korn] Style fixes
afbb6da [Uwe L. Korn] Lint fixes
04fb0f2 [Uwe L. Korn] Add JIRA tickets to TODOs
02e0ba5 [Uwe L. Korn] More code cleanup
36b7d1d [Uwe L. Korn] Code cleanups
b581da1 [Uwe L. Korn] Get rid of num_nulls
f3ec4de [Uwe L. Korn] Correct order of definition and repetition levels
e306b04 [Uwe L. Korn] Running write unit test
ad351e8 [Uwe L. Korn] Added more tests for PageWriter
63c06f0 [Uwe L. Korn] Refactored unit tests
ebd9648 [Uwe L. Korn] We can now at least open a self-written file
7bedf34 [Uwe L. Korn] Lint fixes
aa84251 [Uwe L. Korn] Add support for writing repetition levels
b354e07 [Uwe L. Korn] Add support for writing definition levels
7298f96 [Uwe L. Korn] Lint fixes
9fe1d89 [Uwe L. Korn] Some unit test refactorings
9e21c56 [Uwe L. Korn] First version that can write a page
0662386 [Uwe L. Korn] First draft of write API
wesm pushed a commit to wesm/arrow that referenced this pull request Sep 4, 2018
Basic write support with only DataPage as page type, no compression and fixed page sizes. Also allocates and consumes more memory than the optimal solution. But I hope that it is in a state where we can review & merge, so that afterwards the work can be split up.

One outstanding design question for me is `WriteDataPage`. We could pass here a `DataPage` instance but would also need to take care of `num_rows` and `num_values` correctly for the thrift metadata, hoping for good suggestions here.

Also there are numerous TODOs in here for which we need to decide if they should be done as part of the PR or if we should open JIRAs for them.

Author: Uwe L. Korn <[email protected]>

Closes apache#90 from xhochy/parquet-436 and squashes the following commits:

ebfc3e0 [Uwe L. Korn] Address review comments
4a81afe [Uwe L. Korn] Address review comments
80af8ca [Uwe L. Korn] Address review comments
0cf2aa1 [Uwe L. Korn] Address code review comments
77da3c7 [Uwe L. Korn] Style fixes
afbb6da [Uwe L. Korn] Lint fixes
04fb0f2 [Uwe L. Korn] Add JIRA tickets to TODOs
02e0ba5 [Uwe L. Korn] More code cleanup
36b7d1d [Uwe L. Korn] Code cleanups
b581da1 [Uwe L. Korn] Get rid of num_nulls
f3ec4de [Uwe L. Korn] Correct order of definition and repetition levels
e306b04 [Uwe L. Korn] Running write unit test
ad351e8 [Uwe L. Korn] Added more tests for PageWriter
63c06f0 [Uwe L. Korn] Refactored unit tests
ebd9648 [Uwe L. Korn] We can now at least open a self-written file
7bedf34 [Uwe L. Korn] Lint fixes
aa84251 [Uwe L. Korn] Add support for writing repetition levels
b354e07 [Uwe L. Korn] Add support for writing definition levels
7298f96 [Uwe L. Korn] Lint fixes
9fe1d89 [Uwe L. Korn] Some unit test refactorings
9e21c56 [Uwe L. Korn] First version that can write a page
0662386 [Uwe L. Korn] First draft of write API

Change-Id: I32392ecdf46c89cb8a66feabb384af5c7ec3dc4f
wesm pushed a commit to wesm/arrow that referenced this pull request Sep 6, 2018
Basic write support with only DataPage as page type, no compression and fixed page sizes. Also allocates and consumes more memory than the optimal solution. But I hope that it is in a state where we can review & merge, so that afterwards the work can be split up.

One outstanding design question for me is `WriteDataPage`. We could pass here a `DataPage` instance but would also need to take care of `num_rows` and `num_values` correctly for the thrift metadata, hoping for good suggestions here.

Also there are numerous TODOs in here for which we need to decide if they should be done as part of the PR or if we should open JIRAs for them.

Author: Uwe L. Korn <[email protected]>

Closes apache#90 from xhochy/parquet-436 and squashes the following commits:

ebfc3e0 [Uwe L. Korn] Address review comments
4a81afe [Uwe L. Korn] Address review comments
80af8ca [Uwe L. Korn] Address review comments
0cf2aa1 [Uwe L. Korn] Address code review comments
77da3c7 [Uwe L. Korn] Style fixes
afbb6da [Uwe L. Korn] Lint fixes
04fb0f2 [Uwe L. Korn] Add JIRA tickets to TODOs
02e0ba5 [Uwe L. Korn] More code cleanup
36b7d1d [Uwe L. Korn] Code cleanups
b581da1 [Uwe L. Korn] Get rid of num_nulls
f3ec4de [Uwe L. Korn] Correct order of definition and repetition levels
e306b04 [Uwe L. Korn] Running write unit test
ad351e8 [Uwe L. Korn] Added more tests for PageWriter
63c06f0 [Uwe L. Korn] Refactored unit tests
ebd9648 [Uwe L. Korn] We can now at least open a self-written file
7bedf34 [Uwe L. Korn] Lint fixes
aa84251 [Uwe L. Korn] Add support for writing repetition levels
b354e07 [Uwe L. Korn] Add support for writing definition levels
7298f96 [Uwe L. Korn] Lint fixes
9fe1d89 [Uwe L. Korn] Some unit test refactorings
9e21c56 [Uwe L. Korn] First version that can write a page
0662386 [Uwe L. Korn] First draft of write API

Change-Id: I32392ecdf46c89cb8a66feabb384af5c7ec3dc4f
wesm pushed a commit to wesm/arrow that referenced this pull request Sep 7, 2018
Basic write support with only DataPage as page type, no compression and fixed page sizes. Also allocates and consumes more memory than the optimal solution. But I hope that it is in a state where we can review & merge, so that afterwards the work can be split up.

One outstanding design question for me is `WriteDataPage`. We could pass here a `DataPage` instance but would also need to take care of `num_rows` and `num_values` correctly for the thrift metadata, hoping for good suggestions here.

Also there are numerous TODOs in here for which we need to decide if they should be done as part of the PR or if we should open JIRAs for them.

Author: Uwe L. Korn <[email protected]>

Closes apache#90 from xhochy/parquet-436 and squashes the following commits:

ebfc3e0 [Uwe L. Korn] Address review comments
4a81afe [Uwe L. Korn] Address review comments
80af8ca [Uwe L. Korn] Address review comments
0cf2aa1 [Uwe L. Korn] Address code review comments
77da3c7 [Uwe L. Korn] Style fixes
afbb6da [Uwe L. Korn] Lint fixes
04fb0f2 [Uwe L. Korn] Add JIRA tickets to TODOs
02e0ba5 [Uwe L. Korn] More code cleanup
36b7d1d [Uwe L. Korn] Code cleanups
b581da1 [Uwe L. Korn] Get rid of num_nulls
f3ec4de [Uwe L. Korn] Correct order of definition and repetition levels
e306b04 [Uwe L. Korn] Running write unit test
ad351e8 [Uwe L. Korn] Added more tests for PageWriter
63c06f0 [Uwe L. Korn] Refactored unit tests
ebd9648 [Uwe L. Korn] We can now at least open a self-written file
7bedf34 [Uwe L. Korn] Lint fixes
aa84251 [Uwe L. Korn] Add support for writing repetition levels
b354e07 [Uwe L. Korn] Add support for writing definition levels
7298f96 [Uwe L. Korn] Lint fixes
9fe1d89 [Uwe L. Korn] Some unit test refactorings
9e21c56 [Uwe L. Korn] First version that can write a page
0662386 [Uwe L. Korn] First draft of write API

Change-Id: I32392ecdf46c89cb8a66feabb384af5c7ec3dc4f
wesm pushed a commit to wesm/arrow that referenced this pull request Sep 8, 2018
Basic write support with only DataPage as page type, no compression and fixed page sizes. Also allocates and consumes more memory than the optimal solution. But I hope that it is in a state where we can review & merge, so that afterwards the work can be split up.

One outstanding design question for me is `WriteDataPage`. We could pass here a `DataPage` instance but would also need to take care of `num_rows` and `num_values` correctly for the thrift metadata, hoping for good suggestions here.

Also there are numerous TODOs in here for which we need to decide if they should be done as part of the PR or if we should open JIRAs for them.

Author: Uwe L. Korn <[email protected]>

Closes apache#90 from xhochy/parquet-436 and squashes the following commits:

ebfc3e0 [Uwe L. Korn] Address review comments
4a81afe [Uwe L. Korn] Address review comments
80af8ca [Uwe L. Korn] Address review comments
0cf2aa1 [Uwe L. Korn] Address code review comments
77da3c7 [Uwe L. Korn] Style fixes
afbb6da [Uwe L. Korn] Lint fixes
04fb0f2 [Uwe L. Korn] Add JIRA tickets to TODOs
02e0ba5 [Uwe L. Korn] More code cleanup
36b7d1d [Uwe L. Korn] Code cleanups
b581da1 [Uwe L. Korn] Get rid of num_nulls
f3ec4de [Uwe L. Korn] Correct order of definition and repetition levels
e306b04 [Uwe L. Korn] Running write unit test
ad351e8 [Uwe L. Korn] Added more tests for PageWriter
63c06f0 [Uwe L. Korn] Refactored unit tests
ebd9648 [Uwe L. Korn] We can now at least open a self-written file
7bedf34 [Uwe L. Korn] Lint fixes
aa84251 [Uwe L. Korn] Add support for writing repetition levels
b354e07 [Uwe L. Korn] Add support for writing definition levels
7298f96 [Uwe L. Korn] Lint fixes
9fe1d89 [Uwe L. Korn] Some unit test refactorings
9e21c56 [Uwe L. Korn] First version that can write a page
0662386 [Uwe L. Korn] First draft of write API

Change-Id: I32392ecdf46c89cb8a66feabb384af5c7ec3dc4f
rafael-telles pushed a commit to rafael-telles/arrow that referenced this pull request Aug 24, 2021
Flight SQL Ratification Based On Community Feedback #5
amoeba added a commit to amoeba/arrow that referenced this pull request Jan 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants