-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ARROW-217: Fix Travis w.r.t conda 4.1.0 changes #90
Conversation
bded63d
to
0137ce2
Compare
For reference, after the
now:
i.e. continuum's channels are missing. Those were listed before we added the channels. |
65db19b
to
a1d6af6
Compare
a1d6af6
to
82e9840
Compare
+1, thank you |
@kalefranz can you or someone on the conda team comment on what may have changed with the channel URLs? thank you =) |
In 4.1.0 we introduced a bug where We've fixed it here: conda/conda#2670 In general, as of 4.1.0, channel order now matters. Before, all channels were thrown together and the package with the highest version comparison won. Now, if a package exists in a higher priority channel (i.e. the channel comes first in the channel list), then that package wins. We'll continue to refine and improve on channel/package priority. This is just a first step. |
@kalefranz thank you for the quick reply. Per the channel priority, am I correct that the last channel added has the highest priority? (so in our case, apache channel would supercede conda-forge?) |
Yes, the last channel added with |
FYI we're also adding a |
Basic write support with only DataPage as page type, no compression and fixed page sizes. Also allocates and consumes more memory than the optimal solution. But I hope that it is in a state where we can review & merge, so that afterwards the work can be split up. One outstanding design question for me is `WriteDataPage`. We could pass here a `DataPage` instance but would also need to take care of `num_rows` and `num_values` correctly for the thrift metadata, hoping for good suggestions here. Also there are numerous TODOs in here for which we need to decide if they should be done as part of the PR or if we should open JIRAs for them. Author: Uwe L. Korn <[email protected]> Closes apache#90 from xhochy/parquet-436 and squashes the following commits: ebfc3e0 [Uwe L. Korn] Address review comments 4a81afe [Uwe L. Korn] Address review comments 80af8ca [Uwe L. Korn] Address review comments 0cf2aa1 [Uwe L. Korn] Address code review comments 77da3c7 [Uwe L. Korn] Style fixes afbb6da [Uwe L. Korn] Lint fixes 04fb0f2 [Uwe L. Korn] Add JIRA tickets to TODOs 02e0ba5 [Uwe L. Korn] More code cleanup 36b7d1d [Uwe L. Korn] Code cleanups b581da1 [Uwe L. Korn] Get rid of num_nulls f3ec4de [Uwe L. Korn] Correct order of definition and repetition levels e306b04 [Uwe L. Korn] Running write unit test ad351e8 [Uwe L. Korn] Added more tests for PageWriter 63c06f0 [Uwe L. Korn] Refactored unit tests ebd9648 [Uwe L. Korn] We can now at least open a self-written file 7bedf34 [Uwe L. Korn] Lint fixes aa84251 [Uwe L. Korn] Add support for writing repetition levels b354e07 [Uwe L. Korn] Add support for writing definition levels 7298f96 [Uwe L. Korn] Lint fixes 9fe1d89 [Uwe L. Korn] Some unit test refactorings 9e21c56 [Uwe L. Korn] First version that can write a page 0662386 [Uwe L. Korn] First draft of write API
Basic write support with only DataPage as page type, no compression and fixed page sizes. Also allocates and consumes more memory than the optimal solution. But I hope that it is in a state where we can review & merge, so that afterwards the work can be split up. One outstanding design question for me is `WriteDataPage`. We could pass here a `DataPage` instance but would also need to take care of `num_rows` and `num_values` correctly for the thrift metadata, hoping for good suggestions here. Also there are numerous TODOs in here for which we need to decide if they should be done as part of the PR or if we should open JIRAs for them. Author: Uwe L. Korn <[email protected]> Closes apache#90 from xhochy/parquet-436 and squashes the following commits: ebfc3e0 [Uwe L. Korn] Address review comments 4a81afe [Uwe L. Korn] Address review comments 80af8ca [Uwe L. Korn] Address review comments 0cf2aa1 [Uwe L. Korn] Address code review comments 77da3c7 [Uwe L. Korn] Style fixes afbb6da [Uwe L. Korn] Lint fixes 04fb0f2 [Uwe L. Korn] Add JIRA tickets to TODOs 02e0ba5 [Uwe L. Korn] More code cleanup 36b7d1d [Uwe L. Korn] Code cleanups b581da1 [Uwe L. Korn] Get rid of num_nulls f3ec4de [Uwe L. Korn] Correct order of definition and repetition levels e306b04 [Uwe L. Korn] Running write unit test ad351e8 [Uwe L. Korn] Added more tests for PageWriter 63c06f0 [Uwe L. Korn] Refactored unit tests ebd9648 [Uwe L. Korn] We can now at least open a self-written file 7bedf34 [Uwe L. Korn] Lint fixes aa84251 [Uwe L. Korn] Add support for writing repetition levels b354e07 [Uwe L. Korn] Add support for writing definition levels 7298f96 [Uwe L. Korn] Lint fixes 9fe1d89 [Uwe L. Korn] Some unit test refactorings 9e21c56 [Uwe L. Korn] First version that can write a page 0662386 [Uwe L. Korn] First draft of write API Change-Id: I32392ecdf46c89cb8a66feabb384af5c7ec3dc4f
Basic write support with only DataPage as page type, no compression and fixed page sizes. Also allocates and consumes more memory than the optimal solution. But I hope that it is in a state where we can review & merge, so that afterwards the work can be split up. One outstanding design question for me is `WriteDataPage`. We could pass here a `DataPage` instance but would also need to take care of `num_rows` and `num_values` correctly for the thrift metadata, hoping for good suggestions here. Also there are numerous TODOs in here for which we need to decide if they should be done as part of the PR or if we should open JIRAs for them. Author: Uwe L. Korn <[email protected]> Closes apache#90 from xhochy/parquet-436 and squashes the following commits: ebfc3e0 [Uwe L. Korn] Address review comments 4a81afe [Uwe L. Korn] Address review comments 80af8ca [Uwe L. Korn] Address review comments 0cf2aa1 [Uwe L. Korn] Address code review comments 77da3c7 [Uwe L. Korn] Style fixes afbb6da [Uwe L. Korn] Lint fixes 04fb0f2 [Uwe L. Korn] Add JIRA tickets to TODOs 02e0ba5 [Uwe L. Korn] More code cleanup 36b7d1d [Uwe L. Korn] Code cleanups b581da1 [Uwe L. Korn] Get rid of num_nulls f3ec4de [Uwe L. Korn] Correct order of definition and repetition levels e306b04 [Uwe L. Korn] Running write unit test ad351e8 [Uwe L. Korn] Added more tests for PageWriter 63c06f0 [Uwe L. Korn] Refactored unit tests ebd9648 [Uwe L. Korn] We can now at least open a self-written file 7bedf34 [Uwe L. Korn] Lint fixes aa84251 [Uwe L. Korn] Add support for writing repetition levels b354e07 [Uwe L. Korn] Add support for writing definition levels 7298f96 [Uwe L. Korn] Lint fixes 9fe1d89 [Uwe L. Korn] Some unit test refactorings 9e21c56 [Uwe L. Korn] First version that can write a page 0662386 [Uwe L. Korn] First draft of write API Change-Id: I32392ecdf46c89cb8a66feabb384af5c7ec3dc4f
Basic write support with only DataPage as page type, no compression and fixed page sizes. Also allocates and consumes more memory than the optimal solution. But I hope that it is in a state where we can review & merge, so that afterwards the work can be split up. One outstanding design question for me is `WriteDataPage`. We could pass here a `DataPage` instance but would also need to take care of `num_rows` and `num_values` correctly for the thrift metadata, hoping for good suggestions here. Also there are numerous TODOs in here for which we need to decide if they should be done as part of the PR or if we should open JIRAs for them. Author: Uwe L. Korn <[email protected]> Closes apache#90 from xhochy/parquet-436 and squashes the following commits: ebfc3e0 [Uwe L. Korn] Address review comments 4a81afe [Uwe L. Korn] Address review comments 80af8ca [Uwe L. Korn] Address review comments 0cf2aa1 [Uwe L. Korn] Address code review comments 77da3c7 [Uwe L. Korn] Style fixes afbb6da [Uwe L. Korn] Lint fixes 04fb0f2 [Uwe L. Korn] Add JIRA tickets to TODOs 02e0ba5 [Uwe L. Korn] More code cleanup 36b7d1d [Uwe L. Korn] Code cleanups b581da1 [Uwe L. Korn] Get rid of num_nulls f3ec4de [Uwe L. Korn] Correct order of definition and repetition levels e306b04 [Uwe L. Korn] Running write unit test ad351e8 [Uwe L. Korn] Added more tests for PageWriter 63c06f0 [Uwe L. Korn] Refactored unit tests ebd9648 [Uwe L. Korn] We can now at least open a self-written file 7bedf34 [Uwe L. Korn] Lint fixes aa84251 [Uwe L. Korn] Add support for writing repetition levels b354e07 [Uwe L. Korn] Add support for writing definition levels 7298f96 [Uwe L. Korn] Lint fixes 9fe1d89 [Uwe L. Korn] Some unit test refactorings 9e21c56 [Uwe L. Korn] First version that can write a page 0662386 [Uwe L. Korn] First draft of write API Change-Id: I32392ecdf46c89cb8a66feabb384af5c7ec3dc4f
Basic write support with only DataPage as page type, no compression and fixed page sizes. Also allocates and consumes more memory than the optimal solution. But I hope that it is in a state where we can review & merge, so that afterwards the work can be split up. One outstanding design question for me is `WriteDataPage`. We could pass here a `DataPage` instance but would also need to take care of `num_rows` and `num_values` correctly for the thrift metadata, hoping for good suggestions here. Also there are numerous TODOs in here for which we need to decide if they should be done as part of the PR or if we should open JIRAs for them. Author: Uwe L. Korn <[email protected]> Closes apache#90 from xhochy/parquet-436 and squashes the following commits: ebfc3e0 [Uwe L. Korn] Address review comments 4a81afe [Uwe L. Korn] Address review comments 80af8ca [Uwe L. Korn] Address review comments 0cf2aa1 [Uwe L. Korn] Address code review comments 77da3c7 [Uwe L. Korn] Style fixes afbb6da [Uwe L. Korn] Lint fixes 04fb0f2 [Uwe L. Korn] Add JIRA tickets to TODOs 02e0ba5 [Uwe L. Korn] More code cleanup 36b7d1d [Uwe L. Korn] Code cleanups b581da1 [Uwe L. Korn] Get rid of num_nulls f3ec4de [Uwe L. Korn] Correct order of definition and repetition levels e306b04 [Uwe L. Korn] Running write unit test ad351e8 [Uwe L. Korn] Added more tests for PageWriter 63c06f0 [Uwe L. Korn] Refactored unit tests ebd9648 [Uwe L. Korn] We can now at least open a self-written file 7bedf34 [Uwe L. Korn] Lint fixes aa84251 [Uwe L. Korn] Add support for writing repetition levels b354e07 [Uwe L. Korn] Add support for writing definition levels 7298f96 [Uwe L. Korn] Lint fixes 9fe1d89 [Uwe L. Korn] Some unit test refactorings 9e21c56 [Uwe L. Korn] First version that can write a page 0662386 [Uwe L. Korn] First draft of write API Change-Id: I32392ecdf46c89cb8a66feabb384af5c7ec3dc4f
Flight SQL Ratification Based On Community Feedback #5
Travis is happy, fixes the problems we see with Travis in #85