REFACTOR-#2957: Restructure project files #3210
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #3210 +/- ##
===========================================
- Coverage 85.66% 60.70% -24.97%
===========================================
Files 150 178 +28
Lines 15969 15450 -519
===========================================
- Hits 13680 9379 -4301
- Misses 2289 6071 +3782
Continue to review full report at Codecov.
|
|
|
|
|
this doesn't move files or anything, why would we want a |
YarShev
reviewed
Jul 5, 2021
There was a problem hiding this comment.
@vnlitvinov , I guess @devin-petersohn wanted to move the structure discussion from the issue to a PR to simplify the structure correction not commenting it each time.
YarShev
reviewed
Aug 24, 2021
@vnlitvinov to help see the overall structure before we move anything. github diff is not super great at showing large restructure like this |
YarShev
reviewed
Sep 2, 2021
devin-petersohn
commented
Sep 2, 2021
YarShev
reviewed
Sep 6, 2021
project_structure.txt
Outdated
| .test/ | ||
| .rest files | ||
| .core/ | ||
| .storage_formats/ |
There was a problem hiding this comment.
Doing the refactor one thought came in my mind. I see three options:
- Leave as it is now
- Put
storage_formatsalong withexecutionindataframe. - Move
executionto the higher level to reside it along withstorage_formatsincore.
I am more inclined towards the third option. What do you think would be the best choice?
There was a problem hiding this comment.
3rd option is currently applied.
There was a problem hiding this comment.
Yes, the 3rd option makes the most sense to me
|
Track current stage of the project structure in |
YarShev
force-pushed
the
refactor/2957
branch
4 times, most recently
from
242e89e to
325f168
Compare
|
@modin-project/modin-core , @modin-project/modin-omnisci , reminder to review. |
|
@YarShev does Teamcity need to be updated? |
|
@devin-petersohn , TeamCity failed (upon timeout) because conda takes a huge time to create an environment. We should merge #3409 first to be unblocked to run tests there. |
YarShev
force-pushed
the
refactor/2957
branch
from
5eeb2f9 to
193a873
Compare
|
@YarShev @devin-petersohn where are we with this? I don't think we need to polish the namings much longer, we can always re-do them partially later when desired (as we haven't hit the I have scanned over the |
@vnlitvinov , current stage of the project structure is about what we wanted to reach. It might be required to introduce some local changes based on potential reviews. I created two issues to polish the code and the docs (#3412 and #3413, respectively) that should be resolved in 0.11.
+1 to default2pandas |
YarShev
force-pushed
the
refactor/2957
branch
from
bea2682 to
193a873
Compare
Signed-off-by: Devin Petersohn <[email protected]>
Signed-off-by: Igoshev, Yaroslav <[email protected]>
Signed-off-by: Igoshev, Yaroslav <[email protected]>
Signed-off-by: Igoshev, Yaroslav <[email protected]>
Signed-off-by: Igoshev, Yaroslav <[email protected]>
Signed-off-by: Igoshev, Yaroslav <[email protected]>
YarShev
force-pushed
the
refactor/2957
branch
from
193a873 to
193e872
Compare
vnlitvinov
reviewed
Sep 10, 2021
| # ANY KIND, either express or implied. See the License for the specific language | ||
| # governing permissions and limitations under the License. | ||
|
|
||
| """Modin's functionality related to dispatching to specific execution.""" |
There was a problem hiding this comment.
I would like this to be a little more elaborate
There was a problem hiding this comment.
I would like to hear an example :) but this should go in a separate issue/PR regarding cosmetic changes.
|
@devin-petersohn can I fix conflicts myself? UPD: since I did not find commits in your branch that would fix conflicts, added them myself. |
Signed-off-by: Devin Petersohn <[email protected]>
Signed-off-by: Devin Petersohn <[email protected]>
Signed-off-by: Devin Petersohn <[email protected]>
YarShev
reviewed
Oct 1, 2021
There was a problem hiding this comment.
Mostly fine to me. There are two comments/questions from my side.
- We should probably move
modin/core/execution/ray/implementations/pandas_on_ray/modin_aqp.py
to
modin/core/execution/ray/generic/modin_aqp.py
or
modin/core/execution/ray/common/modin_aqp.py
- We should probably move the following files:
modin/experimental/core/execution/native/implementations/omnisci_on_native/dataframe/calcite_algebra.py
modin/experimental/core/execution/native/implementations/omnisci_on_native/dataframe/calcite_builder.py
modin/experimental/core/execution/native/implementations/omnisci_on_native/dataframe/calcite_serializer.py
modin/experimental/core/execution/native/implementations/omnisci_on_native/dataframe/df_algebra.py
modin/experimental/core/execution/native/implementations/omnisci_on_native/dataframe/expr.py
modin/experimental/core/execution/native/implementations/omnisci_on_native/dataframe/omnisci_worker.py
to a higher lever or somewhere else. Any options?
...xperimental/core/execution/native/implementations/omnisci_on_native/partitioning/__init__.py
Outdated
Show resolved
Hide resolved
.../experimental/core/execution/native/implementations/omnisci_on_native/dataframe/partition.py
Outdated
Show resolved
Hide resolved
|
Moved |
YarShev
force-pushed
the
refactor/2957
branch
3 times, most recently
from
0cc0980 to
1fe61d8
Compare
YarShev
force-pushed
the
refactor/2957
branch
from
1fe61d8 to
7596c88
Compare
YarShev
force-pushed
the
refactor/2957
branch
from
e7d45e0 to
dfe2d8f
Compare
|
@devin-petersohn , @vnlitvinov , reminder to review |
devin-petersohn
commented
Oct 12, 2021
This was referenced Dec 6, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Devin Petersohn [email protected]
What do these changes do?
flake8 modinblack --check modingit commit -sdocs/developer/architecture.rstis up-to-date