Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sync #1 #4

Merged
merged 172 commits into from
Feb 18, 2021
Merged

Sync #1 #4

merged 172 commits into from
Feb 18, 2021

Conversation

aregm
Copy link
Owner

@aregm aregm commented Feb 18, 2021

What do these changes do?

  • commit message follows format outlined here
  • passes flake8 modin
  • passes black --check modin
  • signed commit with git commit -s
  • Resolves #?
  • tests added and passing

anmyachev and others added 30 commits October 20, 2020 16:36
@anmyachev
FIX-#2195: fix describe error for datasets with datetimes (#2272)
3fd16a9 
* FIX-#2195: fix describe error for datasets with datetimes

Signed-off-by: Anatoly Myachev <[email protected]>

* FIX-#2195: add test

Signed-off-by: Anatoly Myachev <[email protected]>

* FIX-#2195: enable fix

Signed-off-by: Anatoly Myachev <[email protected]>

* FIX-#2195: Update modin/pandas/test/dataframe/test_reduction.py

Co-authored-by: Dmitry Chigarev <[email protected]>

Signed-off-by: Anatoly Myachev <[email protected]>
@dchigarev
FIX-#1906: fixed incorrect behaviour of 'groupby.__getattr' (#2276)
c680077 
Signed-off-by: Dmitry Chigarev <[email protected]>
@dchigarev
FIX-#2277: applied Title Case to the names of DATASET_SIZE_DICT keys (#…
39d69ab 
…2278)

Signed-off-by: Dmitry Chigarev <[email protected]>
@anmyachev
FIX-#2280: use 32 bytes in secrets.token_hex (#2286)
544de0d 
Signed-off-by: Anatoly Myachev <[email protected]>
@anmyachev
TEST-#2260: use recommended pandas testing api (#2273)
b514d6f 
* TEST-#2260: use recommended pandas testing api

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2260: replace getSeriesData with test_data

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2260: remove assert_categories_equal

Signed-off-by: Anatoly Myachev <[email protected]>
@dchigarev
FIX-#2254: handling dict functions at groupby.agg improved (#2267)
64b94f5 
Signed-off-by: Dmitry Chigarev <[email protected]>
@ienkovich
FEAT-#2282: support DataFrame.[count|max|min|sum] for OmniSci backend (…
6697e05 
…#2283)

Signed-off-by: ienkovich <[email protected]>
@dchigarev
FIX-#1976: indices matching at reduction functions fixed (#2270)
55e3459 
Signed-off-by: Dmitry Chigarev <[email protected]>
@ienkovich
FEAT-#2299: support value_counts in OmniSci backend. (#2300)
60d4956 
Signed-off-by: ienkovich <[email protected]>
@prutskov
FIX-#1765: Fix support of s3 in read_parquet (#2287)
a0a3ebb 
Signed-off-by: Alexey Prutskov <[email protected]>
@dchigarev
FIX-#2285: Default to pandas warning message improved (#2302)
e63c042 
Signed-off-by: Dmitry Chigarev <[email protected]>
@ienkovich
FEAT-#2303: fix OmniSci aggregates and add mean (#2304)
e685b8d 
Signed-off-by: ienkovich <[email protected]>
@anmyachev
FIX-#2258: return 'Commit Message formatting' topic (#2306)
a571e10 
Signed-off-by: Anatoly Myachev <[email protected]>
@prutskov
FIX-#2133 #2265: Fix binary operations for modin frames in case when …
8866ca8 
…partitioning isn't aligned (#2256)

Signed-off-by: Alexey Prutskov <[email protected]>
@devin-petersohn
FIX-#2239: Compute row index start using pandas (#2240)
a7d3093 
* FIX-#2239: Compute row index start using pandas

Signed-off-by: Devin Petersohn <[email protected]>

* FIX-#2239: Documentation

Signed-off-by: Devin Petersohn <[email protected]>

* FIX-#2239: Improve testing for case

Signed-off-by: Devin Petersohn <[email protected]>
@dchigarev
FIX-#2253: loc assignment fixed in case of (1, 1) shape frame (#2316)
5c5a8e0 
Signed-off-by: Dmitry Chigarev <[email protected]>
@dchigarev
FIX-#2311: fixed performance bottleneck at reduction operations (#2314)
5cabeb9 
Signed-off-by: Dmitry Chigarev <[email protected]>
@amyskov
TEST-#2288: Cover by tests delimiters parameters of read_csv (#2310)
ed69517 
Signed-off-by: Alexander Myskov <[email protected]>
@anmyachev
FIX-#2234: update dask_deps in setup.py (#2325)
23b4408 
Signed-off-by: Anatoly Myachev <[email protected]>
@anmyachev
FIX-#2326: move s3fs import in _read function (#2327)
e2f628c 
Signed-off-by: Anatoly Myachev <[email protected]>
@anmyachev
FIX-#2329: TypeError while creating cluster (#2330)
9d3ead8 
Signed-off-by: Anatoly Myachev <[email protected]>
@devin-petersohn
FIX-#0000: Indexing regression (#2333)
e08386b 
* FIX-#0000: Indexing regression

Signed-off-by: Devin Petersohn <[email protected]>

* FIX-#0000: Fix `loc`

Signed-off-by: Devin Petersohn <[email protected]>

* FIX-#0000: Fix DatetimeIndex

Signed-off-by: Devin Petersohn <[email protected]>

* FIX-#0000: Fix Datetime and checks

Signed-off-by: Devin Petersohn <[email protected]>
@devin-petersohn
DOCS-#2334: Add tutorials to main repo (#2335)
f9cd119 
Signed-off-by: Devin Petersohn <[email protected]>
@anmyachev
DOCS-#2193: Add contributing doc in checklist (#2216)
a11e7c9 
* DOCS-#2193: update contributing doc

Signed-off-by: Anatoly Myachev <[email protected]>
@anmyachev
REFACTOR-#2343: refactor offset, _read_rows, partitioned_file (#2344)
737ec34 
Signed-off-by: Anatoly Myachev <[email protected]>
@YarShev
FIX-#1927: Fix performance issue related to sparse attribute access (…
c86422a 
…#2318)

Signed-off-by: Igoshev, Yaroslav <[email protected]>
@YarShev @gshimansky @dchigarev
FIX-#2269: Move default_to_pandas logic from API layer to backend (#…
f1f82ee 
…2332)

* FIX-#2269: Move `default_to_pandas` logic from API layer to backend

Signed-off-by: Igoshev, Yaroslav <[email protected]>

* FIX-#2269: Added a test which calls _apply_agg_function

Signed-off-by: Gregory Shimansky <[email protected]>

* FIX-#2269: Added required arguments for groupby_agg

Moved wrap_udf_function into backend because omnisci doesn't support
executing lambdas.

Signed-off-by: Gregory Shimansky <[email protected]>

* FIX-#2269: Use correct default_to_pandas for groupby in backend,
refactor default to pandas functions in BaseQC

Signed-off-by: Igoshev, Yaroslav <[email protected]>

* FIX-#2269: Renamed new default_to_pandas_groupby function

into private function of Pandas backend because it is not used anywhere
else.

Signed-off-by: Gregory Shimansky <[email protected]>

* FIX-#2269: Fixed specification of backend

now it is possible to specify --backend=PandasOnDask,
--backend=PandasOnRay or --backend=PandasOnPython, not just
--backend=BaseOnPython.

Signed-off-by: Gregory Shimansky <[email protected]>

* FIX-#2269: Fix BaseOnPython tests

Signed-off-by: Igoshev, Yaroslav <[email protected]>

* FIX-#2269: Remove default_to_pandas_groupby

Signed-off-by: Igoshev, Yaroslav <[email protected]>

* FIX-#2269: logic of dropping 'by' moved back to API level

Signed-off-by: Dmitry Chigarev <[email protected]>

Co-authored-by: Gregory Shimansky <[email protected]>
Co-authored-by: Dmitry Chigarev <[email protected]>
@amyskov
TEST-#2292: Cover by tests Datetime Handling parameters of read_csv (#…
067b8ac 
…2336)

Signed-off-by: Alexander Myskov <[email protected]>
@prutskov
FEAT-#2271: Add implementation of groupby.shift (#2323)
776d8e2 
Signed-off-by: Alexey Prutskov <[email protected]>
@YarShev
FIX-#2348: Fix default to pandas warnings (#2349)
5251135 
Signed-off-by: Igoshev, Yaroslav <[email protected]>
anmyachev and others added 28 commits February 4, 2021 10:36
@anmyachev
TEST-#2670: some updates to improve asv tests stability (#2671)
77d40ce 
* TEST-#2670: some updates to improve asv tests stability

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2670: fixes

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2670: data_size -> shape

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2670: use dict approach

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2670: use CpuCount when Npartitions isn't defined

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2670: fix ASV_DATASET_SIZE

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2670: update TimeSortValues

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2670: modify asv tests for using with old modin version

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2670: reply to review comments

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2670: use env variables for default values

Signed-off-by: Anatoly Myachev <[email protected]>
@anmyachev
TEST-#2686: add fillna benchmark (#2687)
16fa188 
* TEST-#2686: add fillna benchmark

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2686: reply to review comments

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2686: add inplace parameter

Signed-off-by: Anatoly Myachev <[email protected]>
@anmyachev
TEST-#2692: add drop benchmark (#2693)
0f4ce8a 
* TEST-#2692: add drop benchmark

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2692: add one column case

Signed-off-by: Anatoly Myachev <[email protected]>
@devin-petersohn
FIX-#2688: Update ray.ObjectID to ray.ObjectRef for Ray 2.0 (#2695)
5990e06 
* FIX-#2688: Update ray.ObjectID to ray.ObjectRef for Ray 2.0

Resovles #2688

Signed-off-by: Devin Petersohn <[email protected]>

* FIX-#2688: Address comments

Signed-off-by: Devin Petersohn <[email protected]>
@dchigarev
TEST-#2707: add lint check for ASV benchmarks (#2708)
3a31683 
Signed-off-by: Dmitry Chigarev <[email protected]>
@anmyachev
TEST-#2699: add append benchmark (#2700)
7525f05 
Signed-off-by: Anatoly Myachev <[email protected]>
@prutskov
FIX-#2684: Add method level docs for Modin XGBoost (#2685)
c637d89 
Signed-off-by: Alexey Prutskov <[email protected]>
@anmyachev
TEST-#2694: add head benchmark (#2696)
b93b879 
* TEST-#2694: add head benchmark

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2694: add small number for head op

Signed-off-by: Anatoly Myachev <[email protected]>
@dchigarev
TEST-#2705: add 'value_counts' benchmarks (#2706)
e74e012 
* TEST-#2705: add 'value_counts' benchmarks

Signed-off-by: Dmitry Chigarev <[email protected]>

* TEST-#2705: apply suggestions from review

Signed-off-by: Dmitry Chigarev <[email protected]>
@dchigarev
FIX-#2709: fixed typo in '_copartition' (#2710)
8a50c4a 
Signed-off-by: Dmitry Chigarev <[email protected]>
@YarShev @prutskov
@devin-petersohn @dchigarev
FIX-#2596: Update pandas version to 1.2.1 (#2600)
5cb3283 
Co-authored-by: Alexey Prutskov <[email protected]>
Co-authored-by: Devin Petersohn <[email protected]>
Co-authored-by: Dmitry Chigarev <[email protected]>
Co-authored-by: Devin Petersohn <[email protected]>
Signed-off-by: Igoshev, Yaroslav <[email protected]>
@anmyachev
TEST-#2690: add astype benchmark (#2691)
6f1fe69 
* TEST-#2690: add astype benchmark

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2690: add category dtype; use df.types

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2690: add case with one column

Signed-off-by: Anatoly Myachev <[email protected]>
@anmyachev
TEST-#2702: add loc/iloc benchmark (#2703)
a54875a 
* TEST-#2702: add loc/iloc benchmark

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2702: add multiindex loc bench

Signed-off-by: Anatoly Myachev <[email protected]>

* TEST-#2702: add row_loc check

Signed-off-by: Anatoly Myachev <[email protected]>
@anmyachev
TEST-#2716: add describe bench (#2718)
569337b 
Signed-off-by: Anatoly Myachev <[email protected]>
@prutskov
DOCS-#2717: Fix version of Modin for building latest docs (#2719)
a1238a0 
Signed-off-by: Alexey Prutskov <[email protected]>
@abykovsk
FEAT-#1611: Add mod operation (#2726)
ba82360 
Signed-off-by: Alina <[email protected]>
@anmyachev
TEST-#2725: add index, columns, shape benchmarks (#2727)
f14a3c1 
Signed-off-by: Anatoly Myachev <[email protected]>
@dchigarev
FIX-#2305: fix handling of renaming aggregation (#2732)
04cd912 
Signed-off-by: Dmitry Chigarev <[email protected]>
@dchigarev
FIX-#2362: fix key handling in 'Series.__setitem__' (#2731)
a2ecf31 
Signed-off-by: Dmitry Chigarev <[email protected]>
@amyskov @anmyachev
TEST-#2722: add ASV read_csv skiprows benchmark (#2724)
33a57e3 
* TEST-#2722: add ASV read_csv skiprows benchmark

Co-authored-by: Anatoly Myachev <[email protected]>
Signed-off-by: Alexander Myskov <[email protected]>
@dchigarev
FIX-#2735: move '.reindex' logic about axis dispatching from the base…
7935c59 
… class (#2736)

Signed-off-by: Dmitry Chigarev <[email protected]>
@dchigarev
TEST-#1496: add tests for setting new column with different from fram…
aa818f5 
…e length (#2733)

Signed-off-by: Dmitry Chigarev <[email protected]>
@amyskov
REFACTOR-#2739: io tests refactoring (#2740)
8ebbad9 
Signed-off-by: Alexander Myskov <[email protected]>
@dchigarev
TEST-#2753: add GroupBy benchmarsk with huge amount of groups (#2754)
61c6e99 
Signed-off-by: Dmitry Chigarev <[email protected]>
@dchigarev
FIX-#2362: fix handling slices in 'DataFrame.__setitem__' (#2741)
1f3b514 
Signed-off-by: Dmitry Chigarev <[email protected]>
@dchigarev
FIX-#2742: fix performance degradation for dictionary GroupBy aggrega…
ad0bcab 
…tion (#2743)

* FIX-#2742: changed callable functions to its names in dict aggregation

Signed-off-by: Dmitry Chigarev <[email protected]>

* FIX-#2742: commends added

Signed-off-by: Dmitry Chigarev <[email protected]>
@amyskov @anmyachev
FIX-#2737: fix handling of dates for read_csv with OmniSci backend (#…
84c9ab5 
…2738)

Co-authored-by: Anatoly Myachev <[email protected]>
Signed-off-by: Alexander Myskov <[email protected]>
@devin-petersohn
DOCS-#2584: Add CODEOWNERS file (#2759)
d1321f5 
* Resolves #2584

Signed-off-by: Devin Petersohn <[email protected]>
@aregm
Copy link
Owner Author

aregm commented Feb 18, 2021

Sync

@aregm aregm merged commit 62bbe39 into aregm:master Feb 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.