Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move Dask DataFrame read/to options under read/to keys #2821

Merged
merged 25 commits into from
Sep 4, 2020
Merged

Move Dask DataFrame read/to options under read/to keys #2821

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
0752eac
Place existing DataFrame loader configs under a "read" key
kinghuang Aug 11, 2020
e34c77d
Rename file_type/file_options to read_type/read_options
kinghuang Aug 11, 2020
8b2f7e0
Place existing DataFrame materializer configs under a "to" key
kinghuang Aug 11, 2020
22b4617
Rename file_type/file_options to to_type/to_options
kinghuang Aug 11, 2020
a7db1a6
Allow materialization to multiple formats
kinghuang Aug 11, 2020
361d08a
Move materializer after loader
kinghuang Aug 16, 2020
f880562
Fix capitalization of key option
kinghuang Aug 16, 2020
2794ffa
Fix spelling of storage_options option
kinghuang Aug 16, 2020
6ed9c08
Generate DataFrame loader config
kinghuang Aug 16, 2020
d8d6daf
Read dataframe using function in types
kinghuang Aug 16, 2020
2e7d297
Add support for read types at root of loader config
kinghuang Aug 16, 2020
dd3a5bd
Generate DataFrame materializer config
kinghuang Aug 17, 2020
9b66540
Write dataframe using function in types
kinghuang Aug 17, 2020
f68d701
Consider to specs on config root
kinghuang Aug 17, 2020
a0143ac
Remove dict_without_keys
kinghuang Aug 17, 2020
4a67240
Add filters field to read_parquet
kinghuang Aug 20, 2020
abe6966
Move input tests before output tests
kinghuang Aug 21, 2020
33d44b2
Test that results of old and new form reads are the same
kinghuang Aug 24, 2020
e89aafd
Allow permissive fields on read/to options
kinghuang Aug 24, 2020
0247745
Fix string quote issues from conversion to double quotes
kinghuang Aug 24, 2020
3fd8afb
Warn on use of read/to configs without read/to keys
kinghuang Aug 24, 2020
c89e651
Test for warnings on lack of read/to keys
kinghuang Aug 24, 2020
3ac8504
Change config options to Selector
kinghuang Aug 31, 2020
8e3630a
Note deprecated config options
kinghuang Aug 31, 2020
4eae55d
Adjust code formatting based on Black
kinghuang Sep 3, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading