Code

File partitioning for ListingTable#1141

Code

Merged

alamb merged 18 commits intoapache:masterapache/datafusion:masterfrom rdettai:file-partitioningrdettai/arrow-datafusion:file-partitioningCopy head branch name to clipboardNov 1, 2021

+2,634-1,031

Commits

Commits on Oct 28, 2021

• [feat] adding partition_values to PartitionedFile

  rdettai

  committed
  c184efa

  Copy full SHA for c184efa

• [wip] implementing pruned_partition_list

  rdettai

  committed
  2e82902

  Copy full SHA for 2e82902

• [feat] implement pruned_partition_list

  rdettai

  committed
  cbbf347

  Copy full SHA for cbbf347

• [fix] outdated comment

  rdettai

  committed
  f74ee67

  Copy full SHA for f74ee67

• [fix] adding extra partitioning col for csv avro json

  Show description for fb15b3b
  rdettai

  committed
  fb15b3b

  Copy full SHA for fb15b3b

• [fix] houqp's review

  rdettai

  committed
  95216d8

  Copy full SHA for 95216d8

• [fix] ListingTable takes schema without partition cols

  Show description for 26bf105
  rdettai

  committed
  26bf105

  Copy full SHA for 26bf105

• [feat] added partition columns for parquet

  Show description for c34193d
  rdettai

  committed
  c34193d

  Copy full SHA for c34193d

• [feat] specify supports_filter_pushdown

  rdettai

  committed
  32c5bf6

  Copy full SHA for 32c5bf6

• [fix] parquet rowgroup pruning not kicking in

  rdettai

  committed
  10b52eb

  Copy full SHA for 10b52eb

• [test] added proper integration tests

  rdettai

  committed
  f207b92

  Copy full SHA for f207b92

• [test] check output schemas

  rdettai

  committed
  dfa6d0e

  Copy full SHA for dfa6d0e

• [test] add integ tests for partitioned parquet

  rdettai

  committed
  f8edb38

  Copy full SHA for f8edb38

• [fix] wrong schema passed to get_statistics_with_limit

  rdettai

  committed
  c5cfcfb

  Copy full SHA for c5cfcfb

Commits on Oct 29, 2021

• [fix] improvements following review

  Show description for cb0789e
  rdettai

  committed
  cb0789e

  Copy full SHA for cb0789e

Commits on Nov 1, 2021

• Merge remote-tracking branch 'apache/master' into file-partitioning

  alamb

  committed
  0303913

  Copy full SHA for 0303913

• Add GetIndexedField to the list of exprs that can be in predicates

  alamb

  committed
  5d34be6

  Copy full SHA for 5d34be6

• Update datafusion/src/datasource/listing/helpers.rs

  Show description for a4ec801
  

  rdettai

  and

  jimexist

  authored
  a4ec801

  Copy full SHA for a4ec801