Concating dictionary array leads to duplicated dict values.

[eddyxu]

Describe the bug

I was trying to concatenate a few DictionaryArrays using arrow_select::concat::concat. While the each Dictionary Array shares the same value strings. The resulted array however have all values from each DictionaryArray copied (duplicated) in the final output.

To Reproduce

let arrs: Vec<Arc<DictionaryArray<Int32Type>>> = (0..10).map(|v| {
          let mut dict_builder = StringDictionaryBuilder::<Int32Type>::new();
          dict_builder.append_null();
          dict_builder.append("a").unwrap();
          dict_builder.append("b").unwrap();
          dict_builder.append("c").unwrap();
          Arc::new(dict_builder.finish())
    }).collect();
    let mut arrays: Vec<&dyn Array> = vec![];
    for b in arrs.iter() {
        arrays.push(b.as_ref());
    }
    let b = concat(arrays.as_slice()).unwrap();
    println!("Batch is: {:?}", b);

Output

Batch is: DictionaryArray {keys: PrimitiveArray<Int32>
[
  null,
  0,
  1,
  2,
  null,
  3,
  4,
  5,
] values: StringArray
[
  "a",
  "b",
  "c",
  "a",
  "b",
  "c",
]}

Expected behavior

I'd expect the output will be

DictionaryArray {keys: PrimitiveArray<Int32>
[
  null,
  0,
  1,
  2,
  null,
  0,
  1,
   2
] values: StringArray
[
  "a",
  "b",
  "c",
]}

Additional context

N/A

[tustvold]

This is intentional, #506 tracks doing something different here