Implement unique function

[izveigor]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Part of apache/datafusion#7289

Describe the solution you'd like
unique function removes duplicate values (include NULLs) from Array. Does preserve the original order.

Example (PyArrow):

>>> pc.unique(pa.array([1, 2, 1, 4, 5, 2, 4, 1]))
<pyarrow.lib.Int64Array object at 0x7f849d1f2040>
[
  1,
  2,
  4,
  5
]

with nulls:

>>> pc.unique(pa.array([1, 2, 1, 4, None, 5, 2, None, 4, 1]))
<pyarrow.lib.Int64Array object at 0x7f849d1f2040>
[
  1,
  2,
  4,
  null,
  5
]

Describe alternatives you've considered

Additional context
Documentation: https://arrow.apache.org/docs/cpp/compute.html

[tustvold]

I'm not sure we typically provide these sorts of grouping/aggregation functions in arrow-rs? How is it you intend to integrate such a function with the grouping machinery in DataFusion?

[izveigor]

Hello, @tustvold!
I created the ticket as I found the similar syntax in other versions of Apache Arrow.
If you consider this function is redundant it is better to implement the functionality in Arrow Datafusion and close the ticket.

[tustvold]

What do you think of implementing this in DataFusion first, and we can then assess whether we upstream a version of it?