Make files as the unit of task distribution for HoodieWriteClient.clean()

[kaushikd49]

When partitions have skew (some partitions having more files than others), clean takes time since the unit of task distribution is partition. Change the unit of task distribution to
be files instead, to avoid delays due to such skews.

[vinothchandar]

This needs to be done on top of #171 .. @kaushikd49 when are you planning to work on this

[kaushikd49]

@vinothchandar I am taking this up this week. I had started making some changes actually. Sure, I can make it on top of that. Any idea when that will be complete?

[vinothchandar]

Conservatively, by end of week, mostly by EOD tmrw. I strongly encourage to wait for the filegroups to be in, before you go there, coz what you need is to ultimately parallelize by file groups.

[kaushikd49]

Ok, that would be great. I'll wait for it to go in before.

[vinothchandar]

@kaushikd49 and I chatted offline and decided to decouple the efforts..