tsdb: Switch split16 to be a single file

[pquentin]

That way, it will be split in 16, but each part will start from beginning to end.

[martijnvg]

Thanks @pquentin! I left a question for my own understanding.
LGTM

[martijnvg]

Question: how does Rally know how to split this file into 16 parts? (and each part having the correct date range)

[pquentin]

Splitting the file in 16 is the default behavior of Rally, provided we configure it to use 16 bulk indexing clients. I rearranged the original file to make sure that each of the 16 splits was starting from beginning to end. Given the previous split16 work, all I had to do was:

cat documents-split-0.json documents-split-1.json documents-split-2.json documents-split-3.json documents-split-4.json documents-split-5.json documents-split-6.json documents-split-7.json documents-split-8.json documents-split-9.json documents-split-10.json documents-split-11.json documents-split-12.json documents-split-13.json documents-split-14.json documents-split-15.json > documents-split16.json

Does that make sense?

[martijnvg]

I see, and each of the documents-split-N.json files has an equal size, so that Rally knows exactly how to split the documents-split16.json file?

[pquentin]

Yes, each bulk indexing client will split and mmap its own part of file independently based on its id (from 0 to 15) and the number of lines (which is why we have to provide document-count).

[martijnvg]

👍