Is it possible to fetch a preview of a dataset from a cloud source?

[EsaFazal72]

I'm building a data connector to import datasets from cloud sources such as AWS, Azure, GCP and etc. I would like to show the user a preview of the data prior to importing that dataset. Currently, I believe Tablesaw downloads the entire dataset temporarily and then fetches columns and the required number of rows. This particularly takes a very long time to fetch if the dataset is big. Is there a way around this to only fetch a preview of the dataset instead of downloading the whole dataset. Maybe a Java-based library that does this? Any help is highly appreciated. Thank you inadvance.

[lwhite1]

For remote access we use a Stream-based API and, AFAIK, there is no way to retrieve a preview. If you had a way to stream the first n rows of the file that would work. IDK off the top of my head how to do that. For a local file, you could do something like this: String fileName = "/Users/me/Desktop/file.txt"; try (Stream<String> stream = Files.lines(Paths.get(fileName))) { stream.limit(10).forEach(System.out::println); } catch (IOException e) { e.printStackTrace(); }

…

On Wed, Jul 14, 2021 at 9:39 AM EsaFazal72 ***@***.***> wrote: I'm building a data connector to import datasets from cloud sources such as AWS, Azure, GCP and etc. I would like to show the user a preview of the data prior to importing that dataset. Currently, I believe Tablesaw downloads the entire dataset temporarily and then fetches columns and the required number of rows. This particularly takes a very long time to fetch if the dataset is big. Is there a way around this to only fetch a preview of the dataset instead of downloading the whole dataset. Maybe a Java-based library that does this? Any help is highly appreciated. Thank you inadvance. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#960>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2FPAUFEFSWBMATMCNU6JLTXWHRRANCNFSM5ALPRXNA> .