BUG: Memory exhausted (limit reached?)

[SanderDevisscher]

When uploading a large logger file we get the following error after pushing "add data".

Warning: Error in : memory exhausted (limit reached?)
  1: shiny::runApp

[SanderDevisscher]

the maximum number of records we were able to upload is 5.501.268 (approx. 511mb)

[pmallot]

Sounds like your machine is out of memory/RAM, have you checked the memory load just before the error happens?
Due to the way the app unpacks and handles the data it needs more memory that the just the file size.

Try not loading all the files at once. Rather load them in chunks, filter out the noise and save them as sqlite. Once you have done that the file size should be drastically smaller and you should be able to load all the sqlite files at once.