overly sized ebird data

[peymanmortazavi]

It took me 50 minutes to import the data in the MongoDB. I think it would've been more productive and efficient if the size of the file was more reasonable considering that we need do these things on our laptops not on servers capable of handling real big data tasks.

[dawsbot]

Same thing happened to me. Not enough space to do this assignment on my computer or my roomates'. Going to take the grade hit on this one, we all gave it a fair share of effort.

[ianks]

Make sure you give your VM a hunk of memory if you have it. As far as converting the data goes, you also need to pay attention to things like spatial locality etc, or you will just constantly evicting blocks from cache, and doing too many memory reads.

[dawsbot]

Can't allocate more memory since neither Peyman nor I are in a VM. Natively
I have already allocated my partition the maximum amount. He is also
running natively and has reached max capacity on his SSD.
ᐧ

Dawson Botsford

http://hackcu.org/

Dawson Botsford | *Co-Founder HackCU
*[email protected] [email protected] | 865-804-9305 |
dawsonbotsford.com

[image: Facebook] https://www.facebook.com/dawson.botsford [image:
Twitter] https://twitter.com/DawsonBotsford [image: Linkedin]
https://www.linkedin.com/pub/dawson-botsford/9a/91b/94b

On Sun, Nov 9, 2014 at 6:29 PM, Ian Ker-Seymer [email protected]
wrote:

Make sure you give your VM a hunk of memory if you have it. As far as
converting the data goes, you also need to pay attention to things like
spatial locality etc, or you will just constantly evicting blocks from
cache, and doing too many memory reads.

—
Reply to this email directly or view it on GitHub
#13 (comment)
.