MongoError: can't map file memory on Raspberry PI 3

[megrimm]

Hello,

Trying this out on a Raspberry PI 3. I am just learning MongoDB stuff.

I get the following error after backfilling gdax.BTC-USD & gdax.LTC-USD

MongoError: can't map file memory - mongo requires 64 bit build for larger datasets

I realize now with 32 bit OS the size of the DB is limited to 2gb... SO I did:

$ show dbs
$ use zenbot4
$ show collections
$ db.trades.remove( { selector : 'gdax.LTC-USD'} )

Now zenbot4 DB is still 1.99926GB.

If I do:

$ db.repairDatabase()

 "errmsg" : "exception: can't map file memory - mongo requires 64 fit build for larger datasets"

My questions are:
How do I reduce the size of the DB?
Is there a work around to the database file size limit?
If i just drop the DB I assume I can just backfill again BUT if I continue backfill/trading over the next few weeks I am assuming the DB will max out again?
If data > 90 days can be removed does this mean the database will just expand in size anyway unless repaired each time there is a backfill?

I thinking running zenbot on a Raspberry Pi would be awesome I am just not sure how t get around the database size limitation.

Thanks!

[carlos8f]

• Running mongo on a 32 bit machine is obviously a pretty bad idea :) It pre-acquires disk space and uses padding for performance, and I don't think you can compact it without the repair command, unfortunately. Only other option is to drop the db periodically I guess.
• One thing I'm thinking of to help the disk space issue, is having a "reaper" to periodically trim the dataset to a certain length while the trader is active. At least then it wouldn't balloon to the point where you'd be desperate to compact or drop it. I'm putting that on the list for 4.1 release.
• Another idea I had is a "lite" mode for the trader, which would be a barebones option where Mongo wouldn't be needed. You could run simulations on a different server that has mongo, then deploy the trader without needing all the data still around.

[megrimm]

these are good thoughts... i like the "lite" trader and "reaper" ideas.

i wonder what would be a better board than the RPI for this? would need 64bit OS, etc....

[carlos8f]

A guy named adifferent in our Discord channel is using Pine64 with Ubuntu and a 64GB sd card to run it and says it's working well :D

[megrimm]

hmmm... yeah just saw that on discord. Thanks!

[megrimm]

just a side note: the RPI 3 does have a 64-bit A53 ARM processor which SLES & openSUSE have distribution for yet have not tried. I assume the problems faced by the mongodb 2gb limit will cease but who knows?

[simonwestyn]

@megrimm any luck on getting this to work on a RPI 3? I'm interested in buyin one just for this

[megrimm]

not really... i gave up and am running on an old laptop now. if you give it a shot with a 64 bit distro and it works out let us know! cheers

…

On Tue, Jun 13, 2017 at 6:31 AM, selachiix ***@***.***> wrote: @megrimm <https://github.com/megrimm> any luck on getting this to work on a RPI 3? I'm interested in buyin one just for this — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#171 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAIOr49a6QHQ0wav7iN_bemQ3-iXB0OQks5sDmTxgaJpZM4NhmpR> .

--

____________________ m.e.grimm, m.f.a, ed.m. syracuse u., tc3 megrimm.net

____________________

[talvasconcelos]

i have the bot running on rpi 3...

…

On Tue, Jun 13, 2017, 12:45 PM megrimm ***@***.***> wrote: not really... i gave up and am running on an old laptop now. if you give it a shot with a 64 bit distro and it works out let us know! cheers On Tue, Jun 13, 2017 at 6:31 AM, selachiix ***@***.***> wrote: > @megrimm <https://github.com/megrimm> any luck on getting this to work on > a RPI 3? I'm interested in buyin one just for this > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#171 (comment)>, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/AAIOr49a6QHQ0wav7iN_bemQ3-iXB0OQks5sDmTxgaJpZM4NhmpR > > . > -- ____________________ m.e.grimm, m.f.a, ed.m. syracuse u., tc3 megrimm.net ____________________ — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#171 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALhf8i9aM7eko7Cp3i7iz2WQQCrhSWGnks5sDnZCgaJpZM4NhmpR> .

[pushkarnagle]

@talvasconcelos Which OS you are running on RPI 3? 32-bit or 64-bit?

[talvasconcelos]

32 bit latest jessie

…

On Sat, Jul 1, 2017, 11:52 AM pushkarnagle ***@***.***> wrote: @talvasconcelos <https://github.com/talvasconcelos> Which OS you are running on RPI 3? 32-bit or 64-bit? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#171 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALhf8v5I0U9N-7isk7Lt0PT2zufppg78ks5sJiTRgaJpZM4NhmpR> .

[jimmydeal]

@talvasconcelos I'm working on getting zenbot up and running on the rpi 3 also. I'm running 32 bit w the latest jessie same as you, but mongodb is giving me hell. How'd you get mongo running properly on your system? It works fine until I run zenbot commands that require it's use (then it crashes and won't run).

[benwa]

I know very little about MongoDB, but would a configurable TTL Index work to expire old data to keep the DB small?

[ferdiesletering]

@talvasconcelos, can you describe how you got it running on RPI 3 with jessie 32 bit? I'm sure you will help a lot of folks here:)

[jimmydeal]

Got notified by the last post, but I figured I would drop in because I was able to get Zenbot to run properly on Raspbian OS for the RPI3. First, I installed the required mongodb and nodejs, but the builds specifically meant for the RPI3 (you can find guides specific to the hardware online). Then, test mongodb to make sure that the status is running or active. After that, it was a matter of simply installing zenbot on the raspberry pi. I was able to get it to run, but it will crash mongodb if you run a long simulation. If mongodb crashes and/or the system shuts down before you manually close the process, it will not run properly on startup and you will have to troubleshoot. I no longer run zenbot, using the RPI to stake NAV now, but it was doable given time and effort.

[jimmydeal]

@s-ferdie my reply was geared to help you as much as I can given that I haven't run zenbot in ~6 months

[ferdiesletering]

@jimmydeal thanks for the quick reply! Installing works great but when starting zenbot I get the following error server_1 | standard_init_linux.go:195: exec user process caused "exec format error". Goodluck with your NAV stacking;)

[elRadix]

#1217 same issue here

[DeviaVir]

Dear issue reporter,

We have slightly changed our github issue policy and would now kindly request folks that have questions that they ask them in our zenbot subreddit.

You can find it here: https://reddit.com/r/zenbot

This issue will be closed, but if you disagree with your ticket being marked as a question feel free to leave a comment defending your case.

Thanks for contributing time and effort!

Greetings,
🤖