This repository has been archived by the owner on Feb 15, 2022. It is now read-only.
Genetic algorithm to fine tune strategy parameters #299
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,97 @@ | ||
| FROM node:latest | ||
|
|
||
|
|
||
|
|
||
| # NOTE: THIS DOCKERFILE IS GENERATED VIA "update.sh" | ||
| # | ||
| # PLEASE DO NOT EDIT IT DIRECTLY. | ||
| # | ||
|
|
||
|
|
||
| # ensure local python is preferred over distribution python | ||
| ENV PATH /usr/local/bin:$PATH | ||
|
|
||
| # http://bugs.python.org/issue19846 | ||
| # > At the moment, setting "LANG=C" on a Linux system *fundamentally breaks Python 3*, and that's not OK. | ||
| ENV LANG C.UTF-8 | ||
|
|
||
| # runtime dependencies | ||
| RUN apt-get update && apt-get install -y --no-install-recommends \ | ||
| tcl \ | ||
| tk \ | ||
| && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| ENV GPG_KEY 0D96DF4D4110E5C43FBFB17F2D347EA6AA65421D | ||
| ENV PYTHON_VERSION 3.6.0 | ||
|
|
||
| # if this is called "PIP_VERSION", pip explodes with "ValueError: invalid truth value '<VERSION>'" | ||
| ENV PYTHON_PIP_VERSION 9.0.1 | ||
|
|
||
| RUN set -ex \ | ||
| && buildDeps=' \ | ||
| tcl-dev \ | ||
| tk-dev \ | ||
| ' \ | ||
| && apt-get update && apt-get install -y $buildDeps --no-install-recommends && rm -rf /var/lib/apt/lists/* \ | ||
| \ | ||
| && wget -O python.tar.xz "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz" \ | ||
| && wget -O python.tar.xz.asc "https://www.python.org/ftp/python/${PYTHON_VERSION%%[a-z]*}/Python-$PYTHON_VERSION.tar.xz.asc" \ | ||
| && export GNUPGHOME="$(mktemp -d)" \ | ||
| && gpg --keyserver ha.pool.sks-keyservers.net --recv-keys "$GPG_KEY" \ | ||
| && gpg --batch --verify python.tar.xz.asc python.tar.xz \ | ||
| && rm -r "$GNUPGHOME" python.tar.xz.asc \ | ||
| && mkdir -p /usr/src/python \ | ||
| && tar -xJC /usr/src/python --strip-components=1 -f python.tar.xz \ | ||
| && rm python.tar.xz \ | ||
| \ | ||
| && cd /usr/src/python \ | ||
| && ./configure \ | ||
| --enable-loadable-sqlite-extensions \ | ||
| && make -j$(nproc) \ | ||
| && make install \ | ||
| && ldconfig \ | ||
| \ | ||
| # explicit path to "pip3" to ensure distribution-provided "pip3" cannot interfere | ||
| && if [ ! -e /usr/local/bin/pip3 ]; then : \ | ||
| && wget -O /tmp/get-pip.py 'https://bootstrap.pypa.io/get-pip.py' \ | ||
| && python3 /tmp/get-pip.py "pip==$PYTHON_PIP_VERSION" \ | ||
| && rm /tmp/get-pip.py \ | ||
| ; fi \ | ||
| # we use "--force-reinstall" for the case where the version of pip we're trying to install is the same as the version bundled with Python | ||
| # ("Requirement already up-to-date: pip==8.1.2 in /usr/local/lib/python3.6/site-packages") | ||
| # https://github.com/docker-library/python/pull/143#issuecomment-241032683 | ||
| && pip3 install --no-cache-dir --upgrade --force-reinstall "pip==$PYTHON_PIP_VERSION" \ | ||
| # then we use "pip list" to ensure we don't have more than one pip version installed | ||
| # https://github.com/docker-library/python/pull/100 | ||
| && [ "$(pip list |tac|tac| awk -F '[ ()]+' '$1 == "pip" { print $2; exit }')" = "$PYTHON_PIP_VERSION" ] \ | ||
| \ | ||
| && find /usr/local -depth \ | ||
| \( \ | ||
| \( -type d -a -name test -o -name tests \) \ | ||
| -o \ | ||
| \( -type f -a -name '*.pyc' -o -name '*.pyo' \) \ | ||
| \) -exec rm -rf '{}' + \ | ||
| && apt-get purge -y --auto-remove $buildDeps \ | ||
| && rm -rf /usr/src/python ~/.cache | ||
|
|
||
| # make some useful symlinks that are expected to exist | ||
| RUN cd /usr/local/bin \ | ||
| && { [ -e easy_install ] || ln -s easy_install-* easy_install; } \ | ||
| && ln -s idle3 idle \ | ||
| && ln -s pydoc3 pydoc \ | ||
| && ln -s python3 python \ | ||
| && ln -s python3-config python-config | ||
|
|
||
|
|
||
|
|
||
|
|
||
| RUN apt-get update | ||
| RUN apt install -y graphviz libgraphviz-dev pkg-config | ||
| RUN PKG_CONFIG_ALLOW_SYSTEM_LIBS=OHYESPLEASE pip install pygraphviz | ||
| ADD zen/requirements.txt / | ||
| RUN pip3 install -r /requirements.txt | ||
| WORKDIR /app | ||
| RUN /usr/local/bin/npm install | ||
| RUN npm update | ||
|
|
||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| import datetime | ||
| import shlex | ||
|
|
||
| import subprocess | ||
| from fabric.api import run, cd,local | ||
|
|
||
|
|
||
|
|
||
|
|
||
| def sim(instrument, days, popsize, strategy): | ||
| with cd('zenbot'): | ||
| params = dict(instrument=instrument, days=days, strategy=strategy, popsize=popsize, | ||
| timestamp=datetime.datetime.now().strftime('%Y-%m-%d-%H-%M')) | ||
| cmd = "cd zen && python -m scoop main.py {instrument} {days} {popsize} {strategy}".format(**params) | ||
| total = '(nohup docker-compose exec -T server bash -c "{cmd}" > {instrument}_{strategy}_{days}_{popsize}_{timestamp}.out 2>&1 &) && sleep 1'.format( | ||
| cmd=cmd, **params) | ||
| print(total) | ||
| run(total) | ||
|
|
||
|
|
||
| def remote(cmd,logfile): | ||
| with cd('zenbot'): | ||
| total = '(nohup docker-compose exec -T server bash -c "{cmd}" > {logfile} 2>&1 &) && sleep 1'.format(cmd=cmd, | ||
| logfile=logfile) | ||
| print(total) | ||
| run(total) | ||
|
|
||
| def backfill_remote(TOTAL_DAYS): | ||
| products = ['gdax.BTC-EUR','gdax.BTC-USD','gdax.BTC-GBP']+['gdax.ETH-BTC','poloniex.ETH-BTC'] | ||
| for instrument in products: | ||
| cmd = '/app/zenbot.sh backfill {instrument} --days {days}'.format(days=TOTAL_DAYS, instrument=instrument) | ||
| remote(cmd,'backfill_'+instrument) | ||
| def backfill_local(TOTAL_DAYS): | ||
| products = ['gdax.BTC-EUR','gdax.BTC-USD','gdax.BTC-GBP']+['gdax.ETH-BTC','poloniex.ETH-BTC'] | ||
|
There was a problem hiding this comment. I understand why this is a constant now (for demo purposes) but if we plan on merging this in at some point, we should try and use the |
||
| for instrument in products: | ||
| cmd = '/app/zenbot.sh backfill {instrument} --days {days}'.format(days=TOTAL_DAYS, instrument=instrument) | ||
| local(cmd) | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would suggest to just create your own docker.
Also, to run this, I had to run
mkdir -p logs/hof, so perhaps aRUN mkdir -p zen/logs/hofhere around the end?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I get a weird error related to npm in docker. When you guys were building your docker, did you stumble upon this?
server_1 | module.js:471
server_1 | throw err;
server_1 | ^
server_1 |
server_1 | Error: Cannot find module 'semver'
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you are using arphenos dockerfile you need to copy at least package.json into you workdir before the npm install command.
or just copy the app into the working dir:
COPY . /app