Deal with, document dependencies issue, add first README.md #22
Conversation
vinceatbluelabs
changed the title
| @@ -24,7 +24,7 @@ commands: | |||
| . venv/bin/activate | |||
| # venv/ dir doesn't seem to save enough info to keep the | |||
| # editable installation | |||
| pip install --progress-bar=off -e . | |||
| pip install --progress-bar=off -e '.[movercli]' | |||
There was a problem hiding this comment.
Whoops! This wasn't the same as the pip install below and it should have been. This may speed up our tests a touch.
| export CFLAGS | ||
| LDFLAGS="-L$(brew --prefix openssl)/lib" | ||
| export LDFLAGS | ||
| fi |
There was a problem hiding this comment.
I tested this and this is no longer needed with psycopg2-binary!
vinceatbluelabs
changed the title
| @@ -39,8 +39,19 @@ | |||
| }, | |||
| install_requires=[ | |||
| 'boto>=2,<3', 'boto3', | |||
| 'jsonschema', 'timeout_decorator', 'awscli', | |||
| 'PyYAML', 'psycopg2', | |||
There was a problem hiding this comment.
See notes in INSTALL.md for why I think dropping psycopg2 here is the right change.
|
|
||
| ### pyarrow | ||
|
|
||
| `pyarrow` is a Python wrapper around the Apache Arrow native library. |
There was a problem hiding this comment.
Hmm... is it worth making a mover-cli target that doesn’t include this? I guess it might depend on how hard it is to install these libraries in general. Should we maybe include steps for homebrew and whatever Linux distro we have already figured out for CI/docker?
There was a problem hiding this comment.
Hmm... is it worth making a mover-cli target that doesn’t include this
No objections here.
Should we maybe include steps for homebrew and whatever Linux distro we have already figured out for CI/docker?
That'd be great! I don't have any off-the-shelf to drop in. Same feeling on not holding up this PR, but if I wasn't super clear about it, having spaces for those sorts of instructions are exactly why I added this file.
INSTALL.md
Outdated
| for local Parquet manipulation). | ||
| * `pip3 install records-mover[gsheets]` - Minimal install plus API | ||
| libraries to access Google Sheets. | ||
| * `pip3 install records-mover[mover-cli]` - Install everything and |
There was a problem hiding this comment.
Is it mover-cli or movercli? I’ve seen it both ways in the docs (will flag)
There was a problem hiding this comment.
movercli. Where else did you see mover-cli in the docs?
README.md
Outdated
|
|
||
| # This is a SQLAlchemy database engine. | ||
| # | ||
| # You can instead call job_context.get_db_engine('cred name'). |
There was a problem hiding this comment.
Some old uses of job_context in here
There was a problem hiding this comment.
Fixed; that's all that grep found.
Changes: