Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add configuration option for base Python image for Docker files #524

Merged
merged 2 commits into from
Nov 22, 2024

Conversation

alchzh
Copy link
Collaborator

@alchzh alchzh commented Nov 21, 2024

Python 3.8 is now officially EoL. I've been testing how OFRAK does with newer python versions in the maintenance/newer-python-test-fixes branch. This change just lets us build Docker images off images of newer Python versions.

One sentence summary of this PR (This should go in the CHANGELOG!)
Add configuration option for base Python image for Docker files.

Link to Related Issue(s)

Please describe the changes in your request.

  • Adds a python_image configuration option to docker yaml files. Defaults to the current 3.8-bullseye image for now.
  • Changes the pip and jefferson install to not depend on specific version

Anyone you think should look at this, specifically?

ofrak_core/Dockerstub Outdated Show resolved Hide resolved
@alchzh
Copy link
Collaborator Author

alchzh commented Nov 22, 2024

I ran into a problem where the new setuptools editable installs are broken with our directory layout. Directories in the working directory are now imported before editable installed packages, so running e.g. import ofrak_type from a folder with the parent ofrak_type directory is broken. See pypa/setuptools#3557 and https://setuptools.pypa.io/en/latest/userguide/development_mode.html for details.

$ docker run -it redballoonsecurity/ofrak/dev-3.13 bash
Using service wrapper: yajsw-beta-13.01
Installing Ghidra Server...
Traceback (most recent call last):
  File "<frozen runpy>", line 189, in _run_module_as_main
  File "<frozen runpy>", line 148, in _get_module_details
  File "<frozen runpy>", line 112, in _get_module_details
  File "/ofrak_core/ofrak/__init__.py", line 8, in <module>
    from ofrak.ofrak_context import OFRAK, OFRAKContext
  File "/ofrak_core/ofrak/ofrak_context.py", line 13, in <module>
    from ofrak_type import InvalidStateError
ImportError: cannot import name 'InvalidStateError' from 'ofrak_type' (unknown location)

I see a couple solutions to this problem, not sure which one is the best

  • add --config-settings editable_mode=compat to any pip install -e command
  • Require older setuptools in pyproject.toml
[build-system]
requires = ["setuptools == 63"]
build-backend = "setuptools.build_meta"

@rbs-jacob
Copy link
Member

I ran into a problem where the new setuptools editable installs are broken with our directory layout. Directories in the working directory are now imported before editable installed packages, so running e.g. import ofrak_type from a folder with the parent ofrak_type directory is broken.

Is this only happening because the default working directory in the Docker images we build is the root directory? Would changing the WORKDIR to /root/ as the last step in the Dockerfile assembled from Dockerstub files be an alternative way to address this?

@rbs-jacob rbs-jacob merged commit e601a5e into master Nov 22, 2024
4 checks passed
@whyitfor whyitfor deleted the docker/config-file-python-image branch November 25, 2024 10:24
@ANogin
Copy link
Collaborator

ANogin commented Dec 4, 2024

I ran into a problem where the new setuptools editable installs are broken with our directory layout. Directories in the working directory are now imported before editable installed packages, so running e.g. import ofrak_type from a folder with the parent ofrak_type directory is broken. See pypa/setuptools#3557 and https://setuptools.pypa.io/en/latest/userguide/development_mode.html for details.

$ docker run -it redballoonsecurity/ofrak/dev-3.13 bash
Using service wrapper: yajsw-beta-13.01
Installing Ghidra Server...
Traceback (most recent call last):
  File "<frozen runpy>", line 189, in _run_module_as_main
  File "<frozen runpy>", line 148, in _get_module_details
  File "<frozen runpy>", line 112, in _get_module_details
  File "/ofrak_core/ofrak/__init__.py", line 8, in <module>
    from ofrak.ofrak_context import OFRAK, OFRAKContext
  File "/ofrak_core/ofrak/ofrak_context.py", line 13, in <module>
    from ofrak_type import InvalidStateError
ImportError: cannot import name 'InvalidStateError' from 'ofrak_type' (unknown location)

I see a couple solutions to this problem, not sure which one is the best

  • add --config-settings editable_mode=compat to any pip install -e command
  • Require older setuptools in pyproject.toml
[build-system]
requires = ["setuptools == 63"]
build-backend = "setuptools.build_meta"

@alchzh There is also python/mypy#13392; I ended up using export SETUPTOOLS_ENABLE_FEATURES="legacy-editable" in [stale] https://github.com/redballoonsecurity/ofrak/pull/314/files#diff-9d7ea1b163b7b4693bbc13b0dd627154521dc56ad997cd3b1a7747e413916027

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants