Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make boto3 dependency optional/soft for cases where only PureS3Path is needed #122

Open
echernyavskiy opened this issue Feb 15, 2023 · 5 comments

Comments

@echernyavskiy
Copy link

Looking at the code it seems PureS3Path does not need anything from boto3.
However boto3 is a hard requirement for s3path. This is problematic for environments like AWS Lambda Python runtime that has boto3 "built-in".
Could boto3 be a soft dependency like awacs for troposphere for example?

@liormizr
Copy link
Owner

Hi @echernyavskiy
Thanks for the issue :-)

I don't full understand your use case
in AWS Lambda Python like you say you already have boto3 built-in
So when you do import from s3path you don't have an issue.
You want to install s3path without "reinstalling" boto3 again? is this your use case?

@echernyavskiy
Copy link
Author

@liormizr, right - since boto3 is "built into" the runtime, I don't have to define it as a dependency in my pyproject.toml / requirements.txt. Except s3path requires it, so it gets installed into the venv anyway. Could it be a soft dependency instead?

@liormizr
Copy link
Owner

Hi @echernyavskiy,
Sorry for the delay.

We have an issue to move boto3 from the dependencies.
Basically it means that all the users of this package will need to do change the way that thy install the package and that is a big change.

Now I don't see a nice way to exclude dependency.
If you have an idea how to do so I'm open for suggestions
Also I see that AWS do recommend to specify your boto3 version so there won't be surprises if AWS decide to change boto3 version.

@plondino
Copy link

plondino commented May 20, 2024

There does seem to be a lively discussion about this here: https://discuss.python.org/t/optional-dependency-groups-omitting-package-requirements/20833/1 - I agree that this would be useful. I have some libraries that need to manipulate both local filesystem paths and S3 paths, and the PureS3Path functionality would be extremely useful, but adding the full boto3 library as a required dependency is a non-starter since many consumers won't even use the AWS functionality.

@liormizr
Copy link
Owner

@plondino

Boto3 is currently a hard requirement
How ever in case you are using only PureS3Path it's not going to be loaded.
We changed it so it will be lazy imported only when using S3Path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants