pipreqs future

[alan-barzilay]

Hi! I'm the new and (afaik) sole active maintainer of pipreqs. As you may have noticed, this package has been kinda abandoned in the last couple years. Well, I intend to change this and I would like to share my plan/vision with the community.

Pipreqs current state

I've been closing a few issues and merging some PR's and I've noticed that most of them are quite old and/or duplicated. I've tried engaging with older issues but (understandably) I've been mostly ignored since people have probably moved on and don't remember or care about a 3 years old issue. Apart from these older issues there quite a few duplicated issues which can be divided in feature requests (e.g. jupyter support) and mappings (e.g. package foobar is missing from my requirements).
The mapping problems are by far the majority of issues being reported, but attacking them feels like a wack-a-mole game where we just can't win and they just keep piling up.

Where do we go from here?

The time I can dedicate to this project is limited since I have a thesis to write and other obligations in life, so instead of addressing the symptoms I would like to attack the source of our problems, i.e. our mapping system.

To be honest I'm not yet sure of the best way to address this, I think there are a few easy improvements that could be made but they are mostly band-aids, ideally we would completely remove the mapping file since this is a solution that does not scale very well.
I'm open to suggestions on how to deal with this, but in the mean time I think I will make a PR template for adding package remapping and improve the documentation regarding this issue so users don't have to look at the source code to figure out why they have a missing package.

These are my goals for the next release of pipreqs:

• Add =~ and >= support
• Drop support for python 2
• Update pipreqs to python 3.9 (add missing standard lib packages, bump travis testing versions, etc)
• Add support for .ipynb and .pyw files
• Add FAQ for common problems with package mapping
• Get rid of the mapping system (this may be pushed to a second release if this takes to long)

I will mostly dedicate my time to implement these changes, I won't be looking too much at the open issues and PR's made so I'm sorry if you end up being ignored. All development will be done in the next branch in order to keep the master branch synchronized with the latest release.

PS: I know there are a few open PR's that tackle some of these issues, but I have a few issues with them. I will contact the authors with my grievances and suggestions but some of them are quite old and I doubt they are still interested, nevertheless I will probably use them as a starting point.

[kumiDa]

ideally we would completely remove the mapping file since this is a solution that does not scale very well.

+1 to this suggestion.

Let me know where I can be of help to you in tackling these goals.

[alan-barzilay]

@rahul-kumi any insights on how to get rid of the mappings file? I'm still struggling to decide on the best approach
Issue #76 may have some relevant discussion, but it just reforms the mapping scheme to be more complicated so I'm not particularly fond of its approach.

[ivanlen]

Hey, I've made a wrapper of pipreqs, pipreqsnb that supports .ipynb , single .py files and folders with mixed filetypes, as target and is fully functional.

If you want we can talk and I can prepare some PRs to integrate the single file and .ipynb functionality into pipreqs.

[kumiDa]

I think adding support to .ipynb would be a really nice thing to have in pipreqs.
What are your thoughts @alan-barzilay?

[alan-barzilay]

@ivanlen thank you for your kind offer, but I was thinking about doing something more in line with PRs #210 #229 for notebook support

[ivanlen]

@alan-barzilay , yeah those PRs look more aligned to the project rather than the wrapper that I've made. Still, if you need some help with the ipynb implementation or in other stuff let me know that I will be more than happy to contribute.
Cheers!

[alan-barzilay]

Hey guys, just a quick update. I got selected for the Google Summer of Code 2021 and because of that I no longer have time to dedicate myself to this project. Once GSoC ends I intend to get back to maintaining pipreqs, so I'm not abandoning the project forever.

I would love to finish the ipynb support and make a new release before GSoC starts but I can't guarantee this will happen.
Thank you for your understanding.

[igorkf]

Hi! I think I can try to contribute too.

[mapattacker]

Hi @alan-barzilay, thanks for following up among your busy schedule! Still looking forward to point 1 to be released hehe~ All the best to GSoC too

[alan-barzilay]

hello hello hello everybody, how have you been?

GSoC officially ended a few days ago and I'm back!

While I was gone travis changed its pricing strategy and isn't a viable solution for this project anymore. My top 2 priorities right now are porting our CI pipeline to github actions and making a new release with the work we have so far.

Once this is done I will get back to working on the opened PRs and on the points previously raised in this issue (beginning with the jupyter support).

That's all I have for now, just wanted to let the community know I'm back and to expect new releases (hopefully) soon

[PatMyron]

Manual mapping replacement?
replit/upm#4

[perklet]

Get rid of the mapping system (this may be pushed to a second release if this takes to long)

Are there any efforts on transforming this lib into a way that ultilizes abstract syntax tree and PyPI metadata?

[alan-barzilay]

@yifeikong sorry for the delayed response. Do you have a particular suggestion in mind? we currently do use ASTs iirc. This particular issue hasn' t progressed all that much because I haven' t had the time to explore it nor have I seen/thought of a clear fix for it. I want to take a look at the suggestion raised by @PatMyron but when I skimmed the project I recall thinking it wasnt exactly what I was looking for

[amit-chaudhari1]

@alan-barzilay I saw the about section of this repository mention that it is looking for maintainers. Is this project still looking for maintainers?