Add Apache Airflow classifiers

[ashb]

Request to add a new Trove classifier.

The name of the classifier(s) you would like to add:

• Framework :: Apache Airflow
• Framework :: Apache Airflow :: Provider

Why do you want to add this classifier?

Apache Airflow is a tool to manage workflows (usually data processing related), and for 1 year now it has been split in to "core" (the scheduler, etc) and "providers", which are bindings between Airflow and other tools in the (data) ecosystem.

It currently has over 60 modules on PyPi and (many released by the core team, but not all) and having a classifier would be a great wat to group these all together and improve discoverability.

https://pypi.org/search/?q=%22apache-airflow-providers-%22&o= shows some of the current providers, but misses others such as https://pypi.org/project/airflow-dbt/

A classifier would be a great way to tie all of these plugins together.

I have asked for the "Providers" sub-classifier as we have the scope for other pluggable components (but no one has published any for now), so for future proofing it would be good to have Framework :: Apache Airflow :: Provider classification as well.

[ashb]

Black no likey. One moment.

[ashb]

(web editor left trailing white space)

[pradyunsg]

Hi! Thanks for the request. Because removing classifiers is very difficult, and the longer the classifier list gets the harder it is to browse, the bar to add new ones is relatively high. Currently, PyPI moderators primarily examine two guidelines for Framework classifiers:

• New Framework classifiers should be created if they are "notable" and
• They will be of immediate use to existing projects (in other words, no creating classifiers simply in anticipation of future uses)

We understand that "notability" is subjective and we'd appreciate help judging that. Personally, I'm pretty sure Airflow meets this criterion.

As to the other guideline ('immediate use to existing projects'), the best way we've found to demonstrate that is to have 10 or more other project maintainers submit comments on the requesting issue about their desire to use the classifier. Is that something you could assist with?

[pradyunsg]

Ah, I just noticed that this doesn't have a corresponding issue. Comments directly on this PR would work as well. :)

[ashb]

Yup, I'll chase down some people to leave an appropriate comment here.

[potiuk]

First comment : I confirm It is desired for the whole community.

I am an Apache Airlfow community maintainer and I have been responsible as release manager for betwee 20 and 70 provider packages for Airflow every month for the last 10 months or so and classifiers would be very useful to discover those packages.

My profile at PyPI; https://pypi.org/user/potiuk/

[PedramNavid]

+1! This would be useful! We maintain an Apache Airflow provider at Hightouch and discoverability of providers would be beneficial to us and the larger Airflow community.

[kaxil]

I am one of the release managers of Apache Airflow and sometimes release adhoc providers and I can attest that this will be very useful for Airflow users

My profile: https://pypi.org/user/kaxil/

[itaiad200]

+1
At Treeverse we built the lakeFS airflow provider and this can improve the discoverability of the package for airflow users.

[talagluck]

Agree that this would be useful for us as well. We maintain an Apache Airflow provider for use with Great Expectations and believe that this discoverability of providers would be beneficial for the community

[PubChimps]

At Fivetran working on the Fivetran Airflow provider and would like to see this added as well

[RonSherfey]

Apache framework Airflow update

[dylanbstorey]

I maintain the Kafka Airflow provider and would like to see this added.

[potiuk]

Is it enough of a critical mass :) ?

[kaxil]

We also now have https://github.com/astronomer/astronomer-providers that would benefit from it.

[pradyunsg]

This should be on PyPI in ~24-48 hours; once we get a dependabot update PR on warehouse and that's landed + deployed. :)

[kaxil]

Awesome, thanks

[potiuk]

Fantastic! thanks !

[potiuk]

Almost want me to make an RC2 of the 70+ providers that are being voted right now :)