unrelated files provided in collection release

[bmillemathias]

SUMMARY

ISSUE TYPE

• Bug Report

COMPONENT NAME

I just did a ansible-galaxy collection install community.kubernetes and look what was provided and found such files.

tests
    ├── output
    │   ├── bot
    │   │   ├── ansible-test-sanity-ansible-doc.json
    │   │   ├── ansible-test-sanity-future-import-boilerplate.json
    │   │   ├── ansible-test-sanity-metaclass-boilerplate.json
    │   │   ├── ansible-test-sanity-pep8.json
    │   │   └── ansible-test-sanity-validate-modules.json
    │   ├── coverage
    │   └── data
    │       └── integration-2020-01-30-20-10-01.json

and also .DStore

.DS_Store
plugins/.DS_Store
plugins/module_utils/.DS_Store
plugins/modules/.DS_Store
tests/.DS_Store
tests/integration/targets/kubernetes/.DS_Store

ANSIBLE VERSION

N/A

CONFIGURATION

N/a

OS / ENVIRONMENT

N/a

[bmillemathias]

I don't know if it is possible to ignore directories and files when doing a ansible-galaxy collection publish

[geerlingguy]

@bmillemathias yikes! I did it from my Mac, which is why the .DS_Store files were there. And I had run some tests, which is likely why those other files were there. But we should definitely ignore things like .DS_Store files and test artifacts when building and publishing (no matter what system it's done from).

I'm looking at the docs to see if there's a way to ignore things (e.g. similar to gitignore).

[geerlingguy]

Hmm... from the Building collections docs:

Certain files and folders are excluded when building the collection artifact. This is not currently configurable and is a work in progress so the collection artifact may contain files you would not wish to distribute.

[geerlingguy]

Here's one way to do it—basically a manual pruning of everything pre-release: https://github.com/mattclay/ansible-collection-aws/blob/master/Makefile

I could run git clean -x -d -n (without -n to actually do it, add -f to force it) but I'm not 100% sure if that will remove hidden files and directories too. [Edit: It does]

[geerlingguy]

Opened upstream issue: ansible/ansible#67543

I also just found ansible/ansible#67130 which seems to suggest there's a build_ignore key in galaxy.yml that might work for this purpose? Maybe... but it doesn't seem to cover globbing and such.

[s-hertel]

I think the documentation is out-of-date. Globbing works for build_ignore. As the issue above indicates, things required for building the MANIFEST.json don't currently get excluded.

build_ignore docs in the default galaxy.yml template:

# A list of file glob-like patterns used to filter any files or directories that should not be included in the build
# artifact. A pattern is matched from the relative path of the file or directory of the collection directory. This
# uses 'fnmatch' to match the files or directories. Some directories and files like 'galaxy.yml', '*.pyc', '*.retry',
# and '.git' are always filtered

Edit - Oh, the docs are actually right if you look at docs/docsite/rst/dev_guide/developing_collections.rst. They just haven't been updated on the docs site.

[geerlingguy]

@s-hertel thanks for the update; I think the fix, then, is to make sure whomever is building the artifact uses devel (until it's in a public release) and we should add at least the test files and the macOS files to the build_ignore setting in galaxy.yml.

(Long term, I would like to make it so the releases are built in CI as part of a tag + release process—but that requires a key to be stored somewhere and right now the only key I have is my personal Galaxy key, which I'd rather not upload into GitHub's secret store.

[geerlingguy]

Opened related issue #43

[geerlingguy]

PR for this: #85