Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

main: initial version of image-builder with basic --list-images #1

Merged
merged 2 commits into from
Nov 28, 2024

Conversation

mvo5
Copy link
Collaborator

@mvo5 mvo5 commented Nov 18, 2024

This commit adds the new image-builder binary (name still a bit TBD but this is the best so far IMHO). This binary is meant to build images from the CLI without the need to setup a daemon. The main use-case is CI/CD and admins running this in scripts or ad-hoc. The CLI should be pleasant to use.

This first commit adds the list-images command which is a thin wrapper around functionality from the osbuild/images library.

It will list all buildable images by default and can be trimmed down further via --filter which supports the filtering from the images library, see osbuild/images#1015

It also supports --output which will output the result in the given format. Currently "text" and "json" are supported.

Note that this will not work on it's own yet, it will need an installed image-builder to get the repositories. This will need to get fixed via either:

  1. a dependency package for image-builder that carries all the repos
  2. a shared repo that contains the repos
  3. using go:embed to get them (see images#1038)

[edit: rename from ibuilder->image-builder after the most recent discussion during the standup, while there was no consensus it seems people did like image-builder a bit better]

Part of. COMPOSER-2395

@mvo5 mvo5 requested review from ondrejbudai and thozza November 18, 2024 09:19
@mvo5 mvo5 force-pushed the add-list-image branch 8 times, most recently from 417535f to 59a4386 Compare November 18, 2024 09:59
@mvo5 mvo5 force-pushed the add-list-image branch 2 times, most recently from 80b79f4 to 39060ad Compare November 18, 2024 11:56
@mvo5 mvo5 changed the title main: initial version of ibuilder with basic --list-images main: initial version of image-builder with basic --list-images Nov 20, 2024
Copy link
Member

@ondrejbudai ondrejbudai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! I put some "high-level" thoughts inline, but nothing is blocking (well, maybe the ibuilder one). I guess we should convert them to issues/discussions at some point?

logrus.SetLevel(logrus.WarnLevel)

rootCmd := &cobra.Command{
Use: "ibuilder",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be image-builder

SilenceUsage: true,
}
listImagesCmd.Flags().StringArray("filter", nil, `Filter distributions by a specific criteria (e.g. "type:rhel*")`)
listImagesCmd.Flags().String("output", "", "Output in a specific format (text, json)")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if this should become an argument of rootCmd in the future?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A really good questions, probably yes - most verbs will have an --output switch (list-images, build). Not entirely sure about manifest though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be good to think of it as "console output". Manifests can be considered intermediary artifacts with a set format (JSON). The option could be to print them to stdout or save them into a file.

// that we keep this in sync
// XXX2: means we need to depend on osbuild-composer-common or a new rpm
// that provides the relevant packages *or* we use go:embed (cf images#1038)
var repositoryConfigs = []string{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Food for thought: Should we stop using our custom repo format, and start using the same format as dnf uses in /etc/yum.repos.d? Our depsolver should be already able to consume via the root opion. Definitely not something to address in this PR, but I think it's a good conceptual dicusssion.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no strong opinion here, I would like to centralize what we have though. I would like to be able to build all supported imagetypes from this binary, ideally without much/any external dependencies. I was thinking go:embed for this and having the list of supported reos in images and the ability to override. But there is some pushback for this idea (not sure why though, I'm probably missing some details why this would be an issue).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A dedicated RPM with repo definitions would make sense. We could split it as an osbuild-composer sub-package for the beginning. WRT the format, I see little value in using the DNF format at the moment. Our repo definitions contain more information (such as "use RHSM?", image type tags, etc.).

go.mod Outdated Show resolved Hide resolved
mvo5 added 2 commits November 21, 2024 15:22
This commit adds the new `image-builder` binary. This binary is
meant to build images from the CLI without the need to setup a
daemon. The main use-case is CI/CD and admins running this in
scripts or ad-hoc. The CLI should be pleasant to use.

This first commit adds the `list-images` command which is a thin
wrapper around functionality from the `osbuild/images` library.

It will list all buildable images by default and can be trimmed
down further via `--filter` which supports the filtering from
the `images` library, see osbuild/images#1015

It also supports `--output` which will output the result in the
given format. Currently "text" and "json" are supported.

Note that this will not work on it's own yet, it will need an
installed image-builder to get the repositories. This will need
to get fixed via either:
1. a dependency package for `ibuilder` that carries all the repos
2. a shared repo that contains the repos
3. using go:embed to get them (see images#1038)
This commit updates the GH workflow file to install the required
dependencies to build the images library.
Copy link
Member

@thozza thozza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This LGTM as the initial PR. 👍

The first commit still contains a mention of ibuilder. I've added a few comments, but none of them should be considered blockers at this stage of development.

SilenceUsage: true,
}
listImagesCmd.Flags().StringArray("filter", nil, `Filter distributions by a specific criteria (e.g. "type:rhel*")`)
listImagesCmd.Flags().String("output", "", "Output in a specific format (text, json)")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be good to think of it as "console output". Manifests can be considered intermediary artifacts with a set format (JSON). The option could be to print them to stdout or save them into a file.

// that we keep this in sync
// XXX2: means we need to depend on osbuild-composer-common or a new rpm
// that provides the relevant packages *or* we use go:embed (cf images#1038)
var repositoryConfigs = []string{
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A dedicated RPM with repo definitions would make sense. We could split it as an osbuild-composer sub-package for the beginning. WRT the format, I see little value in using the DNF format at the moment. Our repo definitions contain more information (such as "use RHSM?", image type tags, etc.).

@mvo5 mvo5 enabled auto-merge November 28, 2024 09:46
@mvo5 mvo5 disabled auto-merge November 28, 2024 09:51
@mvo5 mvo5 added this pull request to the merge queue Nov 28, 2024
Merged via the queue into osbuild:main with commit 56f9c69 Nov 28, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants