From c53283f5fc9a7bdbf727478bc5e6e32ce490524a Mon Sep 17 00:00:00 2001 From: Valentin Rothberg Date: Thu, 22 Apr 2021 09:37:54 +0200 Subject: [PATCH] libimage: follow-up changes The following changes were not split into smaller commits since the entire package is still work in progress and I want to keep moving: * Various small fixes. * The internal image cache has been removed as it's a recipe for inconsistencies for longer running processes. This should make libimage easier to use for CRI-O and a Podman service. * LookupImage now returns storage.ErrUnknownImage rather than nil. This simplifies the callers and makes sure we have a consistent error. * LookupImage is now able to handle manifests lists. Unless the platform is explicitly ignored via the options, the matching image within the manifest list is now returned. This greatly simplifies the spec generation in Podman; no callers should have to worry about this kind of detail. * LookupImage has been refactored into smaller-sized and easier to read functions. * RemoveImages has been changed to assemble the data of removed or untagged images. This comes in handy for pruning images. I am heavily against having a dedicated API for pruning since the it's really just a combination of filtering and removing images which RemoveImages already supports. Hence these changes to satisfy the needs of `podman image prune`. Furthermore, it now returns an []error slice rather than a single error. Again to make Podman happy which needs to inspect *all* errors for setting the appropriate exit code. * A rather large refactoring of the removal code along with very verbose comments. Those were largely absent in the Podman code base but there many rules and contracts embedded that I partially could only reconstruct by manually tests and comparing to Docker. * Add a new `containers={true,false}` filter which allows filtering images whether they are used by containers (=true) or if no container is using them (=false). This filter is required for pruning images in Podman. * `libimage/types` has been merged into `libimage`. Podman has to do _a lot of_ massaging for the remote client already and the types are pretty much nailed down for the remote API. Hence, I prefer to do some translation between `libimage` types and what Podman needs rather than splitting `libimage` in half without an obvious reason. This way the package is self-contained allowing for an easier navigation and maintenance. * `libimage.PullPolicy` has been merged into `pkg/config.PullPolicy` to have _one_ central place to deal with pull policies. The type system in `pkg/config` sets "always" as the default unfortunately but I think consistency is more important at that point. * Added `CopyOptions.DirForceCompress` to enforce layer compression when copying to a `dir` destination. * We now use `github.com/disiqueira/gotree` for pretty printing image trees. That greatly simplifies the code and we don't have to worry about the logic of printing a tree. Note that trees are now always printed top down! * Added a new `libimage.ManifestList` type along with an API for local lookups and performing certain operations on it to wrap around `libimage/manifests` as previously done in `libpod/image` and other places in Podman. * Correct caching of `(*Image).Inspect`. * In addition to username, password and credentials, allow for speciying an identity token for copying images. That's needed for Podman's remote API. * Make image removal more tolerant toward corrupted images. * A new "until=timestamp" filter that can be used by all APIs supporting filtering. * An empty string now resolves to PullPolicyMissing. * `(*Runtime) systemContextCopy()` returns a deep copy of the runtime's system context. Golang's shallow copies are very dangerous for long running processes such as Podman's system service. Hence, we need to make sure that base data is not altered over time. That adds another external dependency but I do not see a way around that. Long term, I desire a `(*containers/image/types.SystemContext).Copy()` function. Signed-off-by: Valentin Rothberg --- go.mod | 2 + go.sum | 4 + libimage/copier.go | 50 +- libimage/disk_usage.go | 126 +++++ libimage/filters.go | 43 +- libimage/history.go | 18 +- libimage/image.go | 342 +++++++++--- libimage/image_tree.go | 98 ++-- libimage/import.go | 5 +- libimage/inspect.go | 72 ++- libimage/load.go | 44 +- libimage/manifest_list.go | 389 ++++++++++++++ libimage/normalize.go | 16 +- libimage/oci.go | 2 +- libimage/pull.go | 67 +-- libimage/pull_policy.go | 90 ---- libimage/push.go | 7 +- libimage/runtime.go | 387 +++++++++----- libimage/save.go | 10 +- libimage/search.go | 63 ++- libimage/types/types.go | 58 --- pkg/config/config.go | 29 -- {libimage/types => pkg/config}/pull_policy.go | 27 +- pkg/filters/filters.go | 4 +- .../disiqueira/gotree/v3/.gitignore | 137 +++++ .../disiqueira/gotree/v3/.travis.yml | 11 + .../github.com/disiqueira/gotree/v3/LICENSE | 21 + .../github.com/disiqueira/gotree/v3/README.md | 104 ++++ .../disiqueira/gotree/v3/_config.yml | 1 + vendor/github.com/disiqueira/gotree/v3/go.mod | 3 + .../disiqueira/gotree/v3/gotree-logo.png | Bin 0 -> 24183 bytes .../github.com/disiqueira/gotree/v3/gotree.go | 129 +++++ vendor/github.com/jinzhu/copier/License | 20 + vendor/github.com/jinzhu/copier/README.md | 131 +++++ vendor/github.com/jinzhu/copier/copier.go | 491 ++++++++++++++++++ vendor/github.com/jinzhu/copier/errors.go | 10 + vendor/github.com/jinzhu/copier/go.mod | 3 + vendor/modules.txt | 6 + 38 files changed, 2484 insertions(+), 536 deletions(-) create mode 100644 libimage/disk_usage.go create mode 100644 libimage/manifest_list.go delete mode 100644 libimage/pull_policy.go delete mode 100644 libimage/types/types.go rename {libimage/types => pkg/config}/pull_policy.go (83%) create mode 100644 vendor/github.com/disiqueira/gotree/v3/.gitignore create mode 100644 vendor/github.com/disiqueira/gotree/v3/.travis.yml create mode 100644 vendor/github.com/disiqueira/gotree/v3/LICENSE create mode 100644 vendor/github.com/disiqueira/gotree/v3/README.md create mode 100644 vendor/github.com/disiqueira/gotree/v3/_config.yml create mode 100644 vendor/github.com/disiqueira/gotree/v3/go.mod create mode 100644 vendor/github.com/disiqueira/gotree/v3/gotree-logo.png create mode 100644 vendor/github.com/disiqueira/gotree/v3/gotree.go create mode 100644 vendor/github.com/jinzhu/copier/License create mode 100644 vendor/github.com/jinzhu/copier/README.md create mode 100644 vendor/github.com/jinzhu/copier/copier.go create mode 100644 vendor/github.com/jinzhu/copier/errors.go create mode 100644 vendor/github.com/jinzhu/copier/go.mod diff --git a/go.mod b/go.mod index b187567ce..a1edbc165 100644 --- a/go.mod +++ b/go.mod @@ -7,6 +7,7 @@ require ( github.com/containers/image/v5 v5.11.1 github.com/containers/ocicrypt v1.1.1 github.com/containers/storage v1.30.1 + github.com/disiqueira/gotree/v3 v3.0.2 github.com/docker/distribution v2.7.1+incompatible github.com/docker/docker v20.10.3-0.20210216175712-646072ed6524+incompatible github.com/docker/go-units v0.4.0 @@ -14,6 +15,7 @@ require ( github.com/google/go-cmp v0.5.5 // indirect github.com/gorilla/mux v1.8.0 // indirect github.com/hashicorp/go-multierror v1.1.1 + github.com/jinzhu/copier v0.3.0 github.com/moby/term v0.0.0-20201216013528-df9cb8a40635 // indirect github.com/onsi/ginkgo v1.16.1 github.com/onsi/gomega v1.11.0 diff --git a/go.sum b/go.sum index 49fc0d1bc..1276a1b3c 100644 --- a/go.sum +++ b/go.sum @@ -219,6 +219,8 @@ github.com/denverdino/aliyungo v0.0.0-20190125010748-a747050bb1ba/go.mod h1:dV8l github.com/dgrijalva/jwt-go v0.0.0-20170104182250-a601269ab70c/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ= github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ= github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no= +github.com/disiqueira/gotree/v3 v3.0.2 h1:ik5iuLQQoufZBNPY518dXhiO5056hyNBIK9lWhkNRq8= +github.com/disiqueira/gotree/v3 v3.0.2/go.mod h1:ZuyjE4+mUQZlbpkI24AmruZKhg3VHEgPLDY8Qk+uUu8= github.com/dnaeon/go-vcr v1.0.1/go.mod h1:aBB1+wY4s93YsC3HHjMBMrwTj2R9FHDzUr9KyGc8n1E= github.com/docker/distribution v0.0.0-20190905152932-14b96e55d84c/go.mod h1:0+TTO4EOBfRPhZXAeF1Vu+W3hHZ8eLp8PgKVZlcvtFY= github.com/docker/distribution v2.7.1-0.20190205005809-0d3efadf0154+incompatible/go.mod h1:J2gT2udsDAN96Uj4KfcMRqY0/ypR+oyYUYmja8H+y+w= @@ -405,6 +407,8 @@ github.com/imdario/mergo v0.3.11/go.mod h1:jmQim1M+e3UYxmgPu/WyfjB3N3VflVyUjjjwH github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM= github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8= github.com/j-keck/arping v0.0.0-20160618110441-2cf9dc699c56/go.mod h1:ymszkNOg6tORTn+6F6j+Jc8TOr5osrynvN6ivFWZ2GA= +github.com/jinzhu/copier v0.3.0 h1:P5zN9OYSxmtzZmwgcVmt5Iu8egfP53BGMPAFgEksKPI= +github.com/jinzhu/copier v0.3.0/go.mod h1:24xnZezI2Yqac9J61UC6/dG/k76ttpq0DdJI3QmUvro= github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k= github.com/jmespath/go-jmespath v0.0.0-20160803190731-bd40a432e4c7/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k= github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo= diff --git a/libimage/copier.go b/libimage/copier.go index 91a9f212b..34cc0d45d 100644 --- a/libimage/copier.go +++ b/libimage/copier.go @@ -48,6 +48,8 @@ type CopyOptions struct { BlobInfoCacheDirPath string // Path to the certificates directory. CertDirPath string + // Force layer compression when copying to a `dir` transport destination. + DirForceCompress bool // Allow contacting registries over HTTP, or HTTPS with failed TLS // verification. Note that this does not affect other TLS connections. InsecureSkipTLSVerify types.OptionalBool @@ -115,6 +117,9 @@ type CopyOptions struct { // "username[:password]". Cannot be used in combination with // Username/Password. Credentials string + // IdentityToken is used to authenticate the user and get + // an access token for the registry. + IdentityToken string `json:"identitytoken,omitempty"` // ----- internal ----------------------------------------------------- @@ -146,30 +151,33 @@ var ( // getDockerAuthConfig extracts a docker auth config from the CopyOptions. Returns // nil if no credentials are set. func (options *CopyOptions) getDockerAuthConfig() (*types.DockerAuthConfig, error) { + authConf := &types.DockerAuthConfig{IdentityToken: options.IdentityToken} + if options.Username != "" { if options.Credentials != "" { return nil, errors.New("username/password cannot be used with credentials") } - return &types.DockerAuthConfig{ - Username: options.Username, - Password: options.Password, - }, nil + authConf.Username = options.Username + authConf.Password = options.Password + return authConf, nil } if options.Credentials != "" { - var username, password string split := strings.SplitN(options.Credentials, ":", 2) switch len(split) { case 1: - username = split[0] + authConf.Username = split[0] default: - username = split[0] - password = split[1] + authConf.Username = split[0] + authConf.Password = split[1] } - return &types.DockerAuthConfig{ - Username: username, - Password: password, - }, nil + return authConf, nil + } + + // We should return nil unless a token was set. That's especially + // useful for Podman's remote API. + if options.IdentityToken != "" { + return authConf, nil } return nil, nil @@ -178,8 +186,9 @@ func (options *CopyOptions) getDockerAuthConfig() (*types.DockerAuthConfig, erro // newCopier creates a copier. Note that fields in options *may* overwrite the // counterparts of the specified system context. Please make sure to call // `(*copier).close()`. -func newCopier(sys *types.SystemContext, options *CopyOptions) (*copier, error) { +func (r *Runtime) newCopier(options *CopyOptions) (*copier, error) { c := copier{} + c.systemContext = r.systemContextCopy() if options.SourceLookupReferenceFunc != nil { c.sourceLookup = options.SourceLookupReferenceFunc @@ -189,11 +198,14 @@ func newCopier(sys *types.SystemContext, options *CopyOptions) (*copier, error) c.destinationLookup = options.DestinationLookupReferenceFunc } - c.systemContext = sys - if c.systemContext == nil { - c.systemContext = &types.SystemContext{} + if options.InsecureSkipTLSVerify != types.OptionalBoolUndefined { + c.systemContext.DockerInsecureSkipTLSVerify = options.InsecureSkipTLSVerify + c.systemContext.OCIInsecureSkipTLSVerify = options.InsecureSkipTLSVerify == types.OptionalBoolTrue + c.systemContext.DockerDaemonInsecureSkipTLSVerify = options.InsecureSkipTLSVerify == types.OptionalBoolTrue } + c.systemContext.DirForceCompress = c.systemContext.DirForceCompress || options.DirForceCompress + if options.AuthFilePath != "" { c.systemContext.AuthFilePath = options.AuthFilePath } @@ -226,7 +238,11 @@ func newCopier(sys *types.SystemContext, options *CopyOptions) (*copier, error) c.systemContext.BlobInfoCacheDir = options.BlobInfoCacheDirPath } - policy, err := signature.DefaultPolicy(sys) + if options.CertDirPath != "" { + c.systemContext.DockerCertPath = options.CertDirPath + } + + policy, err := signature.DefaultPolicy(c.systemContext) if err != nil { return nil, err } diff --git a/libimage/disk_usage.go b/libimage/disk_usage.go new file mode 100644 index 000000000..edfd095a0 --- /dev/null +++ b/libimage/disk_usage.go @@ -0,0 +1,126 @@ +package libimage + +import ( + "context" + "time" +) + +// ImageDiskUsage reports the total size of an image. That is the size +type ImageDiskUsage struct { + // Number of containers using the image. + Containers int + // ID of the image. + ID string + // Repository of the image. + Repository string + // Tag of the image. + Tag string + // Created time stamp. + Created time.Time + // The amount of space that an image shares with another one (i.e. their common data). + SharedSize int64 + // The the amount of space that is only used by a given image. + UniqueSize int64 + // Sum of shared an unique size. + Size int64 +} + +// DiskUsage calculates the disk usage for each image in the local containers +// storage. Note that a single image may yield multiple usage reports, one for +// each repository tag. +func (r *Runtime) DiskUsage(ctx context.Context) ([]ImageDiskUsage, error) { + layerTree, err := r.layerTree() + if err != nil { + return nil, err + } + + images, err := r.ListImages(ctx, nil, nil) + if err != nil { + return nil, err + } + + var allUsages []ImageDiskUsage + for _, image := range images { + usages, err := diskUsageForImage(ctx, image, layerTree) + if err != nil { + return nil, err + } + allUsages = append(allUsages, usages...) + } + return allUsages, err +} + +// diskUsageForImage returns the disk-usage baseistics for the specified image. +func diskUsageForImage(ctx context.Context, image *Image, tree *layerTree) ([]ImageDiskUsage, error) { + base := ImageDiskUsage{ + ID: image.ID(), + Created: image.Created(), + Repository: "", + Tag: "", + } + + // Shared, unique and total size. + parent, err := tree.parent(ctx, image) + if err != nil { + return nil, err + } + childIDs, err := tree.children(ctx, image, false) + if err != nil { + return nil, err + } + + // Optimistically set unique size to the full size of the image. + size, err := image.Size() + if err != nil { + return nil, err + } + base.UniqueSize = size + + if len(childIDs) > 0 { + // If we have children, we share everything. + base.SharedSize = base.UniqueSize + base.UniqueSize = 0 + } else if parent != nil { + // If we have no children but a parent, remove the parent + // (shared) size from the unique one. + size, err := parent.Size() + if err != nil { + return nil, err + } + base.UniqueSize -= size + base.SharedSize = size + } + + base.Size = base.SharedSize + base.UniqueSize + + // Number of containers using the image. + containers, err := image.Containers() + if err != nil { + return nil, err + } + base.Containers = len(containers) + + repoTags, err := image.NamedRepoTags() + if err != nil { + return nil, err + } + + if len(repoTags) == 0 { + return []ImageDiskUsage{base}, nil + } + + pairs, err := ToNameTagPairs(repoTags) + if err != nil { + return nil, err + } + + results := make([]ImageDiskUsage, len(pairs)) + for i, pair := range pairs { + res := base + res.Repository = pair.Name + res.Tag = pair.Tag + results[i] = res + } + + return results, nil +} diff --git a/libimage/filters.go b/libimage/filters.go index 34df7626f..eae18fd9c 100644 --- a/libimage/filters.go +++ b/libimage/filters.go @@ -9,6 +9,7 @@ import ( "time" filtersPkg "github.com/containers/common/pkg/filters" + "github.com/containers/common/pkg/timetype" "github.com/pkg/errors" "github.com/sirupsen/logrus" ) @@ -45,15 +46,12 @@ func filterImages(images []*Image, filters []filterFunc) ([]*Image, error) { // compileImageFilters creates `filterFunc`s for the specified filters. The // required format is `key=value` with the following supported keys: -// after, since, before, dangling, id, label, readonly, reference, intermediate +// after, since, before, containers, dangling, id, label, readonly, reference, intermediate func (r *Runtime) compileImageFilters(ctx context.Context, filters []string) ([]filterFunc, error) { logrus.Tracef("Parsing image filters %s", filters) filterFuncs := []filterFunc{} - visitedKeys := make(map[string]bool) - for _, filter := range filters { - // First, parse the filter. var key, value string split := strings.SplitN(filter, "=", 2) if len(split) != 2 { @@ -62,13 +60,6 @@ func (r *Runtime) compileImageFilters(ctx context.Context, filters []string) ([] key = split[0] value = split[1] - - if _, exists := visitedKeys[key]; exists { - return nil, errors.Errorf("image filter %q specified multiple times", key) - } - visitedKeys[key] = true - - // Second, dispatch the filters. switch key { case "after", "since": @@ -85,6 +76,13 @@ func (r *Runtime) compileImageFilters(ctx context.Context, filters []string) ([] } filterFuncs = append(filterFuncs, filterBefore(img.Created())) + case "containers": + containers, err := strconv.ParseBool(value) + if err != nil { + return nil, errors.Wrapf(err, "non-boolean value %q for dangling filter", value) + } + filterFuncs = append(filterFuncs, filterContainers(containers)) + case "dangling": dangling, err := strconv.ParseBool(value) if err != nil { @@ -115,6 +113,18 @@ func (r *Runtime) compileImageFilters(ctx context.Context, filters []string) ([] case "reference": filterFuncs = append(filterFuncs, filterReference(value)) + case "until": + ts, err := timetype.GetTimestamp(value, time.Now()) + if err != nil { + return nil, err + } + seconds, nanoseconds, err := timetype.ParseTimestamps(ts, 0) + if err != nil { + return nil, err + } + until := time.Unix(seconds, nanoseconds) + filterFuncs = append(filterFuncs, filterBefore(until)) + default: return nil, errors.Errorf("unsupported image filter %q", key) } @@ -179,6 +189,17 @@ func filterReadOnly(value bool) filterFunc { } } +// filterContainers creates a container filter for matching the specified value. +func filterContainers(value bool) filterFunc { + return func(img *Image) (bool, error) { + ctrs, err := img.Containers() + if err != nil { + return false, err + } + return (len(ctrs) > 0) == value, nil + } +} + // filterDangling creates a dangling filter for matching the specified value. func filterDangling(value bool) filterFunc { return func(img *Image) (bool, error) { diff --git a/libimage/history.go b/libimage/history.go index b966eb57e..b63fe696b 100644 --- a/libimage/history.go +++ b/libimage/history.go @@ -2,13 +2,23 @@ package libimage import ( "context" + "time" - libimageTypes "github.com/containers/common/libimage/types" "github.com/containers/storage" ) +// ImageHistory contains the history information of an image. +type ImageHistory struct { + ID string `json:"id"` + Created *time.Time `json:"created"` + CreatedBy string `json:"createdBy"` + Size int64 `json:"size"` + Comment string `json:"comment"` + Tags []string `json:"tags"` +} + // History computes the image history of the image including all of its parents. -func (i *Image) History(ctx context.Context) ([]libimageTypes.ImageHistory, error) { +func (i *Image) History(ctx context.Context) ([]ImageHistory, error) { ociImage, err := i.toOCI(ctx) if err != nil { return nil, err @@ -19,7 +29,7 @@ func (i *Image) History(ctx context.Context) ([]libimageTypes.ImageHistory, erro return nil, err } - var allHistory []libimageTypes.ImageHistory + var allHistory []ImageHistory var layer *storage.Layer if i.TopLayer() != "" { layer, err = i.runtime.store.Layer(i.TopLayer()) @@ -33,7 +43,7 @@ func (i *Image) History(ctx context.Context) ([]libimageTypes.ImageHistory, erro numHistories := len(ociImage.History) - 1 usedIDs := make(map[string]bool) // prevents assigning images IDs more than once for x := numHistories; x >= 0; x-- { - history := libimageTypes.ImageHistory{ + history := ImageHistory{ ID: "", // may be overridden below Created: ociImage.History[x].Created, CreatedBy: ociImage.History[x].CreatedBy, diff --git a/libimage/image.go b/libimage/image.go index d6756e6ee..4728565bb 100644 --- a/libimage/image.go +++ b/libimage/image.go @@ -4,9 +4,9 @@ import ( "context" "path/filepath" "sort" + "strings" "time" - libimageTypes "github.com/containers/common/libimage/types" "github.com/containers/image/v5/docker/reference" "github.com/containers/image/v5/manifest" storageTransport "github.com/containers/image/v5/storage" @@ -40,7 +40,7 @@ type Image struct { // Inspect data we get from containers/image. partialInspectData *types.ImageInspectInfo // Fully assembled image data. - completeInspectData *libimageTypes.ImageData + completeInspectData *ImageData // Corresponding OCI image. ociv1Image *ociv1.Image } @@ -131,7 +131,7 @@ func (i *Image) Created() time.Time { func (i *Image) Labels(ctx context.Context) (map[string]string, error) { data, err := i.inspectInfo(ctx) if err != nil { - isManifestList, listErr := i.isManifestList(ctx) + isManifestList, listErr := i.IsManifestList(ctx) if listErr != nil { err = errors.Wrapf(err, "fallback error checking whether image is a manifest list: %v", err) } else if isManifestList { @@ -208,7 +208,9 @@ func (i *Image) removeContainers(fn RemoveContainerFunc) error { // Execute the custom removal func if specified. if fn != nil { logrus.Debugf("Removing containers of image %s with custom removal function", i.ID()) - return fn(i.ID()) + if err := fn(i.ID()); err != nil { + return err + } } containers, err := i.Containers() @@ -234,59 +236,182 @@ func (i *Image) removeContainers(fn RemoveContainerFunc) error { // an image specified by imageID. type RemoveContainerFunc func(imageID string) error -// RemoveImageOptions allow for customizing image removal. -type RemoveImageOptions struct { - // Force will remove all containers from the local storage that are - // using a removed image. Use RemoveContainerFunc for a custom logic. - // If set, all child images will be removed as well. - Force bool - // RemoveContainerFunc allows for a custom logic for removing - // containers using a specific image. By default, all containers in - // the local containers storage will be removed (if Force is set). - RemoveContainerFunc RemoveContainerFunc +// RemoveImagesReport is the assembled data from removing *one* image. +type RemoveImageReport struct { + // ID of the image. + ID string + // Image was removed. + Removed bool + // Size of the removed image. Only set when explicitly requested in + // RemoveImagesOptions. + Size int64 + // The untagged tags. + Untagged []string } -// Remove removes the image along with all dangling parent images that no other +// remove removes the image along with all dangling parent images that no other // image depends on. The image must not be set read-only and not be used by -// containers. Callers must make sure to remove containers before image -// removal and may use `(*Image).Containers()` to get a list of containers -// using the image. +// containers. // // If the image is used by containers return storage.ErrImageUsedByContainer. // Use force to remove these containers. -func (i *Image) Remove(ctx context.Context, options *RemoveImageOptions) error { +// +// NOTE: the rmMap is used to assemble image-removal data across multiple +// invocations of this function. The recursive nature requires some +// bookkeeping to make sure that all data is aggregated correctly. +// +// This function is internal. Users of libimage should always use +// `(*Runtime).RemoveImages()`. +func (i *Image) remove(ctx context.Context, rmMap map[string]*RemoveImageReport, referencedBy string, options *RemoveImagesOptions) error { + // If referencedBy is empty, the image is considered to be removed via + // `image remove --all` which alters the logic below. + + // The removal logic below is complex. There is a number of rules + // inherited from Podman and Buildah (and Docker). This function + // should be the *only* place to extend the removal logic so we keep it + // sealed in one place. Make sure to add verbose comments to leave + // some breadcrumbs for future readers. logrus.Debugf("Removing image %s", i.ID()) + if i.IsReadOnly() { return errors.Errorf("cannot remove read-only image %q", i.ID()) } - if options == nil { - options = &RemoveImageOptions{} + // Check if already visisted this image. + report, exists := rmMap[i.ID()] + if exists { + // If the image has already been removed, we're done. + if report.Removed { + return nil + } + } else { + report = &RemoveImageReport{ID: i.ID()} + rmMap[i.ID()] = report + } + + // The image may have already been (partially) removed, so we need to + // have a closer look at the errors. On top, image removal should be + // tolerant toward corrupted images. + handleError := func(err error) error { + switch errors.Cause(err) { + case storage.ErrImageUnknown, storage.ErrNotAnImage, storage.ErrLayerUnknown: + // The image or layers of the image may already + // have been removed in which case we consider + // the image to be removed. + return nil + default: + return err + } } + // Calculate the size if requested. `podman-image-prune` likes to + // report the regained size. + if options.WithSize { + size, err := i.Size() + if handleError(err) != nil { + return err + } + report.Size = size + } + + skipRemove := false + numNames := len(i.Names()) + + // NOTE: the `numNames == 1` check is not only a performance + // optimization but also preserves exiting Podman/Docker behaviour. + // If image "foo" is used by a container and has only this tag/name, + // an `rmi foo` will not untag "foo" but instead attempt to remove the + // entire image. If there's a container using "foo", we should get an + // error. + if options.Force || referencedBy == "" || numNames == 1 { + // DO NOTHING, the image will be removed + } else { + byID := strings.HasPrefix(i.ID(), referencedBy) + byDigest := strings.HasPrefix(referencedBy, "sha256:") + if byID && numNames > 1 { + return errors.Errorf("unable to delete image %q by ID with more than one tag (%s): please force removal", i.ID(), i.Names()) + } else if byDigest && numNames > 1 { + // FIXME - Docker will remove the digest but containers storage + // does not support that yet, so our hands are tied. + return errors.Errorf("unable to delete image %q by digest with more than one tag (%s): please force removal", i.ID(), i.Names()) + } + + // Only try to untag if we know it's not an ID or digest. + if !byID && !byDigest { + if err := i.Untag(referencedBy); handleError(err) != nil { + return err + } + report.Untagged = append(report.Untagged, referencedBy) + + // If there's still tags left, we cannot delete it. + skipRemove = len(i.Names()) > 0 + } + } + + if skipRemove { + return nil + } + + // Perform the actual removal. First, remove containers if needed. if options.Force { if err := i.removeContainers(options.RemoveContainerFunc); err != nil { return err } } + // Podman/Docker compat: we only report an image as removed if it has + // no children. Otherwise, the data is effectively still present in the + // storage despite the image being removed. + hasChildren, err := i.HasChildren(ctx) + if err != nil { + // We must be tolerant toward corrupted images. + // See containers/podman commit fd9dd7065d44. + logrus.Warnf("error determining if an image is a parent: %v, ignoring the error", err) + hasChildren = false + } + // If there's a dangling parent that no other image depends on, remove // it recursively. parent, err := i.Parent(ctx) if err != nil { - return err + // We must be tolerant toward corrupted images. + // See containers/podman commit fd9dd7065d44. + logrus.Warnf("error determining parent of image: %v, ignoring the error", err) + parent = nil } - if _, err := i.runtime.store.DeleteImage(i.ID(), true); err != nil { + if _, err := i.runtime.store.DeleteImage(i.ID(), true); handleError(err) != nil { return err } - delete(i.runtime.imageIDmap, i.ID()) + report.Untagged = append(report.Untagged, i.Names()...) + + if !hasChildren { + report.Removed = true + } + + // Check if can remove the parent image. + if parent == nil { + return nil + } - if parent == nil || !parent.IsDangling() { + if !parent.IsDangling() { return nil } - return parent.Remove(ctx, options) + // If the image has siblings, we don't remove the parent. + hasSiblings, err := parent.HasChildren(ctx) + if err != nil { + // See Podman commit fd9dd7065d44: we need to + // be tolerant toward corrupted images. + logrus.Warnf("error determining if an image is a parent: %v, ignoring the error", err) + hasSiblings = false + } + if hasSiblings { + return nil + } + + // Recurse into removing the parent. + return parent.remove(ctx, rmMap, "", options) } // Tag the image with the specified name and store it in the local containers @@ -307,16 +432,29 @@ func (i *Image) Tag(name string) error { return i.reload() } +// to have some symmetry with the errors from containers/storage. +var errTagUnknown = errors.New("tag not known") + +// TODO (@vrothberg) - `docker rmi sha256:` will remove the digest from the +// image. However, that's something containers storage does not support. +var errUntagDigest = errors.New("untag by digest not supported") + // Untag the image with the specified name and make the change persistent in // the local containers storage. The name is normalized according to the rules // of NormalizeName. func (i *Image) Untag(name string) error { + if strings.HasPrefix(name, "sha256:") { + return errors.Wrap(errUntagDigest, name) + } + ref, err := NormalizeName(name) if err != nil { return errors.Wrapf(err, "error normalizing name %q", name) } name = ref.String() + logrus.Debugf("Untagging %q from image %s", ref.String(), i.ID()) + removedName := false newNames := []string{} for _, n := range i.Names() { @@ -328,11 +466,9 @@ func (i *Image) Untag(name string) error { } if !removedName { - return nil + return errors.Wrap(errTagUnknown, name) } - logrus.Debugf("Untagging %q from image %s", ref.String(), i.ID()) - if err := i.runtime.store.SetNames(i.ID(), newNames); err != nil { return err } @@ -353,25 +489,78 @@ func (i *Image) RepoTags() ([]string, error) { return repoTags, nil } -// NammedTaggedRepoTags returns the repotags associated with the image as a +// NamedTaggedRepoTags returns the repotags associated with the image as a // slice of reference.NamedTagged. func (i *Image) NamedTaggedRepoTags() ([]reference.NamedTagged, error) { var repoTags []reference.NamedTagged for _, name := range i.Names() { - named, err := reference.ParseNormalizedNamed(name) + parsed, err := reference.Parse(name) if err != nil { return nil, err } - if tagged, isTagged := named.(reference.NamedTagged); isTagged { - repoTags = append(repoTags, tagged) + named, isNamed := parsed.(reference.Named) + if !isNamed { + continue + } + tagged, isTagged := named.(reference.NamedTagged) + if !isTagged { + continue + } + repoTags = append(repoTags, tagged) + } + return repoTags, nil +} + +// NamedRepoTags returns the repotags associated with the image as a +// slice of reference.Named. +func (i *Image) NamedRepoTags() ([]reference.Named, error) { + var repoTags []reference.Named + for _, name := range i.Names() { + parsed, err := reference.Parse(name) + if err != nil { + return nil, err + } + if named, isNamed := parsed.(reference.Named); isNamed { + repoTags = append(repoTags, named) } } return repoTags, nil } -// RepoDigests returns a string array of repodigests associated with the image +// inRepoTags looks for the specified name/tag pair in the image's repo tags. +// Note that tag may be empty. +func (i *Image) inRepoTags(name, tag string) (reference.Named, error) { + repoTags, err := i.NamedRepoTags() + if err != nil { + return nil, err + } + + pairs, err := ToNameTagPairs(repoTags) + if err != nil { + return nil, err + } + + for _, pair := range pairs { + if tag != "" && tag != pair.Tag { + continue + } + if !strings.HasSuffix(pair.Name, name) { + continue + } + if len(pair.Name) == len(name) { // full match + return pair.named, nil + } + if pair.Name[len(pair.Name)-len(name)-1] == '/' { // matches at repo + return pair.named, nil + } + } + + return nil, nil +} + +// RepoDigests returns a string array of repodigests associated with the image. func (i *Image) RepoDigests() ([]string, error) { - var repoDigests []string + repoDigests := []string{} added := make(map[string]struct{}) for _, name := range i.Names() { @@ -416,6 +605,32 @@ func (i *Image) Mount(ctx context.Context, mountOptions []string, mountLabel str return mountPoint, nil } +// Mountpoint returns the path to image's mount point. The path is empty if +// the image is not mounted. +func (i *Image) Mountpoint() (string, error) { + mountedTimes, err := i.runtime.store.Mounted(i.TopLayer()) + if err != nil || mountedTimes == 0 { + if errors.Cause(err) == storage.ErrLayerUnknown { + // Can happen, Podman did it, but there's no + // explanation why. + err = nil + } + return "", err + } + + layer, err := i.runtime.store.Layer(i.TopLayer()) + if err != nil { + return "", err + } + + mountPoint, err := filepath.EvalSymlinks(layer.MountPoint) + if err != nil { + return "", err + } + + return mountPoint, nil +} + // Unmount the image. Use force to ignore the reference counter and forcefully // unmount. func (i *Image) Unmount(force bool) error { @@ -460,7 +675,7 @@ func (i *Image) HasDifferentDigest(ctx context.Context, remoteRef types.ImageRef return false, err } - sys := i.runtime.systemContext + sys := i.runtime.systemContextCopy() sys.ArchitectureChoice = inspectInfo.Architecture // OS and variant may not be set, so let's check to avoid accidental // overrides of the runtime settings. @@ -471,7 +686,7 @@ func (i *Image) HasDifferentDigest(ctx context.Context, remoteRef types.ImageRef sys.VariantChoice = inspectInfo.Variant } - remoteImg, err := remoteRef.NewImage(ctx, &sys) + remoteImg, err := remoteRef.NewImage(ctx, sys) if err != nil { return false, err } @@ -490,7 +705,7 @@ func (i *Image) HasDifferentDigest(ctx context.Context, remoteRef types.ImageRef } // driverData gets the driver data from the store on a layer -func (i *Image) driverData() (*libimageTypes.DriverData, error) { +func (i *Image) driverData() (*DriverData, error) { store := i.runtime.store layerID := i.TopLayer() driver, err := store.GraphDriver() @@ -504,7 +719,7 @@ func (i *Image) driverData() (*libimageTypes.DriverData, error) { if mountTimes, err := store.Mounted(layerID); mountTimes == 0 || err != nil { delete(metaData, "MergedDir") } - return &libimageTypes.DriverData{ + return &DriverData{ Name: driver.String(), Data: metaData, }, nil @@ -524,27 +739,6 @@ func (i *Image) StorageReference() (types.ImageReference, error) { return ref, nil } -// isManifestList returns true if the image is a manifest list (Docker) or an -// image index (OCI). This information may be useful to make certain execution -// paths more robust. -// NOTE: please use this function only to optimize specific execution paths. -// In general, errors should only be suppressed when necessary. -func (i *Image) isManifestList(ctx context.Context) (bool, error) { - ref, err := i.StorageReference() - if err != nil { - return false, err - } - imgRef, err := ref.NewImageSource(ctx, &i.runtime.systemContext) - if err != nil { - return false, err - } - _, manifestType, err := imgRef.GetManifest(ctx, nil) - if err != nil { - return false, err - } - return manifest.MIMETypeIsMultiImage(manifestType), nil -} - // source returns the possibly cached image reference. func (i *Image) source(ctx context.Context) (types.ImageSource, error) { if i.cached.imageSource != nil { @@ -554,7 +748,7 @@ func (i *Image) source(ctx context.Context) (types.ImageSource, error) { if err != nil { return nil, err } - src, err := ref.NewImageSource(ctx, &i.runtime.systemContext) + src, err := ref.NewImageSource(ctx, i.runtime.systemContextCopy()) if err != nil { return nil, err } @@ -562,6 +756,32 @@ func (i *Image) source(ctx context.Context) (types.ImageSource, error) { return src, nil } +// rawConfigBlob returns the image's config as a raw byte slice. Users need to +// unmarshal it to the corresponding type (OCI, Docker v2s{1,2}) +func (i *Image) rawConfigBlob(ctx context.Context) ([]byte, error) { + ref, err := i.StorageReference() + if err != nil { + return nil, err + } + + imageCloser, err := ref.NewImage(ctx, i.runtime.systemContextCopy()) + if err != nil { + return nil, err + } + defer imageCloser.Close() + + return imageCloser.ConfigBlob(ctx) +} + +// Manifest returns the raw data and the MIME type of the image's manifest. +func (i *Image) Manifest(ctx context.Context) (rawManifest []byte, mimeType string, err error) { + src, err := i.source(ctx) + if err != nil { + return nil, "", err + } + return src.GetManifest(ctx, nil) +} + // getImageDigest creates an image object and uses the hex value of the digest as the image ID // for parsing the store reference func getImageDigest(ctx context.Context, src types.ImageReference, sys *types.SystemContext) (string, error) { diff --git a/libimage/image_tree.go b/libimage/image_tree.go index 5ab180836..6583a7007 100644 --- a/libimage/image_tree.go +++ b/libimage/image_tree.go @@ -4,19 +4,14 @@ import ( "fmt" "strings" + "github.com/disiqueira/gotree/v3" "github.com/docker/go-units" ) -const ( - imageTreeMiddleItem = "├── " - imageTreeContinueItem = "│ " - imageTreeLastItem = "└── " -) - // Tree generates a tree for the specified image and its layers. Use // `traverseChildren` to traverse the layers of all children. By default, only // layers of the image are printed. -func (i *Image) Tree(traverseChildren bool) (*strings.Builder, error) { +func (i *Image) Tree(traverseChildren bool) (string, error) { // NOTE: a string builder prevents us from copying to much data around // and compile the string when and where needed. sb := &strings.Builder{} @@ -24,85 +19,78 @@ func (i *Image) Tree(traverseChildren bool) (*strings.Builder, error) { // First print the pretty header for the target image. size, err := i.Size() if err != nil { - return nil, err + return "", err } repoTags, err := i.RepoTags() if err != nil { - return nil, err + return "", err } fmt.Fprintf(sb, "Image ID: %s\n", i.ID()[:12]) fmt.Fprintf(sb, "Tags: %s\n", repoTags) fmt.Fprintf(sb, "Size: %v\n", units.HumanSizeWithPrecision(float64(size), 4)) if i.TopLayer() != "" { - fmt.Fprintf(sb, "Image Layers\n") + fmt.Fprintf(sb, "Image Layers") } else { - fmt.Fprintf(sb, "No Image Layers\n") + fmt.Fprintf(sb, "No Image Layers") } + tree := gotree.New(sb.String()) + layerTree, err := i.runtime.layerTree() if err != nil { - return nil, err + return "", err } imageNode := layerTree.node(i.TopLayer()) + // Traverse the entire tree down to all children. if traverseChildren { - return imageTreeTraverseChildren(sb, imageNode, "", true) - } - - // Walk all layers of the image and assemlbe their data. - for parentNode := imageNode.parent; parentNode != nil; parentNode = parentNode.parent { - indent := imageTreeMiddleItem - if parentNode.parent == nil { - indent = imageTreeLastItem - } - - var tags string - repoTags, err := parentNode.repoTags() - if err != nil { - return nil, err + if err := imageTreeTraverseChildren(imageNode, tree); err != nil { + return "", err } - if len(repoTags) > 0 { - tags = fmt.Sprintf(" Top Layer of: %s", repoTags) + } else { + // Walk all layers of the image and assemlbe their data. + for parentNode := imageNode; parentNode != nil; parentNode = parentNode.parent { + if parentNode.layer == nil { + break // we're done + } + var tags string + repoTags, err := parentNode.repoTags() + if err != nil { + return "", err + } + if len(repoTags) > 0 { + tags = fmt.Sprintf(" Top Layer of: %s", repoTags) + } + tree.Add(fmt.Sprintf("ID: %s Size: %7v%s", parentNode.layer.ID[:12], units.HumanSizeWithPrecision(float64(parentNode.layer.UncompressedSize), 4), tags)) } - fmt.Fprintf(sb, "%s ID: %s Size: %7v%s\n", indent, parentNode.layer.ID[:12], units.HumanSizeWithPrecision(float64(parentNode.layer.UncompressedSize), 4), tags) } - return sb, nil + return tree.Print(), nil } -func imageTreeTraverseChildren(sb *strings.Builder, node *layerNode, prefix string, last bool) (*strings.Builder, error) { - numChildren := len(node.children) - if numChildren == 0 { - return sb, nil +func imageTreeTraverseChildren(node *layerNode, parent gotree.Tree) error { + var tags string + repoTags, err := node.repoTags() + if err != nil { + return err } - sb.WriteString(prefix) - - intend := imageTreeMiddleItem - if !last { - prefix += imageTreeContinueItem - } else { - intend = imageTreeLastItem - prefix += " " + if len(repoTags) > 0 { + tags = fmt.Sprintf(" Top Layer of: %s", repoTags) } + newNode := parent.Add(fmt.Sprintf("ID: %s Size: %7v%s", node.layer.ID[:12], units.HumanSizeWithPrecision(float64(node.layer.UncompressedSize), 4), tags)) + + if len(node.children) <= 1 { + newNode = parent + } for i := range node.children { child := node.children[i] - var tags string - repoTags, err := child.repoTags() - if err != nil { - return nil, err - } - if len(repoTags) > 0 { - tags = fmt.Sprintf(" Top Layer of: %s", repoTags) - } - fmt.Fprintf(sb, "%sID: %s Size: %7v%s\n", intend, child.layer.ID[:12], units.HumanSizeWithPrecision(float64(child.layer.UncompressedSize), 4), tags) - sb, err = imageTreeTraverseChildren(sb, child, prefix, i == numChildren-1) - if err != nil { - return nil, err + if err := imageTreeTraverseChildren(child, newNode); err != nil { + return err } } - return sb, nil + return nil } diff --git a/libimage/import.go b/libimage/import.go index 31dc4a0fa..4cce4c9ca 100644 --- a/libimage/import.go +++ b/libimage/import.go @@ -82,10 +82,11 @@ func (r *Runtime) Import(ctx context.Context, path string, options *ImportOption name := options.Tag if name == "" { - name, err = getImageDigest(ctx, srcRef, &r.systemContext) + name, err = getImageDigest(ctx, srcRef, r.systemContextCopy()) if err != nil { return "", err } + name = "sha256:" + name[1:] // strip leading "@" } destRef, err := storageTransport.Transport.ParseStoreReference(r.store, name) @@ -93,7 +94,7 @@ func (r *Runtime) Import(ctx context.Context, path string, options *ImportOption return "", err } - c, err := newCopier(&r.systemContext, &options.CopyOptions) + c, err := r.newCopier(&options.CopyOptions) if err != nil { return "", err } diff --git a/libimage/inspect.go b/libimage/inspect.go index ebcb7ccd0..349709155 100644 --- a/libimage/inspect.go +++ b/libimage/inspect.go @@ -3,20 +3,67 @@ package libimage import ( "context" "encoding/json" + "time" - libimageTypes "github.com/containers/common/libimage/types" "github.com/containers/image/v5/manifest" "github.com/containers/image/v5/types" + "github.com/opencontainers/go-digest" ociv1 "github.com/opencontainers/image-spec/specs-go/v1" "github.com/sirupsen/logrus" ) +// ImageData contains the inspected data of an image. +type ImageData struct { + ID string `json:"Id"` + Digest digest.Digest `json:"Digest"` + RepoTags []string `json:"RepoTags"` + RepoDigests []string `json:"RepoDigests"` + Parent string `json:"Parent"` + Comment string `json:"Comment"` + Created *time.Time `json:"Created"` + Config *ociv1.ImageConfig `json:"Config"` + Version string `json:"Version"` + Author string `json:"Author"` + Architecture string `json:"Architecture"` + Os string `json:"Os"` + Size int64 `json:"Size"` + VirtualSize int64 `json:"VirtualSize"` + GraphDriver *DriverData `json:"GraphDriver"` + RootFS *RootFS `json:"RootFS"` + Labels map[string]string `json:"Labels"` + Annotations map[string]string `json:"Annotations"` + ManifestType string `json:"ManifestType"` + User string `json:"User"` + History []ociv1.History `json:"History"` + NamesHistory []string `json:"NamesHistory"` + HealthCheck *manifest.Schema2HealthConfig `json:"Healthcheck,omitempty"` +} + +// DriverData includes data on the storage driver of the image. +type DriverData struct { + Name string `json:"Name"` + Data map[string]string `json:"Data"` +} + +// RootFS includes data on the root filesystem of the image. +type RootFS struct { + Type string `json:"Type"` + Layers []digest.Digest `json:"Layers"` +} + // Inspect inspects the image. Use `withSize` to also perform the // comparatively expensive size computation of the image. -func (i *Image) Inspect(ctx context.Context, withSize bool) (*libimageTypes.ImageData, error) { +func (i *Image) Inspect(ctx context.Context, withSize bool) (*ImageData, error) { logrus.Debugf("Inspecting image %s", i.ID()) if i.cached.completeInspectData != nil { + if withSize && i.cached.completeInspectData.Size == int64(-1) { + size, err := i.Size() + if err != nil { + return nil, err + } + i.cached.completeInspectData.Size = size + } return i.cached.completeInspectData, nil } @@ -54,7 +101,7 @@ func (i *Image) Inspect(ctx context.Context, withSize bool) (*libimageTypes.Imag } } - data := &libimageTypes.ImageData{ + data := &ImageData{ ID: i.ID(), RepoTags: repoTags, RepoDigests: repoDigests, @@ -68,7 +115,7 @@ func (i *Image) Inspect(ctx context.Context, withSize bool) (*libimageTypes.Imag VirtualSize: size, // TODO: they should be different (inherited from Podman) Digest: i.Digest(), Labels: info.Labels, - RootFS: &libimageTypes.RootFS{ + RootFS: &RootFS{ Type: ociImage.RootFS.Type, Layers: ociImage.RootFS.DiffIDs, }, @@ -108,15 +155,24 @@ func (i *Image) Inspect(ctx context.Context, withSize bool) (*libimageTypes.Imag } // Docker image - case manifest.DockerV2Schema2MediaType: - var dockerManifest manifest.Schema2Image - if err := json.Unmarshal(manifestRaw, &dockerManifest); err != nil { + case manifest.DockerV2Schema1MediaType, manifest.DockerV2Schema2MediaType: + rawConfig, err := i.rawConfigBlob(ctx) + if err != nil { + return nil, err + } + var dockerManifest manifest.Schema2V1Image + if err := json.Unmarshal(rawConfig, &dockerManifest); err != nil { return nil, err } data.Comment = dockerManifest.Comment data.HealthCheck = dockerManifest.ContainerConfig.Healthcheck } + if data.Annotations == nil { + // Podman compat + data.Annotations = make(map[string]string) + } + i.cached.completeInspectData = data return data, nil @@ -134,7 +190,7 @@ func (i *Image) inspectInfo(ctx context.Context) (*types.ImageInspectInfo, error return nil, err } - img, err := ref.NewImage(ctx, &i.runtime.systemContext) + img, err := ref.NewImage(ctx, i.runtime.systemContextCopy()) if err != nil { return nil, err } diff --git a/libimage/load.go b/libimage/load.go index cfee2740c..c606aca5b 100644 --- a/libimage/load.go +++ b/libimage/load.go @@ -3,11 +3,13 @@ package libimage import ( "context" "errors" + "os" dirTransport "github.com/containers/image/v5/directory" dockerArchiveTransport "github.com/containers/image/v5/docker/archive" ociArchiveTransport "github.com/containers/image/v5/oci/archive" ociTransport "github.com/containers/image/v5/oci/layout" + "github.com/containers/image/v5/types" "github.com/sirupsen/logrus" ) @@ -33,6 +35,7 @@ func (r *Runtime) Load(ctx context.Context, path string, options *LoadOptions) ( for _, f := range []func() ([]string, error){ // OCI func() ([]string, error) { + logrus.Debugf("-> Attempting to load %q as an OCI directory", path) ref, err := ociTransport.NewReference(path, "") if err != nil { return nil, err @@ -42,6 +45,7 @@ func (r *Runtime) Load(ctx context.Context, path string, options *LoadOptions) ( // OCI-ARCHIVE func() ([]string, error) { + logrus.Debugf("-> Attempting to load %q as an OCI archive", path) ref, err := ociArchiveTransport.NewReference(path, "") if err != nil { return nil, err @@ -51,6 +55,7 @@ func (r *Runtime) Load(ctx context.Context, path string, options *LoadOptions) ( // DIR func() ([]string, error) { + logrus.Debugf("-> Attempting to load %q as a Docker dir", path) ref, err := dirTransport.NewReference(path) if err != nil { return nil, err @@ -60,11 +65,12 @@ func (r *Runtime) Load(ctx context.Context, path string, options *LoadOptions) ( // DOCKER-ARCHIVE func() ([]string, error) { + logrus.Debugf("-> Attempting to load %q as a Docker archive", path) ref, err := dockerArchiveTransport.ParseReference(path) if err != nil { return nil, err } - return r.copyFromDockerArchive(ctx, ref, &options.CopyOptions) + return r.loadMultiImageDockerArchive(ctx, ref, &options.CopyOptions) }, // Give a decent error message if nothing above worked. @@ -81,3 +87,39 @@ func (r *Runtime) Load(ctx context.Context, path string, options *LoadOptions) ( return nil, loadError } + +// loadMultiImageDockerArchive loads the docker archive specified by ref. In +// case the path@reference notation was used, only the specifiec image will be +// loaded. Otherwise, all images will be loaded. +func (r *Runtime) loadMultiImageDockerArchive(ctx context.Context, ref types.ImageReference, options *CopyOptions) ([]string, error) { + // If we cannot stat the path, it either does not exist OR the correct + // syntax to reference an image within the archive was used, so we + // should. + path := ref.StringWithinTransport() + if _, err := os.Stat(path); err != nil { + return r.copyFromDockerArchive(ctx, ref, options) + } + + reader, err := dockerArchiveTransport.NewReader(r.systemContextCopy(), path) + if err != nil { + return nil, err + } + + refLists, err := reader.List() + if err != nil { + return nil, err + } + + var copiedImages []string + for _, list := range refLists { + for _, listRef := range list { + names, err := r.copyFromDockerArchiveReaderReference(ctx, reader, listRef, options) + if err != nil { + return nil, err + } + copiedImages = append(copiedImages, names...) + } + } + + return copiedImages, nil +} diff --git a/libimage/manifest_list.go b/libimage/manifest_list.go new file mode 100644 index 000000000..72a2cf55f --- /dev/null +++ b/libimage/manifest_list.go @@ -0,0 +1,389 @@ +package libimage + +import ( + "context" + "fmt" + + "github.com/containers/common/libimage/manifests" + imageCopy "github.com/containers/image/v5/copy" + "github.com/containers/image/v5/docker" + "github.com/containers/image/v5/manifest" + "github.com/containers/image/v5/transports/alltransports" + "github.com/containers/image/v5/types" + "github.com/containers/storage" + "github.com/opencontainers/go-digest" + "github.com/pkg/errors" +) + +// NOTE: the abstractions and APIs here are a first step to further merge +// `libimage/manifests` into `libimage`. + +// ManifestList represents a manifest list (Docker) or an image index (OCI) in +// the local containers storage. +type ManifestList struct { + // NOTE: the *List* suffix is intentional as the term "manifest" is + // used ambiguously across the ecosystem. It may refer to the (JSON) + // manifest of an ordinary image OR to a manifest *list* (Docker) or to + // image index (OCI). + // It's a bit more work when typing but without ambiguity. + + // The underlying image in the containers storage. + image *Image + + // The underlying manifest list. + list manifests.List +} + +// ID returns the ID of the manifest list. +func (m *ManifestList) ID() string { + return m.image.ID() +} + +// CreateManifestList creates a new empty manifest list with the specified +// name. +func (r *Runtime) CreateManifestList(name string) (*ManifestList, error) { + normalized, err := NormalizeName(name) + if err != nil { + return nil, err + } + + list := manifests.Create() + listID, err := list.SaveToImage(r.store, "", []string{normalized.String()}, manifest.DockerV2ListMediaType) + if err != nil { + return nil, err + } + + mList, err := r.LookupManifestList(listID) + if err != nil { + return nil, err + } + + return mList, nil +} + +// LookupManifestList looks up a manifest list with the specified name in the +// containers storage. +func (r *Runtime) LookupManifestList(name string) (*ManifestList, error) { + image, list, err := r.lookupManifestList(name) + if err != nil { + return nil, err + } + return &ManifestList{image: image, list: list}, nil +} + +func (r *Runtime) lookupManifestList(name string) (*Image, manifests.List, error) { + image, _, err := r.LookupImage(name, &LookupImageOptions{IgnorePlatform: true}) + if err != nil { + return nil, nil, err + } + if err := image.reload(); err != nil { + return nil, nil, err + } + list, err := image.getManifestList() + if err != nil { + return nil, nil, err + } + return image, list, nil +} + +// ToManifestList converts the image into a manifest list. An error is thrown +// if the image is no manifest list. +func (i *Image) ToManifestList() (*ManifestList, error) { + list, err := i.getManifestList() + if err != nil { + return nil, err + } + return &ManifestList{image: i, list: list}, nil +} + +// LookupInstance looks up an instance of the manifest list matching the +// specified platform. The local machine's platform is used if left empty. +func (m *ManifestList) LookupInstance(ctx context.Context, architecture, os, variant string) (*Image, error) { + sys := m.image.runtime.systemContextCopy() + if architecture != "" { + sys.ArchitectureChoice = architecture + } + if os != "" { + sys.OSChoice = os + } + if architecture != "" { + sys.VariantChoice = variant + } + + // Now look at the *manifest* and select a matching instance. + rawManifest, manifestType, err := m.image.Manifest(ctx) + if err != nil { + return nil, err + } + list, err := manifest.ListFromBlob(rawManifest, manifestType) + if err != nil { + return nil, err + } + instanceDigest, err := list.ChooseInstance(sys) + if err != nil { + return nil, err + } + + allImages, err := m.image.runtime.ListImages(ctx, nil, nil) + if err != nil { + return nil, err + } + + for _, image := range allImages { + for _, imageDigest := range append(image.Digests(), image.Digest()) { + if imageDigest == instanceDigest { + return image, nil + } + } + } + + return nil, errors.Wrapf(storage.ErrImageUnknown, "could not find image instance %s of manifest list %s in local containers storage", instanceDigest, m.ID()) +} + +// Saves the specified manifest list and reloads it from storage with the new ID. +func (m *ManifestList) saveAndReload() error { + newID, err := m.list.SaveToImage(m.image.runtime.store, m.image.ID(), nil, "") + if err != nil { + return err + } + + // Make sure to reload the image from the containers storage to fetch + // the latest data (e.g., new or delete digests). + if err := m.image.reload(); err != nil { + return err + } + image, list, err := m.image.runtime.lookupManifestList(newID) + if err != nil { + return err + } + m.image = image + m.list = list + return nil +} + +// getManifestList is a helper to obtain a manifest list +func (i *Image) getManifestList() (manifests.List, error) { + _, list, err := manifests.LoadFromImage(i.runtime.store, i.ID()) + return list, err +} + +// IsManifestList returns true if the image is a manifest list (Docker) or an +// image index (OCI). This information may be critical to make certain +// execution paths more robust (e.g., suppress certain errors). +func (i *Image) IsManifestList(ctx context.Context) (bool, error) { + ref, err := i.StorageReference() + if err != nil { + return false, err + } + imgRef, err := ref.NewImageSource(ctx, i.runtime.systemContextCopy()) + if err != nil { + return false, err + } + _, manifestType, err := imgRef.GetManifest(ctx, nil) + if err != nil { + return false, err + } + return manifest.MIMETypeIsMultiImage(manifestType), nil +} + +// Inspect returns a dockerized version of the manifest list. +func (m *ManifestList) Inspect() (*manifest.Schema2List, error) { + return m.list.Docker(), nil +} + +// Options for adding a manifest list. +type ManifestListAddOptions struct { + // Add all images to the list if the to-be-added image itself is a + // manifest list. + All bool `json:"all"` + // containers-auth.json(5) file to use when authenticating against + // container registries. + AuthFilePath string + // Path to the certificates directory. + CertDirPath string + // Allow contacting registries over HTTP, or HTTPS with failed TLS + // verification. Note that this does not affect other TLS connections. + InsecureSkipTLSVerify types.OptionalBool + // Username to use when authenticating at a container registry. + Username string + // Password to use when authenticating at a container registry. + Password string +} + +// Add adds one or more manifests to the manifest list and returns the digest +// of the added instance. +func (m *ManifestList) Add(ctx context.Context, name string, options *ManifestListAddOptions) (digest.Digest, error) { + if options == nil { + options = &ManifestListAddOptions{} + } + + ref, err := alltransports.ParseImageName(name) + if err != nil { + withDocker := fmt.Sprintf("%s://%s", docker.Transport.Name(), name) + ref, err = alltransports.ParseImageName(withDocker) + if err != nil { + return "", err + } + } + + // Now massage in the copy-related options into the system context. + systemContext := m.image.runtime.systemContextCopy() + if options.AuthFilePath != "" { + systemContext.AuthFilePath = options.AuthFilePath + } + if options.CertDirPath != "" { + systemContext.DockerCertPath = options.CertDirPath + } + if options.InsecureSkipTLSVerify != types.OptionalBoolUndefined { + systemContext.DockerInsecureSkipTLSVerify = options.InsecureSkipTLSVerify + systemContext.OCIInsecureSkipTLSVerify = options.InsecureSkipTLSVerify == types.OptionalBoolTrue + systemContext.DockerDaemonInsecureSkipTLSVerify = options.InsecureSkipTLSVerify == types.OptionalBoolTrue + } + if options.Username != "" { + systemContext.DockerAuthConfig = &types.DockerAuthConfig{ + Username: options.Username, + Password: options.Password, + } + } + + newDigest, err := m.list.Add(ctx, systemContext, ref, options.All) + if err != nil { + return "", err + } + + // Write the changes to disk. + if err := m.saveAndReload(); err != nil { + return "", err + } + return newDigest, nil +} + +// Options for annotationg a manifest list. +type ManifestListAnnotateOptions struct { + // Add the specified annotations to the added image. + Annotations map[string]string + // Add the specified architecture to the added image. + Architecture string + // Add the specified features to the added image. + Features []string + // Add the specified OS to the added image. + OS string + // Add the specified OS features to the added image. + OSFeatures []string + // Add the specified OS version to the added image. + OSVersion string + // Add the specified variant to the added image. + Variant string +} + +// Annotate an image instance specified by `d` in the manifest list. +func (m *ManifestList) AnnotateInstance(d digest.Digest, options *ManifestListAnnotateOptions) error { + if options == nil { + return nil + } + + if len(options.OS) > 0 { + if err := m.list.SetOS(d, options.OS); err != nil { + return err + } + } + if len(options.OSVersion) > 0 { + if err := m.list.SetOSVersion(d, options.OSVersion); err != nil { + return err + } + } + if len(options.Features) > 0 { + if err := m.list.SetFeatures(d, options.Features); err != nil { + return err + } + } + if len(options.OSFeatures) > 0 { + if err := m.list.SetOSFeatures(d, options.OSFeatures); err != nil { + return err + } + } + if len(options.Architecture) > 0 { + if err := m.list.SetArchitecture(d, options.Architecture); err != nil { + return err + } + } + if len(options.Variant) > 0 { + if err := m.list.SetVariant(d, options.Variant); err != nil { + return err + } + } + if len(options.Annotations) > 0 { + if err := m.list.SetAnnotations(&d, options.Annotations); err != nil { + return err + } + } + + // Write the changes to disk. + if err := m.saveAndReload(); err != nil { + return err + } + return nil +} + +// RemoveInstance removes the instance specified by `d` from the manifest list. +// Returns the new ID of the image. +func (m *ManifestList) RemoveInstance(d digest.Digest) error { + if err := m.list.Remove(d); err != nil { + return err + } + + // Write the changes to disk. + if err := m.saveAndReload(); err != nil { + return err + } + return nil +} + +// ManifestListPushOptions allow for customizing pushing a manifest list. +type ManifestListPushOptions struct { + CopyOptions + + // For tweaking the list selection. + ImageListSelection imageCopy.ImageListSelection + // Use when selecting only specific imags. + Instances []digest.Digest +} + +// Push pushes a manifest to the specified destination. +func (m *ManifestList) Push(ctx context.Context, destination string, options *ManifestListPushOptions) (digest.Digest, error) { + if options == nil { + options = &ManifestListPushOptions{} + } + + dest, err := alltransports.ParseImageName(destination) + if err != nil { + oldErr := err + dest, err = alltransports.ParseImageName("docker://" + destination) + if err != nil { + return "", oldErr + } + } + + // NOTE: we're using the logic in copier to create a proper + // types.SystemContext. This prevents us from having an error prone + // code duplicate here. + copier, err := m.image.runtime.newCopier(&options.CopyOptions) + if err != nil { + return "", err + } + defer copier.close() + + pushOptions := manifests.PushOptions{ + Store: m.image.runtime.store, + SystemContext: copier.systemContext, + ImageListSelection: options.ImageListSelection, + Instances: options.Instances, + ReportWriter: options.Writer, + SignBy: options.SignBy, + RemoveSignatures: options.RemoveSignatures, + ManifestType: options.ManifestMIMEType, + } + + _, d, err := m.list.Push(ctx, dest, pushOptions) + return d, err +} diff --git a/libimage/normalize.go b/libimage/normalize.go index 5ba94fd48..03d2456de 100644 --- a/libimage/normalize.go +++ b/libimage/normalize.go @@ -17,7 +17,7 @@ func NormalizeName(name string) (reference.Named, error) { // NOTE: this code is in symmetrie with containers/image/pkg/shortnames. ref, err := reference.Parse(name) if err != nil { - return nil, err + return nil, errors.Wrapf(err, "error normalizing name %q", name) } named, ok := ref.(reference.Named) @@ -61,16 +61,24 @@ type NameTagPair struct { Name string // Tag of the RepoTag. Maybe "". Tag string + + // for internal use + named reference.Named } // ToNameTagsPairs splits repoTags into name&tag pairs. // Guaranteed to return at least one pair. -func ToNameTagPairs(repoTags []reference.NamedTagged) ([]NameTagPair, error) { +func ToNameTagPairs(repoTags []reference.Named) ([]NameTagPair, error) { none := "" var pairs []NameTagPair - for _, named := range repoTags { - pair := NameTagPair{Name: named.Name(), Tag: none} + for i, named := range repoTags { + pair := NameTagPair{ + Name: named.Name(), + Tag: none, + named: repoTags[i], + } + if tagged, isTagged := named.(reference.NamedTagged); isTagged { pair.Tag = tagged.Tag() } diff --git a/libimage/oci.go b/libimage/oci.go index d7c6ce1e4..b88d6613d 100644 --- a/libimage/oci.go +++ b/libimage/oci.go @@ -16,7 +16,7 @@ func (i *Image) toOCI(ctx context.Context) (*ociv1.Image, error) { return nil, err } - img, err := ref.NewImage(ctx, &i.runtime.systemContext) + img, err := ref.NewImage(ctx, i.runtime.systemContextCopy()) if err != nil { return nil, err } diff --git a/libimage/pull.go b/libimage/pull.go index e38565d47..b92a5e15e 100644 --- a/libimage/pull.go +++ b/libimage/pull.go @@ -6,7 +6,7 @@ import ( "io" "strings" - libimageTypes "github.com/containers/common/libimage/types" + "github.com/containers/common/pkg/config" dirTransport "github.com/containers/image/v5/directory" dockerTransport "github.com/containers/image/v5/docker" dockerArchiveTransport "github.com/containers/image/v5/docker/archive" @@ -36,7 +36,7 @@ type PullOptions struct { // name will be treated as a reference to a registry (i.e., docker transport). // // Note that pullPolicy is only used when pulling from a container registry but -// it *must* be different than the default value `PullPolicyUnsupported`. This +// it *must* be different than the default value `config.PullPolicyUnsupported`. This // way, callers are forced to decide on the pull behaviour. The reasoning // behind is that some (commands of some) tools have different default pull // policies (e.g., buildah-bud versus podman-build). Making the pull-policy @@ -45,8 +45,8 @@ type PullOptions struct { // The errror is storage.ErrImageUnknown iff the pull policy is set to "never" // and no local image has been found. This allows for an easier integration // into some users of this package (e.g., Buildah). -func (r *Runtime) Pull(ctx context.Context, name string, pullPolicy libimageTypes.PullPolicy, options *PullOptions) ([]*Image, error) { - logrus.Debugf("Pulling image %s", name) +func (r *Runtime) Pull(ctx context.Context, name string, pullPolicy config.PullPolicy, options *PullOptions) ([]*Image, error) { + logrus.Debugf("Pulling image %s (policy: %s)", name, pullPolicy) if options == nil { options = &PullOptions{} @@ -57,16 +57,13 @@ func (r *Runtime) Pull(ctx context.Context, name string, pullPolicy libimageType // If the image clearly refers to a local one, we can look it up directly. // In fact, we need to since they are not parseable. if strings.HasPrefix(name, "sha256:") || (len(name) == 64 && !strings.Contains(name, "/.:@")) { - if pullPolicy == libimageTypes.PullPolicyAlways { + if pullPolicy == config.PullPolicyAlways { return nil, errors.Errorf("pull policy is always but image has been referred to by ID (%s)", name) } local, _, err := r.LookupImage(name, nil) if err != nil { return nil, err } - if local == nil { - return nil, errors.Wrap(storage.ErrImageUnknown, name) - } return []*Image{local}, err } @@ -121,14 +118,12 @@ func (r *Runtime) Pull(ctx context.Context, name string, pullPolicy libimageType } localImages := []*Image{} + lookupOptions := &LookupImageOptions{IgnorePlatform: true} for _, name := range pulledImages { - local, _, err := r.LookupImage(name, nil) + local, _, err := r.LookupImage(name, lookupOptions) if err != nil { return nil, errors.Wrapf(err, "error locating pulled image %q name in containers storage", name) } - if local == nil { - return nil, errors.Wrap(storage.ErrImageUnknown, name) - } localImages = append(localImages, local) } @@ -138,7 +133,7 @@ func (r *Runtime) Pull(ctx context.Context, name string, pullPolicy libimageType // copyFromDefault is the default copier for a number of transports. Other // transports require some specific dancing, sometimes Yoga. func (r *Runtime) copyFromDefault(ctx context.Context, ref types.ImageReference, options *CopyOptions) ([]string, error) { - c, err := newCopier(&r.systemContext, options) + c, err := r.newCopier(options) if err != nil { return nil, err } @@ -228,21 +223,25 @@ func (r *Runtime) storageReferencesReferencesFromArchiveReader(ctx context.Conte return references, imageNames, nil } -// copyFromDockerArchive copies one or more images from the specified -// reference. +// copyFromDockerArchive copies one image from the specified reference. func (r *Runtime) copyFromDockerArchive(ctx context.Context, ref types.ImageReference, options *CopyOptions) ([]string, error) { - c, err := newCopier(&r.systemContext, options) + // There may be more than one image inside the docker archive, so we + // need a quick glimpse inside. + reader, readerRef, err := dockerArchiveTransport.NewReaderForReference(&r.systemContext, ref) if err != nil { return nil, err } - defer c.close() - // There may be more than one image inside the docker archive, so we - // need a quick glimpse inside. - reader, readerRef, err := dockerArchiveTransport.NewReaderForReference(&r.systemContext, ref) + return r.copyFromDockerArchiveReaderReference(ctx, reader, readerRef, options) +} + +// copyFromDockerArchiveReaderReference copies the specified readerRef from reader. +func (r *Runtime) copyFromDockerArchiveReaderReference(ctx context.Context, reader *dockerArchiveTransport.Reader, readerRef types.ImageReference, options *CopyOptions) ([]string, error) { + c, err := r.newCopier(options) if err != nil { return nil, err } + defer c.close() // Get a slice of storage references we can copy. references, destNames, err := r.storageReferencesReferencesFromArchiveReader(ctx, readerRef, reader) @@ -265,7 +264,7 @@ func (r *Runtime) copyFromDockerArchive(ctx context.Context, ref types.ImageRefe // can later be used to look up the image in the local containers storage. // // If options.All is set, all tags from the specified registry will be pulled. -func (r *Runtime) copyFromRegistry(ctx context.Context, ref types.ImageReference, inputName string, pullPolicy libimageTypes.PullPolicy, options *PullOptions) ([]string, error) { +func (r *Runtime) copyFromRegistry(ctx context.Context, ref types.ImageReference, inputName string, pullPolicy config.PullPolicy, options *PullOptions) ([]string, error) { // Sanity check. if err := pullPolicy.Validate(); err != nil { return nil, err @@ -283,6 +282,12 @@ func (r *Runtime) copyFromRegistry(ctx context.Context, ref types.ImageReference pulledTags := []string{} for _, tag := range tags { + select { // Let's be gentle with Podman remote. + case <-ctx.Done(): + return nil, errors.Errorf("pulling cancelled") + default: + // We can continue. + } tagged, err := reference.WithTag(named, tag) if err != nil { return nil, errors.Wrapf(err, "error creating tagged reference (name %s, tag %s)", named.String(), tag) @@ -301,7 +306,7 @@ func (r *Runtime) copyFromRegistry(ctx context.Context, ref types.ImageReference // from a registry. On successful pull it returns the used fully-qualified // name that can later be used to look up the image in the local containers // storage. -func (r *Runtime) copySingleImageFromRegistry(ctx context.Context, imageName string, pullPolicy libimageTypes.PullPolicy, options *PullOptions) ([]string, error) { +func (r *Runtime) copySingleImageFromRegistry(ctx context.Context, imageName string, pullPolicy config.PullPolicy, options *PullOptions) ([]string, error) { // Sanity check. if err := pullPolicy.Validate(); err != nil { return nil, err @@ -318,11 +323,11 @@ func (r *Runtime) copySingleImageFromRegistry(ctx context.Context, imageName str // If there's already a local image "localhost/foo", then we should // attempt pulling that instead of doing the full short-name dance. localImage, resolvedImageName, err = r.LookupImage(imageName, nil) - if err != nil { + if err != nil && errors.Cause(err) != storage.ErrImageUnknown { return nil, errors.Wrap(err, "error looking up local image") } - if pullPolicy == libimageTypes.PullPolicyNever { + if pullPolicy == config.PullPolicyNever { if localImage != nil { logrus.Debugf("Pull policy %q but no local image has been found for %s", pullPolicy, imageName) return []string{resolvedImageName}, nil @@ -331,14 +336,14 @@ func (r *Runtime) copySingleImageFromRegistry(ctx context.Context, imageName str return nil, errors.Wrap(storage.ErrImageUnknown, imageName) } - if pullPolicy == libimageTypes.PullPolicyMissing && localImage != nil { + if pullPolicy == config.PullPolicyMissing && localImage != nil { return []string{resolvedImageName}, nil } // If we looked up the image by ID, we cannot really pull from anywhere. if localImage != nil && strings.HasPrefix(localImage.ID(), imageName) { switch pullPolicy { - case libimageTypes.PullPolicyAlways: + case config.PullPolicyAlways: return nil, errors.Errorf("pull policy is always but image has been referred to by ID (%s)", imageName) default: return []string{resolvedImageName}, nil @@ -353,7 +358,9 @@ func (r *Runtime) copySingleImageFromRegistry(ctx context.Context, imageName str } imageName = resolvedImageName } - resolved, err := shortnames.Resolve(&r.systemContext, imageName) + + sys := r.systemContextCopy() + resolved, err := shortnames.Resolve(sys, imageName) if err != nil { return nil, err } @@ -382,7 +389,7 @@ func (r *Runtime) copySingleImageFromRegistry(ctx context.Context, imageName str return nil } - c, err := newCopier(&r.systemContext, &options.CopyOptions) + c, err := r.newCopier(&options.CopyOptions) if err != nil { return nil, err } @@ -397,7 +404,7 @@ func (r *Runtime) copySingleImageFromRegistry(ctx context.Context, imageName str return nil, err } - if pullPolicy == libimageTypes.PullPolicyNewer && localImage != nil { + if pullPolicy == config.PullPolicyNewer && localImage != nil { isNewer, err := localImage.HasDifferentDigest(ctx, srcRef) if err != nil { pullErrors = append(pullErrors, err) @@ -439,7 +446,7 @@ func (r *Runtime) copySingleImageFromRegistry(ctx context.Context, imageName str return []string{candidate.Value.String()}, nil } - if localImage != nil && pullPolicy == libimageTypes.PullPolicyNewer { + if localImage != nil && pullPolicy == config.PullPolicyNewer { return []string{resolvedImageName}, nil } diff --git a/libimage/pull_policy.go b/libimage/pull_policy.go deleted file mode 100644 index 45c25b186..000000000 --- a/libimage/pull_policy.go +++ /dev/null @@ -1,90 +0,0 @@ -package libimage - -import ( - "fmt" - - "github.com/pkg/errors" -) - -// PullPolicy determines how and which images are being pulled from a container -// registry (i.e., docker transport only). -// -// Supported string values are: -// * "always" <-> PullPolicyAlways -// * "missing" <-> PullPolicyMissing -// * "newer" <-> PullPolicyNewer -// * "never" <-> PullPolicyNever -type PullPolicy int - -const ( - // This default value forces callers to setup a custom default policy. - // Some tools use different policies (e.g., buildah-bud versus - // podman-build). - PullPolicyUnsupported PullPolicy = iota - // Always pull the image. - PullPolicyAlways - // Pull the image only if it could not be found in the local containers - // storage. - PullPolicyMissing - // Pull if the image on the registry is new than the one in the local - // containers storage. An image is considered to be newer when the - // digests are different. Comparing the time stamps is prone to - // errors. - PullPolicyNewer - // Never pull the image but use the one from the local containers - // storage. - PullPolicyNever -) - -// String converts a PullPolicy into a string. -// -// Supported string values are: -// * "always" <-> PullPolicyAlways -// * "missing" <-> PullPolicyMissing -// * "newer" <-> PullPolicyNewer -// * "never" <-> PullPolicyNever -func (p PullPolicy) String() string { - switch p { - case PullPolicyAlways: - return "always" - case PullPolicyMissing: - return "missing" - case PullPolicyNewer: - return "newer" - case PullPolicyNever: - return "never" - } - return fmt.Sprintf("unrecognized policy %d", p) -} - -// Validate returns if the pull policy is not supported. -func (p PullPolicy) Validate() error { - switch p { - case PullPolicyAlways, PullPolicyMissing, PullPolicyNewer, PullPolicyNever: - return nil - default: - return errors.Errorf("unsupported pull policy %d", p) - } -} - -// ParsePullPolicy parses the string into a pull policy. -// -// Supported string values are: -// * "always" <-> PullPolicyAlways -// * "missing" <-> PullPolicyMissing -// * "newer" <-> PullPolicyNewer -// * "never" <-> PullPolicyNever -func ParsePullPolicy(s string) (PullPolicy, error) { - switch s { - case "always": - return PullPolicyAlways, nil - case "missing": - return PullPolicyMissing, nil - case "newer": - return PullPolicyNewer, nil - case "never": - return PullPolicyMissing, nil - default: - return PullPolicyUnsupported, errors.Errorf("unsupported pull policy %q", s) - } -} diff --git a/libimage/push.go b/libimage/push.go index 47ab2ceba..8ff5d5ffd 100644 --- a/libimage/push.go +++ b/libimage/push.go @@ -6,8 +6,6 @@ import ( dockerArchiveTransport "github.com/containers/image/v5/docker/archive" "github.com/containers/image/v5/docker/reference" "github.com/containers/image/v5/transports/alltransports" - "github.com/containers/storage" - "github.com/pkg/errors" "github.com/sirupsen/logrus" ) @@ -35,9 +33,6 @@ func (r *Runtime) Push(ctx context.Context, source, destination string, options if err != nil { return nil, err } - if image == nil { - return nil, errors.Wrap(storage.ErrImageUnknown, source) - } srcRef, err := image.StorageReference() if err != nil { @@ -77,7 +72,7 @@ func (r *Runtime) Push(ctx context.Context, source, destination string, options } } - c, err := newCopier(&r.systemContext, &options.CopyOptions) + c, err := r.newCopier(&options.CopyOptions) if err != nil { return nil, err } diff --git a/libimage/runtime.go b/libimage/runtime.go index 3fcf236ee..4e6bd2cf2 100644 --- a/libimage/runtime.go +++ b/libimage/runtime.go @@ -13,7 +13,7 @@ import ( "github.com/containers/image/v5/transports/alltransports" "github.com/containers/image/v5/types" "github.com/containers/storage" - "github.com/hashicorp/go-multierror" + deepcopy "github.com/jinzhu/copier" "github.com/pkg/errors" "github.com/sirupsen/logrus" ) @@ -46,9 +46,13 @@ type Runtime struct { // Global system context. No pointer to simplify copying and modifying // it. systemContext types.SystemContext - // maps an image ID to an Image pointer. Allows for aggressive - // caching. - imageIDmap map[string]*Image +} + +// Returns a copy of the runtime's system context. +func (r *Runtime) systemContextCopy() *types.SystemContext { + var sys types.SystemContext + deepcopy.Copy(&sys, &r.systemContext) + return &sys } // RuntimeFromStore returns a Runtime for the specified store. @@ -73,7 +77,6 @@ func RuntimeFromStore(store storage.Store, options *RuntimeOptions) (*Runtime, e return &Runtime{ store: store, systemContext: systemContext, - imageIDmap: make(map[string]*Image), }, nil } @@ -101,24 +104,21 @@ func (r *Runtime) Shutdown(force bool) error { // storageToImage transforms a storage.Image to an Image. func (r *Runtime) storageToImage(storageImage *storage.Image, ref types.ImageReference) *Image { - image, exists := r.imageIDmap[storageImage.ID] - if exists { - return image - } - image = &Image{ + return &Image{ runtime: r, storageImage: storageImage, storageReference: ref, } - r.imageIDmap[storageImage.ID] = image - return image } // Exists returns true if the specicifed image exists in the local containers // storage. func (r *Runtime) Exists(name string) (bool, error) { - image, _, err := r.LookupImage(name, nil) - return image != nil, err + image, _, err := r.LookupImage(name, &LookupImageOptions{IgnorePlatform: true}) + if err != nil && errors.Cause(err) != storage.ErrImageUnknown { + return false, err + } + return image != nil, nil } // LookupImageOptions allow for customizing local image lookups. @@ -131,9 +131,9 @@ type LookupImageOptions struct { // Lookup Image looks up `name` in the local container storage matching the // specified SystemContext. Returns the image and the name it has been found -// with. Returns nil if no image has been found. Note that name may also use -// the `containers-storage:` prefix used to refer to the containers-storage -// transport. +// with. Note that name may also use the `containers-storage:` prefix used to +// refer to the containers-storage transport. Returns storage.ErrImageUnknown +// if the image could not be found. // // If the specified name uses the `containers-storage` transport, the resolved // name is empty. @@ -158,79 +158,44 @@ func (r *Runtime) LookupImage(name string, options *LookupImageOptions) (*Image, return r.storageToImage(img, storageRef), "", nil } - byDigest := false + originalName := name + idByDigest := false if strings.HasPrefix(name, "sha256:") { - byDigest = true + // Strip off the sha256 prefix so it can be parsed later on. + idByDigest = true name = strings.TrimPrefix(name, "sha256:") } - // Anonymouns function to lookup the provided image in the storage and - // check whether it's matching the system context. - findImage := func(input string) (*Image, error) { - img, err := r.store.Image(input) - if err != nil && errors.Cause(err) != storage.ErrImageUnknown { - return nil, err - } - if img == nil { - return nil, nil - } - ref, err := storageTransport.Transport.ParseStoreReference(r.store, img.ID) - if err != nil { - return nil, err - } - - if options.IgnorePlatform { - logrus.Debugf("Found image %q as %q in local containers storage", name, input) - return r.storageToImage(img, ref), nil - } - - matches, err := imageReferenceMatchesContext(context.Background(), ref, &r.systemContext) - if err != nil { - return nil, err - } - if !matches { - return nil, nil - } - // Also print the string within the storage transport. That - // may aid in debugging when using additional stores since we - // see explicitly where the store is and which driver (options) - // are used. - logrus.Debugf("Found image %q as %q in local containers storage (%s)", name, input, ref.StringWithinTransport()) - return r.storageToImage(img, ref), nil - } - // First, check if we have an exact match in the storage. Maybe an ID // or a fully-qualified image name. - img, err := findImage(name) + img, err := r.lookupImageInLocalStorage(name, name, options) if err != nil { return nil, "", err } if img != nil { - return img, name, nil + return img, originalName, nil } // If the name clearly referred to a local image, there's nothing we can // do anymore. - if storageRef != nil || byDigest { - return nil, "", nil + if storageRef != nil || idByDigest { + return nil, "", errors.Wrap(storage.ErrImageUnknown, originalName) } // Second, try out the candidates as resolved by shortnames. This takes // "localhost/" prefixed images into account as well. candidates, err := shortnames.ResolveLocally(&r.systemContext, name) if err != nil { - return nil, "", err + return nil, "", errors.Wrap(storage.ErrImageUnknown, originalName) } // Backwards compat: normalize to docker.io as some users may very well // rely on that. - dockerNamed, err := reference.ParseDockerRef(name) - if err != nil { - return nil, "", errors.Wrap(err, "error normalizing to docker.io") + if dockerNamed, err := reference.ParseDockerRef(name); err == nil { + candidates = append(candidates, dockerNamed) } - candidates = append(candidates, dockerNamed) for _, candidate := range candidates { - img, err := findImage(candidate.String()) + img, err := r.lookupImageInLocalStorage(name, candidate.String(), options) if err != nil { return nil, "", err } @@ -239,7 +204,161 @@ func (r *Runtime) LookupImage(name string, options *LookupImageOptions) (*Image, } } - return nil, "", nil + return r.lookupImageInDigestsAndRepoTags(originalName, options) +} + +// lookupImageInLocalStorage looks up the specified candidate for name in the +// storage and checks whether it's matching the system context. +func (r *Runtime) lookupImageInLocalStorage(name, candidate string, options *LookupImageOptions) (*Image, error) { + logrus.Debugf("Trying %q ...", candidate) + img, err := r.store.Image(candidate) + if err != nil && errors.Cause(err) != storage.ErrImageUnknown { + return nil, err + } + if img == nil { + return nil, nil + } + ref, err := storageTransport.Transport.ParseStoreReference(r.store, img.ID) + if err != nil { + return nil, err + } + + image := r.storageToImage(img, ref) + if options.IgnorePlatform { + logrus.Debugf("Found image %q as %q in local containers storage", name, candidate) + return image, nil + } + + // If we referenced a manifest list, we need to check whether we can + // find a matching instance in the local containers storage. + isManifestList, err := image.IsManifestList(context.Background()) + if err != nil { + return nil, err + } + if isManifestList { + manifestList, err := image.ToManifestList() + if err != nil { + return nil, err + } + image, err = manifestList.LookupInstance(context.Background(), "", "", "") + if err != nil { + return nil, err + } + ref, err = storageTransport.Transport.ParseStoreReference(r.store, "@"+image.ID()) + if err != nil { + return nil, err + } + } + + matches, err := imageReferenceMatchesContext(context.Background(), ref, &r.systemContext) + if err != nil { + return nil, err + } + + // NOTE: if the user referenced by ID we must optimistically assume + // that they know what they're doing. Given, we already did the + // manifest limbo above, we may already have resolved it. + if !matches && !strings.HasPrefix(image.ID(), candidate) { + return nil, nil + } + // Also print the string within the storage transport. That may aid in + // debugging when using additional stores since we see explicitly where + // the store is and which driver (options) are used. + logrus.Debugf("Found image %q as %q in local containers storage (%s)", name, candidate, ref.StringWithinTransport()) + return image, nil +} + +// lookupImageInDigestsAndRepoTags attempts to match name against any image in +// the local containers storage. If name is digested, it will be compared +// against image digests. Otherwise, it will be looked up in the repo tags. +func (r *Runtime) lookupImageInDigestsAndRepoTags(name string, options *LookupImageOptions) (*Image, string, error) { + // Until now, we've tried very hard to find an image but now it is time + // for limbo. If the image includes a digest that we couldn't detect + // verbatim in the storage, we must have a look at all digests of all + // images. Those may change over time (e.g., via manifest lists). + // Both Podman and Buildah want us to do that dance. + allImages, err := r.ListImages(context.Background(), nil, nil) + if err != nil { + return nil, "", err + } + + if !shortnames.IsShortName(name) { + named, err := reference.ParseNormalizedNamed(name) + if err != nil { + return nil, "", err + } + digested, hasDigest := named.(reference.Digested) + if !hasDigest { + return nil, "", errors.Wrap(storage.ErrImageUnknown, name) + } + + logrus.Debug("Looking for image with matching recorded digests") + digest := digested.Digest() + for _, image := range allImages { + for _, d := range image.Digests() { + if d == digest { + return image, name, nil + } + } + } + + return nil, "", errors.Wrap(storage.ErrImageUnknown, name) + } + + // Podman compat: if we're looking for a short name but couldn't + // resolve it via the registries.conf dance, we need to look at *all* + // images and check if the name we're looking for matches a repo tag. + // Split the name into a repo/tag pair + split := strings.SplitN(name, ":", 2) + repo := split[0] + tag := "" + if len(split) == 2 { + tag = split[1] + } + for _, image := range allImages { + named, err := image.inRepoTags(repo, tag) + if err != nil { + return nil, "", err + } + if named == nil { + continue + } + img, err := r.lookupImageInLocalStorage(name, named.String(), options) + if err != nil { + return nil, "", err + } + if img != nil { + return img, named.String(), err + } + } + + return nil, "", errors.Wrap(storage.ErrImageUnknown, name) +} + +// ResolveName resolves the specified name. If the name resolves to a local +// image, the fully resolved name will be returned. Otherwise, the name will +// be properly normalized. +// +// Note that an empty string is returned as is. +func (r *Runtime) ResolveName(name string) (string, error) { + if name == "" { + return "", nil + } + image, resolvedName, err := r.LookupImage(name, &LookupImageOptions{IgnorePlatform: true}) + if err != nil && errors.Cause(err) != storage.ErrImageUnknown { + return "", err + } + + if image != nil && !strings.HasPrefix(image.ID(), resolvedName) { + return resolvedName, err + } + + normalized, err := NormalizeName(name) + if err != nil { + return "", err + } + + return normalized.String(), nil } // imageReferenceMatchesContext return true if the specified reference matches @@ -301,9 +420,6 @@ func (r *Runtime) ListImages(ctx context.Context, names []string, options *ListI if err != nil { return nil, err } - if image == nil { - return nil, errors.Wrap(storage.ErrImageUnknown, name) - } images = append(images, image) } } else { @@ -330,8 +446,14 @@ func (r *Runtime) ListImages(ctx context.Context, names []string, options *ListI // RemoveImagesOptions allow for customizing image removal. type RemoveImagesOptions struct { - RemoveImageOptions - + // Force will remove all containers from the local storage that are + // using a removed image. Use RemoveContainerFunc for a custom logic. + // If set, all child images will be removed as well. + Force bool + // RemoveContainerFunc allows for a custom logic for removing + // containers using a specific image. By default, all containers in + // the local containers storage will be removed (if Force is set). + RemoveContainerFunc RemoveContainerFunc // Filters to filter the removed images. Supported filters are // * after,before,since=image // * dangling=true,false @@ -341,6 +463,11 @@ type RemoveImagesOptions struct { // * readonly=true,false // * reference=name[:tag] (wildcards allowed) Filters []string + // The RemoveImagesReport will include the size of the removed image. + // This information may be useful when pruning images to figure out how + // much space was freed. However, computing the size of an image is + // comparatively expensive, so it is made optional. + WithSize bool } // RemoveImages removes images specified by names. All images are expected to @@ -349,100 +476,98 @@ type RemoveImagesOptions struct { // If an image has more names than one name, the image will be untagged with // the specified name. RemoveImages returns a slice of untagged and removed // images. -func (r *Runtime) RemoveImages(ctx context.Context, names []string, options *RemoveImagesOptions) (untagged, removed []string, rmError error) { +// +// Note that most errors are non-fatal and collected into `rmErrors` return +// value. +func (r *Runtime) RemoveImages(ctx context.Context, names []string, options *RemoveImagesOptions) (reports []*RemoveImageReport, rmErrors []error) { if options == nil { options = &RemoveImagesOptions{} } - // deleteMe bundles an image with a possibly empty string value it has - // been looked up with. The string value is required to implement the - // untagging logic. + // The logic here may require some explanation. Image removal is + // surprisingly complex since it is recursive (intermediate parents are + // removed) and since multiple items in `names` may resolve to the + // *same* image. On top, the data in the containers storage is shared, + // so we need to be careful and the code must be robust. That is why + // users can only remove images via this function; the logic may be + // complex but the execution path is clear. + + // Bundle an image with a possible empty slice of names to untag. That + // allows for a decent untagging logic and to bundle multiple + // references to the same *Image (and circumvent consistency issues). type deleteMe struct { - image *Image - name string + image *Image + referencedBy []string } - var images []*deleteMe + appendError := func(err error) { + rmErrors = append(rmErrors, err) + } + + orderedIDs := []string{} // determinism and relative order + deleteMap := make(map[string]*deleteMe) // ID -> deleteMe + + // Look up images in the local containers storage and fill out + // orderedIDs and the deleteMap. switch { case len(names) > 0: lookupOptions := LookupImageOptions{IgnorePlatform: true} for _, name := range names { img, resolvedName, err := r.LookupImage(name, &lookupOptions) if err != nil { - return nil, nil, err + appendError(err) + continue } - if img == nil { - return nil, nil, errors.Wrap(storage.ErrImageUnknown, name) + dm, exists := deleteMap[img.ID()] + if !exists { + orderedIDs = append(orderedIDs, img.ID()) + dm = &deleteMe{image: img} + deleteMap[img.ID()] = dm } - images = append(images, &deleteMe{image: img, name: resolvedName}) + dm.referencedBy = append(dm.referencedBy, resolvedName) } - if len(images) == 0 { - return nil, nil, errors.New("no images found") + if len(orderedIDs) == 0 { + return nil, rmErrors } case len(options.Filters) > 0: filteredImages, err := r.ListImages(ctx, nil, &ListImagesOptions{Filters: options.Filters}) if err != nil { - return nil, nil, err + appendError(err) + return nil, rmErrors } for _, img := range filteredImages { - images = append(images, &deleteMe{image: img}) + orderedIDs = append(orderedIDs, img.ID()) + deleteMap[img.ID()] = &deleteMe{image: img} } } - // Now remove the images. - for _, delete := range images { - numNames := len(delete.image.Names()) - - skipRemove := false - if len(names) > 0 { - hasChildren, err := delete.image.HasChildren(ctx) - if err != nil { - rmError = multierror.Append(rmError, err) - continue - } - skipRemove = hasChildren + // Now remove the images in the given order. + rmMap := make(map[string]*RemoveImageReport) + for _, id := range orderedIDs { + del, exists := deleteMap[id] + if !exists { + appendError(errors.Errorf("internal error: ID %s not in found in image-deletion map", id)) + continue } - - if delete.name != "" { - untagged = append(untagged, delete.name) + if len(del.referencedBy) == 0 { + del.referencedBy = []string{""} } - - mustUntag := !options.Force && delete.name != "" && (numNames > 1 || skipRemove) - if mustUntag { - if err := delete.image.Untag(delete.name); err != nil { - rmError = multierror.Append(rmError, err) - continue - } - // If the untag did not reduce the image names, name - // must have been an ID in which case we should throw - // an error. UNLESS there is only one tag left. - newNumNames := len(delete.image.Names()) - if newNumNames == numNames && newNumNames != 1 { - err := errors.Errorf("unable to delete image %q by ID with more than one tag (%s): use force removal", delete.image.ID(), delete.image.Names()) - rmError = multierror.Append(rmError, err) - continue - } - - // If we deleted the last tag/name, we can continue - // removing the image. Otherwise, we mark it as - // untagged and need to continue. - if newNumNames >= 1 || skipRemove { + for _, ref := range del.referencedBy { + if err := del.image.remove(ctx, rmMap, ref, options); err != nil { + appendError(err) continue } } + } - if err := delete.image.Remove(ctx, &options.RemoveImageOptions); err != nil { - // If the image does not exist (anymore) we are good. - // We already performed a presence check in the image - // look up when `names` are specified. - if errors.Cause(err) != storage.ErrImageUnknown { - rmError = multierror.Append(rmError, err) - continue - } + // Finally, we can assemble the reports slice. + for _, id := range orderedIDs { + report, exists := rmMap[id] + if exists { + reports = append(reports, report) } - removed = append(removed, delete.image.ID()) } - return untagged, removed, rmError + return reports, rmErrors } diff --git a/libimage/save.go b/libimage/save.go index 5b842896a..c03437682 100644 --- a/libimage/save.go +++ b/libimage/save.go @@ -77,7 +77,7 @@ func (r *Runtime) saveSingleImage(ctx context.Context, name, format, path string // Unless the image was referenced by ID, use the resolved name as a // tag. var tag string - if strings.HasPrefix(image.ID(), imageName) { + if !strings.HasPrefix(image.ID(), imageName) { tag = imageName } @@ -108,8 +108,7 @@ func (r *Runtime) saveSingleImage(ctx context.Context, name, format, path string return err } - sys := r.systemContext - c, err := newCopier(&sys, &options.CopyOptions) + c, err := r.newCopier(&options.CopyOptions) if err != nil { return err } @@ -163,7 +162,7 @@ func (r *Runtime) saveDockerArchive(ctx context.Context, names []string, path st localImages[image.ID()] = local } - writer, err := dockerArchiveTransport.NewWriter(&r.systemContext, path) + writer, err := dockerArchiveTransport.NewWriter(r.systemContextCopy(), path) if err != nil { return err } @@ -177,9 +176,8 @@ func (r *Runtime) saveDockerArchive(ctx context.Context, names []string, path st copyOpts := options.CopyOptions copyOpts.dockerArchiveAdditionalTags = local.tags - sys := r.systemContext // prevent copier from modifying the runtime's context - c, err := newCopier(&sys, ©Opts) + c, err := r.newCopier(©Opts) if err != nil { return err } diff --git a/libimage/search.go b/libimage/search.go index d58d50ba3..b36b6d2a3 100644 --- a/libimage/search.go +++ b/libimage/search.go @@ -3,6 +3,7 @@ package libimage import ( "context" "fmt" + "strconv" "strings" "sync" @@ -69,13 +70,47 @@ type SearchFilter struct { IsOfficial types.OptionalBool } -func (r *Runtime) Search(ctx context.Context, term string, options SearchOptions) ([]SearchResult, error) { - searchRegistries, err := sysregistriesv2.UnqualifiedSearchRegistries(&r.systemContext) - if err != nil { - return nil, err +// ParseSearchFilter turns the filter into a SearchFilter that can be used for +// searching images. +func ParseSearchFilter(filter []string) (*SearchFilter, error) { + sFilter := new(SearchFilter) + for _, f := range filter { + arr := strings.SplitN(f, "=", 2) + switch arr[0] { + case "stars": + if len(arr) < 2 { + return nil, errors.Errorf("invalid `stars` filter %q, should be stars=", filter) + } + stars, err := strconv.Atoi(arr[1]) + if err != nil { + return nil, errors.Wrapf(err, "incorrect value type for stars filter") + } + sFilter.Stars = stars + case "is-automated": + if len(arr) == 2 && arr[1] == "false" { + sFilter.IsAutomated = types.OptionalBoolFalse + } else { + sFilter.IsAutomated = types.OptionalBoolTrue + } + case "is-official": + if len(arr) == 2 && arr[1] == "false" { + sFilter.IsOfficial = types.OptionalBoolFalse + } else { + sFilter.IsOfficial = types.OptionalBoolTrue + } + default: + return nil, errors.Errorf("invalid filter type %q", f) + } + } + return sFilter, nil +} + +func (r *Runtime) Search(ctx context.Context, term string, options *SearchOptions) ([]SearchResult, error) { + if options == nil { + options = &SearchOptions{} } - logrus.Debugf("Searching images matching term %s at the following registries %s", term, searchRegistries) + var searchRegistries []string // Try to extract a registry from the specified search term. We // consider everything before the first slash to be the registry. Note @@ -85,8 +120,16 @@ func (r *Runtime) Search(ctx context.Context, term string, options SearchOptions if spl := strings.SplitN(term, "/", 2); len(spl) > 1 { searchRegistries = append(searchRegistries, spl[0]) term = spl[1] + } else { + regs, err := sysregistriesv2.UnqualifiedSearchRegistries(r.systemContextCopy()) + if err != nil { + return nil, err + } + searchRegistries = regs } + logrus.Debugf("Searching images matching term %s at the following registries %s", term, searchRegistries) + // searchOutputData is used as a return value for searching in parallel. type searchOutputData struct { data []SearchResult @@ -130,27 +173,27 @@ func (r *Runtime) Search(ctx context.Context, term string, options SearchOptions return results, multiErr } -func (r *Runtime) searchImageInRegistry(ctx context.Context, term, registry string, options SearchOptions) ([]SearchResult, error) { +func (r *Runtime) searchImageInRegistry(ctx context.Context, term, registry string, options *SearchOptions) ([]SearchResult, error) { // Max number of queries by default is 25 limit := searchMaxQueries if options.Limit > 0 { limit = options.Limit } - sys := r.systemContext + sys := r.systemContextCopy() if options.InsecureSkipTLSVerify != types.OptionalBoolUndefined { sys.DockerInsecureSkipTLSVerify = options.InsecureSkipTLSVerify } if options.ListTags { - results, err := searchRepositoryTags(ctx, &sys, registry, term, options) + results, err := searchRepositoryTags(ctx, sys, registry, term, options) if err != nil { return []SearchResult{}, err } return results, nil } - results, err := dockerTransport.SearchRegistry(ctx, &sys, registry, term, limit) + results, err := dockerTransport.SearchRegistry(ctx, sys, registry, term, limit) if err != nil { return []SearchResult{}, err } @@ -209,7 +252,7 @@ func (r *Runtime) searchImageInRegistry(ctx context.Context, term, registry stri return paramsArr, nil } -func searchRepositoryTags(ctx context.Context, sys *types.SystemContext, registry, term string, options SearchOptions) ([]SearchResult, error) { +func searchRepositoryTags(ctx context.Context, sys *types.SystemContext, registry, term string, options *SearchOptions) ([]SearchResult, error) { dockerPrefix := "docker://" imageRef, err := alltransports.ParseImageName(fmt.Sprintf("%s/%s", registry, term)) if err == nil && imageRef.Transport().Name() != dockerTransport.Transport.Name() { diff --git a/libimage/types/types.go b/libimage/types/types.go deleted file mode 100644 index 9924bc813..000000000 --- a/libimage/types/types.go +++ /dev/null @@ -1,58 +0,0 @@ -package types - -import ( - "time" - - "github.com/containers/image/v5/manifest" - "github.com/opencontainers/go-digest" - ociv1 "github.com/opencontainers/image-spec/specs-go/v1" -) - -// ImageData contains the inspected data of an image. -type ImageData struct { - ID string `json:"Id"` - Digest digest.Digest `json:"Digest"` - RepoTags []string `json:"RepoTags"` - RepoDigests []string `json:"RepoDigests"` - Parent string `json:"Parent"` - Comment string `json:"Comment"` - Created *time.Time `json:"Created"` - Config *ociv1.ImageConfig `json:"Config"` - Version string `json:"Version"` - Author string `json:"Author"` - Architecture string `json:"Architecture"` - Os string `json:"Os"` - Size int64 `json:"Size"` - VirtualSize int64 `json:"VirtualSize"` - GraphDriver *DriverData `json:"GraphDriver"` - RootFS *RootFS `json:"RootFS"` - Labels map[string]string `json:"Labels"` - Annotations map[string]string `json:"Annotations"` - ManifestType string `json:"ManifestType"` - User string `json:"User"` - History []ociv1.History `json:"History"` - NamesHistory []string `json:"NamesHistory"` - HealthCheck *manifest.Schema2HealthConfig `json:"Healthcheck,omitempty"` -} - -// DriverData includes data on the storage driver of the image. -type DriverData struct { - Name string `json:"Name"` - Data map[string]string `json:"Data"` -} - -// RootFS includes data on the root filesystem of the image. -type RootFS struct { - Type string `json:"Type"` - Layers []digest.Digest `json:"Layers"` -} - -// ImageHistory contains the history information of an image. -type ImageHistory struct { - ID string `json:"id"` - Created *time.Time `json:"created"` - CreatedBy string `json:"createdBy"` - Size int64 `json:"size"` - Comment string `json:"comment"` - Tags []string `json:"tags"` -} diff --git a/pkg/config/config.go b/pkg/config/config.go index 1531422cd..371dd3667 100644 --- a/pkg/config/config.go +++ b/pkg/config/config.go @@ -47,18 +47,6 @@ const ( BoltDBStateStore RuntimeStateStore = iota ) -// PullPolicy whether to pull new image -type PullPolicy int - -const ( - // PullImageAlways always try to pull new image when create or run - PullImageAlways PullPolicy = iota - // PullImageMissing pulls image if it is not locally - PullImageMissing - // PullImageNever will never pull new image - PullImageNever -) - // Config contains configuration options for container tools type Config struct { // Containers specify settings that configure how containers will run ont the system @@ -700,23 +688,6 @@ func (c *NetworkConfig) Validate() error { return errors.Errorf("invalid cni_plugin_dirs: %s", strings.Join(c.CNIPluginDirs, ",")) } -// ValidatePullPolicy check if the pullPolicy from CLI is valid and returns the valid enum type -// if the value from CLI or containers.conf is invalid returns the error -func ValidatePullPolicy(pullPolicy string) (PullPolicy, error) { - switch strings.ToLower(pullPolicy) { - case "always": - return PullImageAlways, nil - case "missing", "ifnotpresent": - return PullImageMissing, nil - case "never": - return PullImageNever, nil - case "": - return PullImageMissing, nil - default: - return PullImageMissing, errors.Errorf("invalid pull policy %q", pullPolicy) - } -} - // FindConmon iterates over (*Config).ConmonPath and returns the path // to first (version) matching conmon binary. If non is found, we try // to do a path lookup of "conmon". diff --git a/libimage/types/pull_policy.go b/pkg/config/pull_policy.go similarity index 83% rename from libimage/types/pull_policy.go rename to pkg/config/pull_policy.go index 69e36ed6c..7c32dd660 100644 --- a/libimage/types/pull_policy.go +++ b/pkg/config/pull_policy.go @@ -1,4 +1,4 @@ -package types +package config import ( "fmt" @@ -17,23 +17,23 @@ import ( type PullPolicy int const ( - // This default value forces callers to setup a custom default policy. - // Some tools use different policies (e.g., buildah-bud versus - // podman-build). - PullPolicyUnsupported PullPolicy = iota // Always pull the image. - PullPolicyAlways + PullPolicyAlways PullPolicy = iota // Pull the image only if it could not be found in the local containers // storage. PullPolicyMissing + // Never pull the image but use the one from the local containers + // storage. + PullPolicyNever // Pull if the image on the registry is new than the one in the local // containers storage. An image is considered to be newer when the // digests are different. Comparing the time stamps is prone to // errors. PullPolicyNewer - // Never pull the image but use the one from the local containers - // storage. - PullPolicyNever + + // Ideally this should be the first `ioata` but backwards compatibility + // prevents us from changing the values. + PullPolicyUnsupported = -1 ) // String converts a PullPolicy into a string. @@ -71,14 +71,14 @@ func (p PullPolicy) Validate() error { // // Supported string values are: // * "always" <-> PullPolicyAlways -// * "missing" <-> PullPolicyMissing +// * "missing" <-> PullPolicyMissing (also "ifnotpresent" and "") // * "newer" <-> PullPolicyNewer (also "ifnewer") // * "never" <-> PullPolicyNever func ParsePullPolicy(s string) (PullPolicy, error) { switch s { case "always": return PullPolicyAlways, nil - case "missing": + case "missing", "ifnotpresent", "": return PullPolicyMissing, nil case "newer", "ifnewer": return PullPolicyNewer, nil @@ -88,3 +88,8 @@ func ParsePullPolicy(s string) (PullPolicy, error) { return PullPolicyUnsupported, errors.Errorf("unsupported pull policy %q", s) } } + +// Deprecated: please use `ParsePullPolicy` instead. +func ValidatePullPolicy(s string) (PullPolicy, error) { + return ParsePullPolicy(s) +} diff --git a/pkg/filters/filters.go b/pkg/filters/filters.go index 22222d33d..53f420db2 100644 --- a/pkg/filters/filters.go +++ b/pkg/filters/filters.go @@ -69,9 +69,11 @@ func FiltersFromRequest(r *http.Request) ([]string, error) { } for filterKey, filterSlice := range filters { + f := filterKey for _, filterValue := range filterSlice { - libpodFilters = append(libpodFilters, fmt.Sprintf("%s=%s", filterKey, filterValue)) + f += "=" + filterValue } + libpodFilters = append(libpodFilters, f) } return libpodFilters, nil diff --git a/vendor/github.com/disiqueira/gotree/v3/.gitignore b/vendor/github.com/disiqueira/gotree/v3/.gitignore new file mode 100644 index 000000000..3236c30ab --- /dev/null +++ b/vendor/github.com/disiqueira/gotree/v3/.gitignore @@ -0,0 +1,137 @@ +# Compiled Object files, Static and Dynamic libs (Shared Objects) +*.o +*.a +*.so + +# Folders +_obj +_test + +# Architecture specific extensions/prefixes +*.[568vq] +[568vq].out + +*.cgo1.go +*.cgo2.c +_cgo_defun.c +_cgo_gotypes.go +_cgo_export.* + +_testmain.go + +*.exe +*.test +*.prof + +.idea/ +GoTree.iml +### Linux template +*~ + +# temporary files which can be created if a process still has a handle open of a deleted file +.fuse_hidden* + +# KDE directory preferences +.directory + +# Linux trash folder which might appear on any partition or disk +.Trash-* +### Windows template +# Windows image file caches +Thumbs.db +ehthumbs.db + +# Folder config file +Desktop.ini + +# Recycle Bin used on file shares +$RECYCLE.BIN/ + +# Windows Installer files +*.cab +*.msi +*.msm +*.msp + +# Windows shortcuts +*.lnk +### JetBrains template +# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and Webstorm +# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839 + +# User-specific stuff: +.idea/workspace.xml +.idea/tasks.xml +.idea/dictionaries +.idea/vcs.xml +.idea/jsLibraryMappings.xml + +# Sensitive or high-churn files: +.idea/dataSources.ids +.idea/dataSources.xml +.idea/dataSources.local.xml +.idea/sqlDataSources.xml +.idea/dynamic.xml +.idea/uiDesigner.xml + +# Gradle: +.idea/gradle.xml +.idea/libraries + +# Mongo Explorer plugin: +.idea/mongoSettings.xml + +## File-based project format: +*.iws + +## Plugin-specific files: + +# IntelliJ +/out/ + +# mpeltonen/sbt-idea plugin +.idea_modules/ + +# JIRA plugin +atlassian-ide-plugin.xml + +# Crashlytics plugin (for Android Studio and IntelliJ) +com_crashlytics_export_strings.xml +crashlytics.properties +crashlytics-build.properties +fabric.properties +### Go template +# Compiled Object files, Static and Dynamic libs (Shared Objects) + +# Folders + +# Architecture specific extensions/prefixes + + + +### OSX template +*.DS_Store +.AppleDouble +.LSOverride + +# Icon must end with two \r +Icon + +# Thumbnails +._* + +# Files that might appear in the root of a volume +.DocumentRevisions-V100 +.fseventsd +.Spotlight-V100 +.TemporaryItems +.Trashes +.VolumeIcon.icns +.com.apple.timemachine.donotpresent + +# Directories potentially created on remote AFP share +.AppleDB +.AppleDesktop +Network Trash Folder +Temporary Items +.apdisk diff --git a/vendor/github.com/disiqueira/gotree/v3/.travis.yml b/vendor/github.com/disiqueira/gotree/v3/.travis.yml new file mode 100644 index 000000000..29261dfff --- /dev/null +++ b/vendor/github.com/disiqueira/gotree/v3/.travis.yml @@ -0,0 +1,11 @@ +language: go +go_import_path: github.com/disiqueira/gotree +git: + depth: 1 +env: + - GO111MODULE=on + - GO111MODULE=off +go: [ 1.11.x, 1.12.x, 1.13.x ] +os: [ linux, osx ] +script: + - go test -race -v ./... diff --git a/vendor/github.com/disiqueira/gotree/v3/LICENSE b/vendor/github.com/disiqueira/gotree/v3/LICENSE new file mode 100644 index 000000000..e790b5a52 --- /dev/null +++ b/vendor/github.com/disiqueira/gotree/v3/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2017 Diego Siqueira + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/vendor/github.com/disiqueira/gotree/v3/README.md b/vendor/github.com/disiqueira/gotree/v3/README.md new file mode 100644 index 000000000..d09d4a98c --- /dev/null +++ b/vendor/github.com/disiqueira/gotree/v3/README.md @@ -0,0 +1,104 @@ +# ![GoTree](https://rawgit.com/DiSiqueira/GoTree/master/gotree-logo.png) + +# GoTree ![Language Badge](https://img.shields.io/badge/Language-Go-blue.svg) ![Go Report](https://goreportcard.com/badge/github.com/DiSiqueira/GoTree) ![License Badge](https://img.shields.io/badge/License-MIT-blue.svg) ![Status Badge](https://img.shields.io/badge/Status-Beta-brightgreen.svg) [![GoDoc](https://godoc.org/github.com/DiSiqueira/GoTree?status.svg)](https://godoc.org/github.com/DiSiqueira/GoTree) [![Build Status](https://travis-ci.org/DiSiqueira/GoTree.svg?branch=master)](https://travis-ci.org/DiSiqueira/GoTree) + +Simple Go module to print tree structures in terminal. Heavily inpired by [The Tree Command for Linux][treecommand] + +The GoTree's goal is to be a simple tool providing a stupidly easy-to-use and fast way to print recursive structures. + +[treecommand]: http://mama.indstate.edu/users/ice/tree/ + +## Project Status + +GoTree is on beta. Pull Requests [are welcome](https://github.com/DiSiqueira/GoTree#social-coding) + +![](http://image.prntscr.com/image/2a0dbf0777454446b8083fb6a0dc51fe.png) + +## Features + +- Very simple and fast code +- Intuitive names +- Easy to extend +- Uses only native libs +- STUPIDLY [EASY TO USE](https://github.com/DiSiqueira/GoTree#usage) + +## Installation + +### Go Get + +```bash +$ go get github.com/disiqueira/gotree +``` + +## Usage + +### Simple create, populate and print example + +![](http://image.prntscr.com/image/dd2fe3737e6543f7b21941a6953598c2.png) + +```golang +package main + +import ( + "fmt" + + "github.com/disiqueira/gotree" +) + +func main() { + artist := gotree.New("Pantera") + album := artist.Add("Far Beyond Driven") + album.Add("5 minutes Alone") + + fmt.Println(artist.Print()) +} +``` + +## Contributing + +### Bug Reports & Feature Requests + +Please use the [issue tracker](https://github.com/DiSiqueira/GoTree/issues) to report any bugs or file feature requests. + +### Developing + +PRs are welcome. To begin developing, do this: + +```bash +$ git clone --recursive git@github.com:DiSiqueira/GoTree.git +$ cd GoTree/ +``` + +## Social Coding + +1. Create an issue to discuss about your idea +2. [Fork it] (https://github.com/DiSiqueira/GoTree/fork) +3. Create your feature branch (`git checkout -b my-new-feature`) +4. Commit your changes (`git commit -am 'Add some feature'`) +5. Push to the branch (`git push origin my-new-feature`) +6. Create a new Pull Request +7. Profit! :white_check_mark: + +## License + +The MIT License (MIT) + +Copyright (c) 2013-2018 Diego Siqueira + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in +all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN +THE SOFTWARE. diff --git a/vendor/github.com/disiqueira/gotree/v3/_config.yml b/vendor/github.com/disiqueira/gotree/v3/_config.yml new file mode 100644 index 000000000..c74188174 --- /dev/null +++ b/vendor/github.com/disiqueira/gotree/v3/_config.yml @@ -0,0 +1 @@ +theme: jekyll-theme-slate \ No newline at end of file diff --git a/vendor/github.com/disiqueira/gotree/v3/go.mod b/vendor/github.com/disiqueira/gotree/v3/go.mod new file mode 100644 index 000000000..7e17c637e --- /dev/null +++ b/vendor/github.com/disiqueira/gotree/v3/go.mod @@ -0,0 +1,3 @@ +module github.com/disiqueira/gotree/v3 + +go 1.13 diff --git a/vendor/github.com/disiqueira/gotree/v3/gotree-logo.png b/vendor/github.com/disiqueira/gotree/v3/gotree-logo.png new file mode 100644 index 0000000000000000000000000000000000000000..1735c6008d6f1f4a92944dcf32337b33534ec27e GIT binary patch literal 24183 zcmV)=K!m@EP)4Tx062|}Rb6NtRTMtEb7vzY&QokOg>Hg1+lHrgWS zWcKdPn90sKGrRqvPeo9CG3uKX#J{(IASm?@+di}}l?o-=)F3E6wD^Ni=!>T7nL9I? zX}YoAW$t|Qo$sD|?zw001?ah|SeB6#0T!CBEf+H4bBB+JJu8rehoBb*p;u8ID_yBf z0ya+zcePvJL&AGs+11_tpRKn>9TgyPA7ZoSs0)aX0r00)%XR^J`jH<$>RKN5V(7Oq zK*TS4xZz{h!*f1C3ECFkK$#7nA@pGN!$;%jYvwjAKwmYb0gKL(K8 z-kPtb5${A?tlI~wzMrJ6wTdBr=Y%%%EaEMQ&o}4FQ^DA)s*}Z>!FI&AHCpoWI|RUq zx?7s@$8!5^Q=anY%X@i5{QA6kNcMelpE>R6eCYFpmMsVTrI(b06~u#xf1yS} z_UGdMvD``!0~u->P=lA4?YN`hilQ|3tHka)7T{2CGqw zjZfMwx$5irQN_*|e4l)UHmiYuz74Yp1t^#>hrJ3-SOXDcC_o0^7T9R1gAN8V6s;5) zieI5-7aQlmJn}lUna#nz!j%5V$X|o`xX!dHWQRV27P1=rj;t2bW$~+pTw@bIek?Zv zKPDL<64`^#UNTAck#RBsB6*5DP4<%UA_FqU$I>2EH_cM;u)Q~SI+rg`Rn{L_AC5qq~L$#SMj%U z$6Cz0vP{G5Y*=%5RT^yu;}-DInZ=349rJPVM6C3K^oO)8y(fJr{l>k`ead~!ea?NsT>_Ci%bnxC;Vy6=b6>{xYV#Ue-+LB$ z7`JEXmTRm^AtP)R9u{)KHsMiWGV&)32xCG~*nyU<>-!d;FP=Re4r3qYr~6#KE>;1F z`>_J_P5xC?ROxV(DIHdCO*p$HRQI@7^PwV@Pvuf+5K}u-6REM(K@W$s zrgorh0{i?O)v0c>QtHxU-hBdD(>iYJ4b2sIOVX2K8m~4gmYVA5h^QEb$V`rCQ-|7Z zS{nuL-t>?3n=-o(6I(7vocj#GzCZEo`!3>+v;dYIfPu#&ZWzzX2i^rZ^Mu;6+rb@? zNPG+6)c5T6zxpzGe*M(x+{AON=PiJ>H#?ob-|uwRK0yDg0B4PV0id6JRZw95ZvX&5 z07*naRCodHod=v%wbH-?NLzaEO_U~00a5e`s33NI3ijT6#rEuCL$T}kEIfO~h6*AU zik+fT1OcV@URMN!FaKQTY;JDOEnDv1#hv}#J?EsIoFtRXOlBq}rc9Z#qf(_xs!5Y3 zsu+q}ptuF1TcBk0WARgQ3lz72-2x@IZr!T3Y}sOdJ9g}_uh3+NUov3~t} z!=x9tMsW)iw?J$Ql+bimG2;>2F8LNR`}Y}Y<+3Ge-kd+wiWSS$>Xpl)%3if%g<8F8 zWmH)88Z}joTD79XYSvM;YSmFqnl)E7>(o=b@6j?UzWAYFTA&y~@~K^yFI}vDo$|9r zgjs6gygAA@@bl?aK{0|z4VyI6h_k2KtKHtJcHR0xeDO`*Tc8+0a@mH05YwklQnRK{ zjadb9S!|xOfi!!zZmU|i?V$GE?;uq{?{O7F?pvT3K~iWt$fdDgeX3?oo2nKqn4dz< z!b{qqQDfDjOE{Wa6sZT*Z`fF6Rmf5`YSvb@YS#%0s9C$N zey$fhuUNiBtz5n|c;;rs@?~n(%H{gELakY~Dk?6w4;}XItoH7DKrtDXpaF{!#A+ZA zWBAZ_)R&)qtp3c-wqlaM1rZuGX{wqu-9t6meGfBJ=QraFskcH)#^!5*0LU}*w_kIO zIOVgl)FFo*t&TXhUopSp>E>bt(T$Q9!~h{$Y8tuQG)09_? zcA7ltwD10E-~D@}j1W0ZuhiU?lhmM(&QX7E*{BZhbe=k)%Z+xlx&M9jE9$331Jr&^ z2CBxDkBf;93lM~vIPQBjB?U535T*N}hZplHx^e%p2r}fg!RnI_-VPgIG2wf(Y^4tD z(JRkLAq9gR1K#Z`kD&-MVR3);+pSqRdlD#p4hVRCzZ?vFc@xtV!Z-OX& zPd_i&*X)w%{&5I0^_K~1@H3AjwQo_be3m-2&ylM8p+|aWQ)YLdah0Q0nNrn~9w3la zH@)7|tQ^%U)m3XYELF|xbx=1SJ0zxzk-s0UmTwrZDwJuUwr<~`j%YbDA-NS(h!p^` zz-N8`^=P&E&%}Hd`zq&MdaY{Pv9ndmKj-!zgCGoy*IsxkDXCPo8a33hC!U&cwHQ6; zRJCyJSE^#!25SE%gH)XgJ(3zZ?~Q*@eL8WlL5e??Ofq@GNGw5+s#z`7+Uyx>_nOD4 zZjJt%)O+G3itt@;sc*-Al@x=I)a&%~FZWIq@!nI^RlmmWYSo4%CTY3S71cp4PxEGLRr4|vzGd5b1+tVW z6^%&GkGe{&SU=aB*A1j@qvxEa#{O}!+PEbr50e3tLLx+=4dL5pQ`+{Q0HyBKuq%Ej zyvBfAuTjgELttboIS12>WZ>CRI7BhMF?>YqeX+GU}|Jcd4ee+6VFWO_@?vRaV*BYWb!gRIQ3# zRsAYGRh6=lRfTl__t{nI%PFs_^_!Qg=5;!0MD^SCfDm8)af({Lal9$MQME&3cp2K@ zUp)@fhI2c#sCb%Pji~$8D_5$oMt-82?%d~zirmE>c1a#283`4l+DopwIUyP4_BS)9 ztpFJC2<7$cu2+%WtM^Sx(u5Ri;+zqB$mAq1H^uSoI3x~>vAQPq5O>-H$*A@G5Eflf0<#{Th%#3wXbtaltha> zsKKvXwp1;c8yVb(G~dl2){2W~aGRFFghmk-`YZQi=kkZg-r%~n0y z_EV=HI6$J!RYb+Y=o7e4!~du@wXTRt=X+o!e&LxxaT)nNTeVi#+%zC+;_#JK6vK8I z1VQ~`;9WPyRT1gg`v}$d)bn!Hk(57C>u15(h5^VT;@(l84^!Wc9vK%|lNF)Lw;giW zkumRo?9+49)Op{URRmEn_b*&LGpdRoPU~ctdzgGg#z3zYZzU~t13CZi@%!VhChVKs zaL0pDlSfn~is-{G!K5?G#X}BwjZ3e+MKhsStI}E9Qtm1S=%b86d;jjBqyJyofa<$c+LFlNukAG}@lJMFxL zdjJgP0sn~z1{g>eUfExDIdX5cs_abFrQ^Y>M~`Dvhfe!y#Q9ZiSidfK$FGxrR%J?; zRxPxllMTH4fjh1@X>Y#!VO61Gv;>Q^wJJAJRVq|hGZ#-()hpLgZ5r>db-E6;i&0BA zjZsai9jj)qdf$Y1sNY}Jt=K&(O)?MgJzcvUq<)?JQ&jyWFMaUc*e_MpYE^S3^pJ8g z=~|>$5Rl`gXP*phPiE}a9US2dD*?8`H6T|lNJpdH%-X1Fqoel zc$XpZK#m*ld{Aw!zDPataYR+)?sxxf-h(aqtpgs7*>`y7_5Udl%!Zerf86Z&qvjG% zBe;9()6`Sv{iJ%g_v>?YsCS(f>&`d3@yYTYc0KHS>Oq^jtD&8SKA~@R1t_vS$V<;X zrbc}>JhTsx>D$b#B>Zl{#cKA-59QHZYggJ=^=S5Tz&}W&ARy#f*8~jpGmi{VKYlyb zWR>(+{BIx4JU3UXH!d^kB-b82IMCPy8Sc6|;PY8G(+d~TXKYlY>J^jdi>Y58r&3FEQ z2?*p3<4c&p=&Pa(MGiq2K%++bR}LX<;wjcOsd`ksO0A=0)hNi z6#+>gC*ALmchgsb^b%H-u0;w#Xy`DW17`f5ho4C(W~^4OqoKv6lMs8de*r99H$sAy ztR6u?oSUB}d>6#ZY%i7>4*T9&*6=+B@9wzM)g2+l)RE zX;?*eD_O-L&6d9-`bF>uzQGr-Oo@}(URDdlu6?^*7qfau`Xi1#S?f0qkw*eQy z=goiq8QgQ@n8rYSxLz6Lk?7C9{Iml%^rylIoP~hwc|^5{#BP5enILn_N$!L zQo?e3zKll@7~rr?KxWPhuIO(%28NZi$pSVN2Q~k%*+9ej zDmqHdt49X%w__uG0A=OH|J8*DKM}QZgbj{PU8DCceBarf;{%X?EyV*H!2-) z%)d`oAHManI_reqMv{YAl>0{p+>(^UhLxm8vp3Z7drdT}3j7J3XCnoj3QrIZ%4g;5 zUdQYFd9$NcK%`8hUNUec%)SnL|1I_QtAm3ez8&O%LH2=66u+;2O@aLISt8oT5Dc(a zDJcU>$5enlBvj{7eq5D3CCHixNzf@PDv;3#f{4%_tahVjY26<-*L;4f2t$P<6=oD3 z2gE{T>cY?=uf;_;95@I})t*Jd;AibXAq-ly+AAzdeXLbcY*OpwGtY~Q93mpd8wvoQ zg&r1Q``|JG*1v%tovg;s{@4l$zeNFZ^5Q3h1oAEk5G0Xj`-&hUmA=_Wrjr{Begc*f2i3iO1AO;BKLpJh)q{O3R5!2K!k`Uf%4HzZu!u6BdFefM+kH<2zlWZFNx`@Z#1q%@ zo_lY*CK$MoIc(ms(MUO5A!TGQgqaV1$VC!(0LVRnG|xPGpS<2&$7vER?0Y0(qRmLs z@m)vi_l%kTx|!+UA3spCn=7Oh@2^lHLh@BD*Up4v`vLg~%$xnYx#HYT0E8GlCF0=) zrN35c;es}MuQ_} zg>ef|sgv_&7tO>I}F}wbVq_HjeVdgNek1kWF{)admFDthB zue|{)>EwFW1y`vB^XA659HNiuYf-O}^qF+cxD^E2lv&UP2FQWh@iGIR87YiGm3)_8 zduu@U;G)xoA#V&0_#0??7dhPYAQ&P{fR1Hf!3Ewa>Lb*X?+`?1bvo_b%iJtc&#iW9 z97hH2d;o!2soweiArmrV(RkIMW^*-q>N|l=y7=^kXJP_i)sU5gOfdV@2}HCfjPGWC zxbrQM3#u{k$Se?}PUSAzLUD%40Doh@C#o?0K)tWI`R<@U?0D)!y)qRt3PB(dzWi*M zx2+h&+1VRR+=-{2>rLYZ)~H$24KmM=NW$MFqUr4dps6A205;u3T4ble<^=MLa%DFg zqB$^Rz*lhjJ5#27ZnqP8kBX> zl{cAi_uqakb9BSzt(@#e2hY3L=x21=g!%8$>=olB^0jYnQ}++;t04U*&W`R!R^xLo zxjHDrzCj4#YfC##VclmGiG|-PnQ`p~krm^fTdp+0Li*Tg+74 zLK>w4_RcptSmej5uV$r&>i90#8+&l-mrS`hQdqiitjgZHK-2SkX&Z_|yj79a13PvB z5T&}-@3MhaVdSv)4VlKbW3C+|~M1Zj! zJ@sw%qo&JAHFu*ACED@4oqxn*7tasEG?T`fdju zVm!jDa{73N_48HxX8+PgTR#UY9Bz%S?DK*t$NmQ&-^=S#ipC#ha$S=TZaYpw$ z%p=J0uO_D(k+zQ)BoX`-N{?6%UU9=6s#ev;YUTRH2038#d$vC*l*wm;fz(3T(gnuA zqtCtJ1{dmcMj!}`IgRcDVol#5c2^7r1FOWCFNUe#rcE(3JO|bEIi_z6-7lKA9ro$$ zzAkun%MT)U-ndBGs76bL3(?J6HkkqY?)QUK=iQHrala$Tfdr_uW^_@p2e20xXAdyy zwj^txE?sxNHFA;JUILHmbg|s#=?X%_<1C-JS>66l42o`D`@-NJ1h>Iu+$j_9B_JfLhi|2_^Cx>B5qn=3wfeGp9oO%4?<9rf z38tUP;)SOkQujR;OHRdmH+_5C^_RwI{vxCa(@Kt4UO~kbx)e*)BHRyzF?EqZq%>kw zp?1ZviamTXBl zS4g%!TeUIjC77zBVsgvX=bLAdjKQeebUq9z>g~TTo}>>qTlf4@A&?9db8TgC9ih#I}ULIs8B8>!0%UZ7RfKwF0u(BOFjKhnJdGk6r@jWd@`gK2qq4lm^CxBJrLox)unm!X zn}>ED+4RfSGa3jfh(Q{}&WKBGz}+Y6?U!_ZUe(;vO8q)%qWMXhj&*M}Y8fB}JO@&% z-g|r;@E)Yg-WWY7WbjED_SKbW*>R9^09UkaSy9Ah!%oLNUR&hbAgMr(MEf;%867)C zcqI4-$mLW3u*(0K0H9Z^Vc%%&f9<8%I)p_$W-M#l-#z$N|o zmLYEr4z&;GY>fkZ_ST+7&W!7~cly+yqpE~ND>q=f7)4Tss{86qzp37Peh`I_sLzN3 zB{Mm@-R`#~dVhrq6@ny*gd0IZ>xNb6^ASU%BzHt-3NKP<2$JmTv10vPHDT@u^{3W3 z0%}xjpt^TF&DfvAOtYgMCXO^2LjG|&Y%;(s7f&`1U*j|AVm9X3=j=_{W)LzEKh!E9 z=U?7Gr5QSzI=X4fl*-Yt%_>u0uUs>X#BB1i`!q=x;h{ikdqrjC@F7Q{y@_Po(XFHX z48`D@Q_CviBem_k1$|#A2$JmTfy%=0i8;~{PGM?NtF6&Js$I9Kb|-qPI`P!SioK0g z&flX^7tWuXQrkkJY}z_6O6Q30aFGM`5?2bt%qqw%V$o#C7$5mbrP)tel|4E1=r_(0 z5g)u9yaP$cNm!_;bKVFi<}yKvXdPy|cH^>Y^|`ve;~*vKlO|2X(^!rUCUKg@b8 z(@0!ies82vQAhVl^q}O5>bk9Voh;df?E{C-yfeAAnz!~7sj9i6uqcBV66KBupESY+ zm$9afATxC#{TnYn6ID*wgIh-=Q!w7cruQ*Sxa^&{`5RTqkq)7bj{h(AUVu$o=bEdK zx1w${dD#QTCVzFdRvy&l3v-XwFGy`m08IhyL>X(TUVW1A0|y=tgxTlWg>ZZ#rU zKovz0TQrPe=$CK!(AaqJj-M9ZqZ+q4&g_A_`SP;{Ne(&eXsxo+L6eOe)ktj^CZsKW z7p4FybxL&Y(d$U{k=|Qz4;E#t#N$a5m7CZ2w67qL8=?zkSDu}1nSq8*cp`{G_2GvZ z9|y=mMA&(LL8g!!*+-G+i9dcLw^3I}nBg;e#~??UQdNR7aowOqL@!6-l zn}VcAKtO$9v>EiAbPI4$j{s?0IH1Yn#^R6g$STsvB*Clu=fj7Zd%NDPYM*Dq*xfHt zA|e5ky%Z#e*tVlyf@LM18U;C6Il}lCQ(%(+=zE`b17jss=~$AlOe$I05kz#M+}fcQ z0>G5ZN`b=LtR)kT&XJwIbJHBPY&$&BJ({RCOxBIh3~d7|$LF*9nL))&3V9+>BP(K| z$Mf$qrb)0lCOUXXa3fNm?2VgZQu>%=Ry{~N_z1P5L{KmLymWVEq+qmck83k7aBs9a zd#X7!XW^XXfmjz_NdN~HczHqUC1PT6qzCfjS_p&!A^2hU0!fK-WU`ZkyesS~W5A6M z8#8XYf_P3liRIf+LgFKhZ1WfJ94O@3fU;h)N0!<7xR-ZGxX`%v5)LiSj0)-px90qN{%7_9V@h)<(qFiurq=W7JsodN4>g026)7(0alX+@B+Uw-QSMq$p0 zT(m0IJeF{fx5*KWf&+1vNwTH!%0|p)B=(XpNs3Viz*&nBA=vZN=UAknY z;2d(KWSF{w80d82%mF$+9=Xs(pWx(z?dj2tljk{ql=*Z3{%wiAz=21HF{N z6(mAU1~nd}U2GJ_YftuL@Q;6o!p-}h$Xg7O>u;b)@ z`qW?K)?5*EuxEseAOW$3u`Mi#0|^9)pBQ+jn)}Br)n)&PV+IEWWA^_Mn@V+rY8^lA@u=nIOJZ((6xTJ9_Y|og?A{X@@q_tEv1L>Ug zO+Fe3Ju88xySMOm9?V`m!yD*tdk=&15L@EO5&bK6nxVtv0R}{rfi#QsC?BFh8%TiG zZcyFY{nto&;*~=jk~>@Q03GhCUa%9-W}^?VOYzOH*o)vIs7-?dR91-^#t_REDu5Ut z4tdplJ9`=NJU05?OIXD+Imj#$Frj{ZS0<|hdHZtW|HdN)At8Wb1@da_876f7D^NDw*V|^dSAmxxt5QBk?E*1J#kQtCc z_?aRMq#=#Zk(=4y*s-?r|Is$&a7vIvXmAjK z<=>SY(FlZbcn)d8NDvT;W1O$M^o+ofnlvD*fOtZbFuC&nMCbW2xMUT=BZqILlJevV zSp{;yoI-<#LD(wdQ*>mtq#g`>K_Ey~94VZB-~hva5GO#|Fx@eP6z~$P0B8c^oSI`4 z;z6rg5j7RkkT0urV^c<09zk5QN8X`f#)z#HvAI8-xki@JAIVvhU%z%%# z5|9u6%c#QaEM)n{@73-#j*C*$k@C5M5H%|{G$zV-^nKrJo=m^W(I1&;AD+*!|cmXJ15qykG*drb&@kmeEMasu9i5k zmb~tj4M}*Ze(h$dUM=2=QjB!d$}_v}ZR=fbRt{8RYF9kK>?cV)S0+F9ShBX>vA) zQFciyJAK2((USvluGbekeL>vsRuF}{Wv*Vi(hXAXjy4G(t!z?2JPO8S_5?87_{cCc zQgO;gr$cTvY5JB?t7hQy12Bo*r_26^#&y-LiAPd}qqA?d+sxCoyBbIu8}Jr5;MQx5 z4iWXoBMJ`H1aa-O+_QSvfU}1%;F}-=#g{PYs1GM^*{*hEZ-Y2?-tEmucTn-&tfId5 za_fQqS)B@QsQ(%D6n^ZkS18@Yv^f&UkA}SNZS%;ZXa=%i16>tXgL)@;rZF*9&DF4`umFo%Vl z@*#))KGJc_8zNGhSrjr)NCLhW0fUc+!O7g7_4if_Rz-A+putZ&;2!%mG53&x?g|GY zQV#*zO47Irp)%>nN=F{{KMD%iSlybvTOfh-^*`?hQSmE4o4(``*ZYJxVPzH6w-h*n z*z~^r5$@n;z)44vu4S9HcJjFQs8HCd+pHSM;l*>IeSRJV@oq>iq+m9p#C?esiHOui z6jlouUqKR*gLH_g?KU{9{rRd<%~nR#$;tr@?(X|ouL^GAv+S~opdPkJS`_^lL=w*A z!Ky;uK1W5gg2~I^lr%gi0$U87C@o|wq^8}Uo z5>`;x0w=NLFc9)gA4Ilo_I>HegMYIy@|F1fgi?2E$j<`Zvc4btxti%5In2z_=jaoo zlDg?ywB9SqD|=L8>tXAbEvinv2v5My_1M`Lnb>l>)lysaNk~BLO8ct16%R7^zJGKC zx<^NA@@vnQtyRNDP1Kq-tJH?|>wL*$!k~+P(JJS|K72>5UA02(+pW9$W7Z6{ZQIr$ zEosJl@u`|KYr3jbxr%qN1c^g#5La*8x;dsjf6*%J9Xjo6+F^k{byZx>dw??7W|;f?B;&R2xgEIRnfhwjn! z+-gX^dmnjLb=3S_1?~932ZYo1)Rpfb-?;C_=r+{qMmlc9$L~e8BW2X7t6e;WY!Ue0 zk(aQNdC8niUY}c-Wc)q3jAhun?AdSpf;&wz+ccGc>RHK?Ys|8EU0ky|B3OW~hZI8# z*Cq+fi#-3}Q=qHE_!;6wie{5uo}#=#h<<%)GR(Y4+H5-yRtS+GqE!IMDk6pB7un{g zB%GDwsn0G6HuDG%Q=^wWT!o~gt=OZo`^Qd)dc4@sz%R1e(Z&EPi71cyBw0AnP8Sq7 zf^bHK3*z+I4tEc^ktNzr6YrkgUg0@VS!csapt@!nlK}<_`yNQF=p>{^bTWgAfywUu z*%w`r6rv=0A3Q)Zh~l%eKpN*F3H7DkY@`V@4lg3)R1>Y(DR6whGv(G?WiZ--uYY@+ zK@L~qW~cGpXK#$G+GrX0kc00*#|tkppU*r=ODTe-|jb##w$0G^S``(}5VFivL zz6$$1tGi1k9i9pkxpn(m^D0&k8#z#(3sd=!x`7x-gkZes7&?hn1L=|5$;@;h3Thh6 zcp^EHvQoVYqSz8CyNuCu&NLoPTy+m0l8_IcWb>b~GptRsgYbexM$S6k=y>sr<8GWqAW#u^M79nVQcd4vDrKePS||wO`!M(b zSEBUxtAoAEzc0VBxZ6!TIIDi5LLW$=Thl>u8*_ym5X6Al1{m}p0D~wVFyNT+&=2EG zLRJ=59KOMoHpMGP;&?yjwflDMu`|KTig+23JYh(}fVaVISK>H+z^*5xmPm)hHUmXR zaiGCU5^qya-iMPpz{g;t_$aG9$N}Ixmfp6r(gaVghrSfqs{g533 ztRlDfztkMiV$XmQ?_OKfI!miJG%^j26l=dG4+V}EBrOAQ+z(^b=fge-BrU{2vOvGn zv5)~_=B-kWeI<`OAb5|h;(;`8<+7#0QPFmOxraXqI~EyBMWl0p?a>4zQEGtEMKh6} ze{8t9Kz&g;!3ns)lLPNk0|q^-n$(P#C70iMBpdPly7^V#;Y&y_qH6ky6`1e0S6|w6 z=89L$t3=`MvSu1H#@EFYvMFMU`eTfKzjnR$WdUzxIH4blkmAc1Ppx~-AeELk+qNSZv^ z6w)pvRYIaH+cZ|K$^K21)k=d|+AZk(wZqk(HBVQiN|q0@BL!Nwc9pRN3<9m2LytJl zC~R6m=9c5A~vzzA^MqRHy z{pESB9Z*v>s20)cEvvtM%a*^QCXG33hp2gLJ~4SJmunt%MhcTz*M^UnIQojC-4a2c zqfbbbBwykSd4-9A%0d>$yl`EpG(>qG9y zzCpt(eT*qDmWq%iYqN1AH%V<;QZH&R0DcKcmF$by6}2^Er>%HOEnGL!9OQ!v4ti+z zp_;5>kez2;+Fu_Av_?I6dhAhhcG-)yTPBn*pB;Ifn$dQgIq4TZ&6UtC-}Ify1~SmL zHngpKW$+lbZx$rbH@-1t$LHsLfg^~ovhFhKrU6OV{CoX>PX|&MNdsNVk6r3i0mc-i zzC@d2;z1DxAEP1du`Te5p2|DfeV)Gfd*hkjc%|3p016~R63$zM$HDZsb|uY==|gd# z)gezXQkpPXTQ6#I#s&r^(y9mVyfF|n^UXr);A5Dh-%u)?z3M%awN0%n%yAMnX$wEp zdargy^lF#yyBC{s;xPW2wPb?2;h48#csPz<0OZh8^PvB=s&j^-4g!hJZIN{8x)ut8 zB=d1}KI$Maj<|f0gCmp4D`{X(8#US87~e39%0Q9$uxq_a|1wdVwk$Gg+;~NoV*|qS z*x|Ub;jD1ZG8e#r$pzH#7tEU*<0y@E_|F1C01n!L!A0%7OT)X=+-WN{^BNJSngrHA z+6~)sSxH38mrYcV1E0PN*M4D`WzjG|Ac2rU@=%=IgLKINXRYI&6t6VwykFHPhqocW zFy~ShlTW8cx5e20;{+p#A?tg^7RLL{ca_k9&_q zjF}p827@8U0yeXJ-c9Vhh(|G_Wso2uWzfhBiz5mbKv;CFP;7L;B_q(16?lHgRYq+^ zp53eM3{QBO1?z?d$z-s~N=XEWfZqTN&jeCI6222dD5PFf=6<8b&;Ho>7oaSAFNb!; zz)Cc%Y`7)IP5h+TEBIR-4%Z$%TCmVWp zJgcWmMnT&zm^UvbT=KDZB*?`cjqq5jHKPxrV@eBfzen}cC7ZeW$UcySv*(XI{^Y2% z^1!z)UNzf1Zq`U5&)?2?-$Iae#_J2}KJoUXl0E7`HWQ@|1$ixS1VOFDMUb!$5?gI% zBYa|gFyRriVO70i{m_Hy!^&cC&71SP5v{_!*3^!x-N6=?mv9gQ=L?V~*hC;*L7KKn zlQtFf3Aw1FpqyxvZ5Tr_Kx$N~Z)Q=-Nk*N$_mThB2$G|^PCg)e=dN_*De@oFUY{3q zNn@`bqTMTTHvNwX+$YNRAqlE8^XAT0XPtjV)b~)pn9mRg(O9y%sovq*B2bN8J=OEGJ3fI|&#)|>akL!CzN-Ge^fvo}(G#B1}<$1-K8GTPb&qP5|b!%~4P;Zc< zZ@23b*4vgc+3#>qY+|>Y)gjCaI&O|+^HYxZs%n(u*p6NGrs@kWIdoka*kb5EXMff0xBU^nD7dhnpAPJ1S4GdtU zU5J7CUVzT~TXoW4 z92Pc#A-Yc&6gYzTGtSJDCcBE z6>2?zKmwHQ&d3tS4mJqSB!KC!kNlCeVOQ~bXvYix00N5gp{u4&ddM9HsttHHIqQVp zc96N}n1qIno2Z6OnyFf~>V`T?a5I8zeCI)j93HxPi}y>QdX1W@My*<^W$TDO7s8AR zSrv@o7V$HtPY%9^XCFY?$wc5AE6G}$`pQlKwVfP^V56%FW6#&{?W^Kqj z7x{^Ik%RY0ySVQp9Ap!OLpsIme?a#`lR_4dpUqo$MYi8VyYiEF*l~&O5#OG2)f)X?Dd7ciE$|h@o%Q4+Tp54k8f_#h z>dNYIeCKO}Fk4zf!(Maq-En94Amh%P@>w}6l^d7aXI23a&n2%Csm;TCxFSrXPn>Up zEsOxC%-Snhs>tC^3Jd4YkJ(!QNnw_Mp76c;&pqL$vQj6mOgQjm{AdHL8QG1;iqz#h z?R?xbu{CPa)Og9ak-|=ga|R!{?d(gFUttA~AXJnY#@#GLUBpHne969C`i>}hI(d7x zKPk#k$M)xhJwq5HM_wQNnH`bWr!wpLq;7WEcx18p12z{YEqO$H7u1dtEEN$u53?iE zG#eN+o%H6T&09f_+XKSKbJ%6@U0CH|coa|n3^J9SYtA7TzKn8MQE=H44@u~g^7f8MYc!zBhLpXaoHnK}v-_@tMm`cC-%1kvZEePVXeRO7{&04AaFtlsD_A)UlCdr)|s8=D{TC4-Cv*;z3rFS|c5DFs1%Ks{_E zAPuV_JKuIVyod}q@*vHU@3+e(FY5a+(Ka&KY214TDP{9x_D7s5vp#JUX1FD8l_(E& z6?Wv;{`spu>fp&Jtz6=Hhdl`r2C(NfWAQk(PqW@$(S0P9K6T2YxR}mebm~#yJP0rG ziRFA}J;=EhzlOc}*X+aQiIb+uq=>#*E4LESd_8H`GPqbxV*`$DoL22wm+npH{gEr~i@BUk% zSt|fJ^UNay%vpgrV~~L-@xH68|ARBMcmQ&sC5tLYJYYjnRxwG(3IH!6$ibk;-!A~; zjJB^&nnSu|q<4HjPNU-?IqzaM>37<>+!}c6<>%zqT;&XTR*s5g8mgZb>eFp^dXKR+ zgQGGz4=Xk-FzQ6JmP|DL{Lc3eIVmEK)YbRRuJ;>n=6noH3yCEv*KWO`-CZ8yoSyeZ z**Hrco0f&V5;_(&lusr;qYm$Mp89lRgs~;;i4o_U^D)TJ%$f{R$h)BP-OjPC{3M-R zAv-`022N?M(jnju8um2UP;|jYMj7;c?;9^^?O?rn`G6G$B$9UUr2#RpGq=Bm-EUjz z5IYmJdPU0?vX#R=A-BlKK;<2fk77e!rc~t^lUY|Y*jFFI#50O$J+mRqs^XHya%7xa zPS{v-)8*H5T2>H{z{gzqHpo~-e5&EL4Tw#Lvf9wtt2Zt-GmE&5S5uT6kbeReMamJ_m#JdtvM+Y7M=%7FEq|FLcRIni2y_w zvWnG)4Lz9fXJ7j;V8m^Hj`HB0V7MWr?3GDqez7T+v}gu%5|!+DOWhBey{+ceMe*WHV3FbLDhePKJV%D!ICA5LK&oJ#$~Zg5C(Nae~5LKN;)F zpny0xOm6&KkkfkIa1qS7;RX4eHUxoogp6ROw1JaZlj(fhw=DU;{A`%nR1?vrG&X$4 zPjfy~GqraOOhH$ypC5`K%UTESBip+I`j!F|jlq#%d^Kr$h& zouid51;l8Jjlgul^l6i0Ud7l+wH;y#$wffg5QMsMD2|IDLS`jmzCZ+KblH(-7H3oK zoQtoH(%*r}x6k_v+Y^C_jVC3dzFo8net?bvN&Zi-1}yQc>b_i^PzE!?F+6!UcWobNNh#v9C;4`5W5kCVF;G|WZMrcyMdZrhu#pxdL2?9Q zz3|MS08#CBhLmD83R{uvwC+9cV<#OlBCj)L`K*Y(*@_iQ-Pf3B`@679$I1}W3aQ)Y zLq7{h!hO2rkXqh?3Lr3yykhJW2BZ~1+*Lq6I9W0a`g!EAp)sqFA$2slgX(q*y-GD{ zwx_CHyRHR@m}DFlzK@0VhH@kOXEMX&Wc#0@O`-yu8#U)x3-a7nnSRly>6^ z`Z=u#!pA_@yY|P)l3Cz3X6KprPoAArg&u|#Y1_L zJOFkX=?2=-*MA3UdzHe1w267;1BI@aXSre*kqj`CxQAsXlg~|qga@?~$R^)Fbx+~t z#9VZaIR6`QsC^Uxbz*g6#$D|>Fi76bOvw94>wBl9%k7mDn*iz~;%WFJj#UT~rxT9m zfE*e`erz*PK}TH&U8zsCzb{*LhcpENc+m7g*qHI9&F`?ZuOJKzW>I&R#Y_~bQX+DI z1`s59{>g`8$gM;T#K1YQN3VqY2_n6rJTz6=A;)d!>{Ss{+^I_+j}qsS4yl!I1|$y1 zsu0^^pUwC`q5Slu6X?RbRQ)-d6;r6I{O?}QstvbfP*UV{CW9p0LqL5ac zGF78e12t*xC^dY-bLzTdUJR88=Jjjnd)Km(G8rocF_|P=-?SqL6@$cZ5rn0Ev_5n` z8Idn#fFvAV16lONh+zS9O`Hqx7--8P+VnBC&dz2#udu_@bzdk5f}}={E4DxMixD3sjUbYb*^a#e zjDOq>FjS+lUk9Oy%_s!>PLC}!qm z5@93MHH?zFu+!eD>w(@j5>*gfFtOu}rh-qgP|C{7m3=K2nAi$qbI`6wp&$seGn-%{ zF_H=yF{$oMX!KNo^2n1)K4@#Sfa!DqSEt=Oo@OLkBI*t5P(r&c*RWB0dlx{RM1a-9 z*TE8B5Lb90LD+^>4+&16;$7^D+hDg1b;YEY6*v)Ku!aE}8n-BguFFY+^F-w& ziU)yxtB33SE>SrJ5ubJ@T|Rv%%#mtB4z(=ozH= za9US=deUbm>FC*K8^t>}AG%pMSH>Ac*OrIt17Cho!zaF=W-a=~B!%1nm}vz#<;ezC z4^h%fW%cm+c(Th)_MV9#Vt-OuafXQv*oWg!I6nZ51o#iY_Pv`QWlI^hVx@#}&4ks1 zdK4Oh@ba)#1l4eRfRZ*Iz^sm$eW9Vpcdy>pI80ywO~l8j-2QU zz^}lt+6G#KBMtDaNtJ`Mk4oG0YRJERPJ0^qS- zKn$MbIWJfG*k=ZiDXRzHR;UQVDuUjT3k*2OmSh2`jSLnHdc@nt>{Uafia?Q3cn)De zNdTF3e3$+~TBJGlAwSr4pwtI)^xAVslr}48d8YzwhUFDjXHN}^fW$JY=X?DS<1`^i zl*gOH+h3}#J$kU=L-LdV2qR51W%Z!h3WZ6x`wM5#-+w(iMobIS%L)S$#RJF<(MC1L zi>=G?3J=oQCqvl=R7hp^tiQK9uw~yUT`|8=&hu)Vm=HIl^{+?R@S%JMPKy z0B&A+u2`nAIgqACR{N;>NPIzDA-C@Co71*=eFqzqu8@W@GimjpK821Th$4LwEmjfn zm`+8*fD>N@Dwb^!Ad7^d8X|y^&V>idQ~ayDU*euC_F)y;mV& z`FRy@AGs;hXHuG*a*A3W_B?!~6@5L*}hAvdZ(p7W|Uf^9|^$ygv2E~G-~EBed_^l_q%U|+s;hGisz&;Up@F#YraHtp_C zHye5B+(7fKaK5_63$qUjjhC*d#<7Rx`X7*Qp>H!$Y9;lPYsMl76@(wbj@t*w((^;F z$yatVVI1t4SMZsb=C#aB$xq+7Cum9C;5+Esq5v6Oy@i7 zcFlMMp|WVdhB57hg(SXHv+RaV_CB`;V#$`D&UEpRgL=@1dDUc2=lAfgB7q=O6o&5X zi>}}{20)S*o_Z*)MKe)2(RH6ewMKk2N{_@WX~WD$f0#(!=+B;g?X@!yY8&9oG(TJ@R*(kCri zw^if58>2R?Uzg|dw{6|3y{+sBJ+3a#FUW@iP>;Fy)@zIx%dp}=j-pIf+22)ShXy;K zDBh!X^5nw~fGyj!_Z3lO!*+=kL|UJYeB}QBsNZM&CP6V* z&JjHMtP8!)nqv~D7;y7d=LeqDQ_LEZJX6RNR%gy!kEgwiZWU_lMQMjTrolr3b}0mr zZ{$b>pU)+U=gL_b7hZ8=zLGM~=eJ#dX(*{@zdzAEWZ|G^UM;ji$G6~`$7dpI*B!0g zqM~?c*E`w!U4q1NKbBqYJzKU`KaKz1OLCDOgQw4jf2c`@Evl(LA|XE{M-?hoQWYy! z2$ih4^^ErnDU+SOQ6a5MC$I_+9rBubdeFUUnMRT@fTX(S=DP}vuM2x`y2E$L6+{{_ z8Ip^%kX^_0J#`a|XM<58_&7gfn z)70+KZ=NIXT6Bk5gPLgo8zH|!saAT$JI*BCu;5}lIkHX0)0y*ux(TDRLf;mUILg{LI7TDkA= zJ$KM#&b9iS^<;o=8~^=SHRIRG2}QMbd4d?p96e#@+g0!X(Fh`)$_&e4E~z34`I*`G z)cRPR_w8nMsNBsmG<3r}d&v7!xrS{Z@(sUEo~Wk%`iq)0VO&B5$N#Bf_A@?4A{ zR=eZ{QKT67mWGX+sFtnUE6j4UDpZWCCnouly;A<=*WIeJH*Hk&=KP@+%+)`wlo@ZQ zJ6_qvh!L-K^ih4gg0f=#PfR@)M8P#^*xBN#mza4ukV}-fCBjAStvTB2tcBn=x%_ zkwF%aqGg-5MmxBuc@#;ZYYHrBF@of>X@v~q%)yzz{hDu*s$5nl$i}`!i&lH7)@?f! zlTSgb1@L2}=AQ8^*SkRq*IzEp8`$f}hq3LI%bh8i_%sa9>;shYLx8Yx;P8d^m) zR1te9MvwyTFvzkM%a$lkq}sG;lUguucGMevh$I8wPKYPR3Kc4-##-~ZR-JkY_h0P1 z#rL^yfno&7ecKi1{0Fo^k#zg>59pWTPt1@OC`OPB>4@TLWM&H#BS>cUOL46-qy>r* zBttr)xEh(+0>ucDnf+2+s|;y@Vg$*Mjwr51X12g?nOLnq|I9WUZtK^sQET;Y%U^#+ zs#&6>%F>2Bcn_&ryN;?-wYn;wRUzX(VXO9(f^iZ~>hg2rZriqPYW0exs#=ZOs&tt$ z8TZ}0VudmWK~^kXqCWfNJvHISZ`GRBiOpf9Gb(6x@&@`qO>~YLH`_xUtdB=1QKE!I z7s~agao?#I|L-yLZt9Yyk+^i(GOBdxGP$bTmMwp&ty{LLO&d3u`~#mHtZLV5SSa82 zPkU*hBgks4lJWdg1J$%Cle}#Y0_ag;Tweb5*XEcso3vfT%<0q2uUxtEs{0{FsNHre z+|NSgL;%;v)KP7cki1_@lqi+=LW)aBss##lBz^eKE9!%{UyG?|n|2*kpJPtcB+A}V zqy!|!l11~358lyVe5y9Ai}t8qx=iVqveF#@Ld>#!^n&13s#RBI^^pdUMPbdHNufI~yJ^7IO`|rO^IatT#mb?P_mMvbWmM>eZvN!4(U&EHI zf0;pDvvxhze2-R9$Z8UV{M)i+iz-#Bl)7`^Q>t;(=H66)Z{MaaJoVTpXVBAKWx(eg zeZnbffBoJtL|ngajas#0x$1ZNxuM8G*?;}HQ}AHDi28u+J`xdu#08gMr|!J*O0{Fh z-}<*h-EuX~vKN>VSebkP{{j$?xo)-UHB^Uvx~PNu9Iek*2|qI>+49|Y(k?;@U(x?I z)p@_j0Y;O59;XHkxYZ;l4XbEEfR)sTbMOsJ9)texwUqidVfjfUNV~G)?=z>Vdv1h(-r#2=$XN5=*&~kzEJhh$Di5aQ__Aj>NDM; zPeu)7nC{0OxKqI|Tz{laV7;9$zyoxLt4DIzf>KoRtkJ~B?_}j0)M9rU@Lryhry1QPzp71XD zl>4uP`=|qZ9;P0?=VsmcKSh=Q)A%3Noi|*r?tJj+v@iWp74bY&)q;UK4H_8ag*?0Q z&WF^uqeiOdo_aVc^?Pr=qQ9?KTyiTa!_J2+ZJW}5pRO_A-Aj|o@4xw?8aaHZs-%(6 z2SWDQac$anR4rP!vxCh2F(;m?E<3xgHXDwOi%DDn0cqPwXK;Qs;^UZ4I6x1+{DAW= zyIx&=^WAZadhUq_qmToV=8gxS@*;;_5Qx(M_6N)?XNPkS@AB^JgWcdlfA*cUbwJPF zs@p-)tK0VNf9sVc^+QNa`T9IXA8>A?kB{=Dm9TAFw#jYOwRGvSdPBIHDz#hTNGQ{W zAXwG85*SFEJzBJiYLJ3@nEdm2_1)O7qKZHDoQqQGNRV)MEeLP{Qf2L$m2PlGJ|BP5 znK83|*x=0V2ej?jMLqT6yHTWPZp&hFdia@F)B)NXOJP9T5M-&QCq*19A6mI;mC$8a z(&WdL)$6_2{})vht4EjpyGMm3dZ4o|xV(Ri1dH$b@SWEZWzUHCij^v<5+&2t?HTdS zd8tI&T593EIWhJ8Tkk!j1D0riR`V>rgLm%HqD?~5%2&7gjhd)dd&SbjzfAa1vA0pw z0Hm)~w_Z`JcG~KjHUwFry?qoGj2$&1MqJyrW2anK=ZNF_#pGSLcCA{sdQD7trUkGg zzhnFMpjx%+)b%pQL4uH*-QWpAq=u~6D_1NF$>|T?txUMQ2v0GHxC;ZqYn9KnDW&-h z7fPk5GsS)f2TNA0T*ZiE+59S_)k}8M;?KXgZqpl7h#UXX=U!}5b?Y@y|32j$^U8H= zVh3)SJZQr<(MpjF5StO+v17Xy1xGgJ-0#e!=kLG%G9YzXJ+8R^wiuoynfhjH4=s~_ z`q7Aa`F8e=MAZAScGyinJpFR=e8{hpe%5;?b<{SU50LVvXxKv?q@7$39r8x-DeOO?0i7FQx17D8FkoXZY{S!+=bO=AD1JUbnA3W%j=rb@4=Ypz zVYy$ka(UAGAPpKf4bI2-W7c%DBM$=DaLLs-#cb4-E}g$3!mO-{dIgJKLF_tY!abXr zqdxmstKYOVLWQ68iNDP3v==MKSr=X&(@v?@8(W;R<;ogWd?qh@e?wM+RLZQSZG5t` zv(2U~Wl7(%LKPOI4MFzmu#fuu(~qLQlLlb(?O?sTnkqEbN{eOmZth>3b99=RX?&Ya z($MxhZ1TtsPv(9%jc^!k{i!=MQ(=oHH(wq6jJJCA8Z^rD@y%gsF>RVv1X!_htUjn#H*R zv;y3fs&%`LDKVwFd2hTtsIl7s2`Qu{lfX@P42)XAiF@q++ZCqH$$(paygWU%*)w)K zVSuzMSwI{3$m36rs(Z3|^CMl_5QNGebK>cmZ-|}Er%#=%ZPCBU^Y@@vw02E2=|iEd z_UfSOHHhYO7$EOzrMld}mm`J-gOB_m|53UVGY#w+o)!A|lTMGSA}h;XH(yZ*dkh|W zDyn9g^N>~q!Fj=HAx2S8KYXuN{QEsmEyF;Cv~Yp&lg>Eb4OTO!{i-HuS$-LUsMNctT^-+47kdIkA% z(UrNdXR)XEoo`QO&?Q|_PeLY&8}EKNW)(*x`N@HId2)mYZPtwQ_HE(f*FR=XSN$(L z+ZalPmy#2+eqA(gniCVbyGz^fpst^8NIQb?f!F=#UWFB*3k=u??zq9Ik26yz0-t0h zEDwX@x?Atpa{Fj2KM?b_>n>G4jr%q%YY_g;=+D&9cXKw{C=(U?N_uBMRam}caSR^= zU*nSurCK)W!;pafsRL6y2cMe!GCZb{H@? z9)9L!g+{9lAm-x_-lZP7_dm)<7GQe%)WdhH=bs!DWMj|e*83h!S_%|o+Nyw3yattyrym&*Q!f&_<D^0z>{V zklDa}PfN%st8BS)G4CK?vuFQk#;;Qif=(;I(k&^5ubJfFO~+Vb*-|g}05FsU7a#)49*+f#C8I zl8znS+FF+jE#MvcZ_TPz5@oKCYFBCRBbckE3Xl@mqYu)Zj-v?|&=fxFqN~haLS)B2 zzHw;y@}qwDjn8xH{+J zYfLgGrT&+lm9n!2GSl`YwGGQ7&%UO=AzGXrR)_ow&lm);KZwR%IDfWUq+Qx9U$#gu zxob3Sz9Vp)z-(BnP94>F_ZBJntFqtW-mhA@OiR)J(C=GnqRMJlp_?{rR5kT(Go>|5Sv<2Sg9SiGtc3n!K(R6cXt4scL3!=x%XT7RBAkCB*(r_sxt0XsPKUDaA|mf0 zi*E&Sic)snh})|5q0FYJz88Hq%8!&CVbG~B>7?G|Ls4{trVOfShY}2K0qR9MLilDB zDv7thow5KakN2Wd#J42#4JLCw{>w-N$;a<2E~>Z%^3(!_!ldV^#>JV7TOf@sP>djH zZ0zD!6}=WHMv$V{XT>#6V+#}`NE#cv_*F%(1xjuxcB^0Xdaby|X={O!3l=Ph*(FWe z+lyaR+yWWi0wuR=7cfhgMr8jPUf1I46}LdzTfp!j_J&g0Yf${c;ugr@7WjW*KALIM SRH@tm0000 0 { + spacesChild := append(spaces, last) + result += p.printItems(f.Items(), spacesChild) + } + } + return result +} diff --git a/vendor/github.com/jinzhu/copier/License b/vendor/github.com/jinzhu/copier/License new file mode 100644 index 000000000..e2dc5381e --- /dev/null +++ b/vendor/github.com/jinzhu/copier/License @@ -0,0 +1,20 @@ +The MIT License (MIT) + +Copyright (c) 2015 Jinzhu + +Permission is hereby granted, free of charge, to any person obtaining a copy of +this software and associated documentation files (the "Software"), to deal in +the Software without restriction, including without limitation the rights to +use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of +the Software, and to permit persons to whom the Software is furnished to do so, +subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS +FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR +COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER +IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. diff --git a/vendor/github.com/jinzhu/copier/README.md b/vendor/github.com/jinzhu/copier/README.md new file mode 100644 index 000000000..cff72405c --- /dev/null +++ b/vendor/github.com/jinzhu/copier/README.md @@ -0,0 +1,131 @@ +# Copier + + I am a copier, I copy everything from one to another + +[![test status](https://github.com/jinzhu/copier/workflows/tests/badge.svg?branch=master "test status")](https://github.com/jinzhu/copier/actions) + +## Features + +* Copy from field to field with same name +* Copy from method to field with same name +* Copy from field to method with same name +* Copy from slice to slice +* Copy from struct to slice +* Copy from map to map +* Enforce copying a field with a tag +* Ignore a field with a tag +* Deep Copy + +## Usage + +```go +package main + +import ( + "fmt" + "github.com/jinzhu/copier" +) + +type User struct { + Name string + Role string + Age int32 + + // Explicitly ignored in the destination struct. + Salary int +} + +func (user *User) DoubleAge() int32 { + return 2 * user.Age +} + +// Tags in the destination Struct provide instructions to copier.Copy to ignore +// or enforce copying and to panic or return an error if a field was not copied. +type Employee struct { + // Tell copier.Copy to panic if this field is not copied. + Name string `copier:"must"` + + // Tell copier.Copy to return an error if this field is not copied. + Age int32 `copier:"must,nopanic"` + + // Tell copier.Copy to explicitly ignore copying this field. + Salary int `copier:"-"` + + DoubleAge int32 + EmployeId int64 + SuperRole string +} + +func (employee *Employee) Role(role string) { + employee.SuperRole = "Super " + role +} + +func main() { + var ( + user = User{Name: "Jinzhu", Age: 18, Role: "Admin", Salary: 200000} + users = []User{{Name: "Jinzhu", Age: 18, Role: "Admin", Salary: 100000}, {Name: "jinzhu 2", Age: 30, Role: "Dev", Salary: 60000}} + employee = Employee{Salary: 150000} + employees = []Employee{} + ) + + copier.Copy(&employee, &user) + + fmt.Printf("%#v \n", employee) + // Employee{ + // Name: "Jinzhu", // Copy from field + // Age: 18, // Copy from field + // Salary:150000, // Copying explicitly ignored + // DoubleAge: 36, // Copy from method + // EmployeeId: 0, // Ignored + // SuperRole: "Super Admin", // Copy to method + // } + + // Copy struct to slice + copier.Copy(&employees, &user) + + fmt.Printf("%#v \n", employees) + // []Employee{ + // {Name: "Jinzhu", Age: 18, Salary:0, DoubleAge: 36, EmployeId: 0, SuperRole: "Super Admin"} + // } + + // Copy slice to slice + employees = []Employee{} + copier.Copy(&employees, &users) + + fmt.Printf("%#v \n", employees) + // []Employee{ + // {Name: "Jinzhu", Age: 18, Salary:0, DoubleAge: 36, EmployeId: 0, SuperRole: "Super Admin"}, + // {Name: "jinzhu 2", Age: 30, Salary:0, DoubleAge: 60, EmployeId: 0, SuperRole: "Super Dev"}, + // } + + // Copy map to map + map1 := map[int]int{3: 6, 4: 8} + map2 := map[int32]int8{} + copier.Copy(&map2, map1) + + fmt.Printf("%#v \n", map2) + // map[int32]int8{3:6, 4:8} +} +``` + +### Copy with Option + +```go +copier.CopyWithOption(&to, &from, copier.Option{IgnoreEmpty: true, DeepCopy: true}) +``` + +## Contributing + +You can help to make the project better, check out [http://gorm.io/contribute.html](http://gorm.io/contribute.html) for things you can do. + +# Author + +**jinzhu** + +* +* +* + +## License + +Released under the [MIT License](https://github.com/jinzhu/copier/blob/master/License). diff --git a/vendor/github.com/jinzhu/copier/copier.go b/vendor/github.com/jinzhu/copier/copier.go new file mode 100644 index 000000000..72bf65c78 --- /dev/null +++ b/vendor/github.com/jinzhu/copier/copier.go @@ -0,0 +1,491 @@ +package copier + +import ( + "database/sql" + "database/sql/driver" + "fmt" + "reflect" + "strings" +) + +// These flags define options for tag handling +const ( + // Denotes that a destination field must be copied to. If copying fails then a panic will ensue. + tagMust uint8 = 1 << iota + + // Denotes that the program should not panic when the must flag is on and + // value is not copied. The program will return an error instead. + tagNoPanic + + // Ignore a destination field from being copied to. + tagIgnore + + // Denotes that the value as been copied + hasCopied +) + +// Option sets copy options +type Option struct { + // setting this value to true will ignore copying zero values of all the fields, including bools, as well as a + // struct having all it's fields set to their zero values respectively (see IsZero() in reflect/value.go) + IgnoreEmpty bool + DeepCopy bool +} + +// Copy copy things +func Copy(toValue interface{}, fromValue interface{}) (err error) { + return copier(toValue, fromValue, Option{}) +} + +// CopyWithOption copy with option +func CopyWithOption(toValue interface{}, fromValue interface{}, opt Option) (err error) { + return copier(toValue, fromValue, opt) +} + +func copier(toValue interface{}, fromValue interface{}, opt Option) (err error) { + var ( + isSlice bool + amount = 1 + from = indirect(reflect.ValueOf(fromValue)) + to = indirect(reflect.ValueOf(toValue)) + ) + + if !to.CanAddr() { + return ErrInvalidCopyDestination + } + + // Return is from value is invalid + if !from.IsValid() { + return ErrInvalidCopyFrom + } + + fromType, isPtrFrom := indirectType(from.Type()) + toType, _ := indirectType(to.Type()) + + if fromType.Kind() == reflect.Interface { + fromType = reflect.TypeOf(from.Interface()) + } + + if toType.Kind() == reflect.Interface { + toType, _ = indirectType(reflect.TypeOf(to.Interface())) + oldTo := to + to = reflect.New(reflect.TypeOf(to.Interface())).Elem() + defer func() { + oldTo.Set(to) + }() + } + + // Just set it if possible to assign for normal types + if from.Kind() != reflect.Slice && from.Kind() != reflect.Struct && from.Kind() != reflect.Map && (from.Type().AssignableTo(to.Type()) || from.Type().ConvertibleTo(to.Type())) { + if !isPtrFrom || !opt.DeepCopy { + to.Set(from.Convert(to.Type())) + } else { + fromCopy := reflect.New(from.Type()) + fromCopy.Set(from.Elem()) + to.Set(fromCopy.Convert(to.Type())) + } + return + } + + if from.Kind() != reflect.Slice && fromType.Kind() == reflect.Map && toType.Kind() == reflect.Map { + if !fromType.Key().ConvertibleTo(toType.Key()) { + return ErrMapKeyNotMatch + } + + if to.IsNil() { + to.Set(reflect.MakeMapWithSize(toType, from.Len())) + } + + for _, k := range from.MapKeys() { + toKey := indirect(reflect.New(toType.Key())) + if !set(toKey, k, opt.DeepCopy) { + return fmt.Errorf("%w map, old key: %v, new key: %v", ErrNotSupported, k.Type(), toType.Key()) + } + + elemType, _ := indirectType(toType.Elem()) + toValue := indirect(reflect.New(elemType)) + if !set(toValue, from.MapIndex(k), opt.DeepCopy) { + if err = copier(toValue.Addr().Interface(), from.MapIndex(k).Interface(), opt); err != nil { + return err + } + } + + for { + if elemType == toType.Elem() { + to.SetMapIndex(toKey, toValue) + break + } + elemType = reflect.PtrTo(elemType) + toValue = toValue.Addr() + } + } + return + } + + if from.Kind() == reflect.Slice && to.Kind() == reflect.Slice && fromType.ConvertibleTo(toType) { + if to.IsNil() { + slice := reflect.MakeSlice(reflect.SliceOf(to.Type().Elem()), from.Len(), from.Cap()) + to.Set(slice) + } + + for i := 0; i < from.Len(); i++ { + if to.Len() < i+1 { + to.Set(reflect.Append(to, reflect.New(to.Type().Elem()).Elem())) + } + + if !set(to.Index(i), from.Index(i), opt.DeepCopy) { + err = CopyWithOption(to.Index(i).Addr().Interface(), from.Index(i).Interface(), opt) + if err != nil { + continue + } + } + } + return + } + + if fromType.Kind() != reflect.Struct || toType.Kind() != reflect.Struct { + // skip not supported type + return + } + + if to.Kind() == reflect.Slice { + isSlice = true + if from.Kind() == reflect.Slice { + amount = from.Len() + } + } + + for i := 0; i < amount; i++ { + var dest, source reflect.Value + + if isSlice { + // source + if from.Kind() == reflect.Slice { + source = indirect(from.Index(i)) + } else { + source = indirect(from) + } + // dest + dest = indirect(reflect.New(toType).Elem()) + } else { + source = indirect(from) + dest = indirect(to) + } + + destKind := dest.Kind() + initDest := false + if destKind == reflect.Interface { + initDest = true + dest = indirect(reflect.New(toType)) + } + + // Get tag options + tagBitFlags := map[string]uint8{} + if dest.IsValid() { + tagBitFlags = getBitFlags(toType) + } + + // check source + if source.IsValid() { + // Copy from source field to dest field or method + fromTypeFields := deepFields(fromType) + for _, field := range fromTypeFields { + name := field.Name + + // Get bit flags for field + fieldFlags, _ := tagBitFlags[name] + + // Check if we should ignore copying + if (fieldFlags & tagIgnore) != 0 { + continue + } + + if fromField := source.FieldByName(name); fromField.IsValid() && !shouldIgnore(fromField, opt.IgnoreEmpty) { + // process for nested anonymous field + destFieldNotSet := false + if f, ok := dest.Type().FieldByName(name); ok { + for idx := range f.Index { + destField := dest.FieldByIndex(f.Index[:idx+1]) + + if destField.Kind() != reflect.Ptr { + continue + } + + if !destField.IsNil() { + continue + } + if !destField.CanSet() { + destFieldNotSet = true + break + } + + // destField is a nil pointer that can be set + newValue := reflect.New(destField.Type().Elem()) + destField.Set(newValue) + } + } + + if destFieldNotSet { + break + } + + toField := dest.FieldByName(name) + if toField.IsValid() { + if toField.CanSet() { + if !set(toField, fromField, opt.DeepCopy) { + if err := copier(toField.Addr().Interface(), fromField.Interface(), opt); err != nil { + return err + } + } + if fieldFlags != 0 { + // Note that a copy was made + tagBitFlags[name] = fieldFlags | hasCopied + } + } + } else { + // try to set to method + var toMethod reflect.Value + if dest.CanAddr() { + toMethod = dest.Addr().MethodByName(name) + } else { + toMethod = dest.MethodByName(name) + } + + if toMethod.IsValid() && toMethod.Type().NumIn() == 1 && fromField.Type().AssignableTo(toMethod.Type().In(0)) { + toMethod.Call([]reflect.Value{fromField}) + } + } + } + } + + // Copy from from method to dest field + for _, field := range deepFields(toType) { + name := field.Name + + var fromMethod reflect.Value + if source.CanAddr() { + fromMethod = source.Addr().MethodByName(name) + } else { + fromMethod = source.MethodByName(name) + } + + if fromMethod.IsValid() && fromMethod.Type().NumIn() == 0 && fromMethod.Type().NumOut() == 1 && !shouldIgnore(fromMethod, opt.IgnoreEmpty) { + if toField := dest.FieldByName(name); toField.IsValid() && toField.CanSet() { + values := fromMethod.Call([]reflect.Value{}) + if len(values) >= 1 { + set(toField, values[0], opt.DeepCopy) + } + } + } + } + } + + if isSlice { + if dest.Addr().Type().AssignableTo(to.Type().Elem()) { + if to.Len() < i+1 { + to.Set(reflect.Append(to, dest.Addr())) + } else { + set(to.Index(i), dest.Addr(), opt.DeepCopy) + } + } else if dest.Type().AssignableTo(to.Type().Elem()) { + if to.Len() < i+1 { + to.Set(reflect.Append(to, dest)) + } else { + set(to.Index(i), dest, opt.DeepCopy) + } + } + } else if initDest { + to.Set(dest) + } + + err = checkBitFlags(tagBitFlags) + } + + return +} + +func shouldIgnore(v reflect.Value, ignoreEmpty bool) bool { + if !ignoreEmpty { + return false + } + + return v.IsZero() +} + +func deepFields(reflectType reflect.Type) []reflect.StructField { + if reflectType, _ = indirectType(reflectType); reflectType.Kind() == reflect.Struct { + fields := make([]reflect.StructField, 0, reflectType.NumField()) + + for i := 0; i < reflectType.NumField(); i++ { + v := reflectType.Field(i) + if v.Anonymous { + fields = append(fields, deepFields(v.Type)...) + } else { + fields = append(fields, v) + } + } + + return fields + } + + return nil +} + +func indirect(reflectValue reflect.Value) reflect.Value { + for reflectValue.Kind() == reflect.Ptr { + reflectValue = reflectValue.Elem() + } + return reflectValue +} + +func indirectType(reflectType reflect.Type) (_ reflect.Type, isPtr bool) { + for reflectType.Kind() == reflect.Ptr || reflectType.Kind() == reflect.Slice { + reflectType = reflectType.Elem() + isPtr = true + } + return reflectType, isPtr +} + +func set(to, from reflect.Value, deepCopy bool) bool { + if from.IsValid() { + if to.Kind() == reflect.Ptr { + // set `to` to nil if from is nil + if from.Kind() == reflect.Ptr && from.IsNil() { + to.Set(reflect.Zero(to.Type())) + return true + } else if to.IsNil() { + // `from` -> `to` + // sql.NullString -> *string + if fromValuer, ok := driverValuer(from); ok { + v, err := fromValuer.Value() + if err != nil { + return false + } + // if `from` is not valid do nothing with `to` + if v == nil { + return true + } + } + // allocate new `to` variable with default value (eg. *string -> new(string)) + to.Set(reflect.New(to.Type().Elem())) + } + // depointer `to` + to = to.Elem() + } + + if deepCopy { + toKind := to.Kind() + if toKind == reflect.Interface && to.IsNil() { + if reflect.TypeOf(from.Interface()) != nil { + to.Set(reflect.New(reflect.TypeOf(from.Interface())).Elem()) + toKind = reflect.TypeOf(to.Interface()).Kind() + } + } + if toKind == reflect.Struct || toKind == reflect.Map || toKind == reflect.Slice { + return false + } + } + + if from.Type().ConvertibleTo(to.Type()) { + to.Set(from.Convert(to.Type())) + } else if toScanner, ok := to.Addr().Interface().(sql.Scanner); ok { + // `from` -> `to` + // *string -> sql.NullString + if from.Kind() == reflect.Ptr { + // if `from` is nil do nothing with `to` + if from.IsNil() { + return true + } + // depointer `from` + from = indirect(from) + } + // `from` -> `to` + // string -> sql.NullString + // set `to` by invoking method Scan(`from`) + err := toScanner.Scan(from.Interface()) + if err != nil { + return false + } + } else if fromValuer, ok := driverValuer(from); ok { + // `from` -> `to` + // sql.NullString -> string + v, err := fromValuer.Value() + if err != nil { + return false + } + // if `from` is not valid do nothing with `to` + if v == nil { + return true + } + rv := reflect.ValueOf(v) + if rv.Type().AssignableTo(to.Type()) { + to.Set(rv) + } + } else if from.Kind() == reflect.Ptr { + return set(to, from.Elem(), deepCopy) + } else { + return false + } + } + + return true +} + +// parseTags Parses struct tags and returns uint8 bit flags. +func parseTags(tag string) (flags uint8) { + for _, t := range strings.Split(tag, ",") { + switch t { + case "-": + flags = tagIgnore + return + case "must": + flags = flags | tagMust + case "nopanic": + flags = flags | tagNoPanic + } + } + return +} + +// getBitFlags Parses struct tags for bit flags. +func getBitFlags(toType reflect.Type) map[string]uint8 { + flags := map[string]uint8{} + toTypeFields := deepFields(toType) + + // Get a list dest of tags + for _, field := range toTypeFields { + tags := field.Tag.Get("copier") + if tags != "" { + flags[field.Name] = parseTags(tags) + } + } + return flags +} + +// checkBitFlags Checks flags for error or panic conditions. +func checkBitFlags(flagsList map[string]uint8) (err error) { + // Check flag conditions were met + for name, flags := range flagsList { + if flags&hasCopied == 0 { + switch { + case flags&tagMust != 0 && flags&tagNoPanic != 0: + err = fmt.Errorf("field %s has must tag but was not copied", name) + return + case flags&(tagMust) != 0: + panic(fmt.Sprintf("Field %s has must tag but was not copied", name)) + } + } + } + return +} + +func driverValuer(v reflect.Value) (i driver.Valuer, ok bool) { + + if !v.CanAddr() { + i, ok = v.Interface().(driver.Valuer) + return + } + + i, ok = v.Addr().Interface().(driver.Valuer) + return +} diff --git a/vendor/github.com/jinzhu/copier/errors.go b/vendor/github.com/jinzhu/copier/errors.go new file mode 100644 index 000000000..cf7c5e74b --- /dev/null +++ b/vendor/github.com/jinzhu/copier/errors.go @@ -0,0 +1,10 @@ +package copier + +import "errors" + +var ( + ErrInvalidCopyDestination = errors.New("copy destination is invalid") + ErrInvalidCopyFrom = errors.New("copy from is invalid") + ErrMapKeyNotMatch = errors.New("map's key type doesn't match") + ErrNotSupported = errors.New("not supported") +) diff --git a/vendor/github.com/jinzhu/copier/go.mod b/vendor/github.com/jinzhu/copier/go.mod new file mode 100644 index 000000000..531422dcb --- /dev/null +++ b/vendor/github.com/jinzhu/copier/go.mod @@ -0,0 +1,3 @@ +module github.com/jinzhu/copier + +go 1.15 diff --git a/vendor/modules.txt b/vendor/modules.txt index dd0eab57d..3724cb7eb 100644 --- a/vendor/modules.txt +++ b/vendor/modules.txt @@ -159,6 +159,9 @@ github.com/coreos/go-systemd/v22/dbus github.com/cyphar/filepath-securejoin # github.com/davecgh/go-spew v1.1.1 github.com/davecgh/go-spew/spew +# github.com/disiqueira/gotree/v3 v3.0.2 +## explicit +github.com/disiqueira/gotree/v3 # github.com/docker/distribution v2.7.1+incompatible ## explicit github.com/docker/distribution @@ -244,6 +247,9 @@ github.com/hashicorp/go-multierror github.com/imdario/mergo # github.com/inconshreveable/mousetrap v1.0.0 github.com/inconshreveable/mousetrap +# github.com/jinzhu/copier v0.3.0 +## explicit +github.com/jinzhu/copier # github.com/juju/ansiterm v0.0.0-20180109212912-720a0952cc2a github.com/juju/ansiterm github.com/juju/ansiterm/tabwriter