Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adds support for the /v2/_catalog API #548

Merged
merged 10 commits into from
Oct 25, 2019
45 changes: 45 additions & 0 deletions cmd/crane/cmd/catalog.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
// Copyright 2018 Google LLC All Rights Reserved.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: 2019

//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package cmd

import (
"fmt"
"log"

"github.com/google/go-containerregistry/pkg/crane"
"github.com/spf13/cobra"
)

func init() { Root.AddCommand(NewCmdGetCatalog()) }

// NewCmdGetCatalog creates a new cobra.Command for the repos subcommand.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: NewCmdCatalog

func NewCmdGetCatalog() *cobra.Command {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I'd name this "NewCmdCatalog" to be consistent with the rest of the commands.

return &cobra.Command{
Use: "repos",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd rather this just be catalog than repos

Short: "List the repos in a registry",
Args: cobra.ExactArgs(1),
Run: func(_ *cobra.Command, args []string) {
reg := args[0]
repos, err := crane.GetCatalog(reg)
if err != nil {
log.Fatalf("reading repos for %s: %v", reg, err)
}

for _, repo := range repos {
fmt.Println(repo)
}
},
}
}
51 changes: 51 additions & 0 deletions pkg/crane/catalog.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
// Copyright 2018 Google LLC All Rights Reserved.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: 2019

//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package crane

import (
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote"
)

// GetCatalog returns the repositories in a registry's catalog
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

supernit: add a period at the end of this sentence

func GetCatalog(src string) (res []string, err error) {
reg, err := name.NewRegistry(src)
if err != nil {
return nil, err
}

n := 100
last := ""
for {

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

supernit: drop this line

page, err := remote.GetCatalogPage(reg, last, n, remote.WithAuthFromKeychain(authn.DefaultKeychain))
if err != nil {
return nil, err
}

if len(page) > 0 {
last = page[len(page)-1]
res = append(res, page...)
}

if len(page) < n {
break
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

supernit: drop this line

}

return res, nil
}
70 changes: 70 additions & 0 deletions pkg/v1/remote/catalog.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
// Copyright 2018 Google LLC All Rights Reserved.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: 2019

//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package remote

import (
"encoding/json"
"fmt"
"net/http"
"net/url"

"github.com/google/go-containerregistry/pkg/name"
"github.com/google/go-containerregistry/pkg/v1/remote/transport"
)

type catalog struct {
Repos []string `json:"repositories"`
}

// GetCatalogPage calls /_catalog, returning the list of repositories on the registry
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

supernit: add a period at the end of this sentence

func GetCatalogPage(target name.Registry, last string, n int, options ...Option) ([]string, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the idea is to just make this a private method in the crane/catalog.go file. At some point in the future it can get moved to the remote library when we have interface concensus.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'm actually okay with leaving this in. If we ever come up with a good API for this, we can name it GetCatalog, and GetCatalogPage would still be useful.

I am a little bit unhappy with this because it doesn't allow clients to adhere to the spec:

Compliant client implementations should always use the Link header value when proceeding through results linearly.

However, this does enable you to skip ahead, which is immediately after:

The client may construct URLs to skip forward in the catalog.

I think we're basically making it impossible (sometimes, depends on implementation) for a registry to implement this efficiently by handing us a cursor in the Link header, but maybe we don't care.

o, err := makeOptions(target, options...)
if err != nil {
return nil, err
}

scopes := []string{target.Scope(transport.PullScope)}
tr, err := transport.New(target, o.auth, o.transport, scopes)
if err != nil {
return nil, err
}

query := fmt.Sprintf("last=%s&n=%d", url.QueryEscape(last), n)

uri := url.URL{
Scheme: target.Scheme(),
Host: target.RegistryStr(),
Path: "/v2/_catalog",
RawQuery: query,
}

client := http.Client{Transport: tr}
resp, err := client.Get(uri.String())
if err != nil {
return nil, err
}
defer resp.Body.Close()

if err := transport.CheckError(resp, http.StatusOK); err != nil {
return nil, err
}

var parsed catalog
if err := json.NewDecoder(resp.Body).Decode(&parsed); err != nil {
return nil, err
}

return parsed.Repos, nil
}
84 changes: 84 additions & 0 deletions pkg/v1/remote/catalog_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
// Copyright 2018 Google LLC All Rights Reserved.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: 2019

//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package remote

import (
"net/http"
"net/http/httptest"
"net/url"
"testing"

"github.com/google/go-cmp/cmp"
"github.com/google/go-containerregistry/pkg/authn"
"github.com/google/go-containerregistry/pkg/name"
)

func TestGetCatalog(t *testing.T) {
cases := []struct {
name string
responseBody []byte
wantErr bool
wantRepos []string
}{{
name: "success",
responseBody: []byte(`{"repositories":["test/test","foo/bar"]}`),
wantErr: false,
wantRepos: []string{"test/test", "foo/bar"},
}, {
name: "not json",
responseBody: []byte("notjson"),
wantErr: true,
}}
//TODO: add test cases for pagination
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Planning to do this later? 😄

supernit: add a space between // and TODO

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potentially, but I just wanted to note that they are missing if anyone runs into issues there later and wants to write some.
Do you need this?


for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
catalogPath := "/v2/_catalog"
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.URL.Path {
case "/v2/":
w.WriteHeader(http.StatusOK)
case catalogPath:
if r.Method != http.MethodGet {
t.Errorf("Method; got %v, want %v", r.Method, http.MethodGet)
}

w.Write(tc.responseBody)
default:
t.Fatalf("Unexpected path: %v", r.URL.Path)
}
}))
defer server.Close()
u, err := url.Parse(server.URL)
if err != nil {
t.Fatalf("url.Parse(%v) = %v", server.URL, err)
}

reg, err := name.NewRegistry(u.Host)
if err != nil {
t.Fatalf("name.NewRegistry(%v) = %v", u.Host, err)
}

repos, err := GetCatalog(reg, "", 100, WithAuthFromKeychain(authn.DefaultKeychain))
if (err != nil) != tc.wantErr {
t.Errorf("GetCatalog() wrong error: %v, want %v: %v\n", (err != nil), tc.wantErr, err)
}

if diff := cmp.Diff(tc.wantRepos, repos); diff != "" {
t.Errorf("GetCatalog() wrong repos (-want +got) = %s", diff)
}
})
}
}