Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[vtadmin-api] Add /api/schemas endpoint to vtadmin-api #7432

Merged

Conversation

doeg
Copy link
Contributor

@doeg doeg commented Feb 2, 2021

Signed-off-by: Sara Bee [email protected]

Description

A simplified version of our Slack-internal /api/schemas endpoint. Notably:

  • This does not provide a mechanism to filter by anything other than cluster ID
  • This does not implement any caching

"Don't forget" to expand the api_test.go diff 😭

@ajm188 thanks as always for weathering my incessant whining questions!

Queries against local Vitess

curl "http://localhost:14200/api/schemas" | jq .
{
  "result": {
    "schemas": [
      {
        "cluster": {
          "id": "id1",
          "name": "cluster1"
        },
        "keyspace": "commerce",
        "table_definitions": [
          {
            "name": "customer_seq",
            "schema": "CREATE TABLE `customer_seq` (\n  `id` int(11) NOT NULL,\n  `next_id` bigint(20) DEFAULT NULL,\n  `cache` bigint(20) DEFAULT NULL,\n  PRIMARY KEY (`id`)\n) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='vitess_sequence'",
            "columns": [
              "id",
              "next_id",
              "cache"
            ],
            "primary_key_columns": [
              "id"
            ],
            "type": "BASE TABLE",
            "data_length": 16384,
            "row_count": 1,
            "fields": [
              {
                "name": "id",
                "type": 263,
                "table": "customer_seq",
                "org_table": "customer_seq",
                "database": "vt_commerce",
                "org_name": "id",
                "column_length": 11,
                "charset": 63,
                "flags": 53251
              },
              {
                "name": "next_id",
                "type": 265,
                "table": "customer_seq",
                "org_table": "customer_seq",
                "database": "vt_commerce",
                "org_name": "next_id",
                "column_length": 20,
                "charset": 63,
                "flags": 32768
              },
              {
                "name": "cache",
                "type": 265,
                "table": "customer_seq",
                "org_table": "customer_seq",
                "database": "vt_commerce",
                "org_name": "cache",
                "column_length": 20,
                "charset": 63,
                "flags": 32768
              }
            ]
          },
          {
            "name": "order_seq",
            "schema": "CREATE TABLE `order_seq` (\n  `id` int(11) NOT NULL,\n  `next_id` bigint(20) DEFAULT NULL,\n  `cache` bigint(20) DEFAULT NULL,\n  PRIMARY KEY (`id`)\n) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='vitess_sequence'",
            "columns": [
              "id",
              "next_id",
              "cache"
            ],
            "primary_key_columns": [
              "id"
            ],
            "type": "BASE TABLE",
            "data_length": 16384,
            "row_count": 1,
            "fields": [
              {
                "name": "id",
                "type": 263,
                "table": "order_seq",
                "org_table": "order_seq",
                "database": "vt_commerce",
                "org_name": "id",
                "column_length": 11,
                "charset": 63,
                "flags": 53251
              },
              {
                "name": "next_id",
                "type": 265,
                "table": "order_seq",
                "org_table": "order_seq",
                "database": "vt_commerce",
                "org_name": "next_id",
                "column_length": 20,
                "charset": 63,
                "flags": 32768
              },
              {
                "name": "cache",
                "type": 265,
                "table": "order_seq",
                "org_table": "order_seq",
                "database": "vt_commerce",
                "org_name": "cache",
                "column_length": 20,
                "charset": 63,
                "flags": 32768
              }
            ]
          },
          {
            "name": "product",
            "schema": "CREATE TABLE `product` (\n  `sku` varbinary(128) NOT NULL,\n  `description` varbinary(128) DEFAULT NULL,\n  `price` bigint(20) DEFAULT NULL,\n  PRIMARY KEY (`sku`)\n) ENGINE=InnoDB DEFAULT CHARSET=utf8",
            "columns": [
              "sku",
              "description",
              "price"
            ],
            "primary_key_columns": [
              "sku"
            ],
            "type": "BASE TABLE",
            "data_length": 16384,
            "fields": [
              {
                "name": "sku",
                "type": 10262,
                "table": "product",
                "org_table": "product",
                "database": "vt_commerce",
                "org_name": "sku",
                "column_length": 128,
                "charset": 63,
                "flags": 20611
              },
              {
                "name": "description",
                "type": 10262,
                "table": "product",
                "org_table": "product",
                "database": "vt_commerce",
                "org_name": "description",
                "column_length": 128,
                "charset": 63,
                "flags": 128
              },
              {
                "name": "price",
                "type": 265,
                "table": "product",
                "org_table": "product",
                "database": "vt_commerce",
                "org_name": "price",
                "column_length": 20,
                "charset": 63,
                "flags": 32768
              }
            ]
          }
        ]
      }
    ]
  },
  "ok": true
}
curl "http://localhost:14200/api/schemas?cluster=nope" | jq .
{
  "result": {},
  "ok": true
}

Related Issue(s)

Checklist

  • Should this PR be backported?
  • Tests were added or are not required
  • Documentation was added or is not required

Deployment Notes

N/A

Impacted Areas in Vitess

Components that this PR will affect:

  • Query Serving
  • VReplication
  • Cluster Management
  • Build/CI
  • VTAdmin

@doeg doeg requested review from ajm188 and rohit-nayak-ps February 2, 2021 22:03
@doeg doeg force-pushed the sarabee-vtadmin-schemas-endpoint branch from aea8170 to 8bf8622 Compare February 2, 2021 22:19
Copy link
Contributor

@ajm188 ajm188 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lg2m!

}

// getSchemas returns all of the schemas across all keyspaces in the given cluster.
func (api *API) getSchemas(ctx context.Context, c *cluster.Cluster, tablets []*vtadminpb.Tablet) ([]*vtadminpb.Schema, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIOLI: I know we talked a bit about this file potentially growing very large as the API surface area grows, and these methods don't actually depend on api for anything, so if you wanted to you could make these top-level private functions in another file, but also this is fine as-is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll leave it for now. I agree with you on one file offering better readability. Even though these methods don't use api directly, the colocation seems good. One day I'll get used to the verbosity.

go/vt/vtadmin/api_test.go Outdated Show resolved Hide resolved
tabletSchemas map[string]*tabletmanagerdata.SchemaDefinition
req *vtadminpb.GetSchemasRequest
expected *vtadminpb.GetSchemasResponse
}{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ugh, I am sorry lol. These are great tests though!

@rohit-nayak-ps rohit-nayak-ps merged commit 77fd148 into vitessio:master Feb 4, 2021
@askdba askdba added the Component: VTAdmin VTadmin interface label Feb 8, 2021
@doeg doeg deleted the sarabee-vtadmin-schemas-endpoint branch March 1, 2021 00:50
@doeg doeg changed the title Add /api/schemas endpoint to vtadmin-api [vtadmin-api] Add /api/schemas endpoint to vtadmin-api Mar 16, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Component: VTAdmin VTadmin interface
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants