Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(bigquery): link types on package docs #11036

Merged
merged 3 commits into from
Oct 29, 2024
Merged

docs(bigquery): link types on package docs #11036

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
bbf4d7e
docs(bigquery): link types on package docs
alvarowolfx Oct 25, 2024
8715e0d
docs(bigquery): link NewClient on DetectProjectID comments
alvarowolfx Oct 25, 2024
7e1db85
Merge branch 'main' into bq-docs-bq-link-types
alvarowolfx Oct 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions bigquery/bigquery.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,19 +65,19 @@ type Client struct {
enableQueryPreview bool
}

// DetectProjectID is a sentinel value that instructs NewClient to detect the
// project ID. It is given in place of the projectID argument. NewClient will
// DetectProjectID is a sentinel value that instructs [NewClient] to detect the
// project ID. It is given in place of the projectID argument. [NewClient] will
// use the project ID from the given credentials or the default credentials
// (https://developers.google.com/accounts/docs/application-default-credentials)
// if no credentials were provided. When providing credentials, not all
// options will allow NewClient to extract the project ID. Specifically a JWT
// options will allow [NewClient] to extract the project ID. Specifically a JWT
// does not have the project ID encoded.
const DetectProjectID = "*detect-project-id*"

// NewClient constructs a new Client which can perform BigQuery operations.
// NewClient constructs a new [Client] which can perform BigQuery operations.
// Operations performed via the client are billed to the specified GCP project.
//
// If the project ID is set to DetectProjectID, NewClient will attempt to detect
// If the project ID is set to [DetectProjectID], NewClient will attempt to detect
// the project ID from credentials.
//
// This client supports enabling query-related preview features via environmental
Expand Down
81 changes: 42 additions & 39 deletions bigquery/doc.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ connection pooling and similar aspects of this package.

# Creating a Client

To start working with this package, create a client:
To start working with this package, create a client with [NewClient]:

ctx := context.Background()
client, err := bigquery.NewClient(ctx, projectID)
Expand All @@ -33,7 +33,7 @@ To start working with this package, create a client:

# Querying

To query existing tables, create a Query and call its Read method, which starts the
To query existing tables, create a [Client.Query] and call its [Query.Read] method, which starts the
query and waits for it to complete:

q := client.Query(`
Expand All @@ -52,7 +52,7 @@ query and waits for it to complete:
}

Then iterate through the resulting rows. You can store a row using
anything that implements the ValueLoader interface, or with a slice or map of bigquery.Value.
anything that implements the [ValueLoader] interface, or with a slice or map of [Value].
A slice is simplest:

for {
Expand Down Expand Up @@ -86,7 +86,7 @@ You can also use a struct whose exported fields match the query:
}

You can also start the query running and get the results later.
Create the query as above, but call Run instead of Read. This returns a Job,
Create the query as above, but call [Query.Run] instead of [Query.Read]. This returns a [Job],
which represents an asynchronous operation.

job, err := q.Run(ctx)
Expand All @@ -100,17 +100,17 @@ the results at a later time, even in another process.
jobID := job.ID()
fmt.Printf("The job ID is %s\n", jobID)

To retrieve the job's results from the ID, first look up the Job:
To retrieve the job's results from the ID, first look up the [Job] with the [Client.JobFromID] method:

job, err = client.JobFromID(ctx, jobID)
if err != nil {
// TODO: Handle error.
}

Use the Job.Read method to obtain an iterator, and loop over the rows.
Calling Query.Read is preferred for queries with a relatively small result set,
Use the [Job.Read] method to obtain an iterator, and loop over the rows.
Calling [Query.Read] is preferred for queries with a relatively small result set,
as it will call BigQuery jobs.query API for a optimized query path. If the query
doesn't meet that criteria, the method will just combine Query.Run and Job.Read.
doesn't meet that criteria, the method will just combine [Query.Run] and [Job.Read].

it, err = job.Read(ctx)
if err != nil {
Expand All @@ -120,26 +120,26 @@ doesn't meet that criteria, the method will just combine Query.Run and Job.Read.

# Datasets and Tables

You can refer to datasets in the client's project with the Dataset method, and
in other projects with the DatasetInProject method:
You can refer to datasets in the client's project with the [Client.Dataset] method, and
in other projects with the [Client.DatasetInProject] method:

myDataset := client.Dataset("my_dataset")
yourDataset := client.DatasetInProject("your-project-id", "your_dataset")

These methods create references to datasets, not the datasets themselves. You can have
a dataset reference even if the dataset doesn't exist yet. Use Dataset.Create to
a dataset reference even if the dataset doesn't exist yet. Use [Dataset.Create] to
create a dataset from a reference:

if err := myDataset.Create(ctx, nil); err != nil {
// TODO: Handle error.
}

You can refer to tables with Dataset.Table. Like bigquery.Dataset, bigquery.Table is a reference
You can refer to tables with [Dataset.Table]. Like [Dataset], [Table] is a reference
to an object in BigQuery that may or may not exist.

table := myDataset.Table("my_table")

You can create, delete and update the metadata of tables with methods on Table.
You can create, delete and update the metadata of tables with methods on [Table].
For instance, you could create a temporary table with:

err = myDataset.Table("temp").Create(ctx, &bigquery.TableMetadata{
Expand All @@ -153,15 +153,15 @@ We'll see how to create a table with a schema in the next section.
# Schemas

There are two ways to construct schemas with this package.
You can build a schema by hand, like so:
You can build a schema by hand with the [Schema] struct, like so:

schema1 := bigquery.Schema{
{Name: "Name", Required: true, Type: bigquery.StringFieldType},
{Name: "Grades", Repeated: true, Type: bigquery.IntegerFieldType},
{Name: "Optional", Required: false, Type: bigquery.IntegerFieldType},
}

Or you can infer the schema from a struct:
Or you can infer the schema from a struct with the [InferSchema] method:

type student struct {
Name string
Expand All @@ -174,10 +174,10 @@ Or you can infer the schema from a struct:
}
// schema1 and schema2 are identical.

Struct inference supports tags like those of the encoding/json package, so you can
Struct inference supports tags like those of the [encoding/json] package, so you can
change names, ignore fields, or mark a field as nullable (non-required). Fields
declared as one of the Null types (NullInt64, NullFloat64, NullString, NullBool,
NullTimestamp, NullDate, NullTime, NullDateTime, and NullGeography) are
declared as one of the Null types ([NullInt64], [NullFloat64], [NullString], [NullBool],
[NullTimestamp], [NullDate], [NullTime], [NullDateTime], [NullGeography], and [NullJSON]) are
automatically inferred as nullable, so the "nullable" tag is only needed for []byte,
*big.Rat and pointer-to-struct fields.

Expand All @@ -193,16 +193,17 @@ automatically inferred as nullable, so the "nullable" tag is only needed for []b
}
// schema3 has required fields "full_name" and "Grade", and nullable BYTES field "Optional".

Having constructed a schema, you can create a table with it like so:
Having constructed a schema, you can create a table with it using the [Table.Create] method like so:

if err := table.Create(ctx, &bigquery.TableMetadata{Schema: schema1}); err != nil {
// TODO: Handle error.
}

# Copying

You can copy one or more tables to another table. Begin by constructing a Copier
describing the copy. Then set any desired copy options, and finally call Run to get a Job:
You can copy one or more tables to another table. Begin by constructing a [Copier]
describing the copy using the [Table.CopierFrom]. Then set any desired copy options,
and finally call [Copier.Run] to get a [Job]:

copier := myDataset.Table("dest").CopierFrom(myDataset.Table("src"))
copier.WriteDisposition = bigquery.WriteTruncate
Expand All @@ -211,21 +212,21 @@ describing the copy. Then set any desired copy options, and finally call Run to
// TODO: Handle error.
}

You can chain the call to Run if you don't want to set options:
You can chain the call to [Copier.Run] if you don't want to set options:

job, err = myDataset.Table("dest").CopierFrom(myDataset.Table("src")).Run(ctx)
if err != nil {
// TODO: Handle error.
}

You can wait for your job to complete:
You can wait for your job to complete with the [Job.Wait] method:

status, err := job.Wait(ctx)
if err != nil {
// TODO: Handle error.
}

Job.Wait polls with exponential backoff. You can also poll yourself, if you
[Job.Wait] polls with exponential backoff. You can also poll yourself, if you
wish:

for {
Expand All @@ -247,8 +248,9 @@ wish:
There are two ways to populate a table with this package: load the data from a Google Cloud Storage
object, or upload rows directly from your program.

For loading, first create a GCSReference, configuring it if desired. Then make a Loader, optionally configure
it as well, and call its Run method.
For loading, first create a [GCSReference] with the [NewGCSReference] method, configuring it if desired.
Then make a [Loader] from a table with the [Table.LoaderFrom] method with the reference,
optionally configure it as well, and call its [Loader.Run] method.

gcsRef := bigquery.NewGCSReference("gs://my-bucket/my-object")
gcsRef.AllowJaggedRows = true
Expand All @@ -257,8 +259,9 @@ it as well, and call its Run method.
job, err = loader.Run(ctx)
// Poll the job for completion if desired, as above.

To upload, first define a type that implements the ValueSaver interface, which has a single method named Save.
Then create an Inserter, and call its Put method with a slice of values.
To upload, first define a type that implements the [ValueSaver] interface, which has
a single method named Save. Then create an [Inserter], and call its [Inserter.Put]
method with a slice of values.

type Item struct {
Name string
Expand Down Expand Up @@ -286,7 +289,7 @@ Then create an Inserter, and call its Put method with a slice of values.
// TODO: Handle error.
}

You can also upload a struct that doesn't implement ValueSaver. Use the StructSaver type
You can also upload a struct that doesn't implement [ValueSaver]. Use the [StructSaver] type
to specify the schema and insert ID by hand:

type item struct {
Expand Down Expand Up @@ -324,13 +327,13 @@ Lastly, but not least, you can just supply the struct or struct pointer directly
}

BigQuery allows for higher throughput when omitting insertion IDs. To enable this,
specify the sentinel `NoDedupeID` value for the insertion ID when implementing a ValueSaver.
specify the sentinel [NoDedupeID] value for the insertion ID when implementing a [ValueSaver].

# Extracting

If you've been following so far, extracting data from a BigQuery table
into a Google Cloud Storage object will feel familiar. First create an
Extractor, then optionally configure it, and lastly call its Run method.
[Extractor], then optionally configure it, and lastly call its [Extractor.Run] method.

extractor := table.ExtractorTo(gcsRef)
extractor.DisableHeader = true
Expand All @@ -339,16 +342,16 @@ Extractor, then optionally configure it, and lastly call its Run method.

# Errors

Errors returned by this client are often of the type googleapi.Error: https://godoc.org/google.golang.org/api/googleapi#Error
Errors returned by this client are often of the type [googleapi.Error].
These errors can be introspected for more information by using [errors.As]
with the richer [googleapi.Error] type. For example:

These errors can be introspected for more information by using `xerrors.As` with the richer *googleapi.Error type. For example:

var e *googleapi.Error
if ok := xerrors.As(err, &e); ok {
if e.Code == 409 { ... }
}
var e *googleapi.Error
if ok := errors.As(err, &e); ok {
if e.Code == 409 { ... }
}

In some cases, your client may received unstructured googleapi.Error error responses. In such cases, it is likely that
In some cases, your client may received unstructured [googleapi.Error] error responses. In such cases, it is likely that
you have exceeded BigQuery request limits, documented at: https://cloud.google.com/bigquery/quotas
*/
package bigquery // import "cloud.google.com/go/bigquery"
Loading