-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Record leaf nodes support #2943
Labels
api: bigquery
Issues related to the BigQuery API.
priority: p1
Important issue which blocks shipping the next release. Will be fixed prior to next release.
type: bug
Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Comments
@daspecster Here is an excerpt of the code I used from google.cloud import bigquery
import os
import json
from datetime import datetime
def transform(item):
newItem = {"insertId": getUUID(), "json": item}
return newItem
def getTemplateSuffix():
utcDate = datetime.utcnow().strftime("%Y%m%d")
templateSuffix = "_" + utcDate
return templateSuffix
def uploadToBigQuery(data):
rowData = json.loads(data)
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "credentials.json"
# Instantiates a client
bigquery_client = bigquery.Client("logs")
dataset = bigquery_client.dataset("eventlogs")
table = dataset.table("events")
# Reload the table to get the schema.
table.reload()
rows = [rowData]
print(rows)
errors = table.insert_data(rows=rows, template_suffix=getTemplateSuffix())
if not errors:
print('Loaded 1 row into {}:{}'.format(dataset, table))
else:
print('Errors:')
print(errors)
data = '{"user_dim":{"user_id":"12345","user_properties":[{"key":"subdomain_id","value":{"string_value":"custom.zendesk.com"}}]},"event_dim":{"date":"20170116","category":"ALERT","name":"NUMBER_NOT_A_MOBILE","timestamp_micros":1484580530384000,"params":[{"key":"network","value":{"string_value":"Singapore"}},{"key":"country","value":{"string_value":"Singapore"}}]}}'
uploadToBigQuery(data) |
Is this a duplicate of #2951? (It seems like it is, but I am hesitant to close without being sure.) |
lukesneeringer
added
the
priority: p1
Important issue which blocks shipping the next release. Will be fixed prior to next release.
label
Feb 28, 2017
bjwatson
added
the
type: bug
Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
label
Feb 28, 2017
@tswast Can you confirm / deny the duplicateness? |
These two issues look like duplicates to me, too. |
Okay. I marked the other one as important, and am going to close this one for personal sanity / tracking. :-) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
api: bigquery
Issues related to the BigQuery API.
priority: p1
Important issue which blocks shipping the next release. Will be fixed prior to next release.
type: bug
Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Python version - 2.7.13
Google-cloud-bigquery - 0.22.1
I am trying to insert data using the big query python client library. I am getting an error that record leaf types are not supported. The schema contains nested records that are repeatable. Is there a way using the client to insert nested repeatable records ?
my schema is as follows:
The text was updated successfully, but these errors were encountered: