Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Hubspot Connector 0.1.11 Fails on Some Tables due to outputting empty strings as dates #5714

Closed
RKdev opened this issue Aug 27, 2021 · 5 comments · Fixed by #5798
Closed

Hubspot Connector 0.1.11 Fails on Some Tables due to outputting empty strings as dates #5714

RKdev opened this issue Aug 27, 2021 · 5 comments · Fixed by #5798
Assignees
@avida
Labels
area/connectors Connector related issues connectors/destination/bigquery connectors/destinations-warehouse connectors/source/hubspot connectors/sources-api type/bug Something isn't working
Milestone
Connectors, Septe...

Comments

@RKdev
Copy link

RKdev commented Aug 27, 2021 •
edited
Loading

Enviroment

  • Airbyte version: 0.29.12-alpha
  • OS Version / Instance: Debian GNU/Linux 10 (buster) on GCP, e2-medium
  • Deployment: Docker
  • Source Connector and version: Hubspot 0.1.11
  • Destination Connector and version: BigQuery 0.3.12
  • Severity: High - Cannot currently sync contacts, deals, companies
  • Step where error happened: Sync job

Current Behavior

The following tables are not syncable from Hubspot

Failed On SYNC

  • Deals
  • Companies
  • Contacts

Normalization then also fails

Expected Behavior

All tables should sync and go through basic normalization

Logs

If applicable, please upload the logs from the failing operation.
For sync jobs, you can download the full logs from the UI by going to the sync attempt page and
clicking the download logs button at the top right of the logs display window.

LOG 
2021-08-27 19:24:38 INFO () LineGobbler(voidCall):85 - 19:24:38 | 
2021-08-27 19:24:38 INFO () LineGobbler(voidCall):85 - 19:24:38 | Finished running 134 table models in 146.91s.
2021-08-27 19:24:40 INFO () LineGobbler(voidCall):85 - 
2021-08-27 19:24:40 INFO () LineGobbler(voidCall):85 - �[31mCompleted with 3 errors and 0 warnings:�[0m
2021-08-27 19:24:40 INFO () LineGobbler(voidCall):85 - 
2021-08-27 19:24:40 INFO () LineGobbler(voidCall):85 - �[33mDatabase Error in model deals_properties (models/generated/airbyte_tables/airbyte_hubspot_test/deals_properties.sql)�[0m
2021-08-27 19:24:40 INFO () LineGobbler(voidCall):85 -   Invalid timestamp: ''
2021-08-27 19:24:40 INFO () LineGobbler(voidCall):85 -   compiled SQL at ../build/run/airbyte_utils/models/generated/airbyte_tables/airbyte_hubspot_test/deals_properties.sql
2021-08-27 19:24:40 INFO () LineGobbler(voidCall):85 - 
2021-08-27 19:24:40 INFO () LineGobbler(voidCall):85 - �[33mDatabase Error in model companies_properties (models/generated/airbyte_tables/airbyte_hubspot_test/companies_properties.sql)�[0m
2021-08-27 19:24:40 INFO () LineGobbler(voidCall):85 -   Invalid timestamp: ''
2021-08-27 19:24:40 INFO () LineGobbler(voidCall):85 -   compiled SQL at ../build/run/airbyte_utils/models/generated/airbyte_tables/airbyte_hubspot_test/companies_properties.sql
2021-08-27 19:24:40 INFO () LineGobbler(voidCall):85 - 
2021-08-27 19:24:40 INFO () LineGobbler(voidCall):85 - �[33mDatabase Error in model contacts_properties (models/generated/airbyte_tables/airbyte_hubspot_test/contacts_properties.sql)�[0m
2021-08-27 19:24:40 INFO () LineGobbler(voidCall):85 -   Invalid timestamp: ''
2021-08-27 19:24:40 INFO () LineGobbler(voidCall):85 -   compiled SQL at ../build/run/airbyte_utils/models/generated/airbyte_tables/airbyte_hubspot_test/contacts_properties.sql
2021-08-27 19:24:40 INFO () LineGobbler(voidCall):85 - 
2021-08-27 19:24:40 INFO () LineGobbler(voidCall):85 - Done. PASS=131 WARN=0 ERROR=3 SKIP=0 TOTAL=134
2021-08-27 19:24:41 INFO () TemporalAttemptExecution(lambda$getWorkerThread$2):149 - Completing future exceptionally...
io.airbyte.workers.WorkerException: Normalization Failed.
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:69) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:36) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:146) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
	at java.lang.Thread.run(Thread.java:832) [?:?]
Caused by: io.airbyte.workers.WorkerException: Normalization Failed.
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:66) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
	... 3 more
	Suppressed: io.airbyte.workers.WorkerException: Normalization process wasn't successful
		at io.airbyte.workers.normalization.DefaultNormalizationRunner.close(DefaultNormalizationRunner.java:151) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
		at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:60) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
		at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:36) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
		at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:146) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
		at java.lang.Thread.run(Thread.java:832) [?:?]
2021-08-27 19:24:41 INFO () TemporalAttemptExecution(get):133 - Stopping cancellation check scheduling...
2021-08-27 19:24:41 WARN () POJOActivityTaskHandler$POJOActivityImplementation(execute):243 - Activity failure. ActivityId=9f2db00f-d115-3d21-a97b-54e2c9854484, activityType=Normalize, attempt=1
java.util.concurrent.ExecutionException: io.airbyte.workers.WorkerException: Normalization Failed.
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395) ~[?:?]
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2063) ~[?:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:131) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
	at io.airbyte.workers.temporal.SyncWorkflow$NormalizationActivityImpl.normalize(SyncWorkflow.java:291) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:564) ~[?:?]
	at io.temporal.internal.sync.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:277) ~[temporal-sdk-1.0.4.jar:?]
	at io.temporal.internal.sync.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:216) ~[temporal-sdk-1.0.4.jar:?]
	at io.temporal.internal.sync.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:181) ~[temporal-sdk-1.0.4.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:192) ~[temporal-sdk-1.0.4.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:154) ~[temporal-sdk-1.0.4.jar:?]
	at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:73) ~[temporal-sdk-1.0.4.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
	at java.lang.Thread.run(Thread.java:832) [?:?]
Caused by: io.airbyte.workers.WorkerException: Normalization Failed.
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:69) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:36) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:146) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
	... 1 more
Caused by: io.airbyte.workers.WorkerException: Normalization Failed.
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:66) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:36) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:146) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
	... 1 more
	Suppressed: io.airbyte.workers.WorkerException: Normalization process wasn't successful
		at io.airbyte.workers.normalization.DefaultNormalizationRunner.close(DefaultNormalizationRunner.java:151) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
		at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:60) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
		at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:36) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
		at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:146) ~[io.airbyte-airbyte-workers-0.29.12-alpha.jar:?]
		at java.lang.Thread.run(Thread.java:832) [?:?]

Steps to Reproduce

  1. wget https://raw.githubusercontent.com/airbytehq/airbyte/v0.29.12-alpha/{.env,docker-compose.yaml}
  2. docker-compose up
  3. Update Hubspot connector to 0.1.11
  4. select companies, contacts, deals, deal_pipelines (deal_pipelines seems to be fine)
  5. run sync

Are you willing to submit a PR?

Someday
@RKdev RKdev added the type/bug Something isn't working label Aug 27, 2021
@RKdev
Copy link
Author

RKdev commented Aug 29, 2021 •
edited
Loading

Still fails on Airbyte v0.29.13-Alpha, Hubspot Connector 0.1.11

Latest test restricted to contacts, companies, and deals

Using current connector...

2021-08-29 15:35:34 INFO () TemporalAttemptExecution(get):110 - Executing worker wrapper. Airbyte version: 0.29.13-alpha
2021-08-29 15:35:34 INFO () DefaultReplicationWorker(run):102 - start sync worker. job id: 6 attempt id: 0
2021-08-29 15:35:34 INFO () DefaultReplicationWorker(run):111 - configured sync modes: {null.companies=full_refresh - append, null.deals=full_refresh - append, null.contacts=full_refresh - append}
2021-08-29 15:35:34 INFO () DefaultAirbyteDestination(start):78 - Running destination...
2021-08-29 15:35:34 INFO () LineGobbler(voidCall):85 - Checking if airbyte/destination-bigquery:0.3.12 exists...
2021-08-29 15:35:34 INFO () DockerProcessFactory(create):146 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/6/0 --network host --log-driver none airbyte/destination-bigquery:0.3.12 write --config destination_config.json --catalog destination_catalog.json
2021-08-29 15:35:34 INFO () LineGobbler(voidCall):85 - airbyte/destination-bigquery:0.3.12 was found locally.
2021-08-29 15:35:34 INFO () LineGobbler(voidCall):85 - Checking if airbyte/source-hubspot:0.1.11 exists...
2021-08-29 15:35:34 INFO () LineGobbler(voidCall):85 - airbyte/source-hubspot:0.1.11 was found locally.
2021-08-29 15:35:34 INFO () DockerProcessFactory(create):146 - Preparing command: docker run --rm --init -i -v airbyte_workspace:/data -v /tmp/airbyte_local:/local -w /data/6/0 --network host --log-driver none airbyte/source-hubspot:0.1.11 read --config source_config.json --catalog source_catalog.json --state input_state.json
2021-08-29 15:35:34 INFO () DefaultReplicationWorker(lambda$getDestinationOutputRunnable$3):246 - Destination output thread started.
2021-08-29 15:35:34 INFO () DefaultReplicationWorker(run):139 - Waiting for source thread to join.
2021-08-29 15:35:34 INFO () DefaultReplicationWorker(lambda$getReplicationRunnable$2):210 - Replication thread started.
2021-08-29 15:35:36 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Starting syncing SourceHubspot
2021-08-29 15:35:36 INFO () DefaultAirbyteStreamFactory(internalLog):110 - Syncing companies stream
2021-08-29 15:35:37 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-08-29 15:35:37 �[32mINFO�[m i.a.i.d.b.BigQueryDestination(main):305 - {} - starting destination: class io.airbyte.integrations.destination.bigquery.BigQueryDestination
2021-08-29 15:35:37 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-08-29 15:35:37 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):96 - {} - Running integration: io.airbyte.integrations.destination.bigquery.BigQueryDestination
2021-08-29 15:35:37 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-08-29 15:35:37 �[32mINFO�[m i.a.i.b.IntegrationCliParser(parseOptions):135 - {} - integration args: {catalog=destination_catalog.json, write=null, config=destination_config.json}
2021-08-29 15:35:37 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-08-29 15:35:37 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):100 - {} - Command: WRITE
2021-08-29 15:35:37 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-08-29 15:35:37 �[32mINFO�[m i.a.i.b.IntegrationRunner(run):101 - {} - Integration config: IntegrationConfig{command=WRITE, configPath='destination_config.json', catalogPath='destination_catalog.json', statePath='null'}
2021-08-29 15:35:37 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-08-29 15:35:37 �[33mWARN�[m c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword examples - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2021-08-29 15:35:37 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-08-29 15:35:37 �[33mWARN�[m c.n.s.JsonMetaSchema(newValidator):338 - {} - Unknown keyword airbyte_secret - you should define your own Meta Schema. If the keyword is irrelevant for validation, just use a NonValidationKeyword
2021-08-29 15:35:40 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-08-29 15:35:40 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):85 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=airbyte_hubspot_test, tableId=_airbyte_tmp_xmk_companies}} created successfully
2021-08-29 15:35:40 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-08-29 15:35:40 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):85 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=airbyte_hubspot_test, tableId=_airbyte_tmp_anl_contacts}} created successfully
2021-08-29 15:35:41 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-08-29 15:35:41 �[32mINFO�[m i.a.i.d.b.BigQueryUtils(createTable):85 - {} - Table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=airbyte_hubspot_test, tableId=_airbyte_tmp_xbk_deals}} created successfully

JSON schema validation failed...

2021-08-29 15:41:00 INFO () DefaultReplicationWorker(run):141 - Source thread complete.
2021-08-29 15:41:00 INFO () DefaultReplicationWorker(run):142 - Waiting for destination thread to join.
2021-08-29 15:41:00 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-08-29 15:41:00 �[32mINFO�[m i.a.i.b.FailureTrackingAirbyteMessageConsumer(close):80 - {} - Airbyte message consumer: succeeded.
2021-08-29 15:41:00 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-08-29 15:41:00 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(close):133 - {} - Started closing all connections
2021-08-29 15:41:02 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-08-29 15:41:02 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(close):147 - {} - Waiting for jobs to be finished/closed
2021-08-29 15:41:14 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-08-29 15:41:14 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(close):164 - {} - Migration finished with no explicit errors. Copying data from tmp tables to permanent
2021-08-29 15:41:17 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-08-29 15:41:17 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):200 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=airbyte_hubspot_test, tableId=_airbyte_tmp_anl_contacts}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=airbyte_hubspot_test, tableId=_airbyte_raw_contacts}}
2021-08-29 15:41:19 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-08-29 15:41:19 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):200 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=airbyte_hubspot_test, tableId=_airbyte_tmp_xmk_companies}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=airbyte_hubspot_test, tableId=_airbyte_raw_companies}}
2021-08-29 15:41:21 INFO () DefaultAirbyteStreamFactory(lambda$create$0):73 - 2021-08-29 15:41:21 �[32mINFO�[m i.a.i.d.b.BigQueryRecordConsumer(copyTable):200 - {} - successfully copied tmp table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=airbyte_hubspot_test, tableId=_airbyte_tmp_xbk_deals}} to final table: GenericData{classInfo=[datasetId, projectId, tableId], {datasetId=airbyte_hubspot_test, tableId=_airbyte_raw_deals}}
2021-08-29 15:41:21 INFO () JsonSchemaValidator(test):76 - JSON schema validation failed. 
errors: $: null found, object expected
2021-08-29 15:41:21 ERROR () DefaultAirbyteStreamFactory(lambda$create$1):83 - Validation failed: null

Configuration paths warning...

2021-08-29 15:41:45 INFO () LineGobbler(voidCall):85 - [�[33mWARNING�[0m]: Configuration paths exist in your dbt_project.yml file which do not apply to any resources.
2021-08-29 15:41:45 INFO () LineGobbler(voidCall):85 - There are 1 unused configuration paths:
2021-08-29 15:41:45 INFO () LineGobbler(voidCall):85 - - models.airbyte_utils.generated.airbyte_views
2021-08-29 15:41:45 INFO () LineGobbler(voidCall):85 - 
2021-08-29 15:41:48 INFO () LineGobbler(voidCall):85 - Found 524 models, 0 tests, 0 snapshots, 0 analyses, 405 macros, 0 operations, 0 seed files, 3 sources, 0 exposures
2021-08-29 15:41:48 INFO () LineGobbler(voidCall):85 -

invalid timestamp errors still...

2021-08-29 15:44:37 INFO () LineGobbler(voidCall):85 - 
2021-08-29 15:44:37 INFO () LineGobbler(voidCall):85 - �[31mCompleted with 3 errors and 0 warnings:�[0m
2021-08-29 15:44:37 INFO () LineGobbler(voidCall):85 - 
2021-08-29 15:44:37 INFO () LineGobbler(voidCall):85 - �[33mDatabase Error in model companies_properties (models/generated/airbyte_tables/airbyte_hubspot_test/companies_properties.sql)�[0m
2021-08-29 15:44:37 INFO () LineGobbler(voidCall):85 -   Invalid timestamp: ''
2021-08-29 15:44:37 INFO () LineGobbler(voidCall):85 -   compiled SQL at ../build/run/airbyte_utils/models/generated/airbyte_tables/airbyte_hubspot_test/companies_properties.sql
2021-08-29 15:44:37 INFO () LineGobbler(voidCall):85 - 
2021-08-29 15:44:37 INFO () LineGobbler(voidCall):85 - �[33mDatabase Error in model deals_properties (models/generated/airbyte_tables/airbyte_hubspot_test/deals_properties.sql)�[0m
2021-08-29 15:44:37 INFO () LineGobbler(voidCall):85 -   Invalid timestamp: ''
2021-08-29 15:44:37 INFO () LineGobbler(voidCall):85 -   compiled SQL at ../build/run/airbyte_utils/models/generated/airbyte_tables/airbyte_hubspot_test/deals_properties.sql
2021-08-29 15:44:37 INFO () LineGobbler(voidCall):85 - 
2021-08-29 15:44:37 INFO () LineGobbler(voidCall):85 - �[33mDatabase Error in model contacts_properties (models/generated/airbyte_tables/airbyte_hubspot_test/contacts_properties.sql)�[0m
2021-08-29 15:44:37 INFO () LineGobbler(voidCall):85 -   Invalid timestamp: ''
2021-08-29 15:44:37 INFO () LineGobbler(voidCall):85 -   compiled SQL at ../build/run/airbyte_utils/models/generated/airbyte_tables/airbyte_hubspot_test/contacts_properties.sql
2021-08-29 15:44:37 INFO () LineGobbler(voidCall):85 - 
2021-08-29 15:44:37 INFO () LineGobbler(voidCall):85 - Done. PASS=128 WARN=0 ERROR=3 SKIP=0 TOTAL=131
2021-08-29 15:44:37 INFO () TemporalAttemptExecution(lambda$getWorkerThread$2):149 - Completing future exceptionally...
io.airbyte.workers.WorkerException: Normalization Failed.
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:69) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:36) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:146) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
	at java.lang.Thread.run(Thread.java:832) [?:?]
Caused by: io.airbyte.workers.WorkerException: Normalization Failed.
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:66) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
	... 3 more
	Suppressed: io.airbyte.workers.WorkerException: Normalization process wasn't successful
		at io.airbyte.workers.normalization.DefaultNormalizationRunner.close(DefaultNormalizationRunner.java:151) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
		at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:60) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
		at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:36) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
		at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:146) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
		at java.lang.Thread.run(Thread.java:832) [?:?]
2021-08-29 15:44:37 INFO () TemporalAttemptExecution(get):133 - Stopping cancellation check scheduling...
2021-08-29 15:44:37 WARN () POJOActivityTaskHandler$POJOActivityImplementation(execute):243 - Activity failure. ActivityId=9a800fcf-fb60-36d0-ac73-9fa65ad8ce47, activityType=Normalize, attempt=1
java.util.concurrent.ExecutionException: io.airbyte.workers.WorkerException: Normalization Failed.
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395) ~[?:?]
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2063) ~[?:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.get(TemporalAttemptExecution.java:131) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
	at io.airbyte.workers.temporal.SyncWorkflow$NormalizationActivityImpl.normalize(SyncWorkflow.java:291) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:564) ~[?:?]
	at io.temporal.internal.sync.POJOActivityTaskHandler$POJOActivityInboundCallsInterceptor.execute(POJOActivityTaskHandler.java:277) ~[temporal-sdk-1.0.4.jar:?]
	at io.temporal.internal.sync.POJOActivityTaskHandler$POJOActivityImplementation.execute(POJOActivityTaskHandler.java:216) ~[temporal-sdk-1.0.4.jar:?]
	at io.temporal.internal.sync.POJOActivityTaskHandler.handle(POJOActivityTaskHandler.java:181) ~[temporal-sdk-1.0.4.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:192) ~[temporal-sdk-1.0.4.jar:?]
	at io.temporal.internal.worker.ActivityWorker$TaskHandlerImpl.handle(ActivityWorker.java:154) ~[temporal-sdk-1.0.4.jar:?]
	at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:73) ~[temporal-sdk-1.0.4.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
	at java.lang.Thread.run(Thread.java:832) [?:?]
Caused by: io.airbyte.workers.WorkerException: Normalization Failed.
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:69) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:36) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:146) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
	... 1 more
Caused by: io.airbyte.workers.WorkerException: Normalization Failed.
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:66) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
	at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:36) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
	at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:146) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
	... 1 more
	Suppressed: io.airbyte.workers.WorkerException: Normalization process wasn't successful
		at io.airbyte.workers.normalization.DefaultNormalizationRunner.close(DefaultNormalizationRunner.java:151) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
		at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:60) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
		at io.airbyte.workers.DefaultNormalizationWorker.run(DefaultNormalizationWorker.java:36) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
		at io.airbyte.workers.temporal.TemporalAttemptExecution.lambda$getWorkerThread$2(TemporalAttemptExecution.java:146) ~[io.airbyte-airbyte-workers-0.29.13-alpha.jar:?]
		at java.lang.Thread.run(Thread.java:832) [?:?]

@RKdev
Copy link
Author

RKdev commented Aug 29, 2021

@marcosmarxm
Is it possibly an issue with Big Query rather than Hubspot?

@RKdev
Copy link
Author

RKdev commented Aug 29, 2021 •
edited
Loading

I have performed the step "Update latest source schema." I have also done completely fresh install with same issue. If there's anything else I can provide or any "grunt work" I can do to help please let me know - I'm happy to help but need guidance still.

@sherifnada sherifnada added the area/connectors Connector related issues label Aug 31, 2021
@sherifnada sherifnada added this to the Connectors, September 3rd milestone Aug 31, 2021
@sherifnada sherifnada changed the title Hubspot Connector 0.1.11 Fails on Some Tables Hubspot Connector 0.1.11 Fails on Some Tables due to outputting empty strings as dates Aug 31, 2021
@sherifnada
Copy link
Contributor

the issue seems to be that the timestamp fields are output as empty strings. We'll fix this soon @RKdev

@RKdev
Copy link
Author

RKdev commented Aug 31, 2021

Thanks @sherifnada - I know some work on this issue has been done in this ticket as well #5555

Also this may or may not be relevant
#5688

I would love to work on this but I don't enough experience developing on Airbyte yet. But if there's anything useful I can do such as providing logs or sanitized records from our actual data please let me know

@avida avida self-assigned this Aug 31, 2021
@avida avida mentioned this issue Sep 2, 2021
Merged
36 tasks
@avida avida linked a pull request Sep 3, 2021 that will close this issue
🐛 Fix hubspot datetime empty string #5798
Merged
36 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@avida avida

Labels
area/connectors Connector related issues connectors/destination/bigquery connectors/destinations-warehouse connectors/source/hubspot connectors/sources-api type/bug Something isn't working
Projects
None yet
Milestone
Connectors, September 17th 2021
Development

Successfully merging a pull request may close this issue.

🐛 Fix hubspot datetime empty string airbytehq/airbyte
5 participants
@avida @RKdev @sherifnada @karinakuz @igrankova