-
Notifications
You must be signed in to change notification settings - Fork 9.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
r/aws_kinesis_firehose_delivery_stream: Add OpenSearch Serverless support #33101
r/aws_kinesis_firehose_delivery_stream: Add OpenSearch Serverless support #33101
Conversation
AWS [added][1] Amazon Kinesis Data Firehose stream delivery to OpenSearch Serverless. In order to support this with terraform we now add a new `opensearchserverless_configuration` block that we can map to a desired `AmazonOpenSearchServerlessDestinationConfiguration` (see [here][2]). [1]: https://aws.amazon.com/about-aws/whats-new/2022/11/amazon-kinesis-data-firehose-support-data-stream-delivery-opensearch-serverless [2]: https://docs.aws.amazon.com/firehose/latest/APIReference/API_AmazonOpenSearchServerlessDestinationConfiguration.html
We adopted the Kineses Firehost Delivery Stream resource in order to also stream into OpenSearch Serverless. Adapt the documentation to describe the new section.
Community NoteVoting for Prioritization
For Submitters
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Welcome @stschulte 👋
It looks like this is your first Pull Request submission to the Terraform AWS Provider! If you haven’t already done so please make sure you have checked out our CONTRIBUTOR guide and FAQ to make sure your contribution is adhering to best practice and has all the necessary elements in place for a successful approval.
Also take a look at our FAQ which details how we prioritize Pull Requests for inclusion.
Thanks again, and welcome to the community! 😃
I also have two general questions:
|
Hey @stschulte 👋 Thank you for your contribution! You're correct in that these are tested via acceptance tests only; not unit tests. Regarding examples, you're again correct; we don't always include an entire example in the documentation examples, often omitting details for brevity. |
Hey @justinretzolk are the non-existing unit tests by design? I wanted to implement some unit tests for the correct conversion of AWS responses to the slightly different terraform structure that delivery_stream uses. But Since this is the first go package I am looking into is there some guidance on how tests are particulary structured inside the provider in regards to acceptance tests vs unit tests? Or do they follow some best practice you can point me to? |
% ACCTEST_TIMEOUT=720m make testacc TESTARGS='-run=TestAccFirehoseDeliveryStream_' PKG=firehose ACCTEST_PARALLELISM=2 ==> Checking that code complies with gofmt requirements... TF_ACC=1 go test ./internal/service/firehose/... -v -count 1 -parallel 2 -run=TestAccFirehoseDeliveryStream_ -timeout 720m === RUN TestAccFirehoseDeliveryStream_basic === PAUSE TestAccFirehoseDeliveryStream_basic === RUN TestAccFirehoseDeliveryStream_disappears === PAUSE TestAccFirehoseDeliveryStream_disappears === RUN TestAccFirehoseDeliveryStream_tags === PAUSE TestAccFirehoseDeliveryStream_tags === RUN TestAccFirehoseDeliveryStream_s3WithCloudWatchLogging === PAUSE TestAccFirehoseDeliveryStream_s3WithCloudWatchLogging === RUN TestAccFirehoseDeliveryStream_extendedS3basic === PAUSE TestAccFirehoseDeliveryStream_extendedS3basic === RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversion_enabled === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversion_enabled === RUN TestAccFirehoseDeliveryStream_ExtendedS3_externalUpdate === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3_externalUpdate === RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionDeserializer_update === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionDeserializer_update === RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionHiveJSONSerDe_empty === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionHiveJSONSerDe_empty === RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOpenXJSONSerDe_empty === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOpenXJSONSerDe_empty === RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOrcSerDe_empty === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOrcSerDe_empty === RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionParquetSerDe_empty === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionParquetSerDe_empty === RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionSerializer_update === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionSerializer_update === RUN TestAccFirehoseDeliveryStream_ExtendedS3_errorOutputPrefix === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3_errorOutputPrefix === RUN TestAccFirehoseDeliveryStream_ExtendedS3_S3BackupConfiguration_ErrorOutputPrefix === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3_S3BackupConfiguration_ErrorOutputPrefix === RUN TestAccFirehoseDeliveryStream_ExtendedS3Processing_empty === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3Processing_empty === RUN TestAccFirehoseDeliveryStream_extendedS3KMSKeyARN === PAUSE TestAccFirehoseDeliveryStream_extendedS3KMSKeyARN === RUN TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioning === PAUSE TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioning === RUN TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioningUpdate === PAUSE TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioningUpdate === RUN TestAccFirehoseDeliveryStream_extendedS3Updates === PAUSE TestAccFirehoseDeliveryStream_extendedS3Updates === RUN TestAccFirehoseDeliveryStream_ExtendedS3_kinesisStreamSource === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3_kinesisStreamSource === RUN TestAccFirehoseDeliveryStream_redshiftUpdates === PAUSE TestAccFirehoseDeliveryStream_redshiftUpdates === RUN TestAccFirehoseDeliveryStream_splunkUpdates === PAUSE TestAccFirehoseDeliveryStream_splunkUpdates === RUN TestAccFirehoseDeliveryStream_Splunk_ErrorOutputPrefix === PAUSE TestAccFirehoseDeliveryStream_Splunk_ErrorOutputPrefix === RUN TestAccFirehoseDeliveryStream_httpEndpoint === PAUSE TestAccFirehoseDeliveryStream_httpEndpoint === RUN TestAccFirehoseDeliveryStream_HTTPEndpoint_ErrorOutputPrefix === PAUSE TestAccFirehoseDeliveryStream_HTTPEndpoint_ErrorOutputPrefix === RUN TestAccFirehoseDeliveryStream_HTTPEndpoint_retryDuration === PAUSE TestAccFirehoseDeliveryStream_HTTPEndpoint_retryDuration === RUN TestAccFirehoseDeliveryStream_elasticSearchUpdates === PAUSE TestAccFirehoseDeliveryStream_elasticSearchUpdates === RUN TestAccFirehoseDeliveryStream_elasticSearchEndpointUpdates === PAUSE TestAccFirehoseDeliveryStream_elasticSearchEndpointUpdates === RUN TestAccFirehoseDeliveryStream_elasticSearchWithVPCUpdates === PAUSE TestAccFirehoseDeliveryStream_elasticSearchWithVPCUpdates === RUN TestAccFirehoseDeliveryStream_Elasticsearch_ErrorOutputPrefix === PAUSE TestAccFirehoseDeliveryStream_Elasticsearch_ErrorOutputPrefix === RUN TestAccFirehoseDeliveryStream_openSearchUpdates === PAUSE TestAccFirehoseDeliveryStream_openSearchUpdates === RUN TestAccFirehoseDeliveryStream_openSearchEndpointUpdates === PAUSE TestAccFirehoseDeliveryStream_openSearchEndpointUpdates === RUN TestAccFirehoseDeliveryStream_openSearchWithVPCUpdates === PAUSE TestAccFirehoseDeliveryStream_openSearchWithVPCUpdates === RUN TestAccFirehoseDeliveryStream_Opensearch_ErrorOutputPrefix === PAUSE TestAccFirehoseDeliveryStream_Opensearch_ErrorOutputPrefix === RUN TestAccFirehoseDeliveryStream_missingProcessing === PAUSE TestAccFirehoseDeliveryStream_missingProcessing === CONT TestAccFirehoseDeliveryStream_basic === CONT TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioningUpdate --- PASS: TestAccFirehoseDeliveryStream_basic (174.70s) === CONT TestAccFirehoseDeliveryStream_elasticSearchUpdates --- PASS: TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioningUpdate (460.34s) === CONT TestAccFirehoseDeliveryStream_missingProcessing --- PASS: TestAccFirehoseDeliveryStream_missingProcessing (143.27s) === CONT TestAccFirehoseDeliveryStream_Opensearch_ErrorOutputPrefix --- PASS: TestAccFirehoseDeliveryStream_elasticSearchUpdates (1700.64s) === CONT TestAccFirehoseDeliveryStream_openSearchWithVPCUpdates --- PASS: TestAccFirehoseDeliveryStream_Opensearch_ErrorOutputPrefix (2030.91s) === CONT TestAccFirehoseDeliveryStream_openSearchEndpointUpdates --- PASS: TestAccFirehoseDeliveryStream_openSearchEndpointUpdates (2030.85s) === CONT TestAccFirehoseDeliveryStream_openSearchUpdates --- PASS: TestAccFirehoseDeliveryStream_openSearchWithVPCUpdates (3820.77s) === CONT TestAccFirehoseDeliveryStream_Elasticsearch_ErrorOutputPrefix --- PASS: TestAccFirehoseDeliveryStream_openSearchUpdates (2051.62s) === CONT TestAccFirehoseDeliveryStream_elasticSearchWithVPCUpdates --- PASS: TestAccFirehoseDeliveryStream_Elasticsearch_ErrorOutputPrefix (1804.43s) === CONT TestAccFirehoseDeliveryStream_elasticSearchEndpointUpdates --- PASS: TestAccFirehoseDeliveryStream_elasticSearchWithVPCUpdates (2610.03s) === CONT TestAccFirehoseDeliveryStream_HTTPEndpoint_ErrorOutputPrefix --- PASS: TestAccFirehoseDeliveryStream_elasticSearchEndpointUpdates (1908.30s) === CONT TestAccFirehoseDeliveryStream_HTTPEndpoint_retryDuration --- PASS: TestAccFirehoseDeliveryStream_HTTPEndpoint_ErrorOutputPrefix (169.34s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOpenXJSONSerDe_empty --- PASS: TestAccFirehoseDeliveryStream_HTTPEndpoint_retryDuration (156.73s) === CONT TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioning --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOpenXJSONSerDe_empty (172.49s) === CONT TestAccFirehoseDeliveryStream_Splunk_ErrorOutputPrefix --- PASS: TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioning (203.51s) === CONT TestAccFirehoseDeliveryStream_httpEndpoint --- PASS: TestAccFirehoseDeliveryStream_Splunk_ErrorOutputPrefix (225.47s) === CONT TestAccFirehoseDeliveryStream_redshiftUpdates --- PASS: TestAccFirehoseDeliveryStream_httpEndpoint (203.72s) === CONT TestAccFirehoseDeliveryStream_splunkUpdates --- PASS: TestAccFirehoseDeliveryStream_splunkUpdates (201.89s) === CONT TestAccFirehoseDeliveryStream_extendedS3KMSKeyARN --- PASS: TestAccFirehoseDeliveryStream_extendedS3KMSKeyARN (110.55s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3_errorOutputPrefix --- PASS: TestAccFirehoseDeliveryStream_redshiftUpdates (398.05s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionSerializer_update --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_errorOutputPrefix (166.30s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3Processing_empty --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionSerializer_update (164.88s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3_S3BackupConfiguration_ErrorOutputPrefix --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3Processing_empty (136.18s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversion_enabled --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_S3BackupConfiguration_ErrorOutputPrefix (169.05s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionHiveJSONSerDe_empty --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionHiveJSONSerDe_empty (168.12s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionDeserializer_update --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversion_enabled (212.98s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3_externalUpdate --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_externalUpdate (176.92s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3_kinesisStreamSource --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionDeserializer_update (204.10s) === CONT TestAccFirehoseDeliveryStream_extendedS3Updates --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_kinesisStreamSource (86.58s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionParquetSerDe_empty --- PASS: TestAccFirehoseDeliveryStream_extendedS3Updates (142.89s) === CONT TestAccFirehoseDeliveryStream_s3WithCloudWatchLogging --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionParquetSerDe_empty (97.95s) === CONT TestAccFirehoseDeliveryStream_extendedS3basic --- PASS: TestAccFirehoseDeliveryStream_extendedS3basic (94.03s) === CONT TestAccFirehoseDeliveryStream_disappears --- PASS: TestAccFirehoseDeliveryStream_s3WithCloudWatchLogging (118.31s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOrcSerDe_empty --- PASS: TestAccFirehoseDeliveryStream_disappears (125.64s) === CONT TestAccFirehoseDeliveryStream_tags --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOrcSerDe_empty (139.03s) --- PASS: TestAccFirehoseDeliveryStream_tags (155.50s) PASS ok github.com/hashicorp/terraform-provider-aws/internal/service/firehose 11542.446s
% ACCTEST_TIMEOUT=720m make testacc TESTARGS='-run=TestAccFirehoseDeliveryStream_' PKG=firehose ACCTEST_PARALLELISM=3 ==> Checking that code complies with gofmt requirements... TF_ACC=1 go test ./internal/service/firehose/... -v -count 1 -parallel 3 -run=TestAccFirehoseDeliveryStream_ -timeout 720m === RUN TestAccFirehoseDeliveryStream_basic === PAUSE TestAccFirehoseDeliveryStream_basic === RUN TestAccFirehoseDeliveryStream_disappears === PAUSE TestAccFirehoseDeliveryStream_disappears === RUN TestAccFirehoseDeliveryStream_tags === PAUSE TestAccFirehoseDeliveryStream_tags === RUN TestAccFirehoseDeliveryStream_s3WithCloudWatchLogging === PAUSE TestAccFirehoseDeliveryStream_s3WithCloudWatchLogging === RUN TestAccFirehoseDeliveryStream_extendedS3basic === PAUSE TestAccFirehoseDeliveryStream_extendedS3basic === RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversion_enabled === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversion_enabled === RUN TestAccFirehoseDeliveryStream_ExtendedS3_externalUpdate === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3_externalUpdate === RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionDeserializer_update === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionDeserializer_update === RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionHiveJSONSerDe_empty === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionHiveJSONSerDe_empty === RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOpenXJSONSerDe_empty === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOpenXJSONSerDe_empty === RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOrcSerDe_empty === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOrcSerDe_empty === RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionParquetSerDe_empty === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionParquetSerDe_empty === RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionSerializer_update === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionSerializer_update === RUN TestAccFirehoseDeliveryStream_ExtendedS3_errorOutputPrefix === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3_errorOutputPrefix === RUN TestAccFirehoseDeliveryStream_ExtendedS3_S3BackupConfiguration_ErrorOutputPrefix === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3_S3BackupConfiguration_ErrorOutputPrefix === RUN TestAccFirehoseDeliveryStream_ExtendedS3Processing_empty === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3Processing_empty === RUN TestAccFirehoseDeliveryStream_extendedS3KMSKeyARN === PAUSE TestAccFirehoseDeliveryStream_extendedS3KMSKeyARN === RUN TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioning === PAUSE TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioning === RUN TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioningUpdate === PAUSE TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioningUpdate === RUN TestAccFirehoseDeliveryStream_extendedS3Updates === PAUSE TestAccFirehoseDeliveryStream_extendedS3Updates === RUN TestAccFirehoseDeliveryStream_ExtendedS3_kinesisStreamSource === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3_kinesisStreamSource === RUN TestAccFirehoseDeliveryStream_redshiftUpdates === PAUSE TestAccFirehoseDeliveryStream_redshiftUpdates === RUN TestAccFirehoseDeliveryStream_splunkUpdates === PAUSE TestAccFirehoseDeliveryStream_splunkUpdates === RUN TestAccFirehoseDeliveryStream_Splunk_ErrorOutputPrefix === PAUSE TestAccFirehoseDeliveryStream_Splunk_ErrorOutputPrefix === RUN TestAccFirehoseDeliveryStream_httpEndpoint === PAUSE TestAccFirehoseDeliveryStream_httpEndpoint === RUN TestAccFirehoseDeliveryStream_HTTPEndpoint_ErrorOutputPrefix === PAUSE TestAccFirehoseDeliveryStream_HTTPEndpoint_ErrorOutputPrefix === RUN TestAccFirehoseDeliveryStream_HTTPEndpoint_retryDuration === PAUSE TestAccFirehoseDeliveryStream_HTTPEndpoint_retryDuration === RUN TestAccFirehoseDeliveryStream_elasticSearchUpdates === PAUSE TestAccFirehoseDeliveryStream_elasticSearchUpdates === RUN TestAccFirehoseDeliveryStream_elasticSearchEndpointUpdates === PAUSE TestAccFirehoseDeliveryStream_elasticSearchEndpointUpdates === RUN TestAccFirehoseDeliveryStream_elasticSearchWithVPCUpdates === PAUSE TestAccFirehoseDeliveryStream_elasticSearchWithVPCUpdates === RUN TestAccFirehoseDeliveryStream_Elasticsearch_ErrorOutputPrefix === PAUSE TestAccFirehoseDeliveryStream_Elasticsearch_ErrorOutputPrefix === RUN TestAccFirehoseDeliveryStream_openSearchUpdates === PAUSE TestAccFirehoseDeliveryStream_openSearchUpdates === RUN TestAccFirehoseDeliveryStream_openSearchEndpointUpdates === PAUSE TestAccFirehoseDeliveryStream_openSearchEndpointUpdates === RUN TestAccFirehoseDeliveryStream_openSearchWithVPCUpdates === PAUSE TestAccFirehoseDeliveryStream_openSearchWithVPCUpdates === RUN TestAccFirehoseDeliveryStream_Opensearch_ErrorOutputPrefix === PAUSE TestAccFirehoseDeliveryStream_Opensearch_ErrorOutputPrefix === RUN TestAccFirehoseDeliveryStream_missingProcessing === PAUSE TestAccFirehoseDeliveryStream_missingProcessing === CONT TestAccFirehoseDeliveryStream_basic === CONT TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioningUpdate === CONT TestAccFirehoseDeliveryStream_elasticSearchUpdates --- PASS: TestAccFirehoseDeliveryStream_basic (138.57s) === CONT TestAccFirehoseDeliveryStream_openSearchEndpointUpdates --- PASS: TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioningUpdate (233.70s) === CONT TestAccFirehoseDeliveryStream_missingProcessing --- PASS: TestAccFirehoseDeliveryStream_missingProcessing (101.86s) === CONT TestAccFirehoseDeliveryStream_Opensearch_ErrorOutputPrefix --- PASS: TestAccFirehoseDeliveryStream_elasticSearchUpdates (1604.96s) === CONT TestAccFirehoseDeliveryStream_openSearchWithVPCUpdates --- PASS: TestAccFirehoseDeliveryStream_openSearchEndpointUpdates (1880.22s) === CONT TestAccFirehoseDeliveryStream_Splunk_ErrorOutputPrefix --- PASS: TestAccFirehoseDeliveryStream_Splunk_ErrorOutputPrefix (145.81s) === CONT TestAccFirehoseDeliveryStream_openSearchUpdates --- PASS: TestAccFirehoseDeliveryStream_Opensearch_ErrorOutputPrefix (1947.13s) === CONT TestAccFirehoseDeliveryStream_HTTPEndpoint_retryDuration --- PASS: TestAccFirehoseDeliveryStream_HTTPEndpoint_retryDuration (130.87s) === CONT TestAccFirehoseDeliveryStream_Elasticsearch_ErrorOutputPrefix --- PASS: TestAccFirehoseDeliveryStream_Elasticsearch_ErrorOutputPrefix (1533.31s) === CONT TestAccFirehoseDeliveryStream_HTTPEndpoint_ErrorOutputPrefix --- PASS: TestAccFirehoseDeliveryStream_openSearchUpdates (1907.01s) === CONT TestAccFirehoseDeliveryStream_elasticSearchWithVPCUpdates --- PASS: TestAccFirehoseDeliveryStream_HTTPEndpoint_ErrorOutputPrefix (144.20s) === CONT TestAccFirehoseDeliveryStream_elasticSearchEndpointUpdates --- PASS: TestAccFirehoseDeliveryStream_openSearchWithVPCUpdates (2975.71s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOpenXJSONSerDe_empty --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOpenXJSONSerDe_empty (79.46s) === CONT TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioning --- PASS: TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioning (120.08s) === CONT TestAccFirehoseDeliveryStream_httpEndpoint --- PASS: TestAccFirehoseDeliveryStream_httpEndpoint (138.26s) === CONT TestAccFirehoseDeliveryStream_extendedS3KMSKeyARN --- PASS: TestAccFirehoseDeliveryStream_extendedS3KMSKeyARN (120.22s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3Processing_empty --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3Processing_empty (123.52s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversion_enabled --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversion_enabled (146.58s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3_S3BackupConfiguration_ErrorOutputPrefix --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_S3BackupConfiguration_ErrorOutputPrefix (185.40s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionHiveJSONSerDe_empty --- PASS: TestAccFirehoseDeliveryStream_elasticSearchEndpointUpdates (1477.62s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3_errorOutputPrefix --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionHiveJSONSerDe_empty (103.58s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionDeserializer_update --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionDeserializer_update (127.91s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionSerializer_update --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_errorOutputPrefix (179.48s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionParquetSerDe_empty --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionSerializer_update (127.44s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOrcSerDe_empty --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionParquetSerDe_empty (106.58s) === CONT TestAccFirehoseDeliveryStream_redshiftUpdates --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOrcSerDe_empty (96.27s) === CONT TestAccFirehoseDeliveryStream_splunkUpdates --- PASS: TestAccFirehoseDeliveryStream_splunkUpdates (127.85s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3_kinesisStreamSource --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_kinesisStreamSource (81.78s) === CONT TestAccFirehoseDeliveryStream_tags --- PASS: TestAccFirehoseDeliveryStream_redshiftUpdates (404.97s) === CONT TestAccFirehoseDeliveryStream_extendedS3basic --- PASS: TestAccFirehoseDeliveryStream_tags (140.61s) === CONT TestAccFirehoseDeliveryStream_ExtendedS3_externalUpdate --- PASS: TestAccFirehoseDeliveryStream_extendedS3basic (109.33s) === CONT TestAccFirehoseDeliveryStream_extendedS3Updates --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_externalUpdate (121.18s) === CONT TestAccFirehoseDeliveryStream_disappears --- PASS: TestAccFirehoseDeliveryStream_extendedS3Updates (129.48s) === CONT TestAccFirehoseDeliveryStream_s3WithCloudWatchLogging --- PASS: TestAccFirehoseDeliveryStream_disappears (95.62s) --- PASS: TestAccFirehoseDeliveryStream_elasticSearchWithVPCUpdates (2480.50s) --- PASS: TestAccFirehoseDeliveryStream_s3WithCloudWatchLogging (110.48s) PASS ok github.com/hashicorp/terraform-provider-aws/internal/service/firehose 6614.206s
This reverts commit d7ab30e.
% make testacc TESTARGS='-run=TestAccFirehoseDeliveryStream_openSearchServerlessUpdates' PKG=firehose ==> Checking that code complies with gofmt requirements... TF_ACC=1 go test ./internal/service/firehose/... -v -count 1 -parallel 20 -run=TestAccFirehoseDeliveryStream_openSearchServerlessUpdates -timeout 360m === RUN TestAccFirehoseDeliveryStream_openSearchServerlessUpdates === PAUSE TestAccFirehoseDeliveryStream_openSearchServerlessUpdates === CONT TestAccFirehoseDeliveryStream_openSearchServerlessUpdates --- PASS: TestAccFirehoseDeliveryStream_openSearchServerlessUpdates (536.74s) PASS ok github.com/hashicorp/terraform-provider-aws/internal/service/firehose 541.892s
@ewbankkit I saw you reworked the whole resource quite a bit and reverted and reimplemented the change. I hope the initial contribution was still useful and thanks a lot for your work. |
@stschulte Your contributions were great and I lifted them verbatim into the refactored code 👏. |
% make testacc TESTARGS='-run=TestAccFirehoseDeliveryStream_basic' PKG=firehose ==> Checking that code complies with gofmt requirements... TF_ACC=1 go test ./internal/service/firehose/... -v -count 1 -parallel 20 -run=TestAccFirehoseDeliveryStream_basic -timeout 360m === RUN TestAccFirehoseDeliveryStream_basic === PAUSE TestAccFirehoseDeliveryStream_basic === CONT TestAccFirehoseDeliveryStream_basic --- PASS: TestAccFirehoseDeliveryStream_basic (191.32s) PASS ok github.com/hashicorp/terraform-provider-aws/internal/service/firehose 196.775s
% make testacc TESTARGS='-run=TestAccFirehoseDeliveryStream_ExtendedS3_mskClusterSource' PKG=firehose ==> Checking that code complies with gofmt requirements... TF_ACC=1 go test ./internal/service/firehose/... -v -count 1 -parallel 20 -run=TestAccFirehoseDeliveryStream_ExtendedS3_mskClusterSource -timeout 360m === RUN TestAccFirehoseDeliveryStream_ExtendedS3_mskClusterSource === PAUSE TestAccFirehoseDeliveryStream_ExtendedS3_mskClusterSource === CONT TestAccFirehoseDeliveryStream_ExtendedS3_mskClusterSource --- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_mskClusterSource (562.46s) PASS ok github.com/hashicorp/terraform-provider-aws/internal/service/firehose 579.573s
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM 🚀.
% ACCTEST_TIMEOUT=720m make testacc TESTARGS='-run=TestAccFirehoseDeliveryStream_' PKG=firehose ACCTEST_PARALLELISM=3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/firehose/... -v -count 1 -parallel 3 -run=TestAccFirehoseDeliveryStream_ -timeout 720m
=== RUN TestAccFirehoseDeliveryStream_basic
=== PAUSE TestAccFirehoseDeliveryStream_basic
=== RUN TestAccFirehoseDeliveryStream_disappears
=== PAUSE TestAccFirehoseDeliveryStream_disappears
=== RUN TestAccFirehoseDeliveryStream_tags
=== PAUSE TestAccFirehoseDeliveryStream_tags
=== RUN TestAccFirehoseDeliveryStream_s3WithCloudWatchLogging
=== PAUSE TestAccFirehoseDeliveryStream_s3WithCloudWatchLogging
=== RUN TestAccFirehoseDeliveryStream_extendedS3basic
=== PAUSE TestAccFirehoseDeliveryStream_extendedS3basic
=== RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversion_enabled
=== PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversion_enabled
=== RUN TestAccFirehoseDeliveryStream_ExtendedS3_externalUpdate
=== PAUSE TestAccFirehoseDeliveryStream_ExtendedS3_externalUpdate
=== RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionDeserializer_update
=== PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionDeserializer_update
=== RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionHiveJSONSerDe_empty
=== PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionHiveJSONSerDe_empty
=== RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOpenXJSONSerDe_empty
=== PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOpenXJSONSerDe_empty
=== RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOrcSerDe_empty
=== PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOrcSerDe_empty
=== RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionParquetSerDe_empty
=== PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionParquetSerDe_empty
=== RUN TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionSerializer_update
=== PAUSE TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionSerializer_update
=== RUN TestAccFirehoseDeliveryStream_ExtendedS3_errorOutputPrefix
=== PAUSE TestAccFirehoseDeliveryStream_ExtendedS3_errorOutputPrefix
=== RUN TestAccFirehoseDeliveryStream_ExtendedS3_S3BackupConfiguration_ErrorOutputPrefix
=== PAUSE TestAccFirehoseDeliveryStream_ExtendedS3_S3BackupConfiguration_ErrorOutputPrefix
=== RUN TestAccFirehoseDeliveryStream_ExtendedS3Processing_empty
=== PAUSE TestAccFirehoseDeliveryStream_ExtendedS3Processing_empty
=== RUN TestAccFirehoseDeliveryStream_extendedS3KMSKeyARN
=== PAUSE TestAccFirehoseDeliveryStream_extendedS3KMSKeyARN
=== RUN TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioning
=== PAUSE TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioning
=== RUN TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioningUpdate
=== PAUSE TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioningUpdate
=== RUN TestAccFirehoseDeliveryStream_extendedS3Updates
=== PAUSE TestAccFirehoseDeliveryStream_extendedS3Updates
=== RUN TestAccFirehoseDeliveryStream_ExtendedS3_kinesisStreamSource
=== PAUSE TestAccFirehoseDeliveryStream_ExtendedS3_kinesisStreamSource
=== RUN TestAccFirehoseDeliveryStream_ExtendedS3_mskClusterSource
=== PAUSE TestAccFirehoseDeliveryStream_ExtendedS3_mskClusterSource
=== RUN TestAccFirehoseDeliveryStream_redshiftUpdates
=== PAUSE TestAccFirehoseDeliveryStream_redshiftUpdates
=== RUN TestAccFirehoseDeliveryStream_splunkUpdates
=== PAUSE TestAccFirehoseDeliveryStream_splunkUpdates
=== RUN TestAccFirehoseDeliveryStream_Splunk_ErrorOutputPrefix
=== PAUSE TestAccFirehoseDeliveryStream_Splunk_ErrorOutputPrefix
=== RUN TestAccFirehoseDeliveryStream_httpEndpoint
=== PAUSE TestAccFirehoseDeliveryStream_httpEndpoint
=== RUN TestAccFirehoseDeliveryStream_HTTPEndpoint_ErrorOutputPrefix
=== PAUSE TestAccFirehoseDeliveryStream_HTTPEndpoint_ErrorOutputPrefix
=== RUN TestAccFirehoseDeliveryStream_HTTPEndpoint_retryDuration
=== PAUSE TestAccFirehoseDeliveryStream_HTTPEndpoint_retryDuration
=== RUN TestAccFirehoseDeliveryStream_elasticSearchUpdates
=== PAUSE TestAccFirehoseDeliveryStream_elasticSearchUpdates
=== RUN TestAccFirehoseDeliveryStream_elasticSearchEndpointUpdates
=== PAUSE TestAccFirehoseDeliveryStream_elasticSearchEndpointUpdates
=== RUN TestAccFirehoseDeliveryStream_elasticSearchWithVPCUpdates
=== PAUSE TestAccFirehoseDeliveryStream_elasticSearchWithVPCUpdates
=== RUN TestAccFirehoseDeliveryStream_Elasticsearch_ErrorOutputPrefix
=== PAUSE TestAccFirehoseDeliveryStream_Elasticsearch_ErrorOutputPrefix
=== RUN TestAccFirehoseDeliveryStream_openSearchUpdates
=== PAUSE TestAccFirehoseDeliveryStream_openSearchUpdates
=== RUN TestAccFirehoseDeliveryStream_openSearchEndpointUpdates
=== PAUSE TestAccFirehoseDeliveryStream_openSearchEndpointUpdates
=== RUN TestAccFirehoseDeliveryStream_openSearchWithVPCUpdates
=== PAUSE TestAccFirehoseDeliveryStream_openSearchWithVPCUpdates
=== RUN TestAccFirehoseDeliveryStream_Opensearch_ErrorOutputPrefix
=== PAUSE TestAccFirehoseDeliveryStream_Opensearch_ErrorOutputPrefix
=== RUN TestAccFirehoseDeliveryStream_openSearchServerlessUpdates
=== PAUSE TestAccFirehoseDeliveryStream_openSearchServerlessUpdates
=== RUN TestAccFirehoseDeliveryStream_missingProcessing
=== PAUSE TestAccFirehoseDeliveryStream_missingProcessing
=== CONT TestAccFirehoseDeliveryStream_basic
=== CONT TestAccFirehoseDeliveryStream_httpEndpoint
=== CONT TestAccFirehoseDeliveryStream_ExtendedS3_errorOutputPrefix
--- PASS: TestAccFirehoseDeliveryStream_basic (130.83s)
=== CONT TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioningUpdate
--- PASS: TestAccFirehoseDeliveryStream_httpEndpoint (165.24s)
=== CONT TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioning
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_errorOutputPrefix (185.87s)
=== CONT TestAccFirehoseDeliveryStream_extendedS3KMSKeyARN
--- PASS: TestAccFirehoseDeliveryStream_extendedS3KMSKeyARN (121.75s)
=== CONT TestAccFirehoseDeliveryStream_ExtendedS3Processing_empty
--- PASS: TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioning (152.44s)
=== CONT TestAccFirehoseDeliveryStream_ExtendedS3_S3BackupConfiguration_ErrorOutputPrefix
--- PASS: TestAccFirehoseDeliveryStream_extendedS3DynamicPartitioningUpdate (293.09s)
=== CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionDeserializer_update
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3Processing_empty (144.75s)
=== CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionSerializer_update
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_S3BackupConfiguration_ErrorOutputPrefix (199.95s)
=== CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionParquetSerDe_empty
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionSerializer_update (213.83s)
=== CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOrcSerDe_empty
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionDeserializer_update (242.49s)
=== CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOpenXJSONSerDe_empty
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionParquetSerDe_empty (168.54s)
=== CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionHiveJSONSerDe_empty
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOpenXJSONSerDe_empty (139.41s)
=== CONT TestAccFirehoseDeliveryStream_extendedS3Updates
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionOrcSerDe_empty (140.03s)
=== CONT TestAccFirehoseDeliveryStream_Splunk_ErrorOutputPrefix
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversionHiveJSONSerDe_empty (129.39s)
=== CONT TestAccFirehoseDeliveryStream_splunkUpdates
--- PASS: TestAccFirehoseDeliveryStream_Splunk_ErrorOutputPrefix (149.89s)
=== CONT TestAccFirehoseDeliveryStream_extendedS3basic
--- PASS: TestAccFirehoseDeliveryStream_extendedS3Updates (162.08s)
=== CONT TestAccFirehoseDeliveryStream_redshiftUpdates
--- PASS: TestAccFirehoseDeliveryStream_splunkUpdates (162.14s)
=== CONT TestAccFirehoseDeliveryStream_ExtendedS3_externalUpdate
--- PASS: TestAccFirehoseDeliveryStream_extendedS3basic (129.86s)
=== CONT TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversion_enabled
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_externalUpdate (141.27s)
=== CONT TestAccFirehoseDeliveryStream_tags
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3DataFormatConversion_enabled (172.05s)
=== CONT TestAccFirehoseDeliveryStream_ExtendedS3_mskClusterSource
--- PASS: TestAccFirehoseDeliveryStream_tags (175.63s)
=== CONT TestAccFirehoseDeliveryStream_s3WithCloudWatchLogging
--- PASS: TestAccFirehoseDeliveryStream_redshiftUpdates (395.16s)
=== CONT TestAccFirehoseDeliveryStream_ExtendedS3_kinesisStreamSource
--- PASS: TestAccFirehoseDeliveryStream_s3WithCloudWatchLogging (101.70s)
=== CONT TestAccFirehoseDeliveryStream_disappears
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_kinesisStreamSource (81.31s)
=== CONT TestAccFirehoseDeliveryStream_openSearchUpdates
--- PASS: TestAccFirehoseDeliveryStream_disappears (96.37s)
=== CONT TestAccFirehoseDeliveryStream_openSearchWithVPCUpdates
--- PASS: TestAccFirehoseDeliveryStream_ExtendedS3_mskClusterSource (538.83s)
=== CONT TestAccFirehoseDeliveryStream_openSearchEndpointUpdates
--- PASS: TestAccFirehoseDeliveryStream_openSearchUpdates (1959.72s)
=== CONT TestAccFirehoseDeliveryStream_Opensearch_ErrorOutputPrefix
--- PASS: TestAccFirehoseDeliveryStream_openSearchEndpointUpdates (1983.42s)
=== CONT TestAccFirehoseDeliveryStream_missingProcessing
--- PASS: TestAccFirehoseDeliveryStream_missingProcessing (112.75s)
=== CONT TestAccFirehoseDeliveryStream_openSearchServerlessUpdates
--- PASS: TestAccFirehoseDeliveryStream_openSearchWithVPCUpdates (2763.90s)
=== CONT TestAccFirehoseDeliveryStream_elasticSearchEndpointUpdates
--- PASS: TestAccFirehoseDeliveryStream_openSearchServerlessUpdates (505.40s)
=== CONT TestAccFirehoseDeliveryStream_HTTPEndpoint_retryDuration
--- PASS: TestAccFirehoseDeliveryStream_HTTPEndpoint_retryDuration (196.31s)
=== CONT TestAccFirehoseDeliveryStream_Elasticsearch_ErrorOutputPrefix
--- PASS: TestAccFirehoseDeliveryStream_Opensearch_ErrorOutputPrefix (1937.63s)
=== CONT TestAccFirehoseDeliveryStream_HTTPEndpoint_ErrorOutputPrefix
--- PASS: TestAccFirehoseDeliveryStream_HTTPEndpoint_ErrorOutputPrefix (191.20s)
=== CONT TestAccFirehoseDeliveryStream_elasticSearchUpdates
--- PASS: TestAccFirehoseDeliveryStream_elasticSearchEndpointUpdates (1718.77s)
=== CONT TestAccFirehoseDeliveryStream_elasticSearchWithVPCUpdates
--- PASS: TestAccFirehoseDeliveryStream_Elasticsearch_ErrorOutputPrefix (1804.50s)
--- PASS: TestAccFirehoseDeliveryStream_elasticSearchUpdates (1678.97s)
--- PASS: TestAccFirehoseDeliveryStream_elasticSearchWithVPCUpdates (2472.77s)
PASS
ok github.com/hashicorp/terraform-provider-aws/internal/service/firehose 8453.764s
This functionality has been released in v5.20.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you! |
I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. |
Description
AWS added Amazon Kinesis Data Firehose stream delivery to OpenSearch Serverless which is currently not supported by the terraform provider. The provider only supports ElasticSearch and the regular OpenSearch.
In order to support this with terraform we now add a new
opensearchserverless_configuration
block that we can map to a desiredAmazonOpenSearchServerlessDestinationConfiguration
(see here).Please note that this is my first contribution to the provider and my first attempt in go. I don't have access to an AWS account right now, so this code is untested and I did not run acceptance tests. Unfortunately the only thing I can say at this point is "it compiles".
I hope I wil be able to run this against an actual AWS account soon but I wanted to get the MR out there in case
someone else is also planning to work on this so we are not duplicating effort.
Relations
Closes #32704
References
Output from Acceptance Testing