Data from region ap-southeast-5 is not being fetched

[agoyburu]

Describe the bug
Plugin is not fetching data from new region ap-southeast-5

Steampipe version (steampipe -v)
Example: v1.0.0

Plugin version (steampipe plugin list)
Example: v1.4.0

To reproduce
Create any resource in region ap-southeast-5 and try to fetch it using steampipe.

Expected behavior
The resource should be present in the result.

Additional context
Region ap-southeast-5 was made available last September so it is kind of new.

Querying select * from aws_region; shows the region as opted-in but still no data is being retrieved from there.

I've also tried using that region exclusively but that didn't work either:

connection "aws_default" {
  plugin = "aws"
  # `regions` defines the list of regions that Steampipe should target for
  # each query. API calls are made to multiple regions in parallel. The regions
  # list may include wildcards (e.g. *, us-*, us-??st-1).
  # If `regions` is not specified, Steampipe will target the `default_region`
  # only.
  regions = ["ap-southeast-5"] # All regions

[ParthaI]

Hi @agoyburu,

Sorry to hear that you’re encountering this issue.

After investigating, I’ve identified the root cause of why resources for the ap-southeast-5 region are not being retrieved. Here are my findings:

• The region ap-southeast-5 was officially made generally available (GA) on August 21, 2024.
• Our AWS plugin is designed to make API calls based on the region configured in the aws.spc file.
• The plugin relies on AWS SDK V1 to fetch the list of supported regions for services.
• However, AWS SDK V1 is in maintenance mode and will reach its end-of-life on July 31, 2025. For details, see the announcement: AWS SDK V1 End-of-Life.
• When fetching the supported region list for the service, the ap-southeast-5 region is categorized as unsupported because it hasn’t been added to the list in AWS SDK V1.
• We currently have an open issue to remove support for AWS SDK V1 from the plugin. Once this transition is complete, the issue should be resolved, as ap-southeast-5 is already supported in AWS SDK V2.

Thanks, and I hope this clarifies the situation!

[agoyburu]

Hi @ParthaI, thanks for replying.

Do you know when this will be worked on?

[ParthaI]

Sorry, @agoyburu, we don’t have an ETA for this issue yet. It’s on our to-do list, and we’ll update you as soon as we begin working on it. Thank you for your patience!

[Luis-Garcia-206]

Hi @ParthaI , happy new year!
Do you have any updates or an ETA for this issue?

[ParthaI]

Hello @agoyburu, @Luis-Garcia-206,

Apologies for the delay in addressing this issue. I’ve created a draft PR to address the problem and have tested several tables. The changes appear to be working well on my end.

With the updated code, the tables can now successfully fetch resources from the region ap-southeast-5. Here's an example query result:

> select vpc_id, region from aws_vpc
+-----------------------+----------------+
| vpc_id                | region         |
+-----------------------+----------------+
| vpc-097ed715748293054 | us-east-1      |
| vpc-070dcf85b82e773be | us-east-1      |
| vpc-0e8e31d2c03750f7a | ap-south-1     |
| vpc-0c34189d09770cdba | ap-south-1     |
| vpc-0308016b          | ap-south-1     |
| vpc-4438322d          | eu-west-3      |
| vpc-70304918          | eu-west-2      |
| vpc-3cb25e45          | eu-west-1      |
| vpc-9c488bf6          | eu-central-1   |
| vpc-3c053f5b          | ap-southeast-2 |
| vpc-4f8e9928          | ap-southeast-1 |
| vpc-00fedf50bbc180cf2 | me-central-1   |
| vpc-06d7cf161df3af8b1 | me-south-1     |
| vpc-0fbdaa2addb65b03f | ap-southeast-3 |
| vpc-2c591f44          | ca-central-1   |
| vpc-5c1ecc35          | eu-north-1     |
| vpc-09b59cc2e412311d4 | ap-northeast-3 |
| vpc-09b8899d28cd2afe0 | ap-northeast-3 |
| vpc-aa63a1c1          | us-east-2      |
| vpc-09afad0360e9e9e61 | us-east-2      |
| vpc-0884cc3e220be8370 | ap-east-1      |
| vpc-2ca9a84b          | ap-northeast-1 |
| vpc-039b616565a5c9b9b | af-south-1     |
| vpc-06c6156d          | ap-northeast-2 |
| vpc-0979059646e244b4c | eu-south-1     |
| vpc-0e41a09234cfac0c8 | us-west-2      |
| vpc-0eb0b270446015d85 | us-west-2      |
| vpc-69f9180f          | us-west-1      |
| vpc-04c48faffa7dc913f | ap-southeast-5 |
| vpc-33d8d954          | sa-east-1      |
+-----------------------+----------------+

I’d appreciate it if you could test the changes in the PR branch and share your observations. Let me know if you encounter any additional issues.

Note: This PR is currently work in progress (WIP) and will be updated with additional changes soon.

Thanks for your patience!