[COST-5657] subs: use subquery in Trino to join from Postgres

[maskarb]

Jira Ticket

COST-5657

Description

This change will filter the list of resource-ids thru a subquery in postgres.

Testing

1. smokes

Release Notes

• proposed release note

* [COST-5657](https://issues.redhat.com/browse/COST-5657) subs: use subquery in Trino to join from Postgres

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.1%. Comparing base (6eea791) to head (637c005).
Report is 1 commits behind head on main.

Additional details and impacted files

@@          Coverage Diff          @@
##            main   #5381   +/-   ##
=====================================
  Coverage   94.1%   94.1%           
=====================================
  Files        371     371           
  Lines      31557   31552    -5     
  Branches    3385    3385           
=====================================
- Hits       29686   29684    -2     
+ Misses      1210    1209    -1     
+ Partials     661     659    -2     

[myersCody]

We likely have the same problem with the determine_ids_for_provider function. There is likely a cleaner way to handle the exclusion in Trino. Right now we use the django ORM to build the exclude list and then populate the exclude filter using that list in jinja creating our huge query problem.

Instead we could just build the exclude resource id list in trino, then left join in the excluded list and a filter to make sure all of our excluded resources are not included in the return:

    SELECT distinct resource_id 
    FROM postgres.{{schema | sqlsage}}.reporting_subs_last_processed_time as lp
    WHERE lp.source_uuid != {{source_uuid}}
)
SELECT lineitem_resourceid, max(lineitem_usagestartdate)
FROM hive.{{schema | sqlsafe}}.aws_line_items
LEFT JOIN excluded_ids AS ex ON aws.lineitem_resource = ex.resource_id
WHERE source={{source_uuid}}
AND year={{year}}
AND month={{month}}
AND lineitem_productcode = 'AmazonEC2'
AND strpos(lower(resourcetags), 'com_redhat_rhel') > 0
AND ex.resource_id IS NULL
GROUP BY lineitem_resourceid

This will allow for the excludes resource ids to be however long it needs to be without changing the length of the query itself.

[maskarb]

/retest

[maskarb]

@myersCody good suggestion. I had not considered making Trino do the postgres query, so that's a good idea!