AmountString field type impractical for energy tariffs

[joshharris3]

Background:
The AmountString field type currently refers to an amount of currency, with valid currencies defined by ISO 4217, including AUD (Australian Dollars). This format ($00.00) is standard but impractical for energy tariffs.

Issue:
Market analysis and feedback from our API users indicate that energy tariffs are typically displayed in cents on retailer websites & third party comparison services. Storing tariff information in cents is strongly preferred because it reduces floating point precision issues and maintains consistency, eliminating the need for additional conversions during data sharing. The current process is cumbersome:

1. Retailers provide tariffs in cents.
2. AER converts these tariffs into dollars to meet CDR standards.
3. Data recipients must convert the tariffs back to cents to align with energy industry standards, such as those used on comparison websites.

Example retailer website

Area Affected
Get Generic Plan Detail
https://consumerdatastandardsaustralia.github.io/standards/#cdr-energy-api_get-generic-plan-detail

Change Proposed
We propose two alternatives to better align with industry practices:

1. Define a new type to support the energy industry's use of cents for tariffs and rebates. For example, a tariff of 5 cents would be represented as "0.05" in AmountString and "5" in a newly defined type.
2. Use the Number type for tariffs and rebates and specify in the attribute descriptions that the values represent cents.

Example Proposed new Tariff Structure

                                   "type": "SHOULDER",
                        "rates": [
                            {
                                "unitPrice": "25.83"
                            }
                        ],

Other instances where energy tariffs are shown in cents:

Retailers:

Origin

Red Energy

API users & third party websites

Finder

Solar Choice

DSB Proposal

The current DSB proposal for this issue is in #644 (comment)

[nils-work]

Hi @joshharris3

The description for AmountString states:

Minimum 2 digits following a decimal point (more digits allowable but only if required)

So "unitPrice": "0.2583" would be valid unless I've misunderstood your concerns.

[joshharris3]

Hi Nils,

I agree with your above comment, however perhaps wasn't as clear above about issue here.

Just to reiterate, we don't believe that the AmountString supports the energy industry which displays tariffs in cents not dollars. What this means practically:

1. Retailers provide tariffs in cents,
2. AER converts into dollars to meet CDR standards,
3. Data recipients must convert back to cents to align with energy industry standards e.g. for comparison websites.

I hope this helps clarify our position.

Thanks

[mattyp]

1. The units to use for an 'AmountString' field is and has always been ambiguous. The definition does not explicitly specify units (dollars or cents):
   
   ISO 4217 is only ever referenced for the "CurrencyString" (currency code), not an amount. In reality today, interpretation of this ambiguity means some AmountString values are provided as dollars and some as cents, even within details of the same plan contract. It would help enormously if this ambiguity were to be explicitly clarified in the definition. For example: "A string representing a monetary amount in main currency units with fractional units after a decimal point (e.g., if working with Australian dollars: "123.45" for one hundred twenty-three dollars and forty-five cents)."

2. Whatever the defined units are, converting between cents and dollars is trivial and not a worthy reason for a CR on its own. In terms of practicality, both dollar and cents are practical and what is "displayed" is irrelevant to what is defined. FWIW, in my implementation I try to detect the units being provided and standardise all to cents. Point #1, above, would help a lot to reduce errors in consumer facing products.

3. As one of many data recipients that will rely on the integrity of data from the AER my plead to the AER is to simply implement the format of main units (e.g. dollar) consistently and not let this reported maintenance issue be an excuse to hold up current much needed rectification work:
   

In summary, I concur with the callout that the definition for "AmountString" is problematically ambiguous, but do not support the notion that a change to the standards beyond clarifying the definition is required.

[ProductCloudCDR]

1. The units to use for an 'AmountString' field is and has always been ambiguous. The definition does not explicitly specify units (dollars or cents):

This is just a type. The clarification is made elsewhere in the standards where there are numerous references indicating the currency that the AmountString is being used to represent (either as a default, or as an optional field). As the default is usually AUD if no specified it clarifies the ambiguity that its in dollars.

[ProductCloudCDR]

We don't agree with this change as it introduces a new type for a common entity (currency amount). AmountString is used throughout the standards and anyone consuming CDR payloads will support it. This change is effectively pushing AER cost onto all future consumers of the data.

[AGL-CDR]

AGL does not support this change. All participants are free to create internal functions that convert dollars to cents and back again as the context of presentation requires, according to their own purposes.

[mattyp]

This is just a type.

Granted, there may be a more appropriate place for the clarification than at the type definition.

The clarification is made elsewhere in the standards where there are numerous references indicating the currency that the AmountString is being used to represent (either as a default, or as an optional field). As the default is usually AUD if no specified it clarifies the ambiguity that its in dollars.

This exemplifies the ambiguity. The clarification is not explicitly made elsewhere. While the units to use may be deduced, there is evidently room for interpretation which is incongruent with the purpose of a specification. From a DR perspective, the data coming through from DHs (primary & secondary) clearly demonstrates the ambiguity is real. A simple explicit clarification can only be helpful for everyone and help improve CDR data quality, which is widely recognised as an ecosystem issue.

Speaking of confusion, I'm probably confusing this CR with this side-issue. Sorry all! It's prob worthy of its own ticket...

[ProductCloudCDR]

This exemplifies the ambiguity. The clarification is not explicitly made elsewhere. While the units to use may be deduced, there is evidently room for interpretation which is incongruent with the purpose of a specification. From a DR perspective, the data coming through from DHs (primary & secondary) clearly demonstrates the ambiguity is real. A simple explicit clarification can only be helpful for everyone and help improve CDR data quality, which is widely recognised as an ecosystem issue.

I just checked and you're right. The energy standard never actually says that the currency of amounts should be assumed to be AUD so that aspect is ambiguous. In the banking standard it's pretty explicit throughout. I'm pretty sure this was discussed in the early days of the energy standard but it never made it into the standards itself.

Speaking of confusion, I'm probably confusing this CR with this side-issue. Sorry all! It's prob worthy of its own ticket...

I agree. If you raise one we would support it being included in MI20. It would be a non-breaking change.

[dempstert]

I think the proposed change is an unnecessary headache for anyone who has already integrated with this API and the change offers no clear upside

[perlboy]

Agree with @dempstert, introducing a type and requiring every energy related value to be reset to cents when implementers have already completed their build is a pointless waste of participant resources. Perhaps the AmountString definition can be clearer but I see no change outside of clarification being required here.

[mattyp]

I agree. If you raise one we would support it being included in MI20. It would be a non-breaking change.

#652 raised and suggested for MI20 👍

[CDR-API-Stream]

Based on the discussion during July 10 MI 20 meeting and the feedback noted by participants in comments above, the DSB is proposing not to proceed with this change request.

During the discussions it was noted that the definition of AmountString in the standards may be ambiguous. This will be discussed as part of a separate CR #652 in this MI which @mattyp has kindly raised.

Feedback on the above is welcome.

[nils-work]

Closing with no change made, as per #353 - Decision Proposal 353 - Maintenance Iteration 20.