FOCUS #174: ListCost - Merge remote-tracking branch 'origin/working_d…

…raft' into 174-create-a-list-cost-metric # Conflicts: # specification/metrics/metrics.mdpp
FinOps-Open-Cost-and-Usage-Spec · Nov 5, 2023 · ca210cd · ca210cd
2 parents f11a7da + 147250f
commit ca210cd
Show file tree

Hide file tree

Showing 10 changed files with 122 additions and 25 deletions.
diff --git a/specification/dimensions/billingaccountid.md b/specification/dimensions/billingaccountid.md
@@ -4,7 +4,7 @@ A billing account is a container for resources and/or services that are billed t
 
 A Billing Account ID is a provider assigned identifier for a billing account.
 
-The BillingAccountId column MUST be present in the billing data. This column must be of type String and MUST NOT contain null values. BillingAccountId MUST be a globally unique identifier within a provider.
+The BillingAccountId column MUST be present in the billing data. This column MUST be of type String and MUST NOT contain null values. BillingAccountId MUST be a globally unique identifier within a provider.
 
 See [Appendix: Grouping constructs for resources and/or services](#groupingconstructsforresourcesand/orservices) for details and examples of the different grouping constructs supported by FOCUS.
 

diff --git a/specification/dimensions/billingcurrency.md b/specification/dimensions/billingcurrency.md
@@ -2,7 +2,7 @@
 
 Billing Currency is an identifier that represents the currency that a charge for resources and/or services was billed in. Billing Currency is commonly used in scenarios where costs need to be grouped or aggregated.
 
-The BillingCurrency column MUST be present in the billing data. BillingCurrency MUST match the currency used in the invoice generated by the invoice issuer. This column must be of type String and MUST NOT contain null values. BillingCurrency MUST conform to [FOCUS Currency Code Format](#currencycodeformat) requirements.
+The BillingCurrency column MUST be present in the billing data. BillingCurrency MUST match the currency used in the invoice generated by the invoice issuer. This column MUST be of type String and MUST NOT contain null values. BillingCurrency MUST conform to [FOCUS Currency Code Format](#currencycodeformat) requirements.
 
 ## Column ID
 

diff --git a/specification/dimensions/commitmentdiscountcategory.md b/specification/dimensions/commitmentdiscountcategory.md
@@ -0,0 +1,39 @@
+# Commitment Discount Category
+
+A commitment-based discount is a commitment for an amount of usage or spend throughout a specified term, in exchange for discounted unit pricing on that amount. The commitment may be based on quantities of resource units or monetary value, with various payment options and time frames.
+
+Commitment Discount Category indicates whether the commitment-based discount identified in the CommitmentDiscountId column is based on usage quantity or cost (aka "spend").
+
+The CommitmentDiscountCategory column MUST be present in the billing data. This column MUST be of type String, MUST be null when CommitmentDiscountId is null, and MUST NOT be null when CommitmentDiscountId is not null. The CommitmentDiscountCategory MUST be one of the allowed values.
+
+## Column ID
+
+CommitmentDiscountCategory
+
+## Display name
+
+Commitment Discount Category
+
+## Description
+
+Indicates whether the commitment-based discount identified in the CommitmentDiscountId column is based on usage quantity or cost (aka "spend").
+
+## Content constraints
+
+|    Constraint   |      Value       |
+|:----------------|:-----------------|
+| Column required | True             |
+| Data type       | String           |
+| Allows nulls    | True             |
+| Value format    | list-of-values   |
+
+Allowed values:
+
+| Value      | Description                                                                                                                                                                   |
+|:--------|:--------------------------------------------------------------------------------|
+| Spend   | Commitment-based discounts that require a predetermined amount of spend.    |
+| Usage   | Commitment-based discounts that require a predetermined amount of usage.    |
+
+## Introduced (version)
+
+1.0
diff --git a/specification/dimensions/commitmentdiscountid.md b/specification/dimensions/commitmentdiscountid.md
@@ -1,10 +1,10 @@
 # Commitment Discount ID
 
-A commitment-based discount is a contractual commitment for an amount of usage or spend throughout a specified term, in exchange for discounted unit pricing on that amount. The commitment may be based on quantities of resource units or monetary value, with various payment options and time frames.
+A commitment-based discount is a commitment for an amount of usage or spend throughout a specified term, in exchange for discounted unit pricing on that amount. The commitment may be based on quantities of resource units or monetary value, with various payment options and time frames.
 
 A Commitment Discount ID is the identifier assigned to a commitment-based discount by the provider. Commitment Discount ID is commonly used for scenarios like chargeback for commitments and savings per commitment-based discount.
 
-The CommitmentDiscountId column MUST be present in the billing data. This column must be of type String and MUST NOT contain null values when a charge is related to a commitment-based discount. When a charge is not associated with a commitment-based discount, the column MUST be null. CommitmentDiscountId MUST be unique within the provider.
+The CommitmentDiscountId column MUST be present in the billing data. This column MUST be of type String and MUST NOT contain null values when a charge is related to a commitment-based discount. When a charge is not associated with a commitment-based discount, the column MUST be null. CommitmentDiscountId MUST be unique within the provider.
 
 ## Column ID
 

diff --git a/specification/dimensions/commitmentdiscountname.md b/specification/dimensions/commitmentdiscountname.md
@@ -1,10 +1,10 @@
 # Commitment Discount Name
 
-A commitment-based discount is a contractual commitment for an amount of usage or spend throughout a specified term, in exchange for discounted unit pricing on that amount. The commitment may be based on quantities of resource units or monetary value, with various payment options and time frames.
+A commitment-based discount is a commitment for an amount of usage or spend throughout a specified term, in exchange for discounted unit pricing on that amount. The commitment may be based on quantities of resource units or monetary value, with various payment options and time frames.
 
 A Commitment Discount Name is the display name assigned to a commitment-based discount.
 
-The CommitmentDiscountName column MUST be present in the billing data. This column must be of type String. The CommitmentDiscountName value MUST be null if the charge is not related to a commitment-based discount and MAY be null if a display name cannot be assigned to a commitment-based discount. CommitmentDiscountName MUST NOT be null if a display name can be assigned to a commitment-based discount.
+The CommitmentDiscountName column MUST be present in the billing data. This column MUST be of type String. The CommitmentDiscountName value MUST be null if the charge is not related to a commitment-based discount and MAY be null if a display name cannot be assigned to a commitment-based discount. CommitmentDiscountName MUST NOT be null if a display name can be assigned to a commitment-based discount.
 
 ## Column ID
 

diff --git a/specification/dimensions/dimensions.mdpp b/specification/dimensions/dimensions.mdpp
@@ -16,6 +16,7 @@ dimensions to categorize, filter, and reveal details in your data when grouped w
 !INCLUDE "chargetype.md",1
 !INCLUDE "commitmentdiscountid.md",1
 !INCLUDE "commitmentdiscountname.md",1
+!INCLUDE "commitmentdiscountcategory.md",1
 !INCLUDE "invoiceissuer.md",1
 !INCLUDE "provider.md",1
 !INCLUDE "publisher.md",1

diff --git a/specification/metrics/billedcost.md b/specification/metrics/billedcost.md
@@ -1,8 +1,8 @@
 # Billed Cost
 
-The Billed Cost represents a charge inclusive of negotiated discounts that a consumer would be charged for each billing period. Billed Cost does not amortize upfront charges (one-time or recurring). The currency for the value specified in Billed Cost can be found in the [Billing Currency](#billingcurrency) column. Billed Cost is often used to perform FinOps capabilities that require cash-basis accounting such as cost allocation, budgeting, and invoice reconciliation.
+The Billed Cost represents a charge serving as the basis for invoicing, inclusive of all reduced rates and discounts while excluding the amortization of upfront charges (one-time or recurring). This cost is denominated in the [Billing Currency](#billingcurrency). The Billed Cost is used to perform FinOps capabilities that require cash-basis accounting such as cost allocation, budgeting, and invoice reconciliation.
 
-The BilledCost column MUST be present in the billing data. This column MUST be a numeric value of type Decimal and MUST NOT contain null values. The aggregated BilledCost for a billing period MUST match the charge received on the invoice for the same billing period.
+The BilledCost column MUST be present in the billing data. This column MUST be a valid numeric value of type Decimal and MUST NOT contain null values. The aggregated BilledCost for a billing period MUST match the charge received on the invoice for the same billing period.
 
 ## Column ID
 
@@ -14,16 +14,17 @@ Billed Cost
 
 ## Description
 
-A charge inclusive of negotiated discounts that a consumer would be charged for each billing period.
+A charge serving as the basis for invoicing, inclusive of all reduced rates and discounts while excluding the amortization of upfront charges (one-time or recurring).
 
 ## Content constraints
 
-|    Constraint   |      Value      |
-|:----------------|:----------------|
-| Column required | True            |
-| Data type       | Decimal         |
-| Allows nulls    | False           |
-| Value format    | Numeric value   |
+|    Constraint   |      Value              |
+|:----------------|:------------------------|
+| Column required | True                    |
+| Data type       | Decimal                 |
+| Allows nulls    | False                   |
+| Value format    | Numeric value           |
+| Number range    | Any valid decimal value |
 
 ## Introduced (version)
 

diff --git a/specification/metrics/metrics.mdpp b/specification/metrics/metrics.mdpp
@@ -5,7 +5,7 @@ The FOCUS specification defines a group of columns that provide numeric values.
 allow for aggregation operations such as arithmetic operations (sum, multiplication, averaging etc.) and statistical
 operations. When combined with dimensions (which are qualitative value columns), metrics reveal details in your data.
 
-!INCLUDE "effectivecost.md",1
 !INCLUDE "billedcost.md",1
+!INCLUDE "effectivecost.md",1
 !INCLUDE "listcost.md",1
 !INCLUDE "listunitprice.md",1
diff --git a/supporting_content/dimensions/commitmentdiscountcategory.md b/supporting_content/dimensions/commitmentdiscountcategory.md
@@ -0,0 +1,59 @@
+# Column: CommitmentDiscountCategory
+
+## Example provider mappings
+
+Current column mappings found in available data sets:
+
+| Provider | Data set                 | Column                    |  Example Values  |
+|----------|--------------------------|---------------------------|------------------|
+| AWS | CUR                      | Not available   | N/A |
+| Google Cloud | BigQuery Billing Export | credit.type              | COMMITTED_USAGE_DISCOUNT, COMMITTED_USAGE_DISCOUNT_DOLLAR_BASE |
+| Microsoft | Cost Details             | Not available               | N/A |
+
+## Example usage scenarios
+
+Current values observed in billing data for various scenarios:
+
+| Provider | Data set                 | CommitmentDiscountCategory     | CommitmentDiscountProgram (Name TBD)     |
+|----------|--------------------------|----------------------------|------------------------------------------|
+| AWS | CUR (PurchaseOption)                   | Usage               | Reserved Instances              |
+| AWS | CUR (PurchaseOption)                   | Spend               | Savings Plans                   |
+| Google Cloud | BigQuery Billing Export | Usage        | Resource-based CUD                            |
+| Google Cloud | BigQuery Billing Export | Spend        | Spend-based CUD                               |
+| Microsoft | Cost Details (PricingModel)| Spend                     | Savings Plan                     |
+| Microsoft | Cost Details (PricingModel)| Usage                     | Reservation                      |
+
+## Documentation
+- Microsoft
+  - Azure:  Understand usage details fields: https://learn.microsoft.com/en-us/azure/cost-management-billing/automate/understand-usage-details-fields
+- GCP
+  - Google Commitment Types:  https://cloud.google.com/docs/cuds#spend_versus_resource_commitments (This is the most similar to our selected implementation)
+  - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/standard-usage - see credit.type
+- AWS
+  - Amazon: Reservation details - https://docs.aws.amazon.com/cur/latest/userguide/reservation-columns.html
+
+## Discussion Topics
+It was discussed whether or not this field should be a normalized list of values OR if we should make it a suggestive, freeform text field as different cloud providers have different names for their implementation of Commitment Usage Discounts. For example:
+  - Savings Plan
+  - RI/CUD
+  - Flexible CUDs
+
+For AWS, this column could possibly calculated as such:
+  if reservation/ReservationARN <> '' then Usage, else if savingsPlan/SavingsPlanArn <> '' then Spend, else nullThis column
+
+It was agreed that another column would be added (ideally in V1.0) that would identify the CUD name as termed by the Cloud Provider and that this column would be normalized to allow practitioners to have a standard interface to group and compare CUDs from multiple sources. This would be a non-normalized string.
+
+The name of this additional column is yet to be determined but could be something like:
+- Commitment Discount Type
+- Commitment Discount Program
+- Commitment Discount Plan
+- Commitment Discount Plan Name
+- Other
+
+- Microsoft
+  - Azure:  PricingModel: https://learn.microsoft.com/en-us/azure/cost-management-billing/automate/understand-usage-details-fields
+- GCP
+  - Google Commitment Types:  https://cloud.google.com/docs/cuds#spend_versus_resource_commitments (This is the most similar to our selected implementation)
+  - https://cloud.google.com/billing/docs/how-to/export-data-bigquery-tables/standard-usage
+- AWS
+  - Amazon: Instance purchasing options - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-purchasing-options.html
diff --git a/supporting_content/metrics/billedcost.md b/supporting_content/metrics/billedcost.md
@@ -1,23 +1,21 @@
-## Supporting content
+# Column: Billed Cost
 
-### Column: Billed Cost
-
-#### Example provider mappings 
+## Example provider mappings
 
 Current column mappings found in available data sets:
 
 | **Provider** | **Data set** | **Column** |
 |---|---|---|
 | AWS | CUR | lineItem/NetUnblendedCost (If you have an EDP, otherwise line_item_unblended_cost) |
-| GCP | Big Query Billing Export | Cost (The cost of the usage before any credits, to a precision of up to six decimal places) |
+| GCP | Big Query Billing Export | credits + cost (The cost of the usage before any credits, to a precision of up to six decimal places. To get the total cost including credits, any credits.amount should be added to cost.) |
 | Microsoft | Cost details | Cost, CostInBillingCurrency, CostInPricingCurrency, EffectivePrice |
-| OCI | Cost reports | cost/myCost |
+| OCI | Cost reports | cost/myCostOverage (The cost billed for overage usage of a resource) |
 
-#### Example usage scenarios
+## Example usage scenarios
 
 See [Cost Metrics Examples Spreadsheet](https://docs.google.com/spreadsheets/d/1bhRELDgf3LTSfQJRrCyovTt65g4ElimYHq6fmKOz83E) for examples of billing data for various scenarios.
 
-### Discussion / Scratch space
+## Discussion / Scratch space
 
 - Aggregations can capture billed cost across different charge types.
   - The cost shown in a single line item may not be the final billed cost of a given resource for a billing period as taxes, discounts and other adjustments for the same resource will be in other rows.
@@ -52,4 +50,3 @@ See [Cost Metrics Examples Spreadsheet](https://docs.google.com/spreadsheets/d/1
 | 369.15 | CNY | 369.15 | CNY |
 
 *Do we just need a THIRD option - CostInBase, BaseCurrency - which is like UTC - it could be USD? OR are we assuming that PricedCurrency is always USD?*
-