Three Phase Power Meter - first pass implementation

[BuongiornoTexas]

I didn't get any response on my earlier three phase question, so I'm guessing interest for this functionality is low.

However, as it is useful for me and may be useful for others in the future, I'll show what I have done and how I have implemented it. It's not a general implementation - just good enough for my needs, so I haven't posted it as an enhancement issue. If more people get interested, we can work on making it more general purpose. I'll go through the work that would need to be done at the end of this post.

First step: What is it?

The plot pretty well summarises it: Per phase details of home, solar, grid and battery usage. So for our house, we have one powerwall and one solar system on phase A (most of our day to day circuits hang on this phase), phase B is nearly unused, and phase C has a second solar system and some heavier intermittent loads (EV, heat pumps and resistance heaters). Both solar systems are non-tesla.

It's a useful set of plots for me, as I can do diagnostics on what is happening on each phase, and, on marginal production days, I can decide if I want to plug in our EV without doing a full circuit of the house to check on the solar system without wifi (yeah - it's basically to save me walking around the house on cloudy days).

Current implementation details:

• Phase monitoring dashboard JSON attached at the bottom of the post.
• At this point, I'm injecting the data I need for monitoring into the raw measurement. This is only retained for 3 days.
• Data injection is by the following addition to telegraf.conf, pasted after the current [[inputs.http]] block:

  [[inputs.http]]
        urls = [
                "http://pypowerwall:8675/api/meters/solar"
        ]
        name_override = "solarmeter"
        method = "GET"
        insecure_skip_verify = true
        timeout = "4s"
        data_format = "json"
        json_query = "0.Cached_readings"
        fieldpass = ["instant_power", "real_power_?", "energy_??ported"]        
  
  [[inputs.http]]
        urls = [
                "http://pypowerwall:8675/api/meters/site"
        ]
        name_override = "sitemeter"
        method = "GET"
        insecure_skip_verify = true
        timeout = "4s"
        data_format = "json"
        json_query = "0.Cached_readings"
        fieldpass = ["instant_power", "real_power_?", "energy_??ported"]
  
  [[inputs.http]]
        urls = [
                "http://pypowerwall:8675/api/system_status"
        ]
        name_override = "batterymeter"
        method = "GET"
        insecure_skip_verify = true
        timeout = "4s"
        data_format = "json"
        json_query = "battery_blocks"
        fieldpass = ["p_out"]

Implementation notes:

• I've gone for multiple measurements and overrides to deal with the problem that the solar and site meter pages use the same template, so influxdb treats them as the same metric (I think "tag_keys" would get around this?). This will be a problem with more than one powerwall, as the power data p_out value is going to face the same issue.
• Separate measurements with name overrides also allows me to use json_query and fieldpass on each, which minimises the data collected and written to the database.
• The dashboard reflects my site. Others will have different line ups of solar systems and powerwalls per phase. This can be addressed by:
  • Checking phase line ups on the following pages: localhost:8675/api/meters/solar (solar phase assignment), localhost:8675/powerwalls (powerwall phase assignment), or by doing some trial and error experimentation.
  • Editing the grafana queries for each phase to match the site lineup:
    

To convert this proof of concept to a general implementation, we need to do the following:

• Automate the phase line up. I think this is doable with a small pypowerwall utility, but this should also be doable with short interactive script to allow the site owner to specify solar and battery phase line ups manually.
• Generate powerwall data for more than one battery. I have some ideas on how to do this with "tag_keys" and the powerwalll serial numbers as tags, but will need to talk to someone with more than one powerwall to test them out.
• Automatically update the json to reflect the phase line ups.
• Create an RP if users want to retain more than three days of data.
• Replace the expression used for home load with a grafana influxdb query (the expression is computationally inefficient - I'm only using it for the proof of concept).

Dashboard JSON

{
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": {
          "type": "grafana",
          "uid": "-- Grafana --"
        },
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "target": {
          "limit": 100,
          "matchAny": false,
          "tags": [],
          "type": "dashboard"
        },
        "type": "dashboard"
      }
    ]
  },
  "editable": true,
  "fiscalYearStartMonth": 0,
  "graphTooltip": 0,
  "id": 2,
  "links": [],
  "liveNow": false,
  "panels": [
    {
      "aliasColors": {
        "Charge": "red",
        "Grid Usage": "purple",
        "Home Usage": "blue",
        "Powerwall": "green",
        "Solar Energy": "yellow",
        "Sun altitude": "rgb(192, 192, 192)"
      },
      "bars": false,
      "dashLength": 10,
      "dashes": false,
      "datasource": {
        "type": "datasource",
        "uid": "-- Mixed --"
      },
      "fieldConfig": {
        "defaults": {
          "links": []
        },
        "overrides": []
      },
      "fill": 3,
      "fillGradient": 0,
      "gridPos": {
        "h": 9,
        "w": 7,
        "x": 0,
        "y": 0
      },
      "hiddenSeries": false,
      "id": 2,
      "interval": "",
      "legend": {
        "alignAsTable": false,
        "avg": false,
        "current": true,
        "max": false,
        "min": false,
        "rightSide": false,
        "show": true,
        "total": false,
        "values": true
      },
      "lines": true,
      "linewidth": 1,
      "nullPointMode": "connected",
      "options": {
        "alertThreshold": true
      },
      "percentage": false,
      "pluginVersion": "9.1.2",
      "pointradius": 2,
      "points": false,
      "renderer": "flot",
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "alias": "Solar Energy",
          "datasource": {
            "type": "influxdb",
            "uid": "ajmBhJ74k"
          },
          "groupBy": [
            {
              "params": [
                "5m"
              ],
              "type": "time"
            },
            {
              "params": [
                "0"
              ],
              "type": "fill"
            }
          ],
          "hide": false,
          "measurement": "solarmeter",
          "orderByTime": "ASC",
          "policy": "raw",
          "refId": "Solar",
          "resultFormat": "time_series",
          "select": [
            [
              {
                "params": [
                  "real_power_a"
                ],
                "type": "field"
              },
              {
                "params": [],
                "type": "mean"
              }
            ]
          ],
          "tags": [],
          "tz": "Australia/Hobart"
        },
        {
          "alias": "Powerwall",
          "datasource": {
            "type": "influxdb",
            "uid": "ajmBhJ74k"
          },
          "groupBy": [
            {
              "params": [
                "5m"
              ],
              "type": "time"
            },
            {
              "params": [
                "0"
              ],
              "type": "fill"
            }
          ],
          "hide": false,
          "measurement": "batterymeter",
          "orderByTime": "ASC",
          "policy": "raw",
          "query": "SELECT mean(from_pw) - mean(to_pw) FROM autogen.http WHERE $timeFilter GROUP BY time(5m) tz('Australia/Hobart')",
          "rawQuery": false,
          "refId": "Powerwall",
          "resultFormat": "time_series",
          "select": [
            [
              {
                "params": [
                  "p_out"
                ],
                "type": "field"
              },
              {
                "params": [],
                "type": "mean"
              }
            ]
          ],
          "tags": [],
          "tz": "Australia/Hobart"
        },
        {
          "alias": "Grid Usage",
          "datasource": {
            "type": "influxdb",
            "uid": "ajmBhJ74k"
          },
          "groupBy": [
            {
              "params": [
                "5m"
              ],
              "type": "time"
            },
            {
              "params": [
                "0"
              ],
              "type": "fill"
            }
          ],
          "hide": false,
          "measurement": "sitemeter",
          "orderByTime": "ASC",
          "policy": "raw",
          "query": "SELECT mean(from_grid) - mean(to_grid) FROM autogen.http WHERE $timeFilter GROUP BY time(5m) tz('Australia/Hobart')",
          "rawQuery": false,
          "refId": "Grid",
          "resultFormat": "time_series",
          "select": [
            [
              {
                "params": [
                  "real_power_a"
                ],
                "type": "field"
              },
              {
                "params": [],
                "type": "mean"
              }
            ]
          ],
          "tags": [],
          "tz": "Australia/Hobart"
        },
        {
          "datasource": {
            "name": "Expression",
            "type": "__expr__",
            "uid": "__expr__"
          },
          "expression": "$Solar + $Powerwall + $Grid\n\n",
          "hide": false,
          "refId": "Home Usage",
          "type": "math"
        }
      ],
      "thresholds": [],
      "timeRegions": [],
      "title": "Phase A",
      "tooltip": {
        "shared": true,
        "sort": 0,
        "value_type": "individual"
      },
      "transparent": true,
      "type": "graph",
      "xaxis": {
        "mode": "time",
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "$$hashKey": "object:359",
          "format": "watt",
          "label": "",
          "logBase": 1,
          "show": true
        },
        {
          "$$hashKey": "object:360",
          "decimals": 0,
          "format": "percent",
          "label": "",
          "logBase": 1,
          "max": "101",
          "min": "-101",
          "show": false
        }
      ],
      "yaxis": {
        "align": true
      }
    },
    {
      "aliasColors": {
        "Charge": "red",
        "Grid Usage": "purple",
        "Home Usage": "blue",
        "Powerwall": "green",
        "Solar Energy": "yellow",
        "Sun altitude": "rgb(192, 192, 192)"
      },
      "bars": false,
      "dashLength": 10,
      "dashes": false,
      "datasource": {
        "type": "datasource",
        "uid": "-- Mixed --"
      },
      "fieldConfig": {
        "defaults": {
          "links": []
        },
        "overrides": []
      },
      "fill": 3,
      "fillGradient": 0,
      "gridPos": {
        "h": 9,
        "w": 7,
        "x": 7,
        "y": 0
      },
      "hiddenSeries": false,
      "id": 3,
      "interval": "",
      "legend": {
        "alignAsTable": false,
        "avg": false,
        "current": true,
        "max": false,
        "min": false,
        "rightSide": false,
        "show": true,
        "total": false,
        "values": true
      },
      "lines": true,
      "linewidth": 1,
      "nullPointMode": "connected",
      "options": {
        "alertThreshold": true
      },
      "percentage": false,
      "pluginVersion": "9.1.2",
      "pointradius": 2,
      "points": false,
      "renderer": "flot",
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "alias": "Solar Energy",
          "datasource": {
            "type": "influxdb",
            "uid": "ajmBhJ74k"
          },
          "groupBy": [
            {
              "params": [
                "5m"
              ],
              "type": "time"
            },
            {
              "params": [
                "0"
              ],
              "type": "fill"
            }
          ],
          "hide": false,
          "measurement": "solarmeter",
          "orderByTime": "ASC",
          "policy": "raw",
          "refId": "Solar",
          "resultFormat": "time_series",
          "select": [
            [
              {
                "params": [
                  "real_power_b"
                ],
                "type": "field"
              },
              {
                "params": [],
                "type": "mean"
              }
            ]
          ],
          "tags": [],
          "tz": "Australia/Hobart"
        },
        {
          "alias": "Grid Usage",
          "datasource": {
            "type": "influxdb",
            "uid": "ajmBhJ74k"
          },
          "groupBy": [
            {
              "params": [
                "5m"
              ],
              "type": "time"
            },
            {
              "params": [
                "0"
              ],
              "type": "fill"
            }
          ],
          "hide": false,
          "measurement": "sitemeter",
          "orderByTime": "ASC",
          "policy": "raw",
          "query": "SELECT mean(from_grid) - mean(to_grid) FROM autogen.http WHERE $timeFilter GROUP BY time(5m) tz('Australia/Hobart')",
          "rawQuery": false,
          "refId": "Grid",
          "resultFormat": "time_series",
          "select": [
            [
              {
                "params": [
                  "real_power_b"
                ],
                "type": "field"
              },
              {
                "params": [],
                "type": "mean"
              }
            ]
          ],
          "tags": [],
          "tz": "Australia/Hobart"
        },
        {
          "datasource": {
            "name": "Expression",
            "type": "__expr__",
            "uid": "__expr__"
          },
          "expression": "$Solar + $Grid\n\n",
          "hide": false,
          "refId": "Home Usage",
          "type": "math"
        }
      ],
      "thresholds": [],
      "timeRegions": [],
      "title": "Phase B",
      "tooltip": {
        "shared": true,
        "sort": 0,
        "value_type": "individual"
      },
      "transparent": true,
      "type": "graph",
      "xaxis": {
        "mode": "time",
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "$$hashKey": "object:359",
          "format": "watt",
          "label": "",
          "logBase": 1,
          "show": true
        },
        {
          "$$hashKey": "object:360",
          "decimals": 0,
          "format": "percent",
          "label": "",
          "logBase": 1,
          "max": "101",
          "min": "-101",
          "show": false
        }
      ],
      "yaxis": {
        "align": true
      }
    },
    {
      "aliasColors": {
        "Charge": "red",
        "Grid Usage": "purple",
        "Home Usage": "blue",
        "Powerwall": "green",
        "Solar Energy": "yellow",
        "Sun altitude": "rgb(192, 192, 192)"
      },
      "bars": false,
      "dashLength": 10,
      "dashes": false,
      "datasource": {
        "type": "datasource",
        "uid": "-- Mixed --"
      },
      "fieldConfig": {
        "defaults": {
          "links": []
        },
        "overrides": []
      },
      "fill": 3,
      "fillGradient": 0,
      "gridPos": {
        "h": 9,
        "w": 7,
        "x": 14,
        "y": 0
      },
      "hiddenSeries": false,
      "id": 4,
      "interval": "",
      "legend": {
        "alignAsTable": false,
        "avg": false,
        "current": true,
        "max": false,
        "min": false,
        "rightSide": false,
        "show": true,
        "total": false,
        "values": true
      },
      "lines": true,
      "linewidth": 1,
      "nullPointMode": "connected",
      "options": {
        "alertThreshold": true
      },
      "percentage": false,
      "pluginVersion": "9.1.2",
      "pointradius": 2,
      "points": false,
      "renderer": "flot",
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "alias": "Solar Energy",
          "datasource": {
            "type": "influxdb",
            "uid": "ajmBhJ74k"
          },
          "groupBy": [
            {
              "params": [
                "5m"
              ],
              "type": "time"
            },
            {
              "params": [
                "0"
              ],
              "type": "fill"
            }
          ],
          "hide": false,
          "measurement": "solarmeter",
          "orderByTime": "ASC",
          "policy": "raw",
          "refId": "Solar",
          "resultFormat": "time_series",
          "select": [
            [
              {
                "params": [
                  "real_power_c"
                ],
                "type": "field"
              },
              {
                "params": [],
                "type": "mean"
              }
            ]
          ],
          "tags": [],
          "tz": "Australia/Hobart"
        },
        {
          "alias": "Grid Usage",
          "datasource": {
            "type": "influxdb",
            "uid": "ajmBhJ74k"
          },
          "groupBy": [
            {
              "params": [
                "5m"
              ],
              "type": "time"
            },
            {
              "params": [
                "0"
              ],
              "type": "fill"
            }
          ],
          "hide": false,
          "measurement": "sitemeter",
          "orderByTime": "ASC",
          "policy": "raw",
          "query": "SELECT mean(from_grid) - mean(to_grid) FROM autogen.http WHERE $timeFilter GROUP BY time(5m) tz('Australia/Hobart')",
          "rawQuery": false,
          "refId": "Grid",
          "resultFormat": "time_series",
          "select": [
            [
              {
                "params": [
                  "real_power_c"
                ],
                "type": "field"
              },
              {
                "params": [],
                "type": "mean"
              }
            ]
          ],
          "tags": [],
          "tz": "Australia/Hobart"
        },
        {
          "datasource": {
            "name": "Expression",
            "type": "__expr__",
            "uid": "__expr__"
          },
          "expression": "$Solar + $Grid\n\n",
          "hide": false,
          "refId": "Home Usage",
          "type": "math"
        }
      ],
      "thresholds": [],
      "timeRegions": [],
      "title": "Phase C",
      "tooltip": {
        "shared": true,
        "sort": 0,
        "value_type": "individual"
      },
      "transparent": true,
      "type": "graph",
      "xaxis": {
        "mode": "time",
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "$$hashKey": "object:359",
          "format": "watt",
          "label": "",
          "logBase": 1,
          "show": true
        },
        {
          "$$hashKey": "object:360",
          "decimals": 0,
          "format": "percent",
          "label": "",
          "logBase": 1,
          "max": "101",
          "min": "-101",
          "show": false
        }
      ],
      "yaxis": {
        "align": true
      }
    }
  ],
  "refresh": "5m",
  "schemaVersion": 37,
  "style": "dark",
  "tags": [],
  "templating": {
    "list": []
  },
  "time": {
    "from": "now-6h",
    "to": "now"
  },
  "timepicker": {},
  "timezone": "",
  "title": "Phase Monitoring",
  "uid": "QVLJwpV4z",
  "version": 26,
  "weekStart": ""
}

[jasonacox]

I think this is a good example of something that could go into the "Dashboard Marketplace" we are discussing in #96

[BuongiornoTexas]

Agreed that it makes sense in the Marketplace given its one off hacky nature, and the requirement for manual updates. If we generalise it and the underlying queries in the future, it might make sense to shift to a library panel module - either way, until others express interest, it's not going to see more development :-).

[BuongiornoTexas]

I've now got a summary panel that breaks down the solar and grid energy flows.

I believe the two export and import values correspond to the values reported in vitals (total import/export) and the site/solar meters (net import/export). Still working on validating these.