Powerwall operation mode does not update from the cloud following HA restart

[jaymunro]

Version of the custom_component

3.12.2

Configuration

From core.config_entries

      {
        "entry_id": "<redacted>",
        "version": 1,
        "domain": "tesla_custom",
        "title": "<redacted>",
        "data": {
          "token": “<redacted>“,
          "access_token": "<redacted>",
          "expiration": 1685400615,
          "username": "<redacted>",
          "domain": "https://auth.tesla.com",
          "include_vehicles": true,
          "include_energysites": true,
          "initial_setup": false
        },
        "options": {},
        "pref_disable_new_entities": false,
        "pref_disable_polling": false,
        "source": "user",
        "unique_id": null,
        "disabled_by": null
      },

Describe the bug

Following HA restart the OperationMode sensor is always empty 'unknown' and does not update from the cloud, even after many hours (tested up to 11 hours).

Debug log

Debug log attached below but the following may be relevant.

Logger: homeassistant.helpers.entity
Source: helpers/entity.py:560
First occurred: 6:11:47 PM (1 occurrences)
Last logged: 6:11:47 PM

Entity select.my_home_operation_mode (<class 'custom_components.tesla_custom.select.TeslaEnergyOperationMode'>) is using self.async_update_ha_state(), without enabling force_update. Instead it should use self.async_write_ha_state(), please report it to the custom integration author.

home-assistant_tesla_custom_2023-05-29T06-46-22.585Z.log

[jaymunro]

Transferring this comment from #619 for reference:
This started May 27 at 4:12PM (NZST), just 2 days ago. Before that the sensor would populate correctly after a restart. Other powerwall tesla custom sensors are populating without issue.

[jaymunro]

Additionally, changes made to the setting in the official phone Tesla app do not propagate down to this custom_component.
Changes made in this custom_component DO propagate up to the cloud and are reflected in the official phone Tesla app.

[jaymunro]

Checking the time it is attempting to update the value against debug logs, I am seeing a "script error":
ERROR (MainThread) [frontend.is.latest.202305033] :0:0 Script error.

Working through this, I see other sensors updating with valid values at the same time, e.g. 'Grid charging'

The sensor is updated from 'unavailable' to 'Yes' at 11:21:34

However for the 'Operation Mode' sensor, it is updated from 'unavailable' to 'unknown' at the same time, 11:21:34

[jaymunro]

@alandtse , I see the response in the debug logs does not return an operationhow does one use the HA service 'Tesla Custom Integration: api' ?
Specifically what does one put in each field? The following does not work and the links are not really helpful unless you're familiar with the process. The following results in Failed to call service tesla_custom.api. Unknown error

command: "SITE_SUMMARY"

parameters: {
    "AUTH": true,
    "TYPE": "GET",
    "URI": "api/1/energy_sites/11xxxxx0-14-J--CNxxxxxxxx4J03Y/site_status"
  }

Some of the ID is redacted above.

[jaymunro]

Looking at the code in `services.yaml', I found the following:

api:
  # Description of the service
  description: Run an API command using controller.api. https://teslajsonpy.readthedocs.io/en/latest/teslajsonpy/teslajsonpy.html#teslajsonpy.Controller.api
  # Different fields that your service accepts
  fields:
    # Key of the field
    email:
      # Description of the field
      description: Email address (optional if only one account)
      # Example value that can be passed for this field
      example: "[email protected]"
      required: false
      selector:
        text:
    command:
      # Description of the field
      description: Command to run. See https://github.com/zabuldon/teslajsonpy/blob/master/teslajsonpy/endpoints.json
      # Example value that can be passed for this field
      example: "WAKE_UP"
      required: true
      selector:
        text:
    parameters:
      # Description of the field
      description: Parameters in a dictionary. `path_vars` replace variables in endpoints.json path. All others are passed directly to controller.api. For command parameters see https://tesla-api.timdorr.com/vehicle/commands.
      # Example value that can be passed for this field
      example: '{"path_vars": {"vehicle_id":"1"}, "wake_if_asleep":True}'
      required: true
      selector:
        object:

However, the example: does not show in the UI

But the above still produces the same error

[jaymunro]

Progressing after viewing in YAML mode but still errored.

[purcell-lab]

@jaymunro thanks for your details on this issue, I have also been having states disappear so hopefully switching to #606 will resolve this issue as well.

[jaymunro]

TLDR: I think the teslajsonpy code needs to change the API call from BATTERY_DATA to SITE_CONFIG to retrieve this data. I'd appreciate comments from people with more background on this than me.

Using debug logs I found that the data is being retrieved by API calls to BATTERY_SUMMARY and BATTERY_DATA, neither of which contain the operation mode. It seems this is defined in teslajsonpy in energy.py on lines 175-178

    @property
    def operation_mode(self) -> str:
        """Return operation mode."""
        return self._battery_data.get("operation")

It expects the operation mode to exist at the top level in battery_data, but it is not there, it is actually in the API call SITE_CONFIG under the property "default_real_mode" (not "operation"). This may have been an API change by Tesla (would be strange) or teslajsonpy using a deprecated API.

I determined this by using Developer Tools > Services with a call to service Tesla Custom Integration: api, and found:

BATTERY_DATA call response. (operation is totally absent)

{"response":{
    "site_name":"My Home",
    "energy_left":9567.05263157895,
    "total_pack_energy":13846,
    "grid_status":"Active",
    "backup":{
        "backup_reserve_percent":15,
        "events":[
            {"timestamp":"2023-01-08T00:07:31+13:00","duration":34626},
            {"timestamp":"2022-07-02T13:36:25+12:00","duration":21030}],
        "events_count":0,
        "total_events":0},
    "user_settings":{
        "storm_mode_enabled":true,
        "powerwall_onboarding_settings_set":true,
        "powerwall_tesla_electric_interested_in":false,
        "sync_grid_alert_enabled":false,
        "breaker_alert_enabled":false},
    "components":{
        "solar":true,
        "solar_type":"pv_panel",
        "battery":true,
        "grid":true,
        "backup":true,
        "gateway":"teg",
        "load_meter":true,
        "tou_capable":true,
        "storm_mode_capable":true,
        "flex_energy_request_capable":false,
        "car_charging_data_supported":false,
        "off_grid_vehicle_charging_reserve_supported":false,
        "vehicle_charging_performance_view_enabled":false,
        "vehicle_charging_solar_offset_view_enabled":false,
        "battery_solar_offset_view_enabled":true,
        "solar_value_enabled":true,
        "energy_value_header":"Energy Value",
        "energy_value_subheader":"Estimated Value",
        "show_grid_import_battery_source_cards":true,
        "backup_time_remaining_enabled":true,
        "battery_type":"ac_powerwall",
        "configurable":false,
        "grid_services_enabled":false,
        "customer_preferred_export_rule":"pv_only"},
    "installation_date":"2022-07-01T15:15:19+12:00",
    "power_reading":[{
        "timestamp":"2023-06-02T18:09:54+12:00",
        "load_power":920,
        "solar_power":0,
        "grid_power":19,
        "battery_power":910,
        "generator_power":0}],
    "battery_count":1}}

SITE_CONFIG call response. (Note the 4th property "default_real_mode":"autonomous")

{
    "response":{
        "id":"<redacted>",
        "site_name":"My Home",
        "backup_reserve_percent":15,
        "default_real_mode":"autonomous",
        "installation_date":"2022-07-01T15:15:19+12:00",
        "user_settings":{
            "storm_mode_enabled":true,
            "powerwall_onboarding_settings_set":true,
            "powerwall_tesla_electric_interested_in":false,
            "sync_grid_alert_enabled":true,
            "breaker_alert_enabled":false},
        "components":{
            "solar":true,
            "solar_type":"pv_panel",
            "battery":true,
            "grid":true,
            "backup":true,
            "gateway":"teg",
            "load_meter":true,"tou_capable":true,
            "storm_mode_capable":true,
            "flex_energy_request_capable":false,
            "car_charging_data_supported":false,
            "off_grid_vehicle_charging_reserve_supported":false,
            "vehicle_charging_performance_view_enabled":false,
            "vehicle_charging_solar_offset_view_enabled":false,
            "battery_solar_offset_view_enabled":true,
            "solar_value_enabled":true,
            "energy_value_header":"Energy Value",
            "energy_value_subheader":"Estimated Value",
            "energy_service_self_scheduling_enabled":true,
            "show_grid_import_battery_source_cards":true,
            "set_islanding_mode_enabled":true,
            "wifi_commissioning_enabled":true,
            "backup_time_remaining_enabled":true,
            "battery_type":"ac_powerwall",
            "configurable":true,
            "grid_services_enabled":false,
            "customer_preferred_export_rule":"pv_only",
            "edit_setting_permission_to_export":true,
            "edit_setting_grid_charging":true,
            "edit_setting_energy_exports":true},
        "version":"23.4.2-1 fe55682a",
        "battery_count":1,
        "tou_settings":{
            "optimization_strategy":"economics",
            "schedule":[
                {
                    "target":"off_peak",
                    "week_days":[1,0],
                    "start_seconds":75600,
                    "end_seconds":0},
                {
                    "target":"peak",
                    "week_days":[1,0],
                    "start_seconds":0,
                    "end_seconds":75600}
            ]},
        "nameplate_power":5000,
        "nameplate_energy":13500,
        "installation_time_zone":"Pacific/Auckland",
        "max_site_meter_power_ac":14.1,
        "min_site_meter_power_ac":-1000000000,
        "geolocation":{
            "latitude":-43.501566,
            "longitude":172.5709272},
        "address":{
            "address_line1":"<redacted>",
            "city":"<redacted>",
            "state":"<redacted>",
            "zip":"<redacted>",
            "country":"NZ"            
        }}}

I tested the default_real_mode by changing the mode in the Tesla iOS app and the tesla_custom API service call to change the operation mode (still works).

Tesla app

Operation Mode     default_real_mode
Self-Powered       "self_consumption"
Time-Based Control "autonomous"
Backup             [no longer exists in the UI]

tesla_custom select. setting

Operation Mode     default_real_mode
Self-Powered       "self_consumption"
Time-Based Control "autonomous"
Backup             "self_consumption" ("backup_reserve_percent" is changed to 100)

[jaymunro]

I’ve opened a fork to start a PR, but first need to test. Does anyone have any pointers on how to get changed teslajsonpy code into HA for testing a change with tesla_custom?