Polling rate? can we do better than 12s?

[tmenguy]

Hi,

I'm on the latest 1.4 beta, SDongle via wired ethernet for connection. I use an automation to update the power meter and inverter active power sensor:

• alias: "Huawei Solar power data polling"
  trigger:
  • platform: time_pattern
    hours: ""
    minutes: ""
    seconds: "/4"
    action:
  • service: homeassistant.update_entity
    target:
    entity_id:
    - sensor.power_meter_active_power
    - sensor.inverter_active_power

and it seems I can't go really beyond 12s (looking at the update graphs and the concrete state update of the sensors) Is there someone that can go below? should I change the polling frequencies directly in the const.py ? I would like to get closer to 5s to implement some control algorithms.

Thanks!
BR
Thomas

[Roving-Ronin]

@tmenguy
...hold my beer ;-)
Seriously though, bit of a how long is a piece of string / can I nitro boost my car situation. How fast is also going to depend upon how many devices you have in your plant. i.e. If you only a single inverter + power meter, you might be able to go fast.... if your like my setup with 2 x Inverters + Meter + 38 Optimisers it will be a bit slower, as it needs to poll 41 devices with 484 entities between them.

Someone (you) mentioned getting 15 seconds previously, I'd be happy with that.... and remember on the flip side the HA Recorder is going to be saying all that data away and the Db is going to get big and slow.

[tmenguy]

Hi,

(first did I say that I love your avatar?)

I would like to implement a kind of EVCC like stuff directly as a custom component to handle my car smart charging ... AND my soon to come battery + some of my loads like pools, etc ( I have a Wallbox with a meter and ecosmart but it probably won't work well with the Huawei optimisation algorithms)

I have 40 optimizers , meter and one inverter (and soon a battery), so not so far from your setup.

By putting directly 5s in the const.py of the integration, I'm now down to 6 or 7 seconds ... but I still don't understand why it is faster this way than using the automations with trigger, @wlcrs if you have an idea?

I may go down a bit to in the code to see if I can squeeze a bit more calls, but I don't know what is the theoretical Modbus limits I may hit (and the real update period of the meter itself).

[wlcrs]

No idea. Mu guess is that the HA automation trigger process is not optimized for such high frequency updates.

Starting from v1.4.0 the integration will only poll the sensors that are effectively enabled in HA. You could look into disabling all but the most essential sensors of this integration and check what the performance you get that way. Afterwards you can try re-enabling sensors a few at a time to check what the performance hit is. Feel free to report back your findings :-)

[Roving-Ronin]

Hi Thijs,

So is there any way to disable all Optimisers, or not have them discovered/installed, to cut down on the amount of polling?

[tmenguy]

yes, thx a lot, I'll fiddle a bit and do a fork to see if we can have something of a higher frequency "for free" (like the Shelly interaction that is more around the second for their refresh)
Good point for the device/entity we can disable, will look at it too, thx

[tmenguy]

interesting findings, here:
-disabled all optimizers : no impact , and it is normal the new 1.40 is really well decoupled on this, the optimizers are pull every 5mn so no real impact
-I missed how the HA polling was implemented in the DataUpdateCoordinator but it is pretty simple : once you have finished your update you add the polling interval to the now, so you can't go faster than real polling interval = [your fetching and update time] + [setup polling interval] (I probably learn you nothing @wlcrs here)

When I set the update to 1s as polling interval I have a result of around a data point every 3s, probably meaning it takes for me around 2s to gather the data from the SDongle ... which seems a lot to me to be honest (it is a bit less to be honest here from my tests more between 1 and 2s, still not insignificant)

All in all I have a solution here :) 3 to 4s of update is plenty fast for what I need

@wlcrs : would it be worthwhile to you if I open a PR to add an option config flow to your component to select a polling interval in the UI configuration, or you think it is too dangerous for general use? (and if so I'll just hardcode stuff for my personal use)

[PulsarFX]

Hm. I changed the values, restarted HA and tested the fast automation with 2s and 5s. The GUI shows updates every 10s, though. On 2s the trigger header is always orange, while 5s has most of the time a blue header.
As I am reading only 4 sensors, traces shows execution times of only 0.0x seconds.

[tmenguy]

Hey, which « GUI » are you referring to?

[heinemannj]

@wlcrs : would it be worthwhile to you if I open a PR to add an option config flow to your component to select a polling interval in the UI configuration, or you think it is too dangerous for general use? (and if so I'll just hardcode stuff for my personal use)

To be honest and crystal clear:

• The mayority of huawei_solar users have already activated battery control via installer authorization - the most important benefit and feature of this beautiful integration!!!
• For controlling and changing inverter/batteries system parameters you (all user itself) are responsible for any change of system parameters
• Everybody using this integration must have an approbiate level of knowledge about interverter/batteries operations

=> Reading an additional little peace of documentation about polling parameters is peanuts compared to above ...

@wlcrs: PLEASE accept this merge request - It makes for the majority of your followers their life a little bit more easier.

[PulsarFX]

Hey, which « GUI » are you referring to?

ops, overlooked this one. I mean the sensors detail page, like:

it always resets at 10s, whilst the automation with call to service: homeassistant.update_entity is set to trigger each 5s.

[PulsarFX]

Found the issue: I had disabled polling by the integration and used my own automations instead.
With them disabled and default polling by the integration enabled, I get about real time updates to 5s updates.
I am using:

INVERTER_UPDATE_INTERVAL = timedelta(seconds=1)
POWER_METER_UPDATE_INTERVAL = timedelta(seconds=1)
ENERGY_STORAGE_UPDATE_INTERVAL = timedelta(seconds=1)
UPDATE_TIMEOUT = timedelta(seconds=29)

(in the meantime I managed to setup modbuxproxy and query against that)

i.e. the dtsu666 sensor is updating like crazy. 8-)

[LotharWoman]

I found this : https://solaranzeige.de/phpBB3/viewtopic.php?t=4400
Can this workaround be used here to make the queries faster and more reliable?

[heinemannj]

Polling via automation as proposed by @wlcrs is also not working fine for me.
Modifying directly the const.py is much more faster and more reliable.

To archive small update frequcencies without any HA error logs, I had to switch my network connection directly to the inverter WLAN (not to the dongle) => this helps a lot!!!

BUT after the switch to from v1.3.x to v1.4.x I recognize time issues: #767

To make this story short:

house_power: "{{ inv_active_power - pm_active_power }}"

Negative House Load (for my setup NOT possible) - potentially due to Poll/Timing mismatch between Inverter Active power and Power meter Active power:

NOT recognized in versions 1.3.x.

My proposal:

• dedicated update interval for all power related sensors (inverter, power-meter, batteries): interval 1-10s
• dedicated update interval for all energy related sensors (inverter, power-meter, batteries): interval 10-30s
• for remaining sensors (inverter, power-meter, batteries): interval 30-60s

Any ideas or proposals for the power sensors???

For the energy and all other sensor's it's not an issue.

[spamadamm]

I've managed stable refresh rate of 3 seconds for EMMA 02 + inverter. I don't have the battery at the moment. I suggest to increase Modbus RTU baudrate from 9600 to 19200, maybe that improves the pooling rate over Modbus TCP a bit. It has to be done on all your devices in order to work.

[LotharWoman]

Changing the baud rate is not a solution. It can reset due to configuration changes or updates. In the worst case scenario, various components no longer communicate with each other.

[tmenguy]

I submitted a PR for this (#828 ) for those who wants to test something, you can test with my repo (see the PR), it is based on the latest (as of today) code, so far so good for me on my install.

[Roving-Ronin]

@tmenguy Is that updated off the latest (v1.4.1) code, or v1.5.1 Alpha 1 ?

Edit: Assuming v.1.5.1 Alpha 1

Appears to work:

[heinemannj]

Unfortunately after installation of your repository the integrations goes into error and stops.
The first step of configuration workflow seems to be OK.

I will make additional debugging when I have a couple of hours free time.

[tmenguy]

@Roving-Ronin yes it is latest code base of the repo, while update it while we test and @wlcrs is ok (or not) to merge it

@heinemannj : where you on 1.4.1? Because the 1.5.x has some auto discovery code for the slaveIDs for me the "AUTO" for the slaveID was not working I had to put back my slaveID (1 in my case) and then it works. If you were in 1.5.x are you in serial or network? (I haven't tested serial....)

And by the way thanks you both for testing!

[heinemannj]

Actual I'm on the origional 1.5.0a1.
After understanding that this new version reacts different to the older versions I was also successful with your patched version!

Polling frequencies for all of the three devices as expected.

Power meter: 3sec
Inverter: 3-6sec
Batteries: 3-8sec

Increasing to 4 or 5 secs seems to be a good compromise.

Thanks for your work!

Actually I'm working on an prototype for an HEMS: https://github.com/heinemannj/huawei_solar_hems
Polling rates at 30sec are far away from the need of an Home Assistent (HA) Home Energy Management System (HEMS).

By the way:
I'm using the 3sec polling rates since more then 9 months without any problems.

[tmenguy]

As an HEMS you can look at mine : tmenguy/quiet-solar it is working for some months for me, EV charging, pool, some loads, etc

[LotharWoman]

I think with the new firmware 121 for the sDongle, no adjustment is necessary anymore.

[tmenguy]

Hum the polling rate is on the HA code part, and not dependent on the SDongle firmware to me, so not sure how to get a faster refresh without changing the HA part.

[spamadamm]

Unfortunately, it is not possible to use the SDongle with the EMMA 02 energy management system, so we still search for a solution.

[heinemannj]

Unfortunately the PR has been rejected by @wlcrs

As already stated above:

• Polling via automation as proposed by @wlcrs is not working fine for me.
• Modifying directly the const.py is much more faster and more reliable.
• Using above patch will simplyfy a lot and is better supportable.

My recommendation:

• The lowest polling time should be limited to 3 sec (From my perspective lower then 3sec makes no sense)

[heinemannj]

You are confusing me ...

There is already a stable workarround by modifying directly the const.py because it is much more faster and more reliable then using HA automations to decrease the Huawei Solar polling frequency.

Disadvantage of this workarround: You will need to apply this hack again and again - after each integration update ...

What file needs to be modified after applying above patch to go below predefined (silly) limits?

I've found nothing at this time.
Please clarify

[Roving-Ronin]

I was referring to IF this PR was included people would not have to di the workaround. It would only be (very few) people who would need to modify a version that includes this PR, after official updates, to solely (unofficially) change the update frequency minimum limitation (solely for their own HA instance).

[heinemannj]

For the moment we have the file: const.py
Which file needs to be modified in future???

If HEMS is common in future => to secure our earth => Demand of just in time actions will be increased!

[tmenguy]

No hard feelings it is Thijs code, I closed the PR will see if I prefer to maintain my repo, or revert back to edit may const.py :)

[heinemannj]

From my perspective maintaining your repro really make sense - the frequency of new Huawei Solar versions is not to high at this time.