Informational - integration with Home Assistant

[lmamakos]

HI! I'm a new genmon user, only a couple weeks now. I'm using a Pi Zero W with a USB attached WiFi and external antenna. Maybe some photos of the build later.

I recently did some work to get the open-source Home Automation software I use, called Home Assistant to integrate with genmon. I used the JSON interface in genserv.py as the means to get the good stuff out, and then some fancy sensor configuration in Home Assistant to parse out the JSON values I was interested in. DIdn't have write one line of code or hack on anything in genmon to solve my problem. See: https://community.home-assistant.io/t/monitor-your-generac-generator-with-home-assistant/62701 on the Home Assistant community forum where I put an example of this out there for others to use/learn from.

If you're looking to do more complex automation and integrate monitoring and control of lots of other types of things, you should seriously look at Home Assistant for that purpose. It integrates with all sorts of devices/products and is easy to do your own thing. And there's a bunch of UI options available, both as part of the basic distribution, to third-party contributions that target, e.g., tablet displays to hang on the wall. Cool stuff, and even my wife likes having it in the house, controlling the lighting (mostly Z-Wave, some Sonoff Wi-Fi switches, some X-10 powerline modules connected off my alarm system, which it also talks to and can see zone states, etc.)

Anyway, back to genmon..

The only suggestion I'd have from going through this is to have some additional JSON fields with strictly numeric values, rather than formatted as "242 V", where I need to strip off the units before doing a conversion to a numeric value. Happily, Home Assistant has a jinja2 template engine embedded in it, with some facilities for doing just that sort of hacking around. E.g.,

value_template: '{{ states.sensor.genmon_monitor_json.attributes["Monitor"]["Platform Stats"]["CPU Temperature"] | regex_replace(" F","") | float | round(1) }}'

Thanks for building this really useful software and sharing it.

[proppilot]

This is very interesting to me. My home automation is done via a Vera Plus and I have been noodling over ways to bring some monitor/control into that platform. I had considered using a Z-wave I/O module and using the GPIO in/out programs in the repository to provide some basic start/stop and status . I recently (like Thursday) even became aware of the Home Assistant project/platform and plan to play around with it as it integrates fairly nicely with Vera so I like what you have done here. If I was aware of HA prior to my adoption a few weeks ago, I probably would have gone that route as I like playing with this stuff. Thanks for posting the information. I have been a Genmon user for just under a year and have really seen the contributions/additions and development accelerate quite rapidly in that time as it is gaining popularity.

[lmamakos]

I happened across Home Assistant almost 2 years ago. For probably 10 years or more, I've had a couple of dozen temperature sensors, and then later, power use sensors in my home. And some X-10 PLC lighting (well, this goes back a few years! :-) I hacked up all sorts of scripts to shove the interesting data into a database and then some ugly RRD-based graphs to present it.

I had built a very simple macro capability so that when this happened over here, turn that light on over there. When I went looking for a better solution to integrate my sensors, I found Home Assistant and was overwhelmed and astonished at how capable and well-designed it is! It nicely complemented the work I had done, and it was very easy to integrate with. I really liked have a "neutral" component in the middle of things, rather than having to buy into one platform or another.

To illustrate - my Home Assistant has an experimental connection to my Elk alarm system. I've been collaborating/supporting someone that's been working on this, and it will eventually get contributed into HA when it's mature enough. In any case, I have access to all of the "zones" in the alarm system - wired sensors on doors and windows, and PIR motion detectors. I've also got some Wi-Fi Sonoff switch modules, so cheap at less than $10 each! And I have some Z-Wave switches for some lighting, as well as just a few X-10 PIR motion floodlights that are driven from the alarm system as well.

All these things get exposed as generic "entities" of various classes in HA, either lights or switches, or sensors. So I have an "automation" that when the door off the kitchen opens when the sun is less than 15 degrees above the horizon, turn on the floor light in the enclosed porch. What's remarkable is that the door opening is sensed by the alarm system, and the floor lamp is controlled by a Sonoff switch that's listening to an MQTT topic. HA lets me effortlessly span all these different technologies that some of these "hub" systems might limit you to.

I've another automation that when either the motion detector on the front porch senses motion, or the Z-Wave lock on the front door is unlocked, to turn on the X-10 floodlights, turn on the Z-Wave porch light, etc. Or a similar automation when HA senses "presence" because it can poll my wireless network and notice the MAC address of my phone connect when I pull up in front of the house in my car.

This is wonderful stuff! And there's a thriving, active community of users and contributors. See also the community web site at https://community.home-assistant.io/ to get sense of the activity going on.

If you're contemplating doing something clever based on genmon, rather than hacking up one-off scripts, it might be worth investing in a general purpose solution with HA. It's easy to get started with a Raspberry Pi and a docker-based deployment for easy upgrades, call HASS.IO

[jgyates]

@lmamakos

This is great stuff. I will look into making a more exportable JSON formatted data as you describe. It sounds pretty straightforward, I just need to find the best way work it into the code.

[baboss80]

I use SmartThings for home automation and would like to integrate my Pi/Genmon into it, not so much for control but for status reporting of the generator in the SmartThings app. I've just started looking into it and am not much of a developer but can muck my way through other's examples.

[proppilot]

baboss80 a simple solution for you (and I am even considering) is the Fortrezz Mimo2+ z-wave module. It has two input and two output channels. You could hang it off your ST (or my Vera) and using the GPIO routine on this project, input two states such as READY and RUNNING and you could input the START/TRANSFER and STOP commands so you could initiate a start/transfer and stop from your ST.

Three would be perfect as I would go READY/RUNNING as inputs and START/START TRANSFER/STOP as outputs. I am still noodling around for a z-wave i/o module with a few more inputs and am going to look seriously at the HA project as a means to do it as it looks like Vera and HA can chat. The more I think about it I could probably use my Genmon Pi to host HA as well just for this specific functionality.

I anticipate some more late nights playing..... ;-)

[USAFPride]

@lmamakos, I've managed to install your package into HA. While I'm a complete NOOB with HA and you are currently at an Alpha stage with your packages, I'm wondering if the best approach is to have Genmon have the ability to send the data to MQTT server? This way the data is being pushed vs. pulled from the generator.

[lmamakos]

@USAFPride , I think it would be interesting to be able to push data via MQTT and get a more prompt response vs. polling for the current state. I think that would be most appropriate if there was some mechanism inside of genmon that had some way to push updates out to clients. Perhaps there is; I just am not familiar enough with the structure of genmon and the assorted libraries.

I started down the path of thinking about making MQTT-based sensors in Home Assistant, and I figure dI'd hack up some sort of daemon I could run to push out the MQTT messages. I took a look at gengpio.py for instance, to see if I could adapt that structure, but ultimately it still polls for state changes. Ideally, there'd be some sort of pubsub pattern inside of the main genmon.py script that polls the generator at whatever frequency that could be a way to hooking for push updates.

In any case, I wasn't really sure what needed to be done and it seemed pretty easy to get started by polling remotely via the web interface. This insulated me from any custom code on the RPi running genmon, as well as possibly any custom component in Home Assistant as well. Ultimately, I wish I polled more frequently and I may try that using the current approach.

[USAFPride]

@jgyates, where would be the best place for the data to be sent out? Genmon is constantly receiving data from the modbus and is being processed continuously, so it's just a matter of knowing where the data is already formatted, adding an if mqtt, and sending the data out at that point. Obviously there would need to be some configuration additions (use MQTT, server, port, etc.), but most of the MQTT transmission is already handled by a python module.

The benefit of the push vs. pull is that if Genmon is off line, the requesting program doesn't need to handle the communication error. The data just isn't there to be rebroadcast by MQTT to the listeners.

[jgyates]

If genmon is running, it is continuously making requesting to the generator for the internal modbus registers (i.e. genmon polls the generator). These registers are held in memory and some basic state is monitored (switch state, engine state and line voltage), if a state change is detected then emails will be sent . The remaining data is not "formatted" until is is requested by an external app like the web interface.

The generator does not push data. It is a modbus slave device so it is silent until data is requested. It is certainly possible to make a layer that pushes data, but that layer would br polling on one end and pushing on the other.

I would think the cleanest method would be to have an external program poll genmon every 1- 2 seconds, change the data as needed, and push it to MQTT.

[jgyates]

also, the /genmonlib/mynotify.py module uses callbacks that could be used for a push type situation. At present it only notifies on state change (running, exercise, stopped, etc)

[lmamakos]

So just for clarity, when one of the external application polls genmon (e.g, like the GPIO script, etc.), that's just retrieving the data out of state inside of genmon? My concern was there being some connection between the individual polls, and causing more traffic on the modbus channel to the generator.

If my understanding is correct, then its probably reasonable to bang away with polls every second to detect changing state and then act on it. While perhaps not quite as "elegant", the practical aspect of it is that specific activity is nicely isolated from the core of genmon.

[jgyates]

@lmamakos ,

Correct. Calling into genmon as fast as the CPU / network will support will not cause additional modbus traffic. The modbus polling is in a separate thread so the only thing that will affect modbus traffic is if the CPU utilization is high and this would cause the modbus reads to be slowed. You can monitor the modbus packets per second and average transaction time on the Monitor page from the web UI.

To give you an idea of throughput, if you have the web interface up the javascript calls into genserv then to genmon about once every 5 seconds to update the page and every 3 seconds to determine the base status. You could poll faster than this but you will eventually reach a point of diminishing returns (around 0.5 seconds would be my guess) since it takes time to read all of the registers.

[lmamakos]

@jgyates ,
Thanks, that's good to know.

I have some desire to spin a small board to piggyback on the Raspberry Pi GPIO connector, and include 3 or 4 thermocouple interfaces and an RS-232/TTL converter for the modbus connection as well. I'd like to be able to monitor some internal temperatures, like the air coming in, the air exhaust, perhaps the oil temperature measured at the oil cooler. As that will need a bit of software to manage, that might also be an opportunity to also include polling the genmon data, and publishing it all via MQTT.

That's nothing but a crazy idea at this point, so something for the back burner. There's already some open-source work in this area that would be a great starting point: https://github.com/mikelawrence/RPi-pHat-Thermocouple

[USAFPride]

@jgyates

I would think the cleanest method would be to have an external program poll genmon every 1- 2 seconds, change the data as needed, and push it to MQTT.

This makes sense. A simple cron/python script is simple enough to be added. Is there currently a way to get all the "data" at once vs. issuing separate calls (eg. status, maint, outage)? If not, could that be implemented?

[jgyates]

I have something working now that imports any changed values from genmon (except the full alarm, start/stop, maintenance log) into a mqtt broker. I just need to put some finishing touches on it and do some more test. I should be able to check it in in a few day.

[USAFPride]

@jgyates Awesome. No rush on my part as I won't be able to test anything until Friday night at the earliest.

[jgyates]

I have added a new program named genmqtt.py to the repository to support importing genmon data into a MQTT server. The program is described here

Let me know if there are any issues. I tested this with the MQTT broker from http://mosquitto.org/ and node-red.

[proppilot]

Wow. this is getting wings. There is a Plugin for the VERA to read MQTT data. Would the RPi with Genmon running need a MQTT broker (like Mosquitto) running ? This is new territory for me but super interesting and something I want (need !!) to implement between my Genmon Rpi and my Vera !

[jgyates]

I am not too familiar with VERA, but a quick search turned up this site (https://github.com/jonferreira/vera-mqtt). This appears to be a MQTT client that can push data to a MQTT broker. This is essentially what the genmqtt.py program is doing (pushing to a MQTT server. Some home automation programs have a MQTT broker underneath the hood or have the ability to read and post to one as a way of exchanging data. Home Assistant and OpenHab.org are two that I know of will support this but I am sure there are others. I use another pi to monitor my home energy usage from https://openenergymonitor.org/. This openenergy monitor has a MQTT backend and it also comes with node-red and openhab built in.

[USAFPride]

@proppilot, a MQTT broker would be needed somewhere in your system. I'd recommend not using the Genmon rPi as those tend to be located in less than ideal locations. A quick googlefu shows that VERA does not have a built in broker, so you will need another device, but it can read MQTT from a broker. Hopefully you have another rPi in your house somewhere.

@jgyates, thanks for the addition. I'll try to test today, but don't know how in depth I can be for a week.

[USAFPride]

the client_id field needs to be implemented. If not declared, it is random each time it starts:
Received PUBLISH from e8f608a7-b3a7-4875-bed3-10088b03b830
Received PUBLISH from 52304f79-4ec0-43c4-91db-d4a9c4ee25ff

It's definitely chatty, but exactly what I was hoping for!!!

[jgyates]

I checked in a new version that has a static client id of "genmon".

[USAFPride]

Much better. Thanks!!! @lmamakos, This will make your HA setup that much easier!!

HA setup:

sensor linevoltage:
  - platform: mqtt
    state_topic: "generator/Status/Line State/Utility Voltage"
    name: 'Utility Voltage'

sensor batteryvoltage:
  - platform: mqtt
    state_topic: "generator/Status/Engine/Battery Voltage"
    name: 'Battery Voltage'

sensor cpu_utilization:
  - platform: mqtt
    state_topic: "generator/Monitor/Platform Stats/CPU Utilization"
    name: 'CPU'
    unit_of_measurement: "%"

sensor wlan_signal_quality:
  - platform: mqtt
    state_topic: "generator/Monitor/Platform Stats/WLAN Signal Quality"
    name: 'WLAN Signal Quality'

[lmamakos]

I'll need to use a value_template in the MQTT sensor to parse out the number so that my sensor is a number and not a text string. I feed all that sensor data into InfluxDB and Grafana so I can make pretty graphs and stuff..

I'm about to leave town on a business trip of a few days, so to avoid wrath from my wife about home automation stuff going weird, I'm in a "Change Freeze" until I return, when I'll be able to give this a try..

Thanks @jgyates for building this!

[kalex1114]

Wow nice. I'm in the process of migrating from smartthings to home assistant

[darkstar2002]

I'll need to use a value_template in the MQTT sensor to parse out the number so that my sensor is a number and not a text string. I feed all that sensor data into InfluxDB and Grafana so I can make pretty graphs and stuff..

I'm about to leave town on a business trip of a few days, so to avoid wrath from my wife about home automation stuff going weird, I'm in a "Change Freeze" until I return, when I'll be able to give this a try..

@lmamakos could you give a little more detail on how you are getting the stuff into influxdb? That is my next goal. I have 3 gens that I want to graph.

[lmamakos]

@lmamakos could you give a little more detail on how you are getting the stuff into influxdb? That is my next goal. I have 3 gens that I want to graph.

As mentioned before, I'm using Home Assistant in my setup at home to capture and manage the stats from genmon via MQTT Sensors. Within the Home Assistant platform, you can have it log the state of some (or all) of the Home Assistant entities in InfluxDB without having to do anything special for the genmon-specific data using the InfluxDB component.

One thing to keep in mind is that it will be much more useful to log numeric values as, well number and not just the text strings that appear in the MQTT payloads. Then you can do all sorts of useful queries in InfluxDB (and, Grafana for presentation). So, in order to do this, my sensor entities in Home Assistant look sort of like this:

sensor:
  - platform: mqtt
    name: Genmon Generator Output Voltage
    friendly_name: 'Generator Output Voltage'
    unit_of_measurement: "VAC"
    state_topic: '19916/generator/Status/Engine/Output Voltage'
    value_template: '{{ value | regex_replace(" V","") | float | round(1) }}'

The thing to note here is the use of the value_template item which is passed the MQTT payload and is transformed by the jinja2 templating language that's used inside of Home Assistant to manipulate the form of data in and out of Home Assistant.

In this example, value is the MQTT payload. For the Voltage measurement, the MQTT payload looks like:

234 V

The template express takes that text string, pushes it through the regex_replace filter to remove the trailing V characters. Then the remaining text string is pushed through the float filter which converts the text into a floating point value. Finally, it's pushed through the round filter to round the value to one decimal digit of precision.

This is an important transformation because data in InfluxDB is typed, and you want these values to be numbers, not text.

Another example:

  - platform: mqtt
    name: Genmon Generator Output Power
    friendly_name: 'Generator Output Power'
    unit_of_measurement: "W"
    state_topic: '19916/generator/Status/Engine/Output Power (Single Phase)'
    # this is normalized to watts from kilowatts
    value_template: '{{ value | regex_replace(" kW","") | float * 1000 | round(0) }}'

Similar to before, except getting rid of the kW suffix, converting it to a float, and then multiplying by 1000 to convert to watts as the unit. For me, this matches other power measurements that I make that are normalized to watts.

There's other stuff that actually are text strings that I don't do that kind of fancy conversion, and I omit the value_template from the entity definition in Home Assistant

  - platform: mqtt
    name: Genmon Generator Last Alarm Log
    friendly_name: 'Generator Last Alarm'
    state_topic: '19916/generator/Status/Last Log Entries/Logs/Alarm Log'

Hope this helps!

[jgyates]

I checked in a change that is ready to test. It adds two new options (available on the Add Ons page)

Full path to CA cert file
TLS version (default is 1)

Let me know how this goes. If there is an error it should log to /var/log/genmqtt.log

[liltux]

@jgyates I tried without then with ca file and no connection. Submitted logfiles, let me know if you recieved.

[jgyates]

Got it, looks like the TLS version parameter is an int not a float. I checked in an update. Try it now. I did not bump the version info but clicking update will still work.

[jgyates]

i.e version is still 1.11.31

[liltux]

Still not connecting. sent new logfiles.

[jgyates]

I got your new logs. I checked in a new version. Let me know if this works.

[liltux]

Let me look into the CA file. I may need to fix my file. I will let you know.

[jgyates]

The call to setup SSL/TLS looks like this:

tls_set(ca_certs=None, certfile=None, keyfile=None, cert_reqs=ssl.CERT_REQUIRED, tls_version=ssl.PROTOCOL_TLS, ciphers=None)

I am not 100% sure about the cert_reqs parameter. It should either be CERT_NONE, CERT_OPTIONAL, or CERT_REQUIRED (default is CERT_REQUIRED), depending on the server config. Here is ther SSL doc for the three options.

https://docs.python.org/3/library/ssl.html#ssl.CERT_NONE

[jgyates]

I checked in an update that give you the options of require,none and optional

[liltux]

@jgyates if it will help I followed this to setup and configure my MQTT broker:
https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-the-mosquitto-mqtt-messaging-broker-on-ubuntu-16-04
So far no connection.

[jgyates]

I will look at the api some more. I should probably get a cert and try to do some testing. I could use it with some other stuff I am running ..

[liltux]

Here is my NodeRed Config...

[kalex1114]

@lmamakos I'm using your configs and looks like some of your configs are going to be deprecated later on.

[lmamakos]

Yes, time marches on.. Here's my present configuration, running with version 0.91.2 of Home Assistant. Of course, you'll need to tweak the MQTT topic names because I happen to use some non-defaults to align with some existing conventions.

################################################
## generator sensors
################################################

sensor:
  - platform: mqtt
    state_topic: '19916/generator/Outage/Status'
    name: Genmon Outage Status

  - platform: mqtt
    state_topic: '19916/generator/Outage/Utility Voltage'
    name: Genmon Outage Utility Voltage
    value_template: '{{ value | regex_replace(" V", "") | float | round(1) }}'
    unit_of_measurement: "VAC"

  # status related sensors
  - platform: mqtt
    name: Genmon Generator Battery
    state_topic: '19916/generator/Status/Engine/Battery Voltage'
    unit_of_measurement: "V"
    value_template: '{{ value | regex_replace(" V", "") | float }}'

  - platform: mqtt
    name:  Genmon Generator RPM
    unit_of_measurement: "RPM"
    state_topic: '19916/generator/Status/Engine/RPM'
    value_template: '{{ value | int }}'

  - platform: mqtt
    name: Genmon Generator Frequency
    unit_of_measurement: "HZ"
    state_topic: '19916/generator/Status/Engine/Frequency'
    value_template: '{{ value | regex_replace(" Hz","") | float }}'

  - platform: mqtt
    name: Genmon Generator Output Voltage
    unit_of_measurement: "VAC"
    state_topic: '19916/generator/Status/Engine/Output Voltage'
    value_template: '{{ value | regex_replace(" V","") | float | round(1) }}'

  # current measrement is garbage on 22KW product, apparently
  - platform: mqtt
    name: Genmon Generator Output Current
    unit_of_measurement: "A"
    state_topic: '19916/generator/Status/Engine/Output Current'
    value_template: '{{ value | regex_replace(" A","") | float | round(1) }}'

  - platform: mqtt
    name: Genmon Generator Output Power
    unit_of_measurement: "W"
    state_topic: '19916/generator/Status/Engine/Output Power (Single Phase)'
    # this is normalized to watts from kilowatts
    value_template: '{{ value | regex_replace(" kW","") | float * 1000 | round(0) }}'

  - platform: mqtt
    name: Genmon Generator Last Alarm Log
    state_topic: '19916/generator/Status/Last Log Entries/Logs/Alarm Log'

  - platform: mqtt
    name: Genmon Generator Last Service Log
    state_topic: '19916/generator/Status/Last Log Entries/Logs/Service Log'

  - platform: mqtt
    name: Genmon Generator Last Action
    state_topic: '19916/generator/Status/Last Log Entries/Logs/Start Stop Log'

    # maint related sensors
  - platform: mqtt
    name: Genmon Generator Capacity
    state_topic: '19916/generator/Maintenance/Rated kW'
    # normalize to watts
    value_template: '{{ value | float * 1000 | round(1) }}'
    unit_of_measurement: 'W'

  - platform: mqtt
    name: Genmon Generator Exercise Time
    state_topic: '19916/generator/Maintenance/Exercise/Exercise Time'
    # might be nice to maybe somehow normalize this, but not sure into what date/time representation
    # value is something like:  "Weekly Sunday 17:35 Quiet Mode On"

  # genmon monitor related sensors
  - platform: mqtt
    name: Genmon Mon CPU Temperature
    unit_of_measurement: "°F"
    device_class: temperature
    state_topic: '19916/generator/Monitor/Platform Stats/CPU Temperature'
    value_template: '{{ value | regex_replace(" F","") | float | round(1) }}'

  - platform: mqtt
    name: Genmon Mon Uptime
    # maybe parse this out into minutes/hours?
    state_topic: '19916/generator/Monitor/Platform Stats/System Uptime'

  - platform: mqtt
    name: Genmon Mon Load
    unit_of_measurement: '%'
    state_topic: '19916/generator/Monitor/Platform Stats/CPU Utilization'
    value_template: '{{ value | regex_replace(" *%","") | float | round(1) }}'

  - platform: template
    sensors:
      generator_outage_summary:
        entity_id:
          - sensor.genmon_outage_status
          - sensor.timep
        value_template: >-
          {% if states.sensor.genmon_outage_status.state | regex_match('System in outage') %}
             {% set duration = (as_timestamp(now()) - as_timestamp(states.sensor.genmon_outage_status.state)) | regex_replace('System in out
age since ') | int %}
             {% set hours = duration // 3600 %}
             {% set duration = duration - hours * 3600 %}
             {% set minutes = duration // 60 %}
             {% set seconds = duration % 60 %}
             Outage active for {{ '%d:%02d:%02d' | format(hours, minutes, seconds) }}
          {% else %}
            {{ states.sensor.genmon_outage_status.state }}
          {% endif %}
#
#  probably the most useful sensor - are we in a power outage condition or not?
#
binary_sensor:
  - platform: mqtt
    name: Genmon Power Outage
    state_topic: '19916/generator/Outage/System In Outage'
    payload_on: 'Yes'
    payload_off: 'No'

automation:
  - alias: 'start power outage'
    initial_state: 'on'
    trigger:
      - platform: state
        entity_id: binary_sensor.genmon_power_outage
        from: 'off'
        to: 'on'
        # so ideally wait a little bit for the generator to time out the utility power
        # loss, start and operate the automatic transfer switch from utility to generator
        # network equipment should be a UPS.. hopefully ISP connectivity is still working
        for:
          seconds: 45
    action:
      - service: notify.pushover_louie
        data:
          message: "Utiliy Power Loss at {{  now().strftime('%Y-%m-%d %H:%M:%S') }}"
          title: "HASS"

# maybe also initiate a zwave heal network or something after power is restored to
# from standby generator after a ~45 second interruption?  Switch back is pretty
# quick, with only a couple hundred millisecond transfer time in the ATS..

#
  - alias: end power outage
    initial_state: true
    trigger:
      - platform: state
        entity_id: binary_sensor.genmon_power_outage
        from: 'on'
        to: 'off'
    action:
      - service: notify.pushover_louie
        data:
          message: "Transfer back to utiliy power at {{  now().strftime('%Y-%m-%d %H:%M:%S') }}"
          title: "HASS"

[GlennGoddard]

I know this is closed; but would adding the MQTT topics from the below link add what we need to Home Assistant. I read above that the client_id was added but I can not find that in my MQTT topics.
https://www.home-assistant.io/docs/mqtt/discovery/

[mefbgil]

I'll need to use a value_template in the MQTT sensor to parse out the number so that my sensor is a number and not a text string. I feed all that sensor data into InfluxDB and Grafana so I can make pretty graphs and stuff..

I'm about to leave town on a business trip of a few days, so to avoid wrath from my wife about home automation stuff going weird, I'm in a "Change Freeze" until I return, when I'll be able to give this a try..

Thanks @jgyates for building this!

I'll need to use a value_template in the MQTT sensor to parse out the number so that my sensor is a number and not a text string. I feed all that sensor data into InfluxDB and Grafana so I can make pretty graphs and stuff..

I'm about to leave town on a business trip of a few days, so to avoid wrath from my wife about home automation stuff going weird, I'm in a "Change Freeze" until I return, when I'll be able to give this a try..

Thanks @jgyates for building this!

@ lmamakos - can you share your InfluxDB and Grafana configs? I'd love to do what you've done!

[lmamakos]

@mefbgil - I don't have any specific configuration for InfluxDB. I use the Home Assistant InfluxDB integration that feeds sensor entity state changes in Home Assistant into InfluxDB.

I've built a couple of dashboards in Grafana, though I don't know of a way to post their configuration since they're constructed from the Grafana GUI. Here is one dashboard that incorporates some data from genmon, as well as a bunch of other entities. The temperature sensors are extra sensors vs. data from the generator controller itself, and I have other power monitoring sensors associated with my main circuit breaker panel in there too. This may not be useful to anyone else..

[GlennGoddard]

You can save it as a JSON file. I don't have it running at the moment so I can't post a screen shot.
You can import this into Grafana.
Name it anything you want with the "json" extension and Grafana can import it.
{ "annotations": { "list": [ { "builtIn": 1, "datasource": "-- Grafana --", "enable": true, "hide": true, "iconColor": "rgba(0, 211, 255, 1)", "name": "Annotations & Alerts", "type": "dashboard" } ] }, "editable": true, "gnetId": null, "graphTooltip": 0, "id": 2, "links": [], "panels": [ { "datasource": "HomeAssistant", "fieldConfig": { "defaults": { "custom": {}, "decimals": 0, "mappings": [], "max": 257, "min": 223, "noValue": "0", "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": null }, { "color": "red", "value": 0 }, { "color": "green", "value": 228 }, { "color": "red", "value": 252 } ] } }, "overrides": [] }, "gridPos": { "h": 7, "w": 5, "x": 0, "y": 0 }, "id": 8, "options": { "reduceOptions": { "calcs": [ "lastNotNull" ], "fields": "/^Utility Voltage$/", "values": false }, "showThresholdLabels": false, "showThresholdMarkers": true }, "pluginVersion": "7.3.1", "targets": [ { "alias": "Utility Voltage", "groupBy": [ { "params": [ "$__interval" ], "type": "time" }, { "params": [ "null" ], "type": "fill" } ], "measurement": "V", "orderByTime": "ASC", "policy": "default", "refId": "A", "resultFormat": "time_series", "select": [ [ { "params": [ "value" ], "type": "field" }, { "params": [], "type": "last" } ] ], "tags": [ { "key": "friendly_name", "operator": "=", "value": "Utility Voltage" } ] } ], "timeFrom": null, "timeShift": null, "title": "Utility Voltage", "transparent": true, "type": "gauge" }, { "datasource": "HomeAssistant", "fieldConfig": { "defaults": { "custom": {}, "decimals": 0, "mappings": [], "max": 64, "min": 56, "noValue": "0", "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": null }, { "color": "red", "value": 0 }, { "color": "#EAB839", "value": 57 }, { "color": "green", "value": 58.5 }, { "color": "yellow", "value": 61.5 }, { "color": "red", "value": 63 } ] } }, "overrides": [] }, "gridPos": { "h": 7, "w": 5, "x": 5, "y": 0 }, "id": 12, "options": { "reduceOptions": { "calcs": [ "lastNotNull" ], "fields": "", "values": false }, "showThresholdLabels": false, "showThresholdMarkers": true }, "pluginVersion": "7.3.1", "targets": [ { "alias": "Hz", "groupBy": [ { "params": [ "$__interval" ], "type": "time" }, { "params": [ "null" ], "type": "fill" } ], "measurement": "Hz", "orderByTime": "ASC", "policy": "default", "refId": "A", "resultFormat": "time_series", "select": [ [ { "params": [ "value" ], "type": "field" }, { "params": [], "type": "last" } ] ], "tags": [ { "key": "entity_id", "operator": "=", "value": "generator_frequency" } ] } ], "timeFrom": null, "timeShift": null, "title": "Generator Frequency", "transparent": true, "type": "gauge" }, { "datasource": "HomeAssistant", "fieldConfig": { "defaults": { "custom": {}, "decimals": 0, "mappings": [], "max": 100, "min": 0, "noValue": "0", "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": null }, { "color": "red", "value": 0 }, { "color": "#EAB839", "value": 20 }, { "color": "green", "value": 30 }, { "color": "yellow", "value": 80 } ] } }, "overrides": [] }, "gridPos": { "h": 7, "w": 5, "x": 10, "y": 0 }, "hideTimeOverride": true, "id": 14, "options": { "reduceOptions": { "calcs": [ "lastNotNull" ], "fields": "", "values": false }, "showThresholdLabels": false, "showThresholdMarkers": true }, "pluginVersion": "7.3.1", "targets": [ { "alias": "%", "groupBy": [ { "params": [ "$__interval" ], "type": "time" }, { "params": [ "null" ], "type": "fill" } ], "measurement": "%", "orderByTime": "ASC", "policy": "default", "refId": "A", "resultFormat": "time_series", "select": [ [ { "params": [ "value" ], "type": "field" }, { "params": [], "type": "last" } ] ], "tags": [ { "key": "entity_id", "operator": "=", "value": "tank_utility_002e001b3733373013473831" } ] } ], "timeFrom": "24h", "timeShift": null, "title": "Propane Tank", "transparent": true, "type": "gauge" }, { "datasource": "HomeAssistant", "fieldConfig": { "defaults": { "custom": {}, "decimals": 0, "mappings": [], "max": 257, "min": 223, "noValue": "0", "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": null }, { "color": "red", "value": 0 }, { "color": "green", "value": 228 }, { "color": "red", "value": 252 } ] } }, "overrides": [] }, "gridPos": { "h": 7, "w": 5, "x": 15, "y": 0 }, "id": 10, "options": { "reduceOptions": { "calcs": [ "lastNotNull" ], "fields": "/^Utility Voltage$/", "values": false }, "showThresholdLabels": false, "showThresholdMarkers": true }, "pluginVersion": "7.3.1", "targets": [ { "alias": "Utility Voltage", "groupBy": [ { "params": [ "$__interval" ], "type": "time" }, { "params": [ "null" ], "type": "fill" } ], "measurement": "V", "orderByTime": "ASC", "policy": "default", "refId": "A", "resultFormat": "time_series", "select": [ [ { "params": [ "value" ], "type": "field" }, { "params": [], "type": "last" } ] ], "tags": [ { "key": "friendly_name", "operator": "=", "value": "Generator Voltage" } ] } ], "timeFrom": null, "timeShift": null, "title": "Generator Voltage", "transparent": true, "type": "gauge" }, { "datasource": "HomeAssistant", "fieldConfig": { "defaults": { "custom": {}, "decimals": 0, "mappings": [], "max": 4100, "min": 3100, "noValue": "0", "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": null }, { "color": "red", "value": 0 }, { "color": "#EAB839", "value": 3240 }, { "color": "green", "value": 3420 }, { "color": "yellow", "value": 3780 }, { "color": "red", "value": 3960 } ] } }, "overrides": [] }, "gridPos": { "h": 7, "w": 5, "x": 0, "y": 7 }, "id": 9, "options": { "reduceOptions": { "calcs": [ "lastNotNull" ], "fields": "", "values": false }, "showThresholdLabels": false, "showThresholdMarkers": true }, "pluginVersion": "7.3.1", "targets": [ { "alias": "RPM", "groupBy": [ { "params": [ "$__interval" ], "type": "time" }, { "params": [ "null" ], "type": "fill" } ], "measurement": "RPM", "orderByTime": "ASC", "policy": "default", "refId": "A", "resultFormat": "time_series", "select": [ [ { "params": [ "value" ], "type": "field" }, { "params": [], "type": "last" } ] ], "tags": [ { "key": "entity_id", "operator": "=", "value": "generator_speed" } ] } ], "timeFrom": null, "timeShift": null, "title": "Generator Speed", "transparent": true, "type": "gauge" }, { "datasource": "HomeAssistant", "description": "", "fieldConfig": { "defaults": { "custom": {}, "decimals": 0, "mappings": [], "max": 1, "min": -99, "noValue": "0", "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": null }, { "color": "red", "value": 0 }, { "color": "green", "value": 1 } ] } }, "overrides": [] }, "gridPos": { "h": 7, "w": 5, "x": 15, "y": 7 }, "hideTimeOverride": true, "id": 15, "options": { "reduceOptions": { "calcs": [ "lastNotNull" ], "fields": "", "values": false }, "showThresholdLabels": false, "showThresholdMarkers": false }, "pluginVersion": "7.3.1", "targets": [ { "alias": "%", "groupBy": [ { "params": [ "$__interval" ], "type": "time" }, { "params": [ "null" ], "type": "fill" } ], "measurement": "binary_sensor.ping_rpigen", "orderByTime": "ASC", "policy": "default", "refId": "A", "resultFormat": "time_series", "select": [ [ { "params": [ "value" ], "type": "field" }, { "params": [], "type": "last" } ] ], "tags": [] } ], "timeFrom": "24h", "timeShift": null, "title": "rpiGen Online", "transparent": true, "type": "gauge" }, { "aliasColors": { "kW": "dark-yellow" }, "bars": false, "dashLength": 10, "dashes": false, "datasource": "HomeAssistant", "decimals": 1, "fieldConfig": { "defaults": { "custom": {} }, "overrides": [] }, "fill": 1, "fillGradient": 0, "gridPos": { "h": 7, "w": 20, "x": 0, "y": 14 }, "hiddenSeries": false, "id": 4, "legend": { "alignAsTable": true, "avg": true, "current": false, "max": true, "min": true, "show": true, "sort": "current", "sortDesc": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "nullPointMode": "null", "options": { "alertThreshold": true }, "percentage": false, "pluginVersion": "7.3.1", "pointradius": 2, "points": true, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "stack": false, "steppedLine": false, "targets": [ { "alias": "kW", "groupBy": [ { "params": [ "$__interval" ], "type": "time" }, { "params": [ "none" ], "type": "fill" } ], "measurement": "kW", "orderByTime": "ASC", "policy": "default", "refId": "A", "resultFormat": "time_series", "select": [ [ { "params": [ "value" ], "type": "field" }, { "params": [], "type": "max" } ] ], "tags": [ { "key": "friendly_name", "operator": "=", "value": "Generator Power" } ] } ], "thresholds": [], "timeFrom": "1y", "timeRegions": [], "timeShift": null, "title": "Generator Load", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "transparent": true, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "decimals": 0, "format": "short", "label": null, "logBase": 1, "max": null, "min": "0", "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ], "yaxis": { "align": false, "alignLevel": null } }, { "aliasColors": { "Tank %": "dark-orange" }, "bars": false, "dashLength": 10, "dashes": false, "datasource": "HomeAssistant", "decimals": 1, "description": "500 Gallon Tank", "fieldConfig": { "defaults": { "custom": {}, "displayName": "Tank %", "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": null }, { "color": "red", "value": 80 } ] }, "unit": "percent" }, "overrides": [] }, "fill": 0, "fillGradient": 1, "gridPos": { "h": 7, "w": 20, "x": 0, "y": 21 }, "hiddenSeries": false, "hideTimeOverride": false, "id": 2, "legend": { "alignAsTable": true, "avg": true, "current": false, "hideEmpty": false, "hideZero": false, "max": true, "min": true, "rightSide": false, "show": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "nullPointMode": "null", "options": { "alertThreshold": true }, "percentage": false, "pluginVersion": "7.3.1", "pointradius": 2, "points": true, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "stack": false, "steppedLine": false, "targets": [ { "alias": "Tank %", "groupBy": [ { "params": [ "$__interval" ], "type": "time" }, { "params": [ "none" ], "type": "fill" } ], "measurement": "%", "orderByTime": "ASC", "policy": "default", "refId": "A", "resultFormat": "time_series", "select": [ [ { "params": [ "value" ], "type": "field" }, { "params": [], "type": "mean" } ] ], "tags": [ { "key": "entity_id", "operator": "=", "value": "tank_utility_002e001b3733373013473831" } ] } ], "thresholds": [], "timeFrom": "1y", "timeRegions": [ { "colorMode": "background6", "fill": true, "fillColor": "rgba(234, 112, 112, 0.12)", "line": false, "lineColor": "rgba(237, 46, 24, 0.60)", "op": "time" } ], "timeShift": null, "title": "Propane Tank", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "transparent": true, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "decimals": 1, "format": "percent", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ], "yaxis": { "align": false, "alignLevel": null } }, { "aliasColors": { "Tank %": "dark-orange", "Vdc": "blue" }, "bars": false, "dashLength": 10, "dashes": false, "datasource": "HomeAssistant", "decimals": 2, "description": "", "fieldConfig": { "defaults": { "custom": {}, "mappings": [], "thresholds": { "mode": "absolute", "steps": [ { "color": "green", "value": null }, { "color": "red", "value": 80 } ] }, "unit": "volt" }, "overrides": [] }, "fill": 0, "fillGradient": 1, "gridPos": { "h": 7, "w": 20, "x": 0, "y": 28 }, "hiddenSeries": false, "hideTimeOverride": false, "id": 16, "legend": { "alignAsTable": true, "avg": true, "current": false, "hideEmpty": false, "hideZero": false, "max": true, "min": true, "rightSide": false, "show": true, "total": false, "values": true }, "lines": true, "linewidth": 1, "nullPointMode": "null", "options": { "alertThreshold": true }, "percentage": false, "pluginVersion": "7.3.1", "pointradius": 2, "points": true, "renderer": "flot", "seriesOverrides": [], "spaceLength": 10, "stack": false, "steppedLine": false, "targets": [ { "alias": "Vdc", "groupBy": [ { "params": [ "$__interval" ], "type": "time" }, { "params": [ "none" ], "type": "fill" } ], "measurement": "V", "orderByTime": "ASC", "policy": "default", "refId": "A", "resultFormat": "time_series", "select": [ [ { "params": [ "value" ], "type": "field" }, { "params": [], "type": "mean" } ] ], "tags": [ { "key": "entity_id", "operator": "=", "value": "generator_battery" } ] } ], "thresholds": [], "timeFrom": "1y", "timeRegions": [ { "colorMode": "background6", "fill": true, "fillColor": "rgba(234, 112, 112, 0.12)", "line": false, "lineColor": "rgba(237, 46, 24, 0.60)", "op": "time" } ], "timeShift": null, "title": "Generator Battery", "tooltip": { "shared": true, "sort": 0, "value_type": "individual" }, "transparent": true, "type": "graph", "xaxis": { "buckets": null, "mode": "time", "name": null, "show": true, "values": [] }, "yaxes": [ { "decimals": 1, "format": "volt", "label": null, "logBase": 1, "max": null, "min": null, "show": true }, { "format": "short", "label": null, "logBase": 1, "max": null, "min": null, "show": true } ], "yaxis": { "align": false, "alignLevel": null } } ], "refresh": "5m", "schemaVersion": 26, "style": "dark", "tags": [], "templating": { "list": [] }, "time": { "from": "now-6h", "to": "now" }, "timepicker": { "hidden": false, "nowDelay": "1m", "refresh_intervals": [ "5s", "10s", "30s", "1m", "5m", "15m", "30m", "1h", "2h", "1d" ] }, "timezone": "", "title": "Generator", "uid": "_TkXhbDMz", "version": 3 }

[lmamakos]

Ok, that's great, except my dashboard has more non-genmon specific items on it with other power and temperature sensors, along with a few that come from the the generator. So I don't know what good my dashboard is likely to be to anyone.

Once you configure Home Assistant's influxdb integration, the data gets fed over and you can build a dashboard to show the data that's useful to you. For example, I show the "Utility Voltage" which is what comes from genmon, along with the voltage from my breaker panel, and total power consumption. I built and tweaked that dashboard as something to watch during outages.

That said, here you go! I'm not sure how helpful this might be to you..
Generator-1610673101483..json.txt

[scyto]

Cool thread, i just started the move to HA (i have an ISY994i i have been using for years), it has improved massively since i looked at 2+ and 4+ years ago!

I have moved to z-wave (using zwavejs2mqtt) so have a tiny understanding of MQTT but not node red.

I am used to MQTT 'just working' because of help apps like zwavej2mqtt (or nodelink on my ISY994i that has support for genmon).

It looks likes for me to get tis to working with HA i need to manually create the MQTT topics. Or do i also need node red (maybe that's generating the topics in this case? in which case if the topics are right why isn't the MQTT integration on HA automatically creating the devices/entities).

tl;dr i am pretty confused (and high maintenance, lol); is there a quick 101 of links you can point me to so that i go down some rabbit holes.

[lmamakos]

@scyto, it's not necessary to create the MQTT topics as a manual step. The primary topic state in the brokers are the subscriptions, and those happen when clients connect and subscribe to the topics they're interested in. Primarily, you just need to match up the topics of the messages that e.g., genmon is sending to with the topic configuration in Home Assistant.

In the example I posted, I use a specific prefix ("19916/") at home for reasons that are not important, and needed to configure genmon's genmqtt program to use that topic prefix. But there's nothing magic there.

In an earlier comment, I posted my Home Assistant entity configuration for populating the MQTT sensors. At one point there was some discussion about generating an auto-discovery message to have this done automagically, but I'm not using that so can't really comment. There's no reason to use Node Red here to make this work (unless you want to use it as a tool for other purposes.) While I have Node Red in my software stack, it's not involved at all with the Home Assistant / genmon communications.

[scyto]

Thanks, in the last 17 days i learnt much more about MQTT and using things like MQTT explorer, i now realize there is an MQTT function in genmon to push the data into MQTT (my nexus controller is down waiting for a replacement so couldn't look as the genmon is off) thanks for your work, appreciate it!

See you on the HA discord maybe :-)

[scyto]

thanks, i added one more sensor for a topic i found that looks like it could be useful

  - platform: mqtt
    name: Genmon Generator Last Run Log
    state_topic: 'generator/Status/Last Log Entries/Logs/Run Log'

Also i saw this in the yaml validation and was unclear if i need to fix or not?

and

thoughts?

[lmamakos]

The errors in the template are because of changes in Home Assistant over the last few months that made it unnecessary to specify the specific entity_id names, as it can now pick out the ones to watch as the template expression is evaluated. So you can just delete the entity_id: key and the two items in that list.

I believe in the currently recommended best-practices, having a sensor that updates that frequently (every minute based on how often the now() thing changes) is not recommended. This is because it ends up pushing that state change into the database which can junk it up. However, in practice, this is only going to happen when you're in an outage, so probably not that big a deal. Or you could exclude it from the recorder database if that's of real concern. I would probably update the template to just skip computing the number of seconds since that template only gets evaluated (I think) once per minute..

For the second one, that's just me being stupid. That should just be true rather than the quoted string "on" on the initial_state: thing.

[scyto]

Thanks for the clarifications and the tips, I switched my database from sql lite to mariadb for recorder, so that should help.

Also about to hook my GEM energy monitor; it outputs 32 channels of kwh, watt and amp thinking of using the exclude function of recorder to exclude any power data and use that via grafana and influxdb - so might do the same with the outage sensors on the generator.

As for the discovery stuff for HA i will put that on my long list of something to come back and see if i can help with :-)

[lmamakos]

@scyto I also have a GEM here, too.. like minds in the same gutter or something. I'm using this "mtools" software to publish to MQTT every two seconds. I pipe all of that into Home Assistant, but exclude it from the history/recorder database.

I do, however, send it all to influxdb (using the integration in Home Assistant) since it deals with higher volume time series data much better and than sqlite does. I do my prospecting and "WTF was going on there?" sort of investigation in influxdb when needed.

[scyto]

@lmamakos yeah I have found bunch of folks in the home automation space are like minded :-)
Thanks for pointing me to mtools, i have used btmon in the past for other things.
I am also trying out the official GEM HA integration, plus I also found my brultech dashbox can push straight to MQTT too. Not sure I have a mental model yet of pros /cons.

[lmamakos]

I was using mtools (really, a modified variation of an earlier version of it) before the component appeared in Home Assistant. Since it was working, I left well-enough alone. I don't have an opinion of the built-in integration; for complicated/weird stuff, I tend to prefer MQTT since it's really easy to eavesdrop on the data by subscribing to the relevant topics to figure out what's really going on..

I've never tried the Brultech applicance; I was using the MQTT integration since before the GEM, when I has the ECM-1240 initially, and Brultech got interested in this cloud stuff more seriously.