Display non-historical data

[jchasey]

I have a use case where I am looking to graph future events - specifically tides and have my own custom component that retrieves the forthcoming tide data into an array.

It would be great to be able to display this data rather than simply the historical data of a sensor. The data can be formatted in a similar manner as a sensor.

At the moment I can fudge this (using mini-graph card) by setting the 'current' value as the value 2 days into the future and therefore when 2 days worth of data is displayed the graph line is accurate, but of course all the timing information is incorrect.

Thanks for considering.

[RomRider]

We could graph things from the future that is not a problem with ApexCharts, but the problem is how the data comes from Home-Assistant...
The same way as mini-graph-card, it retrieves the history so the same comments that I made there applies: kalkih/mini-graph-card#459

[deepcoder]

This is way out there and I most likely don't know what I am talking about....
I have seen javascript graphing tools that seem to run within the HA sandbox that get data from web socket calls. This is an example (not a graph, but getting refreshed JSON data in realtime):

https://github.com/dmulcahey/zha-network-card

It would be up to the writer of the 'future sensor' to create a web socket endpoint and refresh JSON data on it. I am not sure if it is possible to write a python script within HA sandbox that would be able to create this web socket and push data on it.

https://apexcharts.com/docs/update-charts-from-json-api-ajax/

As I said way out there and out of my pay grade! But clearly the ability to use your implementation to graph data from other sources than the history recorder is a use case of interest.

But stay focused on the super work you doing now, too many ideas can make you crazy...

[CDRX2]

I actually have a similar interest, looking to graph future data, specifically rain forecast provided by the openweathermap integration. The data is a state attribute of the weather.openweathermap entity. Here's a screenshot for reference:

There's 7 days of forecast included. Here's someone who's done something similar with chart.js, however he's hardcoded pretty much everything and there's pretty much no flexibility (i.e. color, for instance): https://github.com/Yevgenium/lovelace-weather-card-chart

So I guess it would come down to whether state attributes can be defined as series on this card.

[RomRider]

I see your point. I'd have to find a nice way to remap any kind of data and give you this ability through yaml.
That is going to be challenging, not impossible, but challenging 😅

[CDRX2]

I'd definitely be grateful if you make it, who doesn't love a challenge ;-) In all seriousness, my very limited coding skills allow me to comprehend what you mean. Keeping my fingers crossed and thanks for your work already, looking forward to testing it out in the next few weeks (have a few other things to accomplish in my HA first).

[RomRider]

@jchasey @CDRX2 ,could you please post the full dev-tools state data you'd expect to be able to display as code here, no screenshot please, I need to be able to copy paste.

Thanks :)

[jchasey]

Here is the data from my sensor, an array of times and then array of values to be plotted.

FirstPeak: High
PeakTimes:
  - '2021-01-27 03:43:00'
  - '2021-01-27 10:24:00'
  - '2021-01-27 16:02:00'
  - '2021-01-27 22:38:00'
  - '2021-01-28 04:21:00'
  - '2021-01-28 11:06:00'
  - '2021-01-28 16:40:00'
  - '2021-01-28 23:18:00'
  - '2021-01-29 05:00:00'
  - '2021-01-29 11:45:00'
  - '2021-01-29 17:19:00'
  - '2021-01-29 23:58:00'
  - '2021-01-30 05:39:00'
  - '2021-01-30 12:25:00'
  - '2021-01-30 17:59:00'
PeakHeights:
  - 4.99
  - 1.41
  - 4.96
  - 1.33
  - 5.22
  - 1.19
  - 5.15
  - 1.14
  - 5.42
  - 1.01
  - 5.3
  - 0.99
  - 5.57
  - 0.87
  - 5.39
unit_of_measurement: m
friendly_name: Tides

[RomRider]

Thanks! I think I have an idea which is not too ugly 😊 it will require you to write some javascript in yaml though, let's see!

What it would look like probably for your data @jchasey:

graph_span: 4d
span:
  start: day
series:
  - entity: sensor.future
    remap_attributes: |
      return state.PeakTimes.map((peak, index) => {
        return [new Date(peak), PeakHeights[index]];
      });

What do you think?

Sorry forgot to ask, what is the integration which generates the data so that I can test it with a real use case?

[jchasey]

Its a custom script I wrote which scrapes the data, configuration:

  - platform: command_line
    name: 'Sennen Tides'
    command: 'python3 /config/tide_scrape.py'
    scan_interval: 600            # Scrape every 10 minutes
    value_template: '{{ value_json.fortyeighthours }}'
    unit_of_measurement: 'm'
    json_attributes:
      - FirstPeak
      - PeakTimes
      - PeakHeights

and the script:

#!/usr/bin/python3

import json
import bs4 as bs
import urllib.request
import time
import datetime as dt
from datetime import datetime

url = 'https://www.ntslf.org/tides/tidepred?port=Newlyn'

html = urllib.request.urlopen(url).read()
soup = bs.BeautifulSoup(html, 'html.parser')

# Select the tide table and the elements therein containing the times & heights
tides = soup.find(id="ntslf-tide-table").find_all("span")
tide_data = {}
peak_times = []
peak_heights = []

# Is first tide of the day low or high?
if (tides[2].contents[0][-1] == "L"):
    tide_data['FirstHighTide'] = tides[3].contents[0]
    tide_data['FirstPeak'] = "Low"
else:
    tide_data['FirstHighTide'] = tides[1].contents[0]
    tide_data['FirstPeak'] = "High"

# Get initial date stripping tags
datestr = tides[0].contents[0].get_text()
# Convert th, rd or st to whitespace
datestr = datestr.replace("st"," ")
datestr = datestr.replace("nd"," ")
datestr = datestr.replace("rd"," ")
datestr = datestr.replace("th"," ")
date = dt.datetime.strptime(datestr,"%a %d %b %Y")

t = 1
lasthours = 0
# 18 = scan next 2 days
# 36 = scan next 4 days
while t < 36:
    # Make sure field has data & ignore non-times
    if tides[t].contents and tides[t].contents[0][0].isdigit():
        timestr = tides[t].contents[0].split(':')
        hours = int(timestr[0])
        mins = int(timestr[1])

        # Increment day if needed
        if (hours < lasthours):
            date += dt.timedelta(days=1)
        lasthours = hours

        # Set hours and mins of the tide
        date = date.replace(hour=hours,minute=mins)

        peak_times.append(date)
        peak_heights.append(float(tides[t+1].contents[0][:-3].strip()))

        t += 2
    else:
        # The next day field or a blank field when only 3 peaks in a day
        t += 1

# Rule of twelths to calculate tide height at particular time
# Take height difference between low and high tide either side of the time e.g. 12m
# Divide time between peaks (usually 6 hours)
# 1 = 1/12 increase = 1m
# 2 = 2/12 increase = 3m
# 3 = 3/12 increase = 6m
# 4 = 3/12 increase = 9m
# 5 = 2/12 increase = 11m
# 6 = 1/12 increase = 12m
rule_of_twelths = [ 1,2,3,3,2,1]

# Get the tide heights either side of the target time
target_time = datetime.now() + dt.timedelta(days=2)
t = 0
while peak_times[t] < target_time:
    t += 1

# Calculate how many twelths required based on time between low/high tide
twelths = 0
tide_delta = (peak_times[t] - peak_times[t-1]) / 6
tt = peak_times[t-1]
for x in range(0, 5):
    if tt > target_time:
        break
    tt = tt + tide_delta
    twelths = twelths + rule_of_twelths[x]

# Now add that on taking into account low or high tide
height_delta = twelths * (abs(peak_heights[t] - peak_heights[t-1]) / 12)
if peak_heights[t-1] > peak_heights[t]:
    # High Tide to Low Tide
    tide_height = peak_heights[t-1] - height_delta
else:
    # Low Tide to High Tide
    tide_height = peak_heights[t-1] + height_delta

tide_data['fortyeighthours'] = tide_height
tide_data['PeakTimes'] = peak_times
tide_data['PeakHeights'] = peak_heights
json = json.dumps(tide_data, indent=4, sort_keys=True, default=str)
print (json)

[CDRX2]

Here you go:

temperature: 1.5
humidity: 84
pressure: 1021
wind_bearing: 232
wind_speed: 14.9
attribution: Data provided by OpenWeatherMap
forecast:
  - datetime: '2021-01-27T11:00:00+00:00'
    precipitation: 7
    wind_speed: 3.49
    wind_bearing: 223
    condition: snowy-rainy
    temperature: 3.5
    templow: -4.5
  - datetime: '2021-01-28T11:00:00+00:00'
    precipitation: 27
    wind_speed: 7.3
    wind_bearing: 211
    condition: rainy
    temperature: 7.4
    templow: 3.7
  - datetime: '2021-01-29T11:00:00+00:00'
    precipitation: 40
    wind_speed: 6.88
    wind_bearing: 236
    condition: pouring
    temperature: 7.4
    templow: 4.9
  - datetime: '2021-01-30T11:00:00+00:00'
    precipitation: 23
    wind_speed: 3.95
    wind_bearing: 228
    condition: rainy
    temperature: 4.8
    templow: 3.5
  - datetime: '2021-01-31T11:00:00+00:00'
    precipitation: 4
    wind_speed: 0.43
    wind_bearing: 144
    condition: rainy
    temperature: 5.6
    templow: 2.1
  - datetime: '2021-02-01T11:00:00+00:00'
    precipitation: 17
    wind_speed: 6.76
    wind_bearing: 227
    condition: rainy
    temperature: 5.9
    templow: 2.9
  - datetime: '2021-02-02T11:00:00+00:00'
    precipitation: 20
    wind_speed: 8.17
    wind_bearing: 222
    condition: rainy
    temperature: 9
    templow: 5.3
  - datetime: '2021-02-03T11:00:00+00:00'
    precipitation: null
    wind_speed: 1.07
    wind_bearing: 202
    condition: cloudy
    temperature: 9
    templow: 4.4
friendly_name: OpenWeatherMap

The integration is OpenWeatherMap, but similar data comes from various weather integrations. The data I'd be looking to map is the forecast data.

[github-actions]

🎉 This issue has been resolved in version 1.2.0-dev.4 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

[RomRider]

@jchasey, I've put the code you need in the PR #14
@CDRX2, le me know if you need help to build your data_generator function 🙂

[CDRX2]

Thanks for this. Will give it a go tomorrow or friday and report back.

[github-actions]

🎉 This issue has been resolved in version 1.2.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀