add Webhook #3148

[RaHehl]

#3148

[RaHehl]

Example Payload {"value":"345.90106","valueraw":"0345.90106","error":"no error","rate":"0.000000","timestamp":"2024-08-07T11:55:48+0200","timeUTC":1723024548}

[RaHehl]

PHP Example for collection data of a single meter in a csv file:

<?php
$expectedApiKey = 'exampleapikey';

$csvFile = 'webhook_log.csv';

$receivedApiKey = isset($_SERVER['HTTP_APIKEY']) ? $_SERVER['HTTP_APIKEY'] : '';

if ($receivedApiKey !== $expectedApiKey) {
    http_response_code(403); // 403 Forbidden
    echo json_encode(['status' => 'error', 'message' => 'Invalid API key']);
    exit;
}

$jsonData = file_get_contents('php://input');

$dataArray = json_decode($jsonData, true);
if (!$jsonData || !is_array($dataArray)) {
    http_response_code(400); // 400 Bad Request
    echo json_encode(['status' => 'error', 'message' => 'Invalid JSON data']);
    exit;
}

$writeHeader = !file_exists($csvFile);

$csvHandle = fopen($csvFile, 'a');
if ($csvHandle === false) {
    http_response_code(500); // 500 Internal Server Error
    echo json_encode(['status' => 'error', 'message' => 'Unable to open CSV file']);
    exit;
}

if ($writeHeader) {
    fputcsv($csvHandle, array_keys($dataArray));
}

fputcsv($csvHandle, $dataArray);

fclose($csvHandle);

http_response_code(200); // 200 OK
echo json_encode(['status' => 'success', 'message' => 'Data written to CSV file']);
?>

[caco3]

Thanks for the Pull Request and the example PHP script! I will give it a try on my system and see how it behaves.
I am very impressed that you found out how to add all the config and documentation stuff as it is IMO still not trivial to understand!

Did you also test it with multiple numbers?

[RaHehl]

@caco3 Thanks!

The correct clue came from @henrythasler #3148 (comment)

That reminded me that the Influx connection is quite new, so I just looked at the pull request for Influx https://github.com/jomjol/AI-on-the-edge-device/pull/2004/files and then recreated the whole thing for Webhook.

I don't know exactly what you mean by multiple numbers, but as far as I can tell, the firmware has been running quite smoothly for me with a water meter since yesterday, and I used my water today :D

[SybexX]

[RaHehl]

No, it runs on my water meter, I honestly didn't know until now that there were multiple numbers, of course it has to be built a little differently than I currently have

[SybexX]

you can create several numbers, some user use it to get the digital and analog numbers separately.

[caco3]

I will test it with 2 numbers on my test device, but only in 2 weeks.
As I understand, this would not work yet?
You could adapt the data file writing which we already have (storing the data as csv on the sd card).

[RaHehl]

@caco3 I'm almost done with the changes. I just have to build it, test it and adapt the example:

[caco3]

Also we might need to fix the columns in case of an error:

value,valueraw,error,rate,timestamp,timeUTC
692.67233,692.67233,"no error",0.000945,2024-08-09T09:56:20+0200,1723190180
,692.67230,"Neg. Rate - Read:  - Raw: 692.67230 - Pre: 692.67233 ",,2024-08-09T09:56:20+0200,1723190180
692.67233,692.67232,"no error",0.000000,2024-08-09T10:00:20+0200,1723190420

Again, we should orient on the sd card data file

[RaHehl]

Changes are committed, the sent Json is now an array:

[
  {
    "name": "main",
    "value": "345.42648",
    "valueraw": "0345.42646",
    "error": "no error",
    "rate": "0.000000",
    "timestamp": "2024-08-09T10:57:32+0200",
    "timeUTC": 1723193852
  },
  {
    "name": "test",
    "value": "34",
    "valueraw": "34",
    "error": "no error",
    "rate": "-0.000000",
    "timestamp": "2024-08-09T10:57:32+0200",
    "timeUTC": 1723193852
  }
]

I'm currently looking for the place where the csv is created in order to adapt the php example, although it's really just a simple example

[RaHehl]

[
  {
    "timeUTC": 1723196684,
    "timestamp": "2024-08-09T11:44:44+0200",
    "name": "main",
    "rawValue": "0345.42647",
    "value": "345.42648",
    "preValue": "345.42648",
    "rate": "0.000000",
    "changeAbsolute": "0.00000",
    "error": "no error"
  },
  {
    "timeUTC": 1723196684,
    "timestamp": "2024-08-09T11:44:44+0200",
    "name": "test",
    "rawValue": "34",
    "value": "34",
    "preValue": "34",
    "rate": "0.000000",
    "changeAbsolute": "0",
    "error": "no error"
  }
]

2024-08-09T11:44:44+0200,main,0345.42647,345.42648,345.42648,0.000000,0.00000,no error,0.1,3.0,4.0,5.4,4.2,2.6,6.3,4.7
2024-08-09T11:44:44+0200,test,34,34,34,0.000000,0,no error,3.0,4.0

@caco3 I have now adapted the json closer to the csv log.

The difference is basically the additional timestamp, which can have advantages for further processing, and the fact that the "ReadoutRawString" is missing, as that apparently requires further modifications and the benefit in this case is not yet clear to me.

[RaHehl]

Here is a new php example which creates an identical csv minus the two ReadoutRawStrings

<?php
$expectedApiKey = 'exampleapikey';

$csvFile = 'webhook_log.csv';

$receivedApiKey = isset($_SERVER['HTTP_APIKEY']) ? $_SERVER['HTTP_APIKEY'] : '';

if ($receivedApiKey !== $expectedApiKey) {
    http_response_code(403); // 403 Forbidden
    echo json_encode(['status' => 'error', 'message' => 'Invalid API key']);
    exit;
}

$jsonData = file_get_contents('php://input');

$dataArray = json_decode($jsonData, true);
if (!$jsonData || !is_array($dataArray)) {
    http_response_code(400); // 400 Bad Request
    echo json_encode(['status' => 'error', 'message' => 'Invalid JSON data']);
    exit;
}

$csvHandle = fopen($csvFile, 'a');
if ($csvHandle === false) {
    http_response_code(500); // 500 Internal Server Error
    echo json_encode(['status' => 'error', 'message' => 'Unable to open CSV file']);
    exit;
}

foreach ($dataArray as $data) {
    $csvRow = [
        $data['timestamp'], 
        $data['name'], 
        $data['rawValue'], 
        $data['value'], 
        $data['preValue'], 
        $data['rate'], 
        $data['changeAbsolute'], 
        $data['error']
    ];
    fputcsv($csvHandle, $csvRow);
}

fclose($csvHandle);

http_response_code(200); // 200 OK
echo json_encode(['status' => 'success', 'message' => 'Data written to CSV file']);
?>

[RaHehl]

@caco3 I've created another PR #3174 here (currently not quite finished) for uploading the images.

And what is your opinion about the ReadoutRawStrings? Should I integrate them?

[caco3]

@RaHehl I am currently on holiday and will only be able to look into it after the 20st August.

And what is your opinion about the ReadoutRawStrings? Should I integrate them?

@SybexX It doesn't say anything to me, does it to you?

@RaHehl If you have a spare device, I suggest to use the demo mode for testing, see https://jomjol.github.io/AI-on-the-edge-device-docs/Demo-Mode/

[caco3]

It looks good to me. it worked a few weeks on my system (just tested a single number), so I will merge it.

What we further will need:

• Please add a brief description of this feature to this merge request.
• We also need to add it to our documentation.

@RaHehl Can you do this? I will prepare the documentation and you then can fill in the markdown based text.

[RaHehl]

@caco3 yes, i just Need to know where

[caco3]

Thanks for your contribution!

The documentation will be here: https://jomjol.github.io/AI-on-the-edge-device-docs/Webhook/
The file to edit is is here: https://github.com/jomjol/AI-on-the-edge-device-docs/blob/main/docs/Webhook.md
Check the readme how you can generate and test it locally: https://github.com/jomjol/AI-on-the-edge-device-docs/tree/main