Replies: 5 comments 4 replies
-
|
I don't know if it's the best way, but I wrote this and it's worked for me
for a few years now.
https://www.npmjs.com/package/signalk-to-batch-format
https://github.com/c33howard/signalk-to-timestream-lambda
…On Tue, Nov 21, 2023 at 12:59 PM Jean-Laurent Girod < ***@***.***> wrote:
In the past, to keep trace of my boat trips, I logged all my NMEA0183
streams to files with kplex, and replicate them to my home with rsync cron
jobs.
As I’m moving to SignalK, I’m searching a way to perform same function,
with SignalK data instead of NMEA0183.
The constraint are :
- The boat may not always be connected to the internet
- SignalK is running on a RaspBerry Pi 4 with VenusOS. So I can’t
install InfluxDB locally, or similar software.
- It could be great to be able to replay trips in a home instance of
SignalK, and/or to show dashboards with Grafana
What is the best way to do that ?
—
Reply to this email directly, view it on GitHub
<#84>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAN33OW3XCN3JCQUJZQPES3YFUI4HAVCNFSM6AAAAAA7VGMLDGVHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZVHA3TONRTG4>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Thank you. It seems very interesting. I already have a S3 local instance. So I will try to use it instead of an AWS instance. And I saw another plugin from you that can do what I need : signalk-from-batch-format For this one, I need to understand if I can provide a SQS like URL from my local S3. |
Beta Was this translation helpful? Give feedback.
-
|
signalk-to-timestream look mighty interesting, thanks for sharing! It does not support playback I assume? Another pretty simple solution would be to just activate data logging in your server's connections. This will produce hourly data files with timestamped lines of raw input data that you can rsync like previously. You can then configure a server at home to replay them. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you. It’s a good approach too. But is there a way to automatically ingest the hourly data files, then move/delete them when there are processed ? |
Beta Was this translation helpful? Give feedback.
-
|
Not out of the box. I am not aware of a plugin either. But this would work with your DIY cron approach. Really depends on the destination you want to use: S3, Dropbox, your own server with ssh/rsync.... I have used AWS command line client |
Beta Was this translation helpful? Give feedback.
-
|
I may use rsync or s3. Not a problem for that. The question is : when I have the file near to the destination SignalK server, how to load them? I see there is a Data Connection type « Filestream » but how does it work ? Should I create a FIFO, and put input logs into it when there are ready ? How to be sure they are consumed ? |
Beta Was this translation helpful? Give feedback.
-
|
signalk-to-timestream does not support playback directly today. I don't
think it's hard to do, but I don't need it, so I haven't implemented it.
Craig
…On Tue, Nov 21, 2023, 20:51 Teppo Kurki ***@***.***> wrote:
signalk-to-timestream look mighty interesting, thanks for sharing! It does
not support playback I assume?
Another pretty simple solution would be to just activate data logging in
your server's connections. This will produce hourly data files with
timestamped lines that you can rsync like previously. You can then
configure a server at home to replay them.
—
Reply to this email directly, view it on GitHub
<#84 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAN33OV7HTY5YFNFRTWJJVLYFWOILAVCNFSM6AAAAAA7VGMLDGVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TMMZYGYZDG>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
The companion to signalk-to-batch-format is signalk-from-batch-format.
https://github.com/c33howard/signalk-from-batch-format
https://www.npmjs.com/package/signalk-from-batch-format
…On Sun, Dec 3, 2023 at 3:10 AM Jean-Laurent Girod ***@***.***> wrote:
I may use rsync or s3. Not a problem for that.
The question is : when I have the file near to the destination SignalK
server, how to load them? I see there is a Data Connection type
« Filestream » but how does it work ? Should I create a FIFO, and put input
logs into it when there are ready ?
—
Reply to this email directly, view it on GitHub
<#84 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAN33OQ7JCJTW64O4P5RVALYHRMZLAVCNFSM6AAAAAA7VGMLDGVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TONBTGE3DQ>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
Some 2-3 years ago I made first attempts to keep remote SK sync'd to a vessel as real time as possible. Work was done for demonstrating remote piloting for commercial vessels and real time monitoring of bridge operations from shore using only cellular networks (not failovered to sat link then). In this use case, only real time data was important and if no cellular coverage then no data. At the edge of cellular network or somehow otherwise poor networks, things get complicated. UDP becomes more reliable transport media than TCP etc... Things evolved a bit and I published work related to comms link below. In addition to get data at poor or good networks, also optimizing payload became important part of comms link. Based on lots of trials and testing SK deltas can be batched, compressed, encrypted to get low bandwidth usage with high security over public Internet. I would like to see next steps to improve vessel-to-cloud links. |
Beta Was this translation helpful? Give feedback.
-
In the past, to keep trace of my boat trips, I logged all my NMEA0183 streams to files with kplex, and replicate them to my home with rsync cron jobs.
As I’m moving to SignalK, I’m searching a way to perform same function, with SignalK data instead of NMEA0183.
The constraint are :
What is the best way to do that ?
Beta Was this translation helpful? Give feedback.
All reactions