DietPi-Software | Blynk Server: Logging fails and config change questions

[Phil1988]

Creating a bug report/issue

Required Information

• DietPi version
  G_DIETPI_VERSION_CORE=6
  G_DIETPI_VERSION_SUB=21
  G_DIETPI_VERSION_RC=1
  G_GITBRANCH=master
  G_GITOWNER=Fourdee

• Distro version
  9.9

• Kernel version
  Linux DietPi 4.9.0-8-amd64 Fix warning when running scp over openssh-server #1 SMP Debian 4.9.144-3.1 (2019-02-19) x86_64 GNU/Linux

• SBC device
  Virtual Machine (x86_64)

Steps to reproduce

1. Install blynk server
2. Go to the web admin interface and change the admin login (for example) on the system.properties
3. You cant login with the changed admin login.

Expected behaviour

system.properties in web admin interface should be used and installation paths should be finable :)

Extra details

I installed a Blynk Server from the dietpi software menue yesterday.
The paths are different then the description, because there is no /etc/blynkserver on my system.
This is the reason why I also cant see the log files.
Changing them manually to /mnt/dietpi-userdata/blynk/logs make them findable for me :)
Problem is, that the system.properties (in /mnt/dietpi-userdata/blynk/) and the system.properties that are accessable in the web admin interface (https://localIP:9433/admin/) are not the same.
Changes in the frontend system.properties doesnt change (for example the admin account).
There seem to be another "system.properties" or anything else that is important.

This can be replicated on my Virtual Machine (x86_64).

The main reason why I am here is that its not possible for me to use SSL connections on this Blynk server.

I am using an home server and a dynamic DNS service to find my server.
Problem is, that nextcloud is already running on this server and letsencrypt is running too.
This combination works great and I am redirecting to 443 to only allow ssl connections to my public nextcloud.

Blynk also needs some certificate for SSL communication, but the letsencrypt certificate is not working here (maybe it is possible but I dont know how yet).
Own signed certificate gives me an Error initializin ssl context. Reason : File dies not contain valid private key: ...".
Also the topic in the blynk forum didnt help me out
The problem with the unconsistent installation paths and makes it harder for me to find the problem(s) 🗡️

[MichaIng]

@Phil1988
Many thanks for your report. That is indeed strange since Blynk should search/use for the config in it's install dir in the first place. /etc is not such a good location since that is usually used by OS and APT package configs only. But we simply download the Java executable and place it with the config manually to dietpi_userdata so that all together can be easily backed up and migrated.
However the service definitely uses the config from userdata.

I will check this. If there is no chance to tell Blynk which config file to show (even that the one it uses is clearly defined), then we probably need to place a symlink.

About SSL:
Not sure how you created the self-signed cert (as there seem to be an issue according to the error message), but note that Blynk runs as "blynk" user with limited permissions. So you need to grant it read permissions the cert and key file, e.g. by making them group-readable, and add it to the root group. Alternatively you copy them e.g. to the userdata/blynk dir and chown it accordingly. This needs to be redone whenever the certificate is renewed (every ~2 months).

[MichaIng]

@Phil1988
Okay did some testing.

About server.properties:

• Indeed it uses the one where the script itself is started from by default. So it is not even required to set this via -serverConfig in our case.
• Also I didn't find any /etc/blynkserver defaults or reference in the docs, so it is totally common to use a custom location or it, or fallback to default to use the one besides the blynkserver.jar executable: https://github.com/blynkkk/blynk-server#advanced-local-server-setup
• But now yeah you can configure through the web UI but without any effect to the actual server.properties file. When restarted, the made changes are lost.
• The other way round it work, adjusting the actual server.properties and restarting the server, then the changes are shown in web UI as well.
• Not sure if it is the expected behaviour, but it seems the web UI settings are only for the current server session and possibly not meant to be persistent.

About SSL:

• SSL is enabled by default via build-in self-signed certificates. So there is no point in creating own self-signed certificates.
• As mentioned above grant the blynk user read access to the LetsEncrypt cert+key. Security-wise best would be to create a copy of the cert+key to /mnt/dietpi_userdata/blynk/ and chown this to the blynk user. But this needs to be redone after every cert renewal as mentioned.

I am a bid confused about the required Blynk account. On the one hand there is a default user+password provided by the config file, to login into the web UI. I added the missing info about this to our online docs: https://dietpi.com/phpbb/viewtopic.php?p=5901#p5901
But there is no way to connect to the own server directly with this user from the Blynk Andoid app. Since they explicitly mention the possibility to run an own server beneath the account creation info (in app and their docs), it does not make much sense to me that an external Blynk account is required.

• You know more about this than me, since I never actively used Blynk, so perhaps you know who things are working about this? Are app connections, even to your own server, always relayed through the external Blynk server+account so that it is required?

Another quirk indeed is the logs:

• The default path entry ./logs should create this according to the docs at the dir "from which you are running script".
• But other than server.properties this is not the location of the script, but the current location you are navigated to when starting it. Since systemd by defaults start in root /, it tries to create /logs where it of course does not have permissions to.
• Simply solution is to actively set the WorkingDirectory for the systemd unit. You can do this by running the following:

cat << _EOF_ > /etc/systemd/system/blynkserver.service
[Unit]
Description=Blynk Server (DietPi)
After=network.target

[Service]
User=blynk
Group=dietpi
WorkingDirectory=$G_FP_DIETPI_USERDATA/blynk
ExecStart=$(command -v java) -jar $G_FP_DIETPI_USERDATA/blynk/blynkserver.jar

[Install]
WantedBy=multi-user.target
_EOF_
systemctl daemon-reload
systemctl restart blynkserver

---

PR up to implement the above fixes + enable support on Debian Buster (via Java 11 binary): #2779

---

Hmm actually it would be best to have logs in /var/log/blynk, what you think?

[Phil1988]

Thanks for the deeper look.

The /etc/blynkserver directory has been noted as first point after "Install notes:" but it's gone now ;)
Maybe I have a 'not refreshed tab' open on my browser at home, to make a screenshot :)

Thanks for clearing the things about the server.properties. I highly appreciate it and makes it clear which 'server.properties' file is the real one and how I deal with it.

I am beginner myself at blynk.
The server config in 'Web admin interface' seems to be used to really make the changes on the server.properties, and not only to show it.
That's why there is a save changes button...

Personally it is not important for me, as I now know where and how to change things and also how to make it take effect (by restarting the service).

The logs directory can be easily changed in the server.properties so that will be the way to go for me.
If it's a minor change for you, it might be helpful to set the logs directory to the /mnt/dietpi-userdata/blynk on installation.
Otherwise a hint at the docs would be helping new users (to change it in server.properties).

Indeed there is no need for a blynk account (why do you think so?).
You just have to install the local server and can use the admin account set in the server
properties.
I changed it directly on start at the Web admin interface at the 'user' category (as server.config at the Web interface didn't worked) to my whished account.
Maybe its also working if you change it in the /mnt/dietpi-userdata/blynk/server.properties, but I didn't test that.

Then you can use the app with the admin/user account and connect to the local server.

I will have a look at the SSL thing, when I'm back home.
I hope granting read access to the user 'blynk' will do the job.
If this works I will report back and also try to copy them to the folder you said and chown them.
Can you explain a little, why it is security wise better to separate them?
Maybe it would be possible to do a cron job to copy and chown this after every certification renewal.

I will look deeper into this, when I am back at home.
Thanks for the help so far!

[MichaIng]

@Phil1988

The /etc/blynkserver directory has been noted as first point after "Install notes:" but it's gone now ;)

Ah that is true, this was the first thing I fixed in our online docs (besides adding default account info). Okay so now this is fine/consistent.

The server config in 'Web admin interface' seems to be used to really make the changes on the server.properties, and not only to show it.
That's why there is a save changes button...

Yes it is, but the changes are only made for the current session and not persistent for service restarts. I am not sure if this is the intended way. At best we open an issue to ask this on their GitHub page: https://github.com/blynkkk/blynk-server/issues
If it is desired, then at least there should be some clear info that one needs to edit the file manually outside the GUI to have changes persistent.

If it's a minor change for you, it might be helpful to set the logs directory to the /mnt/dietpi-userdata/blynk on installation.

This is how it is with the open pull request. However what do you think about having the logs in /var/log/blynk/ instead, so within the RAMlog (if enabled) and together file most other log files?

Indeed there is no need for a blynk account (why do you think so?).
You just have to install the local server and can use the admin account set in the server
properties.

But does this local account work with the Blynk Android app as well? Of course with the local account I can access my server via web UI, but since I cannot tell the app which domain/IP to connect to, I doubt that it is able to connect to my local server directly, at least since your local account is not necessarily individual.

Can you explain a little, why it is security wise better to separate them?

Certbot auto-renews the certificate every ~2 months. It runs as root so the files are created as root:root and especially the key files are saved with strict 600 or even 400 permissions. I am currently not sure if Certbot preserves manually changed permission modes, but at least to grant other users read access you either:

• need to make the files world-readable, which is the most dangerous solution. The whole purpose of SSL/TLS is broken if anyone can read/use your private key to decrypt incoming packets.
• need to add the "blynk" user to the "root" group and make the files group-readable. But as root group, the blynk user has muuuch more access to the whole system and in cases even write permissions to critical files. Of course we trust Blynk, but the whole purpose of having clearly separated users is to prevent worst-case scenarios.
• Having a separate copy of the files allows the blynk users, and only the blynk user (chmod blynk:blynk and chown 400 the files btw), additional read access to them, while its access to other system files does not change. This solves the above mentioned issues.

[Phil1988]

I will open an issue on blynks server github regarding the server.properties changes made at the WebUI and ask what it is intended to do and report back to you here.

Regarding the log file:
I don't care where this is stored as long as I know where it is :D
You know dietpi pretty well (obviously) and if the var/log/blynk directory is more consistent with the logs from other projects/programs then it would be fine.
Otherwise I would think, that (almost) everything related for the daily user/admin is in one folder should also be a good idea.
So feel free to place it where it's best (because you are more informed then me :D )

The blynk is only usable from the app. The WebUI is really only to configure the lokal server (and doesn't really work :D... not only because if the not recognised changes but also because there are more errors then that)
So yes, you can use the Android/IOS App with a local account on your local server.
If you start the app you have to choose 'Log in' and then click the symbol on the bottom with the 3 dots to enter your custom/local server.

If you have an dynDNS as well, you can control all your devices remotely.
Pretty awesome idea/concept, that I am testing right now.
Hopefully there will be improvements regarding a Web interface in the future (or you have to use an Android emulator on the PC).

I hope to be able to fix the SSL thing, because that is mandatory for me before I use it on a daily base on my server :)

[MichaIng]

@Phil1988

and then click the symbol on the bottom with the 3 dots to enter your custom/local server.

Ahhh, I missed that 3 dots. Thanks for the hint. I will add this info to our online docs as well, as for my impression it's a bid hidden, ending up probably that many users create a central account that is not required.

You know dietpi pretty well (obviously) and if the var/log/blynk directory is more consistent with the logs from other projects/programs then it would be fine.

/var/log is the default log directory according to the FHS (File Hierarchy Standard), so most UNIX systems and most programs that log to files use this in case. You will as well find several logs there on your system. On DietPi this directory is by default mounted to RAM to increase logging performance and reduce disk I/O. The files are cleared every hour to not fill the RAM unlimited. As long as there is no special reason for the Blynk logs to be persistent, especially survive during reboot, jep having logs in /var/log is the most consistent solution.

[Phil1988]

Ahhh, I missed that 3 dots. Thanks for the hint. I will add this info to our online docs as well, as for my impression it's a bid hidden, ending up probably that many users create a central account that is not required.

Might be a good idea :)

I just arrived back home and opened a report at blynks server github and checked the actual status of the server.
It sais:

Mai 10 16:37:52 DietPi systemd[1]: blynkserver.service: Main process exited, code=exited, status=143/n/a
Mai 10 16:37:52 DietPi systemd[1]: Stopped Blynk Server (DietPi).
Mai 10 16:37:52 DietPi systemd[1]: blynkserver.service: Unit entered failed state.
Mai 10 16:37:52 DietPi systemd[1]: blynkserver.service: Failed with result 'exit-code'.
Mai 10 16:37:52 DietPi systemd[1]: Started Blynk Server (DietPi).
Mai 10 16:37:54 DietPi java[9813]: Blynk Server  successfully started.
Mai 10 16:37:54 DietPi java[9813]: All server output is stored in folder '/mnt/dietpi_userdata/blynk/logs' file.
Mai 10 16:37:54 DietPi java[9813]: Generating own initial certificates...
Mai 10 16:37:54 DietPi java[9813]: Error during certificate generation.
Mai 10 16:37:54 DietPi java[9813]: user.pem (Keine Berechtigung)

'Keine Berechtigung' means 'no permission'

In the logs I can find this:

16:37:53.774 INFO - Region : local. Host : myREAL.DNSadress.de.
16:37:53.800 INFO - Using native epoll transport.
16:37:54.147 INFO - Initializing gmail smtp mail transport. Username : [email protected]. SMTP host : smtp.gmail.com:587
16:37:54.168 INFO - Reports : 0
16:37:54.168 INFO - Didn't find custom user certificates.
16:37:54.169 INFO - Didn't find Let's Encrypt certificates.
16:37:54.169 INFO - Automatic certificate generation is turned ON.
16:37:54.169 INFO - Using native openSSL provider.
16:37:54.169 WARN - ATTENTION. Server certificate paths (cert : '/', key : '/') not valid. Using embedded server certs and one way ssl. This is not secure. Please replace it with your own certs.
16:37:54.440 INFO - HTTP API and WebSockets server listening at 8080 port.
16:37:54.441 INFO - HTTPS API, WebSockets and Admin page server listening at 9443 port.
16:37:54.441 INFO - Mqtt hardware server listening at 8440 port.
16:37:54.448 INFO - Starting up certificate retrieval process for host myREAL.DNSadress.de and email [email protected].

Maybe you already have an idea how to solve this. Just wanted to let you know the issue.
Im now trying to get some information where the letsencrypt certificate is sitting and how I can make this work :)

EDIT3:
The Web admin interface doesnt work like it should.
So its 'normal' that changes on the server.properties at the WebUI doesnt effect the real server.properties file.

Anything I can do more for you, to test anything regarding this?
Otherwise I would say, that this issue is at least fixed with a workaround (meaning the log).
I would say, that a hint at the doc would be usefull again :)

[Phil1988]

Just a quick feedback.
I got SSL working now, as I copied the certificates to the /mnt/dietpi_userdata/blynk/SSLcertificates (that I created) and chmod and chown it.
Then setting the correct path to the certificates in the server.properties and first it did NOT work (after restart of course).

But I had to do some changes in the ESP Sketch.
As I read pretty often that the ESP8266 is on the weak side for SSL and might cause problems I used and ESP32 dev board.
Same problems, made it work (cause I knew the hardware cant be the issue there) and changed the ESP8266 accordingly.
Both running great and since 9 hours without a problem (looks to be stable tho).

Thanks @MichaIng for you fast responses and great help!
I do not only love dietpi but also the great work and kind help here for a non standard support (and this additionally on a free project).

If you need me to test something, let me know.
Otherwise this can be closed I guess?

[MichaIng]

@Phil1988
Wow these Espressif SOCs are really tiny ones. How do you run DietPi on them? Above I see VM, or was this just for testing reasons?

However I am glad it works now.

---

Just for reference, that the web UI config changes are expected to not work yet: https://github.com/blynkkk/blynk-server/issues/1318

• This is also something to mention in our online docs to avoid reasonable confusion.
• Done: https://dietpi.com/phpbb/viewtopic.php?p=5901#p5901

[Phil1988]

No no :D
The Espressif SOCs are just the blynk hardware (that will be controlled by the server/blynk/app)

Dietpi is running on a vm for the blynk server ;)

But the problem was to get a SSL connection between the blynkserver and the "blynk hardware" (what they call for the controled hardware).

By the way something more offtopic:
Yesterday I had set some timers and observed hard time shift between real world time and the dietpi servers time (it was ~55sec less on the dietpi server).

I then went to dietpi-config->advanced options and set "time sync mode" from "Boot + Daily" to "Deamon + Drift".
As far as I understand this sets up a daemon, that constantly checks the real time (from the internet) and drifts time shifts to make the times matching. (so this is the most accurate option)
Correct?

After this I still had 4 sec time shift.
Today I have already 46sec time shift between real time and the server time...

What is the RTC mode (currently "Emulated" instead of "Hardware") and is it possible to get the real time on the dietpi server?

Maybe its a problem that I am running a VM and I need to sync the time between the host and the VM or something?

[MichaIng]

@Phil1988

The Espressif SOCs are just the blynk hardware (that will be controlled by the server/blynk/app)

Ah jep understand now. Was a bid confused about the Blynk Server/target machine thing. First I though that one can only control the machine were the server is installed on. But going through the login/account issue above things became clear now: The server is for controlling multiple projects and authentication and from there you can connect to the actual target devices.

About time sync:

• Jep daemon mode is most accurate. We also have this active on the dietpi.com server. Open htop and you should see systemd-timesyncd running which should regularly adjust your system time.
• 46s is indeed quite much. You could check journalctl -u systemd-timesyncd to see the logs.
• If you require it most accurate, install NTP: G_AGI ntp
  • But then within dietpi-config choose custom/none as time sync mode, so it will not do any checks but instead relies on NTP doing that.

RTC is the hardware clock of the board. It is only required to set the system time on boot, when network is not yet up. Otherwise all scripts that run until network time sync would have the year 1970 as system time 😉. Since many SBCs do not have a real hardware clock (which requires a battery of course), there is an emulation package (fake-hwclock) which simply stores/restores the system time on shutdown. So on next boot the time will be the one from shutdown, until time sync is done. VirtualBox has an included virtual hwclock so there you can choose RTC hardware mode. This will simply remove the emulation package. But keeping it does not hurt as well.