Failed start with json.decoder.JSONDecodeError

[olm52]

Sometime it works, sometime not. This is the error:

feb 01 16:48:44 archlinux python[1003]: json_object = json.loads(content, object_pairs_hook=OrderedDict)
feb 01 16:48:44 archlinux python[1003]: File "/usr/lib/python3.8/json/init.py", line 370, in loads
feb 01 16:48:44 archlinux python[1003]: return cls(**kw).decode(s)
feb 01 16:48:44 archlinux python[1003]: File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
feb 01 16:48:44 archlinux python[1003]: obj, end = self.raw_decode(s, idx=_w(s, 0).end())
feb 01 16:48:44 archlinux python[1003]: File "/usr/lib/python3.8/json/decoder.py", line 353, in raw_decode
feb 01 16:48:44 archlinux python[1003]: obj, end = self.scan_once(s, idx)
feb 01 16:48:44 archlinux python[1003]: json.decoder.JSONDecodeError: Invalid \escape: line 3 column 67 (char 113)

[bugy]

Hi @hifi25nl , thanks for reporting!
Unfortunately I cannot see from the logs when it's happening. When you start the server? Or when you try to open or execute some script?
Also I don't see any script server sources in the stack trace. Are there any other logs?

[bugy]

@hifi25nl I think the problem is in file conf/logging.json
Did you do any changes to it? Seems that your json is corrupted

[olm52]

No, never touched it. I want to copy my previous configuration to a new computer.

Maybe my procedure is not correct:

1. Install script-server on new PC
2. Copy the files in /conf/runners and conf.json to the new installation
3. Run as root the command /usr/bin/python /opt/script-server/tools/init.py --dev --no-npm

After I start script-server as user with /usr/bin/python /opt/script-server/launcher.py

[bugy]

Hi @hifi25nl, from which working directory are you starting the server? I mean /usr/bin/python /opt/script-server/launcher.py
Also, are you using git clone or downloading git sources somehow?
If you take release files, you don't need to run init.py

[olm52]

I am using git clone of the address https://github.com/bugy/script-server.git
Really I am trying to make a PKGBUILD for Archlinux of git version. I had to chmod temp and logs directory otherwise I have permission problems.

PKGBUILD:
pkgname=scriptserver-git
_gitname=script-server
pkgver=r371.78da638
pkgrel=1
pkgdesc="Web UI for your scripts with execution management"
provides=('scriptserver')
conflicts=('scriptserver')
arch=('armv7h' 'x86_64')
license=('Apache License, Version 2.0')
url="https://github.com/bugy/script-server"
depends=('python-tornado')
source=('git+https://github.com/bugy/script-server.git' 'scriptserver.service')
install=scriptserver.install
md5sums=('SKIP' 'SKIP')

pkgver() {
cd $_gitname
printf "r%s.%s" "$(git rev-list --count HEAD)" "$(git rev-parse --short HEAD)"
}

package () {
#tar -xf conf.tar.gz -C "$srcdir/$_gitname"
install -Dm644 $srcdir/scriptserver.service $pkgdir/usr/lib/systemd/user/scriptserver.service
install -Dm644 $srcdir/$_gitname/LICENSE $pkgdir/usr/share/licenses/$pkgname/COPYING
mkdir -p $pkgdir/opt/script-server
cp -r $srcdir/script-server/* $pkgdir/opt/script-server/
mkdir -p $pkgdir/opt/script-server/logs
chmod 777 $pkgdir/opt/script-server/logs
mkdir -p $pkgdir/opt/script-server/temp
chmod 777 $pkgdir/opt/script-server/temp
}

SYSTEMD SERVICE:
[Unit]
Description=Script server

[Service]
Type=simple
ExecStart=/usr/bin/python /opt/script-server/launcher.py

[Install]
WantedBy=default.target

INSTALL FILE (to be executed after installation or upgrade)
post_install() {
/usr/bin/python /opt/script-server/tools/init.py --dev --no-npm
}

post_upgrade() {
/usr/bin/python /opt/script-server/tools/init.py --dev --no-npm
}

[bugy]

Hi @hifi25nl,

regarding the initial problem with json decoding. Could it be, that you have a bit larger log? From this log I cannot see, which json file is corrupted

Run as root the command /usr/bin/python /opt/script-server/tools/init.py --dev --no-npm

I think it's better to always run everything as a normal user, otherwise, you may end up with some files created by root, and the normal user won't have access to them (that's why you have to chmod it)

I am using git clone of the address

I would recommend using releases, instead of building the tool yourself. You just need to download and extract it. You can also extract server version from version.txt file, which is bundled now
(but to be honest, I never worked with pkgbuild, May be this approach is not possible there)

[olm52]

I think that I know the source of the problem.
If I update the package with my runners folder and conf.json already there, launch will fail.
To make it work I must remove or copy to another directory runners and conf.json
After I start the launcher, copy back runners and conf.json, restart the launcher and all is working fine.

I am using now the package at https://github.com/bugy/script-server/archive/1.15.1.tar.gz
With this package I must run
/usr/bin/python /opt/script-server/tools/init.py --dev --no-npm
because the directory /web is missing

[bugy]

Hi @hifi25nl, please use: https://github.com/bugy/script-server/releases/download/1.15.1/script-server.zip
/archive/...tar.gz are sources

(it wasn't available for this release, because of an error during upload yesterday. Fixed now)

[olm52]

Ok, I have rebuild the package with the new link and made a new systemd service that will launch script-server as root.
The same problem occurs. To be clear:

WITH RUNNERS NOT DELETED
[root@archlinux scriptserver]# /usr/bin/python /opt/script-server/launcher.py
2020-02-02 15:20:25,872 [root.INFO] Starting Script Server, v1.15.1
2020-02-02 15:20:25,872 [migrations.INFO] Applying migration migrate_output_files_parameters_substitution
Traceback (most recent call last):
File "/opt/script-server/launcher.py", line 11, in
main.main()
File "src/main.py", line 77, in main
migrations.migrate.migrate(TEMP_FOLDER, CONFIG_FOLDER, SERVER_CONF_PATH, LOG_FOLDER)
File "src/migrations/migrate.py", line 315, in migrate
migration_descriptor.callable(context)
File "src/migrations/migrate.py", line 249, in __introduce_access_config
json_object = json.loads(content, object_pairs_hook=OrderedDict)
File "/usr/lib/python3.8/json/init.py", line 370, in loads
return cls(**kw).decode(s)
File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.8/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Invalid \escape: line 3 column 67 (char 113)

After, if I run one first time without runners and conf.son and copy back runners and conf.json and restart the launcher all is working fine.
It seems that I must run a first time without my scripts in runners.

[bugy]

Hi @hifi25nl , thanks, that helps!
The problem is that one of the script configurations (runners/*.json) is not readable and automatic migrations fail. I'll fix it for dev version

In your case, could you find broken configuration and fix it, please? I think the easiest way would be try to open a main server page (index.html) and then check logs/server.log file. There should be an error, showing which scripts failed to be open

[olm52]

I have found the problematic file in /runners looking at server.log
Now it seems working fine, thanks

[bugy]

Fixed. Now migrations shouldn't fail startup