[bug] Request failed with status code 404 when opening files from Control panel

[maivorbim]

Problem/Motivation

When trying to open any file (such as nodes.json) from ZWaveJS2MQTT control panel > Store I receive the following error message: Request failed with status code 404

ZwaveJS2mqtt addon shows the following message in the logs:

2021-01-29 11:37:01.456 INFO APP: GET /api/store//data/store/settings.json 404 1.298 ms - 136 Error: Not Found at /opt/app.js:443:15 at Layer.handle [as handle_request] (/opt/node_modules/express/lib/router/layer.js:95:5) at trim_prefix (/opt/node_modules/express/lib/router/index.js:317:13) at /opt/node_modules/express/lib/router/index.js:284:7 at Function.process_params (/opt/node_modules/express/lib/router/index.js:335:12) at next (/opt/node_modules/express/lib/router/index.js:275:10) at cors (/opt/node_modules/cors/lib/index.js:188:7) at /opt/node_modules/cors/lib/index.js:224:17 at originCallback (/opt/node_modules/cors/lib/index.js:214:15) at /opt/node_modules/cors/lib/index.js:219:13 at optionsCallback (/opt/node_modules/cors/lib/index.js:199:9) at corsMiddleware (/opt/node_modules/cors/lib/index.js:204:7) at Layer.handle [as handle_request] (/opt/node_modules/express/lib/router/layer.js:95:5) at trim_prefix (/opt/node_modules/express/lib/router/index.js:317:13) at /opt/node_modules/express/lib/router/index.js:284:7 at Function.process_params (/opt/node_modules/express/lib/router/index.js:335:12)

Using Z-Wave JS to MQTT addon version 0.2.0 on core-2021.2.0b0 supervisor-2021.01.8.

This error was not received when using portainer and https://hub.docker.com/r/zwavejs/zwavejs2mqtt.

Expected behavior

Files should open without any errors.

[frenck]

Yes this, seem to be an upstream issue, related to the new STORE_DIR environment variable introduced in the latest version.

The add-on relies on that functionality.

[robertsLando]

Any news on this? @frenck

[frenck]

Didn't manage to dive in yesterday. So, no news yet.

[maivorbim]

quick update: selecting a file and deleting/downloading it via bottom right hand corner wheel works fine, it is only when directly clicking on a file / clicking trash can icon a 404 error is received.

[robertsLando]

selecting a file and deleting/downloading it via bottom right hand corner wheel works fine, it is only when directly clicking on a file / clicking trash can icon a 404 error is received.

Because that is not a GET request and the path is not in the url in that case. The problem is that for some reasons the url is decoded by some middleware, it's not something on zwavejs2mqtt side

[mikepowell]

quick update: selecting a file and deleting/downloading it via bottom right hand corner wheel works fine, it is only when directly clicking on a file / clicking trash can icon a 404 error is received.

When I do this I seem to only be getting empty zero-byte files downloaded, even though the files are shown with nonzero sizes in the web UI.

[robertsLando]

I'm not able to reproduce this on my side, as I said it's something weird with the addon, maybe some middleware that parses the get requests and decodes the path

[dodepier]

I have also the same issue. In addition I would like to add a custom device. Therefore I need to add a file in the folder. Is it possible to add a function to upload or create a new file?

https://zwave-js.github.io/zwavejs2mqtt/#/guide/homeassistant?id=zwave-js-server

You can specify your custom devices configuration inside store/customDevices(.js|.json) file. This allows users that use Docker to create their custom hass devices configuration without the need to build a new container. If using .json format zwavejs2mqtt will watch for file changes and automatically load new components on runtime without need to restart the application.

[robertsLando]

ATM there is not such support but I could think about it

[dodepier]

Another option is to connect the store folder to the config folder of home assistant so one can edit the file with ssh or in the build-in file editor of home assistant

[byackee]

Another option is to connect the store folder to the config folder of home assistant so one can edit the file with ssh or in the build-in file editor of home assistant

Hi,
How do you that?
Thx

[truglodite]

I am having the same problem. As frenck mentioned above, it appears to be permissions related after looking in chrome dev tools. I'm using Hassos. I think the addon has to store in /config, otherwise there is no permission. This is making it impossible to debug/verify some things on my zwave network after moving from zw1.4 to zjs2m.

[TheDK]

I have the same issue on HA OS (as probably everyone running it this way).

As @frenck mentioned downloading the file marking, clicking the options wheel bottom right and select download works (at least in Chrome, in FireFox it produces a weird bug trying to download a file called zwavejs2mqtt-store.xlsx --> Microsoft Excel?! :) ) so we have a workaround.

So I'd say it's not critical, just not very convenient reading logs by downoading .zip files first. For people starting out new (like me) it also took 30min of try and error, google search without result (even though I used the same error message as above) before landing here by chance when I wanted to open an issue and found this one.

Is there a more elegant way to connect to the file system of the ZWaveJS2MQTT container using ssh, ftp or something like that?

Thanks guys!

[scottdfedorov]

+1 for the issue here. Also ran into the same issue @mikepowell mentioned:

When I do this I seem to only be getting empty zero-byte files downloaded, even though the files are shown with nonzero sizes in the web UI.

but based on @TheDK's reply, I realized I was clicking the download button on the blank file displayed on the page. As they noted, clicking the gear, then selecting the download button does in fact download the file as a zip.

An inconvenient, but functional workaround for now I suppose and wanted to highlight for other users noticing the same.

[robertsLando]

@frenck FYI, this is the solution to fix the problem: zwave-js/zwave-js-ui#1013

[frenck]

@robertsLando Thanks!

However, the "fix" proposed there is uh... creative 😬

[robertsLando]

However, the "fix" proposed there is uh... creative

What you mean? What would be a better fix in your opinion?

[frenck]

I dunno at this point!

I'm just saying it feels odd (I'm not saying it is wrong).

[robertsLando]

I searched a lot and seems that it is the only way to fix that kind of problem.

Another solution could be to encode path using base64 instead of using encodeUri on my side.

[frenck]

Another solution could be to encode path using base64 instead of using encodeUri on my side.

That feels equally odd 🤷‍♂️
I'll try to apply the patch today or tomorrow (still running behind from the weekend)

[frenck]

SO I think I've spent around 3 hours on this today now...

The problem is similar, but definitely not the Nginx reverse proxy in this case. There are actually 3 proxies in play here, two of which are based on the Python aiohttp. I suspect it gets lost in there already giving NGinx not even a shot at screwing it up.

I'm not aware of encoding it again using bare NGinx, I could do that through Lua, but Lua can be really problematic on AARCH64 systems (Issues with the Lua API having a bad light userdata pointer, which just keeps on popping up the past years).

So I'm back at square one.

[robertsLando]

hummm, I think the BASE64 solution could be easier to sort this out?

[frenck]

Maybe, or not making the file part of the URI path, but passing it as a query string. As a query string parameter, the encoding doesn't matter that much in that case (as it is still just a query param).

The problem right now, is passing a full path, as part of a path (if that makes sense?)

[robertsLando]

not 100% sure that the query will work, it may be decoded too like it happened with the url? If not I can use that method as it makes more sense then the base64 encoding (last chance could be to set it in a header)

[frenck]

not 100% sure that the query will work, it may be decoded too like it happened with the URL?

Query strings are handled differently.

So right now it's like this-ish: http://example.com/api/store/<path with encoded slashes to prevent is becoming an actual part of the path>/

There is a "path" like variable hidden in a single fragment of the URL path. The path of the URL will be processed, as it is considered a path, not data (which is normal to happen actually). HTTP wasn't designed to have data in the path (the era before we start handling URLs dynamically...).

A query string, however, is handled like actual data and, if passed along, will be encoded.

[robertsLando]

Ok nice, I will go for it so!

[robertsLando]

This can be closed 🚀

[TheDK]

Looking forward to the update of the HA add-on!

[frenck]

fixed in v0.12.0