Node disk space usage rapidly growing (1GB every 2-3 hours)

[DubbaThony]

• OpenEthereum version: 2.7.2-stable; 3.0.0
• Operating system: Linux (Debian)
• Installation: one-line installer for 2.7.2, for 3.0.0 unpacked binaries to /opt/parity and symlink to PATH
• Fully synchronized: yes
• Network: ethereum mainnet
• Restarted: yes

Parity node uses all SSD space we can provide to it. We ran out of our 500 GB ssd (we were running fatdb + tracing, 10 days ago we removed fatdb and tracing from config, and removed chain db). 10 after days of beeing fully synced node uses 400 GB which leaves around 3 GB free on system. Restarting node clears +- 90 GB of disk space but it fills back up relatively fast (1GB every 2-3 hours).

Expected:
Looking at Storage Requirements Overview I would expect node to take no more than 250 GB of space.

Actual behaviour:
Node will graduately fill up disk to point where is nothing available at all

Parity parameters:

/opt/parity/parity --pruning fast --fat-db=off --tracing off --scale-verifiers --pruning-memory=1024 --can-restart --cache-size-state=512 --cache-size-db=8192 --tx-queue-size 8196 --no-secretstore --no-ipc --no-ws --allow-ips=public --min-peers=128 --max-peers=256 --max-pending-peers=256 --auto-update none --no-download --jsonrpc-interface=10.0.0.2

[dvdplm]

Your settings look sane, so not sure what's going on here. Can you share your rocksdb log file? It should be here on linux: ~/.local/share/paritychains/ethereum/db/<your id>/overlayrecent/db/LOG.

Can you also try to start a fresh sync into a different folder, using --base-path=/path/to/somewhere/else and check if it also increases the disk space used quicker than expected?

You might consider upgrading to openethereum 3.0 (released yesterday) as well, but be aware it is a one way upgrade so make sure you have a backup of your data should you ever need to revert back to 2.7.2.

[DubbaThony]

It is resync after insane growth.
This LOG file is very dynamically changing. 5 minutes ago it was 700 KB, than I pulled it out from server it was 74KB, now its 984 bytes. Does it make it still usefull? I guess its purged often.

EDIT: I underestimated how fast the storage is used up, its been 3 GB in 2 hours.

EDIT2: I think it was hectic behaviour of scp on live file

LOG.txt

[DubbaThony]

Also I haven't addressed about the upgrade, Im not planning right now to upgrade node on production enviroment for now (we prefer to wait some time after major update to see if there are any potentially blocker issues reported by community)

[DubbaThony]

Update:

after checking it against version 3.0.0 issue still persists.