`DatabaseHandles' in ethconfig.Config cannot be set through TOML

[barwod]

System information

Geth
Version: 1.10.13-stable
Git Commit: 7a0c19f
Architecture: amd64
Go Version: go1.17.3
Operating System: linux
GOPATH=
GOROOT=go

Expected behaviour

Close files that aren't in use / have a settable limit

Actual behaviour

Every database file it opens stays open for the duration of the sync

Steps to reproduce the behaviour

Snap sync the blockchain on an AWS EFS volume

Backtrace

I use @holiman recomendation from issue #23922
but i get error:

"Fatal: /root/.ethereum/config.toml, line 13: field corresponding to `DatabaseHandles' in ethconfig.Config cannot be set through TOML"

My config.toml:

[Eth]
NetworkId = 4
SyncMode = "snap"
EthDiscoveryURLs = ["enrtree://AKA3AM6LPBYEUDMVNU3BSVQJ5AD45Y7YPOHJLEF6W26QOE4VTUDPE@all.rinkeby.ethdisco.net"]
SnapDiscoveryURLs = ["enrtree://AKA3AM6LPBYEUDMVNU3BSVQJ5AD45Y7YPOHJLEF6W26QOE4VTUDPE@all.rinkeby.ethdisco.net"]
NoPruning = false
NoPrefetch = false
TxLookupLimit = 2350000
LightPeers = 100
UltraLightFraction = 75
DatabaseCache = 512
DatabaseFreezer = ""
DatabaseHandles = 30000
TrieCleanCache = 154
TrieCleanCacheJournal = "triecache"
TrieCleanCacheRejournal = 3600000000000
TrieDirtyCache = 256
TrieTimeout = 3600000000000
SnapshotCache = 102
Preimages = false
EnablePreimageRecording = false
RPCGasCap = 50000000
RPCEVMTimeout = 5000000000
RPCTxFeeCap = 1e+00

Any idea how to fix this?

[ligi]

thanks - seems we need to add it to the TOML - as a workaround you can set it via CLI

[barwod]

How? Could you please provide this command here?

[kkosowsk]

hello, I have the same problem, but I don't see an option to set this parameter via CLI?

[holiman]

RIght, no that's true, it's not settabie, not via config nor cli. It's calculated on the fly,

[holiman]

I think a valid solution would be, not to set the max database handles explicitly, but rather to override the max file limit. So instead of geth checking the max file limit and deriving the db handle limit from that, we'd check use the smallest of fdlimit and override_fd_limit and use that to derive the db handle limit.

[eval-exec]

I think a valid solution would be, not to set the max database handles explicitly, but rather to override the max file limit. So instead of geth checking the max file limit and deriving the db handle limit from that, we'd check use the smallest of fdlimit and override_fd_limit and use that to derive the db handle limit.

Hello holiman, I would like to solve this issue.
I found DatabaseHandles was set by MakeDatabaseHandles() at here:

go-ethereum/cmd/utils/flags.go

Lines 1058 to 1070 in fb3a652

	// MakeDatabaseHandles raises out the number of allowed file handles per process
	// for Geth and returns half of the allowance to assign to the database.
	func MakeDatabaseHandles() int {
	limit, err := fdlimit.Maximum()
	if err != nil {
	Fatalf("Failed to retrieve file descriptor allowance: %v", err)
	}
	raised, err := fdlimit.Raise(uint64(limit))
	if err != nil {
	Fatalf("Failed to raise file descriptor allowance: %v", err)
	}
	return int(raised / 2) // Leave half for networking and other stuff
	}

but, how can I derive the db handle limit from the smallest of fdlimit and override_fd_limit ?
I think the smallest fdlimit of a process is uncertain.

[joaoepj]

but, how can I derive the db handle limit from the smallest of fdlimit and override_fd_limit ?
I think the smallest fdlimit of a process is uncertain.

@eval-exec , I think this issue only arises on very busy nodes running under restricted enviroments like AWS. Using the minimum value between limit and DatabaseHandles as argument to fdlimit.Raise() would suffice to avoid the postponed filesystem errors. In the other hand, geth should free file descriptors as soon as possible.

[tuanggolt]

open

[s1na]

Closed by #24477.