Pending Log Subscription emits logs at extremely slow rate ( roughly 1 pending transaction per second ) #29415
Comments
|
Can you check the contents of your txpool and how well your node is peered? I have a hunch your node is not receiving many pending txes. |
|
Peer connection doesn't seem to be the issue. I have attempted to debug this issue by running the same pending log subscription snippet code against different webscoket connections and eyeballing the printout to get a idea of the throughput.
Observations: Throughput of 3 and 4 are fastest and mathced my impression of my local throughput prior to dencun upgrade. Local pendingLog Throughput local-pendinglogs.mp4Chainstack pendingLog Throughput chainstack-pendinglogs.mp4
Can you suggest what would be the canonical way to do so ? I try to approximate the pending txn throughput of txpool by printing the pending transactions Local PendingTx local-pendingTx.mp4Chainstack PendingTx chainstack-pendingTx.mp4The pending txn throughput through my local tx pool looks to be normal. Here is the config.toml of my geth node |
|
Are you running the current master? We recently changed how pending subscriptions work which makes them a lot slower |
|
Yes. I am running a382917 On which commit was the change enacted ? Is the slow down permanent ? (next release will have the slow version) |
|
Yep, it was changed on this commit: #28623 |
|
I tried play with recommit value. Reduce it by a factor of 5 and the pending log is still very slow. If I reduce it by a factor of 10 then I won't be able to get any pending logs at all, seeing Looks like the only way to get the old throughput is to create a fork myself. Thanks for the info though. Closing this issue. |
|
Are your sure your local geth is fully synced? |
|
Yes. blockHeight looks right. Javascript console says it is not syncing and log doesn't indicate anything funny is going on. Is the pending log subscription throughput abnormally slow even consider the changes we made in #28623 ? |
|
I will check it out, the pending throughput is expected to be higher. |
|
Originally the pending logs are streamed out whenever they are available. Specifically, the pending logs will be emitted when new transaction arrives and be packed into pending block. In the recent miner rewrite, the pending logs are only emitted once a new chain head is produced, with the available transactions in the pool. During the next 12 second time-window, the newly arrived transactions are silently ignored. Will discuss in the issue triage. |
|
Is the current throughput considered unacceptable / buggy ? If so, then I expect a remediation in the future so I don't have to spend my time to fork it . My usecase is that I need to react to live pendingLog as soon as they arrive. The current throughput makes GETH un-usable for that purpose |
|
This was never reliable, and often not useful, since it only ever spat out events for the ~200txs that were possible to include in the pending block. It would not emit logs for all things going into the pool. With the rewrite, this subscription is now moot, and we should fully remove support for it. |
package main
import (
"context"
"fmt"
"log"
"math/big"
"time"
"github.com/ethereum/go-ethereum"
"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/ethclient"
"github.com/ethereum/go-ethereum/rpc"
"github.com/ethereum/go-ethereum/types"
)
const (
LOCAL_GETH_WS_CONN = "ws://localhost:8546"
)
func GetProvider(conn string) *ethclient.Client {
client, err := rpc.Dial(conn)
if err != nil {
log.Fatalf("连接到geth节点的ws失败。错误=%v 连接=%v\n", err, conn)
return nil
}
return ethclient.NewClient(client)
}
func MinimumLoop() {
localCh := make(chan types.Log)
provider := GetProvider(LOCAL_GETH_WS_CONN)
defer provider.Close()
localSub, err := provider.SubscribeFilterLogs(context.Background(), ethereum.FilterQuery{
FromBlock: nil,
ToBlock: nil,
Addresses: []common.Address{},
Topics: [][]common.Hash{},
}, localCh)
if err != nil {
log.Fatalf("订阅失败：%v", err)
}
defer localSub.Unsubscribe()
for {
select {
case logLocal := <-localCh:
fmt.Println(logLocal.TxHash.Hex())
case err := <-localSub.Err():
log.Fatalf("订阅错误：%v", err)
}
}
}
func main() {
MinimumLoop()
}The code utilizes the |
|
Assigned to @fjl to remove it |
|
Removing pending logs: #29574 |
|
Closing this, we've yeeted pending logs now |
System information
CL client & version: e.g. [email protected]
OS & Version: LINUX
Commit hash : a382917
Running this minimal snippet to subscribe to logs emitted by pending transactions.
Expected behaviour
Volume of logs being printed from the loop should match number of actual transaction entering the mempool.
Actual behaviour
Pending logs are printed out very very slowly. At a rate of roughly 1 unique transaction per second. Something is definitely wrong,
The text was updated successfully, but these errors were encountered: