Mirrord fails on incoming traffic when running a react app

[t4lz]

Bug Description

When I'm running the template react app with mirrord, and send an http request to the target service, mirrord exits with an error, both in steal and in mirror mode.

mirrord has encountered an error and is now exiting.
thread 'tokio-runtime-worker' panicked at 'explicit panic', mirrord/layer/src/lib.rs:814:5

Steps to Reproduce

npx create-react-app my-app

cd my-app

MIRRORD_AGENT_RUST_LOG=mirrord=trace PORT=80 RUST_LOG=mirrord=trace MIRRORD_OPERATOR_ENABLE=false mirrord exec --agent-ttl=60 --fs-mode=local -t pod/http-echo-3dmcsq0-646b6bdf8c-jztqc npm -- start

curl localhost:31945

(whatever port the target service is listening on).

Backtrace

No response

Relevant Logs

2023-06-08T12:57:11.584607Z TRACE ThreadId(02) handle_daemon_message:handle_daemon_message: mirrord_layer::tcp: new daemon_message=Tcp(NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 })) NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 })
2023-06-08T12:57:11.584629Z TRACE ThreadId(02) handle_daemon_message:handle_daemon_message:handle_new_connection: mirrord_layer::tcp_mirror: new daemon_message=Tcp(NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 })) NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 }) tcp_connection=NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 }
2023-06-08T12:57:11.584649Z TRACE ThreadId(02) handle_daemon_message:handle_daemon_message:handle_new_connection:create_local_stream: mirrord_layer::tcp: new daemon_message=Tcp(NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 })) NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 }) tcp_connection=NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 } tcp_connection=NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 }
2023-06-08T12:57:11.584659Z TRACE ThreadId(02) handle_daemon_message:handle_daemon_message:handle_new_connection:create_local_stream:new: mirrord_layer::socket: new daemon_message=Tcp(NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 })) NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 }) tcp_connection=NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 } tcp_connection=NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 } remote_address=192.168.65.4:48136 local_address=10.1.20.74:80
2023-06-08T12:57:11.584712Z TRACE ThreadId(02) handle_daemon_message:handle_daemon_message:handle_new_connection:create_local_stream:new: mirrord_layer::socket: close time.busy=1.33µs time.idle=50.7µs daemon_message=Tcp(NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 })) NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 }) tcp_connection=NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 } tcp_connection=NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 } remote_address=192.168.65.4:48136 local_address=10.1.20.74:80
2023-06-08T12:57:11.585508Z TRACE ThreadId(02) handle_daemon_message:handle_daemon_message:handle_new_connection:create_local_stream:add: mirrord_layer::socket: new daemon_message=Tcp(NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 })) NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 }) tcp_connection=NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 } tcp_connection=NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 } id=SocketId(0) info=SocketInformation { remote_address: 192.168.65.4:48136, local_address: 10.1.20.74:80 }
2023-06-08T12:57:11.585552Z TRACE ThreadId(02) handle_daemon_message:handle_daemon_message:handle_new_connection:create_local_stream:add: mirrord_layer::socket: close time.busy=15.4µs time.idle=30.7µs daemon_message=Tcp(NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 })) NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 }) tcp_connection=NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 } tcp_connection=NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 } id=SocketId(0) info=SocketInformation { remote_address: 192.168.65.4:48136, local_address: 10.1.20.74:80 }
2023-06-08T12:57:11.585744Z TRACE ThreadId(02) handle_daemon_message:handle_daemon_message:handle_new_connection:create_local_stream: mirrord_layer::tcp: close time.busy=1.08ms time.idle=15.9µs daemon_message=Tcp(NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 })) NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 }) tcp_connection=NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 } tcp_connection=NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 }
2023-06-08T12:57:11.585757Z TRACE ThreadId(02) handle_daemon_message:handle_daemon_message:handle_new_connection: mirrord_layer::tcp_mirror: close time.busy=1.11ms time.idle=23.0µs daemon_message=Tcp(NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 })) NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 }) tcp_connection=NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 }
2023-06-08T12:57:11.585772Z DEBUG ThreadId(02) handle_daemon_message:handle_daemon_message: mirrord_layer::tcp: handle_incoming_message -> handled Err(
    IO(
        Os {
            code: 61,
            kind: ConnectionRefused,
            message: "Connection refused",
        },
    ),
) daemon_message=Tcp(NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 })) NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 })
2023-06-08T12:57:11.585839Z TRACE ThreadId(02) handle_daemon_message:handle_daemon_message: mirrord_layer::tcp: close time.busy=1.21ms time.idle=25.9µs daemon_message=Tcp(NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 })) NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 })
2023-06-08T12:57:11.585852Z TRACE ThreadId(02) handle_daemon_message: mirrord_layer: close time.busy=1.24ms time.idle=30.5µs daemon_message=Tcp(NewConnection(NewTcpConnection { connection_id: 0, remote_address: 192.168.65.4, destination_port: 80, source_port: 48136, local_address: 10.1.20.74 }))
2023-06-08T12:57:11.585864Z ERROR ThreadId(02) mirrord_layer: Error handling daemon message: IO(Os { code: 61, kind: ConnectionRefused, message: "Connection refused" })

Your operating system and version

macOS 13.4

Local process

node

Local process version

No response

Additional Info

Discovered while trying to reproduce #1524.