The service name cannot be resolved when using a custom network in podman-compose #25032
Labels
kind/bug
Categorizes issue or PR as related to a bug.
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Issue Description
root@ubuntu:
/agentos# podman-compose -v/agentos# podman versionpodman-compose version: 1.0.6
['podman', '--version', '']
using podman version: 4.9.5
podman-compose version 1.0.6
podman --version
podman version 4.9.5
exit code: 0
root@ubuntu:
Client: Podman Engine
Version: 4.9.5
API Version: 4.9.5
Go Version: go1.20.14
Git Commit: 1d51e91
Built: Fri Jan 17 02:06:59 2025
OS/Arch: linux/amd64
root@ubuntu:~/agentos# podman info
host:
arch: amd64
buildahVersion: 1.33.8
cgroupControllers:
cgroupManager: systemd
cgroupVersion: v1
conmon:
package: Unknown
path: /usr/local/libexec/podman/conmon
version: 'conmon version 2.1.12, commit: unknown'
cpuUtilization:
idlePercent: 79.34
systemPercent: 7.79
userPercent: 12.87
cpus: 4
databaseBackend: boltdb
distribution:
codename: focal
distribution: ubuntu
version: "20.04"
eventLogger: journald
freeLocks: 1945
hostname: ubuntu
idMappings:
gidmap: null
uidmap: null
kernel: 5.4.0-204-generic
linkmode: dynamic
logDriver: journald
memFree: 6116114432
memTotal: 8331362304
networkBackend: cni
networkBackendInfo:
backend: cni
dns:
package: podman-plugins_100:1.2.0-1_amd64
path: /usr/libexec/cni/dnsname
version: CNI dnsname plugin version unknown
package: 'podman-plugins, podman-machine-cni: /usr/libexec/cni'
path: /usr/libexec/cni
ociRuntime:
name: crun
package: crun_100:1.3-1_amd64
path: /usr/bin/crun
version: |-
crun version 1.3.7-506ba
commit: 7b4a7042370eea7fb00d3a4da34332b26f080acd
spec: 1.0.0
+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
os: linux
pasta:
executable: ""
package: ""
version: ""
remoteSocket:
exists: false
path: /run/podman/podman.sock
security:
apparmorEnabled: false
capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
rootless: false
seccompEnabled: true
seccompProfilePath: /usr/share/containers/seccomp.json
selinuxEnabled: false
serviceIsRemote: false
slirp4netns:
executable: ""
package: ""
version: ""
swapFree: 4294963200
swapTotal: 4294963200
uptime: 0h 31m 27.00s
variant: ""
plugins:
authorization: null
log:
network:
volume:
registries:
search:
store:
configFile: /etc/containers/storage.conf
containerStore:
number: 0
paused: 0
running: 0
stopped: 0
graphDriverName: overlay
graphOptions:
overlay.mountopt: nodev,metacopy=on
graphRoot: /var/lib/containers/storage
graphRootAllocated: 314865221632
graphRootUsed: 24114126848
graphStatus:
Backing Filesystem: extfs
Native Overlay Diff: "false"
Supports d_type: "true"
Supports shifting: "false"
Supports volatile: "false"
Using metacopy: "true"
imageCopyTmpDir: /var/tmp
imageStore:
number: 11
runRoot: /run/containers/storage
transientStore: false
volumePath: /var/lib/containers/storage/volumes
version:
APIVersion: 4.9.5
Built: 1737079619
BuiltTime: Fri Jan 17 02:06:59 2025
GitCommit: 1d51e91
GoVersion: go1.20.14
Os: linux
OsArch: linux/amd64
Version: 4.9.5
The service name cannot be resolved when using a custom network in podman-compose: {"level":"warn","ts":"2025-01-17T02:51:08.616Z","logger":"etcd-client","caller":"[email protected]/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc000da4540/agentos-etcd:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = "transport: Error while dialing: dial tcp: lookup agentos-etcd on 100.200.4.254:53: dial udp 100.200.4.254:53: connect: network is unreachable""}
docker-compose.yaml such as :
Milvus vector database services
agentos-etcd:
#container_name: milvus-etcd
image: quay.io/coreos/etcd:v3.5.5
environment:
ETCD_AUTO_COMPACTION_MODE: ${ETCD_AUTO_COMPACTION_MODE:-revision}
ETCD_AUTO_COMPACTION_RETENTION: ${ETCD_AUTO_COMPACTION_RETENTION:-1000}
ETCD_QUOTA_BACKEND_BYTES: ${ETCD_QUOTA_BACKEND_BYTES:-4294967296}
ETCD_SNAPSHOT_COUNT: ${ETCD_SNAPSHOT_COUNT:-50000}
volumes:
- ./volumes/milvus/etcd:/etcd
command: etcd -advertise-client-urls=http://127.0.0.1:2379 -listen-client-urls http://0.0.0.0:2379 --data-dir /etcd
healthcheck:
test: ['CMD', 'etcdctl', 'endpoint', 'health']
interval: 30s
timeout: 20s
retries: 3
networks:
- agentos-milvus
agentos-minio:
#container_name: milvus-minio
image: minio/minio:RELEASE.2023-03-20T20-16-18Z
environment:
MINIO_ACCESS_KEY: ${MINIO_ACCESS_KEY:-minioadmin}
MINIO_SECRET_KEY: ${MINIO_SECRET_KEY:-minioadmin}
volumes:
- ./volumes/milvus/minio:/minio_data
command: minio server /minio_data --console-address ":9001"
healthcheck:
test: ['CMD', 'curl', '-f', 'http://localhost:9000/minio/health/live']
interval: 30s
timeout: 20s
retries: 3
networks:
- agentos-milvus
agentos-milvus:
#container_name: milvus
image: milvusdb/milvus:v2.3.1
command: ['milvus', 'run', 'standalone']
environment:
ETCD_ENDPOINTS: ${ETCD_ENDPOINTS:-agentos-etcd:2379}
MINIO_ADDRESS: ${MINIO_ADDRESS:-agentos-minio:9000}
MINIO_ACCESS_KEY: ${MINIO_ACCESS_KEY:-minioadmin}
MINIO_SECRET_KEY: ${MINIO_SECRET_KEY:-minioadmin}
common.security.authorizationEnabled: ${MILVUS_AUTHORIZATION_ENABLED:-true}
volumes:
- ./volumes/milvus/milvus:/var/lib/milvus
healthcheck:
test: ['CMD', 'curl', '-f', 'http://localhost:9091/healthz']
interval: 30s
start_period: 90s
timeout: 20s
retries: 3
#ports:
# - 29530:19530
# - 29091:9091
links:
- agentos-etcd
- agentos-minio
depends_on:
- agentos-etcd
- agentos-minio
networks:
- agentos-milvus
networks:
create a network between sandbox, api and ssrf_proxy, and can not access outside.
agentos_ssrfproxy_network:
driver: bridge
internal: true
agentos-milvus:
driver: bridge
internal: true
default:
driver: bridge
internal: false
Steps to reproduce the issue
Steps to reproduce the issue
Describe the results you received
Describe the results you received
6cd2516f649f localhost/milvusdb/milvus:v2.3.1 milvus run standa... 4 minutes ago Up 4 minutes (unhealthy) agentos_agentos-milvus_1
{"level":"warn","ts":"2025-01-17T02:51:08.616Z","logger":"etcd-client","caller":"[email protected]/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc000da4540/agentos-etcd:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = "transport: Error while dialing: dial tcp: lookup agentos-etcd on 100.200.4.254:53: dial udp 100.200.4.254:53: connect: network is unreachable""}
{"level":"warn","ts":"2025-01-17T02:51:08.699Z","logger":"etcd-client","caller":"[email protected]/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc000c5c540/agentos-etcd:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = "transport: Error while dialing: dial tcp: lookup agentos-etcd on 100.200.4.254:53: dial udp 100.200.4.254:53: connect: network is unreachable""}
[2025/01/17 02:51:08.699 +00:00] [ERROR] [sessionutil/session_util.go:236] ["retry func failed"] ["retry time"=32] [error="context deadline exceeded"] [stack="github.com/milvus-io/milvus/internal/util/sessionutil.NewSession\n\t/go/src/github.com/milvus-io/milvus/internal/util/sessionutil/session_util.go:236\ngithub.com/milvus-io/milvus/internal/datacoord.(*Server).initSession\n\t/go/src/github.com/milvus-io/milvus/internal/datacoord/server.go:277\ngithub.com/milvus-io/milvus/internal/datacoord.(*Server).Init\n\t/go/src/github.com/milvus-io/milvus/internal/datacoord/server.go:297\ngithub.com/milvus-io/milvus/internal/distributed/datacoord.(*Server).init\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/datacoord/service.go:108\ngithub.com/milvus-io/milvus/internal/distributed/datacoord.(*Server).Run\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/datacoord/service.go:229\ngithub.com/milvus-io/milvus/cmd/components.(*DataCoord).Run\n\t/go/src/github.com/milvus-io/milvus/cmd/components/data_coord.go:49\ngithub.com/milvus-io/milvus/cmd/roles.runComponent[...].func1\n\t/go/src/github.com/milvus-io/milvus/cmd/roles/roles.go:112"]
[2025/01/17 02:51:11.523 +00:00] [DEBUG] [sessionutil/session_util.go:227] ["Session try to connect to etcd"]
[2025/01/17 02:51:11.617 +00:00] [DEBUG] [sessionutil/session_util.go:227] ["Session try to connect to etcd"]
[2025/01/17 02:51:11.617 +00:00] [DEBUG] [sessionutil/session_util.go:227] ["Session try to connect to etcd"]
[2025/01/17 02:51:11.617 +00:00] [DEBUG] [sessionutil/session_util.go:227] ["Session try to connect to etcd"]
[2025/01/17 02:51:11.617 +00:00] [DEBUG] [sessionutil/session_util.go:227] ["Session try to connect to etcd"]
[2025/01/17 02:51:11.617 +00:00] [DEBUG] [sessionutil/session_util.go:227] ["Session try to connect to etcd"]
[2025/01/17 02:51:11.700 +00:00] [DEBUG] [sessionutil/session_util.go:227] ["Session try to connect to etcd"]
Describe the results you expected
Describe the results you expected
podman info output
Podman in a container
No
Privileged Or Rootless
Privileged
Upstream Latest Release
Yes
Additional environment details
Additional environment details
Additional information
Additional information like issue happens only occasionally or issue happens with a particular architecture or on a particular setting
The text was updated successfully, but these errors were encountered: