Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Podman not starting oracle enterprise container #11903

Closed
kbul opened this issue Oct 9, 2021 · 26 comments
Closed

Podman not starting oracle enterprise container #11903

kbul opened this issue Oct 9, 2021 · 26 comments
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@kbul
Copy link

kbul commented Oct 9, 2021

Is this a BUG REPORT? (leave only one on its own line)

/kind bug

When executoing the following command:

$ podman-compose up
using podman version: podman version 3.4.0
podman pod create --name=oracle --share net -p 5503:5500 -p 1521:1521
45ad43454fa8238272bb5f64e6fe5795c6de23650ec37491d2e77982ac652c4f

One cannot connect to the DB by means of for example DBEAVER client.

It stucks (providing no error) at some point of container instantiation. Attempting to connect to the database gives the following error.
Listener refused the connection with the following error:
ORA-12514, TNS:listener does not currently know of service requested in connect descriptor

Downgrading to version 3.1.0, and executing the following:

$ podman-compose up
using podman version: podman version 3.1.0
podman pod create --name=oracle --share net -p 5503:5500 -p 1521:1521
298a4e3c427a00c7a83fd84746dba73c1a9893d6dc68a2e37be252c7f54364a9

Works fine.

Url to oracle docker image: https://hub.docker.com/_/oracle-database-enterprise-edition

Steps to reproduce the issue:

  1. Pull the oracle docker image.
  2. Run with the defaults.
  3. Wait, and wait...

Describe the results you received:
Not able to connect to the database.

Describe the results you expected:
Database connection.

Additional information you deem important (e.g. issue happens only occasionally):
Issue permanent.

Output of podman version:

3.4.0

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.23.1
  cgroupControllers: []
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.30-2.fc34.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.30, commit: '
  cpus: 8
  distribution:
    distribution: fedora
    variant: workstation
    version: "34"
  eventLogger: journald
  hostname: fedora-pc
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.14.9-200.fc34.x86_64
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 12105699328
  memTotal: 33521885184
  ociRuntime:
    name: crun
    package: crun-1.1-1.fc34.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.1
      commit: 5b341a145c4f515f96f55e3e7760d1c79ec3cf1f
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc34.x86_64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.0
  swapFree: 8589930496
  swapTotal: 8589930496
  uptime: 7h 29m 47.27s (Approximately 0.29 days)
plugins:
  log:
  - k8s-file
  - none
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /home/xxx/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/xxx/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 2
  runRoot: /run/user/1000/containers
  volumePath: /home/xxx/.local/share/containers/storage/volumes
version:
  APIVersion: 3.4.0
  Built: 1633030821
  BuiltTime: Thu Sep 30 21:40:21 2021
  GitCommit: ""
  GoVersion: go1.16.8
  OsArch: linux/amd64
  Version: 3.4.0


Package info (e.g. output of rpm -q podman or apt list podman):

podman-3.4.0-1.fc34.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):
Host running Fedora.

@openshift-ci openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Oct 9, 2021
@mheon
Copy link
Member

mheon commented Oct 9, 2021

Can you provide a reproducer that doesn't use podman-compose?

@kbul
Copy link
Author

kbul commented Oct 11, 2021

Hello.
Although not entirely without podman-compose, but please below find commands (bold) that are being invoked by the podman-compose.

$ podman-compose up
using podman version: podman version 3.4.0
podman pod create --name=oracle --share net -p 5503:5500 -p 1521:1521
4534e7bd727a9af6c6a5677313761f797148a94d350989bc9f7dbd9ab501e63b
0
podman create --name=oracle_db --pod=oracle --label io.podman.compose.config-hash=123 --label io.podman.compose.project=oracle --label io.podman.compose.version=0.0.1 --label com.docker.compose.container-number=1 --label com.docker.compose.service=oracle --add-host oracle:127.0.0.1 --add-host oracle_db:127.0.0.1 store/oracle/database-enterprise:12.2.0.1-slim
d542c602d9fa466a89e61307f24bfdf7932ce45a7fbfcecaf43d22ad3aa7f613
0
podman start -a oracle_db
Setup Oracle Database
Oracle Database 12.2.0.1 Setup
Mon Oct 11 09:01:30 UTC 2021

@Luap99
Copy link
Member

Luap99 commented Oct 11, 2021

It looks like podman successfully starts the container. This looks like an configuration error. Can you inspect the container and provide the output please.

@kbul
Copy link
Author

kbul commented Oct 11, 2021

@Luap99, i have attached few files for comparison.
podman-3-1-0-inspect.txt
podman-3-4-0-inspect.txt
podman-3_1_0-command-output.txt
podman-3_4_0-command-output.txt
podman-error-ctrl-plus-c-output.txt
sql-deleoper-connection-error.txt

Please follow the names as descriptions.

Regards and thank you.

@kbul
Copy link
Author

kbul commented Oct 11, 2021

docker-compose.yaml.txt

Attaching docker-compose file.

@Luap99
Copy link
Member

Luap99 commented Oct 11, 2021

@kbul Thanks, this is helpful. Could also inspect the infra container of the pod. You can get the infra container ID with podman pod inspect --format {{.InfraContainerID}} <pod name>

@kbul
Copy link
Author

kbul commented Oct 11, 2021

@Luap99, attaching two files just in case (executed commands inside the fil)
podman-3-4-0-ps-podman-inspect.txt
podman-3-4-0-pod-inspect.txt
:

@kbul
Copy link
Author

kbul commented Oct 11, 2021

@Luap99, if any more information is needed, or the one i provided is not sufficient, do not hesitate to let me know. Regards.

@Luap99
Copy link
Member

Luap99 commented Oct 11, 2021

OK I cannot find something wrong in the inspect output. How do you connect to the db, from the host via port forwarding or from another container within the pod?

@kbul
Copy link
Author

kbul commented Oct 11, 2021

I have left the container startup for a longer time, attaching command output
podman-3_4_0-command-output-longer-time.txt

Please observe time stamps. There is a situation in which after 10 minutes it started to do some work. I was not that patience before. But still no luck in connecting to the database.

@kbul
Copy link
Author

kbul commented Oct 11, 2021

I am connecting from host using SQL-Developer from Oracle with the following configuration:
Connection url: jdbc:oracle:thin:@localhost:1521/ORCLCDB.localdomain
username: SYS
password: Oradoc_db1
Role: SYSDBA

That configuration works with the older version of podman.

@kbul
Copy link
Author

kbul commented Oct 12, 2021

Another observation that i think is worth to share.

  1. Started successfully oracle container with older version of the podman (podman-compose).
  2. Stopped the container without removing it, i.e. podman ps does not show the running container, while podman ps -a shows that the oracle container exists.
  3. Upgraded the podman to version 3.4.0 and tried to start the stopped container with the podman start <container_id> command.

Result. I am able to connect to the DB.

@kbul
Copy link
Author

kbul commented Oct 17, 2021

Hello @mheon, @Luap99.
Any update on the issue? Do you guys perhaps know when the bug will be fixed?
Regards.

@Luap99
Copy link
Member

Luap99 commented Oct 26, 2021

Sorry for the delay. The inspect output looks correct.

Can you compare the /etc/hosts content in the container between the old and new version. This might be related to this issue #12003.

@kbul
Copy link
Author

kbul commented Oct 26, 2021

Hello.
Below please find the both outputs for the new and older version of the podman. there is a difference in the hosts file. Just curious, what impact has this on the container operation?

$ podman version
Version: 3.4.0
API Version: 3.4.0
Go Version: go1.16.8
Built: Thu Sep 30 21:40:21 2021
OS/Arch: linux/amd64

sh-4.2$ cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

used by slirp4netns

10.0.2.100 oracle 1abb44ab84d9-infra
10.0.2.2 host.containers.internal
127.0.0.1 oracle
127.0.0.1 oracle_db

$ podman version
Version: 3.1.0
API Version: 3.1.0
Go Version: go1.16
Built: Tue Mar 30 15:29:36 2021
OS/Arch: linux/amd64

sh-4.2$ cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

used by slirp4netns

10.0.2.100 oracle 2b314aade7bc-infra
127.0.0.1 oracle
127.0.0.1 oracle_db
127.0.1.1 0883d5c561b4 oracle_db

@kbul
Copy link
Author

kbul commented Oct 26, 2021

$ podman version
Version: 3.4.0
API Version: 3.4.0
Go Version: go1.16.8
Built: Thu Sep 30 21:40:21 2021
OS/Arch: linux/amd64

sh-4.2$ cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

used by slirp4netns

10.0.2.100 oracle 1abb44ab84d9-infra
10.0.2.2 host.containers.internal
127.0.0.1 oracle
127.0.0.1 oracle_db

$ podman version
Version: 3.1.0
API Version: 3.1.0
Go Version: go1.16
Built: Tue Mar 30 15:29:36 2021
OS/Arch: linux/amd64

sh-4.2$ cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

used by slirp4netns

10.0.2.100 oracle 2b314aade7bc-infra
127.0.0.1 oracle
127.0.0.1 oracle_db
127.0.1.1 0883d5c561b4 oracle_db

@kbul kbul closed this as completed Oct 26, 2021
@kbul kbul reopened this Oct 26, 2021
@kbul
Copy link
Author

kbul commented Oct 26, 2021

Sorry i accidentally pressed the close issue button.
Just so you know, "used by slirp4netns" phrase in the command output is formatted (bold) by markdown.

@github-actions
Copy link

A friendly reminder that this issue had no activity for 30 days.

@rhatdan
Copy link
Member

rhatdan commented Nov 29, 2021

@kbul is this still broken?

@kbul
Copy link
Author

kbul commented Nov 30, 2021

Good day @rhatdan,
Unfortunately the problem remains unchanged even though there was two updates in fedora repository since then.
Regards.

@rhatdan
Copy link
Member

rhatdan commented Nov 30, 2021

@Luap99 any more thoughts?

@kbul
Copy link
Author

kbul commented Nov 30, 2021

Recently we were discussing an option in which /etc/hosts entries changed. Would it work if i add those entries manually and restart the container (not recreate the image, but podman start previously running container)?

@rhatdan
Copy link
Member

rhatdan commented Nov 30, 2021

Try it out, that should help.

@github-actions
Copy link

A friendly reminder that this issue had no activity for 30 days.

@kbul
Copy link
Author

kbul commented Jan 1, 2022

Hello. Recent update to version 3.4.4 still does not solve the problem. Regards.

@kbul
Copy link
Author

kbul commented Jan 14, 2022

Hello. It started work again. Not sure how it happened. I did not configure anything specific. Anyway, thank you for handling the issue.

@kbul kbul closed this as completed Jan 14, 2022
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 21, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

No branches or pull requests

4 participants