podman --remote API backward compatibility

[valleedelisle]

/kind feature

When running podman --remote, it would be good if we could specify the API version.

There's no garantee that the client and the API are using the same version so having backward compatibility should be provided.

Currently, there's no way to specify an API version and we endup with an obscur message

# podman --remote info
Error: cannot connect to the Podman socket, please verify that Podman REST API service is running: ping response was 404

After snooping the socket, I realized that the remote client was running podman-3.1 and called /v3.1.0-dev api

    GET /v3.1.0-dev/libpod/_ping HTTP/1.1\r
    Host: d\r
    User-Agent: Go-http-client/1.1\r
    \r
    < 2021/07/16 11:38:49.163010  length=167 from=0 to=166
    HTTP/1.1 404 Not Found\r
    Content-Type: text/plain; charset=utf-8\r
    X-Content-Type-Options: nosniff\r
    Date: Fri, 16 Jul 2021 11:38:49 GMT\r
    Content-Length: 10\r
    \r
    Not Found

As opposed to the native client that was running podman-3.0 and was calling /v2.0.0:

    > 2021/07/16 11:39:09.068753  length=78 from=0 to=77
    GET /v2.0.0/libpod/_ping HTTP/1.1\r
    Host: d\r
    User-Agent: Go-http-client/1.1\r
    \r
    < 2021/07/16 11:39:09.229345  length=335 from=0 to=334
    HTTP/1.1 200 OK\r

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

host:
  arch: amd64
  buildahVersion: 1.19.8
  cgroupManager: systemd
  cgroupVersion: v1
  conmon:
    package: conmon-2.0.26-1.module_el8.5.0+736+58cc1a5a.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.26, commit: 4fbca32554ff2f8218d059dc2f0a68ed16f32092'
  cpus: 2
  distribution:
    distribution: '"centos"'
    version: "8"
  eventLogger: journald
  hostname: compute-0.alejandro.ftw
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 4.18.0-315.el8.x86_64
  linkmode: dynamic
  memFree: 5293477888
  memTotal: 7946698752
  ociRuntime:
    name: runc
    package: runc-1.0.0-70.rc92.module_el8.5.0+736+58cc1a5a.x86_64
    path: /usr/bin/runc
    version: 'runc version spec: 1.0.2-dev'
  os: linux
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_NET_RAW,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    selinuxEnabled: true
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 0
  swapTotal: 0
  uptime: 3h 22m 58.2s (Approximately 0.12 days)
registries:
  search:
  - registry.redhat.io
  - registry.access.redhat.com
  - registry.fedoraproject.org
  - registry.centos.org
  - docker.io
  undercloud-0.ctlplane.alejandro.ftw:8787:
    Blocked: false
    Insecure: true
    Location: undercloud-0.ctlplane.alejandro.ftw:8787
    MirrorByDigestOnly: false
    Mirrors: null
    Prefix: undercloud-0.ctlplane.alejandro.ftw:8787
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 19
    paused: 0
    running: 8
    stopped: 11
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageStore:
    number: 7
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 3.0.0
  Built: 1616791199
  BuiltTime: Fri Mar 26 20:39:59 2021
  GitCommit: ""
  GoVersion: go1.16.1
  OsArch: linux/amd64
  Version: 3.0.2-dev

Package info (e.g. output of rpm -q podman or apt list podman):

remote: podman-3.0.1-6.module_el8.5.0+736+58cc1a5a.x86_64
client: podman-3.1.0-0.13.module_el8.5.0+733+9bb5dffa.x86_64

[jwhonce]

This should have been resolved by #9878 . I am going to close this issue. Please open another if you find additional errors.