Skip to content
This repository has been archived by the owner on Oct 22, 2024. It is now read-only.

OLM: update to v0.19.1 broke installation #1050

Open
pohly opened this issue Nov 12, 2021 · 2 comments
Open

OLM: update to v0.19.1 broke installation #1050

pohly opened this issue Nov 12, 2021 · 2 comments
Assignees

Comments

@pohly
Copy link
Contributor

pohly commented Nov 12, 2021

Found on release-1.0 branch during make test_e2e TEST_E2E_FOCUS=olm.API TEST_E2E_ARGS=-ginkgo.failFast:

+ /nvme/gopath/src/github.com/intel/pmem-csi/_work/bin/operator-sdk run bundle --namespace default --timeout 5m 172.17.42.1:5001/pmem-csi-bundle:v1.0.0 --skip-tls
          time="2021-11-12T08:05:39+01:00" level=info msg="Successfully created registry pod: 172-17-42-1-5001-pmem-csi-bundle-v1-0-0"
          time="2021-11-12T08:05:39+01:00" level=info msg="Created CatalogSource: pmem-csi-operator-catalog"
          time="2021-11-12T08:05:39+01:00" level=info msg="OperatorGroup \"operator-sdk-og\" created"
          time="2021-11-12T08:05:39+01:00" level=info msg="Created Subscription: pmem-csi-operator-v1-0-0-sub"
          time="2021-11-12T08:10:32+01:00" level=fatal msg="Failed to run bundle: install plan is not available for the subscription pmem-csi-operator-v1-0-0-sub: timed out waiting for the condition\n"

It works when installing OLM v0.18.3.

There are similar issues on the master branch:

/mnt/workspace/pmem-csi-periodic/_work/bin/operator-sdk run bundle --namespace default --timeout 5m 172.17.0.1:5000/pmem-csi-bundle:v100.0.0 --skip-tls
    time="2021-11-11T02:22:42Z" level=fatal msg="Failed to run bundle: create catalog: error creating catalog source: catalogsources.operators.coreos.com \"pmem-csi-operator-catalog\" already exists\n"
@pohly
Copy link
Contributor Author

pohly commented Nov 12, 2021

As a workaround I am going to fix the OLM version to 0.18.3, but this needs to be investigated.

pohly added a commit that referenced this issue Nov 12, 2021
Locking to a specific version is good practice because it avoids unexpected
compatibility issues when a new version gets released. That seems to have
happened with OLM v0.19.0 which currently doesn't work for
us (#1050).
pohly added a commit to pohly/pmem-CSI that referenced this issue Nov 12, 2021
Locking to a specific version is good practice because it avoids unexpected
compatibility issues when a new version gets released. That seems to have
happened with OLM v0.19.0 which currently doesn't work for
us (intel#1050).

(cherry picked from commit 08505ea)
@pohly
Copy link
Contributor Author

pohly commented Nov 17, 2021

More diagnostics, now with our devel branch:

$ kubectl describe --all-namespaces subscriptions
Name:         pmem-csi-operator-v100-0-0-sub
Namespace:    default
Labels:       operators.coreos.com/pmem-csi-operator.default=
Annotations:  <none>
API Version:  operators.coreos.com/v1alpha1
Kind:         Subscription
Metadata:
  Creation Timestamp:  2021-11-17T13:21:35Z
  Generation:          1
  Managed Fields:
    API Version:  operators.coreos.com/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:catalogHealth:
        f:conditions:
        f:lastUpdated:
    Manager:      catalog
    Operation:    Update
    Subresource:  status
    Time:         2021-11-17T13:21:35Z
    API Version:  operators.coreos.com/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          .:
          f:operators.coreos.com/pmem-csi-operator.default:
    Manager:      olm
    Operation:    Update
    Time:         2021-11-17T13:21:35Z
    API Version:  operators.coreos.com/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:spec:
        .:
        f:channel:
        f:installPlanApproval:
        f:name:
        f:source:
        f:sourceNamespace:
        f:startingCSV:
    Manager:         operator-sdk
    Operation:       Update
    Time:            2021-11-17T13:21:35Z
  Resource Version:  21120
  UID:               d96c7834-6a74-4b6a-9f91-28e8841103f6
Spec:
  Channel:                alpha
  Install Plan Approval:  Manual
  Name:                   pmem-csi-operator
  Source:                 pmem-csi-operator-catalog
  Source Namespace:       default
  Starting CSV:           pmem-csi-operator.v100.0.0
Status:
  Catalog Health:
    Catalog Source Ref:
      API Version:       operators.coreos.com/v1alpha1
      Kind:              CatalogSource
      Name:              pmem-csi-operator-catalog
      Namespace:         default
      Resource Version:  21117
      UID:               f7422ba4-9ffb-43c0-b4dc-0a530a6c847f
    Healthy:             true
    Last Updated:        2021-11-17T13:21:36Z
    Catalog Source Ref:
      API Version:       operators.coreos.com/v1alpha1
      Kind:              CatalogSource
      Name:              operatorhubio-catalog
      Namespace:         olm
      Resource Version:  17519
      UID:               a8fb807c-b7b3-425f-bba6-4ef890b5c060
    Healthy:             true
    Last Updated:        2021-11-17T13:21:36Z
  Conditions:
    Last Transition Time:  2021-11-17T13:21:36Z
    Message:               all available catalogsources are healthy
    Reason:                AllCatalogSourcesHealthy
    Status:                False
    Type:                  CatalogSourcesUnhealthy
  Last Updated:            2021-11-17T13:21:36Z
Events:                    <none>

Note the odd "AllCatalogSourcesHealthy: False". This pod here might be responsible for it (not sure) and reports an error:

$ kubectl logs -n olm pods/catalog-operator-84976fd7df-x5tmr
...
E1117 13:21:35.798270       1 queueinformer_operator.go:290] sync {"update" "default/pmem-csi-operator-v100-0-0-sub"} failed: Operation cannot be fulfilled on subscriptions.operators.coreos.com "pmem-csi-operator-v100-0-0-sub": the object has been modified; please apply your changes to the latest version and try again
time="2021-11-17T13:21:35Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=
time="2021-11-17T13:21:35Z" level=info msg=syncing event=update reconciling="*v1alpha1.Subscription" selflink=
time="2021-11-17T13:21:36Z" level=warning msg="an error was encountered during reconciliation" error="Operation cannot be fulfilled on subscriptions.operators.coreos.com \"pmem-csi-operator-v100-0-0-sub\": the object has been modified; please apply your changes to the latest version and try again" event=update reconciling="*v1alpha1.Subscription" selflink=
...

This repeats a few times but then not anymore. Deleting that pod doesn't help, the recreated one has the same problem.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants