Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update dependent CPAs for Secret add/modify/delete operation #210

Merged
merged 1 commit into from
May 5, 2023

Conversation

Anandkumar26
Copy link
Contributor

@Anandkumar26 Anandkumar26 commented Apr 13, 2023

  • Add: Updates account only if its error state with error as Secret does not exist.
  • Update/Delete: Updates account to recompute account state, as Secret is changed
  • Cleanup account state when account add/update fails.

@Anandkumar26 Anandkumar26 changed the title Cleanup stale inventory when account update fails [WIP] Cleanup stale inventory when account update fails Apr 13, 2023
@Anandkumar26 Anandkumar26 changed the title [WIP] Cleanup stale inventory when account update fails Cleanup stale inventory when account update fails Apr 13, 2023
@Anandkumar26 Anandkumar26 force-pushed the updatecpa branch 3 times, most recently from 48299b7 to ecd2061 Compare April 14, 2023 09:29
@Anandkumar26 Anandkumar26 changed the title Cleanup stale inventory when account update fails Update dependent CPAs for Secret add/modify/delete operation Apr 14, 2023
@Anandkumar26
Copy link
Contributor Author

/nephe-test-e2e-kind

continue
}
}
if err := r.processCreateOrUpdate(accountNamespacedName, cpa); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

processCreateOrUpdate function can be called from reconciler too. Let's say, before this function acquires the lock, reconciler calls processCreateOrUpdate for the same CPA(CPA attached to the secret in this function).
When the execution is complete and this function acquires the lock, this might be operating on stale secret event.

Do you see any issue in this scenario?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At any point of time only one of them will acquire the lock and update the account's internal state.

You are talking about a scenario where both events occur at the same time.

  • update of CPA object, which triggers reconciler.
  • update on the Secret object.

In this particular scenario, processCreateOrUpdate will be called twice(once via Secret and once via reconciler), as a result we would restart poller and perform polling twice(which cannot be avoided).

I don't see any issue here.

}
}
}
return cpaItems, nil
}

// watchSecret watch the Secret objects.
// updateDependentCpaBySecretNamespace updates all dependent accounts for a given Secret Namespace.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the comment state "for a given Secret" instead of Secret namespace

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. Secret watcher watches for Secrets only from POD_NAMESPACE. This function will be called only for the Secret namespace.

@@ -199,8 +196,7 @@ func (p *accountPoller) updateAccountStatus(cloudInterface common.CloudInterface

if account.Status != discoveredStatus {
account.Status.Error = discoveredStatus.Error
e = p.Client.Status().Update(context.TODO(), account)
if e != nil {
if e = p.Client.Status().Update(context.TODO(), account); e != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename e to err?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

}
if err = r.AccManager.AddAccount(namespacedName, accountCloudType, account); err != nil {
account.Status.Error = err.Error()
r.Log.Info("Setting account status", "account", namespacedName, "err", err)
if err = r.Client.Status().Update(context.TODO(), account); err != nil {
return fmt.Errorf("failed to update account status, account %v err %v", namespacedName, err)
}
_ = r.AccManager.RemoveAccount(namespacedName)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we really need to call removal? the AddAccount will be retried right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. We don't want to reconciler to keep trying whenever there is an error returned by CPA.
If for any reason, we are not able to process CPA request, we will set the error status on the CPA object and clean up our internal state.

}
namespacedName := &types.NamespacedName{Namespace: secret.Namespace, Name: secret.Name}
r.Log.WithName("Secret").Info("Received request", "Secret", namespacedName, "operation", event.Type)
r.updateDependentCpaBySecretNamespace(namespacedName, event.Type)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

call it processSecretUpdateEvent..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

// For add operation, update CPA only when status contains an error ErrorMsgSecretDoesNotExist.
if eventType == watch.Added {
if !strings.Contains(cpa.Status.Error, util.ErrorMsgSecretDoesNotExist) {
continue
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a log here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

- Add: Updates account only if its error state
  with error as Secret does not exist.

- Update/Delete: Updates account to recompute account state,
  as Secret is changed

- Cleanup account state when account add/update fails.

- Fix imports

Signed-off-by: Anand Kumar <[email protected]>
@reachjainrahul
Copy link
Contributor

/nephe-test-e2e-aws

Copy link
Contributor

@reachjainrahul reachjainrahul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/LGTM

@reachjainrahul reachjainrahul merged commit c776acb into main May 5, 2023
@reachjainrahul reachjainrahul deleted the updatecpa branch May 5, 2023 17:37
@reachjainrahul reachjainrahul added this to the Nephe v0.5 release milestone Jun 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants