Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Should cylc config be able to report the configuration being used by a running workflow? #4756

Open
dpmatthews opened this issue Mar 23, 2022 · 6 comments
Labels
question Flag this as a question for the next Cylc project meeting.
Milestone

Comments

@dpmatthews
Copy link
Contributor

dpmatthews commented Mar 23, 2022

Given a workflow id, cylc config prints the "workflow configuration" (according to the help text).
But what does that really mean?
What (I think) it means is that it parses whatever configuration files it finds in the workflow directory.
However, this may be very different from the configuration in use by a running workflow because:
a) the config may have been changed by reinstall but not reloaded
b) the config may have been altered via cylc play options (--set #4740, --fcp, etc)

The parsed config file is recorded in the log/flow-config directory.
This includes the result of any --set options although note that it still doesn't reflect any changes made by --fcp for example.
(See also #4755)

The Rose fcm make built-in app uses cylc config to check for the existence of "fcm_make2" tasks.
In theory it could get the wrong result because it is not actually querying the configuration in use.
In practise this is very unlikely and not worth worrying about for this legacy use case.
However, it does raise the question: do we need the ability to query the configuration being used by a running workflow?

@dpmatthews dpmatthews added the question Flag this as a question for the next Cylc project meeting. label Mar 23, 2022
@dpmatthews dpmatthews added this to the cylc-8.0.0 milestone Mar 23, 2022
@hjoliver
Copy link
Member

However, it does raise the question: do we need the ability to query the configuration being used by a running workflow?

Good point. I think we should, ideally. However, we could have another file in the run-directory that records the config of the (latest) running scheduler, rather than literally query the scheduler.

@oliver-sanders
Copy link
Member

  • Broadcasts alter the config in memory but not on disk.
  • We could allow the workflow config to be queried via GraphQL.
  • We need to be able to query the [runtime] section for Re-implement edit-run #3751.
  • Does the fcm_make2 issue require [scheduling] information or is [runtime] sufficient?

@dpmatthews
Copy link
Contributor Author

Does the fcm_make2 issue require [scheduling] information or is [runtime] sufficient?

I think [runtime] is sufficient but I'm really not worried about this use case.

@oliver-sanders
Copy link
Member

I don't think cylc config needs to be able to display the running config (note we get a file in log/config on reload), but we might want to open up an interface via GraphQL for other purposes.

Can we bump this to 8.x?

@hjoliver
Copy link
Member

Broadcasts alter the config in memory but not on disk.

We could potentially write every config change to disk, for provenance reasons. Short of that though, getting current active config via GraphQL would be fine IMO.

@hjoliver
Copy link
Member

Can we bump this to 8.x?

I don't this is super urgent, I'll bump it.

@hjoliver hjoliver modified the milestones: cylc-8.0rc4, cylc-8.x May 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Flag this as a question for the next Cylc project meeting.
Projects
None yet
Development

No branches or pull requests

3 participants