Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a metacat / data_dispatcher option equivalient for --dataset-definition #590

Open
marcmengel opened this issue Nov 4, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@marcmengel
Copy link
Contributor

It would be --metacat-query xxx and would basically be equivalent to: -e SAM_PROJECT=$(ddisp project create xxx)
which would start the project with data_dispatcher at launch time and pass its ID in SAM_PROJECT, letting the compatability
code in ifdhc 2_7 and later use data dispatcher calls to get the next file, etc.

Users would need data_dispatcher and metacat setup to use this.

@marcmengel marcmengel added the enhancement New feature or request label Nov 4, 2024
@shreyb shreyb added the proposed Hasn't been reviewed by the jobsub team. Don't do these until post-review (and then remove label) label Nov 4, 2024
@shreyb
Copy link
Collaborator

shreyb commented Dec 18, 2024

Discussion points:

  1. Request: Add Classad Attribute or ENV that indicates in the job which data handler we're using (SAM or Metacat). We should coordinate this with POMS, fife_wrap, fife_launch, FIFE devs.
  2. We should try to isolate the metacat-specific (and while we're at it, the SAM-specific code). So our get_parser.py would, for example from metacat import BLAH) would have this arg (--metacat-query xxx`), call BLAH. The same could be done for SAM-specific options.

@shreyb shreyb removed the proposed Hasn't been reviewed by the jobsub team. Don't do these until post-review (and then remove label) label Dec 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants