Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store true #150

Merged
merged 13 commits into from
Feb 25, 2022
7 changes: 2 additions & 5 deletions lstmcpipe/hiperta/hiperta_r0_to_dl1lstchain.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@

import os
import argparse
from distutils.util import strtobool
from lstmcpipe.hiperta.reorganize_dl1hiperta300_to_dl1lstchain060 import (
main as reorganize_dl1,
)
Expand Down Expand Up @@ -48,19 +47,17 @@ def main():
parser.add_argument(
"--keep_file",
"-k",
type=lambda x: bool(strtobool(x)),
action='store_true',
dest="keep_file",
help="Keep output of hiperta. Set by default to False",
default=False,
)

parser.add_argument(
"--debug_mode",
"-d",
type=lambda x: bool(strtobool(x)),
action='store_true',
dest="debug_mode",
help="Activate debug mode (add cleaned mask in the output hdf5). Set by default to False",
default=False,
)
args = parser.parse_args()

Expand Down
17 changes: 8 additions & 9 deletions lstmcpipe/scripts/script_batch_filelist_rta.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@

import argparse
import subprocess
from distutils.util import strtobool
import os


Expand Down Expand Up @@ -39,18 +38,16 @@ def main():
parser.add_argument(
"--keep_file",
"-k",
type=lambda x: bool(strtobool(x)),
action='store_true',
dest="keep_file",
help="Keep output of hiperta. Set by default to False",
default=False,
)
parser.add_argument(
"--debug_mode",
"-d",
type=lambda x: bool(strtobool(x)),
action='store_true',
dest="debug_mode",
help="Activate debug mode (add cleaned mask in the output hdf5). Set by default to False",
default=False,
)
args = parser.parse_args()

Expand All @@ -66,11 +63,13 @@ def main():
"lstmcpipe_hiperta_r0_to_dl1lstchain",
f"--infile={file}",
f"--outdir={args.output_dir}",
f"--keep_file={args.keep_file}",
f"--debug_mode={args.debug_mode}",
f"--config={args.config_file}",
]
if args.config_file:
cmd.append(f"--config={args.config_file}")

if args.keep_file:
cmd.append(f"--keep_file")
garciagenrique marked this conversation as resolved.
Show resolved Hide resolved
if args.debug_mode:
cmd.append(f"--debug_mode")
garciagenrique marked this conversation as resolved.
Show resolved Hide resolved

subprocess.run(cmd)

Expand Down
19 changes: 15 additions & 4 deletions lstmcpipe/stages/mc_r0_to_dl1.py
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,7 @@ def r0_to_dl1(
workflow_kind="lstchain",
keep_rta_file=False,
n_jobs_parallel=20,
debug_mode=False,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see no way to set this, how do you want to use it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should probably be the debug flag we already have in lstmcpipe_start.py?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hhmm right... 🤔

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed, there are functionalities that were implemented long time ago, but now I'm not sure they make much sense.... (before you could configured HiPeRTA from the config file...). Same for keep_files indeed.

Copy link
Collaborator

@garciagenrique garciagenrique Feb 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would propose.
Nobody uses hiperta functionality but me.
Let's not add extra args to the config file that will never be used.
Either I implement some optional args when hiperta workflow is set (and open an issue otherwise I will forgot), either (what would be most probable) I change the default value of the vars on the script by hand and I re-install if ever I need to used these args 🙈 🙈 🙈 🙈 - not that I do this always 😂

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, now I recall. Whenever I needed these functionalities, I was directly changing things on the old and deprecated batch_files_rta.sh.

):
"""
R0 to DL1 MC onsite conversion.
Expand Down Expand Up @@ -170,12 +171,18 @@ def r0_to_dl1(
! NOTE : train_pipe AND dl1_to_dl2 **MUST** be run with the same environment.
workflow_kind: str
One of the supported pipelines. Defines the command to be run on r0 files
keep_rta_file : bool
Argument to be passed to the hiperta_r0_to_dl1lstchain script, which runs the hiperta_r0_dl1 and
re-organiser stage
n_jobs_parallel: int
Number of jobs to be run at the same time per array.

# HIPERTA ARGUMENTS
keep_rta_file : bool
Flag to indicate whether to keep (keep_rta_file = True) or remove (keep_rta_file = Flase ) the
`dl1v06_reorganized_*.h5` output file (hiperta_r0_dl1 and re-organiser stages).
Argument to be passed to the hiperta_r0_to_dl1lstchain script.
debug_mode : bool
Flag to activate debug_mode. Only compatible with `hiperta` workflow kind, i.e.,
HiPeRTA functionality. DEFAULT=False.

Returns
-------
jobid2log : dict
Expand Down Expand Up @@ -208,8 +215,12 @@ def r0_to_dl1(
elif workflow_kind == "hiperta":
base_cmd = (
f"{source_environment} lstmcpipe_rta_core_r0_dl1 -c {config_file} "
f"--debug_mode False -k {keep_rta_file}"
)
if keep_rta_file:
base_cmd += " --keep_rta_file"
if debug_mode:
base_cmd += " --debug_mode"

jobtype_id = "RTA"
else:
log.critical("Please, selected an allowed workflow kind.")
Expand Down