Skip to content

Commit

Permalink
[SDK] New fields for schedule pipeline (#3201)
Browse files Browse the repository at this point in the history
* [SDK] New fields for schedule pipeline

* Update according to CR comments

* Update snapshot
  • Loading branch information
k8s-ci-robot authored Mar 4, 2020
1 parent 00a2cad commit 75b5fd7
Show file tree
Hide file tree
Showing 3 changed files with 29 additions and 19 deletions.
5 changes: 3 additions & 2 deletions frontend/src/components/Trigger.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -235,11 +235,12 @@ export default class Trigger extends React.Component<TriggerProps, TriggerState>
<p>
For example, if the recurring run is paused for a while and re-enabled
afterwards. If catchup=true, the scheduler will catch up on (backfill) each
missed interval. Otherwise, it only schedules the latest interval.
missed interval. Otherwise, it only schedules the latest interval if more than
one interval is ready to be scheduled.
</p>
<p>
Usually, if your pipeline handles backfill internally, you should turn catchup
off to avoid duplicate work.
off to avoid duplicate backfill.
</p>
</div>
}
Expand Down
20 changes: 10 additions & 10 deletions frontend/src/components/__snapshots__/Trigger.test.tsx.snap
Original file line number Diff line number Diff line change
Expand Up @@ -165,10 +165,10 @@ exports[`Trigger enables a single day on click 1`] = `
Whether the recurring run should catch up if behind schedule. Defaults to true.
</p>
<p>
For example, if the recurring run is paused for a while and re-enabled afterwards. If catchup=true, the scheduler will catch up on (backfill) each missed interval. Otherwise, it only schedules the latest interval.
For example, if the recurring run is paused for a while and re-enabled afterwards. If catchup=true, the scheduler will catch up on (backfill) each missed interval. Otherwise, it only schedules the latest interval if more than one interval is ready to be scheduled.
</p>
<p>
Usually, if your pipeline handles backfill internally, you should turn catchup off to avoid duplicate work.
Usually, if your pipeline handles backfill internally, you should turn catchup off to avoid duplicate backfill.
</p>
</div>
}
Expand Down Expand Up @@ -506,10 +506,10 @@ exports[`Trigger renders all week days enabled 1`] = `
Whether the recurring run should catch up if behind schedule. Defaults to true.
</p>
<p>
For example, if the recurring run is paused for a while and re-enabled afterwards. If catchup=true, the scheduler will catch up on (backfill) each missed interval. Otherwise, it only schedules the latest interval.
For example, if the recurring run is paused for a while and re-enabled afterwards. If catchup=true, the scheduler will catch up on (backfill) each missed interval. Otherwise, it only schedules the latest interval if more than one interval is ready to be scheduled.
</p>
<p>
Usually, if your pipeline handles backfill internally, you should turn catchup off to avoid duplicate work.
Usually, if your pipeline handles backfill internally, you should turn catchup off to avoid duplicate backfill.
</p>
</div>
}
Expand Down Expand Up @@ -847,10 +847,10 @@ exports[`Trigger renders periodic schedule controls for initial render 1`] = `
Whether the recurring run should catch up if behind schedule. Defaults to true.
</p>
<p>
For example, if the recurring run is paused for a while and re-enabled afterwards. If catchup=true, the scheduler will catch up on (backfill) each missed interval. Otherwise, it only schedules the latest interval.
For example, if the recurring run is paused for a while and re-enabled afterwards. If catchup=true, the scheduler will catch up on (backfill) each missed interval. Otherwise, it only schedules the latest interval if more than one interval is ready to be scheduled.
</p>
<p>
Usually, if your pipeline handles backfill internally, you should turn catchup off to avoid duplicate work.
Usually, if your pipeline handles backfill internally, you should turn catchup off to avoid duplicate backfill.
</p>
</div>
}
Expand Down Expand Up @@ -1086,10 +1086,10 @@ exports[`Trigger renders periodic schedule controls if the trigger type is CRON
Whether the recurring run should catch up if behind schedule. Defaults to true.
</p>
<p>
For example, if the recurring run is paused for a while and re-enabled afterwards. If catchup=true, the scheduler will catch up on (backfill) each missed interval. Otherwise, it only schedules the latest interval.
For example, if the recurring run is paused for a while and re-enabled afterwards. If catchup=true, the scheduler will catch up on (backfill) each missed interval. Otherwise, it only schedules the latest interval if more than one interval is ready to be scheduled.
</p>
<p>
Usually, if your pipeline handles backfill internally, you should turn catchup off to avoid duplicate work.
Usually, if your pipeline handles backfill internally, you should turn catchup off to avoid duplicate backfill.
</p>
</div>
}
Expand Down Expand Up @@ -1348,10 +1348,10 @@ exports[`Trigger renders week days if the trigger type is CRON and interval is w
Whether the recurring run should catch up if behind schedule. Defaults to true.
</p>
<p>
For example, if the recurring run is paused for a while and re-enabled afterwards. If catchup=true, the scheduler will catch up on (backfill) each missed interval. Otherwise, it only schedules the latest interval.
For example, if the recurring run is paused for a while and re-enabled afterwards. If catchup=true, the scheduler will catch up on (backfill) each missed interval. Otherwise, it only schedules the latest interval if more than one interval is ready to be scheduled.
</p>
<p>
Usually, if your pipeline handles backfill internally, you should turn catchup off to avoid duplicate work.
Usually, if your pipeline handles backfill internally, you should turn catchup off to avoid duplicate backfill.
</p>
</div>
}
Expand Down
23 changes: 16 additions & 7 deletions sdk/python/kfp/_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -410,18 +410,26 @@ def __repr__(self):
return RunPipelineResult(self, run_info)

def schedule_pipeline(self, experiment_id, job_name, pipeline_package_path=None, params={}, pipeline_id=None,
namespace=None, cron_schedule=None, description=None):
namespace=None, cron_schedule=None, description=None, max_concurrency=10, no_catchup=None):
"""Schedule pipeline on kubeflow to run based upon a cron job
Arguments:
experiment_id {[type]} -- The expriment within which we would like kubeflow
job_name {[type]} -- The name of the scheduled job
experiment_id {string} -- The expriment within which we would like kubeflow
job_name {string} -- The name of the scheduled job
Keyword Arguments:
pipeline_package_path {[type]} -- The path to the pipeline package (default: {None})
pipeline_package_path {string} -- The path to the pipeline package (default: {None})
params {dict} -- The pipeline parameters (default: {{}})
pipeline_id {[type]} -- The id of the pipeline which should run on schedule (default: {None})
namespace {[type]} -- The name space with which the pipeline should run (default: {None})
pipeline_id {string} -- The id of the pipeline which should run on schedule (default: {None})
namespace {string} -- The name space with which the pipeline should run (default: {None})
max_concurrency {int} -- Max number of concurrent runs scheduled (default: {10})
no_catchup {boolean} -- Whether the recurring run should catch up if behind schedule.
For example, if the recurring run is paused for a while and re-enabled
afterwards. If no_catchup=False, the scheduler will catch up on (backfill) each
missed interval. Otherwise, it only schedules the latest interval if more than one interval
is ready to be scheduled.
Usually, if your pipeline handles backfill internally, you should turn catchup
off to avoid duplicate backfill. (default: {False})
"""

pipeline_json_string = None
Expand Down Expand Up @@ -458,7 +466,8 @@ def schedule_pipeline(self, experiment_id, job_name, pipeline_package_path=None,
description=description,
pipeline_spec=spec,
resource_references=resource_references,
max_concurrency=10,
max_concurrency=max_concurrency,
no_catchup=no_catchup,
trigger=trigger,
enabled=True,
)
Expand Down

0 comments on commit 75b5fd7

Please sign in to comment.