Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add seq2seq forecasting training job #1196

Merged
merged 14 commits into from
Jun 3, 2022

Conversation

TheMichaelHu
Copy link
Contributor

@TheMichaelHu TheMichaelHu commented May 4, 2022

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)

Fixes b/229909845 🦕


Adds a SequenceToSequencePlusForecastingTrainingJob to training jobs. This job has the exact same signature as AutoMLForecastingTrainingJob, but we are creating a separate job in case the two models diverge in the future.

The logic for AutoMLForecastingTrainingJob has been moved to a new abstract base class _ForecastingTrainingJob. The only things that differ between the seq2seq and automl training jobs that extend it are the model_type and training_task_definition.

@product-auto-label product-auto-label bot added the size: xl Pull request size is extra large. label May 4, 2022
@TheMichaelHu TheMichaelHu marked this pull request as draft May 4, 2022 22:13
@TheMichaelHu TheMichaelHu requested a review from geraint0923 May 4, 2022 22:16
@geraint0923
Copy link
Contributor

Overall LG, just one minor comment.

Copy link
Contributor

@geraint0923 geraint0923 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@product-auto-label product-auto-label bot added the api: vertex-ai Issues related to the googleapis/python-aiplatform API. label May 5, 2022
@sararob sararob added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label May 5, 2022
@TheMichaelHu TheMichaelHu marked this pull request as ready for review May 6, 2022 17:07
@TheMichaelHu TheMichaelHu added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 6, 2022
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 6, 2022
@TheMichaelHu TheMichaelHu force-pushed the mh-seq2seq branch 2 times, most recently from 7f16202 to e16cf4e Compare May 9, 2022 14:21
@TheMichaelHu TheMichaelHu requested a review from ivanmkc May 9, 2022 14:39
@TheMichaelHu TheMichaelHu added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 9, 2022
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 9, 2022
@TheMichaelHu TheMichaelHu added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 9, 2022
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label May 9, 2022
@sararob sararob removed the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label May 10, 2022
@TheMichaelHu TheMichaelHu force-pushed the mh-seq2seq branch 2 times, most recently from 22098ca to 2cf47c9 Compare May 17, 2022 14:17

self._optimization_objective = optimization_objective
self._additional_experiments = []

def run(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you just remove the run override and defer to super?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we remove this, Sphinx will not generate documentation for the run() function. It can pull the docstring from the parent class (tested with nox -s docs), but I couldn't find a way to make it document the function signature other than by writing it out manually.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, that's quite annoying. Let me ask the SDK team for more solutions.

Copy link
Contributor

@ivanmkc ivanmkc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like you can simplify both AutoMLForecastingTrainingJob and SequenceToSequencePlusForecastingTrainingJob by removing all methods, since they are the exact same as the super's implementation.

We also need to add system tests for these. See tests/system/aiplatform/test_e2e_tabular.py for an example.

@ivanmkc
Copy link
Contributor

ivanmkc commented May 19, 2022

Regarding my last comment, you can simplify to:



class AutoMLForecastingTrainingJob(_ForecastingTrainingJob):
    _model_type = "AutoML"
    _training_task_definition = schema.training_job.definition.automl_forecasting
    _supported_training_schemas = (schema.training_job.definition.automl_forecasting,)

class SequenceToSequencePlusForecastingTrainingJob(_ForecastingTrainingJob):
    _model_type = "Seq2Seq"
    _training_task_definition = schema.training_job.definition.seq2seq_plus_forecasting
    _supported_training_schemas = (
        schema.training_job.definition.seq2seq_plus_forecasting,
    )

I ran the unit tests and it works fine. Otherwise, everything else looks pretty good.

@TheMichaelHu
Copy link
Contributor Author

TheMichaelHu commented May 19, 2022

Thanks for the review @ivanmkc! qq: how will simplifying the way you suggested impact the documentation? Is there a way to check/ensure that there will be no impact? Like if the code used to generate the docs has an option to document functions from super()?

@TheMichaelHu TheMichaelHu force-pushed the mh-seq2seq branch 2 times, most recently from 32fb0d4 to 7577c31 Compare May 28, 2022 01:14
@TheMichaelHu TheMichaelHu requested a review from ivanmkc May 28, 2022 01:52
Copy link
Contributor

@ivanmkc ivanmkc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if system tests pass.

@TheMichaelHu TheMichaelHu added the automerge Merge the pull request once unit tests and other checks pass. label Jun 3, 2022
@gcf-merge-on-green gcf-merge-on-green bot merged commit 643d335 into googleapis:main Jun 3, 2022
@gcf-merge-on-green gcf-merge-on-green bot removed the automerge Merge the pull request once unit tests and other checks pass. label Jun 3, 2022
rosiezou pushed a commit that referenced this pull request Jun 16, 2022
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
- [x] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-aiplatform/issues/new/choose) before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)

Fixes b/229909845 🦕

---

Adds a `SequenceToSequencePlusForecastingTrainingJob` to training jobs. This job has the exact same signature as `AutoMLForecastingTrainingJob`, but we are creating a separate job in case the two models diverge in the future.

The logic for `AutoMLForecastingTrainingJob` has been moved to a new abstract base class `_ForecastingTrainingJob`. The only things that differ between the seq2seq and automl training jobs that extend it are the `model_type` and `training_task_definition`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: vertex-ai Issues related to the googleapis/python-aiplatform API. size: xl Pull request size is extra large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants