POST /jobs internal server error if validator references unspecified optional parameter

[jtherrmann]

Jira: https://asfdaac.atlassian.net/browse/TOOL-3335

Note: The above link is accessible only to members of ASF.

---

The POST /jobs endpoint returns an internal server error if a validator attempts to reference an optional job parameter that was not provided. For example, this can be reproduced by attempting to submit either SRG_GSLC or SRG_TIME_SERIES without the optional bounds parameter. Here's an example payload that should validate successfully:

{
  "validate_only": true,
  "jobs": [
    {
      "job_type": "SRG_GSLC",
      "name": "Job Name",
      "job_parameters": {
        "granules": [
          "S1A_IW_RAW__0SDV_20231229T134339_20231229T134411_051870_064437_4F42",
          "S1A_IW_RAW__0SDV_20231229T134404_20231229T134436_051870_064437_5F38"
        ],
        "bounds": [
          -116.583,
          35.714,
          -113.209,
          38.138
        ]
      }
    }
  ]
}

and here's the same payload without bounds, which should return a 500 error:

{
  "validate_only": true,
  "jobs": [
    {
      "job_type": "SRG_GSLC",
      "name": "Job Name",
      "job_parameters": {
        "granules": [
          "S1A_IW_RAW__0SDV_20231229T134339_20231229T134411_051870_064437_4F42",
          "S1A_IW_RAW__0SDV_20231229T134404_20231229T134436_051870_064437_5F38"
        ]
      }
    }
  ]
}

The API Lambda logs show:

[ERROR]	2024-12-05T00:35:11.108Z	ea26392c-cff8-4ddc-8021-32c36437b530	Exception on /jobs [POST]
Traceback (most recent call last):
  File "/var/task/flask/app.py", line 1511, in wsgi_app
    response = self.full_dispatch_request()
  File "/var/task/flask/app.py", line 919, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/var/task/flask_cors/extension.py", line 194, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "/var/task/flask/app.py", line 917, in full_dispatch_request
    rv = self.dispatch_request()
  File "/var/task/flask/app.py", line 902, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
  File "/var/task/openapi_core/contrib/flask/decorators.py", line 52, in decorated
    response = self.handle_request(
  File "/var/task/openapi_core/unmarshalling/processors.py", line 28, in handle_request
    return valid_handler(request_unmarshal_result)
  File "/var/task/openapi_core/contrib/flask/handlers.py", line 75, in __call__
    rv = self.view(*self.view_args, **self.view_kwargs)
  File "/var/task/hyp3_api/routes.py", line 136, in jobs_post
    return jsonify(handlers.post_jobs(request.get_json(), g.user))
  File "/var/task/hyp3_api/handlers.py", line 28, in post_jobs
    validate_jobs(body['jobs'])
  File "/var/task/hyp3_api/validation.py", line 221, in validate_jobs
    validator(job, job_granule_metadata)
  File "/var/task/hyp3_api/validation.py", line 144, in check_bounds_formatting
    bounds = job['job_parameters']['bounds']
KeyError: 'bounds'

The problem is that the post_jobs handler in handlers.py calls validate_jobs before calling dynamo.jobs.put_jobs, which is responsible for populating default parameter values (see jobs.py). The fix might be as simple as calling validate_jobs on the prepared jobs in dynamo.jobs.put_jobs before writing them to the database.

This could potentially affect any job type, but after reviewing validation.py, I'm not seeing any other job types that would currently be affected, so this should only affect the LAVAS deployments right now.