Create Table Schema and register

[PriyaBasker23]

User Story

As a data producer
I want to define a schema for my data product table and register it with the data platform.
So that best i can ensure conformance against that schema in future uploads, and supply useful information to the data platform catalogue for consumers of my data product.

Proposal

1. Write a python function/class that will allow to create a schema, plus unit tests
2. Create an api end point , where schema can be send and created
3. Metadata information about the file extension type and load type should be included Create a schema and store in folder eg: /data-product-bucket/scheme/databasename=x/table_name=x/schema.json

( No Schema Versioning)

Definition of Done

• will use glue schema format and have a schema file for the schema to validate table schema against (using pythons jsonschema.validate)
• define the spec of the schema object in a json schema
• create new shared ecr repo for each version of the lambda container - modernisation-platform repo.
• create new lambda container in data-platform repo, with the code api calls to execute the required actions.
• make all infra in modernisation-platform-environments. api-gateway, lambda, iam
• Once above tasks complete, add documentation for api endpoint.