[IBCDPE-792] Implement Great Expectations for the proteomics_distribution_data Dataset

[BWMac]

Description:

This PR implements a GX expectation suite for the proteomics_distribution_data dataset.

Please let me know if there are any additional expectations I should add, or if there are any adjustments to be made to the expectations I have already implemented.

Notes:

• Values I have chosen for expectations like expect_column_values_to_be_between are based on looking at the example dataset on Synapse.
• This change has been tested by running adt test_config.yaml. Example results can be downloaded from here.
• I also fixed a variable naming typo in the neuropath_corr notebook which was not affecting performance/proper functioning.

[sonarcloud]

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

[thomasyu888]

🔥 LGTM! Will wait for @jaclynbeck-sage or @JessterB to comment on the rules themselves

[jaclynbeck-sage]

Looks good! I'm a little torn on having min/max values for the numerical columns. On the one hand, generically these values could be anything and may vary based on the type of proteomics data, so it doesn't entirely make sense to bound them. On the other hand, this matches our current data, and if our current data goes outside these bounds that means something is wrong. Last time this came up we decided to leave it in for that very reason so, it's probably ok?

[JessterB]

This looks great, thanks @BWMac! I agree with @jaclynbeck-sage about leaving in the min/max checks on the numeric values - we don't expect these to change unless we add new data, and if we add new data we'll likely need to adjust the expectations anyway. Having a check in place that lets us know if things change unexpectedly is a good thing.