Add data validation checks to data analysis

[agendazhang]

• Correct data file format
• Correct column names
• No empty observations
• Missingness not beyond expected threshold
• Correct data types in each column
• No duplicate observations
• No outlier or anomalous values
• Correct category levels (i.e., no string mismatches or single values)
• Target/response variable follows expected distribution
• No anomalous correlations between target/response variable and features/explanatory variables
• No anomalous correlations between features/explanatory variables

[ch3ch20h]

I just did Correct data file format, and I pulled it to the branch data_validation. But there is a merge conflict. Could anyone check?

[agamsanghera]

I saw and commented on the PR, it is best to copy the work you did in a new notebook, and let the original analysis.ipynb be unchanged, good thing that you did it in a separate branch, this way the original work is safe

[diwanashita]

I will start on the new notebook and @ch3ch20h can continue in that notebook.

[diwanashita]

@ch3ch20h the work pushed isn't on par with the rubric given to us. I am fixing this file to include the proper pre-processing and validation we learned from the lecture and readings.
Most of the work there looks like it is copy pasted from the analysis file, which isn't completely wrong, but it is missing the pre-processing part, rendering it ineffective for the data validation step.

[diwanashita]

Created Schema earlier in the day but when tried to run the file later tonight, it was not reading in the data for some reason. Will troubleshoot tmr morning unless someone knows why this is happening