Data Analysis with Python I

Classwork examples and exercises for the Data Analysis with Python I module of the Degree Apprenticeship in Data Analytics and Visualisation.

If you spot any mistakes or problems, or have any suggestions, please raise an issue.

Import, clean

• Read csv
• Datetime format
• Ordering ordinal data
• Ordinal categories
• Read tab separated values
• Deal with the thousands comma
• Convert a column to integer
• Drop a single row of data

Index, sort, search, filter and pivot

• Setting an index
• Sorting by the index
• Accessing columns
• Basic sorting and filtering
• Groupby
• Filter with 'like'
• loc and iloc
• Filter with 'regex'
• The query function
• The (numpy) where function
• Categorising data with cut
• Crosstabulation
• Select by data type
• Get the index back as a column

Concatenate and join

• Make new columns from existing columns
• Make a new, constant, column
• Combining by concatenation
• Combining by joining on a common column
• When the shared column has a different name

Measure and summarise

• Summary statistics
• Correlation
• Describe subsets

Visualise

• Overview of visualisation libraries
• Scatter plots with pandas and seaborn
• Time series
• Also time series
• Scatter plot with seaborn with markers and colours
• Seaborn distribution plot
• Grouped box plots
• Seaborn catplot for grouped count plots
• Adding data labels in pyplot/seaborn
• Interactivity in notebooks
• Chartify from Spotify
• Holoviews
• Grouped plots in seaborn - relplot
• Multiple scatter plots - pairplot
• Maps with geopandas

Test

• t-tests
• An example of a non-parametric test on a skewed distribution
• Chi squared test
• An interactive t-test
• Binomial hypothesis testing
• Groundhog Day case study

Simulate

• Generating some simple data to understand a problem better
• Simulate normally distributed data