04-mutating.Rmd

# Mutating data

One of the most common data analysis techniques is to look at change over time. The most common way of comparing change over time is through percent change. The math behind calculating percent change is very simple, and you should know it off the top of your head. The easy way to remember it is:

`(new - old) / old` 

Or new minus old divided by old. Your new number minus the old number, the result of which is divided by the old number. To do that in R, we can use `dplyr` and `mutate` to calculate new metrics in a new field using existing fields of data. 

So first we'll import the tidyverse so we can read in our data and begin to work with it.

```{r, message=FALSE,  warning=FALSE}
library(tidyverse)
```

Now you'll need a common and simple dataset of total attendance at NCAA football games over the last few seasons. 

```{r echo=FALSE, class.output="bg-info", results="asis",  message=FALSE,  warning=FALSE}
library(downloadthis)
library(glue)

dllink <- download_link(
  link = "http://mattwaite.github.io/sportsdatafiles/attendance.csv",
  button_label = "Download csv file",
  button_type = "danger",
  has_icon = TRUE,
  icon = "fa fa-save",
  self_contained = FALSE
)

glue("<pre><p><strong>For this walkthrough:</strong></p><p>{dllink}</p></pre>")

```

You'll import it something like this.

```{r}
attendance <- read_csv('data/attendance.csv')
```

If you want to see the first six rows -- handy to take a peek at your data -- you can use the function `head`.

```{r}
head(attendance)
```

The code to calculate percent change is pretty simple. Remember, with `summarize`, we used `n()` to count things. With `mutate`, we use very similar syntax to calculate a new value using other values in our dataset. So in this case, we're trying to do (new-old)/old, but we're doing it with fields. If we look at what we got when we did `head`, you'll see there's \`2018\` as the new data, and we'll use \`2017\` as the old data. So we're looking at one year. Then, to help us, we'll use arrange again to sort it, so we get the fastest growing school over one year. 

```{r}
attendance %>% mutate(
  change = (`2018` - `2017`)/`2017`
) 
```
What do we see right away? Do those numbers look like we expect them to? No. They're a decimal expressed as a percentage. So let's fix that by multiplying by 100. 

```{r}
attendance %>% mutate(
  change = ((`2018` - `2017`)/`2017`)*100
) 
```
Now, does this ordering do anything for us? No. Let's fix that with arrange. 

```{r}
attendance %>% mutate(
  change = ((`2018` - `2017`)/`2017`)*100
) %>% arrange(desc(change))
```

So who had the most growth last year from the year before? Something going on at Georgia Southern. 

## A more complex example

There's metric in basketball that's easy to understand -- shooting percentage. It's the number of shots made divided by the number of shots attempted. Simple, right? Except it's a little too simple. Because what about three point shooters? They tend to be more vailable because the three point shot is worth more. What about players who get to the line? In shooting percentage, free throws are nowhere to be found. 

Basketball nerds, because of these weaknesses, have created a new metric called [True Shooting Percentage](https://en.wikipedia.org/wiki/True_shooting_percentage). True shooting percentage takes into account all aspects of a players shooting to determine who the real shooters are. 

Using `dplyr` and `mutate`, we can calculate true shooting percentage. So let's look at a new dataset, one of every college basketball player's season stats in 2018-19 season. It's a dataset of 5,386 players, and we've got 59 variables -- one of them is True Shooting Percentage, but we're going to ignore that. 

```{r echo=FALSE, class.output="bg-info", results="asis",  message=FALSE,  warning=FALSE}
library(downloadthis)
library(glue)

dllink <- download_link(
  link = "http://mattwaite.github.io/sportsdatafiles/players20.csv",
  button_label = "Download csv file",
  button_type = "danger",
  has_icon = TRUE,
  icon = "fa fa-save",
  self_contained = FALSE
)

glue("<pre><p><strong>For this walkthrough:</strong></p><p>{dllink}</p></pre>")

```

Import it like this:

```{r}
players <- read_csv("data/players20.csv")
```

The basic true shooting percentage formula is `(Points / (2*(FieldGoalAttempts + (.44 * FreeThrowAttempts)))) * 100`. Let's talk that through. Points divided by a lot. It's really field goal attempts plus 44 percent of the free throw attempts. Why? Because that's about what a free throw is worth, compared to other ways to score. After adding those things together, you double it. And after you divide points by that number, you multiply the whole lot by 100. 

In our data, we need to be able to find the fields so we can complete the formula. To do that, one way is to use the Environment tab in R Studio. In the Environment tab is a listing of all the data you've imported, and if you click the triangle next to it, it'll list all the field names, giving you a bit of information about each one. 

```{r, echo=FALSE}
knitr::include_graphics(rep("images/environment.png"))
```

So what does True Shooting Percentage look like in code? 

Let's think about this differently. Who had the best true shooting season last year? 

```{r}
players %>%
  mutate(trueshooting = (PTS/(2*(FGA + (.44*FTA))))*100) %>%
  arrange(desc(trueshooting))
```

You'll be forgiven if you did not hear about Vanderbilt's shooting sensation Trace Arbuckle. He played in one game, took one shot and actually hit it. It happened to be a three pointer, which is one more three pointer than I've hit in college basketball. So props to him. Does that mean he had the best true shooting season in college basketball last year? 

Not hardly. 

We'll talk about how to narrow the pile and filter out data in the next chapter.