The apply family
===

R offers a family of **apply functions**, which allow you to apply a function across different chunks of data. Offers an alternative to explicit iteration using `for()` loop; can be simpler and faster, **though not always** and can lack readability. Summary of functions:

- `apply()`: apply a function to rows or columns of a matrix or data frame
- `lapply()`: apply a function to elements of a list or vector
- `sapply()` and `vapply()`: same as the above, but simplify the output (if possible)
- `tapply()`: apply a function to levels of a factor vector

`apply()`: rows or columns of a matrix or data frame
===

The `apply()` function takes inputs of the following form:

- `apply(x, MARGIN=1, FUN=my.fun)`, to apply `my.fun()` across rows of a matrix or data frame `x`
- `apply(x, MARGIN=2, FUN=my.fun)`, to apply `my.fun()` across columns of a matrix or data frame `x`

```{r}
set.seed(123)

X <- matrix(rnorm(20), nrow = 5, ncol = 4)

out <- numeric(nrow(X))

for (i in 1:nrow(X)) {
  out[i] <- mean(X[i, ])
}

out <- apply(X, 1, mean)
```

`apply()` is still a loop. You just don’t see the counter.

```{r}
class(state.x77) # Built-in matrix of states data, 50 states x 8 variables
head(state.x77) 

apply(state.x77, MARGIN=2, FUN=min) # Minimum entry in each column
apply(state.x77, MARGIN=2, FUN=max) # Maximum entry in each column
apply(state.x77, MARGIN=2, FUN=which.max) # Index of the max in each column
```

Optimized functions for special tasks
===

**Don't overuse** the apply paradigm! There's lots of special functions that **optimized** will be both simpler and faster than using `apply()`. E.g.,

- `rowSums()`, `colSums()`: for computing row, column sums of a matrix
- `rowMeans()`, `colMeans()`: for computing row, column means of a matrix
- `max.col()`: for finding the maximum position in each row of a matrix

Combining these functions with logical indexing and vectorized operations will enable you to do quite a lot. E.g., how to count the number of positives in each row of a matrix?

```{r}
x = matrix(rnorm(9), 3, 3)
# Don't do this (much slower for big matrices)
apply(x, MARGIN=1, function(v) { return(sum(v > 0)) })
# Do this insted (much faster, simpler)
rowSums(x > 0)
```

`lapply()`: elements of a list or vector
===

`lapply()` applies a function to each element of a list (or vector, which is treated as a list of length-1 elements).

Suppose we have a function called `my.fun()`. Don't worry about this syntax too much yet, we'll talk about how to define custom functions in the next set of slides. 

```{r}
my_fun <- function(x) {
  mean(x) + sd(x)
}

my_list <- list(a = 1:5, b = 1:10, c = rnorm(20))
my_list
```

```{r}
out <- vector("list", length(my_list))

for (i in 1:length(my_list)) {
  out[[i]] <- my_fun(my_list[[i]])
}

out
```

`lapply()` version: 
```{r}
out <- lapply(my_list, my_fun)
out
```

`lapply()` is still a loop. The counter is hidden.


```{r}

lapply(my_list, mean)
lapply(my_list, length)

```
Note: the output is always a list.

`sapply()`: elements of a list or vector
===

The `sapply()` function works just like `lapply()`, but tries to **simplify** the return value whenever possible. E.g., most common is the conversion from a list to a vector

- If results are dimension one:  returns a vector

- If results are same dimension > one: returns a matrix
 
- Otherwise:  returns a list

Observe the difference in structure.

```{r}
my_list <- list(a = 1:5, b = 1:10, c = rnorm(20))

lapply(my_list, FUN=mean)
sapply(my_list, FUN=mean) # Simplifies the result, now a vector
```

Why be cautious with `sapply()`?

Because the output type depends on the result.
This can cause subtle bugs if structure changes.

For example:

```{r}
lapply(my_list, FUN=summary) 
sapply(my_list, FUN=summary) ## returns a matrix
```
The output may be a matrix — or not — depending on what `summary()` returns.


`vapply()`: Type-safe version of `sapply()`
===
`vapply()` is a safer version of `sapply()`.

You must explicitly specify the expected output type.

Form:
`vapply(X, FUN, FUN.VALUE = numeric(1))`

`FUN.VALUE` tells R what each return value should look like.

```{r}
vapply(my_list, mean, FUN.VALUE = numeric(1))
```

This guarantees:

  - Each element returns a numeric scalar
  - Output will be a numeric vector
  - If a function returns the wrong type or length, R throws an error.

More robust inside larger functions. 

`tapply()`: levels of a factor vector
===

`tapply()` applies a function to subsets of a vector defined by a factor.

Form: `tapply(X, INDEX = group, FUN = my.fun)`

```{r}
# Compute the mean and sd of the Frost variable, within each region
tapply(state.x77[,"Frost"], INDEX=state.region, FUN=mean)
tapply(state.x77[,"Frost"], INDEX=state.region, FUN=sd)
```

```{r}
class(state.region)
table(state.region)
```

Equivalent loop logic:

  1. Split vector by group
  2. Apply function within each group
  3. Collect results

`split()`: split by levels of a factor
===

The function `split()` split up the rows of a data frame by levels of a factor, as in: `split(x, f=my.index)` to split a data frame `x` according to levels of `my.index`

```{r}
# Split up the state.x77 matrix according to region
state.by.reg = split(data.frame(state.x77), f=state.region)
class(state.by.reg) # The result is a list
names(state.by.reg) # This has 4 elements for the 4 regions
class(state.by.reg[[1]]) # Each element is a data frame
```

---

```{r}
# For each region, display the first 3 rows of the data frame
lapply(state.by.reg, FUN=head, 3) 
```

---

```{r}
# For each region, average each of the 8 numeric variables
lapply(state.by.reg, FUN=function(df) { 
  return(apply(df, MARGIN=2, mean)) 
})
```

Conceptual hierarchy
===

These functions are structured abstractions over loops:

  - for() → explicit iteration

  - apply() → loop over rows/column

  - lapply() → loop over list elements

  - sapply() → loop + auto-simplify

  - vapply() → loop + enforced type

  - tapply() → loop over groups

All of them are still iteration.

The only thing removed is the visible counter.

When should you use `apply()` vs `for()`?
===
Use `for()` when:

  - The logic is complex
  - You need conditional branching
  - You need multiple outputs
  - You are debugging

Use `apply()` family when:

  - The operation is clean and repeated
  - The structure is regular
  - The loop body is simple
