README.Rmd

---
title: "Corpus MilesDavis CompMu 2020"
author: "Willem Pleiter"
date: "2020"
output: github_document

---

### __The Miles Davis Corpus__
For my corpus I have decided to investigate the jazz trumpet player Miles Davis. Davis is known within the jazz and global community as a constant innovator and key figure of jazz music. His method for innovation was to constantly attract upcoming talents and stars within the upcoming current in jazz. He then instructed these youngsters with his own vision of jazz and this was reciprocated by the youngsters how provided him with their own new views. 

In my corpus I will try to analyse Davis’ music by answering some of the following questions:

-	How did Davis’ music change over time, what are the constants in his music and what are the variables?

-	How did Davis’ music reflect other albums of each specific time period that he was adapting to?


The following phases of Miles Davis’ oeuvre could be examined

-	Early bebop/ Charlie Parker phase

-	Hard Bop phase: (Miles Davis quartet with Horace Silver, Max Roach, Charles Mingus, Art Blakey)

-	Gil Evans orchestral phase (Sketches in Spain)

-	Miles Davis first quintet (Coltrane, Adderley, Evans, Kelly). This coincides with Gil Evans

-	Miles Davis second quintet (Carter, Hancock, Williams, Shorter)

-	Miles Davis electric/funk fusion (Corea, Jarrett) (Bitches Brew, 

-	Miles Davis post 80ies Hip-hop modern contemporary phase

I intend to use atleast one album of each period and compare them to each other and perhaps compare them to other big albums. This comparison will be mostly analysed through spotify data, but when allowed I would like to also harmonically and culturally analyse them. For instance, I know that Miles Davis was heavily influenced by Ahmad Jamal prior to his Cool Jazz phase, which could be interesting to compare.



#### __Preliminary Analysis__
For the preliminary analysis, I will just get the entire discography of Miles from Spotify and run some general statistic analyses:

loading libraries
```{r message=FALSE, warning=FALSE}
# Load libraries (every time)

library(tidyverse)
library(spotifyr)
library(ggplot2)
library(compmus)
source('spotify.R')
```

__Audio Features__ <br>
Getting the entire Miles Davis catalogue. This is a very rough representation of his statistics, since a lot of compilation albums, posthumous releases containing the same tracks are included.

```{r message=FALSE, warning=FALSE}
miles_davis <- get_artist_audio_features("Miles Davis")
```

__Histograms__ <br>
Showing Histograms of the main audio features of all songs:
```{r echo = FALSE}
par(mfrow = c(2, 2))
hist(miles_davis$danceability, main = "Miles Davis Danceability", xlab = "Danceability")
hist(miles_davis$energy, main = "Miles Davis Energy", xlab = "Energy")
hist(miles_davis$speechiness, main = "Miles Davis Speechiness", xlab = "Speechiness")
hist(miles_davis$acousticness, main = "Miles Davis Acousticness", xlab = "Acousticness")
par(mfrow = c(2, 2))
hist(miles_davis$instrumentalness, main = "Miles Davis Instrumentalness", xlab = "Instrumentalness")
hist(miles_davis$liveness, main = "Miles Davis Liveness", xlab = "Liveness")
hist(miles_davis$valence, main = "Miles Davis Valence", xlab = "Valence")
hist(miles_davis$loudness, main = "Miles Davis Loudness", xlab = "Loudness")

```

Here are the histograms of music features like key, mode, tempo of each song
```{r}
par(mfrow = c(2,2))
plot(table(miles_davis$key), main = "Miles Davis Key", xlab = "key, C = 1, C# = 2 etc.", ylab = "Frequency")
hist(miles_davis$mode, main = "Miles Davis Mode", xlab = "Mode (0 = minor, 1 = major)")
hist(miles_davis$tempo, main = "Miles Davis Tempo", xlab = "Tempo in bpm")

```

__Descriptive statistics and outliers__ <br>
Here are some descriptive statistics:
```{r, echo = FALSE}
Miles_matrix <- matrix(0, 11, 4)
rownames(Miles_matrix) = c("Danceability", "Energy", "Key", "Loudness", "Mode", "Speechiness", "Acousticness", "Instrumentalness", "Liveness", "Valence", "Tempo") 
colnames(Miles_matrix) = c("Mean", "SD", "Median", "Mode")

for(i in 1:11){
  Miles_matrix[i, 1] <- mean(miles_davis[,i + 8])
  Miles_matrix[i, 1] <- round(Miles_matrix[i, 1], digits = 2)
}
for(i in 1:11){
  Miles_matrix[i, 2] <- sd(miles_davis[,i + 8])
  Miles_matrix[i, 2] <- round(Miles_matrix[i, 2], digits = 2)
}
for(i in 1:11){
  Miles_matrix[i, 3] <- median(miles_davis[,i + 8])
  Miles_matrix[i, 3] <- round(Miles_matrix[i, 3], digits = 2)
}


getmode <- function(v) {
   uniqv <- unique(v)
   uniqv[which.max(tabulate(match(v, uniqv)))]
}

for(i in 1:11){
  Miles_matrix[i, 4] <- getmode(miles_davis[,i + 8])
  Miles_matrix[i, 4] <- round(Miles_matrix[i, 4], digits = 2)
}

```


#### Week 7
In this week I want to further get into my preliminary analysis by using ggplot. The following goals are formulated:
-	Filter out compilation/posthumous albums

-	Subset data according to decades

- Create plots comparing different characteristics from each decade

__Ggplot__ <br>
Working with ggplot to plot danceability vs popularity scatterplot and valence added as well.
```{r}
Dance_v_pop <- ggplot(miles_davis, aes(x = danceability, y = energy, color = valence)) + geom_point()
Dance_v_pop
```


__Subsets__ <br>
Lets create subsets based on decades:
```{r}
#50ies
miles_50s <- subset(miles_davis, album_release_year > 1950 & album_release_year < 1960)
miles_50s$Time_Period <- "1950ies"

#60ies
miles_60s <- subset(miles_davis, album_release_year > 1959 & album_release_year < 1970)
miles_60s$Time_Period <- "1960ies"

#70ies
miles_70s <- subset(miles_davis, album_release_year > 1969 & album_release_year < 1980)
miles_70s$Time_Period <- "1970ies"

#80ies_todeath
miles_80s <- subset(miles_davis, album_release_year > 1979 & album_release_year < 1992)
miles_80s$Time_Period <- "1980ies"



miles_decades <- rbind(miles_50s, miles_60s, miles_70s, miles_80s)

```


Making Facet grids based on decade

```{r}
ggplot(miles_decades, aes(x = danceability, y = energy, col = valence)) + 
  geom_point(size = 3, alpha = 0.6) +
  facet_wrap(. ~ Time_Period)
```

__Barplots and descriptive statistics__ <br>
Checking keys with barplots:
```{r}
ggplot(miles_decades, aes(key_name, fill = key_name)) + 
  geom_bar() + 
  theme(legend.position = "none") + 
  labs(x = "key") + 
  facet_grid(. ~ Time_Period)
```
Checking modes:
```{r}
ggplot(miles_decades, aes(as.factor(mode_name), fill = as.factor(mode_name))) + 
  geom_bar() + 
  theme(legend.position = "none") + 
  labs(x = "mode") + 
  facet_grid(. ~ Time_Period)
```



Comparing means with a barplot:
```{r echo = FALSE}
Mean_danceability <- ggplot(miles_decades, aes(x = Time_Period, y = danceability, fill = Time_Period)) + 
  geom_bar(stat = "summary", position = "dodge", fun.y = "mean") + 
  theme(legend.position = "none") + 
  labs(x = "", y = "Mean Danceability")

Mean_Energy <- ggplot(miles_decades, aes(x = Time_Period, y = energy, fill = Time_Period)) + 
  geom_bar(stat = "summary", position = "dodge", fun.y = "mean") + 
  theme(legend.position = "none") + 
  labs(x = "", y = "Mean Energy")

Mean_val <- ggplot(miles_decades, aes(x = Time_Period, y = valence, fill = Time_Period)) + 
  geom_bar(stat = "summary", position = "dodge", fun.y = "mean") + 
  theme(legend.position = "none") + 
  labs(x = "", y = "Mean Valence")


Mean_live <- ggplot(miles_decades, aes(x = Time_Period, y = liveness, fill = Time_Period)) + 
  geom_bar(stat = "summary", position = "dodge", fun.y = "mean") + 
  theme(legend.position = "none") + 
  labs(x = "", y = "Mean Liveness")


library(gridExtra)
grid.arrange(Mean_danceability, Mean_Energy, Mean_val, Mean_live, nrow = 2, ncol=2)


```

__Conclusion Week 7__ <br>
This week I've created subsets and found some interesting differences between decades with regards to acousticness, energy and liveness and I can already get a pretty clear picture of each decade. However the decade subset is still a bit arbitrary, perhaps some historic research is required to correctly split the time periods to reflect Miles' innovations. Furthermore the following could be done:
-	Get audio-analysis features from spotify or track popularity data (although not historic)

- Subset time periods more accurately

- maybe start with running some statistical tests or analysis

- formulate hypotheses about what you want to test and why.

- include some examples via Spotify Embed, polish up portfolio and start to actually create a timeline.


#### Week 8

__Subsets version 2 (week 8__ <br>
I've done some research and created the following subsets based on historical data when Miles switched quartets and genres (according to wikipedia:
```{r}
#Miles_Parker(1944-1950)

miles40ies <- get_playlist_audio_features("Wpleiter", "1v8oL4Clr0LliAh6HNK4C2")
miles40ies_cut <- as.data.frame(miles40ies[,c(6:16)])
miles40ies_cut$Time_Period <- "1944-1950"
#1950-1955
Miles_early_50s <- get_playlist_audio_features("Wpleiter", "3lv2u17pbcXIoNZZtP8wZ5")
milesE50ies <- as.data.frame(c(Miles_early_50s[,c(6:16)]))
milesE50ies$Time_Period <- "1951-1954"

#first quintet and 1955-1962
milesmodal <- subset(miles_davis, album_release_year > 1954 & album_release_year < 1963)
milesmodal_short <- as.data.frame(milesmodal[,c(9:19)])
milesmodal_short$Time_Period <- "1955-1962"


#second quintet 1963-1968
miles_herbie <- subset(miles_davis, album_release_year > 1963 & album_release_year < 1969)
miles_herbie_short <- as.data.frame(miles_herbie[,c(9:19)])
miles_herbie_short$Time_Period <- "1963-1968"

#Milesfusion:
miles_fusion <- subset(miles_davis, album_release_year > 1969 & album_release_year < 1975)
miles_fusionshort <- as.data.frame(miles_fusion[,c(9:19)])
miles_fusionshort$Time_Period <- "1969-1975"

#Miles_hiatus
miles_hiatus <- subset(miles_davis, album_release_year > 1976 & album_release_year < 1979)
miles_hiatus_short <- as.data.frame(miles_hiatus[,c(9:19)])
miles_hiatus_short$Time_Period <- "1976 - 1979"

#80ies_todeath
miles_death <- subset(miles_davis, album_release_year > 1979 & album_release_year < 1992)
miles_death_short <- as.data.frame(miles_death[,c(9:19)])
miles_death_short$Time_Period <- "1980 - 1992"

miles_long <- rbind(miles40ies_cut, milesE50ies, milesmodal_short, miles_herbie_short, miles_fusionshort, miles_hiatus_short, miles_death_short)
miles_wide <- miles_long %>%
  gather("danceability", "energy", "speechiness", "acousticness", "instrumentalness", "liveness", "valence", key = Feature, value = Parameter)



```

__Scatterplot__ <br>
```{r echo = FALSE}
ggplot(miles_long, aes(x = danceability, y = energy, col = valence)) + 
  geom_point(size = 3, alpha = 0.6) +
  facet_wrap(. ~ Time_Period)


```

__Ggslopegraph__ <br>
```{r echo = FALSE}
#Slopegraph is not in my gg-package for some reason, trying something else:
if (!require("remotes")) {
    install.packages("remotes")
}
remotes::install_github("leeper/slopegraph")

library("slopegraph")

#Creating a unique matrix to fulfill the conditions of ggplot
Miles_slope <- matrix(0, 7, 7)
colnames(Miles_slope) <- unique(miles_wide$Time_Period)
rownames(Miles_slope) <- names(miles_death_short[,c(1,2,6,7,8,9,10)])


#40ies
Miles_slope[1,1] <- mean(miles40ies_cut[,1])
Miles_slope[2,1] <- mean(miles40ies_cut[,2])
Miles_slope[3,1] <- mean(miles40ies_cut[,6])
Miles_slope[4,1] <- mean(miles40ies_cut[,7])
Miles_slope[5,1] <- mean(miles40ies_cut[,8])
Miles_slope[6,1] <- mean(miles40ies_cut[,9])
Miles_slope[7,1] <- mean(miles40ies_cut[,10])



#second
Miles_slope[1,2] <- mean(milesE50ies[,1])
Miles_slope[2,2] <- mean(milesE50ies[,2])
Miles_slope[3,2] <- mean(milesE50ies[,6])
Miles_slope[4,2] <- mean(milesE50ies[,7])
Miles_slope[5,2] <- mean(milesE50ies[,8])
Miles_slope[6,2] <- mean(milesE50ies[,9])
Miles_slope[7,2] <- mean(milesE50ies[,10])


#Third
Miles_slope[1,3] <- mean(milesmodal_short[,1])
Miles_slope[2,3] <- mean(milesmodal_short[,2])
Miles_slope[3,3] <- mean(milesmodal_short[,6])
Miles_slope[4,3] <- mean(milesmodal_short[,7])
Miles_slope[5,3] <- mean(milesmodal_short[,8])
Miles_slope[6,3] <- mean(milesmodal_short[,9])
Miles_slope[7,3] <- mean(milesmodal_short[,10])


#4
Miles_slope[1,4] <- mean(miles_herbie_short[,1])
Miles_slope[2,4] <- mean(miles_herbie_short[,2])
Miles_slope[3,4] <- mean(miles_herbie_short[,6])
Miles_slope[4,4] <- mean(miles_herbie_short[,7])
Miles_slope[5,4] <- mean(miles_herbie_short[,8])
Miles_slope[6,4] <- mean(miles_herbie_short[,9])
Miles_slope[7,4] <- mean(miles_herbie_short[,10])

#5
Miles_slope[1,5] <- mean(miles_fusionshort[,1])
Miles_slope[2,5] <- mean(miles_fusionshort[,2])
Miles_slope[3,5] <- mean(miles_fusionshort[,6])
Miles_slope[4,5] <- mean(miles_fusionshort[,7])
Miles_slope[5,5] <- mean(miles_fusionshort[,8])
Miles_slope[6,5] <- mean(miles_fusionshort[,9])
Miles_slope[7,5] <- mean(miles_fusionshort[,10])

#6
Miles_slope[1,6] <- mean(miles_hiatus_short[,1])
Miles_slope[2,6] <- mean(miles_hiatus_short[,2])
Miles_slope[3,6] <- mean(miles_hiatus_short[,6])
Miles_slope[4,6] <- mean(miles_hiatus_short[,7])
Miles_slope[5,6] <- mean(miles_hiatus_short[,8])
Miles_slope[6,6] <- mean(miles_hiatus_short[,9])
Miles_slope[7,6] <- mean(miles_hiatus_short[,10])

#7
Miles_slope[1,7] <- mean(miles_death_short[,1])
Miles_slope[2,7] <- mean(miles_death_short[,2])
Miles_slope[3,7] <- mean(miles_death_short[,6])
Miles_slope[4,7] <- mean(miles_death_short[,7])
Miles_slope[5,7] <- mean(miles_death_short[,8])
Miles_slope[6,7] <- mean(miles_death_short[,9])
Miles_slope[7,7] <- mean(miles_death_short[,10])


Miles_slope <- as.data.frame(Miles_slope)

data <- Miles_slope
xlabels <- names(data)

palette <- c("#000000", "#E69F00", "#56B4E9", "#009E73", "#F0E442", "#0072B2", "#D55E00")
ggslopegraph(Miles_slope, offset.x = 0.06,xlim = c(-0.5,7.9),
  decimals = 2, col.lab = palette, col.lines = palette, col.num = palette, cex.lab = 3.5) + 
  theme(panel.grid =element_blank(), panel.background = element_blank())



```
### Week 10
Making self-similarity matrices

```{r}
#four levels of structure
#Segments are not regularly spaced, but the smallest units possible, Spotify makes a very finely grained self-similarity-matrix
#Beats, Bars and sections
#segment for spotify is super low short thing. Section is a chorus, verse, bridge etc. Used interchangeably in literature but section is higher level

#You can use euclidian or manhattan
#Chebyshev is not useful for distance
#If you're working with Chroma + manhattan, use manhattan and aitchison, Aitchison most musical
#Euclidean is also possible
#Chebyshev is not great for similarity matrix, chromagram it could be used, but try to stick to manhattan or Euclidean

#TIMBRE FEATURES
#Spotify gives you average values of zero, but range is Inf to -Inf. Positive is high, negative is low, should be centered at 0



get_track_audio_analysis()
```





```{r}
ST1958 <- 
  get_tidy_audio_analysis("URI")
  compmus_align(bars, segments %>%#you can use beats, or sections as well) 
                select(bars) %>% unnest(bars) %>%
                  mutate(
                    pitches = 
                      map(segments,
                          compus_summarise, pitches,
                          method = 'rms', norm = 'euclidian')) %>% #aitchision is also possible (lower case)
                  mutate(
                    timbre = 
                      map(segments,
                          compmus_summarise, timbre,
                          method = "mean")
                  )
                  
```

Plots See Canvas
```{r}

```