class: center, middle, inverse, title-slide .title[ # Intro to <code>R</code> (Part 2) ] .author[ ### Prof. Weldzius ] .institute[ ### Villanova University ] .date[ ### Lecture Date: Slides Updated: 2025-05-28 ] --- <style type="text/css"> .small .remark-code { /*Change made here*/ font-size: 85% !important; } .tiny .remark-code { /*Change made here*/ font-size: 50% !important; } </style> --- # How `RStudio` Works -- - `RStudio` allows us to: -- 1. Write scripts 2. Run scripts 3. See results -- - It is **deeply interactive** -- - We can highlight a line and press `ctrl+enter` / `cmd+enter` and see the result -- - **We can even do this with single objects!** --- # Give it a try - Comment **everything** -- - Create two objects - `a` contains the product of 3 and 5 (`3*5`) - `b` contains five numbers `c(10,21,43,87,175)` -- - Now create object `c` which is `a - b` ``` r # INSERT CODE HERE ``` --- # Functions & Packages -- - What are `packages`? -- - Basically, **functions** that someone else wrote -- - `R` has many functions already installed -- - These are known as "base `R`" and contain many useful functions -- - For example, `sum()` will add up a vector of numbers -- ``` r sum(object1) ``` -- - Quiz: What does the `mean()` function do? The `median()`? The `range()`? -- - Other base `R` functions interact with other files -- - For example, `read.csv()` will load a `.csv` file -- - And there are MANY **MANY** more --- # Installing Packages -- - In addition to the functions included in base `R`, we want more -- - For this class, we want one called `tidyverse` -- - `tidyverse` contains many (hundreds?) of functions that make `R` easier -- - But it is NOT included in the base `R` set of functions -- - Therefore, we need to add it -- - Use the base `R` function `install.packages("[PACKAGE NAME]")` -- - Specifically, `install.packages("tidyverse")` --- # Requiring Packages -- - Once installed, a package will live somewhere on your computer -- - However, any new *instance* of `R` will not automatically load the packages -- - We need to `require()` them to tell `R` to load them -- - Alternatively, we can use `library()` (but it's the same result) -- - So load the `tidyverse` package with `require(tidyverse)` -- - NB: you need quotes for the `install.packages()` function... -- - i.e., `install.packages("tidyverse")` -- - but NOT for the `require()` function -- - i.e., `require(tidyverse)` --- # Loading Data -- - So you should be using `R` via `RStudio` with the `tidyverse` package loaded -- - Now let's load some data -- - You can save it locally from the course [github page](https://github.com/rweldzius/PSC4175_SUM2025/raw/main/Data/sc_debt.Rds) and then load it from your computer -- - Or you can load it directly from the internet with the `read_rds()` function from `tidyverse` -- - NB: `R` is an "object-oriented language" (OOL) -- - We **create** an "object" to store the data using a left-arrow: `<-` -- ``` r df<-read_rds("https://github.com/rweldzius/PSC4175_SUM2025/raw/main/Data/sc_debt.Rds") ``` -- - url: https://github.com/rweldzius/PSC4175_SUM20256/raw/main/Data/sc_debt.Rds --- # Loading Data -- - We now have the contents of `sc_debt.Rds` stored in the object `df` -- - This is a "tabular data frame", aka a `tibble` - **Rows** are observations - **Columns** are values -- - We can look at this object directly ``` r df ``` ``` ## # A tibble: 2,546 × 16 ## unitid instnm stabbr grad_debt_mdn control region preddeg ## <int> <chr> <chr> <int> <chr> <chr> <chr> ## 1 100654 Alaba… AL 33375 Public South… Bachel… ## 2 100663 Unive… AL 22500 Public South… Bachel… ## 3 100690 Amrid… AL 27334 Private South… Associ… ## 4 100706 Unive… AL 21607 Public South… Bachel… ## 5 100724 Alaba… AL 32000 Public South… Bachel… ## 6 100751 The U… AL 23250 Public South… Bachel… ## 7 100760 Centr… AL 12500 Public South… Associ… ## 8 100812 Athen… AL 19500 Public South… Bachel… ## 9 100830 Aubur… AL 24826 Public South… Bachel… ## 10 100858 Aubur… AL 21281 Public South… Bachel… ## # ℹ 2,536 more rows ## # ℹ 9 more variables: openadmp <int>, adm_rate <dbl>, ## # ccbasic <int>, sat_avg <int>, md_earn_wne_p6 <int>, ## # ugds <int>, costt4_a <int>, selective <dbl>, ## # research_u <dbl> ``` --- # Loading Data -- - Or we can look at its columns ``` r names(df) ``` ``` ## [1] "unitid" "instnm" "stabbr" ## [4] "grad_debt_mdn" "control" "region" ## [7] "preddeg" "openadmp" "adm_rate" ## [10] "ccbasic" "sat_avg" "md_earn_wne_p6" ## [13] "ugds" "costt4_a" "selective" ## [16] "research_u" ``` --- # Loading Data -- <table class=" lightable-paper lightable-hover" style='font-size: 13px; color: black; font-family: "Arial Narrow", arial, helvetica, sans-serif; width: auto !important; margin-left: auto; margin-right: auto;'> <thead> <tr> <th style="text-align:left;"> Name </th> <th style="text-align:left;"> Definition </th> </tr> </thead> <tbody> <tr> <td style="text-align:left;"> unitid </td> <td style="text-align:left;"> Unit ID </td> </tr> <tr> <td style="text-align:left;"> instnm </td> <td style="text-align:left;"> Institution Name </td> </tr> <tr> <td style="text-align:left;"> stabbr </td> <td style="text-align:left;"> State Abbreviation </td> </tr> <tr> <td style="text-align:left;"> grad_debt_mdn </td> <td style="text-align:left;"> Median Debt of Graduates </td> </tr> <tr> <td style="text-align:left;"> control </td> <td style="text-align:left;"> Control Public or Private </td> </tr> <tr> <td style="text-align:left;"> region </td> <td style="text-align:left;"> Census Region </td> </tr> <tr> <td style="text-align:left;"> preddeg </td> <td style="text-align:left;"> Predominant Degree Offered: Assocates or Bachelors </td> </tr> <tr> <td style="text-align:left;"> openadmp </td> <td style="text-align:left;"> Open Admissions Policy: 1=Yes, 2=No, 3=No 1st time students </td> </tr> <tr> <td style="text-align:left;"> adm_rate </td> <td style="text-align:left;"> Admissions Rate: proportion of applications accepted </td> </tr> <tr> <td style="text-align:left;"> ccbasic </td> <td style="text-align:left;"> Type of institution* </td> </tr> <tr> <td style="text-align:left;"> sat_avg </td> <td style="text-align:left;"> Average SAT scores </td> </tr> <tr> <td style="text-align:left;"> md_earn_wne_p6 </td> <td style="text-align:left;"> Average Earnings of Recent Graduates </td> </tr> <tr> <td style="text-align:left;"> ugds </td> <td style="text-align:left;"> Number of undergraduates </td> </tr> <tr> <td style="text-align:left;"> costt4_a </td> <td style="text-align:left;"> Average cost of attendance (tuition-grants) </td> </tr> <tr> <td style="text-align:left;"> selective </td> <td style="text-align:left;"> Institution admits fewer than 10% of applications, 1=Yes, 0=No </td> </tr> <tr> <td style="text-align:left;"> research_u </td> <td style="text-align:left;"> Institution is a research university, 1=Yes, 0=No </td> </tr> </tbody> </table> &ast;<font size="2">See [here](https://data.ed.gov/dataset/9dc70e6b-8426-4d71-b9d5-70ce6094a3f4/resource/658b5b83-ac9f-4e41-913e-9ba9411d7967/download/collegescorecarddatadictionary_01192021.xlsx)</font> --- # Manipulating the Data -- - These data are cool! -- - But TMI at first -- - I want to know... -- - Where is `Villanova University`? -- - Who is the most selective? -- - Which schools produce the richest grads? -- - There are `tidyverse` functions to answer all of these questions --- # Manipulating with `tidyverse` - The code process of `tidyverse` relies on a "pipe" symbol: `%>%` -- - Think of it as a "chain" that **links code together** - Or maybe a "do" symbol because it tells `R` what to do -- - The basic grammar of `R` is: object, `%>%`, verb ``` r object %>% # This is the object function() # This is the verb ``` --- # Manipulating with `tidyverse` - `tidyverse` has many useful "verbs" (i.e., functions) - `filter()`: subsets **rows** - `select()`: subsets **columns** - `arrange()`: sorts **rows** based on **columns** - `summarise()`: collapses **rows** - `group_by()`: groups **rows** by **columns** --- # Manipulating: `filter()` - So let's look at Villanova -- - `filter` will select **rows** of the data based on some criteria -- ``` r df %>% filter(instnm == "Villanova University") # Only select rows with Villanova ``` ``` ## # A tibble: 1 × 16 ## unitid instnm stabbr grad_debt_mdn control region preddeg ## <int> <chr> <chr> <int> <chr> <chr> <chr> ## 1 216597 Villan… PA 26000 Private North… Bachel… ## # ℹ 9 more variables: openadmp <int>, adm_rate <dbl>, ## # ccbasic <int>, sat_avg <int>, md_earn_wne_p6 <int>, ## # ugds <int>, costt4_a <int>, selective <dbl>, ## # research_u <dbl> ``` --- # Manipulating: `select()` - Still TMI! -- - I only care about the admissions rate (`adm_rate`), the SAT scores (`sat_avg`), and the future earnings (`md_earn_wne_p6`) -- - `select` will select **columns** ``` r df %>% filter(instnm == "Villanova University") %>% select(instnm,adm_rate,sat_avg,md_earn_wne_p6) # Only select four columns ``` ``` ## # A tibble: 1 × 4 ## instnm adm_rate sat_avg md_earn_wne_p6 ## <chr> <dbl> <int> <int> ## 1 Villanova University 0.282 1422 62600 ``` --- # Manipulating: `arrange()` - How does Villanova compare...? -- - to other schools in terms of SAT scores? - to other schools in terms of future earnings? - to other schools in terms of admissions rates? -- - `arrange` will sort the data based on a column (ascending!) ``` r df %>% arrange(sat_avg) %>% # Sort data by SAT scores select(instnm,sat_avg) # Only look at name and SAT scores ``` ``` ## # A tibble: 2,546 × 2 ## instnm sat_avg ## <chr> <int> ## 1 Morgan State University 737 ## 2 Saint Augustine's University 847 ## 3 Albany State University 849 ## 4 Holy Names University 851 ## 5 Livingstone College 854 ## 6 Virginia Union University 855 ## 7 Manor College 861 ## 8 Saint Louis Christian College 865 ## 9 Bacone College 875 ## 10 Paine College 876 ## # ℹ 2,536 more rows ``` --- # Manipulating: `arrange()` - Villanova is not in the bottom 10 schools ``` r df %>% arrange(sat_avg) %>% # Sort data by SAT scores select(instnm,sat_avg) # Only look at name and SAT scores ``` ``` ## # A tibble: 2,546 × 2 ## instnm sat_avg ## <chr> <int> ## 1 Morgan State University 737 ## 2 Saint Augustine's University 847 ## 3 Albany State University 849 ## 4 Holy Names University 851 ## 5 Livingstone College 854 ## 6 Virginia Union University 855 ## 7 Manor College 861 ## 8 Saint Louis Christian College 865 ## 9 Bacone College 875 ## 10 Paine College 876 ## # ℹ 2,536 more rows ``` --- # Manipulating: `arrange()` - Use `desc()` to order in descending Villanova not in top 10 either ``` r df %>% arrange(desc(sat_avg)) %>% # Sort data by SAT scores (descending) select(instnm,sat_avg) # Only look at name and SAT scores ``` ``` ## # A tibble: 2,546 × 2 ## instnm sat_avg ## <chr> <int> ## 1 California Institute of Technology 1557 ## 2 Massachusetts Institute of Technology 1547 ## 3 University of Chicago 1528 ## 4 Harvey Mudd College 1526 ## 5 Duke University 1522 ## 6 Franklin W Olin College of Engineering 1522 ## 7 Washington University in St Louis 1520 ## 8 Rice University 1520 ## 9 Yale University 1517 ## 10 Harvard University 1517 ## # ℹ 2,536 more rows ``` --- # Manipulating: `arrange()` - What if we look only at "selective" schools (i.e., those who accept less than 10% of applicants)? -- ``` r df %>% filter(adm_rate < .1) %>% # Only look at schools who accept less than 10% arrange(sat_avg,adm_rate) %>% # Sort data by SAT scores AND THEN admissions rates (breaks ties) select(instnm,adm_rate,sat_avg) # Only look at name, admissions rate, and SAT scores ``` ``` ## # A tibble: 25 × 3 ## instnm adm_rate sat_avg ## <chr> <dbl> <int> ## 1 Colby College 0.0967 1456 ## 2 Swarthmore College 0.0893 1469 ## 3 Pomona College 0.074 1480 ## 4 Dartmouth College 0.0793 1500 ## 5 Stanford University 0.0434 1503 ## 6 Northwestern University 0.0905 1506 ## 7 Columbia University in the City of New … 0.0545 1511 ## 8 Brown University 0.0707 1511 ## 9 University of Pennsylvania 0.0766 1511 ## 10 Vanderbilt University 0.0912 1515 ## # ℹ 15 more rows ``` --- # How does Villanova compare? -- - `arrange` in descending order ``` r df %>% filter(adm_rate < .1) %>% arrange(desc(sat_avg),adm_rate) %>% select(instnm,adm_rate,sat_avg) ``` ``` ## # A tibble: 25 × 3 ## instnm adm_rate sat_avg ## <chr> <dbl> <int> ## 1 California Institute of Technology 0.0642 1557 ## 2 Massachusetts Institute of Technology 0.067 1547 ## 3 University of Chicago 0.0617 1528 ## 4 Duke University 0.076 1522 ## 5 Rice University 0.0872 1520 ## 6 Harvard University 0.0464 1517 ## 7 Princeton University 0.0578 1517 ## 8 Yale University 0.0608 1517 ## 9 Vanderbilt University 0.0912 1515 ## 10 Columbia University in the City of New … 0.0545 1511 ## # ℹ 15 more rows ``` --- # More complicated? More `%>%`! -- - Less selective schools by SAT with debt and state ``` r df %>% filter(adm_rate > .2 & adm_rate < .3) %>% # Less selective schools (accept between 20% and 30%) arrange(stabbr,desc(sat_avg)) %>% # Sort by state name, then by SAT scores select(instnm,sat_avg,grad_debt_mdn,stabbr) # Only look at some columns ``` ``` ## # A tibble: 37 × 4 ## instnm sat_avg grad_debt_mdn stabbr ## <chr> <int> <int> <chr> ## 1 Heritage Christian Universi… NA NA AL ## 2 University of California-Sa… 1370 15000 CA ## 3 California Polytechnic Stat… 1342 19501 CA ## 4 University of California-Ir… 1306 15488 CA ## 5 California Institute of the… NA 27000 CA ## 6 University of Miami 1371 17125 FL ## 7 Georgia Institute of Techno… 1418 23000 GA ## 8 Point University 986 26000 GA ## 9 Grinnell College 1457 17500 IA ## 10 St Luke's College NA 17750 IA ## # ℹ 27 more rows ``` --- # A quick aside on missingness -- - Some rows have `NA` in some columns -- - `NA` is the standard code for **missing data** in `R` - Data can be missing for many different reasons (i.e., some schools don't require SAT scores or record them) - We can use a base `R` function called `is.na()` which will be `TRUE` if the value is `NA` or `FALSE` otherwise -- - And we can combine `is.na()` with the `filter()` function from `tidyverse` -- - We will return to this in the lectures on **data wrangling** -- - For now, how many schools don't report SAT scores? ``` r df %>% filter(is.na(sat_avg)) %>% # Only look at schools that DON'T report SATs select(instnm,stabbr) # Only look at the name and the state ``` ``` ## # A tibble: 1,317 × 2 ## instnm stabbr ## <chr> <chr> ## 1 Amridge University AL ## 2 Central Alabama Community College AL ## 3 Athens State University AL ## 4 Chattahoochee Valley Community College AL ## 5 Coastal Alabama Community College AL ## 6 Gadsden State Community College AL ## 7 George C Wallace State Community College-Selma AL ## 8 Heritage Christian University AL ## 9 Jefferson State Community College AL ## 10 Lurleen B Wallace Community College AL ## # ℹ 1,307 more rows ``` --- # Stepping back -- - Thus far, lots of .red[data] -- - Not a lot of .blue[science] -- - But remember the .blue[Research] camp! -- 1. .red[Observation] &rarr; .blue[Question] 2. .blue[Theory] &rarr; .blue[Hypothesis] 3. .red[Data Collection / Wrangling] &rarr; .red[Analysis] 4. .red[Results] &rarr; .blue[Conclusion] -- - We have been doing lots of .red[Observation]! -- - Do we have any good .blue[Research questions]? --- # Stepping back - .blue[RQ]: How might admissions and SAT scores be **related**? -- - .blue[Theory]: selective schools have stricter criteria -- - .blue[Hypothesis]: admissions and SAT scores should be **negatively** related -- - How can we test this hypothesis? --- # Summarizing Data: `summarise()` + `mean()` -- - We can combine base `R` functions with `tidyverse` functions! -- - Base `R`: `mean()` - `tidyverse`: `summarise()` (aka `summarize()`) - Overall average SAT scores ``` r df %>% summarise(mean_sat = mean(sat_avg,na.rm=T)) # Average SAT scores for entire data ``` ``` ## # A tibble: 1 × 1 ## mean_sat ## <dbl> ## 1 1141. ``` --- # Summarizing Data -- - Let's unpack this ``` r df %>% summarise(mean_sat = mean(sat_avg,na.rm=T)) ``` -- - Create new variable `mean_sat` that contains the `mean()` of every school's average SAT score -- - `na.rm=T` means we want to ignore missing data. If not? -- ``` r df %>% summarise(mean_sat = mean(sat_avg)) ``` ``` ## # A tibble: 1 × 1 ## mean_sat ## <dbl> ## 1 NA ``` --- # Summarizing Data -- - Recall we want see if more selective schools have higher SAT scores -- ``` r df %>% filter(adm_rate < .1) %>% # Only look at schools who accept less than 10% of applicants summarise(mean_sat_LT10 = mean(sat_avg,na.rm=T)) # Calculate the average SAT score ``` ``` ## # A tibble: 1 × 1 ## mean_sat_LT10 ## <dbl> ## 1 1510. ``` ``` r df %>% filter(adm_rate > .1) %>% # Only look at schools who accept more than 10% of applicants summarise(mean_sat_GT10 = mean(sat_avg,na.rm=T)) # Calculate the average SAT score ``` ``` ## # A tibble: 1 × 1 ## mean_sat_GT10 ## <dbl> ## 1 1135. ``` --- # Summarizing Data: `group_by()` -- - One final `tidyverse` function: `group_by()` -- - There is a column called `selective` which is either 1 or 0 -- - 1: the admissions rate is less than 10% - 0: otherwise -- ``` r df %>% select(instnm,selective,adm_rate) ``` ``` ## # A tibble: 2,546 × 3 ## instnm selective adm_rate ## <chr> <dbl> <dbl> ## 1 Alabama A & M University 0 0.918 ## 2 University of Alabama at Birmingham 0 0.737 ## 3 Amridge University NA NA ## 4 University of Alabama in Huntsville 0 0.826 ## 5 Alabama State University 0 0.969 ## 6 The University of Alabama 0 0.827 ## 7 Central Alabama Community College NA NA ## 8 Athens State University NA NA ## 9 Auburn University at Montgomery 0 0.904 ## 10 Auburn University 0 0.807 ## # ℹ 2,536 more rows ``` --- # Summarizing Data: `group_by()` - Instead of running two separate `filter()` commands, use `group_by()` -- ``` r df %>% group_by(selective) %>% # Group the data by selective (either 1 or 0) summarise(mean_sat = mean(sat_avg,na.rm=T)) # Calculate average SAT for each group ``` ``` ## # A tibble: 3 × 2 ## selective mean_sat ## <dbl> <dbl> ## 1 0 1135. ## 2 1 1510. ## 3 NA NaN ``` --- # Results - Do more selective schools have higher SAT scores? -- - Yes -- - This .red[Result] **confirms** our .blue[Hypothesis] and **answers** our .blue[Research Question] --- # Conclusion -- - What we've done today is a microcosm of data science -- 1. Opened .red[data] (`readRDS`) -- 2. Looked at .red[data] (`tidyverse` + `select()`, `filter()`, `arrange()`) -- 3. Generated .blue[hypotheses] (Admissions versus SAT scores) -- 4. .red[Tested] .blue[hypotheses] (`summarise()` + `mean()`) --- # Homework 1. Work through psc4175_hw_2.Rmd 2. Complete Problem Set 0 by Friday at midnight