Lab 07 Visualizing in ggplot2

URBAN 520 Quantitative Methods in Urban Design and Planning


In this lab, we are going to explore tidy data, start to visualize data using R, and produce nice diagram using ggplot2. Please try to understand each step, including the rational, functions, and arguments. The dataset we are using is residential real estate rental listing data, from RentHub and can be accessed through Dewey Data Platform (UW NetID required).

Recommended Reading
Chapter 2, 6, Modern Data Science with R. Baumer et al. 2024.
Tidy Data, The Journal of Statistical Software, Hadley Wickham, 2014.


library('tidyverse')
library('ggplot2') # the major package for visualization in R
library('skimr')

Data Preparation

Instead of using API or R data package as we did in previous labs, we will use data on our local device. We have two datasets in csv file rent_city_wide.csv and rental_data_wa.csv. Comma-separated values (CSV) is a text data format that uses commas to separate delimiter-separated values, and newlines to separate records.

Rental Listing Data

RentHub provides us residential real estate rental listing data, including pricing, property information, location, and more from 2014 to present. The whole dataset is huge (over 100 GB). Therefore, we only select all data for the State of Washington during 2019-01-01 to 2023-12-31. Note: the time refers to when RentHub collected the data, rather than when the listing was originally posted. To keep the file small, we further keep listings posted between 2019 and 2023 and 17 of the 25 columns. You can access the whole data through Dewey Data Platform (UW NetID required).

# load the rental listing data, it may take a while because its size
rental <- read_csv("rental_data_wa.csv")
## Rows: 2268590 Columns: 17
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr  (11): CITY, BUILDING_TYPE, GRANITE, STAINLESS, POOL, GYM, DOORMAN, CLUB...
## dbl   (5): ZIP, BEDS, BATHS, SQFT, RENT_PRICE
## date  (1): DATE_POSTED
## 
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

There are 2,268,590 observations (rows) and 17 variables (columns) in total. The dataset is already tidy, so no further transformation is required. You can take a look at the meaning of each column.

Click to Read Data Dictionary
# skim the whole dataset
# skim(rental)

Aggregation Rental Listing Data by City

We have organized the average monthly rent of the 10 Washington cities with the most rental listings from 2019 to 2023 to have a smaller dataset.

rent_city <- read_csv("rent_city_wide.csv")
## Rows: 10 Columns: 63
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr  (2): CITY, COUNTY
## dbl (61): LISTINGS, 2019-01-01, 2019-02-01, 2019-03-01, 2019-04-01, 2019-05-...
## 
## ℹ Use `spec()` to retrieve the full column specification for this data.
## ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

CSV files are just plain text. You can use your text editor (TextEdit on Mac and Notepad on Windows) to have a look. They do not store metadata such as column types, units, or formats. So. when we use read_csv(), it automatically guesses column types by inspecting the data. Sometimes its guess is wrong, we need to specify the types using spec().

rent_city <- read_csv("rent_city_wide.csv",
  col_types = cols(
    CITY     = col_character(),
    COUNTY   = col_character(),
    LISTINGS = col_number(),
    .default = col_double()   # for other columns
  )
)
head(rent_city, 3)
## # A tibble: 3 × 63
##   CITY    COUNTY LISTINGS `2019-01-01` `2019-02-01` `2019-03-01` `2019-04-01`
##   <chr>   <chr>     <dbl>        <dbl>        <dbl>        <dbl>        <dbl>
## 1 Seattle King     585257         2049         2140         2178         2199
## 2 Renton  King     126745         1861         1874         1872         1935
## 3 Tacoma  Pierce   115160         1557         1584         1549         1587
## # ℹ 56 more variables: `2019-05-01` <dbl>, `2019-06-01` <dbl>,
## #   `2019-07-01` <dbl>, `2019-08-01` <dbl>, `2019-09-01` <dbl>,
## #   `2019-10-01` <dbl>, `2019-11-01` <dbl>, `2019-12-01` <dbl>,
## #   `2020-01-01` <dbl>, `2020-02-01` <dbl>, `2020-03-01` <dbl>,
## #   `2020-04-01` <dbl>, `2020-05-01` <dbl>, `2020-06-01` <dbl>,
## #   `2020-07-01` <dbl>, `2020-08-01` <dbl>, `2020-09-01` <dbl>,
## #   `2020-10-01` <dbl>, `2020-11-01` <dbl>, `2020-12-01` <dbl>, …

We will not see the message if we specified the data types. let’s have a look of the first 3 rows of this data. The data is the average rent of all listings posted in each month. The data includes average rent for 10 cities from 2019 to 2023 by month!

Currently, the rent_city dataset has 10 rows and 63 columns. This presentation of the data has some advantages: we can easily look through the average rent for single city by date (one row). But, we have 60 columns for average rent, which treated as separate variables now. In data science, researchers defined a particular format of data table: tidy data - A research paper about tidy data.

Tidy Data

Data Tidying

This section was adopted from Tidy data page, authored by Hadley Wickham.

Happy families are all alike; every unhappy family is unhappy in its own way — Leo Tolstoy

Like families, tidy datasets are all alike but every messy dataset is messy in its own way. The principles of tidy data provide a standard way to organize data values within a dataset. A standard makes initial data cleaning easier because you don’t need to start from scratch and reinvent the wheel every time. The tidy data standard has been designed to facilitate initial exploration and analysis of the data, and to simplify the development of data analysis tools that work well together.

Data Semantics

A dataset is a collection of values, usually either numbers or strings. Values are organised in two ways: every value belongs to a variable and an observation.

head(rent_city, 3)
## # A tibble: 3 × 63
##   CITY    COUNTY LISTINGS `2019-01-01` `2019-02-01` `2019-03-01` `2019-04-01`
##   <chr>   <chr>     <dbl>        <dbl>        <dbl>        <dbl>        <dbl>
## 1 Seattle King     585257         2049         2140         2178         2199
## 2 Renton  King     126745         1861         1874         1872         1935
## 3 Tacoma  Pierce   115160         1557         1584         1549         1587
## # ℹ 56 more variables: `2019-05-01` <dbl>, `2019-06-01` <dbl>,
## #   `2019-07-01` <dbl>, `2019-08-01` <dbl>, `2019-09-01` <dbl>,
## #   `2019-10-01` <dbl>, `2019-11-01` <dbl>, `2019-12-01` <dbl>,
## #   `2020-01-01` <dbl>, `2020-02-01` <dbl>, `2020-03-01` <dbl>,
## #   `2020-04-01` <dbl>, `2020-05-01` <dbl>, `2020-06-01` <dbl>,
## #   `2020-07-01` <dbl>, `2020-08-01` <dbl>, `2020-09-01` <dbl>,
## #   `2020-10-01` <dbl>, `2020-11-01` <dbl>, `2020-12-01` <dbl>, …

Look at our rent_city dataset, it contains 630 values representing 63 variables and 10 observations. The variables are:

A tidy version of the data looks like this:

rent_city_tidy <- rent_city %>%
  pivot_longer( # using `pivot_longer` to reshape wide → long
    cols = -c("CITY", "COUNTY", "LISTINGS"), # keep these 3 as identifier variables (don’t pivot them)
    names_to = "month",  # old column names (like "2019-01-01") become a new column called month
    values_to = "mean_rent" # the cell values go into a new column called mean_rent
    # values_drop_na = TRUE; default as FALSE; if TRUE, will remove rows with missing mean_rent.
  ) %>%
  mutate(
    month = as.Date(month, format = "%Y-%m-%d")
  )
head(rent_city_tidy,3)
## # A tibble: 3 × 5
##   CITY    COUNTY LISTINGS month      mean_rent
##   <chr>   <chr>     <dbl> <date>         <dbl>
## 1 Seattle King     585257 2019-01-01      2049
## 2 Seattle King     585257 2019-02-01      2140
## 3 Seattle King     585257 2019-03-01      2178
From Wider to Longer. Source: Cheatsheet of tidyr.

Look at our rent_city_tidy dataset again, it contains 3,000 values representing 5 variables and 600 observations. The variables are:

The tidy data frame explicitly tells us the definition of an observation. Every combination of CITY and month is a single measured observation (mean_rent). It’s important to clearly define what is an observation. In the case of rent, we are interested in average rent for sure! For an observation, such as 2049, we need know two information: city and month.

Tidy data

In tidy data:

Tidy data makes it easy for an analyst or a computer to extract needed variables because it provides a standard way of structuring a dataset. Tidy data is particularly well suited for vectorised programming languages like R, because the layout ensures that values of different variables from the same observation are always paired. Many R packages, such as ggplot2 are designed around the tidy data principles.

From Tidy Data to Wider Data

We can use pivot_wider() to perform the inverse operation.

rent_city_wide <- rent_city_tidy %>%
  pivot_wider(
    names_from = month,   # expand `month` to multiple columns
    values_from = mean_rent    # extract `mean_rent` column to those new columns
  )
head(rent_city_wide, 3)
## # A tibble: 3 × 63
##   CITY    COUNTY LISTINGS `2019-01-01` `2019-02-01` `2019-03-01` `2019-04-01`
##   <chr>   <chr>     <dbl>        <dbl>        <dbl>        <dbl>        <dbl>
## 1 Seattle King     585257         2049         2140         2178         2199
## 2 Renton  King     126745         1861         1874         1872         1935
## 3 Tacoma  Pierce   115160         1557         1584         1549         1587
## # ℹ 56 more variables: `2019-05-01` <dbl>, `2019-06-01` <dbl>,
## #   `2019-07-01` <dbl>, `2019-08-01` <dbl>, `2019-09-01` <dbl>,
## #   `2019-10-01` <dbl>, `2019-11-01` <dbl>, `2019-12-01` <dbl>,
## #   `2020-01-01` <dbl>, `2020-02-01` <dbl>, `2020-03-01` <dbl>,
## #   `2020-04-01` <dbl>, `2020-05-01` <dbl>, `2020-06-01` <dbl>,
## #   `2020-07-01` <dbl>, `2020-08-01` <dbl>, `2020-09-01` <dbl>,
## #   `2020-10-01` <dbl>, `2020-11-01` <dbl>, `2020-12-01` <dbl>, …
From Longer to Wider. Source: Cheatsheet of tidyr.

Missing Values

In pivot_longer, there is an optional argument called values_drop_na =, default as FALSE. If setting as TRUE, it will remove rows with missing mean_rent. We set as FALSE but is there any NA (Not Available) values?

We can use is.na() to check and there are 8 missing value in mean_rent column:

colSums(is.na(rent_city_tidy))
##      CITY    COUNTY  LISTINGS     month mean_rent 
##         0         0         0         0         8

We can remove NA or smooth/impute them based on their neighbors. We will just remove them here because those 8 missing values do not impact too much on our visualization. But, there are many ways you can do data interpolation. It can based on known temporal neighbors, spatial neighbors, group-based neighbors. It goes beyond this course but it’s an important problem for data science. If you are interested in missing data, you can refer to this book: Statistical Analysis with Missing Data.

# remove NA then we can see there are 592 rows afterwards, 8 missing values are removed
rent_city_tidy_NA_remove <- rent_city_tidy %>%
  filter(!is.na(mean_rent))
dim(rent_city_tidy_NA_remove)
## [1] 592   5

📚 TODO: Handle Missing Values

Instead of omitting those missing values, let’s try filling missing values in rent_city_tidy using a linear interpolation method. You can use the zoo package, which provides a convenient function na.approx() for handling time-series data.

The documentation and this post may be helpful to you!

# TODO

First Use of ggplot2

Starting from this lab session, we are going to conduct data visualization using R and ggplot2. You don’t have to install ggplot2 separately because it is already a part of tidyverse. ggplot2 is built on the Grammar of Graphics — a system that describes plots as combinations of independent components. This is a typical chunk of ggplot2:

ggplot(data,   aes(x = ..., y = ...)) +
  geom_*() +
  facet_*() +
  scale_*() +
  coord_*() +
  theme()

Let’s use the average rent of Seattle and Tacoma to complete ggplot2 figure step by step to understand how each component contributes. We defined one objects with average rent of both Seattle and Tacoma (rent_sea_tac).

rent_sea_tac <- rent_city_tidy_NA_remove %>%
  filter(CITY %in% c("Seattle", "Tacoma"))

Starting from ggplot(), we create a plot object. We can define the data and and aes (or later). We have argument aes(), which defines how data fields (e.g., categorical, ordinal) map to visual properties/channels (e.g., x, y, color, shape, size).

If we assign this ggplot() object to g, we can later add anything upon this object using +. Otherwise, the plot will show here. ggplot() alone initializes a coordinate system and data mapping, so there is nothing on the diagram.

ggplot(rent_sea_tac, aes(x = month, y = mean_rent, group = CITY)) # will show below the chunk

# what will happen if we do not set group = CITY
g <- ggplot(rent_sea_tac, aes(x = month, y = mean_rent, group = CITY)) # will not show the plots

In ggplot2, defining geom_*() (geometric object) is required. What do we want in this plot? Points or lines? We can stack more than one geom_*() in a plot and each geom_*() function will return a layer.

Within geom_*() functions, we can modify the visual attributes of this layer. Note: different geometric objects have different arguments.

g <- g + geom_line(aes(color = CITY),
              alpha = 0.8, # Opacity: 0 means fully transparent (invisible)
              linetype = 6, # 6 = twodash, you can check more option using `?linetype`
              linewidth = 0.4) # linewidth
g

This is a confusing thing here: we added aes(color = CITY) within geom_line(). Why did we use CITY twice, one in ggplot() one in geom_line()?

ggplot(rent_sea_tac, aes(x = month, y = mean_rent, group = CITY)) defines global mappings: defaults that all layers geom_*() will inherit. This line just say we have month as X and average rent as y, and data are organized into two groups based on CITY. Color is a new aesthetic mapping that we can introduce to this layer! Also, it’s good to know that if we want ggplot2 to both group and color by city, we could directly define (introduce color to the global setting): ggplot(rent_sea_tac, aes(x = month, y = mean_rent, color = CITY)) + geom_line().

Visualizing Time: A Deeper Example

Between 2019 and 2024, one of the most significant global events was the COVID-19 pandemic, which profoundly affected housing and rental markets. In this analysis, we are interested in rent resiliency across different groups after the pandemic — that is, how quickly and to what extent rents in various apartment types and amenity levels recovered following the COVID-19 shock.

First, we filter the rental dataset to apartment buildings only and add four useful variables:

We call it the amenity group rather than quality: amenities_sum counts words in the marketing description, not the condition of the unit. In December 2019, 1- and 2-bedroom listings with 0-1 amenities actually rented for more than those with 2-4.

rental_apt <- rental %>%
  filter(BUILDING_TYPE == "apartment building") %>%
  mutate(
    # converts posting date into the first day of its month for monthly analysis
    month = as.Date(paste0(format(as.Date(DATE_POSTED), "%Y-%m"), "-01")),
    # categorizes apartments by the number of bedrooms
    bed_group = case_when(
      BEDS == 0 ~ "Studio",
      BEDS == 1 ~ "1 Bedroom",
      BEDS == 2 ~ "2 Bedrooms",
      BEDS >= 3 ~ "3+ Bedrooms",
      TRUE ~ "Unknown"),
    # creates a variable counting how many of six amenities each listing offers
    amenities_sum = (
      (CLUBHOUSE == "Y") +
      (GRANITE == "Y") +
      (STAINLESS == "Y") +
      (POOL == "Y") +
      (GYM == "Y") +
      (DOORMAN == "Y")),
    # groups apartments by how many of the six amenities are mentioned
    amenity_group = case_when(
      amenities_sum <= 1 ~ "0-1",
      amenities_sum %in% 2:4 ~ "2-4",
      amenities_sum >= 5 ~ "5-6",
      TRUE ~ "Unknown"
    ),
    # convert two variables from categorical to ordinal
    bed_group = factor(bed_group,
                       levels = c("Studio", "1 Bedroom", "2 Bedrooms", "3+ Bedrooms")),
    amenity_group = factor(amenity_group,
                           levels = c("0-1", "2-4", "5-6"))
  )

Then, let’s calculate median monthly rent broken down by amenity group and bedroom category. We use the median rather than the mean because a few listings above $10,000 pull the mean around (see the histogram and boxplot in More Visualization below). We also count the listings in each group. Notes: It is important to note that decisions about grouping, variable construction, and filtering should ideally be informed by exploratory data analysis and domain knowledge. For simplicity and clarity, these exploratory steps are not shown in this tutorial.

apt_rent_by_month <- rental_apt %>%
  # group by three variables: month, amenity_group, bed_group
  group_by(month, amenity_group, bed_group) %>%
  dplyr::summarise(median_rent = median(RENT_PRICE, na.rm = TRUE),
                   listings = n(), # number of listings in each group
                   .groups = "keep")

Let’s produce the first plot about monthly rent trends for apartment buildings, separated by bedroom type and amenity group. We have four variables: time, median rent, bedroom group, amenity group. We need to determine the aesthetics (channels/encoding variables).

Recall the Expressiveness + Effectiveness we discussed: positions are always the best, so we put two important variables (time, median rent) as x and y.

Amenity group and bedroom group are ordinal categorical variables. We need to think about whether we want to preserve this order. If preserving order is not essential for our analytically purpose, we may treat them simply as nominal categories. In the current design, amenity group (amenity_group) is represented by color, while bedroom type (bed_group) is shown through faceting.

ggplot(apt_rent_by_month, aes(x = month, y = median_rent, color = amenity_group)) +
  geom_line() +
  facet_wrap(~bed_group)+
  labs(
    title = "Monthly Apartment Rent Trend 2019–2023",
    x = "Time",
    y = "Median Rent Price",
    color = "Amenities Advertised (of 6)"
  ) +
  theme_minimal() +
  theme(legend.position = "bottom")

In general, the figure looks quite good after some adjustments if our goal is simply to show overall rent trends. However, the key message I’d like to emphasize is the rent resiliency across different groups after the pandemic—that is, how quickly and strongly rents in various apartment types and amenity levels recovered following the COVID-19 shock. The absolute rent levels are not particularly meaningful on their own, so we can make an adjustment by converting them into relative rents, measured as rent changes compared to December 2019 levels (pre-pandemic).

The lines are also jumpy. Some groups have only 30-50 listings a month (3+ bedrooms with 5-6 amenities, for example), so one unusual month can move a line a lot. We smooth each line with a 3-month moving average (the previous, current and next month) before comparing it with December 2019.

apt_rent_rel <- apt_rent_by_month %>%
  # compute relative change within each group
  group_by(amenity_group, bed_group) %>%
  # put months in order, lag() and lead() use the row order
  arrange(month, .by_group = TRUE) %>%
  mutate(
    # 3-month moving average: previous, this and next month
    rent_smooth = (lag(median_rent) + median_rent + lead(median_rent)) / 3,
    # smoothed rent in Dec 2019 as baseline
    base_rent = rent_smooth[month == as.Date("2019-12-01")],
    # rent change (e.g., +10%)
    rent_change = (rent_smooth - base_rent)/base_rent
  ) %>%
  ungroup()

We are going to create a relatively complicated plot and you can have a look of the final result at the end To achieve that, we need many functions in ggplot2 and other software, such PowerPoint and PhotoShop, to do final edits (like the notes inside).

Basic Structure

Similar as previous plot. We started from the data: apt_rent_rel. We can do filter here to make sure the plot will only show the time period we want: between December 2019 (just before the first Covid-19 case in Washington State) and December 2022.

The we need to determine the aesthetics (channels/encoding variables), we map time to the x-axis, rent changes to the y-axis. There are two additional dimensions in the data: amenity group (amenity_group), which is represented by color, and bedroom type (bed_group), which is shown through faceting.

After that, we are going to add the first geometry: geom_line. Its location is decided by the global aes we already defined. We also add the labels, including title, subtitle, caption (data source and definitions), x and y axis title, and the label for color group.

g <- ggplot(
  filter(apt_rent_rel, month >= as.Date("2019-12-01") & month <= as.Date("2022-12-01")),
  aes(x = month, y = rent_change, color = amenity_group)
) +
  geom_line(linewidth = 1.2) +
  facet_wrap(~bed_group, nrow = 1) +
  labs(
    title = "Washington State Monthly Apartment Rent Changes Since Covid-19",
    subtitle = "Change in median asking rent compared to December 2019, by number of bedrooms and amenities advertised in the listing",
    caption = "Data: RentHub listings, December 2019 to December 2022. Lines are 3-month moving averages of the monthly median rent.\nAmenities: clubhouse, granite, stainless, pool, gym, doorman mentioned in the listing text.",
    x = "Time",
    y = "Relative Rent Change",
    color = "Amenities Advertised (of 6)"
  )
g

Graphical Primitives

We would like emphasize the line \(y=0\) because it means no change compared to the beginning of the pandemic. Meanwhile, we need to let readers know the time period of the pandemic. So, we use geom_hline() to add a horizontal line as another layer. (Remember to check Cheatsheet, R documentation, and using ?function when you are not familiar with a function.)

There is another way we can simply draw a line is adding an annotation layer. But unlike a typical geom function, the properties of the geoms are not mapped from variables of a data frame, but are instead passed in as vectors. This is useful for adding small annotations (such as text labels or a line). We add a rectangular annotation to highlight a time period, from the first Covid-19 case in Washington State (January 2020) to the statewide reopening (June 30, 2021). -Inf and Inf stretch it to the edges of each panel, and alpha makes it transparent.

annotate() repeats the same text in every panel. To label only one panel, we give geom_text() a one-row data frame whose bed_group is Studio, so the text only shows in the Studio panel.

# a one-row data frame for the label: bed_group decides the panel
band_label <- tibble(
  bed_group = factor("Studio", levels = levels(apt_rent_rel$bed_group)),
  month = as.Date("2020-01-20"),
  rent_change = 0.57,
  label = "Pandemic period\nfirst case to statewide reopening"
)

g_line_rec <- g +
  # add a horizontal line across the plot at y = 0
  geom_hline(yintercept = 0, linewidth = 0.6) +
  annotate("rect", # a rectangular
           # we need 4 points to determine the location
           xmin = as.Date("2020-01-01"), xmax = as.Date("2021-07-01"),
           ymin = -Inf, ymax = Inf, # from the bottom to the top of each panel
           fill = "grey50", alpha = 0.12) + # explain below
  geom_text(data = band_label, # the label only shows in the Studio panel
            aes(label = label),
            color = "black",
            size = 3.6,
            hjust = 0, vjust = 1, # the top-left corner of the text sits at (x, y)
            fontface = "italic") # explain below
g_line_rec

Fontface

fontface/face is a catch-all argument that describes the style of the typeface to use. “plain” is an upright normal-weight font, “italic” is a slanted normal-weight font, “bold” is an upright bold-weight font, and “bold.italic” is a slanted bold-weight font.

Colors

We can use R built-in colors (600+), you can check their names using colors().

We can also use Hex colors and palette functions. Hex colors are written as #RRGGBB, where each pair (RR, GG, BB) represents the intensity of Red, Green, and Blue in hexadecimal (from 00 to FF, i.e., 0–255). ggplot2 provides built-in palette functions that automatically generate harmonious color sets for categorical or continuous variables.

R also supports numeric color systems like RGB, HCL, and HSV.

# colors()
# scale_color_brewer(palette = "Set2")
# scale_color_viridis_d(option = "plasma")

Scale

Scale controls how data values are mapped to aesthetics (channels/encoding variables). For example, we use color = amenity_group and by default ggplot2 will choose some colors for that. But we can change how color encodes through scale_color_*.

Aesthetic Scale function Controls
x-axis scale_x_*() axis range, tick marks, labels
y-axis scale_y_*() axis range, tick marks, labels
color scale_color_*() how color encodes categories or values
fill scale_fill_*() fill colors for bars, areas, etc.
size scale_size_*() point or line thickness
shape scale_shape_*() point shapes
alpha scale_alpha_*() transparency

Viridis is the most colorblind-friendly, perceptually-uniform color maps used in data visualization. Read more about viridis palettes. You can use those common commands for color or fill:

The Viridis Palettes
g_scale <- g_line_rec +
  # change how assigns colors to the variable amenity_group
  scale_color_manual(values = c("0-1" = "#ACA3CB", "2-4" = "#685A8E", "5-6" = "#32006E")) + 
  # how to use an viridis palette here? 
  scale_x_date(
    breaks = as.Date(c("2020-01-01", "2021-01-01", "2022-01-01")),
    labels = c("2020", "2021", "2022")
  ) +
  # formats the numeric values into percentages with a plus or minus sign
  scale_y_continuous(
    breaks = seq(-0.2, 0.6, by = 0.2),
    labels = function(x) sprintf("%+.0f%%", x * 100)
  )
g_scale

Coordinate System

The coordinate system determines how data coordinates are translated to positions on the plot. We will talk about this more in the future class on mapping and geospatial data.

g_coordinate <- g_scale +
  # limits the y-axis to between −0.2 and 0.6, without extra space around the data
  coord_cartesian(ylim = c(-0.2, 0.6), expand = FALSE)
g_coordinate

Theme

Theme controls all non-data visual components, text, background, grid lines, margins, etc.

Components

First, we need to know how ggplot2 names those components:

Theme Elements Reference. Source: Reproducible Science for Busy Researchers by Dr. Andrew P. Lapointe.

You can see there is a hierarchy among those plot components. We list some here as the example, you can read all components of a theme from its official document.

Elements

There are several types of components, ggplot2 calls them elements, such as text, line, and rectangular. For example, the line of axis is element_line while the title of axis is element_text. We remove some components through assigning them as element_blank().

We change the attributes of those components (such as legend.title, axis.line) by assigning them an appropriate element_*() function inside theme(). Please refer to its official document or ?element_*() for detailed arguments - Theme elements.

Complete Themes

Beyond those specific theme configuration, we can start with existing complete themes by + theme_*(). The one I like most is theme_minimal() because it is super clean. Within theme_*(), we can set a few argument for the whole plot: base_family for default font, base_size for default font size, for example.

Theme Inheritance

It is useful to know theme inheritance feature of ggplot2: theme elements inherit properties from other theme elements hierarchically. For example, there is the hierarchy for text elements:

text
 ├── plot.title
 ├── plot.subtitle
 ├── axis.title
      ├── axis.title.x / axis.title.y
 └── legend.text / legend.title

Modify our Plot using Theme

g_theme <- g_coordinate +
  theme_minimal(base_family = "Helvetica", base_size = 13) +
  theme(
    # ---------------- for the whole plot ----------------
    text = element_text(lineheight = 1.1), # set line height for all text
    plot.title = element_text(face = "bold", size = 18), # make title bold and large
    plot.subtitle = element_text(margin = margin(b = 10)), # space below the subtitle
    plot.caption = element_text(hjust = 0, color = "grey30", margin = margin(t = 12)), # left-aligned caption
    plot.title.position = "plot", # align title and caption with the whole plot, not the panels
    plot.caption.position = "plot",
    plot.margin = margin(4, 12, 4, 4), # external margins (top, right, bottom, left)
    # ---------------- for x and y axis ----------------
    axis.title.x = element_blank(), # remove axis.title.x
    axis.title.y = element_blank(),
    axis.ticks.x = element_line(linewidth = 0.4), # style tick marks
    # ---------------- for grid and panel ----------------
    panel.grid.major.x = element_blank(), # remove vertical grid lines
    panel.grid.minor = element_blank(), # remove minor grid lines
    panel.grid.major.y = element_line(color = "grey85", linewidth = 0.3), # keep light horizontal lines to read values
    panel.spacing.x = unit(2, "lines"), # spacing between facets horizontally
    # ---------------- for the legend ----------------
    legend.position = "top", # place legend above the plot
    legend.justification = "left",
    legend.title = element_text(face = "bold"),
    # ---------------- for facet ----------------
    strip.background = element_blank(),
    strip.text.x = element_text(face = "bold", # facet labels
                                size = 14,
                                hjust = 0, # left-aligned
                                margin = margin(b = 8)) # bottom margin (t/r/b/l)
  )
g_theme

The Complete PLot

ggplot(
  filter(apt_rent_rel, month >= as.Date("2019-12-01") & month <= as.Date("2022-12-01")),
  aes(x = month, y = rent_change, color = amenity_group)
) +
  # 1️⃣ geometry
  annotate("rect",
           xmin = as.Date("2020-01-01"), xmax = as.Date("2021-07-01"),
           ymin = -Inf, ymax = Inf,
           fill = "grey50", alpha = 0.12) +
  geom_hline(yintercept = 0, linewidth = 0.6) +
  geom_line(linewidth = 1.2) +
  geom_text(data = band_label, aes(label = label),
            color = "black", size = 3.6, hjust = 0, vjust = 1, fontface = "italic") +

  # 2️⃣ scale
  scale_color_manual(values = c("0-1" = "#ACA3CB", "2-4" = "#685A8E", "5-6" = "#32006E")) +
  scale_x_date(
    breaks = as.Date(c("2020-01-01", "2021-01-01", "2022-01-01")),
    labels = c("2020", "2021", "2022")
  ) +
  scale_y_continuous(
    labels = function(x) sprintf("%+.0f%%", x * 100),
    breaks = seq(-0.2, 0.6, by = 0.2)
  ) +

  # 3️⃣ facet and coordinate system
  facet_wrap(~bed_group, nrow = 1) +
  coord_cartesian(clip = "off", ylim = c(-0.2, 0.6), expand = FALSE) +

  # 4️⃣ labels
  labs(
    title = "Washington State Monthly Apartment Rent Changes Since Covid-19",
    subtitle = "Change in median asking rent compared to December 2019, by number of bedrooms and amenities advertised in the listing",
    caption = "Data: RentHub listings, December 2019 to December 2022. Lines are 3-month moving averages of the monthly median rent.\nAmenities: clubhouse, granite, stainless, pool, gym, doorman mentioned in the listing text.",
    x = "Time",
    y = "Relative Rent Change",
    color = "Amenities Advertised (of 6)"
  ) +

  # 5️⃣ theme
  theme_minimal(base_family = "Helvetica", base_size = 13) +
  theme(
    text = element_text(lineheight = 1.1),
    plot.title = element_text(face = "bold", size = 18),
    plot.subtitle = element_text(margin = margin(b = 10)),
    plot.caption = element_text(hjust = 0, color = "grey30", margin = margin(t = 12)),
    plot.title.position = "plot",
    plot.caption.position = "plot",
    axis.title.x = element_blank(),
    axis.ticks.x = element_line(linewidth = 0.4),
    axis.title.y = element_blank(),
    legend.position = "top",
    legend.justification = "left",
    legend.title = element_text(face = "bold"),
    panel.grid.major.x = element_blank(),
    panel.grid.minor = element_blank(),
    panel.grid.major.y = element_line(color = "grey85", linewidth = 0.3),
    strip.background = element_blank(),
    strip.text.x = element_text(face = "bold", size = 14, hjust = 0, margin = margin(b = 8)),
    panel.spacing.x = unit(2, "lines"),
    plot.margin = margin(4, 12, 4, 4)
  )

# save it as 14 inch * 7 inch
ggsave("rent_change_plot.png", width = 14, height = 7, dpi = 200)
# save it as 7 inch * 3.5 inch
ggsave("rent_change_plot_735.png", width = 7, height = 3.5, dpi = 200)

When working with ggplot2, customizing the theme() controls visual style — but the final output size and aspect ratio are just as important. RStudio preview IS NOT the actual exported figure. I will recommend that you work within a single chunk and check changes in exported figure all the time. I decided to use 14 * 7 inch at the beginning so it is perfect when I export as 14 * 7 inch. But the following figure was exported as 7 * 3.5 inch: everything squeezed together.

The Exported Figure

Final Edits

Not every figure — or every visual component within a figure — is best created entirely in R.

While ggplot2 is powerful for data-driven visualization, it’s not always flexible for fine-grained graphic design tasks. We can always combine R/ggplot2 with other tools, like PowerPoint and Photoshop. Here, we added an explanation on pandemic and a directional line using Photoshop.

Washington State Monthly Apartment Rent Changes Since Covid-19

More Visualization in R

Histograms

A histogram summarizes the distribution of the data by placing observations into intervals (also called classes or bins) and counting the number of observations in each interval.

ggplot(data = rental, aes(x = RENT_PRICE)) +
  geom_histogram(
    binwidth = 200,   # width of each bin, in dollars
    color = "white",  # border color
    alpha = 0.7       # transparency
  ) +
  scale_x_continuous(labels = scales::dollar) +
  scale_y_continuous(labels = scales::comma) +
  coord_cartesian(xlim = c(0, 8000)) + # zoom in; listings above $8,000 are still counted
  labs(title = "Histogram of Rent Price", x = "Rent Price", y = "Count of Listings") +
  theme_minimal()

Boxplots

A box and whisker plot (also referred to as boxplot) provides a compact summary of the distribution of a variable.

ggplot(
  data = rental %>%
    filter(CITY %in% rent_city$CITY),  # the 10 cities in `rent_city`
  aes(x = RENT_PRICE, y = reorder(CITY, RENT_PRICE, FUN = median)) # horizontal boxes, sorted by median rent
) +
  geom_boxplot(
    outlier.size = 0.2,
    fill = "black",
    alpha = 0.3,    # transparency
    width = 0.5     # box width
  ) +
  labs(title = "Boxplot of Rent Price by City", x = "Rent Price", y = NULL) +
  scale_x_continuous(labels = scales::dollar) +
  coord_cartesian(xlim = c(0, 8000)) + # zoom in; the boxes are still computed from all listings
  theme_minimal()

Cumulative Distribution Functions (CDF)

The cumulative distribution function CDF is a function \(F(x)\) that is the probability that the observations of a variable are not larger than a specified value. In constructing the CDF, weights (e.g., inclusion probabilities from a probability design) can be used. In this way the probability that a value of the variable in the statistical population is less than a specified value is estimated. Otherwise, for equal weighting of observations, the CDF applies only to the observed values.

\[F(x) = P(X \leq x) = \text{Probability of Random Variable}\space X \space \text{is not greater than} \space x\]

ggplot(data = rental %>%
         filter(CITY %in% c("Seattle", "Tacoma", "Spokane"))) +
  stat_ecdf(aes(x = RENT_PRICE),) + # stat builds new variables to plot, we build CDF
  facet_wrap(~CITY)+ # it is not the best option to use facet
  scale_x_continuous(labels = scales::dollar, breaks = c(0, 2000, 4000)) +
  coord_cartesian(xlim = c(0, 6000)) + # zoom in on where the curves rise
  labs(
    title = "CDF of Rent Price in Seattle, Spokane and Tacoma",
    subtitle = "F(x) = P(X ≤ x)",
    y = "Cumulative Distribution Function (CDF)",
    x = "Rent Price"
  ) +
  theme_minimal()

Scatter Plot

Scatterplots are graphical displays of matched data plotted with one variable on the horizontal axis and the other variable on the vertical axis. Data are usually plotted with measures of an influential parameter on the horizontal axis (independent variable) and measures of an attribute that may respond to the influential parameter on the vertical axis (dependent variable). Scatterplots are a useful first step in any analysis because they help visualize relationships and identify possible issues (e.g., outliers) that can influence subsequent statistical analyses.

set.seed(520) # set seed, then the random sample below stays the same
ggplot(
  data = rental %>%
    filter(!is.na(SQFT)) %>%
    sample_n(2000), # a random sample of 2,000 listings
  aes(x = SQFT, y = RENT_PRICE)
) +
  geom_point(size = 0.5, alpha = 0.6) +
  geom_smooth( # we use a sample linear regression to model the relationship (trends)
    method = "lm",
    formula = y ~ x,
    se = TRUE, # show confidence interval
    color = "#4B2E83"
  ) +
  scale_x_continuous(labels = scales::comma) +
  scale_y_continuous(labels = scales::dollar) +
  labs(title = "Scatter Plot of Square Footage and Rent Price", x = "Square Footage", y = "Rent Price") +
  theme_minimal()

2D Histogram

With 2 million listings, the points of a scatter plot cover each other, which is why we took a sample of 2,000 above. A 2D histogram (geom_bin_2d()) uses all of them: like a histogram, it cuts the plane into small rectangles, counts the listings in each, and colors each rectangle by the count. The color uses a log scale, because the densest rectangles hold over 10,000 listings and most hold far fewer.

ggplot(
  data = rental %>%
    filter(!is.na(SQFT)), # all listings with square footage, no sampling
  aes(x = SQFT, y = RENT_PRICE)
) +
  geom_bin_2d(binwidth = c(50, 100)) + # each rectangle is 50 sq ft wide and $100 tall
  scale_fill_viridis_c(option = "mako", direction = -1,
                       transform = "log10", # log scale: 1, 10, 100, 1,000, ...
                       labels = scales::comma) +
  scale_x_continuous(labels = scales::comma) +
  scale_y_continuous(labels = scales::dollar) +
  coord_cartesian(xlim = c(0, 3000), ylim = c(0, 6000)) + # zoom in; all listings are still counted
  labs(title = "2D Histogram of Square Footage and Rent Price",
       x = "Square Footage", y = "Rent Price", fill = "Listings") +
  theme_minimal()

Bar Chart

A bar chart shows the count of each category; sorting the bars makes the ranking easy to read. Here it also finds a data problem: the same building type is written in different ways (apartment building, Apartment, apartment), and many listings are unknown or missing.

ggplot(
  data = rental %>%
    mutate(BUILDING_TYPE = replace_na(BUILDING_TYPE, "(missing)")), # name the missing values, so they are sorted too
  aes(y = fct_rev(fct_infreq(BUILDING_TYPE))) # sort the bars by count, largest on top
) +
  geom_bar(alpha = 0.7) +
  scale_x_continuous(labels = scales::comma) +
  labs(title = "Listings by Building Type", x = "Count of Listings", y = NULL) +
  theme_minimal()

Heatmap

A heatmap is a graphical representation of data where the individual values contained in a matrix are represented as colors. It’s like looking at a data table from above, where each cell’s color intensity indicates the magnitude of the value it represents. Heatmaps are especially useful for identifying patterns, trends, and outliers in large datasets.

Here each cell is the median rent of one city and one bedroom group: reading across a row shows how rent rises with size, and reading down a column compares cities. Seattle studios ($1,530) rent for less than studios in Renton or Lynnwood, but Seattle 2-bedrooms are the second most expensive.

rent_city_bed <- rental %>%
  filter(CITY %in% rent_city$CITY) %>% # the 10 cities in `rent_city`
  mutate(
    bed_group = case_when(
      BEDS == 0 ~ "Studio",
      BEDS == 1 ~ "1 Bedroom",
      BEDS == 2 ~ "2 Bedrooms",
      BEDS >= 3 ~ "3+ Bedrooms"),
    bed_group = factor(bed_group, levels = c("Studio", "1 Bedroom", "2 Bedrooms", "3+ Bedrooms"))
  ) %>%
  group_by(CITY, bed_group) %>%
  summarise(median_rent = median(RENT_PRICE), .groups = "drop")

ggplot(rent_city_bed, aes(x = bed_group, y = reorder(CITY, median_rent), fill = median_rent)) + # sort cities by rent
  geom_tile(color = "white", linewidth = 0.25) + # white lines between the cells
  scale_fill_viridis_c(option = "mako", direction = -1, labels = scales::dollar) +
  labs(title = "Median Rent by City and Number of Bedrooms", x = NULL, y = NULL, fill = "Median Rent") +
  theme_minimal() +
  theme(panel.grid = element_blank(),
        legend.key.width = unit(2, "cm"), # a wider color bar
        legend.position = "bottom") # 

Stacked Area Chart

We combine the labels into five groups with case_when() and use them in the next chart. A stacked area chart shows how a total and its parts change over time.

rental <- rental %>%
  mutate(
    building = case_when(
      str_to_lower(BUILDING_TYPE) %in% c("apartment building", "apartment") ~ "Apartment",
      str_to_lower(BUILDING_TYPE) %in% c("house", "single family house") ~ "House",
      str_to_lower(BUILDING_TYPE) == "condo" ~ "Condo",
      str_to_lower(BUILDING_TYPE) %in% c("townhouse", "duplex") ~ "Townhouse/Duplex",
      TRUE ~ "Other/Unknown" # unknown, missing, Other, RNT, and Multi-family (see the next chart)
    ),
    building = factor(building,
                      levels = c("Apartment", "House", "Condo", "Townhouse/Duplex", "Other/Unknown"))
  )
ggplot(
  data = rental %>%
    count(month = floor_date(DATE_POSTED, "month"), building) %>% # listings per month and group
    complete(month, building, fill = list(n = 0)), # a group with no listings in a month gets 0, not a gap
  aes(x = month, y = n, fill = building)
) +
  geom_area(position = position_stack(reverse = TRUE)) + # the first group (Apartment) at the bottom
  scale_fill_viridis_d(option = "mako", begin = 0.15, end = 0.85) + # skip the near-black and near-white ends
  scale_x_date(date_breaks = "1 year", date_labels = "%Y") +
  scale_y_continuous(labels = scales::comma) +
  labs(title = "Listings per Month by Building Type", x = NULL, y = "Count of Listings", fill = NULL) +
  theme_minimal() +
  theme(legend.position = "bottom")

Dumbbell Chart

A dumbbell chart connects two values of the same category, so the length of the line is the message. Among 1-bedroom listings in apartment buildings, those that mention a doorman ask $660 more (median) than those that do not, and those that mention a pool ask $301 less. That does not mean a pool lowers rent: only 9% of these listings in Seattle mention a pool, compared with 45% elsewhere, and Seattle rents are higher. A simple comparison like this mixes the amenity with location.

amenity_gap <- rental %>%
  filter(BUILDING_TYPE == "apartment building", BEDS == 1) %>%
  pivot_longer(GRANITE:GARAGE, # the nine amenity columns
               names_to = "amenity", values_to = "mentioned") %>%
  group_by(amenity) %>%
  summarise(without = median(RENT_PRICE[mentioned == "N"]),
            with = median(RENT_PRICE[mentioned == "Y"])) %>%
  mutate(amenity = fct_reorder(str_to_title(amenity), with - without)) # sort by the gap

ggplot(amenity_gap, aes(y = amenity)) +
  geom_segment(aes(x = without, xend = with, yend = amenity), color = "grey75", linewidth = 1) +
  geom_point(aes(x = without), color = "coral4", size = 3) +
  geom_point(aes(x = with), color = "navyblue", size = 3) + # you can use `colors()` to check built-in color names
  scale_x_continuous(labels = scales::dollar) +
  labs(title = "1-Bedroom Apartment Rent With and Without Each Amenity",
       subtitle = "Median rent. Blue: amenity mentioned in the listing; red: not mentioned",
       x = NULL, y = NULL) +
  theme_minimal()