Lab 08 Distributions and Confidence Intervals in R

URBAN 520 Quantitative Methods in Urban Design and Planning


In Labs 8 and 9, we will move from describing data to making statements about a population from a sample. The topic is household energy cost: how much King County households pay for electricity, gas, and other fuels, and how heavy that cost is for them. We can simply define the energy burden of a household is the share of its income spent on home energy. Researchers commonly call a burden above 6% high and above 10% severe (ACEEE, 2020).


library('tidyverse')
library('tidycensus') # we used it in Lab 3 for getting ACS tables via API

Data: ACS Public Use Microdata

In Lab 3, we downloaded ACS tables: estimates that the Census Bureau has already computed for each geography, such as the median household income of a tract. The Census Bureau also publishes the Public Use Microdata Sample (PUMS): the anonymized, sample-level records from the American Community Survey (ACS), one row per household or person, with names and addresses removed. The smallest geography in PUMS is the Public Use Microdata Area (PUMA), an area with at least 100,000 people. King County has 18 PUMAs, and 7 of them are in Seattle. You can check the King County PUMA Map here.

We prepared kc_energy_2024.csv from the 2020-2024 ACS 5-year PUMS: every occupied housing unit in King County that answered the survey, 42,178 households. All dollar amounts are adjusted to 2024 dollars. Below is the code we used to download and clean it. The download takes about a few minutes, so the two chunks below are not run when you knit this file (eval=FALSE). And we will just read the data from saved CSV file.

Downloading PUMS

get_pums() from tidycensus works like get_acs() in Lab 3, but instead of table IDs, we name PUMS variables. You can look up every variable and its codes in pums_variables, a data frame that comes with tidycensus, or in the PUMS documentation. Some variables we will use:

Variable Meaning
TEN Tenure: 1 owned with a mortgage, 2 owned free and clear, 3 rented, 4 occupied without paying rent
BLD Units in the structure
YRBLT When the structure was first built (1939 means 1939 or earlier)
HFL House heating fuel
ELEP, GASP Last month’s electricity cost and gas cost
FULP Cost of other fuels (oil, wood, propane, …) in the past 12 months
ELEFP, GASFP, FULFP Flags: is the cost valid, included in rent, or not used?
HINCP Household income in the past 12 months
ADJHSG, ADJINC Factors that adjust housing costs and income to 2024 dollars
NP, BDSP Number of persons and number of bedrooms
TYPEHUGQ Housing unit or group quarters (dorms, nursing homes, …)
# View(pums_variables) # look up variable names and their codes
kc_pums <- get_pums(
  variables = c("TEN", "BLD", "YRBLT", "HFL", "ELEP", "ELEFP", "GASP", "GASFP",
                "FULP", "FULFP", "HINCP", "ADJHSG", "ADJINC", "NP", "BDSP", "TYPEHUGQ"),
  state = "WA",
  puma = as.character(23301:23318), # the 18 King County PUMAs
  survey = "acs5", # 5-year PUMS
  year = 2024 # 2020-2024
)

Cleaning PUMS

get_pums() returns one row per person (106,190 rows), and the household variables repeat on the row of every person in the household. It also adds the IDs and weights (SERIALNO, SPORDER, WGTP, PWGTP). We keep one row per occupied housing unit: the first person in the household (SPORDER == 1), living in a housing unit rather than group quarters (TYPEHUGQ == "1"), in a unit that is occupied (TEN from 1 to 4). Three more details:

  1. 2024 dollars. The five survey years are in different dollars. We multiply housing costs by ADJHSG and income by ADJINC. tidycensus returns ADJHSG with six implied decimals (1214548 means 1.214548), so we divide it by 1,000,000; ADJINC already has its decimal point.
  2. Flags. A cost only counts when its flag says it is valid. Electricity: ELEFP 3 is a valid cost; 1 (included in rent or condo fee) and 2 (no charge or not used) give NA. Gas: GASFP 4 is valid; 2 (included in the electricity bill) and 3 (not used) give 0; 1 (included in rent) gives NA. Other fuels: FULFP 3 is valid; 2 (not used) gives 0; 1 (included in rent) gives NA.
  3. Energy burden. energy_burden = 12 Γ— energy_month / income, and NA when the income is 0 or less.
# short names for the codes
area_names <- c(
  "23301" = "Shoreline, Kenmore & Bothell (South)",
  "23302" = "Snoqualmie, Cottage Lake & Union Hill",
  "23303" = "Redmond & Kirkland",
  "23304" = "Bellevue",
  "23305" = "Sammamish, Issaquah & Mercer Island",
  "23306" = "Maple Valley, Covington & Enumclaw",
  "23307" = "Auburn",
  "23308" = "Federal Way, Des Moines & Vashon",
  "23309" = "Kent",
  "23310" = "Renton, Fairwood & Skyway",
  "23311" = "Burien, SeaTac, Tukwila & White Center",
  "23312" = "Seattle: West Seattle & Industrial",
  "23313" = "Seattle: Southeast",
  "23314" = "Seattle: Central",
  "23315" = "Seattle: Lake Union & Downtown",
  "23316" = "Seattle: Northwest",
  "23317" = "Seattle: Northeast",
  "23318" = "Seattle: North"
)
fuel_names <- c("1" = "Utility gas", "2" = "Bottled, tank, or LP gas", "3" = "Electricity",
                "4" = "Fuel oil or kerosene", "5" = "Coal or coke", "6" = "Wood",
                "7" = "Solar energy", "8" = "Other fuel", "9" = "No fuel used")
bld_names <- c("1" = "Mobile home", "2" = "Single-family detached",
               "3" = "Single-family attached", "4" = "2-4 units", "5" = "2-4 units",
               "6" = "5-19 units", "7" = "5-19 units", "8" = "20 or more units",
               "9" = "20 or more units", "10" = "Boat, RV, or van")

energy <- kc_pums %>%
  filter(SPORDER == 1, TYPEHUGQ == "1", TEN %in% c("1", "2", "3", "4")) %>% # one row per occupied housing unit
  mutate(
    across(c(ELEP, GASP, FULP, HINCP, ADJHSG, ADJINC, NP, BDSP, YRBLT), as.numeric), # text to numbers
    adjhsg = ADJHSG / 1e6, # six implied decimals
    elec_month = if_else(ELEFP == "3", ELEP * adjhsg, NA),
    gas_month = case_when(GASFP == "4" ~ GASP * adjhsg,
                          GASFP %in% c("2", "3") ~ 0), # all other cases become NA
    other_fuel_year = case_when(FULFP == "3" ~ FULP * adjhsg,
                                FULFP == "2" ~ 0),
    energy_month = elec_month + gas_month + other_fuel_year / 12, # NA if any part is NA
    income = HINCP * ADJINC,
    energy_burden = if_else(income > 0, 12 * energy_month / income, NA)
  ) %>%
  transmute( # like mutate(), but keeps only the columns created here
    household_id = SERIALNO,
    puma = PUMA,
    area = area_names[PUMA],
    seattle = PUMA %in% as.character(23312:23318),
    tenure = case_when(TEN %in% c("1", "2") ~ "Owner",
                       TEN == "3" ~ "Renter",
                       TEN == "4" ~ "No rent paid"),
    building = bld_names[BLD],
    year_built = case_when(YRBLT == 1939 ~ "1939 or earlier",
                           YRBLT < 2020 ~ paste0(YRBLT, "-", YRBLT + 9),
                           TRUE ~ as.character(YRBLT)), # single years from 2020
    era = case_when(YRBLT <= 1939 ~ "Before 1940",
                    YRBLT <= 1979 ~ "1940-1979",
                    YRBLT <= 1999 ~ "1980-1999",
                    TRUE ~ "2000 or later"),
    heat_fuel = fuel_names[HFL],
    persons = NP,
    bedrooms = BDSP,
    income = round(income),
    elec_month = round(elec_month, 2),
    gas_month = round(gas_month, 2),
    other_fuel_year = round(other_fuel_year, 2),
    energy_month = round(energy_month, 2),
    energy_burden = round(energy_burden, 5),
    weight = WGTP
  )

write_csv(energy, "kc_energy_2024.csv")

Reading the Data

energy <- read_csv("kc_energy_2024.csv")
## Rows: 42178 Columns: 18
## ── Column specification ────────────────────────────────────────────────────────
## Delimiter: ","
## chr  (7): household_id, area, tenure, building, year_built, era, heat_fuel
## dbl (10): puma, persons, bedrooms, income, elec_month, gas_month, other_fuel...
## lgl  (1): seattle
## 
## β„Ή Use `spec()` to retrieve the full column specification for this data.
## β„Ή Specify the column types or set `show_col_types = FALSE` to quiet this message.
Click to Read Data Dictionary

A note on weights. The ACS is not a simple random sample: some addresses are more likely to be selected than others, and the weight column corrects for that. To keep the analysis simple, we will ignore the weights and treat the 42,178 households as a simple random sample of King County households, as in Assignment 2. Our numbers will therefore differ a little from published ACS tables, and we will see by how much at the end of this lab.

Comparing with the Normal Distribution

Let’s start with the monthly electricity cost, elec_month. 1,737 households have no electricity cost of their own (it is included in their rent or not paid), so we drop them.

elec <- energy %>%
  filter(!is.na(elec_month)) %>%
  pull(elec_month) # pull() returns the column as a vector
elec_mean <- mean(elec)
elec_sd <- sd(elec)
summary(elec)
##    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
##    4.00   85.02  139.34  181.04  220.13 2882.26

Let’s see whether a normal distribution with the same mean and standard deviation describe these costs? Draw the histogram on the density scale and stack with the normal curve.

ggplot(data.frame(elec), aes(x = elec)) +
  geom_histogram(aes(y = after_stat(density)), # density instead of count, to match the curve
                 binwidth = 20, fill = "grey70", color = "white") +
  stat_function(fun = dnorm, args = list(mean = elec_mean, sd = elec_sd),
                xlim = c(-300, 1000), # draw the curve below $0 too
                linewidth = 1) + # the normal curve with the same mean and SD
  scale_x_continuous(labels = scales::dollar) +
  coord_cartesian(xlim = c(-300, 1000)) + # zoom in; the costs above $1,000 are still counted
  labs(title = "Monthly Electricity Cost of King County Households",
       x = "Monthly Electricity Cost", y = "Density") +
  theme_minimal()

The d/p/q/r Functions

dnorm() above is one of four functions R has for every distribution. The first letter says what you get, and the rest names the distribution (norm, binom, pois, unif, t, …):

Function Returns Example
d...() Density (height of the curve), or probability of exactly one value for a discrete distribution dbinom(5, size = 12, prob = 0.4)
p...() Probability of a value at or below q (area to the left) pnorm(50, mean = 181, sd = 209)
q...() The value with probability p below it (a percentile) qnorm(0.95, mean = 181, sd = 209)
r...() Random draws from the distribution rnorm(10, mean = 181, sd = 209)

pnorm() tells us what share of households would pay less than a value if the costs were normal. Compare it with the share that actually does:

pnorm(50, mean = elec_mean, sd = elec_sd) # if normal: share below $50
## [1] 0.2653752
mean(elec < 50) # data: share below $50 (TRUE counts as 1, FALSE as 0)
## [1] 0.08691674
pnorm(0, mean = elec_mean, sd = elec_sd) # if normal: share with a negative bill!
## [1] 0.1932335

The normal distribution says 26.5% of households pay less than $50 a month; in the data it is 8.7%. It also says 19.3% pay less than nothing.

Q-Q Plot

A quantile-quantile (Q-Q) plot compares the data with the normal distribution directly. It sorts the data and plots each value against the value we would expect at the same position in a normal sample. If the data is perfectly normally distributed, it should follow the dash line.

ggplot(data.frame(elec), aes(sample = elec)) +
  geom_qq(size = 0.5, alpha = 0.3) + # the data against the normal quantiles
  geom_qq_line(color = "#32006E") + # the line the points would follow if the data were normal
  scale_y_continuous(labels = scales::dollar) +
  labs(title = "Q-Q Plot of Monthly Electricity Cost",
       x = "Normal Quantiles", y = "Monthly Electricity Cost") +
  theme_minimal()

Log Transformation

For right-skewed data like costs and incomes, the logarithm often helps. It pulls in the long right tail and spreads out the small values. And when a variable is close to normal on the log scale, we say it is roughly log-normal.

ggplot(data.frame(elec), aes(sample = log(elec))) +
  geom_qq(size = 0.5, alpha = 0.3) +
  geom_qq_line(color = "#32006E") +
  labs(title = "Q-Q Plot of Log Monthly Electricity Cost",
       x = "Normal Quantiles", y = "Log Monthly Electricity Cost") +
  theme_minimal()

Confidence Intervals

Mean

What is the average monthly electricity cost of all King County households, and how precisely do our 40,441 households tell us? A 95% confidence interval for a mean is

\[\bar{x} \pm t^* \times \frac{s}{\sqrt{n}}\]

where \(t^*\) comes from the t distribution with \(n-1\) degrees of freedom. Let’s compute it by hand:

n <- length(elec)
se <- elec_sd / sqrt(n) # standard error of the mean
t_star <- qt(0.975, df = n - 1) # 2.5% in each tail
c(elec_mean - t_star * se, elec_mean + t_star * se)
## [1] 179.0055 183.0805

t.test() does the same in one line:

t.test(elec)$conf.int
## [1] 179.0055 183.0805
## attr(,"conf.level")
## [1] 0.95

We are 95% confident that the mean monthly electricity cost of King County households is between $179.01 and $183.08. The interval is narrow because \(n\) is large.

Comparing with a Benchmark

The U.S. Energy Information Administration (EIA) reports that the average residential electricity bill in Washington State was $113.68 a month in 2024 (EIA, Table 5A). We may check our interval with the data: our interval is far higher than the EIA data.

But why are the two numbers so different? The interval itself cannot tell us. Some explanation could be King County residents spend much more than the state average; or the ACS asks each household for last month’s electricity cost, and people maybe answer from memory. Therefore, a narrow confidence interval only measures sampling error. It says nothing about whether the question measures what we want, the validity/bias problem.

Proportion

What share of King County households has a high energy burden (above 6%)? For a proportion, the 95% interval is

\[\hat{p} \pm 1.96 \times \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}\]

burden <- energy %>%
  filter(!is.na(energy_burden)) %>%
  pull(energy_burden)
x <- sum(burden > 0.06) # number of households with a high burden
n_b <- length(burden)
p_hat <- x / n_b
se_p <- sqrt(p_hat * (1 - p_hat) / n_b)
c(p_hat, p_hat - 1.96 * se_p, p_hat + 1.96 * se_p)
## [1] 0.1091980 0.1061107 0.1122854

prop.test() computes the interval for us:

prop.test(x, n_b)$conf.int
## [1] 0.1061363 0.1123366
## attr(,"conf.level")
## [1] 0.95

10.9% of King County households have a high burden (95% CI: 10.6% to 11.2%). The interval from prop.test() uses a slightly different formula (Wilson’s method with a continuity correction), which matters for small samples; with 39,204 households it matches ours.

Confidence Intervals by Group

The county share hides differences between places. Let’s compute the share and its 95% interval for each PUMA, and plot them.

burden_area <- energy %>%
  filter(!is.na(energy_burden)) %>%
  group_by(area, seattle) %>%
  summarise(
    n = n(),
    p = mean(energy_burden > 0.06),
    se = sqrt(p * (1 - p) / n),
    lower = p - 1.96 * se,
    upper = p + 1.96 * se,
    .groups = "drop"
  )

ggplot(burden_area, aes(x = p, y = reorder(area, p), color = seattle)) + # sort the areas by share
  geom_vline(xintercept = p_hat, color = "grey60", linetype = "dashed") + # King County as a whole
  geom_pointrange(aes(xmin = lower, xmax = upper)) +
  scale_color_manual(values = c("TRUE" = "#32006E", "FALSE" = "grey50"),
                     labels = c("TRUE" = "Seattle", "FALSE" = "Rest of King County"), name = NULL) +
  scale_x_continuous(labels = scales::percent) +
  labs(title = "Share of Households with a High Energy Burden",
       subtitle = "Energy costs above 6% of income, with 95% confidence intervals",
       x = NULL, y = NULL) +
  theme_minimal() +
  theme(legend.position = "bottom")

How can we confidently say there is a difference between two groups if their confidence interval overlapped with each other?

Margins of Error in Published ACS Tables

When planners use ACS data, we download tables. Every ACS estimate comes with a margin of error (MOE) at the 90% level:

\[\text{MOE} = 1.645 \times \text{SE}, \qquad \text{SE} = \frac{\text{MOE}}{1.645}, \qquad \text{CV} = \frac{\text{SE}}{\text{estimate}}\]

Table B25040 House Heating Fuel counts occupied housing units by heating fuel. Let’s download the number of homes heated with fuel oil in each King County tract.

heat_tract <- get_acs(
  geography = "tract", state = "WA", county = "King",
  year = 2024, survey = "acs5",
  variables = c(total = "B25040_001", oil = "B25040_005"), # all occupied units; fuel oil, kerosene, etc.
  output = "wide" # one row per tract
)
head(heat_tract, 3)
## # A tibble: 3 Γ— 6
##   GEOID       NAME                                     totalE totalM  oilE  oilM
##   <chr>       <chr>                                     <dbl>  <dbl> <dbl> <dbl>
## 1 53033000101 Census Tract 1.01; King County; Washing…   1746    283    10    17
## 2 53033000102 Census Tract 1.02; King County; Washing…   2358    294    21    34
## 3 53033000201 Census Tract 2.01; King County; Washing…   2486    387    58    51

The coefficient of variation (CV) measures the SE relative to the estimate. A common convention (used by Esri) calls a CV of 12% or less high reliability, 12% to 40% medium, and above 40% low.

heat_tract <- heat_tract %>%
  mutate(
    oil_se = oilM / 1.645,
    oil_cv = oil_se / oilE, # Inf or NaN when the estimate is 0
    reliability = case_when(
      oilE == 0 ~ "Estimate is 0",
      oil_cv <= 0.12 ~ "High (CV 12% or less)",
      oil_cv <= 0.40 ~ "Medium (CV 12% to 40%)",
      TRUE ~ "Low (CV above 40%)"
    )
  )
count(heat_tract, reliability)
## # A tibble: 3 Γ— 2
##   reliability                n
##   <chr>                  <int>
## 1 Estimate is 0            182
## 2 Low (CV above 40%)       256
## 3 Medium (CV 12% to 40%)    57

Not one of the 495 tracts has a high-reliability count of oil-heated homes, and most are low. Small counts in small areas are noisy. Adding tracts together helps: moe_sum() from tidycensus combines the MOEs of a sum (the square root of the sum of squared MOEs).

oil_kc <- sum(heat_tract$oilE)
oil_kc_moe <- moe_sum(heat_tract$oilM, heat_tract$oilE)
c(estimate = oil_kc, moe = round(oil_kc_moe), cv = round(oil_kc_moe / 1.645 / oil_kc, 3))
##  estimate       moe        cv 
## 20311.000  1095.000     0.033

For the whole county, the estimate is 20,311 oil-heated homes Β± 1,095, a CV of 3.3%.

Finally, let’s check on the weights. The published share of oil-heated homes in King County is 2.16%. In our microdata:

oil <- energy$heat_fuel == "Fuel oil or kerosene"
mean(oil) # unweighted
## [1] 0.0243966
weighted.mean(oil, energy$weight) # weighted
## [1] 0.02155874

πŸ“š TODO: Compare with the Normal Distribution

Pick energy_month or income. Draw its histogram with a normal curve and its Q-Q plot. Is it close to normal? Does the log help? (Hint: log() of 0 is -Inf; filter out zeros first.) Use pnorm() to check one share against the data, as we did for $50.

# TODO

πŸ“š TODO: Low-Income Households

Repeat the proportion analysis for households with an income under $50,000. What share has a high energy burden? Give a 95% confidence interval, and compare it with the national 25%. Write one sentence for a city council member.

# TODO