Lab 08 Distributions and Confidence Intervals in R
URBAN 520 Quantitative Methods in Urban Design and Planning
- Data: ACS Public Use Microdata
- Comparing with the Normal Distribution
- Confidence Intervals
- Margins of Error in Published ACS Tables
- π TODO: Compare with the Normal Distribution
- π TODO: Low-Income Households
In Labs 8 and 9, we will move from describing data to making statements about a population from a sample. The topic is household energy cost: how much King County households pay for electricity, gas, and other fuels, and how heavy that cost is for them. We can simply define the energy burden of a household is the share of its income spent on home energy. Researchers commonly call a burden above 6% high and above 10% severe (ACEEE, 2020).
library('tidyverse')
library('tidycensus') # we used it in Lab 3 for getting ACS tables via APIData: ACS Public Use Microdata
In Lab 3, we downloaded ACS tables: estimates that the Census Bureau has already computed for each geography, such as the median household income of a tract. The Census Bureau also publishes the Public Use Microdata Sample (PUMS): the anonymized, sample-level records from the American Community Survey (ACS), one row per household or person, with names and addresses removed. The smallest geography in PUMS is the Public Use Microdata Area (PUMA), an area with at least 100,000 people. King County has 18 PUMAs, and 7 of them are in Seattle. You can check the King County PUMA Map here.
We prepared kc_energy_2024.csv from the 2020-2024 ACS
5-year PUMS: every occupied housing unit in King County that answered
the survey, 42,178 households. All dollar amounts are
adjusted to 2024 dollars. Below is the code we used to download and
clean it. The download takes about a few minutes, so the two chunks
below are not run when you knit this file (eval=FALSE). And
we will just read the data from saved CSV file.
Downloading PUMS
get_pums() from tidycensus works like
get_acs() in Lab 3, but instead of table IDs, we name PUMS
variables. You can look up every variable and its codes in
pums_variables, a data frame that comes with
tidycensus, or in the PUMS
documentation. Some variables we will use:
| Variable | Meaning |
|---|---|
TEN |
Tenure: 1 owned with a mortgage, 2 owned free and clear, 3 rented, 4 occupied without paying rent |
BLD |
Units in the structure |
YRBLT |
When the structure was first built (1939 means 1939 or earlier) |
HFL |
House heating fuel |
ELEP, GASP |
Last monthβs electricity cost and gas cost |
FULP |
Cost of other fuels (oil, wood, propane, β¦) in the past 12 months |
ELEFP, GASFP, FULFP |
Flags: is the cost valid, included in rent, or not used? |
HINCP |
Household income in the past 12 months |
ADJHSG, ADJINC |
Factors that adjust housing costs and income to 2024 dollars |
NP, BDSP |
Number of persons and number of bedrooms |
TYPEHUGQ |
Housing unit or group quarters (dorms, nursing homes, β¦) |
# View(pums_variables) # look up variable names and their codes
kc_pums <- get_pums(
variables = c("TEN", "BLD", "YRBLT", "HFL", "ELEP", "ELEFP", "GASP", "GASFP",
"FULP", "FULFP", "HINCP", "ADJHSG", "ADJINC", "NP", "BDSP", "TYPEHUGQ"),
state = "WA",
puma = as.character(23301:23318), # the 18 King County PUMAs
survey = "acs5", # 5-year PUMS
year = 2024 # 2020-2024
)Cleaning PUMS
get_pums() returns one row per person
(106,190 rows), and the household variables repeat on the row of every
person in the household. It also adds the IDs and weights
(SERIALNO, SPORDER, WGTP,
PWGTP). We keep one row per occupied housing unit: the
first person in the household (SPORDER == 1), living in a
housing unit rather than group quarters (TYPEHUGQ == "1"),
in a unit that is occupied (TEN from 1 to 4). Three more
details:
- 2024 dollars. The five survey years are in
different dollars. We multiply housing costs by
ADJHSGand income byADJINC.tidycensusreturnsADJHSGwith six implied decimals (1214548 means 1.214548), so we divide it by 1,000,000;ADJINCalready has its decimal point. - Flags. A cost only counts when its flag says it is
valid. Electricity:
ELEFP3 is a valid cost; 1 (included in rent or condo fee) and 2 (no charge or not used) giveNA. Gas:GASFP4 is valid; 2 (included in the electricity bill) and 3 (not used) give 0; 1 (included in rent) givesNA. Other fuels:FULFP3 is valid; 2 (not used) gives 0; 1 (included in rent) givesNA. - Energy burden.
energy_burden= 12 Γenergy_month/income, andNAwhen the income is 0 or less.
# short names for the codes
area_names <- c(
"23301" = "Shoreline, Kenmore & Bothell (South)",
"23302" = "Snoqualmie, Cottage Lake & Union Hill",
"23303" = "Redmond & Kirkland",
"23304" = "Bellevue",
"23305" = "Sammamish, Issaquah & Mercer Island",
"23306" = "Maple Valley, Covington & Enumclaw",
"23307" = "Auburn",
"23308" = "Federal Way, Des Moines & Vashon",
"23309" = "Kent",
"23310" = "Renton, Fairwood & Skyway",
"23311" = "Burien, SeaTac, Tukwila & White Center",
"23312" = "Seattle: West Seattle & Industrial",
"23313" = "Seattle: Southeast",
"23314" = "Seattle: Central",
"23315" = "Seattle: Lake Union & Downtown",
"23316" = "Seattle: Northwest",
"23317" = "Seattle: Northeast",
"23318" = "Seattle: North"
)
fuel_names <- c("1" = "Utility gas", "2" = "Bottled, tank, or LP gas", "3" = "Electricity",
"4" = "Fuel oil or kerosene", "5" = "Coal or coke", "6" = "Wood",
"7" = "Solar energy", "8" = "Other fuel", "9" = "No fuel used")
bld_names <- c("1" = "Mobile home", "2" = "Single-family detached",
"3" = "Single-family attached", "4" = "2-4 units", "5" = "2-4 units",
"6" = "5-19 units", "7" = "5-19 units", "8" = "20 or more units",
"9" = "20 or more units", "10" = "Boat, RV, or van")
energy <- kc_pums %>%
filter(SPORDER == 1, TYPEHUGQ == "1", TEN %in% c("1", "2", "3", "4")) %>% # one row per occupied housing unit
mutate(
across(c(ELEP, GASP, FULP, HINCP, ADJHSG, ADJINC, NP, BDSP, YRBLT), as.numeric), # text to numbers
adjhsg = ADJHSG / 1e6, # six implied decimals
elec_month = if_else(ELEFP == "3", ELEP * adjhsg, NA),
gas_month = case_when(GASFP == "4" ~ GASP * adjhsg,
GASFP %in% c("2", "3") ~ 0), # all other cases become NA
other_fuel_year = case_when(FULFP == "3" ~ FULP * adjhsg,
FULFP == "2" ~ 0),
energy_month = elec_month + gas_month + other_fuel_year / 12, # NA if any part is NA
income = HINCP * ADJINC,
energy_burden = if_else(income > 0, 12 * energy_month / income, NA)
) %>%
transmute( # like mutate(), but keeps only the columns created here
household_id = SERIALNO,
puma = PUMA,
area = area_names[PUMA],
seattle = PUMA %in% as.character(23312:23318),
tenure = case_when(TEN %in% c("1", "2") ~ "Owner",
TEN == "3" ~ "Renter",
TEN == "4" ~ "No rent paid"),
building = bld_names[BLD],
year_built = case_when(YRBLT == 1939 ~ "1939 or earlier",
YRBLT < 2020 ~ paste0(YRBLT, "-", YRBLT + 9),
TRUE ~ as.character(YRBLT)), # single years from 2020
era = case_when(YRBLT <= 1939 ~ "Before 1940",
YRBLT <= 1979 ~ "1940-1979",
YRBLT <= 1999 ~ "1980-1999",
TRUE ~ "2000 or later"),
heat_fuel = fuel_names[HFL],
persons = NP,
bedrooms = BDSP,
income = round(income),
elec_month = round(elec_month, 2),
gas_month = round(gas_month, 2),
other_fuel_year = round(other_fuel_year, 2),
energy_month = round(energy_month, 2),
energy_burden = round(energy_burden, 5),
weight = WGTP
)
write_csv(energy, "kc_energy_2024.csv")Reading the Data
energy <- read_csv("kc_energy_2024.csv")## Rows: 42178 Columns: 18
## ββ Column specification ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
## Delimiter: ","
## chr (7): household_id, area, tenure, building, year_built, era, heat_fuel
## dbl (10): puma, persons, bedrooms, income, elec_month, gas_month, other_fuel...
## lgl (1): seattle
##
## βΉ Use `spec()` to retrieve the full column specification for this data.
## βΉ Specify the column types or set `show_col_types = FALSE` to quiet this message.
Click to Read Data Dictionary
- household_id: ACS housing unit ID
- puma: PUMA code (23301-23318)
- area: Short name of the PUMA
- seattle:
TRUEif the PUMA is in Seattle - tenure: Owner, Renter, or No rent paid
- building: Units in the structure (Single-family detached, 20 or more units, β¦)
- year_built: When the structure was first built, by decade (single years from 2020)
- era:
year_builtin four groups: Before 1940, 1940-1979, 1980-1999, 2000 or later - heat_fuel: Fuel used most for heating the home
- persons: Number of people in the household
- bedrooms: Number of bedrooms
- income: Household income in the past 12 months
- elec_month: Last monthβs electricity cost;
NAif it is included in rent or not paid - gas_month: Last monthβs gas cost; 0 if no gas or included in the
electricity bill,
NAif included in rent - other_fuel_year: Cost of oil, wood, propane, and other fuels in the
past 12 months;
NAif included in rent - energy_month:
elec_month+gas_month+other_fuel_year/ 12;NAif any part is unknown - energy_burden: 12 Γ
energy_month/income;NAifenergy_monthis unknown or income is 0 or less - weight: Household weight (how many King County households this row stands for)
A note on weights. The ACS is not a simple random
sample: some addresses are more likely to be selected than others, and
the weight column corrects for that. To keep the analysis
simple, we will ignore the weights and treat the 42,178 households as a
simple random sample of King County households, as in Assignment 2. Our
numbers will therefore differ a little from published ACS tables, and we
will see by how much at the end of this lab.
Comparing with the Normal Distribution
Letβs start with the monthly electricity cost,
elec_month. 1,737 households have no electricity cost of
their own (it is included in their rent or not paid), so we drop
them.
elec <- energy %>%
filter(!is.na(elec_month)) %>%
pull(elec_month) # pull() returns the column as a vector
elec_mean <- mean(elec)
elec_sd <- sd(elec)
summary(elec)## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 4.00 85.02 139.34 181.04 220.13 2882.26
Letβs see whether a normal distribution with the same mean and standard deviation describe these costs? Draw the histogram on the density scale and stack with the normal curve.
ggplot(data.frame(elec), aes(x = elec)) +
geom_histogram(aes(y = after_stat(density)), # density instead of count, to match the curve
binwidth = 20, fill = "grey70", color = "white") +
stat_function(fun = dnorm, args = list(mean = elec_mean, sd = elec_sd),
xlim = c(-300, 1000), # draw the curve below $0 too
linewidth = 1) + # the normal curve with the same mean and SD
scale_x_continuous(labels = scales::dollar) +
coord_cartesian(xlim = c(-300, 1000)) + # zoom in; the costs above $1,000 are still counted
labs(title = "Monthly Electricity Cost of King County Households",
x = "Monthly Electricity Cost", y = "Density") +
theme_minimal()The d/p/q/r Functions
dnorm() above is one of four functions R has for every
distribution. The first letter says what you get, and the rest names the
distribution (norm, binom, pois,
unif, t, β¦):
| Function | Returns | Example |
|---|---|---|
d...() |
Density (height of the curve), or probability of exactly one value for a discrete distribution | dbinom(5, size = 12, prob = 0.4) |
p...() |
Probability of a value at or below q (area to the
left) |
pnorm(50, mean = 181, sd = 209) |
q...() |
The value with probability p below it (a
percentile) |
qnorm(0.95, mean = 181, sd = 209) |
r...() |
Random draws from the distribution | rnorm(10, mean = 181, sd = 209) |
pnorm() tells us what share of households would
pay less than a value if the costs were normal. Compare it with the
share that actually does:
pnorm(50, mean = elec_mean, sd = elec_sd) # if normal: share below $50## [1] 0.2653752
mean(elec < 50) # data: share below $50 (TRUE counts as 1, FALSE as 0)## [1] 0.08691674
pnorm(0, mean = elec_mean, sd = elec_sd) # if normal: share with a negative bill!## [1] 0.1932335
The normal distribution says 26.5% of households pay less than $50 a month; in the data it is 8.7%. It also says 19.3% pay less than nothing.
Q-Q Plot
A quantile-quantile (Q-Q) plot compares the data with the normal distribution directly. It sorts the data and plots each value against the value we would expect at the same position in a normal sample. If the data is perfectly normally distributed, it should follow the dash line.
- S-shaped curve: heavier tails or more peaked than the normal distribution
- Curved upward or downward: skewness (the data are not symmetric).
- Clearly deviating from the straight line: Suggests the data are not normally distributed.
ggplot(data.frame(elec), aes(sample = elec)) +
geom_qq(size = 0.5, alpha = 0.3) + # the data against the normal quantiles
geom_qq_line(color = "#32006E") + # the line the points would follow if the data were normal
scale_y_continuous(labels = scales::dollar) +
labs(title = "Q-Q Plot of Monthly Electricity Cost",
x = "Normal Quantiles", y = "Monthly Electricity Cost") +
theme_minimal()Log Transformation
For right-skewed data like costs and incomes, the logarithm often helps. It pulls in the long right tail and spreads out the small values. And when a variable is close to normal on the log scale, we say it is roughly log-normal.
ggplot(data.frame(elec), aes(sample = log(elec))) +
geom_qq(size = 0.5, alpha = 0.3) +
geom_qq_line(color = "#32006E") +
labs(title = "Q-Q Plot of Log Monthly Electricity Cost",
x = "Normal Quantiles", y = "Log Monthly Electricity Cost") +
theme_minimal()Confidence Intervals
Mean
What is the average monthly electricity cost of all King County households, and how precisely do our 40,441 households tell us? A 95% confidence interval for a mean is
\[\bar{x} \pm t^* \times \frac{s}{\sqrt{n}}\]
where \(t^*\) comes from the t distribution with \(n-1\) degrees of freedom. Letβs compute it by hand:
n <- length(elec)
se <- elec_sd / sqrt(n) # standard error of the mean
t_star <- qt(0.975, df = n - 1) # 2.5% in each tail
c(elec_mean - t_star * se, elec_mean + t_star * se)## [1] 179.0055 183.0805
t.test() does the same in one line:
t.test(elec)$conf.int## [1] 179.0055 183.0805
## attr(,"conf.level")
## [1] 0.95
We are 95% confident that the mean monthly electricity cost of King County households is between $179.01 and $183.08. The interval is narrow because \(n\) is large.
Comparing with a Benchmark
The U.S. Energy Information Administration (EIA) reports that the average residential electricity bill in Washington State was $113.68 a month in 2024 (EIA, Table 5A). We may check our interval with the data: our interval is far higher than the EIA data.
But why are the two numbers so different? The interval itself cannot tell us. Some explanation could be King County residents spend much more than the state average; or the ACS asks each household for last monthβs electricity cost, and people maybe answer from memory. Therefore, a narrow confidence interval only measures sampling error. It says nothing about whether the question measures what we want, the validity/bias problem.
Proportion
What share of King County households has a high energy burden (above 6%)? For a proportion, the 95% interval is
\[\hat{p} \pm 1.96 \times \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}\]
burden <- energy %>%
filter(!is.na(energy_burden)) %>%
pull(energy_burden)
x <- sum(burden > 0.06) # number of households with a high burden
n_b <- length(burden)
p_hat <- x / n_b
se_p <- sqrt(p_hat * (1 - p_hat) / n_b)
c(p_hat, p_hat - 1.96 * se_p, p_hat + 1.96 * se_p)## [1] 0.1091980 0.1061107 0.1122854
prop.test() computes the interval for us:
prop.test(x, n_b)$conf.int## [1] 0.1061363 0.1123366
## attr(,"conf.level")
## [1] 0.95
10.9% of King County households have a high burden (95% CI: 10.6% to
11.2%). The interval from prop.test() uses a slightly
different formula (Wilsonβs method with a continuity correction), which
matters for small samples; with 39,204 households it matches ours.
Confidence Intervals by Group
The county share hides differences between places. Letβs compute the share and its 95% interval for each PUMA, and plot them.
burden_area <- energy %>%
filter(!is.na(energy_burden)) %>%
group_by(area, seattle) %>%
summarise(
n = n(),
p = mean(energy_burden > 0.06),
se = sqrt(p * (1 - p) / n),
lower = p - 1.96 * se,
upper = p + 1.96 * se,
.groups = "drop"
)
ggplot(burden_area, aes(x = p, y = reorder(area, p), color = seattle)) + # sort the areas by share
geom_vline(xintercept = p_hat, color = "grey60", linetype = "dashed") + # King County as a whole
geom_pointrange(aes(xmin = lower, xmax = upper)) +
scale_color_manual(values = c("TRUE" = "#32006E", "FALSE" = "grey50"),
labels = c("TRUE" = "Seattle", "FALSE" = "Rest of King County"), name = NULL) +
scale_x_continuous(labels = scales::percent) +
labs(title = "Share of Households with a High Energy Burden",
subtitle = "Energy costs above 6% of income, with 95% confidence intervals",
x = NULL, y = NULL) +
theme_minimal() +
theme(legend.position = "bottom")How can we confidently say there is a difference between two groups if their confidence interval overlapped with each other?
Margins of Error in Published ACS Tables
When planners use ACS data, we download tables. Every ACS estimate comes with a margin of error (MOE) at the 90% level:
\[\text{MOE} = 1.645 \times \text{SE}, \qquad \text{SE} = \frac{\text{MOE}}{1.645}, \qquad \text{CV} = \frac{\text{SE}}{\text{estimate}}\]
Table B25040 House Heating Fuel counts occupied housing units by heating fuel. Letβs download the number of homes heated with fuel oil in each King County tract.
heat_tract <- get_acs(
geography = "tract", state = "WA", county = "King",
year = 2024, survey = "acs5",
variables = c(total = "B25040_001", oil = "B25040_005"), # all occupied units; fuel oil, kerosene, etc.
output = "wide" # one row per tract
)
head(heat_tract, 3)## # A tibble: 3 Γ 6
## GEOID NAME totalE totalM oilE oilM
## <chr> <chr> <dbl> <dbl> <dbl> <dbl>
## 1 53033000101 Census Tract 1.01; King County; Washing⦠1746 283 10 17
## 2 53033000102 Census Tract 1.02; King County; Washing⦠2358 294 21 34
## 3 53033000201 Census Tract 2.01; King County; Washing⦠2486 387 58 51
The coefficient of variation (CV) measures the SE relative to the estimate. A common convention (used by Esri) calls a CV of 12% or less high reliability, 12% to 40% medium, and above 40% low.
heat_tract <- heat_tract %>%
mutate(
oil_se = oilM / 1.645,
oil_cv = oil_se / oilE, # Inf or NaN when the estimate is 0
reliability = case_when(
oilE == 0 ~ "Estimate is 0",
oil_cv <= 0.12 ~ "High (CV 12% or less)",
oil_cv <= 0.40 ~ "Medium (CV 12% to 40%)",
TRUE ~ "Low (CV above 40%)"
)
)
count(heat_tract, reliability)## # A tibble: 3 Γ 2
## reliability n
## <chr> <int>
## 1 Estimate is 0 182
## 2 Low (CV above 40%) 256
## 3 Medium (CV 12% to 40%) 57
Not one of the 495 tracts has a high-reliability count of oil-heated
homes, and most are low. Small counts in small areas are noisy. Adding
tracts together helps: moe_sum() from
tidycensus combines the MOEs of a sum (the square root of
the sum of squared MOEs).
oil_kc <- sum(heat_tract$oilE)
oil_kc_moe <- moe_sum(heat_tract$oilM, heat_tract$oilE)
c(estimate = oil_kc, moe = round(oil_kc_moe), cv = round(oil_kc_moe / 1.645 / oil_kc, 3))## estimate moe cv
## 20311.000 1095.000 0.033
For the whole county, the estimate is 20,311 oil-heated homes Β± 1,095, a CV of 3.3%.
Finally, letβs check on the weights. The published share of oil-heated homes in King County is 2.16%. In our microdata:
oil <- energy$heat_fuel == "Fuel oil or kerosene"
mean(oil) # unweighted## [1] 0.0243966
weighted.mean(oil, energy$weight) # weighted## [1] 0.02155874
π TODO: Compare with the Normal Distribution
Pick energy_month or income. Draw its
histogram with a normal curve and its Q-Q plot. Is it close to normal?
Does the log help? (Hint: log() of 0 is -Inf;
filter out zeros first.) Use pnorm() to check one share
against the data, as we did for $50.
# TODOπ TODO: Low-Income Households
Repeat the proportion analysis for households with an income under $50,000. What share has a high energy burden? Give a 95% confidence interval, and compare it with the national 25%. Write one sentence for a city council member.
# TODO