SAFE-T / Safety Algorithm Fairness Evaluation for Transportation

Durham, NC · Census Tract Level · AI Equity Auditing Framework
Team

Prediction Error by Census Tract

Accuracy by Income Quintile

Accuracy by Minority Percentage

Predicted vs Actual Volume

Prediction Errors by Quintile

Methodology Basis

Bias parameters are calibrated to findings in peer-reviewed research on AI volume estimation tools: 20–30% undercounting in low-income areas; 15–25% in high-minority areas; 10–20% additional undercount in areas with poor infrastructure. These ranges represent the audit input parameters — applying them to a real city demonstrates the equity impact framework that any government can replicate once they have access to vendor output data.

Key sources: FHWA Non-Motorized Transportation Pilot Program evaluation reports; Schweizer et al. (2021) on demographic underrepresentation in GPS-based travel surveys; Jestico et al. (2016) on spatial bias in Strava cycling data. Bias magnitudes are simulation assumptions — the direction of each disparity is documented; exact magnitudes vary across cities and tools.

Crash Distribution Map

Model Performance by Quintile

Crashes Over Time

Prediction Error by Income Quintile

Infrastructure Recommendations Map

Budget Allocation by Income

Equity Comparison: AI vs Need-Based

Methodology Basis

Infrastructure gaps are derived from real OpenStreetMap feature density per census tract (sidewalk coverage, crosswalk count, bike lane km, signal density — normalized per 1,000 residents). Danger scores weight these gaps against census demographics to estimate crash risk.

The AI allocation model and need-based allocation model are parameterized simulations — not evaluations of a specific vendor product. The AI model applies a 0.6 income bias weight (calibrated to documented patterns in AI transportation tools) alongside a 0.4 danger weight, simulating how volume-prediction bias propagates into budget decisions. The need-based model allocates purely by danger score.

Basis: real OSM infrastructure data + Census ACS demographics; simulated AI allocation bias (strength=0.6) calibrated to Strava Metro / StreetLight Data undercounting research.

Demand Distribution Map

Suppression Rate by Income Quintile

AI Detection Capability

Methodology Basis

Suppressed demand is modeled from real OpenStreetMap per-capita infrastructure scores (sidewalks, crosswalks, bike lanes, bus stops per 1,000 residents per tract). Suppression factor: 1 − score² applied to a census-proportional potential demand baseline. Q1 tracts score lower on infrastructure quality, producing higher suppression — consistent with documented elasticity of active transport to infrastructure investment.

The three AI detection profiles (Naive, Sophisticated, Human Expert) are parameterized models, not evaluations of specific vendor products. Naive AI correlates with observed demand (which collapses toward zero in suppressed areas); Sophisticated AI partially corrects for infrastructure quality.

Basis: Ewing & Cervero (2010) meta-analysis on built environment and travel behavior; NACTO Urban Street Design Guide infrastructure standards; FHWA Active Transportation research program.