StreetLight Data Issues Public Comment on DOT Notice
* * *
Executive Order 13985 on Advancing Racial Equity and Support for Underserved Communities Through the Federal Government (Equity E.O.) directs the Federal Government to pursue a comprehensive approach to advance equity, civil rights, racial justice, and equal opportunity to strengthen communities that have been historically underserved, marginalized and adversely affected by persistent poverty and inequality.
In response, the
Similarly,
Methods and Assessment Tools to Measure Equity
Q: What are feasible methods for the Department to assess equity in transportation, including whether and to what extent, Departmental programs and policies perpetuate systemic barriers to opportunities and benefits for underserved communities?
There are two key methods:
1. Use big data to measure equity imbalances in transportation systemically and with granularity.
2. Update the data regularly and perform equity before-and-after studies to build a detailed understanding of what policies and interventions improve equity in transportation.
First - the Department needs to determine a set of metrics to assess equity that are simple and can be measured consistently nationwide. More than one metric will be required, including: Accessibility: The transportation community has centered on a concept called "accessibility," which, in short, measures how many places a resident can get to by car, foot, bicycle, or transit within X minutes. This can be calculated for different types of places (e.g., jobs, healthcare, groceries, voting, daycare, etc.). If low-income neighborhoods in a metro area have worse accessibility scores than high-income neighborhoods, that is one way to quantify inequity. This method is best summarized in a recent paper by the State Smart Transportation Initiative/1 - we recommend following their methods.
Actual Travel Time to Access Jobs and Services, or "What is Accessed": The Accessibility metrics mentioned above describe what could be accessed by different communities. This has many applications, especially in forecasting. However, these metrics don't measure what actually happens.
For example - what if a neighborhood has 1000 jobs within 15 minutes, but none of them match the skills or needed salary for neighborhood residents? The average commute time for the neighborhood might be 45 minutes to get to jobs. With the availability of Big Data about mobility patterns (from smartphones, connected cars, and other devices), fused with land use and point-of-interest data, it is possible to get granular travel time distribution for any neighborhood to any set of locations that are actually accessed. For example - distribution of travel time to actual work locations for neighborhood residents, segmented by mode (bus travel time vs. car). The same set of travel time distributions can be developed for each neighborhood in the
Second - we need to acknowledge that we don't have a good idea of the impact different interventions may have on equity. Here are some questions that we do not know the complete answer to and only have anecdotal evidence regarding:
* When we build new highways, sponsors make promises (e.g., reduced congestion, access to jobs, etc.). For which highways constructed in the past decade have such promises come true and not come true? Did all communities benefit equally?
* Where does adding a bicycle lane increase access to critical services?
* If people work from home, will their overall VMT go down? Will it only go down for white-collar workers? (The pandemic has clearly challenged this long-held assumption.)
We're still in the dark because we do not have a practice of consistent, widespread before-and-after (also called ex-post) studies. This problem exists for classic transportation metrics, like congestion, and new metrics for analyzing equity. However, when it comes to new equity metrics, the problem is much more profound. Many have remarked on this gap - for example, a past review of published impact studies says:
Over the last few decades, there has been increasing attention given to the lack of demand forecast accuracy. However, since data availability for comprehensive ex-post appraisals is problematic, such studies are still relatively rare...Mandatory, systematic ex-post evaluation programmes are suggested as a necessary tool to improve decision support, as data availability for ex-post studies is often remarkably poor even for internal audits. (Nicolaisen and Driscoll, 2014)./2
The rise of using passive Big Data (derived from mobile devices, smartphones, connected cars, IoT sensors, etc.) over the past ten years has reduced this data barrier. As a result, the Department can leverage this type of data to dramatically accelerate understanding of the current equity problems of the transportation system with great local context and begin to improve them.
The Department needs to require that agencies regularly update these metrics to check progress in general, and specifically require that agencies do before-and-after studies of major infrastructure investments, interventions, and policy changes to see if the predicted outcomes were realized or additional adjustments needed to be made. Most predictions will be wrong in some way - we do not have a good history of clear documentation of which interventions cause which outcomes (and the world is constantly changing, as the pandemic shows). Thus, an incorrect prediction need not be penalized. Instead, if the agency makes adjustments based on updated data, that type of continuous and iterative data-driven planning and management cycle should be applauded.
We do not have to wait to get started on these impact studies. StreetLight
Q: What assessment tools currently exist to analyze equity in transportation investments, policies, and programs? Can these tools be scaled to a national level? If so, please describe the nature and level of detail of the data and how the data are collected or retrieved. If possible, please discuss any privacy concerns or barriers for collection of these data.
Accessibility: Several tools are available that focus on specific modes, such as Conveyal's tool that focuses on Transit Accessibility. These tools require several types of data:
* Road segment speed by time of day: This allows the tool to consider actual travel times for motorized transportation (cars, buses). This data is widely available, though not free, and is collected by the mobile devices of passengers using the nation's roadways. It can be provided by StreetLight Data (my firm) or other firms.
* Availability of active mode infrastructure (bike lanes, sidewalks, bus routes, etc.): This data is available within the OpenStreetMap library. However, it is not complete. The Department should encourage or sponsor cities to update their active mode infrastructure in OpenStreetMap to benefit all. In addition, some new technologies based on vehicle-mounted cameras are beginning to map and code this infrastructure. However, without encouragement and/or incentive, it may be kept in proprietary databases. The agency should review restrictions in the OpenStreetMap open source license as well.
* Point-of-interest (POI) data (where are jobs, daycare centers, groceries, etc.): This data is widely available from private providers, and limited jobs data is also available from existing federal data products.
Actual Travel Time Distribution by Mode, or "What is Accessed":
To measure actual travel time distribution by mode, it is required to have a sample of actual trips that is large enough to capture some of every trip type (e.g., job, grocery, daycare, etc.) for each mode used (e.g., bicycle, car, transit, etc.) for every Census Tract (as a proxy for neighborhood) in America. In addition, this data must be updated regularly and must be available in a usable and privacy-protected format.
StreetLight is an industry leader in privacy practices, with a full-time Vice President of Privacy and a commitment to Privacy by Design ("PbD"). StreetLight has put robust technical safeguards in place to protect data throughout the production process. Administrative safeguards and training complement these technical safeguards.
StreetLight provides agencies with aggregated data or "Metrics" (e.g., the travel time distribution of all residents of Tract X to walk to the grocery). We never provide data for individuals. We strongly recommend that the Department work with a third-party provider of anonymized, processed, and aggregated Metrics and not acquire individual trip traces for several reasons:
1) These trip traces reveal individual behavior, such as trips to cancer or abortion clinics, political meetings, or protests. Many Americans are very uncomfortable with the concept of the government holding such individual data at a nationwide scale, especially with the potential to combine this data with other government-held data.
While transportation departments have not been explicitly barred from licensing and using this raw data at this time, recent sources indicate that other government entities (military, law enforcement, national security) have been bypassing Fourth Amendment requirements for search warrants and subpoenas and licensing individual location data directly from data brokers./3
2) This has caught the attention of lawmakers and organizations defending civil liberties, resulting in a number of proposals to close the loophole and ensure that these government entities secure a search warrant to access commercial location data about individuals./4
3) Data from mobile devices is a constantly evolving field. A few years ago, data from cell towers was the best. Now, data from smartphones is the most commonly used, and new options are coming online all the time. As smartphone technology and regulation and usage patterns change, the characteristics and quantity of the data are constantly changing. There is no single algorithm that can be run or rerun each month to update metrics. Instead, it requires a large team (or teams) of data scientists constantly updating algorithms and scanning the market for new data to keep metrics fresh and accurate.
Q: What assessment tools and best practices currently exist to analyze equity in state and metropolitan transportation planning processes?
For each Census Tract (the best proxy for neighborhood and easiest to join to other data such as demographics), the following accessibility criteria should be measured. These metrics can then be turned into a set of indices using a combination of the average and 90th percentile travel time for each mode for each key service.
Link to table below.
In addition, Metrics can be calculated at the tract level that estimate the level of noise and criteria air emissions generated in each tract from transportation. These are derivatives of Vehicle-Miles Travelled (binned by speed and truck vs. car), which data are also readily available from Big Data suppliers like StreetLight. The "Study of Capabilities and Limitations of Vehicle Telematics Data for Emission Inventories"/6 paper provides additional methodological detail.
Then, each region (e.g., city, MPO, state, or county) needs to characterize the population of each tract by demographics - income, race, age, education level - this can be done using data from the
Finally, each region can create metrics that show each equity score in relation to the demographic of interest. For example, the mean and 90th percentile travel time to work for low, medium, and highincome tracts can be shown easily in a graph, as illustrated in Figure 1. It's important to use local percentiles to "bin" criteria like income, not national averages. This dual "granular" and "aggregate" approach has the benefit of
a) Allowing relatively simple, regional metrics to be communicated, and
b) Neighborhoods with disproportionately bad scores can be highlighted and prioritized for investment in the right type of access (i.e., actual vs. hypothetical, grocery vs. daycare, etc.).
Link to figure below.
This should be updated quarterly. For places where new interventions/infrastructure or policies have been implemented, a detailed before and after study of these metrics for the surrounding area should be performed.
Q: Transportation plays a critical role in how people access what they need (e.g., jobs, school, healthcare) and facilitates the movement of essential goods. What methodologies exist for measuring access to goods, services, education, recreation, and employment; well-being; and transportation reliability for people of color and other underserved groups? What are the limitations of the current measures or methods? What data is needed to overcome those limitations? How should the Department capture transportation's ability to contribute to opportunities that help improve equity for underserved communities or individuals?
There are several methods for measuring access. We believe the best approaches are those outlined in a recent paper by the
Recent advances in speed data availability by mode and road segment and point of interest (POI) data make this approach scalable.
However, the concept of accessibility alone is not sufficient to inform a meaningful understanding of access and equity in transportation. It has two key gaps:
* It doesn't measure what is actually used (or what is actually accessed) by people. It only measures what they could access. This creates many problems. For example - what if a neighborhood has 1000 jobs within 15 minutes, but none of them match the skills or needed salary for neighborhood residents? The average commute time for the neighborhood might be 45 minutes to get to jobs that match residents' needs and skills. Or what if, in theory, a grocery store can be accessed by foot in 10 minutes, but the road is down a huge hill (which no one wants to carry their grocery bags up) or dangerous. A different approach is needed to measure "What is Accessed" (described below) based on modern Big Data.
* It isn't usually implemented with an eye to before-and-after studies. If a project is done because, in theory, it will improve access to jobs by several minutes, it's essential to follow up in 1, 3, or 5 years to see if the travel time to work for that neighborhood has actually decreased.
Actual Travel Time to Access Jobs and Services, or "What is Accessed": With the availability of Big Data about mobility patterns (from smartphones, connected cars, and other devices), fused with land use and point-of-interest data, it is possible to get granular travel time distribution for any neighborhood to any set of locations that are actually used. For example - a distribution of travel time to work for neighborhood residents, separated by mode (bus travel time vs. car) for where those people actually work. The same set of travel time distributions can be developed for each neighborhood in the US for work, daycare, healthcare, grocery, and more for each mode of travel. Comparing these actual experienced travel times for different neighborhoods can reveal actual equity gaps.
The Need for Before-and-After Studies: We don't have a practice of consistent, widespread before-and-after (also called ex-post) studies when we build infrastructure projects. This problem exists for classic transportation metrics, like congestion. However, when it comes to new equity metrics, the problem is many-fold more profound. Also, in terms of measurement, equity metrics should be measured in per-capita, per unit of travel, or per dollar to be consistent and unbiased.
The rise of using passive Big Data (derived from mobile devices, smartphones, connected cars, IoT sensors, etc.) in the past ten years has reduced this data barrier. As a result, the Department can leverage this type of data to dramatically accelerate our understanding of the current equity problems in our system with great local context and begin to improve them.
The Department needs to foster or require that agencies update these metrics regularly to check progress in general, and specifically require that agencies do before-and-after studies of major infrastructure investments, interventions, and policy changes to see if the predicted outcomes came true or if additional adjustments should be made. Most predictions will be wrong in some way - we do not have a good history of clear documentation of which interventions cause which outcomes (and the world is constantly changing, as the pandemic showed). Thus, an incorrect prediction should not be penalized. Instead, if the agency takes updated data and adjusts, that type of iterative data-driven planning and management cycle should be applauded.
But we don't have to wait to get started on these impact studies. StreetLight has data that allows measurement of Accessibility and Travel Time Distribution by Mode/"What is Accessed" going back over five years and updated ~weekly. So we can immediately begin to develop a knowledge base for the practicing community by looking at projects implemented during those years.
Q: What data or data collection methods can be employed or augmented to better capture impacts of transportation on the safety and security of underserved populations, especially when people from underserved populations are walking or biking?
Traditionally, planners have relied on incomplete statistics captured with sensors, surveys, and police reports. But focusing efforts based on population density or number of fatalities (separately) may not return the most significant bike safety results. Models based on this data alone may under- or over-inflate bicycling danger based on overall population numbers.
Using big data, planners can zero in on the real experiences of cyclists. By overlaying crash data points with metrics that represent the distribution of actual bicycle trips, and the demographics of those bicyclists, planners can hone in on the complete picture of bike safety, exposure, and equity. In addition, bringing in vehicle trips to identify areas with high vehicle volumes and speeds and few bicyclists may help understand the impact of transportation on walk and bike safety of underserved populations.
For example, traditional wisdom for any city might lead it to prioritize bike safety efforts in the densest areas where cyclists ride more. Or planners might choose to focus on where more fatal crashes happen. In
But when StreetLight analyzed the number of bike trips on all corridors across the boroughs and then compared that to the number of crashes (not just fatalities), we found that exposure is actually higher elsewhere in NYC. Next, streetLight plotted the comparison of bike crashes to bike trips for select areas in
The
City Planners used StreetLight Metrics to identify and rank the business districts with high bicycle activity, which later helped the city efficiently deploy bike parking infrastructure.
The study proved the importance of real-world travel data to verify anecdotal claims and assumptions. For
Data from connected vehicles is an additional valuable data source for analyzing exposure and safety. For example, combining such data about "near Misses" from Ford with data about total county VMT from StreetLight, we calculated the number of harsh events per million VMT and analyzed which counties in the
Q: What assessment tools and practices are currently being used at any level of government that do not address equity or worsen disparities felt by underserved groups? What data are being used in a way that widens disparities in safety and access to transportation by traditionally underserved groups?
1) Using surveys that only cover a tiny percentage of the population on a small set of days. Surveys consistently undersample low income and non-native English speakers, and younger people, despite valiant efforts to fix this. A more modern approach, such as Big Data, has vastly better sampling and thus captures all sorts of people.
2) Systemic measurement of modes beyond cars. We have no nationwide estimates of bicycle miles traveled or pedestrian miles traveled as we do for Vehicle Miles Traveled. Most regions have just a handful of temporary bicycle and pedestrian counters. Without data on where these travelers go, we can't improve their safety. Big Data can correct this data gap.
3) Transportation planners need to know the income (and race, language, etc.) of people by where they travel, not just where they live. When we talk about transportation equity, we often use local residential income as a proxy, but most people spend their day away from their residence.
We need to know the income breakdown of travelers on X road and how it changed from 2020 to 2021? Again, Big Data can solve this question.
Equity Data Considerations
Q: How should the Department amend the transportation data it collects to meet equity analysis needs at the necessary spatial granularity (the geographic level of detail, i.e., national, state, local)? Since most of the Department's funding is not directed at individuals, what is the appropriate level of spatial granularity to accurately evaluate the impact of transportation investments on underserved communities?
We believe Census Tract is best - it is the approximate scale of a neighborhood, and it is easily joined to other data sets. In addition, it rolls cleanly up into counties and states.
Q: What actions can the Department take with its data to make it more useful for equity research and analysis?
Provide the data via a tool (or enable a set of tools made feasible via an API) that employ best practices in usability for less technical users. Many past federal data access tools have been clunky, have become quickly out of date, and have such a high learning curve that they are not widely used.
Partnering with third-party software designers to implement best practices and understanding that these tools need constant and significant maintenance funding is the best path. Also, maintaining an API, clearly documented, that allows other approved third parties to develop tools on the purchased data set would foster innovation in visualization and utilization of this data.
Q: What data exist that track people in historically underserved groups over time (i.e., panel surveys) that may be useful to evaluating transportation equity? What metadata is useful in determining that a data collection effort is equitable (e.g., demographic profile of the researchers, method of questionnaire administration, language of questionnaire)? What methods or data would be useful in addressing nonresponse bias in equity data collection?
We believe that no technique will viably address non-response bias using survey techniques. All the best practices (e.g., phone-based, internet-based, oversampling in some neighborhoods) have been attempted, and while they may have small localized success, over time, bias has gotten worse, not better. This will continue. The only way to get a representative sample is from passive data collection from widespread devices, such as smartphones which enjoy near-universal penetration.
Detail on StreetLight's data sources, their lack of bias compared to surveys, and methodology is provided in our StreetLight InSight(R) Methodology and Data Sources white paper, which is available for review and download at streetlightdata.com/whitepapers or upon request.
Q: Transportation plays a large role in localized pollution and negative environmental outcomes for those living near certain transportation routes and facilities. These negative environmental outcomes can have disproportionately high and adverse effects on underserved populations. How can the Department better analyze these effects, what are the data gaps, and what data sources can help address this problem? For example, what data are needed to measure the impact of vehicle electrification on the shift from mobile-source emissions to point-source (e.g., power plant) emissions on disadvantaged populations?
Up to date measurements of internal combustion engine (ICE) vehicle and ICE truck miles (with speed) that are updated regularly. A method to do this using Big Data is described in the "Study of Capabilities and Limitations of Vehicle Telematics Data for Emission Inventories"/10 paper. To do this well, the Department should collect and publish EV, PHEV, HEV registration data by tract as is done in
Q: What data are required to model equity outcomes at the individual person level? How can the Department gather this information while protecting personal privacy?
We do not think this is possible while protecting privacy. Instead, the Department should seek to understand equity at the group and population levels. Furthermore, a model is just a model - even one based on individuals. It would not be possible to verify the accuracy of that model (i.e., if the improvements to equity were realized or not) at an individual level at scale. Thus the efficacy of the model would only be theoretical.
For individual projects, it would be possible to partner with an academic institution to do detailed survey work of a neighborhood with coded identifiers and then follow up with those individuals after the project/intervention was implemented to measure individual outcomes. This is a good idea that should be done often but would be too costly on a national scale.
Q: What are approaches that DOT can take to ensure that individuals from underserved populations are represented in our data collection efforts?
The DOT should establish a plan to use and apply big data resources throughout the planning and policymaking process. Big Data can supplement, and in many cases replace, traditional transportation data sources that can be costly and under-represent underserved populations.
Fundamentally, we believe (and research backs this up) that more equitable data and sampling will lead to more equitable transportation policies and infrastructure planning. These policies and plans have generational impacts due to the extent of the life cycles for these investments.
As a simple example -- if no one from a specific block group is included in a survey, then planners may never know that it takes twice as long for that community to get to work compared to the average and not take actions to correct this inequitable distribution of transportation and accessibility. If lower-income block groups are more often the ones not included, then a systemic community-wide bias in the application of transportation planning can develop.
Two of the questions we're often asked at StreetLight Data are: "What percentage of the population does your sample capture?" and "Is the location data in your sample biased in some way, or does it fairly represent all groups?" Our analysis shows that the location-based services (LBS) data we use closely represents the population at large - more so than typical surveys. Broader representation of all groups is an important benefit of big data in transportation. We have detailed this in our white paper, "
One of the
Q: How should the Department develop a data collection framework, gather new and existing data, set data standards, and analyze and aggregate it into useful information for policymaking?
The Department should acknowledge that success will require a sample of actual trips that is large enough to capture some of every trip type (e.g., job, grocery, daycare, etc.) for each mode used (e.g., bicycle, car, transit, etc.) for every Census Tract (as a proxy for neighborhood) in America. This data must be updated regularly and must be available in a usable and privacy-protected format. The Department should partner with private firms and potentially academic partners to conduct the data collection and aggregation for the various metrics and perform validation tests regularly. The data should be easily accessible and shareable across multiple groups working on analysis and policy, which is more likely to be feasible with cloud-based data solution(s).
This should be published via modern, best-in-class UX and open API.
Q: How should the Department engage industry to increase the data available to understand electric vehicles and vehicle hybridization with the intent of understanding how these technologies can benefit different income and demographic groups; and to improve the distribution and fairness in the use of these technologies for all citizens?
Up to date measurements of ICE vehicle and ICE truck miles (with speed) that are updated regularly. A method to do this using Big Data is described in the "Study of Capabilities and Limitations of Vehicle Telematics Data for Emission Inventories"/12 paper. To do this well, the Department should collect and publish EV, PHEV, HEV registration data by tract as is done in
About StreetLight Data: StreetLight Data applies proprietary machine-learning algorithms to measure travel patterns and makes them available on-demand via StreetLight InSight(R), the world's first software as a service (SaaS) platform for mobility. StreetLight powers thousands of global projects every month.
For more information, please visit www.streetlightdata.com.
View figure and table at: https://downloads.regulations.gov/DOT-OST-2021-0056-0230/attachment_1.pdf
* * *
Footnotes:
1/ Sundquist, Eric,
2/ Nicolaisen,
3/ https://www.vice.com/en/article/jgqm5x/us-military-location-data-xmode-locate-x
6/ https://crcao.org/wp-content/uploads/2020/11/ERG_CRC-E-131-Final-Report_Oct-30-2020.pdf
7/ Sundquist, Eric,
8/
9/ https://www.streetlightdata.com/connected-car-data-creates-road-safety-insights/?type=blog/
10/ https://crcao.org/wp-content/uploads/2020/11/ERG_CRC-E-131-Final-Report_Oct-30-2020.pdf
12/ https://crcao.org/wp-content/uploads/2020/11/ERG_CRC-E-131-Final-Report_Oct-30-2020.pdf
13/ https://www.energy.ca.gov/files/zev-and-infrastructure-stats-data
* * *
The notice can be viewed at: https://www.regulations.gov/document/DOT-OST-2021-0056-0001
TARGETED NEWS SERVICE (founded 2004) features non-partisan 'edited journalism' news briefs and information for news organizations, public policy groups and individuals; as well as 'gathered' public policy information, including news releases, reports, speeches. For more information contact
Senate Special Committee on Aging Issues Testimony From Personal Finance Author Brooks
South East flood protection receives record government investment
Advisor News
- How much could failure to fund Social Security cost average Americans?
- How can more Americans achieve financial independence?
- Savers vs. spenders: How money management attitudes impact financial confidence
- Demonstrating the value of life insurance to Gen Z
- Poor money habits are a dealbreaker in a new relationship
More Advisor NewsAnnuity News
- Canvas steps into the direct-to-consumer market that has yet to take off
- The next growth phase in life/annuities depends on modernization
- CA judge certifies class action in teachers’ lawsuit over in-plan annuity fees
- Globe Life Inc. (NYSE: GL) Records 52-Week High Thursday Morning
- AM Best Managing Director Joins ‘Target Topics’ Podcast to Discuss State of Delegated Underwriting Authority Enterprises Market
More Annuity NewsHealth/Employee Benefits News
- Covered California rates to jump nearly 10% as thousands struggle to afford coverage
- A Tale of Two Behavioral Health Systems, or How the State Border Determines Who Gets Access to Mental Healthcare
- Recent Findings from Iwate Medical University Provide New Insights into Science (Nonlinear Associations Between Sleep Duration and Incident LTCI Certification in Older Adults: Findings From the Iwate-Kenpoku Cohort Study): Science
- Researchers from Capital Medical University Detail New Studies and Findings in the Area of Managed Care (Factors Contributing To the Enrollment of Ambiguous Medical Insurance Cases: a Retrospective Study At a Tertiary Hospital In Beijing): Managed Care
- Studies from Cornell University Further Understanding of Managed Care (IRA Reforms To Medicare Part D Reinsurance Were Associated With Plan Exits And Higher Premiums, 2024-25): Managed Care
More Health/Employee Benefits NewsLife Insurance News
- USAA introduces Secure Start whole life program for children
- Best’s Market Segment Report: AM Best Maintains Stable Outlook on South Korea’s Non-Life Insurance Market
- Horace Mann Strengthens Customer Relationships and Accelerates Long-Term Growth Through Transactions with Medical Mutual of Ohio
- Regulators: ‘No firm conclusions’ from first offshore reinsurance filings
- Allianz Life Study Finds Americans Struggle to Shift From Retirement Saving to Spending
More Life Insurance News