Coming on the heels of a government-issued report saying the Federal Motor Carrier Safety Administration’s Compliance, Safety, Accountability system’s scoring method is faulty and unfair for smaller carriers, the agency itself has released a study on CSA and its Safety Measurement System concluding the opposite: The system does work to identify carriers who pose the greatest crash risk, the agency says.
One of the agency’s key conclusions in its report signals SMS scores and their intervention thresholds identify carriers with crash rates 79 percent higher than those not identified for agency intervention.
The Government Accountability Office, the non-partisan government watchdog, released Monday, Feb. 3, a report that concluded CSA and its SMS scoring were based on a bad or incomplete data set, which yielded flawed and unfair scoring, particularly for smaller carriers, for whom data is limited.
Overdrive actually beat the GAO to the punch last year in its CSA’s Data Trail series, where it pointed out that the CSA scoring method was calculated by poor or an overall lack of data and crash rates didn’t match up with CSA-related intervention. Click here to see the CSA’s Data Trail website.
Though the agency’s report isn’t a direct rebuttal to the GAO report and its findings, it does attempt to make a case that its scoring system is effective in targeting carriers at a higher risk for crash involvement — something the GAO report said CSA was not doing.
The agency says it used an Effectiveness Test to calculate that number by simulating results for carriers based on its own data and then cross-checking that with actual crash involvement.
FMCSA measured its findings based on crashes per 100 power units, and of the 43,000 carriers it identified as above an intervention threshold per SMS scores, 51,763 subsequently were involved in a crash, yielding a crash rate of 4.82 per 100 power units.
Those carriers not identified — just more than 235,000 — as above the threshold were involved in 54,222 crashes, yielding a crash rate of 2.69 per 100 power units.
The agency also backed up criticism of CSA’s so-called bias against smaller carriers, pointing to their higher rate of crashes as the reason they’re seen as picked on, rather than a flaw in the system that unfairly singles them out.
The crash rate of carriers with fewer than five power units (which make up 75 percent of the companies registered with the DOT), have a crash rate of 3.84 per 100 power units, compared to just 3.51 for carriers with more than 50 power units and 2.98 for carriers with more than 500.
The agency does note it has a limited amount of data on smaller carriers.
“However, many of these small carriers have very little safety information to make a meaningful safety assessment. FMCSA also has limited resources for interventions. For [SMS] to work most effectively in this industry environment, the system must strike a balance of being highly selective with identifying small carriers for interventions (i.e., the group of carriers with the very worst safety problems) relative to large carriers while still holding all carriers accountable,” the study reads.
FMCSA’s study also makes the point that the agency is more selective in targeting smaller carriers, as just 12 percent of carriers with five or fewer power units have been identified for intervention, relative to their CSA scores, whereas 49 percent of carriers with more than 500 power units have been targeted for intervention and 35 percent of carriers with between 50 and 500 power units have.
That would seem partly due to the sheer amount of small companies there are relative to large companies, as 209,915 carriers have five or fewer power units, according to FMCSA, whereas just 548 have more than 500, but FMCSA chalks up the difference in intervention percentages to greater selectivity by SMS scoring:
“This means that the CSMS is being more selective with identifying smaller-sized carriers for interventions, while also being effective in finding sets of small carriers with high future crashes rates,” the study reads.
The agency also points to data that says for the most part, its BASICs (behavioral analytical and statistical category) correlate with crash risk — and oft-cited point of contention between carriers and CSA.
Its report says that in all but one of the seven BASICs, carriers targeted for intervention have a higher crash rate per 100 power units than the national average of 3.43.
The Unsafe Driving, Crash and HOS Compliance BASICS have the highest correlation with crash rate, as carriers above the intervention threshold in those categories have 93 percent, 85 percent and 83 percent higher crash rates than the national average.
The Vehicle Maintenance BASIC has a lower correlation, 65 percent, while the Controlled Substances/Alcohol and Hazmat Compliance BASIC have 34 percent and 31 percent, respectively, greater crash rates.
The Driver Fitness BASIC was the only one where carriers above the intervention threshold actually had a lower crash rate than the national average.
Lastly, the agency concluded that the more BASIC thresholds carriers cross, the greater the crash rate. Carriers who are not over the threshold in any BASICs had a crash rate of 2.69 per 100 power units. Those above in one BASIC had a crash rate of 4.26, while those above in two, three to four and five or more BASICS had crash rates of 5.77, 6.24 and 7.17, respectively. “
“In conclusion, these three analyses provide solid evidence that the CSMS as a tool is effectively supporting FMCSA in its mission to reduce crashes, injuries, and fatalities involving large trucks and buses by improving safety and compliance.,” the study reads.
Note: This story will be updated as requested responses from trucking groups become available.