“The data is the data,” said David Yessen of the Federal Motor Carrier Safety Administration’s compliance office. Yessen, speaking about halfway through a hearing this morning in Washington, D.C., was reiterating the agency’s strategy toward reforming the Compliance, Safety, Accountability system. That strategy follows the National Academies of Science’s June 2017 recommendations to FMCSA, in which it urged the agency to build a new statistical model to underly the SMS, one based on so-called “item response theory,” to better prioritize at-risk carriers for investigation or other intervention. FMCSA has begun that process, and Wednesday’s meeting was part and parcel of its engagement with stakeholders to address concerns and solicit ideas.
Yessen outlined steps the agency has taken in recent times during the hearing, including building small-scale versions of a new statistical model to test and considering new ways to utilize the data it has — its MCMIS database of inspections and violations, crash data, etc. — as well as new data inputs.
As it identifies a potential new source of data to use over the coming year as it builds the system, the agency will be dropping the data into the IRT model to see if it has the appropriate effects. Yessen likened the model to “a funnel, and you can keep dropping information into that funnel, and we’re looking for opportunities to do that.” At today’s hearing, representatives from the regulated industry communities were asked to suggest improvements related specifically to Recommendations 2, 3 and 4 from the NAS’ report on the CSA SMS.
New data under consideration include driver pay method, carriers’ driver turnover rates
Improving the quality of MCMIS data, part of recommendation No. 2, was the context for Yessen’s “data is the data” notion, a response to a commenter’s recommendation that the agency take into account geographical variations of carrier operations — and the attendant well-documented and widely varying violation priorities in different states — when normalizing the data to provide more accurate safety assessments.
Irwin Shires of Panther Expedited was quick to point out an example where the data might not be the data, so to speak, when the wrong assumptions are made. Shires is a member of the Motor Carriers for Regulatory Reform coalition, a group with CSA and other reforms at the top of their agenda. The MCRR, as member attorney Henry Seaton noted in conversation with Overdrive last week, continues to view an at least biennial “desktop audit” conducted offsite (akin to an offsite version of FMCSA’s new entrant audits in some ways) of every carrier in the nation as perhaps the best solution to safety assessment in light of the difficulty of accurately assessing small carriers with a data-based approach.
As the agency nonetheless examines bolstering data in the new system with different information, Shires cautioned that the agency to “be careful about assuming” a simple relationship to safety performance. Among new items the agency urged commentary on — driver method and level of compensation, a carrier’s driver-turnover rate, driver age and other factors — Shires noted turnover could easily suggest more than a single interpretation. For instance, “if a carrier has adopted a proactive culture of safety, which I’ve been told that’s what they want us to do, if I have methods in place to identify bad eggs … and I get them out of my fleet before they have an accident, I still have to replace the driver. That’s a kind of positive turnover.”
While Shires recognized “there are a number of carriers out there who churn through drivers” as a matter of course, he said, “take into account the exceptions to the rule before you implement them into an IRT model.”
