Do CSA Scores Matter?

CSA scores do matter.

Why?

Because significant people think they matter. People who can help you. And, people who can hurt you. While we would like to live in a society based on facts, it’s often perceptions that carry the day.

Read the rest of the article here……

Mark G. Gardner, Chief Executive Officer.  Avatar Management Services, Inc

Fleet Owner: DOT IG faults CSA data quality, interventions

DOT IG faults CSA data quality, interventions

Avery Vise Fleet Owner

In an audit report that focuses primarily on the limited issue of the timeliness and completeness of state- and carrier-reported data, the U.S. Dept. of Transportation’s Office of Inspector General (DOT IG) said that the Federal Motor Carrier Safety Administration has not fully implemented various planned steps to improve the data underlying the Compliance, Safety, Accountability (CSA) program.

The audit report released on March 7 also notes that only 10 states had fully implemented CSA enforcement interventions due to delays in receiving and being trained on new software needed for assessing and monitoring interventions. FMCSA expects to release the software by May 2015.

Given the limited implementation of enforcement interventions to date, the DOT IG did not assess the effectiveness of the interventions. In a footnote, the DOT IG report notes that the National Transportation Safety Board had recommended an audit of FMCSA’s use of focused compliance reviews but did not comment on whether it would be conducting such an audit.

Unlike a recent critical Government Accountability Office study, the DOT IG doesn’t address at all CSA’s effectiveness in identifying high-risk carriers. And unlike with the GAO report, which prompted FMCSA to respond with its own analysis defending CSA, the agency agreed with all of the DOT IG’s recommendations. The DOT IG recommendations are:

• Issue updated DataQs guidance to clarify how data challenges should be reviewed and to provide additional measures to ensure that data challenges are closed consistently;
• Implement the process for deactivating USDOT numbers when carriers do not submit required census data, as outlined in a policy issued by FMCSA in November 2013;
• Develop a comprehensive plan to fully implement CSA enforcement interventions in the remaining 40 states and the District of Columbia. The plan should include an estimated completion date and milestones for releasing Sentri software, developing and delivering training, and using the enforcement interventions, the DOT IG said;
• Update the Carrier Safety Measurement System (CSMS) requirements document to specify all sources of CSMS data – including each of the Motor Carrier Management Information System fields used – and to fully describe CSMS validation procedures;
• Develop and implement a process for managing CSMS system documentation that includes a central file for validation records and testing results; and
• Develop and implement a configuration management policy that includes documentation of system changes and associated testing for CSMS.

Although the DOT IG report was not as critical of FMCSA’s implementation of CSA as the GAO report, the American Trucking Assns. argued that it highlighted significant problems with CSA, including some that ATA had identified in a white paper late last year.

“The Inspector General’s report confirms what industry stakeholders, independent researchers and other government watchdogs have found: there continue to be significant flaws in the data FMCSA is using to evaluate and score carriers under CSA,” said ATA Executive Vice President Dave Osiecki. “ATA continues to support the oversight mission and safety goals of CSA – but FMCSA must acknowledge the program’s many problems – and commit to addressing them.”

ATA cited the DOT IG’s findings that FMCSA (1) had taken “limited action” to address previously identified flaws; (2) had failed to get 40 states and DC to implement CSA interventions; (3) had not followed standard technology validation and testing; and (4) failed to push states to be consistent in correcting reams of inaccurate data.

“The audit found that while FMCSA claims to have enough data to evaluate 40% of the industry – 13% of those companies report not owning or operating any trucks,” Osiecki said.  “Serious inaccuracies like this affect the scores of everyone scored under CSA – since carriers are compared to one another.”

Motor carriers themselves are not blameless, however, Osiecki acknowledged, noting the DOT IG’s finding that many motor carriers need to improve their reporting of operational and exposure data, such as the number of trucks they operate.

ATA said it was disappointed that the DOT IG took FMCSA’s self-assessment of its State Safety Data Quality (SSDQ) system at face value. The SSDQ scores states’ ability to upload timely and accurate data – the very data that drives the SMS.

“While the SSDQ indicates states are performing better than they have in the past, the IG failed to examine under what circumstances a state might obtain a ‘good’ rating,” Osiecki said. “This is a critical link in CSA data quality that deserves greater attention.”

A December 2012 ATA analysis found that FMCSA’s thresholds are so broad that a state could receive a “good” rating even if it underreported crashes by several thousand. In a large state like Texas, for example, the state could obtain a “good” rating for uploading between 12,551 and 25,163 accidents – a range of 12,612 accidents.

Also, FMCSA estimates of total expected crashes is based on the number of known fatal crashes, ATA notes. But ATA argues that fatal crashes may not be a reliable predictor of non-fatal crashes. Thorough examination of state police accidents reports remains the only way to accurately measure state accident reporting to MCMIS, ATA says.