Changes to CSA Scoring System – Facts and Opinion

By Dennis McGee; Dennis McGee and Associates Consulting

1. Because every motor carrier is not inspected, data is not available for every carrier.
2. SMS compares all rated motor carriers against all other rated carriers, rather than comparing small carriers with small carriers and large carriers with large carriers.  For accuracy, both large and small carriers should be compared with similarly sized carriers.
3. All crashes are weighted equally to determine an SMS score. CSA looks at crash involvement as the measurement, not fault. Is it true that a driver is less safe because of involvement in crashes, even when the accident is not their fault?
4. This is extremely challenging.  An example of this is, if a truck was stopped waiting for a traffic signal to change from red, or a school bus driver was in the same situation and is struck from behind by another vehicle, why should the truck company or school bus company be penalized for the “accident” with no background that neither the truck or school bus driver were not at fault. With no perspective to these accidents, the readers would believe that the information is reflective of the truck driver transporting freight or the school bus driver transporting their child.
5. Crash scenarios that could be classified as non-preventable and currently charged to a motor carrier’s CSA score are if a CMV vehicle was struck by another who was: driving under the influence, driving the wrong direction, striking the rear of a CMV; and/or striking the CMV while it was legally stopped.
6. There is a lack of sufficient data to reliably assess the performance of carriers. According to the CSA’s findings, they only had enough data to assign scores to three percent of active carriers in all seven of the BASICS.
7. States also have differences in their enforcement of certain laws and regulations. A driver is much more likely to get a moving violation in Indiana (29%) than Mississippi (1.4%). Vigillo Inc. (8) found that about half of all speeding tickets were written in ten states. A driver operating in one of these states may have higher scores due to stricter enforcement.

The GAO report concluded among other things:

1. “For SMS to be effective in identifying carriers more likely to crash, the violations that FMCSA uses to calculate SMS scores should have a strong predictive relationship with crashes.  However, based on GAO’s analysis of available information, most regulations used to calculate SMS scores are not violated often enough to strongly associate them with crash risk for individual carriers.”
2. “SMS is intended to provide a safety measure for individual carriers, and FMCSA has not demonstrated relationships between groups of violations and the risk that an individual motor carrier will crash.”
3. “To improve the CSA program, the Secretary of Transportation should direct the FMCSA Administrator to take the following two actions:  Revise the SMS methodology to better account for limitations in drawing comparisons of safety performance information across carriers; in doing so, conduct a formal analysis that specifically identifies:
4. Limitations in the data used to calculate SMS scores including variability in the carrier population and the quality and quantity of data available for carrier safety performance assessments, and
5. Limitations in the resulting SMS scores including their precision, confidence, and reliability for the purposes for which they are used.”
6. “Data used to determine a carrier’s score is inconsistent due to differences in inspection and enforcement policies among the states.”
7. “Scores for small carriers may be inflated and fluctuate greatly because there is less data available.”
8. “Some of the data is self-reported, such as data used to calculate scores in the Unsafe Driving BASIC the Crash Indicator BASIC. This leads to inaccurate, missing or misleading reports from carriers.”
9. “Most regulations factoring into the calculations aren’t violated enough to be tied to crash risk. When the GAO studied the violations, 593 of the 750 violations studied were only violated by less than 1% of carriers.”
10. “A majority of carriers that are determined to be “high risk” by CSA have not crashed at all, showing a weak relationship between BASICs scores and crash occurrence.”(5)
11. The GAO report also noted that “most carriers lack sufficient safety performance data to ensure that FMCSA can reliably compare them with other carriers.”  Further, in recent testimony before the Senate Commerce, Science and Transportation Committee, U.S. GAO Comptroller General Gene Dodaro noted that SMS scores are so flawed that they should be removed from public view.

Congress commissioned the GAO Report due to concern over the effectiveness of the SMS scoring model. The Report suggests that only motor carriers who have sufficient data should be scored. Fewer carriers would be scored, but scores would be more accurate.

The key changes that FMCSA made to the SMS public website by March 25, 2011, were:

• Replace ALERT symbol currently displayed in orange on the SMS website with the symbol of an exclamation mark inside a yellow triangle.
• Place the following guidance and disclaimer language on the SMS website to read:
• The data in the Safety Measurement System (SMS) is performance data used by the Agency and Enforcement Community. A     symbol, based on that data, indicates that FMCSA may prioritize a motor carrier for further monitoring.
• The    symbol is not intended to imply any federal safety rating of the carrier pursuant to 49 USC 3114.  Readers should not draw conclusions about a carrier’s overall safety condition simply based on the data displayed in this system.  Unless a motor carrier in the SMS has received an UNSATISFACTORY safety rating pursuant to 49 CFR Part 385, or has otherwise been order to discontinue operations by the FMCSA, it is authorized to operate on the nation’s roadways.”

“As of Dec. 4, 2015, pursuant to the FAST Act of 2015, much of the information previously available on the Federal Motor Carrier Safety Administration’s (FMCSA) website related to property carrier’s compliance and safety performance will no longer be displayed publicly.

“The data in the Safety Measurement System (SMS) is performance data used by the Agency and Enforcement Community. A     symbol, based on that data, indicates that FMCSA may prioritize a motor carrier for further monitoring.

The    symbol is not intended to imply any federal safety rating of the carrier pursuant to 49 USC 3114.  Readers should not draw conclusions about a carrier’s overall safety condition simply based on the data displayed in this system.  Unless a motor carrier in the SMS has received an UNSATISFACTORY safety rating pursuant to 49 CFR Part 385, or has otherwise been ordered to discontinue operations by the FMCSA, it is authorized to operate on the nation’s roadways.”

The FMCSA website offers the below information to its readers for an “unsatisfactory” rated motor carrier.

“U.S. DOT# XXXXXXX is currently under an Out-of-Service order from FMCSA and shall not operate. Out-of-Service Reason: Unsatisfactory = Unfit.”