STATE OF CALIFORNIA
DEPARTMENT OF MOTOR VEHICLES
ENHANCED NEGLIGENT OPERATOR TREATMENT EVALUATION SYSTEM 
 
PROGRAM EFFECTIVENESS REPORT # 1 
( SUMMARY OF FINDINGS)
 
June 2009 
 
 
 
 
Research and Development Branch Author: Michael A. Gebers
Licensing Operations Division
© California Department of Motor Vehicles, 2009
RSS- 09- 230 REPORT DOCUMENTATION PAGE
Form Approved
OMB No. 0704- 0188
Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202- 4302, and to the Office of Management and Budget, Paperwork Reduction Project ( 0704- 0188), Washington, DC 20503.
1. AGENCY USE ONLY ( Leave blank)
2. REPORT DATE
June 2009
3. REPORT TYPE AND DATES COVERED
Final Report
4. TITLE AND SUBTITLE
Enhanced Negligent Operator Treatment Evaluation System Program Effectiveness Report # 1 ( Summary Of Findings)
6. AUTHOR( S)
Michael A. Gebers
5. FUNDING NUMBERS
7. PERFORMING ORGANIZATION NAME( S) AND ADDRESS( ES)
California Department of Motor Vehicles
Research and Development Branch
P. O. Box 932382
Sacramento, CA 94232- 3820
8. PERFORMING ORGANIZATION
REPORT NUMBER
CAL- DMV- RSS- 09- 230
9. SPONSORING/ MONITORING AGENCY NAME( S) AND ADDRESS( ES)
Office of Traffic Safety
2208 Kausen Drive., Suite 300
Elk Grove, CA 95758- 7115
10. SPONSORING/ MONITORING
AGENCY REPORT NUMBER
11. SUPPLEMENTARY NOTES
12a. DISTRIBUTION/ AVAILABILITY STATEMENT
Unlimited
12b. DISTRIBUTION CODE
UL
13. ABSTRACT ( Maximum 200 words)
The Enhanced Negligent Operator Treatment Evaluation System ( ENOTES) provides periodic cost- effectiveness analyses of the California Negligent Operator Treatment System ( NOTS). The evaluation system is based on a comparison of the driving records of negligent operators ( drivers with multiple traffic conviction and/ or crash points) who are randomly assigned to an intervention or to a no- contact delayed treatment comparison group. Three levels of progressively more severe negligent operator interventions were evaluated in this manner; the fourth- level intervention ( the most severe) was evaluated indirectly by using data from the Level 3 intervention. Drivers at Levels 1 and 2 were eligible to receive the standard, non- alcohol letter or a letter based on the Transtheoretical Model ( TTM) of behavior change. The results found NOTS to be effective in reducing subsequent total crashes and citations of treated drivers. Consistent with prior evaluations of NOTS, the largest effects were found for the probation and probation violator sanctions, and the smallest effects were associated with the Level 1 and 2 letters. Within Levels 1 and 2, the TTM letters were found to be more effective than the standard letters in reducing subsequent traffic crashes and convictions. A cost effectiveness analysis of NOTS provided positive estimates at all four levels.
15. NUMBER OF PAGES
63
14. SUBJECT TERMS
Safety Program Effectiveness, Traffic Crashes, Crash Prevention, Traffic Law Violators, Warning Letters, Driver License Suspension, Negligence, Point Systems, Driver Improvement, Driver License Probation, Negligent Operators.
16. PRICE CODE
17. SECURITY CLASSIFICATION
OF REPORT
Unclassified
18. SECURITY CLASSIFICATION
OF THIS PAGE
Unclassified
19. SECURITY CLASSIFICATION
OF ABSTRACT
Unclassified
20. LIMITATION OF ABSTRACT
None
NSN 7540- 01- 280- 5500 Standard Form 298 ( 2- 89)
Prescribed by ANSI Std. Z39- 18
298- 102 Enhanced Negligent Operator Treatment Evaluation System
i
PREFACE
 
 
This project is part of the California Traffic Safety Program and was made possible through the support of the California Office of Traffic Safety, the State of California, and the National Highway Traffic Safety Administration. The report was prepared by the Research and Development Branch of the California Department of Motor Vehicles. The opinions, findings, and conclusions expressed in this publication are those of the author and not necessarily those of any or all of the supporting organizations named above.
Enhanced Negligent Operator Treatment Evaluation System
 
 
ACKNOWLEDGEMENTS
The author wishes to acknowledge with appreciation the individuals who contributed to this project. The study was conducted under the general direction of David DeYoung, Research Chief, and the supervision of Robert Hagge, Research Manager. Karen Coyle of the California Office of Traffic Safety was very helpful in guiding the project’s grant process. Douglas Rickard, Staff Services Analyst, helped type, organize, and proof the many drafts of this report. Senior Programmer Analysts Richard Viscuso and Kevin Nguyen developed and executed the computer programs to select and track drivers in the study. Associate Information System Analyst Edita Purnell oversaw the development and implementation of the program specifications and testing of the computer programs. Bill Rahlf, formerly Manager III in the department’s Licensing Policy Section, provided the NOTS program costing data used for the cost- effectiveness analysis.
Special acknowledgement is also extended to former Research and Development Branch staff member Robert A. Roberts, Research Program Specialist, who developed the theory- based intervention letters evaluated in this study and contributed heavily to the project’s research design.
Credit is also given to former Research and Development Branch Chief Raymond C. Peck and to former Research Program Specialist William C. Marsh for developing and implementing the two prior Negligent Operator Treatment System evaluation components known as the Post License Control Reporting and Evaluation System and the Negligent Operator Treatment Evaluation System.
ii
Enhanced Negligent Operator Treatment Evaluation System
EXECUTIVE SUMMARY
Background
The Enhanced Negligent Operator Treatment Evaluation System ( ENOTES) is the successor to the Negligent Operator Treatment Evaluation System ( NOTES), which provided decision makers with biennial assessments of the cost- effectiveness of the Department of Motor Vehicles’ Negligent Operator Treatment System ( NOTS) from 1976 through 1995. The present report is the first in an ongoing series of ENOTES reports evaluating NOTS that will be produced every few years depending on the needs of the department.
There are currently four NOTS intervention levels:
Level 1 – Warning letter ( W/ L)
Level 2 – Notice of intent to suspend ( N/ I) – a more severe warning letter
Level 3 – Probation with suspension hearing ( P/ H)
Level 4 – Probation- violator ( P/ V) suspensions and revocations
The Level 1 W/ L is sent when a driver accumulates two neg- op points in 1 year, four neg- op points in 2 years, or six neg- op points in 3 years. The letter gives a low- threat, general warning to the driver to change his/ her driving behavior to avoid future crashes and convictions and more severe licensing actions. There are two types of Level 1 W/ Ls. One is a standard letter triggered by a one– point conviction. The second is an alcohol letter triggered by a two- point ( major) conviction that warns of the dangers of drinking and driving.
The Level 2 N/ I letter is sent when a driver is one point below being classified as a prima facie negligent operator. That is, the letter is mailed when a driver accumulates three points in 1 year, five points in 2 years, or seven points in 3 years. The N/ I is a more severe warning than the Level 1 W/ L in that it threatens license suspension if the driver accumulates an additional point within the specified time windows. There are two types of N/ I: a standard letter and an alcohol letter.
A Level 3 P/ H intervention is imposed when a driver accumulates enough points to meet the legal definition of a negligent operator, which is four or more points in 1 year, six or more points
iii
Enhanced Negligent Operator Treatment Evaluation System
 
 
in 2 years, or eight or more points in 3 years. Level 3 drivers are allowed to schedule and attend a hearing, at which time probation and license suspension may be imposed. Probation is initially for 1 year and remains in effect during any license suspension within that period.
If the department receives any notice of failure to appear in court, countable traffic conviction, responsible police- reported crash, or any evidence of driving while suspended for a driver when he or she is on probation, a Level 4 P/ V intervention ( additional probation and/ or suspension time or possible revocation) will be imposed.
Several NOTES evaluations were conducted from 1976 through 1995. The primary NOTES findings are summarized below:
1.
Each NOTS intervention significantly reduced the rate of subsequent traffic citations. In general, the amount of the reduction increased with the severity of the intervention, with warning letters producing the smallest effect and probation- violator suspensions producing the largest effect.
2.
Overall, each NOTS intervention probably reduced the rate of subsequent crashes. However, the crash effects were smaller and less consistent than the citation effects, particularly the crash effects for warning letters, which in some evaluations did not reach statistical significance.
3.
The NOTS program was highly cost- beneficial in that the economic cost of the crashes prevented greatly exceeded the cost of the program. Somewhat paradoxically, the most cost- beneficial treatment ( warning letters) was the least effective in terms of crashes prevented per unit of treatment. However, this finding should not be surprising because the warning letters were extremely inexpensive and were applied to a large number of drivers.
4.
NOTS interventions prevented approximately 30,000 crashes from 1976 through 1994.
iv
Enhanced Negligent Operator Treatment Evaluation System
Current Study
The department terminated NOTES in 1995 and then replaced it in 2005 with ENOTES. The new evaluation system replicated the earlier analyses and also included new ones to evaluate whether the traffic safety benefits of the Level 1 and 2 non- alcohol warning letters can be increased through the use of the Transtheoretical Model ( TTM) of behavior change.
As a brief intervention, the TTM has a long and distinguished history of motivating narcotic abusers and alcoholics to change their high- risk behaviors. It was hypothesized for the present study that incorporating the TTM in the design and content of the NOTS intervention letters would also reduce crash and violation rates among negligent drivers.
Method
The present study used an experimental design to scientifically evaluate the effectiveness of Level 3 and 4 interventions in reducing crash and conviction rates and to assess whether the TTM letters were more ( or less) effective than the current standard letters.
The evaluation was based on the accumulated post- treatment driving records of 260,647 violators who were randomly assigned, within each NOTS level, to receive an intervention appropriate for their NOTS level or instead to a delayed- treatment comparison group. Within Level 1 ( W/ L) and Level 2 ( N/ I), treated drivers were randomly chosen to be sent either the TTM letter or the standard letter. The effect of Level 4 was evaluated indirectly using data from the Level 3 groups, who received a P/ H intervention.
Cox proportional hazards survival analysis was used to assess the efficacy of the treatment interventions by analyzing the number of days between intervention and first subsequent reported crash or citation. This technique made it possible to assess the separate and independent impact of each NOTS intervention level.
Results

NOTS was found to be effective in reducing subsequent total crashes and citations of treated drivers. Consistent with the prior NOTES evaluations, the largest effects were
v
Enhanced Negligent Operator Treatment Evaluation System
 
 
found for the probation and probation- violator sanctions, and the smallest effects were associated with the Level 1 and Level 2 letters.

Within Levels 1 and 2, the TTM letters were found to be more effective than the standard letters in reducing crashes. It was estimated that the NOTS program using only the TTM letters would have prevented 1,310 total crashes during FY 2007/ 08 at a cost of $ 690 per crash prevented. This is superior to the 1,001 total crashes that would have been prevented, at a program cost of $ 903 per crash, by the NOTS program using the standard letters. The estimated costs per crash prevented with NOTS using either type of letter are far below any recently published estimates for crash costs ( even for non- injury crashes), showing that the department’s NOTS program is highly cost- beneficial.

Similar results were found for total citations. It was estimated that by using the TTM letters, the NOTS program would have prevented 15,532 total citations at a cost of $ 61 per citation prevented. In contrast, it was estimated that by using the standard letters, NOTS would have prevented only 10,840 total citations at a cost of $ 72 per citation prevented.
Recommendations
Based on the findings in this evaluation and previous research cited in this report, the following recommendations are offered.

The department should continue all four levels of NOTS and re- evaluate the program every few years as needed.

The department should replace the standard non- alcohol letters at Levels 1 and 2 with the TTM letters.

The department should revise the alcohol letters at Levels 1 and 2 based on the TTM and evaluate their effectiveness in reducing crashes and citations.

The department should evaluate the effects of NOTS on fatal/ injury crashes when the sample sizes for the ENOTES groups increase sufficiently to do so. This evaluation will produce more precise estimates of the cost effectiveness of the NOTS interventions.
vi
Enhanced Negligent Operator Treatment Evaluation System
vii

There is some preliminary evidence from both the current and prior evaluations that Level 3 interventions may not be effective for drivers entering this level as the result of being cited for driving with a suspended/ revoked license. The department should consider investigating this further, and if the results validate the earlier findings ( and there is no reduction in traffic safety), should consider proposing legislation to permit it to deal with habitual traffic offenders through mandatory actions outside the standard point system.

The department should consider alternative and innovative ways to identify and treat pre- negligent drivers who, while not qualifying for a licensing action on the basis of their point count, exceed the crash risk level of prima facie negligent operators. The department’s in- progress study on the traffic safety impact of aggressive driving is already exploring this possibility for drivers who exhibit this behavior.
Enhanced Negligent Operator Treatment Evaluation System
 
TABLE OF CONTENTS 
PAGE
viii
PREFACE........................................................................................................................ .............. i
ACKNOWLEDGEMENTS............................................................................................................ ii
EXECUTIVE SUMMARY........................................................................................................... iii
INTRODUCTION................................................................................................................... ...... 1
Background..................................................................................................................... .......... 1
Negligent Operator Treatment Evaluation System.................................................................... 3
Enhanced Negligent Operator Treatment Evaluation System................................................... 4
METHOD......................................................................................................................... ............. 5
Subject Selection and Treatment Assignment............................................................................ 5
ENOTES Evaluation.................................................................................................................. 7
Assessing the Effects of Interventions on Crashes and Citations............................................ 10
RESULTS........................................................................................................................ ............ 14
Comparability of Intervention and Delayed- Treatment Groups.............................................. 14
Statistical Assumptions............................................................................................................ 15
Intervention Effects on Crashes............................................................................................... 16
Warning Letter ( Level 1 – W/ L)............................................................................................ 20
Notice of Intent ( Level 2 – N/ I)............................................................................................. 21
Probation Hearing ( Level 3 – P/ H)........................................................................................ 22
Probation- Violator Sanction ( Level 4 – P/ V)........................................................................ 22
Intervention Effects on Total Citations.................................................................................... 22
Warning Letter ( Level 1 – W/ L)............................................................................................ 23
Notice of Intent to Suspend ( Level 2 – N/ I).......................................................................... 23
Probation Hearing ( Level 3 – P/ H)........................................................................................ 28
Probation- Violator Sanctions ( Level 4 – P/ V)....................................................................... 28
Cost- Effectiveness Analysis..................................................................................................... 28
Warning Letter ( Level 1 – W/ L)............................................................................................ 31
Notice of Intent ( Level 2 – N/ I)............................................................................................. 32
Probation Hearing ( Level 3 – P/ H)........................................................................................ 34
Probation Violator Sanctions ( Level 4 – P/ V)....................................................................... 34
DISCUSSION/ RECOMMENDATIONS..................................................................................... 36
Study Limitations.................................................................................................................... 36
Discussion..................................................................................................................... .......... 36
Recommendations................................................................................................................ ... 37
REFERENCES..................................................................................................................... ....... 41 Enhanced Negligent Operator Treatment Evaluation System
 
TABLE OF CONTENTS ( continued)
 
ix
APPENDICES
NUMBER PAGE
A Report Titles and Dates of Prior California Department of Motor Vehicles Evaluations of the Negligent Operator Treatment System.................................................... 43
B Transtheoretical Model of Behavior Change........................................................................ 46
C Transtheoretical Model Level 1 Treatment Letter Including Specific Behavioral Change Strategies.................................................................................................................. 50
D Transtheoretical Model Level 2 Treatment Letter Including Specific Behavioral Change Strategies.................................................................................................................. 54
E Current NOTS Levels 1 and 2 Letters.................................................................................. 58
F Estimated Cost of Interventions............................................................................................ 61
LIST OF TABLES
1 ENOTES Group Assignment Process Using Random Numbers (. 000 to .999) to Create Three Groups at Both Levels 1 and 2, Two Groups at Level 3, and One Group at Level 4.............................................................................................................................. ... 6
2 Number of Subjects ( n) and Mean Number of Days in the ENOTES Study for Each Intervention Group Within Each NOTS Level...................................................................... 11
3 Intervention Effects on Total Crashes– Survival Analysis Results for Drivers Entering ENOTES from November 2005 through October 2007........................................................ 20
4 Intervention Effects on Total Citations– Survival Analysis Results for Drivers Entering ENOTES from November 2005 through October 2007......................................... 27
5 Results of Cost- Effectiveness Analysis of the Negligent Operator Treatment Program– Prevention of Crashes............................................................................................ 29
6 Results of Cost- Effectiveness Analysis of the Negligent Operator Treatment Program– Prevention of Citations........................................................................................... 30
F1 Estimated Reducible NOTS Program Cost for FY 2007/ 08.................................................. 63
Enhanced Negligent Operator Treatment Evaluation System
TABLE OF CONTENTS ( continued)
 
 
LIST OF FIGURES
NUMBER PAGE
1 Schematic of the ENOTES evaluation project......................................................................... 9
2 Survival chart for Level 1 W/ L ( cumulative percentage of crash- free drivers)..................... 17
3 Survival chart for Level 2 N/ I ( cumulative percentage of crash- free drivers)....................... 18
4 Survival chart for Level 3 P/ H ( cumulative percentage of crash- free drivers)...................... 19
5 Survival chart for Level 1 W/ L ( cumulative percentage of citation- free drivers)................. 24
6 Survival chart for Level 2 N/ I ( cumulative percentage of citation- free drivers)................... 25
7 Survival chart for Level 3 P/ H ( cumulative percentage of citation- free drivers).................. 26
x
Enhanced Negligent Operator Treatment Evaluation System
INTRODUCTION
Background
Almost all motor vehicle driver licensing agencies assign penalty points for various traffic- law violations and establish accumulated point counts at which licensing actions are taken. The primary objective of driver post licensing point systems is to identify drivers who are most likely to be involved in crashes and then to initiate driver improvement and license control actions to reduce their future crash risk. The existence of point systems and the threat of punishment also act as general deterrents to unsafe driving and the violation of traffic laws.
Section 12810.5a of the California Vehicle Code ( CVC) defines a prima facie negligent operator as any Class C vehicle ( usually passenger car) driver “ whose driving record shows a violation point count of four or more points in 12 months, six or more points in 24 months, or eight or more points in 36 months.” Minor traffic convictions ( e. g., for speeding or illegal turning) and crashes for which a driver is considered at least partially responsible by DMV count one point each. Major traffic convictions ( e. g., for drunk driving, reckless driving, and hit- and- run) count two points each. When the accumulated point count reaches a specified level, the driver is exposed to a “ treatment,” which can range from a warning letter at the lowest level to revocation of the driving privilege at the highest level.
CVC Sections 13800 and 14250 grant the department discretionary authority to take a variety of license control actions, including license suspension, against drivers who meet the CVC’s definition of a negligent operator. In addition, CVC Section 13950 requires the department to offer to drivers an administrative hearing when an action is taken under the negligent operator provisions, since the program is discretionary.
There are currently four Negligent Operator Treatment System ( NOTS) intervention levels:
Level 1 ─ Warning letter ( W/ L)
Level 2 ─ Notice of intent to suspend ( N/ I) – a more severe warning letter
Level 3 ─ Probation and suspension hearing ( P/ H)
Level 4 ─ Probation- violator ( P/ V) suspensions and revocations
1
Enhanced Negligent Operator Treatment Evaluation System
 
The Level 1 W/ L is sent when a driver accumulates two neg- op points in 1 year, four neg- op points in 2 years, or six neg- op points in 3 years. The letter gives a low- threat, general warning to the driver to change his/ her driving behavior to avoid future crashes and convictions and more severe licensing actions. Approximately 250,000 warning letters are mailed to drivers annually.
There are two types of letters at Level 1. One is a standard letter triggered by a 1- point conviction or responsible crash. The second is an alcohol letter triggered by a 2- point ( major) conviction. The assumption behind the alcohol letter is that most 2- point convictions involve some degree of alcohol use. The two letters are very similar in content, except that only the alcohol letter contains some brief wording warning of the dangers of drinking and driving.
The Level 2 N/ I letter is sent when a driver is one point below being classified as a prima facie negligent operator. Specifically, the letter is mailed when a driver accumulates three points in 1 year, five points in 2 years, or seven points in 3 years. The N/ I is a more severe warning than the Level 1 W/ L in that it threatens license suspension if the driver accumulates an additional point within the specified time windows. There are two types of Level 2 notices: a standard letter that is triggered by a 1- point incident and an alcohol letter that is triggered by a 2- point incident. Approximately 60,000 N/ I letters are mailed to drivers annually.
A Level 3 P/ H intervention is imposed when a driver accumulates enough points to meet the legal definition of a negligent operator, which is four or more points in 1 year, six or more points in 2 years, or eight or more points in 3 years. Level 3 drivers are allowed to schedule and attend a hearing, at which time probation and a license suspension may be imposed. If a driver does not request a hearing within 34 days of the mailing of the Level 3 notice, the probation and suspension actions go into effect. Probation is initially for 1 year and remains in effect during any license suspension within that period. Annually, the department treats approximately 40,000 drivers at Level 3.
If DMV receives any failure to appear ( FTA) notice, countable traffic conviction or responsible police- reported crash during the period of probation, or any evidence of driving during the period of suspension ( e. g., a driver- reported crash), a Level 4 P/ V intervention ( additional probation and/ or suspension time) will be imposed. The Level 4 treatment is normally a maximum of 6 months suspension for each of the first and second probation violations. The third such violation results in a 1- year revocation, followed by an additional year of revocation for each subsequent 2
Enhanced Negligent Operator Treatment Evaluation System
violation of probation. A Level 4 license suspension also extends probation by 1 year from the date of the triggering incident.
Negligent Operator Treatment Evaluation System
The Negligent Operator Treatment Evaluation System ( NOTES) was created to provide the DMV, the Department of Finance, and the Legislature with periodic evaluation reports on the NOTS program. 1 Peck and Healey ( 1995) presented a detailed review of NOTES from its inception in 1976 to its termination in 1995. The primary NOTES findings are summarized below:
1.
Each NOTS intervention significantly reduced the rate of subsequent traffic citations throughout the 20 years covered. In general, the amount of the reduction increased with the severity of the intervention, with warning letters producing the smallest effect and probation- violator suspensions producing the largest effect.
2.
Overall, each NOTS intervention probably reduced the rate of subsequent crashes. However, the crash effects were smaller and less consistent than the citation effects, particularly the crash effects for warning letters, which in some of the evaluations did not reach statistical significance or became significant only when the effects of warning letters at Level 1 and Level 2 were combined.
3.
The total number of crashes prevented increased with increases in the number of drivers receiving interventions.
4.
The NOTS program was highly cost- beneficial in that the economic cost of the crashes prevented greatly exceeded the cost of the program. Somewhat paradoxically, the most cost- beneficial treatment ( warning letters) was the least effective in terms of crashes prevented per unit of treatment. However, this finding shouldn’t be surprising because the warning letters were extremely inexpensive and were applied to large volumes of drivers. This result demonstrates that seemingly- trivial crash effects can become highly cost- beneficial when the net per- item dollar benefits are multiplied by a very large number of treated drivers.
1 Appendix A presents the report titles and dates. 3
Enhanced Negligent Operator Treatment Evaluation System
 
4
5.
NOTS interventions prevented approximately 30,000 crashes from 1976 through 1994.
Enhanced Negligent Operator Treatment Evaluation System
The department terminated the original NOTS evaluation system in 1995 and then resumed it in a new form in 2005. The new evaluation system is titled the Enhanced Negligent Operator Treatment Evaluation System ( ENOTES). One of the goals of ENOTES is to evaluate whether the traffic safety benefits of the Level 1 and Level 2 non- alcohol letters can be increased through the use of the Transtheoretical Model ( TTM) of behavior change. 2
The use of the TTM in brief interventions has a long and distinguished history of motivating the most resistant groups of narcotic abusers and alcoholics to change their high- risk behaviors. Given these successes, it is reasonable to assume that incorporating the TTM in the design and content of the NOTS intervention letters might also reduce crash and violation rates among negligent drivers.
ENOTES uses an experimental design to scientifically evaluate the effectiveness of Level 3 and Level 4 interventions in reducing crash and conviction rates and to assess whether the TTM letters are more ( or less) effective than the current standard letters. The prior NOTES evaluations found that the current administratively- crafted letters had marginal and inconsistent effectiveness in reducing crashes among treated drivers.
This is the first in an ongoing series of ENOTES reports that will be produced every few years depending on the needs of the department.
2 A detailed presentation of the TTM theory and its application to NOTS letters is provided in: Roberts, R. ( 2002). Application of behavior change theory to the development of an enhanced California negligent operator treatment evaluation system ( Report No. 199). Sacramento: California Department of Motor Vehicles. Enhanced Negligent Operator Treatment Evaluation System 
METHOD
This section describes the ENOTES study method. Some methodological details are reserved for the Results section because they are more understandable within the context of the study findings.
Before proceeding, it is suggested that the reader review the results of the prior NOTES evaluations ( summarized by Peck and Healey, 1995) and details of the current NOTS treatments and the newly designed letters using the TTM approach ( Roberts, 2002; Gebers & Roberts, 2004). Appendices B through E contain the current NOTS letters, the new TTM letters, and a brief introduction to the theory of behavior change supporting the TTM letters used in the present evaluation.
Subject Selection and Treatment Assignment
The department developed and maintains computer programs for ENOTES subject selection and treatment assignment. These programs create an electronic file containing the driver license numbers and driver record histories for all drivers receiving NOTS interventions.
Table 1 presents a schematic of the ENOTES subject assignment process that creates three groups at Level 1 and also at Level 2, two groups at Level 3, and one group at Level 4. The assignment of subjects to these groups was based on an electronically generated random fractional number ( from .000 to .999) for each subject that indicated the group to which the subject was to be assigned. The proportion of NOTS- treated drivers assigned to each group within each NOTS level is noted in the table. The group proportions were chosen to provide enough subjects in each group to be able to statistically detect any meaningful program effects. 3 The table identifies the following groups:
1.
D refers to drivers within each level who had the treatment delayed. Drivers assigned to this group served as the comparison group for the treatment group at that level. The driving records for subjects assigned to the delayed- treatment group were electronically monitored after the date of group assignment for 18 months or until the driver was involved in a responsible crash or received a conviction point. Upon the first update of a responsible crash or receipt of a conviction point, a delayed- treatment group driver would
3 The issue of statistical power and sample size is noted in the Results section. 5
Enhanced Negligent Operator Treatment Evaluation System
immediately receive the appropriate NOTS treatment at the current level. Inclusion of the delayed- treatment group was essential for determining whether the proposed changes to the NOTS system were more effective and cost- beneficial than the current NOTS interventions.
2.
T1 refers to drivers in Level 1 and Level 2 who received the new letters based on the Transtheoretical Model of behavior change.
3.
T2 refers to drivers who received the current treatments at Level 1 through Level 4.
4.
The shaded areas represent drivers who were randomly excluded from the evaluation at their current level. These drivers received the normal NOTS interventions appropriate for their point counts at their current level.
Table 1
ENOTES Group Assignment Process Using Random Numbers (. 000 to .999) to Create Three Groups at Both Levels 1 and 2, Two Groups at Level 3, and One Group at Level 4
Random number range
ENOTES study group (% of drivers within level)
NOTS treatment level
.000 - .039
.040 - .209
.210 - .409
.410 - .499
.500 - 849
.850 - 999
Level 1 ( W/ L)
D ( 4%)
T1 ( 35%)
T2 ( 15%)
Level 2 ( N/ I)
D ( 17%)
T1 ( 35%)
T2 ( 15%)
Level 3 ( P/ H)
D ( 20%)
T2 ( 50%)
Level 4 ( P/ V)
T2 ( 59%)
T1: Transtheorethical Model Treatment Group
T2: Standard Treatment Group
D: Delayed Treatment Group
Shaded: Rejects ( not selected for the study)
6
Enhanced Negligent Operator Treatment Evaluation System 
As indicated in the table, a subject could have qualified for assignment to a treatment group ( T1 or T2) at more than one level. However, a subject could not be assigned to a delayed- treatment group at more than one level. Additionally, the assignment process did not alter the Level 4 probation- violator sanctions ( i. e., all drivers received the prescribed standard interventions normally due them).
The driver records were also screened to identify and exclude from the evaluation any drivers who do not meet the specified subject- selection criteria. As previously stated, these drivers still received the NOTS interventions appropriate for their accumulated point counts. The excluded drivers are listed below.

Drivers of commercial vehicles ( Classes A and B)

Drivers with any of the following endorsements and certificates:
o
Ambulance
o
School bus
o
Farm labor vehicle
o
Youth bus
o
Public paratransit vehicle
o
Passenger transport vehicle
o
Vehicle carrying hazardous materials

Drivers with physical or mental ( P& M) conditions

Drivers with an X- prefix driver- record number ( essentially unlicensed drivers)

Drivers younger than 18 or older than 68.5 years of age when the incident triggering the
NOTS intervention was entered on the driver record
ENOTES Evaluation
Figure 1 illustrates the components of the ENOTES evaluation. As indicated in the figure, only the applicable standard ( non- alcohol) letters were evaluated at Level 1 and Level 2; the alcohol letters were not included in the evaluation. No distinction between standard and alcohol treatment formats is made at Level 3 and Level 4 because there are no separate alcohol interventions at these levels.
7
Enhanced Negligent Operator Treatment Evaluation System
As indicated in the figure, ENOTES uses an experimental approach ( random assignment of subjects to groups) to evaluate the first three levels of NOTS interventions. As was the case with NOTES, a direct evaluation of the Level 4 probation- violator sanction is not included as part of ENOTES. The main reason for this is that the department is not willing to adopt a blanket policy of delaying treatment against probation violators due to the potential risk involved. However, as described below, ENOTES still provides sufficient data to roughly estimate the independent effects of Level 4 intervention indirectly through a subsidiary analysis of Level 3 subjects at 12 and 18 months after intervention.
As described above, in each of the first three NOTS levels, a proportion of NOTS eligible drivers was randomly assigned to either an intervention group that received a prescribed NOTS intervention or a delayed- treatment group that did not receive a NOTS contact, unless they were involved in a subsequent countable incident. Since the assignment of drivers to receive an immediate or delayed intervention was random, the only systematic difference between the groups should have been the presence or absence of a NOTS intervention. This random assignment process should have minimized or eliminated any biasing influence of uncontrolled extraneous variables. In addition, consistent with the NOTES methodology, subjects remained in the intervention groups to which they were assigned whether or not they actually received the treatment ( e. g., a warning letter). This strategy was used to avoid a “ self- selection” bias that could have occurred if these subjects had been removed from the intervention groups while comparable drivers had to remain in the delayed- treatment groups because they could not be identified. Peck ( 1976) provides a detailed discussion of this topic.
The department’s normal sanctioning activities outside the NOTS program are not affected by a driver’s assignment to an intervention or delayed- treatment group in ENOTES. For example, drivers in the intervention and delayed groups may have their licenses suspended or revoked by the department acting in accordance with laws and policies governing other programs, ( such as suspending a driver for a DUI offense). The random assignment protocol used in this study would help ensure that these sanctioning activities outside the NOTS program would be experienced equally among drivers in the treatment and delayed- treatment groups, and thus not be an alternative rival hypothesis to that of the effects of the NOTS treatments.
8
Apply group treatments Extract driver record data Compare group crash and citation rates Delayed treatment group TTM treatment group Random assignment Random assignment Random assignment Population of drivers eligible for NOTS Drivers eligible for Level 1 standard warning letter ( W/ L) Drivers eligible for Level 3 probation/ suspension hearing ( P/ H) Drivers eligible for Level 2 standard notice of intent ( N/ I) Standard treatment group Delayed treatment group TTM treatment group Standard treatment group Delayed treatment group Standard treatment group Enhanced Negligent Operator Treatment Evaluation System
The driving records for the delayed- treatment groups are monitored electronically on the department’s DL Master File. When a delayed- treatment driver receives a countable incident ( i. e., a responsible crash or conviction), the driver is removed from the delayed- treatment group, and the appropriate NOTS action is initiated. For example, in the survival analysis of crashes ( described below), the number of days from ENOTES entry until a driver’s first subsequent crash was used to calculate the criterion measure for the evaluation, and, therefore, any intervention following a driver’s first crash after entering the delayed group would have had no effect on the outcome of the survival analysis. In addition, any driver who received an intervention after entering the delayed- treatment group was censored ( excluded from further analysis) immediately after the date of the intervention, which should have eliminated all intervention effects within the delayed- treatment group.
Assessing the Effects of Interventions on Crashes and Citations
This section presents the methods used to assess the intervention effects of NOTS on subsequent crashes and traffic citations. The techniques described here are essentially the same as those used in the prior NOTES evaluations.
Driver record extractions to obtain the criterion measures ( e. g., time to first subsequent crash) for Level 1 through Level 3 occurred on January 7, 2008. These data were for individuals entering NOTS between November 5, 2005 ( the first day of subject selection) and October 9, 2007. Only crashes and citations that occurred after the driver’s scheduled intervention date and before or on October 9, 2007 were considered as post- intervention criterion incidents. The 90- day buffer period between the last day of the evaluation period ( October 9, 2007) and the January 7, 2008 extract date was intended to provide enough time for almost all of the crashes and citations that occurred during the criterion period to be reported to DMV and updated on the driver record in the DL Master File. The sample size and average number of days in the study for each group are presented in Table 2.
10
Enhanced Negligent Operator Treatment Evaluation System 
Table 2
Number of Subjects ( n) and Mean Number of Days in the ENOTES Study for Each Intervention Group Within Each NOTS Level
NOTS treatment group level
Treatment group
n
Mean days in study
Level 1 ( W/ L)
TTM letter
112,597
352
Standard letter
48,585
353
Delayed
12,508
352
Level 2 ( N/ I)
TTM letter
25,572
352
Standard letter
11,222
359
Delayed
12,395
352
Level 3 ( P/ H)
Probation/ suspension
27,145
353
Delayed
10,623
355
A statistical technique known as survival analysis was employed to compare the proportions of crash- and citation- free drivers in the intervention groups to those in the delayed- treatment groups at various points in time following their scheduled intervention dates. This technique was used for ENOTES ( and NOTES) because it is one of the most appropriate and powerful methods of evaluating program effects when subjects are observed for differing amounts of time in a study. In addition, the technique is desirable because it can eliminate the effects of higher- level interventions that occur when a driver in an intervention group at one level subsequently accumulates enough neg- op points or traffic incidents to become eligible for a treatment at a higher level.
For example, contacts at Level 2 through Level 4 are considered higher- level interventions for drivers previously treated at Level 1. Since the effects of any higher- level interventions are 11
Enhanced Negligent Operator Treatment Evaluation System
present in the raw data for the intervention groups at Level 1 through Level 3, it was important to use a statistical technique to eliminate these effects and make it possible to estimate the separate impact of each level of intervention.
The type of survival analysis used in this study ( and in NOTES) is known as Cox regression. This technique analyzes the time to an event or outcome, which in the present study is the number of days to first subsequent reported crash or citation. An important feature of Cox regression is that covariates can be used in the model. To be consistent with the prior NOTES studies, demographic and prior driving histories were used as covariates in an attempt to control for any residual bias that may not have been controlled through the study’s random assignment process. In building the hierarchical Cox regression model, the study- group ( intervention vs. delayed- treatment) indicator variable was entered into the model after the covariates were entered to assess whether the intervention affected the criterion measure ( crashes or citations) after controlling or adjusting for any effects of the covariates.
SAS PROC LIFETEST was used to produce Kaplan- Meier estimates of the sample survivor and hazard plots. These plots were used to examine the raw survivor hazard functions for the intervention and comparison groups, providing information on the groups’ survival ( crash- and citation- free) status over the time period in which they were evaluated. The plots were also used to assess whether the hazards for the groups were proportional over time, a requirement of Cox regression.
The proportional hazards assumption was further examined by calculating the differences between the log of the negative- log of the survivor functions for the different levels of each covariate and intervention variable in the model as suggested by Cantor ( 2003). A final statistical check of the proportional hazards assumption was performed by using SAS PROC PHREG to run and plot Cox regression models to test for possible interactions ( e. g., intervention group by time). As discussed by Hosmer and Lemeshow ( 1999) and Allison ( 1995), a violation of the proportional hazards assumption is not fatal to the analysis but rather is simply one of several possible model misspecifications that can be appropriately handled by leaving the significant interaction( s) in the final models.
The final Cox regression models were built by entering all of the covariates entered as a block and then entering the treatment ( group- indicator) variable. This approach enabled the causal effect of the intervention on the criterion measure to be evaluated. In this report, an observed
12
Enhanced Negligent Operator Treatment Evaluation System 
13
intervention effect is considered statistically significant if the associated p- value is .10 or lower, meaning that one would expect to find an effect that large or larger in no more than 10 out of 100 random samples if the intervention had no real effect on the criterion. All statistical tests and the resulting probability estimates are two- tailed to allow for the possibility of treatment effects being either positive or negative.
An assessment of the effect of Level 4 on crashes and citations was conducted in a different manner than the analyses for Level 1 through Level 3. The independent effects of Level 4 could not be directly evaluated because there was no delayed- treatment comparison group at Level 4. Therefore, it was necessary to indirectly evaluate the unique Level 4 effects using data for the Level 3 study groups. As explained in more detail later in this report, a comparison of the Level 3 groups on subsequent crashes at 18 months after the scheduled intervention reveals the combined effects of treatments at Level 3 and Level 4. The unique effect of Level 4 treatment was estimated by subtracting from the combined effects the unique crash effect of Level 3 treatment which was measured 12 months after the scheduled intervention. The remainder represents the unique effect of the Level 4 intervention. The same approach was used to evaluate the unique effect of Level 4 on citations. Enhanced Negligent Operator Treatment Evaluation System
RESULTS
Comparability of Intervention and Delayed- Treatment Groups
In any evaluation that estimates program effects by comparing a group that received an intervention to a comparison or control group that did not receive the intervention, it is vital that the groups be equivalent before treatment. If the random subject- assignment process is successful, the only systematic differences between the treatment and comparison groups will be due to the presence or absence of the intervention. Unfortunately, random assignment processes that are thought to be inherently sound may occasionally produce groups that before treatment are significantly different from one another on one or more variables that correlate with the criterion measure. If this occurs, the problem can be remedied to some extent by statistically adjusting the criterion measures to control the influence of these preexisting differences.
A statistical procedure known as multiple logistic regression was used to verify the initial comparability of the intervention and delayed- treatment groups at each NOTS level. This procedure can detect any statistically significant pre- existing differences between the groups on a variety of demographic and driver- record variables. The study groups were compared on the following variables in each of three separate multiple logistic regression analyses ( one for each of the first three NOTS treatment levels):
1.
Sex ─ Coded 1 for men and 0 for women.
2.
Age ─ As of the scheduled intervention date.
3.
Days in Study ─ Number of days from the scheduled intervention date to the end of the data- collection period.
4.
Motorcycle License ─ Coded 1 for drivers with a motorcycle license, endorsement, or permit and 0 for all others.
5.
Prior 36- Month Minor Citations ─ Number of convictions, failure- to- appear ( FTA) notices, and traffic violator school ( TVS) citation dismissals for minor violations accruing within 36 months before the subject’s intervention date. A conviction for a
14
Enhanced Negligent Operator Treatment Evaluation System 
minor violation would have a negligent operator point count of one. The violation date was used to place a citation in time.
6.
Prior 36- Month Major Citations ─ Same as in # 5 above except the count is for major violations. A conviction for a major violation ( e. g., driving under the influence of alcohol or drugs, reckless driving, hit and run) is assigned a negligent operator point count of two.
7.
Prior 36- Month Total Crashes ─ Number of crashes reported by drivers and/ or law enforcement that occurred during the 36 months before the scheduled intervention date.
The results of the multiple logistic regression analyses indicate that the treatment and comparison groups did not differ significantly on the set of variables described above at any of the three NOTS treatment levels tested. The probability ( p- value) for the Likelihood Ratio Chi- Square test of the global null hypothesis was .68 for Level 1, .55 for Level 2, and .32 for Level 3. These probability estimates are well above the .10 significance ( alpha) level adopted for these analyses. Since the overall statistical tests did not find significant differences between the groups, the results of the separate tests conducted on the individual variables are not included here. These unreported results did not reveal a consistent source of bias in favor of either group.
Statistical Assumptions
Before assessing the effectiveness of the NOTS interventions, procedures were implemented to check the accuracy of the proportional hazards assumption, which is one of the main assumptions underlying the Cox regression model. This assumption specifies that the hazard rates for the levels or groups defined by each predictor variable are proportional over time. This assumption was initially examined for both the total crash and total citation criteria by creating difference plots of the log negative log function for each level of a predictor variable ( SAS, 2006). Additionally, SAS PROC LIFETEST was used to produce sample survivor and hazard plots, which provided a visual indication as to the direction and magnitude of any possible non- proportionality.
The final test for checking the proportional hazards assumption was to create interaction terms of each predictor by time ( i. e., days to first subsequent total crash or total citation) in a Cox 15
Enhanced Negligent Operator Treatment Evaluation System
regression model and determine whether the interaction was statistically significant. When the statistical interaction tests were run for models consisting of the covariates and treatment indicator variables in the Cox regression models predicting days to first subsequent total crash and days to first subsequent total citations, the results showed that none of the interactions were statistically significant. Therefore, the proportional hazards assumption was not violated for either outcome criteria evaluated in the present study.
Intervention Effects on Crashes
Figures 2, 3, and 4 illustrate the results of the survival analyses for Level 1 through Level 3, respectively. Figures 2 and 3 are based on reported crashes occurring within 6 months after intervention. Figure 4 is based on reported crashes occurring within 1 year after intervention. These covariate- adjusted survival curves obtained from the Cox proportional hazards regression models are graphs of the cumulative percentages of drivers in the intervention and delayed- treatment groups remaining crash- free (“ surviving”) following intervention. In each figure, the vertical distances between the curves for the intervention groups and the curve for the delayed- treatment group indicate the cumulative intervention effects.
Table 3 summarizes the survival analysis results for reported crashes. The estimated effect for each treatment was computed by subtracting the failure rate for the delayed group from the failure rate for the associated intervention- treatment group. The p- value for each effect is also shown in the table.
A 6- month follow- up period was selected for the Level 1 and Level 2 analyses because the prior NOTES evaluations indicated ( as do the present data) that the duration of the program effects was about 6 months. Similarly, the selection of the 12- month period for evaluating the effects at Level 3 was based upon findings in prior NOTES reports ( and the present data) that indicated the effects of Level 3 on crashes lasted for at least 12 months.
16
9596979899100PERCENT CRASH- FREETTM StandardDelayed 9596979899100PERCENT CRASH- FREETTM StandardDelayed Enhanced Negligent Operator Treatment Evaluation System 
949596979899100PERCENT CRASH- FREE xxxxxP/ HDelayed
19
Enhanced Negligent Operator Treatment Evaluation System
Table 3
Intervention Effects on Total Crashes– Survival Analysis Results for Drivers Entering ENOTES from November 2005 through October 2007
NOTS Level
Treatment group
Sample size
Number of crash- free
drivers per 1,000
( S: survival rate)
Number of crash- involved
drivers per 1,000
( F: failure rate;
F = 1,000 - S)
Intervention
effecta
Percentage
effectb
p
Level 1 ( W/ L) c
TTM
112,597
958.95
41.05
- 1.99
- 4.62
.095
Standard
48,585
957.97
42.03
- 1.01
- 2.35
.105
Delayed
12,508
956.96
43.04
Level 2 ( N/ I) c
TTM
25,572
949.68
50.32
- 4.40
- 8.04
.078
Standard
11,222
947.52
52.48
- 2.24
- 4.09
.088
Delayed
12,395
945.28
54.72
Level 3 ( P/ H) d
P/ H
27,145
950.17
49.83
- 8.38
- 14.40
.005
Delayed
10,623
941.79
58.21
a The intervention effect is equal to the difference in failure rates ( Fintervention - Fdelayed). A minus sign indicates that the intervention group had a lower proportion of crash- involved drivers than did the delayed group. The intervention effect was computed from the failure rates before those rates were rounded for the table, and thus the effect cannot always be derived from the table rates. b The intervention effect expressed as a percentage. c The survival and failure rates are for the 6 months following the scheduled intervention date. d The survival and failure rates are for the 12 months following the scheduled intervention date.
A detailed presentation of findings for total crashes is provided for each intervention level below.
Warning Letter ( Level 1 – W/ L)
The results indicate that the TTM warning letter probably reduced total crashes. At 6 months, the TTM group had a survival rate of 958.95 per 1,000 drivers, compared to 956.96 per 1,000 for the delayed- treatment group ( a difference of 1.99 per 1,000). This effect is statistically significant ( p = .095) and represents a 4.62% reduction in crash risk ( i. e., 4.62% fewer crash- involved drivers) attributable to the TTM warning letter. The survival rates for the standard letter treatment group and the comparison group were 957.97 and 956.96 respectively ( a difference of
20
Enhanced Negligent Operator Treatment Evaluation System 
1.01 per 1,000 drivers). This observed 2.35% reduction in crash risk, however, barely failed to reach statistical significance ( p = .105). 4
Although not displayed in Table 3, a comparison of the survival rates for TTM versus the standard NOTS letters ( 958.95 versus 957.97 per 1,000 drivers, respectively) yielded a difference of 0.98 per 1,000 drivers. This difference is not statistically significant ( p = .102), although it is suggestive that the TTM letter is superior to the standard letter.
Notice of Intent ( Level 2 – N/ I)
The results shown in Figure 3 and Table 3 indicate that each intervention group had a lower crash risk than did the delayed- treatment group during the first 6 months following the scheduled intervention date.
Specifically, at 6 months the group receiving the TTM notice of intent letter had a survival rate of 949.68 per 1,000 drivers, which was higher than the 945.28 rate for the delayed- treatment group ( a difference of 4.40 per 1,000). This effect is statistically significant ( p = .078) and represents an 8.0% reduction in crash risk.
The standard notice of intent letter also appears to have reduced crash risk. The group receiving this treatment had a survival rate of 947.52, which is higher than the 945.28 rate for the delayed- treatment group ( a difference of 2.24 per 1,000). This effect is also statistically significant ( p = .088) and represents a 4.09% reduction in the crash involvement rate for drivers who were sent this letter.
A comparison of the survival rates for the TTM and NOTS intent letters ( 949.68 and 947.52 per 1,000, respectively) found a difference of 2.16 in favor of the TTM letter. This effect is statistically significant ( p = .081), indicating the superiority of the TTM letter in reducing crash risk.
4 When testing the difference of intervention effects between two or more groups, statistical power is the probability that a statistical test will produce a significant result if there is a real difference of a given size. In the present study, power analyses were computed for the differences between the intervention and delayed- treatment groups on the total crash and citation criteria. The power for each comparison equaled or exceeded .60.
21
Enhanced Negligent Operator Treatment Evaluation System
Probation Hearing ( Level 3 – P/ H)
Figure 4 shows the survival curves for the Level 3 intervention and delayed- treatment groups. The difference between these two curves is much larger than that between the curves for the groups compared at Level 1 and Level 2. Table 3 shows that the survival rate at 12 months for the P/ H intervention group was 950.17 per 1,000 drivers, which is 8.38 higher than the 941.79 rate for the delayed- treatment group. This difference is statistically significant ( p = .005) and represents a 14.40% lower crash risk for the P/ H group. This effect represents the impact of all interventions applied at Level 3, including suspending the licenses of those who do not request or do not appear for the hearing.
Probation- Violator Sanction ( Level 4 – P/ V)
As previously stated, there is no way to directly estimate the independent effect of the Level 4 intervention because there was no delayed- treatment group established at Level 4. Therefore, it is necessary to compute a less precise estimate of the Level 4 effect using data available for the Level 3 intervention and comparison groups. This was done by computing the difference between the two groups’ crash involvement rates 18 months after intervention ( for drivers with at least 18 months of post- intervention driver record data) and then comparing that difference to the difference between the two group’s survival rates at 12 months. This difference ( between the 12- month and 18- month effects) is assumed to roughly represent the unique effect of the Level 4 interventions. The results of these computations are presented below.
At 18 months after intervention, the average crash rate for the P/ H group was approximately 95.76 per 1,000 drivers, which is 21.24 ( 18.15%) lower than the 117.00 rate for the delayed- treatment group over the same period. This difference is statistically significant ( p = .040). This effect is much larger than the effect found at 12 months after intervention ( 8.38 per 1,000 drivers). The difference between these two estimated effects ( 12.86 per 1,000 drivers) represents an estimated effect of the Level 4 interventions. While not precise, this is the best possible estimate of the Level 4 effect that can be made with the available data.
Intervention Effects on Total Citations
Traffic citations analyzed for this evaluation consist of reported convictions of violations, notices of failure- to- appear in court ( FTAs) or pay a fine ( FTPs), and dismissals of citations for drivers
22
Enhanced Negligent Operator Treatment Evaluation System 
who attend a traffic violator school ( TVS). Figures 5, 6, and 7 show the survival curves from the survival analyses of citations for Level 1 through Level 3, respectively. The adjusted survival curves, obtained from the results of the Cox proportional hazards regression models, are based on total citations occurring up to 6 months after the scheduled intervention date. These survival curves display the cumulative percentage of drivers in the groups remaining citation- free following intervention. The vertical distances between the lines for the intervention groups and the line for the delayed- treatment group indicate the cumulative effects of each of the two types of warning letters.
The specific findings for each intervention level are presented below.
Warning Letter ( Level 1 – W/ L)
As displayed in Table 4, the results for the TTM warning letter indicate that this treatment reduced the citation rates for those who were scheduled to receive it. Specifically, the citation failure rate of 179.42 per 1,000 drivers for the TTM group was 9.49 ( 5.02%) lower that the 188.91 citation failure rate for the delayed- treatment group. This effect is statistically significant ( p = .082).
Drivers sent the standard warning letter had a citation failure rate of 184.11 per 1,000 drivers, which is 4.80 ( 2.54%) lower that the 188.91 citation failure rate for drivers in the delayed- treatment group. However, this observed reduction is not statistically significant ( p = .128).
A comparison of the survival rates for the TTM and standard groups found a difference of 4.69 cited drivers per 1,000 drivers, which is statistically significant ( p = .001). This finding suggests that the TTM letter yields a significantly greater benefit in terms of reducing citations than does the standard letter.
Notice of Intent to Suspend ( Level 2 – N/ I)
Drivers sent the TTM intent letter were cited at a rate of 237.18 per 1,000 drivers, which is 15.91 ( 6.29%) lower than the 253.09 rate for the delayed- treatment group. This effect estimate is statistically significant ( p = .001), indicating it is highly probable that the TTM notice letter reduced citations.
23
TTM StandardDelayed80828486889092949698100PERCENT CITATION- FREE 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 7984899499PERCENT CITATION- FREETTM StandardDelayed 7782879297PERCENT CITATION- FREEPHDelayed Enhanced Negligent Operator Treatment Evaluation System 
Table 4
Intervention Effects on Total Citations– Survival Analysis Results for Drivers Entering ENOTES from November 2005 through October 2007
NOTS Level
Treatment groupa
Sample size
Number of citation- free
drivers per 1,000
( S: survival rate)
Number of cited
drivers per 1,000
( F: failure rate;
F = 1,000 - S)
Intervention
effectb
Percentage
effectc
p
Level 1 ( W/ L)
TTM
112,597
820.58
179.42
- 9.49
- 5.02
.082
Standard
48,585
815.89
184.11
- 4.80
- 2.54
.128
Delayed
12,508
811.09
188.91
Level 2 ( N/ I)
TTM
25,572
762.82
237.18
- 15.91
- 6.29
.001
Standard
11,222
754.97
245.03
- 8.06
- 3.18
.066
Delayed
12,395
746.91
253.09
Level 3 ( P/ H)
P/ H
27,145
785.30
214.70
- 34.72
- 13.92
.000
Delayed
10,623
750.58
249.42
a The survival and failure rates shown for each NOTS level are for the 6 months following the driver's scheduled intervention date. b The intervention effect is equal to the difference in failure rates ( Fintervention - Fdelayed). c The intervention effect expressed as a percentage. A minus sign indicates that the intervention group had a lower proportion of cited drivers than did the delayed group. The intervention effect was computed from the failure rates before those rates were rounded for the table, and thus the effect cannot always be derived from the table rates.
Drivers sent the standard notice letter had a citation rate of 245.03 per 1,000 drivers, which is 8.06 ( 3.18%) lower than the rate for the delayed- treatment group. The reduction is statistically significant ( p = .066), indicating that the standard notice letter was also effective in reducing citations.
A comparison of the rates of cited drivers for the TTM and standard groups sent notice of intent letters ( 237.18 versus 245.03, respectively) found a difference of 7.85 per 1,000 drivers. This difference is statistically significant ( p = .091), indicating that the TTM notice letter was more effective than the standard notice letter in reducing citations during the first 6 months after intervention.
27
Enhanced Negligent Operator Treatment Evaluation System
Probation Hearing ( Level 3 – P/ H)
Subjects in the P/ H group were cited for violations at a rate of 214.70 per 1,000 cited drivers during the 6 months after intervention. This rate is 34.72 ( or 13.92%) lower than the 249.42 rate for subjects in the delayed- treatment group. This difference is highly statistically significant ( p = .000) indicating that the combination of Level 3 interventions resulted in a reduction in negligent driving.
Probation- Violator Sanctions ( Level 4 – P/ V)
As displayed in Table 4, the difference between the 6- month survival rates for the Level 3 intervention and delayed- treatment groups is 34.72 per 1,000 drivers. As was explained earlier, this estimate represents the independent effect of the Level 3 interventions and excludes the effect of higher- level interventions. When the two groups were compared at 12 months after intervention ( using only subjects for which 12 months of post- intervention driver record data were available), it was found that the delayed group had 285.72 more citations per 1,000 drivers than did the treatment group. This difference was highly statistically significant ( p < .000). This estimate represent the combined effects of Level 3 and Level 4 intervention, since subjects were not censored or removed from the analysis as they were in the Level 3 survival analysis. The difference of 251.00 between the unique 6- month effect of Level 3 and the 12- month combined effects of Level 3 and Level 4 represents the upper- bound unique effect of Level 4. While not precise, this estimate provides strong evidence that the Level 4 intervention reduced the proportion of drivers cited for traffic violations.
Cost- Effectiveness Analysis
Tables 5 and 6 summarize the results of the cost- effectiveness analyses of the NOTS program based on the estimated prevention of crashes and citations attributed to the various NOTS interventions.
28
Enhanced Negligent Operator Treatment Evaluation System 
Table 5
Results of Cost- Effectiveness Analysis of the Negligent Operator Treatment Program– Prevention of Crashes
NOTS level
Intervention
Number of drivers
treated in
FY 2007/ 08
Reducible cost during
FY 2007/ 08
Number
of crashes
prevented
Cost per crash
prevented
Level 1 ( W/ L)
212,328
$ 184,767
TTM
436
$ 424
Standard
223
$ 829
Level 2 ( N/ I)
43,815
$ 38,128
TTM
200
$ 191
Standard
104
$ 367
Level 3 ( P/ H)
31,777
$ 525,779
303
$ 1,735
Level 4 ( P/ V)
13,909
$ 155,244
371*
$ 418
All levels ( using TTM letters)
301,829
$ 903,918
1,310
$ 690
All levels ( using standard letters)
301,829
$ 903,918
1,001
$ 903
* Part of the reduction in crashes attributed to Level 4 may have been caused by earlier Level 3 interventions.
During FY 2007/ 08, the program treated 301,829 drivers at an estimated reducible cost of $ 903,918. ( Appendix F provides the estimated cost of each NOTS level.) If the estimated effects of NOTS interventions on crashes are real, then it is estimated that the NOTS program using the TTM letters at Level 1 and Level 2 would have prevented 1,310 crashes in FY 2007/ 08. This yields a cost- effectiveness ratio of $ 690 per crash prevented. Similarly, it is estimated that the NOTS program using the standard letters at Level 1 and Level 2 would have prevented 1,001 crashes in FY 2007/ 08, yielding a cost- effectiveness ratio of $ 903 per crash prevented. Those results indicate that the department would recover its costs if the average cost of a prevented crash is at least $ 690 if the TTM letters were used or at least $ 903 if the standard letters were used. The National Safety Council ( 2007) estimates that the average property damage crash, including minor injuries, cost approximately $ 7,500 in 2007. The cost for California crashes would be somewhat higher because the state’s labor costs are higher than the 29
Enhanced Negligent Operator Treatment Evaluation System
Table 6
Results of Cost- Effectiveness Analysis of the Negligent Operator Treatment Program– Prevention of Citations
NOTS Level
Intervention
Number of drivers
treated in FY 2007- 08
Reducible program cost in
FY 2007- 08
Number of citations
prevented
Cost avoidance at higher levels due to citation prevention
Net program cost for interventiona
Net program cost
per citation
prevented
Level 1 ( W/ L)
212,328
$ 184,767
TTM
2,519
$ 30,089
$ 154,678
$ 61
Standard
1,284
$ 15,398
$ 169,369
$ 132
Level 2 ( N/ I)
43,815
$ 38,128
TTM
934
$ 13,083
$ 25,045
$ 27
Standard
477
$ 6,754
$ 31,374
$ 66
Level 3 ( P/ H)
31,777
$ 525,779
1,765
$ 17,423
$ 508,356
$ 288
Level 4 ( P/ V)
13,909
$ 155,244
7,314
$ 81,749
$ 73,495
$ 10
All levels ( using TTM letters)
301,829
$ 903,918
12,532
NAb
$ 761,574c
$ 61
All levels ( using standard letters)
301,829
$ 903,918
10,840
NAb
$ 782,594c
$ 72
a The net program cost represents the estimated savings achievable if the program at that level were cancelled. In that case, cost avoidance credited to that level would not be realized at higher levels. Therefore, projected savings from eliminating a level should be reduced by the resulting increased costs at higher levels. b Cost avoidance figures are not summed across levels because the results would be paradoxical. For example, the cost avoidance credited to Level 2 results from the increase in interventions at Levels 3 and 4 that would occur if interventions at Level 2 were totally eliminated. However, if interventions at Level 2 were eliminated, then much of the cost avoidance credited to Level 1 would disappear. c Represents cost savings to DMV if the entire NOTS program were eliminated.
national average. Obviously, inclusion of crash costs involving more severe injuries and fatalities would produce an average cost far in excess of $ 7,500. Based on this crash- cost estimate, it is clear that the cost of crashes prevented by NOTS far exceeds the cost of the program.
The citations prevented by NOTS provide additional cost savings, since a reduction in citations results in fewer higher- level interventions. In FY 2007/ 08, NOTS interventions are credited with the following savings through the prevention of citations and the associated avoidance of higher- level interventions: $ 142,344 if TTM letters are used and $ 121,324 if the standard NOTS letters are used.
30
Enhanced Negligent Operator Treatment Evaluation System 
The detailed results of the cost- effectiveness analysis for crashes and citations at each intervention level are presented below.
Warning Letter ( Level 1 – W/ L)
The evaluation results provide some evidence that the warning letters reduced crashes. The estimated effect of the TTM warning letters during the first 6 months is statistically significant. In addition, the estimated effect of the standard W/ L nearly reached statistical significance ( p = .105) and is therefore suggestive of a real treatment effect. For purposes of comparing the two types of letters on cost- effectiveness, these estimated effects were assumed to be valid.
As displayed in Table 3, the TTM W/ L was estimated to have reduced the number of crash- involved drivers by 1.99 per 1,000 drivers treated at Level 1. This represents a reduction of 423 crash- involved drivers during FY 2007/ 08. It was determined that crash- involved drivers treated with the TTM letter have an average of 1.03 crashes during the 6 months following intervention. Based on this, it is estimated that reducing the number of crash- involved drivers by 423 would have prevented 436 crashes in the first 6 months after intervention ( 423 x 1.03 = 436). The department would therefore have spent $ 424 on TTM warning letters for each crash prevented ($ 184,767 ÷ 436 = $ 424).
The standard NOTS W/ L was estimated to have reduced the number of crash- involved drivers by 1.01 per 1,000 drivers treated at Level 1. This represents a reduction of 214 crash- involved drivers during FY 2007/ 08. It was determined that the crash- involved drivers who were sent the standard warning letter had an average of 1.04 crashes during the first 6 months after intervention. Based on this finding, it is estimated that reducing the number of crash- involved drivers by 214 would have prevented 223 crashes in the first 6 months after intervention ( 214 x 1.04 = 223). The department therefore would have spent $ 829 on the standard W/ L for each crash prevented ($ 184,767 ÷ 223 = $ 829).
As reported in the Results section, there is evidence that the W/ Ls also reduced citations. Specifically, the TTM letter reduced the number of cited drivers by 9.49 per 1,000 treated drivers. If all 212,328 drivers treated in Level 1 had been sent the TTM W/ L, it is estimated that the number of cited drivers would have been reduced by 2,015 during FY 2007/ 08 (. 00949 x 212,328 = 2,015). It was determined that an average of 1.25 citations were issued to cited drivers during the first 6 months after intervention with the TTM W/ L. Based on this, it is 31
Enhanced Negligent Operator Treatment Evaluation System
estimated that 2,519 citations would have been prevented by sending 212,328 TTM W/ Ls during FY 2007/ 08 ( 1.25 x 2,015 = 2,519).
The standard W/ L is estimated to have reduced the number of cited drivers by 4.80 per 1,000 drivers sent this letter. If the standard W/ L had been sent to all 212,328 Level 1 drivers, it is estimated that the number of cited drivers would have been reduced by 1,019 during FY 2007/ 08 (. 00480 x 212,328 = 1,019). Drivers who were sent the standard W/ L had an average of 1.26 citations during the first 6 months after intervention. Based on this, it is estimated that 1,284 citations would have been prevented by sending 212,328 standard W/ Ls during FY 2007/ 08 ( 1.26 x 1,019 = 1,284).
The department spent 87 cents in reducible costs per W/ L ($ 184,767 in total) to send letters during FY 2007/ 08. However, the W/ Ls also reduced costs by preventing citations and consequently reducing the number of interventions at Level 2 through Level 4.
It is estimated that the TTM W/ L would have saved $ 30,089 by preventing 1,479 Level 2; 1,072 Level 3; and 990 Level 4 interventions. 5 Offsetting the total cost with this saving, the net cost of the TTM W/ Ls would have been $ 154,678 for FY 2007/ 08. Since the avoidance of higher- level interventions by using the W/ Ls is already reflected in the numbers of interventions at Level 2 through Level 4, there is no need to adjust the cost of the entire NOTS program to reflect the overall cost avoidance achieved by the W/ Ls.
It is estimated that the standard W/ L would have saved $ 15,398 by preventing 757 Level 2; 549 Level 3; and 507 Level 4 interventions, resulting in a net cost of the current W/ L of $ 169,369 for FY 2007/ 08.
Notice of Intent ( Level 2 – N/ I)
The evaluation results indicate that the TTM and standard N/ I letters reduced crash involvements during the 6 months following treatment intervention. Specifically, it is estimated that the TTM N/ I intervention reduced the number of crash- involved drivers by 4.40 per 1,000 interventions. Applying this effect to all 43,815 Level 2 drivers treated during FY 2007/ 08, it is estimated that
5 The procedures used in the cost effectiveness analysis to estimate the number of higher- level treatments avoided are contained in a series of internal working papers and spreadsheets in the possession of the author. 32
Enhanced Negligent Operator Treatment Evaluation System 
the TTM N/ I letter would have resulted in 193 fewer of the drivers being involved in a crash than if they had not been sent a Level 1 treatment letter. Based on the average number of crashes per crash- involved drivers ( 1.04), it is estimated that 200 crashes would have been prevented in the first 6 months after intervention with the TTM N/ I. This represents a cost of $ 191 per crash prevented, given that it cost $ 38,128 to send N/ I letters during FY 2007/ 08.
It is estimated that the standard NOTS N/ I letter reduced the number of crash- involved drivers by 2.24 per 1,000 interventions. If all 43,815 Level 2 drivers treated during FY 2007/ 08 had been sent this letter, it is expected that 98 fewer drivers would have crashed. In addition, given that crash- involved drivers had an average of 1.06 crashes, 104 fewer crashes would have occurred in the first 6 months after intervention. This represents a cost of $ 367 per crash prevented.
The evaluation also found the TTM and standard NOTS N/ I letters were each effective in reducing citations.
In the TTM treated sample, the N/ I letter reduced cited drivers by 15.91 per 1,000 drivers. If all 43,815 Level 2 drivers had received this treatment, it is estimated that there would have been 697 fewer cited drivers and, with such drivers having an average of 1.34 citations, 934 fewer citations issued.
In the sample treated with the standard letter, the N/ I reduced cited drivers by approximately 8.06 per 1,000 interventions. If the standard N/ I letter had been sent to all 43,815 Level 2 drivers, it is estimated that this would have decreased the number of cited drivers by about 353 and the number of citations by 477 ( based on such drivers having an average of 1.35 citations).
The reduction in citations achieved by the N/ I letters saved program costs by enabling higher- level interventions to be avoided. During FY 2007/ 08, these TTM N/ I letters could have saved $ 13,083 by avoiding 487 Level 3 and 450 Level 4 interventions. Since the Department spent only $ 38,128 on the N/ I treatments, sending the TTM N/ I letter would have resulted in a net cost of $ 25,045 for treated drivers after subtracting the cost savings resulting from the reduction in higher- level interventions.
It is estimated that the standard N/ I would have saved $ 6,754 by preventing 251 Level 3 and 232 Level 4 interventions. The net cost of the standard N/ I is estimated to have been $ 31,374 for FY 2007/ 08.
33
Enhanced Negligent Operator Treatment Evaluation System
Because these reductions in interventions are already reflected in counts of interventions at Level 3 and Level 4 during FY 2007/ 08, there is no need to adjust the overall program costs for these savings.
Probation Hearing ( Level 3 – P/ H)
The P/ H interventions had a significant effect on crashes in the first year following treatment. The difference between the intervention and delayed- treatment groups’ survival curves at 12 months is 8.38 crash involvements per 1,000 drivers. It was determined that drivers in the Level 3 sample with at least 12 months in the study had an average of about 1.14 crashes per crash- involved driver in that period. Applying this average to the 31,777 drivers treated at Level 3 in FY 2007/ 08, it is estimated that the interventions prevented 266 (. 00838 x 31,777 = 266) drivers from being involved in 303 crashes ( 266 x 1.14 = 303).
The Level 3 intervention also saved money by reducing citations. At 6 months, the intervention was found to have reduced citations by 34.72 per 1,000 treated drivers. Cited drivers in this group had an average of 1.6 citations each. Based on these figures, it is estimated that the P/ H prevented 1,765 citations ( 1,103 x 1.60 = 1,765) among 1,103 drivers (. 03472 x 31,777 = 1,103). The survival analysis found that 89% of these citations occurred while the drivers were on probation. Based on this finding, it is estimated that the Level 3 treatment resulted in the avoidance of 1,571 Level 4 interventions and $ 17,423 in associated program costs.
Probation Violator Sanctions ( Level 4 – P/ V)
As stated earlier, the difference between the average crash rates for the Level 3 intervention and delayed- treatment groups 18 months after intervention was 21.24 per 1,000 drivers ( for drivers with at least 18 months of post- intervention driver record data). This estimate represents the combined effects of Level 3 and Level 4 interventions, since no subjects were censored for the comparison of group averages. Applying this effect estimate to the 31,777 drivers treated at Level 3 in FY 2007/ 08 yields an estimate of 675 crashes prevented by the Level 3 and Level 4 interventions (. 02124 x 31,777 = 675). The independent contribution of Level 4 is estimated to be 371 crashes saved, which is derived by subtracting the 304 crashes attributed to Level 3 ( see Table 5) from the 675 total. The associated monetary cost per crash prevented for this effect is $ 418.
34
Enhanced Negligent Operator Treatment Evaluation System 
As stated earlier, the combined effects of Level 3 and Level 4 on citations are represented by the difference between the group averages for citations at 12 months after initial intervention ( for drivers with at least 12 months of post- intervention driving records). This difference was found to be 286 citations per 1,000 drivers. Applying this effect ( prior to rounding) to the 31,777 drivers treated at Level 3 in FY 2007/ 08 yields a total of 9,079 citations jointly prevented by the P/ H and P/ V interventions. The number of citations prevented by Level 4 interventions was obtained by subtracting the estimated number prevented by Level 3 ( 1,765) from the reduction in citations attributed jointly to Level 3 and Level 4 ( 9,079 - 1,765 = 7,314). This effect translates into a total cost avoidance of $ 81,749 attributed to Level 4.
35
Enhanced Negligent Operator Treatment Evaluation System
DISCUSSION/ RECOMMENDATIONS
Study Limitations
Before discussing the evaluation results and offering recommendations, the following limitations of the study need to be mentioned so that appropriate conclusions can be drawn.
1.
General deterrence. The present evaluation applies only to the specific deterrence of interventions in relation to drivers who have become eligible for one or more of the NOTS interventions, which are fewer than 3% of all licensed California drivers. However, it is possible that the existence of the NOTS program and the associated threat of interventions produces a general deterrent effect on all drivers. Any possible crash or citation reduction due to a general deterrent effect is not considered in this report and, in fact, would be extremely difficult to measure.
2.
External effects. The program’s effectiveness may be moderated by a host of external variables, such as lag time in reporting crashes and citations, or court- related interventions, such as traffic violator schools and penalties. ENOTES cannot determine the potential effectiveness of the NOTS interventions in an ideal world. Rather, it can only measure the program effects in the environment in which the program operates.
3.
Secondary costs not included. Costs incurred by the negligent operators treated under NOTS, for example the expenses of taking time off work and traveling to DMV, are not considered in this evaluation. The rationale behind this is that the driver should bear the economic consequences resulting from his/ her negligent behavior; therefore, these costs should not be counted against the program. Although this point of view could be reversed with a change in philosophical orientation, the reader should keep in mind that these secondary costs are excluded from the reported cost figures at this time.
Discussion
ENOTES was conceived with an emphasis on improving and assessing, through a rigorous scientific design, the effectiveness of the Level I and Level II standard, non- alcohol letters. A close inspection of the standard letters that were being used at Levels 1 and 2 revealed that they are impersonal, difficult to read, and nearly indistinguishable from each other. In addition, it was
36
Enhanced Negligent Operator Treatment Evaluation System 
not unusual for the department’s administrators to make uncoordinated changes to the wording of these letters based on what they anecdotally believed would be effective rather than adhering to a valid behavior- modification paradigm supported by scientific research findings.
One of the goals of ENOTES is to determine and assess what motivates negligent operators and pre- negligent operators to change their hazardous driving behaviors. For more than 50 years, traffic safety researchers have been attempting to answer that question by evaluating a variety of treatments and commenting on the elements needed to promote behavior change through the use of letters sent to negligent operators. In addition, research in the field of psychology was discovering that treatment elements similar to those identified by traffic safety experts formed the motivational basis of behavior change generally. These similar treatment elements form the foundation of the Transtheoretical Model ( TTM) of behavior change ( Prochaska & DiClemente, 1982).
The hypothesis tested in the present evaluation is that the TTM, when incorporated into a treatment intervention letter and applied to negligent operators, will show potent effects in reducing rates of traffic crashes and citations. In addition to evaluating the effects of the letters at Levels 1 and 2, ENOTES also evaluated the effectiveness of licensing actions ( e. g., probation and suspension) taken at NOTS Levels 3 and 4.
The current evaluation found NOTS to be effective in reducing subsequent total crashes and citations of treated drivers. Consistent with the prior NOTES evaluations, the largest effects were found for the probation and probation violator sanctions, and the smallest effects were associated with the Level 1 and 2 letters. Within Levels 1 and 2, the TTM letters were found to be more effective than the standard letters in reducing subsequent traffic crashes and convictions among treated drivers. The following recommendations are based on these and other findings in this report, and the results of previous research as cited.
Recommendations
1.
The department should continue all four levels of the NOTS program. The present report provides strong evidence that the overall program is very cost effective. In the prior section, it was estimated that the NOTS program using the TTM letters would have prevented 1,310 crashes during FY 2007/ 08 at a cost of $ 690 per crash prevented. This is in contrast to the 1,001 crashes prevented at a program cost of $ 903 per crash prevented by the NOTS program 37
Enhanced Negligent Operator Treatment Evaluation System
using the standard letters. The estimated program costs per crash prevented with NOTS using either type of letter are far below any recently published estimates for crash costs ( even for non- injury crashes). Similar savings associated with NOTS reducing total citations were also demonstrated in this evaluation.
2.
The department should replace the standard non- alcohol letters at Levels 1 and 2 with the TTM letters. It was estimated that by using the new TTM letters, the overall NOTS program would have prevented 1,310 crashes in FY 2007/ 08 at a cost of $ 690 per crash prevented. Using the standard letter instead would have prevented only 1,001 ( 309 fewer) crashes at a cost of $ 902 per crash prevented. Similar results were found for total citations. Specifically, it is estimated that using the new TTM letters across all levels, the NOTS program would have prevented 12,532 total citations at a cost of $ 61 per citation prevented. In contrast, it was estimated that NOTS would have prevented only 10,840 ( 1,692 fewer) total citations at a cost of $ 72 per citation prevented using the standard letters.
3.
The department should revise the alcohol letters at Levels 1 and 2 based on the TTM and then evaluate whether they are more effective than the current alcohol letters in reducing total crashes and total citations, as well as alcohol- related incidents. The prior NOTES studies found very small ( and sometime negative) effects associated with the current alcohol letters. A possible explanation for these disappointing effects is that, with the exception of one or two brief sentences explaining the risks of drinking and driving, the alcohol letters are nearly identical to the standard letters. This raises serious doubts about their potential efficacy in treating the drinking- driver subpopulation, a group that is normally recalcitrant to treatment interventions. Modifying the alcohol letters based on the TTM is supported by research showing the effectiveness of TTM- based interventions in producing positive behavior change across a wide spectrum of individuals who are highly resistant to intervention and change, including alcoholics.
4.
When the sample sizes in the ENOTES groups increase sufficiently, the department should evaluate the effects of NOTS on fatal/ injury crashes. The sample sizes available for the present evaluation were not large enough to reliably measure this effect due to the relative infrequency of fatal/ injury crashes among treated drivers. However, as new subjects are added to the ENOTES groups, it will be possible in future evaluations to assess the effect of each NOTS level on fatal/ injury crashes. This will allow more precise estimates to be made of the cost effectiveness of NOTS interventions.
38
Enhanced Negligent Operator Treatment Evaluation System 
5.
The department should consider including in subsequent ENOTES evaluations an assessment of the impact of Level 3 interventions on drivers cited for driving with a suspended/ revoked license per California Vehicle Code Section 14601. Gebers and Roberts ( 2004) reported that 56% of Level 3 interventions are triggered as the result of a 2- point conviction and that almost half of these 2- point convictions are for driving with a suspended or revoked license. Prior NOTES evaluations as well as preliminary analyses of data used in the current evaluation provide some evidence that Level 3 interventions may not be effective in reducing crashes and citations among drivers already under a suspension or revocation action. If subsequent and more formal ENOTES analyses were to validate these findings, it may be advisable for the department to seek legislation that would permit it to deal with habitual traffic offenders through mandatory actions outside of the standard point system. This change would reduce costs by removing these drivers from Level 3 contact. However, it would be initiated only if it can be shown that it would not reduce traffic safety.
6.
The department should investigate the feasibility and desirability of modifying the assignment of points for various types and combinations of violations and crashes involving aggressive driving. The department has already initiated preliminary research in this area and is currently developing a statewide definition of aggressive driving based on an assessment of the crash risk posed by drivers with single and multiple prior aggressive- driving violations and crash involvements. The safety impact of such behaviors has typically been assessed by determining how often the individual behavior or violation is the primary factor contributing to crashes. However, very little research has been done to evaluate the linkage between multiple aggressive driving violations and future crash risk. The lack of an objective risk prediction model incorporating longitudinal patterns of aggressive driving makes it impossible to develop coherent, consistent, effective, and defensible sanctions and countermeasures to treat these high- risk drivers.
The department’s in- progress empirical study of aggressive driving is analyzing the historical driving records of a large representative sample of California drivers to determine what patterns and combinations of driving behaviors thought to be aggressive in nature would be good predictors that drivers’ who exhibit them would have a future crash risk greater than that posed by prima facie negligent operators treated at Level 3. Establishing that chronic aggressive driving tends to lead to high future crash risk would provide justification for taking away the licensing privileges of these drivers earlier than otherwise would be required under the existing NOTS point- count structure. Modifying NOTS to increase the swiftness
39
Enhanced Negligent Operator Treatment Evaluation System
40
and severity of sanctions against high- risk aggressive drivers is also supported explicitly in the California Strategic Highway Safety Plan.
7.
The department should convene a task force to review the above recommendations and develop implementation plans as deemed necessary. The task force should also identify opportunities for improving the training of driver safety personnel, operating procedure manuals, and action criteria in an effort to increase the cost- effectiveness of NOTS. Enhanced Negligent Operator Treatment Evaluation System 
REFERENCES
Allison, P. D. ( 1995). Survival analysis using SAS, a practical guide. Cary, NC: SAS Institute Inc.
Cantor, A. B. ( 2003). SAS survival analysis techniques for medical research, second edition. Cary, NC: SAS Institute Inc.
Gebers, M. A., & Roberts, R. A. ( 2004). Characteristics of negligent operators in California ( Report No. 209). Sacramento: California Department of Motor Vehicles.
Hosmer, D. W., & Lemeshow, S. ( 1999). Applied survival analysis. New York: John Wiley & Sons, Inc.
National Safety Council. ( 2007). Estimating the costs of unintentional injuries. April 15, 2009 from http:// www. nsc. org/ resources/ issues/ estcost. aspx
Peck, R. C. ( 1976). Toward a dynamic system of driver improvement program evaluation. Human Factors, 18( 5), 493- 506.
Peck, R. C., & Healey, E. J. ( 1995). California’s negligent operator treatment program evaluation system, 1976- 1995: An overview of findings and program improvements ( Report No. 155). Sacramento: California Department of Motor Vehicles.
Prochaska, J. O., & DiClemente, C. C. ( 1982). Transtheoretical therapy: Toward a more integrative model of change. Psychotherapy: Theory, Research, and Practice, 19, 276- 288.
Roberts, R. ( 2002). Application of behavior change theory to the development of an enhanced California negligent operator Treatment Evaluation system ( Report No. 199). Sacramento: California Department of Motor Vehicles.
SAS. ( 2006). Survival analysis using the proportional hazards model -- instructor based training. Cary, NC: SAS Institute Inc.
41
Enhanced Negligent Operator Treatment Evaluation System
APPENDICES
42
Enhanced Negligent Operator Treatment Evaluation System 
Appendix A
Report Titles and Dates of Prior California Department of Motor Vehicles Evaluations of the Negligent Operator Treatment System 6
6 The title of the evaluation system changed after 1981 from Post License Control and Evaluation System ( PLCRES) to Negligent Operator Treatment Evaluation System ( NOTES). Copies of the referenced reports are available upon request from the department’s Research and Devlopment Branch.
43
Enhanced Negligent Operator Treatment Evaluation System
Report Titles and Dates of Prior California Department of Motor Vehicles Evaluations of the Negligent Operator Treatment System
1.
Post Licensing Control Reporting and Evaluation System: Negligent Operator Program Effectiveness. Periodic Status Report. No. 1, October 1976.
2.
Post Licensing Control Reporting and Evaluation System: Negligent Operator Program Costs and Effectiveness. Periodic Status Report. No. 2, March 1977.
3.
Post Licensing Control Reporting and Evaluation System: Negligent Operator Program Costs and Effectiveness. Periodic Status Report. No. 3, March 1978.
4.
Post Licensing Control Reporting and Evaluation System: Negligent Operator Program Costs and Effectiveness. Periodic Status Report. No. 4, February 1979.
5.
Post Licensing Control Reporting and Evaluation System: Negligent Operator Program Costs and Effectiveness. Periodic Status Report. No. 5, April 1980.
6.
Post Licensing Control Reporting and Evaluation System: Negligent Operator Program Costs and Effectiveness. Periodic Status Report. No. 6, December 1980.
7.
Post Licensing Control Reporting and Evaluation System: Negligent Operator Program Costs and Effectiveness. Periodic Status Report. No. 7, ( Summary) December 1981.
8.
Negligent Operator Treatment Evaluation System: Progress Report, March 1985.
9.
Negligent Operator Treatment Evaluation System: Program Effectiveness Report # 1, December 1985.
10
. Negligent Operator Treatment Evaluation System: Program Effectiveness Report # 2, December 1986.
11.
Negligent Operator Treatment Evaluation System: Program Effectiveness Report # 3, December 1987.
44
Enhanced Negligent Operator Treatment Evaluation System 
45
12.
Negligent Operator Treatment Evaluation System: Program Effectiveness Report # 4 ( Detailed Findings), December 1988.
13.
Negligent Operator Treatment Evaluation System: Program Effectiveness Report # 5 ( Detailed Findings), December 1990.
14.
Negligent Operator Treatment Evaluation System: Program Effectiveness Report # 6 ( Detailed Findings), December 1992.
15.
Negligent Operator Treatment Evaluation System: Program Effectiveness Report # 7 ( Summary of Findings), June 1995. Enhanced Negligent Operator Treatment Evaluation System
Appendix B
Transtheoretical Model of Behavior Change
46
Enhanced Negligent Operator Treatment Evaluation System
Transtheoretical Model of Behavior Change
People change. To the behavioral scientist, this knowledge sustains practice, but treatment efforts that aim to change behavior are often narrowly focused, and ignore the general processes of change that all people go through. Prochaska and DiClemente ( 1982) adopted a more universal approach by studying the steps traversed by individuals in the course of unassisted self- change efforts and, in the process, discovered an underlying, systematic process capable of predicting readiness to change. Their subsequent research confirmed that the change phenomenon progresses through the same steps with or without professional assistance ( Prochaska and DiClemente, 1984).
The stages through which individuals pass in the process of changing a behavior include: precontemplation, contemplation, determination, action, maintenance, and relapse. Each stage describes a person's readiness to change and specifies effective strategies to motivate the individual to move toward the next stage. In this context, motivation can be defined as the probability a person will persevere in a change strategy. In order to develop more effective treatments for negligent operators, the treatments must address the following issues at each stage of change.
• Precontemplation: Raise doubts about the advisability of continuing the hazardous behaviors.
• Contemplation: Influence the decisional balance away from the status quo by presenting reasons to change and stressing risks associated with a decision not to change.
• Determination: Encourage change with suggestions regarding courses of actions that will lead to positive change.
• Action: Promote change by offering assistance in plan development.
• Maintenance: Help identify and implement strategies to prevent relapse.
• Relapse: Assist reentry into the change process as soon as possible.
Miller and Brown ( 1991) reported that brief interventions are potent agents for change because their major impact is motivational. Specifically, the authors believe these brief interventions elicit commitments from subjects to try changing their behaviors and to persevere in their efforts. Previous research has identified three types of elements useful to the change process: General elements necessary to any change strategy; Early Stage elements to promote movement through precontemplation, contemplation, and determination; and Late Stage elements to elicit movement through the action, maintenance, and relapse phases. The General and Early Stage elements listed below are most relevant to this research.
Effective Elements For Any Intervention
Supplying systematic feedback: Provide clear knowledge of the present . situation for change to occur.
Stressing personal responsibility: This can be stated implicitly or explicitly, but the message is the same, " If change is to occur, you are the one who has to do it."
Providing direct advice: Clear advice has been shown to be very effective with behaviors that are hard to change. In some cases, providing specific goals has been successful but, in others, the opposite is true. There appears to be personality differences at work in the way specific goals are tolerated.
47
Enhanced Negligent Operator Treatment Evaluation System
Offering choice of strategy: Enhances intrinsic motivation by allowing the negligent operator to freely choose a course of action.
Expressing empathy: Communicates respect for the driver as a person. The letter is a blend of support and consultation.
Strengthening self- efficacy: The goal is to persuade the driver that he or she can make a successful change in the problem area. If a person is persuaded of a serious and threatening condition, but perceives no way in which change is feasible, the result is likely to be defensiveness rather than behavior change.
Early Stage Transitions
Consciousness Raising: Drivers in the precontemplation stage are not even thinking about changing their driving behaviors. They deny having poor driving habits and may even blame other drivers or law enforcement for their violations. Providing information that raises a concern about their hazardous driving habits will engender doubt about their complacency.
Self- Reevaluation: Doubt causes a " cognitive dissonance" between behavior and self- image. The treatment should view dissonance as an opportunity to prompt drivers to align their self- images with responsible driving behaviors.
Dramatic Relief: The treatment should foster the identification, experiencing, and expression of emotions related to the risk and the safer alternatives as a means of promoting change. The treatment then must lower the elevated emotions with a reminder that the risk is within the control of the driver. If the treatment leaves the negligent operator in a heightened state of arousal, a feeling of helplessness may give rise to resentment and recalcitrance toward authority.
Environmental Reevaluation: The treatment should assist the driver to reflect upon the consequences of the behavior for other people. The' driver should be left with doubt about the opinions of those who reinforce the negligent operator's current driving practices.
Social Liberation: The treatment should help the negligent operator to understand that the social norms are changing in the direction of supporting responsible driving with the goal of increasing traffic safety.
References:
Prochaska, J. O. & DiClemente, C. C. ( 1982). Transtheoretical therapy: Toward a more integrative model of change. Psychotherapy: Theory, Research, and Practice, 19, 276- 288.
Prochaska, J. O. & DiClemente, C. C. ( 1984). The Transtheoretical approach: Crossing traditional boundaries of therapy. Homewood, IL: Dow Jones/ Irwin.
Miller, W. R. & Brown, 1. M. ( 1991). Self- regulation as a conceptual basis for the prevention and treatment of addictive behaviours, in: Heather, N., Miller, W. R., & Greeley, J. ( Eds.) Self- control and the Addictive Behaviours, PP. 3- 79 ( Sydney, Maxwell Macmillan Publishing Australia.
48
Enhanced Negligent Operator Treatment Evaluation System
49
CHANGING HAZARDOUS BEHAVIOR
MOTIVATIONAL INTERVENTIONS
EFFECTIVE ELEMENTS FOR ANY INTERVENTION
• Systematic Feedback
• Stress Personal Responsibility
• Provide Direct Advice
• Offer Choice of Strategy
• Express Empathy
• Strengthen Self Efficacy
STRATEGIES IMPORTANT TO PROMOTE CHANGE PROCESSES
Early Stage Transitions Late Stage Transitions
• Consciousness Raising • Contingency Management
• Self- Reevaluation • Helping Relationships
• Dramatic Relief • Counter Conditioning
• Environmental Reevaluation • Self- Liberation
• Social Liberation • Stimulus Control Enhanced Negligent Operator Treatment Evaluation System
Appendix C
Transtheoretical Model Level 1 Treatment Letter Including Specific Behavioral Change Strategies
50
  Enhanced Negligent Operator Treatment Evaluation System 
AUGUST 29, 2003
TINA MARIE PULLANO DRIVER LICENSE NO. I4071280
2415 1ST AVENUE
SACRAMENTO, CA 95818
DEAR CALIFORNIA DRIVER:
DRIVERS WITH CRASHES AND TRAFFIC CONVICTIONS ON THEIR RECORDS ARE AT GREATER RISK OF CAUSING FUTURE CRASHES. YOUR RECENT RECORD OF BAD DRIVING ( SEE BELOW) PLACES YOU AT INCREASED RISK OF CAUSING CRASHES, INJURY AND DEATH. WE DO NOT WANT YOU TO SUFFER THOSE CONSEQUENCES AND WANT TO HELP YOU AVOID THEM.
WE UNDERSTAND MAT YOU MAY BELIEVE YOU ARE A GOOD DRIVER, AND YET, YOUR DRIVING
RECORD IS MUCH WORSE THAN THE AVERAGE CALIFORNIA DRIVER. WHILE YOU MAY BE A GOOD
AND SAFE DRIVER MOST OF THE TIME, YOUR RECORD REFLECTS AT LEAST MOMENTARY LAPSES IN DRIVING JUDGMENT. AT HIGHWAY SPEEDS, A MOMENT'S CARELESSNESS CAN BECOME A TRAGEDY. GOOD, CARING PEOPLE WHO MAKE CARELESS DECISIONS WHILE DRIVING CAN CAUSE INJURY OR DEATH.
IN OUR EFFORT TO URGE YOU TO DRIVE SAFER, WE ARE OFFERING YOU A CHOICE. YOU CAN CHOOSE TO PREVENT FURTHER ACTION FROM DMV BY AVOIDING ADDITIONAL TRAFFIC CONVICTIONS AND BY NOT CAUSING ANY CRASHES. HOWEVER, IF YOU CHOOSE TO CONTINUE YOUR UNSAFE DRIVING, THE PENALTIES WILL INCREASE AND EVENTUALLY LEAD TO PROBATION, SUSPENSION OR REVOCATION OF YOUR DRIVING PRIVILEGE.
WE BELIEVE YOU ARE CAPABLE OF MAKING A CHANGE TO BECOME A SAFER DRIVER, BUT IT IS UP TO YOU TO DO SO. IF YOU DO NOT WANT TO CHANGE, YOUR DRIVING WILL CONTINUE TO PRESENT A RISK TO YOURSELF AND OTHER ROAD USERS. IF YOU WANT TO CHANGE AND BECOME A SAFER, MORE RESPONSIBLE DRIVER, YOU CAN. WE HOPE THAT YOU WILL DECIDE TO CHANGE, BUT THE CHOICE IS YOURS.
IF THERE IS A DISCREPANCY IN YOUR DRIVING RECORD, YOU MAY CONTACT THE SACRAMENTO DEPARTMENT OF MOTOR VEHICLES AT ( 916) 657- 9074.
POINT COUNT DATA
DOES NOT NEED TO BE BOXED
DEPARTMENT OF MOTOR VELES
HICDIVISION OF DRIVER SAFETY
ENOTS Level 1 Treatment Letter 51
Enhanced Negligent Operator Treatment Evaluation System
NOTS Level 1 Treatment Letter/ Transtheoretical Model Table
The first letter has three objectives: 1) To help the negligent operator understand there is a problem; 2) To promote the idea that change is possible; and 3) To encourage those already contemplating change to continue in that direction.
We do not know what stage of change Level negligent operators occupy. Therefore, the main thrust is to use all general and early stage change techniques to increase the probability that the driver will move to the next stage, and closer to permanent change.
DRIVERS WITH CRASHES AND TRAFFIC CONVICTIONS ON THEIR RECORDS ARE AT GREATER RISK OF CAUSING FUTURE CRASHES. ( CONSCIOUSNESS RAISING) ( PERSONAL RESPONSIBILITY) YOUR RECENT RECORD OF BAD DRIVING ( SEE BELOW) PLACES YOU AT INCREASED RISK OF CAUSING CRASHES, INJURY AND DEATH. ( FEEDBACK) ( CONSCIOUSNESS RAISING) ( SELF REEVALUATION) ( ENVIRONMENTAL REEVALUATION) WE DO NOT WANT YOU TO SUFFER THOSE CONSEQUENCES AND WANT TO HELP YOU AVOID THEM. ( EMPATHY) ( HELPING RELATIONSHIP) ( PERSONAL – RESPONSIBILITY)
After setting the stage in the first paragraph, the goal here is to assist the negligent operator to align self image with safe driving behaviors by creating cognitive dissonance between self- image and the current driving record.
WE UNDERSTAND THAT YOU MAY BELIEVE YOU ARE A GOOD DRIVER, AND YET, YOUR DRIVING RECORD IS MUCH WORSE THAN THE AVERAGE CALIFORNIA DRIVER. ( FEEDBACK) ( SELF REEVALUATION) ( SOCIAL LIBERATION) WHILE YOU MAY BE A GOOD AND SAFE DRIVER MOST OF THE TIME, YOUR RECORD REFLECTS AT LEAST MOMENTARY LAPSES IN DRIVING JUDGMENT. ( SELF EFFICACY) ( SELF REEVALUATION) ( CONSCIOUSNESS RAISING) ( PERSONAL RESPONSIBILITY) AT HIGHWAY SPEEDS, A MOMENT'S CARELESSNESS CAN BECOME A TRAGEDY. ( CONSCIOUSNESS RAISING) ( ENVIRONMENTAL REEVALUATION) ( PERSONAL RESPONSIBILITY). GOOD, CARING PEOPLE WHO MAKE CARELESS DECISIONS WHILE DRIVING CAN CAUSE INJURY OR DEATH. ( CONSCIOUSNESS RAISING) ( SELF REEVALUATION) ( ENVIRONMENTAL REEVALUATION) ( PERSONAL RESPONSIBILITY)
This paragraph introduces choice and the idea that the driver is responsible for the consequences that result from driving habits. In the process, cognitive dissonance that was introduced in the previous paragraph is further developed and elevated.
IN OUR EFFORT TO URGE YOU TO DRIVE SAFER, WE ARE OFFERING YOU A CHOICE. ( HELPING RELATIONSHIP) ( CHOICE) ( EMPATHY). YOU CAN CHOOSE TO PREVENT FURTHER ACTION FROM DMV BY AVOIDING ADDITIONAL TRAFFIC CONVICTIONS AND BY NOT CAUSING ANY CRASHES. ( CHOICE) ( DRAMATIC RELIEF) ( PERSONAL RESPONSIBILITY) ( CONSCIOUSNESS RAISING) HOWEVER, IF YOU CHOOSE TO CONTINUE YOUR UNSAFE DRIVING, THE PENALTIES W1LL INCREASE AND EVENTUALLY LEAD TO PROBATION, SUSPENSION OR REVOCATION OF YOUR DRIVING PRIVILEGE. ( CHOICE) ( PERSONAL RESPONSIBILITY) ( CONSCIOUSNESS RAISING) ( SELF REEVALUATION) ( ENVIRONMENTAL REEVALUATION)
NOTS Level 1 Treatment Letter/ Transtheoretical Model Table 52
  Enhanced Negligent Operator Treatment Evaluation System 
53
Choice is a strong motivator for recalcitrant persons. This short paragraph is loaded with choices that are linked with personal responsibility, self- efficacy, and dramatic relief.
Choices serve double- duty in terms of the TTM because inherent in the concept of choice is the reality of personal responsibility. In addition, making a choice is an act of self efficacy that leads to dramatic relief.
WE BELIEVE YOU ARE CAPABLE OF MAKING A CHANGE TO BECOME A SAFER DRIVER, BUT IT IS UP TO YOU TO DO SO. ( SELF- EFFICACY) ( SELF REEVALUATION) ( PERSONAL RESPONSIBILITY) ( CHOICE) IF YOU DO NOT WANT TO CHANGE, YOUR DRIVING WILL CONTINUE TO PRESENT A RISK TO YOURSELF AND OTHER ROAD USERS. ( CHOICE) ( PERSONAL RESPONSIBILITY) ( ENVIRONMENTAL REEVALUATION) IF YOU WANT TO CHANGE AND BECOME A SAFER, MORE RESPONSIBLE DRIVER, YOU CAN. ( CHOICE) ( SELF REEVALUATION) ( SOCIAL LIBERATION) ( PERSONAL RESPONSIBILITY) ( SELF- EFFICACY) ( DRAMATIC RELIEF) WE HOPE THAT YOU WILL DECIDE TO CHANGE, BUT THE CHOICE IS YOURS. ( CHOICE) .( PERSONAL RESPONSIBILITY) ( SELF EFFICACY)
The warning letter ends by reinforcing DMV's desire to be helpful.
IF THERE IS A DISCREPANCY IN YOUR DRIVING RECORD, YOU MAY CONTACT THE SACRAMENTO DEPARTMENT OF MOTOR VEHICLES AT ( 916) 657- 9074. ( HELPING RELATIONSHIP) ( DRAMATIC RELIEF)
NOTS Level 1 Treatment Letter/ Transtheoretical Model Table Enhanced Negligent Operator Treatment Evaluation System
Appendix D
Transtheoretical Model Level 2 Treatment Letter Including Specific Behavioral Change Strategies
54
  Enhanced Negligent Operator Treatment Evaluation System 
AUGUST 29, 2003
TIMOTHY WILLIAM BRICKER DRIVER LICENSE NO I4071930
2415 1ST AVENUE ( 01)
SACRAMENTO, CA 95818
DEAR CALIFORNIA DRIVER:
DUE TO THE LATEST ENTRIES TO YOUR DRIVER RECORD ( SEE BELOW), YOU ARE NOW AMONG CALIFORNIA'S WORST DRIVERS, ONE OF THOSE WHO CAUSE THE MOST CRASHES, INJURIES, AND DEATHS.
ONE MORE POINT ADDED TO YOUR ALREADY POOR DRIVING RECORD AND YOU WILL MEET THE LEGAL DEFINITION OF A NEGLIGENT OPERATOR. IF THAT HAPPENS, YOUR DRIVING PRIVILEGE WILL BE IN JEOPARDY.
CONSIDER LIFE WITHOUT THE CONVENIENCE OF A CAR. WOULD BEING WITHOUT A CAR CAUSE DIFFICULTIES? COULD YOU GET TO WORK, PAY YOUR BILLS? YOU MAKE CHOICES ABOUT HOW YOU ARE GOING TO DRIVE AND THOSE CHOICES LEAD TO CONSEQUENCES. IF YOU CHOOSE TO CONTINUE YOUR BAD DRIVING, THERE ARE ONLY THREE POSSIBLE OUTCOMES: LICENSE PROBATION, SUSPENSION, OR REVOCATION OF YOUR DRIVING PRIVILEGE. THAT IS IT! YOU CHOOSE.
IF YOU DRIVE WITHOUT A VALID LICENSE, YOU CAN BE ARRESTED. YOUR CAR CAN BE IMPOUNDED AND SOLD. YOU ARE PAINTING YOURSELF INTO A CORNER WITH YOUR NEGLIGENT DRIVING AND IT IS NOT NECESSARY. IF YOU HAVE EVER CONSIDERED IMPROVING YOUR DRIVING, NOW IS THE TIME TO DO IT. DO IT WHILE YOU STILL HAVE AN UNRESTRICTED DRIVING PRIVILEGE. DO NOT GET THAT NEXT NEGLIGENT OPERATOR POINT ON YOUR DRIVING RECORD.
THIS IS YOUR LAST CHANCE TO AVOID ACTION AGAINST YOUR DRIVING PRIVILEGE. DO NOT THROW IT AWAY. YOU CAN IMPROVE YOUR DRIVING HABITS. IT IS YOUR RESPONSIBILITY TO IMPROVE THEM. OTHERS HAVE IMPROVED THEIR DRIVING WHEN CONFRONTED WITH THE CHOICES THAT FACE YOU NOW. WE BELIEVE THAT YOU CAN CHANGE, TOO, AND HOPE THAT YOU CHANGE BEFORE YOU FACE HARSH PENALTIES. BUT THE CHOICE IS YOURS.
POINT COUNT DATA
DOES NOT NEED TO BE BOXED
DEPARTMENT OF MOTOR VEHICLES
DIVISION OF DRIVER SAFETY
ENOTS Level 2 Treatment Letter
55
Enhanced Negligent Operator Treatment Evaluation System
NOTS Level 2 Treatment Letter/ Transtheoretical Model Table
Drivers still in NOTS at Level 2 are more resistant than the average driver who enters Level 1. Therefore, the primary goal of this letter is to use methods to create cognitive dissonance between reality and their belief of reality. Tipping the decisional balance away from the status quo toward change is the challenge. This letter is a more focused attempt to get the driver to think about change.
Level 2 effects have had disappointing results with previous treatment letters. The reason may have been our failure to recognize that those who graduated to level 2 were drivers who were highly resistant to change. This letter focuses directly upon that resistance and upon the denial of reality.
DUE TO THE LATEST ENTRIES TO YOUR DRIVER RECORD ( SEE BELOW), YOU ARE NOW AMONG CALIFORNIA'S WORST DRIVERS, ONE OF THOSE WHO CAUSE THE MOST CRASHES, INJURIES, AND DEATHS. ( FEEDBACK) ( CONSCIOUSNESS RAISING) ( SELF REEVALUATION) ( ENVIRONMENTAL REEVALUATION)
Drivers in the precontemplation stage regarding a problem behavior such as driving are not even thinking about changing that behavior. There are many reasons to remain in precontemplation. These can be summarized as the " four R's" of resistance: reluctance, rebellion, resignation, and rationalization.
CONSIDER LIFE WITHOUT THE CONVENIENCE OF A CAR. ( ENVIRONMENTAL REEVALUATION) ( CONSCIOUSNESS RAISING) WOULD BEING WITHOUT A CAR CAUSE DIFFICULTIES? COULD YOU GET TO WORK, PAY YOUR BILLS? YOU MAKE CHOICES ABOUT HOW YOU ARE GOING TO DRIVE AND THOSE CHOICES LEAD TO CONSEQUENCES. ( CHOICE) ( PERSONAL RESPONSIBILITY) ( SELF REEVALUATION) IF YOU CHOOSE TO CONTINUE YOUR BAD DRIVING, THERE ARE ONLY THREE POSSIBLE OUTCOMES: LICENSE PROBATION, SUSPENSION, OR REVOCATION OF YOUR DRIVING PRIVILEGE. ( CHOICE) ( ENVIRONMENTAL REEVALUATION) ( SELF REEVALUATION) ( CONSCIOUSNESS RAISING) THAT IS IT! YOU CHOOSE. ( CHOICE)
Reluctant precontemplators are those drivers who through lack of knowledge or inertia do not want to consider change. The technique of raising the reluctant precontemplator's consciousness with
IF YOU DRIVE WITHOUT A VALID LICENSE, YOU CAN BE ARRESTED. ( CONSCIOUSNESS RAISING) ( SELF REEVALUATION) YOUR CAR CAN BE IMPOUNDED AND SOLD. ( CONSCIOUSNESS RAISING) ( SELF)
NOTS Level 2 Treatment Letter/ Transtheoretical Model Table 56
  Enhanced Negligent Operator Treatment Evaluation System 
57
relevant information can be most helpful for this group.
Rebellious precontemplators have a heavy investment in the problem behavior and in making their own decisions. Providing choices seems to be the best strategy for working with this type of person. The Level 2 treatment letter provides a dozen reminders, of the choices either made or available to the rebellious precontemplator. These drivers have a lot invested in their current driving behaviors and the real task is trying to shift some of that energy into contemplating change.
REEVALUATION) YOU ARE PAINTING YOURSELF INTO A CORNER WITH YOUR NEGLIGENT DRIVING AND IT IS NOT NECESSARY. ( CONSCIOUSNESS RAISING) ( PERSONAL RESPONSIBILITY) ( CHOICE) ( DRAMATIC RELIEF) IF YOU HAVE EVER CONSIDERED IMPROVING YOUR DRIVING, NOW IS THE TIME TO DO IT. ( DIRECT ADVICE) ( CHOICE) DO IT WHILE YOU STILL HAVE AN UNRESTRICTED DRIVING PRIVILEGE. ( DIRECT ADVICE) ( CHOICE) DO NOT GET THAT NEXT NEGLIGENT OPERATOR POINT ON YOUR DRIVING RECORD. ( DIRECT ADVICE) ( CHOICE)
Resigned precontemplators are characterized by a lack of energy and investment. Instilling hope is a powerful motivator for the resigned precontemplator. Without some hope of the possibility for change, these drivers will not be motivated to contemplate change. Building self- efficacy and reinforcing the idea that the correct choice is within the driver's ability to select and maintain are used to instill hope.
Rationalizing precontemplators have all the answers where the resigned precontemplator has none. These drivers are not considering change because they have figured out the odds of personal risk, or they have plenty of reasons why the problem is not a problem or is a problem for others but not for them. These drivers are challenged by this treatment letter, but are left with the conviction that the solutions are within their ability to control.
THIS IS YOUR LAST CHANCE TO AVOID ACTION AGAINST YOUR DRIVING PRIVILEGE. ( CONSCIOUSNESS RAISING) ( CHOICE) DO NOT THROW IT AWAY. ( DIRECT ADVICE) YOU CAN IMPROVE YOUR DRIVING HABITS. ( SELF- EFFICACY) ( CHOICE) IT IS YOUR RESPONSIBILITY TO IMPROVE THEM. ( PERSONAL RESPONSIBILITY) ( SELF REEVALUATION) ( ENVIRONMENTAL REEVALUATION) OTHERS HAVE IMPROVED THEIR DRIVING WHEN CONFRONTED WITH THE CHOICES THAT FACE YOU NOW. ( ENVIRONMENTAL REEVALUATION) ( CHOICE) ( SOCIAL LIBERATION) WE BELIEVE THAT YOU CAN CHANGE, TOO, AND HOPE THAT YOU CHANGE BEFORE YOU FACE HARSH PENALTIES. ( SELF- EFFICACY) ( CHOICE) ( PERSONAL RESPONSIBILITY) BUT THE CHOICE IS YOURS. ( CHOICE) ( PERSONAL RESPONSIBILITY) ( SELF REEVALUATION)
NOTS Level 2 Treatment Letter/ Transtheoretical Model Table Enhanced Negligent Operator Treatment Evaluation System
Appendix E
Current NOTS Levels 1 and 2 Letters
58
Enhanced Negligent Operator Treatment Evaluation System
STATE OF CALIFORNIA – BUSINESS, TRANSPORTATION AND HOUSING AGENCY GRAY DAVIS. Governor
DEPARTMENT OF MOTOR VEHICLES
LICENSING OPERATIONS DIVISION
P. O. BOX 932345
SACRAMENTO. CA 94232- 3450
JULY 11, 2003
PLEASE SHOW THIS NUMBER ON
YOUR CORRESPONDENCE
TINA MARIE PULLANO DRIVERS LICENSE NO. I4071280
2415 1ST AVENUE
SACRAMENTO, CA 95818
DEAR CALIFORNIA DRIVER:
PLEASE TAKE A FEW MOMENTS TO REVIEW THIS NOTICE. IT IS THE RESULT OF A RECENT ENTRY
TO YOUR DRIVING RECORD, AND CAREFUL CONSIDERATION OF ITS' CONTENTS MAY SAVE YOUR
DRIVING PRIVILEGE.
YOU MAY NOT BE AWARE THAT CALIFORNIA VEHICLE CODE SECTION 12810.5 DEFINES SOME
DRIVERS AS " NEGLIGENT OPERATORS". TRAFFIC CONVICTIONS AND RESPONSIBLE ACCIDENTS
ADD UP TO POINTS ON YOUR DRIVING RECORD. THE VEHICLE CODE DEFINES YOU AS NEGLIGENT
IF YOU HAVE FOUR POINTS IN 12 MONTHS, SIX POINTS IN 24 MONTHS, OR EIGHT POINTS IN 36
MONTHS. THE DEPARTMENT WILL SUSPEND YOUR DRIVING PRIVILEGE IF YOU MEET THE
DEFINITION OF A NEGLIGENT OPERATOR.
WE ARE CONCERNED WITH YOUR DRIVING RECORD AND WANT TO ASSIST YOU TO AVOID BEING
CLASSIFIED AS A NEGLIGENT OPERATOR. RESEARCH SHOWS THAT DRIVERS WHO HAVE A
PATTERNOF VIOLATIONS ARE AT GREATER RISK OF CAUSING, OR CONTRIBUTING TO THE CAUSE
OF AN ACCIDENT.
PLEASE REVIEW YOUR CONVICTIONS AND POINTS ( LISTED BELOW), AND THEN ASK YOURSELF IF
THERE IS ANYTHING YOU CAN DO TO AVOID FUTURE VIOLATIONS AND/ OR ACCIDENTS. YOU CAN
MAKE THE HIGHWAYS SAFER FOR ALL OF US, AND RETAIN YOUR DRIVING PRIVILEGE, BY TAKING
THIS OPPORTUNITY TO REEVALUATE YOUR DRIVING HABITS AND RESPONSIBILITIES.
IF THERE IS A DISCREPANCY IN YOUR DRIVING RECORD, YOU MAY CONTACT THE SACRAMENTO
DEPARTMENT OF MOTOR VEHICLES, AT ( 916) 657- 9074.
ACCIDENT:
DATE LOCATION VEH REPORT NO POINTS
10- 23- 93 LODI 83741346174 1
07- 28- 93 IONE 90776320181 1
THIS ACTION IS INDEPENDENT OF ANY OTHER ACTION TAKEN BY THE COURT OR THIS
DEPARTMENT.
DEPARTMENT OF MOTOR VEHICLES
DIVISION OF DRIVER SAFETY
A Public Service Agency
CURRENT NOTS LEVEL 1
59
Enhanced Negligent Operator Treatment Evaluation System
60
STATE OF CALIFORNIA – BUSINESS, TRANSPORTATION AND HOUSING AGENCY GRAY DAVIS. Governor
DEPARTMENT OF MOTOR VEHICLES
LICENSING OPERATIONS DIVISION
P. O. BOX 932345
SACRAMENTO, CA 94232- 3450
JULY 11, 2003
PLEASE SHOW THIS NUMBER ON
YOUR CORRESPONDENCE
TIMOTHY WILLIAM BRICKER DRIVER LICENSE NO I4071930
2415 1ST AVENUE
SACRAMENTO, CA 95818
DEAR CALIFORNIA DRIVER:
PLEASE REVIEW THIS WARNING LETTER. IT SAVE YOUR DRIVING PRIVILEGE.
EACH YEAR THIS DEPARTMENT SUSPENDS OR REVOKES THE DRIVING PRIVILEGE OF OVER ONE
MILLION DRIVERS. A RECENT ENTRY TO YOUR DRIVING RECORD PLACES YOU DANGEROUSLY
CLOSE TO JOINING THIS GROUP. WE DO NOT WANT THIS TO HAPPEN.
CALIFORNIA VEHICLE CODE SECTION 12810.5 DEFINES A NEGLIGENT OPERATOR AS ANYONE
' WHOSE DRIVING RECORD SHOWS A POINT COUNT OF FOUR POINTS IN 12 MONTHS, SIX POINTS IN
24 MONTHS, OR EIGHT. POINTS IN 36 MONTHS. BASED ON THE POINT COUNT ACCUMULATED ON
YOUR DRIVING RECORD, YOU ARE IN DANGER OF BEING CLASSIFIED AS A NEGLIGENT OPERATOR.
YOU ARE ONE OF A VERY SMALL PERCENTAGE OF CALIFORNIA DRIVERS WHO HAVE HAD SUCH
RECORDS IN THE PAST YEAR. RESEARCH SHOWS THAT DRIVERS WHO HAVE A PATTERN OF
VIOLATIONS ARE AT GREATER RISK OF CAUSING OR CONTRIGUTING TO THE CAUSE OF, AN
ACCIDENT. TICKETS AND ACCIDENTS COST MONEY AND LIVES. PLEASE REVIEW YOUR
CONVICTIONS AND POINTS ( LISTED BELOW), AND REVIEW YOUR DRIVING HABITS, THEN ASK
YOURSELF IF THERE IS ANYTHING YOU CAN DO TO AVOID FUTURE VIOLATIONS AND/ OR
ACCIDENTS.
LOSING YOUR DRIVING PRIVILEGE IS NOT JUST AN INCONVENIENCE. IT CAN HAVE A
DEVASTATING IMPACT ON YOUR ABILITY TO CONDUCT YOUR DAILY LIFE. CHANGING YOUR
DRIVING HABITS WILL HELP SAVE YOUR DRIVING PRIVILEGE. THE CHOICE IS YOURS.
ACCIDENTS:
DATES LOCATION VEH LIC REPORT NO POINTS
10- 11- 95 LOS ALTOS CAL AAA111 44444444444 1
10- 10- 95 SAN FRANCISCO AAA111 3333333333 1
12- 01- 94 SACRAMENTO AAA2222 3333333333 1
THIS ACTION IS INDEPENDENT OF ANY OTHER ACTION TAKEN BY THE COURT OR THIS
DEPARTMENT.
DEPARTMENT OF MOTOR VEHICLES
DIVISION OF DRIVER SAFETY
A Public Service Agency
CURRENT NOTS LEVEL 2 Enhanced Negligent Operator Treatment Evaluation System
Appendix F
Estimated Cost of Interventions
61
Enhanced Negligent Operator Treatment Evaluation System
Estimated Cost of Interventions
A primary objective of ENOTES is to provide periodic cost- effectiveness analyses of the negligent operator program. To conduct the cost- effectiveness analyses for the present report, it was necessary to estimate what portion of the total cost of an intervention would be saved if the intervention were eliminated. This is necessary since some