This paper has been mechanically scanned. Some
errors may have been inadvertently introduced.
CALIFORNIA PATH PROGRAM
INSTITUTE OF TRANSPORTATION STUDIES
UNIVERSITY OF CALIFORNIA, BERKELEY
TravInfo Evaluation ( Technology Element)
Traveler Information Center ( TIC) Study:
Operator Response Time Analysis
Mark A. Miller
Dimitri Loukakos
California PATH Working Paper
UCB- ITS- PWP- 2000- 9
This work was performed as part of the California PATH Program of
the University of California, in cooperation with the State of California
Business, Transportation, and Housing Agency, Department of Trans-portation;
and the United States Department Transportation, Federal
Highway Administration.
The contents of this report reflect the views of the authors who are
responsible for the facts and the accuracy of the data presented herein.
The contents do not necessarily reflect the official views or policies of
the State of California. This report does not constitute a standard,
specification, or regulation.
Report for MOU 365
August 2000
ISSN 1055- 1417
CALIFORNIA PARTNERS FOR ADVANCED TRANSIT AND HIGHWAYS
TravInfo Evaluation ( Technology Element)
Traveler Information Center ( TIC) Study
Operator Response Time Analysis
TM
Mark A. Miller
Dimitri Loukakos
July 31,2000
ABSTRACT
TravInfoTMis an advanced traveler information system for the San Francisco Bay Area that
began operation in September 1996 under a publidprivate partnership seeking to compile,
integrate, and broadly disseminate timely and accurate multi- modal traveler information through
commercial products and services. The public sector component centers on the Traveler
Information Center ( TIC), TravInfoT” s information gathering, processing, and dissemination
hub. For two years, until September 1998, it was a Field Operational Test ( FOT) sponsored by
the Federal Highway Administration. Since the end of the test TravInfoTMh as been in a
transitional or “ as- is” phase continuing its operation without major changes to the TIC relative to
the test phase. September 2000 is the tentative date when a new system manager for the TIC is
expected to assume its managerial and operational responsibilities. At this time, TravInfoTM
moves into a more stable and likely permanent part of the Bay Area’s information technology
infrastructure.
During the Field Operational Test, TravInfoT” s TIC was evaluated with respect to system
reliability, the communications interface, the operator interface, and operator response time. This
report documents the analysis of operator response time that is a significant part of the evaluation
because TravInfoTM’s operations are substantially less automated than originally envisioned and
so depends to a significant degree on operator performance.
Focus was placed on incident information generated by the California Highway Patrol’s
Computer- Aided- Dispatch ( CAD) system. Response time is defined as the elapsed time once
information is collected from the CAD system until it is disseminated to the public via the
Traveler Advisory Telephone System. Response times remained stable during the FOT with an
overall average operator response time of approximately 11 minutes though there were
significant response time differences among the operators. It is expected that once TravInfoTM
enters its more permanent and stable phase with a new system manager, operations will become
substantially more automated and this should have a significant impact on reducing operator
response times as well as increasing overall operator productivity. Automating operations
coupled with the much stricter operator quality control measures that were put in place near the
end of the FOT should combine to make operations more like the “ timely” undertaking it was
intended to be.
Key Words: TravInfoTMF, ield Operational Test, evaluation, traveler information center,
advanced traveler information systems, operator response time
i
TABLE OF CONTENTS
SECTION PAGE
ABSTRACT
EXECUTIVE SUMMARY
LIST OF FIGURES
LIST OF TABLES
1. INTRODUCTION
1.1 Background and Objectives
2. STUDY DESIGN
3. METHODOLOGY AND PERFORMANCE MEASURES
4. STUDY FINDINGS
4.1 CAD to TIC
4.1.1 Descriptive and Aggregate Statistics
4.1.1.1 Operator Work Efficiency
4.1.1.2 Aggregate Measures by Operator Shift, Time of Day,
4.1.1.3 Response Time Variability by Incident Type
4.1.2 Statistical Hypotheses and Tests
and Operator Work Station
4.2 TIC to TATS
5. CONCLUSIONS
6. REFERENCES
1
11
..
iv
V
1
1
2
5
6
6
7
7
10
18
21
23
24
25
EXECUTIVE SUMMARY
TravInfoTM is an advanced traveler information system for the San Francisco Bay Area that
began operation in September 1996 under a publidprivate partnership seeking to compile,
integrate, and broadly disseminate timely and accurate multi- modal traveler information through
commercial products and services. The public sector component centers on the Traveler
Information Center ( TIC), TravInfoTM’s information gathering, processing, and dissemination
hub. For two years, until September 1998, it was a Field Operational Test ( FOT) sponsored by
the Federal Highway Administration. Since the end of the test TravInfoTMh as been in a
transitional or “ as- is’’ phase continuing its operation without major changes to the TIC relative to
the test phase. September 2000 is the tentative date when a new system manager for the TIC is
expected to assume its managerial and operational responsibilities. At this time, TravInfoTM
moves into a more stable and likely permanent part of the Bay Area’s information technology
infrastructure.
The evaluation of TravInfoTMc onsisted of three major elements: ( 1) institutional, ( 2) technology,
and ( 3) traveler response. The TIC study was part of the technology evaluation and consisted of
four primary elements: system reliability, communications interface, operator interface and operator
response time analysis. System reliability examined system problems. The communications
interface examined TIC data access on the part of both the public and private sectors. Operator
interface investigated the human element by considering the role of the operator in the flow of
information through the TIC, the operators’ tasks and responsibilities and the operators’ physical
working environment. Response time measured the operations’ processing time of incidents
between entry into the TIC and dissemination to the public and private sector. TravInfoTM
operations were considerably less automated than originally envisioned and more dependent on
operator performance and so operator response time took on added significance.
One of TravInfoTM’sp rimary goals has been to provide timely traveler information, but there is
no unambiguous and generally accepted meaning of “ timely” in this context. The operator’s
response time is critical to how well the center meets this ambiguous goal. However, with no
baseline value for what a “ timely” response is, the timeliness of operators’ responses could not
be comparatively assessed. Factors that could influence operator response times, however, were
identified and their influence assessed.
Two one- week analyses approximately six months apart were conducted during the final nine
months of the field test. Response times were examined between 5 a. m. and 8 p. m. for each day
of these two weeks. Response time was defined as the time between an incident first being
posted on the California Highway Patrol’s Computer- Aided Dispatch ( CAD) system and it being
entered into the TravInfoTMp hone system and available to the public.
During the FOT, as reflected in the two- wave test, average response times for processing
incidents from the Highway Patrol’s Computer- Aided Dispatch system and disseminating the
information to the public through the Traveler Advisory Telephone System remained stable with
an average value of approximately 11 minutes. Approximately 20% of the total number of
incoming Computer- Aided Dispatch incidents were entered. Operators’ job performance and
11
..
level of operator workload were the two primary factors influencing the number of incidents
entered into the system and operator response times for those entered incidents. Other factors
were also examined to assess their influence on operator response times, such as number and
type of incidents ( e. g., traffic hazard, accident with major injuries) reported by the California
Highway Patrol, time of day, the location ( Operator Work Station) where the incident was
processed, i. e., the specific geographic workstation, level of operator work experience at the TIC,
incident arrival rate, operator work- break periods, weather conditions, and system software and
hardware problems ( documented by operators). These factors, however, played only a minor role
in influencing response times.
Numerous operator work activities also affect operator response times and help provide the fuller
context within which response times were assessed. These activities included attempts to verify
an incident, disruptions caused by their own and other operators’ shift changes, operator call-checking
into the phone advisory system for quality control purposes, updating an incident
already entered into the system, searching for multiple listings for the same incident or those not
delaying traffic flow. However, these activities were difficult to isolate and individually quantify
their contribution to operator response times.
It is anticipated that once TravInfoTMe nters its more permanent and stable phase with a new
system manager, operations will soon become substantially more automated and this should have
a significant impact on reducing operator response times as well as increasing overall operator
productivity. It is recommended that once these changes are implemented, subsequent operator
response time assessments be made. Automating operations coupled with the much stricter
operator quality control measures that were put in place near the end of the FOT should combine
to make operations more like the “ timely” undertaking it was intended to be.
111
...
LIST OF FIGURES PAGE
Figure 1: Response Time Distribution for TIC Processed Incidents 10
Figure 2: Time- of- Day Distribution for TIC Processed Incidents 11
Figure 3: Response Time vs. Incident Arrivals for Wave 1 12
Figure 4: Wave 1 Incident Type Distribution Per Response Time Categories 20
Figure 5: Wave 2 ( M - F) Incident Type Distribution Per
Response Time Categories
Figure 6: Wave 2 ( M, W, F) Incident Type Distribution
Per Response Time Categories
20
21
iv
LIST OF TABLES PAGE
Table 1: Wave 1 Aggregate Statistics for Each Shift per Day- of- Week
Table 2: Wave 2 ( Monday, Wednesday, Friday) Aggregate Statistics for
Each Shift
Table 3: Wave 1 Response Times and Average Item Processing Rates of
Operators
Table 4a: Wave 1 Response Times and Items Processed by Morning Shift
Operators 5A. M.- 12 Noon
Table 4b: Wave 1 Response Times and Items Processed by Afternoon
Shift Operators 1P. M. 4P. M.
Table 5: Wave 2 ( Monday, Wednesday, Friday) Response Times and
Average Item Processing Rates of Operators
13
14
15
15
16
16
Table 6a: Wave 2 ( Monday, Wednesday, Friday) Response Times and Items
Processed by Morning Shift Operators, 5A. M.- 12 Noon 17
Table 6b: Wave 2 ( Monday, Wednesday, Friday) Response Times and Items
Processed by Afternoon Shift Operators, 1P. M. 4P. M. 17
Table 7: Wave 1 TIC to TATS Response Times ( Minutes) Average
( Standard Deviation) 23
Table 8: Wave 2 TIC to TATS Response Times ( Minutes) Average
( Standard Deviation) 24
V
1. INTRODUCTION
This report documents the Response Time element of the Traveler Information Center ( TIC)
evaluation as described in the TIC evaluation plan ( 1). This TIC evaluation element focuses on
the time cycle for data acquisition, processing, and dissemination through the TIC. Providing
timely information is a primary goal of TravInfoTM( 2,3). There is, however, no unambiguous and
generally accepted meaning of “ timely” in this context.
The System Requirements document ( 4) produced by the consultant- led ( TRW) TIC design and
development team states that:
0 Freeway incident data ( dynamic data type) would arrive at the TIC from the Traffic
0 Average latency of dynamic data would be no more than one minute, i. e., “ the time delay
Operations System
from the time of dynamic information receipt to the time of TravInfom / TIC output results”
into the Landline Data Server ( LDS) and the Traveler Advisory Telephone System ( TATS).
This, however, does not take into account the time between the incident’s occurrence and the
center first being aware of it that would include the time until the incident is first observed,
reported to proper authorities, and forwarded onto the TIC. Thus, “ timely” does not necessarily
mean “ instantaneously” and remains somewhat of an ambiguous goal. Also, with no baseline
value for what a “ timely” response is, the timeliness of operators’ responses could not be
comparatively assessed with any reference response time measure. Factors that could influence
response times, however, were identified and analyzed to determine their impact on operator
response times. Based on these findings, recommendations are made for altering certain factors
in order to reduce operator response times.
The California Highway Patrol Computer- Aided Dispatch system is the primary Traveler
Information Center data source ( 5) for entering, processing and disseminating data. Since data
from this source needs to be manually entered, it would very likely require longer than one
minute and could create a potentially serious bottleneck in Traveler Information Center
information flow. Thus, investigating operator response time, in particular since it depends to a
great extent on operator work performance, is an important component of the overall TIC
evaluation. The remainder of this section explains the specific objectives for this part of the
evaluation and provides additional background material before discussing the study design,
methodology, performance measures, and findings.
1.1 Background and Objectives
The objective of the work reported in this document is to assess TIC performance over the course
of the TravInfoTMF ield Operational Test ( FOT) relative to response time, that is, the amount of
elapsed time once data enters the TIC to be processed and disseminated. Information is
disseminated to the public via the Traveler Advisory Telephone System and to the private sector
via the Landline Data Server. Data transfer from the TIC to the LDS is instantaneous, whereas
transfer from the TIC to TATS is not, so focus in this analysis was placed on the TIC to TATS
connection. The operator plays a significant role in all phases of data manipulation at the TIC
( 5,6), resulting in time being a relevant factor to overall TIC performance. Studying response
time will also allow evaluators to make recommendations for improvements in overall TIC
performance where needed.
2. STUDY DESIGN
The response time evaluation was conducted by analyzing data from a specially designed and
constructed database consisting of all relevant response time data. As described in ( l), focus was
placed on incidents. The schedule outlined in ( 1) called for a response time analysis to be
conducted every six months with approximately four waves during the FOT. There were issues at
the TIC that impacted the collection of relevant response time data and these issues were not
completely resolved until fall 1997 when the two- year FOT was half complete’. With one year
remaining until the conclusion of the FOT, it was decided that response time would be assessed
twice, i. e., in two waves. 2 Each wave would consist of examining one week’s worth of data3 to
analyze this aspect of TIC operational effectiveness. Each wave’s results would provide a
description of the timeliness and responsiveness of TIC operations. Comparing and contrasting
each wave’s results would also document changes over time. This document reports the findings
of both waves.
Two one- week, i. e., the workweek ( Monday through Friday), analyses approximately six months
apart were conducted during the final nine months of the field test4. The time periods for Wave 1
and 2 were, respectively, the weeks of January 12 - 16 and June 22 - 26, 1998. Response times
were examined between 5 a. m. and 8 p. m. for each day of these two weeks. Response time was
defined as the time between an incident first being posted on the California Highway Patrol’s
Computer- Aided Dispatch system and it being entered into TATS and available to the public.
There are some notable intermediate times between these two times, such as the time between an
incident being posted and an operator being aware of its posting and the gap between an
operator’s initial awareness of a posting and its verification by the California Highway Patrol.
The time between the first appearance of an incident record in the California Highway Patrol’s
Computer- Aided Dispatch system and operator awareness of it depends mainly on the operator’s
workload and attentiveness. Operators are instructed to confirm an incident before entering it into
the database and they make attempts to do this. However, according to operators, actual
I Issues associated with supporting the response time study that needed resolution prior to its start included: 1. PATH
acceptance of outstanding evaluation- related TIC Acceptance Testing procedures, 2. Implementation of a change in
data entry procedures for incidents, 3. Resolution of a particular TIC workstation problem ( cross pollination among
workstations).
* Performing additional waves was considered but could not be accommodated due to evaluation resource constraints
in fulfilling other TIC evaluation responsibilities.
Originally each wave was to consist of a two- week period. Due to the enormous amount of data that had to be
collected, reduced, and analyzed, even for one week, coupled with staffing limitations, each wave was reduced in
length to one week.
and the most heavily traveled hours for such days.
4 No weekend data was collected or analyzed. Emphasis was placed on the most heavily traveled days of the week
2
verification does not always occur before an entry is made in the interest of remaining timely.
Operators use several means to confirm an incident, including an on- site California Highway
Patrol officer report, an airborne record, a web- service report, multiple Computer- Aided
Dispatch entries for the same incident, and operators’ experience about incidents occurring at
certain locations during certain time periods.
During the time between the first appearance of an incident record in the California Highway
Patrol’s Computer- Aided Dispatch system and incident verification ( if it actually occurs),
operators are making incident verification attempts for that particular incident, as well as
performing other activities related to other incidents and other Traveler Information Center
operational matters. Because operators take an active role in trying to confirm an incident before
entering it into the TIC database, operator performance is a factor and including this time in the
definition of operator response time helps document how close to “ timely” the information is.
During this time, incident information is available for processing and dissemination. The time
between incident verification ( again, if it occurs) and entering the processed incident into the
Traveler Information Center database is related more to the specific incident under examination.
The following factors could potentially influence operator response times:
Number of incidents to be processed; the rate at which incidents arrive
Type of incident ( e. g., traffic hazard, accident with major injuries)
Time of day
Location where the incident is processed ( i. e., the specific geographic workstation)
Length of the operator’s experience working at the Traveler Information Center
Time and nature of the California Highway Patrol’s Computer- Aided Dispatch system
problems ( documented by operators)
Time and nature of the Traveler Information Center’s system problems ( documented by
operators)
Operators’ work- break periods
Weather conditions.
Approximately 90% of TIC data processed by the three geographic workstations originates with
the CHP CAD. As documented in ( 5), all operators use the CHP CAD as a source of information
for approximately 75% of their time. During the FOT, operators ranked the CAD as the number
one TIC data source and stated it was “ very useful,” if not critical, to the functioning of the TIC.
Moreover, other incident- related data sources were incomplete and could not be used in
constructing the response time database. For all these reasons, only CHP CAD- generated incident
data was used in the response time analysis.
The response time database was constructed by compiling and organizing data from the
following five sources. Multiple sources were required to obtain all necessary data and to be able
to match data from multiple sources associated with the same incident.
CHP CAD Incident Retrieval Report
Operator Audit File ( OAF)
Operator Activity Report ( OAR)
0 TATS Tracker Sheet
0 Operator Activity Check Sheet
Relevant data contained within each of these sources are listed below:
CHP CAD Incident Retrieval Report ( hard copy’ record of each incident entered into the CAD
system):
0 CAD incident identification number
0 Incident date
0 Incident time ( when incident information was entered into the CAD
0 Incident location
0 Incident type ( as defined by CHP Aural Brevity Codes, e. g., 1125 ( traffic hazard),
1180 ( accident- major injury))
Operator Audit File ( electronic record of each item entered into the TIC’S database):
0 Entry date for TIC item
Entry time for TIC item
0 Operator workstation for TIC item
TIC item identification number
TATS Tracker Sheet ( hard copy record of each TIC item entered into TATS):
0 Entry date for TIC item
0 Entry time into TATS
0 Operator workstation
Operator
0 CHP CAD incident identification number for incidents
0 TIC item identification number
TATS mailbox number
Operator Activity Report ( electronic record of each TIC item entered into TATS):
0 Entry date of TIC item
Entry time of TIC item
0 TIC item identification number
0 Operator identification number
0 TATS mailbox number
’ The CHP was able to provide electronic files for Wave 1 and a hard copy record for Wave 2.
For Wave 1, CHP CAD incident times were provided with up- to- the- second accuracy, but only up- to- the- minute
accuracy for Wave 2 incident times. In order to continue entering times into the master response time database in
hour: minute: second format to be consistent with other database entries to calculate response times and with Wave 1,
CHP CAD times for Wave 2 records were entered has having been time stamped at the halfway point in the minute
during which they occurred, e. g., a record with a CHP CAD time stamp of “ 10: 08 A. M.” was entered into the
database as “ 10: 08: 30 A. M.”. This procedure, done consistently for all Wave 2 records introduces up to a 30 second
margin of error in the calculation of response times.
4
Operator Activity Check Sheet:
Operator identification number
Operator
The entry time into TATS for a TIC item recorded on the TATS Tracker Sheet by each operator
is not always a reliable source of actual TATS entry time. The operators do not use the same
timepiece nor are there any assurances that the timepiece each does use is accurate. Moreover,
some operators do not always enter the time when they actually make the entry, but enter it later
once they have realized they omitted this information. An even less accurate estimate of entry
time is then recorded. Because of these problems, the Operator Activity Report was used to
obtain an accurate TATS entry time for each CAD- generated TIC item.
After obtaining and compiling this data7, the constructed master response time database consisted
of 708 and 634* records for the two waves, respectively with the following fields:
CAD incident identification number
Incident date
Incident entry time into CAD ( CAD Time)
CAD incident type code
Location
Incident TIC- identification number
Operator workstation
Incident entry time into TIC ( TIC Time)
Incident entry time into TATS ( TATS Time)
Before analyzing the data, changes were made to data in both the TIC time and TATS time fields
to insure that all three times ( CAD, TIC, and TATS) were synchronized with each other. The
CAD system clock is synchronized daily with an accurate time measure so no changes were
necessary for CAD system time.
3. METHODOLOGY AND PERFORMANCE MEASURES
The response time analysis covers the time from when an incident is initially input into the CAD
system until it is input into TATS and available to the public. The two components of this total
response time, CAD to TIC time and TIC to TATS time, were performed separately. The method
used to investigate both components consisted of the writing and running of queries in the
response time database relative to different criteria.
Data from the Operator Activity Check Sheet was used in conjunction with the Operator Activity Report to help
associate different data with each other and was not specifically entered into the master response time database.
For Wave 2, operators entered 738 records into the TIC database. Of these records, 104 correspond to missing
records from the CHP CAD log and so were not included in the master response time database as response times for
them could not be calculated.
5
For the CAD to TIC component, for each query run, i. e., for each specific criterion, the response
time distribution was also calculated. The performance measures used were 1. Average response
time, 2. Standard deviation of response time, 3. Number of incidents with a particular response
time range, 4. Number of incidents processed per hour and 5. Ratio of the number of TIC items
processed to the number of CAD items ( adjusted for repeat records and records not considered by
operators by incident type). Criteria include the following:
0 Day of week
0 Time of day ( individual hours, A. M. P. M. shifts, A. M./ P. M. peak periods)
0 Type of incident
0 Operator geographic workstation
In addition to these measures of performance, statistical hypothesis testing was performed to
assess the significance of operator response time variability ( See Section 4.1.2).
For the TIC to TATS component, the performance measures used were 1. Average response
time and 2. Standard deviation of response time. Criteria include the following:
0 Day of week
0 Time of day ( A. M. P. M. shifts)
0 Operator geographic workstation
4. STUDY FINDINGS
This section provides the major findings of the response time analysis with separate sections for
the CAD to TIC and the TIC to TATS components. The main reasons for dividing the analysis
into these two parts are 1. Differences in the set of records for which there were complete data, 2.
Substantially different response time ranges between these two components, and 3. Different sets
of explanatory variables to consider in order to understand the magnitude and variation in
response times.
4.1 CAD to TIC
The response time analysis covers the time from when an incident is initially input onto the CAD
by a CHP dispatcher until it is input into TATS and available to the public. Incident information
appears on a CAD terminal at the TIC as soon as it is entered by a CHP dispatcher. As TIC
operators may be performing other tasks, especially during busy times, and would not necessarily
stare at the CAD screen waiting for the next incident to appear, the CAD to TIC time for an
incident includes the time until it is observed by an operator. While operators do work to
confirm the validity of an incident before entering it on the TIC, incidents are definitely entered
into the TIC without official CHP confirmation. This is based on operator interviews and
observations by the evaluators. Means of incident confirmation include Metro Networks
Airborne and Instatrack ( a Web- based tool), operators’ experiential knowledge about incidents at
6
particular locations and during particular times, Closed Circuit Television, and confirmation by a
CHP officer at the site of the incident.
The objective in performing the CAD to TIC response time analysis was to explain the
magnitude and variability in these response times. This is accomplished by considering the
explanatory variables individually ( See Section 4.1.1) and simultaneously ( See Section 4.1.2)
4.1.1 Descriptive and Aggregate Statistics
This section of the report discusses the response time study findings from a descriptive and
aggregate statistical perspective. The primary objective is to examine operator response times
and explain their magnitude and variability. As explained in the study design section, data was
collected for two one- week periods from 5 A. M. to 8 P. M on Monday January 12 through Friday
January 16, 1998 ( Wave l), and on Monday June 22 through Friday June 26, 1998 ( Wave 2).
This data consisted exclusively of incidents arriving on the CHP CAD terminal that were
processed by operators then entered into the TIC database. There are four on- duty TIC operators
from 5 A. M. to 9 P. M. Three of these operators handle mostly incident data and the remaining
operator processes roadwork and slowdown information, i. e. information pertaining to commute
hour slowdowns on freeways. This section focuses on the response time of the operators located
at the three workstations handling incident data ( operators at the TIC rotate according to a
morning and afternoon shift. Hence, what is measured in these study findings is the response
time of different operators located at the three workstations handling incident data) from the CHP
CAD. For purposes of the analysis below, these three workstations will be referred to as
workstations 1,2 and 3.
4.1.1.1 Operator Work Efficiency
During Waves 1 and 2, a total of 10,619 and 10,967 records were entered onto the CAD by CHP
dispatchers, respectively. Operators at each of the three geographic workstations examined these
records to determine which ones would subsequently be entered into the TIC. An important
measure of operator performance is their work efficiency, i. e., the ratio of the number of records
entered into the TIC out of the total number of records coming into the CAD. It would be
incorrect to divide the number of TIC records for each wave by the number of CAD records for
each wave ( 10,619 or 10,967) for this time period to estimate work efficiency. There are
incidents that are not considered because they do not impact traffic and there are incidents for
which there are multiple records. Both these incident types should first be removed from the
CAD record population prior to the calculation of work efficiency. The former group consists of
the following CHP CAD Codes’ and total 2,801 and 3,7531° out of the 10,619 and 10,967
These codes are referred to as CHP Aural Brevity Codes with which CHP staff communicates for brevity.
The CHP CAD log contained missing records for Tuesday ( June 23) and Thursday ( June 25). This total ( 3,753)
represents an exact calculation for records that do not impact traffic for three of the five days of Wave 2 ( no missing
records) plus an estimate of records that do not impact traffic during the two days with missing records.
IO
7
records respectively for Waves 1 and 2: 1126, 1125C, and 3A" with 7,812 and 7,213 records
remaining.
An estimate for the number of records in the latter group, i. e., records that are input into the CAD
multiple times, was determined by the following methodology. As it was infeasible to
individually examine each of the 7,812 and 7,213 records, i. e., the record populations for Wave 1
and Wave 2, respectively, to determine which ones were previously recorded into the CAD, a
random sample of 100 records from each population was selected. Each member of the two
samples was subsequently examined to determine whether or not it had been previously recorded
in the population. After this examination, each sample of 100 consisted of the following two
components: those records that had not been previously recorded and those that had been. For
each record in each sample, all population records with a CAD timestamp within fifteen minutes
prior to the sample record CAD timestamp were examined. It was assumed that if a record had
been previously recorded, fifteen minutes would be an acceptably sufficient time period with
which to capture that repeat ( at least one of them, for multiple repeats). The percentages of
sample records previously recorded for Wave 1 and 2 were 32% and 34% respectively.
The issue of duplicates, triplicates, quadruplicates, and so on arises since a record may be
replicated a multiple number of times. The sample records were examined only to determine if
they were repeats of previous records and did not specifically count the number of such
replications. Varying number of replications is, nevertheless, accounted for in the following way:
There is only one replication for each duplicate, two replications for each triplicate, three
replications for each quadruplicate, and so on. Thus, the probability of finding a replication for a
duplicate is less than or equal to the probability of finding a replication for a triplicate, which are
less than or equal to the chances of finding a replication for a quadruplicate, etc. Thus, the
different kinds of replications are represented in the percentage breakdown of records
with/ without replications12.
Another issue is determining how representative of the actual record population is the random
sample of 100. The objective in choosing a random sample of a particular size, N, is to have N be
large enough so that the following statistical statement is justifiably valid:
With 95% confidence, the percentage of sample records that were previously recorded is
within +/- X% of the actual percentage of population records that were previously
recorded. That is, there is only a 5% chance that the actual percentage of the population
that had been previously recorded is beyond the range of +/- X% from the sample
percentage.
'' CAD Code 1126 = Stalled vehicle not blocking a lane, 1125C = Stalled vehicle in the center divide, 3A =
American Automobile Association ( AAA) has been called to tow the vehicle.
l2 For example, consider the following two populations: One population consisting of records with duplicates ( one
replication) and records with no replications. The second population consists of records with triplicates ( two
replications) only and records with no replications. Examine a random sample from each population. The first
population would have a percentage of records that had been previously recorded less than or equal to the analogous
percentage of records for the second population.
8
The smaller one desires “ X” to be, the larger the sample size must be. With a random sample of
size 100, X takes on the values of 9.4 and 9.5 for Waves 1 and 2, re~ pectively’T~ h. us, with 95%
confidence, the actual percentage of Wave 1 population records that were replicated is 32% +/-
9.4%, i. e., an interval ranging from 22.6% to 41.2%. The “ 9.4%” is larger than usually accepted
in practice. It is accepted and used here because operator work efficiency is the final measure to
be estimated and efficiency is fairly insensitive to values of X and hence, examining a larger
random sample was unnecessary. Operator work efficiency ranges from 11.7% to 15.5% with a
value of 13.3% that corresponds to the empirical result of 34% replicated records from the
random sample of lOOI4. Being conservative and erring on the side of the larger value for work
efficiency, the upper limit of 15.5% efficiency is henceforth used. This corresponds to 4,567
recordsI5. For Wave 2, the corresponding 95% confidence interval ranges between 24.5% and
43.5% corresponding to a range of operator work efficiency between 13.6% and 18.1 % I6. Again
being conservative and erring on the side of the larger value for work efficiency yields the upper
limit of 18.1% efficiency corresponding to 4,064 records.
The estimated number of distinct records coming from the CAD on which operators focus their
attention, i. e., the 4,567 items for Wave 1 and 4,064 records for Wave 2, provide an approximate
upper bound on the volume of records that could be processed. Even with significant increased
work efficiency, it is not likely that this volume of processed items could ever be achieved
without additional operators all else being equal. Operators state that it takes approximately 3 to
4 minutes to process each item arriving from the CAD17 ( 1). This rate translates into a total
ranging from approximately 3,375 to 4,500 records that could be processed by 3 operators ( 3
workstations) over 15 hours ( 5 A. M. TO 8 P. M.) over 5 days ( Monday through Friday). This
means that currently from 15.7% ( 708/ 4,500) to 21 . O% ( 708/ 3,375) of the total number of CAD
records that operators say could be processed are being processed for Wave 1 and between
16.4% ( 738/ 4,500) and 21.9% ( 738/ 3,375) for Wave 2. Again, even with significant increased
work efficiency, it is not likely that this volume of processed items could ever be achieved
without additional operators.
Thus, during Wave 1 and Wave 2,708 incidents out of 4,567 CAD items and 738 out of 4,064
CAD items, respectively were processed or entered into the TIC database at workstations 1 , 2
and 3. Since these incidents were fairly evenly distributed by day- of- week and there were no
statistically significant differences in response times by day of week, the response time data was
initially aggregated by hour of the day and/ or operator workstation for the whole week. More
detailed analysis will be presented later in this section.
l3 This is based on the values for the sample size, standard deviation, and the standard error.
( 32%* 7,812)).
l5 The complete discussion of this methodology may be found in ( 7). Using this methodology depends on the
probability of a sample member having a replication being independent of the probability of another sample member
having a replication. While this is not guaranteed, it is, nevertheless, fairly likely to be true in this case.
l6 This calculation is based on the number of operator- entered records into the TIC database, i. e., 738, not the
reduced number of 634 after missing records have been accounted for: 13.6% = 738/( 7,213-( 24.5%* 7,213)); 18.1%
= 738/( 7,213-( 43.5%* 7,213)). Otherwise, operator efficiency would be incorrectly reduced.
l7 This is the process starting with incident record retrieval from the CAD to inputting the incident message onto
TATS.
14 11.7% 708/( 7,812-( 22.6%* 7,812)); 15.5% = 708/( 7,812-( 41.4%* 7,812)); 13.3% = 708/( 7,812-
9
4.1.1.2 Aggregate Measures by Operator Shift, Time of Day, and Operator Work Station
If we look at Monday through Friday for Wave 2 to reflect the actual number of incidents entered
into the TIC, we won’t be able to calculate response times for each of the five workdays because
we don’t have all data for CAD times for Tuesday and Thursday. Alternatively, if we use
Monday through Friday for the modified data set, i. e., removing records corresponding to the
missing CAD data, then we can estimate response times for all the data in this modified data set
but we have now excluded incidents for which operators entered data into the TIC, so operator
statistics would be biased downward. For these reasons, we have concentrated on Monday,
Wednesday, and Friday for the Wave 2 analysis; however, we occasionally display Monday
through Friday response time measures ( on the modified data set after removal of records
corresponding to missing CAD data) to show performance measure stability with the abbreviated
Wave 2 week for Monday, Wednesday, and Friday.
Figure 1 displays an overview of the distribution of response times for Waves 1 and 2 with two
distributions ( 1. Monday through Friday and 2. Monday, Wednesday, Friday) depicted for Wave
2 to account for the missing data in the CHP CAD data on Tuesday and Thursday during the
Wave 2 week. While there are differences between Waves 1 and 2, a single pattern emerges. For
each Wave, approximately 63% of the incidents arriving through the CAD and entered into the
TIC have a response time of less than or equal to ten minutes. However, approximately 20% of
these incidents have a response time of fifteen minutes or more. The overall average and standard
deviation for CAD to TIC response times are 10.1 minutes and 7.4 minutes for Wave 1, 10.1
minutes and 8.5 minutes for Wave 2 ( Monday through Friday), and 10.3 minutes and 9.2 minutes
for Wave 2 ( Monday, Wednesday, Friday) respectively.
Figure 1: Response Time Distribution for TIC Processed Incidents
45.0
40.0
28 35.0
z3 30.0
25.0
Ow 20.0
g 15.0 10.0
u 5.0
fc;
0.0
l-
Wave 2 ( M - F)
[ 7 Wave 2 ( M , W,
0- 5 minutes 5- 10 10- 1 5 15- 20 20t
minutes minutes minutes minutes
RESPONSETIMECATEGORIES
10
The analysis now focuses on response time data by hour- of- day for the three workstations
aggregated. Figure 2 displays the percentage breakdown of processed incidents by hour- of- day.
Again, while differences exist for these results between Waves 1 and 2 ( Monday, Wednesday,
Friday) generally the same patterns emerge. As expected, peaks occur mostly during morning and
afternoon commute hours. The morning peak occurs from 8 to 9 A. M. wherein approximately
9% of overall incidents were processed during Wave 1. However, approximately 6.4% of overall
incidents were processed for Wave 2 ( Monday, Wednesday, and Friday). During the afternoon,
the pattern corresponds more to a plateau with a marked increase from 1- 2 P. M. and a
pronounced fall from 7 to 8 P. M.
Figure 3 shows graphs of the average response time per incident and the average time between
the processing of incidents by hour of day for Wave I ( Wave 2 data shows similar behavior and
is not explicitly depicted here). Points on the graph with the lowest average time between the
processing of incidents correspond to peaks in the number of incidents being processed. The
trends in Figure 3 show a very unexpected relationship. One would expect an inverse relationship
between the average response time per incident and the average time between the processing of
incidents. Indeed, as the number of incidents increases leading to a lower average time between
the processing of incidents, in turn, the response time to incidents should increase as the
operators are becoming more burdened with work. Instead, the trends in Figure 3 show either
mostly a parallel relationship or no discernible relationship at all. During the morning hours
from 5 to 9 A. M., the number of incidents processed close to quadruples in response to the
morning commute, while at the same time the average response time to incidents is halved from
16 to 8 minutes. From 10- 1 1 A. M., the number of incidents processed is three times lower than at
the 8- 9 A. M. peak, yet the average response time, instead of declining, remains essentially
unchanged at about 8 minutes per incident. Between 11 A. M. and 2 P. M., the number of
Figure 2: Time of Day Distribution for TIC Processed Incidents
14.0
0 Wave 2 ( M, W, F
5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
TIME OF DAY
11
incidents per hour remains essentially stable at between 6% and 7% of overall incidents ( between
43 and 50 incidents), yet the average response time per hour for this three- hour period goes
successively from 11 to 9 to 17 minutes. From 1 to 2 P. M., there is a 50% increase in the number
of incidents processed ( from approximately 43 to 64 incidents) and the number of incidents
remains close to that level until 7 P. M., yet response time continuously decreases from 17
minutes from 1- 2 P. M. to a low point of 9 minutes for the 5- 6 P. M. hour.
Figure 3: Response Time vs. Incident Arrivals for Wave 1
AVERAGE " E
BElWEEN INCIDENTS
. . - - . . - AVERAGE RFSPO NSE
5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
TIME OF DAY
There is thus no discernible relationship between the number of incidents processed and the
average response time to incidents. In fact, the response time pattern that emerges from Figure 3
is the following: response time is generally at its greatest before and after commute time peaks
and we surmise this is due mostly to a lack of diligence by some operators. These operators
become gradually more diligent and responsive to incidents as the morning and afternoon
commute peaks approach as is evidenced by the rapid decrease in response times between 5 A. M.
and 8 A. M. ( despite a rapid increase in the number of incidents) and between 1 P. M. and 5 P. M.
( when the number of incidents remains stable). After the commute peaks, response times increase
as these operators become less diligent as is clear from the crests in response time from 11 A. M.
to noon and from 6 to 7 P. M.
Thus, the response time to incidents depends only little on the number of incidents coming into
the TIC ( i. e. how burdened the operators are) but mostly on the quality of work produced by
those specific operators. For Wave 1, Table 1 displays the number of incidents processed at the
TIC, the average response times and the number and arrival rate of items coming into the CAD
for the morning and afternoon shifts'*. When analyzing the numbers displayed in Table 1 no
clear relationship emerges between the number and arrival rate of CAD items, the number of
'* The noon hour was omitted from this table resulting in an equal number of hours per time period. Shift changes
occur during the 5am and lpm hours.
12
items entered into the TIC and the average response times. For example, during the morning
shift, the best average response time was logged on Monday at 7.8 minutes per incident yet
Monday was the busiest day of the week with 900 items coming into the CAD and 76 items
processed into the TIC. With regard to the CAD incident arrival rate, the numbers for the
afternoon shift similarly speak of a lack of correlation: while the CAD item arrival rate is
generally stable at between 19 and 23 seconds, the average response time varies between 9.7 and
12.8 minutes. The numbers from the Wednesday afternoon shift represent another counter-intuitive
case: this was the second least busy afternoon in terms of items coming into the CAD
yet represented the most busy afternoon in terms of items entered into the TIC and the second-best
response time for the afternoon shift. While it is clear that the number of items coming into
the CAD effects the number of TIC items processed and response times, the variability in these
two numbers cannot be ascribed to the volume of CAD items alone. Table 2 displays the
corresponding information for Wave 2 ( Monday, Wednesday, Friday).
Table 1: Wave 1 Aggregate Statistics For Each Shift per Day- of- Week
l9 For each day and time period, the original number of CAD items was reduced to factor out incidents not
considered by operators and repeated records ( See discussion given in earlier part of this section of the report).
This 20 is defined as the ratio of the number of TIC items processed to the adjusted number of CAD items.
13
Table 2: Wave 2 ( Monday, Wednesday, Friday)
Aggregate Statistics For Each Shift
In fact, when observing incident processing rates operator by operator, it becomes clear that the
TIC incident processing and response time variability are due to differences in operator
productivity. Tables 3,4a and 4b display for Wave 1 the incident processing and response time
rates for each operator in the morning and afternoon shifts. Two things are clear from these tables
for Wave 1 ( Comparable results for Wave 2 are shown in Tables 5,6a and 6b): 1. Substantial
differences in operator productivity, as measured both by the average response time and by the
average number of items created per hour, are evident and 2. The absolute numbers of items
processed into the TIC appear low.
21 For each day and time period, the original number of CAD items was reduced to factor out incidents not
This is defined as the ratio of the number of TIC items processed to the adjusted number of CAD items.
considered by operators and repeated records ( See discussion given in earlier part of this section of the report).
22
14
Table 3: Wave 1 Response Times and Average Item Processing Rates of Operators
A. M. IAl -+ A4
= iF P. M. P1
Response
( minutes) Times ( min.)
Time Range Response
Average
6.3 - 7.8 I 7.3
5.8 - 12.5
8.9 - 10.5 9.6
6.9
10.4- 15.9 I 13.0
7.5 - 7.6 I 6.9
Items
Range Per Hour
Created Items Created
Average # of
+ 18 - 25
15- 21 12.5
18 - 44 14.4
31 - 40 14.6
Average # of Items
Created Per Hour
During Peak Commute
Times
2.7
3.0
4.4
3.2
3 . O
2.7
6.2
4.9
Table 4a: Wave 1 Response Times and Items Processed by Morning Shift Operators
5A. M.- 12 Noon
23 OWS = Operator Workstation
15
Table 4b: Wave 1 Response Times and Items Processed by Afternoon Shift Operators
1 P. M. 4 P. M.
Table 5: Wave 2 ( Monday, Wednesday, Friday)
Response Times and Average Item Processing Rates of Operators
Crew
A. M.
P. M.
Operator
AI
A2
A3
P1
P2
P3
P4
P5
Response
Time Range
( minutes)
4.7 - 5.3
7.5 - 9.0
4.2 - 7.0
12.7 - 12.9
12.8 - 12.9
13.1 - 15.7
7.9
11.6 - 12.8
Average
Response
Times ( min.)
5 . O
8.2
4.7
12.8
12.9
14.2
7.9
12.1
Items
Range Per Hour During Peak Commute
Created Items Created Created Per Hour
Average # of Average # of Items
22- 24 3.3 4.7
Times
11- 23
17- 24 2.9 3.8
26- 29 3.9 5.0
49 7.0 8.7
2- 16 1.4 1. o
2.4 2.9
23 3.3 2.7
20- 26 3.3 4.7
24 Replacement operator 1.
25 Replacement operator 2.
16
Table 6a: Wave 2 ( Monday, Wednesday, Friday)
Response Times and Items Processed by Morning Shift Operators, 5A. M.- 12 Noon
Average response time ( minutes)
Number of items created in TIC 11 23 2
4.2 9.0 7.0
Table 6b: Wave 2 ( Monday, Wednesday, Friday)
Response Times and Items Processed by Afternoon Shift Operators, 1 P. M. 4 P. M.
M W F
OWS 1 Onerator PI P2 PI
Average response time ( minutes)
Number of items created in TIC 49 26 49
12.7 12.8 12.9
OWS 2 ODerator P2 P3 P4
Average response time ( minutes)
Number of items created in TIC 29 17 23
12.9 15.7 7.9
OWS 3 Onerator P3 P5 P5
Average response time ( minutes)
Number of items created in TIC 24 26 20
13.1 11.6 12.8
For Wave 1, the two most productive morning operators, in terms of average response time and
items created, are A2 and A3, although A2 displays less consistency. Operator A1 has the best
average response time for the morning crew. For the afternoon crew, the most productive
operators are P3 and P4. These operators handle approximately 95% and 55% more incidents,
respectively, than P1 and P2 ( 62 records were processed by P1 and by P2) yet still manage to
have lower response times. The comparison between P4 and P2 is striking: P4 handles
approximately 55% more incidents than P2 yet has an average response time that is only 43% of
P2’ s average response time. These differences in operator productivity are the main explanatory
variable for variations in average response times and the number of items processed into the TIC.
Indeed, when looking at the best performances for the morning crew that occur on Monday and
Wednesday, these are in large part due to both A2 and A3 being on staff on those days. For the
26 OWS = Operator Workstation
27 Replacement operator 3.
17
afternoon shift, the influence of productive operators is even more noticeable. The best days for
the afternoon shift occurred on Wednesday and Friday with a high number of items handled and
low response times relative to the other days. The performance for those days was mostly due to
the presence of operators P3 and P4. Thursday afternoon had the second highest number of items
input into the TIC but over 50% of those items were processed by one operator: P3.
Another factor, linked to operator productivity that is disturbing is how few items operators
actually process and enter into the TIC, generally between 13.3% and 21.8% of CAD items
( Table 1). A majority of the operators ( 5/ 8) processed approximately three items per hour during
peak commute times ( 6 A. M. to 9 A. M. and 4 P. M. to 7 P. M.). Considering item- processing rates
on an hourly basis, the week as a whole for the three operator workstations, from 5 A. M. to 8
P. M., contains 225 one- hour segments. For 10.7% of these segments, or of the overall TIC
worktime, 0 items were processed and for another 14.7% only one item per hour was processed.
The aforementioned numbers are particularly low if one considers the following facts: 1. The
operators themselves have stated during interviews that it takes them approximately two to three
minutes to process an item into the TIC and an additional minute for input into the Traveler
Advisory Telephone System, ( at this rate one would expect operators to process close to between
15 and 20 items per hour), 2. The operators have little non- CAD work to execute ( inputting CAD
items represents over 90% of their work) and 3. The week under study witnessed rains every day
of the week and there was hence a high volume of incidents on the CAD.
4.1.1.3 Response Time Variability by Incident Type
Thus far, the discussion has focused on issues such as the number of CAD items, their rate of
arrival, and operator work efficiency, in terms of both the number of TIC items processed per
time period per operator and the number of TIC items processed per number of CAD items
entering the system. The following additional factors could also influence response time during a
given hour of the day: 1. type of incident, 2. equipment malfunctions, 3. percentage of other work
to which operators must attend ( other than inputting CAD incidents) during that hour.
As stated previously, the percentage of non- CAD work is less than 10% of the overall work. In
addition, this non- CAD work is distributed fairly evenly among the three workstations and hence
cannot really explain the significant variations in response times among the different operators.
During the week under observation for Wave 1, there were only two documented non- recurring
TIC problems, documenting the same problem. These were of “ critical” severity and had no
effect upon response times. In fact, they occurred within the Acquisition sub- system of the TIC
and concerned the unavailability of loop detector data outside the Contra Costa county area.
Recurring TIC problems were not documented from January to July 1998. In any event, TIC
problems, whether they were recurring or non- recurring, only very rarely interrupt or stop item
processing for more than a few minutes. Thus, equipment malfunctions should not be considered
a contributing factor to variations in response times. Aside from worker productivity, the last
variable that could influence response times is the type of incidents handled by the operators.
The type of incident being processed by the operator is a variable that has only minor effect on
operator response time. Indeed, the incident entry processes do not change or lengthen
18
considerably depending upon the type of incident being input. Whether the incident is severe or
not, the operators still need to observe the CAD terminal, select items of interest and enter them
into the TIC. The only difference between a severe and non- severe incident relative to response
time is that the former would perhaps require slightly more typing or explanation in the
description box of the Traffic Incident Manager window of the TIC and more checking of the
CAD for updates on the incident. If one operator relative to another had to process mostly severe
incidents, as opposed to the normal mix of incidents, one would expect his or her response time
to be slower as the CAD terminal would need to be checked more often for updates on such
incidents. However, since each operator rotates among workstations ( see Tables 3a and 3b) and
each workstation covers a whole area ( such as the East Bay or Peninsula), each operator is
exposed to different types of incidents throughout a shift and even more so throughout a given
week. Thus, while incident types influence processing time to some extent they do not help
explain variations in response times among operators.
One would expect that the response time would be smaller for serious incidents having a major
impact on traffic. Figure 4 displays for Wave 1 results response time categories in increments of
five minutes for the following major types of incidents handled by the operators ( Wave 2 results
are shown in Figures 5 ( Monday through Friday) and 6 ( Monday, Wednesday, Friday), again with
Figure 5 based on the modified data set to account for the missing records on Tuesday and
Thursday):
1125: Unspecified roadway hazard ( A = animal in roadway, C = in center divide,
D = debris in roadway, V = stalled vehicle in the roadway)
1 179: Injury accident, ambulance rolling
1182: Non- injury accident ( vehicle/ property damage only)
1 183: Accident- No details
Other: All other incident types
As is clear from Figure 4, the distribution among incident types remains essentially the same for
response times up to 20 minutes. Over 20 minutes, the “ Other” incident type category doubles in
size while the “ 1 183” and “ 1 182” categories shrink. Of particular note in the “ Other” category
are the incidents “ 1 126”, “ CLFIRE” ( Car Fire), and “ FLOOD’. “ 1 126” type incidents do not
impact traffic and should not be input into the TIC. In adjusting the number of CAD incidents,
“ 1 126s” were removed because that is TIC policy, yet were included here in the response time
database since operators spent time processing them and entering them into the TIC28. These
three “ Other” category incidents ( 1 126, CFIRE, FLOOD) consist of 6.6%, 9%, 8.6%, 6.4%, and
15.8% of all incidents from each of the response time categories from 0- 5 minutes through 20
minutes or more, respectively. Thus, in the “ 20 minutes or more” category, approximately 16%
of the whole database account for approximately 40% of the “ Other” category. Unfortunately,
these incident types are too general and do not allow the evaluators to separate incidents by
severity groups and then measure response times.
28 There were 26 “ 1 126s” out of 708 in the response time database. Their average response time and standard
deviation were 13.3 minutes and 11.5 minutes, respectively. Deleting them from the response time database would
have reduced the percentage of incidents with response times greater than or equal to fifteen minutes by only a
minimal amount yet it would also have reduced the work efficiency by a similar minimal amount.
19
Figure 4: Wave 1 Incident Type Distribution Per Response Time Categories
r
100%
80%
60%
20%
0%
0- 5 MIN. 5- 10 MIN. 10- 15 MIN. 15- 20 MIN. 20 MIN.+
RESPONSE TIME DISTRIBUTION CATEGORIES
Figure 5: Wave 2 ( M - F) Incident Type Distribution Per Response Time Categories
100%
80%
60%
40%
20%
0%
0- 5 MIN. 5- 10 MIN. 10- 15 MIN. 15- 20 MIN. 20 MIN.+
RESPONSE TIME DISTRIBUTION CATEGORIES
20
Figure 6: Wave 2 ( M, W, F) Incident Type Distribution Per Response Time Categories
100%
80%
W 2 60%
CI
* ecl 40%
IC)
E
E
20%
0%
0- 5 MIN. 5- 10 MIN. 10- 15 MIN. 15- 20 MIN. 20 MIN.+
RESPONSETIMEDISTRIBUTION CATEGORIES
0 OTHER
1183
0 1182
1179
1125
From the analysis above, it is the evaluators’ preliminary judgment that operator productivity is
the primary explanation for the magnitude and variability of CAD to TIC response times. The
following section describes the statistical hypothesis testing that was conducted to provide
additional evidence concerning this assertion.
4.1.2 Statistical Hypotheses and Tests
The objective of the analysis described in this section is to demonstrate that operator productivity
as measured by response time differs among the operators and that such differences are
statistically significant. For Wave 1 the 708 entries in the CAD to TIC response time database
were partitioned into groups according to the following categories: operator workstation, shift,
and incident type29. The resulting groups ( where OWS, shift, and incident type are held constant)
leave the day of the week as the primary variable, which of course, corresponds, to specific
operators. Thus, operator productivity would then be the primary variable with which to explain
response time, both in magnitude and variability. Groupings by OWS and shift are shown in
Tables 4a and 4b. A similar partitioning was done for Wave 2 ( Tables 6a and 6b).
To select an appropriate statistical tool to use initially depends on whether or not the underlying
distribution of the data can be identified coupled with satisfying the assumption that the
distribution may be approximated by the normal distribution ( bell- shaped curve). Statistical tests
also exist that do not depend on knowing the underlying distribution of the data, called non-parametric
tests. Such a non- parametric test was selected for the analysis of these groups. The
29 Partitioning by incident type for subsequent statistical hypothesis testing was not always possible due to an
insufficient number of incidents within each cell describing a particular OWS, shift, and incident type. To
circumvent this problem, the incident types for accidents ( 1 179, 1 180, 1 18 1, 1 182, and 1 183) were pooled and
considered a single category.
21
name of the test used was the Kruskal- Wallis Test. A brief description will be given here while
full details may be found in either ( 8) or ( 9).
Each test is conducted for a fixed OWS ( 1,2,3), shift ( A. M., P. M.), and incident type. Tests were
also conducted in which just the OWS and shift were held constant. The variable is the day of the
week providing five populations for Wave 130. The objective is to test whether the five
population medians are all the same ( Null Hypothesis) or are not all equal ( Alternative
Hypothesis). What this essentially says is that productivity is not equal across operators under the
Alternative Hypothesis and is equal under the Null Hypothesis. The five populations are initially
pooled and their response times ranked31. Following this ranking procedure, the five individual
rank sums are calculated corresponding to each day of the week ( specific operator). Finally, an
expression, the Kruskal- Wallis statistic, is calculated that possesses approximately a known
distribution ( Chi- square). In statistical hypothesis testing, there are no absolute assurances, only
likelihoods or probabilities. Thus one must also select, prior to performing the test, the
probability that the Null Hypothesis will be rejected when, in fact, it is true ( called the Type I
Error). Customary practice places this value at 5%. All that remains to perform is a table look- up
to identify the “ critical value” corresponding to the chi- square distribution with a particular
number for the degrees of freedom and a Type I Error of 5%. If the Kruskal- Wallis statistic is
larger than this look- up value, then the Null Hypothesis is rejected, i. e., the medians for the five
populations are significantly ( statistically) different and productivity is not equal across
operators.
Approximately thirty- five to forty individual tests were performed and covered each of the three
operator workstations, the two shifts, specific incident types ( e. g. accidents only, or traffic
hazards only) as well as all types aggregated together and spanned different day- of- the- week
combinations ( Monday- Friday, Tuesday- Thursday, Monday- Thursday, Tuesday- Friday).
Of these tests, 75% resulted in a rejection of the Null Hypothesis, i. e., a statistically significant
difference among the population medians under study. Hence, operator productivity is a
statistically significant parameter explaining differences among the observed response times.
Similar tests were performed for Wave 2 ( Monday, Wednesday, Friday) and again, in general, the
result indicated that operator productivity is a statistically significant parameter explaining
differences among observed response times.
Furthermore, it should be noted that during the last set of TIC interviews, approximately one-third
of the operators and one of the three supervisors mentioned operator performance as an area
of concern in achieving TIC operational goals ( 10).
30 The use of this test depends on the assumption that the populations are independent samples. Tables 3a and 3b
show that the operator for Monday and Friday is the same person for each OWS and shift, which weakens the
independence assumption. This can be avoided by omitting either Monday or Friday from the tests. Moreover, tests
were also conducted for the three- day period of Tuesday, Wednesday, and Thursday to factor out the possibility of
differences associated with weekend- related travel, i. e., on Monday and Friday.
3’ Ties are handled by assigning the half- way point to each record in a set of tied records, e. g., if records 21 and 22
each have the same response time, then each is ranked “ 2 1.5”, if records 2 1,22, and 23 each have the same response
time, then each is ranked “ 22”.
22
4.2 TIC to TATS
The Operator Activity Report was the only data source providing actual3* TATS time using a
single source of time ( TATS clock) across all operators. However, this data source was
incomplete, and contained TATS time for only 489 of the 708 CAD- generated incidents.
Moreover, for 39 of the 489 records, TATS data was entered before TIC data, resulting in a
negative value for the TIC to TATS time period and larger than expected CAD to TIC times.
Several operators were responsible for these 39 records, which could have also affected their
CAD to TIC times as well. For analysis purposes, only the remaining 450 records were used.
From Table 7, there is variability in TIC to TATS response time across operators and for each
operator. With the exception of the morning shift for OWS2 on Thursday33, the TIC to TATS
response times are within the range of 1.6 to 2.6 minutes. This is, nevertheless, large for the work
involved. The overall mean and standard deviation for the 450 records is 1.8 minutes and 0.8,
respectively.
Table 7: Wave 1 TIC to TATS Response Times ( Minutes)
Average ( Standard Deviation)
ows 1
Morning
Afternoon
ows 2
Morning
Afternoon
ows 3
Morning
1.8 ( 0.2) 1.7 ( 0.6) 2.6 ( 0.5) No records Afternoon , 2.3 ( 0.6)
1.9 ( 0.4) 1.7 ( 0.7) 1.8 ( 0.7) 2.0 ( 0.4) 2.3 ( 1.0)
For Wave 234 the Operator Activity Report was also incomplete, though only minimally so
compared to Wave 1 with only 33 missing records. However, as in the case for Wave 1, some
operators sometimes “ TATed” before “ TICing”, i. e., entered data into TATS before it was
formally entered into the TIC database. This occurred for 136 incident records, resulting in a
negative value for the computed TIC to TATS time and larger than expected CAD to TIC times
for those operators who did this. Three out of nine operators were responsible for two- thirds of
these records occurring on Monday, Wednesday, and Friday. For analysis purposes, only the
32 As previously mentioned TATS time from the OAR was adjusted to be synchronized with TIC and CAD time.
33 This time period includes only six data points one of which is an extreme outlier and if removed, would reduce the
average and standard deviation of the response time for this period to 1.4 and 0.7, respectively.
34 This table is based on Monday through Friday data modified to account for the missing CAD records.
23
remaining 465 records were used. From Table 8, there is variability in TIC to TATS response
times across operators and for each operator. Also note that there are three time periods during
which there are rather large averages and very large standard deviations for the TIC to TATS
time ( Monday at operator work station # 2 during the morning shift; Thursday at operator work
station # 2 during the afternoon shift; Friday at operator work station # 3 during the afternoon
shift). This is due to only four records with extreme outlier behavior. If they are removed from
the database, then the response time averages and standard deviations during these three periods
become, respectively: 1.6 ( 0.7); 1.2 ( 0.6); 1.4 ( OX), which are much more in line with all other
periods. The overall mean and standard deviation for the 463 records is 1.4 minutes and 2.9,
respectively. The overall mean and standard deviation after the outliers have been removed, i. e.,
461 records, is 1.2 minutes and 0.6, respectively.
Table 8: Wave 2 TIC to TATS Response Times ( Minutes)
Average ( Standard Deviation)
I MONDAY I TUESDAY I WEDNESDAY 1 THURSDAY 1 FRIDAY I ~ ows 1
Morning
Afternoon 1. O ( 0.7) 1.3 ( 0.2) 1.1 ( 0.4) 1.3 ( 0.3) 1.1 ( 0.4)
1.1 ( 0.2) 0.9 ( 0.2) 1.1 ( 0.6) 1.3 ( 0.2) 1.4 ( 0.5)
- 1 Mornin 2.8 ( 3.8) 1.3 ( 0.3) 1.2 ( 0.4) 1.1 ( 0.5) 1.5 ( 0.3)
No records 1.3 ( 0.2) 1.7 ( 2.9) 1.3 ( 0.3)
,
OWS 3
Morning
Afternoon 1.6 ( 1.1) No records 1.3 (----) 1.1 ( 0.2) 8.5 ( 18.7)
0.9 ( 0.1) No records 1.3 ( 0.6) 1.3 ( 0.5) 1.4 ( 0.2)
5. CONCLUSIONS
The response time analyses conducted over two waves indicate overall that for Wave 1 from the
CAD to the TIC and from the TIC to TATS the mean response times are 10.2 minutes and 1.8
minutes, respectively. For Wave 2 restricted to Monday, Wednesday, and Friday analogous times
are 10.1 minutes and I .4 minutes35. Thus there was a small decrease in the overall CAD to TATS
response times between the two waves.
Operator response times have generally remained fairly constant and stable during the Field
Operational Test with significant differences in response times across operators. The findings
indicate that improvement is needed in both overall operator work efficiency and operator
35 The CAD to TIC times for both waves are slightly different than reported in an early section of this report. This is
due to the fact that the estimates reported in this section have removed those records corresponding to those
operators who “ TATed” before they “ TICed”. Otherwise, the CAD to TIC plus TIC to TATS response times for
these records would have in essence double counted the time required to prepare a message for TATS.
24
response time. However, a formal operator quality control plan was not implemented until March
1998. Furthermore, based on frequent evaluation visits to the TIC throughout the FOT,
interviews with operators over the course of the FOT, and a lack of operator performance reviews
until March 1998, evaluators believe the findings from these two Waves are reflective of operator
performance that have existed throughout the FOT.
Other factors were also examined to assess their influence on operator response times including
the number and type of incidents ( e. g., traffic hazard, accident with major injuries) reported by
the California Highway Patrol, time of day, the location where the incident was processed, i. e.,
the specific geographic workstation, level of operator work experience at the TIC, incident
arrival rate, operator work- break periods, weather conditions, and system software and hardware
problems ( documented by operators). These factors, however, played only a minor role in
influencing response times.
Numerous operator work activities also affect operator response times and help provide the fuller
context within which response times were assessed. These activities included attempts to verify
an incident, disruptions caused by their own and other operators’ shift changes, operator call-checking
into the phone advisory system for quality control purposes, updating an incident
already entered into the system, searching for multiple listings for the same incident or those not
delaying traffic flow. However, these activities were difficult to isolate and individually quantify
their contribution to operator response times.
It is anticipated that once TravInfoTMe nters its more permanent and stable phase with a new
system manager, operations will soon become substantially more automated and this should have
a significant impact on reducing operator response times as well as increasing overall operator
productivity. It is recommended that once these changes are implemented, subsequent operator
response time assessments be made. Automating operations coupled with the much stricter
operator quality control measures that were put in place near the end of the FOT should combine
to make Traveler Information Center operations more like the “ timely” undertaking it was
intended to be.
6.
1.
2.
3.
REFERENCES
Miller M. A. and Hall, R. “ TravInfo Field Operational Test Traveler Information Center ( TIC)
Study ( Technology Evaluation Element) Implementation Plan”, California PATH Working
Paper, UCB- ITS- PWP- 95- 14, California PATH Program, University of California Berkeley,
( 1 995).
Bay Area Ad HOC IVHS Committee, “ TravInfo, Bay Area Intermodal Traveler Information
System,’’ October 1992.
Randolph Hall, Youngbin Yim, Asad Khattak, Mark Miller, Stein Weissenberger, “ Travhfo
Field Operational Test Evaluation Plan,” PATH Working Paper UCB- ITS- PWP- 95- 4, May
1995.
25
4. TRW, Inc. “ TravInfo Deliverable 2.1 System Requirements”, May 1994.
5. Miller M. A. and Loukakos, D. “ TravInfo Evaluation ( Technology Element) Traveler
Information Center ( TIC) Study ( September 1996 - June 1997)”, California PATH Working
Paper, UCB- ITS- PWP- 98- 7, California PATH Program, University of California Berkeley,
( 1 998).
6. Miller M. A. and Loukakos, D. “ TravInfo Evaluation ( Technology Element) Traveler
Information Center ( TIC) Study Operator Znterjizce Analysis- Phase HZ”, California PATH
Working Paper, UCB- ITS- PWP- 98- 22, California PATH Program, University of California
Berkeley, ( 1 998).
7. Freedman, D., Pisani, R., Purves, R., and Adhikari, A., Statistics, 2” d Edition, W. W. Norton
& Company, New York, 1991.
8. Freund, J. E., Statistics: A First Course 3rd Edition, Prentice- Hall, Inc., Englewood Cliffs,
New Jersey, 198 I .
9. Larsen, R. J. and Marx, M. L., An Introduction to Mathematical Statistics and its
Applications 2” d Edition, Prentice- Hall, Inc., Englewood Cliffs, New Jersey, 1986.
10. Miller M. A. and Loukakos, D. “ TravInfo Evaluation ( Technology Element) Traveler
Information Center ( TIC) Study Operator Inte$ ace Component- Phase ZV: Institutional
Analysis”, California PATH Working Paper, UCB- ITS- PWP- 98- 30, California PATH
Program, University of California Berkeley, ( 1998).
26