ISSN 1055- 1425
August 2007
This work was performed as part of the California PATH Program of the
University of California, in cooperation with the State of California Business,
Transportation, and Housing Agency, Department of Transportation, and the
United States Department of Transportation, Federal Highway Administration.
The contents of this report reflect the views of the authors who are responsible
for the facts and the accuracy of the data presented herein. The contents do not
necessarily reflect the official views or policies of the State of California. This
report does not constitute a standard, specification, or regulation.
Final Report for Task Order 6300
CALIFORNIA PATH PROGRAM
INSTITUTE OF TRANSPORTATION STUDIES
UNIVERSITY OF CALIFORNIA, BERKELEY
Health of California’s Loop Detector System
UCB- ITS- PRR- 2007- 13
California PATH Research Report
Ram Rajagopal, Pravin Varaiya
University of California, Berkeley
CALIFORNIA PARTNERS FOR ADVANCED TRANSIT AND HIGHWAYS
Health of California’s Loop Detector System:
Final Report for PATH TO 6300
Ram Rajagopal and Pravin Varaiya
Department of Electrical Engineering and Computer Science
University of California, Berkeley, CA 94720
Tel: ( 510) 642- 5270, Fax: ( 510) 642- 7815
{ ramr, varaiya}@ eecs. berkeley. edu
July 4, 2007
Abstract
The California Department of Transportation ( Caltrans) freeway sensor network has two com-ponents:
the sensor system of 25,000 inductive loop sensors grouped into 8,000 vehicle detector
stations ( VDS) and covering 30,500 freeway direction- miles; and the communication network
over which the sensor measurements are transported to Caltrans Traffic Management Centers.
The sensor network is virtually the only source of data for use in traffic operations, performance
measurement, planning and traveler information. However, the value of these data is greatly
reduced by the poor reliability of the sensor network: On a typical day in 2005, only 60 percent
of the statewide sensor network provided reliable measurements.
This is the report of an empirical study of the reliability of the sensor network based on two
data sets. The first set, obtained from PeMS, consists of a daily summary of the quality of data
received from each loop sensor in Caltrans Districts 4, 7 and 11 during the 27 month observation
period January 2005– March 2007. The second data set consists of reports of field inspections of
more than 4,000 loops each in Districts 4 and 7 during December 2005– December 2006 as part
of Caltrans’ Detector Fitness Program.
The study proposes and calculates three metrics of system performance: productivity is the
fraction of days that sensors provide reliable measurements; stability is the frequency with
which sensors switch from being reliable to becoming unreliable; and lifetime and fixing time—
the number of consecutive days that sensors are continuously working or failed, respectively.
Productivity measures the performance of the sensor system; stability measures the reliability
of the communication network; lifetime and fixing time provide more detailed views of both
components of the sensor network.
These metrics are used to evaluate the differences in system performance in Districts 4, 7 and
11. Productivity in District 11 is much better than in Districts 4 and 7; District 4 is slightly
worse than District 7. A significant part of the productivity difference is due to the large number
of sensors in Districts 4 and 7 that never worked during the 27 month observation period.
The stability metric shows that the communication network in all three Districts suffer short-term
outages; again, District 4 is the worst and District 11 is the best. The outages are likely
due to the communication network technology, including protocols, that is used in the different
Districts.
The metrics are also used to evaluate the effectiveness of the Detector Fitness Program ( DFP).
The DFP is unlikely to be cost- effective: two- thirds of the visited loops show no improvement in
system performance, the remaining one- third show marginal improvement. Simple suggestions
for a more effective design of the DFP are offered.
Lastly, the report proposes a statistical model of sensor failure that could be used in a scientific
approach to the maintenance and replacement of the sensor system.
1
Keywords: Sensor network reliability; loop detectors; Detector Fitness Program; freeway detector
system
Executive Summary
The California Department of Transportation ( Caltrans) freeway sensor network has two compo-nents.
The first component is the sensor system comprising 8,000 vehicle detector stations ( VDS)
that group 25,000 inductive loop sensors located on the mainline and ramps. The sensor system
produces 30- second averages of vehicle occupancy and volume measured by each sensor. The second
component is the communication network which transports these data to the Traffic Management
Centers ( TMCs) in each District.
The sensor network is virtually the only source of data used by a District TMC to make freeway
operational decisions. The data are also archived in the Performance Measurement System or PeMS,
which processes them for use in performance analysis, planning and traveler information.
The value of the data from the sensor network is greatly reduced by its poor reliability. On a
typical day in 2005, only 60 percent of the sensor network provides reliable measurements. The poor
reliability seriously degrades the quality of traffic operations and planning decisions. In an attempt
to improve reliability, Caltrans launched the Detector Fitness Program ( DFP) for the 12- month
period beginning December 2005.
This is a report of an empirical study of the reliability of the Caltrans sensor network and the
effectiveness of the DFP. The study was conducted under PATH Task Order 6300, “ Innovative
Topic: Maintaining the health of the Caltrans loop detector system.”
The study proposes three sensor network performance metrics: productivity, stability and lifetimes.
Productivity is the fraction of days that sensors provide reliable measurements; it measures reli-ability
of the sensor system. Stability is the frequency with which sensors switch from providing
reliable measurements to becoming unreliable; it measures reliability of the communication network.
Lifetime is the number of consecutive days that sensors continue working before they fail, and fixing
time is the number of successive days that sensors remain in a failed state before being fixed; they
provide detailed views of the reliability both of sensor system and communication network.
The conclusions of the study are grouped under District Performance, and DFP Evaluation.
District Performance
District 11’ s sensor network is much more reliable than District 7, which is more reliable than District
4. The difference is reliability can be analyzed in terms of three sensor groups:
1. A significant fraction of sensors had always failed during the study period in District 4 ( 24%)
and District 7 ( 16%); District 11 had few ( 3%) always failed sensors. Always failed sensors are
likely caused by permanent hardware faults in the sensor system ( broken loops, missing parts
in the controller, etc.) or, in some cases, misconfiguration.
2. A significant fraction of sensors had always worked in District 11 ( 40%) in contrast with District
4 ( 0%) and District 7 ( 4%). The difference is likely due to a communication failure diagnosed
by PeMS as ‘ insufficient number of samples’ received. Elimination of this failure state would
increase the fraction of always working sensors from 0 to 40% in District 4, from 4 to 30% in
District 7, and from 44 to 79% in District 11.
3. A majority of sensors are neither always failed nor always working: they fail and ‘ sponta-neously’
recover more than once. Sensors in this group fail more frequently and for longer
2
periods in Districts 4 and 7 compared with District 11. The failures are likely due to failures
in the communication network.
Detector Fitness Program
The DFP resulted in 4,578 visits to 3,244 individual loops in District 4 and 4,732 visits to 3,192
individual loops in District 7. The suspect loops were selected on the basis of the PeMS report of
loop status on a single day. The DFP records claim that 33% of visited loops in District 4 and 52%
in District 7 were fixed. In fact the ‘ success’ rate is much lower.
1. 27% of the visited sensors in District 4 and 16% in District 7 had always failed. The records
report that only 9% of these sensors were fixed. In fact, of the sensors that were claimed fixed,
only 40% in District 4 and only 24% in District 7 worked for two or more days after the visit.
Thus the actual success rate for visits to always failed sensors is must less than 5%.
2. The productivity of loops that were claimed to be fixed improved slightly after the visit;
surprisingly, the productivity of loops that were not claimed to be fixed improved by almost
the same amount. It seems that the improvement comes largely from loops for which crews
found hardware faults: ‘ missing equipment’ or ‘ modem issues’.
3. The stability of loops that were visited was unchanged by the visits. Thus the DFP program
in its present form cannot improve the communication network.
4. The maximum productivity improvement in Districts 4 and 7 from the Detector Fitness Pro-gram
is about 10%. Thus the DFP in its present form cannot significantly improve the per-formance
of the Caltrans sensor network.
Recommendations
Loop detector technology is obsolete. The sensor system consists of many subsystems, failure in any
of subsystems puts the associated sensor in a failed state. The sensor system cannot be remotely
diagnosed, so the cause of failure can be accurately determined only by an expensive field visit.
The communication network technology is also obsolete. In Districts 4 and 7, the communication
protocol does not re- transmit lost packets. The tests that PeMS performs on received samples does
not check for accuracy of the sensors. If such a check were done, the performance of the sensor
network would be significantly worse. The following tentative recommendations are suggested by
the analysis.
1. The always failed sensors— which account for between 14 and 25% of failed sensors in Districts
4 and 7— cannot be fixed by the DFP. It would be much more cost- effective to replace them.
More generally, loop sensors should not be replaced by other loop sensors.
2. The selection of loops to be visited by the DFP should not be based on a single day of data
from PeMS, but on the time series of performance as done in this study. A selection procedure
can be designed that will increase the chance of success from a DFP visit.
3. Sensors on ramps perform worse than mainline sensors, with performance in District 7 much
worse than in District 11. Since District 4 has no ramp sensors registered in PeMS, it is not
possible to determine their performance.
4. The analysis cannot pinpoint the reason why the sensor network in District 11 is so much
superior to District 4 and 6. It is likely that the disparity is the result of a history of good
maintenance in District 11 and a history of neglect in Districts 4 and 7. It is unlikely that
this neglect can be compensated by more resources devoted to the DFP. A more cost- effective
process is likely to result from selective replacement by a modern sensor network.
3
In conclusion, if the sensor network is to provide the data of a quality that is adequate to meet the
needs of the Strategic Growth Program, the sensor network should receive more serious attention
and resources than that provided by the Detector Fitness Program.
1 Introduction
The freeway sensor network of the California Department of Transportation ( Caltrans) has two
components: a sensor system and a communication network. The statewide sensor network is
divided into twelve parts, each built, operated and maintained by one Caltrans District.
The statewide sensor system consists of 25,000 sensors located on the mainline and ramps, and
grouped into 8,000 vehicle detector stations ( VDS). Each sensor records the presence of a vehicle
above it. The measurements are electronically processed in the VDS to produce 30- second averages
of vehicle counts or volume and vehicle occupancy. 1 Over 90 percent of the sensors use inductive
loops, most of the remaining use radar detectors. Following successful tests, wireless sensor network
technology is being introduced. The current loop- based system has a replacement cost of $ 320
million and is very expensive to maintain.
The communication network transports data packets from each VDS to its District Traffic Man-agement
Center ( TMC). Based on these data the District Advanced Traffic Management System
( ATMS) makes decisions about traffic operations. A copy of the data packets is also sent to the
freeway Performance Measurement System ( PeMS), which archives the data and processes them in
different ways to generate a variety of freeway system performance measures. The communication
network is built out of communication links that employ different technologies. For example, wireless
GPRS links predominate in District 4; telephone land lines are widely used in Districts 7 and 11.
The communication network comprises owned and leased facilities. The owned facilities cost $ 325
million. Caltrans incurs an annual communication network operating cost of $ 19 million.
As noted above, each District builds, operates and maintains the sensor network in its area. The
sensor system in the largest District ( District 7) covering Los Angeles and Ventura counties has
8,700 loop sensors; the nine- county San Francisco Bay Area District 4 has 4,600 loop sensors; and
District 11, covering San Diego and Imperial counties, has 3,100 loop sensors. We study the sensor
networks in these three Districts, first using data from PeMS.
PeMS expects to receive from each sensor one sample ( packet) every 30 seconds. Based on the
number and quality of the samples that it actually receives from a sensor on a given day, PeMS
designates that sensor as ‘ good’ or ‘ failed’ for that day. The fraction of failed sensors summarizes
the reliability of the sensor network for the entire state, an individual District or freeway.
Figure 1 plots the percentage of failed sensors for each day from 10/ 10/ 2005 to 12/ 31/ 2005 for the
whole state and for Districts 4, 7 and 11. The sensor network has very poor reliability, with 35
percent of sensors statewide considered failed on any given day. The reliability varies widely by
District: District 4 had 40 percent, District 11 had 5 percent, and District 7 had 35 percent failed
sensors.
The poor reliability of the sensor network, and the pressing need to use sensor data for better
freeway operations decisions, led Caltrans to launch the Detector Fitness Program ( DFP) beginning
December 2005. The goal of the DFP was to significantly raise the reliability of the statewide
sensor network. Over the next 12 months, crews made 9310 visits to sensors in the field in order to
diagnose why they had failed and, if possible, to fix the failure. As will be seen later, the resulting
improvement in the reliability of the sensor network is measurable. But the improvement is small
and it seems clear after 12 months of the DFP that by itself it cannot significantly improve reliability.
The DFP produced detailed reports, and our analysis of those reports explains why the impact of
1In some cases, a pair of sensors form a ‘ speed trap’ and provide a measurement of speed.
4
Figure 1: Daily fraction of failed sensors for the statewide system, and Districts 4, 7 and 11 from
10/ 10/ 2005 to 12/ 31/ 2005.
the DFP is limited.
Most sensors behave like light bulbs: once they fail, they stop functioning for ever. The freeway
sensors are quite different: they repeatedly fail and then ‘ spontaneously’ recover from failure, as is
evident from the oscillations in Figure 1. Thus, metrics designed to measure the reliability of systems
with light bulb- like failures cannot be used for the freeway sensor system. The study proposes three
different ways of measuring the reliability of the freeway sensor system: productivity, stability, and
lifetime and fixing time.
Productivity is the distribution of the fraction of days that sensors provide reliable measurements.
Stability is the distribution of the frequency with which sensors switch from providing reliable mea-surements
to becoming unreliable. Lifetime is the distribution of the number of successive days
that sensors continue working before they fail, and fixing time is the distribution of the number of
successive days that sensors remain in a failed state before being fixed. The study uses these metrics
to compare the reliability of the sensor networks in Districts 4, 7 and 11, as well as to evaluate the
effectiveness of the Detector Fitness Program.
The remainder of this report is organized as follows. Section 2 describes the sensor network in a
way that shows the kinds of hardware and software faults that can lead PeMS to declare a sensor
failure. Section 3 summarizes the two data sets that are used. Section 4 gives the distribution of
sensors on different highways.
Section 5 considers sensors that are always in a failed state, and argues that these are likely to be a
software configuration error; hence they are excluded from most of the subsequent analysis.
Section 6 introduces the scope chart, which gives a visual summary of the state of the sensor network
in an entire District over a period of two years.
5
Section 7 defines productivity and computes the productivity of the three Districts. Section 8 defines
and evaluates stability. Section 9 calculates the lifetime and fixing time distributes.
Section 10 focuses on the overall communication network and attempts to evaluate its failures.
Section 11 attempts to evaluate the failures of individual communication links. Section 12 analyzes
the Detector Fitness Program and attempts to evaluate its effectiveness. Section 13 collects some
conclusions.
Appendix A proposes a Markov chain model of failure and recovery. Appendix B exhibits some
fragments of the DFP data sheets. Appendix C explains why the number of sensors in District 4
varies from year to year.
2 Sensor fault description and PeMS failure states
To understand how data are collected and what faults can occur we describe how the sensor network
works. Figure 2 is a schematic of the sensor network in District 4. At a particular VDS location,
there is a sensor in each lane of the freeway. In more than 90 percent of the locations the sensor is
an inductive loop, represented by the little circles in the figure. Sensors from the different lanes are
connected through a pull box to a controller cabinet on the side of the road.
The cabinet includes a 170 controller and a modem. The controller detector cards process each
sensor’s measurements to produce 30- second averages of vehicle occupancy and volume, and format
these data into a packet, which includes fields indicating the VDS and sensor IDs ( identifiers). The
cabinet receives power from a local power line.
The TMC receives the data packets from the controller over a digital communication network. The
network has two parts, one of which is Caltrans- operated and the other is Telco- operated.
A Caltrans- operated field line connects the controller cabinet and modem to the Telco demarc box;
optionally a field bridge connects multiple controllers to the Telco demarc box. A Telco bridge
connects multiple demarc boxes to a TMC Line inside the Telco network. The TMC Line connects
to the front- end processor ( FEPT) of the District TMC. Up to 20 controllers may share the same
Telco line, the different controllers being distinguished by a Drop ID.
The FEPT received data by polling the controller modems. The received packets are forwarded to
the District ATMS; a copy is also forwarded to PeMS.
Caltrans deploys several variations of the sensor network. A small fraction of the sensors use radar
to detect the presence of a vehicle. However, the radar- based systems also produce data packets
with the same format. There is a greater difference in the communication network. Some controller
cabinets in District 7 are connected to the TMC over Caltrans- owned optical fiber links. More
significant is the use of wireless links rather than land lines as in the figure. For example, District 4
uses the GPRS data service.
Thus the overall sensor network combines several hardware and software subsystems. Each sub-system
is a potential source of failure. The main subsystems are the inductive loop; the detector
card for each sensor; the controller; the Caltrans- operated communication sub- network; and the
Telco- operated communication sub- network.
When PeMS receives a data packet, it consults a configuration table to interpret the data packet.
The table contains meta information that helps determine whether the VDS and sensor IDs in the
packet are valid and where the sensors are located ( mainline, ramps). If a packet contains an ID that
is not recognized by the table, the packet is discarded. Conversely, if there is no packet corresponding
to an ID in the table, it is assumed that there is a failure in the system corresponding to that sensor.
Every midnight, PeMS examines the sequence of ( data) samples received from each detector; subjects
6
Figure 2: The configuration of the sensor network in District 7
the sequence to a set of statistical tests; and classifies the detector ‘ health’ for that day into one of
10 diagnostic states displayed in the first column of Table 1. A correctly functioning detector should
daily provide 2,880 30- sec samples with reasonable values. The statistical tests involve the number
of received samples and their values. The second column of the table indicates the nature of the
tests, and a detailed description is available in [ 1].
The first nine diagnostic states indicate failure; the tenth state, ‘ good’, indicates a functioning
detector. The plots in Figure 1 refer to the daily fraction of failed sensors.
Comparing the configuration of the sensor network in Figure 2 with Table 1 we see that knowledge
of the sample sequence received by PeMS from a particular sensor is not enough to uniquely relate a
failure diagnostic state with an actual hardware or software fault. For instance, if a sensor delivers
insufficiently many samples or no samples at all, this may be due to the controller being down or to
a failure of a communication link or an error in the configuration table. Therefore we will aggregate
the 10 diagnostic states provided by PeMS into two or three ‘ macro’ states: good, sensor system
failure, and communication network failure. Detailed fault analysis will be conducted on data from
the Detector Fitness Program since those manually compiled records are obtained from a field visit.
7
Diagnostic
State
Description Detector
Types
Line Down No detector on the same communication line as the selected detec-tor
is reporting data. If information about communication lines is
not available this state is omitted.
ML,
Ramps
Controller
Down
No detector attached to the same controller as the selected detector
is reporting data. This may indicate no power at this location or
the communication link is broken.
ML,
Ramps
No Data The individual detector is not reporting any data, but others on the
same controller are sending samples. This may indicate a software
configuration error or bad wiring.
ML,
Ramps
Insufficient
Data
Insufficient number of samples are received to perform PeMS diag-nostic
tests, while other detectors reported more samples.
ML,
Ramps
Card Off Too many samples with an occupancy ( for ML and HOV detectors)
or flow ( for ramps) of zero. The detector card ( in the case of loop
detectors) is probably off.
ML,
Ramps
High Val Too many samples with either occupancy above 70% ( for ML and
HOV detectors) or flow above 20 veh/ 30- sec ( for ramps). The
detector is probably stuck on.
ML,
Ramps
Intermittent Too many samples with zero flow and non- zero occupancy. This
could be caused by the detector hanging on.
ML
Constant Detector is stuck at some value. ( PeMS counts the number succes-sive
occurrences of the same non- zero occupancy value.)
ML
Feed Unsta-ble
The data feed itself died and there were insufficiently many samples
during the day to run the tests. On days where this occurs we mark
the detectors that were previously good as good and the ones that
were previously bad as Feed Unstable.
ML,
Ramps
Good Detector passed all tests ML,
Ramps
Table 1: Diagnostic states
3 Data used
We describe the data used in the study and then the pre- processing steps taken to convert the data
into a standard format used in the subsequent analysis. Two data sets are used.
The first data set consists of the sequence or time series of daily sensor diagnostic states in PeMS
for each loop in Districts 4, 7 and 11 as described in Table 1.2 The loops considered in the study
are those that were listed in the PeMS configuration table on March 31, 2007. For each loop the
sequence of days spans 27 months from January 1, 2005 to March 31, 2007; for loops that were
installed on a later date, the sequence begins later.
The second data set comprises records from the Detector Fitness Program ( DFP) for Districts 4 and
7. These records were created by crews following a field visit to a loop. The records are textual and
their format is not standardized; hence they require some interpretation on our part, as explained
next.
For District 7, the visits occurred in clusters between December 12, 2005 and August 2, 2006. The
records corresponds to a total of 4,732 visits. 3192 individual detectors were visited, implying that
several detectors were visited more than once. For different clusters, the data are recorded in different
ways on a spreadsheet, possibly because there were different crews. Each row of the spreadsheet
2We use ‘ sensor’, ‘ loop’ and ‘ detector’ interchangeably.
8
corresponds to a visit to an individual loop. Typically, the records contain the following fields:
• Location: comprising the Detector Station ( VDS) to which the loop is connected, lane number,
highway name, highway direction, and postmile.
• Visit date: typically the date the loop was visited, but sometimes the date the record was
entered in the system. It is safe to say that the sensor was visited before this date.
• Problem type: typically a textual description of either the diagnostic state reported by PeMS
or the type of problem encountered.
• Related cause: typically the failure cause as surmised by the crew, frequently the name of a
broken or missing hardware part.
• Solution: typically the steps taken towards the solution of the problem, and whether the
problem was successfully resolved.
• Status: the PeMS diagnostic state after visit.
The column titles are not uniform across different spreadsheets for District 7. For example, some-times
solution and problem type are described in an extra column labeled ‘ Comments’.
For District 4, the records summarize a total of 4,578 visits between December 12, 2005 and December
30th, 2006. 3244 individual detectors were visited. No visits were reported during February 7, 2006–
April 10, 2006 and August 3, 2006– December 3, 2006.
District 4 records are similar to those for District 7, although for some records, a manually entered
code is given for the cause of failure together with the solution. This code is not fully utilized in
this paper because it mixes the cause of failure with the possibility of solution.
3.1 PeMS data pre- processing
Let the time series si( n) denote the diagnostic state as determined by PeMS for sensor i on day n;
si( n) takes one of the 10 values listed in the first column of Table 1. We merge several diagnostic
states together to obtain a new time series s  i
:
s  i
( n) = 8 < :
1 if si( n) 2 { Good} 0 if si( n) / 2 { No Data, Controller Down, Good}
− 1 if si( n) 2 { No Data, Controller Down}
. ( 1)
In some cases ( as will be made clear), we modify the conditions above to
s  i
( n) = 8 < :
1 if si( n) 2 { Good} 0 if si( n) / 2 { No Data, Insufficient Data, Controller Down, Good}
− 1 if si( n) 2 { No Data, Insufficient Data, Controller Down}
. ( 2)
The aim of this coding scheme is this. s  i
( n) = 1 means that both the sensor system and the
communication network associated with loop i were functioning on day n. s  i
( n) = 0 means that
the communication network associated with loop i on day n was functional, since some packets were
received ( si( n) 6 = No Data), but there was a failure in some part of the sensor system. Lastly,
s  i
( n) = − 1 corresponds to a communication network failure as no data were received. Sometimes,
following ( 2), ‘ Insufficient Data’ is treated as a communication failure.
Thus s  i
( n) encodes three conditions:
s  i ( n) = 1 ) good, s  i
( n) = 0 ) sensor system failure, s  i
( n) = − 1 ) communication network failure.
( 3)
9
The aim of the analysis is to understand the persistence of the ‘ good’ state and the occurrence of the
two failure states. Note that by using ( 1) more failures are attributed to the sensor system, whereas
by using ( 2) more failures are attributed to the communication network.
If there is a ‘ communication network failure’ ( s  i
( n) = − 1), we cannot say whether the sensor system
has also failed, because a communication failure masks or censors the corresponding sensor system
observation. How can we estimate the sensor system state when there is a communication failure?
One approach is to build censored estimators, which can be computationally very expensive and
requires a parametric model for sensor system failures. A simple alternative, in keeping with the
non- parametric approach we have adopted here, is to fill in the sensor system failure value with its
last known value. Thus if s  i
( n) = − 1, we set s  i
( n) = s  i
( n − 1). If s  i
( n) = − 1 on the first day of
the series for loop i we ignore the series until the very first day for which s  i
( n) 6 = − 1.
We call the new resulting sequence a filled sequence. Filling can be done with respect to a { No
Data, Controller Down} state ( see ( 1)) or with respect to a { No Data, Controller Down, Insufficient
Data} state ( see ( 2)). Both reflect communication failures. The difference lies in the interpretation
of insufficiently many samples received. Could it be that the controller card is damaged, in which
case it is a sensor system failure? Or are samples lost, indicating a communication network failure?
Wherever necessary we make the distinction during the analysis. A filled sequence is one in which
{ No Data, Controller Down} are filled. A ND- filled sequence is one in which states { No Data,
Controller Down, Insufficient Data} have been filled.
3.2 Detector Fitness Program data pre- processing
The DFP maintenance records do not follow a uniform format. The recording frequently was not
very careful. For example in District 4, 25% of the records report no underlying cause of failure. In
District 7, only 4% of the records suffer from this deficiency. Such recording procedures poses an
additional challenge to the data analysis.
For each visit to a sensor an entry in the maintenance record is created and the observed failure
and actions taken are recorded. One problem is that no systematic way of recording the observed
failures was followed. For example the following are typical entries for observed failures:
1. open loops SB lane 3. disabled channels
2. open loop
3. open loops/ DLC. should be eb lanes
4. no EBML. DLC not present in cabinet
5. no WBML lane 7
6. new cabinet. no equipment
The first item claims that the sensor was damaged ( open loop), and the resulting action was to
disable the communication channels. In the third and fourth items, DLC, EMBL and WBML are
parts of the sensing equipment, so they characterize “ Bad or Missing Equipment” cases. Appendix
B displays a fragment of typical spreadsheets that contained the data and some more explanations.
Another problem as seen in the examples above is that the cause for the observed failure and the
actions taken are sometimes encoded in the same sentence. More examples can be seen in appendix
B. Given the thousands of DFP records, it is not practicable to read the records one by one and
encode the information manually for subsequent analysis.
10
Therefore we used a simple parsing scheme to encode the textual records. We created 9 non- mutually
exclusive classes: Upgrade Firmware, Under Construction, Open Loop, Connection Issues, Modem/ Card
Issues, Reset Equipment, No Power, Other Issues and No reported cause. For each class we seek specific
keywords or combination of keywords in the text. If the keywords are observed a ‘ 1’ is entered for
that class for that record; otherwise a ‘ 0’ is entered. We manually checked many of the assignments
made in this way and they worked reasonably well, because failure descriptions use a much smaller
vocabulary than freeform text.
Each record also contains an entry that tells us if the sensor was fixed ( and thus left working) or not
fixed. This entry will be important in analyzing the effectiveness of the DFP. Lastly the analysis
uses the recorded visit date as a proxy for the true visit date.
Thus for each visit we end up with 13 variables: the sensor visited, the visit date, whether the sensor
was fixed or not, and an indicator of 9 possible non- exclusive failure causes.
4 Sensor distributions
Table 2 gives the total number of sensors in District 43, and the highways with the largest number
of sensors. The corresponding numbers for Districts 7 and 11 are in Tables 3 and 4. Table 5 shows
the sensor distributions per lane for each District. These sensors cover 2,870 miles in District 4,
2,318 miles in District 7, and 2,060 miles in District 11.
Highway Sensors (%)
101 1315 22.7%
680 922 15.9%
80 846 14.6%
880 815 14.1%
Other 1884 32.6%
Total 5782 100.0%
Table 2: Sensor distribution for District 4 on March 31, 2007.
Highway Sensors (%)
10 1374 15.8%
405 1153 13.2%
5 1068 12.3%
101 733 8.4%
210 604 6.9%
605 592 6.8%
60 556 6.4%
Other 2627 30.2%
Total 8707 100.0%
Table 3: Sensor distribution for District 7 on March 31, 2007.
We consider on- ramp and off- ramp as well as mainline sensors.
3See Appendix C for disabled sensors in District 4, which explains differences with respect to PeMS.
11
Highway Sensors (%)
5 716 21.9%
15 656 20.1%
805 426 13.1%
8 479 14.7%
Other 987 30.2%
Total 3264 100.0%
Table 4: Sensor distribution for District 11 on March 31, 2007.
District Lane 1 Lane 2 Lane 3 Lane 4 Lane 5 Lane 6 Lane 7
4 1531 1515 1379 1043 255 51 7
7 3831 1788 1478 1193 349 52 8
11 1108 956 567 401 177 48 5
Table 5: Sensor distributions in lanes 1- 7 on March 31, 2007.
12
5 always- failed and always- working sensors
Examination of the PeMS diagnostic sequences reveals a significant fraction of sensors that are
always failed ( i. e., never worked) or always working. We begin by analyzing the statistics of such
sensors. Sensor i is called always- 0 if the sensor is assigned a failed state for the entire period T,
i. e., s  i
( n) = 0, n 2 T. It is called always- 1 if a failed state is never observed for the entire period
T, i. e., s  i
( n) = 1, n 2 T. T is taken to be one quarter or one year. ( See ( 1) for definition of s  i
.)
We use filled sequences to count always- 0 and always- 1 sensors, unless explicitly indicated oth-erwise.
This means that communication failures are not considered to be faults for this analysis.
The type of filled sequence does not affect the always- 0 status, but it does affect always- 1 status,
because if ‘ Insufficient Samples or Data’ is regarded as a communication failure, some sequences
that include 0 values can become always- 1 .
Table 6 shows the number and percent of always- 0 and always- 1 sensors in the three different
Districts. There is a large number of always- 0 sensors in District 4 and District 7 compared
to District 11. This discrepancy by itself accounts for a considerable portion of the performance
difference between Districts.
Furthermore, District 4 and District 7, in contrast with District 11, have almost no always- 1
sensors. The increase of always- 1 sensors from 2005 to 2006 in District 7 is due in part to sensor
misconfigurations that were corrected and in part to the DFP. Notice also that the increase in
always- 0 sensors in District 11 is mainly due to the inclusion of ramp sensors in PeMS starting in
2006.
District ( Year) Total always- 1 always- 0
District 4 ( 2005) 5140 9 ( 0%) 1171 ( 23%)
District 4 ( 2006) 5271 0 ( 0%) 1327 ( 25%)
District 7 ( 2005) 6478 21 ( 0%) 1090 ( 17%)
District 7 ( 2006) 8613 319 ( 4%) 1399 ( 16%)
District 11 ( 2005) 1750 604 ( 35%) 38 ( 2%)
District 11 ( 2006) 3223 1402 ( 44%) 116 ( 4%)
Table 6: Failure summary for always failed and always working sensors.
For 2006, we separate mainline from other sensors ( located on ramps) and show the distribution of
always- 0 sensors. Table 7 shows that ramps have a larger fraction of always- failed sensors. Also,
District 4 has no ramp sensors registered in PeMS.
District ( Year) Total Mainline always- 0 (%) Total Other always- 0 (%)
District 4 ( 2006) 5269 1327 25% 2 0 0%
District 7 ( 2006) 5932 847 14% 2681 552 21%
District 11 ( 2006) 2162 49 2% 1061 67 6%
Table 7: Failure summary for always failed sensors classified by type ( Mainline or Other).
In Tables 6 and 7, we considered communication failures as an actual fault. If we use an ND-filled
sequence, which classifies ‘ Insufficient Data’ as a communication failure, then the number of
always- 1 sensors increases while the number of always- 0 sensors remains the same, as expected
( Table 8). The increase in the number of always- 1 sensors assumes that the sensor system reliability
is unchanged during a communication failure. This indicates that at least part of the performance
difference among Districts can be attributed to communication network failures.
Comparison of Tables 6 and 8 leads to the following conclusion.
13
District ( Year) Total always- 1 always- 0
District 4 ( 2006) 5271 2106 ( 40%) 1327 ( 25%)
District 7 ( 2006) 8613 2590 ( 30%) 1399 ( 16%)
District 11 ( 2006) 3223 2544 ( 79%) 116 ( 4%)
Table 8: Failure summary for always failed sensors in ND- filled sequences.
Conclusion 1 The large number of always- 0 sensors in Districts 4 and 7 account for a significant
of the poor reliability of their sensor system. If the diagnostic state ‘ Insufficient Data’ is caused
by a communication network failure and if this failure can be eliminated, the percent of always- 1
sensors will increase from 0 to 40% for District 4, from 4 to 30% for District 7, and from 44 to 79%
for District 11, resulting in a dramatic improvement in the performance of the sensor system as a
whole.
To conclude the section, we check how always failed sensors are distributed among different highways
and lanes. We only report on District 4, as the results are similar for the other Districts. Figure 3
shows that the distribution of failures is even across highways. Highways with a very large percentage
of always- 0 sensors have very few sensors as shown in Figure 3.
Figure 3: Distribution of number ( top) and percent ( bottom) of always- 0 sensors by highway for
District 4 ( 2004)
We can similarly analyze the always failed sensor distribution by lane. We choose a particular
highway ( US- 101) for this plot. This highway has a high volume of trucks, which tend to use the
slower lanes ( lanes with higher number). Figure 4 shows that the outermost lanes have a higher
number of failures, suggesting that heavy traffic use may cause more permanently failed sensors. A
statewide view is shown in Figure 5.
We can also investigate whether permanent failures of individual loops are caused by failure of the
controller itself, in which case all loops attached to the controller would fail. ( See ‘ Controller Down’
14
Figure 4: Distribution of number always- 0 by lane for highway US- 101 ( top) and I- 680 ( bottom)
in District 4 ( 2004)
in Table 1.) In our case we can conclude that a controller has failed if all the sensors attached to the
controller are in always- 0 state for the chosen time period. Figure 6 shows this distribution. About
100 controllers are always failed, whereas about 600 controllers have a few ( but not all) always failed
loops. We may conclude that an always failed sensor is not strongly related to the a possible failure
of the controller.
Conclusion 2 Always failed loops are not primarily caused by ‘ Controller Down’ failures.
6 System view
In this section we introduce a novel view— the scope chart— of the sensor system state, based on
visualizing the fault sequence over time and across highways. This visualization technique provides
15
Figure 5: Distribution of always- 0 loops by lane for District 4 ( 2004)
16
Figure 6: Distribution of fraction of always- 0 loops in a controller ( top) and distribution of possible
broken controllers by highway in District 4 ( bottom) ( 2004)
a global view of the system or of parts of the system. 4 We first describe how such plot is constructed.
For each sensor i, compute the state sequence s  i
( n), which assumes values 1 ( sensor is good on
day n), - 1 ( communication network failure on day n) and 0 ( sensor system failure on day n) ( see
( 1)). The plot is a two- dimensional ‘ heat’ map ( 1 = red, - 1 = blue, 0 = green). The horizontal
axis is time in days. ( The sequences cover 27 months or 810 days.) The vertical axis corresponds to
some ordering of all sensors. In Figure 7 for District 7 all sensors on the same highway are grouped
together ( beginning with highway I- 5 and progressing to I- 710) and within each highway group they
are ordered by postmile and lanes.
In the chart we can clearly see horizontal red lines representing sensors that worked for long periods.
A blue streak in the horizontal direction indicates a sensor that did not report data for a long period.
Blue streaks in the vertical direction correspond to days when many sensors sent no samples. This
could be caused by a communication network failure in which several TMC lines failed or the FEPT
4Scope charts are now a feature of PeMS7.3.
17
was unable to poll many modems ( see Figure 2). Such streaks explain the oscillations observed
in the total number of failed sensors in Figure 1. The scope chart also allows us to compare the
reliability of different highways.
The scope charts in Figure 7 can be compared with each other. The charts suggest that in general
District 11 has a much more reliable sensor system. In particular, there are fewer communication
failure streaks in District 11 than in the other two Districts ( the blue streaks at the leftmost side
of the chart usually corresponds to dates before the sensor was installed into the system). This
reinforces the importance of the communication network between the controller modems and the
FEPT.
Another ordering is by the number of observed working days, with an increased weight for more
recent days. An exponential weight function is chosen, with the parameter tuned so that working
days further back in time are considered less valuable then those more current. This gives a boundary
curve for the working sensors, such as that shown in Figure 8 for District 7. If the boundary curve
is concave, then the system performance is clearly improving over time, whereas a convex boundary
curve indicating the system performance is becoming worse.
Two boundary curves can be compared using their shapes as intuitively seen in the figure. By
comparing the shape resulting from the dark region, we see that the figure on the right has a larger
dark region, implying that more sensors were working in 2006.
Conclusion 3 The scope chart provides an excellent summary of the performance of the sensor
network in a District or on a particular highway. It permits comparison of performance across
Districts and over time. It is now a feature of PeMS7.3.
7 System productivity
In this section we propose a measure of productivity of a District’s sensor network. The measure
is computed as follows. Consider a time interval T and a sensor set M of size M. For each sensor
m 2 M we calculate the percent of days dm that the sensor is working as wm = 100[ dm/ T ]. The
productivity of M, PM( x), x 2 [ 0, 100] is the cumulative frequency distribution of wm:
PM( x) =
1
M
M X m= 1
1 ( wm  x) . ( 4)
PM( x) is the fraction of the sensors that worked for at most x% of days. Evidently, sensor set Ma
has strictly better productivity than Mb if PMa ( x) < PMb ( x) for all x. A single number to compare
two sensor sets is the total productivity ( TP) defined as the area above the productivity function,
TPM = 1 − Z 100
0
PM( x) dx, ( 5)
which of course is the empirical average of wm,
TPM =
1
M
M X m= 1
wm. ( 6)
If we model the sensor state as a two- state (‘ good’ and ‘ failed’) stationary Markov chain, TP is the
steady- state probability of the chain being in the ‘ good’ state.
18
Figure 7: Scope chart ordered by highway, postmile and lane for Districts 4 ( top), District 7 ( middle)
and District 11 ( bottom), 2005- 2007. Red streaks corresponding to Good state, green to Bad and
blue to Communication network failure.
19
Figure 8: Scope chart ordered by number of ones ( more recent), District 7, 2005 ( left) and 2006
( right)
7.1 Empirical estimates of productivity
We compute the productivity of the sensor networks in Districts 4, 7 and 11, using the raw ( non-filled)
data sequence ( 1). We omit all sensors that are always- 0 for the chosen time horizon. The
reason for this choice is that the always- 0 analysis has already been carried out, and these sensors
affect the obtained curves and make the interpretation more difficult.
Figure 9 displays the results. For any point on the curve take the y- ordinate ( say 20%), determine
the corresponding x- ordinate ( say 40 days), and interpret the point to mean 20% of the sensors
worked for less than 40% of the time. Alternatively, 80 % ( 100%- 20%) of the sensors worked for
more than 40% of the time. The total productivity of the sensor network is the area above the
productivity curve.
For District 7, productivity in 2006 is strictly better than in 2005, presumably a result of the
Detector Fitness Program ( DFP). For District 11 productivity remained unchanged, and is strictly
better than the productivity of both Districts 4 and 7. For District 4, the median productivity for
both years was unchanged ( y = 50%), with an improvement for sensors with performance below
median in 2005 and worse for those above the median. The effect of the DFP for District 4 is mixed.
To improve our understanding of the productivity of Districts 4 and 7, we calculate the productivity
in 2006 of the sensor network in specific highways with the largest number of sensors. Figure 10
shows the results. In District 4, the choice of highway has little influence. In District 7, the sensor
network productivity for US- 101 and I- 405 is significantly better than for I- 10 and I- 5. Comparing
the productivity for these highways during 2005 ( Figure 11) we see that I- 5 is the worst in both
years.
We probe further by examining productivity by lane for selected highways, Figure 12. For US- 101
in District 4 and I- 10 in District 7 the productivity across lanes is almost identical. But for I- 5 in
District 7, lane 2- 4 are similar, but lane 1 is worse.
20
Figure 9: Productivity of District 4 ( top), District 7 ( middle) and District 11 ( bottom), 2005 and
2006
21
Figure 10: Productivity of District 4 ( left) and District 7 ( right) for some highways ( 2006)
Figure 11: Productivity of District 7 for some highways ( 2005)
22
Figure 12: Productivity of US- 101 in District 4 ( top left), I- 5 ( top right) and I- 10 ( bottom) in
District 7 ( 2006)
23
Conclusion 4 The productivity metric is the most important measure of performance of the sensor
network in a District. For District 7, productivity improved from 2005 to 2006, possibly as a result
of the Detector Fitness Program. For District 4, there was an improvement in sensors that were
performing poorly in 2005. For District 11, productivity was unchanged and remained at its high
level. I- 5 in District 7 continues to perform poorly. There is no significant variation in productivity
by lane.
8 System stability
From Table 6 we know that the majority of sensors switch between good and failed states one or
more times. ( These are the sensors that are not always- 0 or always- 1 .) Sensors with the same
productivity may switch different number of times. We propose a simple system metric that captures
this difference.
For a sensor set M of size M and time interval T, we compute the normalized number of state
changes sm = ( r10, m + r01, m)/ T , where r10, m is the number of times sensor m switches from the
good state to a failed state during T and r01, m is the number of switches from a failed to the good
state. The stability of M, SM( x), x 2 [ 0, 100] is the cumulative distribution of sm:
SM( x) =
1
M
M X m= 1
1  sm 
x
100  . ( 7)
SM( x) is the fraction of sensors that switched states on at most x% of the days. The total stability
TS is the area below SM( x):
TSM = Z 100
0
SM( x) dx. ( 8)
Sensor setMa is strictly more stable than a setMb if SMa ( x) > SMb ( x) for all x. Ma is on average
more stable than Mb if TSMa > TSMb . If we model the sensor state as a stationary two- state
Markov chain, its two transitional probabilities are determined by its total productivity and total
stability ( see Appendix A).
8.1 Empirical estimates of stability
Using the raw data, we estimate the stability of different Districts. We discard sensors that are
always- 0 , as they were considered separately. The average stability of the system is just the area
below the stability distribution curve.
Figure 13 compares the stability of districts District 4, District 7 and District 11 for 2005 and 2006.
For a point on the plot, suppose its y- ordinate is 50% and the corresponding x- ordinate is 5. This
means that 50% of the sensors switched 5 or fewer times during a 100 day period. In the figure,
District 4 is less stable in 2006 than in 2005, as the stability curve in 2005 strictly dominates 2006.
The median number of switches increased from 5 in 2005 to 7 in 2006. This may be due to the large
number of changes in the system configuration in 2006 ( see Appendix C).
The sensor network in District 7 was more stable in 2006 ( median 3) than in 2005 ( median 4). Also
this District’s sensor network is more stable than District 4. In District 11 we see no change in
stability between 2005 and 2006. The median number of switches is 1, much better than Districts 7
and 11. Notice also the large number of sensors with 0 number of switches. These are the always- 1
sensors.
24
Figure 13: Stability of Districts 4, 7 and 11 ( 2005- 2006)
Figure 14: Stability of District 4 and District 7 for some highways ( 2006)
We drill down further by computing the stability for specific highways in 2006 for District 4 and
District 11. Figure 14 displays the results. For District 4, only I- 80 is different, being more unstable
than US- 101, I- 680 and I- 880. Since the productivity is essentially the same for these highways,
this implies that the sensors in I- 80 although they worked on average as much as the other sensors,
switch more frequently.
In District 7 we see a similar phenomenon for I- 5 compared to other highways. In this case I- 5 is
strictly dominated by its counterparts. This also matches up with the poor productivity of I- 5 when
compared to other highways. The difference is not extreme ( for example the median switching for
25
Figure 15: Stability of District 4 for US- 101, District 7 for I- 5 and I- 10( 2006)
I- 5 is 5, and for the other highways is about 4). This reinforces the hypothesis that there could be
some essential difference between the sensor network in I- 5 and in other highways.
To conclude this section, we compute the stability in 2006 for selected highways, group the sensors
by lane. Figure 15 displays the results. Observe that the stability does not vary by lane in US- 101
in District 4 or I- 5 and I- 10 in District 7.
Conclusion 5 Stability is a measure of how frequently individual sensors switch between working
and failed states. The sensor network in District 4 was less stable and in District 7 it was more
stable in 2006 compared with 2005; District 11’ s stability was unchanged and continued to be much
better than Districts 4 and 7.
9 Lifetime Estimates
Estimation of lifetime or survival curves is the standard approach in statistics for characterizing
system failures ([ 3, 4]). In this approach a number of individuals are observed starting at varying
initial times and their failure times are recorded. Records of individuals that did not experience
failures during the observation period will be right- censored as we don’t know when they would have
failed. The survival curve is the complement of the cumulative distribution of time to failure. The
standard non- parametric estimators of the survival curve are the Nelson- Aalen and Kaplan- Meyer
estimators [ 3, 4]. These estimators are appropriate only for individuals experiencing a permanent
26
failure rather than recurring failures.
In the California sensor network, many failed sensors ‘ spontaneously’ start working again, which
is different from the standard survival analysis setting. In the sensor network literature as well
recurring failures are usually ignored, but it is an important phenomenon that should be understood
[ 8, 9]. Spontaneous failure and recovery processes could indicate that the loss of performance is not
a result of failures in the underlying hardware ( which are likely to be permanent), but is rooted in
the design choices for the communication network and sensor unit.
We use simple estimates of survival curves, which account for spontaneous recovery. More compli-cated
estimators can be calculated and, in future work, we will investigate and develop parametric
lifetime models for the system. We now describe our estimates.
Choose a time period T. The data comprise filled or a ND- filled sequences. For each sensor i,
compute the runs of 0’ s and 1’ s. A 0- run is the count of the number of successive days si( n) = 0;
a 1- run is the count of the number of successive days si( n) = 1. Each sensor’s 0- runs and 1- runs
alternate. Denote the set of 0- runs and 1- runs for sensor i by R 0i
and R 1i
respectively. We normalize
all run lengths by the total number of days the sensor is in the system during T.
Observe that the length of a 1- run is the number of days a sensor remains in a working state before
it fails, which we can regard as its lifetime. The length of a 0- run is the number of days a sensor
remains in a failed state before it begins to work, which we can regard as the time it takes to get
‘ fixed’ or fixing time. This observation leads to the following estimators.
The first estimate is the lifetime distribution, which is the empirical cumulative distribution function
for R 1 = S i 2 A R 1i
, while the mean lifetime of sensor i is
μ1( i) = P ri 2 R 1i
ri
| R 1i
|
. ( 9)
The second estimate is the fixing time distribution, which is the empirical cumulative distribution
function for R 0 = S i 2 A R 0i
, while the mean fixing time of sensor i is
μ0( i) = P ri 2 R 0i
ri
| R 0i |
. ( 10)
μ1( i) is the average time sensor i is working before it fails and μ0( i) is the average time it takes to
become fixed after it has failed.
We can compute the empirical distributions of the mean lifetime μ1( i) and the mean fixing time
μ0( i). In these distributions, each sensor contributes a single number. The difference between the
1- run distribution and the mean lifetime distribution, is that the former represents a system property
( for example, sensors that are always- 1 contribute less to the distribution, as they have a smaller
number of runs), whereas the latter is a distribution of the lifetime property of individual sensors.
9.1 Runs distributions
Figure 16 shows the 1- run distribution for Districts 4, 7 and 11. As usual, we do not consider
always- 0 sensors. For District 4, 80% of the 1- runs last 50 or fewer days during a one- year period,
with little difference between 2005 and 2006. For District 7 we see an improvement in 2006 over
2005. For District 11 the distribution remains the same over both years, and is strictly better than
both Districts 4 and 7, mainly due to sensors that have very long runs of 1’ s.
The 0- runs remain the same year over year for all Districts as shown in Figure 17. District 4 exhibits
a slight improvement. An interesting phenomenon is worth noting. For District 4, 61% of the 0-
runs have length 1 in 2006; the corresponding numbers are 48% for District 7 and 49% for District
27
11. That is, many sensors experience failures that last one day. To check if such failures are the
result of insufficient samples being received, we consider run plots for ND- filled sequences. Figure 18
display the results. As expected, the 1- run distributions have improved, but interestingly the 0- run
distributions remain almost the same. For District 4, 42% of the 0- runs are one day long in 2006
( Figure 19). For District 7, this number is 50% and for District 11 it is 33%. This means that the
underlying causes of one- day failures are not due to an insufficient number of samples, and they are
concentrated on a group of sensors.
Figure 16: 1- runs distribution of District 4, 7 and 11 ( 2005- 2006, filled)
9.2 Mean lifetime
Figure 20 shows the mean lifetime distributions for sensors in Districts 4, 7 and 11. For District 7
we observe an improvement in 2006 over 2005, with sensors taking longer to fail. For District 4, the
performance is worse. For District 11 performance remains the same, with a large number of sensors
never failing. District 7 also has longer average working runs than District 4, especially for sensors
with average runs of more than 50 days.
The fixing time distribution curves in Figure 21 show that on average sensors remain failed for only
a few days, making clear the oscillatory nature of the system. Furthermore, notice that District 11
has a distribution that has shorter fixing times than in the other districts.
We can analyze the effect of the insufficiently many samples by considering runs with ND- filled
sequences. Figure 22 shows the results for lifetime. Notice how the mean time to failure per sensor
is a lot better for all districts. Districts 4 and 11 perform similarly with about 60% of the sensors
working for the entire period of observation. This suggests that communication failures play a major
part in the failure states of sensors for Districts 4 and 11. Furthermore, the fixing time distributions
28
Figure 17: 0- runs distribution of Districts 4, 7 and 11 ( 2005- 2006, filled)
( Figure 23) show that 0- runs are relatively short on average, even after insufficient sample states are
filtered out. This means that the other error states do not force a sensor to be permanently broken.
For District 4 there is also a shift to lower average 0- run lengths from 2005 to 2006, showing that
after sensors were fixed, temporary faults other than communication failures are observed. This
means that the sensor system could be essentially less reliable.
To conclude this section, Figure 24 shows the time- to- failure and time- to- fix distribution curves for
District 7 disaggregated by highways in 2006. All highways have very close performance, except for
I- 5 for which the mean 1- runs are shorter than average and the 0- runs are longer than average. This
could mean that I- 5 has some underlying faults that cause the sensor to stay in the failed state much
longer.
Conclusion 6 The 1- run distribution for District 11 is strictly better than for Districts 5 and 7,
implying that sensors in District 11 keep working much longer before they fail. There is a large
number of one- day long failures: 61% in District 11, 48% in District 4 and 49% in District 7. The
one- day failures do not appear to be the result of ‘ insufficient number of samples’ and they seem to
be concentrated in a group of sensors.
29
Figure 18: 1- runs distribution of Districts 4, 7 and 11 ( 2005- 2006, ND filled)
30
Figure 19: 0- runs distribution of Districts 4, 7 and 11 ( 2005- 2006, ND filled)
31
Figure 20: Sensor mean lifetime distribution of District 4, 7 and 11 ( 2005- 2006, filled)
32
Figure 21: Sensor mean fixing time distribution of District 4, 7 and 11 ( 2005- 2006, filled)
33
Figure 22: Sensor mean lifetime distribution of District 4, 7 and 11 ( 2005- 2006, ND- filled)
34
Figure 23: Sensor mean fixing time distribution of District 4, 7 and 11 ( 2005- 2006, ND- filled)
Figure 24: Sensor mean lifetime ( left) and mean fixing time ( right) distributions for District 7 by
highway ( 2005- 2006, filled)
35
10 Communication network failures
Figure 25: Mean length distribution of Comm Up period per sensor for Districts 4, 7 and 11
( 2006, filled)
In this section we analyze communication network failures. For this purpose, instead of the sequence
( 1) or ( 2) we use the sequence
s  i
( n) =  1 if si( n) / 2 { No Data, Insufficient Data} 0 if si( n) 2 { No Data, Insufficient Data}
.
Thus this sequence captures failures that only relate to communication failure events. We focus on
data for 2006.
First, we plot the distribution of the mean length of 1- runs, which corresponds to the average length
of a communications up ( Comm Up) period for each sensor. Figure 25 shows the results. We have
normalized the periods with respect to the number of days a sensor was observed in an year. District
11 has the best comm up average time distribution, with 80% of the sensors having Comm Up runs
of 100 days or longer. For District 4, the average Comm Up period for a sensor is less than 30 days
for 90% of the sensors. For District 7, 30% of the sensors have Comm Up average run lengths of 30
days or less, and 30% of the sensors have Comm Up average periods of 100 days or more.
Next, we plot the distribution of the mean length of 0- runs, which corresponds to the average length
of a communications down ( Comm Down) period for each sensor, Figure 26. District 4 and District
11 have very similar behaviors. 70% of the sensors in District 4 have average Comm Down run
lengths of 5 days or less. In District 11, 70% of sensors have average Comm Down run lengths of 3
days or less. District 7 has a different behavior. 50% of the sensors have an average Comm Down
run length of 5 days or less and 30% have average Comm Down run lengths between 5 days and 25
36
Figure 26: Mean length distribution of Comm Down period per sensor for Districts 4, 7 and 11
( 2005, filled)
Figure 27: Mean length distribution of Comm Down period per sensor for non- filled data for District
7 ( 2005- 2006)
days. One might suspect that this is due to the fact that District 7 added more sensors in 2006 than
other Districts. However, Figure 27 shows this is not the case. Still it is interesting that the Comm
Down periods lengths are short. This shows the unreliable nature of the communication network,
and may be due to the communication protocol or equipment being used.
Figure 28 strengthens our conclusions by plotting the distribution of the number of 1 day fail-ures
for each sensor. Notice the huge number of number of 1 day failure events, confirming that
37
Figure 28: Distribution of number of 1 day failures for each sensor in Districts 4, 7 and 11 ( 2005- 2006)
communications is unreliable. This holds true for all districts.
Conclusion 7 The communication networks in Districts 4 and 7 are very unreliable, compared with
District 11. In District 4, communication with 90% of sensors fails within 30 days compared with
30% in District 7 and 5% in District 11. Generally communication failures have short duration. In
District 4, 70% of failures last for at most five days; in District 7, 50% of failures last for at most
five days; and in District 11, 70% of failures last for at most three days. The failures could be the
result of a poor choice of communication protocols.
11 Link reliability
In the previous section we saw that the communication network in District 4 is very unreliable. We
can quantify the reliability more directly, instead of simply using the diagnostic state classification
of PeMS. We use the number of 30- sec samples actually received by PeMS from each sensor and for
the District. Our non- parametric choice of estimator is again the histogram.
For this estimate we exclude days when no samples were received. Those are accounted separately.
A second feature in the data is also considered. In any given day, a different number of samples may
be received from each sensor. But there is also a maximum number over all sensors in the network
for that given day. We observe that for some days, this maximum is consistently smaller than the
theoretical maximum of 2,880 samples per day. Figure 29 shows the un- normalized histogram of
38
Figure 29: Distribution of maximum number of samples received for District 7 ( 2005- 2006)
Figure 30: Normalized distribution of number of samples received for ( a) District 4, ( b) District 7
and ( c) District 11 ( 2005- 2006)
samples received for District 7. Notice the small repeated structure in the figure, which may indicate
that some modem banks are down.
To overcome the effect of this kind of failure, we use a normalized number of samples values: for each
day, we multiply the number of samples received from a sensor by a coefficient so that the maximum
number received from all sensors for that day is 2,000. Results of the estimated probability densities
are shown in Figures 30 and 31. In these plots, better performance is indicated by higher values
towards the right end of the plot ( high probability of receiving a high number of samples). Comparing
39
Figure 31: Comparison of normalized distribution of number of samples received ( a) full view and
( b) Zoom ( 2006)
across districts, in Figure 31, we can see that District 11 has a much better link quality than Districts
4 and 7. This accounts for the phenomenon observed in Section 5 of an increased number of always- 1
sensors, when the fault sequence considers Insufficient Samples state as a communication failure.
This analysis reinforces the conclusion of Section 10: communication network failures are both very
significant and unlikely to be fixed by the Detector Fitness Program.
12 Detector Fitness Program
The Detector Fitness Program for Districts 4 and 7 is an attempt to improve the reliability of their
sensor networks. The Program sent crews to fix sensors which were suspected on the basis of their
PeMS diagnostic state for a single day. We have seen above that the sensor network in these Districts
is very unstable. Hence it is a poor idea to determine the suspect list on the basis of a single day,
especially if the failed state is due to a communication failure.
In this section we investigate the effectiveness of the fitness program, using the metrics developed
earlier. We compute these metrics for periods before and after the visit, focusing attention on visited
sensors and comparing visited and non- visited sensors.
12.1 Summary
Tables 9 and 10 summarize the effort expended in the fitness program for Districts 4 and 7. The
column “ fixed” is based on the reported claim that a particular sensor was fixed during the visit.
Notice that only 33% of the visited sensors in District 4 and 52% in District 7 were fixed. The
number for District 7 is higher because the crew could replace the loop in some locations.
Thus the DFP records claim a ‘ success’ rate between 30 and 50%. We will see below that this claim
is illusory.
Tables 11 and 12 display a summary of the most common failure causes. The rows do not add up
to 100% because some reports record multiple causes and some records report no cause. Modem,
detector card issues and bad/ open loop are the most common causes. The first two can be fixed by
possibly replacing the equipment or resetting it, but fixing open or bad loops requires construction
work. Notice also a significant number of non- operational loops: sensors with missing parts, no
power, or are at locations in a construction site. Such loops may nevertheless report samples
40
Highway Total Investigated Fixed
80 441 20.4%
101 696 37.5%
680 485 31.5%
880 576 35.1%
All 3244 33.4%
Table 9: Fitness Program summary Dis-trict
4.
Highway Total Investigated Fixed
5 638 46.1%
405 443 41.8%
10 401 45.9%
605 359 71.9%
101 238 44.1%
All 3192 51.7%
Table 10: Fitness Program Summary
District 7.
depending on the configuration table and the communication network. Maintaining the configuration
table should improve sensor network reliability.
Highway Bad/ Open Missing Modem/ Card Under No Other
Loops Parts Issues Construction Power Issues
80 10.0% 15.0% 13.6% 7.3% 0.9% 3.6%
101 14.7% 12.9% 29.9% 1.7% 2.9% 1.3%
680 11.1% 20.8% 29.1% 12.8% 2.5% 1.2%
880 6.3% 6.9% 11.6% 2.6% 2.4% 0.7%
All 12.5% 15.1% 26.2% 5.6% 4.0% 3.7%
Table 11: Fitness Program summary of failures District 4.
Highway Bad/ Open Missing Modem/ Card Under No Other
Loops Parts Issues Construction Power Issues
5 35.7% 21.8% 39.5% 10.8% 3.8% 8.3%
405 20.8% 17.4% 32.1% 23.0% 5.0% 4.5%
10 18.0% 11.0% 46.4% 1.0% 4.5% 22.7%
605 30.6% 22.3% 43.5% 1.4% 11.4% 4.2%
101 17.6% 21.0% 36.1% 3.8% 16.0% 7.6%
All 21.6% 17.3% 40.1% 7.9% 8.5% 11.6%
Table 12: Fitness Program summary of failures District 7.
12.2 always- 0 sensors
Table 13 summarizes the information on always- 0 sensors that were visited. Almost 70% of the
always- 0 sensors in both Districts were visited ( see the total of always- 0 sensors in Table 6 for
2006). Furthermore, almost 30% of the visited sensors were in the class of always- 0 , which is only
slightly less than the proportion of always- 0 in the population in Table 6. Thus our analysis of the
always- 0 sensors based on DFP reports may apply to the entire population of always- 0 sensors.
Information District 4 District 7
always- 0 visited 65% 68%
visited that are always- 0 27% 30%
total sensors that are always- 0 25% 16%
Table 13: Fitness Program Summary for visited always- 0 sensors ( 867 visited sensors for District
4, 958 for District 7)
The effectiveness of the visits to always- 0 sensors can be seen in Tables 14 and 15. Only 9% of
the always- 0 sensors were claimed fixed. Most of the always- 0 sensors that were not fixed were
41
because of ‘ Bad or No Equipment’,‘ Non- existing lanes’, or ‘ Open/ bad loops’.
Type of always- 0 Number of Sensors (%)
Fixed 73 8.5%
Bad or No Equipment 370 42.7%
Open or bad loops 193 22.2%
Lane not existent 73 8.5%
No Power 28 3.2%
Other 130 15%
Total Visited 867 100.0%
Table 14: Fitness Program Summary for District 4, always- 0 sensors
Type of always- 0 Number of Sensors (%)
Fixed 84 8.8%
Open or bad loops 249 25.9%
Bad or No Equipment 142 14.8%
Lane not existent 137 14.3%
No Power 86 9.0%
Other 260 27.0%
Total Visited 958 100.0%
Table 15: Fitness Program Summary for District 7, always- 0 sensors
One important question is what happens to the 9% of always- 0 sensors that were claimed fixed.
Do any one of them return to being always- 0 ? For this purpose we check if any of the fixed sensors
return to an always- 0 state after the reported visit date. Instead of requiring that all samples after
the fixing date be zero ( until March 2007), we require that at least 89 days out of 90 have reported
zeros ( we relax our condition as some sensors have been observed for a smaller number of days after
fixing).
The results are shown in Table 16. Notice that in District 4, about 40% of the sensors revert back to
the always- 0 state, although some sort of fixing was done. For District 7, this is the case for 24%
of the sensors. If this information is taken into account, only 44 always- 0 sensors in District 4 ( 5%
of visited always- 0 sensors) and 64 in District 7 ( 7% of visited always- 0 sensors) were effectively
fixed. Thus the actual success rate of DFP visits to always- 0 sensors is only about 5%.
District Claimed fixed Continued always- 0 (%)
District 4 73 29 40%
District 7 84 20 24%
Table 16: Fitness Program Summary for District 4, District 7, always- 0 sensors
12.3 Productivity and stability
Figure 32 shows the productivity of Districts 4 and 7 for all visited sensors before and after being
visited. For both districts the productivity curve indicates an improvement. However the improve-ment
in stability is insignificant in both Districts ( Figure 33). This confirms our earlier conclusion
that communication network failure is an “ independent” failure, which the DFP does not effectively
address.
To evaluate further the productivity improvement, we investigate the performance of sensors that
were visited but not claimed to be fixed ( Figure 34). Notice that after the visit, there is a slight
performance improvement in both Districts. This could be result of misreporting the effects of
42
Figure 32: Productivity of visited sensors in District 4 ( left) and District 7 ( right) before and after
visit ( 2005- 2007)
Figure 33: Stability of visited sensors in District 4 ( left) and District 7 ( right) before and after visit
( 2005- 2007)
Figure 34: Productivity of visited but not fixed sensors in District 4 ( left) and District 7 ( right)
before and after visit ( 2005- 2007)
fixing. It could also be the case that on the day the list of suspect sensors was compiled, these
sensor had failed but spontaneously recovered later as frequently happens. Again notice that the
stability ( Figure 35) remains the same, and the curves are very close to yearly estimates.
43
Figure 35: Stability of visited but not fixed sensors in District 4 ( left) and District 7 ( right) before
and after visit ( 2005- 2007)
Figure 36: Productivity of visited and fixed sensors in Districts 4 and 7 before and after visit
( 2005- 2007)
Figure 37: Stability of visited and fixed sensors in Districts 4 and 7 before and after visit ( 2005- 2007)
Figure 36 shows the productivity estimates for sensors that were visited and claimed fixed. In this
case there is a very significant improvement in performance, confirming that the Detector Fitness
Program has an effect in system performance. The productivity of these sensors is now the same as
for the District as a whole ( compare with 9).
44
On the other hand, the stability shows no improvement ( Figure 37), again implying the independence
of communications failures and their immunity to the DFP.
To conclude this section Figures 38 and 39 show productivity estimates for visited sensors with
different classes of failures. Notice that the highest improvement obtains for sensors that had missing
equipment or modem/ card problems. Fixing crews made more difference to these failure classes.
Unfortunately, in the absence of remote diagnosis, one cannot tell whether a sensor has failed because
of these causes.
Figure 38: Productivity of visited sensors in District 4 with observed failures: Open loop, Modem
issues and Missing equipment before and after visit ( 2005- 2007)
45
Figure 39: Productivity of visited sensors in District 7 with observed failures: Open loop, Modem
issues and Missing equipment before and after visit ( 2005- 2007)
46
12.4 Lifetime and fixing Time
Figure 40: Lifetime distribution of visited and not fixed sensors in Districts 4 7 before and after
visit ( 2005- 2007)
Figure 41: Lifetime distribution of visited and fixed sensors in Districts 4 7 before and after visit
( 2005- 2007)
Figure 42: Time to fix distribution of visited and not fixed sensors in Districts 4 7 before and after
visit ( 2005- 2007)
In this subsection we investigate the improvement to lifetime resulting from the Detector Fitness
47
Figure 43: Time to fix distribution of visited and fixed sensors in Districts 4 and 7 before and after
visit ( 2005- 2007)
Program. Interestingly, the 1- runs of a sensor are deeply affected by communication failures, and
thus we will see that the improvement obtained in the average time to failure for each sensor is
not as much as seen in productivity. To improve lifetime one needs to improve both key metrics:
productivity and stability. This concept is verified with our data.
Figures 40 and 41 show the average per sensor lifetime curves for both districts for visited sensors
that were not fixed and those that were fixed, respectively. Notice that for sensors that were not
fixed, there is no improvement for District 4, and some improvement for District 7. This implies
that the average behavior improved for District 7 without any direct intervention on the sensors
( result of statistical variation). Fixed sensors improved their average length of 1- runs for Districts
4 and 7. Still the improvement is not as remarkable as the improvements observed in productivity.
Notice that for District 7, the after visit curve of fixed sensors is slightly worse than for non- fixed
sensors. For example, for fixed sensors, 15% of the sensors work continuously for an average of more
than 100 days. The same number for non- fixed sensors is 25% for an average of more than 100 days.
The average time to fixing distributions in Figures 42 and 43 show some improvement as well. For
District 7 we see a reduction on the average 0- run length. This evidence supports the thesis that
inherent failures in the sensors show longer 0- run lengths, whereas communication failures cause
short ( mostly 1 or 2 day long) 0- run lengths. It is interesting to observe also that for sensors that
were not fixed, there is a large number of always- 0 sensors which we chose to plot in these figures
( they show up as sensors that are never fixed).
Conclusion 8 DFP reports claim that between 30% and 50% of visited sensors were fixed. The
actual success rate is much lower. Nearly 30% of the visits were to always- 0 sensors, only 5% of
which were fixed. The productivity of sensors that were visited and fixed improved to match the
average for the District. Stability showed no improvement. The most improvement came from visits
to sensors whose observed failures were due to modem issues or missing equipment.
48
13 Conclusions
In this report we performed a systematic analysis of failures and the actions taken against them in
2 districts in California. Our analysis did not rely in any specific parametric models, avoiding any
particular assumptions about the sensor behavior. Instead we devised simple metrics that can be
easily computed for very large systems to tackle the problem. Another innovation was the use of a
whole day ( or block) of samples to attribute a sensor state. In other problems, maybe a day might
be a block too long, but for transportation networks, a day of missing samples can be reasonably
interpolated.
We group our conclusions under three headings: methodology, district performance, and Detector
Fitness Program.
Methodology
These conclusions relate to the methodology we have followed.
• always- 0 sensors should always be treated separately as they represent a clearly different class
of behavior than other sensors.
• Productivity and Stability capture independent aspects of the system performance. A system’s
productivity can be improved without affecting its stability.
• Productivity captures the underlying sensor system performance and Stability capture commu-nication
network performance. The latter is related to the choice of communication technology.
• Highly productive systems could still have poor Lifetime ( or runs distribution). The average
1- run length in a sensor is a good metric of uptime. Its average 0- run length allows us to infer
more about the underlying cause of instability. Short average 0- runs usually indicates a poorly
functioning communication network.
• Metrics should be normalized to account for the number of days in operation in order to provide
meaningful insights into the system. Although this seems a minor detail, it greatly improves
our understanding of plots.
• The simple way we treated missing data - which in this case corresponded to days where
communication failed ( No Data) and/ or there were insufficient samples to make a daily decision
- was to use the previous day’s state. This worked well as we got many insights by comparing
the inclusion and exclusion of Insufficient Samples state as a com fault. In a more parametric
setting, maximum likelihood estimators can be used to estimate parameters even with missing
values, but the estimation complexity becomes a lot higher, which might be an issue if huge
volumes of data are to be addressed.
• The scope chart allows an easy visual comparison of the performance over time. It also captures
the same information available by plotting the number of samples acquired, but it is visually
much less burdensome. Less information displayed does not necessarily mean less information.
• The number of samples received can be used to estimate link quality metrics, such as the
probability of receiving a particular number of samples. Link quality is almost exclusively a
metric of the communication network.
• Communication failures can be studied just like other types of failure, by considering an
alternate sequence where the only fault is a communication fault, and the remaining states are
” Good” states. The same metrics apply.
49
• When failures are fixed or repaired, performance evaluation after the fact should be done over
collections of samples, not just based on a single day observation, as sensor systems can be
unstable. The metrics proposed in this report capture such improvements accurately.
• Fixing actions should take into account the possible classes of failures the sensor experience,
as the effectiveness of fixing can be very different in different classes.
• It is essentially to have proper bookkeeping of maintenance data, with a repeatable ( instead
of an ad- hoc) reporting system. Even when the maintenance records are not so good, the use
of automated parsing and other techniques can make the data useful.
District performance
These are the main conclusions regarding of District performance based on PeMS data.
• District 11 has much better performance than District 7 whose performance is slightly better
than District 4.
• Large discrepancies in the percentage of Good sensors among Districts are caused mainly by
permanently failed ( always- 0 ) sensors.
• Systemwide oscillations in the percentage of failed sensors are caused by instabilities in the
communication network, which can “ black out” large sections of a District.
• Lanes that experience more intense traffic of heavy vehicles seem to have more always- 0
sensors, but on the other metrics, lane is not an influence.
• The highway variable also does not seem to play an important part in determining the pro-ductivity,
stability and lifetime of sensors.
• Communication technology choice appears to be a huge variable in determining stability.
• No District’s stability improved after the DFP. All three districts exhibit almost the same
stability pattern, with District 4 being slightly worse than others.
• Communication link quality is better for District 11 than for District 7, which in turn is
better than District 4. Nevertheless, in all Districts, significantly many samples are lost and
affect the data collection. Sensors in all Districts experience a large number of one- day long
communication failures.
Detector Fitness Program
• Determining which sensors to visit based on a single day of observation is not a good choice.
• always- 0 sensors experience a very low success rate, and should be low on the priority of the
fixing program.
• A considerable percentage of always- 0 sensors that are claimed to be fixed, in fact never work.
• Productivity improves after fixing, but stability does not.
• Modem/ Card issues and no equipment problems are the failure causes that most benefit from
fixing, whereas Open Loop failures cannot be significantly repaired. This could be because
only a very small fraction of Open Loop sensors are actually fixed.
• Fixed sensors in Districts 4 and 7 exhibit almost the same productivity pattern, possibly
implying fixing crew performance was consistent across districts.
• District 4 has poorer DFP records than District 7.
50
14 Acknowledgement
The work reported here has benefited from advice, comments and interest of Jaimyoung Kwon
of Cal State University East Bay, Karl Petty of Berkeley Transportation Systems, and Joe Palen
and William Okwu of Caltrans. The contents of this report reflect the views of the authors who
are responsible for the facts and the accuracy of the data presented herein. The contents do not
necessarily reflect the official views of or policy of the California Department of Transportation. This
report does not constitute a standard, specification or regulation.
References
[ 1] Chen, C., Kwon, J., Rice, J., Skabardonis, A. and Varaiya, P., “ Detecting errors and imputing
missing data for single loop surveillance systems,” Transportation Research Record, no. 1855,
160- 167.
[ 2] Rao, C. R., Linear Statistical Inference and Its Applications, Wiley, 2nd Ed. 2002.
[ 3] Nikulin, M. S. Parametric and semiparametric models with applications to reliability, survival
analysis, and quality of life, Birkhuser, 2004.
[ 4] Klein, J. P. and Moeschberger, M., Survival Analysis Techniques for Cnesored and Truncated
Data, 2nd Ed., Springer- Verlag, 2003.
[ 5] Brillinger, D., Stat 215B Class Notes, Revision, University of California, Berkeley, CA 2005.
[ 6] Gupta, B. C. and Mathai, A. M., Regression and Analysis of Variance Techniques, Instituto de
Matematica, Universidade Federal do Rio de Janeiro.
[ 7] Veneables, W. N. and Ripley, B. D., Modern Applied Statistics with S- Plus, 3rd Ed., Springer.
[ 8] Koushanfar, F. , Potkonjak, M. and Sangiovanni- Vincentelli, A., “ On- line Fault Detection of
Sensor Measurements, Proc. IEEE Sensors, pp. 974- 980, October 2003.
[ 9] Zhou, Z. and Guo, J., “ Simulations using the Monte Carlo method to estimate life distribution
for sensors,” Proc. SPIE Vol. 3374, pp. 451- 455, 1998.
51
Appendix
A System metrics and sensor Markov models
Consider a two- state Markov chain with states labeled { 0, 1}. The transition probabilities between
states are labeled p01 for transitions from state 0 to 1 and p10 for the opposite transition. We identify
state 0 with a failed state of a sensor, and state 1 with a working state. The stationary distribution is
denoted by  , with elements  0 and  1 corresponding to states 0 and 1. In this section we show how
the productivity and stability metrics introduced in Sections 7 and 8 relate to the Markov model.
Suppose the Markov chain is stationary. Let the state of the chain at time n be denoted by Xn.
Then the individual productivity estimate w for a time horizon T is given by
wT =
T X n= 1
1 ( Xn = 1). ( 11)
We can compute the expectation this random variable as
1
T
E[ wT ] =
1
T
T X n= 1
E[ 1 ( Xn = 1)]
=
1
T
T X n= 1
P( Xn = 1)
=
1
T
T X n= 1
 1
=  1 ( 12)
Standard results from the theory of Markov chain show that limT ! 1
1
T wT =  1 almost surely.
Similarly, for stability we have,
sT =
T X n= 1
1 ( Xn 6 = Xn− 1) ( 13)
Computing the expectation and noting that for a Markov Chain
P( Xn 6 = Xn− 1) = P( Xn = 1| Xn− 1 = 0) P( Xn− 1 = 0) + P( Xn = 0| Xn− 1 = 1) P( Xn− 1 = 1)
= p01  0 + p10  1, ( 14)
we have
lim
T ! 1
1
T
sT = p01  0 + p10  1 ( 15)
Furthermore, using the relations
 0 = p10
p10 + p01
,  1 = p01
p10 + p01
( 16)
We can thus express stability and productivity in terms of the unknowns of the model.
B Detector Fitness Program data sheets
In this appendix we present a few examples of the data obtained from the detector fitness program.
See Figures 44, 46, 47 and 48. The sheets don’t have a clear pattern. Furthermore, in some cases it
52
is not clearly reported if a sensor was fixed or not ( the default was assumed to be not fixed, but a
machine interpretation was done based on the remaining text). Also notice that in some cases, the
entries are mangled up, with actions and causes entered in incompatible columns.
C Disabled sensors for District 4
An analysis of the number of sensors in District 4 shows a discrepancy between those reported in
Table 2 in section 4 and those in the PeMS website. In this we explain the differences.
Table 17 shows the sensors that reported any data for each quarter, and during 2005 and 2006. The
table indicates that sensors have been added and disabled in the system. Sensors that are disabled
do not report any data from the date they are disabled. In our earlier analysis, disabled sensors
were not counted after the period they were disabled. Sensors enabled for any part of a year, that
reported data on that year, are accounted for in the statistics of that year.
Period Sensors (%) Total
Q1 ( 2005) 4809 83.2%
Q2 ( 2005) 4848 83.9%
Q3 ( 2005) 4857 84.0%
Q4 ( 2005) 4912 84.9%
2005 5140 88.9%
Q1 ( 2006) 4895 84.7%
Q2 ( 2006) 4110 71.1%
Q3 ( 2006) 4236 73.3%
Q4 ( 2006) 4515 78.1%
2006 5271 91.2%
Q1 ( 2007) 4633 80.1%
Total 5782 100.0%
Table 17: Number of sensors reporting data in District 4.
Table 18 shows the number of sensors added and disabled for each year. Notice the large number
of disabled sensors. We investigated if sensor added were replacing existing disabled sensors, but
this did not seem to be the case. In fact, 6 sensors were disabled twice, meaning they were dis-abled,
enabled and then disabled again remaining in a disabled state. We can explore further the
characteristics of disabled sensors.
Period Sensors added Sensors disabled
2005 331 228
2006 584 982
2007 ( Q1) 160 42
Total 1075 1252
Table 18: Number of sensors added and disabled in District 4.
First to make clear the connection between sensors reporting data, and disabled sensors during a
period, we compare the numbers reported by PeMS and the number of disabled sensors during each
quarter ( Table 19). Notice that the difference between the number of data reporting sensors, and
the PeMS reported sensors is exactly the number of sensors disabled in a given period.
1151 of the 1252 disabled sensors correspond to complete Detector Stations being disabled ( as
opposed to an individual sensor being disabled). Furthermore, 890 of the disabled sensors were
visited, with 231 of the sensors reported ones after the visit. 101 of the sensors were claimed fixed.
53
Period PeMS Disabled Data reporting
Q1 ( 2005) 4670 139 4809
Q2 ( 2005) 4834 14 4848
Q3 ( 2005) 4782 75 4857
Q4 ( 2005) 4787 125 4912
Q1 ( 2006) 4082 813 4895
Q2 ( 2006) 4090 20 4110
Q3 ( 2006) 4212 24 4236
Q4 ( 2006) 4515 42 4515
Table 19: Comparing PeMS, sensors reporting data and disabled sensors District 4.
Interestingly, 554 of the 890 visited, had missing equipment, no power or were under construction,
thus corresponding to sites where there could be missing sensors.
The consequences of disabling are that for periods after disabling, the sensor is accounted for in the
always- 0 category. In aggregate analysis, if a sensor was disabled in 2005, it will show in always- 0
only in 2006, as during 2005 the sensor reported data for a period of time before disabling. Of course
all statistics are normalized by the period of time in 2006 the sensor was enabled.
54
Figure 44: Typical DFP spreadsheet information for District 4 ( a)
55
Figure 45: Typical DFP spreadsheet information for District 4 ( b)
56
Figure 46: Typical DFP spreadsheet information for District 7 ( a)
57
Figure 47: Typical DFP spreadsheet information for District 7 ( b)
58
Figure 48: Typical DFP spreadsheet information for District 7 ( c)
59