ISSN 1055- 1425
August 2009
This work was performed as part of the California PATH Program of the
University of California, in cooperation with the State of California Business,
Transportation, and Housing Agency, Department of Transportation, and the
United States Department of Transportation, Federal Highway Administration.
The contents of this report reflect the views of the authors who are responsible
for the facts and the accuracy of the data presented herein. The contents do not
necessarily reflect the official views or policies of the State of California. This
report does not constitute a standard, specification, or regulation.
Final Report for Task Order 6500
CALIFORNIA PATH PROGRAM
INSTITUTE OF TRANSPORTATION STUDIES
UNIVERSITY OF CALIFORNIA, BERKELEY
The Naturalistic Driver Model: Development,
Integration, and Verification of Lane Change
Maneuver, Driver Emergency and Impairment
Modules: Final Report
UCB- ITS- PRR- 2009- 34
California PATH Research Report
Delphine Cody, Swekuang Tan
CALIFORNIA PARTNERS FOR ADVANCED TRANSIT AND HIGHWAYS
The Naturalistic Driver Model: Development, Integration, and Verification of Lane
Change Maneuver, Driver Emergency and Impairment Modules
Final Report to Caltrans on
PATH Task Order 6500
Delphine Cody, Swekuang Tan
ii
Abstract
This report describes the development of a driver model from a Human Science
perspective, with the goal of integrating this model into a simulation environment
supporting the design and support of Intelligent Transportation Systems. It also provides
a discussion regarding the challenges faced in such an enterprise, concluding with a
discussion concerning the need to develop a driver simulator.
iii
iv
Executive Summary
This report documents the development of a driver model that describes how humans
process information. The need for such a model comes from the goal of improving
driving safety by designing systems that will support drivers by either allowing drivers to
avert errors or, in the most extreme cases, take control over the driver. The systems
developed to this day relay neither a good description of driving activity nor the
perceptive and cognitive processes underlying it, and are therefore limited in the type of
support that can be provided. A driver model would also provide the basis for an
evaluation of the benefit of the driver support systems, in terms of safety improvement
and impact on traffic.
The architecture of the model is presented, along with a description of the modules that
constitute it, with various degree of detail based on the closeness to implementation of
the module. The method for integrating data into the development of the model is also
discussed.
The conclusions of the report focus on the challenges met for designing such a model and
the requirements for pursuing this line of work.
v
vi
Table of content
Abstract ............................................................................................................................. iii
Executive Summary .......................................................................................................... v
1. Introduction ................................................................................................................... 1
2. PATH DRIver Cognitive ( PADRIC) model architecture .......................................... 2
3. Perception Module ........................................................................................................ 6
3.1. Field of view characteristics .................................................................................... 6
3.2. Mechanics of visual field of view ............................................................................. 7
3.3. Visual attention allocation ....................................................................................... 9
4. Cognitive Module ........................................................................................................ 12
4.1. Driving knowledge stored in memory .................................................................... 13
4.2. Processing information .......................................................................................... 16
4.2.1. Manipulation and interpretation of information ............................................. 16
4.2.2. Processes constraining the manipulation of information ............................... 17
5. Future research: specifications for a “ driver simulator” ........................................ 20
Bibliography .................................................................................................................... 22
vii
viii
1. Introduction
The terminology of driver models is used abundantly in several research fields, such as
Human Factors, design of Intelligent Transportation Systems ( ITS), and micro- traffic
simulation. The need for driver models in these fields arises for different reasons and
care has to be applied when trying to couple the development of models to serve several
purposes. The level of modeling carried on by these fields is defined below.
In the field of psychology, the requisite for a driver model development stems from a
necessity to organize micro- models describing steps of information processing that need
to be articulated together to render Human information processing. In other words, we
need a " driver simulator" that we can use to sort out our understanding of cognition as it
applies to driving. This need results in part from the " divide and conquer" approach
( Anderson & Lebiere, 1998), which led to overlapping theories making it difficult to have
a comprehensive view of how the human brain works. Some researchers are trying to go
back to a more comprehensive description of the cognitive system, but these cognitive
architectures are still not mature enough for a direct application to driving activity; hence,
when trying to develop a driver model, one has to first develop a model of cognition.
For designing ITS, there is a dual need: on one hand, there is a need of understanding
what provokes driver errors so that we can identify what has to be done to support the
driver, and on the other hand, there is a need of models that can be integrated in
simulations in order to evaluate the impact of an ITS on traffic and safety, and therefore
evaluate the benefit of the deployment of such a system.
At the level of traffic simulation, there is a desire to include more human- like behavior
for the simulated driver. The requirement at that level is rather towards the development
of driver performance or behavior, which requires sets of data covering behavior for a
specific scenario, with a distribution covering driver behavior variability based on factors
such as age or experience.
This work addresses the development of ITS, in terms of supporting their design and
evaluation through the integration of the developed model into a traffic simulation tool.
From this perspective, we propose an information processing architecture in the first
section of this report, followed by a description of the modules that constitute it, and we
conclude this report with a presentation of the limits of this approach and
recommendations for the development of a tool that would be a first step towards
overcoming these limits.
1
2. PATH DRIver Cognitive ( PADRIC) model architecture
The development of the PATH DRIver Cognitive ( PADRIC) model supports the
development of simulations tools for assessing the impact of Intelligent Transportation
System on traffic. From this perspective, the model needs to output a behavior that
corresponds to the one that can be observed in driving and also the interactions with
various types of interfaces. In order to accomplish this, it is necessary to reproduce some
aspects of information processing and activity control. Our approach consists of
developing a modular architecture of the different processes involved while drivers are
processing information. In this section, we present the architecture that we developed.
The bases of the PADRIC architecture have been developed in the frame of two previous
projects as presented in the figure below.
Tactical module
Perception
module
Operational
module Vehicle
Road
environment
Vehicles in
other lanes
Landmarks
Visual attention
allocation
Relative velocity
scaling
Lower
Regulation Layer
Driving subschemas
Expectancies
Drivers’ preferences
Goal
management
Representation
of current
driving scene
Current
driving action
Working Memory
Strategic module
Itinerary plans: goals
Figure 1: PADRIC architecture ( TO4222 – Human Model driver development)
As we worked further with this architecture, we came to consider that the integration of
cognitive processes ( e. g. working memory) in driving levels ( strategic, tactical and
operational) became a problem in order to describe control and automatic behaviors
involved with driving in terms of cognitive resources management.
In a now classical paper, Rasmussen ( 1983) categorized behavior based on the level of
processing required for generating a response. From this perspective, he distinguishes
information processed by a human between stimuli, signs and signals, and assigns the
level of processing as a response, in terms of skill- based behavior, ( automatic response),
rule- based behavior ( requires some cognitive attentional resources) and knowledge based
behavior ( requires all attentional resources, involves reasoning). Driving activity is
2
usually categorized in three levels based on a time scale: strategic ( or planning), tactical
( or maneuvering) and operational ( or vehicle control) 1. These categorizations
integrated in the matrix below in table 1, where planni
have been
ng stands for strategic,
maneuvering for tac
and Michon, 1992 p et al. 19 )
Control
tical and control for operational.
Table 1: Matrix of tasks hierarchy vs. task performance
( in Asman .170 , from Hale 90, p. 1383
Planning Maneuvering
Knowledge ting in unfamiliar skid
naviga on icy road learner on a first lesson
town
controlling a
Rule between familiar
routes passing other cars driving an unfamiliar car
commuter travel familiar
junctions
owing around
corners
choice
Skilled negotiating road foll
The examples given in this table provides a mix of pairs where the difference can be
either within a driver or between two drivers. For example, at the planning level, the
factor between skill, rules and knowledge is the level of familiarity with the route to
perform, while the difference between control and maneuvering at the knowledge level is
the difference between a novice and experienced driver. A within driver difference at the
knowledge level would be an urban driver used to city streets and urban highways driving
a mountain, and not used to negotiating sharp curves with poor surface conditions.
n task
, such
rience will be selected when the driver
haracteristics are selected prior to simulation.
ng,
o,
order to
present automatic control actions that do not require cognitive processing.
in
We propose to adapt this matrix and distinguish between situations where the shift i
performance ( skill, rule, knowledge) is happening within the driver, due to outside
conditions, such as weather ( road/ rain reducing visibility) or to “ inside” conditions
as learning, and situations where the shift in task performance is explained by the
difference in experience. The differences in expe
c
In this architecture, we identified the three main functions of information processi
with a perceptive, cognitive and executive module, where the perceptive module
“ interfaces” with the world, the cognitive module processes information received from
the perceptive module and generate “ orders” for the executive module to carry on. Als
a more automatic loop between the perceptive and executive module exists in
re
1 The terms strategic, tactical and operational are used to describe human activities, when applied to driving,
this hierarchy is often described as planning, maneuvering and control ( of the vehicle). These terms will be
used interchangeably in the report.
3
Driving
Environment
Perception Module Cognitive Module
Vehicle Execution module
Driver Model
Figure 2: PADRIC General structure
In figure 2, the solid line arrows describe the loop of processing for controlled driving
behavior ( which includes rule and knowledge based behavior), while the dashed line
arrows describe the loop for automatic behavior ( skill based behavior). We decided to
extract the task hierarchy ( strategic, tactical and operational) out of the model.
This architecture is to be seen as a metaphor, like the blueprint of a house, i. e. a
specification of where the processing happens. Once the system becomes active, only
parts of the architecture are activated, and the activation of these parts is achieved by
rules regulating how many parts can be activated at once, for how long. In other words,
the architecture has to be as exhaustive as possible, but its activation during a simulation
will show up in only parts of it. A difference with the blueprint of a house is that most
houses are customized, and most efforts for driver model development aim at producing
one driver model, which in turns has to be able to display the variability either at the
driver level ( for example a driver can show variable state of arousal) or at the driving
population level ( replication of the variety of drivers based on the most investigated
characteristics, such as age, gender and experience).
We proceeded with the development of a single architecture, and handled the differences
with variables that are input to the model and associated rules for the behavior of the
system’s module. For example, a set of “ prototypical” drivers can be set up. By
prototypical, we mean that we describe the basis of information processing and identify
factors influencing this processing, and therefore simulate different types of driver
models based on the identified factors. For example, in order to retain information and
process it, we all use a structure called working memory. What differs is the amount of
information that can be manipulated at once, how fast it can manipulated, how it is
selected. These variations can happen at the individual level, for example, the baseline is
a well rested driver, if this driver is tired, or under the influence, or distracted, then the
information processed in working memory will be affected based on these factors.
4
This approach allows us to build a model that can be populated as more research is
completed regarding the effect of specific factors, such as age, on each of the functions
described above, and as more work is conducted on the model, more prototypical drivers
could be created.
We propose to following set of characteristics:
Static characteristics ( set prior to the simulation)
• Driver level of performance:
o Novice: learning to drive – any driving tasks is “ cognitive”
o In- between: in the process of building automatic behavior – some driving
tasks do not rely anymore of verbal control, better and faster execution, in
between rule and knowledge
o Experienced:
􀂃 automatic processing for lane position, speed control and gap
regulation
􀂃 non verbal, but some degree of cognitive attention for lane change,
merge when level of traffic higher than xxx
• Destination:
o Commute: does not look for direction signs, street name of specific
landmarks
o Route involving known roads: some degree of attention directed toward
landmarks and road sign and where to make direction change
o Unknown route: high degree of attention to direction signs, street names
• Driving environment familiarity
o Highway: not familiar, familiar, very familiar
o Surface street urban: not familiar, familiar, very familiar
o Rural ( mountain, curves): not familiar, familiar, very familiar
• Driver alertness/ fatigue before to start driving
o Very alert: well rested, short reaction time
o Alert: rested, reaction time normal
o Tired ( less than x hours of sleep the night before): slow reaction time
• Driver level of intoxication
Dynamic Characteristic ( change during the simulation)
• Level of alertness: drivers level of alertness will be going down with a
combination of:
o Time:
o Demand from the driving task: if conditions are bad, like weather, night,
high traffic density ( stop and go)
o Demand from other tasks: cell phone, conversation …
• Driver level of performance
o If conditions ( e. g. visibility) degrades then:
􀂃 Automatic processing becomes controlled, but might remain non
verbal
􀂃 Non verbal can become verbal
5
3. Perception Module
The perception module modeling addresses three dimensions of the visual perception:
1 – Human Field of view characteristics
2 - The mechanics of visual perception
3 – Allocation of visual attention
3.1. Field of view characteristics
The field of view consists of five sub zones providing different types of information:
• The full binocular field, 180°
• The fovea or focus point, covering a zone of 3°
• The para- fovea, covering a zone of 10°
• The useful field of view ( UFOV), covering a zone of 30°
• The peripheral field of view, what is left between the UFOV and the full
binocular field of view
The “ size” of the sub zones varies, either because of a physical cause ( some drivers see
only with one eye, some eye sickness can alter the fields of view and age indicates a
reduction in its width) or due to the tasks demanded ( phenomenon of tunnel vision or
general interference), especially for the UFOV. Therefore, these fields are treated as
inputs variables to the model, for which there are two sets: ( i) baseline, using the values
above, and ( ii) customized, where the user enters the size of the simulated driver UFOV
prior to running the simulation.
The information usually processed during an eye fixation is obtained from the fovea and
para- fovea field. While driving, drivers usually extract information contained in the
UFOV. The information processed in the peripheral field of view is usually motion;
change of color, this type of information will likely attract a driver’s attention. For
example, a driver would be focusing his/ her visual attention on a lead vehicle, react to a
light change perceived in their peripheral vision that would drive them to divert their
visual focus, to a traffic light for example.
6
0 m 10 m 20 m
Peripheral
UFOV
foveal Para- foveal
Figure 3: Top view of driver's fields of view
The figure above displays a top view of a simulated driver’s field of view using the
baseline data, where the focus point is on the vehicle ahead.
The following equation is used in order to compute the coverage of the field of view:
Coverage = object distance * tan ( angle* π/ 180)
3.2. Mechanics of visual field of view
The mechanics described here address the changes of field of view related to the task
demand and the scaling of relative velocity.
Field of view and task demand
The width of the field of view can vary based on age, experience and task demand. Age
and experience are inputs to the model in order to build a prototypical driver prior to
running the simulation, while the task demand effect will happen during the simulation,
based on the characteristics of the scenario.
In terms of experience and detection, data from Crundall et al. ( 1999) ( see table below)
provide a description of detection rate for novice and experienced drivers of targets
within the drivers’ UFOV ( although considered as peripheral field by the authors) based
on task demand.
7
Table 2: Peripheral target detection characteristics ( p. 1082 in Crundall et al. 1999)
High demand Low demand
< 52 5 6 >= 7 < 5 5 6 >= 7
Hit rates (%)
Experienced drivers 66 69 67 45 73 77 73 58
Novice drivers 65 63 61 43 72 76 76 48
Reaction time ( ms)
Experienced drivers 569 595 566 569 542 532 531 566
Novice drivers 589 568 583 629 563 557 550 569
This table describes the results for two groups based on their driving experience, where
the targets are presented at various degrees of eccentricity and when the participant is
required to conduct another task, which presents a high or low demand on the participant
perceptive and cognitive resources. The results are expressed in terms of detection rate
and reaction time for each population and show when the drivers are conducting a
primary task with a high demand on their perceptive- cognitive resources. The detection
rate is considerably affected when the target is at more than 7 degrees from the point of
fixation for both demand condition, experience does not play a meaningfully significant
difference between the two groups and the reaction time are slightly shorter for the low
demand condition.
Accepting this data set as valid for describing detection rates and time reaction, we
integrate it to the model in order to describe the expected object detection mechanism. In
the current version of the model, the schema describing the maneuver provides a list of
objects from the environment that are expected. Even though the drivers’ experience did
not seem to affect the detection performance, the data for each condition are used as an
input to the model, translated in chances to detect the target based on its eccentricity
relative to the current focus point. The model using these data as input will then provide
a response based on the simulated driver characteristic ( i. e. experienced or novice) and
the level of demand of the current task ( i. e. high or low).
The table above is valid only for expected/ looked for objects. For highway driving,
expected/ look for objects will be signs ( direction, HOV), markings ( used for entrance and
exits, merge), other vehicles.
Another dimension to integrate to the model is the distance at which drivers start to look
for specific objects. Serafin ( 1993) provided the following:
• 22 to 75 meters
o left lane and left edge line
o right lane and right line edge
• 60 to 75 meters
o traffic signs
• 75 to 90 meters
2 Onset eccentricity of object from focus point
8
o road ahead
• over 90 meters
o left and right scenery, sky, far field
Scaling relative velocity with surrounding vehicles
For describing how and when drivers sense the relative velocity with the surrounding
vehicles, we apply the model from Hoffman and Mortimer ( 1996). The model proposes
thresholds of subtended angle change and angular velocity to describe drivers' ability to
perceive and scale the relative velocity ( Hoffmann, 1996). At distances where the visual
angle changes at a rate below 0.003 rad/ sec, drivers' perception based on a “ looming”
effect does not allow perceiving relative velocity with a car ( 1.8 m width). In other
words, the driver sees the vehicle but cannot appreciate how fast he/ she is closing on this
vehicle. Using R⋅ θ = d and differentiating the geometric equation with respect to time,
the following result can be derived:
R2
d R&
& ⋅
θ = − ( 1)
where R is the range to the lead vehicle and d the width of the lead vehicle, is the
perceived relative velocity,
R&
θ and represent the visual angle and the rate of change of
visual angle. These parameters are displayed on the figure below.
θ&
Figure 4: Parameters used for modeling the perception of relative speed
d
R
θ
At R < R& / 0.00164 from the equation ( 1) and just- noticeable increments of δR / R = 0.12,
drivers scale perceived range- rate in a practically linear relationship to R.
3.3. Visual attention allocation
Visual perception functions are based on two principles:
• proactive: it is allocated to specific location of the environment in order to find
information supporting the task at hand, e. g., when planning to overtake, the
driver will look in rear- view, side mirror or behind his shoulder to look for a
vehicle in the back
• reactive: when the driver focuses on one area, the peripheral field of view will be
reactive to
o changes in contrast ( e. g. traffic light),
o motion ( e. g. start to look in the central mirror when a tailgater races
towards him)
9
o specific objects ( e. g. slowing down when spotting a CHP vehicle).
At this stage of the modeling effort, we focus mainly on the proactive form, i. e., the
cognitive module sends a request to the perception module about where to look and what
to look for. In future development, we intend to add the reactive mode, so the model
structure needs to allow a place holder for it, in order to allow for its integration at a
future time.
At this level, the visual allocation rules addresses mainly where the driver looks and for
how long:
The default mode for the focus point is looking straight ahead. If there is a lead vehicle
within 90 m of the driver, then the focus point is that vehicle.
Scan radio find/ select
station/ turn volume/
sound quality
( 1.5/ 1.5/ 0.97/ 1.51 sec)
Fuzzy vehicle, fuzzy road
sign/ landmark
Road enviornment Vehicle
grade; banking; curvature, position;
number of lane, lane number
Left_ range; left_ range_ rate;
right_ range;
right_ range_ rate; range;
range_ rate;
xDot;
yDot
position; number_ of_ lane; grade; banking, curvature
right_ range, left_ range, front_ range, back_ range, back_ right_ range, back_ left_ range;
right_ range_ rate, left_ range_ rate, front_ range _ rate, back_ range _ rate, back_ right _ range_ rate,
back_ left_ range _ rate;
Road ahead
Lead vehicle
Cognit ve
system
i
Vehicle- Roadway Environment Processor Distance, infra Road Sign + Landmarks
info
Front- Scanning ( default)
Front vehicle,
fuzzy front- left vehicle, fuzzy front-right
vehicle, fuzzy road
sign/ landmark
Scan fuel gauge ( 1.3
sec)
Fuel level, Fuzzy velocity, Fuzzy
vehicle, fuzzy road
sign/ landmark
Right- Scanning ( 2
sec)
Right vehicle,
Fuzzy front- right vehicle,
fuzzy rear- right vehicle, road
sign/ landmark
Front Right- Scanning ( 2
sec)
Front right vehicle,
Fuzzy front vehicle, fuzzy right
vehicle, road sign/ landmark
Read sign
direction ( 2 sec)
road sign/ landmark
Fuzzy vehicle
Scan speedometer
glance/ read/ exact
( 0.5 / 0.8 / 1.2 sec)
Fuzzy/ Clear velocity, Fuzzy
vehicle, fuzzy road
sign/ landmark
Left- Scanning ( 2 sec)
Left vehicle,
Fuzzy front- left vehicle, fuzzy
rear- right vehicle, road
sign/ landmark
Front Left- Scanning ( 2
sec)
Right vehicle,
Fuzzy front vehicle, fuzzy left
vehicle, road sign/ landmark
Scan left mirror
( 1.22 sec)
Rear- left vehicle,
Fuzzy vehicle, fuzzy road
sign/ landmark
Scan right mirror
( 1.5 sec)
Rear- right vehicle,
Fuzzy vehicle, fuzzy road
sign/ landmark
Scan in- vehicle
display/ mis- controls
( 1.1/ 1.0 sec)
Fuzzy vehicle, fuzzy road
sign/ landmark
Scan central mirror
( 0.6 sec)
Rear vehicle,
Fuzzy vehicle, fuzzy road
sign/ landmark
Perception
Module
xDot, yDot
Figure 5: Visual attention allocation as implemented in SmartAHS
The figure above provides the different areas that are scanned with the time it takes the
driver to extract the information. For example, if the driver is looking at the vehicle ahead,
and then scans the in- vehicle display, this means that for 1 second, he/ she does not
perceive any input from the front scene, and the data from the display is not available to
the cognitive module until the 1 second is over. Here also, we will distinguish between a
“ default” set of values and a customized set of values that can be entered by the user of
the simulation. For the current version of the model, the main rule is that every glance or
10
glance combination superior to 2 sec. is followed by a return to the front scene. This rule
will need to be adapted in the future based on other factors such as driver inattention. For
example, some empirical data describes that drivers are willing to engage in glances out
of the road or driving context for more than 2 sec. The mechanics underlying drivers’
willingness to engage in activities in which their visual attention is not engaged with the
front scene is starting to be quantified from an empirical standpoint and is likely to be
correlated to the demand of the driving task. In other words, when drivers feel like the
driving task is not very demanding ( e. g. driving on a highway with a low level of traffic,
the “ urge” to keep up with the front scene is diminishing.
Another element to include is the number of glances toward an information source for
gaining information, where drivers need to glance several times at a display, such as a
radio or in- vehicle display. In order to quantify these two dimension, we gathered
empirical data describing the time spent on location, number of glances necessary to
retrieve the information. As the data available in the literature vary in presentation
format, we propose to make sets available to the user to select for the simulation, and also
allow for the possibility to enter other sets. The table below lists possible sets specifying
glance duration and number of glances necessary for extracting information for specific
in- vehicle locations. For example, a radio control would demand between 2 to 7 glances
to extract the relevant information for the driver.
Table 3: Glance location and duration
Focus
Bhises et al. ( 1986) Time # of glances
Left mirror .5 to 1 1
Speedometer glance .4 to .7 1
Speedometer reading .8 1
Speedometer exact value 1.2 1
Central mirror .5 to .7
Radio control 1.1 2 to 7
In- vehicle display 1 to 1.2 7 to 15
Rockwell ( 1988)
Left mirror
Group A B C
Mean 1.06 1.22 1.1
Median .96 1.15 1.1
Standard deviation .4 .28 .33
5% .80 .94 .7
95% 1.2 1.8 1.7
Radio
Group A B C
Mean 1.27 1.28 1.42
Median 1.2 1.29 1.3
Standard deviation .48 .5 .42
5% .82 .89 .8
95% 2.16 1.83 2.5
11
The data set presented above is not exhaustive, as considerable effort has been devoted to
measuring glances to various locations in and out of the vehicle. The interest in gathering
a more comprehensive set depends on the setup that will be used to test this aspect of the
model.
The development of the model is iterative; the data cited above are descriptions of the
results of experiments in very controlled settings. Although they do not have a predictive
value by themselves, the goal of the model development is to associate them in order to
identify what part of driver behavior they do describe and which part they do not, which
also orients the need for new research.
4. Cognitive Module
The cognitive module manipulates the information sent by/ received from the perceptual
module. The manipulation of information is affected by the nature of the tasks
supporting the driving activity at any given time, which can be characterized in terms of
cognitive resources demand based on the cognitive mode ( verbal/ non verbal) on which
they are carried.
The driving activity, from a performance control perspective, is a succession of discrete
tasks, within one level of the activity ( e. g. switching from accelerating to braking) and
between levels of activity. The mechanisms allowing for these switches to happen are
key mechanisms to understand and describe in order to develop a driver model.
Another way to describe driving is that there is a main goal, reaching a destination, which
is going to be declined on a succession of sub- goals at different level of granularity based
on the goal of the modeling effort. In order to accomplish this goal, the driver will
alternate between states of anticipation, planning behavior, and states of reactions and
adaptation to the driving he/ she is interacting with. From this point of view, the cognitive
module has to support:
1. Activity planning: setting goal hierarchy and switching from goal to goal, for
example, the itinerary of the trip with a set of sub- goals between the origin and
destination
2. Adaptation of sub- goals to the situation, for example a sub- goal is to take an exit,
determine how to proceed based on traffic conditions
3. Reaction to event, for example, while changing lane to take an exit, there was a
vehicle in the blind spot, the driver either can react to the event or not
A caveat of the cognitive module is that only one of these three states can be controlled at
once, and however rapidly the switch between them can be done will rely on several
factors, that will be the “ modeled” with driver’s characteristics, in static terms such as the
level of experience, or dynamic terms, such as the level of fatigue. Also, while driving is
often described as a very complex task or activity by researchers, from the driver’s
perspective, it is viewed as a simple activity, inviting the driver to engage into other
activities either in order to fight boredom and/ or maintain vigilance. Therefore, another
12
dimension to represent in the model is the switching from one activity to another and the
cost of re- acquiring focus on the driving activity.
From this perspective, the cognitive module contains three entities:
• The driving knowledge, which entails all of the knowledge required for driving,
and at some levels is better described as skills
• Processes, and here there are two types:
o The ones tied to processing the information itself, such as decision making,
o The ones tied to the mechanics of processing the information, for example,
how long can information be held while making a decision?
• The ‘ space’ where information is manipulated, a space that is constrained based
on the rules from the second type of processes described above.
We will provide a description of our approach concerning the two first entities below.
The development of the third entity would be strongly associated with the method chosen
to implement the model, and this is a choice that will have to be made when the model
would move toward an implementation effort.
4.1. Driving knowledge stored in memory
For the purpose of the modeling effort, we distinguish between memories supporting
specific knowledge and generic knowledge. Each of these types of knowledge exists at
any level of driving performance, i. e., strategic, tactical and operational. Until now, most
of the work we conducted on driver model addressed the operational and tactical level,
here; we will describe how we organize the knowledge at the strategic level.
A description of the specific knowledge at his level is the one a driver uses when
commuting. In order to develop this aspect of the model, we are using data that was
collected previously, based on written description that one participant provided about his
commute to work, as well as the observation of the behavior.
In the table below, we illustrate how from the narrative that the participant provided
about the commute, we extract an itinerary composed of sub- destination, and the
association of these sub- destinations with landmark and actions.
13
Table 4: Driver commute
Sub- destination Landmark Action Preferences Part of
commute
Castle street Make a
right
Urban
driving
hillside blvd straight
7- 11 right turn
stop sign/
Mission St
Go straight
Walgreen’s
on right
Continue
straight
76 gas
station
Make left stay on the
far right
on ramp to 280
S
stay on the
left side
On- ramp
280 S Cruising 1
exit 380
101 S Continue
on 101
Cruising 2
3rd street in
San Mateo
Start
moving to
the right
lanes
take the 92 E cross over
the San
Mateo
Bridge
Cruising 3
After the
bridge
stay on the
left lane.
Urban
jackson st Continue
straight
on the left
lane
watkins Go 2
blocks
On the
third block
make a
right into
the 2 story
parking
structure
The next step once this table is made is to identify how it fits with the description of
behavior alternating between goal- oriented behavior and planning and reactive/ adaptive
behavior. Here, the “ sub destinations” become local goals at the strategic level, and
translate into local goals at the tactical level, such as “ make a left ( at an intersection)”
that will eventually translate into an execution of the maneuver ( i. e. steering and
acceleration/ deceleration). However, the sub destinations are discrete and, especially for
highway driving, separated by relatively long distances. For these sections, the behavior
14
becomes more reactive/ adaptive, and is regulated by looser rules. As a reminder, we
distinguish three type of zones in which the driver operates:
• Absolute: where a physical element constrains the behavior; for example, a driver
has to merge onto the highway before the end of the entrance lane, or merge
before turning at an intersection
• Relative: where there is no physical element relative to which the driver has to
execute a specific maneuver, the best example is when the driver is cruising on
the highway and the speed decisions are mostly governed by the driver
preferences and traffic
• Transition: when a driver is getting near the next section of the itinerary and needs
to adapt behavior.
In order to apply this taxonomy to the itinerary described by the participant, we used the
data collected while the participant drove on his or her commute for a period of two
weeks. From these data, we can identify the transition and absolute zone by plotting lane
change locations and directions. The results are shown below in Figure 5. This commute
section is approximately 12 kilometers long, and on a 4 to 5 lane highway, with a
relatively dense and fast traffic at the times of the participant’s commutes and where the
participant’s preferred lanes are the second and third lanes from the entrance.
Driver 1 Morning Commute Cruise 2 Lane change
0
2000
4000
6000
8000
10000
12000
0 1 2 3
Lane change direction ( 1 = to left, 2 = to right)
Position ( meters)
Trip 05 Trip 08 trip 12 Trip 23 Trip 26 Trip 30 Trip 33 Trip 37
TraTnsraitniosint i1o n 1
toward exit lane
Transition 2
toward exit lane
Relative
Transition to
preferred lane
Figure 6: Determination of a driver's lane preferences
We identify the transition to the preferred lane within the first four kilometers of the
section, followed by the same distance in a relative zone, and then followed by the
transition toward the exit lane. The interest of this approach is to support a better
15
characterization of lane changes decisions and the parameters that are used while
conducting the lane changes, in terms of time gap with vehicles in origin and destination
lane for example. Unfortunately, due to difficulties isolating the lead and following
vehicles out of the data files collected, we cannot at this time provide the data quantifying
this behavior. Also, with a longer data collection which would allow to control for
conditions such as the density of traffic, it would be possible to determine a driver’s
preference for moving toward the exit based on traffic density, and also distinguish
between commute and unfamiliar itineraries, which would in turn allow to recreate the
diversity of behaviors that can be observed on the road.
4.2. Processing information
4.2.1. Manipulation and interpretation of information
In this section, we introduce a set of processes that support the interpretation of the world
and the set of rules used for that purpose. This set would have to be considerably
increased based on the type of implementation that would be done of the model, and are
therefore provided in order to illustrate the type of rule that would be used for
characterizing the environment, which would support the identification of the relevant
knowledge for the driving situation.
Traffic density determination:
If number of vehicles within the next 150 m is between 0 and 2 and speed is above 50
mph, then traffic condition is light
If number of vehicles within the next 150 m is between 2 and 4 and speed is above 40
mph, then traffic condition is medium
If number of vehicles within the next 150 m is between 4 and 6 and speed is above 30
mph, then traffic condition is heavy
If number of vehicles within the next 150 m is above 6 and speed is below 30 mph, then
traffic condition is congested
Collision estimator:
If gaze is no more than 6 degree ( combination of horizontal and vertical) from a vehicle
for which Time To Collision ( TTC) is equal or below 3 sec, then send “ imminent
collision” to operational
Lane change need estimator:
If tactical is under infra control and current lane is different from desired lane, then lane
change need is very high ( e. g. change lane to take an exit)
16
If tactical is under transition zone or following an undesirable vehicle ( type ( truck) or
speed), then need is high
If tactical is under relative zone, then need is medium or low, depending on how
unsatisfying is either the lead type ( e. g. a truck is more undesirable than a SUV) or the
lead speed ( the slower relative to preferred speed, the higher the need)
4.2.2. Processes constraining the manipulation of information
The attention controller will be a cognitive process describing how the driver switches
focus of attention from non- driving activities to the driving activity and switches from
tasks to tasks within the driving activity.
Three mechanisms are involved with the focus and shifting of attention:
• Selective attention: filtering of relevant information among all incoming
information, e. g. a conversation among the noise produced by a crowd, or, applied
to driving, focus on the light for through traffic and ignoring the light for turning
vehicles.
• Focused attention: allocation of attention to one known source of information
( target) and the effort made to keep attention directed to this target. For example,
the target can be the lead vehicle; this effort would demand little resource when
driving under clear weather and low traffic volume but require more resources
under degraded conditions, such as fog or when the task is more demanding, such
as stop and go traffic.
• Sustained attention: Actively process incoming information over a period of time.
Sustained attention is the process involved in vigilant tasks.
These mechanisms, or attentional processes, have to be implemented in order to control
the flow of information that the cognitive system is processing. This means that for each
task there is a list of relevant information to use in order to complete the task, this is the
static aspect of the model. The attentional processes will have a set of rules relative to
their activation, and when the simulation is run, based on the availability of the selective
attention process, the relevant information will be selected or not. For example, when
simulating fatigue, one of the processes that will be degraded will be the one of sustained
attention. If selective attention is degraded, then the driver will be processing multiple
sources of information, not necessarily relevant to the driving task, which will be
accompanied by a time cost.
17
Driving Activity
Operational
control
Tactical control
Strategic control
Non Driving
related Activity 1
Non driving
related activity 2
Figure 7: Attention switching between activities and within the driving activity
The attention controller performs at two levels, within the driving activity and in between
activities, such as talking to a passenger or using a cell phone, and each of these non
driving related activity come with an associated cost in terms of “ reacquisition” of the
driving activity or driving relevant information that can be perceived while driving based
on the other activity that the driver is engaged with.
The non driving activities considered for the model development are the ones either the
most commonly studied for their impact on the driving activity ( e. g. cell phone
conversation) or the ones that are pointed as a cause of distraction leading to crash ( e. g.
disciplining children).
Each activity impact driving in different ways, although it might become practical to
order these activities into categories as a function of the driving tasks they interfere the
most with. For example, the impact of a cell phone conversation with a hands- free phone
would be the reduction of the useful field of view, which in turn would decrease the
chances of reaction to an event which would be in the useful field of view otherwise. We
can also integrate the different case of a handheld cell phone vs. a hands free cell phone
in order to reproduce the impact of the different phases of the call ( calling, picking- up,
talking) has on the different tasks involved in driving. For example, picking up a cell
phone or searching a cell phone can influence the steering control, while some
conversations can influence the scanning ability.
The switching between these activities is based on events ( e. g. phone ring) or set by the
simulation user ( e. g. set listen to radio before starting simulation). In some cases, the
activities are discrete, such as a cell phone conversation, or can be continuous, such as
18
conversation with a passenger or listening to the radio, with different level of engagement
from the driver, for example the radio could be a background supporting day dreaming or
the driver could be very engaged into listening to the program.
19
5. Future research: specifications for a “ driver simulator”
The presentation of the work conducted under this project focused on a “ paper”
development of a model. Although it was originally proposed to implement the model
that was being developed, we lost the key member of the team for the implementation of
the project shortly after the beginning of the project. Therefore, the presentation of this
work is alternating with relatively detail description for part that where near
implementation, and more general for sections that were mostly in discussion but also
depends of the implementation scheme in order to be further specified. Therefore, in
conclusion of this report, we propose to discuss the following points:
• What are the challenges for providing an information processing architecture and
consequent models to support the design of an ITS and its evaluation via
simulation?
• What are the requirements for developing and implementing a driver model?
The main challenge for providing an information processing architecture is the lack of
existing validated architectures from the human sciences community that could be
applied to driving activity. This architecture would, among other things, integrate the
mechanics of the process, in terms of time and space, e. g., even though there is empirical
data as to how much information can be held in working memory, there is no dynamic
model of how the information is kept and replaced, and how it fits within the demand of
an activity such as driving.
From this perspective, the first step before aiming at implementing a driver model that
has some cognitive aspects into a broader micro- traffic simulation consist of developing
an environment that will allow for assembling cognitive models into an architecture and
testing the resulting performance of the model, in other words, a driver simulator.
This simulator would allow reproducing driver errors and behaviors that are leading to
crashes or near- crashes, and would therefore require to recreate all of the driving levels
usually described in psychology and human factors as they apply to driving, i. e.,
navigation, tactical and operational. The need for such a model arises from a common
representation that driving can be summarized to controlling a vehicle’s motion.
Although controlling a vehicle motion is what can be observed externally, the elements
underlying this control, and, more importantly, interfering with the control of a vehicle,
run much deeper than the skills involved with maintaining a vehicle within a lane or
regulating a speed or a gap.
Developing such a comprehensive driver model consists of associating parts of research
that are conducted on specific mechanisms with no or little acknowledgment of the role
of other mechanisms, be it at the theoretical level ( e. g. working memory behavior) or
applied level ( e. g. in vehicle system demand). The studies that need to be associated also
investigate driving at different levels of focus, with data collection focused on glances
duration and others aiming to describe tasks and maneuvers. In other words, there is a
need to reconcile the description of the cognitive processes with the description of the
driving activity and the set of tasks composing it.
20
As important as the need for a comprehensive model of driving is the need for a tool
permitting to visualize the behavior of the model, but also as a support for
communicating between peers involved in the design of an ITS or of a driving
environment. For example, a 3D rendering of a driving scene with an overlap of the
different fields of view and the area they cover would be a very powerful media to
communicate either the differences between some age groups or how its reduction based
on events such as a phone call could impact the chances of seeing relevant information.
In order to develop such a tool, a team composed of researchers issued from the fields of
Human Sciences, Computer Sciences and Engineering would have to be assembled, and
their area of knowledge would have to include cognition, perception, performance,
implementation of cognitive systems, and finally development of software integrating
modules as varied as models of the Human, vehicle and traffic models.
21
22
Bibliography
Aasman, J., & Michon, J. A. ( 1992). Multitasking in driving. In J. A. Michon & A.
Akyürek ( Eds.), Soar: A cognitive architecture in perspective ( pp. 81- 108). Dordrecht,
The Netherlands: Kluwer.
Anderson J. R., Lebiere C. ( 1998) The atomic components of thought. Lawrence
Erlbaum Mahwah, NJ: Erlbaum.
Bhise, V. D., Forbes, L. M., Farber, E. I., ( 1986) Driver behavioral data and
considerations in evaluating in- vehicle control and displays Paper presented at the
Transportation research board, National academy of Sciences, 65th Annual Meeting,
Washington D. C.
Crundall, D., Underwood, G., and Chapman, P. ( 1999) Driving experience and the
functional field of view Perception, 28, 1075- 1087.
Hoffmann E. R. and Mortimer R. G. ( 1996) Scaling of relative velocity between
vehicles. Accident Analysis and Prevention, 28- 4, 415- 421.
Rasmussen J. ( 1983) Skills, rules, and knowledge; signals, signs, and symbols, and other
distinctions in Human performance model IEEE Transactions on systems, man and
cybernetics, Vol. SMC- 13, N° 3, May/ June 257- 266
Rockwell, T. ( 1988). Spare visual capacity in driving revisited. In A. Gale, M. Freeman,
C. Haslegrave, P. Smith, & S. Taylor ( Eds.), Vision in vehicles II ( pp. 317- 324).
Amsterdam: Elsevier.
Serafin C. ( 1993) Preliminary examination of driver eye fixation on rural roads: insight
into safe driving behavior UMTRI report - UMTRI- 93- 29, 68 p.