
small (250x250 max)
medium (500x500 max)
Large
Extra Large
large ( > 500x500)
Full Resolution


ISSN 1055 1425 November 2007 This work was performed as part of the California PATH Program of the University of California, in cooperation with the State of California Business, Transportation, and Housing Agency, Department of Transportation, and the United States Department of Transportation, Federal Highway Administration. The contents of this report reflect the views of the authors who are responsible for the facts and the accuracy of the data presented herein. The contents do not necessarily reflect the official views or policies of the State of California. This report does not constitute a standard, specification, or regulation. Final Report for Task Order 5400 CALIFORNIA PATH PROGRAM INSTITUTE OF TRANSPORTATION STUDIES UNIVERSITY OF CALIFORNIA, BERKELEY Vehicle/ Driver Monitoring for Enhanced Safety of Transit Buses UCB ITS PRR 2007 25 California PATH Research Report Mingyu Shi, Masayoshi Tomizuka University of California, Berkeley CALIFORNIA PARTNERS FOR ADVANCED TRANSIT AND HIGHWAYS 1 Project Title Vehicle/ Driver Monitoring for Enhanced Safety of Transit Buses By Mingyu Shi Masayoshi Tomizuka Department of Mechanical Engineering University of California at Berkeley Berkeley, CA 94720 Final report for project TO5400 January 2007 2 Contents List of Figures ............................................................................................................................... ................ 3 List of Tables......................................................................................................................... ........................ 3 Abstract ............................................................................................................................... .......................... 4 1. Executive Summary ............................................................................................................................... 5 2. Experimental Settings ............................................................................................................................ 7 3. Driver/ Vehicle Operation in A Closed loop System............................................................................ 10 4. Off line ARMAX Driver Model Identification.................................................................................... 11 4.1. Model Order Selection ............................................................................................................. 13 4.2. Sample results of Off line data analysis................................................................................... 14 5. Driver’s Response Time....................................................................................................................... 17 6. Vehicle Lateral Dynamics .................................................................................................................... 20 7. Collision and Road Departure Warning Systems ................................................................................. 22 7.1. Frontal Collision Warning System ........................................................................................... 22 7.2. Road/ Lane Departure Warning System.................................................................................... 24 8. On line Data Processing..................................................................................................................... . 26 9. Warning Message Issuing Mechanism................................................................................................. 29 10. Conclusion..................................................................................................................... .................. 30 Reference ............................................................................................................................... ..................... 31 Appendix....................................................................................................................... .............................. 35 A. Brief review of ARMAX model................................................................................................... 35 B. Recursive Least Square Method................................................................................................... 37 C. Review of closed loop identification methods ............................................................................. 39 3 List of Figures Figure 1. Test vehicle – 2002 Ford Taurus Sedan ( CA license # 1046993)............................................ 7 Figure 2a. SafeTRAC system 2b. Sample view from SafeTRAC video camera ......................... 8 Figure 3. Test route.......................................................................................................................... ...... 9 Figure 4. Block diagram of the driver/ vehicle closed loop system ...................................................... 10 Figure 5. ARMAX driver model............................................................................................................ 11 Figure 6. Look ahead distance and Look ahead lateral position ........................................................ 12 Figure 7. Histogram of the ARMAX model orders ............................................................................... 14 Figure 8. Sample results from the data analysis on one data subset.................................................... 15 Figure 9. Scatter plot of error and QQ plot f the model ...................................................................... 16 Figure 10. Inverse of reaction time versus time on task plot [ 28]........................................................ 17 Figure 11. Driver’s response time estimation from the estimated driver model, .................................. 18 Figure 12. Time to collision notion illustrated with vehicle trajectories [ 19] ..................................... 23 Figure 13. Time to lane crossing ( TLC) assuming constant steer angle. [ 14] ..................................... 24 Figure 14. Sample results of the online estimation for coefficients of A( q  1 ) and the poles ................. 27 Figure 15. Closed loop prediction and calculation of TLC ................................................................. 28 Figure 16. Block diagram of the alarm issuing mechanism................................................................. 29 Figure 17. Block diagram of an ARMAX model................................................................................... 35 Figure 18. Feedback closed loop system ............................................................................................. 39 List of Tables Table 1. Collected data and the corresponding units ............................................................................. 8 Table 2. Vehicle Parameters................................................................................................................. 21 4 Abstract This report documents the findings of the study “ Vehicle/ Driver Monitoring for Enhanced Safety of Transit Buses” conducted as part of the PATH project TO5400. The goal of study TO5400 is to design and implementation of a reliable vehicle/ driver monitoring system with aim to enhance driving safety of transit buses. The main objectives of the project are: ( 1) to identify a simply model that best describes the driving patterns of human drivers, and ( 2) to develop an algorithm which can generate warning messages when there is a provision of danger. In this study, it is assumed that the driver sets the angle of the steering wheel in response to the lateral deviation of the vehicle from the center line of the road and/ or the road curvature ahead of the vehicle which he/ she infers from the visual perception. It is also assumed that the dynamic relationship between the lateral deviation and the road curvature ( i. e. the inputs to the driver) and the steering angle ( i. e. the output from the driver) may be represented by a linear model with Auto Regressive Moving Average with eXogeneous ( ARMAX) structure. The order of the ARMAX model is determined by off line analysis of experimental data collected from actual driving tests. Once the order of the model is determined, the parameters of the model can be estimated by processing the input and output data in real time during actual driving. The on line data processing and prediction can also estimate three critical parameters for driving safety: the driver’s response time ( tr), the time to collision ( TTC) and the time to lane crossing ( TLC) TLC. A timely warning message scheme, which incorporates all the three parameters, is designed to determine whether an alarm should be issued. Keywords: ARMAX driver model, closed loop system identification, collision warning system, response time, road/ lane departure warning system, time to collision, time to lane crossing. 5 Executive Summary Vehicle safety is one of the most important concerns of transit authorities and automotive manufacturers. Vehicle crashes cause more than 41,000 fatalities per year, as well as 5.3 million injuries, and cost the society more than $ 230 billion per year. A large portion of these accidents is due to drivers’ inattention or inability to properly perform driving tasks. Automotive manufacturers have introduced driver assistant systems such as collision warning systems ( CWS) and road departure warning systems ( RDW) to enhance safety. However, most CWS/ RDW systems cannot directly monitor the driver while the negligence of the driver is the major cause of highway accidents. If a system is capable of monitoring the driver’s state from available data and incorporating it into the driver assistant systems, then it will be possible to alert the driver in a timely and effective manner and greatly increase the safety of highway driving. In this study, we develop an effective and reliable vehicle/ driver monitoring system, where the driver’s state can be retrieved from data collected on line and a warning message can be issued to the driver if his/ her state deviates from the normal state in a significant manner. The driver is modeled as a dynamic process with an ARMAX ( Auto Regressive, Moving Average with eXogenous input) structure, where the inputs of the model are the look ahead error and the road curvature, and the output is the angle of the steering wheel. From off line data analysis, the best combination of the orders of the ARMAX model ( which is applicable to all drivers) is first found. This particular structure is employed in on line data processing, where it is coupled with the vehicle lateral dynamics for closed loop identification and prediction. The driver state ( for example the response time) can be determined from the parameter estimates. A simplified linear bicycle model is utilized to describe the vehicle’s lateral motion, under the assumptions of small steering angle, negligible roll and pitch motion and linear tire model. Key parameters used in the CWS/ RDW systems, such as the time to collision ( TTC) and the 6 time to lane crossing ( TLC), are predicted based on the driver model and vehicle’s lateral dynamics in a closed loop system. A warning system, which takes into account the driver’s response time, the TTC and TLC, is designed to determine whether a warning message should be issued to the driver. The system uses prescribed thresholds. The major contributions of this project include: 1. A simple yet effective method is developed to monitor the driver from the view point of driving skills and their variations due to inattentiveness and/ or drowsiness. 2. The resulting vehicle/ driver monitoring system can send an early warning to the driver to alert his/ her abnormal driving behavior. It can improve the effectiveness and reliability of current CWS/ RDW systems, and enhance the safety of highway traffic. 7 2. Experimental Settings Experiments have been conducted on a passenger car 1 at the Richmond Field Station of University of California at Berkeley. The test vehicle is a 2002 Ford Taurus Sedan ( CA license # 1046993). We show a photo of the test vehicle in Figure 1. Detailed information regarding the test vehicle can be obtained from PATH. SafeTRAC Figure 1. Test vehicle – 2002 Ford Taurus Sedan ( CA license # 1046993) For any collision warning and vehicle monitoring system, it is essential to have a reliable sensing system that senses the environment in front of and around the vehicle. A SafeTRAC 2 video camera is installed under the windshield of the test vehicle, which monitors the road in front of the vehicle. The video system tracks the lane markings, as shown in Figure 2b, and retrieves road geometry information such as the lane width, the relative lateral position of the vehicle with respect to the center of the lane, and the road curvature. Other information is measured by on board sensors. For example, the steering angle is measured by the encoder installed under the steering wheel. A list of collected variables is shown in Table 1. The sampling time is 0.075 second, which corresponds to a sampling frequency of 13Hz. There 1 As suggested by the title of the project, the original plan was to study vehicle/ driver monitoring for transit buses. It turned out, however, that it was not practical to use a bus for experimental data gathering because of an experimental buses and qualified bus drivers were both difficult to obtain. 2 SafeTRAC is an on board electronic warning system that prevents crashes caused by driver drowsiness or distraction. Manufactured and developed by Assistware Technology, SafeTRAC uses a forward looking video camera to track a vehicle’s position in its lane. 8 is also a computer installed on the test vehicle, which is used to record and process collected data. Figure 2a. SafeTRAC system 2b. Sample view from SafeTRAC video camera Data Unit clock time of day HH: MM: SS: sss steering wheel angle deg acceleration x m/ s/ s acceleration y m/ s/ s throttle position % Brake 0= off, 10= on left turn signal 0= off, 10= on right turn signal 0= off, 10= on speed over ground km/ h lateral offset cm lateral velocity cm/ s road curvature 1/ m lane width cm Table 1. Collected data and the corresponding units Driving tests are conducted on a specified segment of Interstate Highway I 80 between Richmond and Fairfield, CA. Figure 3 shows a map of the test route. We choose this segment because: ( 1) there are both straight and curved sections, which assure that the collected data is 9 informative enough ( refer to Appendix C for more detailed information), ( 2) this segment can be easily accessed from Richmond Field Station ( RFS) where the test vehicle is maintained, and ( 3) the traffic on this segment is normally not too congested to properly carry out the experiments. A typical testing route consists of the following: ( 1) the tests start from RFS; ( 2) the drivers go on I 80 from Carson Blvd entrance ( point A on the map); ( 3) the drivers turn back at Red Top Road ( point B on the map); and ( 4) the tests end at RFS. The length of the round trip route is approximately 50 miles. Figure 3. Test route from Yahoo map Twenty experienced drivers participated in the experiment. They consisted of 15 males and 5 females. In the experiment, they were asked to drive the test vehicle in the normal way they would drive everyday. The test for each driver took about one hour, and it was either between 9 and 11am or between 1pm and 4pm, when traffic was relatively low. Also, the experiments were normally taken under clear weather conditions, with only one exception of raining. On board sensors and the SafeTRAC video camera were used to collect data for each test, as shown in Table 1. 10 3. Driver/ Vehicle Operation in A Closed loop System The vehicle lateral dynamics appears in a feedback closed loop system in the sense that the human driver observes how the vehicle goes and takes corresponding control of the vehicle. The closed loop system can be represented by the diagram in Figure 4. Driver Vehicle steering angle lateral position road curvature  Figure 4. Block diagram of the driver/ vehicle closed loop system One of the major tasks of this project is to identify a proper driver model. This is a closed loop identification problem, since it is not possible to open the loop and study the driver. A brief review of closed loop identification techniques is provided in Appendix C. The direct method, as described in the appendix, is adopted in this project. 11 4. Off line ARMAX Driver Model Identification The modeling of human driver’s behavior has attracted great attentions from many researchers recently. In order to model the driving pattern, complex input output models, such as the nonlinear regression models and the neural network and fuzzy systems have been used. It is not our intention to introduce another model for a comprehensive description of driver behaviors. Rather, we are interested in a driver model that will benefit the monitoring and warning system, or more specifically, a model that may represent the driver as a time varying controller of the vehicle. Previous studies show that the ARMAX ( Auto Regressive and Moving Average with eXogenous) model is a promising candidate for human driver models [ 4]. In this project, we follow this direction and test the ARMAX model with real experimental data. A brief review of the model is presented in Appendix A. In this project, we construct a model of the human driving behavior based on its representation as an ARMAX model. The structure is shown in Figure 5. One advantage of this model is that a rich body of knowledge is available in the field of modeling and identification of dynamic systems. Another advantage is that many of its coefficients have clear physical interpretations; for example, the delay time may be estimated based on the poles of the identified model, and the delay time may be interpreted as the driver’s response time. We refer to Section 5 for more details. e δ 1 1 ( ) ( ) C z A z − − + + y l ρ Input ( ) Output ( ) 1 1 − − − A z B z z nk Figure 5. ARMAX driver model 12 The inputs of the ARMAX driver model are the look ahead lateral position of the vehicle l y and the look ahead road curvatureρ ; and the output is the steering angleδ . l y and ρ are measured by the SafeTRAC video system, which captures the road condition in front of the vehicle. A graphic representation is shown in Figure 6. The look ahead distance can be adjusted by changing the angle of the lens when the SafeTRAC camera is installed, and we set it as 20 meters in this study. In other words, the look ahead lateral position is the lateral position of the vehicle with respect to the road condition 20 meters ahead of the vehicle. Similar interpretation holds for the road curvature. The output δ is measured by the encoder installed under the steering wheel. y y l look ahead distance Figure 6. Look ahead distance and Look ahead lateral position The ARMAX model can be expressed as, [ ] ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 1 1 2 1 1 1 C q e t t y t A q t q B q B q n l k − − − − − + = ρ δ ( 1) where, the noise e( t) is assumed to be a white Gaussian process, and ( ) − 1 A q , ( ) 1 1 − B q , ( ) 1 2 − B q and ( ) − 1 C q are the model polynomial operators, with orders a b b n , n , n and c n respectively, i. e., a a n n A q a q a q a q − = + − + − 2 + Λ + − 2 1 1 1 ( ) 1 ( 2) b b n n B q b q b q b q − = − + − + + − 1 2 12 1 11 1 1 ( ) Λ ( 3) 13 b b n B q b q b q b n q − = − + − + + − 2 2 22 1 21 1 2 ( ) Λ ( 4) c c n n C q c q c q c q − = + − + − 2 + Λ + − 2 1 1 1 ( ) 1 ( 5) 4.1. Model Order Selection First of all, we investigate what order set ( a b c k n , n , n , n ) the ARMAX model should take to yield the best representation of human driver. Since data is collected from 20 different drivers, and parameters may vary from time to time, the data are divided into 966 subsets, with each one contains data of 30 seconds long. The selection of orders is carried out using the following approach, 1) For each subset of the data, and for each combination of orders of the ARMAX model ( a b c k n , n , n , n ), we estimate the model and calculate the final prediction error ( FPE) 3 with the prediction error method ( PEM) 4 . The choices of the model orders include all possible combinations of ∈[ 1,5] a n , ∈[ 1,10] b n , ∈[ 1,20] c n and ∈[ 1,10] k n . 2) The lowest FPE value is found for each subset of data, and the corresponding orders ( a b c k n , n , n , n ) are detected. The FPE can be used as a model selection criterion. The ARMAX model with orders corresponding to the smallest FPE, is selected as the optimal model for this subset. 3) For the 966 subsets, there are 966 combinations of ( a b c k n , n , n , n ) with the lowest FPEs. A histogram is plotted for each component to determine the overall orders. 3 FPE is a model selection criterion, defined as d N d N FPE V − + = 1 1 , where V is the loss function calculated from the model estimation, d is the number of model parameters, and N is the number of available data. FPE compensates the effect of increasing order, i. e. the bigger the order, the smaller the loss function. Other model selection criteria, which compensate such order effects, include Akaike’s Information Criterion ( AIC), Bayesian Information Criterion ( BIC), Mallow’s Cp, and Minimum Description Length ( MDL), etc. 4 The prediction error method estimates model parameters θ by minimizing the loss function = Σ = N t t N VN 1 ( , ) 1 ( ) 2 θ ε θ , where ε is as defined in Appendix A and N is the time duration, and N = 400 for each data subset in this project. 14 MATLAB 6.5 System Identification Toolkit was adopted to perform task ( 1). Given the data and the prescribed order combination ( a b c k n , n , n , n ), the armax function from the toolkit can estimate the parameters of the model. The FPE of the model can also be calculated. The armax function uses an iterative search algorithm to minimize the prediction error. The iterations are terminated when the expected improvement between adjacent iterations is within a predetermined tolerance, or when a fixed number of iterations has been reached. The iterative algorithm is robust; however, it may take a long time to converge. On a Pentium M 1.6GHz computer, it takes around 5 minutes to obtain results for one subset of data. Thus the toolkit can only be used for the off line data analysis. We resort to a recursive algorithm for our on line data processing. We refer to Section 8 for more details. The histograms in step 3) are shown in Figure 7. We observe that = 3 a n , = 1 b n , = 17 c n 5 and = 1 k n occur most frequently, which means for all the drivers and any 30 seconds time intervals, ARMAX( 3,1,17,1) model can best describe human driver’s behavior. On line parameter identification will be based on this selected model. 1 2 3 4 5 0 1 00 2 00 3 00 4 00 5 00 600 1 2 3 4 5 0 5 0 1 00 1 5 0 2 00 2 5 0 3 00 1 2 1 3 1 4 1 5 1 6 1 7 1 8 1 9 2 0 0 5 0 1 00 1 5 0 2 00 1 2 3 4 5 6 7 8 9 1 0 0 5 0 1 00 1 5 0 2 00 nb= 1 nc= 17 nk= 1 na= 3 Figure 7. Histogram of the ARMAX model orders 4.2. Sample results of Off line data analysis The model estimation results from one subset of data are shown in Figure 8 and Figure 9. Each 5 From the histogram of c n , we see that = 16 c n is almost as frequent as = 17 c n . However, since the noise itself is unknown exactly, we defer the investigation of the effect that small difference in choice of c n will cause to a later study. 15 subset contains observations from a 30 second period for one driver. The estimation is based on the above selected ARMAX( 3, 1, 17, 1) model. Figure 8 shows the inputs of the model – the look ahead lateral position and the road curvature, in the upper right and lower left plots, respectively. It also shows the results from the data analysis based on the model, i. e. the estimated output and error. 6 60 670 6 80 690  2 0 2 4 6 S teer ing Ang le 6 60 670 6 80 690  4 0  2 0 0 2 0 4 0 Lateral po sitio n 6 60 670 6 80 690  1  0. 5 0 0. 5 1 1 . 5 x 1 0  3 Road Curvature time 6 60 670 6 80 690  2  1 0 1 2 E rror time Figure 8. Sample results from the data analysis on one data subset The output of the model – the steering angle – is shown in the upper left plot of Figure 8. The blue line is the measured steering angle, and the red line is the estimated output from the model. There is only small discrepancy between the measured and estimated outputs. The R 2 value 6 of the fitted model is 92.7%, which indicates that the selected model can explain a great majority of the variations and thus satisfactory fitting. 6 R 2 value is the fraction of the variance in the data that is explained by the model. It is also called as the coefficient of determination. It indicates how well the model fits the data, e. g. an R 2 close to 100% indicates that we have accounted for almost all of the variability with the variables specified in the model.. 16  2 0 2 4 6  2 0 2 4 6 estimated steering angle steering angle 660 670 680 690  1 . 5  1  0. 5 0 0. 5 1 1 . 5 Error time Figure 9. Scatter plot of error and QQ plot f the model Comparison of the measured data versus estimated data is also shown in the QQ plot 7 in Figure 9. The measured steering angle is plotted on the horizontal axis, and the estimated steering angle from the model is plotted on the vertical axis. The data points are all close to the reference line – the straight line with unit slope and zero intercept. It indicates that the underlying model can sufficiently explain the data. Furthermore, there is no significant trend in the scatter plot of the error. Thus the model assumption that it has an ARMAX( 3,1,17,1) structure is validated. 7 The quantile quantile ( QQ plot) is a plot of the quantiles of the first data set against the quantiles of the second data set. It is a graphical technique for determining if two data sets come from populations with a common distribution. A 45 degree reference line is also plotted. If the two sets come from a population with the same distribution, the points should fall approximately along this reference line. The greater the departure from this reference line, the greater the evidence for the conclusion that the two data sets have come from populations with different distribution. 17 5. Driver’s Response Time The response ( reaction) time is defined as the difference between the time the driver observes a lateral deviation and the time a corrective action is taken. It consists of basically components processing time and transmission time ( neuromuscular lag) [ 3]. Processing time is the time needed to decide what corrective action should be taken, and transmission time is the time necessary to initiate that action after the decision has been made. Various studies, in many different science fields, have been conducted to investigate the human response time. It has been shown that age, drunkenness, and drowsiness can affect human’s driving performance, especially the response time. For example, about 60% of young drivers ( 23 years old or less) exhibit the control delay on straight lanes less than 0.4 sec while less than 20% of old drivers ( 75 years old or more) exhibit the same statistics [ 12]. Figure 10 shows qualitatively the effects of another factor: drowsiness on the reaction time [ 28]. Figure 10. Inverse of reaction time versus time on task plot [ 28] In this project, we propose a new method to retrieve information about the driver’s response time. The method is based on the identified model and its corresponding parameters. First, we note that the 3 rd order ARMAX model has 3 poles, i. e. ( ) ( 1)( 2 ) 2 2 n n A s = τ s + s + ξω s + ω ( 6) The first term in equation ( 6) usually corresponds to the delay in the system, and we can interpret it as the driver’s response time, i. e. = τ r t . 18 Figure 11 presents an example of how to determine the driver’s response time. First, since the model is estimated in discrete time, we have to transform it into the continuous time domain in order to find the response time. A relationship between the poles for discrete system and its corresponding continuous system, using the zero order hold equivalent transformation, is given by, exp( ) d c s p = p T or c d s p = ln( p ) / T ( 7) where pc is the continuous time pole, pd is the discrete time pole, and Ts is the sampling time. In this project we set Ts as 0.075 second. Once the continuous poles are determined, we choose the one that corresponds to the pole in the first term on the right side of equation ( 6) 8 , i. e. = − 1 τ c p . The response time is then computed as the negative inverse of that pole. For example, the red crosses in Figure 11 are the results from one subset of data, and the pole corresponding to the delay time is chosen as = − 6.7 c p , which leads to the drivers response time at this instant = τ ≈ 0.15 r t seconds. Discrete Poles Continuous Poles  1  0 . 5 0 0 . 5 1  1  0 . 5 0 0 . 5 1  1 0  5 0  3  2  1 0 1 2 3 τ 1 − p d = exp( p c T s ) Figure 11. Driver’s response time estimation from the estimated driver model, Red and Green points are the results from two different data subset. 8 In the case that the system has three real poles, the one closest to the negative real axis is chosen as the desired pole. This is the worst case of the possible delay time. In designing the warning system, if the worst case – maximum response time – stays in the range of normal driving, other possible delays are also in the range. Otherwise, there is a possibility that the delay time is long, and a warning shall be given. 19 The driver’s response time tr indicates how fast the driver can react if emergency occurs during driving. The designed warning system takes this information into account to determine whether to issue a warning message or not. In section 9, we set the threshold of the driver’s response time to 0.5 second. Furthermore, the thresholds of the time to collision and the time to lane crossing are also related to the driver’s response time. With a more accurate estimation of the driver’s response time, the warning system may perform better by issuing necessary warnings and reducing the false alarm rate. Further discussions on issuance of warning messages are in Sections 7 and 9. 20 6. Vehicle Lateral Dynamics The vehicle and driver interact with each other in a close loop form, as shown in Figure 4. Thus a dynamic model of the vehicle is necessary for the on line data prediction when the measured data is not available. This research project focuses on the lateral movement of the vehicle and its lane crossing/ departure events. For this purpose, a simplified bicycle model for the vehicle lateral dynamic is adopted. The bicycle model assumes a small steering angle, negligible roll and pitch motion and a linear tire model [ 9]. It can be described as a fourth order system in the state space as, [ ] = + ρ δ 1 2 x& Ax B B ( 8) ( ) T CG CG r r x = y y& ε ε& ( 9) where x is the state variable, δ is the front wheel steering angle, ρ is the road curvature ( disturbance), CG y is the lateral deviation at the vehicle CG ( Center of Gravity), and r ε is the yaw angle of the vehicle sprung mass relative to the road reference frame. The system matrices are − − + = − − − + + + − + + z f s r s z f r z x f r x s s x I l C d l l C d l I l C l C I v l C l C v d l d l v A ( ) ( ) ( ) ( ) 1 2 1 2 1 2 1 1 2 2 1 2 1 1 2 2 0 0 0 0 1 0 0 1 0 0 α α α α α α φ φ φ φ φ φ ( 10) ( ) T I l C z f B α φ 1 0 0 1 1 = ( 11) T I x l C l C B l l v z f r = − − − + 2 2 2 2 1 1 0 0 2 2 2 1 φ φ α α ( 12) where 2 1 m C f α φ = and m C r α φ 2 2 = . The physical meanings and values of the symbols used in the above equations are listed in Table 2. 21 Symbol Physical Meaning Value m mass of the vehicle 1485 kg L relative longitudinal distance between vehicles 5 m s d distance of rear bumper to CG 2.1 m z I yaw moment of inertia 2872 kg/ m2 f Cα front wheel cornering stiffness 4200 N/ rad r Cα rear wheel cornering stiffness 4200 N/ rad 1 l distance between front wheel and the CG 1.1 m 2 l distance between rear wheel and the CG 1.58 m Table 2. Vehicle Parameters 22 7. Collision and Road Departure Warning Systems There are several types of collision warning systems on the market nowadays, which include frontal collision, intersection collision and road departure warning systems. A frontal collision warning system evaluates the time to collision to determine whether a warning should be issued; an intersection or side collision warning system targets at alleviating the hazards in lane changing, and a road departure waning system issues warnings whenever the vehicle tends to depart from the road without a turning signal. In addition to monitor the vehicle’s lateral position, a warning may be issued by observing the driver’s inappropriate states ( e. g. the driver’s response time). For example, a drowsy driver needs longer time to react to any emergency situation. In this case, if the driver gets warning messages right before the vehicle is too close to the lane, he/ she may not have enough time to react. Also, a drowsy driver may stop reacting to the lateral deviation of the vehicle, which will result in a large prediction error in the identification algorithm. In this case, a timely warning shall be issued before the vehicle gets too close to the lane boundary. A warning system, which takes account of the driver’s response time, will further enhance the driving safety. 7.1. Frontal Collision Warning System Frontal collision accidents account for approximately 30% of all accidents in transit bus operations. Current commercial frontal collision warning systems reduce as much as 51% of crashes [ 27], and in general they evaluate the time to collision ( TTC) to determine whether a warning should be issued. The TTC concept was introduced in 1972 by Hayward [ 1]. A TTC value at an instant t is defined as the time that remains until a collision between two vehicles would have occurred if the collision course and speed difference were maintained. TTC can be calculated simply as the ratio of relative distance to the relative speed [ 19], i. e. 23 ( ) ( ) ( ) x t x t TTC t & = ( 13) or more elaborately by the equation, 2 ( ) ( ) ( ) 2 ( ) ( ) ( ) x t x t x t x t x t TTC t & ± & + && = ( 14) under the assumption that the relative acceleration is constant, where x is the relative distance to the preceding vehicle. The reference TTC is the smaller of the two roots [ 2]. Figure 12 illustrates the notion of TTC. If TTC is less than the prescribed threshold, a warning is issued. Figure 12. Time to collision notion illustrated with vehicle trajectories [ 19] In the study, TTC is compared with a prescribed threshold γ 1 to determine whether the vehicle is too close to the preceding vehicle, i. e., if TTC < γ 1 and there is no breaking reaction detected, a warning message shall be issued to the driver. The threshold γ 1 may also be adjusted according to the estimated driver response time tr, for example, r t α γ = γ + 1 1 , where α and 1 γ are two fixed numbers to be determined. The investigation on α and 1 γ is deferred to future research. We set a fixed threshold 4sec 1 γ = in this project, as suggested by Van der Horst [ 32]. 24 7.2. Road/ Lane Departure Warning System Road/ Lane departure warning systems ( RDW) issue alarms when they predict that the vehicle is going to leave the road/ lane without a turning signal. A vision based system is adopted in sensing the vehicle’s lateral position and the road geometry, and the RDW system uses time to lane crossing ( TLC), a metric similar as TTC, to determine the conditions to issue warning messages. TLC is defined as the remaining time until the vehicle’s center of gravity ( CG) crosses either edge of the roadway [ 14], as shown in Figure 13. Figure 13. Time to lane crossing ( TLC) assuming constant steer angle. [ 14] However, the above defined TLC does not take into account of the width of the vehicle itself, and safety is not ensured if a part of the vehicle is outside of lane/ road while the vehicle’s CG is still inside. In this project, we revise the definition and calculate TLC as the remaining time until either side of the vehicle crosses the lane or road. In other words, to ensure safety, the following condition should be satisfied: ≤ ( − ) − α 2 1 l lane vehicle y W W ( 15) where l y is the look ahead lateral position of the vehicle as defined in section 4. Wlane is the look ahead lane width in front of the vehicle, and is obtained from the on board video camera. Wvehicle is the width of the vehicle, and for the test vehicle – Ford Taurus 2002, it equals to 186cm. 25 The last term, α is a small positive number for confidence. We set α = 5cm in this project. As mentioned above, the road geometry and a vehicle path are needed in order to calculate the TLC. The road geometry in front of the vehicle is estimated based on the camera images, and the vehicle path is predicted under the assumption of a constant steering angle and constant vehicle speed in [ 14]. The assumption that the steering angle is fixed over the next a few seconds is not realistic. In this project, we predict the vehicle path based on the lateral dynamics of the vehicle; more specifically, the simplified bicycle model for the vehicle lateral dynamics is adopted, and is coupled with the driver model to yield prediction for the next a few seconds. We refer to section 8 for more information. Once the TLC is determined, it is compared with a prescribed threshold 2 γ . A warning message will be issued to the driver if the predicted TLC is less than or equal to the threshold. The bandwidth of TLC is found to be 2.5Hz in normal driving [ 31], and accordingly we choose the threshold as 0.4 sec 2 γ = . 26 8. On line Data Processing The on line driver model estimation is based on the ARMAX( 3,1,17,1) model we identify in Section 4. However, since the on line data processing has a higher demand on the computing speed, the iterative method used to solve the ARMAX model is no longer appropriate. We adopt the recursive method ( see Appendix B) for on line model estimation. At each time step once a new data point is collected, the recursive method updates the parameter estimate utilizing the new data point and the previous estimate, instead of calculating the new estimate based on the whole data set. For example, θ ˆ ( t) = θ ˆ ( t − 1) + correction term ( 16) The on line data processing has three main tasks: 1. estimate the parameters of the driver model, and obtain the driver’s response time, 2. calculate the time to collision and time to lane crossing, and 3. check if the driver stays in the safe driving mode, otherwise a warning message will be issued. At each time step, once we update the estimate of the driver model, θ ˆ ( t) becomes available, We can then predict the driver’s performance at the next time step, i. e. ˆ ( t 1) ( t) ˆ ( t) δ + = ψ Tθ ( 17) where δ is the angle of the steering wheel, l y is the lateral position of the vehicle, and T l k l k l k b c a y t n y t n y t n n t t n t t t t n ( ), ( 1), , ( ), ( 1), , ( )) ( ) ( ( 1), ( 2), , ( ), − − − − − − − = − − − − − − ε ε ψ δ δ δ Λ Λ Λ ( 18) The driver’s response time tr, is also obtained from the parameter estimate θ ˆ ( t) , as described in Section 5. A sample results of the estimated model parameters and the response time is shown in 27 Figure 14, where the coefficients of the model polynomial A( q  1 ), a1, a2, a3 are shown in the left plot. The x axis is time. The plot also reveals the time variant property of the human driver. 660 670 680 690  1 . 5  1  0. 5 0 0. 5 time Estimate of A( q  1 )  1 0 1  1  0. 5 0 0. 5 1 a 1 a 2 a 3 ploes of A( q 1) Figure 14. Sample results of the online estimation for coefficients of A( q  1 ) and the poles The estimation of δ and l y over the next a few time steps, i. e. δ ˆ ( t + i) and y ˆ ( t i) l + , are all based on the current estimate θ ˆ ( t) since no new measured data is available unless we proceed to the next time step. The estimation of δ and l y will also require knowledge of the vehicle dynamics, which is based on the simplified bicycle model. The bicycle model takes the estimated δ ˆ ( t + 1) from the driver model as its input, and predicts y ˆ ( t + 1) l for the next time step. y ˆ ( t + 1) l is then fed back to the driver model to get δ ˆ ( t + 2) 9 , and the bicycle model predicts y ˆ ( t + 2) l , and so on. The estimation process continues in the closed loop until enough information has been acquired about the TLC. The process is depicted in Figure 15. 9 Notice here, the estimation ˆ ( t 2) ( t 1) ˆ ( t ) T δ + = ψ + θ , and it is different from ˆ ( t + 2) = ( t + 1) ˆ ( t + 1) T δ ψ θ if θ ˆ ( t + 1) were available. 28 y ˆ TLC ? ( 20) or 2 > γ Driver ( ARMAX) Vehicle ( Bicycle)  δ ˆ TLC Yes No θ ˆ ( t) Figure 15. Closed loop prediction and calculation of TLC The TLC is then simply obtained by accumulating the time steps elapsed in the closed loop prediction, and multiplying the sampling time = 0.075 sec s T . Suppose that the number of time steps that have elapsed is m . Then the TLC can be calculated as, TLC = 0.075m ( 19) and, m is the smallest integer that satisfies, + ≥ ( − )− α lane vehicle y t m W W 2 1 ˆ ( ) ( 20) where again, lane W is the look ahead lane width obtained from the video camera, = 186cm vehicle W is the width of the test vehicle, and α = 5cm is the confidence parameter. As shown in Figure 15, we can acquire enough information on the TLC, if ( 1) the TLC is obtained; or ( 2) 2 TLC > γ is satisfied even though it has not been obtained. By allowing condition ( 2), the computation time can be saved, since we now can stop incrementing TLC as soon as it reaches the prescribed threshold 2 γ . The value of the TLC beyond the threshold provides no further information and will not be needed. 29 9. Warning Message Issuing Mechanism Alarms should be generated in a timely manner since they will be useless if the driver cannot make use of them for corrective actions. On the other hand, a system that gives excessive warning may either desensitize the driver, causing future warnings to be ignored or distract the driver. Undesired warnings may also make the driver turn off the system completely. To reduce the nuisance alarms, we should issue at most one alarm at each possible collision or lane departure. If the driver has already taken an appropriate action ( e. g. decelerate, or turn the steering wheel towards the correct direction) right before a warning is issued, the warning should be discarded. We propose such a lane departure warning system and a frontal collision warning system. The block diagram is shown in Figure 16, where the threshold of the driver’s response time is set to 0.5sec, and the thresholds for TTC and TLC are chosen as 4sec 1 γ = and 0.4sec 2 γ = , respectively. response time > 0.5 ? Yes No TLC < γ 2 ? No Yes is turning signal on? No Issue Warning Message No Go to next time step Data process at each time step Data collection TTC < γ 1 ? is brake on? Yes Yes No No Figure 16. Block diagram of the alarm issuing mechanism 30 10. Conclusion This research project, entitled “ Vehicle/ Driver Monitoring for Enhanced Safety of Transit Buses”, is performed as part of the California PATH Program at the University of California at Berkeley. The main contributions of the project are: ( 1) to find a simple model for human drivers which represents the driving pattern with minimum errors, and ( 2) to develop the algorithm which gives warning messages when there is a provision of danger. Based on analysis of data collected from 20 human subjects in actual driving tests, it has been found that human driver can be modeled as a dynamic process with the ARMAX( 3,1,17,1) model. The inputs of the model are the look ahead error and the road curvature, and the output is the angle of the steering wheel. A simple yet effective way to retrieve information about the driver’s response time from the identified model parameters has also been developed. The performance of the driver and vehicle is predicted while they are in closed loop operation. A simplified bicycle model is adopted for the vehicle lateral dynamics. The on line data processing also provides the estimate of the driver’s response time, the time to collision and the time to lane crossing. A warning system, which takes account of the response time of the driver, the TTC and TLC, is designed to determine whether a warning message should be issued to the driver to alert his/ her abnormal driver behavior. The system uses prescribed thresholds. It will improve the effectiveness and reliability of current CWS/ RDW systems, and thus enhance the safety of highway driving. 31 Reference [ 1] S. Bajikar, el al, Evaluation of In Vehicle GPS Based Lane Position Sensing for Preventing Road Departure, IEEE Conference on Intelligent Transportation System, 1997, pp397~ 402 [ 2] P. Barber and N. Clarke, Advanced Collision Warning Systems, IEE Colloquium on Industrial Automation and Control: Applications in the Automotive Industry, 1998 [ 3] J. M. Carson and W. W. Wierwille, Development of a strategy model of the driver in lane keeping, Vehicle System Dynamics, 1978, pp. 233 253. [ 4] L. K. Chen and A. G. Ulsoy, Identification of a driver steering model, and model uncertainty, from driving simulator data, ASME Journal of Dynamic Systems, Measurement, and Control, 2001, pp. 623 629 [ 5] L. K. Chen and A. G. Ulsoy, Identification of a nonlinear driver model via NARMAX modeling, Proceedings of the American Control Conference, Vol. 4, pp. 2533 2537. [ 6] W. Fincham, Data Recorders for Accident Investigation, IEE Colloquium on Monitoring of Driver and Vehicle Performance, 1997 [ 7] FJJ Hermans, R. Conlong, Road prediction for intelligent vehicles using video, IEE Colloquium on Industrial Automation and Control: Applications in the Automative Industry ( Digest No. 1998/ 234). IEE. 1998, pp. 1/ 1 4 [ 8] R. A. Hess and A. Modjtahedzadeh, A control theoretic model of driver steering behavior, IEEE Control Systems Magazine, pp. 3 8. [ 9] P. S. Hingwe, Robustness and Performance Issues in the Lateral Control of Vehicles in Automated Highway Systems, Ph. D. Dissertation, 1997 [ 10] S. Kato, K. Tomita, and S. Tsugawa, Visual navigation along reference lines and collision avoidance for autonomous vehicles, Proceedings of the 1996 IEEE Intelligent Vehicles Symposium. IEEE. 1996, pp. 385 90. New York, NY, USA. [ 11] U. Kiencke, R. Majjad, S. Kramer, Modeling and performance analysis of a hybrid driver model, Control Engineering Practice 7, 1999, 985 991. [ 12] Y. Kuriyagawa el al, A research on analytical method of driver vehicle environment system 32 of construction of intelligent driver support system, Vehicle System Dynamics, 2002, Vol. 37, No. 5, pp. 339 358. [ 13] M. Land and J. Horweed, Which parts of the road guide steering? Nature, Vol. 377, September 1995. [ 14] D. J. LeBlanc et al, CAPC: An Implementation of a Road departure Warning System, Proceedings of Conference on Control Applications, 1996, pp590~ 595 [ 15] J. S. Lim, Two dimensional Signal and Image Processing, Prentice Hall, 1990 [ 16] C F Lin, A. G. Ulsoy, Vehicle Dynamics and External Disturbance Estimation for Vehicle Path Prediction, IEEE Transactions on Control Systems Technology, Vol. 8, No. 3, 2000, pp508~ 518 [ 17] J. Malik, D. Koller, and T. Luong, A machine vision based system for guiding lane change maneuver, California PATH Research Report, UCB ITS PRR 95 34 [ 18] D. T. McRuer and E. Krendel, Mathematical models of human pilot behavior, AGARDograph No. 188, Jan. 1974. [ 19] C. Mertz, S. McNeil and C. Thorpe, Side Collision Warning Systems for Transit Buses, Proceedings of the IEEE Intelligent Vehicle Symposium, 2000, pp344~ 349 [ 20] M. M. Minderhoud, P. H. L. Bovy, Extended time to collision Safety Measurements for Road Traffic Safety Assessment, Accident Analysis and Prevention, Vol 33, pp. 89 97 [ 21] R. Miller and Q. Huang, An Adaptive Peer to Peer Collision Warning System, IEEE Vehicular Technology Conference, 2002, pp317~ 321 [ 22] T. Pilutti and A. G. Ulsoy, Decision Making for Road Departure Warning Systems, Proceedings of the American Control Conference, 1998, pp1838~ 1842 [ 23] T. Pilutti and A. G. Ulsoy, Identification of Driver State for Lane Keeping Tasks, IEEE Transactions on Systems, Man and Cybernetics PART A: Systems and Humans, Vol. 29, No. 5, 1999, pp486~ 502 [ 24] T. Pilutti and A. G. Ulsoy, Identification of driver state for lane keeping tasks: experimental results, Proceedings of the American Control Conference, June 1997, pp. 3370 3374. 33 [ 25] J. W. Senders et al, The attentional demand of automobile driving, Highway Research Record, 1967, 195, 15 33. [ 26] J. G. Wohl, Man machine steering dynamics, Human Factors, Vol. 3, No. 4, pp. 2220228, 1961. [ 27] NHTSA Crash Avoidance Systems Benefits Estimation Report, 1996 [ 28] NHTSA Drowsy Driving and Automobile Crashes Report. [ 29] M. Tomizuka and Y. Takahashi, Deterministic Disturbance Rejection in Linear Identification, Trans. of ASME, Journal of Dynamic Systems, Meas. and Control, Vol. 99, No. 4, pp. 307 310, December 1977 [ 30] J. Ch. Hayward, Near Miss Determination through Use of a Scale of Danger, Report no. TTSC 7115, The Pennsylvania State University, Pennsylvania, 1972. [ 31] C. F. Liu, A. G. Ulsoy, Calculation of the time to lane crossing and analysis of its frequency distribution, Proceedings of ACC, June 1995. [ 32] R. Van der Horst, Time To Collision as a Cue for decision making in braking. Vision in Vehicles III, pp 19 26, 1991. [ 33] B. Wang, M. Abe, Y. Kano, Influence of driver’s reaction time and gain on driver vehicle system performance with rear wheel steering control systems: part of a study on vehicle control suitable for the aged driver, JSAE Review, No. 3, pp. 75 82, 2002. [ 34] K. Lee, H. Peng, Identification of a longitudinal human driving model for adaptive cruise control performance assessment, Proceedings of IMECE, November 2002. [ 35] I. D. Landau, A. Karimi, A Recursive Algorithm for ARMAX Model Identification in Closed Loop, IEEE Transaction on Automatic Control, Vol. 44, No. 4, April 1999. [ 36] I. D. Landau, An Approach for Closed Loop System Identification, Proceedings of the 33 rd Conference on Decision and Control, December 1994. [ 37] Y. Zhu, F. Butoyi, Multivariable and Closed Loop Identification for Model Predictive Control, Technical paper. [ 38] A. Monin, ARMAX Identification Via Hereditary Algorithm, IEEE Transaction on 34 Automatic Control, Vol. 49, No. 2, February 2004. [ 39] L. Ljung, System Identification  Theory For the User, 2nd ed, PTR Prentice Hall, 1999 35 Appendix A. Brief review of ARMAX model The standard ARMAX model is shown in Figure 17, and it is described by, ( ) ( 1) ( ) ( 1) ( 2) ( ) ( ) ( 1) ( 2) ( ) 1 1 2 1 2 n c k k n k b n a e t c e t c e t n b u t n b u t n b u t n n y t a y t a y t a y t n c b a + + − + + − + − − + − − + − − = − − − − − − − Λ Λ Λ ( A. 1) or, ( ) ( ) ( ) ( ) ( ) ( ) 1 1 1 A q y t q k B q u t C q e t − = − n − + − ( A. 2) where, y and u are the output and input of the system, e is the noise and is assumed to have a Gaussian distribution with variance σ 2 . Also, a n , b n , c n and k n are the orders of the ARMAX model, and na a , a , , a 1 2 Λ , nb b , b , , b 0 1 Λ , nc c , , c 1 Λ are the model parameters. G H u + y v e Figure 17. Block diagram of an ARMAX model By using the vector form, we have T na nb nc ( a , a , , a , b , b , , b , c , , c ) 1 2 0 1 1 θ = Λ Λ Λ ( A. 3) T k k k b c a u t n u t n u t n n e t e t n t y t y t y t n ( ), ( 1), , ( ), ( 1), , ( )) ( ) ( ( 1), ( 2), , ( ), − − − − − − − = − − − − − − Λ Λ ψ Λ ( A. 4) Equation ( A. 1) can be rewritten as, y( t) ( t) e( t) = ψ Tθ + ( A. 5) From the above equation, we recognize that the identification of the model parameters is a least squares problem. It should be noted, however, that the regressor ψ ( t) contains noise at previous 36 time steps ( 1), , ( ) c e t − Λ e t − n , which are not available., so that the two terms on the right hand side of ( A. 5) are coupled. Thus, the stand least squares algorithm needs to be modified as discussed in Appendix B. 37 B. Recursive Least Square Method As described previously, the noise terms ( 1), , ( ) c e t − Λ e t − n in the regressor ψ ( t) are unknown. Since there is no way to measure the noise e( t) , we substitute it with the estimated error ε = y − y ˆ , where y ˆ is the estimate of the output. Thus, from equation ( A. 5), ψ Tθ y ˆ ( t) = ( t) ( B. 1) T k k k b c a u t n u t n u t n n t t n t y t y t y t n ( ), ( 1), , ( ), ( 1), , ( )) ( ) ( ( 1), ( 2), , ( ), − − − − − − − = − − − − − − ε ε ψ Λ Λ Λ ( B. 2) In the regular Least Square Method, the parameters are estimated as, ( t t ) t y T ˆ ( ) ( ) 1 ( ) θ ψ ψ ψ − = ( B. 3) which requires a large amount of computation and involves the inverse of ( ) 1 ( ) ( ) − T ψ t ψ t at each time t . The Recursive Least Square Method, on the other hand, uses current knowledge and estimates at time t to obtain a new estimate at next time step t + 1, i. e. θ ˆ ( t + 1) = θ ˆ ( t) + correction term ( B. 4) There are several approaches to get the correction term in ( B. 4), and we adopt the a posteriori version in this project. Define y ˆ ( t) = ( t − 1) ˆ ( t − 1) o ψ Tθ as the a priori output estimate, and ( t) y( t) y ˆ ( t) ε o = − o as the a priori output estimation error. Similarly, y ˆ ( t) ( t 1) ˆ ( t) = ψ − Tθ and ε ( t) = y( t) − y ˆ ( t) are the a posteriori output estimate and a posteriori output estimation error, respectively. Then, the complete algorithm can be summarized as, 38 ( t 1) y( t 1) ( t) ˆ ( t) ε o + = + − ψ Tθ ( B. 5) 1 ( ) ( ) ( ) ( 1) ( 1) t F t t t t T o ψ ψ ε ε + + + = ( B. 6) θ ˆ ( t + 1) = θ ˆ ( t) + F( t) ψ ( t) ε ( t + 1) ( B. 7) 1 ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( 1) ( ) t F t t F t t t F t F t F t T T ψ ψ ψ ψ + + = − ( B. 8) 39 C. Review of closed loop identification methods The basic problem with closed loop data is that it typically has less information than the open loop system – any important purpose of feedback is to make the closed loop system less sensitive to changes in the open loop system. Figure 18 shows a standard feedback closed loop system, where u is the input, y is the output, e is the noise, and r is a reference signal. The goal is to identify the plant model G and the noise model H. Generally, there is no requirement on the knowledge of the regulator K. Also notice that u( t) = r( t) − K( q) y( t) , so r carries no further information about the system, if u is measured. G K r u y v H e  + Figure 18. Feedback closed loop system The plant G is identifiable under the closed loop system if and only if the data is informative, which requires at least one of the following two conditions to hold [ 39], 1. The external signal r is persistently exciting, or 2. [ ( ) ( ) ( ) ( )] 0 2 E * W q y t + * W q u t ≠ y u , i. e. there is no linear time invariant and noise free relationship between y and u such that W ( q) u( t) W ( q) y( t) u y * ≈ * . In our study, the external signal r is always equal to 0. However, since human is a very complicated nonlinear, time varying system, usually there is no linear time invariant and noise free relationship between the steering angle and look ahead lateral. Thus the second condition can be satisfied. The non zero road curvature further assures that the collected data is 40 informative, thus the driver’s model is identifiable. There are three major approaches to solve this closed loop system identification problem, 1. The direct approach – Apply the basic prediction error method in a straightforward manner, i. e. use the output y of the process and the input u in the same way as for open loop operation, ignoring any possible feedback, and not using the reference signal r. 2. The indirect approach – If the regulator K is known, we can identify the closed loop system from reference input r to output y. The plant model G can be retrieved from that information and the knowledge of K, and we don’t even have to estimate the noise model H. Bear in mind that this approach does require true knowledge about K. 3. The joint input output approach – If the reference signal r is known or can be measured, the joint input output approach can be pursued. We consider y and u as outputs of a system driven by r and noise; then the plant G can be recovered from the joint model as Gry/ Gru. Also notice that the reference r can not be 0 for this method to work. Each approach has its own advantages or disadvantages. For example, the indirect method requires the knowledge of the regulator, and the joint input out method assumes that the reference signal r can not be 0 and it can be measured. These conditions restrict them to be applied in this project, since either the vehicle dynamics ( equivalent to the regulator K) is not known exactly, or the reference signal is 0, as can be seen from Figure 4. The direct method, on the other hand, works regardless of the complexity of the regulator, and requires no knowledge about the character of the feedback. There are no special algorithms and software required either, and it has also been proved that consistency and optimal accuracy can be obtained using the direct method if the data is informative ( see next paragraph for explanation) and the model structure contains the true system [ 39]. Furthermore, unstable systems can also be handled, as long as the closed loop system is stable. 41 The only drawback of the direct approach is that the identified plant model may be biased, unless we have a good noise model. In open loop operation we can always obtain unbiased estimates of model parameters of G without requirements on the noise model H. Thus, we can not handle model approximation issues with full control in the feedback case. However, consistency and optimal accuracy is guaranteed, just as in the open loop case, if the true system is contained in the model structure. No knowledge of the feedback or reference signal is required. The prediction error method, applied in a direct fashion, is chosen to solve the off line closed loop driver model identification problem in this project. No knowledge of the vehicle dynamics or the reference signal is required. The method works well under the assumption that the identified noise model can describe the true noise properties, and they give consistent estimates and optimal accuracy. This direct PEM method can be further improved by first building a higher order model G ˆ , with small bias, and then reducing the identified model to lower order with a proper frequency weighting.
Click tabs to swap between content that is broken into logical sections.
Rating  
Title  Vehicle/driver monitoring for enhanced safety of transit buses 
Subject  TE228.A1 P36 no. 200725; BusesAutomatic control.; Driver assistance systems.; BusesSafety measures. 
Description  Performed in cooperation with the California Dept. of Transportation and the Federal Highway Administration.; "November 2007."; Includes bibliographical references (p. 3134).; Harvested from the web on 2/1/08 
Creator  Shi, Mingu. 
Publisher  California PATH Program, Institute of Transportation Studies, University of California at Berkeley 
Contributors  Tomizuka, Masayoshi.; California. Dept. of Transportation.; University of California, Berkeley. Institute of Transportation Studies.; Partners for Advanced Transit and Highways (Calif.) 
Type  Text 
Language  eng 
Relation  Also available online.; http://www.path.berkeley.edu/PATH/Publications/PDF/PRR/2007/PRR200725.pdf 
DateIssued  [2007] 
FormatExtent  41 p. : ill., charts ; 28 cm. 
RelationIs Part Of  California PATH research report, UCBITSPRR200725; PATH research report ; UCBITSPRR200725. 
Transcript  ISSN 1055 1425 November 2007 This work was performed as part of the California PATH Program of the University of California, in cooperation with the State of California Business, Transportation, and Housing Agency, Department of Transportation, and the United States Department of Transportation, Federal Highway Administration. The contents of this report reflect the views of the authors who are responsible for the facts and the accuracy of the data presented herein. The contents do not necessarily reflect the official views or policies of the State of California. This report does not constitute a standard, specification, or regulation. Final Report for Task Order 5400 CALIFORNIA PATH PROGRAM INSTITUTE OF TRANSPORTATION STUDIES UNIVERSITY OF CALIFORNIA, BERKELEY Vehicle/ Driver Monitoring for Enhanced Safety of Transit Buses UCB ITS PRR 2007 25 California PATH Research Report Mingyu Shi, Masayoshi Tomizuka University of California, Berkeley CALIFORNIA PARTNERS FOR ADVANCED TRANSIT AND HIGHWAYS 1 Project Title Vehicle/ Driver Monitoring for Enhanced Safety of Transit Buses By Mingyu Shi Masayoshi Tomizuka Department of Mechanical Engineering University of California at Berkeley Berkeley, CA 94720 Final report for project TO5400 January 2007 2 Contents List of Figures ............................................................................................................................... ................ 3 List of Tables......................................................................................................................... ........................ 3 Abstract ............................................................................................................................... .......................... 4 1. Executive Summary ............................................................................................................................... 5 2. Experimental Settings ............................................................................................................................ 7 3. Driver/ Vehicle Operation in A Closed loop System............................................................................ 10 4. Off line ARMAX Driver Model Identification.................................................................................... 11 4.1. Model Order Selection ............................................................................................................. 13 4.2. Sample results of Off line data analysis................................................................................... 14 5. Driver’s Response Time....................................................................................................................... 17 6. Vehicle Lateral Dynamics .................................................................................................................... 20 7. Collision and Road Departure Warning Systems ................................................................................. 22 7.1. Frontal Collision Warning System ........................................................................................... 22 7.2. Road/ Lane Departure Warning System.................................................................................... 24 8. On line Data Processing..................................................................................................................... . 26 9. Warning Message Issuing Mechanism................................................................................................. 29 10. Conclusion..................................................................................................................... .................. 30 Reference ............................................................................................................................... ..................... 31 Appendix....................................................................................................................... .............................. 35 A. Brief review of ARMAX model................................................................................................... 35 B. Recursive Least Square Method................................................................................................... 37 C. Review of closed loop identification methods ............................................................................. 39 3 List of Figures Figure 1. Test vehicle – 2002 Ford Taurus Sedan ( CA license # 1046993)............................................ 7 Figure 2a. SafeTRAC system 2b. Sample view from SafeTRAC video camera ......................... 8 Figure 3. Test route.......................................................................................................................... ...... 9 Figure 4. Block diagram of the driver/ vehicle closed loop system ...................................................... 10 Figure 5. ARMAX driver model............................................................................................................ 11 Figure 6. Look ahead distance and Look ahead lateral position ........................................................ 12 Figure 7. Histogram of the ARMAX model orders ............................................................................... 14 Figure 8. Sample results from the data analysis on one data subset.................................................... 15 Figure 9. Scatter plot of error and QQ plot f the model ...................................................................... 16 Figure 10. Inverse of reaction time versus time on task plot [ 28]........................................................ 17 Figure 11. Driver’s response time estimation from the estimated driver model, .................................. 18 Figure 12. Time to collision notion illustrated with vehicle trajectories [ 19] ..................................... 23 Figure 13. Time to lane crossing ( TLC) assuming constant steer angle. [ 14] ..................................... 24 Figure 14. Sample results of the online estimation for coefficients of A( q  1 ) and the poles ................. 27 Figure 15. Closed loop prediction and calculation of TLC ................................................................. 28 Figure 16. Block diagram of the alarm issuing mechanism................................................................. 29 Figure 17. Block diagram of an ARMAX model................................................................................... 35 Figure 18. Feedback closed loop system ............................................................................................. 39 List of Tables Table 1. Collected data and the corresponding units ............................................................................. 8 Table 2. Vehicle Parameters................................................................................................................. 21 4 Abstract This report documents the findings of the study “ Vehicle/ Driver Monitoring for Enhanced Safety of Transit Buses” conducted as part of the PATH project TO5400. The goal of study TO5400 is to design and implementation of a reliable vehicle/ driver monitoring system with aim to enhance driving safety of transit buses. The main objectives of the project are: ( 1) to identify a simply model that best describes the driving patterns of human drivers, and ( 2) to develop an algorithm which can generate warning messages when there is a provision of danger. In this study, it is assumed that the driver sets the angle of the steering wheel in response to the lateral deviation of the vehicle from the center line of the road and/ or the road curvature ahead of the vehicle which he/ she infers from the visual perception. It is also assumed that the dynamic relationship between the lateral deviation and the road curvature ( i. e. the inputs to the driver) and the steering angle ( i. e. the output from the driver) may be represented by a linear model with Auto Regressive Moving Average with eXogeneous ( ARMAX) structure. The order of the ARMAX model is determined by off line analysis of experimental data collected from actual driving tests. Once the order of the model is determined, the parameters of the model can be estimated by processing the input and output data in real time during actual driving. The on line data processing and prediction can also estimate three critical parameters for driving safety: the driver’s response time ( tr), the time to collision ( TTC) and the time to lane crossing ( TLC) TLC. A timely warning message scheme, which incorporates all the three parameters, is designed to determine whether an alarm should be issued. Keywords: ARMAX driver model, closed loop system identification, collision warning system, response time, road/ lane departure warning system, time to collision, time to lane crossing. 5 Executive Summary Vehicle safety is one of the most important concerns of transit authorities and automotive manufacturers. Vehicle crashes cause more than 41,000 fatalities per year, as well as 5.3 million injuries, and cost the society more than $ 230 billion per year. A large portion of these accidents is due to drivers’ inattention or inability to properly perform driving tasks. Automotive manufacturers have introduced driver assistant systems such as collision warning systems ( CWS) and road departure warning systems ( RDW) to enhance safety. However, most CWS/ RDW systems cannot directly monitor the driver while the negligence of the driver is the major cause of highway accidents. If a system is capable of monitoring the driver’s state from available data and incorporating it into the driver assistant systems, then it will be possible to alert the driver in a timely and effective manner and greatly increase the safety of highway driving. In this study, we develop an effective and reliable vehicle/ driver monitoring system, where the driver’s state can be retrieved from data collected on line and a warning message can be issued to the driver if his/ her state deviates from the normal state in a significant manner. The driver is modeled as a dynamic process with an ARMAX ( Auto Regressive, Moving Average with eXogenous input) structure, where the inputs of the model are the look ahead error and the road curvature, and the output is the angle of the steering wheel. From off line data analysis, the best combination of the orders of the ARMAX model ( which is applicable to all drivers) is first found. This particular structure is employed in on line data processing, where it is coupled with the vehicle lateral dynamics for closed loop identification and prediction. The driver state ( for example the response time) can be determined from the parameter estimates. A simplified linear bicycle model is utilized to describe the vehicle’s lateral motion, under the assumptions of small steering angle, negligible roll and pitch motion and linear tire model. Key parameters used in the CWS/ RDW systems, such as the time to collision ( TTC) and the 6 time to lane crossing ( TLC), are predicted based on the driver model and vehicle’s lateral dynamics in a closed loop system. A warning system, which takes into account the driver’s response time, the TTC and TLC, is designed to determine whether a warning message should be issued to the driver. The system uses prescribed thresholds. The major contributions of this project include: 1. A simple yet effective method is developed to monitor the driver from the view point of driving skills and their variations due to inattentiveness and/ or drowsiness. 2. The resulting vehicle/ driver monitoring system can send an early warning to the driver to alert his/ her abnormal driving behavior. It can improve the effectiveness and reliability of current CWS/ RDW systems, and enhance the safety of highway traffic. 7 2. Experimental Settings Experiments have been conducted on a passenger car 1 at the Richmond Field Station of University of California at Berkeley. The test vehicle is a 2002 Ford Taurus Sedan ( CA license # 1046993). We show a photo of the test vehicle in Figure 1. Detailed information regarding the test vehicle can be obtained from PATH. SafeTRAC Figure 1. Test vehicle – 2002 Ford Taurus Sedan ( CA license # 1046993) For any collision warning and vehicle monitoring system, it is essential to have a reliable sensing system that senses the environment in front of and around the vehicle. A SafeTRAC 2 video camera is installed under the windshield of the test vehicle, which monitors the road in front of the vehicle. The video system tracks the lane markings, as shown in Figure 2b, and retrieves road geometry information such as the lane width, the relative lateral position of the vehicle with respect to the center of the lane, and the road curvature. Other information is measured by on board sensors. For example, the steering angle is measured by the encoder installed under the steering wheel. A list of collected variables is shown in Table 1. The sampling time is 0.075 second, which corresponds to a sampling frequency of 13Hz. There 1 As suggested by the title of the project, the original plan was to study vehicle/ driver monitoring for transit buses. It turned out, however, that it was not practical to use a bus for experimental data gathering because of an experimental buses and qualified bus drivers were both difficult to obtain. 2 SafeTRAC is an on board electronic warning system that prevents crashes caused by driver drowsiness or distraction. Manufactured and developed by Assistware Technology, SafeTRAC uses a forward looking video camera to track a vehicle’s position in its lane. 8 is also a computer installed on the test vehicle, which is used to record and process collected data. Figure 2a. SafeTRAC system 2b. Sample view from SafeTRAC video camera Data Unit clock time of day HH: MM: SS: sss steering wheel angle deg acceleration x m/ s/ s acceleration y m/ s/ s throttle position % Brake 0= off, 10= on left turn signal 0= off, 10= on right turn signal 0= off, 10= on speed over ground km/ h lateral offset cm lateral velocity cm/ s road curvature 1/ m lane width cm Table 1. Collected data and the corresponding units Driving tests are conducted on a specified segment of Interstate Highway I 80 between Richmond and Fairfield, CA. Figure 3 shows a map of the test route. We choose this segment because: ( 1) there are both straight and curved sections, which assure that the collected data is 9 informative enough ( refer to Appendix C for more detailed information), ( 2) this segment can be easily accessed from Richmond Field Station ( RFS) where the test vehicle is maintained, and ( 3) the traffic on this segment is normally not too congested to properly carry out the experiments. A typical testing route consists of the following: ( 1) the tests start from RFS; ( 2) the drivers go on I 80 from Carson Blvd entrance ( point A on the map); ( 3) the drivers turn back at Red Top Road ( point B on the map); and ( 4) the tests end at RFS. The length of the round trip route is approximately 50 miles. Figure 3. Test route from Yahoo map Twenty experienced drivers participated in the experiment. They consisted of 15 males and 5 females. In the experiment, they were asked to drive the test vehicle in the normal way they would drive everyday. The test for each driver took about one hour, and it was either between 9 and 11am or between 1pm and 4pm, when traffic was relatively low. Also, the experiments were normally taken under clear weather conditions, with only one exception of raining. On board sensors and the SafeTRAC video camera were used to collect data for each test, as shown in Table 1. 10 3. Driver/ Vehicle Operation in A Closed loop System The vehicle lateral dynamics appears in a feedback closed loop system in the sense that the human driver observes how the vehicle goes and takes corresponding control of the vehicle. The closed loop system can be represented by the diagram in Figure 4. Driver Vehicle steering angle lateral position road curvature  Figure 4. Block diagram of the driver/ vehicle closed loop system One of the major tasks of this project is to identify a proper driver model. This is a closed loop identification problem, since it is not possible to open the loop and study the driver. A brief review of closed loop identification techniques is provided in Appendix C. The direct method, as described in the appendix, is adopted in this project. 11 4. Off line ARMAX Driver Model Identification The modeling of human driver’s behavior has attracted great attentions from many researchers recently. In order to model the driving pattern, complex input output models, such as the nonlinear regression models and the neural network and fuzzy systems have been used. It is not our intention to introduce another model for a comprehensive description of driver behaviors. Rather, we are interested in a driver model that will benefit the monitoring and warning system, or more specifically, a model that may represent the driver as a time varying controller of the vehicle. Previous studies show that the ARMAX ( Auto Regressive and Moving Average with eXogenous) model is a promising candidate for human driver models [ 4]. In this project, we follow this direction and test the ARMAX model with real experimental data. A brief review of the model is presented in Appendix A. In this project, we construct a model of the human driving behavior based on its representation as an ARMAX model. The structure is shown in Figure 5. One advantage of this model is that a rich body of knowledge is available in the field of modeling and identification of dynamic systems. Another advantage is that many of its coefficients have clear physical interpretations; for example, the delay time may be estimated based on the poles of the identified model, and the delay time may be interpreted as the driver’s response time. We refer to Section 5 for more details. e δ 1 1 ( ) ( ) C z A z − − + + y l ρ Input ( ) Output ( ) 1 1 − − − A z B z z nk Figure 5. ARMAX driver model 12 The inputs of the ARMAX driver model are the look ahead lateral position of the vehicle l y and the look ahead road curvatureρ ; and the output is the steering angleδ . l y and ρ are measured by the SafeTRAC video system, which captures the road condition in front of the vehicle. A graphic representation is shown in Figure 6. The look ahead distance can be adjusted by changing the angle of the lens when the SafeTRAC camera is installed, and we set it as 20 meters in this study. In other words, the look ahead lateral position is the lateral position of the vehicle with respect to the road condition 20 meters ahead of the vehicle. Similar interpretation holds for the road curvature. The output δ is measured by the encoder installed under the steering wheel. y y l look ahead distance Figure 6. Look ahead distance and Look ahead lateral position The ARMAX model can be expressed as, [ ] ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 1 1 2 1 1 1 C q e t t y t A q t q B q B q n l k − − − − − + = ρ δ ( 1) where, the noise e( t) is assumed to be a white Gaussian process, and ( ) − 1 A q , ( ) 1 1 − B q , ( ) 1 2 − B q and ( ) − 1 C q are the model polynomial operators, with orders a b b n , n , n and c n respectively, i. e., a a n n A q a q a q a q − = + − + − 2 + Λ + − 2 1 1 1 ( ) 1 ( 2) b b n n B q b q b q b q − = − + − + + − 1 2 12 1 11 1 1 ( ) Λ ( 3) 13 b b n B q b q b q b n q − = − + − + + − 2 2 22 1 21 1 2 ( ) Λ ( 4) c c n n C q c q c q c q − = + − + − 2 + Λ + − 2 1 1 1 ( ) 1 ( 5) 4.1. Model Order Selection First of all, we investigate what order set ( a b c k n , n , n , n ) the ARMAX model should take to yield the best representation of human driver. Since data is collected from 20 different drivers, and parameters may vary from time to time, the data are divided into 966 subsets, with each one contains data of 30 seconds long. The selection of orders is carried out using the following approach, 1) For each subset of the data, and for each combination of orders of the ARMAX model ( a b c k n , n , n , n ), we estimate the model and calculate the final prediction error ( FPE) 3 with the prediction error method ( PEM) 4 . The choices of the model orders include all possible combinations of ∈[ 1,5] a n , ∈[ 1,10] b n , ∈[ 1,20] c n and ∈[ 1,10] k n . 2) The lowest FPE value is found for each subset of data, and the corresponding orders ( a b c k n , n , n , n ) are detected. The FPE can be used as a model selection criterion. The ARMAX model with orders corresponding to the smallest FPE, is selected as the optimal model for this subset. 3) For the 966 subsets, there are 966 combinations of ( a b c k n , n , n , n ) with the lowest FPEs. A histogram is plotted for each component to determine the overall orders. 3 FPE is a model selection criterion, defined as d N d N FPE V − + = 1 1 , where V is the loss function calculated from the model estimation, d is the number of model parameters, and N is the number of available data. FPE compensates the effect of increasing order, i. e. the bigger the order, the smaller the loss function. Other model selection criteria, which compensate such order effects, include Akaike’s Information Criterion ( AIC), Bayesian Information Criterion ( BIC), Mallow’s Cp, and Minimum Description Length ( MDL), etc. 4 The prediction error method estimates model parameters θ by minimizing the loss function = Σ = N t t N VN 1 ( , ) 1 ( ) 2 θ ε θ , where ε is as defined in Appendix A and N is the time duration, and N = 400 for each data subset in this project. 14 MATLAB 6.5 System Identification Toolkit was adopted to perform task ( 1). Given the data and the prescribed order combination ( a b c k n , n , n , n ), the armax function from the toolkit can estimate the parameters of the model. The FPE of the model can also be calculated. The armax function uses an iterative search algorithm to minimize the prediction error. The iterations are terminated when the expected improvement between adjacent iterations is within a predetermined tolerance, or when a fixed number of iterations has been reached. The iterative algorithm is robust; however, it may take a long time to converge. On a Pentium M 1.6GHz computer, it takes around 5 minutes to obtain results for one subset of data. Thus the toolkit can only be used for the off line data analysis. We resort to a recursive algorithm for our on line data processing. We refer to Section 8 for more details. The histograms in step 3) are shown in Figure 7. We observe that = 3 a n , = 1 b n , = 17 c n 5 and = 1 k n occur most frequently, which means for all the drivers and any 30 seconds time intervals, ARMAX( 3,1,17,1) model can best describe human driver’s behavior. On line parameter identification will be based on this selected model. 1 2 3 4 5 0 1 00 2 00 3 00 4 00 5 00 600 1 2 3 4 5 0 5 0 1 00 1 5 0 2 00 2 5 0 3 00 1 2 1 3 1 4 1 5 1 6 1 7 1 8 1 9 2 0 0 5 0 1 00 1 5 0 2 00 1 2 3 4 5 6 7 8 9 1 0 0 5 0 1 00 1 5 0 2 00 nb= 1 nc= 17 nk= 1 na= 3 Figure 7. Histogram of the ARMAX model orders 4.2. Sample results of Off line data analysis The model estimation results from one subset of data are shown in Figure 8 and Figure 9. Each 5 From the histogram of c n , we see that = 16 c n is almost as frequent as = 17 c n . However, since the noise itself is unknown exactly, we defer the investigation of the effect that small difference in choice of c n will cause to a later study. 15 subset contains observations from a 30 second period for one driver. The estimation is based on the above selected ARMAX( 3, 1, 17, 1) model. Figure 8 shows the inputs of the model – the look ahead lateral position and the road curvature, in the upper right and lower left plots, respectively. It also shows the results from the data analysis based on the model, i. e. the estimated output and error. 6 60 670 6 80 690  2 0 2 4 6 S teer ing Ang le 6 60 670 6 80 690  4 0  2 0 0 2 0 4 0 Lateral po sitio n 6 60 670 6 80 690  1  0. 5 0 0. 5 1 1 . 5 x 1 0  3 Road Curvature time 6 60 670 6 80 690  2  1 0 1 2 E rror time Figure 8. Sample results from the data analysis on one data subset The output of the model – the steering angle – is shown in the upper left plot of Figure 8. The blue line is the measured steering angle, and the red line is the estimated output from the model. There is only small discrepancy between the measured and estimated outputs. The R 2 value 6 of the fitted model is 92.7%, which indicates that the selected model can explain a great majority of the variations and thus satisfactory fitting. 6 R 2 value is the fraction of the variance in the data that is explained by the model. It is also called as the coefficient of determination. It indicates how well the model fits the data, e. g. an R 2 close to 100% indicates that we have accounted for almost all of the variability with the variables specified in the model.. 16  2 0 2 4 6  2 0 2 4 6 estimated steering angle steering angle 660 670 680 690  1 . 5  1  0. 5 0 0. 5 1 1 . 5 Error time Figure 9. Scatter plot of error and QQ plot f the model Comparison of the measured data versus estimated data is also shown in the QQ plot 7 in Figure 9. The measured steering angle is plotted on the horizontal axis, and the estimated steering angle from the model is plotted on the vertical axis. The data points are all close to the reference line – the straight line with unit slope and zero intercept. It indicates that the underlying model can sufficiently explain the data. Furthermore, there is no significant trend in the scatter plot of the error. Thus the model assumption that it has an ARMAX( 3,1,17,1) structure is validated. 7 The quantile quantile ( QQ plot) is a plot of the quantiles of the first data set against the quantiles of the second data set. It is a graphical technique for determining if two data sets come from populations with a common distribution. A 45 degree reference line is also plotted. If the two sets come from a population with the same distribution, the points should fall approximately along this reference line. The greater the departure from this reference line, the greater the evidence for the conclusion that the two data sets have come from populations with different distribution. 17 5. Driver’s Response Time The response ( reaction) time is defined as the difference between the time the driver observes a lateral deviation and the time a corrective action is taken. It consists of basically components processing time and transmission time ( neuromuscular lag) [ 3]. Processing time is the time needed to decide what corrective action should be taken, and transmission time is the time necessary to initiate that action after the decision has been made. Various studies, in many different science fields, have been conducted to investigate the human response time. It has been shown that age, drunkenness, and drowsiness can affect human’s driving performance, especially the response time. For example, about 60% of young drivers ( 23 years old or less) exhibit the control delay on straight lanes less than 0.4 sec while less than 20% of old drivers ( 75 years old or more) exhibit the same statistics [ 12]. Figure 10 shows qualitatively the effects of another factor: drowsiness on the reaction time [ 28]. Figure 10. Inverse of reaction time versus time on task plot [ 28] In this project, we propose a new method to retrieve information about the driver’s response time. The method is based on the identified model and its corresponding parameters. First, we note that the 3 rd order ARMAX model has 3 poles, i. e. ( ) ( 1)( 2 ) 2 2 n n A s = τ s + s + ξω s + ω ( 6) The first term in equation ( 6) usually corresponds to the delay in the system, and we can interpret it as the driver’s response time, i. e. = τ r t . 18 Figure 11 presents an example of how to determine the driver’s response time. First, since the model is estimated in discrete time, we have to transform it into the continuous time domain in order to find the response time. A relationship between the poles for discrete system and its corresponding continuous system, using the zero order hold equivalent transformation, is given by, exp( ) d c s p = p T or c d s p = ln( p ) / T ( 7) where pc is the continuous time pole, pd is the discrete time pole, and Ts is the sampling time. In this project we set Ts as 0.075 second. Once the continuous poles are determined, we choose the one that corresponds to the pole in the first term on the right side of equation ( 6) 8 , i. e. = − 1 τ c p . The response time is then computed as the negative inverse of that pole. For example, the red crosses in Figure 11 are the results from one subset of data, and the pole corresponding to the delay time is chosen as = − 6.7 c p , which leads to the drivers response time at this instant = τ ≈ 0.15 r t seconds. Discrete Poles Continuous Poles  1  0 . 5 0 0 . 5 1  1  0 . 5 0 0 . 5 1  1 0  5 0  3  2  1 0 1 2 3 τ 1 − p d = exp( p c T s ) Figure 11. Driver’s response time estimation from the estimated driver model, Red and Green points are the results from two different data subset. 8 In the case that the system has three real poles, the one closest to the negative real axis is chosen as the desired pole. This is the worst case of the possible delay time. In designing the warning system, if the worst case – maximum response time – stays in the range of normal driving, other possible delays are also in the range. Otherwise, there is a possibility that the delay time is long, and a warning shall be given. 19 The driver’s response time tr indicates how fast the driver can react if emergency occurs during driving. The designed warning system takes this information into account to determine whether to issue a warning message or not. In section 9, we set the threshold of the driver’s response time to 0.5 second. Furthermore, the thresholds of the time to collision and the time to lane crossing are also related to the driver’s response time. With a more accurate estimation of the driver’s response time, the warning system may perform better by issuing necessary warnings and reducing the false alarm rate. Further discussions on issuance of warning messages are in Sections 7 and 9. 20 6. Vehicle Lateral Dynamics The vehicle and driver interact with each other in a close loop form, as shown in Figure 4. Thus a dynamic model of the vehicle is necessary for the on line data prediction when the measured data is not available. This research project focuses on the lateral movement of the vehicle and its lane crossing/ departure events. For this purpose, a simplified bicycle model for the vehicle lateral dynamic is adopted. The bicycle model assumes a small steering angle, negligible roll and pitch motion and a linear tire model [ 9]. It can be described as a fourth order system in the state space as, [ ] = + ρ δ 1 2 x& Ax B B ( 8) ( ) T CG CG r r x = y y& ε ε& ( 9) where x is the state variable, δ is the front wheel steering angle, ρ is the road curvature ( disturbance), CG y is the lateral deviation at the vehicle CG ( Center of Gravity), and r ε is the yaw angle of the vehicle sprung mass relative to the road reference frame. The system matrices are − − + = − − − + + + − + + z f s r s z f r z x f r x s s x I l C d l l C d l I l C l C I v l C l C v d l d l v A ( ) ( ) ( ) ( ) 1 2 1 2 1 2 1 1 2 2 1 2 1 1 2 2 0 0 0 0 1 0 0 1 0 0 α α α α α α φ φ φ φ φ φ ( 10) ( ) T I l C z f B α φ 1 0 0 1 1 = ( 11) T I x l C l C B l l v z f r = − − − + 2 2 2 2 1 1 0 0 2 2 2 1 φ φ α α ( 12) where 2 1 m C f α φ = and m C r α φ 2 2 = . The physical meanings and values of the symbols used in the above equations are listed in Table 2. 21 Symbol Physical Meaning Value m mass of the vehicle 1485 kg L relative longitudinal distance between vehicles 5 m s d distance of rear bumper to CG 2.1 m z I yaw moment of inertia 2872 kg/ m2 f Cα front wheel cornering stiffness 4200 N/ rad r Cα rear wheel cornering stiffness 4200 N/ rad 1 l distance between front wheel and the CG 1.1 m 2 l distance between rear wheel and the CG 1.58 m Table 2. Vehicle Parameters 22 7. Collision and Road Departure Warning Systems There are several types of collision warning systems on the market nowadays, which include frontal collision, intersection collision and road departure warning systems. A frontal collision warning system evaluates the time to collision to determine whether a warning should be issued; an intersection or side collision warning system targets at alleviating the hazards in lane changing, and a road departure waning system issues warnings whenever the vehicle tends to depart from the road without a turning signal. In addition to monitor the vehicle’s lateral position, a warning may be issued by observing the driver’s inappropriate states ( e. g. the driver’s response time). For example, a drowsy driver needs longer time to react to any emergency situation. In this case, if the driver gets warning messages right before the vehicle is too close to the lane, he/ she may not have enough time to react. Also, a drowsy driver may stop reacting to the lateral deviation of the vehicle, which will result in a large prediction error in the identification algorithm. In this case, a timely warning shall be issued before the vehicle gets too close to the lane boundary. A warning system, which takes account of the driver’s response time, will further enhance the driving safety. 7.1. Frontal Collision Warning System Frontal collision accidents account for approximately 30% of all accidents in transit bus operations. Current commercial frontal collision warning systems reduce as much as 51% of crashes [ 27], and in general they evaluate the time to collision ( TTC) to determine whether a warning should be issued. The TTC concept was introduced in 1972 by Hayward [ 1]. A TTC value at an instant t is defined as the time that remains until a collision between two vehicles would have occurred if the collision course and speed difference were maintained. TTC can be calculated simply as the ratio of relative distance to the relative speed [ 19], i. e. 23 ( ) ( ) ( ) x t x t TTC t & = ( 13) or more elaborately by the equation, 2 ( ) ( ) ( ) 2 ( ) ( ) ( ) x t x t x t x t x t TTC t & ± & + && = ( 14) under the assumption that the relative acceleration is constant, where x is the relative distance to the preceding vehicle. The reference TTC is the smaller of the two roots [ 2]. Figure 12 illustrates the notion of TTC. If TTC is less than the prescribed threshold, a warning is issued. Figure 12. Time to collision notion illustrated with vehicle trajectories [ 19] In the study, TTC is compared with a prescribed threshold γ 1 to determine whether the vehicle is too close to the preceding vehicle, i. e., if TTC < γ 1 and there is no breaking reaction detected, a warning message shall be issued to the driver. The threshold γ 1 may also be adjusted according to the estimated driver response time tr, for example, r t α γ = γ + 1 1 , where α and 1 γ are two fixed numbers to be determined. The investigation on α and 1 γ is deferred to future research. We set a fixed threshold 4sec 1 γ = in this project, as suggested by Van der Horst [ 32]. 24 7.2. Road/ Lane Departure Warning System Road/ Lane departure warning systems ( RDW) issue alarms when they predict that the vehicle is going to leave the road/ lane without a turning signal. A vision based system is adopted in sensing the vehicle’s lateral position and the road geometry, and the RDW system uses time to lane crossing ( TLC), a metric similar as TTC, to determine the conditions to issue warning messages. TLC is defined as the remaining time until the vehicle’s center of gravity ( CG) crosses either edge of the roadway [ 14], as shown in Figure 13. Figure 13. Time to lane crossing ( TLC) assuming constant steer angle. [ 14] However, the above defined TLC does not take into account of the width of the vehicle itself, and safety is not ensured if a part of the vehicle is outside of lane/ road while the vehicle’s CG is still inside. In this project, we revise the definition and calculate TLC as the remaining time until either side of the vehicle crosses the lane or road. In other words, to ensure safety, the following condition should be satisfied: ≤ ( − ) − α 2 1 l lane vehicle y W W ( 15) where l y is the look ahead lateral position of the vehicle as defined in section 4. Wlane is the look ahead lane width in front of the vehicle, and is obtained from the on board video camera. Wvehicle is the width of the vehicle, and for the test vehicle – Ford Taurus 2002, it equals to 186cm. 25 The last term, α is a small positive number for confidence. We set α = 5cm in this project. As mentioned above, the road geometry and a vehicle path are needed in order to calculate the TLC. The road geometry in front of the vehicle is estimated based on the camera images, and the vehicle path is predicted under the assumption of a constant steering angle and constant vehicle speed in [ 14]. The assumption that the steering angle is fixed over the next a few seconds is not realistic. In this project, we predict the vehicle path based on the lateral dynamics of the vehicle; more specifically, the simplified bicycle model for the vehicle lateral dynamics is adopted, and is coupled with the driver model to yield prediction for the next a few seconds. We refer to section 8 for more information. Once the TLC is determined, it is compared with a prescribed threshold 2 γ . A warning message will be issued to the driver if the predicted TLC is less than or equal to the threshold. The bandwidth of TLC is found to be 2.5Hz in normal driving [ 31], and accordingly we choose the threshold as 0.4 sec 2 γ = . 26 8. On line Data Processing The on line driver model estimation is based on the ARMAX( 3,1,17,1) model we identify in Section 4. However, since the on line data processing has a higher demand on the computing speed, the iterative method used to solve the ARMAX model is no longer appropriate. We adopt the recursive method ( see Appendix B) for on line model estimation. At each time step once a new data point is collected, the recursive method updates the parameter estimate utilizing the new data point and the previous estimate, instead of calculating the new estimate based on the whole data set. For example, θ ˆ ( t) = θ ˆ ( t − 1) + correction term ( 16) The on line data processing has three main tasks: 1. estimate the parameters of the driver model, and obtain the driver’s response time, 2. calculate the time to collision and time to lane crossing, and 3. check if the driver stays in the safe driving mode, otherwise a warning message will be issued. At each time step, once we update the estimate of the driver model, θ ˆ ( t) becomes available, We can then predict the driver’s performance at the next time step, i. e. ˆ ( t 1) ( t) ˆ ( t) δ + = ψ Tθ ( 17) where δ is the angle of the steering wheel, l y is the lateral position of the vehicle, and T l k l k l k b c a y t n y t n y t n n t t n t t t t n ( ), ( 1), , ( ), ( 1), , ( )) ( ) ( ( 1), ( 2), , ( ), − − − − − − − = − − − − − − ε ε ψ δ δ δ Λ Λ Λ ( 18) The driver’s response time tr, is also obtained from the parameter estimate θ ˆ ( t) , as described in Section 5. A sample results of the estimated model parameters and the response time is shown in 27 Figure 14, where the coefficients of the model polynomial A( q  1 ), a1, a2, a3 are shown in the left plot. The x axis is time. The plot also reveals the time variant property of the human driver. 660 670 680 690  1 . 5  1  0. 5 0 0. 5 time Estimate of A( q  1 )  1 0 1  1  0. 5 0 0. 5 1 a 1 a 2 a 3 ploes of A( q 1) Figure 14. Sample results of the online estimation for coefficients of A( q  1 ) and the poles The estimation of δ and l y over the next a few time steps, i. e. δ ˆ ( t + i) and y ˆ ( t i) l + , are all based on the current estimate θ ˆ ( t) since no new measured data is available unless we proceed to the next time step. The estimation of δ and l y will also require knowledge of the vehicle dynamics, which is based on the simplified bicycle model. The bicycle model takes the estimated δ ˆ ( t + 1) from the driver model as its input, and predicts y ˆ ( t + 1) l for the next time step. y ˆ ( t + 1) l is then fed back to the driver model to get δ ˆ ( t + 2) 9 , and the bicycle model predicts y ˆ ( t + 2) l , and so on. The estimation process continues in the closed loop until enough information has been acquired about the TLC. The process is depicted in Figure 15. 9 Notice here, the estimation ˆ ( t 2) ( t 1) ˆ ( t ) T δ + = ψ + θ , and it is different from ˆ ( t + 2) = ( t + 1) ˆ ( t + 1) T δ ψ θ if θ ˆ ( t + 1) were available. 28 y ˆ TLC ? ( 20) or 2 > γ Driver ( ARMAX) Vehicle ( Bicycle)  δ ˆ TLC Yes No θ ˆ ( t) Figure 15. Closed loop prediction and calculation of TLC The TLC is then simply obtained by accumulating the time steps elapsed in the closed loop prediction, and multiplying the sampling time = 0.075 sec s T . Suppose that the number of time steps that have elapsed is m . Then the TLC can be calculated as, TLC = 0.075m ( 19) and, m is the smallest integer that satisfies, + ≥ ( − )− α lane vehicle y t m W W 2 1 ˆ ( ) ( 20) where again, lane W is the look ahead lane width obtained from the video camera, = 186cm vehicle W is the width of the test vehicle, and α = 5cm is the confidence parameter. As shown in Figure 15, we can acquire enough information on the TLC, if ( 1) the TLC is obtained; or ( 2) 2 TLC > γ is satisfied even though it has not been obtained. By allowing condition ( 2), the computation time can be saved, since we now can stop incrementing TLC as soon as it reaches the prescribed threshold 2 γ . The value of the TLC beyond the threshold provides no further information and will not be needed. 29 9. Warning Message Issuing Mechanism Alarms should be generated in a timely manner since they will be useless if the driver cannot make use of them for corrective actions. On the other hand, a system that gives excessive warning may either desensitize the driver, causing future warnings to be ignored or distract the driver. Undesired warnings may also make the driver turn off the system completely. To reduce the nuisance alarms, we should issue at most one alarm at each possible collision or lane departure. If the driver has already taken an appropriate action ( e. g. decelerate, or turn the steering wheel towards the correct direction) right before a warning is issued, the warning should be discarded. We propose such a lane departure warning system and a frontal collision warning system. The block diagram is shown in Figure 16, where the threshold of the driver’s response time is set to 0.5sec, and the thresholds for TTC and TLC are chosen as 4sec 1 γ = and 0.4sec 2 γ = , respectively. response time > 0.5 ? Yes No TLC < γ 2 ? No Yes is turning signal on? No Issue Warning Message No Go to next time step Data process at each time step Data collection TTC < γ 1 ? is brake on? Yes Yes No No Figure 16. Block diagram of the alarm issuing mechanism 30 10. Conclusion This research project, entitled “ Vehicle/ Driver Monitoring for Enhanced Safety of Transit Buses”, is performed as part of the California PATH Program at the University of California at Berkeley. The main contributions of the project are: ( 1) to find a simple model for human drivers which represents the driving pattern with minimum errors, and ( 2) to develop the algorithm which gives warning messages when there is a provision of danger. Based on analysis of data collected from 20 human subjects in actual driving tests, it has been found that human driver can be modeled as a dynamic process with the ARMAX( 3,1,17,1) model. The inputs of the model are the look ahead error and the road curvature, and the output is the angle of the steering wheel. A simple yet effective way to retrieve information about the driver’s response time from the identified model parameters has also been developed. The performance of the driver and vehicle is predicted while they are in closed loop operation. A simplified bicycle model is adopted for the vehicle lateral dynamics. The on line data processing also provides the estimate of the driver’s response time, the time to collision and the time to lane crossing. A warning system, which takes account of the response time of the driver, the TTC and TLC, is designed to determine whether a warning message should be issued to the driver to alert his/ her abnormal driver behavior. The system uses prescribed thresholds. It will improve the effectiveness and reliability of current CWS/ RDW systems, and thus enhance the safety of highway driving. 31 Reference [ 1] S. Bajikar, el al, Evaluation of In Vehicle GPS Based Lane Position Sensing for Preventing Road Departure, IEEE Conference on Intelligent Transportation System, 1997, pp397~ 402 [ 2] P. Barber and N. Clarke, Advanced Collision Warning Systems, IEE Colloquium on Industrial Automation and Control: Applications in the Automotive Industry, 1998 [ 3] J. M. Carson and W. W. Wierwille, Development of a strategy model of the driver in lane keeping, Vehicle System Dynamics, 1978, pp. 233 253. [ 4] L. K. Chen and A. G. Ulsoy, Identification of a driver steering model, and model uncertainty, from driving simulator data, ASME Journal of Dynamic Systems, Measurement, and Control, 2001, pp. 623 629 [ 5] L. K. Chen and A. G. Ulsoy, Identification of a nonlinear driver model via NARMAX modeling, Proceedings of the American Control Conference, Vol. 4, pp. 2533 2537. [ 6] W. Fincham, Data Recorders for Accident Investigation, IEE Colloquium on Monitoring of Driver and Vehicle Performance, 1997 [ 7] FJJ Hermans, R. Conlong, Road prediction for intelligent vehicles using video, IEE Colloquium on Industrial Automation and Control: Applications in the Automative Industry ( Digest No. 1998/ 234). IEE. 1998, pp. 1/ 1 4 [ 8] R. A. Hess and A. Modjtahedzadeh, A control theoretic model of driver steering behavior, IEEE Control Systems Magazine, pp. 3 8. [ 9] P. S. Hingwe, Robustness and Performance Issues in the Lateral Control of Vehicles in Automated Highway Systems, Ph. D. Dissertation, 1997 [ 10] S. Kato, K. Tomita, and S. Tsugawa, Visual navigation along reference lines and collision avoidance for autonomous vehicles, Proceedings of the 1996 IEEE Intelligent Vehicles Symposium. IEEE. 1996, pp. 385 90. New York, NY, USA. [ 11] U. Kiencke, R. Majjad, S. Kramer, Modeling and performance analysis of a hybrid driver model, Control Engineering Practice 7, 1999, 985 991. [ 12] Y. Kuriyagawa el al, A research on analytical method of driver vehicle environment system 32 of construction of intelligent driver support system, Vehicle System Dynamics, 2002, Vol. 37, No. 5, pp. 339 358. [ 13] M. Land and J. Horweed, Which parts of the road guide steering? Nature, Vol. 377, September 1995. [ 14] D. J. LeBlanc et al, CAPC: An Implementation of a Road departure Warning System, Proceedings of Conference on Control Applications, 1996, pp590~ 595 [ 15] J. S. Lim, Two dimensional Signal and Image Processing, Prentice Hall, 1990 [ 16] C F Lin, A. G. Ulsoy, Vehicle Dynamics and External Disturbance Estimation for Vehicle Path Prediction, IEEE Transactions on Control Systems Technology, Vol. 8, No. 3, 2000, pp508~ 518 [ 17] J. Malik, D. Koller, and T. Luong, A machine vision based system for guiding lane change maneuver, California PATH Research Report, UCB ITS PRR 95 34 [ 18] D. T. McRuer and E. Krendel, Mathematical models of human pilot behavior, AGARDograph No. 188, Jan. 1974. [ 19] C. Mertz, S. McNeil and C. Thorpe, Side Collision Warning Systems for Transit Buses, Proceedings of the IEEE Intelligent Vehicle Symposium, 2000, pp344~ 349 [ 20] M. M. Minderhoud, P. H. L. Bovy, Extended time to collision Safety Measurements for Road Traffic Safety Assessment, Accident Analysis and Prevention, Vol 33, pp. 89 97 [ 21] R. Miller and Q. Huang, An Adaptive Peer to Peer Collision Warning System, IEEE Vehicular Technology Conference, 2002, pp317~ 321 [ 22] T. Pilutti and A. G. Ulsoy, Decision Making for Road Departure Warning Systems, Proceedings of the American Control Conference, 1998, pp1838~ 1842 [ 23] T. Pilutti and A. G. Ulsoy, Identification of Driver State for Lane Keeping Tasks, IEEE Transactions on Systems, Man and Cybernetics PART A: Systems and Humans, Vol. 29, No. 5, 1999, pp486~ 502 [ 24] T. Pilutti and A. G. Ulsoy, Identification of driver state for lane keeping tasks: experimental results, Proceedings of the American Control Conference, June 1997, pp. 3370 3374. 33 [ 25] J. W. Senders et al, The attentional demand of automobile driving, Highway Research Record, 1967, 195, 15 33. [ 26] J. G. Wohl, Man machine steering dynamics, Human Factors, Vol. 3, No. 4, pp. 2220228, 1961. [ 27] NHTSA Crash Avoidance Systems Benefits Estimation Report, 1996 [ 28] NHTSA Drowsy Driving and Automobile Crashes Report. [ 29] M. Tomizuka and Y. Takahashi, Deterministic Disturbance Rejection in Linear Identification, Trans. of ASME, Journal of Dynamic Systems, Meas. and Control, Vol. 99, No. 4, pp. 307 310, December 1977 [ 30] J. Ch. Hayward, Near Miss Determination through Use of a Scale of Danger, Report no. TTSC 7115, The Pennsylvania State University, Pennsylvania, 1972. [ 31] C. F. Liu, A. G. Ulsoy, Calculation of the time to lane crossing and analysis of its frequency distribution, Proceedings of ACC, June 1995. [ 32] R. Van der Horst, Time To Collision as a Cue for decision making in braking. Vision in Vehicles III, pp 19 26, 1991. [ 33] B. Wang, M. Abe, Y. Kano, Influence of driver’s reaction time and gain on driver vehicle system performance with rear wheel steering control systems: part of a study on vehicle control suitable for the aged driver, JSAE Review, No. 3, pp. 75 82, 2002. [ 34] K. Lee, H. Peng, Identification of a longitudinal human driving model for adaptive cruise control performance assessment, Proceedings of IMECE, November 2002. [ 35] I. D. Landau, A. Karimi, A Recursive Algorithm for ARMAX Model Identification in Closed Loop, IEEE Transaction on Automatic Control, Vol. 44, No. 4, April 1999. [ 36] I. D. Landau, An Approach for Closed Loop System Identification, Proceedings of the 33 rd Conference on Decision and Control, December 1994. [ 37] Y. Zhu, F. Butoyi, Multivariable and Closed Loop Identification for Model Predictive Control, Technical paper. [ 38] A. Monin, ARMAX Identification Via Hereditary Algorithm, IEEE Transaction on 34 Automatic Control, Vol. 49, No. 2, February 2004. [ 39] L. Ljung, System Identification  Theory For the User, 2nd ed, PTR Prentice Hall, 1999 35 Appendix A. Brief review of ARMAX model The standard ARMAX model is shown in Figure 17, and it is described by, ( ) ( 1) ( ) ( 1) ( 2) ( ) ( ) ( 1) ( 2) ( ) 1 1 2 1 2 n c k k n k b n a e t c e t c e t n b u t n b u t n b u t n n y t a y t a y t a y t n c b a + + − + + − + − − + − − + − − = − − − − − − − Λ Λ Λ ( A. 1) or, ( ) ( ) ( ) ( ) ( ) ( ) 1 1 1 A q y t q k B q u t C q e t − = − n − + − ( A. 2) where, y and u are the output and input of the system, e is the noise and is assumed to have a Gaussian distribution with variance σ 2 . Also, a n , b n , c n and k n are the orders of the ARMAX model, and na a , a , , a 1 2 Λ , nb b , b , , b 0 1 Λ , nc c , , c 1 Λ are the model parameters. G H u + y v e Figure 17. Block diagram of an ARMAX model By using the vector form, we have T na nb nc ( a , a , , a , b , b , , b , c , , c ) 1 2 0 1 1 θ = Λ Λ Λ ( A. 3) T k k k b c a u t n u t n u t n n e t e t n t y t y t y t n ( ), ( 1), , ( ), ( 1), , ( )) ( ) ( ( 1), ( 2), , ( ), − − − − − − − = − − − − − − Λ Λ ψ Λ ( A. 4) Equation ( A. 1) can be rewritten as, y( t) ( t) e( t) = ψ Tθ + ( A. 5) From the above equation, we recognize that the identification of the model parameters is a least squares problem. It should be noted, however, that the regressor ψ ( t) contains noise at previous 36 time steps ( 1), , ( ) c e t − Λ e t − n , which are not available., so that the two terms on the right hand side of ( A. 5) are coupled. Thus, the stand least squares algorithm needs to be modified as discussed in Appendix B. 37 B. Recursive Least Square Method As described previously, the noise terms ( 1), , ( ) c e t − Λ e t − n in the regressor ψ ( t) are unknown. Since there is no way to measure the noise e( t) , we substitute it with the estimated error ε = y − y ˆ , where y ˆ is the estimate of the output. Thus, from equation ( A. 5), ψ Tθ y ˆ ( t) = ( t) ( B. 1) T k k k b c a u t n u t n u t n n t t n t y t y t y t n ( ), ( 1), , ( ), ( 1), , ( )) ( ) ( ( 1), ( 2), , ( ), − − − − − − − = − − − − − − ε ε ψ Λ Λ Λ ( B. 2) In the regular Least Square Method, the parameters are estimated as, ( t t ) t y T ˆ ( ) ( ) 1 ( ) θ ψ ψ ψ − = ( B. 3) which requires a large amount of computation and involves the inverse of ( ) 1 ( ) ( ) − T ψ t ψ t at each time t . The Recursive Least Square Method, on the other hand, uses current knowledge and estimates at time t to obtain a new estimate at next time step t + 1, i. e. θ ˆ ( t + 1) = θ ˆ ( t) + correction term ( B. 4) There are several approaches to get the correction term in ( B. 4), and we adopt the a posteriori version in this project. Define y ˆ ( t) = ( t − 1) ˆ ( t − 1) o ψ Tθ as the a priori output estimate, and ( t) y( t) y ˆ ( t) ε o = − o as the a priori output estimation error. Similarly, y ˆ ( t) ( t 1) ˆ ( t) = ψ − Tθ and ε ( t) = y( t) − y ˆ ( t) are the a posteriori output estimate and a posteriori output estimation error, respectively. Then, the complete algorithm can be summarized as, 38 ( t 1) y( t 1) ( t) ˆ ( t) ε o + = + − ψ Tθ ( B. 5) 1 ( ) ( ) ( ) ( 1) ( 1) t F t t t t T o ψ ψ ε ε + + + = ( B. 6) θ ˆ ( t + 1) = θ ˆ ( t) + F( t) ψ ( t) ε ( t + 1) ( B. 7) 1 ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( 1) ( ) t F t t F t t t F t F t F t T T ψ ψ ψ ψ + + = − ( B. 8) 39 C. Review of closed loop identification methods The basic problem with closed loop data is that it typically has less information than the open loop system – any important purpose of feedback is to make the closed loop system less sensitive to changes in the open loop system. Figure 18 shows a standard feedback closed loop system, where u is the input, y is the output, e is the noise, and r is a reference signal. The goal is to identify the plant model G and the noise model H. Generally, there is no requirement on the knowledge of the regulator K. Also notice that u( t) = r( t) − K( q) y( t) , so r carries no further information about the system, if u is measured. G K r u y v H e  + Figure 18. Feedback closed loop system The plant G is identifiable under the closed loop system if and only if the data is informative, which requires at least one of the following two conditions to hold [ 39], 1. The external signal r is persistently exciting, or 2. [ ( ) ( ) ( ) ( )] 0 2 E * W q y t + * W q u t ≠ y u , i. e. there is no linear time invariant and noise free relationship between y and u such that W ( q) u( t) W ( q) y( t) u y * ≈ * . In our study, the external signal r is always equal to 0. However, since human is a very complicated nonlinear, time varying system, usually there is no linear time invariant and noise free relationship between the steering angle and look ahead lateral. Thus the second condition can be satisfied. The non zero road curvature further assures that the collected data is 40 informative, thus the driver’s model is identifiable. There are three major approaches to solve this closed loop system identification problem, 1. The direct approach – Apply the basic prediction error method in a straightforward manner, i. e. use the output y of the process and the input u in the same way as for open loop operation, ignoring any possible feedback, and not using the reference signal r. 2. The indirect approach – If the regulator K is known, we can identify the closed loop system from reference input r to output y. The plant model G can be retrieved from that information and the knowledge of K, and we don’t even have to estimate the noise model H. Bear in mind that this approach does require true knowledge about K. 3. The joint input output approach – If the reference signal r is known or can be measured, the joint input output approach can be pursued. We consider y and u as outputs of a system driven by r and noise; then the plant G can be recovered from the joint model as Gry/ Gru. Also notice that the reference r can not be 0 for this method to work. Each approach has its own advantages or disadvantages. For example, the indirect method requires the knowledge of the regulator, and the joint input out method assumes that the reference signal r can not be 0 and it can be measured. These conditions restrict them to be applied in this project, since either the vehicle dynamics ( equivalent to the regulator K) is not known exactly, or the reference signal is 0, as can be seen from Figure 4. The direct method, on the other hand, works regardless of the complexity of the regulator, and requires no knowledge about the character of the feedback. There are no special algorithms and software required either, and it has also been proved that consistency and optimal accuracy can be obtained using the direct method if the data is informative ( see next paragraph for explanation) and the model structure contains the true system [ 39]. Furthermore, unstable systems can also be handled, as long as the closed loop system is stable. 41 The only drawback of the direct approach is that the identified plant model may be biased, unless we have a good noise model. In open loop operation we can always obtain unbiased estimates of model parameters of G without requirements on the noise model H. Thus, we can not handle model approximation issues with full control in the feedback case. However, consistency and optimal accuracy is guaranteed, just as in the open loop case, if the true system is contained in the model structure. No knowledge of the feedback or reference signal is required. The prediction error method, applied in a direct fashion, is chosen to solve the off line closed loop driver model identification problem in this project. No knowledge of the vehicle dynamics or the reference signal is required. The method works well under the assumption that the identified noise model can describe the true noise properties, and they give consistent estimates and optimal accuracy. This direct PEM method can be further improved by first building a higher order model G ˆ , with small bias, and then reducing the identified model to lower order with a proper frequency weighting. 
PDI.Date  2007 
PDI.Title  Vehicle/driver monitoring for enhanced safety of transit buses 



B 

C 

I 

S 


