MIT Autonomous Vehicle Technology Study：Large-Scale Deep Learning Based Analysis of Driver Behavior and Interaction with Automation.pdf

aKun

242

17页

0次

2021-02-23

50墨值下载

MIT Autonomous Vehicle Technology Study:

Large-Scale Deep Learning Based Analysis of

Driver Behavior and Interaction with Automation

Lex Fridman

∗

, Daniel E. Brown, Michael Glazer, William Angell, Spencer Dodd, Benedikt Jenik,

Jack Terwilliger, Julia Kindelsberger, Li Ding, Sean Seaman, Hillary Abraham, Alea Mehler,

Andrew Sipperley, Anthony Pettinato, Bobbie Seppelt, Linda Angell, Bruce Mehler, Bryan Reimer

∗

Abstract—Today, and possibly for a long time to come, the

full driving task is too complex an activity to be fully formalized

as a sensing-acting robotics system that can be explicitly solved

through model-based and learning-based approaches in order to

achieve full unconstrained vehicle autonomy. Localization, map-

ping, scene perception, vehicle control, trajectory optimization,

and higher-level planning decisions associated with autonomous

vehicle development remain full of open challenges. This is

especially true for unconstrained, real-world operation where the

margin of allowable error is extremely small and the number of

edge-cases is extremely large. Until these problems are solved,

human beings will remain an integral part of the driving task,

monitoring the AI system as it performs anywhere from just over

0% to just under 100% of the driving. The governing objectives of

the MIT Autonomous Vehicle Technology (MIT-AVT) study are

to (1) undertake large-scale real-world driving data collection

that includes high-deﬁnition video to fuel the development of

deep learning based internal and external perception systems,

(2) gain a holistic understanding of how human beings interact

with vehicle automation technology by integrating video data

with vehicle state data, driver characteristics, mental models,

and self-reported experiences with technology, and (3) identify

how technology and other factors related to automation adoption

and use can be improved in ways that save lives. In pursuing

these objectives, we have instrumented 21 Tesla Model S and

Model X vehicles, 2 Volvo S90 vehicles, and 2 Range Rover

Evoque vehicles for both long-term (over a year per driver)

and medium term (one month per driver) naturalistic driving

data collection. Furthermore, we are continually developing new

methods for analysis of the massive-scale dataset collected from

the instrumented vehicle ﬂeet. The recorded data streams include

IMU, GPS, CAN messages, and high-deﬁnition video streams

of the driver face, the driver cabin, the forward roadway, and

the instrument cluster (on select vehicles). The study is on-

going and growing. To date, we have 78 participants, 7,146 days

of participation, 275,589 miles, and 3.5 billion video frames.

This paper presents the design of the study, the data collection

hardware, the processing of the data, and the computer vision

algorithms currently being used to extract actionable knowledge

from the data.

MIT Autonomous Vehicle

Technology Study

Study months to-date: 21

Participant days: 7,146

Drivers: 78

Vehicles: 25

Miles driven: 275,589

Video frames: 3.48 billion

Study data collection is ongoing.

Statistics updated on: Oct 23, 2017.

Tesla Model S

24,657 miles

588 days in study

Tesla Model X

22,001 miles

421 days in study

Tesla Model S

18,896 miles

435 days in study

Tesla Model S

18,666 miles

353 days in study

Range Rover

Evoque

18,130 miles

483 days in study

Tesla Model S

15,735 miles

322 days in study

Tesla Model X

15,074 miles

276 days in study

Range Rover

Evoque

14,499 miles

440 days in study

Tesla Model S

14,410 miles

371 days in study

Tesla Model S

14,117 miles

248 days in study

Volvo S90

13,970 miles

325 days in study

Tesla Model S

12,353 miles

321 days in study

Volvo S90

11,072 miles

412 days in study

Tesla Model X

10,271 miles

366 days in study

Tesla Model S

9,188 miles

183 days in study

Tesla Model S

8,319 miles

374 days in study

Tesla Model S

6,720 miles

194 days in study

Tesla Model S

5,186 miles

91 days in study

Tesla Model X

5,111 miles

232 days in study

Tesla Model S

4,596 miles

132 days in study

Tesla Model X

4,587 miles

233 days in study

Tesla Model X

3,719 miles

133 days in study

Tesla Model S

3,006 miles

144 days in study

Tesla Model X

1,306 miles

69 days in study

Tesla Model S

(Ooad pending)

Fig. 1: Dataset statistics for the MIT-AVT study as a whole and for the individual vehicles in the study.

* Lex Fridman (fridman@mit.edu) and Bryan Reimer (reimer@mit.edu) are primary contacts. Linda Angell and Sean Seaman are afﬁliated with

Touchstone Evaluations Inc. All other authors are afﬁliated with Massachusetts Institute of Technology (MIT).

arXiv:1711.06976v1 [cs.CY] 19 Nov 2017

(a) Face Camera for Driver State.

(b) Driver Cabin Camera for Driver Body Position.

(d) Instrument Cluster Camera for Vehicle State.

Fig. 2: Video frames from MIT-AVT cameras and visualization

of computer vision tasks performed for each.

I. INTRODUCTION

The idea that human beings are poor drivers is well-

documented in popular culture [1], [2]. While this idea is

often over-dramatized, there is some truth to it in that we’re

at times distracted, drowsy, drunk, drugged, and irrational

decision makers [3]. However, this does not mean it is easy

to design and build an autonomous perception-control system

that drives better than the average human driver. The 2007

DARPA Urban Challenge [4] was a landmark achievement

in robotics, when 6 of the 11 autonomous vehicles in the

ﬁnals successfully navigated an urban environment to reach

the ﬁnish line, with the ﬁrst place ﬁnisher traveling at an

average speed of 15 mph. The success of this competition

led many to declare the fully autonomous driving task a

“solved problem”, one with only a few remaining messy

details to be resolved by automakers as part of delivering a

commercial product. Today, over ten years later, the problems

of localization, mapping, scene perception, vehicle control,

trajectory optimization, and higher-level planning decisions

associated with autonomous vehicle development remain full

of open challenges that have yet to be fully solved by systems

incorporated into a production platforms (e.g. offered for

sale) for even a restricted operational space. The testing of

prototype vehicles with a human supervisor responsible for

taking control during periods where the AI system is “unsure”

or unable to safely proceed remains the norm [5], [6].

The belief underlying the MIT Autonomous Vehicle Tech-

nology (MIT-AVT) study is that the DARPA Urban Challenge

was only a ﬁrst step down a long road toward developing

autonomous vehicle systems. The Urban Challenge had no

people participating in the scenario except the professional

drivers controlling the other 30 cars on the road that day. The

authors believe that the current real-world challenge is one

that has the human being as an integral part of every aspect

of the system. This challenge is made especially difﬁcult due

to the immense variability inherent to the driving task due to

the following factors:

• The underlying uncertainty of human behavior as rep-

resented by every type of social interaction and conﬂict

resolution between vehicles, pedestrians, and cyclists.

• The variability between driver styles, experience, and

other characteristics that contribute to their understand-

ing, trust, and use of automation.

• The complexities and edge-cases of the scene perception

and understanding problem.

• The underactuated nature of the control problem [7] for

every human-in-the-loop mechanical system in the car:

from the driver interaction with the steering wheel to the

tires contacting the road surface.

• The expected and unexpected limitation of and imperfec-

tions in the sensors.

• The reliance on software with all the challenges inherent

to software-based systems: bugs, vulnerabilities, and the

constantly changing feature set from minor and major

version updates.

of 17

50墨值下载

database

关注

评论