LG电子-基于环视监控系统的场景理解自主驾驶网络.pdf

poPoq

9页

0次

2021-05-01

50墨值下载

Scene Understanding Networks for Autonomous Driving based on Around View

Monitoring System

JeongYeol Baek

1,*

, Ioana Veronica Chelu

2,*

, Livia Iordache

, Vlad Paunescu

, HyunJoo Ryu

Alexandru Ghiuta

, Andrei Petreanu

, YunSung Soh

, Andrei Leica

, and ByeongMoon Jeon

1,†

Convergence Center, LG Electronics, Korea

Arnia Software, Romania

Abstract

Modern driver assistance systems rely on a wide range

of sensors (RADAR, LIDAR, ultrasound and cameras) for

scene understanding and prediction. These sensors are typ-

ically used for detecting trafﬁc participants and scene ele-

ments required for navigation. In this paper we argue that

relying on camera based systems, speciﬁcally Around View

Monitoring (AVM) system has great potential to achieve

these goals in both parking and driving modes with de-

creased costs. The contributions of this paper are as fol-

lows: we present a new end-to-end solution for delimiting

the safe drivable area for each frame by means of identi-

fying the closest obstacle in each direction from the driving

vehicle, we use this approach to calculate the distance to the

nearest obstacles and we incorporate it into a uniﬁed end-

to-end architecture capable of joint object detection, curb

detection and safe drivable area detection. Furthermore,

we describe the family of networks for both a high accuracy

solution and a low complexity solution. We also introduce

further augmentation of the base architecture with 3D ob-

ject detection.

1. Introduction

Visual environment perception plays a key role in the

development of autonomous vehicles, providing fundamen-

tal information on the driving scene, including free space

area and surrounding obstacles. These perception tasks can

gather information from various sensors - LIDARs, cam-

eras, RADARs or a fusion of them. Dense laser scanners

are capable of creating a dynamic three-dimensional map of

the environment and are best-suited for the task. However,

their costs are still very high to be integrated in reasonably

* These two authors contributed equally to this work.

† E-mail: bm.jeon@lge.com

priced vehicles. Driven by the latest advances in the ﬁeld

of computer vision, we propose using only camera-based

systems, which have the potential to reach dense laser-scan

performance with lower cost. In particular, deep learning

has fueled an improvement in accuracy of classic object de-

tection and segmentation at an accelerated rate. However,

object detection systems alone are usually not sufﬁcient for

autonomous emergency-braking and forward-collision sys-

tems, since the variety of possible road obstacles (e.g. tree

branches, small animals) and road structure (e.g. country

roads, different textures) make it impractical to train only

typical object detection networks and road segmentation

networks for scene understanding. To tackle these prob-

lems, we present two main contributions:

• A new solution for delimiting the closest obstacles in

all directions of the driving vehicle through bottom

point estimation and curb detection, while also deter-

mining the exact distance to the nearest obstacles in

each direction.

• Integrating the obstacle detection network into a uni-

ﬁed end-to-end solution capable of jointly delimiting

the free drivable area by means of obstacle bottom

point estimation, curb detection and 2D multi-scale ob-

ject detection for a low complexity solution.

Scene understanding systems require high accuracy to

ensure safety. However, model deployment on embedded

platforms calls for real-time inference speed for prompt

control and small model size for power-efﬁciency. We ad-

dress these requirements by developing a low complexity

solution which uses a light encoder network, beneﬁts from

sharing computations (i.e sharing the encoder) amongst the

proposed perception tasks and uses single shot detection.

We demonstrate the viability of our uniﬁed solution by

showing that it achieves 16.7 fps on the Nvidia TX2 em-

bedded platform.

arXiv:1805.07029v1 [cs.CV] 18 May 2018

Figure 1. AVM camera system

Since 2D object detection sometimes provides insufﬁ-

cient information for scene understanding, we need to aug-

ment current solutions with 3D information to ﬁnd the exact

pose of objects in the 3D world. We propose augmenting

the high accuracy solution to detect the orientation and di-

mension of each object.

This paper mainly focuses on detecting fundamental

scene information for safe driving: object detection, curb

detection, free drivable area segmentation, object distance

from the camera and object orientation. We test our ap-

proach on a novel dataset consisting of ﬁsheye images. A

ﬁsheye image is a wide-angle and distorted image which

is generally used for Around View Monitoring (AVM) sys-

tems (Figure 1). The advantage of using ﬁsheye cameras in

the ﬁeld of autonomous driving is obvious, as they offer a

much wider ﬁeld of view (190°) than conventional cameras,

thus capturing more relevant information of the car’s sur-

roundings (pedestrians, obstacles, etc.). The four cameras

are positioned at the front, rear, left, and right side of the

vehicle and give drivers a 360° view of their surroundings

so as to check for obstructions around the vehicle.

2. Related Work

In this section we present a review on recent approaches

for the tasks that we explore in the rest of the paper, i.e.

object detection, classiﬁcation, free space segmentation and

3D orientation.

Free space detection: State-of-the-art methods for de-

tecting drivable area surface usually frame the problem in

terms of road segmentation. Fully Convolutional Networks

(FCN) [8] use a convolutional network to perform spatially

dense prediction tasks like semantic segmentation using

transposed convolutions to model upsample layers. Later,

dilated convolutions [16] were also introduced to augment

the receptive ﬁeld of the network. The existing research

generally tackles pixel segmentation networks or depth map

derivation using stereo cameras. With these methods, un-

classiﬁed pixels require complex post-processing to handle

them. In this paper we propose using a simpler architecture

for detecting obstacles and free space detection by identi-

fying the bottom points of each obstacle in all directions of

the driving vehicle.

Object detection: Modern neural network approaches

to object detection can be divided into two categories: re-

gion proposal based methods and single-shot methods. The

former category covers approaches like Faster R-CNN [10]

that have a two-step process which involves ﬁrst generating

region proposals using an RPN (region proposal network)

and then scoring them using a secondary module. In the

single-shot network approach [7], the region proposal and

classiﬁcation stages are integrated into one single stage, by

using predeﬁned anchor boxes (priors) like a sliding win-

dow that moves through each spatial position on the feature

map to concurrently predict bounding boxes and class conﬁ-

dence scores. Performing region proposal and classiﬁcation

network simultaneously makes this approach extremely fast

in comparison with two-stage methods.

3D Object detection: 3D object detection has gathered

signiﬁcant consideration lately due to its key contribution

in applications that require interactions with objects in real-

world scenarios, as in autonomous driving. This issue has

been addressed from a purely geometric point of view (e.g.

estimating the pose of an object with 6DoF from a single

image), as well as using DCNNs (deep convolutional neu-

ral networks) in order to reconstruct 3D models. In [11],

Rothganger et al. use local afﬁne-invariant image descrip-

tors in order to construct 3D models of object instances in

2D images and then matching them with 3D poses in the

image. In [3], Hara et al. demonstrate DCNN effectiveness

in estimating the 0° to 360° orientation of objects. Mousa-

vian et al. [9] use a DCNN to regress stable 3D object fea-

tures, while other methods exploit depth information from

stereo images [1], or combine temporal information using

structure from motion algorithms in order to augment 2D

detections with 3D information.

3. Networks for Scene Understanding

In this section we give a detailed description of network

architectures which we propose for AVM scene understand-

ing, including object detection, free/drivable area segmen-

tation, object distance and object orientation. For 2D object

detection we investigate standard object detection networks

such as Faster R-CNN and SSD [7]. We also experiment