TerrainNet: Visual Modeling of Complex Terrain for High-speed, Off-road Navigation
Xiangyun Meng, Nathan Hatch, Alexander Lambert, Anqi Li, Nolan Wagener, Matthew Schmittle, JoonHo Lee, Wentao Yuan, Zoey Chen, Samuel Deng, and
4 more authors
In Robotics: Science and Systems (RSS), 2023
Effective use of camera-based vision systems is
essential for robust performance in autonomous off-road driving,
particularly in the high-speed regime. Despite success in structured, on-road settings, current end-to-end approaches for scene
prediction have yet to be successfully adapted for complex outdoor
terrain. To this end, we present TerrainNet, a vision-based terrain
perception system for semantic and geometric terrain prediction
for aggressive, off-road navigation. The approach relies on several
key insights and practical considerations for achieving reliable
terrain modeling. The network includes a multi-headed output
representation to capture fine- and coarse-grained terrain features
necessary for estimating traversability. Accurate depth estimation
is achieved using self-supervised depth completion with multi-view
RGB and stereo inputs. Requirements for real-time performance
and fast inference speeds are met using efficient, learned image
feature projections.