The dataset comprises the following information, captured and synchronized at 10 Hz:

• Raw (unsynced+unrectified) and processed (synced+rectified) grayscale stereo sequences (0.5 Megapixels, stored in png format)
• Raw (unsynced+unrectified) and processed (synced+rectified) color stereo sequences (0.5 Megapixels, stored in png format)
• 3D Velodyne point clouds (100k points per frame, stored as binary float matrix)
• 3D GPS/IMU data (location, speed, acceleration, meta information, stored as text file)
• Calibration (Camera, Camera-to-GPS/IMU, Camera-to-Velodyne, stored as text file)
• 3D object tracklet labels (cars, trucks, trams, pedestrians, cyclists, stored as xml file)

Here, "unsynced+unrectified" refers to the raw input frames where images are distorted and the frame indices do not correspond, while "synced+rectified" refers to the processed data where images have been rectified and undistorted and where the data frame numbers correspond across all sensor streams.

Data Collection

Our recording platform is a Volkswagen Passat B6, which has been modified with actuators for the pedals (acceleration and brake) and the steering wheel. The data is recorded using an eight core i7 computer equipped with a RAID system, running Ubuntu Linux and a real-time database. We use the following sensors:

• 1 Inertial Navigation System (GPS/IMU): OXTS RT 3003
• 1 Laserscanner: Velodyne HDL-64E
• 2 Grayscale cameras, 1.4 Megapixels: Point Grey Flea 2 (FL2-14S3M-C)
• 2 Color cameras, 1.4 Megapixels: Point Grey Flea 2 (FL2-14S3C-C)
• 4 Varifocal lenses, 4-8 mm: Edmund Optics NT59-917

The laser scanner spins at 10 frames per second, capturing approximately 100k points per cycle. The vertical resolution of the laser scanner is 64. The cameras are mounted approximately level with the ground plane. The camera images are cropped to a size of 1382 x 512 pixels using libdc's format 7 mode. After rectification, the images get slightly smaller. The cameras are triggered at 10 frames per second by the laser scanner (when facing forward) with shutter time adjusted dynamically (maximum shutter time: 2 ms). Our sensor setup with respect to the vehicle is illustrated in the following figure. Note that more information on calibration parameters is given in the calibration files and the development kit (see raw data section).