comma.ai presents comma2k19, a dataset of over 33 hours of commute in California's 280 highway. This means 2019 segments, 1 minute long each, on a 20km section of highway driving between California's San Jose and San Francisco. comma2k19 is a fully reproducible and scalable dataset. The data was collected using comma EONs that has sensors similar to those of any modern smartphone including a road-facing camera, phone GPS, thermometers and 9-axis IMU. Additionally, the EON captures raw GNSS measurements and all CAN data sent by the car with a comma grey panda.

Here we also introduced Laika, an open-source GNSS processing library. Laika produces 40% more accurate positions than the GNSS module used to collect the raw data. This dataset includes pose (position + orientation) estimates in a global reference frame of the recording camera. These poses were computed with a tightly coupled INS/GNSS/Vision optimizer that relies on data processed by Laika. comma2k19 is ideal for development and validation of tightly coupled GNSS algorithms and mapping algorithms that work with commodity sensors.

Dataset Structure

Directory Structure

The data is split into 10 chunks of each about 200 minutes of driving. Chunks 1-2 of the dataset are of the RAV4 and the rest is the civic. The dongle_id of the RAV4 is b0c9d2329ad1606b and that of the civic is 99c94dc769b5d96e.

Dataset_chunk_n
|
+-- route_id (dongle_id|start_time)
    |
    +-- segment_number
        |
        +-- preview.png (first frame video)
        +-- raw_log.bz2 (raw capnp log, can be read with openpilot-tools: logreader)
        +-- video.hevc (video file, can be read with openpilot-tools: framereader)
        +-- processed_log/ (processed logs as numpy arrays, see format for details)
        +-- global_pos/ (global poses of camera as numpy arrays, see format for details)

Log Format

Every log type in processed_log director contains 2 numpy arrays. A timestamp array in seconds using the system device's boot time and a value array.

processed_log
|
+--IMU ([forward, right, down])
|  |
|  +--acceleration: (m^2/s)
|  +--gyro_uncalibrated (rad/s)
|  +--gyro_bias: android gyro bias estimate (rad/s)
|  +--gyro: with android bias correction (rad/s)
|  +--magnetic_uncalibrated: (T)
|  +--magnetic: with android calibration(T)
|
+--CAN data:
|  |
|  +--car_speed (m/s)
|  +--steering_angle (deg)
|  +--wheel_speeds: [front_left, front_right, rear_left, rear_right] (m/s)
|  +--radar: [forward distance (m),
|  |          left distance (m),
|  |          relative speed (m/s),
|  |          nan,
|  |          nan,
|  |          address,
|  |          new_track (bool)]
|  +--raw CAN: This not stored as a value array but as three seperate arrays [src, address, data]
|
+--GNSS
   |
   +--live_gnss_qcom: [latitude (deg),
   |                   longitude (deg),
   |                   speed (m/s),
   |                   utc_timestamp (s),
   |                   altitude (m),
   |                   bearing (deg)]
   +--live_gnss_ublox: [latitude (deg),
   |                    longitude (deg),
   |                    speed (m/s),
   |                    utc_timestamp (s),
   |                    altitude (m),
   |                    bearing (deg)]
   |
   +--raw_gnss_qcom: every row represents a measurement
   |                 of 1 satellite at 1 epoch can easily
   |                 be manipulated with laika.
   |                 [prn (nmea_id, see laika),
   |                  week of gps_time of reception (gps_week),
   |                  time pf week of gps_time of reception (s),
   |                  nan,
   |                  pseudorange (m),
   |                  pseudorange_std (m),
   |                  pseudorange_rate (m/s),
   |                  pseudorange_rate_std (m/s)]
   +--raw_gnss_ublox: every row represents a measurement
                      of 1 satellite at 1 epoch can easily
                      be manipulated with laika.
                      [prn (nmea_id, see laika),
                       week of gps_time of reception (gps_week),
                       time pf week of gps_time of reception (s),
                       GLONASS channel number (-7..6) nan if not GLONASS,
                       pseudorange (m),
                       pseudorange_std (m),
                       pseudorange_rate (m/s),
                       pseudorange_rate_std (m/s)]

Pose Format

The poses of the camera and timestamps of every frame of the video are stored
as follows:
  frame_times: timestamps of video frames in boot time (s)
  frame_gps_times: timestamps of video frames in gps_time: ([gps week (weeks), time-of-week (s)])
  frame_positions: global positions in ECEF of camera(m)
  frame_velocities: global velocity in ECEF of camera (m/s)
  frame_orientations: global orientations as quaternion needed to
                      rotate from ECEF  frame to local camera frame
                      defined as [forward, right, down] (hamilton quaternion!!!!)

Workaround for Microsoft-based filesystems

ExFat and NTFS, among others, do not allow for |, the pipe character, in file and directory names. If you are using Microsoft-based filesystems, a workaround is to move the dataset to a non-Microsoft filesystem, decompress the archives there and run python3 utils/unzip_msft_fs.py <dataset dir> <goal dir> where <dataset dir> contains all chunks as zip files and <goal dir> is the directory you want the zip files extracted to. This will expand all the zip files in parallel and replace the pipe character with an underscore at every occurrence on the fly.