San Francisco Thermal Dataset



San Francisco Thermal Dataset


FLIR San Francisco Regional Thermal Dataset for Algorithm Training

The FLIR Enhanced San Francisco Thermal Dataset is available for sale to automotive developers.  It enables developers to start training convolutional neural networks (CNN), empowering the automotive community to create the next generation of safer and more efficient ADAS and driverless vehicle systems using cost-effective thermal cameras from FLIR.


Why Use FLIR Thermal Sensing for ADAS?

The ability to sense thermal infrared radiation, or heat, within the ADAS context provides both complementary and distinct advantages to existing sensor technologies such as visible cameras, Lidar and radar systems:

  • With over 15 years of experience working with Veoneer to make the only automotive-qualified thermal camera, FLIR’s thermal sensors are deployed in over 600,000 cars today for driver warning systems.
  • The FLIR thermal cameras can be used to detect and classify objects in challenging conditions including total darkness, fog, smoke, inclement weather and glare, providing a supplemental dataset beyond LiDAR, radar and visible cameras.
  • When combined with visible-light data and distance scanning data from LiDAR and radar, thermal data paired with machine learning creates a more comprehensive detection and classification system.


Dataset Details & Specifications


  • Synced annotated thermal imagery and non-annotated RGB imagery for reference. Camera centerlines approximately 2 inches apart and collimated to minimize parallax.


  • ~10K total images with ~10K from short video segments and random image samples, plus ~6K BONUS images from video

Frame Annotation Labels: '

  • Car: 96,686
    Sign: 31,711
    Light: 30,568
    Person: 15,987
    Truck: 1,992
    Bus: 1,579
    Hydrant: 994
    Bike: 804
    Rider: 791
    Motor: 410
    Train: 360
    Vehicle Other: 360
    Total: 181,882


  • Clear: 7,526
    Partly Cloudy: 954
    Overcast: 745
    Rainy: 402
    Foggy: 6
    Total: 9,633


  • City Street: 5,578
    Highway: 3,215
    Residential: 717
    Parking Lot: 112
    Tunnel: 78
    Gas Station: 2
    Total: 9,702


  • Day: 8,432
    Night: 1,327
    Dawn/Dusk: 18
    Total: 9,777

Image Capture Refresh Rate: 

  • Recorded at 30Hz. Dataset sequences sampled at 2 frames/sec or 1 frame/ second. Video annotations were performed at 30 frames/sec recording.

Driving Conditions:

  • Day (86%) and night (14%) driving on San Francisco, CA bay area streets and highways from November 2018 to May 2019 with varying weather conditions.

Capture Camera Specifications: 

  • IR Tau2 640x512, 13mm f/1.0 (HFOV 45°, VFOV 37°) FLIR BlackFly (BFS-U3-51S5C-C) 1280x1024, 4-8mm f/1.4-16 megapixel lens (FOV set to match Tau2)

Dataset File Format: 

  • 1. Thermal - 14-bit TIFF (no AGC)
    2. Thermal 8-bit JPEG (AGC applied) w/o bounding boxes embedded in images
    3. Thermal 8-bit JPEG (AGC applied) with bounding boxes embedded in images for viewing purposes
    4. RGB - 8-bit JPEG
    5. Annotations: JSON (MSCOCO format)
    6. No temporal filter applied

Sample Results : 

  • mAP score coming soon.

FLIR ADK Training and Development Settings: 

  • FLIR ADK Training and Development Settings     Use the FLIR ADK with default settings to begin data collection