A dataset to compare the performance of different non-stationary video based fire detectors. This is a new dataset of fire videos containing frame by frame annotations which may be used for non-stationary fire detection algorithms evaluation. The dataset includes hand-held, robot attached and drone attached footages and aims to boost the development of fully autonomous fire fighter robots. The presented ground truth and metrics may adapt to any state-of-the-art technique and provide a reliable and unbiased solution to compare them.