The Emotionet database includes 950,000 images with annotated AUs. These were annotated with the algorithm described in

Benitez-Quiroz, C. F., Srinivasan, R., & Martinez, A. M. (2016). EmotioNet: An accurate, real-time algorithm for the automatic annotation of a million facial expressions in the wild. In Proceedings of IEEE International Conference on Computer Vision & Pattern Recognition (CVPR'16), Las Vegas, NV, USA.

You can train your system using this set. You can also use any other annotated dataset you think appropriate. This dataset has been used to successfully train a variety of classifiers, including several deep networks.We also include 25K (24,600 to be precise) manually annotated AUs. You may want to use this dataset to see how well your algorithm works or to optimize the parameters of your algorithm.