Request access to the AM-FED dataset here.

Public datasets are play an important role in research. They are used to compare results between research groups, and offer a benchmark on which to evaluate competing methodologies.

In this paper, Affectiva presents Affectiva-MIT Facial Expression Dataset (AM-FED). The creation of this dataset was motivated by the gap between typical facial action unit datasets and the real world conditions observed by Affectiva when deploying this technology in the field. Unlike datasets collected in controlled conditions with posed expressions, this dataset is meant to challenge systems to perform in real-world conditions where uncontrolled factors such as lighting come into play. In addition, facial responses in the AM-FED dataset are spontaneous, which can manifest as more subtle expressions that typically seen in posed datasets. Ultimately, the hope is that more researchers focus on solving the problem of facial action unit detection in real-world environments.


Computer classification of facial expressions requires large amounts of data and this data needs to reflect the diversity of conditions seen in real applications. Public datasets help accelerate the progress of research by providing researchers with a benchmark resource. We present a comprehensively labeled dataset of ecologically valid spontaneous facial responses recorded in natural settings over the Internet. To collect the data, online viewers watched one of three intentionally amusing Super Bowl commercials and were simultaneously filmed using their webcam. They answered three self-report questions about their experience. A subset of viewers additionally gave consent for their data to be shared publicly with other researchers. This subset consists of 242 facial videos (168,359 frames) recorded in real world conditions. The dataset is comprehensively labeled for the following: 1) frame-by-frame labels for the presence of 10 symmetrical FACS action units, 4 asymmetric (unilateral) FACS action units, 2 head movements, smile, general expressiveness, feature tracker fails and gender; 2) the location of 22 automatically detected landmark points; 3) self-report responses of familiarity with, liking of, and desire to watch again for the stimuli videos and 4) baseline performance of detection algorithms on this dataset. This data is available for distribution to researchers online, the EULA can be found at: