Introducing the 3MT_French Dataset

Public speaking constitutes a real challenge for a large part of the population: estimates indicate that 15 to 30% of the population suffers from public speaking anxiety (Tillfors & Furmark, 2007). Several existing corpora were previously used to model public speaking behavior. Those created ad-hoc for research purposes (e.g., Wörtwein et al., 2015) often provide a limited number of speakers, and are collected in an experimental setting without a real human audience. In monologues (e.g., Chen et al., 2017), the interaction with the audience is mostly asynchronous, and they are collected in the context of job interviews so the annotations are focused on hireability. TED Talks are a great resource but risk to contain mostly high-quality presentations, making it difficult to investigate the behaviors related to low-quality speeches or to anxious speaking behavior. Moreover, the videos are relatively long and the annotation protocol quite complex. In most of public speaking datasets, judgements are given after watching the entire performance, or on thin slices randomly selected from the presentations (e.g., Chollet & Scherer, 2017), without focusing on the temporal location of these slices. This does not allow to investigate how people's judgments develop over time during presentations, under the perspective of socio-cognitive theories such as Primacy and recency (Ebbinghaus, 2013) or first impressions (Ambady & Skowronski, 2008). To provide novel insights on this phenomenon, we present the 3MT_French dataset. It contains a set of presentations of PhD students participating in the French edition of 3-minute Thesis competition. The jury and audience prizes have been integrated with a set of ratings collected online through a novel annotation scheme and protocol. Global evaluation, persuasiveness, perceived self-confidence of the speaker and audience engagement were annotated on different time windows (i.e., the beginning, middle or end of the presentation, or the full video). We aim at providing two types of contributions. First, the 3MT_dataset with its particular properties: ● A relatively large amount (248) of naturalistic presentations; ● The quality of the presentations is highly heterogeneous; ● The presentations all have similar duration (180s) and structure. On the other hand, we also provide the following methodological contributions: ● A novel annotation scheme, which aims at providing a quick way to rate the quality of a presentation, considering the dimensions in common between other existing schemes; ● The annotations are collected for both the entire video and different time windows. This new resource would interest several researchers working on public speaking assessment and training, as well as it will allow for perceptive studies, both under a behavioral and linguistic point of view. It will allow for investigating whether a speaker's behaviors have a different impact on the observers' perception of their performance according to when these behaviors are realized during the speech. The automatic assessment of a speaker's performance could benefit from this information by assigning different weights to segments of behavior according to their relative position in the speech. In addition, a training system could be more efficient by focusing on improving the speaker's behavior during the most important moments of their performance.