Errare humanum est? A Pilot Study to Evaluate the Human-Likeness of a AI Othello Playing Agent

septembre 2022
Apprendre et Innover
Ingénierie & Outils numériques
Communications avec actes dans un congrès international
Auteurs : Enrico Lauletta (Departement d'Informatique), Beatrice Biancardi (LINEACT), Antonio Noreli (Departement d'Informatique), Maurizio Mancini (Departement d'Informatique), Alessandro Panconesi (Departement d'Informatique)
Conférence : International Conference on Intelligent Virtual Agents, 5 septembre 2022

Olivaw is an AI Othello playing agent which autonomously learns how to improve its gameplay by playing against itself. Some topnotch players (including former World Champions) reported that they had the impression that Olivaw’s gameplay was human-like. To better investigate the processes related to these impressions, we conducted a pilot study using the Othello Game Evaluation App, a computer application we developed to evaluate pre-recorded Othello games in a controlled setting while assuring an adequate user experience. An exploratory analysis of the results shows that the participants mostly evaluated Olivaw as a human. When asked for a motivation for their choice, some of them reported that they evaluate poor game moves (and, consequently, losing the game) as an indication of the human-likeness of the player.