Abstract
MODALSIM94 is a multimodal dataset collected from 94 participants performing simulated flight workload tasks in a fixed-base research flight simulator. The dataset was designed to support research on cognitive workload assessment, pilot monitoring, adaptive automation, and human–machine interaction in aviation-relevant environments. Participants completed three five-minute flight scenarios corresponding to low, medium, and high workload conditions. Workload was manipulated through a secondary N-back working-memory task, with no N-back task in the low workload condition, a 1-back task in the medium workload condition, and a 2-back task in the high workload condition.
The dataset includes synchronised physiological, behavioural, performance, subjective, and demographic data. Physiological measures comprise cardiac activity from an eSENSE Pulse sensor, respiration data from an eSENSE Respiratory sensor, and eye-tracking data from Dikablis 3 goggles. Flight and performance data include aircraft dynamics, control-related variables, trajectory information, tunnel-deviation metrics, and N-back task responses, accuracy, and reaction times. Subjective measures include NASA-TLX workload ratings, SART situation-awareness ratings, Karolinska Sleepiness Scale scores, Samn–Perelli fatigue ratings, and participant demographic information.
All data are provided in open CSV/TXT formats and organised by modality. Each modality preserves its native temporal resolution, while participant identifiers, scenario labels, and elapsed-time references from scenario onset support temporal alignment across data streams. MODALSIM94 can be used for workload classification and regression, multimodal physiological modelling, flight-performance analysis, personalisation studies, and cross-domain research on cognitive-state monitoring.
Technical Information
The dataset is provided in open, tabular formats and can be read using standard software capable of handling CSV and TXT files, such as Python, R, MATLAB, Microsoft Excel, LibreOffice Calc, or similar data-analysis environments. No proprietary software is required to access the released data files. The file `QuestionnairesAndDemographicDefinition.txt` provides the definition of the fields included in the questionnaires and demographic data.
The data were collected in a fixed-base research flight simulator using a multimodal acquisition setup. Physiological data were recorded using an eSENSE Pulse sensor for cardiac activity, an eSENSE Respiratory sensor for respiration, and Dikablis 3 eye-tracking goggles for ocular measures. Flight and control data were generated in the simulator environment, which integrated X-Plane for visual rendering and MATLAB/Simulink for flight-dynamics modelling, real-time data logging, and event synchronisation. Additional software tools, including MATLAB App Designer, D-Lab, and the eSENSE application, were used to administer tasks, collect questionnaire responses, and export physiological and eye-tracking data.
The final dataset was normalised and exported as CSV/TXT files. Each modality was preserved at its native temporal resolution, and synchronisation across modalities is supported through participant identifiers, scenario labels, and elapsed-time references from scenario onset.
Copyright © 2026 Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V. acting on behalf of its Fraunhofer-Institut für Kognitive Systeme IKS. All rights reserved.
This dataset is subject to the terms and conditions of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0) (https://creativecommons.org/licenses/by-nc-sa/4.0/).
Contact: marcel.ramon.rodriguez@iks.fraunhofer.de
