Avaliação inteligente de usabilidade em um laboratório remoto utilizando neuro-fuzzy

Abstract

Over the past decade, several studies have described attempts to automate usability measurement using different data and techniques. However, so far, these proposals have not yielded significant conclusions. In a previous study, a methodology was proposed for measuring usability and user experience (UX) in the Laboratório Remoto de Microcontroladores at Uninove (LRM-U9), which obtained promising results, albeit at a high cost. LRM-U9 is a remote laboratory (RL) that enables distance learning experiments in the Internet of Things (IoT) field, allowing users to send commands to real equipment composed of a Raspberry Pi connected to two Arduinos, which, in turn, include components such as LEDs, sensors, a stepper motor, and a servo motor. The behavior of these components can be observed through a camera. The present study proposed an intelligent methodology for evaluating LRM-U9. To achieve this, software developers and students from Electrical Engineering-related disciplines were invited to conduct experiments in LRM-U9. Participants followed predefined scripts and, at the end, answered a questionnaire comprising items from the System Usability Scale (SUS), Usability Metrics for User Experience (UMUX), and descriptive questions. Additionally, an Adaptive Neuro-Fuzzy Inference System (ANFIS) was developed, where questionnaire scores were used as target values, while usage and navigation data were employed to train the ANFIS model. In total, 39 samples with 49 attributes were collected, with 33 used for training and 6 for validation. The results were considered satisfactory: 78% of respondents reported that LRM-U9 contributed to their learning, and 63% highlighted its usefulness in teaching and learning. The SUS and UMUX questionnaires indicated usability and UX levels slightly above average, with scores of 72.5 and 76.6, respectively. The ANFIS model achieved a Root Mean Square Error (RMSE) of 4.6486, resulting in values very close to the actual ones. The methodology demonstrated potential for application in other contexts and target audiences.