Análise de textos com aplicação de técnicas de inteligência artificial: estudo comparativo para classificação de fuga ao tema em redações

Abstract

The process of manual correction of essays brings some difficulties, among which is the time spent for the correction and for returning the answer to the student. For institutions such as universities and the Exame Nacional do Ensino Médio (ENEM), which use essays as an evaluation for admission to higher education, as well as elementary schools, this activity demands time and cost to evaluate the texts produced. Changing the subject is one of the items evaluated on ENEM, when a student commits such mistake, his or her text is annulled because the concepts were not developed as requested in the stipulated theme for the essay. In this context, the automatic analysis of essays with the application of Natural Language Processing (NLP), Text Mining (TM) and other Artificial Intelligence (AI) techniques has shown promise in the process of automated assessment of written language. Given the referred context, the aim of this research is to compare different AI techniques for classification of topic avoidance in texts and identify those that brought better results. This is an applied and experimental research executed by applying algorithms and measuring the results obtained. The experiments outlined in particular the classification of 1320 Portuguese language essays with 119 different themes. Besides PLN and MT, the research used the following intelligent classification techniques: Convolutional Neural Networks (CNN), Multilayer Perceptron (MPL), Decision Trees, Random Forests, Gradient Boosting, Ada Boost, Stochastic Gradient Descent, Support Vector Machines, and other techniques to identify patterns in the database through unsupervised algorithms such as clustering. The experiments brought the best results for the RNC classifier, which obtained accuracy up to 89%, with False Positive (FP) rate of 5.7% and True Positive (VP) rate of 49%. Other classifiers with satisfactory results were MLP and Gradient Boosting, with accuracy of 90% and 74%, PV of 33% and 51%, and average FP of 4% and 20%, respectively. It is hoped that the solution developed in this research will contribute to impact the work of teachers and educational institutions by reducing the time and costs associated with the essay evaluation process.