The participation in the academic competition Irony Detection in Portuguese – IDPT 2021, organized by the Federal University of Pelotas as part of the Iberian Languages Evaluation Forum (IberLEF), brought positive results for the institute. In addition to the good position in the ranking, SiDi’s team had the opportunity to interact, learn and live an experience that should contribute to the improvement of their work.
The challenge of the competition, which started in late March and ended this May, was to identify the presence of irony in news texts and tweets written in Portuguese. In the news texts category, the SiDi team came in 4th place in the overall classification, with the BACC score (which measures accuracy) of 0.8970 – the first place reached 0.9215. If the result is considered by teams, SiDi obtained the second place in the ranking, since the first three positions were with competitors of the same team – by the rules, each team could submit several model submissions.
In tweets written in Portuguese, the SiDi team ranked 5th – 10th in the overall ranking, considering all submissions. The team achieved a score of 0.50 in the balanced accuracy measurement (BACC) – the first place obtained 0.52.
For Nataly Leopoldina, who works in the cybersecurity area , the participation in IDPT was positive, mainly because it allowed greater contact with the institute’s specialists in natural language processing (NLP). “This will certainly help the work developed in my area,” she says.
Bruna Osti, also from the cybersecurity area, agrees that the experience was enriching. “It was an opportunity to learn a lot from experts in this other area,” she says. Henrico Brum, who works with NLP, also considers participation in events of this type to be important. “It is good to work with new datasets, because it ends up helping our work in SiDi”, he concludes.