Participation in the academic competition Irony Detection in Portuguese - IDPT 2021, organized by the Federal University of Pelotas as part of the Iberian Language Evaluation Forum (IberLEF), brought positive results for the institute. As well as a good position in the ranking, the SiDi team had the opportunity to interact, learn and experience something that should help them improve their work.
The challenge of the competition, which began at the end of March and ended in May this year, was to identify the presence of irony in news texts and tweets written in Portuguese. In the category of news texts, the SiDi team came 4th overall, with a BACC score (which measures accuracy) of 0.8970 - first place achieved 0.9215. If the result is considered by teams, SiDi came second in the ranking, since the first three positions were occupied by competitors from the same team - according to the rules, each team could send several model submissions.
In tweets written in Portuguese, the SiDi team came between 5th and 10th place in the overall ranking, considering all submissions. The team achieved a score of 0.50 in the balanced accuracy measure (BACC) - first place got 0.52.
For Nataly Leopoldina, who works in the area of cybersecurity, participation in the IDPT was positive, mainly because it allowed for greater contact with the institute's experts in natural language processing (NLP). "This will certainly help in the work carried out in my area," she says.
Bruna Osti, also from the cybersecurity area, agrees that the experience was enriching. "It was an opportunity to learn a lot from specialists in this other area," she says. Henrico Brum, who works with NLP, also considers it important to take part in events of this kind. "It's good to work with new data sets, because it ends up helping our work at SiDi," he concludes.