| Title | : | Towards Improved Evaluation of Natural Language Generation Systems |
| Speaker | : | Ananya Sai B (IITM) |
| Details | : | Thu, 19 Dec, 2024 9:00 AM @ SSB 233 |
| Abstract: | : | Advances in machine learning research have generated a wide range of automatic text-generation applications such as machine translation systems, dialogue agents, etc. These systems are not always reliable, and a careful evaluation is important to measure scientific progress. This has led to an increased interest in automatic evaluation metrics, with more than 50 metrics proposed in the past ~4-5 years. This thesis delves deeper and proposes techniques to analyze and further improve evaluation approaches. Specifically, we (i) propose a taxonomy to categorise various metrics (ii) identify the shortcomings of existing evaluation metrics (iii) develop better resources and techniques for improved evaluation (iv) propose a framework for meta-evaluation and (v) extend the largely English-centric work to Indian languages with relevant data, analysis, and novel models. Other Information: -web conference linkhttps://meet.google.com/rpw-shch-tpi |
