GPTBot Development for Translation Purposes: Flowchart, Practical Case and Future Prospects

Abstract

This paper explores the development and evaluation of a GPTBot tailored for institutional translation tasks. It addresses a gap in applied research on how generative AI can be adapted for domain-specific translation workflows, particularly in academic institutions. The aim of this work is to design and implement UGRBot, a chatbot based on ChatGPT-4 that supports the translation of institutional texts at the University of Granada (UGR) while also outlining a structured and replicable methodology for creating specialised chatbots to enhance translation processes. The methodology includes: (1) chatbot development using a knowledge base of 57 bilingual institutional documents; (2) evaluation of output quality using BLEU scores, comparing UGRBot with DeepL and Google Translate; and (3) a focused assessment on the translation of 100 institutional terms. A reference corpus in English of 14,521 words was compiled from UGR administrative and regulatory documents, with human translations serving as the benchmark. BLEU scores were computed using the Natural Language Toolkit library in Python, employing 4-gram analysis for full-text evaluation and bigram analysis for terminology translation. Results show that UGRBot outperformed both baseline systems in the translation of specialised institutional terminology, achieving the highest BLEU score in this area. However, limitations include lower performance across full-length texts. In conclusion, this research documents the development of a domain-specific GPTBot and its implementation in an institutional context, offering a transferable framework for integrating generative AI into specialised translation workflows.