Question-answering systems as efficient source of terminological information: evaluation Olvera Lobo, María Dolores Gutiérrez Artacho, Juncal Decision support techniques Evaluation studies as topic Information storage and retrieval Natural language processing MedQA START Background:  Question-answering systems (or QA Systems) stand as a new alternative for Information Retrieval Systems. Most users frequently need to retrieve specific information about a factual question to obtain a whole document. Objectives:  The study evaluates the efficiency of QA systems as terminological sources for physicians, specialised translators and users in general. It assesses the performance of one open-domain QA system, START, and one restricted-domain QA system, MedQA. Method:  The study collected two hundred definitional questions (What is…?), either general or specialised, from the health website WebMD. Sources used by the open-domain QA system, START, and the restricted-domain QA system, MedQA, were studied to retrieve answers, and later a range of evaluation measures (precision, Mean Reciprocal Rank, Total Reciprocal Rank, First Hit Success) were applied to mark the quality of answers. Results:  It was established that both systems are useful in the retrieval of valid definitional healthcare information, with an acceptable degree of coherent and precise responses from both. The answers supplied by MedQA were more reliable that those of START in the sense that they came from specialised clinical or academic sources, most of them showing links to further research articles. Conclusions:  Results obtained show the potential of this type of tool in the more general realm of information access, and the retrieval of health information. They may be considered a good, reliable and reasonably precise alternative in alleviating the information overload. Both QA systems can help professionals and users can obtain healthcare information. 2013-03-12T13:32:59Z 2013-03-12T13:32:59Z 2010-12 info:eu-repo/semantics/article Olvera Lobo, M.D.; Gutiérrez-Artacho, J. Question-answering systems as efficient source of terminological information: evaluation. Health Information and Libraries Journal, 27(4): 268-276 (2010). [http://hdl.handle.net/10481/23990] 1471-1842 1471-1834 doi: 10.1111/j.1471-1842.2010.00896.x http://hdl.handle.net/10481/23990 eng http://creativecommons.org/licenses/by-nc-nd/3.0/ info:eu-repo/semantics/openAccess Creative Commons Attribution-NonCommercial-NoDerivs 3.0 License Health Libraries Group