big-qa-architecture - Release v1.1.1

Paper: 📄 BigQA: A Software Reference Architecture for Big Data Question Answering Systems, 2024 Abstract: Querying massive and heterogeneous text data is challenging, transcending different business domains. Our study outlines the BigQA architecture, which is specifically designed to support text data queries on Big Data systems using natural language. The architectural design comprises several layers that are intentionally built to be independent of the programming language, technology, and querying algorithm utilized. Nevertheless, the implementation of this architecture remains unclear. In this study, we showcase the versatility and adaptability of BigQA by offering a comprehensive set of guidelines and three practical implementation pipelines. In addition, we performed 60 experiments on four different datasets and compared the recall results of three popular algorithms: BM25, TF-IDF, and DPR. Based on our experiments, BM25 had the best overall performance as a document query algorithm.

• We outline the design principles.
• We propose and design the BigQA architecture.
• We performed several use cases and experiments.

Leonardo Mauro Pereira Moraes, Pedro Calciolari Jardim, and Cristina Dutra Aguiar. BigQA: A Software Reference Architecture for Big Data Question Answering Systems. Lecture Notes in Business Information Processing, vol 518. Springer, Cham, 2024.

```tex

BigQA: A Software Reference Architecture for Big Data Question Answering Systems

@article{moraes:2024:big-qa-architecture, author = {Leonardo Mauro Pereira Moraes and Pedro Jardim and Cristina Dutra Aguiar}, title = {{BigQA}: A Software Reference Architecture for Big Data Question Answering Systems}, year = {2024}, booktitle = {Enterprise Information Systems}, publisher = {Springer Nature Switzerland}, issn = {1865-1348}, isbn = {978-3-031-64748-2}, pubstate={forthcoming}, journal = {Springer Lecture Notes in Computer Science}, pages = {42--65} } ```

- Jupyter Notebook
Published by leomaurodesenv 12 months ago