Dealing with problems involving natural language and structured tables has become an important task in building smarter and useful AI systems. These systems are generally expected to process content including various data types, such as text mixed with numerical tables, which are often found in commercial documents, research papers, and public reports. Understanding such documents requires AI to perform reasoning covering text interpretation and table-based details, a process that is inherently more complex than traditional text-based questions answering.
One of the main problems in this field is that current language models often fail to interpret documents accurately when it comes to tables. When flattening a table into plain text, the model often loses the relationship between the lines. This distorts the basic structure of the data and reduces the accuracy of the answer, especially when the task involves computing, aggregation, or reasoning that connects multiple facts throughout the document. This limitation allows the use of standard systems for practical multi-hop questioning tasks that require text and table insights.
To address these problems, previous methods have tried to adopt retrieval enhanced generation (RAG) techniques. These involve searching text segments and feeding them into a language model for answers. However, these techniques are not sufficient to require tasks that constitute or global reasoning in large tabular datasets. Tools like Naiverag and TableGPT2 try to simulate this process by converting tables to Markdown format or generating code-based execution in Python. However, these methods are still difficult in maintaining the original structure of the table as necessary to correctly interpret the tasks.
Researchers at Huawei Cloud BU proposed a method called Tablerag that directly addresses these limitations. Research uses TableRag as a hybrid system that alternates between text data retrieval and structured SQL-based execution. This approach preserves the table layout and treats table-based queries as a unified inference unit. This new system not only preserves the table structure, but also performs queries in a way that respects the relational nature of the data, which is organized in rows and columns. The researchers also created a dataset called HeTeQA to benchmark the performance of methods on different domains and multi-step inference tasks.
Tablerag works in two main stages. The offline phase involves parsing heterogeneous documents into a structured database by extracting tables and text content separately. These are stored in parallel corpus, a relational database for tables and are the basis of block knowledge for text. The online stage deals with user problems through iterative four-step process: query decomposition, text retrieval, SQL programming and execution, and intermediate answer generation. When a question is received, the system identifies whether it requires table or text reasoning, dynamically selects the appropriate strategy and combines the output. SQL is used for precise symbolic execution, with better performance in numerical and logical calculations.
During the experiment, TableRAG was tested in several benchmarks, including Hybridqa, WikableSquestions and the newly created Heteqa. Heteqa consists of 304 complex problems in 9 different fields, including 136 unique tables and over 5300 Wikipedia-derived entities. Datasets challenge models such as filtering, aggregation, grouping, computing, and sorting tasks. Tablerag performed better than all baseline methods, including Naiverag, React, and TableGPT2. It always achieves higher accuracy, has document-level inference with up to 5 iteration steps, and utilizes models such as Claude-3.5-Sonnet and Qwen-2.5-72b to verify the results.
This work provides a powerful and structured solution to the challenge of reasoning on mixed-format documents. By maintaining structural integrity and using SQL for structured data manipulation, the researchers demonstrated an effective alternative to retrieval-based systems. TableRag represents an important step forward in a questioning system that processes documents containing tables and text, providing a viable way to more accurate, scalable and interpretable document understanding.
Check Paper and Github page. All credits for this study are to the researchers on the project. Ready to connect with 1 million+ AI development/engineers/researchers? See how NVIDIA, LG AI Research and Advanced AI companies leverage Marktechpost to reach target audiences [Learn More]
Nikhil is an intern consultant at Marktechpost. He is studying for a comprehensive material degree in integrated materials at the Haragpur Indian Technical College. Nikhil is an AI/ML enthusiast and has been studying applications in fields such as biomaterials and biomedical sciences. He has a strong background in materials science, and he is exploring new advancements and creating opportunities for contribution.
