Table of Contents

Capabilities and Benchmarking


Introducing NBI.AI-1: a Specialized Generative BI Model

Enhancing Generative AI for Data Analytics with Hybrid LLM/Rule-Based Approach

Introducing NBI.AI-1

We’re sharing the new research and NBI.AI-1, the latest milestone in Narrative BI’s effort to democratize data insights.

The research by Narrative BI focuses on hybrid approaches to generating business data insights from structured data, combining the strengths of rule-based systems and Large Language Models (LLMs). Rule-based analytics systems are precise but lack adaptability. LLMs are great for pattern recognition but are too generic for specific business cases. We're introducing:

  1. A new hybrid approach that leverages the strengths of both worlds to enhance insight generation. Narrative BI's research explores how integrating these approaches can enhance the extraction of actionable business insights while mitigating LLM hallucinations.
  2. A new specialized model (NBI.AI-1) trained on proprietary data.

The research outlines a hybrid approach that leverages the robustness of rule-based systems with the adaptive power of LLMs. By doing so, we aim to improve the process of data extraction and uncover meaningful data insights from diverse data sources using advanced AI data analysis techniques. The hybrid method combines AI techniques with rule-based systems and supervised document classification, creating a powerful framework for business data analysis. LLMs play a crucial role by modeling linguistic characteristics and generating coherent responses, thereby uncovering personalized user interests, needs, and goals from user journeys and activities.

Key considerations for implementing a hybrid approach include ensuring high data quality, understanding the specific business domain, and having sufficient computational resources. The research highlights the importance of maintaining transparency and trustworthiness in the data extraction process.

Capabilities and Benchmarking

The hybrid approach's effectiveness was benchmarked against purely rule-based and LLM data analytics methods. The results demonstrated that the hybrid model offers a balanced solution, leveraging the precision of rule-based analysis and the flexibility and depth of LLM-generated data insights. This integration enhances the quality of data insights generated, ensuring they are actionable and accessible to decision-makers.

The data used for the benchmarking was collected from 30 corporate Google Analytics 4 and Google Ads accounts via APIs for a time frame of approximately two years.

Data Extraction

In this evaluation, we extracted and calculated business metrics (such as cost-per-click, new users, website sessions, etc.) for the required period using different methods: rule-based query builder, AI answer generator (built with ChatGPT API), and Hybrid Approach (AI Data Analyst based on the NBI.AI-1 model).

Processing pipeline type Precision
Rule-based 100%
LLM (GPT-4) 63%
Hybrid (NBI.AI-1) 87%

Business Data Insights Generation

We evaluated different methods of extracting and presenting all relevant business data insights from the dataset, a crucial factor for comprehensive analysis.

Processing pipeline type Recall
Rule-based 71%
LLM (GPT-4) 67%
Hybrid (NBI.AI-1) 82%

Automated Reporting

User satisfaction can be influenced by factors like the accuracy of the information, the relevance, comprehensiveness, and readability of the reports provided. This metric is measured as the ratio of “likes” (reports marked as “helpful” by business users)  to “dislikes” (marked as “not helpful”). The bigger the number, the higher the overall user satisfaction.

Processing pipeline type Likes-to-dislikes ratio
Rule-based 1.79
LLM (GPT-4) 3.82
Hybrid (NBI.AI-1) 4.60

Hallucination Mitigation

LLM systems are prone to data hallucinations: they confidently generate responses that look plausible, but that are entirely incoherent or inaccurate. We measured the percentage of data hallucinations by comparing responses to actual data in the dataset.

Processing pipeline type Error rate
Rule-based 0%
LLM (GPT-4) 46%
Hybrid (NBI.AI-1) 3%


In conclusion, Narrative BI's hybrid approach offers a more dynamic and precise tool for business analytics. Our findings suggest that the hybrid approach not only enhances the precision of data insights but also improves their contextual relevance, making the AI data analysis process more comprehensive and actionable. By combining the strengths of rule-based systems and LLMs, the hybrid model addresses the limitations of each method when used independently. This research provides a foundation for developing more resilient and insightful practices of using Generative AI for data analytics, driving growth and innovation in today's data-driven business environment.

Share on

Facebook logo
LinkedIn logo
X logo

Related articles

Resources for data-driven founders & growth leaders

Learn how to turn your data into a powerful asset that helps you achieve mission-critical goals.
By signing up, you agree to our
Privacy Policy
Terms of service.

Thank you for your interest!

Please leave your email address to learn more about Narrative BI and be the first to try our platform.
Narrative BI Close button
Thank you!
We’re so glad you’re interested in seeing Narrative BI in action
Narrative BI Close button

Thank you for your interest!

Please leave your email address and we will get back to you to learn more about your specific needs.
Narrative BI Close button