The future of Business Intelligence

At the 2024 Data Vault Conference, Neil Strange delivered a thought-provoking session on the future of Business Intelligence (BI) in an era dominated by AI, automation, and ever-evolving data platforms. If you missed it, here’s your friendly recap—and a reminder to join our forum to keep the conversation going!

Is There a Future for BI?

The session opened with big questions: Are robots coming for our jobs? What does the business really need from BI in the future? What will our workdays look like as AI evolves? Neil Strange set the stage for a lively exploration of where BI is headed.

AI’s Impact: A New Epoch

AI is already reshaping the business landscape, and competition is now “AI vs. AI.” The pace of change is accelerating, and we’re all living through a technological Richter scale—where some innovations are mere tremors, and others (like AI) could be seismic, changing the course of human history. But don’t worry: the robots aren’t ready to take over just yet

The Prize: What’s at Stake?

Generative AI (GenAI) promises huge business benefits:

• 54% increase in revenue performance

• 44% faster time to market

• 62% improved client satisfaction

• 54% increase in profitability

But to unlock these gains, BI must embed itself within the business, becoming invisible yet essential—powering everything from dashboards to AI-driven agents and assistants.

The Data Platform: Invisible, Essential

The future data platform will be invisible to users, seamlessly connecting operational systems (CRM, ERP, SEO, etc.) and enabling AI/Chat, dashboards, and self-service analytics. The platform will learn from, assist, and even direct business operations—essentially becoming another operational system itself.

Challenges Ahead

To get BI right in this new world, organizations must tackle:

• Maintaining a business focus (value-driven)

• Bitemporality (knowing what was known, when)

• Explainability and AI ethics

• Data quality and coverage

• Information governance

• Schema drift and data lineage

Traditional reporting and analytics aren’t going away—85% of use cases are still structured reporting, with the rest in data science, IoT, and unstructured data.

Methods Matter: Data Vault, Data Mesh, Medallion

All methods transform raw data into value, but differ in:

• How much work remains for users

• Scope (platform, modeling, development)

• Technical vs. business focus

• Scalability and organizational principles (centralized, decentralized, federated)

• Retention of historical data and replayability

• Maintainability, testability, and protection against poor data quality

• Performance, parallelism, and automation

Data Vault stands out for its integration, auditability, scalability, and agility—addressing tomorrow’s challenges and supporting federated, business-oriented data modeling. Data Mesh and the Medallion approach are also making a big impact, emphasizing domain ownership, data products, and computational governance.

The Future BI Team: Small, Agile, and Productive

A Data Vault team is typically small (3–5 data engineers per squad), but highly productive—combining data engineering, modeling, platform engineering, and security. Teams work in domains, supported by centers of excellence and platform teams, with oversight and governance baked in.

How Will Your Day Change?

• Automation: Data profiling, ETL, visualization, analytics, and documentation will be increasingly automated.

• AI Support: AI will generate ideas, assign work, assess quality, and even coach and reward team members.

• Business Liaison: Human judgment remains essential—AI-generated ideas will still need human evaluation.

• Error Detection: Preventative maintenance and performance tuning will be AI-assisted, but humans will recover from failed loads and ensure quality.

Join the Data Community!

• Sign up to our forum: Share your experiences, ask questions, and connect with fellow data enthusiasts.

• Stay up to date: We host regular webinars, workshops, and meetups—don’t miss out!

• Shape the future: Your feedback and participation help us build better tools and resources for everyone.

Final Thoughts

The future of BI is bright, but it’s changing fast. With the right mix of automation, AI, and human expertise, BI will become more embedded, more valuable, and more essential than ever. Ready to shape the future of BI? Join the conversation, sign up for our next webinar, and let’s build the future of data together!