Is there a future for business intelligence?

AI IN ‘THIRD AGE’ OF BUSINESS INTELLIGENCE WILL BE REAL GAME-CHANGER SAYS ECKERSON

After 30 years, business intelligence is rapidly evolving into a third age that could provide unequalled insights into data. Developments in software tools by an increasing number of providers now offer the ability and potential to unlock huge, previously untapped value – thanks to artificial intelligence (AI) and machine learning (ML). That was the conclusion from a presentation to the Data Vault User Group by Wayne Eckerson – one of the world’s most respected experts on business intelligence (BI) and data analytics. What passed as the original development of business intelligence, in the early 1990s, has evolved – via the self-service era – to a point now where the industry stands at a crossroads he revealed. Wayne’s Eckerson Group’s experience in BI extends to developing strategies and design work to create modern data programs and platforms. His discussion – which can be viewed here – started by asking the question: ‘Is There a Future for Business Intelligence?’ However, he finished by claiming the advent of the application of AI and ML – in what he termed the third “model-driven” era of Business Intelligence would be “game-changing.” Business intelligence has evolved into augmented intelligence with human actions assisted by input from AI models.

Will ChatGPT be the real future?

All this before the revolutionary development of ChatGPT impacts, on the switch to machine-generated reports, which focus on the future rather than the past or even the present. With research and experiments already working on ways for ChatGPT to be converted from text queries and answers to tabular formats and displayed in a BI tool, Wayne pondered a future where ChatGPT might be able to explore databases through connectors and then enable users to ask questions – and not have to wait for the data analytics team come up with the answers. Wayne said: “We don’t know the full extent of the power of this tool for our space. Obviously, it doesn’t always get it right – there is a lot of crud out there in the internet – but there is also a lot of crud in our databases too.” The paradigm shift in the marketplace over the last couple of years has come at a time when many boardroom executives are questioning business intelligence’s value – one influential industry group even went so far as stating that “Dashboards are dead!”

Seven Symptoms of BI’s ‘Last Mile Problem’

While Wayne would not go as far as that, he admitted that there were seven symptoms limiting BI’s effectiveness, ranging from it being too reactive – only looking at past results – too generalised, and too hard to use – with too many manual processes in an area increasingly dominated by automation. Failing to turn insights into action, only fuels the nagging question – are we getting sufficient value from BI? It also confirms the belief that business intelligence has a “last-mile problem,” said Wayne. The Eckerson Group is known worldwide for is research in the Business Intelligence marketplace, and their own surveys – as well industry-wide ones – have confirmed that the percentage of active BI uses has remained stubbornly low at between a fifth and quarter off all employees. Part of that problem – if it is in fact a real issue, Wayne argued – is the categorisation of those users. Wayne factored the question by dividing the total number of users into four key groups – data consumers, data explorers, data analysts and data scientists. In the case of data consumers and data explorers – who make up what Wayne called “business users” and represent the biggest chunk of the workforce – they require a “silver service” with their own needs to view and enrich reports respectively. Meanwhile the data analysts and data scientists– have self-service requirements. The data analysts create reports from scratch, armed with a set of workbench analytical tools augmented by collaborative intelligence, while the data scientists rely increasing on machine learning ops (MLOps) in a more academic and mathematical approach to create predictive models.

Can ChatGPT replace data analytics?

The latest stage of business intelligence’s evolution sees it going from augmented intelligence – where humans take actions assisted by input from AI models and machine learning to suggest possible actions which human judgement moderates in the business decision – to autonomous intelligence, where all models act independently with the human input taking place up front. In little more than a couple of generations since the early ’90s, it’s a long way from the process of human intelligence, often based on intuition and personal experience, to the first forms of business intelligence where data and dashboards guided human judgements and business decisions, along with the future of the market. Now, with real-time data streams increasingly the norm, Wayne believes the big question is, how far and how quickly can ChatGPT 3 influence developments, and even become the interface for analytics? And if so, how quickly? Or will it eventually replace data analytics full stop? Business monitoring programs produced by the likes of Outlier, ThoughtSpot, Amazon Quick Sight, Qlik and Yellowfin, can already monitor and correlate business metrics on 10 times the scale of a traditional dashboard. The ability to create personalised and automated alerts on critical data – using metrics that ML can use to designate baselines that help identify significant deviations and even suggest root causes – is behind the new revolution. Business monitoring is the equivalent of hiring an army of data analysts, making the real ones, up to 100 times more productive, Wayne believes, helping to point data analysts and data scientists in the right direction.

Throwing the kitchen sink at BI

You can now have a data analyst workbench that covers all four stages from discovering data via the data catalogue and business glossary, to data preparation and virtualisation, through to analysing with automated ML tools before reporting the data with visualisation tools to promote, distribute and share the results and insights created. BI software developers are now throwing the kitchen sink at the situation – adding everything from data preparation and ingestion tools along with data analytics and data science functions, monitoring past performance and predicting the future and recommending courses of action, evaluating results along the way. The alternative to going deep is to move into a broad sweep focusing on decision intelligence, but it is the former that the herd tend to be following in their race to differentiate themselves from Microsoft’s Power BI and Tableau. In conclusion, Wayne would not rule things out as much as ruling things in when it comes to the next developments in business intelligence with AI now being built into BI programs. All round Wayne’s observations made for one of the most thought-provoking presentations in the DVUG’s relatively short history – and returning to it in another five to ten years’ time should be fascinating to see where BI eventually ends up.