Dan Linstedt: The Future of Data Vault

Hear Dan Linstedt, the creator of the Data Vault method, giving his predictions for the future of Business Intelligence and Analytics

The UK Datavault User Group attracted its largest-ever online audience thanks to January’s presentation by Data Vault founder Dan Linstedt. The creator of the original Data Vault methodology, gave a fascinating 45-minute presentation on ‘The Future of Data Vault.’

You can watch the whole presentation by clicking the video above – it really is worth taking the time to listen to what Dan had to say – including his seven predictions for the next five years.

He started with a quick recap of the events of 2020 – as businesses and economies across the whole world had to react – and adapt – to the challenges of the COVID-19 pandemic.

With workers at every level in almost every company and organisation being forced to work from home, technology and data took centre stage. Dan highlighted that real-time streaming of data became increasingly important – none more so than in health systems fight to track COVID cases and infection rates, as well as sadly mortalities, followed by all the research into vaccines which will hopefully restore some normality to life as 2021 wears on.

The Eckerson Group and predicted that 2020 would become the year of Data Vault – and Linstedt, who refined his original ideas and introduced Data Vault 2.0 seven years ago, agrees the method is gaining increasing interest from the market.He believes the current trends in the market make the method as relevant as ever.

He pointed to how information governance – the process of creating effective data management throughout any enterprise or organisation – has become increasingly a topic of conversion in the world of data analytics and business intelligence.Furthermore, over the last 12 months organisations are facing increasing concerns about meeting security and privacy obligations.

Dan believes – along with many leaders in the data world – that teams operating without automation are finding it nigh on impossible to compete effectively, as the legacy of old-style data warehouses and business intelligence systems are restricting organisational agility more and more.

It is also true that the increasing migration of data to the cloud means historical industry paradigms are breaking down.

But enough of the past… what about the future?

Well Dan is concerned about the legal and ethical implications of artificial intelligence – and algorithms based on stored historical data in particular – making crucial decisions in everything from healthcare to financial investments and getting them wrong.

Who will pick up those bills in an increasingly litigious world?

But more importantly, Dan focused on his five-year vision, where he sees the Data Vault 2.0 methodology as critical to increased automation, which relies on repeatable best practices and makes seven predictions covered people, process and technology.

Human engineering – amid increasing use of robotics, machine learning and AI – will remain critical, along with collaboration by data engineers, architects and ontologists speaking the same language, using the same technology to agree on standards in architecture and metadata, which are key to creating agile and scalable data solutions.

Automation will be critical for anyone building a Data Vault with correct standards and patterns to ensure repeatable best practices, avoiding lengthy and costly delays and revisions to any data platform project.

The use of hybrid architecture – using the Hadoop cluster – in conjunction with hybrid systems will be key in Business Intelligence and analytics solutions remaining successful across the planet, Dan predicts.

Privacy and security layers will be built into both the hardware and software platforms going forward to aid governance.

Finally, the sharing of data – across sectors and between stakeholders – will grow and grow in the second decade of the third millennium, and those platforms, such as Snowflake, which enable such sharing will thrive, according to Linstedt, who has spent 30 years helping clients across many industries globally in the public sector including the USA’s defence and security community.

Look out for updates on the upcoming speakers the UKDVUG will be hosting in the coming months – sign up for our newsletter on our home page here.