New features in the AutomateDV Data Vault automation tool

Unwrap AutomateDV new features this Christmas

If you are seeking solutions for your digital transformation project, a Christmas present that can help you build a data warehouse in the Cloud – quickly and cheaply – has just been added to Santa’s sleigh for distribution around the world on December 24th. Developers at Datavault have been working over the last few months on the latest version of their AutomateDV tool for the generating a data warehouse utilising the Data Vault 2.0 method. Alex Higgs, Datavault’s lead developer for the tool, explained the latest developments at the December online meeting of the Data Vault User Group. They are as ever aimed at automating as much of the process of creating a Data Vault as possible and testing it to the nth degree to avoid errors, making the whole process as simple as possible. The AutomateDV tool is based on the open source dbt tool, offering powerful templating to provide the SQL code needed to create the business layer of the vault using Jinja. Because it is highly extensible, custom packages can easily be built and overall dbt augments a developer’s workflow thanks to its rapid iteration and extensive testing capability. And by adding the AutomateDV tool, you can automate the creation of a Data Vault. With a bit of help from Santa’s elves, the latest version of AutomateDV has been unveiled this week – including point-in-time tables (PITs), bridge tables and extended tracking satellites, plus multi-active satellite improvements. The latter two features make the process of replaying loads and rebuilding the source system easier. As with most of the other release versions over the last two years, there have been more bug fixes – aimed at achieving composite primary key consistency, and making incremental loads more consistent with better test coverage. The PIT (point-in-time) tables provide a snapshot over any satellites attached to hubs or links in your data warehouse, and can span across multiple satellites, offering a query helper table for the business vault. Logic is also built in to ensure time variation is accounted for those elements of the data that arrive at different times – the PIT tables also offer equi-join capabilities. The bridge tables offer the same benefits, also with the means to join multiple hubs and links together for reporting performance, while also providing a snapshot of key structures using primary and business keys. And the good news Alex and the team are working on planned improvements for version 0.7.10 of AutomateDV which will concentrate on more improvements to effectivity satellites and more PIT and Bridge tables functionality. Alex said in his presentation: “Effectivity satellites currently require a lot of metadata and can be confusing to implement and use. “Currently, there are too many columns and the naming is confusing. In future releases we will make that significantly simpler with less metadata and more logical column names.” The ‘business effective date name’ can be equally confusing – AutomateDV 0.7.9 currently has ‘effective from,’ in future it will be applied date, he added, while a a Boolean flag column to indicate an active state will replace the ‘start date’ and ‘end date’ columns. Future releases will focus on meeting requests from many in the AutomateDV community to make it compatible with other platforms.