eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
We’re experiencing an uncanny renaissance. Databases, long derided as antiquated, are suddenly the darling of the industry. Years after pundits declared databases all but dead, a new breed of startups has captured Wall Street’s attention. However, IT leaders still struggle to efficiently migrate their existing workloads to these systems.
The vendor lock-in on databases is legendary. No other sector wields this much influence over their users. Naturally, vendor lock-in keeps customers beholden to the existing technology. However, it also keeps the competition and upstarts at bay. Snowflake’s CEO famously lamented about how hard it is to break into this market when he said: “Teradata has [made] it bloody hard to move off their platform.”
With the advent of database virtualization (DBV) – not data virtualization – a new methodology has entered the arena. DBV claims to break vendor lock-in on data warehouse technology. How applicable is it and how can IT leaders evaluate the technology? Here are five things to know about DBV.
Also see: How Database Virtualization Helps Migrate a Data Warehouse to the Cloud
1) How Does DBV Work?
A DBV platform sits between database and applications. It enables applications written for one database to run natively on a different one. All queries and communication are translated in real-time. For example, applications written for Teradata can run directly on Microsoft Azure Synapse and won’t even “know” they are no longer running on Teradata.
DBV is fully transparent with the aim that applications wouldn’t require any or only minimal adjustments. This includes not only standard SQL but also proprietary extensions. And to be successful in practice, loaders, drivers, and utilities must also be supported.
Since DBV systems implement the translation of queries and data, they can operate with rather low overhead. The actual data processing is always performed on the data warehouse itself and leverages the massively parallel processing (MPP) capabilities of that system.
2) When to Use DBV Over Conventional Migration?
In a conventional migration, all existing SQL code, drivers, tools and utilities are replaced with their counterparts of the new destination system. For compact data warehouse systems with a small number of applications, this may be the preferred approach. Data marts with only by a limited number of users may qualify for this.
However, in the case of complex Enterprise Data Warehouses (EDW), DBV can outperform conventional migration significantly. DBV accomplishes the migration of workloads at a fraction of the time, the cost, and the risk.
3) Can DBV Cover My Workload?
Some workloads use specialized features extensively. Yet others use features that pre-date standardization efforts. In other words, no two data warehouse workloads are the same. This can make it difficult to gauge the coverage of a DBV system.
A full documentation of supported features seems desirable but is actually not very helpful. Most customers cannot succinctly describe which features their workloads are currently using. To make matters more complicated, often the original authors of queries or functions are no longer with the company.
This need not be an obstacle to the adoption of DBV, however. Since DBV comes with a low risk of adoption, it makes for very effective proof-of-concept (POC) implementations. Customers do not need to alter their applications in order to use DBV. They can test their actual applications directly in a POC.
A POC then can identify any missing coverage quickly. It is important to understand, while a 100% coverage seems desirable, high 90’s is usually enough. Taking care of the remaining issues often requires only negligible effort.
Also see: Top Business Intelligence Software
4) How Does DBV Differ from Data Virtualization?
Data virtualization is a somewhat related, yet quite different approach. For data virtualization to be successful, all applications need to be rewritten first and adopt an abstract SQL dialect. Then, and only then, does it prevent future vendor lock-in. The primary application areas are “green field” scenarios which do not need to take existing applications into account.
In contrast, DBV breaks existing vendor lock-in. It leaves applications as-is. Over time this may lead to a varied assortment of different application technologies. However, this may not be much of a concern and is well offset by the uptake of new data technology.
5) Can DBV Replatform EDWs to Any Technology?
Every few years, a new technology challenges the supremacy of enterprise data warehouse systems. Often the newcomer outperforms the existing stack in one notable dimension. For example, the new technology might be more scalable. Another may simplify the sharing of data. Yet other appeals more to open-source developers.
Typically, expert teams can build bespoke solutions to move workloads from the EDW to most new technology. However, the bigger the gap in functionality, the more software engineering is required to operate these systems. For example, moving an EDW to a NoSQL system maybe technically feasible, but is economically not always advisable.
For DBV to be successful, source and destination must be meaningfully similar. However, this does not require equivalent functionality. DBV can compensate for the lack of most advanced features like stored procedure, macros, or even unsupported data types. Currently, cloud-native PaaS solutions are most successful in the context of DBV.
Choosing the Right Approach for Migrations
When choosing a new destination system for an existing EDW, many factors need to be considered. One often overlooked is that of the migration approach. DBV is highly effective at breaking vendor lock-in on legacy data warehouses.
By factoring the migration approach into their choice of the destination system, IT leaders can optimize for fast uptake while controlling risk and cost.
Also see: Tech Predictions for 2022: Cloud, Data, Cybersecurity, AI and More
About the Author:
Mike Waas is the founder and CEO of Datometry