eWEEK content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.
A recent survey commissioned by data integration platform provider Xplenty indicates that nearly one-third of business intelligence (BI) professionals are little more than “data janitors,” as they spend a majority of their time cleaning raw data for analytics.
The study focused on several areas of the Extract, Transform and Load (ETL) process, including preferences for on-premises or cloud-based solutions, perceived challenges, and the amount of time spent on ETL. Ninety-seven percent of those surveyed said ETL was critical for their business intelligence efforts.
Moreover, more than half (51 percent) of BI professionals polled said that they currently leverage on-premises ETL solutions, versus 49 percent using cloud-based tools. However, of those who said they presently use on-premises ETL tools, 51 percent said that they were “strongly considering” moving all ETL processes to the cloud.
ETL is the process of extracting data from homogeneous or heterogeneous data sources, transforming the data for storage in the proper format or structure for querying and analysis purposes, and loading the data into the final target – such as a database, data store, data mart, or data warehouse.
“While many organizations still rely heavily on existing on-premises IT for ETL, the desire to shift to a more cloud-based model has never been stronger,” said Yaniv Mor, CEO and co-founder of Xplenty, in a statement. “Cloud ETL offers a host of benefits over on-premises, from increased agility in resource deployment to reduced costs. As such, the cloud is an increasingly attractive option from both a performance and operational perspective.”
When asked what the biggest challenges were in making data “analytics-ready,” 55 percent of respondents said integrating data from different platforms, followed by transforming, cleansing and formatting incoming data (39 percent), integrating relational and non-relational data (32 percent), and the sheer volume of data that needs to be managed (21 percent) at any given time.
“Reformatting, cleansing and consolidating large volumes of data from multiple sources can be overwhelming,” Mor said. “BI professionals are still struggling with the best approach to shorten the time between integration and analytics. As a result, businesses are often slow to unlock their data’s true potential for revenue or operational improvements.”
Meanwhile nearly one-third of respondents (30 percent) said that they spend between 50 percent and 90 percent of their time just on ETL.
“BI professionals should be spending the majority of their time evaluating data and deciphering patterns gleaned through the analytics process—not readying data for analytics,” Mor noted. “The more time they spend making raw data analytics usable, the less time they have to generate real value from it. We have to accelerate big data’s ‘time-to-insight,’ boosting efficiency and bringing more immediate answers to an organization so that they can more quickly take advantage of them.”
The Xplenty survey was aimed at understanding the challenges business intelligence professionals face in preparing raw data for analytics. For the study, more than 200 BI professionals from across the U.S. were polled between May 1 and May 11, 2015. Xplenty’s data integration platform enables companies to make raw data “analytics-ready” for analytics on the cloud, the company said.