Free tools to drive big data adoption

Big data vendors are releasing more free tools with which companies can experiment, thus, increasing the knowledge and skills needed to utilize these software as well as improving how analytics are being conducted today.

Philip Carter, vice president of IT services, application software, channels, green IT and sustainability at IDC Asia-Pacific's practice group, said organizations are only recently beginning to discuss big data and creating an information management strategy that incorporates such big data tools into their IT infrastructure.

Carter noted in an e-mail interview that free big data software, or "freemium", entering the market will yield two main benefits. First, developers and analysts will find it easier to experiment with emerging types of data structure that big data represents. In turn, these groups of people will begin to develop new and different analytical procedures, he added.

Second, the overall IT community will stand to gain as developers and IT professionals contribute their findings and know-how back into the industry to drive knowledge exchange, he said.

A host of free big data tools was recently released to the public, including those offered by Karmasphere and EMC.

Data warehousing specialist, Teradata, also think that such freemium will help drive market adoption. Noel Pettitt, its South Asia vice president, told ZDNet Asia that the company believes open source and "try-and-buy" offerings help customers evaluate and determine the best option for their business.

Pettitt said in an e-mail interview that customers use these tools "extensively" for development projects, proof-of-concepts and testing. These free big data tools save them money and allow them to work untethered from a server since it can also run on laptops, he elaborated.

"[However,] depending on the actual size of the data volume and mining activities needed, customers may or may not be able to use these freemium tools, because large-scale solutions are not well served by tools that scale poorly," he noted.

Bernard Spang, director of strategy and marketing for database software systems, IBM's software group, revealed that the company's customers are using free big data software for trial, development and even production use.

Spang said noted that most of its clients have started asking for big data tools to better help them analyze the growing volume of information flowing into their corporate environment, but keeps them using these tools is the ability to handle the variety of information.

Another pressing need companies ask for is flexibility, in particular, the ability to experiment with data quickly and at low cost and effort, he said.

"As existing warehouse and master data management systems become mission-critical, they lose some of their flexibility as these systems cannot be changed on a hunch or to test a hypothesis," he explained, adding that IBM's InfoSphere BigInsights can provide the flexibility to manage both purposes.

However, Carter noted that adoption of big data is currently inhibited by a shortage of sophisticated in-house skills to perform the new types of analysis required. Companies also still do not recognize the benefits of analyzing big data, he added.

That said, because of the exponential increase of data that is expected over the next few years, big data tools will definitely see strong demand, the analyst predicted, noting that IDC has been tracking the amount of digital information created and replicated each year, called "Digital Universe", since 2007.

In 2009, he said the Digital Universe grew to 800 petabytes and the research firm expects the amount of information generated to grow 44 times by 2020, he noted.

"There will be an increasing need to analyze this exponential growth of data, which is why we expect the demand for big data analytics tools to grow significantly in 2011," Carter said.

An earlier ZDNet Asia report noted that as businesses inevitably become more social, whereby organizations take a proactive stance in approaching customers with problems instead of them engaging companies, is also fueling demand for big data analytic.