Data volume to hit 1.8ZB in 2011

Worldwide information is more than doubling every two years, with 1.8 zettabytes or 1.8 trillion gigabytes projected to be created and replicated this year alone, according to a new study released Thursday.

Sponsored by storage vendor EMC and conducted by research firm IDC, the study found that in terms of volume, this amount can fill up 57.5 billion 32GB Apple iPads or is equivalent to every person in Singapore posting three tweets per minute for 1,670,000 years non-stop.

According to the study, enterprise investments in the this "digital universe" have increased 50 percent to US$4 trillion since 2005. EMC defines such investments to include cloud, hardware, software and services, as well as the manpower needed to create, manage, store and generate revenue from the information.

"Big data is forcing change in the way businesses manage and extract value from their most important asset--information," said Jeremy Burton, chief marketing officer of EMC, in the report.

IDC noted that the skills, experience and resources required to manage the flood of data and resources were not keeping pace with staffing.

And the data growth is expected to escalate, pushing up the volume of systems administrators will need to manage.

In fact, the research firm noted that by 2020, IT departments worldwide will need to administer 10 times the number of servers--both virtual and physical--50 times the amount of data, and 75 times more files which will be driven by the use of embedded systems including sensors. The number of IT administrators needed to manage this will grow by 1.5 times, IDC estimated.

In addition, while cloud computing still accounts for 2 percent of today's IT spending, IDC estimated that, by 2015, nearly 20 percent of information will be "touched" by cloud computing service providers and as much as 10 percent will be maintained in a cloud.

It identified technology and money as the forces driving this data growth. And while "information taming" technologies had driven down the cost of creating, capturing, managing and storing information to one-sixth of what it was in 2005, the digital universe's growth will continue to outpace that of storage capacity.

These new technologies which are increasingly dealing with real-time data can help organizations gain insights from their unstructured data, which accounted for more than 90 percent of the digital universe, IDC noted, adding that they would also aid in cutting costs such as deduplication, auto-tiering and virtualization.

However, while the data growth can be overwhelming, there is a silver lining. Burton said: "The chaotic volume of information that continues growing relentlessly presents an endless amount of opportunity, driving transformational societal, technological, scientific and economic changes."