All types of business – from retail to financial to telecommunications – have to familiarize themselves with the term compliance. Whether they are keeping billing information and order histories, transactional data or detailed call logs, government regulations force businesses to store tons of data and logs for over years or even decades.

“For many businesses log management becomes like organizing a huge library – just too overwhelming,” officials from Cirtas (News - Alert), a leading cloud storage provider, wrote in a recent blog post.

“Data stored to comply with regulations and to secure the organizations IT infrastructure isn’t the data that the company accesses frequently in order to operate, but that doesn’t mean that companies don’t produce a lot of it,” the blog states. “Many companies produce tens of gigabytes or even a terabyte of this data per day. Data is generated rapidly and consistently, like a faucet that never shuts off.

To make an otherwise daunting task more manageable, companies can deploy security information and event management solutions (SIEM) like Arcsight, Splunk (News - Alert) and RSA to help manage the growing influx of log data. Like in the instance of a library, SIEMs take in lots of data and organizes it in a way so that IT administrators can easily search and access it.

“Need to know who had access to a particular credit card transaction? Query your SIEM solution,” Cirtas officials wrote in the blog. “Need to correlate the time of an attack on your firewall with subsequent authentication requests in your customer database? Your SIEM solution can tell you.”

Utilizing a SIEM solution in conjunction with Cirtas’ Bluejet Cloud Storage Controller is the best solution as the SIEM solution focuses on correlation and indexing tasks while relying on the efficiency of Cirtas’ cloud storage product for compression and retention tasks, according to company officials.

When logging data, efficiency is not the only thing that IT managers value as security is also paramount. Every second of the day private, critical information is collected and the amount of data collected off an organization’s servers, firewalls, intrusion detection systems, virus scanners, and other critical pieces of infrastructure is “truly staggering,” according to Cirtas officials.

“Data stored to comply with regulations and to secure the organizations IT infrastructure isn’t the data that the company accesses frequently in order to operate, but that doesn’t mean that companies don’t produce a lot of it,” the blog explains. “Many companies produce tens of gigabytes or even a terabyte of this data per day. Data is generated rapidly and consistently, like a faucet that never shuts off.”

SIEM and log management solutions help sift through large amounts of log data and makes sense of it – with its correlation/search engine – and the repository of the actual log data. Both aspects have different requirements when it comes to data storage.

Correlation and search, a performance-intensive aspect, is database-driven and often implemented in a map-reduce function to enable distributed processing. This type of data is usually active and, accordingly, needs to run on high performance storage.

The log repository is the place for long-term retention – where all the data that already has been correlated and indexed goes. This information is rarely accessed but continues to accumulate. According to Cirtas, many customers collect and retain over 100GB per day of log information that they have to keep for seven years.

In other news, while cloud storage has “tremendous potential” for reducing a company’s overall cost by eliminating upfront and ongoing investments in infrastructure, floor space, power, cooling and management, enterprises are not turning to the cloud as often as one might think.

According to a report put out by the Enterprise Strategy Group (News - Alert) (ESG), this result is because for the most part “services on the market lack enterprise-class features.”

Cirtas is challenging that fact, however, as it provides a feature set that makes remote cloud storage function like local, SAN-attached storage, with all the enterprise-class features expected in today’s data centers.

“Cloud storage service providers are offering data storage services at a fraction of the price IT departments pay to store and manage data themselves,” ESG states in the report, available on Cirtas’ website. “But it is not all good news; despite the attractive cost of cloud storage services, many of these services were initially designed to support consumer-class applications and don’t have the advanced data protection and security functionality IT expects and requires.”