IT groups are struggling to measure their very own performance, especially around cloud services, new research says.
Vendor surveys is usually a snooze, but they are able to also offer interesting insights, sometimes unintentionally.
Case in point: The primary ever IT Operations Analytics Benchmark Survey, from Continuity Software, has a couple of things worth noting.
For starters, analytics could be a big buzzword, nevertheless it doesn’t yet pervade corporate IT. Continuity asked whether companies were using analytics to measure the performance of internal IT, and a majority (56%) don’t. Its survey did find that 57% of huge companies (people with greater than 2,500 employees) use analytic tools to measure performance. But only 29% of businesses with fewer than 2,500 employees use such tools.
At their worst, vendor surveys reinforce company talking points or remind investors just how much upside still exists in a market. The Continuity survey does a number of that, because the company’s product involves helping companies analyze their systems for risk. But Continuity will not be a monitoring company. Executives told Information Week they thought the survey was useful for understanding what the market wants, but that not anyone tool could be prone to fill those needs.
[What is the difference between the 3 key forms of data analytics? Read Big Data Analytics: Predictive Vs. Prescriptive.]
What is surprising is that only a few companies seem to measure their cloud services — only 14% of respondents reported doing this. As for other parts of IT infrastructure, the numbers range from 49% to 75%. Storage and network tied on the top, followed by applications and databases. Clusters were cited by only 49%.
Doron Pinhas, Continuity’s CTO, believes the explanation many companies aren’t monitoring their cloud services is as the technology remains maturing, and it’s hard to measure how these services will be performing. For clusters, Pinhas explained, “The velocity of change during the last five years was spectacular, with [the] rise of virtualization around the board.” Such rapid change has created a much more complex IT environment that changes rapidly, making it hard to measure.
Measurement is a mantra of management, and the survey found that uptime was the end metric tracked among organizations which are tracking. Here is a breakdown showing the proportion of respondents that track each metric (note that the survey enables multiple responses):
- Uptime: 92%
- Performance/response time: 80%
- Data loss: 56%
- Number of open issues: 52%
- Average time to mend: 51%
- Security breaches: 49%
- Mean time between failures: 38%
Less than 1/2 the businesses track in real time. Once they do, uptime remains the foremost tracked metric, at 43% of respondents, and security breaches jumps to number two, at 33%.
Respondents said the toughest metrics to fulfill were performance (29% rated it primary) and uptime (17% rated it no 1). Performance was ranked within the top three by 74% of respondents.
Analytics consultants are happy to inform us that many companies are usually not data-driven. The survey suggests not up to 1/2 IT departments are. Of respondents, 48% said that almost all or all critical decisions are informed by operational analytics, meaning greater than half use analytics to tell operational decisions rarely or only a few of the time.
The survey, conducted online from companies in Continuity’s database, received 90 responses. Nearly all of respondents weren’t Continuity customers, in keeping with Eran Livneh, the company’s VP of selling.
Of responding companies, 44% had greater than 5,000 employees, 7% had between 2,500 and 5,000, 18% had from 501 to two,500, and 31% had between 1 and 500 employees. Greater than half the respondents had greater than 500 servers of their datacenters, and of these 25% had greater than 2,500 servers. Only 19% of respondents had fewer than 50 servers.
Private clouds are moving rapidly from concept to production. But some fears about expertise and integration still linger. Also within the Private Clouds Step Up issue of InformationWeek: The general public cloud and the steam engine have more in common than you would possibly think. (Free registration required.)
More Insights