How DigitalGlobe Handles 2 Petabytes Of Satellite Data Yearly

DigitalGlobe’s soon-to-be six satellites gather data at a freakish pace — capturing images of everything from natural disasters to shipping containers in ports.

DigitalGlobe’s soon-to-be six satellites orbiting the globe are amassing image data at a freakish pace. Its archive would be ready to offer new services with that data, provided it could maintain the gathering, storage, indexing, and repair-delivery infrastructure had to support it.

DigitalGlobe offers image services to forestry agencies, mining companies, government crop-estimating experts, and other organizations seeking information on everything from the choice of cars in a competitor’s car parking zone to the variety of shipping containers in port.

The Longmont, Colo., firm has varieties of satellites in orbit, some collecting color images, some black and white. It applies different software to the resulting data — as an instance, stretching upward when it’s viewing the Himalayas and other mountain ranges, and chopping the shadows off tall buildings lest the picture misrepresent their true height.

But one in all its biggest tasks is data collection. Its five satellites (number six, WorldView-3, could be launched in early 2014) are scanning 3.3 million kilometers of earth imagery per day. The data is downloaded because the satellites omit the equator, over the poles and points in between.

[DigitalGlobe was concerned with the 2007 oceanic look for computer researcher Jim Gray. To be told more, see Remembering The quest For Jim Gray, A Year Later.]

Most collection points may be hardwired right into a network. However the satellite collection dish on the South Pole has to beam the knowledge to a communications satellite, which in turn relays it to the DigitalGlobe datacenter near its headquarters in Colorado. All in all, it collects an ordinary of 2 petabytes of knowledge annually. The firm maintains 40 PBs on spinning disks and 20 PBs on tape, with a duplicate in a second datacenter.

“Our archive goes back 7-8 years now. We are able to show changes at the earth’s surface over the years. We are able to show how such things as the Peterman Glacier in Greenland or the Amazon rain forest looked years before, versus the manner they give the impression of being now,” said Scott Hicar, CIO and senior VP of worldwide technology services, in an interview.

This ability may lend itself to future projects in climate research or planning inside the face of habitat changes in several parts of the planet. One satellite in DigitalGlobe’s “constellation,” QuickBird, can take color imagery, that’s useful in agriculture, similar to in assessing the dangerous effects of drought on particular crops.

But to construct modern services on its wealth of knowledge, it’s needed to build out more of a personal cloud infrastructure. Each datacenter has a high-performance, x86 server cluster together with “several thousand servers” running Red Hat Enterprise Linux, Hicar said. It has virtualized and manages the clusters with VMware vSphere software and uses IBM Moab software for job scheduling. It analyzes images and creates indexes of its vast library using Oracle. Services in accordance with the clusters are monitored by CA Technologies Nimsoft, which identifies any hiccups or service slowdowns. The firm uses SAP ERP for each of the information had to keep satellites aloft.

DigitalGlobe recently established a FirstLook service that could zero in on a specific portion of the sector to generate up-to-date imagery to fulfill a customer’s need. Take disaster first responders: DigitalGlobe offered images showing damage from the tsunami that washed over the Fukushima nuclear power plant in Japan and the Haiyan typhoon that hit the Philippines.

Emergency planners and disaster-relief organizations need such information to understand how hard hit a local is, where they’ll stage relief efforts, and what supplies and gear are needed. Along with images, DigitalGlobe may additionally supply its Analysis Reports service, which gives detailed feedback at the image. When the Sakurajima volcano erupted in Japan in 2012, DigitalGlobe provided imagery and analysis of the scale and rate of spread of its plume.

Satellite imaging might also meet more-routine needs. The Thio town council, a citizen body governing a remote area of Pacific island New Caledonia, contracted for satellite imagery and analysis to inform it where nickel mine waste and sediment were infiltrating its main drainage, the Thio River. In Africa, Namibia measures the effectiveness of mine rehabilitation efforts through satellite imagery.

Other governments want coastal mapping to envision on changes to low-lying areas. Marine biologists wish to map the growth or regression of barely visible coral reefs.

Automotive geo-positioning systems and truck fleet management are a number of the newer users of its services.

DigitalGlobes new WorldView-3 satellite will launch in 2014.

DigitalGlobe’s new WorldView-3 satellite will launch in 2014.

Hicar said DigitalGlobe would be capable of keep expanding the kinds of services it offers because it increases the sophistication of analytics utilized in building its reports. The new WorldView-3 satellite may be more sensitive and higher equipped than its black-and-white image-collecting predecessors. It will likely be a high-resolution, “super-spectral” data collector, in a position to see beyond the bands of visible light into infrared and other parts of the spectrum.

Web-based services similar to FirstLook, accessible to any customer equipped with a desktop computer, are a sample of what is to come back from satellite imaging services, Hicar said. It’s already utilized by Microsoft’s Bing search engine in supplying maps and as an information platform for the Esri geo-positioning system company. What’s more likely to follow shall be increasingly sophisticated services for smartphones and other mobile devices, as smartphone suppliers and alertness builders realize what could be done with the satellite data, Hicar said.

The ability to focus on attractions of a neighborhood and specific landmarks would lend itself to travel applications. Direction-finding systems may benefit from showing what a destination seems like in a picture. “Mining, forestry, natural resources, there are lots of uses,” Hicar predicted.

Charles Babcock is an editor-at-large for InformationWeek, having joined the publication in 2003. He’s the previous editor-in-chief of Digital News, former software editor of Computerworld and previous technology editor of Interactive Week.

Too many companies treat digital and mobile strategies as pet projects. Listed below are four ideas to shake up your organization. Also within the Digital Disruption issue of InformationWeek: Six enduring truths about selecting enterprise software. (Free registration required.)

More Insights