The Weather Company builds a brand new forecasting platform using Basho’s Riak NoSQL database and Amazon Web Services.

“Weather is the unique big data application,” says Bryson Koehler, executive VP and CIO on the Weather Company. “When mainframes first happened, one of several first applications was a weather forecasting model.”

Flash forward to today and the elements Company ingests some 20 terabytes of knowledge per day to spin out what Keohler bills because the world’s most accurate forecasts. To remain before its competition, the elements Company is inside the strategy of rolling out a brand new platform built on Basho’s Riak NoSQL database and running globally within the Amazon Web Services (AWS) cloud. Greater than a year inside the making, the recent platform will bring the company’s scale of research to a complete new level.

The Weather Company is the parent behind popular brands including the elements Channel, WeatherFX, Weather Underground, and Intellicast. It serves thousands of consumers, including 30 airlines, emergency services, shippers, utilities, insurers, media giants, and the developers behind thousands of mobile weather apps. The demand adds as much as billions of computer-based data requests per day. Performance expectations are as fast as 10-millisecond latency.

[ Want more on one of the crucial fastest-growing options in information management? Read Why NoSQL Databases Are Gaining Fans. ]

The Weather Company’s incumbent platform is more like a loose-knit choice of aging applications running across 13 data centers. Mainframes aren’t at the list, but Koeler said the corporate uses a “one-of-everything” mixture of databases, including MySQL, Microsoft SQL Server, Cassandra, MongoDB, and PostgreSQL. This generation of technology, which runs totally on MySQL, captures 2.2 million current-weather-condition data points from around the world four times per hour. The company’s new consolidated platform, called SUN (Storage Utility Network), will capture 2.25 billion (with a “b”) weather data points 15 times per hour.

“As with every large-scale, algorithmic-type modeling, the more data you’ve, the simpler the predictions might be,” Koehler said, explaining the planned exponential increase in data capacity.

It’s a significant win for NoSQL technology and, mainly, Basho, which gives Riak Enterprise edition Multi-Datacenter Replication for ultra-high availability. NoSQL won out over relational databases primarily for its scalability, but Riak was chosen for its simplicity and simplicity of administration at high scale. It won out over Apache Cassandra, the runner-up choice, in addition to MongoDB and Hadoop, which also got serious consideration, in line with Koehler.

“If you end up globally distributing massive amounts of information across Amazon nodes or Google Compute nodes, you desire something that’s simple to take advantage of and configure,” said Koehler. “Cassandra, for instance, is excellent at distributing data, but it’s complicated and sophisticated to run. Riak was built to deal with massive data movement, replication, and information-synchronization on a cloud-based, globally distributed data platform.”

The Weather Channel’s new platform may be a win for Amazon, which highlighted the elements Channel story at its Invent event in Las Vegas earlier this month. The SUN system may be deployed across four AWS availability zones: US East, US West, Europe, and Asia.

“We needed to break out from our 13 data centers and move everything to an infrastructure-as-a-service model,” Koehler said.

Amazon is the elements Company’s primary cloud provider, however the firm may be planning so as to add cloud capacity from Google and other providers.

“Competition within the compute space is crucial, so we’re ensuring that we abstract ourselves from being stuck on someone platform,” Koehler said.

The SUN system made its debut in August, powering WeatherFX, the company’s advertising targeting engine, which works ads with weather-driven demand patterns and forecasts. The platform was also rolled out in beta stage for the company’s Forecast on-Demand platform. The elements Underground site will move to the platform by Thanksgiving, in line with Koehler, and The elements Channel is predicted to roll out between now and the primary quarter of next year.

With the brand new platform in place, Koehler said, the elements Company has divided the globe — land, sea, ice caps, and all — into greater than 30,000 four-square-mile squares. It will possibly accurately report current conditions and offer predicted weather for every square hours, days, or maybe weeks earlier.

IT groups need data analytics software that’s visual and accessible. Vendors are becoming the message. Also within the State Of Analytics issue of InformationWeek: SAP CEO envisions a younger, greener, cloudier company. (Free registration required.)

More Insights