Really urgent analytics

A Forrester report entitled "Really Urgent Analytics: The Sweet Spot for Real-Time Data Warehousing" makes the connection between event-stream processing (ESP) and data warehousing, and Intelligent Enterprise published a nice summary.

A traditional data warehouse contained huge amounts of data but was loaded infrequently: say monthly, or nightly at best. Modern businesses demand actions at lower latencies, and data warehousing professionals have been able tune the traditional data warehouse load process to reduce latency.

But even when cranked up to the maximum, the load process cannot achieve latencies of less than a few seconds, whereas many business processes need their answers faster than that. And this performance comes at the cost of higher complexity, so it takes more time and effort to modify the load process to incorporate new data or ask new questions.

To solve the squeeze between lower latency and increasing complexity -- and, I would mention, ever-increasing data volumes and a trend towards distributed systems -- data warehousing needs a new architectural component, and Forrester rightly point to Event Stream Processing to fill that gap. I would add that, given the skill set of data warehousing professionals, it makes a lot of sense for that Event Stream Processing to be in SQL.

At SQLstream, we saw this need four years ago, and are dedicated to solving the latency-complexity problem using SQL.