What Is Windowing?
If you’re creating an application with Kafka Streams or ksqlDB, and that application involves aggregations, it’s likely that you’ll use windowing.
Why is that?
Well, aggregations of data accumulate over time, and without a limit, these aggregations won’t stop accumulating. Enter windowing, which defines the amount of data that can accumulate.
Note that windowing takes four forms, depending on whether the window is defined by size and period, or whether the window is event-triggered.
|
Size and Period |
Event-Triggered |
|
Hopping |
Session |
|
Tumbling |
Sliding |
For example, say that you’re designing an application using moisture sensors to alert when a houseplant gets too dry. You can perform a filter for plants with low moisture readings in the past hour. If that filter returns anything, that event can trigger an aggregation over a window tumbling on every hour. The result is written to the alerts.
Let’s take a look at each of the types of windowing in finer detail.
Hopping
What is a hopping window?
A hopping window is bound by time and defined by a window size and the size of the time block at which it advances (both measured in seconds).
Consider the red window in the above diagram and think about it advancing through time: if you had a window size of 50 seconds but an advance size of 15 seconds, there would be an overlap of 35 seconds between the windows. In a hopping window, the window size is usually set to a larger amount of time than the advance size. The above diagram shows what an overlap might look like if you took the window as a rectangle and superimposed it over a wavy stream of data—the rectangles that overlap represent the overlapping window.
Note: Tumbling windows never overlap or have gaps. Hopping windows might have gaps or overlaps, or they might not.
A classic example of a hopping window implementation is a dashboard with moving averages—say, average clicks on a certain e-commerce page, like a product details page for an air fryer, for 2-minute windows in the past 24 hours.
Creating a hopping window
As stated above, there are two key pieces of information you need to configure hopping windows: window size and advance size.
When you’re working with Kafka Streams, you need to set both a windowSize and advanceSize. See the more complete example in this Kafka S
