---

ε-approximate count over a Sliding Window

---

• ε-approximate count revisited
  • Manku's algorithm can find the ε-approximate count for frequent items in a data stream.

  • But... Manku's algorithm computes the frequent data items and their counts over a stream from the beginning of the stream

  • Often, database applications make queries on the most recent items in the stream...
    For example, what are the top 10% of movies rented in the last 3 months ?

  • Can we modify Manku's algorithm to find the most frequent items and their frequencies in the most recent items in the stream...

• Fixed Sized (Sliding) Windows
  • The following figure shows the effect of a fixed sized window

    W = 4 (fixed size) Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +-+ Sum = 3 Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +----+ Sum = 8 Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +-------+ Sum = 10 Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +----------+ Sum = 17 Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +----------+ Sum = 15

• Variable Sized (Sliding) Windows
  • The following figure shows the effect of a variable sized window

    Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +-+ Sum = 3 Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +----+ Sum = 8 Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +-------+ Sum = 10 Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +----------+ Sum = 17 Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +-------------+ Sum = 18 Current item | V Input: 3 5 2 7 1 8 9 1 6 8 9 | | +----------+ Window can shrink Sum = 15

• Formal definitions
  • S_W,ε(m, N) = ε-approximate statistics on a fixed window of size W when the current window is N and the number of elements processed is m

    (Note: N < W can only happen when m < W)

  • S_ε(m, N) = ε-approximate statistics on a variable size window when the current window is N and the number of elements processed is m

• Operations allowed on S_{W, ε}(m, N) and S_ε(m, N)
  • For a variable window summary information S_ε(m, N), we can do:
    • Query: return the ε-approximate value (count or quantile) of the maintained statistics over the "bag" of elements {e_m-N+1, e_m-N+2, ..., e_m}

    • Insert: insert element e_m into the bag. It will transform:
      S_ε(m, N) ---> S_ε(m+1, N+1)

      (One extra element is processed, the element is added to the bag so the window is increased by 1)

    • Delete: remove the earliest element. It will transform:
      S_ε(m, N) ---> S_ε(m, N-1)

      (No element is processed, but one element is removed in the bag, so the window is decreased by 1)

  • For a fixed size window summary information S_W,ε(m, N), we can do:
    • Query: return the ε×(W/N)-approximate value (count or quantile) of the maintained statistics over the "bag" of elements {e_m-N+1, e_m-N+2, ..., e_m}

    • Insert: insert element e_m into the bag. It will transform:
      S_{W, ε}(m, N) ---> S_{W, ε}(m+1, N+1)

      It will only succeed when N ≤ W − 1 (When the window is full, you need to delete an element before you insert a new one)

    • Delete: remove the earliest element. It will transform:
      S_{W, ε}(m, N) ---> S_{W, ε}(m, N-1)

      (No element is processed, but one element is removed in the bag, so the window is decreased by 1)

• Important Assumption
  • The authors assume that:
    • W is some power of 2
    • 1/ε is some power of 2

  • The reason is that the window W will be divided in half repeated...

  • If W and 1/ε is not some power of 2; then use the follow derived values:
    • W' is a power of 2 and W ≤ W' ≤ 2×W
    • 1/ε' and W' × ε' ≤ W× ε

  • Example:
    • Suppose W = 1000 and ε = 0.01
    • Use W' = 1024 (1000 ≤ 1024 ≤ 2000)
    • Use 1024 × ε' ≤ 1000 × 0.01 ==> ε' ≤ 10/1024 ==> ε' = 8/1024 ==> ε' = 2^-7

  ---

  Part 1: Fixed Sized Window

  ---

• Basic idea to adapt Manku's algorithm to Construct sketches over a FIXED SIZE window
  • Goal:: compute the ε-approximate count/quantile for a window of inputs

    input: a b c d e f g h i j k .... | | window: +---------------------+ (imagine that the window is huge) Window moves: input: a b c d e f g h i j k l m.... | | window: +---------------------+

    • We must do relatively little work to compute ε-approximate statitics (count/quantile) for new window.

  • Basic idea:
    1. Divide the input stream into blocks

       Current window +-----------------------------+ | | input: a b c d e f g h i j k l m .... | | | | | | | | +---------+ +---------+ +---------+ +---------+ block 1 block 2 block 3 block 4

    2. Run Manku's algorithm for each block to compute an ε'-approximate frequency count (or run Greenwald's algorithm for each block to compute the ε-approximate quantile)

       NOTE: the accuracy that you need to run Manku or Greenwald on the values in a block is ε' and ε' may be different from ε !!!

    3. Design a method to combine the different outputs for different blocks to obtain an ε-approximate frequency count for the current window

  • Refined Idea

    • I haven't checked, but I believe that if you use this idea using a one level block division, you will need to use very high accuracy for ε'.

      As a result, the amount of memory space used by Manku's/Greenwald's algorithm will be too large

    • In order to keep the space needed for the computation to a minimum, they use a multi-layered approach:
      

    • They use L layers
      • Blocks in each layer is the same size (number of items is same)

      • Blocks in different layers have different sizes

      • Block size in the layer l + 1 is twice the block size in the layer l

      • The number of layers L that are needed depends on the desired accuracy ε... (the more accurate, i.e., the smaller ε, the larger number of layers

      • The block size in the last layer (layer L) is W elements

      • Since the block size of each lower layer is half that of the above layer, and the block size of layer 0 is equal to:
        W/(2^L)

      • In order to compute an ε-approximate count/quantile for a window W of inputs, the block size in layer 0 must be equal to:
        (ε/4) × W

        NOTE: because ε × W is a power of 2, (ε/4) × W will always be an integer :-)

      • These 2 equations allow us to determine the number of layers L that are needed to achieve the desired accuracy:

        (ε/4) × W = W/(2L) or: 2L = 4/ε or: L = 2 + lg(1/ε)

  ---

• Number of items in a level l block
  • From the above discussion, we saw that the number of items in a layer 0 block is equal to:
    N₀ = (ε/4) × W

  • The number of items in a layer 1 block is twice as many, so:
    N₁ = 2¹ × (ε/4) × W

  • The number of items in a layer 2 block is twice as many as a level 1 block, therefore,
    N₂ = 2² × (ε/4) × W

  • So in general, the number of items in a layer l block is equal to:
    N_l = 2^l × (ε/4) × W

  ---

• Error parameter for a level l block
  • The Arasu's algorithm execute Manku's/Greenwald's algorithm L + 1 times - but using L + 1 error parameters

  • The gist on how to conserve space in Manku/Greenwald algorithms:
    At the end of each period, the algorithms remove some "unneccessary data items" from the data structure !!

    How can we now conserve space when block sizes get smaller ?

    In order to conserve space using a smaller block size, the period must be reduced.

    The period in Manku/Greenwald algorithm depends on the error parameter.

    A larger error parameter will make the period in Manku or Greenwald algorithm smaller !!!

  • In order to compute an ε-approximate frequency count/ quantile in a window W, the error-parameter for a level i block (or the allowable error in the sketch for a level l block) must be equal to:

    	 1       2L
       εi = ---- × ------  × ε
    	 2i     4L+4
    

    You can see that ε_i decreases when i increases - so the accuracy parameter for a higher level block (with more elements in them), are smaller.

  ---

• Estimation Errors for data in a level i block
  • You may wonder why is ε_i chosen that way...

  • Well, for the simple reason that Arasu wants to keep the estimation error in every block the same - regardless (i.e., independent of) the block size.

  • In both Manku's and Greenwald's algorithm, the maximum error that the algorithm makes is proportional to ε × N:
    • In Manku's algorithm, the estimated frequency count f when N items are processes is bounded by:
      (f_e − ε) × N ≤ f ≤ f_e
      or:
      f_eN − ε N ≤ f ≤ f_e

    • In Greenwald's algorithm, the estimated error in the φ-quantile (position) when N items are processes is bounded by:
      (φ − ε) × N ≤ r ≤ (φ + ε) × N
      or:
      φ N − ε N ≤ f ≤ φ N + ε N

  • As we have seen above, Arasu's algorithm:
    • Forms a level i block consisting of
      N_i = 2ⁱ × (ε/4) × W         number of items

    • Furthermore, the error-parameter used in a level i block is:

               1       2L
         εi = ---- × ------  × ε
               2i     4L+4
      

    • You will also need to use the following relationship (see: click here )
      2^L = 4/ε

  • From these 3 equation, we conclude that the factor "ε_i × N_i" for a level i block has a very interesting value... let's do the computation:
    ε_i × N_i = { 2ⁱ × (ε/4) × W } × { 1/(2ⁱ) × 2^L/(4L + 4) } × ε
    or:
    ε_i × N_i = { (ε/4) × W } × { 2^L/(4L + 4) } × ε
    or:
    ε_i × N_i = { W } × { 1/(4L + 4) } × ε
    or:
    ε_i × N_i = ε × W/(4L + 4)

  • Notice that the error term ε_i × N_i for a block at any level is independent of the level (L is a constant !)

• Some terminology in Arasu's Algorithm
  • Before I can discuss Arasu's algorithm, let me clarify the terminology in his algorithm.
    • A block is ACTIVE if every element in the block reside inside the current window

      - only ACTIVE blocks will be used to compute the query on a window W

    • A block is EXPIRED if at least 1 element in the block reside outside the current window

    • A block is UNDER-CONSTRUCTION if every element in the block reside inside the current window, but some element(s) of the block has not yet arrived

      - UNDER-CONSTRUCTION blocks are NOT USED to compute the query on a window W

    • A block is INACTIVE if no element in the block has arrived yet.

  • Example:

    The light-blue shaded blocks are all ACTIVE. The unshaded shaded blocks at left are EXPIRED The unshaded shaded blocks at right are either UNDER-CONSTRUCTION or INACTIVE

  • Example continued:

    One level 0 ACTIVE block has become EXPIRED One level 0 INACTIVE block has become UNDER-CONSTRUCTION

  • Example continued:

    More data arrive, but the state of the blocks did not changed...

  • Example continued:

    A level 0 block has changed its state from UNDER-CONSTRUCTION to ACTIVE A level 1 block has changed its state from UNDER-CONSTRUCTION to ACTIVE

  • Example continued:

    4 blocks (one from each level) have changed from ACTIVE to EXPIRED !!

  ---

• Merging disjoint sketches to obtain an ε-approximate sketch for a window
  • Arasu's algorithm compute the approximate frequency count over a window by merging the information from a number of blocks

  • It selects the best (highest level) ACTIVE blocks that covers the window.

  • Example covering:

    Covering the current window with ACTIVE blocks

  • Notice that not every element in the current window will be including in the ACTIVE windows (see figure: elements in the left and right edges of the window are not accounted for by any ACTIVE windows)

    These uncovered (unaccounted) elements will contribute to the total error

    Arasu's method will make certain that the error contributed by these uncovered elements will not be significant (that is done by many the blocks at the lowest level very small...)

  • Arasu's method continues on the next webpage...