Slideshow:
Setup for the
2-pass algorithms based on
hashing
Buffer allocation:
Pass 1 of the
2-pass algorithms based on
hashing
Pass 1 of the
2-pass algorithms based on
hashing
- Result
Property 1 of the hash results
Because hash(k) = hash(k) when input keys are equal !!
Property 2 of the hash results
Because hash(k1) ≠ hash(k2) or hash(k1) = hash(k2) (possible)
Assumption about the
hash results
Principle of the
2-pass algorithms based on
hashing
- System assumption:
- We have M buffers available
- An input relation
is preprocessed using
hashing as follows:
- Use 1 buffer to
scan the relation
- Use M−1 buffers for
output
Associate 1 output buffer with 1 hash bucket
- The input relation is
scanned 1 block at
a time into the
input buffer.
Tuples in input buffer are hashed into M−1 buckets:
When a hash bucket is full (1 block), the hash bucket (block) is written to disk
- Use 1 buffer to
scan the relation
- Result after
preprocessing with
hashing:
We have M−1 sub-relations
- The sub-relations are not sorted
The size of each sub-relation is approximately B(R)/(M−1) blocks
- Property 1:
- 2 tuples with the
same search key
will
be hashed into the
same bucket:
- 2 tuples with the
same search key
will
be hashed into the
same bucket:
- Property 2:
- Tuples with
different
search keys
can end up
in the same bucket:
- Usually are
hashed into
different buckets:
- But they
can be
hashed into the
same bucket:
- Usually are
hashed into
different buckets:
- Tuples with
different
search keys
can end up
in the same bucket:
- Performance of
hashing-based
2-pass algorithms:
- The performance of
2-pass (data processing) algorithms
depends on the
distribution property
of the hash function
use:
- How uniform will the hash function scatter the input tuples over the different buckets
- The performance of
2-pass (data processing) algorithms
depends on the
distribution property
of the hash function
use:
- Assumption:
- The hash function is
uniform
I.e.:
- The tuples are distributed evenly among the M−1 bucket
- The hash function is
uniform
- Therefore:
Original relation R: size = B(R) After hashing using M−1 buckets: B(R) Size of each sub-relation Ri = ------ M−1
- Basic idea:
- Use Hashing to
partition the
input relation(s)
into
disjoint buckets (subsets)
that can be
processed
independently from
each other:
- Each bucket
is a fraction of the
original input relation
Furthermore:
- Each bucket can be processed individually/independently using a one-pass algorithm
- Use Hashing to
partition the
input relation(s)
into
disjoint buckets (subsets)
that can be
processed
independently from
each other: