Redis HyperLogLog

Redis added the HyperLogLog structure in version 2.8.9.

Redis HyperLogLog is an algorithm used for cardinality statistics. The advantage of HyperLogLog is that when the number or volume of input elements is very, very large, the space required to calculate the cardinality is always fixed and very small.

In Redis, each HyperLogLog key only needs 12 KB of memory to calculate the cardinality of nearly 2^64 distinct elements. This is in sharp contrast to sets, where the more elements there are, the more memory is consumed when calculating cardinality.

However, because HyperLogLog only calculates the cardinality based on the input elements and does not store the input elements themselves, HyperLogLog cannot return the individual input elements like sets can.

What is cardinality?

For example, given the data set {1, 3, 5, 7, 5, 7, 8}, the cardinality set of this data set is {1, 3, 5, 7, 8}, and the cardinality (distinct elements) is 5. Cardinality estimation is to quickly calculate the cardinality within an acceptable error range.

Example

The following example demonstrates how HyperLogLog works:

redis 127.0.0.1:6379> PFADD examplekey "redis"

1) (integer) 1

redis 127.0.0.1:6379> PFADD examplekey "mongodb"

1) (integer) 1

redis 127.0.0.1:6379> PFADD examplekey "mysql"

1) (integer) 1

redis 127.0.0.1:6379> PFCOUNT examplekey

(integer) 3