Benchmarking performance between Redis mget and hmget(pipeline) commands

Published: at 12:00 AM

Redis provides a benchmarking tool called redis-benchmark for testing the performance of common redis commands. The benchmarks in this blog are run on below configuration.

redis-benchmark runs the tests over 100000 commands by default. You can tweak the numbers as required.

$redis-benchmark -q

PING_INLINE: 23860.65 requests per second
PING_BULK: 21008.40 requests per second
SET: 19124.12 requests per second
GET: 19782.39 requests per second
INCR: 20833.33 requests per second
LPUSH: 22999.08 requests per second
RPUSH: 25316.46 requests per second
LPOP: 24685.26 requests per second
RPOP: 22993.79 requests per second
SADD: 22016.73 requests per second
HSET: 22286.61 requests per second
SPOP: 26688.02 requests per second
LPUSH (needed to benchmark LRANGE): 21767.52 requests per second
LRANGE_100 (first 100 elements): 9999.00 requests per second
LRANGE_300 (first 300 elements): 4682.31 requests per second
LRANGE_500 (first 450 elements): 3155.77 requests per second
LRANGE_600 (first 600 elements): 2412.25 requests per second
MSET (10 keys): 13802.62 requests per second

You might’ve noticed there aren’t tests for hmget and hget commands. Only a subset of all the possible commands on Redis are provided. For most purposes this would suffice. Also, we aren’t using pipelining while benchmarking. Now, let’s try that as well.

With a pipeline

Below benchmark runs 1000 commands in each pipeline over all the available tests.

$redis-benchmark -q -P 1000

PING_INLINE: 200803.22 requests per second
PING_BULK: 401606.44 requests per second
SET: 236406.61 requests per second
GET: 277008.31 requests per second
INCR: 229885.06 requests per second
LPUSH: 184501.84 requests per second
RPUSH: 220750.55 requests per second
LPOP: 183150.19 requests per second
RPOP: 223713.64 requests per second
SADD: 224719.11 requests per second
HSET: 172117.05 requests per second
SPOP: 335570.47 requests per second
LPUSH (needed to benchmark LRANGE): 186219.73 requests per second
LRANGE_100 (first 100 elements): 18278.19 requests per second
LRANGE_300 (first 300 elements): 6242.20 requests per second
LRANGE_500 (first 450 elements): 4006.73 requests per second
LRANGE_600 (first 600 elements): 2906.55 requests per second
MSET (10 keys): 29299.74 requests per second

You can also run a set of the tests with -t flag and get %ile numbers by removing -q flag.

$redis-benchmark -t set -P 1000

====== SET ======
  100000 requests completed in 0.49 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

0.00% <= 2 milliseconds
1.00% <= 4 milliseconds
2.00% <= 40 milliseconds
4.00% <= 41 milliseconds
7.00% <= 42 milliseconds
12.00% <= 43 milliseconds
19.00% <= 44 milliseconds
20.00% <= 101 milliseconds
24.00% <= 102 milliseconds
29.00% <= 103 milliseconds
36.00% <= 104 milliseconds
44.90% <= 105 milliseconds
49.27% <= 136 milliseconds
52.27% <= 137 milliseconds
57.27% <= 138 milliseconds
62.27% <= 139 milliseconds
71.27% <= 140 milliseconds
80.00% <= 141 milliseconds
86.00% <= 142 milliseconds
92.00% <= 143 milliseconds
97.00% <= 144 milliseconds
100.00% <= 144 milliseconds
202429.16 requests per second

Latency sampling

You can get the latency via below commands. First one runs until interrupted, while the second one samples over a 15 sec period by default on each iteration.

$redis-cli --latency

min: 0, max: 27, avg: 9.32 (309 samples)

$redis-cli --latency-history

min: 0, max: 181, avg: 10.37 (685 samples) -- 15.01 seconds range
min: 0, max: 125, avg: 10.61 (670 samples) -- 15.04 seconds range
min: 0, max: 183, avg: 10.82 (665 samples) -- 15.01 seconds range

Finally, let get to benchmarking hmget in pipeline vs mget for 1000 keys.

mget vs hmget pipeline

hset hash1 key1 "The golden rule of a useful benchmark is to only compare apples and apples"
hset hash2 key2 "The golden rule of a useful benchmark is to only compare apples and apples"
hset hash3 key3 "The golden rule of a useful benchmark is to only compare apples and apples"
$redis-benchmark -P 1000 hmget hash1 key1 hmget hash2 key2 hmget hash3 key3

====== hmget hash1 key1 hmget hash2 key2 hmget hash3 key3 ======
  100000 requests completed in 0.95 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

0.00% <= 2 milliseconds
1.00% <= 30 milliseconds
3.00% <= 31 milliseconds
7.00% <= 32 milliseconds
9.00% <= 88 milliseconds
10.00% <= 89 milliseconds
13.00% <= 90 milliseconds
18.00% <= 91 milliseconds
24.00% <= 92 milliseconds
29.00% <= 93 milliseconds
36.00% <= 94 milliseconds
42.00% <= 95 milliseconds
49.00% <= 96 milliseconds
55.00% <= 97 milliseconds
62.00% <= 98 milliseconds
67.00% <= 100 milliseconds
70.00% <= 101 milliseconds
73.00% <= 102 milliseconds
77.00% <= 103 milliseconds
80.00% <= 104 milliseconds
84.00% <= 105 milliseconds
88.00% <= 106 milliseconds
93.00% <= 107 milliseconds
97.00% <= 108 milliseconds
100.00% <= 108 milliseconds
105263.16 requests per second

Let’s set some keys to benchmark mget

set hash1_key1 "The golden rule of a useful benchmark is to only compare apples and apples"
set hash2_key2 "The golden rule of a useful benchmark is to only compare apples and apples"
set hash3_key3 "The golden rule of a useful benchmark is to only compare apples and apples"
$redis-benchmark mget hash1_key1 hash2_key2 hash3_key3 ... (1000 keys)

  100000 requests completed in 113.54 seconds
  50 parallel clients
  3 bytes payload
  keep alive: 1

0.00% <= 1 milliseconds
0.06% <= 2 milliseconds
15.38% <= 44 milliseconds
20.01% <= 45 milliseconds
25.82% <= 46 milliseconds
32.60% <= 47 milliseconds
40.24% <= 48 milliseconds
48.40% <= 49 milliseconds
56.21% <= 50 milliseconds
63.03% <= 51 milliseconds
68.34% <= 52 milliseconds
72.26% <= 53 milliseconds
75.10% <= 54 milliseconds
90.14% <= 78 milliseconds
95.77% <= 83 milliseconds
99.99% <= 234 milliseconds
100.00% <= 235 milliseconds
100.00% <= 236 milliseconds
100.00% <= 237 milliseconds
100.00% <= 241 milliseconds
880.72 requests per second

So, we were able to fetch 880*1000 = 880000 keys/sec via mget compared to 105263 keys/sec via hmget in pipeline.