redis-trib: use pipeline to speed up moving slot #2711
Conversation
* "keysinslot" now retrieves more keys (10 -> 10000) for every iteration * Use pipeline to run "migrate" commands much faster then current
|
That's great! But it's a little disappointing that pipeline can only shorten the time for half. I thought the pipeline have huge potential... |
|
@Calabor-Hoc Please note that it is just a benchmark, but I also expect more speedup than 2x. ps. Maybe we need to increase bulk size. (10000 could be small.) |
|
The reason why the speedup is not the one expected is the way MIGRATE is implemented. It requires to chat with other nodes and get a reply. So instead of implementing pipelining, if we truly want much faster migration, we should pipeline MIGRATE itself internally. I'm trying to do this right now for 3.2 and possibly will back port to 3.0. Migrate in this use case will have a new option where it is possible to specify multiple keys, so that the source node will send all the keys and will later retrieve all the replies, making the process much faster. News soon, on it. |
|
Multi keys |
|
Just tested with a larger data set, from 20 to 5 seconds. If I migrate 100 keys per iteration, 2.7 seconds from 20. 💃 |
We use the new variadic/pipelined MIGRATE for faster migration. Testing is not easy because to see the time it takes for a slot to be migrated requires a very large data set, but even with all the overhead of migrating multiple slots and to setup them properly, what used to take 4 seconds (1 million keys, 200 slots migrated) is now 1.6 which is a good improvement. However the improvement can be a lot larger if: 1. We use large datasets where a single slot has many keys. 2. By moving more than 10 keys per iteration, making this configurable, which is planned. Close #2710 Close #2711
We use the new variadic/pipelined MIGRATE for faster migration. Testing is not easy because to see the time it takes for a slot to be migrated requires a very large data set, but even with all the overhead of migrating multiple slots and to setup them properly, what used to take 4 seconds (1 million keys, 200 slots migrated) is now 1.6 which is a good improvement. However the improvement can be a lot larger if: 1. We use large datasets where a single slot has many keys. 2. By moving more than 10 keys per iteration, making this configurable, which is planned. Close #2710 Close #2711
|
@antirez 👍 |
We use the new variadic/pipelined MIGRATE for faster migration. Testing is not easy because to see the time it takes for a slot to be migrated requires a very large data set, but even with all the overhead of migrating multiple slots and to setup them properly, what used to take 4 seconds (1 million keys, 200 slots migrated) is now 1.6 which is a good improvement. However the improvement can be a lot larger if: 1. We use large datasets where a single slot has many keys. 2. By moving more than 10 keys per iteration, making this configurable, which is planned. Close redis#2710 Close redis#2711
We use the new variadic/pipelined MIGRATE for faster migration. Testing is not easy because to see the time it takes for a slot to be migrated requires a very large data set, but even with all the overhead of migrating multiple slots and to setup them properly, what used to take 4 seconds (1 million keys, 200 slots migrated) is now 1.6 which is a good improvement. However the improvement can be a lot larger if: 1. We use large datasets where a single slot has many keys. 2. By moving more than 10 keys per iteration, making this configurable, which is planned. Close redis#2710 Close redis#2711
Closes #2710
Test environment is below,
Test result is here,
a) verbose = true
b) verbose = false
Increasing fetch size of keysinslot and using pipeline seems to be 2 ~ 2.5 times faster than current.
I couldn't test it with multiple machines since I don't have three or more physical machines.
Btw, just modifying verbose option to 'false' also reduces more than 25 secs.