osd: delay populating in-memory PG log hashmaps #6425
Conversation
|
This is a rebased and improved version of #5855 with some of code moved out of it into separate PRs and with some other suggested changes. |
| @@ -599,7 +599,8 @@ class coll_t { | |||
| } | |||
| return false; | |||
| } | |||
|
|
|||
| spg_t* get_pgid() { return &pgid; } | |||
There was a problem hiding this comment.
You can remove this change because is_pg(spg_t *) validates that the collection is TYPE_PG and if so fills in the spg_t passed by pointer.
There was a problem hiding this comment.
All right, reverting this one.
|
The white space and code cleanup changes should be in its own commit for clarity and backporting. |
branch-predictor
force-pushed
the
bp-delayed-pglog-index-v2
branch
from
6e672cc
to
350f884
Compare
branch-predictor
force-pushed
the
bp-delayed-pglog-index-v2
branch
from
350f884
to
abfed1a
Compare
branch-predictor
force-pushed
the
bp-delayed-pglog-index-v2
branch
from
abfed1a
to
c50c8e4
Compare
|
It seems like we should skip the work in index(pg_log_entry_t& e) if the index flag(s) aren't set, since before we can ever look at it we'll see the missing flag, call index(), and throw it all out anyway |
|
Right, I'll change it. |
branch-predictor
force-pushed
the
bp-delayed-pglog-index-v2
branch
from
c50c8e4
to
b515a1a
Compare
branch-predictor
force-pushed
the
bp-delayed-pglog-index-v2
branch
from
916b490
to
db8c959
Compare
branch-predictor
changed the title
|
Please do not merge until #6801 is merged. |
branch-predictor
changed the title
|
Went through teuthology-openstack: http://149.202.167.15:8081/ubuntu-2016-01-07_13:38:00-rados-bp-delayed-pglog-index-v2---basic-openstack/ |
When booting up OSD, it loads all PGs and their respective logs. To speed up processing later, these logs are accompanied by separate unordered_maps which are also populated during PG load. Delay that until we actually need to access it, so we don't occupy too much memory right from start - and when we need it, populate just the map that we want to use, not all 3 of them at once. Signed-off-by: Piotr Dałek <piotr.dalek@ts.fujitsu.com>
branch-predictor
force-pushed
the
bp-delayed-pglog-index-v2
branch
from
db8c959
to
b858e86
Compare
|
Thanks! I'll include this in the next batch of PRs. :) |
|
Thanks! |
osd: delay populating in-memory PG log hashmaps Reviewed-by: Sage Weil <sage@redhat.com>
|
\o/ |
|
:)
|
ghost
changed the title
When booting up OSD, it loads all PGs and their respective logs. To speed
up processing later, these logs are accompanied by separate unordered_maps
which are also populated during PG load.
Delay that until we actually need to access it, so we don't occupy too much
memory right from start - and when we need it, populate just the map that
we want to use, not all 3 of them at once.
Additionally, removed adding all PGs to interim set, because it is actually
unnecessary and code executed during that loop could be moved to other loop.
On my 4-node dev cluster with 98 osds and 52667 objects, combined RSS memory usage for all nodes during cluster bootup looks like this:
Also, there's a tangible difference when rebalancing:
(at the beginning, OSDs on one of nodes are being SIGTERMed one by one, after a while recovery kicks in, when it stabilizes, I turn OSDs back online).
Signed-off-by: Piotr Dałek piotr.dalek@ts.fujitsu.com