Rob’s SprintFaster index creation
time to read 1 min | 174 words

RavenDB previously had a really nice feature for temporary indexes. Since we expected most of them to be temporary, we indexed them directly into memory, greatly saving in IO costs. With the removal of temporary indexes, that left us the option of just removing the entire code path and moving on to other things.

But we sat down and thought about this for a while. Typically, the busiest part in the index’s life is its creation, because the database needs to go through all the documents in the db and index them. We have changed things so during this creation period, we will actually index to memory, without hitting the disk. Only if we reached a configurable size or finished indexing everything will we spill everything to disk.

This, in turn, gives us the best of both worlds. We get a really nice optimization for new indexes, and we don’t have to hit the disk for indexes that would soon go away. And, of course, we get the perf boost for all indexes now.

Tweet Share Share 0 comments
Tags:

More posts in "Rob’s Sprint" series:

  1. (08 Mar 2013) The cost of getting data from LevelDB
  2. (07 Mar 2013) Result Transformers
  3. (06 Mar 2013) Query optimizer jumped a grade
  4. (05 Mar 2013) Faster index creation
  5. (04 Mar 2013) Indexes and the death of temporary indexes
  6. (28 Feb 2013) Idly indexing