-
Steinar H. Gunderson authored
Instead of allocating one big filesort buffer up-front, start with a small buffer, only increasing it (exponentially) once we actually get more data. If we actually need all of the buffer, we do somewhat more mallocs than in the allocate-once case, but on Linux, the mallocs disappear entirely in the rest of the work. (On Windows, we do see a regression in heavily concurrent sort benchmarks, but that will be fixed in a future worklog.) For the case where we have a large buffer but don't use it (e.g. 256 kB of data with 32 MB large buffer), we do save a fair amount of CPU time (on the order of 7–8%, although of course this will vary with the exact circumstances). Also, of course, we save the RAM we don't use. Note that we never move data between the sort buffers -- once rows are written into a buffer, they never move. This means we need separate storage for the record pointers, which means we also need some care to make sure we don't overrun the sort buffer budget too much with the hidden cost of storing the pointers. Change-Id: I0599bc10947aec95ae80a3dbbf04e81d2017cfb0
Steinar H. Gunderson authoredInstead of allocating one big filesort buffer up-front, start with a small buffer, only increasing it (exponentially) once we actually get more data. If we actually need all of the buffer, we do somewhat more mallocs than in the allocate-once case, but on Linux, the mallocs disappear entirely in the rest of the work. (On Windows, we do see a regression in heavily concurrent sort benchmarks, but that will be fixed in a future worklog.) For the case where we have a large buffer but don't use it (e.g. 256 kB of data with 32 MB large buffer), we do save a fair amount of CPU time (on the order of 7–8%, although of course this will vary with the exact circumstances). Also, of course, we save the RAM we don't use. Note that we never move data between the sort buffers -- once rows are written into a buffer, they never move. This means we need separate storage for the record pointers, which means we also need some care to make sure we don't overrun the sort buffer budget too much with the hidden cost of storing the pointers. Change-Id: I0599bc10947aec95ae80a3dbbf04e81d2017cfb0
Loading