Skip to content
  • Olav Sandstaa's avatar
    4d8d09ea
    Fix for Bug#12365385 STRAIGHT_JOIN QUERY QUICKLY EXHAUSTS SYSTEM+VIRT. · 4d8d09ea
    Olav Sandstaa authored
                         MEMORY LEADING TO SYSTEM CRASH
    
    The memory leak is caused by the following:
    
    1. One of the tables in the join is accessed using "Dynamic Range" quick
       select access (the explain output contains "Range checked for each
       record").
    2. Each time the join operation needs to check a record in this table,
       a new QUICK_RANGE_SELECT object is created. This is allocated on
       the heap (and the previous object is deleted from the heap).
    3. The QUICK_RANGE_SELECT object finds the handler object for the table and
       instructs this to use DS-MRR. Note that this is an existing handler object
       that it gets from the table object (no memory allocated).
    4. When the QUICK_RANGE_SELECT object calls
       handler::multi_range_read_init() the DS-MRR implementation will
       create a second handler object ("h2") that will be used for
       accessing the index during the DS-MRR scan. This second handler object
       is created using the current THD's memroot.
    5. When the range scan is completed and the QUICK_SELECT_OBJECT is
       deleted, this will end the index scan on the handler which will
       clean up after the DS-MRR scan. Among the things it will do it to
       "delete" the second handler object ("h2"). Given that this was
       created on a memroot the actual memory used by it is not freed.
    
    As a consequence the statement's memroot gets filled with 
    handler objects until the server runs out of memory.
    
    The fix for this problem is to be able to re-use the allocated "h2"
    handler for the entire statement. The main changes done to achieve this
    is:
    
    1. When completing the DS-MRR scan we no longer delete the "h2" handler 
       object but keep it in case a new scan is started on the same handler
       within the statement.
    2. Create a new function DsMrr_impl::reset() that takes care of
       closing and freeing the "h2" handler. This must be called when
       the statement is completed. This is done by adding calls for this
       into MyISAM and InnoDB's ::reset() methods (which are called from
       handler::ha_reset() when returning the open table objects).
    3. The initialization code in DsMrr_impl::dsmrr_init() has been 
       re-written to be able to handle re-use of an already allocated
       "h2" handler.
    
    A reduced version of the test case for this bug is included in 
    mrr_tests.inc. This will be run when the main suite is run. This test
    is too small to reproduce the "out of memory" problem but it will test
    the  functionality of the changed code. To reproduce and have a regression
    test for the "out of memory problem" an new test is included
    (ds_mrr-big.test). Due to the time to run this it will only be run when MTR
    is started with the --big option.
    4d8d09ea
    Fix for Bug#12365385 STRAIGHT_JOIN QUERY QUICKLY EXHAUSTS SYSTEM+VIRT.
    Olav Sandstaa authored
                         MEMORY LEADING TO SYSTEM CRASH
    
    The memory leak is caused by the following:
    
    1. One of the tables in the join is accessed using "Dynamic Range" quick
       select access (the explain output contains "Range checked for each
       record").
    2. Each time the join operation needs to check a record in this table,
       a new QUICK_RANGE_SELECT object is created. This is allocated on
       the heap (and the previous object is deleted from the heap).
    3. The QUICK_RANGE_SELECT object finds the handler object for the table and
       instructs this to use DS-MRR. Note that this is an existing handler object
       that it gets from the table object (no memory allocated).
    4. When the QUICK_RANGE_SELECT object calls
       handler::multi_range_read_init() the DS-MRR implementation will
       create a second handler object ("h2") that will be used for
       accessing the index during the DS-MRR scan. This second handler object
       is created using the current THD's memroot.
    5. When the range scan is completed and the QUICK_SELECT_OBJECT is
       deleted, this will end the index scan on the handler which will
       clean up after the DS-MRR scan. Among the things it will do it to
       "delete" the second handler object ("h2"). Given that this was
       created on a memroot the actual memory used by it is not freed.
    
    As a consequence the statement's memroot gets filled with 
    handler objects until the server runs out of memory.
    
    The fix for this problem is to be able to re-use the allocated "h2"
    handler for the entire statement. The main changes done to achieve this
    is:
    
    1. When completing the DS-MRR scan we no longer delete the "h2" handler 
       object but keep it in case a new scan is started on the same handler
       within the statement.
    2. Create a new function DsMrr_impl::reset() that takes care of
       closing and freeing the "h2" handler. This must be called when
       the statement is completed. This is done by adding calls for this
       into MyISAM and InnoDB's ::reset() methods (which are called from
       handler::ha_reset() when returning the open table objects).
    3. The initialization code in DsMrr_impl::dsmrr_init() has been 
       re-written to be able to handle re-use of an already allocated
       "h2" handler.
    
    A reduced version of the test case for this bug is included in 
    mrr_tests.inc. This will be run when the main suite is run. This test
    is too small to reproduce the "out of memory" problem but it will test
    the  functionality of the changed code. To reproduce and have a regression
    test for the "out of memory problem" an new test is included
    (ds_mrr-big.test). Due to the time to run this it will only be run when MTR
    is started with the --big option.
Loading