Skip to content
  • Dag Wanvik's avatar
    598fe997
    Bug#21761044 CONDITIONAL JUMP AT TEST_IF_ORDER_BY_KEY IN SQL_OPTIMIZER.CC · 598fe997
    Dag Wanvik authored
    In test_if_order_by_key, the pointer key_part is incremented over a
    key's parts. For some storage engines, a secondary key may contain, in
    addition to the user defined keys, the primary key of the table.  The
    logic detects when we pass from testing a user defined key over to
    testing the "hidden" primary key parts using a conditional:
    
       if (key_part == key_part_end)
    
    However, in this case, the optimizer has detected that the first key
    of a primary key (pk) is constant in a WHERE condition (encoded in
    const_key_parts). This causes the little loop just in front of the
    test above to trigger, even though we have exhausted the user defined
    parts of the key.
    
      /*
        Skip key parts that are constants in the WHERE clause.
        These are already skipped in the ORDER BY by const_expression_in_where()
      */
      for (; const_key_parts & 1 ; const_key_parts>>= 1)
        key_part++;
    
    This causes key_part to be increased *beyond* key_part_end, causing
    the test above to fail and we get no entry into the section for
    handling parts stemming from the "hidden" primary key.  This causes
    the code to eventually increase key_part beyond even the hidden
    primary key parts, i.e. it points into unallocated memory, and an
    access is detected by ASAN (no crash seen).
    
    The fix makes the loop contingent on key_part < key_part_end. This
    makes the code in the repro enter the section for handling the hidden
    key parts as intended.
    
    Repro added to regression tests.
    598fe997
    Bug#21761044 CONDITIONAL JUMP AT TEST_IF_ORDER_BY_KEY IN SQL_OPTIMIZER.CC
    Dag Wanvik authored
    In test_if_order_by_key, the pointer key_part is incremented over a
    key's parts. For some storage engines, a secondary key may contain, in
    addition to the user defined keys, the primary key of the table.  The
    logic detects when we pass from testing a user defined key over to
    testing the "hidden" primary key parts using a conditional:
    
       if (key_part == key_part_end)
    
    However, in this case, the optimizer has detected that the first key
    of a primary key (pk) is constant in a WHERE condition (encoded in
    const_key_parts). This causes the little loop just in front of the
    test above to trigger, even though we have exhausted the user defined
    parts of the key.
    
      /*
        Skip key parts that are constants in the WHERE clause.
        These are already skipped in the ORDER BY by const_expression_in_where()
      */
      for (; const_key_parts & 1 ; const_key_parts>>= 1)
        key_part++;
    
    This causes key_part to be increased *beyond* key_part_end, causing
    the test above to fail and we get no entry into the section for
    handling parts stemming from the "hidden" primary key.  This causes
    the code to eventually increase key_part beyond even the hidden
    primary key parts, i.e. it points into unallocated memory, and an
    access is detected by ASAN (no crash seen).
    
    The fix makes the loop contingent on key_part < key_part_end. This
    makes the code in the repro enter the section for handling the hidden
    key parts as intended.
    
    Repro added to regression tests.
Loading