Skip to content
  • Nirbhay Choubey's avatar
    9ec3d37a
    Bug#11761614 MYSQLD SEGFAULTS WHEN BUILT USING · 9ec3d37a
    Nirbhay Choubey authored
        --WITH-MAX-INDEXES=128
    
    The MySQL server bitmap implementation defines two
    template Bitmap classes. One optimized for 64-bit
    (default) wide bitmaps while the other one is used for
    all bitmaps with non-default width.
    
    In order to optimize the computations, Bitmap<64> class
    has defined its own member functions for bitmap operations,
    the other one, however, relies on mysys' implementation of
    bitmap (mysys/my_bitmap.c).
    
    Issue 1:
    In mysys bitmap implementation, while setting the prefix
    bits, an extra byte beyond the bitmap buffer can be set
    to 0. Now, as this piece of code is used only for bitmaps
    with non-default width, the issue wasn't visible in the
    64-bit wide (default) builds.
    
    Fixed the offending mysys bitmap function by properly
    adjusting the number of prefix bytes, which is used to
    bzero the remaining bitmap buffer.
    
    Issue 2:
    In case of non-default Bitmap class, the intersect function
    wrongly resets the received bitmap while initialising a new
    local bitmap structure (bitmap_init() clears the bitmap
    buffer) and hence the received bitmap was getting lost.
    
    Fixed by initializing the local bitmap structure by using a
    temporary buffer and later copying the received bitmap
    to the initialised bitmap structure.
    
    Also added a cmake variable to hold the MAX_INDEXES
    value when supplied from the command prompt.
    (eg. cmake .. -DMAX_INDEXES=128U). Checks has been put
    in place to trigger a build failure if MAX_INDEXES value
    is greater than 255.
    
    Test modifications:
    * Introduced include/have_max_indexes_[64|128].inc to
      facilitate skipping of tests for which the output
      differs with different MAX_INDEXES.
    
    * Introduced include/max_indexes.inc which would get
      modified by cmake to reflect the MAX_INDEXES value
      used to build the server. This file simply sets an
      mtr variable '$max_indexes' to show the MAX_INDEXES
      value, which will then be consumed by the above
      introduced include file.
    
    * Some test's outputs were dependent on MAX_INDEXES
      value, they have been moved to new test files.
    
    * Added gunit tests for the fixed issues.
    9ec3d37a
    Bug#11761614 MYSQLD SEGFAULTS WHEN BUILT USING
    Nirbhay Choubey authored
        --WITH-MAX-INDEXES=128
    
    The MySQL server bitmap implementation defines two
    template Bitmap classes. One optimized for 64-bit
    (default) wide bitmaps while the other one is used for
    all bitmaps with non-default width.
    
    In order to optimize the computations, Bitmap<64> class
    has defined its own member functions for bitmap operations,
    the other one, however, relies on mysys' implementation of
    bitmap (mysys/my_bitmap.c).
    
    Issue 1:
    In mysys bitmap implementation, while setting the prefix
    bits, an extra byte beyond the bitmap buffer can be set
    to 0. Now, as this piece of code is used only for bitmaps
    with non-default width, the issue wasn't visible in the
    64-bit wide (default) builds.
    
    Fixed the offending mysys bitmap function by properly
    adjusting the number of prefix bytes, which is used to
    bzero the remaining bitmap buffer.
    
    Issue 2:
    In case of non-default Bitmap class, the intersect function
    wrongly resets the received bitmap while initialising a new
    local bitmap structure (bitmap_init() clears the bitmap
    buffer) and hence the received bitmap was getting lost.
    
    Fixed by initializing the local bitmap structure by using a
    temporary buffer and later copying the received bitmap
    to the initialised bitmap structure.
    
    Also added a cmake variable to hold the MAX_INDEXES
    value when supplied from the command prompt.
    (eg. cmake .. -DMAX_INDEXES=128U). Checks has been put
    in place to trigger a build failure if MAX_INDEXES value
    is greater than 255.
    
    Test modifications:
    * Introduced include/have_max_indexes_[64|128].inc to
      facilitate skipping of tests for which the output
      differs with different MAX_INDEXES.
    
    * Introduced include/max_indexes.inc which would get
      modified by cmake to reflect the MAX_INDEXES value
      used to build the server. This file simply sets an
      mtr variable '$max_indexes' to show the MAX_INDEXES
      value, which will then be consumed by the above
      introduced include file.
    
    * Some test's outputs were dependent on MAX_INDEXES
      value, they have been moved to new test files.
    
    * Added gunit tests for the fixed issues.
Loading