Skip to content
  • Norvald H. Ryeng's avatar
    936f5002
    Bug#11766143 59185: ASSERTION FAILED: (FIXED == 1), FILE · 936f5002
    Norvald H. Ryeng authored
    ITEM_STRFUNC.CC, LINE 2760
    
    Problem: The server crashes when a subquery that is involved in a
    comparison requiring a character set conversion causes an error.
    
    During optimization, predicates are combined into multiple equality
    predicates by a call to optimize_cond(). As part of this,
    Item_equal::compare_const() is called to evaluate the comparison. Part
    of this evaluation is to set the comparison function by a call to
    Item_func_eq::set_cmp_func(), which calls
    Arg_comparator::set_cmp_func(). Since the character sets of the two
    arguments differ, it needs to call agg_item_set_converter() to set up
    a conversion. There, a new Item_func_conv_charset is constructed, and
    the constructor calls val_str() on its argument (the subquery). The
    subquery is evaluated and stores an error in the diagnostics
    area.
    
    Later, agg_item_set_converter() calls fix_fields() on the character
    set converter. Item_func::fix_fields() checks for thd->is_error() and
    misinterprets the error as if it occurred in
    Item_func::fix_length_and_dec(). This error is propagated by return
    values back to Arg_comparator::set_cmp_func(), which therefore skips
    setting the func attribute of the comparator.
    
    After the error occurs, Item_equal::compare_const() does not do any
    error checking before calling Item_func_eq::val_int(), where the crash
    occurs when Arg_comparator::compare() tries to call the function
    pointed to by the func attribute, which is a null pointer.
    
    Fix: Skip the call to Item_func_eq::val_int() in
    Item_equal::compare_const() if an error has been stored in the
    diagnostics area. The diagnostics area is checked and the error is
    caught in JOIN::optimize() right after the call to optimize_cond()
    returns.
    
    This patch also fixes bug #11765734.
    936f5002
    Bug#11766143 59185: ASSERTION FAILED: (FIXED == 1), FILE
    Norvald H. Ryeng authored
    ITEM_STRFUNC.CC, LINE 2760
    
    Problem: The server crashes when a subquery that is involved in a
    comparison requiring a character set conversion causes an error.
    
    During optimization, predicates are combined into multiple equality
    predicates by a call to optimize_cond(). As part of this,
    Item_equal::compare_const() is called to evaluate the comparison. Part
    of this evaluation is to set the comparison function by a call to
    Item_func_eq::set_cmp_func(), which calls
    Arg_comparator::set_cmp_func(). Since the character sets of the two
    arguments differ, it needs to call agg_item_set_converter() to set up
    a conversion. There, a new Item_func_conv_charset is constructed, and
    the constructor calls val_str() on its argument (the subquery). The
    subquery is evaluated and stores an error in the diagnostics
    area.
    
    Later, agg_item_set_converter() calls fix_fields() on the character
    set converter. Item_func::fix_fields() checks for thd->is_error() and
    misinterprets the error as if it occurred in
    Item_func::fix_length_and_dec(). This error is propagated by return
    values back to Arg_comparator::set_cmp_func(), which therefore skips
    setting the func attribute of the comparator.
    
    After the error occurs, Item_equal::compare_const() does not do any
    error checking before calling Item_func_eq::val_int(), where the crash
    occurs when Arg_comparator::compare() tries to call the function
    pointed to by the func attribute, which is a null pointer.
    
    Fix: Skip the call to Item_func_eq::val_int() in
    Item_equal::compare_const() if an error has been stored in the
    diagnostics area. The diagnostics area is checked and the error is
    caught in JOIN::optimize() right after the call to optimize_cond()
    returns.
    
    This patch also fixes bug #11765734.
Loading