Skip to content
  • Dag Wanvik's avatar
    14811129
    Bug#25051195 WRONG CALCULATION OF DECIMALS AFTER POINT FOR IFNULL W/ GROUP BY USING TMP TBL · 14811129
    Dag Wanvik authored
    IFNULL is used with type combination {decimal,int}, the resulting
    decimal loses on digit after the decimal point in the calculation of
    ::max_length. This leads to a loss of a decimal digit after point in
    the corner case of GROUP BY which needs a temporary file, e.g. this
    kind of query:
    
       SELECT IFNULL(e2,i1) in, SUM(d1) FROM t GROUP BY e2,i1 ORDER BY in;
    
    This bug will be more visible when we get window functions which rely on
    temporary files in non-trivial cases.
    
    FIX
    ------------------------------
    
    Add space for decimal point to the calculation in
    Item_func_ifnull::resolve_type.
    
    Repro test added.
    
    ANALYSIS (refer to added test case):
    ------------------------------------
    
    The calculation of char_length in Item_func_ifnull::resolve_type is
    used as an argument to fix_char_length. The calculation forgets to add
    space for decimal point if present.
    
    The calculation for NULLIF looks like this:
    
    Item_func_ifnull::resolve_type:
       :
       char_length= max(len0, len1) + decimals + (unsigned_flag ? 0 : 1);
    
    where "decimals" is 2 in our case.
    
    In the similar computation of IF, this is calculated with another (correct)
    code path:
    
    cf Item_func_if::resolve_type ->
       fix_num_type_shared_for_case->
       Item_func::count_decimal_length ->
       my_decimal_precision_to_length_no_truncation
    
    which has this line:
       uint32 retval= (uint32)(precision + (scale > 0 ? 1 : 0) +
                      (unsigned_flag || !precision ? 0 : 1));
    
    the "scale" here being the number of digits after point, making the
    calculation add 1 in our case, since "scale" is 2.
    
    Change-Id: If54750e9e3b456d063591a359514d69f2bc0c510
    14811129
    Bug#25051195 WRONG CALCULATION OF DECIMALS AFTER POINT FOR IFNULL W/ GROUP BY USING TMP TBL
    Dag Wanvik authored
    IFNULL is used with type combination {decimal,int}, the resulting
    decimal loses on digit after the decimal point in the calculation of
    ::max_length. This leads to a loss of a decimal digit after point in
    the corner case of GROUP BY which needs a temporary file, e.g. this
    kind of query:
    
       SELECT IFNULL(e2,i1) in, SUM(d1) FROM t GROUP BY e2,i1 ORDER BY in;
    
    This bug will be more visible when we get window functions which rely on
    temporary files in non-trivial cases.
    
    FIX
    ------------------------------
    
    Add space for decimal point to the calculation in
    Item_func_ifnull::resolve_type.
    
    Repro test added.
    
    ANALYSIS (refer to added test case):
    ------------------------------------
    
    The calculation of char_length in Item_func_ifnull::resolve_type is
    used as an argument to fix_char_length. The calculation forgets to add
    space for decimal point if present.
    
    The calculation for NULLIF looks like this:
    
    Item_func_ifnull::resolve_type:
       :
       char_length= max(len0, len1) + decimals + (unsigned_flag ? 0 : 1);
    
    where "decimals" is 2 in our case.
    
    In the similar computation of IF, this is calculated with another (correct)
    code path:
    
    cf Item_func_if::resolve_type ->
       fix_num_type_shared_for_case->
       Item_func::count_decimal_length ->
       my_decimal_precision_to_length_no_truncation
    
    which has this line:
       uint32 retval= (uint32)(precision + (scale > 0 ? 1 : 0) +
                      (unsigned_flag || !precision ? 0 : 1));
    
    the "scale" here being the number of digits after point, making the
    calculation add 1 in our case, since "scale" is 2.
    
    Change-Id: If54750e9e3b456d063591a359514d69f2bc0c510
Loading