-
Dag Wanvik authored
IFNULL is used with type combination {decimal,int}, the resulting decimal loses on digit after the decimal point in the calculation of ::max_length. This leads to a loss of a decimal digit after point in the corner case of GROUP BY which needs a temporary file, e.g. this kind of query: SELECT IFNULL(e2,i1) in, SUM(d1) FROM t GROUP BY e2,i1 ORDER BY in; This bug will be more visible when we get window functions which rely on temporary files in non-trivial cases. FIX ------------------------------ Add space for decimal point to the calculation in Item_func_ifnull::resolve_type. Repro test added. ANALYSIS (refer to added test case): ------------------------------------ The calculation of char_length in Item_func_ifnull::resolve_type is used as an argument to fix_char_length. The calculation forgets to add space for decimal point if present. The calculation for NULLIF looks like this: Item_func_ifnull::resolve_type: : char_length= max(len0, len1) + decimals + (unsigned_flag ? 0 : 1); where "decimals" is 2 in our case. In the similar computation of IF, this is calculated with another (correct) code path: cf Item_func_if::resolve_type -> fix_num_type_shared_for_case-> Item_func::count_decimal_length -> my_decimal_precision_to_length_no_truncation which has this line: uint32 retval= (uint32)(precision + (scale > 0 ? 1 : 0) + (unsigned_flag || !precision ? 0 : 1)); the "scale" here being the number of digits after point, making the calculation add 1 in our case, since "scale" is 2. Change-Id: If54750e9e3b456d063591a359514d69f2bc0c510
Dag Wanvik authoredIFNULL is used with type combination {decimal,int}, the resulting decimal loses on digit after the decimal point in the calculation of ::max_length. This leads to a loss of a decimal digit after point in the corner case of GROUP BY which needs a temporary file, e.g. this kind of query: SELECT IFNULL(e2,i1) in, SUM(d1) FROM t GROUP BY e2,i1 ORDER BY in; This bug will be more visible when we get window functions which rely on temporary files in non-trivial cases. FIX ------------------------------ Add space for decimal point to the calculation in Item_func_ifnull::resolve_type. Repro test added. ANALYSIS (refer to added test case): ------------------------------------ The calculation of char_length in Item_func_ifnull::resolve_type is used as an argument to fix_char_length. The calculation forgets to add space for decimal point if present. The calculation for NULLIF looks like this: Item_func_ifnull::resolve_type: : char_length= max(len0, len1) + decimals + (unsigned_flag ? 0 : 1); where "decimals" is 2 in our case. In the similar computation of IF, this is calculated with another (correct) code path: cf Item_func_if::resolve_type -> fix_num_type_shared_for_case-> Item_func::count_decimal_length -> my_decimal_precision_to_length_no_truncation which has this line: uint32 retval= (uint32)(precision + (scale > 0 ? 1 : 0) + (unsigned_flag || !precision ? 0 : 1)); the "scale" here being the number of digits after point, making the calculation add 1 in our case, since "scale" is 2. Change-Id: If54750e9e3b456d063591a359514d69f2bc0c510
Loading