-
Oystein Grovlen authored
WHEN USING IN & VAR_POP This bug is caused by the fact that number of decimals in VAR_POP result, was calculated as the number of decimals in the argument (here int => 0) plus 4. Hence, in the example in the bug report, 3.5556 is stored in t2.X. When materialization is off, result of inner query is compared using Arg_comparator::compare_real_fixed() which takes number of decimals into account. When inner query is materialized, no truncation happens and the materialized column contains 3.555555555... The IN-expression is evaluated by doing hash-lookup into the materialized table, and since no record matches 3.5556, the result set is empty. According to the SQL standard the result of VAR_POP should be a floating point number. Hence, the result should not be truncated to a certain number of decimals. This patch changes VAR_POP() and similar functions (VAR_SAMP(), VARIANCE(), STD(), STDDEV_POP(), STDDEV_SAMP(), and STDDEV()) By changing this, results for these functions will never be truncated Hence, materialization should no longer give a different value from what is normally returned to users.
Oystein Grovlen authoredWHEN USING IN & VAR_POP This bug is caused by the fact that number of decimals in VAR_POP result, was calculated as the number of decimals in the argument (here int => 0) plus 4. Hence, in the example in the bug report, 3.5556 is stored in t2.X. When materialization is off, result of inner query is compared using Arg_comparator::compare_real_fixed() which takes number of decimals into account. When inner query is materialized, no truncation happens and the materialized column contains 3.555555555... The IN-expression is evaluated by doing hash-lookup into the materialized table, and since no record matches 3.5556, the result set is empty. According to the SQL standard the result of VAR_POP should be a floating point number. Hence, the result should not be truncated to a certain number of decimals. This patch changes VAR_POP() and similar functions (VAR_SAMP(), VARIANCE(), STD(), STDDEV_POP(), STDDEV_SAMP(), and STDDEV()) By changing this, results for these functions will never be truncated Hence, materialization should no longer give a different value from what is normally returned to users.
Loading