-
Chaithra Gopalareddy authored
Bug#17413040 : USING MANY WHERE CONDITIONS MAKES RANGE SCAN DISABLED Problem: Currently range optimizer checks for excessive memory consumption in a minimalistic way by checking on the MAX_SEL_ARGS created by clone(). This leads to two problems, 1. As there are other ways to create SEL_ARG objects and also other ways of memory consumption during range analysis, current check will not suffice and as a result range analysis can take excessive memory 2. If a where condition is written in such a way that only clone() is invoked to create SEL_ARG objects, it becomes very restrictive as in the Bug#17413040. In this case, range optimizer might consume memory, but the plan generated is much more efficient than the current plan. Solution: In case of Bug#17769777 range optimizer requires a lot of memory and CPU resources because of the conditions present in the query. Currently there is no way to limit the range optimizer memory usage in such cases. But given that range optimizer uses its own memroot to allocate memory, we are putting a cap on this memroot's memory consumption. To keep a check on the memory consumption, a user configurable session variable named "range_optimizer_max_mem_size" is introduced. A value of "0" means there is no limit on the memory consumption. A new member 'max_capacity' is introduced in mem_root structure to limit the memory consumption for a mem_root. A new member 'allocated_size' keeps track of the memory being allocated in mem_root Helper functions like is_mem_avaiable() - to check if the requested memory can be allocated, set_memroot_max_capacity() - Max capacity for this memroot, set_memroot_error_handling() - to report error when capacity is exceeded, are also added. Before allocation in memroot we check if memory is available. If yes proceed else report error is error_for_capacity_exceeded is set. Otherwise return NULL. All the range optimizer's allocations are now made only in range_optimizer's memroot. Earlier SEL_ARG and SEL_TREE allocations were not done consistently in memroot. Error Handling for Out of memory caused due to exceeding max_capacity: To handle such cases a new error code is added. For the memroot that has max_capacity set, error_for_capacity_exceeded needs to be set for alloc_root to report error when capacity is exceeded. Else it will return NULL. For range optimizer, as the query should not be aborted when this error is thrown, instead a warning should be given to user about it, a Internal_error_handler is introduced. This handler reduces the error to a warning and is given out after the query execution is complete.
Chaithra Gopalareddy authoredBug#17413040 : USING MANY WHERE CONDITIONS MAKES RANGE SCAN DISABLED Problem: Currently range optimizer checks for excessive memory consumption in a minimalistic way by checking on the MAX_SEL_ARGS created by clone(). This leads to two problems, 1. As there are other ways to create SEL_ARG objects and also other ways of memory consumption during range analysis, current check will not suffice and as a result range analysis can take excessive memory 2. If a where condition is written in such a way that only clone() is invoked to create SEL_ARG objects, it becomes very restrictive as in the Bug#17413040. In this case, range optimizer might consume memory, but the plan generated is much more efficient than the current plan. Solution: In case of Bug#17769777 range optimizer requires a lot of memory and CPU resources because of the conditions present in the query. Currently there is no way to limit the range optimizer memory usage in such cases. But given that range optimizer uses its own memroot to allocate memory, we are putting a cap on this memroot's memory consumption. To keep a check on the memory consumption, a user configurable session variable named "range_optimizer_max_mem_size" is introduced. A value of "0" means there is no limit on the memory consumption. A new member 'max_capacity' is introduced in mem_root structure to limit the memory consumption for a mem_root. A new member 'allocated_size' keeps track of the memory being allocated in mem_root Helper functions like is_mem_avaiable() - to check if the requested memory can be allocated, set_memroot_max_capacity() - Max capacity for this memroot, set_memroot_error_handling() - to report error when capacity is exceeded, are also added. Before allocation in memroot we check if memory is available. If yes proceed else report error is error_for_capacity_exceeded is set. Otherwise return NULL. All the range optimizer's allocations are now made only in range_optimizer's memroot. Earlier SEL_ARG and SEL_TREE allocations were not done consistently in memroot. Error Handling for Out of memory caused due to exceeding max_capacity: To handle such cases a new error code is added. For the memroot that has max_capacity set, error_for_capacity_exceeded needs to be set for alloc_root to report error when capacity is exceeded. Else it will return NULL. For range optimizer, as the query should not be aborted when this error is thrown, instead a warning should be given to user about it, a Internal_error_handler is introduced. This handler reduces the error to a warning and is given out after the query execution is complete.
Loading