-
Venkatesh Duggirala authored
SMALL SLAVE_PENDING_JOBS_SIZE_MAX Problem: MTS is unable to process an event which is smaller than slave_max_allowed_packet but bigger than slave_pending_jobs_size_max. Analysis and Fix: In the current code , slave_pending_jobs_size_max is hard limit i.e., any packet greater than slave_pending_jobs_size_max is rejected by Coordinator. Slave SQL thread will be stopped with error ER_MTS_EVENT_BIGGER_PENDING_JOBS_SIZE_MAX. Now in this patch, this limitation is changed to soft limit. If there is an event whose size falls in between soft limit (slave_pending_jobs_size_max) and the hard limit (slave_max_allowed_packet), then we consider it as big event and MTS coordinator will wait till all workers queue becomes empty. After it is empty, then this big event will be processed. i.e., only one event of this size will be executed at any given time. All the next events ( big or small) will wait in the coordinator for this big event to finish it's execution. IO thread has already having logic to protect event size with slave_max_allowed_packet size (as the hard limit). When IO thread is downloading the event from the master, if the event size is greater than slave_max_allowed_packet size, IO thread stop by throwing error ER_NET_PACKET_TOO_LARGE. Hence there are no changes required there.
Venkatesh Duggirala authoredSMALL SLAVE_PENDING_JOBS_SIZE_MAX Problem: MTS is unable to process an event which is smaller than slave_max_allowed_packet but bigger than slave_pending_jobs_size_max. Analysis and Fix: In the current code , slave_pending_jobs_size_max is hard limit i.e., any packet greater than slave_pending_jobs_size_max is rejected by Coordinator. Slave SQL thread will be stopped with error ER_MTS_EVENT_BIGGER_PENDING_JOBS_SIZE_MAX. Now in this patch, this limitation is changed to soft limit. If there is an event whose size falls in between soft limit (slave_pending_jobs_size_max) and the hard limit (slave_max_allowed_packet), then we consider it as big event and MTS coordinator will wait till all workers queue becomes empty. After it is empty, then this big event will be processed. i.e., only one event of this size will be executed at any given time. All the next events ( big or small) will wait in the coordinator for this big event to finish it's execution. IO thread has already having logic to protect event size with slave_max_allowed_packet size (as the hard limit). When IO thread is downloading the event from the master, if the event size is greater than slave_max_allowed_packet size, IO thread stop by throwing error ER_NET_PACKET_TOO_LARGE. Hence there are no changes required there.
Loading