-
Jusufadis Bakamovic authored
Problem: * TempTable segfaults in certain scenarios when ThreadPool (TP) plugin is used. * Cursor-protocol only accentuates the crash condition more. Analysis: * TempTable implements two of its critical parts through the means of thread-local variables, namely its data-backend (storage for tables) and custom memory allocator. * This becomes a design and functional issue as soon as MySQL server is run with the ThreadPool (TP) plugin because presence of that plugin will change the traditional thread-handling MySQL model, which is 1-thread-per-client-connection, to the alternative model where this assumption no longer holds and code relying on such assumption automatically becomes broken. * In particular, TP plugin will not be allocating a new thread for each new connection arriving but instead it will pick a thread from pre-allocated pool of threads and assign one to the statement execution engine. * In practice, this means that it is possible that statements within one client connection are now served with a multitude of different threads (but not simultaneously), which with traditional thread-handling model, such condition was not possible (it was always the same thread). * Furthermore, cursor-protocol makes it possible that even an execution of a single statement within one client connection is served by different threads. * E.g. Cursor-protocol enables fetching the result set of a query (data) in chunks. And TP plugin makes a situation possible where each chunk is fetched from a different thread. * TL;DR thread-local-storage variables cannot really co-exist with TP plugin in their natural form. Solution: * Replace TempTable parts which are implemented in terms of thread-local-storage variables with a solution which: * Is functionally correct in both cases, with and without TP plugin running. * Does not sacrifice the overall performance at any rate. * Does not lock the design but keeps the door open for future advancements (e.g. more control over memory consumption). * Basic idea is to: * Organize data-backend (table-storage) into shards * Ditto for shared-blocks (custom memory allocator external state) * Use unique connection identifier to map between connections and their corresponding shards Addendum: * This patch also fixes some other issues which have been reported but which are directly or indirectly caused by the same limitation in design: * BUG #30050420 CRASHES WHEN CLOSING CONNECTION WITH UNCLOSED CURSOR (WITH THREAD-POOL) * BUG #31116036 TEMPTABLE WASTES 1MB FOR EACH CONNECTION IN THREAD CACHE * BUG #31471863 TEMPTABLE ENGINE USES MUCH MORE MEMORY THAN MEMORY ENGINE FOR TEMPORARY TABLE Reviewed-by:
Pawel Olchawa <pawel.olchawa@oracle.com> RB: 24497
Jusufadis Bakamovic authoredProblem: * TempTable segfaults in certain scenarios when ThreadPool (TP) plugin is used. * Cursor-protocol only accentuates the crash condition more. Analysis: * TempTable implements two of its critical parts through the means of thread-local variables, namely its data-backend (storage for tables) and custom memory allocator. * This becomes a design and functional issue as soon as MySQL server is run with the ThreadPool (TP) plugin because presence of that plugin will change the traditional thread-handling MySQL model, which is 1-thread-per-client-connection, to the alternative model where this assumption no longer holds and code relying on such assumption automatically becomes broken. * In particular, TP plugin will not be allocating a new thread for each new connection arriving but instead it will pick a thread from pre-allocated pool of threads and assign one to the statement execution engine. * In practice, this means that it is possible that statements within one client connection are now served with a multitude of different threads (but not simultaneously), which with traditional thread-handling model, such condition was not possible (it was always the same thread). * Furthermore, cursor-protocol makes it possible that even an execution of a single statement within one client connection is served by different threads. * E.g. Cursor-protocol enables fetching the result set of a query (data) in chunks. And TP plugin makes a situation possible where each chunk is fetched from a different thread. * TL;DR thread-local-storage variables cannot really co-exist with TP plugin in their natural form. Solution: * Replace TempTable parts which are implemented in terms of thread-local-storage variables with a solution which: * Is functionally correct in both cases, with and without TP plugin running. * Does not sacrifice the overall performance at any rate. * Does not lock the design but keeps the door open for future advancements (e.g. more control over memory consumption). * Basic idea is to: * Organize data-backend (table-storage) into shards * Ditto for shared-blocks (custom memory allocator external state) * Use unique connection identifier to map between connections and their corresponding shards Addendum: * This patch also fixes some other issues which have been reported but which are directly or indirectly caused by the same limitation in design: * BUG #30050420 CRASHES WHEN CLOSING CONNECTION WITH UNCLOSED CURSOR (WITH THREAD-POOL) * BUG #31116036 TEMPTABLE WASTES 1MB FOR EACH CONNECTION IN THREAD CACHE * BUG #31471863 TEMPTABLE ENGINE USES MUCH MORE MEMORY THAN MEMORY ENGINE FOR TEMPORARY TABLE Reviewed-by:
Pawel Olchawa <pawel.olchawa@oracle.com> RB: 24497
Loading