Skip to content
  • Jusufadis Bakamovic's avatar
    a36bd8b2
    Bug#30050452: CRASH WHEN EXECUTING "SELECT SLEEP(10)" THROUGH CURSOR (WITH THREAD-POOL) · a36bd8b2
    Jusufadis Bakamovic authored
    
    
    Problem:
      * TempTable segfaults in certain scenarios when ThreadPool (TP) plugin
        is used.
      * Cursor-protocol only accentuates the crash condition more.
    
    Analysis:
      * TempTable implements two of its critical parts through the means of
        thread-local variables, namely its data-backend (storage for tables)
        and custom memory allocator.
      * This becomes a design and functional issue as soon as MySQL server is
        run with the ThreadPool (TP) plugin because presence of that plugin
        will change the traditional thread-handling MySQL model, which is
        1-thread-per-client-connection, to the alternative model where this
        assumption no longer holds and code relying on such assumption
        automatically becomes broken.
      * In particular, TP plugin will not be allocating a new thread for each
        new connection arriving but instead it will pick a thread from
        pre-allocated pool of threads and assign one to the statement
        execution engine.
      * In practice, this means that it is possible that statements within
        one client connection are now served with a multitude of different
        threads (but not simultaneously), which with traditional
        thread-handling model, such condition was not possible (it was always
        the same thread).
      * Furthermore, cursor-protocol makes it possible that even an execution
        of a single statement within one client connection is served by
        different threads.
          * E.g. Cursor-protocol enables fetching the result set of a query
            (data) in chunks. And TP plugin makes a situation possible where
            each chunk is fetched from a different thread.
      * TL;DR thread-local-storage variables cannot really co-exist with TP
        plugin in their natural form.
    
    Solution:
      * Replace TempTable parts which are implemented in terms of
        thread-local-storage variables with a solution which:
          * Is functionally correct in both cases, with and without TP plugin
            running.
          * Does not sacrifice the overall performance at any rate.
          * Does not lock the design but keeps the door open for future
            advancements (e.g. more control over memory consumption).
      * Basic idea is to:
        * Organize data-backend (table-storage) into shards
        * Ditto for shared-blocks (custom memory allocator external state)
        * Use unique connection identifier to map between connections and
          their corresponding shards
    
    Addendum:
      * This patch also fixes some other issues which have been reported but
        which are directly or indirectly caused by the same limitation in
        design:
          * BUG #30050420 CRASHES WHEN CLOSING CONNECTION WITH UNCLOSED
            CURSOR (WITH THREAD-POOL)
          * BUG #31116036 TEMPTABLE WASTES 1MB FOR EACH CONNECTION IN THREAD
            CACHE
          * BUG #31471863 TEMPTABLE ENGINE USES MUCH MORE MEMORY THAN MEMORY
            ENGINE FOR TEMPORARY TABLE
    
    Reviewed-by: default avatarPawel Olchawa <pawel.olchawa@oracle.com>
    RB: 24497
    a36bd8b2
    Bug#30050452: CRASH WHEN EXECUTING "SELECT SLEEP(10)" THROUGH CURSOR (WITH THREAD-POOL)
    Jusufadis Bakamovic authored
    
    
    Problem:
      * TempTable segfaults in certain scenarios when ThreadPool (TP) plugin
        is used.
      * Cursor-protocol only accentuates the crash condition more.
    
    Analysis:
      * TempTable implements two of its critical parts through the means of
        thread-local variables, namely its data-backend (storage for tables)
        and custom memory allocator.
      * This becomes a design and functional issue as soon as MySQL server is
        run with the ThreadPool (TP) plugin because presence of that plugin
        will change the traditional thread-handling MySQL model, which is
        1-thread-per-client-connection, to the alternative model where this
        assumption no longer holds and code relying on such assumption
        automatically becomes broken.
      * In particular, TP plugin will not be allocating a new thread for each
        new connection arriving but instead it will pick a thread from
        pre-allocated pool of threads and assign one to the statement
        execution engine.
      * In practice, this means that it is possible that statements within
        one client connection are now served with a multitude of different
        threads (but not simultaneously), which with traditional
        thread-handling model, such condition was not possible (it was always
        the same thread).
      * Furthermore, cursor-protocol makes it possible that even an execution
        of a single statement within one client connection is served by
        different threads.
          * E.g. Cursor-protocol enables fetching the result set of a query
            (data) in chunks. And TP plugin makes a situation possible where
            each chunk is fetched from a different thread.
      * TL;DR thread-local-storage variables cannot really co-exist with TP
        plugin in their natural form.
    
    Solution:
      * Replace TempTable parts which are implemented in terms of
        thread-local-storage variables with a solution which:
          * Is functionally correct in both cases, with and without TP plugin
            running.
          * Does not sacrifice the overall performance at any rate.
          * Does not lock the design but keeps the door open for future
            advancements (e.g. more control over memory consumption).
      * Basic idea is to:
        * Organize data-backend (table-storage) into shards
        * Ditto for shared-blocks (custom memory allocator external state)
        * Use unique connection identifier to map between connections and
          their corresponding shards
    
    Addendum:
      * This patch also fixes some other issues which have been reported but
        which are directly or indirectly caused by the same limitation in
        design:
          * BUG #30050420 CRASHES WHEN CLOSING CONNECTION WITH UNCLOSED
            CURSOR (WITH THREAD-POOL)
          * BUG #31116036 TEMPTABLE WASTES 1MB FOR EACH CONNECTION IN THREAD
            CACHE
          * BUG #31471863 TEMPTABLE ENGINE USES MUCH MORE MEMORY THAN MEMORY
            ENGINE FOR TEMPORARY TABLE
    
    Reviewed-by: default avatarPawel Olchawa <pawel.olchawa@oracle.com>
    RB: 24497
Loading