Skip to content
  • Ajo Robert's avatar
    3b633bae
    Bug #20201006 spamming show processlist prevents old · 3b633bae
    Ajo Robert authored
    connection threads from cleaning up.
    
    Analysis
    =========
    Issue here is, delay in connection cleanup for which global
    connection counter is already decremented, makes room for
    new connections. Hence more than expected connections are
    observed in the server.
    
    Connections running statement "SHOW PROCESSLIST" or "SELECT
    on INFORMATION_SCHEMA.PROCESSLIST" acquires mutex
    LOCK_thd_remove for reading information of all the connections
    in server. Connections in cleanup phase, acquires mutex to
    remove thread from global thread list. Many such concurrent
    connections increases contention on mutex LOCK_thd_remove.
    
    In connection cleanup phase, connection count is decreased
    first and then the thd is removed from global thd list. This
    order makes new connection (above max_connections) possible
    while existing connections removal is still pending because
    of mutex LOCK_thd_remove being held by show processlist.
    
    Fix:
    =====
    In connection clean phase, thd is removed from the global
    thd list first and then global connection count is
    decremented. Added code to wait for connection_count be
    be zero during shutdown to ensure connection threads are
    done with their task.
    3b633bae
    Bug #20201006 spamming show processlist prevents old
    Ajo Robert authored
    connection threads from cleaning up.
    
    Analysis
    =========
    Issue here is, delay in connection cleanup for which global
    connection counter is already decremented, makes room for
    new connections. Hence more than expected connections are
    observed in the server.
    
    Connections running statement "SHOW PROCESSLIST" or "SELECT
    on INFORMATION_SCHEMA.PROCESSLIST" acquires mutex
    LOCK_thd_remove for reading information of all the connections
    in server. Connections in cleanup phase, acquires mutex to
    remove thread from global thread list. Many such concurrent
    connections increases contention on mutex LOCK_thd_remove.
    
    In connection cleanup phase, connection count is decreased
    first and then the thd is removed from global thd list. This
    order makes new connection (above max_connections) possible
    while existing connections removal is still pending because
    of mutex LOCK_thd_remove being held by show processlist.
    
    Fix:
    =====
    In connection clean phase, thd is removed from the global
    thd list first and then global connection count is
    decremented. Added code to wait for connection_count be
    be zero during shutdown to ensure connection threads are
    done with their task.
Loading