Skip to content
  • Ajo Robert's avatar
    852d7b37
    Bug #20201006 spamming show processlist prevents old · 852d7b37
    Ajo Robert authored
    connection threads from cleaning up.
    
    Analysis
    =========
    Issue here is, delay in connection cleanup for which global
    connection counter is already decremented, makes room for
    new connections. Hence more than expected connections are
    observed in the server.
    
    Connections running statement "SHOW PROCESSLIST" or "SELECT
    on INFORMATION_SCHEMA.PROCESSLIST" acquires mutex
    LOCK_thd_remove for reading information of all the connections
    in server. Connections in cleanup phase, acquires mutex to
    remove thread from global thread list. Many such concurrent
    connections increases contention on mutex LOCK_thd_remove.
    
    In connection cleanup phase, connection count is decreased
    first and then the thd is removed from global thd list. This
    order makes new connection (above max_connections) possible
    while existing connections removal is still pending because
    of mutex LOCK_thd_remove being held by show processlist.
    
    Fix:
    =====
    In connection clean phase, thd is removed from the global
    thd list first and then global connection count is
    decremented. Added code to wait for connection_count be
    be zero during shutdown to ensure connection threads are
    done with their task.
    
    Because of this change, tests which verifies PROCESSLIST and
    then status variable "threads_connected" might fail on slower
    machines. As part of this patch, changed test cases to wait
    until "threads_connected" is set to expected value instead
    of PROCESSLIST count.
    852d7b37
    Bug #20201006 spamming show processlist prevents old
    Ajo Robert authored
    connection threads from cleaning up.
    
    Analysis
    =========
    Issue here is, delay in connection cleanup for which global
    connection counter is already decremented, makes room for
    new connections. Hence more than expected connections are
    observed in the server.
    
    Connections running statement "SHOW PROCESSLIST" or "SELECT
    on INFORMATION_SCHEMA.PROCESSLIST" acquires mutex
    LOCK_thd_remove for reading information of all the connections
    in server. Connections in cleanup phase, acquires mutex to
    remove thread from global thread list. Many such concurrent
    connections increases contention on mutex LOCK_thd_remove.
    
    In connection cleanup phase, connection count is decreased
    first and then the thd is removed from global thd list. This
    order makes new connection (above max_connections) possible
    while existing connections removal is still pending because
    of mutex LOCK_thd_remove being held by show processlist.
    
    Fix:
    =====
    In connection clean phase, thd is removed from the global
    thd list first and then global connection count is
    decremented. Added code to wait for connection_count be
    be zero during shutdown to ensure connection threads are
    done with their task.
    
    Because of this change, tests which verifies PROCESSLIST and
    then status variable "threads_connected" might fail on slower
    machines. As part of this patch, changed test cases to wait
    until "threads_connected" is set to expected value instead
    of PROCESSLIST count.
Loading