Skip to content
  • Frazer Clement's avatar
    4c949a26
    Add NdbApi usage counters to NdbApi and MySQLD · 4c949a26
    Frazer Clement authored
    A set of counters are maintained by each Ndb object in the NdbApi :
    
       WaitExecCompleteCount    
         Number of times blocked waiting for key / scan start operation execution to complete.
         execute([no]commit|rollback), pollNdb() for async transactions
         Includes implicit executes for Blobs + autoincrement not visible to users.
       WaitScanResultCount      
         Number of times blocked waiting for Scan related response (batch(es), close acknowledgement etc)
       WaitMetaRequestCount     
         Number of times blocked waiting for Meta data based signal (TC Seize, DDL etc)
    
       WaitNanosCount          
         Total nanoseconds elapsed waiting for a signal of some sort.  
         Achieved resolution is OS dependent.
    
       BytesSentCount           
         Bytes sent to data nodes
       BytesRecvdCount          
         Bytes received from data nodes
    
       TransStartCount          
         Transactions started
       TransCommitCount
         Transactions committed
       TransAbortCount
         Transactions explicitly aborted
       TransCloseCount          
         Transactions closed 
         (Not all are Committed/Aborted)
       PkOpCount
         Primary key based ops
         (Read, Insert, Update, Write, Unlock, Delete)                
         Includes Blob part table ops, implicit unlock ops, autoincrement ops etc
       UkOpCount                
         Unique key based ops
         (Read, Update, Write, Delete)
       TableScanCount
         Table scans started
       RangeScanCount
         Range (Ordered index) scans started
    
       PrunedScanCount          
         Count of scans pruned to 1 partition 
       ScanBatchCount           
         Number of row batches received.  1 batch = set of rows from 1 table fragment.
       ReadRowCount             
         Number of rows read in total from pk, uk + scans.
       TransLocalReadRowCount   
         Number of rows read from the same node the transaction was started on.
         Can indicate performance of 'distribution awareness'.
    
       DataEventsRecvdCount     
         Number of row change events received over the NdbApi event Api
       NonDataEventsRecvdCount  
         Number of non row change events received over the NdbApi event Api
       EventBytesRecvdCount     
         Number of bytes of events received over the NdbApi event Api
    
    These per-Ndb counters can be used to understand and reason about the behaviour
    of NdbApi applications.
    
    Additionally, methods are added to ndb_cluster_connection to sum counters across all
    Ndb objects created by the connection.  These can be used to get a snapshot of the
    NdbApi related activity initiated over a particular connection.
    
    These counters are exposed to SQL users of MySQLD as status variables of
    4 kinds, all with the Ndb_api prefix.  As normal, all status variables can
    also be read and queried from the information_schema.session_status 
    (or information_schema.global_status) tables.
    
    The counters are based on the underlying Ndb objects, and are only reset
    when a new Ndb object is created.  They are not reset by FLUSH STATUS
    commands.
    Rather than relying on absolute values of counters, they should be used 
    to observe a difference over time.
    
    Session status variables, all suffixed with _session.
    -----------------------------------------------------
    
    These are the variable values for the user's current session.  
    Useful for understanding/debugging performance.
    
    Ndb_api_bytes_received_session	
    Ndb_api_bytes_sent_session	
    Ndb_api_pk_op_count_session	
    Ndb_api_pruned_scan_count_session	
    Ndb_api_range_scan_count_session	
    Ndb_api_read_row_count_session	
    Ndb_api_scan_batch_count_session	
    Ndb_api_table_scan_count_session	
    Ndb_api_trans_abort_count_session	
    Ndb_api_trans_close_count_session	
    Ndb_api_trans_commit_count_session	
    Ndb_api_trans_local_read_row_count_session	
    Ndb_api_trans_start_count_session	
    Ndb_api_uk_op_count_session	
    Ndb_api_wait_exec_complete_count_session	
    Ndb_api_wait_meta_request_count_session	
    Ndb_api_wait_nanos_session	
    Ndb_api_wait_scan_result_count_session	
    
    Global status variables, no suffix
    ----------------------------------
    These are for all MySQL connections in the MySQL server.  
    Useful for getting a notion about throughput, bottlenecks, system 
    performance etc.
    Note that slave, injector, utility thread etc are included here.
    Also all connections in a MySQL Server using multiple cluster connections
    are summed.
    
    Ndb_api_bytes_received	
    Ndb_api_bytes_sent	
    Ndb_api_event_bytes_count	
    Ndb_api_event_data_count	
    Ndb_api_event_nondata_count	
    Ndb_api_pk_op_count	
    Ndb_api_pruned_scan_count	
    Ndb_api_range_scan_count	
    Ndb_api_read_row_count	
    Ndb_api_scan_batch_count	
    Ndb_api_table_scan_count	
    Ndb_api_trans_abort_count	
    Ndb_api_trans_close_count	
    Ndb_api_trans_commit_count	
    Ndb_api_trans_local_read_row_count	
    Ndb_api_trans_start_count	
    Ndb_api_uk_op_count	
    Ndb_api_wait_exec_complete_count	
    Ndb_api_wait_meta_request_count	
    Ndb_api_wait_nanos	
    Ndb_api_wait_scan_result_count	
    
    Slave variables, suffixed _slave
    --------------------------------
    These are the counters for the NdbApi work performed by 
    the Slave SQL thread.  If the MySQLD is not acting as a
    Slave then these will remain at 0.
    Useful for verifying / understanding Slave NdbAPI
     performance.
    
    Ndb_api_bytes_received_slave	
    Ndb_api_bytes_sent_slave	
    Ndb_api_pk_op_count_slave	
    Ndb_api_pruned_scan_count_slave	
    Ndb_api_range_scan_count_slave	
    Ndb_api_read_row_count_slave	
    Ndb_api_scan_batch_count_slave	
    Ndb_api_table_scan_count_slave	
    Ndb_api_trans_abort_count_slave	
    Ndb_api_trans_close_count_slave	
    Ndb_api_trans_commit_count_slave	
    Ndb_api_trans_local_read_row_count_slave	
    Ndb_api_trans_start_count_slave	
    Ndb_api_uk_op_count_slave	
    Ndb_api_wait_exec_complete_count_slave	
    Ndb_api_wait_meta_request_count_slave	
    Ndb_api_wait_nanos_slave	
    Ndb_api_wait_scan_result_count_slave	
    
    Binlog injector thread variables, suffixed _injector
    ----------------------------------------------------
    These are the counters for the Binlog injector thread which listens
    to events from the Cluster and writes them into the Binlog.
    In current MySQLD releases, this is the only consumer of NdbApi events
    in the process, so these counters will be the same as the Global counters.
    This may change in future.
    Useful for quantifying the rate of data and schema change in the Cluster.
    
    Ndb_api_event_bytes_count_injector	
    Ndb_api_event_data_count_injector	
    Ndb_api_event_nondata_count_injector	
    4c949a26
    Add NdbApi usage counters to NdbApi and MySQLD
    Frazer Clement authored
    A set of counters are maintained by each Ndb object in the NdbApi :
    
       WaitExecCompleteCount    
         Number of times blocked waiting for key / scan start operation execution to complete.
         execute([no]commit|rollback), pollNdb() for async transactions
         Includes implicit executes for Blobs + autoincrement not visible to users.
       WaitScanResultCount      
         Number of times blocked waiting for Scan related response (batch(es), close acknowledgement etc)
       WaitMetaRequestCount     
         Number of times blocked waiting for Meta data based signal (TC Seize, DDL etc)
    
       WaitNanosCount          
         Total nanoseconds elapsed waiting for a signal of some sort.  
         Achieved resolution is OS dependent.
    
       BytesSentCount           
         Bytes sent to data nodes
       BytesRecvdCount          
         Bytes received from data nodes
    
       TransStartCount          
         Transactions started
       TransCommitCount
         Transactions committed
       TransAbortCount
         Transactions explicitly aborted
       TransCloseCount          
         Transactions closed 
         (Not all are Committed/Aborted)
       PkOpCount
         Primary key based ops
         (Read, Insert, Update, Write, Unlock, Delete)                
         Includes Blob part table ops, implicit unlock ops, autoincrement ops etc
       UkOpCount                
         Unique key based ops
         (Read, Update, Write, Delete)
       TableScanCount
         Table scans started
       RangeScanCount
         Range (Ordered index) scans started
    
       PrunedScanCount          
         Count of scans pruned to 1 partition 
       ScanBatchCount           
         Number of row batches received.  1 batch = set of rows from 1 table fragment.
       ReadRowCount             
         Number of rows read in total from pk, uk + scans.
       TransLocalReadRowCount   
         Number of rows read from the same node the transaction was started on.
         Can indicate performance of 'distribution awareness'.
    
       DataEventsRecvdCount     
         Number of row change events received over the NdbApi event Api
       NonDataEventsRecvdCount  
         Number of non row change events received over the NdbApi event Api
       EventBytesRecvdCount     
         Number of bytes of events received over the NdbApi event Api
    
    These per-Ndb counters can be used to understand and reason about the behaviour
    of NdbApi applications.
    
    Additionally, methods are added to ndb_cluster_connection to sum counters across all
    Ndb objects created by the connection.  These can be used to get a snapshot of the
    NdbApi related activity initiated over a particular connection.
    
    These counters are exposed to SQL users of MySQLD as status variables of
    4 kinds, all with the Ndb_api prefix.  As normal, all status variables can
    also be read and queried from the information_schema.session_status 
    (or information_schema.global_status) tables.
    
    The counters are based on the underlying Ndb objects, and are only reset
    when a new Ndb object is created.  They are not reset by FLUSH STATUS
    commands.
    Rather than relying on absolute values of counters, they should be used 
    to observe a difference over time.
    
    Session status variables, all suffixed with _session.
    -----------------------------------------------------
    
    These are the variable values for the user's current session.  
    Useful for understanding/debugging performance.
    
    Ndb_api_bytes_received_session	
    Ndb_api_bytes_sent_session	
    Ndb_api_pk_op_count_session	
    Ndb_api_pruned_scan_count_session	
    Ndb_api_range_scan_count_session	
    Ndb_api_read_row_count_session	
    Ndb_api_scan_batch_count_session	
    Ndb_api_table_scan_count_session	
    Ndb_api_trans_abort_count_session	
    Ndb_api_trans_close_count_session	
    Ndb_api_trans_commit_count_session	
    Ndb_api_trans_local_read_row_count_session	
    Ndb_api_trans_start_count_session	
    Ndb_api_uk_op_count_session	
    Ndb_api_wait_exec_complete_count_session	
    Ndb_api_wait_meta_request_count_session	
    Ndb_api_wait_nanos_session	
    Ndb_api_wait_scan_result_count_session	
    
    Global status variables, no suffix
    ----------------------------------
    These are for all MySQL connections in the MySQL server.  
    Useful for getting a notion about throughput, bottlenecks, system 
    performance etc.
    Note that slave, injector, utility thread etc are included here.
    Also all connections in a MySQL Server using multiple cluster connections
    are summed.
    
    Ndb_api_bytes_received	
    Ndb_api_bytes_sent	
    Ndb_api_event_bytes_count	
    Ndb_api_event_data_count	
    Ndb_api_event_nondata_count	
    Ndb_api_pk_op_count	
    Ndb_api_pruned_scan_count	
    Ndb_api_range_scan_count	
    Ndb_api_read_row_count	
    Ndb_api_scan_batch_count	
    Ndb_api_table_scan_count	
    Ndb_api_trans_abort_count	
    Ndb_api_trans_close_count	
    Ndb_api_trans_commit_count	
    Ndb_api_trans_local_read_row_count	
    Ndb_api_trans_start_count	
    Ndb_api_uk_op_count	
    Ndb_api_wait_exec_complete_count	
    Ndb_api_wait_meta_request_count	
    Ndb_api_wait_nanos	
    Ndb_api_wait_scan_result_count	
    
    Slave variables, suffixed _slave
    --------------------------------
    These are the counters for the NdbApi work performed by 
    the Slave SQL thread.  If the MySQLD is not acting as a
    Slave then these will remain at 0.
    Useful for verifying / understanding Slave NdbAPI
     performance.
    
    Ndb_api_bytes_received_slave	
    Ndb_api_bytes_sent_slave	
    Ndb_api_pk_op_count_slave	
    Ndb_api_pruned_scan_count_slave	
    Ndb_api_range_scan_count_slave	
    Ndb_api_read_row_count_slave	
    Ndb_api_scan_batch_count_slave	
    Ndb_api_table_scan_count_slave	
    Ndb_api_trans_abort_count_slave	
    Ndb_api_trans_close_count_slave	
    Ndb_api_trans_commit_count_slave	
    Ndb_api_trans_local_read_row_count_slave	
    Ndb_api_trans_start_count_slave	
    Ndb_api_uk_op_count_slave	
    Ndb_api_wait_exec_complete_count_slave	
    Ndb_api_wait_meta_request_count_slave	
    Ndb_api_wait_nanos_slave	
    Ndb_api_wait_scan_result_count_slave	
    
    Binlog injector thread variables, suffixed _injector
    ----------------------------------------------------
    These are the counters for the Binlog injector thread which listens
    to events from the Cluster and writes them into the Binlog.
    In current MySQLD releases, this is the only consumer of NdbApi events
    in the process, so these counters will be the same as the Global counters.
    This may change in future.
    Useful for quantifying the rate of data and schema change in the Cluster.
    
    Ndb_api_event_bytes_count_injector	
    Ndb_api_event_data_count_injector	
    Ndb_api_event_nondata_count_injector	
Loading