Skip to content
  • Sven Sandberg's avatar
    bd6771b7
    BUG#17827018: SLAVE WITH GTID_MODE=OFF LOSES GTID OF GTID-TRANSACTION IN RELAY LOG · bd6771b7
    Sven Sandberg authored
    BUG#17813449: SLAVE WITH GTID_MODE=ON GENERATES A GTID FOR ANONYMOUS TRANSACTIONS IN RELAY LOG
    
    Background:
    In replication, all transactions that have a GTID are supposed to be
    replicated with the same GTID, and all transactions that don't have a
    GTID are supposed to be replicated without a GTID. If the current
    GTID_MODE prevents preserving GTIDs or non-GTIDs in this way, an error
    should be generated.
    
    Problem 1:
    If a transaction was replicated to the slave and stored in the relay
    log but not executed, and then the slave was restarted with the
    opposite GTID_MODE, then the transaction would be executed and
    assigned a wrong GTID:
     1.1. (BUG#17813449) If the transaction was anonymous and the slave
          changed from GTID_MODE=OFF to ON, then a new GTID was generated
          for the transactions.
     1.2. (BUG#17827018) If the transaction was a GTID-transaction and the
          slave changed from GTID_MODE=ON to OFF, then the transaction
          would lose its GTID.  If the slave was built in debug-mode, an
          assertion was raised.
    The correct behavior in both cases is to generate an error and not
    execute the transaction.
    
    Problem 2:
    The server would generate an error message when it started with
    gtid_mode=off and there exists GTIDs in some old binary log.
    
    Problem 3:
    Gtid_log_event::do_apply_event did not check if the event was an
    Anonymous_gtid_log_event. So an Anonymous_gtid_log_event would be
    applied like a Gtid_log_event and thus it would set a wrong gtid on
    the transaction.
    
    Problem 4:
    Implicit commits inside a transaction is disallowed in
    GTID-transactions, and generate an error.  However, the check for this
    was too strong: it generated an error also if
    gtid_mode='anonymous'. That will cause replication errors in NEW->OLD
    once the server generates Anonymous_gtid_log_events.
    
    Fix 1.1:
    The slave applier thread has to use gtid_next=anonymous when it
    executes transactions from an old relay log. However, the slave only
    knows that it executes an old relay log by the absence of
    Gtid_log_event before a transaction. So the solution is:
     1.1.1. Set thd->variables.gtid_next.type to the new value
            NOT_YET_DETERMINED_GROUP when executing a
            Format_description_log_event originating from another server,
            and:
             1.1.1.1. In Gtid_log_event::do_apply_event, silently convert
                      this value to the correct GTID (this gets executed
                      for relay logs generated with GTID_MODE=ON).
                      Generate an error if gtid_mode=off.
             1.1.1.2. In gtid_pre_statement_checks, silently convert this
                      value to ANONYMOUS (this gets executed for relay
                      logs generated with GTID_MODE=OFF).  Generate an
                      error if gtid_mode=on. (This error fixes the bug.)
     1.1.2. Setting NOT_YET_DETERMINED_GROUP must be done both for
            parallel slave workers and for the legacy SQL thread; thus
            both in rpl_rli.cc and rpl_rli_pdb.h.
     1.1.3. In order for the NOT_YET_DETERMINED_GROUP flag to be correctly
            converted to ANONYMOUS_GROUP in RBR, we need to call
            gtid_pre_statement_checks for row events even if opt_bin_log
            is off.
     1.1.4. The change in row events would generate an assertion for
            setting the diagnostics area twice, because
            gtid_pre_statement_checks was called before
            mysql_reset_thd_for_next_command. Thus, we move the call
            to gtid_pre_statement_checks to just after
            mysql_reset_thd_for_next_command in
            Rows_log_event::do_apply_event
     1.1.5. When using mysqlbinlog --skip-gtids, the user expects a client
            that replays the output to generate new GTIDs. However, with
            this change, the client is 'too smart'; when it processes the
            Format_description_log_event it sets gtid_next to
            NOT_YET_DETERMINED, and later when it executes a statement
            without having seen any GTID (as the GTIDs are filtered out by
            --skip-gtids), it will force the transaction to be
            anonymous. So when --skip-gtids is given, mysqlbinlog now
            prints a SET GTID_NEXT='AUTOMATIC' statement, just after the
            BINLOG base64 statement.
    
    Fix 1.2:
     1.2.1. Remove the assertion: It is perfectly possible that the
            applier thread reads a Gtid_log_event when gtid_mode is not
            off.
     1.2.2. Add a check to ensure error is generated when trying to
            execute a Gtid_log_event when gtid_mode=off.
    
    Fix 2:
    This error message should never be generated, so just remove it.
    
    Fix 3:
    In Gtid_log_event::do_apply_event, check if
    spec.type==ANONYMOUS_GROUP, if yes set gtid_next='anonymous'. If
    gtid_mode=on, generate an error.
    
    Fix 4:
    Generate error only if gtid_next is a GTID.
    bd6771b7
    BUG#17827018: SLAVE WITH GTID_MODE=OFF LOSES GTID OF GTID-TRANSACTION IN RELAY LOG
    Sven Sandberg authored
    BUG#17813449: SLAVE WITH GTID_MODE=ON GENERATES A GTID FOR ANONYMOUS TRANSACTIONS IN RELAY LOG
    
    Background:
    In replication, all transactions that have a GTID are supposed to be
    replicated with the same GTID, and all transactions that don't have a
    GTID are supposed to be replicated without a GTID. If the current
    GTID_MODE prevents preserving GTIDs or non-GTIDs in this way, an error
    should be generated.
    
    Problem 1:
    If a transaction was replicated to the slave and stored in the relay
    log but not executed, and then the slave was restarted with the
    opposite GTID_MODE, then the transaction would be executed and
    assigned a wrong GTID:
     1.1. (BUG#17813449) If the transaction was anonymous and the slave
          changed from GTID_MODE=OFF to ON, then a new GTID was generated
          for the transactions.
     1.2. (BUG#17827018) If the transaction was a GTID-transaction and the
          slave changed from GTID_MODE=ON to OFF, then the transaction
          would lose its GTID.  If the slave was built in debug-mode, an
          assertion was raised.
    The correct behavior in both cases is to generate an error and not
    execute the transaction.
    
    Problem 2:
    The server would generate an error message when it started with
    gtid_mode=off and there exists GTIDs in some old binary log.
    
    Problem 3:
    Gtid_log_event::do_apply_event did not check if the event was an
    Anonymous_gtid_log_event. So an Anonymous_gtid_log_event would be
    applied like a Gtid_log_event and thus it would set a wrong gtid on
    the transaction.
    
    Problem 4:
    Implicit commits inside a transaction is disallowed in
    GTID-transactions, and generate an error.  However, the check for this
    was too strong: it generated an error also if
    gtid_mode='anonymous'. That will cause replication errors in NEW->OLD
    once the server generates Anonymous_gtid_log_events.
    
    Fix 1.1:
    The slave applier thread has to use gtid_next=anonymous when it
    executes transactions from an old relay log. However, the slave only
    knows that it executes an old relay log by the absence of
    Gtid_log_event before a transaction. So the solution is:
     1.1.1. Set thd->variables.gtid_next.type to the new value
            NOT_YET_DETERMINED_GROUP when executing a
            Format_description_log_event originating from another server,
            and:
             1.1.1.1. In Gtid_log_event::do_apply_event, silently convert
                      this value to the correct GTID (this gets executed
                      for relay logs generated with GTID_MODE=ON).
                      Generate an error if gtid_mode=off.
             1.1.1.2. In gtid_pre_statement_checks, silently convert this
                      value to ANONYMOUS (this gets executed for relay
                      logs generated with GTID_MODE=OFF).  Generate an
                      error if gtid_mode=on. (This error fixes the bug.)
     1.1.2. Setting NOT_YET_DETERMINED_GROUP must be done both for
            parallel slave workers and for the legacy SQL thread; thus
            both in rpl_rli.cc and rpl_rli_pdb.h.
     1.1.3. In order for the NOT_YET_DETERMINED_GROUP flag to be correctly
            converted to ANONYMOUS_GROUP in RBR, we need to call
            gtid_pre_statement_checks for row events even if opt_bin_log
            is off.
     1.1.4. The change in row events would generate an assertion for
            setting the diagnostics area twice, because
            gtid_pre_statement_checks was called before
            mysql_reset_thd_for_next_command. Thus, we move the call
            to gtid_pre_statement_checks to just after
            mysql_reset_thd_for_next_command in
            Rows_log_event::do_apply_event
     1.1.5. When using mysqlbinlog --skip-gtids, the user expects a client
            that replays the output to generate new GTIDs. However, with
            this change, the client is 'too smart'; when it processes the
            Format_description_log_event it sets gtid_next to
            NOT_YET_DETERMINED, and later when it executes a statement
            without having seen any GTID (as the GTIDs are filtered out by
            --skip-gtids), it will force the transaction to be
            anonymous. So when --skip-gtids is given, mysqlbinlog now
            prints a SET GTID_NEXT='AUTOMATIC' statement, just after the
            BINLOG base64 statement.
    
    Fix 1.2:
     1.2.1. Remove the assertion: It is perfectly possible that the
            applier thread reads a Gtid_log_event when gtid_mode is not
            off.
     1.2.2. Add a check to ensure error is generated when trying to
            execute a Gtid_log_event when gtid_mode=off.
    
    Fix 2:
    This error message should never be generated, so just remove it.
    
    Fix 3:
    In Gtid_log_event::do_apply_event, check if
    spec.type==ANONYMOUS_GROUP, if yes set gtid_next='anonymous'. If
    gtid_mode=on, generate an error.
    
    Fix 4:
    Generate error only if gtid_next is a GTID.
Loading