Skip to content
  • Venkatesh Duggirala's avatar
    3319fcb3
    Bug#18258933 CONCATENATION OF MYSQLBINLOG OUTPUT DOES NOT · 3319fcb3
    Venkatesh Duggirala authored
                   WORK WITH GTID
    
      Problem:  When mysqlbinlog processes and concatenates
      multiple binary log files into one output file, this
      output file is not in useful state. When it is later
      used for point-in-time recovery, it is producing
      ER_GTID_NEXT_TYPE_UNDEFINED_GROUP "ERROR 1837: When
      @@SESSION.GTID_NEXT is set to a GTID, you must explicitly
      set it to a different value after a COMMIT or ROLLBACK.
      Please check GTID_NEXT variable manual page for detailed
      explanation. Current @@SESSION.GTID_NEXT is 'xyz'".
    
      Analysis: When mysqlbinlog processes a GTID enabled
      binary log, it outputs SET GTID_NEXT statements. As
      GTID_NEXT is set to 'undefined' when a transaction
      commits (or DDL commits implicitly), this will leave
      GTID_NEXT as 'undefined' when the server has processed
      all output from mysqlbinlog. Also, there can be some
      anonymous statements/transactions at the beginning of
      binary log (i.e., without SET GTID_NEXT statements).   
      When mysqlbinlog processes and concatenates two (or more)
      binary log files into one output file, the combination
      (GTID_NEXT is undefined at the end of first binary log
      and second binary log can contain anonymous transactions)
      will not work and will result in ERROR 1837
      (ER_GTID_NEXT_TYPE_UNDEFINED_GROUP).
     
      Fix: When mysqlbinlog processes second binary log's
           start_log_event, it can output
           "SET GTID_NEXT=AUTOMATIC" if required i.e., if
           previous binary log is leaving the GTID_NEXT state
           in 'undefined' state. In the old code (before this
           patch), when mysqlbinlog processes multiple binary
           log files from multiple command line arguments, it
           used a new PRINT_EVENT_INFO object for each file,
           so the state at the end of the previous file is lost.
           To solve this problem, this patch changes the logic
           to use the same PRINT_EVENT_INFO structure for all
           command line arguments. And to keep track of
           gtid_next state, we are introducing
           is_gtid_next_valid and is_gtid_next_set variables.
           At the beginning of mysqlbinlog process
           is_gtid_next_set is set to false and
           is_gtid_next_valid is set to true.
           When mysqlbinlog encounters a
             i) GTID_LOG_EVENT: it sets is_gtid_next_set to
                                true and is_gtid_next_valid to
                                true.
            ii) XID_LOG_EVENT
                 /end_group() : it sets is_gtid_next_valid to
                                false if is_gtid_next_set to
                                true.
           And when mysqlbinlog is processing start_log_event,
           if it finds gtid_next is not valid (i.e., undefined)
           then it produces "SET GTID_NEXT=AUTOMATIC" in the
           output file to avoid mentioned
           ER_GTID_NEXT_TYPE_UNDEFINED_GROUP error and it resets
           is_gtid_next_set to false and is_gtid_next_valid to
           true (the initial values). mysqlbinlog also produces
           'SET GTID_NEXT=AUTOMATIC' in the output file at the
           end of processing all the command line arguments if
           it finds GTID_NEXT is in invalid state.
    
           After this patch, if you are merging two or more
           binary log files, you will see two differences in
           the output when compared to the before the patch
    
           a) When mysqlbinlog start processing the second
              binary log file, if required (i.e., if the 
              previous binary log left gtid_next in a invalid
              state), it generates SET GTID_NEXT=AUTOMATIC.
           b) Before the patch, mysqlbinlog outputs session
              variable information for every binary log.
              Now that we will have only one print_event_info
              per run, this session variables value will be
              printed only once *unless there is a change*.
              Example of session variables:
                SET @@session.pseudo_thread_id=999999999/*!*/;
                SET @@session.foreign_key_checks=1
                SET @@session.sql_mode=1073741824/*!*/;
                SET @@session.auto_increment_increment=1,
                SET @@session.auto_increment_offset=1/*!*/;
                SET @@session.lc_time_names=0/*!*/;
                SET @@session.collation_database=DEFAULT/*!*/;
    3319fcb3
    Bug#18258933 CONCATENATION OF MYSQLBINLOG OUTPUT DOES NOT
    Venkatesh Duggirala authored
                   WORK WITH GTID
    
      Problem:  When mysqlbinlog processes and concatenates
      multiple binary log files into one output file, this
      output file is not in useful state. When it is later
      used for point-in-time recovery, it is producing
      ER_GTID_NEXT_TYPE_UNDEFINED_GROUP "ERROR 1837: When
      @@SESSION.GTID_NEXT is set to a GTID, you must explicitly
      set it to a different value after a COMMIT or ROLLBACK.
      Please check GTID_NEXT variable manual page for detailed
      explanation. Current @@SESSION.GTID_NEXT is 'xyz'".
    
      Analysis: When mysqlbinlog processes a GTID enabled
      binary log, it outputs SET GTID_NEXT statements. As
      GTID_NEXT is set to 'undefined' when a transaction
      commits (or DDL commits implicitly), this will leave
      GTID_NEXT as 'undefined' when the server has processed
      all output from mysqlbinlog. Also, there can be some
      anonymous statements/transactions at the beginning of
      binary log (i.e., without SET GTID_NEXT statements).   
      When mysqlbinlog processes and concatenates two (or more)
      binary log files into one output file, the combination
      (GTID_NEXT is undefined at the end of first binary log
      and second binary log can contain anonymous transactions)
      will not work and will result in ERROR 1837
      (ER_GTID_NEXT_TYPE_UNDEFINED_GROUP).
     
      Fix: When mysqlbinlog processes second binary log's
           start_log_event, it can output
           "SET GTID_NEXT=AUTOMATIC" if required i.e., if
           previous binary log is leaving the GTID_NEXT state
           in 'undefined' state. In the old code (before this
           patch), when mysqlbinlog processes multiple binary
           log files from multiple command line arguments, it
           used a new PRINT_EVENT_INFO object for each file,
           so the state at the end of the previous file is lost.
           To solve this problem, this patch changes the logic
           to use the same PRINT_EVENT_INFO structure for all
           command line arguments. And to keep track of
           gtid_next state, we are introducing
           is_gtid_next_valid and is_gtid_next_set variables.
           At the beginning of mysqlbinlog process
           is_gtid_next_set is set to false and
           is_gtid_next_valid is set to true.
           When mysqlbinlog encounters a
             i) GTID_LOG_EVENT: it sets is_gtid_next_set to
                                true and is_gtid_next_valid to
                                true.
            ii) XID_LOG_EVENT
                 /end_group() : it sets is_gtid_next_valid to
                                false if is_gtid_next_set to
                                true.
           And when mysqlbinlog is processing start_log_event,
           if it finds gtid_next is not valid (i.e., undefined)
           then it produces "SET GTID_NEXT=AUTOMATIC" in the
           output file to avoid mentioned
           ER_GTID_NEXT_TYPE_UNDEFINED_GROUP error and it resets
           is_gtid_next_set to false and is_gtid_next_valid to
           true (the initial values). mysqlbinlog also produces
           'SET GTID_NEXT=AUTOMATIC' in the output file at the
           end of processing all the command line arguments if
           it finds GTID_NEXT is in invalid state.
    
           After this patch, if you are merging two or more
           binary log files, you will see two differences in
           the output when compared to the before the patch
    
           a) When mysqlbinlog start processing the second
              binary log file, if required (i.e., if the 
              previous binary log left gtid_next in a invalid
              state), it generates SET GTID_NEXT=AUTOMATIC.
           b) Before the patch, mysqlbinlog outputs session
              variable information for every binary log.
              Now that we will have only one print_event_info
              per run, this session variables value will be
              printed only once *unless there is a change*.
              Example of session variables:
                SET @@session.pseudo_thread_id=999999999/*!*/;
                SET @@session.foreign_key_checks=1
                SET @@session.sql_mode=1073741824/*!*/;
                SET @@session.auto_increment_increment=1,
                SET @@session.auto_increment_offset=1/*!*/;
                SET @@session.lc_time_names=0/*!*/;
                SET @@session.collation_database=DEFAULT/*!*/;
Loading