Skip to content
  • Joao Gramacho's avatar
    66d0aa22
    WL#10957: Binary log encryption at rest (Step 1) · 66d0aa22
    Joao Gramacho authored
    This patch introduces the infrastructure to allow a server to
    read content from encrypted binary/relay log files and to write
    into an existing encrypted binary log file.
    
    Reading encrypted binary/relay log files requires:
    - Deserializing the file encryption header;
    - Retrieving the replication encryption key from keyring;
    - Decrypting the file password;
    - Reading the encrypted content as a stream.
    
    Writing to existing encrypted binary log files requires:
    - Deserializing the file encryption header;
    - Retrieving the replication encryption key from keyring;
    - Decrypting the file password;
    - Encrypting the new content to be written;
    - Write the encrypted content into the lower stream.
    
    @ client/CMakeLists.txt
    
      As Mysqlbinlog_ifile inherits from Basic_binlog_ifile that depends on
      new sql/rpl_log_encryption.cc, an entry for the latter was added to
      mysqlbinlog dependencies.
    
    @ client/mysqlbinlog.cc
    
      A minor refactoring was needed because of changes in the open_file
      function that is overridden by Mysqlbinlog_ifile.
    
    @ share/errmsg-utf8.txt
    
      Introduced the following errors:
      - ER[_SERVER]_RPL_ENCRYPTION_FAILED_TO_FETCH_KEY;
      - ER[_SERVER]_RPL_ENCRYPTION_KEY_NOT_FOUND;
      - ER[_SERVER]_RPL_ENCRYPTION_INVALID_KEY;
      - ER[_SERVER]_RPL_ENCRYPTION_HEADER_ERROR;
    
    @ sql/CMakeLists.txt
    
      Added an entry for sql/rpl_log_encryption.cc to the BINLOG_SOURCE set
      of dependencies.
    
    @ sql/basic_ostream.h
    
      Added flush() and sync() virtual function to Truncatable_ostream as
      part of Binlog_ofile refactoring.
    
    @ sql/binlog.h
    
      Added a encrypted_header_size field to LOG_INFO. This is needed to
      display the correct file size in "SHOW BINARY LOGS" for the last
      active binary log file.
    
    @ sql/binlog.cc
    
      Did some refactoring in Binlog_ofile and added a static function
      open_existing() to allow to open an existing encrypted file to change
      its content or truncate it.
    
      Changed any function relying on file size to state the binary log file
      size to consult the size of the "binary log data stream". It is
      actually the file size for plain binary/relay log files and
      (file size - encrypted header size) for encrypted binary/relay log
      files.
    
      Changed binary log file truncation to use Binlog_ofile (that supports
      encryption) when truncating and clearing LOG_EVENT_BINLOG_IN_USE_F
      flag.
    
    @ sql/binlog_istream.{h|cc}
    
      Added new entries to Binlog_read_error:
      - INVALID_ENCRYPTION_HEADER;
      - CANNOT_GET_FILE_PASSWORD;
    
      Introduced the Binlog_encryption_istream class (a Seekable_istream
      with decryption feature).
    
      Did some refactoring in open_file function.
    
    @ sql/binlog_ostream.{h|cc}
    
      Introduced the Binlog_encryption_ostream class (a Truncatable_ostream
      with encryption feature).
    
    @ sql/rpl_master.cc
    
      Made "SHOW BINARY LOGS" to display proper file size for a last
      encrypted binary log file and also added the "Encrypted" column to
      the "SHOW BINARY LOGS" output.
    
    @ sql/rpl_log_encryption.{h|cc}
    
      This is a new pair of files.
    
      They introduce both encryption and decryption logics using
      Aes_ctr_cipher, the Rpl_encryption_header class and general purpose
      Rpl_encryption class.
    
      The Rpl_encryption_header classes shall handle serialization and
      deserialization of the replication log files encryption header.
    
      The Rpl_encryption class shall provide general functionality related
      to encryption for the MySQL server. In this patch, it is providing
      a function to retrieve encryption keys from the keyring.
    
    Test case
    =========
    
    @ rpl_nogtid.rpl_nogtid_encryption_read
    
      This script aims at testing read and truncate operations on encrypted
      binary log files, as well as related error conditions.
    66d0aa22
    WL#10957: Binary log encryption at rest (Step 1)
    Joao Gramacho authored
    This patch introduces the infrastructure to allow a server to
    read content from encrypted binary/relay log files and to write
    into an existing encrypted binary log file.
    
    Reading encrypted binary/relay log files requires:
    - Deserializing the file encryption header;
    - Retrieving the replication encryption key from keyring;
    - Decrypting the file password;
    - Reading the encrypted content as a stream.
    
    Writing to existing encrypted binary log files requires:
    - Deserializing the file encryption header;
    - Retrieving the replication encryption key from keyring;
    - Decrypting the file password;
    - Encrypting the new content to be written;
    - Write the encrypted content into the lower stream.
    
    @ client/CMakeLists.txt
    
      As Mysqlbinlog_ifile inherits from Basic_binlog_ifile that depends on
      new sql/rpl_log_encryption.cc, an entry for the latter was added to
      mysqlbinlog dependencies.
    
    @ client/mysqlbinlog.cc
    
      A minor refactoring was needed because of changes in the open_file
      function that is overridden by Mysqlbinlog_ifile.
    
    @ share/errmsg-utf8.txt
    
      Introduced the following errors:
      - ER[_SERVER]_RPL_ENCRYPTION_FAILED_TO_FETCH_KEY;
      - ER[_SERVER]_RPL_ENCRYPTION_KEY_NOT_FOUND;
      - ER[_SERVER]_RPL_ENCRYPTION_INVALID_KEY;
      - ER[_SERVER]_RPL_ENCRYPTION_HEADER_ERROR;
    
    @ sql/CMakeLists.txt
    
      Added an entry for sql/rpl_log_encryption.cc to the BINLOG_SOURCE set
      of dependencies.
    
    @ sql/basic_ostream.h
    
      Added flush() and sync() virtual function to Truncatable_ostream as
      part of Binlog_ofile refactoring.
    
    @ sql/binlog.h
    
      Added a encrypted_header_size field to LOG_INFO. This is needed to
      display the correct file size in "SHOW BINARY LOGS" for the last
      active binary log file.
    
    @ sql/binlog.cc
    
      Did some refactoring in Binlog_ofile and added a static function
      open_existing() to allow to open an existing encrypted file to change
      its content or truncate it.
    
      Changed any function relying on file size to state the binary log file
      size to consult the size of the "binary log data stream". It is
      actually the file size for plain binary/relay log files and
      (file size - encrypted header size) for encrypted binary/relay log
      files.
    
      Changed binary log file truncation to use Binlog_ofile (that supports
      encryption) when truncating and clearing LOG_EVENT_BINLOG_IN_USE_F
      flag.
    
    @ sql/binlog_istream.{h|cc}
    
      Added new entries to Binlog_read_error:
      - INVALID_ENCRYPTION_HEADER;
      - CANNOT_GET_FILE_PASSWORD;
    
      Introduced the Binlog_encryption_istream class (a Seekable_istream
      with decryption feature).
    
      Did some refactoring in open_file function.
    
    @ sql/binlog_ostream.{h|cc}
    
      Introduced the Binlog_encryption_ostream class (a Truncatable_ostream
      with encryption feature).
    
    @ sql/rpl_master.cc
    
      Made "SHOW BINARY LOGS" to display proper file size for a last
      encrypted binary log file and also added the "Encrypted" column to
      the "SHOW BINARY LOGS" output.
    
    @ sql/rpl_log_encryption.{h|cc}
    
      This is a new pair of files.
    
      They introduce both encryption and decryption logics using
      Aes_ctr_cipher, the Rpl_encryption_header class and general purpose
      Rpl_encryption class.
    
      The Rpl_encryption_header classes shall handle serialization and
      deserialization of the replication log files encryption header.
    
      The Rpl_encryption class shall provide general functionality related
      to encryption for the MySQL server. In this patch, it is providing
      a function to retrieve encryption keys from the keyring.
    
    Test case
    =========
    
    @ rpl_nogtid.rpl_nogtid_encryption_read
    
      This script aims at testing read and truncate operations on encrypted
      binary log files, as well as related error conditions.
Loading