Skip to content
  • Srikanth B R's avatar
    0fe8569c
    Bug#25984429 MTR: BACKPORT OPTION --report-unstable-tests TO 5.7 · 0fe8569c
    Srikanth B R authored
    
    
    Problem:
    --------
    Various Linux distributions are running the MySQL Test Suite during
    build. As known, MySQL test suite has some flaky tests. If a build
    is unlucky and gets a sporadic test failure, it is cancelled and
    in the worst case, the distribution is blocked from upgrading MySQL
    to a new one. Even if MTR is run with --retry > 1 and all failed
    tests pass on retry, exit code from MTR is non zero and build stops.
    
    Solution:
    ---------
    An MTR option named --report-unstable-tests is added which does the
    following:
    - Reports any test which has passed on at least one retry attempt
      in a separate category called "Unstable tests" in the summary.
      They will be in the format :
      <test_name>(<number_of_failures>/<number_of_attempts>)
    - In case all failures are due to unstable tests, it causes MTR
      to print a warning and terminate with a zero exit code.
    
    Note:
    This is a backport of the patch for Bug#24473420 from mysql-8.0
    
    Reviewed-by: default avatarPavan Naik <pavan.naik@oracle.com>
    0fe8569c
    Bug#25984429 MTR: BACKPORT OPTION --report-unstable-tests TO 5.7
    Srikanth B R authored
    
    
    Problem:
    --------
    Various Linux distributions are running the MySQL Test Suite during
    build. As known, MySQL test suite has some flaky tests. If a build
    is unlucky and gets a sporadic test failure, it is cancelled and
    in the worst case, the distribution is blocked from upgrading MySQL
    to a new one. Even if MTR is run with --retry > 1 and all failed
    tests pass on retry, exit code from MTR is non zero and build stops.
    
    Solution:
    ---------
    An MTR option named --report-unstable-tests is added which does the
    following:
    - Reports any test which has passed on at least one retry attempt
      in a separate category called "Unstable tests" in the summary.
      They will be in the format :
      <test_name>(<number_of_failures>/<number_of_attempts>)
    - In case all failures are due to unstable tests, it causes MTR
      to print a warning and terminate with a zero exit code.
    
    Note:
    This is a backport of the patch for Bug#24473420 from mysql-8.0
    
    Reviewed-by: default avatarPavan Naik <pavan.naik@oracle.com>
Loading