When we need to upgrade from MySQL 5.7 to MySQL 8, we could choose to do an in-place upgrade or set up another MySQL 8 server(s) from the existing running MySQL 5.7 replica.
This article will explain how to set up MySQL 8 as a replica from an existing MySQL 5.7 server with Percona XtraBackup.
In my lab, we have two test nodes:
PRIMARY: mysql57 (192.168.56.111) with Percona Server 5.7 and xtrabackup 2.4 installed REPLICA: mysql8 (192.168.56.113) with Percona Server 8 and xtrabackup 8.0 installed
0. Pre-flight check with MySQL upgrade checker utility
Let’s run the MySQL upgrade checker utility to verify whether MySQL 5.7 server instances are ready for an upgrade.
MySQL localhost JS > util.checkForServerUpgrade('root@localhost:3306', {"password":"####", "targetVersion":"8.0.32", "configPath":"/etc/my.cnf"})
The MySQL server at localhost:3306, version 5.7.26-29-log - Percona Server
(GPL), Release 29, Revision 11ad961, will now be checked for compatibility
issues for upgrade to MySQL 8.0.32...
1) Usage of old temporal type
No issues found
.....
20) Tables recognized by InnoDB that belong to a different engine
No issues found
21) Issues reported by 'check table x for upgrade' command
No issues found
22) New default authentication plugin considerations
Warning: The new default authentication plugin 'caching_sha2_password' offers
more secure password hashing than previously used 'mysql_native_password'
....
23) Columns which cannot have default values
No issues found
....
25) Check for orphaned routines in 5.7
No issues found
Errors: 0
Warnings: 5
Notices: 0For details about MySQL Upgrade Checker, please refer to Upgrading to MySQL 8? Meet the MySQL Shell Upgrade Checker Utility or mysql-shell-utilities-upgrade for details.
First, let’s try streaming copy from mysql57 to mysql8 server as this does not need additional storage.
1. On the replica mysql8 server, run:
[root@mysql8 ~]# nc -l -p 2222 | unpigz -c | xbstream -x -C /var/lib/mysql
The above command would be there to receive the backup until the stream copy was done.
2. On primary server mysql5.7, stream copy to mysql8 server:
[root@mysql57 ~]# xtrabackup --stream=xbstream --parallel=4 /tmp | pigz -c --fast | nc -w 2 192.168.56.113 2222
xtrabackup: recognized server arguments: --datadir=/var/lib/mysql --server-id=1 --log_bin=mysql-bin --parallel=4
xtrabackup: recognized client arguments:
230228 22:58:57 innobackupex: Starting the backup operation
IMPORTANT: Please check that the backup run completes successfully.
At the end of a successful backup run innobackupex
prints "completed OK!".
230228 22:58:57 version_check Connecting to MySQL server with DSN 'dbi:mysql:;mysql_read_default_group=xtrabackup' as 'root' (using password: YES).
230228 22:58:58 version_check Connected to MySQL server
230228 22:58:58 version_check Executing a version check against the server...
230228 22:58:58 version_check Done.
230228 22:59:11 [00] ...done
230228 22:59:11 [00] Streaming <STDOUT>
230228 22:59:11 [00] ...done
xtrabackup: Transaction log of lsn (31871969) to (31872026) was copied.
Shutting down plugin 'keyring_file'
230228 22:59:12 completed OK!3. Prepare the backup on MySQL 8 with Percona XtraBackup 8.0, but you will get this error – [Xtrabackup] Unsupported redo log format 1] because of the version incompatible issue.
xtrabackup --prepare --use-memory=3G --apply-log-only --target-dir=/var/lib/mysql [root@mysql8 mysql]# xtrabackup --prepare --use-memory=3G --apply-log-only --target-dir=/var/lib/mysql xtrabackup: [Warning] option 'innodb_undo_tablespaces': unsigned value 0 adjusted to 2. 2023-02-28T23:05:23.993903-05:00 0 [Note] [MY-011825] [Xtrabackup] recognized server arguments: --innodb_checksum_algorithm=crc32 --innodb_data_file_path=ibdata1:12M:autoextend --innodb_log_files_in_group=2 --innodb_log_file_size=50331648 --innodb_page_size=16384 --innodb_log_block_size=512 --innodb_undo_directory=./ --innodb_undo_tablespaces=0 --server-id=1 2023-02-28T23:05:23.994329-05:00 0 [Note] [MY-011825] [Xtrabackup] recognized client arguments: --prepare=1 --use-memory=3G --apply-log-only=1 --target-dir=/var/lib/mysql xtrabackup version 8.0.32-25 based on MySQL server 8.0.32 Linux (x86_64) (revision id: 14f007fb) 2023-02-28T23:05:23.994404-05:00 0 [Note] [MY-011825] [Xtrabackup] cd to /var/lib/mysql/ 2023-02-28T23:05:24.001029-05:00 0 [Note] [MY-011825] [Xtrabackup] This target seems to be not prepared yet. 2023-02-28T23:05:24.003369-05:00 0 [ERROR] [MY-011825] [Xtrabackup] Unsupported redo log format 1 2023-02-28T23:05:24.003425-05:00 0 [ERROR] [MY-011825] [Xtrabackup] This version of Percona XtraBackup can only perform backups and restores against MySQL 8.0 and Percona Server 8.0, please use Percona Xtrabackup 2.4 for this database. [root@mysql8 mysql]#
4, You need to uninstall percona-xtrabackup-80 and install percona-xtrabackup-24 on the MySQL 8 server.
sudo yum remove percona-xtrabackup-80 sudo yum list | grep percona sudo yum install percona-xtrabackup-24 [root@mysql8 mysql]# xtrabackup --version xtrabackup: recognized server arguments: --datadir=/var/lib/mysql xtrabackup version 2.4.27 based on MySQL server 5.7.40 Linux (x86_64) (revision id: aae8e82) [root@mysql8 mysql]#
Then, prepare the backup on the mysql8 server,
[root@mysql8 mysql]# xtrabackup --prepare --use-memory=3G --apply-log-only --target-dir=/var/lib/mysql xtrabackup: recognized server arguments: --innodb_checksum_algorithm=crc32 --innodb_log_checksum_algorithm=strict_crc32 --innodb_data_file_path=ibdata1:12M:autoextend --innodb_log_files_in_group=2 --innodb_log_file_size=50331648 --innodb_fast_checksum=0 --innodb_page_size=16384 --innodb_log_block_size=512 --innodb_undo_directory=./ --innodb_undo_tablespaces=0 --server-id=1 --redo-log-version=1 xtrabackup: recognized client arguments: --prepare=1 --use-memory=3G --apply-log-only=1 --target-dir=/var/lib/mysql xtrabackup version 2.4.27 based on MySQL server 5.7.40 Linux (x86_64) (revision id: aae8e82) xtrabackup: cd to /var/lib/mysql/ xtrabackup: This target seems to be not prepared yet. InnoDB: Number of pools: 1 ……. InnoDB: xtrabackup: Last MySQL binlog file position 234, file name mysql-bin.000029 InnoDB: xtrabackup: Last MySQL binlog file position 234, file name mysql-bin.000029 xtrabackup: starting shutdown with innodb_fast_shutdown = 1 InnoDB: Starting shutdown... InnoDB: Shutdown completed; log sequence number 31872035 InnoDB: Number of pools: 1 Shutting down plugin 'keyring_file' 230228 23:27:52 completed OK! [root@mysql8 mysql]#
5. You could start the MySQL service on MySQL 8 server now, and MySQL 8 will upgrade for you.
You will get the server upgrade from ‘50700’ to ‘80031’ completed, as below:
[root@mysql8 lib]# chown -R mysql:mysql /var/lib/mysql [root@mysql8 lib]# systemctl start mysql 2023-03-01T02:38:23.887913Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.31-23) initializing of server in progress as process 6660 2023-03-01T02:38:23.939215Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started. 2023-03-01T04:28:38.397836Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.31-23) starting as process 8229 2023-03-01T04:28:38.530856Z 0 [Warning] [MY-010075] [Server] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 89c92c7e-b7e9-11ed-9369-080027f01654. 2023-03-01T04:28:38.749996Z 1 [System] [MY-011012] [Server] Starting upgrade of data directory. 2023-03-01T04:34:02.516924Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.31-23) starting as process 8388 2023-03-01T04:34:02.827172Z 1 [System] [MY-011012] [Server] Starting upgrade of data directory. 2023-03-01T04:34:02.827297Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started. 2023-03-01T04:34:47.547671Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended. 2023-03-01T04:35:10.890987Z 3 [System] [MY-011003] [Server] Finished populating Data Dictionary tables with data. 2023-03-01T04:35:17.864546Z 5 [System] [MY-013381] [Server] Server upgrade from '50700' to '80031' started. 2023-03-01T04:35:41.732506Z 5 [System] [MY-013381] [Server] Server upgrade from '50700' to '80031' completed. 2023-03-01T04:35:52.304549Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed. 2023-03-01T04:35:52.974222Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/lib/mysql/mysqlx.sock
6. You could set up replication with the steps below.
[root@mysql8 mysql]# cat /var/lib/mysql/xtrabackup_binlog_info
mysql-bin.000029 234 2fb728db-eb40-11eb-96c5-080027b6340a:1-78,
30b9be00-c21a-11ec-8eb2-0800275411eb:1
[root@mysql8 mysql]#
mysql8>STOP REPLICA;
mysql8>RESET MASTER;
mysql8>SET GLOBAL gtid_purged='2fb728db-eb40-11eb-96c5-080027b6340a:1-78,30b9be00-c21a-11ec-8eb2-0800275411eb:1';
mysql8>CHANGE REPLICATION SOURCE TO
SOURCE_HOST = '192.168.56.111',
SOURCE_PORT = 3306,
SOURCE_USER = 'repl',
SOURCE_PASSWORD = ########',
SOURCE_AUTO_POSITION = 1;
mysql8>START REPLICA;
mysql8> SHOW REPLICA STATUSG
*************************** 1. row ***************************
Slave_IO_State: Waiting for source to send event
Master_Host: 192.168.56.111
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000029
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
…
Retrieved_Gtid_Set: 2fb728db-eb40-11eb-96c5-080027b6340a:79
Executed_Gtid_Set: 2fb728db-eb40-11eb-96c5-080027b6340a:1-79,
30b9be00-c21a-11ec-8eb2-0800275411eb:1
Auto_Position: 1For now, we have set up the replication, and you could uninstall Percona XtraBackup 2.4 and reinstall Percona XtraBackup 8.0 for future backups on the MySQL 8 server.
If you feel that’s annoying, you could choose to prepare the backup first on the mysql57 server, but this might need additional storage. Please see below.
#1 Take the backup on mysql5.7
xtrabackup --parallel=4 --backup --socket=/var/lib/mysql/mysql.sock --target-dir=/opt/mysql57backup/ xtrabackup: recognized server arguments: --datadir=/var/lib/mysql --server-id=1 --log_bin=mysql-bin --parallel=4 xtrabackup: recognized client arguments: --user=root --password=* --backup=1 --socket=/var/lib/mysql/mysql.sock --target-dir=/opt/mysql57backup/ 230301 13:43:05 version_check Connecting to MySQL server with DSN 'dbi:mysql:;mysql_read_default_group=xtrabackup;mysql_socket=/var/lib/mysql/mysql.sock' as 'root' (using password: YES). 230301 13:43:05 version_check Connected to MySQL server 230301 13:43:05 version_check Executing a version check against the server... …. Using server version 5.7.26-29-log xtrabackup version 2.4.24 based on MySQL server 5.7.35 Linux (x86_64) (revision id: b4ee263) ........ 230301 13:43:25 Executing UNLOCK TABLES 230301 13:43:25 All tables unlocked 230301 13:43:25 [00] Copying ib_buffer_pool to /opt/mysql57backup/ib_buffer_pool 230301 13:43:25 [00] ...done 230301 13:43:25 Backup created in directory '/opt/mysql57backup/' MySQL binlog position: filename 'mysql-bin.000029', position '408', GTID of the last change '2fb728db-eb40-11eb-96c5-080027b6340a:1-79, 30b9be00-c21a-11ec-8eb2-0800275411eb:1' 230301 13:43:26 completed OK!
#2. Prepare the backup on mysql57, then transfer to the mysql8 server.
xtrabackup --prepare --socket=/var/lib/mysql/mysql.sock --target-dir=/opt/mysql57backup/ xtrabackup version 2.4.24 based on MySQL server 5.7.35 Linux (x86_64) (revision id: b4ee263) xtrabackup: cd to /opt/mysql57backup/ xtrabackup: This target seems to be not prepared yet. InnoDB: Number of pools: 1 xtrabackup: xtrabackup_logfile detected: size=8388608, start_lsn=(31872026) xtrabackup: using the following InnoDB configuration for recovery: xtrabackup: innodb_data_home_dir = . xtrabackup: innodb_data_file_path = ibdata1:12M:autoextend xtrabackup: innodb_log_group_hom .... InnoDB: 5.7.35 started; log sequence number 31872533 xtrabackup: starting shutdown with innodb_fast_shutdown = 1 InnoDB: page_cleaner: 1000ms intended loop took 12196ms. The settings might not be optimal. (flushed=0 and evicted=0, during the time.) InnoDB: FTS optimize thread exiting. InnoDB: Starting shutdown... InnoDB: Shutdown completed; log sequence number 31872552 Shutting down plugin 'keyring_file' 230301 13:47:16 completed OK!
Then transfer to mysql8:
[root@mysql8 lib]# chown -R percona:percona /var/lib/mysql/
[root@mysql57 ~]# rsync -avpP -e ssh /opt/mysql57backup/ /var/lib/mysql/
percona@192.168.56.113:/var/lib/mysql/
percona@192.168.56.113's password:
sending incremental file list
./
backup-my.cnf
515 100% 0.00kB/s 0:00:00 (xfr#1, to-chk=443/445)
ib_buffer_pool
686 100% 669.92kB/s 0:00:00 (xfr#2, to-chk=442/445)
ib_logfile0
50,331,648 100% 1.87MB/s 0:00:25 (xfr#3, to-chk=441/445)
ib_logfile1
50,331,648 100% 16.09MB/s 0:00:02 (xfr#4, to-chk=440/445)
......................#3: Start MySQL service, and it will upgrade for you.
[root@mysql8 lib]# chown -R mysql:mysql /var/lib/mysql [root@mysql8 lib]# systemctl start mysql …. 2023-03-01T19:12:38.752622Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.31-23) starting as process 21608 2023-03-01T19:12:40.212880Z 1 [System] [MY-011012] [Server] Starting upgrade of data directory. 2023-03-01T19:12:40.220504Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started. 2023-03-01T19:17:48.087404Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended. 2023-03-01T19:18:10.663895Z 3 [System] [MY-011003] [Server] Finished populating Data Dictionary tables with data. 2023-03-01T19:18:13.814450Z 5 [System] [MY-013381] [Server] Server upgrade from '50700' to '80031' started. 2023-03-01T19:18:57.570769Z 5 [System] [MY-013381] [Server] Server upgrade from '50700' to '80031' completed. ……… 2023-03-01T19:19:06.029463Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.31-23' socket: '/var/lib/mysql/mysql.sock' port: 3306 Percona Server (GPL), Release 23, Revision 71449379.
#4. Set up replication, similar to step 6 above:
[root@mysql57 ~]# cat /opt/mysql57backup/xtrabackup_binlog_info
mysql-bin.000029 408 2fb728db-eb40-11eb-96c5-080027b6340a:1-79,30b9be00-c21a-11ec-8eb2-0800275411eb:1
[root@mysql57 ~]#
mysql8>STOP REPLICA;
mysql8>RESET MASTER;
mysql8>SET GLOBAL gtid_purged='2fb728db-eb40-11eb-96c5-080027b6340a:1-79,30b9be00-c21a-11ec-8eb2-0800275411eb:1';
mysql8> CHANGE REPLICATION SOURCE TO
SOURCE_HOST = '192.168.56.111',
SOURCE_PORT = 3306,
SOURCE_USER = 'repl',
SOURCE_PASSWORD = ########',
SOURCE_AUTO_POSITION = 1;
mysql8>START REPLICA;
mysql8> SHOW REPLICA STATUSG
*************************** 1. row ***************************
Slave_IO_State: Waiting for source to send event
Master_Host: 192.168.56.111
Master_User: repl
Master_Port: 3306
……….
Relay_Master_Log_File: mysql-bin.000029
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Conclusion
You could either use Percona XtraBackup 2.4 to prepare the MySQL 5.7 backup on MySQL 8 servers after the stream copied, or use Percona XtraBackup 2.4 and prepare the backup on MySQL 5.7 first and then transfer to MySQL 8. Finally, let MySQL 8 binary upgrade it automatically for you.
I hope this is helpful for you to upgrade from MySQL 5.7 to MySQL 8 with Percona XtraBackup!
Percona XtraBackup is a free, open source, complete online backup solution for all versions of Percona Server for MySQL and MySQL. It performs online non-blocking, tightly compressed, highly secure backups on transactional systems so that applications remain fully available during planned maintenance windows.