At Percona Managed Services, we manage Percona Server for MySQL, Community MySQL, and MariaDB. There are slight differences when configuring and managing MariaDB GTID replication. In this blog, we’ll show you how to convert MariaDB binary log file and position-based replication to GTID replication.
In my lab, we have two test nodes; both servers will have Debian 11 and MariaDB10.5.19 with mariabackup installed.
The PRIMARY server is deb11m8m (IP: 192.168.56.190), and the REPLICA server name is deb11m8s (IP: 192.168.56.191).
1. Install MariaDB on Debian 11 on PRIMARY and REPLICA
wget https://downloads.mariadb.com/MariaDB/mariadb_repo_setup chmod +x ./mariadb_repo_setup ./mariadb_repo_setup --mariadb-server-version="mariadb-10.5.19" apt-get install mariadb-server mariadb-backup systemctl daemon-reload root@deb11m8:~#wget https://downloads.mariadb.com/MariaDB/mariadb_repo_setup https://downloads.mariadb.com/MariaDB/mariadb_repo_setup downloads.mariadb.com (downloads.mariadb.com)|104.17.191.14|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 36188 (35K) [application/octet-stream] Saving to: ‘mariadb_repo_setup’ 2023-08-30 14:42:51 (26.0 MB/s) - ‘mariadb_repo_setup’ saved [36188/36188] root@deb11m8m:~# chmod +x ./mariadb_repo_setup root@deb11m8m:~# ./mariadb_repo_setup --mariadb-server-version="mariadb-10.5.19" # [info] Checking for script prerequisites. # [warning] Found existing file at /etc/apt/sources.list.d/mariadb.list. Moving to /etc/apt/sources.list.d/mariadb.list.old_1 # [info] MariaDB Server version 10.5.19 is valid # [info] Repository file successfully written to /etc/apt/sources.list.d/mariadb.list # [info] Adding trusted package signing keys... # [info] Running apt-get update… # [info] Done adding trusted package signing keys root@deb11m8m:~# root@deb11m8m:~# apt-get install mariadb-server mariadb-backup Reading package lists... Done Building dependency tree... Done Reading state information... Done The following packages were automatically installed and are no longer required: libconfig-inifiles-perl libmecab2 libopengl0 Use 'apt autoremove' to remove them. The following additional packages will be installed: libdbd-mariadb-perl libmariadb3 mariadb-client-10.5 mariadb-client-core-10.5 mariadb-common mariadb-server-10.5 mariadb-server-core-10.5 Suggested packages: mailx mariadb-test netcat-openbsd TSelecting previously unselected package mariadb-client-core-10.5. Preparing to unpack .../mariadb-client-core-10.5_1%3a10.5.19+maria~deb11_amd64.deb ... Unpacking mariadb-client-core-10.5 (1:10.5.19+maria~deb11) ... …… Setting up mariadb-server (1:10.5.19+maria~deb11) ... Processing triggers for man-db (2.9.4-2) ... Processing triggers for libc-bin (2.31-13+deb11u6) ... root@deb11m8m:~#
1.1 Reset root user password and create a replication user
MariaDB [(none)]>Alter user 'root'@'localhost' identified by ‘#######’; MariaDB [(none)]>GRANT PROXY ON ''@'%' TO 'root'@'localhost' WITH GRANT OPTION ; MariaDB [(none)]flush privileges; MariaDB [(none)]CREATE USER 'repl'@'%' IDENTIFIED BY ‘#######’; MariaDB [(none)]GRANT RELOAD, SUPER, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'repl'@'%'; MariaDB [(none)]flush privileges;
1.2 Configure my.cnf on PRIMARY and REPLICA
PRIMARY
[mysqld] pid-file = /run/mysqld/mysqld.pid basedir = /usr server_id=1 bind-address=192.168.56.190 log-bin = /var/lib/mysql/bin_log expire_logs_days = 10 character-set-server = utf8mb4 collation-server = utf8mb4_general_ci
REPLICA [mysqld] pid-file = /run/mysqld/mysqld.pid basedir = /usr <b>server_id=2</b> <b>bind-address=192.168.56.191</b> log-bin = /var/lib/mysql/bin_log expire_logs_days = 10 character-set-server = utf8mb4 collation-server = utf8mb4_general_ci
2. Let’s set up binary log and position-based replication between PRIMARY and REPLICA with the stream copy method.
2.1 on REPLICA
root@deb11m8s: nc -l -p 2222 | mbstream -x -C /var/lib/mysql
2.2 on PRIMARY
root@deb11m8m:/var/lib/mysql# mariabackup --stream=mbstream --parallel=4 --backup | nc -w 2 192.168.56.191 2222 [00] 2023-08-30 18:02:08 Connecting to server host: localhost, user: root, password: set, port: not set, socket: /run/mysqld/mysqld.sock [00] 2023-08-30 18:02:08 Using server version 10.5.19-MariaDB-1:10.5.19+maria~deb11-log mariabackup based on MariaDB server 10.5.19-MariaDB debian-linux-gnu (x86_64) [00] 2023-08-30 18:02:08 uses posix_fadvise(). [00] 2023-08-30 18:02:08 cd to /var/lib/mysql/ [00] 2023-08-30 18:02:08 open files limit requested 0, set to 1024 [00] 2023-08-30 18:02:08 mariabackup: using the following InnoDB configuration: [00] 2023-08-30 18:02:08 innodb_data_home_dir = [00] 2023-08-30 18:02:08 innodb_data_file_path = ibdata1:12M:autoextend [00] 2023-08-30 18:02:08 innodb_log_group_home_dir = ./ [00] 2023-08-30 18:02:08 InnoDB: Using Linux native AIO 2023-08-30 18:02:08 0 [Note] InnoDB: Number of pools: 1 [00] 2023-08-30 18:02:08 mariabackup: Generating a list of tablespaces [00] 2023-08-30 18:02:08 >> log scanned up to (10912597) [00] 2023-08-30 18:02:08 mariabackup: Starting 4 threads for parallel data files transfer [03] 2023-08-30 18:02:08 Streaming ./mysql/innodb_index_stats.ibd [04] 2023-08-30 18:02:08 Streaming ibdata1 …….. '/var/lib/mysql/xtrabackup_backupfiles/' [00] 2023-08-30 18:02:10 MySQL binlog position: filename 'bin_log.000002', position '326', GTID of the last change '' [00] 2023-08-30 18:02:10 Streaming backup-my.cnf [00] 2023-08-30 18:02:10 Streaming xtrabackup_info [00] 2023-08-30 18:02:10 Redo log (from LSN 10912585 to 10912597) was copied. [00] 2023-08-30 18:02:10 completed OK!
2.3 On REPLICA, get binlog information
<root@deb11m8s:/var/lib/mysql# cat xtrabackup_binlog_info bin_log.000002 326
2.4 On REPLICA, prepare the backup
root@deb11m8s:/var/lib/mysql# mariabackup --prepare --use-memory=1G --target-dir=/var/lib/mysql mariabackup based on MariaDB server 10.5.19-MariaDB debian-linux-gnu (x86_64) [00] 2023-08-30 18:08:32 cd to /var/lib/mysql/ [00] 2023-08-30 18:08:32 open files limit requested 0, set to 1024 . 2023-08-30 18:08:32 0 [Note] InnoDB: Initializing buffer pool, total size = 1073741824, chunk size = 1073741824 2023-08-30 18:08:32 0 [Note] InnoDB: Completed initialization of buffer pool 2023-08-30 18:08:32 0 [Note] InnoDB: Starting crash recovery from checkpoint LSN=10912585,10912585 [00] 2023-08-30 18:08:32 Last binlog file , position 0 [00] 2023-08-30 18:08:32 completed OK! root@deb11m8s:/var/lib/mysql#
2.5 Set up the replication
root@deb11m8s:/var/lib/mysql# chown -R mysql:mysql /var/lib/mysql root@deb11m8s:/var/lib/mysql# systemctl start mysql root@deb11m8s:/var/lib/mysql# mysql Welcome to the MariaDB monitor. Commands end with ; or g. Your MariaDB connection id is 6 Server version: 10.5.19-MariaDB-0+deb11u2-log Debian 11 MariaDB [(none)]> stop slave; reset slave; reset slave all; reset master; Query OK, 0 rows affected, 1 warning (0.000 sec) Query OK, 0 rows affected (0.001 sec) Query OK, 0 rows affected (0.000 sec) Query OK, 0 rows affected (0.009 sec) ariaDB [(none)]> CHANGE MASTER TO MASTER_HOST='192.168.56.190', MASTER_USER='repl', MASTER_PASSWORD='#######', MASTER_PORT=3306, MASTER_LOG_FILE='bin_log.000002', MASTER_LOG_POS=326, MASTER_CONNECT_RETRY=10; Query OK, 0 rows affected (0.022 sec) MariaDB [(none)]> start slave; Query OK, 0 rows affected (0.003 sec) MariaDB [(none)]> show slave statusG *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 192.168.56.190 Master_User: repl Master_Port: 3306 Connect_Retry: 10 Master_Log_File: bin_log.000002 Read_Master_Log_Pos: 326 Relay_Log_File: mysqld-relay-bin.000002 Relay_Log_Pos: 553 Relay_Master_Log_File: bin_log.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes … Exec_Master_Log_Pos: 326 Relay_Log_Space: 863 …. Master_SSL_Crlpath: Using_Gtid: No Gtid_IO_Pos: Replicate_Do_Domain_Ids: Replicate_Ignore_Domain_Ids: Parallel_Mode: optimistic SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Slave_DDL_Groups: 0 Slave_Non_Transactional_Groups: 0 Slave_Transactional_Groups: 0 1 row in set (0.000 sec)
We have now set up binary log and position-based replication between the PRIMARY and REPLICA servers. Our next step is to convert this replication to GTID-based replication.
MariaDB and MySQL have different GTID implementations.
MariaDB GTID consists of three numbers separated with dashes ‘-‘. For example:
0-1-10: The first number, 0, is the domain ID, the second number is the server ID, and the third number is the sequence number.
MySQL GTID has two parts, source_id and transaction_id, separated by a colon character (:) .
e.g., 3E11FA47-71CA-11E1-9E33-C80AA9429562:23
The source_id is the source server’s server_uuid, and the transaction_id is the transaction sequence number committed on the source.
Let’s continue.
On PRIMARY, we enable GTID by running the below command.
MariaDB [(none)] SET GLOBAL gtid_domain_id = 1; MariaDB [(none)] set global gtid_strict_mode=1 ; And persist in my.cnf Verify GTID is enabled with below query MariaDB [(none)]> select @@server_id,@@bind_address,@@log_bin,@@gtid_strict_mode,@@gtid_domain_id; +-------------+----------------+-----------+--------------------+------------------+ | @@server_id | @@bind_address | @@log_bin | @@gtid_strict_mode | @@gtid_domain_id | +-------------+----------------+-----------+--------------------+------------------+ | 1 | 192.168.56.190 | 1 | <b>1 | 1 | +-------------+----------------+-----------+--------------------+------------------+ 1 row in set (0.000 sec)</b>
ON REPLICA
MariaDB [(none)]>SET GLOBAL gtid_domain_id = 2; MariaDB [(none)]set global gtid_strict_mode=1 ; And persist in my.cnf Verify GTID is enabled with below query MariaDB [(none)]select @@server_id,@@bind_address,@@log_bin,@@gtid_strict_mode,@@gtid_domain_id; +-------------+----------------+-----------+--------------------+------------------+ | @@server_id | @@bind_address | @@log_bin | @@gtid_strict_mode | @@gtid_domain_id | +-------------+----------------+-----------+--------------------+------------------+ | 2 | 192.168.56.191 | 1 | 1 |2 |; +-------------+----------------+-----------+--------------------+------------------+ 1 row in set (0.000 sec)
On REPLICA, we need to stop replication and extract the values of Relay_Master_Log_File and Exec_Master_Log_Pos from the output.
MariaDB [test_db]STOP SLAVE; Query OK, 0 rows affected (0.007 sec) MariaDB [test_db] show slave statusG *************************** 1. row *************************** Slave_IO_State: Master_Host: 192.168.56.190 Master_User: repl Master_Port: 3306 Connect_Retry: 10 Master_Log_File: bin_log.000002 Read_Master_Log_Pos: 1206 Relay_Log_File: mysqld-relay-bin.000007 Relay_Log_Pos: 553 Relay_Master_Log_File: bin_log.000002 Slave_IO_Running: No Slave_SQL_Running: No Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 1206 Relay_Log_Space: 863 …….. Slave_Non_Transactional_Groups: 0 Slave_Transactional_Groups: 0 1 row in set (0.000 sec)
In our case, the Relay_Master_Log_File value is bin_log.000002, and the Exec_Master_Log_Pos is 1206.
On PRIMARY, we could use the below query to get the GTID position that corresponds to these binary log coordinates on PRIMARY.
MariaDB[test_db] SELECT BINLOG_GTID_POS('bin_log.000002', 1206),@@hostname;
+-----------------------------------------+------------+
| BINLOG_GTID_POS('bin_log.000002', 1206) | @@hostname |
+-----------------------------------------+------------+
| 1-1-1,0-1-4 |
+-----------------------------------------+------------+
1 row in set (0.000 sec)We can now modify the replica configuration on the REPLICA server deb11m8s by executing the following statements:
MariaDB [test_db]SET GLOBAL gtid_slave_pos = '1-1-1,0-1-4'; MariaDB [test_db] CHANGE MASTER TO master_use_gtid=slave_pos; MariaDB [test_db]START SLAVE; MariaDB [test_db] show slave statusG *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 192.168.56.190 Master_User: repl Master_Port: 3306 Connect_Retry: 10 Master_Log_File: bin_log.000002 Read_Master_Log_Pos: 1391 Relay_Log_File: mysqld-relay-bin.000002 Relay_Log_Pos: 910 Relay_Master_Log_File: bin_log.000002 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 1391 Relay_Log_Space: 1220 ……. Master_SSL_Key: Seconds_Behind_Master: 0 Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 1 Master_SSL_Crl: Master_SSL_Crlpath: Using_Gtid: Slave_Pos Gtid_IO_Pos: 1-1-2,0-1-4 Replicate_Do_Domain_Ids: Replicate_Ignore_Domain_Ids: Parallel_Mode: optimistic SQL_Delay: 0 SQL_Remaining_Delay: NULL Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates Slave_DDL_Groups: 0 Slave_Non_Transactional_Groups: 0 Slave_Transactional_Groups: 1 1 row in set (0.000 sec)
The SHOW SLAVE STATUS output Using_Gtid: Slave_Pos and Gtid_IO_Pos: 1-1-2,0-1-4 indicates that the slave uses GTIDs to track replication. We could use the above command to set up GTID replication on other replicas as well.
When running CHANGE MASTER TO master_use_gtid=slave_pos; and setting the MASTER_USE_GTID replication parameter, you have the option of enabling Global Transaction IDs to use either the current_pos or slave_pos values.
Using the value current_pos causes the replica to set its position based on the gtid_current_pos system variable, which is a union of gtid_binlog_pos and gtid_slave_pos. Using the value slave_pos causes the replica to instead set its position based on the gtid_slave_pos system variable. You may run into issues when you use the value current_pos if you write any local transactions on the replica. For details, please refer to https://mariadb.com/kb/en/gtid/.
On Community MySQL or Percona Server for MySQL, the parameter to enable GTID replication is
gtid-mode=ON enforce-gtid-consistency.
The command is:
change master to master_host = '192.168.1.120', master_port=3306, master_user = 'repl', master_password = 'password', master_auto_position=1;
Conclusion
We can see that the procedure to enable GTIDs on MariaDB replication is simple, but the commands and parameters are a bit different.
As mentioned, Percona offers Support and Managed Services for Community versions of MariaDB. At the same time, we encourage MariaDB users to explore Percona Server for MySQL as an alternative. Should you make the decision to migrate to Percona Software for MySQL, our experts will assist throughout the migration process and support you after the migration is complete.
Contact us to discuss migration options
Hope this is helpful for your daily MySQL and MariaDB management.