Percona Toolkit 3.5.6 was released on December 21, 2023. This blog post covers the main changes in this release.
New tool: pt-galera-log-explainer
We continue adding more instruments for the Support teams. pt-galera-log-explainer was written by my Percona Support colleague Yoann La Cancellera. pt-galera-log-explainer filters, aggregates, and summarizes multiple Galera library logs together. The purpose of this tool is to help find useful information in Percona XtraDB Cluster/Galera library logs.
pt-galera-log-explainer takes raw log files, usually very verbose as the one available in the regression test suite, and makes short summary out of them:
$ ./bin/pt-galera-log-explainer list --all src/go/pt-galera-log-explainer/tests/logs/merge_rotated_daily/node1.20230315.log identifier node1 current path .../tests/logs/merge_rotated_daily/node1.20230315.log last known ip last known name node1 mysql version 2023-03-15T20:10:57.784904+02:00 node2 joined 2023-03-15T20:10:57.785568+02:00 node3 left 2023-03-15T20:10:57.791959+02:00 node3 left 2023-03-15T20:10:57.797221+02:00 PRIMARY(n=2) 2023-03-15T20:20:12.714291+02:00 node2 joined 2023-03-15T20:20:12.714331+02:00 node3 joined 2023-03-15T20:20:13.776977+02:00 PRIMARY(n=3) 2023-03-15T20:20:14.839684+02:00 local node will resync node3 2023-03-15T20:20:14.839723+02:00 SYNCED -> DONOR 2023-03-15T20:20:15.799020+02:00 IST will be used 2023-03-15T20:20:16.850525+02:00 finished sending IST to node3 2023-03-15T20:20:16.850549+02:00 DESYNCED -> JOINED 2023-03-15T20:20:16.865312+02:00 JOINED -> SYNCED
The tool can process logs from multiple nodes and draw a timeline. You can find a usage example with sample output in the user reference manual. You can also filter events to have a more compact output.
pt-galera-log-explainer is an excellent tool for analyzing large log files that produce the Galera library.
Better macOS and ARM support
This release contains a few improvements for platforms and operating systems that Percona does not officially support.
PR-516, contributed by Ivan Kruglov, makes pt-online-schema-changeand other tools respect case-insensitive lookup on Windows and macOS: the default option for these operating systems.
PR-720 simplifies the build process for the Percona Toolkit. Historically, Percona Toolkit was written in Perl and Shell programming languages. Build instructions for all code were simple:
perl Makefile.PL make (optionally) make test make install
Since the first tool, written in the Go programming language, was introduced, package maintainers had to perform one extra step:
cd src/go make <PLATFORM such as linux-amd64>
Now, this extra step is optional because the top-level Makefilehas instructions for building Go tools on the current platform.
PR-712 adds the darwin-arm64 platform to the list of platforms for which Go binaries could be built. While we do not officially support ARM and macOS, you can build the tools yourself without extra effort.
To create macOS ARM binaries on the same platform, simply run the following from the top-level directory.
perl Makefile.PL make
If you want to create binaries on another platform, change the directory to src/go, then run
make darwin-arm64
We also have unofficial ARM packages for Percona Toolkit and PMM Dump in our labs at https://github.com/Percona-Lab/percona-on-arm.
Tools improvements
Explain output for slow query in JSON report for pt-query-digest
Earlier, pt-query-digest did not print EXPLAIN output with option --output=json . PR-471, sent by Ayush Goyal, adds this possibility.
Hook before_die for pt-online-schema-change
PR-509, sent by Ilaria Migliozzi, introduces a new hook for pt-online-schema-change . If pt-online-schema-change stops ungracefully, this hook lets you print diagnostic information about failing operations. You can find an example plugin on GitHub.
Option –unstop for pt-archiver
pt-archiver supports the option –stop that terminates running instances by creating a sentinel file. However, there was no option to remove this file and restart pt-archiver . PR-429 by fraff resolves this gap by adding the option --unstop . If pt-archiver is called with this option, it removes the sentinel file and restarts the operation.
Quality improvements
In my last Percona Toolkit release blog, What’s New in Percona Toolkit 3.5.5, I wrote about how we plan to make regression tests part of the release process. This partially happened, and we are testing Percona Toolkit with Percona Server for MySQL 5.7 and 8.0 on all supported platforms now. There are still a few issues with tests, as reported at PT-2295. We are planning to fix them and then add support for other products.
Viktor Szépe continued to contribute to the quality of the Percona Toolkit source code. He introduced EditorConfig for our GitHub repository, fixed typos, and made style improvements for Go code.
In addition to Kushal Haldar’s reports about vulnerabilities in Go, we enabled better automation for our GitHub repository. This release was built with the latest version of Go and fixes all known module vulnerabilities.
Percona Toolkit Docker images
We released an official Docker image for the Percona Toolkit at https://hub.docker.com/r/percona/percona-toolkit. It can be installed using the following command.
docker pull percona/percona-toolkit
To call any tool using docker, run the following.
docker run <TOOL NAME> <OPTIONS>
For example:
$ docker run --network="host" percona/percona-toolkit pt-online-schema-change > h=127.0.0.1,P=12345,u=msandbox,p=msandbox,D=test,t=t1 --alter='ADD COLUMN f2 INT' > --execute Found 2 slaves: s76 -> 127.0.0.1:12346 s76 -> 127.0.0.1:12347 Will check slave lag on: s76 -> 127.0.0.1:12346 s76 -> 127.0.0.1:12347 Operation, tries, wait: analyze_table, 10, 1 copy_rows, 10, 0.25 create_triggers, 10, 1 drop_triggers, 10, 1 swap_tables, 10, 1 update_foreign_keys, 10, 1 Altering `test`.`t1`... Creating new table... Created new table test._t1_new OK. Altering new table... Altered `test`.`_t1_new` OK. 2023-12-21T15:29:22 Creating triggers... 2023-12-21T15:29:22 Created triggers OK. 2023-12-21T15:29:22 Copying approximately 1 rows... 2023-12-21T15:29:22 Copied rows OK. 2023-12-21T15:29:22 Analyzing new table... 2023-12-21T15:29:22 Swapping tables... 2023-12-21T15:29:22 Swapped original and new tables OK. 2023-12-21T15:29:22 Dropping old table... 2023-12-21T15:29:22 Dropped old table `test`.`_t1_old` OK. 2023-12-21T15:29:22 Dropping triggers... 2023-12-21T15:29:22 Dropped triggers OK. Successfully altered `test`.`t1`
Docker images are not only needed for those who prefer to install software via docker but will allow us to easily ship the Percona Toolkit together with other software, such as Percona Monitoring and Management or Percona Kubernetes Operators.
Community contributions
This release includes many contributions from community members. We want to thank:
- Ivan Kruglov for fixing case-sensitivity issues in pt-online-schema-change and other tools
- Chrys Swingler for fixing a bug with the option --skip-check-slave-lag in pt-online-schema-change and pt-table-checksum
- Ayush Goyal for adding EXPLAIN output for slow query in JSON report for pt-query-digest
- Ilaria Migliozzi for new hook after_die for pt-online-schema-change
- Jakob for fixing pt-archiver bug PT-2064.
- Viktor Szépe for introducing EditorConfig, fixing typos and issues with the Go code
- fraff for the option --unstop for pt-archiver
- Zongzhi Chen for fixing pt-pmp bug PT-2211
- Jason Ng for fixing pt-visual-explain bug PT-2277
- Kushal Haldar for his reports about vulnerabilities in Go-based tools
Percona Toolkit: Free your DBAs with advanced open source command-line tools.