Move Percona Monitoring and Management Server Data From One Instance Type to Another

Percona Monitoring and Management (PMM2) Server runs as a Docker container, a Virtual appliance, or as an instance on Amazon or Azure cloud services. Here I’ll show how to move the PMM Server and its data from one type to another.

Note, this is only for PMM2 to PMM2—you can’t migrate data from PMM Server version 1 to version 2 because of significant architectural differences.

For this exercise, imagine that your PMM server:

• Is running on an Amazon EC2 instance (Server A) from an AMI,
• You want to move it to a dedicated server (Server B) running as a Docker container.
• Server A monitors one client instance (node1) with a MongoDB service (mongodb1).

Here’s the output of pmm-admin status for this instance.

Export Data

PMM2 data is stored in the /srv folder for all types of installations. So first make a backup archive of it.

tar -cv /srv | gzip > pmm-data.tar.gz

Copy this archive to Server B.

scp pmm-data.tar.gz user1@172.17.0.2:~/

Prepare New Server

Connect to Server B run all further commands on this server. Prepare the Docker container.

docker create -v /srv --name pmm-data percona/pmm-server:2 /bin/true

Next extract exported data from the archive.

tar -zxvf pmm-data.tar.gz -C /tmp

Create a container for the new PMM Server with a

/srv

partition on a separate container (

pmm-data

).

docker run -d -p 443:443 --volumes-from pmm-data --name pmm-server --restart always percona/pmm-server:2

Stop all services and copy the exported data into the container.

docker exec -it pmm-server supervisorctl stop all
docker exec -it pmm-server sh -c 'cd /; rm -rf /srv/victoriametrics/data'
docker cp /tmp/srv pmm-data:/

Restore permissions for migrated data folders.

docker exec -it pmm-server chown -R root:pmm /srv/clickhouse /srv/ia /srv/nginx /srv/pmm-distribution /srv/update
docker exec -it pmm-server chown -R pmm:pmm /srv/logs /srv/victoriametrics /srv/alertmanager /srv/prometheus
docker exec -it pmm-server chown -R grafana:grafana /srv/grafana
docker exec -it pmm-server chown -R postgres:postgres /srv/postgres /srv/logs/postgresql.log

Restart PMM Server so that it reloads files with the correct permissions.

docker restart pmm-server

Switch Services to New Server

That’s it! Now you can switch your monitored node1 to use the new server (Server B).

Edit the the PMM agent configuration file

/usr/local/percona/pmm2/config/pmm-agent.yaml

.

Set the IP address of Server B (

172.17.0.2

) and restart

pmm-agent

.

systemctl restart pmm-agent

Check Status

Check the status of

pmm-agent

and monitored services with

pmm-admin status

.

The agent is now connected to your new server.

In the Grafana UI, you can see the migrated data of Server B. (The time gap in the data is how long it took to run the import and switch node1 to the new server.)

If historical data is here then we are done. Otherwise, please follow the commands that are provided in the next section.

Export/Import VictoriaMetrics Data

Copy the metrics in the VictoriaMetrics time-series database using an API request for export/import data. (You can do the export remotely and run all further commands on Server B.)

curl -k -G -u admin:admin https://3.86.222.201/prometheus/api/v1/export/native -d 'match={__name__!=""}' > exported_data.dump

Next import the VictoriaMetrics data.

curl -k -u admin:admin -X POST https://172.17.0.2/prometheus/api/v1/import/native -T exported_data.dump

By default, the maximum allowed size of the client request body for PMM Server’s Nginx service is 10Mb. If

exported_data.dump

is bigger than this you must increase the limit and repeat the import.

docker exec -it pmm-server bash -c "sed -i 's/client_max_body_size 10m;/client_max_body_size 1000m;/g' /etc/nginx/conf.d/pmm.conf"
docker exec -it pmm-server bash -c "supervisorctl restart nginx"

Conclusion

You can use the same process to move from any instance type to another. Also, we have got a separate blog post about how to migrate if the pmm-data container isn’t used. Check it out!