Managing MySQL Configurations with the PXC Kubernetes Operator V1.10.0 Part 2: Walkthrough

In part one of this series, we introduced the different ways to manage MySQL configurations. In this post, we will walk through different possibilities and the changes happening while modifying MySQL configurations with the operator.

Percona Distribution for MySQL Operator based on Percona XtraDB Cluster (PXC) provides three ways for managing MySQL, but the question is, what is the precedence among options? We will walk through several cases of using MySQL configs. For the sake of simplicity, we will play with the values of the Galera cache to see the effects.

configuration: |
   [mysqld]
   wsrep_debug=CLIENT
   wsrep_provider_options="gcache.size=64M; gcache.recover=yes"

CASE-1: Modify Percona XtraDB Cluster object

If PXC object is not yet present, configurations can be edited in the cr.yaml and applied with

kubectl apply -f cr.yaml

.spec.pxc.configuration

pxc:
   size: 3
   image: percona/percona-xtradb-cluster:8.0.25
   autoRecovery: true
   configuration: |
     [mysqld]
     wsrep_debug=CLIENT
     wsrep_provider_options="gcache.size=64M; gcache.recover=yes"

After applying the changes and verifying the configuration:

# kubectl apply -f deploy/cr.yaml
perconaxtradbcluster.pxc.percona.com/cluster1 configured
# kubectl get pxc cluster1 -ojson | jq .spec.pxc.configuration
"[mysqld]\nwsrep_debug=CLIENT\nwsrep_provider_options=\"gcache.size=64M; gcache.recover=yes\"\n"

An interesting observation is when we add MySQL configuration into the PXC object, the ConfigMap named cluster1-pxc (cluster1-pxc in this case) is automatically created. This could be observed in the following snippet where the loop instruction was started before the kubectl apply command:

# for i in `seq 1 100`; do kubectl get cm cluster1-pxc; sleep 3 ;done
Error from server (NotFound): configmaps "cluster1-pxc" not found
NAME           DATA   AGE
cluster1-pxc   1      3s
NAME           DATA   AGE
cluster1-pxc   1      7s

If we check the content of ConfigMap, we can see that MySQL configuration from PXC object is written.

# kubectl get cm cluster1-pxc -ojson | jq .data
{
 "init.cnf": "[mysqld]\nwsrep_debug=CLIENT\nwsrep_provider_options=\"gcache.size=64M; gcache.recover=yes\"\n"
}

Now let’s change the gcache size from 64M to 128M in cr.yaml and apply the changes with

kubectl apply -f cr.yaml

configuration: |
     [mysqld]
     wsrep_debug=CLIENT
     wsrep_provider_options="gcache.size=128M; gcache.recover=yes"

PXC object got changed and also the ConfigMap values will be changed:

# kubectl get pxc cluster1 -ojson | jq .spec.pxc.configuration
"[mysqld]\nwsrep_debug=CLIENT\nwsrep_provider_options=\"gcache.size=128M; gcache.recover=yes\"\n"

for i in `seq 1 100`; do kubectl get cm cluster1-pxc -ojson | jq .data; sleep 3 ;done
{
 "init.cnf": "[mysqld]\nwsrep_debug=CLIENT\nwsrep_provider_options=\"gcache.size=64M; gcache.recover=yes\"\n"
}
{
 "init.cnf": "[mysqld]\nwsrep_debug=CLIENT\nwsrep_provider_options=\"gcache.size=128M; gcache.recover=yes\"\n"
}

ConfigMap cluster1-pxc  is automatically reconciled by the operator to reflect the values of PXC object.

CASE-2: When MySQL configuration is present in both PXC object and ConfigMap and ConfigMap is modified

Following up on CASE-1, let’s try to edit the ConfigMap and see the behavior.

Configmap changes of gcache size from 128M to 256M (changed with

kubectl edit cm cluster1-pxc

# apiVersion: v1
data:
 init.cnf: |
   [mysqld]
   wsrep_debug=CLIENT
   wsrep_provider_options="gcache.size=256M; gcache.recover=yes"

As it can be seen from the below output, the operator reconciles the ConfigMap and reverts the setting of the PXC object to ConfigMap.Configuration in PXC object takes precedence over ConfigMap if both are present.

Added comments to make explain the transition:

# for i in `seq 1 100`; do kubectl get cm cluster1-pxc -ojson | jq .data; sleep 1 ;done  
{
 "init.cnf": "[mysqld]\nwsrep_debug=CLIENT\nwsrep_provider_options=\"gcache.size=128M; gcache.recover=yes\"\n"
}
{
 "init.cnf": "[mysqld]\nwsrep_debug=CLIENT\nwsrep_provider_options=\"gcache.size=256M; gcache.recover=yes\"\n"
}
{
 "init.cnf": "[mysqld]\nwsrep_debug=CLIENT\nwsrep_provider_options=\"gcache.size=256M; gcache.recover=yes\"\n"
}
{
 "init.cnf": "[mysqld]\nwsrep_debug=CLIENT\nwsrep_provider_options=\"gcache.size=128M; gcache.recover=yes\"\n"
}
{
 "init.cnf": "[mysqld]\nwsrep_debug=CLIENT\nwsrep_provider_options=\"gcache.size=128M; gcache.recover=yes\"\n"
}

CASE-3: Modifying the ConfigMap cluster1-pxc when there is no MySQL configuration in PXC object

Let’s clean up the MySQL configuration in PXC object. This is done by removing the configuration section of PXC in cr.yaml and applying changes with

kubectl apply -f cr.yaml

After applying the changes, PXC object can be verified:

# kubectl get pxc cluster1 -ojson | jq .spec.pxc.configuration
 
#

Now let’s create a ConfigMap cluster1-pxc with MySQL configurations:

# cat my.cnf
[mysqld]
wsrep_debug=CLIENT
wsrep_provider_options="gcache.size=256M; gcache.recover=yes"
 
# kubectl create configmap cluster1-pxc --from-file my.cnf
configmap/cluster1-pxc created

An interesting point to observe is PXC object is not updated:

# kubectl get pxc cluster1 -ojson | jq .spec.pxc.configuration
 
#

However, the changes are reflected in DB after pods are recycled and updated with the new configuration:

# kubectl run -i --rm --tty percona-client --image=percona:8.0 --restart=Never -- bash -il
 
[mysql@percona-client /]$ mysql -h cluster1-pxc-2.cluster1-pxc.pxc.svc.cluster.local -uroot -proot_password
 
mysql> SHOW VARIABLES LIKE 'wsrep_provider_options'\G
*************************** 1. row ***************************
<snip> …  gcache.size = 256M; … </snip>
 
mysql>

In the next post, we will see the precedence when secrets are used for MySQL configurations in PXC Operator. Stay tuned!