Help with Percona Cluster to Cluster Replication

I’m trying to build two 3 node MySQL clusters and then replicate databases from cluster A to Cluster B.
I’m using the information provided in the Percona Cluster charm on the charmed store.
When I add the master slave relation some units go into error state saying that the relation hook failed and no databases are actually replicated.

I’ve even gone so far as to set the root-password to the same value on both clusters in the charm config because I was seeing failed authentication notices in the mysqld log after creating the master slave relation.

Any body have any luck with this?

 juju deploy percona-cluster master --series bionic --constraints "mem=2048"
 juju config master vip="10.10.10.10"
 juju config master cluster-id=1
 juju config master min-cluster-size=3
 juju deploy hacluster master-hacluster
 juju relate master-hacluster master
 juju add-unit master -n 2

 juju deploy percona-cluster slave --series bionic --constraints "mem=2048"
 juju config slave vip="10.10.10.11"
 juju config slave cluster-id=2
 juju config slave min-cluster-size=3
 juju deploy hacluster slave-hacluster
 juju relate slave-hacluster slave
 juju add-unit slave -n 2

 juju status 

Model            Controller                      Cloud/Region                       Version  SLA          Timestamp
percona-cluster  juju28.openstack.devzero.co.za  openstack.devzero.co.za/RegionOne  2.8-rc2  unsupported  16:59:55Z

App               Version  Status  Scale  Charm            Store       Rev  OS      Notes
master            5.7.20   active      3  percona-cluster  jujucharms  288  ubuntu
master-hacluster           active      3  hacluster        jujucharms   66  ubuntu
slave             5.7.20   active      3  percona-cluster  jujucharms  288  ubuntu
slave-hacluster            active      3  hacluster        jujucharms   66  ubuntu

Unit                   Workload  Agent  Machine  Public address  Ports     Message
master/0*              active    idle   0        10.10.10.187    3306/tcp  Unit is ready
  master-hacluster/2   active    idle            10.10.10.187              Unit is ready and clustered
master/1               active    idle   1        10.10.10.229    3306/tcp  Unit is ready
  master-hacluster/0*  active    idle            10.10.10.229              Unit is ready and clustered
master/2               active    idle   5        10.10.10.15     3306/tcp  Unit is ready
  master-hacluster/1   active    idle            10.10.10.15               Unit is ready and clustered
slave/0*               active    idle   2        10.10.10.6      3306/tcp  Unit is ready
  slave-hacluster/2    active    idle            10.10.10.6                Unit is ready and clustered
slave/1                active    idle   3        10.10.10.112    3306/tcp  Unit is ready
  slave-hacluster/0*   active    idle            10.10.10.112              Unit is ready and clustered
slave/2                active    idle   4        10.10.10.133    3306/tcp  Unit is ready
  slave-hacluster/1    active    idle            10.10.10.133              Unit is ready and clustered

Machine  State    DNS           Inst id                               Series  AZ    Message
0        started  10.10.10.187  a54825c1-170c-413c-87e0-5657b4d42bcc  bionic  nova  ACTIVE
1        started  10.10.10.229  a8af6773-fbfb-4811-a2a4-054d745c7b7d  bionic  nova  ACTIVE
2        started  10.10.10.6    ad3342d6-eda8-4884-970e-51b9d1e8f7b3  bionic  nova  ACTIVE
3        started  10.10.10.112  316b38ad-26fb-4289-8118-24caa873e12c  bionic  nova  ACTIVE
4        started  10.10.10.133  cddab01b-8c28-4e2b-beb6-2123a0a8b0d9  bionic  nova  ACTIVE
5        started  10.10.10.15   9b5bc5cc-9ec8-4a7e-970b-c498e8013b47  bionic  nova  ACTIVE




 juju config master databases-to-replicate="SeanShupingDB"
 juju config slave databases-to-replicate="SeanShupingDB"

 juju relate master:master slave:slave

Model            Controller                      Cloud/Region                       Version  SLA          Timestamp
percona-cluster  juju28.openstack.devzero.co.za  openstack.devzero.co.za/RegionOne  2.8-rc2  unsupported  17:02:32Z

App               Version  Status  Scale  Charm            Store       Rev  OS      Notes
master            5.7.20   active      3  percona-cluster  jujucharms  288  ubuntu
master-hacluster           active      3  hacluster        jujucharms   66  ubuntu
slave             5.7.20   error       3  percona-cluster  jujucharms  288  ubuntu
slave-hacluster            active      3  hacluster        jujucharms   66  ubuntu

Unit                   Workload  Agent  Machine  Public address  Ports     Message
master/0*              active    idle   0        10.10.10.187    3306/tcp  Unit is ready
  master-hacluster/2   active    idle            10.10.10.187              Unit is ready and clustered
master/1               active    idle   1        10.10.10.229    3306/tcp  Unit is ready
  master-hacluster/0*  active    idle            10.10.10.229              Unit is ready and clustered
master/2               active    idle   5        10.10.10.15     3306/tcp  Unit is ready
  master-hacluster/1   active    idle            10.10.10.15               Unit is ready and clustered
slave/0*               active    idle   2        10.10.10.6      3306/tcp  Unit is ready
  slave-hacluster/2    active    idle            10.10.10.6                Unit is ready and clustered
slave/1                error     idle   3        10.10.10.112    3306/tcp  hook failed: "slave-relation-changed"
  slave-hacluster/0*   active    idle            10.10.10.112              Unit is ready and clustered
slave/2                error     idle   4        10.10.10.133    3306/tcp  hook failed: "slave-relation-changed"
  slave-hacluster/1    active    idle            10.10.10.133              Unit is ready and clustered

Machine  State    DNS           Inst id                               Series  AZ    Message
0        started  10.10.10.187  a54825c1-170c-413c-87e0-5657b4d42bcc  bionic  nova  ACTIVE
1        started  10.10.10.229  a8af6773-fbfb-4811-a2a4-054d745c7b7d  bionic  nova  ACTIVE
2        started  10.10.10.6    ad3342d6-eda8-4884-970e-51b9d1e8f7b3  bionic  nova  ACTIVE
3        started  10.10.10.112  316b38ad-26fb-4289-8118-24caa873e12c  bionic  nova  ACTIVE
4        started  10.10.10.133  cddab01b-8c28-4e2b-beb6-2123a0a8b0d9  bionic  nova  ACTIVE
5        started  10.10.10.15   9b5bc5cc-9ec8-4a7e-970b-c498e8013b47  bionic  nova  ACTIVE

Sean, thank you for reporting this. It seems you are running into a bug. I have filed that bug here where you can follow along until we have a fix.

2 Likes

Thanks for confirming and opening the bug @thedac.
Appreciate it, will follow the bug id

I have the same problem, fixed adding “log_slave_updates = true” to
/etc/mysql/percona-xtradb-cluster.conf.d/mysqld.cnf
and restart the mysql service. I opened a bug asking to add this property available in the charm