Deployment of openstack/base on MaaS fails with 'certificates missing'/ 'Failed at step KEYRING spawning /bin/mount: Required key not available' errors

juju
,
1

After having successfully bootstrapped a maas-controller on a maas cloud with four servers, I am now trying to deploy openstack/base with juju. All containers come up to now be in the ‘active/idle’ state. The only exceptions are the ovn-central and the ovn-chassis apps. They are in a ‘blocked’ state, with the message being ‘certificates missing’. When I juju ssh into one of the ovn-central units and journalctl -p err -b, I get following error:

May 26 22:24:26 juju-58128b-0-lxd-5 systemd[4316]: snap-snapd-7264.mount: Failed to set up kernel keyring: Required key not available
May 26 22:24:26 juju-58128b-0-lxd-5 systemd[4316]: snap-snapd-7264.mount: Failed at step KEYRING spawning /bin/mount: Required key not available
May 26 22:24:26 juju-58128b-0-lxd-5 systemd[1]: Failed to mount Mount unit for snapd, revision 7264.
May 26 22:37:48 juju-58128b-0-lxd-5 systemd-udevd[6433]: Failed to chown '/dev/net/tun' 0 0: Operation not permitted
May 26 22:37:48 juju-58128b-0-lxd-5 systemd-udevd[6433]: Failed to apply permissions on static device nodes: Operation not permitted
May 26 22:38:09 juju-58128b-0-lxd-5 systemd[1]: Failed to start Execute cloud user/final scripts.
May 27 08:56:34 juju-58128b-0-lxd-5 systemd[185756]: snap-lxd-15161.mount: Failed to link user keyring into session keyring: Required key not available
May 27 08:56:34 juju-58128b-0-lxd-5 systemd[185756]: snap-lxd-15161.mount: Failed to set up kernel keyring: Required key not available
May 27 08:56:34 juju-58128b-0-lxd-5 systemd[185756]: snap-lxd-15161.mount: Failed at step KEYRING spawning /bin/mount: Required key not available
May 27 08:56:34 juju-58128b-0-lxd-5 systemd[1]: Failed to mount Mount unit for lxd, revision 15161.

Does anyone know what this could be due to?

2

Hi, specifically what instructions did you follow, especially for deploying OVN?

3

Thanks for the reply. I have used juju deploy cs:bundle/openstack-base as indicated here. Could there be a link with vault having to be unsealed before this container is requesting the key ?

4

I do not see OVN as part of the openstack-base bundle. How did you deploy OVN?

5

It seems it is, as you can see here and here.

6

You’re right. I was looking at the source repository which is exceptionally not synced at the moment.

Yes, there is probably an issue with Vault and certificates. I will look into it.

7

Thanks a lot. I am trying to remove and redeploy the ovn applications, let’s see if that works. Edit: I have tried this, but it yields the same error. So it is not a question of vault not being unsealed at the moment of installation.

8

So you do need to unseal Vault but what you also need is an SSL server certificate. You can either do this automatically or manually (with the vault charm’s actions).

  1. Automatically
    Have Vault generate a CA certificate and sign a certificate:

    juju run-action --wait vault/leader generate-root-ca

  2. Manually
    Provide Vault with a signed certificate:

    juju run-action --wait vault/leader get-csr
<sign CSR>
juju run-action --wait vault/leader upload-signed-csr pem=<certificate-chain> root-ca=<intermediate-CAs,root-CA>

I haven’t actually done it manually before.

Once you’ve done that, wait a few minutes for churn to settle. Your deployment should then be all green.

1 Like
9

Thanks, that was it. I could have sworn I tried that before, but now it works.