Unable to install OpenStack, radoswg stuck in blocked state

I have been having a reoccurring error with the OpenStack-base module the radoswg is always blocked with the error “Services not running that should be: ceph-radosgw@rgw.juju-e94665-3-lxd-0”

I have tried a fresh install multiple times, I’m a new to OpenStack and there ts a good chance its a user-error.

Thank you.

Welcome @strafe. Sorry to hear that you’re encountering issues.

On the face of it, it looks like you’re missing a relation.

Could you please provide the steps that you’re taking? Perhaps also run juju status and juju debug-log and submit the output to https://paste.ubuntu.com?

As background, I also recommend reading through the following guide:

https://docs.openstack.org/project-deploy-guide/charm-deployment-guide/latest/install-openstack.html

$ sudo juju status
Model     Controller       Cloud/Region    Version  SLA          Timestamp
devstack  maas-controller  mymaas/default  2.7.4    unsupported  19:32:17Z

App                    Version  Status   Scale  Charm                  Store       Rev  OS      Notes
ceph-mon               14.2.8   active       3  ceph-mon               jujucharms   44  ubuntu
ceph-osd               14.2.8   active       3  ceph-osd               jujucharms  294  ubuntu
ceph-radosgw           14.2.8   blocked      1  ceph-radosgw           jujucharms  283  ubuntu
cinder                 15.0.1   active       1  cinder                 jujucharms  297  ubuntu
cinder-ceph            15.0.1   active       1  cinder-ceph            jujucharms  251  ubuntu
glance                 19.0.0   active       1  glance                 jujucharms  291  ubuntu
keystone               16.0.0   active       1  keystone               jujucharms  309  ubuntu
mysql                  5.7.20   active       1  percona-cluster        jujucharms  281  ubuntu
neutron-api            15.0.1   active       1  neutron-api            jujucharms  282  ubuntu
neutron-gateway        15.0.1   active       1  neutron-gateway        jujucharms  276  ubuntu
neutron-openvswitch    15.0.1   active       3  neutron-openvswitch    jujucharms  269  ubuntu
nova-cloud-controller  20.0.1   active       1  nova-cloud-controller  jujucharms  339  ubuntu
nova-compute           20.0.1   active       3  nova-compute           jujucharms  309  ubuntu
ntp                    3.2      active       4  ntp                    jujucharms   36  ubuntu
openstack-dashboard    16.0.0   active       1  openstack-dashboard    jujucharms  297  ubuntu
placement              2.0.0    active       1  placement              jujucharms    9  ubuntu
rabbitmq-server        3.6.10   active       1  rabbitmq-server        jujucharms   97  ubuntu

Unit                      Workload  Agent  Machine  Public address  Ports              Message
ceph-mon/0                active    idle   1/lxd/0  192.168.20.76                      Unit is ready and clustered
ceph-mon/1                active    idle   2/lxd/0  192.168.20.79                      Unit is ready and clustered
ceph-mon/2*               active    idle   0/lxd/1  192.168.20.72                      Unit is ready and clustered
ceph-osd/0                active    idle   0        192.168.20.68                      Unit is ready (1 OSD)
ceph-osd/1*               active    idle   2        192.168.20.70                      Unit is ready (1 OSD)
ceph-osd/2                active    idle   1        192.168.20.69                      Unit is ready (1 OSD)
ceph-radosgw/0*           blocked   idle   3/lxd/0  192.168.20.78   80/tcp             Services not running that should be: ceph-radosgw@rgw.juju-e94665-3-lxd-0
cinder/0*                 active    idle   1/lxd/1  192.168.20.81   8776/tcp           Unit is ready
  cinder-ceph/0*          active    idle            192.168.20.81                      Unit is ready
glance/0*                 active    idle   0/lxd/0  192.168.20.82   9292/tcp           Unit is ready
keystone/0*               active    idle   2/lxd/1  192.168.20.75   5000/tcp           Unit is ready
mysql/0*                  active    idle   3/lxd/2  192.168.20.73   3306/tcp           Unit is ready
neutron-api/0*            active    idle   1/lxd/2  192.168.20.80   9696/tcp           Unit is ready
neutron-gateway/0*        active    idle   3        192.168.20.71                      Unit is ready
  ntp/0*                  active    idle            192.168.20.71   123/udp            chrony: Ready
nova-cloud-controller/0*  active    idle   0/lxd/3  192.168.20.83   8774/tcp,8775/tcp  Unit is ready
nova-compute/0            active    idle   1        192.168.20.69                      Unit is ready
  neutron-openvswitch/0*  active    idle            192.168.20.69                      Unit is ready
  ntp/1                   active    idle            192.168.20.69   123/udp            chrony: Ready
nova-compute/1            active    idle   0        192.168.20.68                      Unit is ready
  neutron-openvswitch/2   active    idle            192.168.20.68                      Unit is ready
  ntp/3                   active    idle            192.168.20.68   123/udp            chrony: Ready
nova-compute/2*           active    idle   2        192.168.20.70                      Unit is ready
  neutron-openvswitch/1   active    idle            192.168.20.70                      Unit is ready
  ntp/2                   active    idle            192.168.20.70   123/udp            chrony: Ready
openstack-dashboard/0*    active    idle   2/lxd/2  192.168.20.77   80/tcp,443/tcp     Unit is ready
placement/0*              active    idle   0/lxd/2  192.168.20.84   8778/tcp           Unit is ready
rabbitmq-server/0*        active    idle   3/lxd/1  192.168.20.74   5672/tcp           Unit is ready

Machine  State    DNS            Inst id              Series  AZ       Message
0        started  192.168.20.68  pumped-sloth         bionic  default  Deployed
0/lxd/0  started  192.168.20.82  juju-e94665-0-lxd-0  bionic  default  Container started
0/lxd/1  started  192.168.20.72  juju-e94665-0-lxd-1  bionic  default  Container started
0/lxd/2  started  192.168.20.84  juju-e94665-0-lxd-2  bionic  default  Container started
0/lxd/3  started  192.168.20.83  juju-e94665-0-lxd-3  bionic  default  Container started
1        started  192.168.20.69  honest-bedbug        bionic  default  Deployed
1/lxd/0  started  192.168.20.76  juju-e94665-1-lxd-0  bionic  default  Container started
1/lxd/1  started  192.168.20.81  juju-e94665-1-lxd-1  bionic  default  Container started
1/lxd/2  started  192.168.20.80  juju-e94665-1-lxd-2  bionic  default  Container started
2        started  192.168.20.70  just-cod             bionic  default  Deployed
2/lxd/0  started  192.168.20.79  juju-e94665-2-lxd-0  bionic  default  Container started
2/lxd/1  started  192.168.20.75  juju-e94665-2-lxd-1  bionic  default  Container started
2/lxd/2  started  192.168.20.77  juju-e94665-2-lxd-2  bionic  default  Container started
3        started  192.168.20.71  casual-swine         bionic  default  Deployed
3/lxd/0  started  192.168.20.78  juju-e94665-3-lxd-0  bionic  default  Container started
3/lxd/1  started  192.168.20.74  juju-e94665-3-lxd-1  bionic  default  Container started
3/lxd/2  started  192.168.20.73  juju-e94665-3-lxd-2  bionic  default  Container started

Sorry, I should have asked for a more thorough command. Could you please re-run juju status with the following 2 options added?

juju status --relations --storage
$ sudo juju debug-log
unit-nova-cloud-controller-0: 19:35:09 DEBUG unit.nova-cloud-controller/0.juju-l                                                                              og 0 section(s) found
unit-nova-cloud-controller-0: 19:35:10 INFO unit.nova-cloud-controller/0.juju-lo                                                                              g HAProxy context is incomplete, this unit has no peers.
unit-nova-cloud-controller-0: 19:35:10 INFO unit.nova-cloud-controller/0.juju-lo                                                                              g HAProxy context is incomplete, this unit has no peers.
unit-nova-cloud-controller-0: 19:35:10 INFO unit.nova-cloud-controller/0.juju-lo                                                                              g Generating template context from neutron api relation
unit-nova-cloud-controller-0: 19:35:10 DEBUG unit.nova-cloud-controller/0.update                                                                              -status active
unit-nova-cloud-controller-0: 19:35:10 DEBUG unit.nova-cloud-controller/0.update                                                                              -status active
unit-nova-cloud-controller-0: 19:35:10 DEBUG unit.nova-cloud-controller/0.update                                                                              -status active
unit-nova-cloud-controller-0: 19:35:10 DEBUG unit.nova-cloud-controller/0.update                                                                              -status active
unit-nova-cloud-controller-0: 19:35:10 DEBUG unit.nova-cloud-controller/0.update                                                                              -status active
unit-nova-cloud-controller-0: 19:35:10 INFO unit.nova-cloud-controller/0.juju-lo                                                                              g Unit is ready
unit-openstack-dashboard-0: 19:34:08 INFO unit.openstack-dashboard/0.juju-log En                                                                              suring haproxy enabled in /etc/default/haproxy.
unit-openstack-dashboard-0: 19:34:08 DEBUG unit.openstack-dashboard/0.juju-log E                                                                              nsuring haproxy enabled in /etc/default/haproxy.
unit-openstack-dashboard-0: 19:34:08 DEBUG unit.openstack-dashboard/0.update-sta                                                                              tus lxc
unit-openstack-dashboard-0: 19:34:08 DEBUG unit.openstack-dashboard/0.update-sta                                                                              tus active
unit-openstack-dashboard-0: 19:34:08 DEBUG unit.openstack-dashboard/0.update-sta                                                                              tus active
unit-openstack-dashboard-0: 19:34:08 DEBUG unit.openstack-dashboard/0.update-sta                                                                              tus active
unit-openstack-dashboard-0: 19:34:08 INFO unit.openstack-dashboard/0.juju-log Un                                                                              it is ready
unit-neutron-gateway-0: 19:34:42 INFO unit.neutron-gateway/0.juju-log Registered                                                                               config file: /etc/neutron/neutron.conf
unit-neutron-gateway-0: 19:34:42 INFO unit.neutron-gateway/0.juju-log Registered                                                                               config file: /etc/neutron/l3_agent.ini
unit-neutron-gateway-0: 19:34:42 INFO unit.neutron-gateway/0.juju-log Registered                                                                               config file: /etc/neutron/metering_agent.ini
unit-neutron-gateway-0: 19:34:42 INFO unit.neutron-gateway/0.juju-log Registered                                                                               config file: /etc/neutron/vpn_agent.ini
unit-neutron-gateway-0: 19:34:42 INFO unit.neutron-gateway/0.juju-log Registered                                                                               config file: /etc/neutron/fwaas_driver.ini
unit-neutron-gateway-0: 19:34:42 INFO unit.neutron-gateway/0.juju-log Registered                                                                               config file: /etc/neutron/plugins/ml2/openvswitch_agent.ini
unit-neutron-gateway-0: 19:34:42 INFO unit.neutron-gateway/0.juju-log Registered                                                                               config file: /etc/apparmor.d/usr.bin.neutron-openvswitch-agent
unit-neutron-gateway-0: 19:34:42 INFO unit.neutron-gateway/0.juju-log Registered                                                                               config file: /etc/apparmor.d/usr.bin.neutron-l3-agent
unit-neutron-gateway-0: 19:34:42 INFO unit.neutron-gateway/0.juju-log Registered                                                                               config file: /etc/neutron/dhcp_agent.ini
unit-neutron-gateway-0: 19:34:42 INFO unit.neutron-gateway/0.juju-log Registered                                                                               config file: /etc/neutron/dnsmasq.conf
unit-neutron-gateway-0: 19:34:43 INFO unit.neutron-gateway/0.juju-log Registered                                                                               config file: /etc/neutron/metadata_agent.ini
unit-neutron-gateway-0: 19:34:43 INFO unit.neutron-gateway/0.juju-log Registered                                                                               config file: /etc/apparmor.d/usr.bin.neutron-dhcp-agent
unit-neutron-gateway-0: 19:34:43 INFO unit.neutron-gateway/0.juju-log Registered                                                                               config file: /etc/apparmor.d/usr.bin.neutron-metadata-agent
unit-neutron-gateway-0: 19:34:43 INFO unit.neutron-gateway/0.juju-log Registered                                                                               config file: /etc/apparmor.d/usr.bin.neutron-metering-agent
unit-neutron-gateway-0: 19:34:43 DEBUG unit.neutron-gateway/0.juju-log Hardening                                                                               function 'update_status'
unit-neutron-gateway-0: 19:34:43 DEBUG unit.neutron-gateway/0.juju-log No harden                                                                              ing applied to 'update_status'
unit-neutron-gateway-0: 19:34:43 INFO unit.neutron-gateway/0.juju-log Updating s                                                                              tatus.
unit-neutron-gateway-0: 19:34:44 DEBUG unit.neutron-gateway/0.juju-log Generatin                                                                              g template context for amqp
unit-neutron-gateway-0: 19:34:45 DEBUG unit.neutron-gateway/0.update-status acti                                                                              ve
unit-neutron-gateway-0: 19:34:45 DEBUG unit.neutron-gateway/0.update-status acti                                                                              ve
unit-neutron-gateway-0: 19:34:45 DEBUG unit.neutron-gateway/0.update-status acti                                                                              ve
unit-neutron-gateway-0: 19:34:45 DEBUG unit.neutron-gateway/0.update-status acti                                                                              ve
unit-neutron-gateway-0: 19:34:45 DEBUG unit.neutron-gateway/0.update-status acti                                                                              ve
unit-neutron-gateway-0: 19:34:45 INFO unit.neutron-gateway/0.juju-log Unit is re                                                                              ady
unit-ntp-3: 19:35:00 INFO unit.ntp/3.juju-log Reactive main running for hook upd                                                                              ate-status
unit-ntp-3: 19:35:00 ERROR unit.ntp/3.juju-log Unable to find implementation for                                                                               relation: requires of ntp
unit-nova-cloud-controller-0: 19:35:00 DEBUG unit.nova-cloud-controller/0.juju-l                                                                              og 0 section(s) found
unit-ntp-3: 19:35:00 ERROR unit.ntp/3.juju-log Unable to find implementation for                                                                               relation: provides of ntp
unit-ntp-3: 19:35:00 DEBUG unit.ntp/3.juju-log tracer>
tracer: starting handler dispatch, 43 flags set
tracer: set flag config.changed
tracer: set flag config.changed.auto_peers
tracer: set flag config.changed.auto_peers_upstream
tracer: set flag config.changed.nagios_context
tracer: set flag config.changed.nagios_ntpmon_checks
tracer: set flag config.changed.nagios_servicegroups
tracer: set flag config.changed.ntp_package
tracer: set flag config.changed.orphan_stratum
tracer: set flag config.changed.peers
tracer: set flag config.changed.pools
tracer: set flag config.changed.source
tracer: set flag config.changed.use_iburst
tracer: set flag config.changed.verify_ntp_servers
tracer: set flag config.default.auto_peers
tracer: set flag config.default.auto_peers_upstream
tracer: set flag config.default.nagios_context
tracer: set flag config.default.nagios_ntpmon_checks
tracer: set flag config.default.nagios_servicegroups
tracer: set flag config.default.ntp_package
tracer: set flag config.default.orphan_stratum
tracer: set flag config.default.peers
tracer: set flag config.default.pools
tracer: set flag config.default.source
tracer: set flag config.default.use_iburst
tracer: set flag config.default.verify_ntp_servers
tracer: set flag config.set.auto_peers_upstream
tracer: set flag config.set.nagios_context
tracer: set flag config.set.nagios_ntpmon_checks
tracer: set flag config.set.pools
tracer: set flag config.set.use_iburst
tracer: set flag endpoint.juju-info.joined
tracer: set flag endpoint.ntp-peers.changed
tracer: set flag endpoint.ntp-peers.changed.egress-subnets
tracer: set flag endpoint.ntp-peers.changed.ingress-address
tracer: set flag endpoint.ntp-peers.changed.private-address
tracer: set flag endpoint.ntp-peers.joined
tracer: set flag juju-info.available
tracer: set flag juju-info.connected
tracer: set flag ntp.configured
tracer: set flag ntp.installed
tracer: set flag ntpmon.configured
tracer: set flag ntpmon.installed
tracer: set flag ntpmon.started
unit-nova-cloud-controller-0: 19:35:00 INFO unit.nova-cloud-controller/0.juju-lo                                                                              g Registered config file: /etc/nova/nova.conf
unit-ntp-3: 19:35:00 DEBUG unit.ntp/3.juju-log tracer>
tracer: hooks phase, 1 handlers queued
tracer: ++   queue handler reactive/ntp.py:291:assess_status
unit-nova-cloud-controller-0: 19:35:00 INFO unit.nova-cloud-controller/0.juju-lo                                                                              g Registered config file: /etc/nova/api-paste.ini
unit-ntp-3: 19:35:00 INFO unit.ntp/3.juju-log Invoking reactive handler: reactiv                                                                              e/ntp.py:291:assess_status
unit-nova-cloud-controller-0: 19:35:00 INFO unit.nova-cloud-controller/0.juju-lo                                                                              g Registered config file: /etc/nova/vendor_data.json
unit-nova-cloud-controller-0: 19:35:00 INFO unit.nova-cloud-controller/0.juju-lo                                                                              g Registered config file: /etc/haproxy/haproxy.cfg
unit-nova-cloud-controller-0: 19:35:00 INFO unit.nova-cloud-controller/0.juju-lo                                                                              g Registered config file: /etc/apache2/sites-available/openstack_https_frontend.                                                                              conf
unit-nova-cloud-controller-0: 19:35:01 INFO unit.nova-cloud-controller/0.juju-lo                                                                              g Registered config file: /etc/memcached.conf
unit-nova-cloud-controller-0: 19:35:01 INFO unit.nova-cloud-controller/0.juju-lo                                                                              g Registered config file: /etc/apache2/sites-enabled/wsgi-api-os-compute.conf
unit-ntp-3: 19:35:01 DEBUG unit.ntp/3.update-status active
unit-nova-cloud-controller-0: 19:35:01 INFO unit.nova-cloud-controller/0.juju-lo                                                                              g Registered config file: /etc/apache2/sites-enabled/wsgi-openstack-metadata.con                                                                              f
unit-nova-cloud-controller-0: 19:35:01 DEBUG unit.nova-cloud-controller/0.juju-l                                                                              og Hardening function 'update_status'
unit-nova-cloud-controller-0: 19:35:01 DEBUG unit.nova-cloud-controller/0.juju-l                                                                              og No hardening applied to 'update_status'
unit-ntp-3: 19:35:01 DEBUG unit.ntp/3.juju-log tracer>
tracer: main dispatch loop, 2 handlers queued
tracer: ++   queue handler hooks/relations/juju-info/requires.py:19:changed:juju                                                                              -info
tracer: ++   queue handler reactive/ntp.py:169:config_changed
unit-nova-cloud-controller-0: 19:35:01 INFO unit.nova-cloud-controller/0.juju-lo                                                                              g Updating status.
unit-ntp-3: 19:35:01 INFO unit.ntp/3.juju-log Invoking reactive handler: reactiv                                                                              e/ntp.py:169:config_changed
unit-nova-cloud-controller-0: 19:35:01 INFO unit.nova-cloud-controller/0.juju-lo                                                                              g Registered config file:
unit-ntp-3: 19:35:01 DEBUG unit.ntp/3.juju-log tracer>
tracer: cleared flag ntp.configured
tracer: ++   queue handler ../.venv/lib/python3.6/site-packages/charmhelpers/cor                                                                              e/host.py:715:wrapped_f
unit-ntp-3: 19:35:01 INFO unit.ntp/3.juju-log Invoking reactive handler: hooks/r                                                                              elations/juju-info/requires.py:19:changed:juju-info
unit-ntp-3: 19:35:01 INFO unit.ntp/3.juju-log Invoking reactive handler: ../.ven                                                                              v/lib/python3.6/site-packages/charmhelpers/core/host.py:715:wrapped_f
unit-nova-cloud-controller-0: 19:35:01 DEBUG unit.nova-cloud-controller/0.juju-l                                                                              og 0 section(s) found
unit-ntp-3: 19:35:02 DEBUG unit.ntp/3.update-status Synchronizing state of chron                                                                              y.service with SysV service script with /lib/systemd/systemd-sysv-install.
unit-ntp-3: 19:35:02 DEBUG unit.ntp/3.update-status Executing: /lib/systemd/syst                                                                              emd-sysv-install enable chrony
unit-ntp-3: 19:35:03 DEBUG unit.ntp/3.update-status active
unit-ntp-3: 19:35:03 DEBUG unit.ntp/3.juju-log tracer>
tracer: set flag ntp.configured
tracer: -- dequeue handler ../.venv/lib/python3.6/site-packages/charmhelpers/cor                                                                              e/host.py:715:wrapped_f
unit-ntp-3: 19:35:03 DEBUG unit.ntp/3.update-status active
unit-keystone-0: 19:35:05 DEBUG unit.keystone/0.juju-log No rotation until at le                                                                              ast Mon Apr 13 19:45:08 2020
unit-nova-cloud-controller-0: 19:35:05 DEBUG unit.nova-cloud-controller/0.juju-l                                                                              og Generating template context for amqp
unit-nova-cloud-controller-0: 19:35:06 DEBUG unit.nova-cloud-controller/0.update                                                                              -status ERROR no relation id specified
unit-nova-cloud-controller-0: 19:35:06 DEBUG unit.nova-cloud-controller/0.juju-l                                                                              og Generating template context for image-service.
unit-nova-cloud-controller-0: 19:35:06 DEBUG unit.nova-cloud-controller/0.juju-l                                                                              og 0 section(s) found
unit-nova-cloud-controller-0: 19:35:06 INFO unit.nova-cloud-controller/0.juju-lo                                                                              g HAProxy context is incomplete, this unit has no peers.
unit-nova-cloud-controller-0: 19:35:07 DEBUG unit.nova-cloud-controller/0.juju-l                                                                              og Generating template context for identity-service
unit-nova-cloud-controller-0: 19:35:08 INFO unit.nova-cloud-controller/0.juju-lo                                                                              g Generating template context from neutron api relation
unit-nova-cloud-controller-0: 19:35:08 DEBUG unit.nova-cloud-controller/0.juju-l                                                                              og Generating template context for identity-service
unit-nova-cloud-controller-0: 19:35:09 DEBUG unit.nova-cloud-controller/0.juju-l                                                                              og Ensuring haproxy enabled in /etc/default/haproxy.
unit-nova-cloud-controller-0: 19:35:09 INFO unit.nova-cloud-controller/0.juju-lo                                                                              g HAProxy context is incomplete, this unit has no peers.
unit-rabbitmq-server-0: 19:35:40 DEBUG unit.rabbitmq-server/0.juju-log Hardening function 'update_status'
unit-rabbitmq-server-0: 19:35:40 DEBUG unit.rabbitmq-server/0.juju-log No hardening applied to 'update_status'
unit-rabbitmq-server-0: 19:35:40 INFO unit.rabbitmq-server/0.juju-log Updating status.
unit-rabbitmq-server-0: 19:35:47 DEBUG unit.rabbitmq-server/0.juju-log Calculating erl vm io thread pool size based on num_cpus=2 and multiplier=24
unit-rabbitmq-server-0: 19:35:47 DEBUG unit.rabbitmq-server/0.update-status lxc
unit-rabbitmq-server-0: 19:35:47 DEBUG unit.rabbitmq-server/0.update-status lxc
unit-rabbitmq-server-0: 19:35:47 DEBUG unit.rabbitmq-server/0.juju-log erl vm io thread pool size = 48 (capped=True)
unit-rabbitmq-server-0: 19:35:47 WARNING unit.rabbitmq-server/0.juju-log min-cluster-size is not defined, race conditions may occur if this is not a single unit deployment.
unit-rabbitmq-server-0: 19:35:47 DEBUG unit.rabbitmq-server/0.juju-log Waiting for rabbitmq app to start: /var/lib/rabbitmq/mnesia/rabbit@juju-e94665-3-lxd-1.pid
unit-rabbitmq-server-0: 19:35:47 DEBUG unit.rabbitmq-server/0.juju-log Running ['timeout', '180', '/usr/sbin/rabbitmqctl', 'wait', '/var/lib/rabbitmq/mnesia/rabbit@juju-e94665-3-lxd-1.pid']
unit-ceph-radosgw-0: 19:35:47 INFO unit.ceph-radosgw/0.juju-log Registered config file: /etc/haproxy/haproxy.cfg
unit-ceph-radosgw-0: 19:35:47 INFO unit.ceph-radosgw/0.juju-log Registered config file: /etc/ceph/ceph.conf
unit-ceph-radosgw-0: 19:35:48 DEBUG unit.ceph-radosgw/0.juju-log Hardening function 'update_status'
unit-ceph-radosgw-0: 19:35:48 DEBUG unit.ceph-radosgw/0.juju-log No hardening applied to 'update_status'
unit-ceph-radosgw-0: 19:35:48 INFO unit.ceph-radosgw/0.juju-log Updating status.
unit-rabbitmq-server-0: 19:35:49 DEBUG unit.rabbitmq-server/0.update-status Waiting for 'rabbit@juju-e94665-3-lxd-1'
unit-rabbitmq-server-0: 19:35:49 DEBUG unit.rabbitmq-server/0.update-status pid is 11072
unit-rabbitmq-server-0: 19:35:49 DEBUG unit.rabbitmq-server/0.juju-log Confirmed rabbitmq app is running
unit-rabbitmq-server-0: 19:35:49 DEBUG unit.rabbitmq-server/0.update-status active
unit-rabbitmq-server-0: 19:35:49 INFO unit.rabbitmq-server/0.juju-log Unit is ready
unit-ceph-radosgw-0: 19:35:51 DEBUG unit.ceph-radosgw/0.juju-log Ensuring haproxy enabled in /etc/default/haproxy.
unit-ceph-radosgw-0: 19:35:51 INFO unit.ceph-radosgw/0.juju-log HAProxy context is incomplete, this unit has no peers.
unit-ceph-radosgw-0: 19:35:52 DEBUG unit.ceph-radosgw/0.juju-log Generating template context for identity-service
unit-ceph-radosgw-0: 19:35:53 DEBUG unit.ceph-radosgw/0.update-status failed
unit-ceph-radosgw-0: 19:35:53 DEBUG unit.ceph-radosgw/0.update-status active
unit-ntp-2: 19:36:01 INFO unit.ntp/2.juju-log Reactive main running for hook update-status
unit-ntp-2: 19:36:01 ERROR unit.ntp/2.juju-log Unable to find implementation for relation: requires of ntp
unit-ntp-2: 19:36:01 ERROR unit.ntp/2.juju-log Unable to find implementation for relation: provides of ntp
unit-ntp-2: 19:36:01 DEBUG unit.ntp/2.juju-log tracer>
tracer: starting handler dispatch, 43 flags set
tracer: set flag config.changed
tracer: set flag config.changed.auto_peers
tracer: set flag config.changed.auto_peers_upstream
tracer: set flag config.changed.nagios_context
tracer: set flag config.changed.nagios_ntpmon_checks
tracer: set flag config.changed.nagios_servicegroups
tracer: set flag config.changed.ntp_package
tracer: set flag config.changed.orphan_stratum
tracer: set flag config.changed.peers
tracer: set flag config.changed.pools
tracer: set flag config.changed.source
tracer: set flag config.changed.use_iburst
tracer: set flag config.changed.verify_ntp_servers
tracer: set flag config.default.auto_peers
tracer: set flag config.default.auto_peers_upstream
tracer: set flag config.default.nagios_context
tracer: set flag config.default.nagios_ntpmon_checks
tracer: set flag config.default.nagios_servicegroups
tracer: set flag config.default.ntp_package
tracer: set flag config.default.orphan_stratum
tracer: set flag config.default.peers
tracer: set flag config.default.pools
tracer: set flag config.default.source
tracer: set flag config.default.use_iburst
tracer: set flag config.default.verify_ntp_servers
tracer: set flag config.set.auto_peers_upstream
tracer: set flag config.set.nagios_context
tracer: set flag config.set.nagios_ntpmon_checks
tracer: set flag config.set.pools
tracer: set flag config.set.use_iburst
tracer: set flag endpoint.juju-info.joined
tracer: set flag endpoint.ntp-peers.changed
tracer: set flag endpoint.ntp-peers.changed.egress-subnets
tracer: set flag endpoint.ntp-peers.changed.ingress-address
tracer: set flag endpoint.ntp-peers.changed.private-address
tracer: set flag endpoint.ntp-peers.joined
tracer: set flag juju-info.available
tracer: set flag juju-info.connected
tracer: set flag ntp.configured
tracer: set flag ntp.installed
tracer: set flag ntpmon.configured
tracer: set flag ntpmon.installed
tracer: set flag ntpmon.started
unit-ntp-2: 19:36:02 DEBUG unit.ntp/2.juju-log tracer>
tracer: hooks phase, 1 handlers queued
tracer: ++   queue handler reactive/ntp.py:291:assess_status
unit-ntp-2: 19:36:02 INFO unit.ntp/2.juju-log Invoking reactive handler: reactive/ntp.py:291:assess_status
unit-ntp-2: 19:36:02 DEBUG unit.ntp/2.update-status active
unit-ntp-2: 19:36:02 DEBUG unit.ntp/2.juju-log tracer>
tracer: main dispatch loop, 2 handlers queued
tracer: ++   queue handler hooks/relations/juju-info/requires.py:19:changed:juju-info
tracer: ++   queue handler reactive/ntp.py:169:config_changed
unit-ntp-2: 19:36:02 INFO unit.ntp/2.juju-log Invoking reactive handler: reactive/ntp.py:169:config_changed
unit-ntp-2: 19:36:02 DEBUG unit.ntp/2.juju-log tracer>
tracer: cleared flag ntp.configured
tracer: ++   queue handler ../.venv/lib/python3.6/site-packages/charmhelpers/core/host.py:715:wrapped_f
unit-ntp-2: 19:36:02 INFO unit.ntp/2.juju-log Invoking reactive handler: hooks/relations/juju-info/requires.py:19:changed:juju-info
unit-ntp-2: 19:36:02 INFO unit.ntp/2.juju-log Invoking reactive handler: ../.venv/lib/python3.6/site-packages/charmhelpers/core/host.py:715:wrapped_f
unit-ntp-2: 19:36:03 DEBUG unit.ntp/2.update-status Synchronizing state of chrony.service with SysV service script with /lib/systemd/systemd-sysv-install.
unit-ntp-2: 19:36:03 DEBUG unit.ntp/2.update-status Executing: /lib/systemd/systemd-sysv-install enable chrony
unit-ntp-2: 19:36:03 DEBUG unit.ntp/2.update-status active
unit-ntp-2: 19:36:03 DEBUG unit.ntp/2.juju-log tracer>
tracer: set flag ntp.configured
tracer: -- dequeue handler ../.venv/lib/python3.6/site-packages/charmhelpers/core/host.py:715:wrapped_f
unit-ntp-2: 19:36:04 DEBUG unit.ntp/2.update-status active
unit-cinder-ceph-0: 19:36:14 INFO unit.cinder-ceph/0.juju-log Making dir /var/lib/charm/cinder-ceph root:root 555
unit-cinder-ceph-0: 19:36:14 INFO unit.cinder-ceph/0.juju-log Making dir /etc/ceph root:root 555
unit-cinder-ceph-0: 19:36:14 INFO unit.cinder-ceph/0.juju-log Registered config file: /var/lib/charm/cinder-ceph/ceph.conf
unit-cinder-ceph-0: 19:36:15 DEBUG unit.cinder-ceph/0.juju-log Generating template context for ceph
unit-cinder-ceph-0: 19:36:16 INFO unit.cinder-ceph/0.juju-log Unit is ready
unit-ceph-osd-1: 19:36:16 DEBUG unit.ceph-osd/1.juju-log Hardening function 'update_status'
unit-ceph-osd-1: 19:36:16 DEBUG unit.ceph-osd/1.juju-log No hardening applied to 'update_status'
unit-ceph-osd-1: 19:36:17 INFO unit.ceph-osd/1.juju-log Updating status.
unit-ceph-osd-1: 19:36:18 DEBUG unit.ceph-osd/1.update-status b'smartctl 7.0 2018-12-30 r4883 [x86_64-linux-4.15.0-96-generic] (local build)\nCopyright (C) 2002-18, Bruce Allen, Christian Franke, www.smartmontools.org\n\n=== START OF INFORMATION SECTION ===\nVendor:               VMware,\nProduct:              VMware Virtual S\nRevision:             1.0\nUser Capacity:        85,899,345,920 bytes [85.8 GB]\nLogical block size:   512 bytes\nDevice type:          disk\nLocal Time is:        Mon Apr 13 19:36:18 2020 UTC\nSMART support is:     Unavailable - device lacks SMART capability.\n\n'
unit-neutron-api-0: 19:36:24 INFO unit.neutron-api/0.juju-log Registered config file: /etc/neutron/neutron.conf
unit-neutron-api-0: 19:36:24 INFO unit.neutron-api/0.juju-log Registered config file: /etc/default/neutron-server
unit-neutron-api-0: 19:36:24 INFO unit.neutron-api/0.juju-log Registered config file: /etc/neutron/api-paste.ini
unit-neutron-api-0: 19:36:24 INFO unit.neutron-api/0.juju-log Registered config file: /etc/apache2/sites-available/openstack_https_frontend.conf
unit-neutron-api-0: 19:36:24 INFO unit.neutron-api/0.juju-log Registered config file: /etc/haproxy/haproxy.cfg
unit-neutron-api-0: 19:36:24 INFO unit.neutron-api/0.juju-log Registered config file: /etc/neutron/neutron_vpnaas.conf
unit-neutron-api-0: 19:36:24 INFO unit.neutron-api/0.juju-log Registered config file: /etc/neutron/plugins/ml2/ml2_conf.ini
unit-neutron-api-0: 19:36:24 INFO unit.neutron-api/0.juju-log Registered config file: /etc/memcached.conf
unit-neutron-api-0: 19:36:25 DEBUG unit.neutron-api/0.juju-log Hardening function '_harden_inner2'
unit-neutron-api-0: 19:36:25 DEBUG unit.neutron-api/0.juju-log No hardening applied to '_harden_inner2'
unit-neutron-api-0: 19:36:25 DEBUG unit.neutron-api/0.juju-log Hardening function 'update_status'
unit-neutron-api-0: 19:36:25 DEBUG unit.neutron-api/0.juju-log No hardening applied to 'update_status'
unit-neutron-api-0: 19:36:25 INFO unit.neutron-api/0.juju-log Updating status.
unit-neutron-api-0: 19:36:28 DEBUG unit.neutron-api/0.juju-log Generating template context for amqp
unit-neutron-api-0: 19:36:28 DEBUG unit.neutron-api/0.update-status ERROR no relation id specified
unit-neutron-api-0: 19:36:28 DEBUG unit.neutron-api/0.juju-log Generating template context for identity-service
unit-neutron-api-0: 19:36:29 DEBUG unit.neutron-api/0.juju-log ML2 DNS Extensions are not enabled.
unit-neutron-api-0: 19:36:29 DEBUG unit.neutron-api/0.juju-log ML2 DNS Extensions are not enabled.
unit-neutron-api-0: 19:36:30 DEBUG unit.neutron-api/0.juju-log Ensuring haproxy enabled in /etc/default/haproxy.
unit-neutron-api-0: 19:36:30 INFO unit.neutron-api/0.juju-log HAProxy context is incomplete, this unit has no peers.
unit-neutron-api-0: 19:36:30 DEBUG unit.neutron-api/0.juju-log ML2 DNS Extensions are not enabled.
unit-neutron-api-0: 19:36:30 DEBUG unit.neutron-api/0.update-status active
unit-neutron-api-0: 19:36:30 DEBUG unit.neutron-api/0.update-status active
unit-neutron-api-0: 19:36:30 DEBUG unit.neutron-api/0.update-status active
unit-neutron-api-0: 19:36:30 DEBUG unit.neutron-api/0.update-status active
unit-neutron-api-0: 19:36:30 INFO unit.neutron-api/0.juju-log Unit is ready
unit-glance-0: 19:36:34 INFO unit.glance/0.juju-log Making dir /var/lib/charm/glance root:root 555
unit-glance-0: 19:36:34 INFO unit.glance/0.juju-log Making dir /etc/ceph root:root 555
unit-glance-0: 19:36:34 INFO unit.glance/0.juju-log Registered config file: /etc/glance/glance-api.conf
unit-glance-0: 19:36:34 INFO unit.glance/0.juju-log Registered config file: /etc/haproxy/haproxy.cfg
unit-glance-0: 19:36:34 INFO unit.glance/0.juju-log Registered config file: /var/lib/charm/glance/ceph.conf
unit-glance-0: 19:36:34 INFO unit.glance/0.juju-log Registered config file: /etc/apache2/sites-available/openstack_https_frontend.conf
unit-glance-0: 19:36:34 INFO unit.glance/0.juju-log Registered config file: /etc/memcached.conf
unit-glance-0: 19:36:35 INFO unit.glance/0.juju-log Registered config file: /etc/glance/glance-swift.conf
unit-glance-0: 19:36:35 DEBUG unit.glance/0.juju-log Hardening function 'update_status'
unit-glance-0: 19:36:35 DEBUG unit.glance/0.juju-log No hardening applied to 'update_status'
unit-glance-0: 19:36:35 INFO unit.glance/0.juju-log Updating status.
unit-glance-0: 19:36:37 DEBUG unit.glance/0.update-status ERROR no relation id specified
unit-glance-0: 19:36:37 DEBUG unit.glance/0.juju-log Generating template context for amqp
unit-glance-0: 19:36:38 DEBUG unit.glance/0.juju-log Generating template context for identity-service
unit-glance-0: 19:36:38 DEBUG unit.glance/0.juju-log 0 section(s) found
unit-glance-0: 19:36:39 DEBUG unit.glance/0.juju-log Ensuring haproxy enabled in /etc/default/haproxy.
unit-glance-0: 19:36:39 DEBUG unit.glance/0.juju-log Generating template context for ceph
unit-glance-0: 19:36:40 DEBUG unit.glance/0.juju-log Generating template context for identity-service
unit-glance-0: 19:36:40 DEBUG unit.glance/0.update-status active
unit-glance-0: 19:36:40 DEBUG unit.glance/0.update-status active
unit-glance-0: 19:36:40 DEBUG unit.glance/0.update-status active
unit-glance-0: 19:36:40 DEBUG unit.glance/0.update-status active
unit-glance-0: 19:36:40 INFO unit.glance/0.juju-log Unit is ready
unit-keystone-0: 19:36:44 INFO unit.keystone/0.juju-log Registered config file: /etc/keystone/keystone.conf
unit-keystone-0: 19:36:44 INFO unit.keystone/0.juju-log Registered config file: /etc/keystone/logging.conf
unit-keystone-0: 19:36:44 INFO unit.keystone/0.juju-log Registered config file: /etc/haproxy/haproxy.cfg
unit-keystone-0: 19:36:44 INFO unit.keystone/0.juju-log Registered config file: /etc/apache2/sites-available/openstack_https_frontend.conf
unit-keystone-0: 19:36:44 INFO unit.keystone/0.juju-log Registered config file: /etc/keystone/policy.json
unit-keystone-0: 19:36:44 INFO unit.keystone/0.juju-log Registered config file: /etc/cron.d/keystone-token-flush
unit-keystone-0: 19:36:44 INFO unit.keystone/0.juju-log Registered config file: /etc/cron.d/keystone-fernet-rotate-sync
unit-keystone-0: 19:36:44 INFO unit.keystone/0.juju-log Registered config file: /etc/apache2/sites-enabled/wsgi-openstack-api.conf
unit-keystone-0: 19:36:44 INFO unit.keystone/0.juju-log Registered config file: /etc/memcached.conf
unit-keystone-0: 19:36:44 DEBUG unit.keystone/0.juju-log Hardening function 'update_status'
unit-keystone-0: 19:36:45 DEBUG unit.keystone/0.juju-log No hardening applied to 'update_status'
unit-keystone-0: 19:36:45 INFO unit.keystone/0.juju-log Updating status.
unit-keystone-0: 19:36:47 INFO unit.keystone/0.juju-log Configuring Keystone to use a random admin token.
unit-keystone-0: 19:36:47 INFO unit.keystone/0.juju-log Loading a previously generated admin token from /var/lib/keystone/keystone.token
unit-keystone-0: 19:36:48 DEBUG unit.keystone/0.update-status ERROR no relation id specified
unit-keystone-0: 19:36:49 INFO unit.keystone/0.juju-log HAProxy context is incomplete, this unit has no peers.
unit-keystone-0: 19:36:49 DEBUG unit.keystone/0.juju-log 0 section(s) found
unit-keystone-0: 19:36:49 DEBUG unit.keystone/0.juju-log Ensuring haproxy enabled in /etc/default/haproxy.
unit-keystone-0: 19:36:49 INFO unit.keystone/0.juju-log HAProxy context is incomplete, this unit has no peers.
unit-keystone-0: 19:36:50 INFO unit.keystone/0.juju-log Configuring Keystone to use a random admin token.
unit-keystone-0: 19:36:50 INFO unit.keystone/0.juju-log Loading a previously generated admin token from /var/lib/keystone/keystone.token
unit-keystone-0: 19:36:50 INFO unit.keystone/0.juju-log Configuring Keystone to use a random admin token.
unit-keystone-0: 19:36:50 INFO unit.keystone/0.juju-log Loading a previously generated admin token from /var/lib/keystone/keystone.token
unit-keystone-0: 19:36:50 DEBUG unit.keystone/0.update-status active
unit-keystone-0: 19:36:50 DEBUG unit.keystone/0.update-status active
unit-keystone-0: 19:36:50 DEBUG unit.keystone/0.update-status active
unit-keystone-0: 19:36:50 INFO unit.keystone/0.juju-log Unit is ready
$ sudo juju status --relations --storage
Model     Controller       Cloud/Region    Version  SLA          Timestamp
devstack  maas-controller  mymaas/default  2.7.4    unsupported  19:38:58Z

App                    Version  Status   Scale  Charm                  Store       Rev  OS      Notes
ceph-mon               14.2.8   active       3  ceph-mon               jujucharms   44  ubuntu
ceph-osd               14.2.8   active       3  ceph-osd               jujucharms  294  ubuntu
ceph-radosgw           14.2.8   blocked      1  ceph-radosgw           jujucharms  283  ubuntu
cinder                 15.0.1   active       1  cinder                 jujucharms  297  ubuntu
cinder-ceph            15.0.1   active       1  cinder-ceph            jujucharms  251  ubuntu
glance                 19.0.0   active       1  glance                 jujucharms  291  ubuntu
keystone               16.0.0   active       1  keystone               jujucharms  309  ubuntu
mysql                  5.7.20   active       1  percona-cluster        jujucharms  281  ubuntu
neutron-api            15.0.1   active       1  neutron-api            jujucharms  282  ubuntu
neutron-gateway        15.0.1   active       1  neutron-gateway        jujucharms  276  ubuntu
neutron-openvswitch    15.0.1   active       3  neutron-openvswitch    jujucharms  269  ubuntu
nova-cloud-controller  20.0.1   active       1  nova-cloud-controller  jujucharms  339  ubuntu
nova-compute           20.0.1   active       3  nova-compute           jujucharms  309  ubuntu
ntp                    3.2      active       4  ntp                    jujucharms   36  ubuntu
openstack-dashboard    16.0.0   active       1  openstack-dashboard    jujucharms  297  ubuntu
placement              2.0.0    active       1  placement              jujucharms    9  ubuntu
rabbitmq-server        3.6.10   active       1  rabbitmq-server        jujucharms   97  ubuntu

Unit                      Workload  Agent  Machine  Public address  Ports              Message
ceph-mon/0                active    idle   1/lxd/0  192.168.20.76                      Unit is ready and clustered
ceph-mon/1                active    idle   2/lxd/0  192.168.20.79                      Unit is ready and clustered
ceph-mon/2*               active    idle   0/lxd/1  192.168.20.72                      Unit is ready and clustered
ceph-osd/0                active    idle   0        192.168.20.68                      Unit is ready (1 OSD)
ceph-osd/1*               active    idle   2        192.168.20.70                      Unit is ready (1 OSD)
ceph-osd/2                active    idle   1        192.168.20.69                      Unit is ready (1 OSD)
ceph-radosgw/0*           blocked   idle   3/lxd/0  192.168.20.78   80/tcp             Services not running that should be: ceph-radosgw@rgw.juju-e94665-3-lxd-0
cinder/0*                 active    idle   1/lxd/1  192.168.20.81   8776/tcp           Unit is ready
  cinder-ceph/0*          active    idle            192.168.20.81                      Unit is ready
glance/0*                 active    idle   0/lxd/0  192.168.20.82   9292/tcp           Unit is ready
keystone/0*               active    idle   2/lxd/1  192.168.20.75   5000/tcp           Unit is ready
mysql/0*                  active    idle   3/lxd/2  192.168.20.73   3306/tcp           Unit is ready
neutron-api/0*            active    idle   1/lxd/2  192.168.20.80   9696/tcp           Unit is ready
neutron-gateway/0*        active    idle   3        192.168.20.71                      Unit is ready
  ntp/0*                  active    idle            192.168.20.71   123/udp            chrony: Ready
nova-cloud-controller/0*  active    idle   0/lxd/3  192.168.20.83   8774/tcp,8775/tcp  Unit is ready
nova-compute/0            active    idle   1        192.168.20.69                      Unit is ready
  neutron-openvswitch/0*  active    idle            192.168.20.69                      Unit is ready
  ntp/1                   active    idle            192.168.20.69   123/udp            chrony: Ready
nova-compute/1            active    idle   0        192.168.20.68                      Unit is ready
  neutron-openvswitch/2   active    idle            192.168.20.68                      Unit is ready
  ntp/3                   active    idle            192.168.20.68   123/udp            chrony: Ready
nova-compute/2*           active    idle   2        192.168.20.70                      Unit is ready
  neutron-openvswitch/1   active    idle            192.168.20.70                      Unit is ready
  ntp/2                   active    idle            192.168.20.70   123/udp            chrony: Ready
openstack-dashboard/0*    active    idle   2/lxd/2  192.168.20.77   80/tcp,443/tcp     Unit is ready
placement/0*              active    idle   0/lxd/2  192.168.20.84   8778/tcp           Unit is ready
rabbitmq-server/0*        active    idle   3/lxd/1  192.168.20.74   5672/tcp           Unit is ready

Machine  State    DNS            Inst id              Series  AZ       Message
0        started  192.168.20.68  pumped-sloth         bionic  default  Deployed
0/lxd/0  started  192.168.20.82  juju-e94665-0-lxd-0  bionic  default  Container started
0/lxd/1  started  192.168.20.72  juju-e94665-0-lxd-1  bionic  default  Container started
0/lxd/2  started  192.168.20.84  juju-e94665-0-lxd-2  bionic  default  Container started
0/lxd/3  started  192.168.20.83  juju-e94665-0-lxd-3  bionic  default  Container started
1        started  192.168.20.69  honest-bedbug        bionic  default  Deployed
1/lxd/0  started  192.168.20.76  juju-e94665-1-lxd-0  bionic  default  Container started
1/lxd/1  started  192.168.20.81  juju-e94665-1-lxd-1  bionic  default  Container started
1/lxd/2  started  192.168.20.80  juju-e94665-1-lxd-2  bionic  default  Container started
2        started  192.168.20.70  just-cod             bionic  default  Deployed
2/lxd/0  started  192.168.20.79  juju-e94665-2-lxd-0  bionic  default  Container started
2/lxd/1  started  192.168.20.75  juju-e94665-2-lxd-1  bionic  default  Container started
2/lxd/2  started  192.168.20.77  juju-e94665-2-lxd-2  bionic  default  Container started
3        started  192.168.20.71  casual-swine         bionic  default  Deployed
3/lxd/0  started  192.168.20.78  juju-e94665-3-lxd-0  bionic  default  Container started
3/lxd/1  started  192.168.20.74  juju-e94665-3-lxd-1  bionic  default  Container started
3/lxd/2  started  192.168.20.73  juju-e94665-3-lxd-2  bionic  default  Container started

Relation provider                        Requirer                                       Interface               Type         Message
ceph-mon:client                          cinder-ceph:ceph                               ceph-client             regular
ceph-mon:client                          glance:ceph                                    ceph-client             regular
ceph-mon:client                          nova-compute:ceph                              ceph-client             regular
ceph-mon:mon                             ceph-mon:mon                                   ceph                    peer
ceph-mon:osd                             ceph-osd:mon                                   ceph-osd                regular
ceph-mon:radosgw                         ceph-radosgw:mon                               ceph-radosgw            regular
ceph-radosgw:cluster                     ceph-radosgw:cluster                           swift-ha                peer
cinder-ceph:ceph-access                  nova-compute:ceph-access                       cinder-ceph-key         regular
cinder-ceph:storage-backend              cinder:storage-backend                         cinder-backend          subordinate
cinder:cinder-volume-service             nova-cloud-controller:cinder-volume-service    cinder                  regular
cinder:cluster                           cinder:cluster                                 cinder-ha               peer
glance:cluster                           glance:cluster                                 glance-ha               peer
glance:image-service                     cinder:image-service                           glance                  regular
glance:image-service                     nova-cloud-controller:image-service            glance                  regular
glance:image-service                     nova-compute:image-service                     glance                  regular
keystone:cluster                         keystone:cluster                               keystone-ha             peer
keystone:identity-service                ceph-radosgw:identity-service                  keystone                regular
keystone:identity-service                cinder:identity-service                        keystone                regular
keystone:identity-service                glance:identity-service                        keystone                regular
keystone:identity-service                neutron-api:identity-service                   keystone                regular
keystone:identity-service                nova-cloud-controller:identity-service         keystone                regular
keystone:identity-service                openstack-dashboard:identity-service           keystone                regular
keystone:identity-service                placement:identity-service                     keystone                regular
mysql:cluster                            mysql:cluster                                  percona-cluster         peer
mysql:shared-db                          cinder:shared-db                               mysql-shared            regular
mysql:shared-db                          glance:shared-db                               mysql-shared            regular
mysql:shared-db                          keystone:shared-db                             mysql-shared            regular
mysql:shared-db                          neutron-api:shared-db                          mysql-shared            regular
mysql:shared-db                          nova-cloud-controller:shared-db                mysql-shared            regular
mysql:shared-db                          openstack-dashboard:shared-db                  mysql-shared            regular
mysql:shared-db                          placement:shared-db                            mysql-shared            regular
neutron-api:cluster                      neutron-api:cluster                            neutron-api-ha          peer
neutron-api:neutron-api                  nova-cloud-controller:neutron-api              neutron-api             regular
neutron-api:neutron-plugin-api           neutron-gateway:neutron-plugin-api             neutron-plugin-api      regular
neutron-api:neutron-plugin-api           neutron-openvswitch:neutron-plugin-api         neutron-plugin-api      regular
neutron-gateway:cluster                  neutron-gateway:cluster                        quantum-gateway-ha      peer
neutron-gateway:juju-info                ntp:juju-info                                  juju-info               subordinate
neutron-gateway:quantum-network-service  nova-cloud-controller:quantum-network-service  quantum                 regular
neutron-openvswitch:neutron-plugin       nova-compute:neutron-plugin                    neutron-plugin          subordinate
nova-cloud-controller:cluster            nova-cloud-controller:cluster                  nova-ha                 peer
nova-compute:cloud-compute               nova-cloud-controller:cloud-compute            nova-compute            regular
nova-compute:compute-peer                nova-compute:compute-peer                      nova                    peer
nova-compute:juju-info                   ntp:juju-info                                  juju-info               subordinate
ntp:ntp-peers                            ntp:ntp-peers                                  ntp                     peer
openstack-dashboard:cluster              openstack-dashboard:cluster                    openstack-dashboard-ha  peer
placement:cluster                        placement:cluster                              openstack-ha            peer
placement:placement                      nova-cloud-controller:placement                placement               regular
rabbitmq-server:amqp                     cinder:amqp                                    rabbitmq                regular
rabbitmq-server:amqp                     glance:amqp                                    rabbitmq                regular
rabbitmq-server:amqp                     neutron-api:amqp                               rabbitmq                regular
rabbitmq-server:amqp                     neutron-gateway:amqp                           rabbitmq                regular
rabbitmq-server:amqp                     neutron-openvswitch:amqp                       rabbitmq                regular
rabbitmq-server:amqp                     nova-cloud-controller:amqp                     rabbitmq                regular
rabbitmq-server:amqp                     nova-compute:amqp                              rabbitmq                regular
rabbitmq-server:cluster                  rabbitmq-server:cluster                        rabbitmq-ha             peer

Sidenote: Juju commands shouldn’t require sudo. How have you installed it?

i allways put sudo, its just habit

how have i installed openstack? via maas and juju openstack base charm

Welcome @strafe

Maybe you can try see what’s tripping up the ceph-radosgw@rgw.juju-e94665-3-lxd-0 service.

try run

juju ssh ceph-radosgw/0 sudo systemctl status ceph-radosgw@rgw.juju-e94665-3-lxd-0

The output might be able to point you in the right direction as to why the service isn’t running.
Usually it’s because something has ran out of memory or space or something along those lines

james@maas:~$ sudo juju ssh ceph-radosgw/0 sudo systemctl status ceph-radosgw@rgw.juju-e94665-3-lxd-0
ubuntu@192.168.20.78: Permission denied (publickey).
james@maas:~$

my ssh key must be wrong how do i add my private key and to what directory ?

try without sudo

or

juju add-ssh-key "public key material"

juju has the key but linux doesnt how to i add the private key into ubuntu?

Sound like the configuration is incomplete or one of the relation hooks didn’t run properly.

Is there actually any config in /etc/ceph/ceph.conf?

Hi @strafe, it looks like you may have run into Bug #1868387 “Intermittent deploy failure” : Bugs : OpenStack ceph-radosgw charm which looks fairly similar. You might be able to get away with removing the unit and re-adding a new one (of ceph-radosgw) to see if it’ll come up clean.

However, I don’t think we’ve seen this on the stable charms, so it is a bit strange that you are running into it. Any info that you can collect and add to the bug would be very useful!

Thanks

Have the same problem here. found that by entering the container:

ubuntu@juju-ac0843-0-lxd-0:~$ journalctl -b |grep radosgw
May 03 18:45:17 juju-ac0843-0-lxd-0 systemd[1]: radosgw.service: Failed to reset devices.list: Operation not permitted
May 03 18:45:17 juju-ac0843-0-lxd-0 systemd[1]: Starting LSB: radosgw RESTful rados gateway...
May 03 18:45:17 juju-ac0843-0-lxd-0 systemd[1]: jujud-unit-ceph-radosgw-0.service: Failed to reset devices.list: Operation not permitted
May 03 18:45:20 juju-ac0843-0-lxd-0 radosgw[230]: parse error setting 'debug_rgw' to '/5' (value must take the form N or N/M, where N and M are integers)
May 03 18:45:20 juju-ac0843-0-lxd-0 radosgw[230]: parse error setting 'err_to_syslog' to '' (Expected option value to be integer, got '')
May 03 18:45:20 juju-ac0843-0-lxd-0 radosgw[230]: parse error setting 'log_to_syslog' to '' (Expected option value to be integer, got '')

looking at /etc/ceph/ceph.conf, those variables were empty indeed. It seem something related to that