Openstack base - ovs-system + br-ext down

juju
1

Hi ,
I am installing openstack base bundle charm on Juju + Maas. Deploy task is success. Unit and application show active all. Ubuntu Pastebin
But ports ovs-system,br-ext, br-int in down mode. This is ip a command on comute node : Ubuntu Pastebin
Debug-log show some WARNING about:

unit-ovn-central-0: 07:46:54 WARNING unit.ovn-central/0.juju-log Skipping request for certificate for ip in int space, no local address found
unit-ovn-central-0: 07:46:54 WARNING unit.ovn-central/0.juju-log Skipping request for certificate for ip in admin space, no local address found
unit-ovn-central-0: 07:46:54 WARNING unit.ovn-central/0.juju-log Skipping request for certificate for ip in public space, no local address found

ovsdb-server-nb show many WARN:

2020-06-16T00:16:47.058Z|00197|stream_ssl|WARN|SSL_read: unexpected SSL connection close
2020-06-16T00:16:47.058Z|00198|reconnect|WARN|ssl:192.168.28.19:46526: connection dropped (Protocol error)
2020-06-16T00:16:47.058Z|00199|reconnect|WARN|ssl:192.168.28.19:46520: connection dropped (Protocol error)
2020-06-16T00:16:47.059Z|00200|reconnect|WARN|ssl:192.168.28.19:46538: connection dropped (Protocol error)
2020-06-16T00:16:47.059Z|00201|reconnect|WARN|ssl:192.168.28.19:46474: connection dropped (Protocol error)
2020-06-16T00:16:47.059Z|00202|reconnect|WARN|ssl:192.168.28.19:46536: connection dropped (Protocol error)
2020-06-16T00:16:47.059Z|00203|reconnect|WARN|ssl:192.168.28.19:46484: connection dropped (Protocol error)
2020-06-16T00:16:47.060Z|00204|reconnect|WARN|ssl:192.168.28.19:46502: connection dropped (Protocol error)
2020-06-16T00:16:47.065Z|00205|reconnect|WARN|ssl:192.168.28.19:46534: connection dropped (Protocol error)
2020-06-16T00:16:47.067Z|00206|reconnect|WARN|ssl:192.168.28.19:46514: connection dropped (Protocol error)
2020-06-16T00:16:47.067Z|00207|reconnect|WARN|ssl:192.168.28.19:46540: connection dropped (Protocol error)
2020-06-16T00:16:47.068Z|00208|reconnect|WARN|ssl:192.168.28.19:46544: connection dropped (Protocol error)
2020-06-16T00:16:47.069Z|00209|reconnect|WARN|ssl:192.168.28.19:46542: connection dropped (Protocol error)
2020-06-16T00:16:47.069Z|00210|reconnect|WARN|ssl:192.168.28.19:46506: connection dropped (Protocol error)
2020-06-16T00:16:47.070Z|00211|reconnect|WARN|ssl:192.168.28.19:46528: connection dropped (Protocol error)
2020-06-16T00:16:47.071Z|00212|reconnect|WARN|ssl:192.168.28.19:46548: connection dropped (Protocol error)
2020-06-16T00:18:09.522Z|00213|jsonrpc|WARN|Dropped 27 log messages in last 88 seconds (most recently, 82 seconds ago) due to excessive rate
2020-06-16T00:18:09.522Z|00214|jsonrpc|WARN|unix#16: send error: Broken pipe
2020-06-16T00:18:09.522Z|00215|reconnect|WARN|unix#16: connection dropped (Broken pipe)
2020-06-16T00:18:09.543Z|00216|jsonrpc|WARN|unix#18: send error: Broken pipe
2020-06-16T00:18:09.543Z|00217|reconnect|WARN|unix#18: connection dropped (Broken pipe)
2020-06-16T00:18:09.996Z|00218|stream_ssl|WARN|Dropped 14 log messages in last 83 seconds (most recently, 83 seconds ago) due to excessive rate
2020-06-16T00:18:09.996Z|00219|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-16T00:18:09.996Z|00220|jsonrpc|WARN|ssl:127.0.0.1:37478: receive error: Protocol error
2020-06-16T00:18:09.997Z|00221|reconnect|WARN|ssl:127.0.0.1:37478: connection dropped (Protocol error)
2020-06-16T00:23:23.111Z|00222|jsonrpc|WARN|unix#21: receive error: Connection reset by peer
2020-06-16T00:23:23.111Z|00223|reconnect|WARN|unix#21: connection dropped (Connection reset by peer)
2020-06-16T00:23:23.616Z|00224|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-16T00:23:23.617Z|00225|jsonrpc|WARN|ssl:127.0.0.1:47900: receive error: Protocol error
2020-06-16T00:23:23.617Z|00226|reconnect|WARN|ssl:127.0.0.1:47900: connection dropped (Protocol error)
2020-06-16T00:29:15.526Z|00227|jsonrpc|WARN|unix#27: receive error: Connection reset by peer
2020-06-16T00:29:15.526Z|00228|reconnect|WARN|unix#27: connection dropped (Connection reset by peer)
2020-06-16T00:29:16.013Z|00229|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-16T00:29:16.013Z|00230|jsonrpc|WARN|ssl:127.0.0.1:51890: receive error: Protocol error
2020-06-16T00:29:16.013Z|00231|reconnect|WARN|ssl:127.0.0.1:51890: connection dropped (Protocol error)
2020-06-16T00:34:27.141Z|00232|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-16T00:34:27.141Z|00233|jsonrpc|WARN|ssl:127.0.0.1:55414: receive error: Protocol error
2020-06-16T00:34:27.141Z|00234|reconnect|WARN|ssl:127.0.0.1:55414: connection dropped (Protocol error)
2020-06-16T00:39:44.295Z|00235|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-16T00:39:44.295Z|00236|jsonrpc|WARN|ssl:127.0.0.1:59008: receive error: Protocol error
2020-06-16T00:39:44.295Z|00237|reconnect|WARN|ssl:127.0.0.1:59008: connection dropped (Protocol error)
2020-06-16T00:44:34.215Z|00238|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-16T00:44:34.215Z|00239|jsonrpc|WARN|ssl:127.0.0.1:34070: receive error: Protocol error
2020-06-16T00:44:34.215Z|00240|reconnect|WARN|ssl:127.0.0.1:34070: connection dropped (Protocol error)

Did I do anything wrong or missing config on this bundle ?. I tried to deploy more than 10 times and have the same error. Please help me on this issue

Thanks,

2

Hello Trinh,

What does the output of juju status look like?

Have you unsealed Vault and provided it with means to provide certificates to the deployment?

3

This is the output of juju status : Ubuntu Pastebin
Yes, I already unseal VAULT, create CA, and reissue certificate.

4

Thank you for responding Trinh, that deployment looks great!

The OVS bridges being marked as DOWN in the ip link output is normal.

If you configure networks in OpenStack including an external network and attach a router you should be able to ping the router external IP.

You can find the IP either by querying OpenStack or by looking at the output of ovn-nbctl show from the unit that is leader for the Northbound DB. Please refer to the usage section of the Open Virtual Network appendix in the OpenStack Charms Deployment Guide.

5

Thanks, yes that is my real problem. I can create router, external network, router interfaces, set gateway for external network, assign floating ip. But I can’t ping floating ip or ssh from external network . I can ping router’s ip good. So with my limited knowledge. I guessed it caused by OVS bridges. But if it’s normal . I will check on other things.

This is some output on ovn. If you have time, please give me some advice to troubleshoot

This is ovn-sbctl show, the instance created on S0042 host

Chassis S0046.maas
hostname: S0046.maas
Encap geneve
ip: “192.168.231.46”
options: {csum=“true”}
Chassis S0042.maas
hostname: S0042.maas
Encap geneve
ip: “192.168.231.42”
options: {csum=“true”}
Port_Binding cr-lrp-e10de10b-6b8c-4dfa-9127-1c1fd89d5597
Port_Binding “686ea284-cf8a-48ae-8e89-b6a2671e57c8”
Chassis S0039.maas
hostname: S0039.maas
Encap geneve
ip: “192.168.231.39”
options: {csum=“true”}
Chassis S0043.maas
hostname: S0043.maas
Encap geneve
ip: “192.168.231.43”
options: {csum=“true”}
Chassis S0048.maas
hostname: S0048.maas
Encap geneve
ip: “192.168.231.48”
options: {csum=“true”}
Chassis S0047.maas
hostname: S0047.maas
Encap geneve
ip: “192.168.231.47”
options: {csum=“true”}
Chassis S0050.maas
hostname: S0050.maas
Encap geneve
ip: “192.168.231.50”
options: {csum=“true”}
Chassis S0045.maas
hostname: S0045.maas
Encap geneve
ip: “192.168.231.45”
options: {csum=“true”}
Chassis S0041.maas
hostname: S0041.maas
Encap geneve
ip: “192.168.231.41”
options: {csum=“true”}
Chassis S0044.maas
hostname: S0044.maas
Encap geneve
ip: “192.168.231.44”
options: {csum=“true”}
Chassis S0040.maas
hostname: S0040.maas
Encap geneve
ip: “192.168.231.40”
options: {csum=“true”}
Chassis S0049.maas
hostname: S0049.maas
Encap geneve
ip: “192.168.231.49”
options: {csum=“true”}

This is ovn-nbctl show. I can ping to 29.166 (router) but I can’t ping to 29.180

switch f3561305-b61c-4ed6-8cea-85cfe02e6033 (neutron-c7a81615-f1f8-4b80-9d51-91c706cbf4f5) (aka NET_29)
port provnet-c7a81615-f1f8-4b80-9d51-91c706cbf4f5
type: localnet
addresses: [“unknown”]
port 9dcf7d2a-2392-45ba-a7a5-aedfbcd7de1d
type: localport
addresses: [“fa:16:3e:91:64:58 192.168.29.1”]
port e10de10b-6b8c-4dfa-9127-1c1fd89d5597
type: router
router-port: lrp-e10de10b-6b8c-4dfa-9127-1c1fd89d5597
switch a250e6a2-f35f-4e65-8232-a36f355df114 (neutron-b37d39eb-5b89-4b48-9a24-f6634e2349f0) (aka NET_100)
port 13e2e4d4-6cba-411f-b564-571e7b0538ca
type: router
router-port: lrp-13e2e4d4-6cba-411f-b564-571e7b0538ca
port 686ea284-cf8a-48ae-8e89-b6a2671e57c8
addresses: [“fa:16:3e:03:f5:e9 192.168.100.91”, “unknown”]
port a811aaf1-0964-4532-ab9f-1f78191f1807
type: localport
addresses: [“fa:16:3e:68:fc:22 192.168.100.1”]
router 6e04baf8-fae2-49f8-b5e2-406ed62357a0 (neutron-bc39bb82-db17-42ef-aa72-8b818b5a7d4a) (aka OUCRU-ROUTER)
port lrp-13e2e4d4-6cba-411f-b564-571e7b0538ca
mac: “fa:16:3e:08:6a:a8”
networks: [“192.168.100.254/24”]
port lrp-e10de10b-6b8c-4dfa-9127-1c1fd89d5597
mac: “fa:16:3e:3c:36:43”
networks: [“192.168.29.166/24”]
gateway chassis: [S0042.maas S0043.maas S0049.maas S0044.maas S0046.maas]
nat 138c04db-4883-4d0f-84a5-26f3d1ae70ef
external ip: “192.168.29.166”
logical ip: “192.168.100.0/24”
type: “snat”
nat 20425b3d-02ce-4852-b9bd-e25af8aea49e
external ip: “192.168.29.180”
logical ip: “192.168.100.91”
type: “dnat_and_snat”

Thanks,

6

If you can ping the external router ip that means that the external networking and bridge configuration should be ok.

So on to instance connectivity, what does the security group rules look like for the security group attached to the instances? Does it allow ICMP and SSH to the instances from the network you are attempting to connect from?

If you look at the instance console, openstack console log show <INSTANCE ID or NAME>, does it appear to have successfully retrieved metadata?

7

Thank you so much for helping me
Based on your comment, I can sort out it. My issue caused by a config vlan id on network interface port on MAAS. So when the packet go out it had a vlan tag. Now, I can connect to the instance.

Thanks again for helping me alot.

1 Like