Juju bootstrap inside LXD fails at "Attempting to connect to..."

juju
1

Hello,

I’m trying to run juju bootstrap inside LXD and this happens:

$ juju bootstrap localhost controller
Resolved LXD host address on bridge lxdbr0: https://10.73.222.1:8443
updating credential store
Adding contents of "/home/ubuntu/.local/share/juju/ssh/juju_id_rsa.pub" to authorized-keys
Creating Juju controller "controller" on localhost/localhost
Loading image metadata
Looking for packaged Juju agent version 2.7.3 for amd64
Starting new instance for initial controller
To configure your system to better support LXD containers, please see: https://github.com/lxc/lxd/blob/master/doc/production-setup.md
Launching controller instance(s) on localhost/localhost...
 - juju-1596bf-0 (arch=amd64)
Installing Juju agent on bootstrap instance
Fetching Juju GUI 2.15.0
Waiting for address
Attempting to connect to 10.73.222.120:22
ERROR failed to bootstrap model: waited for 20m0s without being able to connect: /var/lib/juju/nonce.txt does not exist
$ cat deploy.log 
20:48:11 ERROR juju.cmd.juju.commands bootstrap.go:778 failed to bootstrap model: waited for 20m0s without being able to connect: /var/lib/juju/nonce.txt does not exist

I thought that was a network issue but I ran it again and during the execution, I ssh into the instance and created an empty file called /var/lib/juju/nonce.txt. The error changed to:

13:03:07 DEBUG juju.provider.common bootstrap.go:591 connection attempt for 10.73.222.22 failed: /var/lib/juju/nonce.txt contents do not match machine nonce

Then, I added “user-admin:bootstrap” to the file and the execution continued a little more and hanged again:

13:10:47 INFO  cmd bootstrap.go:360 Connected to 10.73.222.22
13:10:47 INFO  juju.cloudconfig userdatacfg_unix.go:565 Fetching agent: curl -sSfw 'agent binaries from %{url_effective} downloaded: HTTP %{http_code}; time %{time_total}s; size %{size_download} bytes; speed %{speed_download} bytes/s ' --retry 10 -o $bin/tools.tar.gz <[https://streams.canonical.com/juju/tools/agent/2.7.3/juju-2.7.3-ubuntu-amd64.tgz]>
13:10:47 INFO  cmd bootstrap.go:430 Running machine configuration script...

What I’m doing wrong? I’m using juju snap version 2.7.3.

2

Hi @pfsmorigo, can you run lxc info and paste back the results.

1 Like
3 
$ lxc info
config: {}
api_extensions:
- storage_zfs_remove_snapshots
- container_host_shutdown_timeout
- container_stop_priority
- container_syscall_filtering
- auth_pki
- container_last_used_at
- etag
- patch
- usb_devices
- https_allowed_credentials
- image_compression_algorithm
- directory_manipulation
- container_cpu_time
- storage_zfs_use_refquota
- storage_lvm_mount_options
- network
- profile_usedby
- container_push
- container_exec_recording
- certificate_update
- container_exec_signal_handling
- gpu_devices
- container_image_properties
- migration_progress
- id_map
- network_firewall_filtering
- network_routes
- storage
- file_delete
- file_append
- network_dhcp_expiry
- storage_lvm_vg_rename
- storage_lvm_thinpool_rename
- network_vlan
- image_create_aliases
- container_stateless_copy
- container_only_migration
- storage_zfs_clone_copy
- unix_device_rename
- storage_lvm_use_thinpool
- storage_rsync_bwlimit
- network_vxlan_interface
- storage_btrfs_mount_options
- entity_description
- image_force_refresh
- storage_lvm_lv_resizing
- id_map_base
- file_symlinks
- container_push_target
- network_vlan_physical
- storage_images_delete
- container_edit_metadata
- container_snapshot_stateful_migration
- storage_driver_ceph
- storage_ceph_user_name
- resource_limits
- storage_volatile_initial_source
- storage_ceph_force_osd_reuse
- storage_block_filesystem_btrfs
- resources
- kernel_limits
- storage_api_volume_rename
- macaroon_authentication
- network_sriov
- console
- restrict_devlxd
- migration_pre_copy
- infiniband
- maas_network
- devlxd_events
- proxy
- network_dhcp_gateway
- file_get_symlink
- network_leases
- unix_device_hotplug
- storage_api_local_volume_handling
- operation_description
- clustering
- event_lifecycle
- storage_api_remote_volume_handling
- nvidia_runtime
- candid_authentication
- candid_config
- candid_config_key
- usb_optional_vendorid
api_status: stable
api_version: "1.0"
auth: trusted
public: false
auth_methods:
- tls
environment:
  addresses: []
  architectures:
  - x86_64
  - i686
  certificate: |
    -----BEGIN CERTIFICATE-----
    MIIFXDCCA0SgAwIBAgIRAIrY90Xy8kZx5mQpbTS4onIwDQYJKoZIhvcNAQELBQAw
    NzEcMBoGA1UEChMTbGludXhjb250YWluZXJzLm9yZzEXMBUGA1UEAwwOcm9vdEBl
    c21weXRob24wHhcNMjAwMjEwMTkxMzAyWhcNMzAwMjA3MTkxMzAyWjA3MRwwGgYD
    VQQKExNsaW51eGNvbnRhaW5lcnMub3JnMRcwFQYDVQQDDA5yb290QGVzbXB5dGhv
    bjCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAMZnITDhCdN0OlOGKl5U
    73LTmcdVGWaMRoXpNTSkAVn3oIpH/tqjW1G0Gw5vnGOsojuxEFYsp0C9XEGDtRV+
    wIXY4oG4A65pW+XHv5g09lx+layHHv3lP7gHw9EGH1klaz/azk2jgrclTrpnZAW4
    SdZMJ+jnIYDXoyJkgHApPP8EZ2SHqgVuP1ApP0y7wAwzg3I8pxJCRDA16RkXzAbG
    0QyV4x9t1x9Pi8PNK+rqJRNoYxJqF4g0AbLuvxlm/cH0UFPvHLJm1j+cR+bEMEd+
    aKGOvbNxqg63zf9uz9P6JBI8YoW082J6PbB06BLAks/+YGIcXN++loL8NRuE+Nwj
    vLrTjDGNtopPRezn3Gj+s+9QGaBzSxu5mE0KN7Aoc3uiYE5oll0FnR9DugaFr6nl
    dsbcSjblNFsdPZ57EUYo7XcZIB6UT/BRZFrY7io3Sivlyys4i6nKgS9WgHLMJSyG
    83pFSFBuDnvEBneuTVXg/zxXAyVctvFCw8LpE+86rWBFNtsA2/oHy6tsdFOZN+5+
    WMjw/s9OGIsbK+sX+dQ09444vYTSohulS4bU0cXB9CmgNsR6d9/vT2wHBw1c/wYH
    G9oR/hlmU5oMSS6f8rcFFc+ZOHxzTZ2KdLb7LMzcenoHZmJcfNJ1/lxJaK2MWAt2
    bjMulwIXexxVf6r5yZfSRzoVAgMBAAGjYzBhMA4GA1UdDwEB/wQEAwIFoDATBgNV
    HSUEDDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMCwGA1UdEQQlMCOCCWVzbXB5
    dGhvbocECuzNT4cQ/UIIQznReN8CFj7//lguZDANBgkqhkiG9w0BAQsFAAOCAgEA
    BnzkdcTOvBsJWKfZ9H/XdrAzc7knJyp9ONuLoa+R+auYF7w1Q0kJZWaQ5fpUA+xM
    IQ0BduswcXGBx6T6jbsa4cVrmL64P5HNBptmlCxpuSe+Q8Ao36gsw0ccowORZyBE
    XlqovD+mLi4lQQLEg2uj+Oa8Jca5NW1D5NS+QejYX3Wr4q8U7lQadV2dyCOpbHZE
    LSWPD4VLS8n1X3HrYmKBIXD8kyKZEFnMy/FTUgNxfamHf56RVjpsAhK/mPvSVQRl
    tbhBuKa7hcs5EsuvkGNgRUaS9vqsmb+mHS6kE945zKuykXjugxmoBU5LXA9nf+HJ
    99n3KLrzWbDAOudLESC5Nrg5ByY/EOl0RjJr9FfKMXppw99hJhpGtbwhAGt3OBkd
    oRU6D4oF4SuYJ3f/xAKAaGLDCVacjcvJxPWbAho2m5YP8M8zOLzXQl4YYxAxSkm1
    YDZIN1/4ToNbzqubaQamCi+XMYJUaoybecsuRlh4/y1bX/gn5T8ZuNFjUGhyTlN0
    gOKMFRJIWvp+0LDherVe/Duiek6gsXpbcxrpyOZCKsWLDIxXHlKNEsA1jQhf8OTS
    6Qr+KSOMcSfM474DMGTU/ZPIGdKeGfLrMKttvEzslWlvRycojFBslbnIEzh6pV1Y
    NXPWDsBLYjdngyMjEulBIVq4KxDQxjrJOIpDMn0kcjY=
    -----END CERTIFICATE-----
  certificate_fingerprint: 76e7584c977a597a30ea66f6abc29c5275e4f0660cde63317a48003b94b0b13f
  driver: lxc
  driver_version: 3.0.3
  kernel: Linux
  kernel_architecture: x86_64
  kernel_version: 5.4.0-14-generic
  server: lxd
  server_pid: 2599
  server_version: 3.0.3
  storage: ""
  storage_version: ""
  server_clustered: false
  server_name: server1
  project: ""
4

Are you using a custom image for ‘ubuntu’ ? Offhand this looks like it isn’t properly running cloud-init. The other thing to look at is to start with a juju bootstrap and then from inside the container it started, look at /var/log/cloud-init-output.log. Usually that contains some hint as to why the instance is failing to initialize. Sometimes it can’t get to the Ubuntu Archive to install necessary packages, or sometimes the internal networking isn’t correct (no route to the outside world).

1 Like