I was trying my “microsample” charm (rev: 9) on google cloud. The charm starts a simple flask application and tries to bind it to a specific ip.
This is the code where is is set and the critical path is here below:
# Get address + port
ADDRESS=$(unit-get public-address)
PORT=$(config-get port)
# set the microsample snap configuration
snap set microsample port=${PORT}
snap set microsample address=${ADDRESS}
So, the problem is now that the service tries to start up on the public ip, which is the real external IP in google-cloud, but which is not available on the deployed host.
This makes the service not able to start.
I have to ask here, what is the correct way to make my service properly set the address to listen to for ?
What address should I use to be able to:
Reach it from internet (after a expose)
To be able to relate it to a reverseproxy deployed with juju?
Using the “public-ip” is clearly the wrong way, but how?
(Update: I discovered differences between GCP and AWS here in relation to “private addresses” described below)
I also see that on GCP, the hooks tool command returns the IP address for the FAN network when requesting the private-address:
Google Cloud Platform
juju run --unit microsample/1 'unit-get private-address'
252.0.64.1
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: ens4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1460 qdisc fq_codel state UP group default qlen 1000
inet 10.166.0.4/32 scope global dynamic ens4
valid_lft 2855sec preferred_lft 2855sec
3: fan-252: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
inet 252.0.64.1/8 scope global fan-252
valid_lft forever preferred_lft forever
Connection to 35.228.109.158 closed.
Where on AWS - the address returned is an private address in the 172.x.x.x range (Class-B).
juju run --unit microsample/1 'unit-get private-address'
172.31.23.235
… which will also properly allow me to reach the service after a “juju expose”. (This is not possible with GCP currently)
There is clearly a difference here which makes my above mentioned charm not work on GCP. Is this a bug?
unit-get private-address is deprecated in favour of network-get
network-get is space and cross model relations aware and provides a more complete view of the model. I am not sure without trying it whether the use of network-get has different behaviour to what which you are seeing with unit-get private-address, but network-get is where any work would be done to fix the issue.
juju help network-get
network-get returns the network config for a given binding name. By default
it returns the list of interfaces and associated addresses in the space for
the binding, as well as the ingress address for the binding. If defined, any
egress subnets are also returned.
If one of the following flags are specified, just that value is returned.
If more than one flag is specified, a map of values is returned.
--bind-address: the address the local unit should listen on to serve connections, as well
as the address that should be advertised to its peers.
--ingress-address: the address the local unit should advertise as being used for incoming connections.
--egress-subnets: subnets (in CIDR notation) from which traffic on this relation will originate.
I’ll now deploy my microsample, that I’ve changed now to try to use the “private-address” instead which I hope would NOT return the fan network. But I will also try to to use “network-get --bind-address” to see if this would also return the fan-network.
juju deploy cs:~erik-lonroth/charm-microsample-10
Located charm “cs:~erik-lonroth/charm-microsample-10”.
Deploying charm “cs:~erik-lonroth/charm-microsample-10”.
My conclusion is that the FAN network is used as the binding address on GCP which is not at all what happens on AWS, which delivers the 172-class-B network which also makes my service accessible when performing “juju expose microsample”.
This must be something wrong? At least, I’m not sure how to retrieve the tennant/project private 10.x.x.x address. I need this to get the service to properly be accessible from the public-address nat:ed to the private 10 address. This is a default behavior on AWS…
Spaces are not yet supported on Google cloud unfortunately.
The fact that subnet discovery is supported means that it should be doable without massive effort to get space support implemented.
In providers which do support spaces, I think the fan network is explicitly filtered out from the bind addresses, but @manadart could weigh in here to correct any misconceptions I have.
So think the issues you are hitting should be resolved once Google provider supports spaces. It’s good there’s a workaround in the meantime.
It’s a bit unfortunate that the current default behavior on gcp would cause charms to break unless modifying the model. Also the bug remains since 2018…
I like gcp really, so if I could contribute to get this fixed upstream I would be glad to assist.
That said, I still don’t know exactly how to get the 10.x.x.x private address, but I’ll figure it out.
Fan addresses are not filtered out in the results of network-get, but we do order the results consistently to ensure that they are listed after their underlay address.
This means that for VMs bound/contrained to a space (subnets) with overlays, they will correctly represent their connectivity to the underlay network. Containers bound/constrained to Fan subnets will not see the underlay, and correctly return the Fan address.
@wallyworld I misremembered my prior GCE recon regarding spaces. What I said in our conversation this morning wasn’t quite right.
As @erik-lonroth demonstrated, subnet discovery is implemented for GCE. So we can allow space definition simply by turning on the appropriate flag.
The work required is actually in the networking config passed at instance creation. Based on a browse through the compute API code, we should be able to support multi-NIC for bindings/constraints by passing a list of compute.NetworkInterface containing values for Subnetwork. It doesn’t look too* hard.
