Flaky CI Tests

Description

In this document, I will try to keep up and update flaky Tests/reasons I will find.

Tests

cmd/juju/status/status_internal_test/TestBranchesOutputTabular [1]

cmd/juju/status/status_internal_test/TestBranchesOutputNonTabular [2]

debugging for [1] [2]:

The status returned from FullStatus occasionally does not return a branches output.
go test -v -check.f TestBranchesOutputTabular -count=30 -check.v

after little debugging and logging:

[LOG] 0:00.842 ERROR juju.cmd.juju.status active branches: "bla"
[LOG] 0:00.842 ERROR juju.cmd.juju.status all Branches: map[]
[LOG] 0:00.842 DEBUG juju.api RPC connection died
[LOG] 0:00.842 DEBUG juju.rpc error closing codec: write tcp 127.0.0.1:38807->127.0.0.1:43316: write: connection reset by peer
[LOG] 0:00.842 DEBUG juju.apiserver [2] user-admin API connection terminated after 25.537732ms
status_internal_test.go:6107:
    c.Assert(strings.Contains(string(stdout), "bla*"), jc.IsTrue)
... obtained bool = false

after debugging it seems that sometimes checking for branches returns 0, even though branches do exist.

status, err := c.getStatus() <- this sometimes does not properly returned processed branches.

status.Branches() -> empty

Update [1]

After talking to @simonrichardson we decided to skip that test for now and discuss this further in the frankfurt sprint.

We assume that the sometimes model cache is slower than the access in the status output test.
We assume that a call to addbranches -> state -> fullstatus -> read from state (which reads from modelcache) is sometimes slower and therefore leads to this problem.