| airship-irc-bot | <sirishagopigiri> Hi Core Reviewers, Requesting some reviews on the below PSs related to #280 : needs another +2 and WF +1 https://review.opendev.org/c/airship/airshipctl/+/790023 https://review.opendev.org/c/airship/airshipctl/+/790026 https://review.opendev.org/c/airship/airshipctl/+/790033 Thank you in advance! | 13:24 |
|---|---|---|
| airship-irc-bot | <hr858f> Hi, looking for one more +2 and WF: https://review.opendev.org/c/airship/treasuremap/+/793484. Thanks. | 18:59 |
| airship-irc-bot | <mattmceuen> I'm running the airshipctl gate scripts locally, and I'm seeing something funny after the `kubectl-wait-pods-any-ephemeral` phase finishes. I see from the utility pod airshipctl output (and its docker logs) that it waits for ephemeral pods to be up, then returns. But that's the last output I see airshipctl, it seems hung and isn't proceeding to the next phase. Has anyone seen this issue? | 20:33 |
| airship-irc-bot | <sidney.shiba> Hi all, after rebasing with recently merged `treasuremap`, I get stuck while building iso images. See trace below. Anything I have to do to get it working? `Build phase plan: iso` `+ airshipctl plan run iso --debug` `[airshipctl] 2021/06/09 20:29:44 opendev.org/airship/airshipctl/pkg/phase/client.go:292: executing phase: iso-cloud-init-data` `{"Message":"starting generic | 20:37 |
| airship-irc-bot | container","Operation":"GenericContainerStart","Timestamp":"2021-06-09T20:29:53.346887285Z","Type":"GenericContainerEvent"}` `{"Message":"execution of the generic container finished","Operation":"GenericContainerStop","Timestamp":"2021-06-09T20:29:54.751612032Z","Type":"GenericContainerEvent"}` `[airshipctl] 2021/06/09 20:29:54 opendev.org/airship/airshipctl/pkg/phase/client.go:292: executing phase: iso-build-image` `{"Message":"starting | 20:37 |
| airship-irc-bot | generic container","Operation":"GenericContainerStart","Timestamp":"2021-06-09T20:29:54.766276515Z","Type":"GenericContainerEvent"}` | 20:37 |
| airship-irc-bot | <mattmceuen> @sidney.shiba it hangs there forever? | 20:51 |
| airship-irc-bot | <sidney.shiba> yeah, tried three times and same place. | 20:51 |
| airship-irc-bot | <mattmceuen> can you check the resource utilization of memory and disk when it hangs? | 20:52 |
| airship-irc-bot | <mattmceuen> It should take a little time to complete that phase, fwiw | 20:53 |
| airship-irc-bot | <sidney.shiba> There is plenty of memory: `ubuntu@dex-test-vm:~/projects/airshipctl$ free -m` `total used free shared buff/cache available` `Mem: 96676 1008 85694 1 9973 94813` `Swap: 0 0 0` | 20:57 |
| airship-irc-bot | <sidney.shiba> Not sure what /dev/loop is but here is what I get: `ubuntu@dex-test-vm:~/projects/airshipctl$ df` `Filesystem 1K-blocks Used Available Use% Mounted on` `udev 49485048 0 49485048 0% /dev` `tmpfs 9899680 1180 9898500 1% /run` `/dev/vda1 507944172 20927540 487000248 5% /` `tmpfs 49498380 0 49498380 0% /dev/shm` `tmpfs 5120 0 5120 | 20:58 |
| airship-irc-bot | 0% /run/lock` `tmpfs 49498380 0 49498380 0% /sys/fs/cgroup` `/dev/vda15 106858 8993 97866 9% /boot/efi` `/dev/loop0 32896 32896 0 100% /snap/snapd/11841` `/dev/loop1 56832 56832 0 100% /snap/core18/2066` `/dev/loop2 10880 10880 0 100% /snap/kubectl/1976` `/dev/loop4 9856 9856 0 100% /snap/helm/336` `/dev/loop5 11136 | 20:58 |
| airship-irc-bot | 11136 0 100% /snap/helm/341` `/dev/loop6 32896 32896 0 100% /snap/snapd/12057` `/dev/loop7 92800 92800 0 100% /snap/go/7696` `/dev/loop8 92800 92800 0 100% /snap/go/7736` `tmpfs 9899676 0 9899676 0% /run/user/1000` | 20:58 |
| airship-irc-bot | <sidney.shiba> It seems that the difference is that it is running now `airshipctl plan run iso` while previous one was not. | 21:02 |
| airship-irc-bot | <mattmceuen> it might be worth running the phases in that iso plan by hand and see if you learn more? | 21:04 |
| airship-irc-bot | <mattmceuen> I wouldn't think that the plan would cause your problem though, since it looks like the plan is invoking a phase that is at least attempting to start a generic container | 21:05 |
| airship-irc-bot | <mattmceuen> For my part, the `kubectl-wait-pods-any-ephemeral` phase doesn't complete when I re-run it by itself either: | 21:15 |
| airship-irc-bot | <mattmceuen> ```madgin@piseag-01:~$ airshipctl phase run kubectl-wait-pods-any-ephemeral {"Message":"starting generic container","Operation":"GenericContainerStart","Timestamp":"2021-06-09T16:08:28.567785119-05:00","Type":"GenericContainerEvent"} [airshipctl] 2021/06/09 16:08:28 Starting container with image: 'localhost/toolbox', cmd: '[]' [airshipctl] 2021/06/09 21:08:30 Filtering input bundle by Group: , Version: , Kind: + N=0 + | 21:16 |
| airship-irc-bot | MAX_RETRY=30 + DELAY=60 + '[' 0 -ge 30 ] + kubectl --context ephemeral-cluster --request-timeout 10s get pods --all-namespaces -o name + wc -l + '[' 7 -ge 1 ] + kubectl --context ephemeral-cluster --request-timeout 10s get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-66bff467f8-dtr68 0/1 Pending 0 8m41s kube-system | 21:16 |
| airship-irc-bot | coredns-66bff467f8-pjctm 0/1 Pending 0 8m41s kube-system etcd-ephemeral 1/1 Running 0 8m51s kube-system kube-apiserver-ephemeral 1/1 Running 0 8m51s kube-system kube-controller-manager-ephemeral 1/1 Running 0 8m51s kube-system kube-proxy-8lh24 1/1 Running 0 8m41s kube-system | 21:16 |
| airship-irc-bot | kube-scheduler-ephemeral 1/1 Running 0 8m51s + break + '[' 0 -ge 30 ] (never returns)``` | 21:16 |
| airship-irc-bot | <mattmceuen> I created a github issue for the problem I'm seeing; not sure why I'd be the only one(?) seeing it, but it seems bad: https://github.com/airshipit/airshipctl/issues/577 | 21:27 |
| airship-irc-bot | <ao129q> may I have reviews here please https://review.opendev.org/c/airship/airshipctl/+/791051 | 21:33 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!