| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
|---|---|---|
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 00:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 00:10 |
| *** med_ has joined #tripleo | 00:16 | |
| *** agopi|off has joined #tripleo | 00:21 | |
| *** mcornea has quit IRC | 00:27 | |
| *** huynq has joined #tripleo | 01:04 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 01:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 01:10 |
| *** hamzy has joined #tripleo | 01:12 | |
| *** mrsoul has joined #tripleo | 01:19 | |
| *** mschuppert has quit IRC | 01:21 | |
| *** moshele has joined #tripleo | 01:26 | |
| *** med_ has quit IRC | 01:30 | |
| *** gfidente has joined #tripleo | 01:56 | |
| *** jbadiapa has quit IRC | 01:56 | |
| gfidente | weshay panda not sure if you guys are still online but I can't create user stories in taiga | 01:57 |
| *** jbadiapa has joined #tripleo | 01:58 | |
| weshay | gfidente, k.. let me see if I can help | 02:00 |
| gfidente | weshay hey :D | 02:00 |
| gfidente | I wanted to edit the epic description as well but I can't, I can only add comments to that | 02:01 |
| weshay | gfidente, try now | 02:02 |
| *** huynq has quit IRC | 02:03 | |
| *** pvc has joined #tripleo | 02:03 | |
| pvc | hi guys | 02:03 |
| pvc | anyone experience the boot mode is set to bios. Failure status code 404 | 02:03 |
| gfidente | weshay++ | 02:04 |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 02:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 02:10 |
| *** moshele has quit IRC | 02:30 | |
| pvc | hi guys | 02:31 |
| *** shyamb has joined #tripleo | 02:36 | |
| *** med_ has joined #tripleo | 02:50 | |
| *** shyam89 has joined #tripleo | 02:55 | |
| *** shyamb has quit IRC | 02:59 | |
| *** sai_p has joined #tripleo | 03:03 | |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 03:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 03:10 |
| *** jaganathan has joined #tripleo | 03:20 | |
| *** shyam89 has quit IRC | 03:25 | |
| *** med_ has quit IRC | 03:37 | |
| *** shyam89 has joined #tripleo | 03:37 | |
| *** med_ has joined #tripleo | 03:40 | |
| shyam89 | Hi | 03:41 |
| *** ykarel has joined #tripleo | 03:47 | |
| pvc | guys | 03:50 |
| pvc | anyone | 03:50 |
| pvc | mandre? | 03:50 |
| pvc | ykarel | 03:50 |
| shyam89 | Hi | 03:51 |
| shyam89 | Pls confirm if this is bug in tripleo | 03:51 |
| shyam89 | http://paste.openstack.org/show/731323/ | 03:51 |
| shyam89 | I am getting this issue on overcloud deploy | 03:52 |
| *** huynq has joined #tripleo | 03:54 | |
| shyam89 | https://bugs.launchpad.net/tripleo/+bug/1795781 | 04:00 |
| openstack | Launchpad bug 1795781 in tripleo "Overcloud deployment failing at early stage: KeyError: 'passwords' " [Undecided,New] | 04:00 |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 04:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 04:10 |
| *** shyam89 has quit IRC | 04:11 | |
| ykarel | pvc, hi | 04:19 |
| stevebaker | quit | 04:20 |
| pvc | hi ykarel you know this error the boot mode is bios then Failure status cdoe 404 | 04:20 |
| stevebaker | ha | 04:20 |
| pvc | when running the instrospection | 04:20 |
| *** med_ has quit IRC | 04:21 | |
| ykarel | pvc, no, as per status code 404, something should be not Found | 04:21 |
| ykarel | what's the exact error | 04:21 |
| *** AJaeger has left #tripleo | 04:21 | |
| pvc | wait ykarel. im running again the instrospection | 04:23 |
| *** gkadam has joined #tripleo | 04:32 | |
| *** gkadam has quit IRC | 04:32 | |
| *** ramishra has joined #tripleo | 04:33 | |
| *** Petersingh has joined #tripleo | 04:40 | |
| *** jaganathan has quit IRC | 04:44 | |
| *** ykarel_ has joined #tripleo | 04:45 | |
| *** chandankumar has quit IRC | 04:46 | |
| *** boazel_ has joined #tripleo | 04:47 | |
| *** chandankumar has joined #tripleo | 04:47 | |
| *** ykarel has quit IRC | 04:48 | |
| *** boazel has quit IRC | 04:49 | |
| *** shyamb has joined #tripleo | 04:53 | |
| *** zaneb has quit IRC | 05:06 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 05:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 05:10 |
| *** jaganathan has joined #tripleo | 05:10 | |
| *** ykarel_ is now known as ykarel | 05:13 | |
| pvc | ykarel WARNING ironic_python_agent.ironic_api_client Failure status code 404 | 05:14 |
| ykarel | shyamb, yes i have seen the issue with missing passwords | 05:14 |
| ykarel | pvc, no idea, but this seems to be WARNING, not sure if this is causing failures in ur case | 05:18 |
| shyamb | ykarel: My overcloud deployment is stopping at this point | 05:22 |
| ykarel | shyamb, yes that's bad, iirc after removing swift container that succeeded, but that's not a workaround(and don't know how it impacts existing deployment) | 05:24 |
| shyamb | ykarel: ok | 05:25 |
| shyamb | You mean remove swift container and redeploy it? or we need to remove swift forever | 05:26 |
| pvc | ykarel LookupNode Error: Error getting configuration from Ironic: Could not look up node oinf | 05:26 |
| ykarel | pvc, no idea, u can get some help on #openstack-ironic | 05:29 |
| ykarel | shyamb, yes remove swift container and redeploy, if u have just test environment u can try that | 05:29 |
| ykarel | otherwise wait for other guys they may have some idea on it as this bug is reproducable | 05:30 |
| pvc | hi guys | 05:30 |
| pvc | dnsmasq.conf dnsmasq.d/ dnsmasq-ironic.conf | 05:30 |
| *** jfrancoa has joined #tripleo | 05:31 | |
| shyamb | ok | 05:32 |
| shyamb | ykarel: this needs to be done on controller nodes? | 05:33 |
| pvc | ykarel how can i change the introspection boot from httpboot to tfptboot? | 05:33 |
| shyamb | like docker stop and docker container rm? | 05:33 |
| ykarel | shyamb, no, it's need to be done on undercloud node, source stackrc; openstack container list; openstack container delete <container name> | 05:34 |
| shyamb | oh ok | 05:34 |
| ykarel | shyamb, u have test environment, right? | 05:35 |
| shyamb | yes | 05:35 |
| ykarel | okk then u can try | 05:35 |
| *** udesale has joined #tripleo | 05:41 | |
| *** ratailor has joined #tripleo | 05:43 | |
| *** zaneb has joined #tripleo | 05:46 | |
| pvc | ykarel | 05:46 |
| pvc | how can i use tftp instead of httpboot for introspection | 05:46 |
| ykarel | no idea | 05:49 |
| ykarel | pvc, ask on #openstack-ironic | 05:49 |
| ykarel | somewhere there might have answer to it | 05:50 |
| *** sl0w has quit IRC | 05:51 | |
| *** shyamb has quit IRC | 05:51 | |
| *** quiquell|off is now known as quiquell | 05:53 | |
| quiquell | Good morning | 05:54 |
| *** yprokule has joined #tripleo | 05:57 | |
| *** ykarel is now known as ykarel|afk | 05:57 | |
| *** moshele has joined #tripleo | 06:00 | |
| *** iranzo has joined #tripleo | 06:00 | |
| quiquell | jaosorior: I see timeouts now :-( | 06:00 |
| bandini | simple backport https://review.openstack.org/#/c/601077/ to pike | 06:04 |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 06:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 06:10 |
| *** shyamb has joined #tripleo | 06:21 | |
| *** hjensas has quit IRC | 06:23 | |
| *** agurenko has joined #tripleo | 06:23 | |
| *** aufi_ has joined #tripleo | 06:24 | |
| *** ksambor has joined #tripleo | 06:25 | |
| *** pvc has quit IRC | 06:26 | |
| *** maufart__ has joined #tripleo | 06:27 | |
| *** ykarel_ has joined #tripleo | 06:27 | |
| *** aufi_ has quit IRC | 06:29 | |
| *** ykarel|afk has quit IRC | 06:29 | |
| Tengu | hello there :) | 06:29 |
| *** saneax has quit IRC | 06:30 | |
| *** maufart__ has quit IRC | 06:33 | |
| *** holser_ has joined #tripleo | 06:35 | |
| *** aufi has joined #tripleo | 06:35 | |
| *** holser_ has quit IRC | 06:39 | |
| *** aufi_ has joined #tripleo | 06:41 | |
| *** aufi has quit IRC | 06:44 | |
| *** ksambor has quit IRC | 06:49 | |
| *** pcaruana has joined #tripleo | 06:51 | |
| *** rdopiera has joined #tripleo | 06:56 | |
| *** moguimar has quit IRC | 06:57 | |
| *** hjensas has joined #tripleo | 07:00 | |
| *** tosky has joined #tripleo | 07:03 | |
| *** moguimar has joined #tripleo | 07:06 | |
| *** aufi_ has quit IRC | 07:08 | |
| *** aufi has joined #tripleo | 07:08 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 07:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 07:10 |
| *** paramite|clone has joined #tripleo | 07:10 | |
| *** ykarel__ has joined #tripleo | 07:13 | |
| *** bugzy_ has joined #tripleo | 07:13 | |
| *** aedc has joined #tripleo | 07:15 | |
| *** holser_ has joined #tripleo | 07:15 | |
| *** ykarel_ has quit IRC | 07:15 | |
| *** bogdando has joined #tripleo | 07:16 | |
| *** ksambor has joined #tripleo | 07:16 | |
| *** pvc_ has joined #tripleo | 07:17 | |
| pvc_ | hi anyone? | 07:17 |
| pvc_ | PXE boot failed. No configuration found for MAC ec:f4:bb:ed:6f:e4 | 07:17 |
| *** safchain_ has joined #tripleo | 07:17 | |
| sshnaidm | d0ugal, hey, maybe you can help with this error in overcloud deployment - The WebSocket timed out before the Workflow completed. https://logs.rdoproject.org/96/15896/35/check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master-vexxhost/2b2cbf3/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 07:18 |
| sshnaidm | d0ugal, this is job that runs on external cloud, the same job on rdo cloud passes | 07:18 |
| *** moguimar has quit IRC | 07:18 | |
| sshnaidm | d0ugal, I suppose it could be something related to performance of cloud maybe, but I don't find something in mistral logs about the error | 07:19 |
| *** jaganathan has quit IRC | 07:19 | |
| *** paramite has quit IRC | 07:19 | |
| *** jroll has quit IRC | 07:19 | |
| *** Tengu has quit IRC | 07:19 | |
| *** bugzy has quit IRC | 07:19 | |
| *** dmellado has quit IRC | 07:19 | |
| *** ajo has quit IRC | 07:19 | |
| *** safchain has quit IRC | 07:19 | |
| *** KingJ has quit IRC | 07:19 | |
| *** odyssey4me has quit IRC | 07:19 | |
| *** safchain_ is now known as safchain | 07:20 | |
| *** ykarel__ is now known as ykarel | 07:20 | |
| *** moguimar has joined #tripleo | 07:25 | |
| *** amoralej|off is now known as amoralej | 07:27 | |
| *** aedc has quit IRC | 07:31 | |
| *** jroll has joined #tripleo | 07:32 | |
| *** Tengu has joined #tripleo | 07:32 | |
| *** dmellado has joined #tripleo | 07:32 | |
| *** ajo has joined #tripleo | 07:32 | |
| *** KingJ has joined #tripleo | 07:32 | |
| *** odyssey4me has joined #tripleo | 07:32 | |
| *** ksambor has quit IRC | 07:35 | |
| *** jpich has joined #tripleo | 07:35 | |
| *** skramaja has joined #tripleo | 07:35 | |
| *** Director100b has quit IRC | 07:37 | |
| *** sai_p has quit IRC | 07:42 | |
| *** ykarel has quit IRC | 07:43 | |
| *** gkadam has joined #tripleo | 07:44 | |
| *** ykarel has joined #tripleo | 07:45 | |
| *** ksambor has joined #tripleo | 07:46 | |
| *** shyamb has quit IRC | 07:46 | |
| *** Petersingh is now known as Petersingh|lunch | 07:50 | |
| *** jpena|off is now known as jpena | 07:51 | |
| *** rcernin has quit IRC | 07:55 | |
| *** ykarel_ has joined #tripleo | 08:07 | |
| *** ykarel_ has quit IRC | 08:07 | |
| jaosorior | quiquell: :( | 08:08 |
| *** ykarel_ has joined #tripleo | 08:09 | |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 08:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 08:10 |
| *** shardy has joined #tripleo | 08:11 | |
| pvc_ | hi shardy may i know if there is a caching on introspection? | 08:13 |
| *** ykarel has quit IRC | 08:13 | |
| *** jroll has quit IRC | 08:13 | |
| *** Tengu has quit IRC | 08:13 | |
| *** dmellado has quit IRC | 08:13 | |
| *** ajo has quit IRC | 08:13 | |
| *** KingJ has quit IRC | 08:13 | |
| *** odyssey4me has quit IRC | 08:13 | |
| *** aufi has quit IRC | 08:16 | |
| *** aufi has joined #tripleo | 08:16 | |
| *** jtomasek_ has joined #tripleo | 08:21 | |
| *** jtomasek has quit IRC | 08:23 | |
| *** jroll has joined #tripleo | 08:27 | |
| *** Tengu has joined #tripleo | 08:27 | |
| *** dmellado has joined #tripleo | 08:27 | |
| *** ajo has joined #tripleo | 08:27 | |
| *** KingJ has joined #tripleo | 08:27 | |
| *** odyssey4me has joined #tripleo | 08:27 | |
| Tengu | jaosorior: ohhh think your https://review.openstack.org/#/c/588291/2/docker/services/haproxy.yaml will also correct some bad things with selinux when using podman :). | 08:31 |
| *** Petersingh|lunch is now known as Petersingh | 08:32 | |
| therve | bogdando: FWIW heat doesn't drop tasks on the floor on SIGHUP | 08:32 |
| therve | Quite the opposite actually | 08:33 |
| therve | There may be bugs around the handling, but not that kind of bug | 08:33 |
| jaosorior | Tengu: nicenice | 08:33 |
| *** derekh has joined #tripleo | 08:33 | |
| bogdando | therve: hm, local testing shows it does :) | 08:34 |
| therve | bogdando: OK, what kind of testing? | 08:34 |
| bogdando | did you see my comment https://bugs.launchpad.net/tripleo/+bug/1795411/comments/13 ? | 08:34 |
| openstack | Launchpad bug 1795411 in tripleo "The scenario001(and 2)-multinode-oooq-container job is failing during the overcloud deploy for "[Run tripleo-container-image-prepare]" " [Critical,Fix released] - Assigned to Marios Andreou (marios-b) | 08:34 |
| *** jtomasek has joined #tripleo | 08:34 | |
| Tengu | jaosorior: so this solves the "update for TLS" issue? | 08:35 |
| *** jtomasek_ has quit IRC | 08:36 | |
| therve | bogdando: yeah it doesn't tell me much :) | 08:36 |
| Tengu | jaosorior: btw, adding right now support for standalone in tripleo-lab. It's interesting :). | 08:36 |
| Tengu | guess it should be up today. | 08:36 |
| *** gfidente has quit IRC | 08:36 | |
| jaosorior | cool! | 08:36 |
| bogdando | therve: well, I can share env to you :) | 08:36 |
| therve | bogdando: Sure | 08:37 |
| *** holser_ has quit IRC | 08:37 | |
| dalvarez | hi folks can anybody take a look at https://review.openstack.org/#/c/568858/ ? already has 2 +2s | 08:40 |
| dalvarez | thanks a lot | 08:40 |
| dalvarez | jfrancoa: ^ | 08:41 |
| *** holser_ has joined #tripleo | 08:42 | |
| *** shyamb has joined #tripleo | 08:42 | |
| jfrancoa | dalvarez: sure, I'll have a look at it | 08:45 |
| *** Petersingh is now known as Petersingh|afk | 08:47 | |
| dalvarez | jfrancoa: thanks! | 08:49 |
| *** jroll has quit IRC | 08:51 | |
| *** Tengu has quit IRC | 08:51 | |
| *** dmellado has quit IRC | 08:51 | |
| *** ajo has quit IRC | 08:51 | |
| *** KingJ has quit IRC | 08:51 | |
| *** odyssey4me has quit IRC | 08:51 | |
| *** ykarel_ is now known as ykarel|lunch | 08:55 | |
| *** derekh has quit IRC | 08:58 | |
| *** saneax has joined #tripleo | 08:58 | |
| *** jtomasek has quit IRC | 08:59 | |
| *** jroll has joined #tripleo | 09:05 | |
| *** Tengu has joined #tripleo | 09:05 | |
| *** dmellado has joined #tripleo | 09:05 | |
| *** ajo has joined #tripleo | 09:05 | |
| *** KingJ has joined #tripleo | 09:05 | |
| *** odyssey4me has joined #tripleo | 09:05 | |
| *** Petersingh|afk is now known as Petersingh | 09:06 | |
| *** aedc has joined #tripleo | 09:07 | |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 09:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 09:10 |
| *** salmankhan has joined #tripleo | 09:11 | |
| Tengu | jaosorior: "TASK [standalone : deploy standalone]" :) | 09:13 |
| Tengu | let's see. | 09:13 |
| *** jtomasek has joined #tripleo | 09:15 | |
| shardy | Morning all, need a second +2A on https://review.openstack.org/#/c/605009 if anyone has a moment | 09:19 |
| Tengu | wow. apparently, things are working so far. standalone create_complete. Now it's time for ansible to take over I guess. | 09:21 |
| sri_ | shardy, Morning, quick question where can we found the default passwords for overcloud ? | 09:25 |
| Tengu | sri_: in overcloud.rc? | 09:26 |
| sri_ | Tengu, I was looking for mariadb and rabbitmq passwords | 09:26 |
| Tengu | oh. they are generated | 09:27 |
| Tengu | might be in hieradata. | 09:28 |
| sri_ | Tengu, I was looking here (https://access.redhat.com/solutions/2839761) " mistral environment-get overcloud" but no luck, maybe that's little bit old | 09:31 |
| Tengu | I think there were some changes for that indeed. not sure about the status, but I saw things in order to encrypt and/or take out of the mistral environment. jaosorior maybe? | 09:32 |
| shardy | sri_: there should be a file written by tripleoclient with them, also you can see them via openstack stack environment show overcloud | 09:36 |
| shardy | or openstack object save overcloud plan-environment.yaml | 09:36 |
| shardy | oh sorry the file is for the undercloud, so try openstack stack environment show | 09:38 |
| *** cylopez has joined #tripleo | 09:38 | |
| Tengu | shardy: they are still in heat? thought they'd be removed :/ | 09:39 |
| shardy | Tengu: they've always been in heat, but they got moved from the mistral environment to the swift container (in plan-environment.yaml) | 09:40 |
| sri_ | shardy,Tengu got it , thanks :) | 09:40 |
| shardy | I know there's been some discussion around vault or Barbican but I'm not sure on the status | 09:40 |
| Tengu | shardy: ok! | 09:41 |
| shardy | sri_: thanks for pointing it out, I just commented on the article so others will know the commands changed | 09:45 |
| sri_ | shardy, np I was just thinking about adding a comment :) | 09:47 |
| jaosorior | Tengu: nope, they're still in mistral/swift | 09:49 |
| jaosorior | It is possible to get them through a mistral action | 09:49 |
| jaosorior | but they're actually in swift | 09:49 |
| Tengu | jaosorior: yep, shardy answered me earlier :). | 09:49 |
| Tengu | I was already in the future with some secure storage... | 09:50 |
| jaosorior | bandini: is this needed in rocky https://review.openstack.org/#/c/607002/ ? | 09:50 |
| ratailor | amoralej, could you please review https://review.openstack.org/#/c/604265/1 | 09:53 |
| bandini | jaosorior: yes! | 09:58 |
| bandini | I'll cherry-pick shortly | 09:59 |
| *** shyamb has quit IRC | 10:01 | |
| *** shyamb has joined #tripleo | 10:03 | |
| *** akrivoka has joined #tripleo | 10:04 | |
| *** akrivoka has quit IRC | 10:04 | |
| *** ykarel|lunch is now known as ykarel | 10:05 | |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 10:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 10:10 |
| sshnaidm | nhicher, I see the last job passed, did you change something? | 10:12 |
| *** dtrainor has quit IRC | 10:14 | |
| *** yprokule is now known as yprokule|afk | 10:21 | |
| *** derekh has joined #tripleo | 10:22 | |
| *** gkadam has quit IRC | 10:24 | |
| *** udesale has quit IRC | 10:26 | |
| *** jtomasek has quit IRC | 10:28 | |
| *** cylopez has quit IRC | 10:28 | |
| *** akrivoka has joined #tripleo | 10:30 | |
| *** shyamb has quit IRC | 10:34 | |
| *** gfidente has joined #tripleo | 10:35 | |
| gfidente | would someone full of joy review https://review.openstack.org/#/c/604357/ and https://review.openstack.org/#/c/604361/ cause we need those in rocky ASAP :D | 10:40 |
| gfidente | marios++ | 10:41 |
| *** med_ has joined #tripleo | 10:41 | |
| *** shyamb has joined #tripleo | 10:41 | |
| jistr | gfidente: +2'd feel free to +A yourself :) | 10:42 |
| fultonj | full of joy! | 10:42 |
| gfidente | thanks I am waiting for the dependency to go in | 10:42 |
| gfidente | fultonj morning | 10:42 |
| fultonj | gfidente: good morning | 10:43 |
| gfidente | wow those reviews got lots of love | 10:44 |
| gfidente | I better don't abuse this again in near future | 10:44 |
| gfidente | thanks :D | 10:44 |
| marios | gfidente: i'll wager a bet that you will abuse it again in the near future. but that isn't a problem becaus we're | 10:45 |
| marios | full of joy | 10:45 |
| gfidente | what are friends for | 10:45 |
| fultonj | :) | 10:45 |
| marios | fultonj: y u no sleep?! | 10:45 |
| fultonj | i couldn't wait to get to the joy | 10:45 |
| marios | ah right. carry on | 10:45 |
| fultonj | it's about 06:45 here | 10:46 |
| marios | and the joy waits for no one! | 10:46 |
| gfidente | I didn't sleep but had a 3h breakfast | 10:46 |
| gfidente | jistr the union with stderr | 10:49 |
| gfidente | I think you know something I don't know | 10:49 |
| gfidente | wasn't 2>&1 supposed to do that already? | 10:49 |
| rascasoft | weshay, here are the logs https://thirdparty.logs.rdoproject.org/jenkins-oooq-rocky-rdo_trunk-bmu-ha-lab-andromeda-float_nic_with_vlans-12/undercloud/home/stack/overcloud_deploy.log.txt.gz, main problem seems to be "Error: Systemd start for xinetd failed!", but I guess it starts earlier | 10:51 |
| jistr | gfidente: yea i just took that Ansible piece from what we have in deploy-steps-tasks.yaml, then realized that stderr might be printed out of place, so i added 2>&1 too | 10:51 |
| jistr | gfidente: i'd keep it there just so that if in the future we change the command and print something to stderr, it doesn't get completely lost | 10:51 |
| *** boazel_ has quit IRC | 10:52 | |
| *** moshele has quit IRC | 11:03 | |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 11:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 11:10 |
| *** Petersingh is now known as Petersingh|mtg | 11:10 | |
| *** jtomasek has joined #tripleo | 11:18 | |
| *** Petersingh|mtg is now known as Petersingh | 11:20 | |
| Tengu | hey guys, PTAL https://review.openstack.org/#/c/607130/ :) - Thanks! | 11:27 |
| *** dtrainor has joined #tripleo | 11:33 | |
| *** shyamb has quit IRC | 11:34 | |
| *** shyamb has joined #tripleo | 11:34 | |
| *** Petersingh is now known as Petersingh|afk | 11:35 | |
| *** quiquell is now known as quiquell|lunch | 11:37 | |
| *** ratailor has quit IRC | 11:37 | |
| gfidente | arxcruz chandankumar do you guys have a link to the submission changing in tempestconf how to enable/disable object storage tests based on if RGW was deployed? | 11:37 |
| gfidente | arxcruz chandankumar I remember I posted in the comments some links to the parts of the tempest tests which would need to be updated | 11:37 |
| *** jpena is now known as jpena|lunch | 11:38 | |
| arxcruz | gfidente: i don't | 11:40 |
| *** huynq has quit IRC | 11:41 | |
| chandankumar | gfidente: we have healthcheck support not specific to ceph https://github.com/openstack/python-tempestconf/commit/5a102fe8fef8471039a02d82067295f9a388b8b5 and https://github.com/openstack/python-tempestconf/commit/f5bfc213511185ec933192b17a1a34c7d7aff5bf | 11:42 |
| *** boazel has joined #tripleo | 11:42 | |
| *** shyamb has quit IRC | 11:42 | |
| *** Petersingh|afk has quit IRC | 11:42 | |
| sshnaidm | cores, please help merge this https://review.openstack.org/#/c/607316/ | 11:44 |
| gfidente | arxcruz chandankumar I was wondering if anybody was willing to look into https://bugs.launchpad.net/tempest/+bug/1783369 and how to bring that up to tempest community | 11:44 |
| openstack | Launchpad bug 1783369 in tempest "Some ojectstorage tests use skip_path() and assume swift is deployed on the /" [Undecided,New] | 11:44 |
| *** rh-jelabarre has joined #tripleo | 11:44 | |
| *** abishop_ has joined #tripleo | 11:45 | |
| arxcruz | gfidente: we do this https://github.com/openstack/python-tempestconf/blob/master/config_tempest/services/object_storage.py#L91 | 11:46 |
| arxcruz | we check both cases on tempestconf | 11:46 |
| gfidente | arxcruz yeah I remember that but as per LP I linked, this affects other tempest tests too | 11:47 |
| chandankumar | gfidente: tomorrow we have qa office hour feel free to bring there, as I will be not avaiable tomorrow | 11:47 |
| gfidente | arxcruz chandankumar okay great, I never joined that before though, is it a call or irc meeting? | 11:48 |
| chandankumar | gfidente: irc meeting https://wiki.openstack.org/wiki/Meetings/QATeamMeeting | 11:48 |
| gfidente | chandankumar thanks | 11:48 |
| *** raildo has joined #tripleo | 11:48 | |
| *** agopi|off is now known as agopi | 11:50 | |
| *** lblanchard has joined #tripleo | 11:56 | |
| *** jtomasek has quit IRC | 11:57 | |
| Tengu | jaosorior: tripleo-lab supports standalone now. enjoy ;). I'm using that new feature in order to check how standalone works with podman+selinux. Apparently: badly. | 11:58 |
| moguimar | jaosorior: meeting? | 12:00 |
| redrobot | o/ | 12:00 |
| *** ansmith has quit IRC | 12:02 | |
| *** shyamb has joined #tripleo | 12:02 | |
| *** morazi has joined #tripleo | 12:03 | |
| *** trown|outtypewww is now known as trown | 12:03 | |
| *** udesale has joined #tripleo | 12:03 | |
| *** jcoufal has joined #tripleo | 12:05 | |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 12:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 12:10 |
| *** moshele has joined #tripleo | 12:15 | |
| *** amoralej is now known as amoralej|lunch | 12:18 | |
| *** Petersingh|afk has joined #tripleo | 12:19 | |
| *** ssbarnea|bkp2 has quit IRC | 12:20 | |
| weshay | EmilienM, have you seen "Systemd start for xinetd failed!" https://thirdparty.logs.rdoproject.org/jenkins-oooq-rocky-rdo_trunk-bmu-ha-lab-andromeda-float_nic_with_vlans-12/undercloud/home/stack/overcloud_deploy.log.txt.gz#_2018-10-02_18_12_04 | 12:22 |
| EmilienM | I've seen that message before | 12:23 |
| EmilienM | let me grab /var/log/messages | 12:23 |
| *** quiquell|lunch is now known as quiquell | 12:24 | |
| *** med_ has quit IRC | 12:25 | |
| *** Petersingh|afk is now known as Petersingh | 12:25 | |
| *** akrivoka has quit IRC | 12:27 | |
| weshay | rascasoft, ^ | 12:27 |
| weshay | EmilienM, we have a live environment as well right? | 12:28 |
| weshay | sorry | 12:28 |
| rascasoft | EmilienM, I have it available yes | 12:28 |
| weshay | EmilienM, we have a live environment as well | 12:28 |
| weshay | I'm tired | 12:28 |
| *** akrivoka has joined #tripleo | 12:28 | |
| EmilienM | I don't see anything in messages | 12:28 |
| rascasoft | weshay, you look beautiful, honey | 12:28 |
| rascasoft | EmilienM, why are you checking messages? The problem happens during the deployment | 12:29 |
| shyamb | Hi | 12:29 |
| EmilienM | if xinetd fails to start, it can be in the journal | 12:29 |
| shyamb | I deployed openstack using redhat director | 12:29 |
| shyamb | with 3 controller nodes and 2 compute nodes | 12:29 |
| shyamb | does it deploy cinder volume also? | 12:30 |
| EmilienM | nothing in /var/log/journal either | 12:30 |
| EmilienM | weshay: sure, give me ssh access. I can look 2 min | 12:30 |
| shyamb | or do I need to explicitly mention a role ? | 12:30 |
| *** dprince has joined #tripleo | 12:31 | |
| weshay | rascasoft, ^ | 12:31 |
| rascasoft | EmilienM, I'm giving you credentials inpriv | 12:31 |
| *** rlandy has joined #tripleo | 12:32 | |
| rascasoft | weshay, EmilienM I retire everything, a new promotion happened so I don't have the env anymore it is redeploying | 12:34 |
| rascasoft | let's see | 12:34 |
| *** ykarel has quit IRC | 12:34 | |
| EmilienM | weshay, rascasoft : ok let me know when you have env again, I'm happy to help | 12:36 |
| rascasoft | EmilienM, thanks for your happiness | 12:37 |
| *** ykarel has joined #tripleo | 12:37 | |
| hjensas | jaosorior: Hey, I am trying to set up a TLS everywhere to try out the novajoin metadata stuff. I did'nt apply my patch yet, but the stack create fails with resources.ServiceServerMetadataHook: No function "concat" matches supplied arguments | 12:39 |
| hjensas | jaosorior: here is the deploy command: http://paste.openstack.org/show/731357/ | 12:39 |
| *** jtomasek has joined #tripleo | 12:39 | |
| *** jpena|lunch is now known as jpena | 12:41 | |
| jaosorior | bandini: did you stumble upon that too? ^^ | 12:42 |
| jaosorior | hjensas: is that in master? | 12:42 |
| jaosorior | hjensas: and what's in my_roles_data.yaml ? | 12:43 |
| hjensas | jaosorior: yes, master. | 12:43 |
| Tengu | jaosorior: care to have a look at this one? https://review.openstack.org/#/c/607130/ would love seeing it merged soon :) | 12:43 |
| hjensas | jaosorior: roles data: http://paste.openstack.org/show/731358/ | 12:44 |
| bandini | jaosorior, hjensas: nope it worked for me but your including the wrong ssl files | 12:44 |
| bandini | hjensas: use environments/ssl/* not the ones in environments/ | 12:45 |
| jaosorior | maybe it's a good time to remove the other ones | 12:45 |
| jaosorior | since we've had the environments/ssl/* files since pike | 12:45 |
| hjensas | jaosorior: bandini: and fix the doc? https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/ssl.html#classic-public-tls-and-certmonger-based-internal-tls | 12:45 |
| jaosorior | hjensas: will do! | 12:45 |
| bandini | hjensas: yeah I say, let's fix doc + remove the old env files at this point? | 12:46 |
| jaosorior | bandini: lets do that. I'm on it. | 12:46 |
| bandini | ack, just CC me | 12:46 |
| hjensas | bandini: jaosorior: thanks, I will do the delete dance and change the environment files to get going then. | 12:46 |
| bandini | hjensas: ack, ping me if you are stuck | 12:47 |
| *** vinaykns has joined #tripleo | 12:47 | |
| nhicher | sshnaidm: hello, no I didn't change anything, I got the same error on https://logs.rdoproject.org/96/15896/35/check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master-vexxhost/2b2cbf3/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 12:53 |
| *** moshele has quit IRC | 12:53 | |
| nhicher | sshnaidm: jobs failed with the same error 95% of the time on vexxhost | 12:53 |
| jaosorior | bandini, hjensas: https://review.openstack.org/607555 | 12:54 |
| sshnaidm | nhicher, I see, I looked at https://review.rdoproject.org/r/#/c/16566/ | 12:54 |
| *** pvc_ has quit IRC | 12:54 | |
| *** tbarron has joined #tripleo | 12:55 | |
| *** artom has quit IRC | 12:56 | |
| jaosorior | bandini, hjensas: https://review.openstack.org/607556 | 12:56 |
| nhicher | sshnaidm: same error on https://review.rdoproject.org/r/#/c/15896 (not periodic) | 12:57 |
| sshnaidm | nhicher, I can think only about performance issue right now.. Of course it doesn't mean that mistral should fail, but seems like there's a bug. | 12:59 |
| sshnaidm | nhicher, it takes 3 hours even before overcloud deploy starts | 12:59 |
| sshnaidm | nhicher, is there something we can speed up? | 13:00 |
| nhicher | sshnaidm: I will try to find what could be the issue | 13:01 |
| *** udesale has quit IRC | 13:05 | |
| hjensas | bandini: still fails http://paste.openstack.org/show/731364/ | 13:05 |
| bandini | hjensas: hohum I did not hit that one | 13:05 |
| *** shyamb has quit IRC | 13:05 | |
| *** holser__ has joined #tripleo | 13:06 | |
| jaosorior | therve, shardy: Any idea what does: No function "concat" matches supplied arguments .... means as a heat error? | 13:06 |
| slagle | yaql | 13:06 |
| * slagle runs | 13:06 | |
| jaosorior | fuuuuuck | 13:07 |
| therve | Yep yaql | 13:07 |
| *** holser_ has quit IRC | 13:07 | |
| therve | jaosorior: Source? | 13:07 |
| *** shyamb has joined #tripleo | 13:07 | |
| jaosorior | therve: hjensas is hitting it like this http://paste.openstack.org/show/731364/ which would be this file https://github.com/openstack/tripleo-heat-templates/blob/master/extraconfig/nova_metadata/krb-service-principals.j2.yaml | 13:08 |
| hjensas | jaosorior: it's a yaql expression .. yes, that one. | 13:08 |
| jaosorior | hjensas: can you show /home/stack/overcloud/templates/my_roles_data.yaml ? | 13:09 |
| therve | Probably need to shove some more coalesce :/ | 13:09 |
| hjensas | jaosorior: h/home/stack/overcloud/templates/my_roles_data.yaml :: http://paste.openstack.org/show/731358/ | 13:10 |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 13:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 13:10 |
| jaosorior | bandini: yours was master too, right? | 13:11 |
| bandini | jaosorior: yes although about 10 days old or so | 13:11 |
| *** ansmith has joined #tripleo | 13:12 | |
| jaosorior | bandini: well, that file hasn't changed since May :D I have no idea what the issue might be | 13:13 |
| jaosorior | bandini: mind telling hjensas how you deployed? maybe he can replicate it that way | 13:14 |
| jaosorior | then we can figure out the difference. | 13:14 |
| *** mcornea has joined #tripleo | 13:14 | |
| hjensas | jaosorior: bandini: hmm, I wonder if it can be because I have vip: false for storage and storage_mgmt networks. | 13:15 |
| *** ssbarnea|bkp2 has joined #tripleo | 13:17 | |
| jaosorior | hjensas: that might be it | 13:17 |
| *** aedc has quit IRC | 13:18 | |
| jaosorior | I've never tested with such a configuration | 13:18 |
| *** agurenko_ has joined #tripleo | 13:18 | |
| *** agurenko has quit IRC | 13:18 | |
| honza | jtomasek: it looks like we'll have to package tempest-tripleo-ui before we can run anything in CI --- the tempest image has all the plugins already installed via the openstack-tempest-all rpm (and we need to add ours to it) | 13:19 |
| bandini | hjensas, jaosorior: sec am in a call | 13:20 |
| therve | hjensas: You can set "$.data" in the expression, and then inspect the value | 13:21 |
| hjensas | jaosorior: bandini: retrying with vips on more networks. (My bad, was using a scaled down network_data.yaml which previously had vip's on other composable networks. | 13:22 |
| hjensas | therve: ack. will try that in the next iteration. | 13:25 |
| nhicher | sshnaidm: I extracted tasks during more than 10s in job-output.txt https://softwarefactory-project.io/paste/show/1231, same for undercloud installation: https://softwarefactory-project.io/paste/show/1229 | 13:27 |
| nhicher | sshnaidm: on rdocloud, we run on local storage, on vexxhost on ceph | 13:27 |
| nhicher | sshnaidm: I checked afs/proxy cache log, it works well | 13:28 |
| *** paramite|clone has quit IRC | 13:29 | |
| *** artom has joined #tripleo | 13:30 | |
| sshnaidm | nhicher, hmm.. I see undercloud install took twice then in rdo cloud: undercloud-deploy : Install the undercloud --------------------------- 4103.78s | 13:30 |
| *** social has joined #tripleo | 13:30 | |
| sshnaidm | nhicher, and rdo: undercloud-deploy : Install the undercloud --------------------------- 2303.82s | 13:31 |
| sshnaidm | nhicher, it's funny, I just start to do script to compare 2 jobs performance.. maybe will finish today and we can see :) | 13:31 |
| hjensas | jaosorior: bandini: It was the vip's in network data missing. | 13:32 |
| *** amoralej|lunch is now known as aoralej | 13:32 | |
| bandini | hjensas: ah, good to know! | 13:32 |
| *** mjturek has joined #tripleo | 13:32 | |
| nhicher | sshnaidm: tristanC wrote LogClassify to compare working and failing jobs, could be useful to http://38.145.32.112/log-classify/list | 13:33 |
| nhicher | to extract relevant logs | 13:34 |
| sshnaidm | nhicher, I compare it by times for tasks, that are taken from ansible | 13:35 |
| nhicher | sshnaidm: yes, was to give you logreduce url =) | 13:36 |
| *** agurenko has joined #tripleo | 13:38 | |
| *** bfournie has joined #tripleo | 13:39 | |
| *** agurenko_ has quit IRC | 13:39 | |
| *** Vorrtex has joined #tripleo | 13:39 | |
| *** ykarel is now known as ykarel|away | 13:39 | |
| *** aoralej is now known as amoralej | 13:41 | |
| *** agopi has quit IRC | 13:42 | |
| *** ykarel_ has joined #tripleo | 13:43 | |
| *** ykarel|away has quit IRC | 13:45 | |
| *** pvc has joined #tripleo | 13:47 | |
| pvc | hi anyone Determining IP information for ens4f0... failed; no link present. Check cable? | 13:47 |
| *** ssbarnea|bkp2 has quit IRC | 13:49 | |
| *** cylopez has joined #tripleo | 13:50 | |
| *** moguimar has quit IRC | 13:51 | |
| *** tosky__ has joined #tripleo | 13:51 | |
| *** tosky has quit IRC | 13:51 | |
| *** moguimar has joined #tripleo | 13:51 | |
| *** tosky__ is now known as tosky | 13:52 | |
| jaosorior | Tengu: so.. how do you run standalone with tripleo-lab then? | 13:53 |
| *** social has quit IRC | 13:55 | |
| *** social has joined #tripleo | 13:56 | |
| *** agurenko has quit IRC | 13:57 | |
| *** shyamb has quit IRC | 13:58 | |
| Tengu | jaosorior: there are some new variables listed at the end of the readme | 13:58 |
| Tengu | jaosorior: standalone: yes and you're on | 13:58 |
| Tengu | do not try "podman" for now, there are some issues :). | 13:59 |
| *** agurenko has joined #tripleo | 13:59 | |
| *** paramite|clone has joined #tripleo | 13:59 | |
| jaosorior | got it | 13:59 |
| jaosorior | Tengu: and that deploys to a VM in my local machine, right? | 13:59 |
| Tengu | jaosorior: well, it's more for a remote builder. | 14:00 |
| Tengu | jaosorior: the whole thing works best when you run it from your laptop and it deploy on a remote computer - you can ask dpeacock - it tried to use it on localhost, it wasn't a real success. | 14:00 |
| jaosorior | :( | 14:00 |
| *** hjensas has quit IRC | 14:01 | |
| *** shyamb has joined #tripleo | 14:01 | |
| chandankumar | EmilienM: hello | 14:03 |
| chandankumar | EmilienM: http://logs.openstack.org/56/605356/17/check/tripleo-ci-centos-7-undercloud-containers/564554e/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2018-10-03_13_29_00 | 14:04 |
| *** ksambor has quit IRC | 14:04 | |
| *** agopi has joined #tripleo | 14:04 | |
| chandankumar | EmilienM: undercloud install is failing while pulling containers | 14:04 |
| *** skramaja has quit IRC | 14:05 | |
| *** skramaja has joined #tripleo | 14:05 | |
| chandankumar | EmilienM: is it a known issue? | 14:05 |
| shyamb | hi ykarel: | 14:06 |
| shyamb | http://paste.openstack.org/show/731373/ | 14:07 |
| shyamb | not able to delete swift-keyrings container | 14:07 |
| shyamb | Am I deleting the right container? | 14:08 |
| nhicher | sshnaidm: undercloud installation https://softwarefactory-project.io/paste/show/1232 for https://review.rdoproject.org/r/#/c/15896 | 14:08 |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 14:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 14:10 |
| Vorrtex | jaosorior could I trouble you to workflow this review? The gate jobs were broken for a bit, but I got a clean run yesterday: https://review.openstack.org/#/c/604401/ | 14:15 |
| jtomasek | honza: ack, sorry my ISP sucks today | 14:17 |
| *** quiquell is now known as quiquell|off | 14:17 | |
| honza | jtomasek: np | 14:19 |
| honza | jtomasek: we might have to +2+1 the project setup patch in order to proceed with packaging... | 14:19 |
| honza | jtomasek: even if it's not 100% tested | 14:19 |
| jtomasek | honza: so how is it going to work when we'll need to add more tests? Will we always need to wait for the package to get updated? | 14:20 |
| jtomasek | honza: I managed to get the env installed correctly finally | 14:20 |
| honza | jtomasek: yisssss | 14:20 |
| honza | jtomasek: yes, it looks like that's how it's set up right now | 14:21 |
| *** chandankumar is now known as chkumar|pyconind | 14:21 | |
| honza | jtomasek: new plugin rpm, new tempest rpm, new image | 14:21 |
| honza | jtomasek: i wonder how other projects do it | 14:22 |
| *** shyamb has quit IRC | 14:24 | |
| *** moguimar has quit IRC | 14:25 | |
| *** moguimar has joined #tripleo | 14:25 | |
| derekh | if anybody has time to review https://review.openstack.org/#/c/582294/ (see depends on patches) I'd appreciate it, trying to get a CI job running that runs ironic in overcloud | 14:27 |
| pvc | hi | 14:28 |
| pvc | derekh http://paste.openstack.org/show/731377/ | 14:28 |
| pvc | should i use use_dhcp : true | 14:28 |
| derekh | this one just needs a A+ https://review.openstack.org/#/c/579601/ | 14:28 |
| derekh | pvc: not sure, sorry | 14:31 |
| *** skramaja has quit IRC | 14:33 | |
| *** aufi has quit IRC | 14:42 | |
| EmilienM | Tengu: new version of podman, pushed from master. It "should" fix storage issue | 14:45 |
| EmilienM | let's see in https://review.openstack.org/#/c/600517/ | 14:46 |
| Tengu | EmilienM: good news :) | 14:46 |
| *** slaweq has quit IRC | 14:48 | |
| sshnaidm | nhicher, so from your paste seems like everything related to containers is slow.. do you have docker proxy as well there? | 14:49 |
| *** ssbarnea|bkp2 has joined #tripleo | 14:51 | |
| nhicher | sshnaidm: by docker proxy you mean registry-mirrors ? | 14:51 |
| sshnaidm | nhicher, yep | 14:51 |
| *** ssbarnea|bkp2 has quit IRC | 14:51 | |
| nhicher | sshnaidm: yes https://logs.rdoproject.org/96/15896/35/check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master-vexxhost/2b2cbf3/logs/undercloud/etc/docker/daemon.json.txt.gz | 14:52 |
| *** ssbarnea|bkp2 has joined #tripleo | 14:52 | |
| *** iurygregory has joined #tripleo | 14:52 | |
| nhicher | sshnaidm: and I saw data on afs-mirror log files | 14:53 |
| *** agurenko has quit IRC | 14:53 | |
| *** ykarel_ has quit IRC | 14:54 | |
| *** slaweq has joined #tripleo | 14:56 | |
| sshnaidm | nhicher, hmm.. then disk performance..? | 14:56 |
| sshnaidm | nhicher, can we compare it anyhow? just to run some tasks in job that will measure it.. | 14:56 |
| nhicher | sshnaidm: yes, ceph versus local storage | 14:56 |
| sshnaidm | nhicher, ceph??? | 14:57 |
| nhicher | on vexxhost, yes | 14:57 |
| sshnaidm | omg | 14:57 |
| sshnaidm | nhicher, well, one reason we already know | 14:57 |
| sshnaidm | nhicher, some container tasks are very disk expensive.. | 14:58 |
| *** Petersingh is now known as Petersingh|gone | 14:59 | |
| sshnaidm | --- are any mistral folks here? need you help to look at error "The WebSocket timed out before the Workflow completed" in https://logs.rdoproject.org/96/15896/35/check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master-vexxhost/2b2cbf3/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 14:59 |
| weshay | toure, ^ | 14:59 |
| *** Petersingh|gone has quit IRC | 15:00 | |
| nhicher | sshnaidm: the final target is to have dedicated computes on vexxhost with local storage, for testing we use public cloud (where no local storage is available) | 15:00 |
| *** hjensas has joined #tripleo | 15:00 | |
| sshnaidm | nhicher, I'm afraid we'll start to hit all bugs that could be caused by slow storage, and which we didn't see in our cloud never.. | 15:01 |
| sshnaidm | nhicher, on one hand it's cool to improve the product and solve it, on the other - it's kinda out of scope of this task.. | 15:02 |
| sshnaidm | nhicher, just worries, maybe not all is so bad.. | 15:02 |
| nhicher | sshnaidm: agree, but we need working jobs to get metrics before going in production with vexxhost. I will discuss with apevec | 15:03 |
| EmilienM | chkumar|pyconind: will look asap. In mtg all morning | 15:03 |
| EmilienM | chkumar|pyconind: yes it's a known issue, being addressed now. I'll let you know status on my EOD | 15:04 |
| pvc | guys anyone can help? | 15:04 |
| pvc | it seems the it cannot up my interface on overcloud deploy | 15:05 |
| sshnaidm | nhicher, ok, I hope to catch somebody from mistral to help anyway.. | 15:05 |
| *** thrash is now known as thrash|biab | 15:06 | |
| *** boazel has quit IRC | 15:06 | |
| *** boazel_ has joined #tripleo | 15:06 | |
| weshay | d0ugal, ^ | 15:07 |
| *** cylopez has quit IRC | 15:07 | |
| *** ykarel has joined #tripleo | 15:07 | |
| *** akrivoka has quit IRC | 15:08 | |
| *** akrivoka has joined #tripleo | 15:08 | |
| *** jtomasek has quit IRC | 15:10 | |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 15:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792872 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1792872 in tripleo "[queens] overcloud prepare image failed by giving IronicAction.node.set_provision_state failed: 'NoneType' object has no attribute '__getitem_" [Critical,Triaged] - Assigned to Quique Llorente (quiquell) | 15:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 15:10 |
| *** jtomasek has joined #tripleo | 15:10 | |
| weshay | mwhahaha, jaosorior FYI http://lists.openstack.org/pipermail/openstack-dev/2018-October/135388.html | 15:11 |
| jaosorior | weshay: ack | 15:12 |
| jaosorior | cgoncalves: around? | 15:12 |
| cgoncalves | jaosorior, yep | 15:12 |
| jaosorior | cgoncalves: not a blocker, but missed a spot in octavia: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/neutron-api.yaml#L284 | 15:13 |
| jaosorior | you need metadata_settings too | 15:13 |
| jaosorior | in octavia-api.yaml | 15:13 |
| *** jfrancoa has quit IRC | 15:15 | |
| cgoncalves | jaosorior, hmm I was told it worked fine without it. and glance-api doesn't have it either https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/glance-api.yaml#L284 | 15:15 |
| jaosorior | cgoncalves: it works fine without... IF you deploy in the controller. Once you move the service away from the controller. The appropriate service principals won't be created | 15:18 |
| jaosorior | cgoncalves: glance not having it is a bug too | 15:18 |
| jaosorior | cgoncalves: so.... stuff will work IF everything is in the controller nodes. It starts breaking once you move towards custom composable roles | 15:19 |
| cgoncalves | jaosorior, I see. I'll add it to octavia-api and glance-api. thank you!! | 15:19 |
| jaosorior | cgoncalves: thank you | 15:19 |
| kopecmartin|ruck | weshay, marios|rover the bug https://bugs.launchpad.net/tripleo/+bug/1792872 seems to affect more jobs, I made a comment there | 15:19 |
| openstack | Launchpad bug 1792872 in tripleo "[queens] overcloud prepare image failed by giving IronicAction.node.set_provision_state failed: 'NoneType' object has no attribute '__getitem_" [Critical,Triaged] - Assigned to Quique Llorente (quiquell) | 15:19 |
| marios|rover | dtantsur|afk: d0ugal o/ hey folks can you please check https://bugs.launchpad.net/tripleo/+bug/1792872 when you next get a chance thanks (comment #4 is from today) | 15:20 |
| *** zaneb has quit IRC | 15:21 | |
| marios|rover | ack kopecmartin|ruck thanks | 15:21 |
| weshay | kopecmartin|ruck, thanks will look in a sec | 15:21 |
| *** zaneb has joined #tripleo | 15:22 | |
| *** iranzo has quit IRC | 15:22 | |
| weshay | EmilienM, do you by chance have a link to the spreadsheet you made the other day, my bookmark tags are not bringing it up | 15:22 |
| weshay | for me | 15:22 |
| weshay | sorry | 15:22 |
| mwhahaha | weshay: https://ethercalc.openstack.org/TripleOStandaloneCICoverage | 15:23 |
| mwhahaha | weshay: speaking of links, is there a doc to 'how to stand up a new pipeline'? re: the aarch64 email that we keep ignoring | 15:23 |
| weshay | thanks | 15:23 |
| weshay | mwhahaha, depends where it's being stood up | 15:24 |
| weshay | for tripleo | 15:24 |
| weshay | ? | 15:24 |
| mwhahaha | weshay: yea it sounded like they wanted it in rdo, but we really should reply with a get bent or here's how to do it | 15:24 |
| mwhahaha | yea for tripleo/rdo | 15:24 |
| mwhahaha | cause they wanted containers or something | 15:24 |
| weshay | mwhahaha, I hada similiar meeting w/ the ppc folks | 15:25 |
| weshay | mwhahaha, I think rain needs to get involved | 15:25 |
| weshay | as they both have the same needs, and I'd rather not do this twice | 15:25 |
| weshay | mwhahaha, 1 sec... home inspectors | 15:25 |
| *** iranzo has joined #tripleo | 15:25 | |
| *** rf0lc0 is now known as rfolco_doc | 15:28 | |
| weshay | mwhahaha, ok.. it passed :) | 15:28 |
| mwhahaha | HEH | 15:29 |
| weshay | mwhahaha, to help these other arch's we need their hardware in rdo-cloud | 15:29 |
| weshay | if it's in ci.centos.. I don't think I can help too much other than to show them the scripts we use | 15:30 |
| mwhahaha | yea i figured that | 15:30 |
| mwhahaha | it sounded like they wanted aarch64 tripleo containers | 15:30 |
| weshay | mwhahaha, there needs to be some coordination of these $things | 15:30 |
| mwhahaha | which i don't know how that'd work | 15:30 |
| weshay | and I have yet to see it | 15:30 |
| *** ramishra has quit IRC | 15:30 | |
| mwhahaha | but i have no idea who would drive such an effort | 15:30 |
| weshay | would need the community leader folks to be in this channel | 15:30 |
| *** tzumainn has joined #tripleo | 15:30 | |
| marios|rover | thrash|biab: bfournie o/ can you please check https://bugs.launchpad.net/tripleo/+bug/1792872 when you next get a chance thanks (comment #4 is from today) | 15:32 |
| openstack | Launchpad bug 1792872 in tripleo "[queens] overcloud prepare image failed by giving IronicAction.node.set_provision_state failed: 'NoneType' object has no attribute '__getitem_" [Critical,Triaged] - Assigned to Quique Llorente (quiquell) | 15:32 |
| weshay | mwhahaha, I'll email rain | 15:33 |
| *** pcaruana has quit IRC | 15:33 | |
| toure | weshay I am looking | 15:36 |
| *** med_ has joined #tripleo | 15:37 | |
| *** pvc has quit IRC | 15:38 | |
| toure | sshnaidm I am looking into the problem | 15:38 |
| weshay | thanks toure | 15:39 |
| weshay | mwhahaha ok.. email sent | 15:39 |
| mwhahaha | k | 15:39 |
| cgoncalves | jaosorior, https://review.openstack.org/#/c/607617/ | 15:40 |
| toure | weshay the timestamp in the mistral logs don't match when the error occurred | 15:40 |
| *** hamzy has quit IRC | 15:40 | |
| jaosorior | cgoncalves: thanks, responded. | 15:41 |
| *** hamzy has joined #tripleo | 15:43 | |
| sshnaidm | toure, thanks | 15:43 |
| sshnaidm | toure, this happens on external cloud only, it passes in rdo cloud.. external cloud has slow storage, maybe this affects it | 15:44 |
| toure | so I see the api call but it never makes it to the engine | 15:44 |
| toure | the logs seem a bit offf | 15:44 |
| toure | so the api registers the call at 04:34 | 15:45 |
| toure | but the engine logs start at 05:05 | 15:45 |
| toure | sshnaidm maybe the logs were rotated | 15:45 |
| toure | but that shouldn't be the case on a new deployment | 15:45 |
| sshnaidm | toure, hmm.. and when the engine did start? | 15:46 |
| sshnaidm | toure, are we talking about heat engine? | 15:47 |
| toure | sorry mistral engine | 15:47 |
| * toure may have found something | 15:48 | |
| * toure is still looking | 15:48 | |
| *** yprokule|afk has quit IRC | 15:50 | |
| *** gfidente is now known as gfidente|afk | 15:57 | |
| *** ade_lee has joined #tripleo | 15:58 | |
| *** vinaykns has quit IRC | 16:00 | |
| *** thrash|biab is now known as thrash | 16:00 | |
| *** saneax has quit IRC | 16:00 | |
| thrash | marios|rover: looking | 16:01 |
| marios|rover | thanks thrash me about to go for the day but will check the bug tomorrow my morning thanks for checking (also emailed you) | 16:02 |
| thrash | marios|rover: ack | 16:02 |
| *** chem has quit IRC | 16:06 | |
| *** slaweq has quit IRC | 16:09 | |
| *** ykarel is now known as ykarel|away | 16:09 | |
| toure | sshnaidm so I think the storage is getting you on this | 16:09 |
| *** paramite|clone has quit IRC | 16:09 | |
| *** slaweq has joined #tripleo | 16:10 | |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 16:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792872 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1792872 in tripleo "[queens] overcloud prepare image failed by giving IronicAction.node.set_provision_state failed: 'NoneType' object has no attribute '__getitem_" [Critical,Triaged] - Assigned to Quique Llorente (quiquell) | 16:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 16:10 |
| toure | sshnaidm tries to lookup some templates and swift is responding with object not found | 16:10 |
| toure | *heat* | 16:10 |
| * toure steps away from keyboard for a few | 16:12 | |
| *** toure is now known as toure|biab | 16:12 | |
| *** rdopiera has quit IRC | 16:12 | |
| sshnaidm | toure|biab, yeah, I suspect it is. But interesting where is bottleneck and can we solve it or not | 16:13 |
| *** trown is now known as trown|lunch | 16:14 | |
| *** shyamb has joined #tripleo | 16:17 | |
| toure|biab | sshnaidm I will have to dig further but we aren't in charge at that point as we just kicked heat off to do its work | 16:23 |
| *** panda is now known as panda|off | 16:24 | |
| *** toure|biab is now known as toure|food | 16:24 | |
| bnemec | Any idea what's going on here: http://paste.openstack.org/show/731414/ ? | 16:24 |
| bnemec | It looks like the network deployment succeeded and then it failed to read the deployed json file which failed the overcloud deployment. | 16:25 |
| Tengu | see you tomorrow folks. | 16:26 |
| *** shyamb has quit IRC | 16:28 | |
| slagle | bnemec: https://bugs.launchpad.net/tripleo/+bug/1792343 | 16:29 |
| openstack | Launchpad bug 1792343 in tripleo "[tripleo] rocky baremetal deployment fails with jq: error: Could not open file /var/lib/heat-config/deployed/<id>.notify.json" [Critical,Fix released] - Assigned to Quique Llorente (quiquell) | 16:29 |
| *** med_ has quit IRC | 16:29 | |
| slagle | bnemec: fixed by https://review.openstack.org/604455 (or perhaps not) | 16:29 |
| slagle | if it's still an issue, we might need to look at just switching to use an ansible async task with NetworkDeployment | 16:31 |
| bnemec | slagle: Thanks, let me check if I have that fix in my environment. | 16:32 |
| *** iurygregory has quit IRC | 16:33 | |
| bnemec | openstack-tripleo-common-9.3.1-0.20180929163758.0748854.el7.noarch | 16:34 |
| bnemec | Seems like that should have the fix. | 16:34 |
| *** med_ has joined #tripleo | 16:34 | |
| slagle | you can check if the options are there in /var/lib/mistral/overcloud/ansible.cfg | 16:34 |
| bnemec | Yep: -o ServerAliveInterval=5 -o ServerAliveCountMax=5 | 16:35 |
| slagle | well anyway the issue is that ansible runs the task twice since the ssh connection dropped on the first one. the second one fails quickly since notify.json does not exist | 16:40 |
| slagle | the first one eventually succeeds (assuming the networking config is applied correctly), but ansible doesn't know anything about it succeeding | 16:40 |
| *** social has quit IRC | 16:41 | |
| honza | jtomasek: btw, the tempest-tripleo-ui instructions are no longer up to date (e.g. they don't work with containerized tempest) | 16:42 |
| *** artom has quit IRC | 16:42 | |
| jtomasek | honza: yeah, I have been able to run the tempest but I did not see ui tests being run | 16:43 |
| jtomasek | honza: do you have the updated steps pls? | 16:43 |
| *** holser__ has quit IRC | 16:44 | |
| honza | jtomasek: nope :) | 16:44 |
| honza | jtomasek: working on it | 16:44 |
| *** med_ has quit IRC | 16:45 | |
| weshay | mwhahaha, ok.. rain is on the case | 16:46 |
| mwhahaha | k | 16:46 |
| *** med_ has joined #tripleo | 16:49 | |
| *** chkumar|pyconind has quit IRC | 16:51 | |
| *** med_ has quit IRC | 16:55 | |
| *** asbishop has joined #tripleo | 16:55 | |
| *** chandankumar has joined #tripleo | 16:56 | |
| *** bogdando has quit IRC | 16:57 | |
| *** abishop_ has quit IRC | 16:58 | |
| *** derekh has quit IRC | 17:00 | |
| honza | arxcruz: hey, are you the person to talk about tempest in ci? | 17:00 |
| *** ykarel|away has quit IRC | 17:02 | |
| *** lblanchard has quit IRC | 17:03 | |
| tosky | honza: arxcruz is one, also chandankumar and kopecmartin|ruck | 17:05 |
| honza | tosky: +1 | 17:05 |
| *** chandankumar has quit IRC | 17:07 | |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792872 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 17:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1792872 in tripleo "[queens] overcloud prepare image failed by giving IronicAction.node.set_provision_state failed: 'NoneType' object has no attribute '__getitem_" [Critical,Triaged] - Assigned to Quique Llorente (quiquell) | 17:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 17:10 |
| *** jpena is now known as jpena|off | 17:11 | |
| *** trown|lunch is now known as trown | 17:11 | |
| arxcruz | honza: hey, yeah, can we talk tomorrow? it's pretty late here already, plus is holiday here :) | 17:12 |
| honza | arxcruz: absolutely! | 17:12 |
| *** med_ has joined #tripleo | 17:13 | |
| *** akrivoka has quit IRC | 17:16 | |
| *** gfidente|afk has quit IRC | 17:16 | |
| *** salmankhan has quit IRC | 17:17 | |
| *** med_ has quit IRC | 17:20 | |
| weshay | bfournie, hey.. looking at one of my deployments | 17:20 |
| weshay | automated_clean=False in deployments /me checks ovb | 17:21 |
| bfournie | weshay: ok, so not cleaning timeout there | 17:21 |
| weshay | ugh.. where is etc :( | 17:22 |
| bfournie | weshay: in https://bugs.launchpad.net/tripleo/+bug/1792872 there is "did not reach state "available", the state is "clean failed"" | 17:23 |
| openstack | Launchpad bug 1792872 in tripleo "[queens] overcloud prepare image failed by giving IronicAction.node.set_provision_state failed: 'NoneType' object has no attribute '__getitem_" [Critical,Triaged] - Assigned to Quique Llorente (quiquell) | 17:23 |
| weshay | bfournie, ya. .see that | 17:23 |
| *** rfolco_doc has quit IRC | 17:23 | |
| weshay | bfournie, hrm.. in ovb it's set to true https://logs.rdoproject.org/21/602721/5/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/a1eb66f/logs/undercloud/var/log/config-data/ironic/etc/ironic/ironic.conf.txt.gz | 17:27 |
| weshay | if that's the right log | 17:28 |
| *** jabadia has joined #tripleo | 17:28 | |
| bfournie | weshay: ok, if use that maybe need to increase clean_callback_timeout | 17:29 |
| weshay | bfournie, http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/puppet/services/ironic-conductor.yaml#n38 | 17:30 |
| weshay | so we would want the default to remain true | 17:30 |
| weshay | if there is an open issue that tells users to turn it off https://access.redhat.com/solutions/3349081 | 17:31 |
| weshay | ? | 17:31 |
| *** vinaykns has joined #tripleo | 17:31 | |
| *** jpich has quit IRC | 17:32 | |
| weshay | can also turn it off via http://git.openstack.org/cgit/openstack/instack-undercloud/tree/elements/puppet-stack-config/puppet-stack-config.yaml.template#n520 | 17:33 |
| bfournie | weshay: yeah I guess we can't increase cleaning timeout through puppet | 17:35 |
| bfournie | TheJulia: ^ looks like cleaning timeouts in CI | 17:37 |
| weshay | bfournie, EmilienM wdyt of https://review.openstack.org/607655 | 17:38 |
| weshay | nhicher, fyi ^ | 17:40 |
| *** shardy has quit IRC | 17:47 | |
| bfournie | weshay: seems reasonable although not sure how undercloud_clean_nodes to IronicAutomatedClean | 17:52 |
| bfournie | weshay: ah I see https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/undercloud-deploy/templates/undercloud.conf.j2#L344 | 17:55 |
| *** asbishop has quit IRC | 17:56 | |
| EmilienM | weshay: you would save reviewer's time by writting commit messages | 17:57 |
| EmilienM | not sure how many time have asked people to write a commit message | 17:57 |
| EmilienM | sounds like it's too hard | 17:57 |
| weshay | EmilienM, yes will do so | 17:57 |
| EmilienM | oh yeah a likely story | 17:57 |
| EmilienM | :D | 17:57 |
| weshay | when there is a bug related | 17:57 |
| weshay | you know.. | 17:58 |
| EmilienM | you lazy | 17:58 |
| EmilienM | you don't deserve +2 | 17:58 |
| EmilienM | not even -2 | 17:58 |
| *** asbishop has joined #tripleo | 17:58 | |
| EmilienM | -999 | 17:58 |
| EmilienM | (and even) | 17:58 |
| weshay | lolz | 17:58 |
| *** holser_ has joined #tripleo | 18:03 | |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 18:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792872 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1792872 in tripleo "[queens] overcloud prepare image failed by giving IronicAction.node.set_provision_state failed: 'NoneType' object has no attribute '__getitem_" [Critical,Triaged] - Assigned to Quique Llorente (quiquell) | 18:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 18:10 |
| *** pcaruana has joined #tripleo | 18:24 | |
| *** itlinux has joined #tripleo | 18:24 | |
| *** ssbarnea|bkp2 has quit IRC | 18:33 | |
| *** artom has joined #tripleo | 18:35 | |
| itlinux | hello all, I know you can allocate ips from the pool, does that include the ctlplane part too. I tried to see if that's an option since I would like to have that set. Thanks | 18:43 |
| *** artom has quit IRC | 18:43 | |
| EmilienM | weshay, mwhahaha : I think we can switch the containerized undercloud on podman | 18:46 |
| EmilienM | https://review.openstack.org/#/c/600517/ | 18:46 |
| EmilienM | it's now passing | 18:47 |
| *** artom has joined #tripleo | 18:47 | |
| *** artom has quit IRC | 18:47 | |
| *** artom has joined #tripleo | 18:48 | |
| EmilienM | beagles, bandini : FYI heads-up on running privileged commands in podman, see https://review.openstack.org/604180 | 18:50 |
| EmilienM | that's what it takes in the containers config to run podman within podman containers | 18:50 |
| *** pcaruana has quit IRC | 18:50 | |
| beagles | EmilienM: ack | 18:52 |
| beagles | EmilienM: so is the intent for the templates in stein to support running with both docker and podman? | 18:53 |
| EmilienM | beagles: good question | 18:54 |
| EmilienM | I don't know | 18:54 |
| EmilienM | at the end of Stein we'll switch to podman by default for sure | 18:54 |
| *** rfolco has joined #tripleo | 18:56 | |
| *** iranzo has quit IRC | 19:00 | |
| *** amoralej is now known as amoralej|off | 19:03 | |
| *** holser_ has quit IRC | 19:07 | |
| *** abishop_ has joined #tripleo | 19:08 | |
| *** asbishop has quit IRC | 19:10 | |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792872 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 19:10 |
| openstack | Launchpad bug 1792872 in tripleo "[queens] overcloud prepare image failed by giving IronicAction.node.set_provision_state failed: 'NoneType' object has no attribute '__getitem_" [Critical,Triaged] - Assigned to Quique Llorente (quiquell) | 19:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 19:10 |
| *** tzumainn has quit IRC | 19:15 | |
| *** bugzy has joined #tripleo | 19:16 | |
| itlinux | question.. I set the variable for ip-pool but somehow I do not see that the overcloud does take those IPs preset in the file. I do have those ips outside of the network-iso and therefore are not in that pool range | 19:17 |
| weshay | mwhahaha, https://review.openstack.org/#/c/607655/ | 19:17 |
| *** ssbarnea|bkp2 has joined #tripleo | 19:18 | |
| mwhahaha | weshay: so i don't think we need to turn that off because i think it should only affect things when we delete the nodes unless something changed? | 19:19 |
| *** bugzy_ has quit IRC | 19:19 | |
| weshay | mwhahaha, just looking to make things more reliable | 19:19 |
| weshay | ovb job pass rate is 45% | 19:20 |
| weshay | compared to 80% upstream | 19:20 |
| * mwhahaha points to ironic | 19:20 | |
| mwhahaha | thats why we have these ovb jobs | 19:20 |
| mwhahaha | not sure how it slipped passed promotion? | 19:20 |
| weshay | mwhahaha, maybe the ironic project would like to stand up some kind of 3rd party | 19:20 |
| mwhahaha | pffft test coverage | 19:20 |
| weshay | mwhahaha, it's not consistent | 19:20 |
| weshay | when you run every 5 hours w/ 3 ovb jobs.. you'll hit yatzee | 19:21 |
| weshay | at some point | 19:21 |
| mwhahaha | yay race conditions | 19:21 |
| mwhahaha | it needs to get fixed | 19:21 |
| weshay | http://cistatus.tripleo.org/promotion/#legacy-periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master | 19:22 |
| weshay | last two times failed on prep-images | 19:22 |
| weshay | sshnaidm, for promotions.. if we could save a longer history that may be helpful | 19:22 |
| * weshay looks at that code | 19:22 | |
| *** spartakos has joined #tripleo | 19:25 | |
| sshnaidm | weshay, yeah, increased it now | 19:26 |
| sshnaidm | weshay, will build and redeploy, will see it later | 19:26 |
| weshay | sshnaidm, ah.. ur the man | 19:26 |
| weshay | thanks | 19:26 |
| *** toure|food is now known as toure | 19:30 | |
| weshay | I think the openstack bot is busted | 19:31 |
| mwhahaha | it's been busted since yesterday | 19:33 |
| weshay | mwhahaha, what has.. ironic? | 19:38 |
| mwhahaha | what do you mean | 19:38 |
| weshay | mwhahaha, it's been busted since yesterday, you are referring to the promotion jobs I linked? | 19:43 |
| weshay | it could have been busted prior to that though if it's a race | 19:43 |
| mwhahaha | openstack bot? | 19:43 |
| mwhahaha | is what i was talking about | 19:43 |
| mwhahaha | broke yesterday | 19:43 |
| weshay | oh.. lolz | 19:43 |
| weshay | k | 19:43 |
| weshay | ya.. not seeing merging | 19:44 |
| weshay | I live for that.. aint got no sunshine.. when it rains | 19:44 |
| mwhahaha | we're missing out on all these glorious merges | 19:44 |
| weshay | lolz | 19:44 |
| weshay | exactly | 19:44 |
| *** mwhahaha changes topic to "Welcome to Rocky | CI Status: YELLOWish? (seems to be stable, please be careful) - ovb is sad still | https://docs.openstack.org/tripleo-docs/latest/" | 19:45 | |
| *** rfolco has quit IRC | 19:47 | |
| * mwhahaha ninja vanish | 19:48 | |
| *** jabadia has quit IRC | 20:02 | |
| *** dprince has quit IRC | 20:02 | |
| *** artom has quit IRC | 20:07 | |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792872 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 20:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1792872 in tripleo "[queens] overcloud prepare image failed by giving IronicAction.node.set_provision_state failed: 'NoneType' object has no attribute '__getitem_" [Critical,Triaged] - Assigned to Quique Llorente (quiquell) | 20:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 20:10 |
| *** slaweq has quit IRC | 20:10 | |
| *** slaweq has joined #tripleo | 20:11 | |
| *** med_ has joined #tripleo | 20:16 | |
| *** spartakos has quit IRC | 20:17 | |
| *** boazel_ has quit IRC | 20:27 | |
| *** ansmith has quit IRC | 20:30 | |
| *** boazel has joined #tripleo | 20:32 | |
| *** med_ has quit IRC | 20:33 | |
| *** boazel has quit IRC | 20:40 | |
| *** mcornea has quit IRC | 20:47 | |
| *** raildo has quit IRC | 20:53 | |
| *** spartakos has joined #tripleo | 20:55 | |
| *** Vorrtex has quit IRC | 20:57 | |
| *** asbishop has joined #tripleo | 20:58 | |
| *** abishop_ has quit IRC | 21:01 | |
| *** holser_ has joined #tripleo | 21:04 | |
| *** trown is now known as trown|outtypewww | 21:05 | |
| *** lblanchard has joined #tripleo | 21:05 | |
| *** spartakos has quit IRC | 21:08 | |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 21:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792872 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1792872 in tripleo "[queens] overcloud prepare image failed by giving IronicAction.node.set_provision_state failed: 'NoneType' object has no attribute '__getitem_" [Critical,Triaged] - Assigned to Quique Llorente (quiquell) | 21:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 21:10 |
| *** morazi has quit IRC | 21:16 | |
| *** mjturek has quit IRC | 21:16 | |
| *** bfournie has quit IRC | 21:18 | |
| *** vinaykns has quit IRC | 21:24 | |
| *** spartakos has joined #tripleo | 21:26 | |
| *** holser_ has quit IRC | 21:28 | |
| *** agopi is now known as agopi|brb | 21:30 | |
| *** slaweq has quit IRC | 21:30 | |
| *** asbishop has quit IRC | 21:34 | |
| *** agopi|brb has quit IRC | 21:37 | |
| *** rh-jelabarre has quit IRC | 21:43 | |
| *** slagle has quit IRC | 21:44 | |
| *** jtomasek has quit IRC | 21:54 | |
| *** dciabrin has quit IRC | 22:02 | |
| *** bcafarel has quit IRC | 22:08 | |
| *** bcafarel has joined #tripleo | 22:08 | |
| *** itlinux has quit IRC | 22:09 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 22:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792872 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1792872 in tripleo "[queens] overcloud prepare image failed by giving IronicAction.node.set_provision_state failed: 'NoneType' object has no attribute '__getitem_" [Critical,Triaged] - Assigned to Quique Llorente (quiquell) | 22:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 22:10 |
| *** agopi|brb has joined #tripleo | 22:13 | |
| *** spartakos has quit IRC | 22:17 | |
| *** sai_p has joined #tripleo | 22:19 | |
| *** tosky has quit IRC | 22:24 | |
| *** rcernin has joined #tripleo | 22:25 | |
| *** flaper87 has quit IRC | 22:32 | |
| *** flaper87 has joined #tripleo | 22:33 | |
| *** agopi_ has joined #tripleo | 22:36 | |
| *** agopi|brb has quit IRC | 22:38 | |
| *** agopi__ has joined #tripleo | 22:38 | |
| *** dsneddon has quit IRC | 22:39 | |
| *** dsneddon has joined #tripleo | 22:39 | |
| *** agopi_ has quit IRC | 22:41 | |
| *** agopi__ has quit IRC | 22:47 | |
| *** agopi__ has joined #tripleo | 22:47 | |
| *** agopi__ has quit IRC | 22:58 | |
| *** agopi__ has joined #tripleo | 22:58 | |
| *** spartakos has joined #tripleo | 23:02 | |
| *** agopi__ has quit IRC | 23:06 | |
| *** agopi__ has joined #tripleo | 23:06 | |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792560 | 23:10 |
| openstack | Launchpad bug 1792560 in tripleo "Upgrades in CI still using Q->master instead of R->master release" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1792872 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1795718 | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1792872 in tripleo "[queens] overcloud prepare image failed by giving IronicAction.node.set_provision_state failed: 'NoneType' object has no attribute '__getitem_" [Critical,Triaged] - Assigned to Quique Llorente (quiquell) | 23:10 |
| openstack | Launchpad bug 1795718 in tripleo "Exception: Error inspecting image: docker://docker.io/ server misbehaving" [Critical,Triaged] | 23:10 |
| *** rlandy is now known as rlandy|afk | 23:21 | |
| *** artom has joined #tripleo | 23:30 | |
| *** dtrainor has quit IRC | 23:43 | |
| *** itlinux has joined #tripleo | 23:48 | |
| *** slagle has joined #tripleo | 23:51 | |
| *** jhebden has quit IRC | 23:57 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!