| openstackgerrit | Merged openstack/tripleo-validations master: Add rabbitmq-limits role https://review.openstack.org/640124 | 00:14 |
|---|---|---|
| openstackgerrit | Merged openstack/paunch stable/queens: docker/compose: quote health-cmd https://review.openstack.org/653067 | 00:14 |
| openstackgerrit | Merged openstack/paunch stable/rocky: docker/compose: quote health-cmd https://review.openstack.org/653063 | 00:14 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: Fixed wrong cinder store user name https://review.openstack.org/653047 | 00:14 |
| openstackgerrit | Merged openstack/tripleo-validations master: Add check-latest-minor-version role https://review.openstack.org/640972 | 00:14 |
| *** absubram has quit IRC | 00:16 | |
| *** boazel_ has joined #tripleo | 00:27 | |
| *** jcoufal has joined #tripleo | 00:29 | |
| *** boazel has quit IRC | 00:30 | |
| *** mjturek has quit IRC | 00:32 | |
| *** dxiri has quit IRC | 00:46 | |
| *** morazi has quit IRC | 00:49 | |
| *** jcoufal has quit IRC | 00:55 | |
| *** jcoufal has joined #tripleo | 00:56 | |
| *** jcoufal has quit IRC | 01:07 | |
| *** dsneddon has quit IRC | 01:10 | |
| *** rlandy|ruck has quit IRC | 01:11 | |
| *** hamzy has joined #tripleo | 01:22 | |
| *** threestrands has joined #tripleo | 01:24 | |
| *** dsneddon has joined #tripleo | 01:35 | |
| *** dsneddon has quit IRC | 01:40 | |
| *** apetrich has quit IRC | 01:59 | |
| *** mschuppert has quit IRC | 01:59 | |
| *** itlinux has joined #tripleo | 02:09 | |
| *** boazel has joined #tripleo | 02:10 | |
| *** boazel_ has quit IRC | 02:13 | |
| *** dsneddon has joined #tripleo | 02:29 | |
| *** dsneddon has quit IRC | 02:36 | |
| *** dsneddon has joined #tripleo | 02:49 | |
| *** itlinux has quit IRC | 02:52 | |
| *** itlinux has joined #tripleo | 02:54 | |
| *** dsneddon has quit IRC | 02:56 | |
| *** dsneddon has joined #tripleo | 02:57 | |
| *** dsneddon has quit IRC | 03:05 | |
| *** dsneddon has joined #tripleo | 03:11 | |
| *** psachin has joined #tripleo | 03:12 | |
| *** dsneddon has quit IRC | 03:15 | |
| *** ykarel|away has joined #tripleo | 03:19 | |
| *** itlinux has quit IRC | 03:43 | |
| *** dsneddon has joined #tripleo | 03:49 | |
| *** boazel_ has joined #tripleo | 03:53 | |
| openstackgerrit | yatin proposed openstack-infra/tripleo-ci master: [DNM] Test integrity of container-build job https://review.openstack.org/652508 | 03:53 |
| *** dsneddon has quit IRC | 03:54 | |
| *** boazel has quit IRC | 03:56 | |
| *** lennyb has joined #tripleo | 04:24 | |
| *** ykarel|away has quit IRC | 04:27 | |
| *** dsneddon has joined #tripleo | 04:29 | |
| *** udesale has joined #tripleo | 04:37 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack-infra/tripleo-ci master: Enable ansible-role-collect-logs role in TOCI https://review.openstack.org/652920 | 04:41 |
| *** ykarel|away has joined #tripleo | 04:43 | |
| *** dsneddon has quit IRC | 04:43 | |
| *** ramishra has joined #tripleo | 04:50 | |
| *** ratailor has joined #tripleo | 04:51 | |
| *** ratailor has joined #tripleo | 04:53 | |
| *** absubram has joined #tripleo | 05:01 | |
| openstackgerrit | Merged openstack/tripleo-common master: Inject validation roles path in ansible config https://review.openstack.org/650852 | 05:04 |
| *** absubram has quit IRC | 05:06 | |
| *** absubram has joined #tripleo | 05:10 | |
| *** ykarel|away is now known as ykarel | 05:10 | |
| Tengu | weshay: why so? | 05:33 |
| Tengu | due to the ovn-northd thingy? if so, it would be better to merge the correction -.- | 05:33 |
| Tengu | uhu, another service is down :D. the validation IS working. | 05:35 |
| *** boazel has joined #tripleo | 05:36 | |
| *** dsneddon has joined #tripleo | 05:37 | |
| *** boazel_ has quit IRC | 05:38 | |
| *** marios has joined #tripleo | 05:39 | |
| Tengu | folks, care to make this one merge? https://review.openstack.org/#/c/652980/ :) | 05:42 |
| *** Chaserjim has quit IRC | 05:43 | |
| *** Chaserjim has joined #tripleo | 05:44 | |
| *** dsneddon has quit IRC | 05:44 | |
| *** quiquell|off is now known as quiquell|rover | 05:47 | |
| quiquell|rover | https://review.openstack.org/#/c/652978 | 05:49 |
| Tengu | yeah, that one will help getting info about the failed container :D | 05:49 |
| quiquell|rover | Tengu I am going to test with this | 05:49 |
| *** janki has joined #tripleo | 05:49 | |
| Tengu | quiquell|rover: lemme know how it goes. | 05:49 |
| quiquell|rover | And we will see stuff going on there | 05:49 |
| Tengu | would love to see it merged as well. | 05:50 |
| Tengu | so many +2, and no +w, it's sad. | 05:50 |
| quiquell|rover | I am not allow to +w them | 05:51 |
| Tengu | sad change is sad. | 05:52 |
| Tengu | oh. OH! we can now run the validations provided by tripleo-validations within tripleo-heat-templates \o/ | 05:56 |
| Tengu | my patch adding the related paths to ansible config has merged \o/\o/ | 05:56 |
| Tengu | in-flight validations are WORKING | 05:56 |
| * Tengu does a little victory dance | 05:56 | |
| *** lmiccini has joined #tripleo | 05:59 | |
| *** paramite has joined #tripleo | 05:59 | |
| *** absubram has quit IRC | 06:01 | |
| marios | o/ Tengu so with that we can re-enable the check on fs 56? | 06:02 |
| marios | https://review.openstack.org/#/c/652980/ | 06:02 |
| marios | Tengu: yea looks like it :D fails on the docker-py thing http://logs.openstack.org/80/652980/1/check/tripleo-ci-centos-7-standalone-upgrade/e2128f0/logs/undercloud/home/zuul/standalone_upgrade.log.txt.gz (https://review.openstack.org/#/c/651789 | 06:04 |
| Tengu | marios: yeah, with the new ovn-northd health check we should be able to re-enable on fs056 :) | 06:08 |
| Tengu | thanks for the vote! | 06:08 |
| Tengu | marios: care to +w that one as well? https://review.openstack.org/#/c/652978/ | 06:09 |
| Tengu | that would be really cool to get some more logs from the containers, especially the ones launched by neutron. | 06:09 |
| marios | Tengu: will check in a minute (only has 4 +2 though is it enough?) | 06:09 |
| Tengu | I think it's enough, especially if beagles voted for it | 06:10 |
| marios | ;) | 06:10 |
| Tengu | brb, breakfast. | 06:10 |
| *** yprokule has joined #tripleo | 06:10 | |
| *** pcaruana has joined #tripleo | 06:11 | |
| *** dciabrin_ has quit IRC | 06:14 | |
| openstackgerrit | Luca Miccini proposed openstack/tripleo-ha-utils master: Switch to {{ undercloud_user }} for rsync https://review.openstack.org/651509 | 06:14 |
| *** dciabrin has joined #tripleo | 06:18 | |
| *** dsneddon has joined #tripleo | 06:20 | |
| marios | Tengu: inline question was looking for the logs | 06:22 |
| marios | Tengu: cant find the keepalived one but dnsmasq one empty? is it ok ? http://logs.openstack.org/78/652978/4/check/tripleo-ci-centos-7-containers-multinode/a2a1ba8/logs/undercloud/var/log/containers/stdouts/neutron-dnsmasq-qdhcp-5f4f320e-0022-4c62-8997-f1ef14e8d2a7.log.txt.gz | 06:22 |
| Tengu | marios: well, that's what beagles said in his own comment. might need to tweak the commands directly | 06:23 |
| Tengu | maybe (probably?) some don't output to the stdout. | 06:23 |
| marios | Tengu: ah missed the comment | 06:23 |
| marios | Tengu: and i only found that dnsmasq one not th eothers but i guess depends on the job | 06:24 |
| marios | ? | 06:24 |
| marios | http://logs.openstack.org/78/652978/4/check/tripleo-ci-centos-7-containers-multinode/a2a1ba8/logs/undercloud/var/log/containers/stdouts/ here i mean | 06:24 |
| Tengu | depends on the job, yeah | 06:24 |
| marios | Tengu: k do we want to tweak it now or you happy to merge as is? | 06:25 |
| *** dsneddon has quit IRC | 06:25 | |
| Tengu | marios: we can merge it, and see how we can get some more output as a follow-up, per service | 06:25 |
| Tengu | at least there's the basis for something good with that patch. tweaking the commands is another task, hence another change | 06:26 |
| marios | Tengu: ack | 06:27 |
| Tengu | thank you :) | 06:29 |
| marios | Tengu: np just clicked some buttons | 06:30 |
| Tengu | well, there were important buttons ;) | 06:30 |
| marios | ;) | 06:33 |
| Tengu | hm. *they, even. | 06:33 |
| Tengu | pfff. morning. brain's still booting up. | 06:33 |
| * marios hugs coffee | 06:33 | |
| Tengu | already had one. need to pump it to the brain, taking time. | 06:34 |
| Tengu | like an old diesel. | 06:34 |
| *** udesale has quit IRC | 06:37 | |
| *** udesale has joined #tripleo | 06:38 | |
| *** udesale has quit IRC | 06:39 | |
| *** khyr0n has quit IRC | 06:40 | |
| *** cylopez has joined #tripleo | 06:43 | |
| *** udesale has joined #tripleo | 06:44 | |
| *** holser_ has joined #tripleo | 06:44 | |
| *** dsneddon has joined #tripleo | 06:45 | |
| *** udesale has quit IRC | 06:46 | |
| *** iurygregory has joined #tripleo | 06:49 | |
| *** udesale has joined #tripleo | 06:56 | |
| *** mschuppert has joined #tripleo | 07:03 | |
| *** amoralej|off is now known as amoralej | 07:07 | |
| *** yolanda_ has joined #tripleo | 07:07 | |
| *** kopecmartin|off is now known as kopecmartin | 07:09 | |
| *** avivgt has joined #tripleo | 07:15 | |
| *** boazel_ has joined #tripleo | 07:19 | |
| *** rpittau|afk is now known as rpittau | 07:19 | |
| *** boazel has quit IRC | 07:21 | |
| *** khyr0n has joined #tripleo | 07:25 | |
| *** tosky has joined #tripleo | 07:29 | |
| *** dtantsur|afk is now known as dtantsur | 07:33 | |
| *** quiquell|rover is now known as quique|rover|brb | 07:33 | |
| *** apetrich has joined #tripleo | 07:34 | |
| *** shyamb has joined #tripleo | 07:37 | |
| *** hberaud|gone is now known as hberaud | 07:39 | |
| *** ykarel is now known as ykarel|lunch | 07:43 | |
| *** dsneddon has quit IRC | 07:51 | |
| gchamoul | Tengu: woot! | 07:52 |
| Tengu | gchamoul: for the in-flight? btw, I'll need to push some doc about them - any idea where? | 07:54 |
| gchamoul | Tengu: to me, the in-flight should be somewhere in tripleo-docs | 07:55 |
| Tengu | sure, and probably not that far from the validation framework or related. | 07:55 |
| Tengu | since it can run those very same validations from within the run :) | 07:55 |
| gchamoul | Tengu: indeed | 07:55 |
| Tengu | gchamoul: new file in the install/validations then? | 07:57 |
| gchamoul | Tengu: are you talking about a tripleo-docs place? | 07:58 |
| Tengu | gchamoul: yeah | 07:58 |
| Tengu | there's a doc/sources/install/validations directory. probably can add something in there. | 07:58 |
| gchamoul | Tengu: maybe I need to check though | 07:58 |
| *** jpich has joined #tripleo | 08:01 | |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-validations master: [WIP] Update tripleo-validations documentation https://review.openstack.org/651822 | 08:02 |
| gchamoul | just a rebase ^^ | 08:02 |
| *** xek has joined #tripleo | 08:04 | |
| *** quique|rover|brb is now known as quiquell|rover | 08:05 | |
| *** dsneddon has joined #tripleo | 08:07 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart master: Run novajoin tempest tests in all TLS job using os_tempest https://review.openstack.org/639324 | 08:09 |
| *** dtrainor has quit IRC | 08:09 | |
| *** dtrainor has joined #tripleo | 08:09 | |
| openstackgerrit | Piotr Kopec proposed openstack/tripleo-heat-templates master: Allow NovaNfs parameters to be role specific https://review.openstack.org/651128 | 08:12 |
| *** rcernin has quit IRC | 08:13 | |
| *** tkajinam has quit IRC | 08:17 | |
| *** florianf has joined #tripleo | 08:18 | |
| *** lucasagomes has joined #tripleo | 08:19 | |
| shyamb | HI | 08:21 |
| shyamb | ironic_inspector_dnsmasq container on undercloud is not starting | 08:22 |
| shyamb | I see a bug opened against this issue | 08:22 |
| shyamb | https://bugzilla.redhat.com/show_bug.cgi?id=1629331 | 08:22 |
| openstack | bugzilla.redhat.com bug 1629331 in openstack-tripleo "docker container ironic_inspector_dnsmasq continuously restarts" [High,New] - Assigned to jslagle | 08:22 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-docs master: New documentation for in-flight validations https://review.openstack.org/653350 | 08:22 |
| shyamb | One workaround is given on the bug | 08:22 |
| shyamb | I am not sure how much correct it is | 08:23 |
| shyamb | Tengu: Any thoughts here? | 08:23 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-docs master: New documentation for in-flight validations https://review.openstack.org/653350 | 08:25 |
| *** rcernin has joined #tripleo | 08:27 | |
| *** suuuper has joined #tripleo | 08:28 | |
| *** shyamb has quit IRC | 08:30 | |
| *** ykarel|lunch is now known as ykarel | 08:31 | |
| *** shyamb has joined #tripleo | 08:31 | |
| openstackgerrit | Merged openstack/tripleo-common master: Correct ovn-dbs health check https://review.openstack.org/652980 | 08:32 |
| quiquell|rover | beagles: ping | 08:34 |
| *** udesale has quit IRC | 08:51 | |
| openstackgerrit | Rajesh Tailor proposed openstack/tripleo-heat-templates master: Disable libvirtd service on controller node https://review.openstack.org/653359 | 08:52 |
| *** ccamacho has joined #tripleo | 08:53 | |
| *** boazel has joined #tripleo | 09:01 | |
| *** boazel_ has quit IRC | 09:04 | |
| openstackgerrit | Merged openstack/puppet-tripleo master: Enable file logging for podman neutron sidecars https://review.openstack.org/652978 | 09:06 |
| Tengu | \«o/ | 09:06 |
| *** bogdando has joined #tripleo | 09:11 | |
| *** cylopez has quit IRC | 09:12 | |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-common master: Adding support for the new validation framework https://review.openstack.org/649288 | 09:15 |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-common master: Use 'DEFAULT_VALIDATIONS_BASEDIR' variable from constants.py https://review.openstack.org/653363 | 09:15 |
| *** udesale has joined #tripleo | 09:20 | |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-validations master: Adds roles support in the script generating the validations doc. https://review.openstack.org/651822 | 09:24 |
| *** rascasoft has quit IRC | 09:27 | |
| *** rascasoft has joined #tripleo | 09:28 | |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-validations master: Deletes validations directory https://review.openstack.org/653367 | 09:31 |
| Tengu | numans: heya! do you know anything about the neutron-haproxy-ovn-metadata container? | 09:33 |
| numans | Tengu, I know a bit. But dalvarez knows it well. | 09:34 |
| Tengu | quiquell|rover: -^^ | 09:34 |
| numans | dalvarez, can you please answer Tengu here. | 09:34 |
| Tengu | numans: thanks :) | 09:34 |
| dalvarez | sure, what's the question Tengu ? | 09:34 |
| dalvarez | i can try to answer, not sure if i'll do well :p | 09:35 |
| Tengu | dalvarez: so quiquell|rover and I are trying to understand why the container fails running on fedora, and is exited with a 137 exit code. | 09:35 |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-heat-templates master: Use oslo_messaging_rpc_port for nova rpc healthchecks https://review.openstack.org/652964 | 09:35 |
| quiquell|rover | dalvarez: LP https://bugs.launchpad.net/tripleo/+bug/1824977 | 09:35 |
| openstack | Launchpad bug 1824977 in tripleo "fedora-28 standalone failing at neutron-haproxy-ovnmeta service" [Critical,In progress] | 09:35 |
| quiquell|rover | dalvarez, Tengu: I have still the node at my tenant | 09:35 |
| quiquell|rover | if you want to poke there | 09:36 |
| *** threestrands has quit IRC | 09:36 | |
| Tengu | quiquell|rover: perfect :) I let you check with dalvarez then | 09:36 |
| Tengu | and if we could get some logs from haproxy, that would be really, really helpful. | 09:36 |
| quiquell|rover | Tengu: do we need something else to have those logs ? | 09:36 |
| Tengu | quiquell|rover: quick question: what's the SELinux state on the vm? | 09:37 |
| Tengu | quiquell|rover: since haproxy doesn't write logs, we have to use the /dev/log socket. not sure about the haproxy.cfg though | 09:37 |
| quiquell|rover | looking | 09:37 |
| dalvarez | quiquell|rover: Tengu that's like a very limited amount of info :P i checked the job log... what's in /var/log/extras/failed_containers.log ? | 09:37 |
| quiquell|rover | Tengu: Permissive | 09:37 |
| Tengu | quiquell|rover: :/ so not related to selinux. | 09:38 |
| dalvarez | quiquell|rover: Tengu not sure if it's specific to haproxy-ovnmeta though, but anyways can you check the ovn metadata agent logs? /var/log/containers/neutron... | 09:38 |
| Tengu | dalvarez: if it was easy... ;) | 09:38 |
| quiquell|rover | dalvarez: we don't have anything else the neutron-haproxy-ovnmeta is not dumping anything | 09:38 |
| quiquell|rover | dalvarez: Tengu has try to run it but nothing appears there | 09:38 |
| quiquell|rover | dalvarez: it's only f28 and tempest is passing though | 09:39 |
| dalvarez | Tengu: i shaked my magic wand but nothing popped up :P | 09:39 |
| dalvarez | quiquell|rover: you mean that even with that error tempest is passing? | 09:39 |
| dalvarez | that's odd, shouldnt be it as instances won't fetch metadata without ovn metadata agent running | 09:39 |
| Tengu | dalvarez: I tried to shake my sleeve, nothing either :( | 09:40 |
| dalvarez | lol | 09:40 |
| quiquell|rover | dalvarez: all green here http://logs.openstack.org/66/652066/2/check/tripleo-ci-fedora-28-standalone/3132cc2/logs/undercloud/home/zuul/tempest/tempest.html.gz | 09:40 |
| quiquell|rover | dalvarez: I can give you access to the node | 09:40 |
| dalvarez | quiquell|rover: then there's something else :) or something's already running and binding on the haproxy port | 09:40 |
| dalvarez | quiquell|rover: k i have no much time now but i can try for 10 mins | 09:41 |
| dalvarez | quiquell|rover: let's go to that node and do some tmux | 09:41 |
| dalvarez | i'll bring some coffee there | 09:41 |
| *** suuuper has quit IRC | 09:42 | |
| quiquell|rover | dalvarez: ack thanks! just let know when you have some time | 09:42 |
| dalvarez | quiquell|rover: go go let's go now :) | 09:42 |
| quiquell|rover | dalvarez: tmate or pub key ? | 09:43 |
| dalvarez | quiquell|rover: tmate works for me :) | 09:45 |
| *** shyamb has quit IRC | 09:46 | |
| *** holser_ is now known as holser|lunch | 09:49 | |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart-extras master: Removing test_update_port_with_two_security_groups_and_extra_attributes https://review.openstack.org/653374 | 09:49 |
| *** sshnaidm|afk is now known as sshnaidm | 09:56 | |
| *** florianf has quit IRC | 10:00 | |
| *** dsneddon has quit IRC | 10:02 | |
| quiquell|rover | Tengu: do you know where are the undercloud creds at standalone ? | 10:03 |
| quiquell|rover | Tengu: I mean at zuul home dir | 10:03 |
| quiquell|rover | A ok is at .config | 10:03 |
| dalvarez | quiquell|rover: Tengu numans mistery solved :) https://github.com/openstack/neutron/blob/master/neutron/agent/linux/external_process.py#L98 | 10:04 |
| dalvarez | quiquell|rover: Tengu numans when haproxy sidecar container is no longer needed, the ovn metadata agent kills it, hence you see that 137. It's all good | 10:04 |
| dalvarez | and expected | 10:04 |
| *** suuuper has joined #tripleo | 10:06 | |
| openstackgerrit | Rabi Mishra proposed openstack/puppet-tripleo master: Set octavia provider_drivers config option correctly https://review.openstack.org/653375 | 10:06 |
| quiquell|rover | confirmed and reproduced | 10:08 |
| *** dsneddon has joined #tripleo | 10:08 | |
| quiquell|rover | Tengu: do you know why validate-services is done before tempest ? | 10:09 |
| *** avivgt has quit IRC | 10:13 | |
| *** jchhatbar has joined #tripleo | 10:14 | |
| *** dsneddon has quit IRC | 10:14 | |
| *** janki has quit IRC | 10:16 | |
| *** gkadam has joined #tripleo | 10:16 | |
| *** janki has joined #tripleo | 10:23 | |
| *** jchhatbar has quit IRC | 10:25 | |
| *** dsneddon has joined #tripleo | 10:27 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Add explicit DNS forwarders to TLS job https://review.openstack.org/653174 | 10:31 |
| *** sanjayu_ has quit IRC | 10:31 | |
| sshnaidm | quiquell|rover, Tengu ^^ | 10:31 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] build containers: make tripleo-repos inclusion optional https://review.openstack.org/652990 | 10:31 |
| *** sanjayu_ has joined #tripleo | 10:31 | |
| sshnaidm | jaosorior, https://review.openstack.org/#/c/653174/ | 10:32 |
| quiquell|rover | sshnaidm: +2 let's wait for fs039 result | 10:33 |
| jaosorior | sshnaidm: Added myself to the review; I'll wait for CI | 10:33 |
| *** dsneddon has quit IRC | 10:37 | |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart-extras master: Run validate-services before tempest https://review.openstack.org/653383 | 10:39 |
| quiquell|rover | Tengu: ^ this make any sense ? | 10:39 |
| quiquell|rover | dalvarez: ^ | 10:39 |
| quiquell|rover | Let's add a depends on with revert on deactivation | 10:39 |
| Tengu | dalvarez: oh wow... would be maybe better to actually "CLI stop container"? | 10:41 |
| Tengu | quiquell|rover: no real reason - I pushed that check there, that's all | 10:42 |
| quiquell|rover | Tengu: let's just validate after the deploy | 10:42 |
| Tengu | if it's better suited elsewhere, move it | 10:42 |
| Tengu | OR | 10:42 |
| Tengu | ah, nah | 10:42 |
| Tengu | wait | 10:42 |
| quiquell|rover | ok | 10:42 |
| Tengu | I did that in order to have a "lifetime" span | 10:42 |
| quiquell|rover | we can filter those containers and all | 10:42 |
| Tengu | BUT | 10:42 |
| *** yprokule has quit IRC | 10:42 | |
| Tengu | we can indeed filter out the exit code 137 | 10:42 |
| quiquell|rover | not sure | 10:42 |
| quiquell|rover | we can end with false positives there | 10:42 |
| Tengu | we usually get something else when it crashes. | 10:42 |
| Tengu | hmm. | 10:43 |
| quiquell|rover | It make sense to check after tempest ? | 10:43 |
| Tengu | well, I wanted to ensure we have the state after some time | 10:43 |
| Tengu | i.e. some time to let things crash if they have to | 10:43 |
| quiquell|rover | I have stop the disabling of validate-services on f28 | 10:43 |
| quiquell|rover | from merging | 10:43 |
| Tengu | if you run the validation right after the deploy, it MIGHT happen things are all good, and a moment later, after some use, things crash. | 10:43 |
| *** hberaud is now known as hberaud|lunch | 10:44 | |
| quiquell|rover | y prefer to do validate-services before tempest than not doing at all | 10:44 |
| quiquell|rover | I see | 10:44 |
| Tengu | quiquell|rover: or we can put a filter for "known" container | 10:44 |
| Tengu | like that ovn-metadata thingy | 10:44 |
| quiquell|rover | but we will have to maintain and all | 10:44 |
| *** boazel_ has joined #tripleo | 10:44 | |
| Tengu | right. hm. | 10:44 |
| quiquell|rover | ok I am going to unblock the disabling of f28 | 10:44 |
| quiquell|rover | and we work this out | 10:44 |
| Tengu | the right thing would be to actually stop the container instead of killing its running process | 10:45 |
| dalvarez | Tengu: quiquell|rover neutron code is not aware of containers so it kills the process it's monitoring | 10:45 |
| dalvarez | when it's no longer needed | 10:45 |
| Tengu | dalvarez: hmm. the script that actually start the container has the needed info. | 10:45 |
| Tengu | dalvarez: maybe time to make some changes in order to make neutron self-aware? | 10:46 |
| dalvarez | Tengu: we deploy a wrapper to replace /bin/haproxy with a script that calls podman/docker... | 10:46 |
| quiquell|rover | Tengu: btw validate-services is deactivated at centos-7-standalone | 10:46 |
| dalvarez | Tengu: i dont think so :) why? anyways it can be proposed and discussed in neutron | 10:46 |
| Tengu | dalvarez: that script generates the container name. so we might push that name somehere, and let neutron kill the container instead of the service? | 10:46 |
| dalvarez | Tengu: that's very tripleo specific as we're using podman there but neutron just monitors the processes | 10:47 |
| dalvarez | this is also true not only for haproxy but for keepalived, dnsmasq, etc. | 10:47 |
| *** boazel has quit IRC | 10:47 | |
| Tengu | dalvarez: yep, there are "some" wrappers for that indeed. | 10:48 |
| Tengu | still... that would prevent dangling containers. | 10:48 |
| Tengu | therefore limit disk usage | 10:48 |
| Tengu | and make things cleaner | 10:48 |
| *** avivgt has joined #tripleo | 10:48 | |
| Tengu | dalvarez: using "kill" is fine in a non-container world. maybe a thing like "if <file> exists, execute that file, else kill" | 10:49 |
| Tengu | dalvarez: that would let the "creation wrapper" generate a file with the right content, and neutron will just call that file. | 10:49 |
| Tengu | dalvarez: consider that as a "nice possibility to put hooks in there" | 10:50 |
| dalvarez | Tengu: yeah i'll raise it in our meetings but not sure it'll be accepted :) ie. to make openstack components container-aware | 10:51 |
| dalvarez | Tengu: this is the reason why the sidecar containers were created | 10:51 |
| dalvarez | anyways let's explore it | 10:51 |
| Tengu | dalvarez: maybe not "container-aware", but more "hooks-aware" | 10:52 |
| dalvarez | right, the hook side can be explored, that sounds reasonable | 10:52 |
| dalvarez | Tengu++ | 10:52 |
| Tengu | that's wider than just container if it can execute a script "if script is present". | 10:52 |
| Tengu | thanks :) | 10:52 |
| quiquell|rover | dalvarez: we have a different one related to neutron https://bugs.launchpad.net/tripleo/+bug/1824315 | 10:52 |
| openstack | Launchpad bug 1824315 in tripleo "periodic fedora28 standalone job failing at test_volume_boot_pattern" [Critical,In progress] - Assigned to Quique Llorente (quiquell) | 10:52 |
| quiquell|rover | dalvarez: maybe you can help there too | 10:52 |
| Tengu | dalvarez: do you want me to take part in the meeting? | 10:52 |
| dalvarez | Tengu: or you can start a thread then we can xplore with the hook idea | 10:53 |
| Tengu | dalvarez: on what ML? | 10:54 |
| dalvarez | Tengu: openstack-dev ? | 10:54 |
| Tengu | *openstack-discuss then | 10:54 |
| dalvarez | yeah sry | 10:54 |
| dalvarez | i keep forgetting the rename | 10:54 |
| Tengu | np :). lemme start a thread in there. | 10:54 |
| dalvarez | Tengu: i believe it may be relevant for other components as well | 10:54 |
| dalvarez | cool :) | 10:54 |
| Tengu | guess I have to add [neutron][tripleo] in the topic right? | 10:54 |
| quiquell|rover | dalvarez: forget about last LP was meaning nova | 10:55 |
| dalvarez | Tengu: +1 ... although i think it'd be nice to call other components attention | 10:55 |
| dalvarez | quiquell|rover: yeah i was reading, confused :) | 10:55 |
| quiquell|rover | dalvarez: is for nova bit brain fart | 10:55 |
| dalvarez | Tengu: not sure if there's such wrappers/sidecars for nova, cinder.. etc | 10:55 |
| dalvarez | lol | 10:55 |
| dalvarez | sry folks need to afk | 10:55 |
| Tengu | dalvarez: not sure there are for other than neutron. | 10:55 |
| dalvarez | ack then | 10:56 |
| Tengu | let's start with that one, and we'll see how it goes. | 10:56 |
| Tengu | thanks for your feedback and time! | 10:56 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Disable validations on f28 standalone https://review.openstack.org/653136 | 10:57 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Activate validate-services for standalone https://review.openstack.org/653390 | 10:59 |
| quiquell|rover | Tengu, dalvarez: ^ activating it with depends on running before tempest, let's see how this pan out | 11:00 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-common master: docker-rm: check if rpm dependency is actually installed https://review.openstack.org/651789 | 11:00 |
| marios | Tengu: EmilienM updated ^ | 11:00 |
| *** gkadam is now known as gkadam-afk | 11:06 | |
| Tengu | lemme finish my mail for that new neutron feature: hooks :) | 11:06 |
| marios | Tengu: ack (also updating i think there is nit doing some testing now) | 11:09 |
| marios | Tengu: no actually is ok i think | 11:10 |
| Tengu | marios: lemme check that. | 11:11 |
| marios | Tengu: thanks | 11:11 |
| Tengu | hmm, that might work. Do you have tested that somewhere? | 11:11 |
| marios | Tengu: just locally i played wit it just now seems like the conditions work. otherwise lets see what it does with that standalone-upgrade job :D | 11:13 |
| Tengu | marios: even in "--check" with ansible-playbook? | 11:13 |
| marios | Tengu: didn't try that i was just running those tasks in a local play | 11:13 |
| Tengu | marios: would be good to ensure it's working in --check as well. apparently there are some downstream job playing that, and it might fail. | 11:14 |
| marios | Tengu: ack.. so -check gives warning about using command and error on the py2_docker_installed.rc == 0): 'dict object' has no attribute 'rc | 11:15 |
| Tengu | hehe | 11:16 |
| Tengu | marios: you can add a condition before the py2_docker_installed.rc: py2_docker_installed is changed | 11:16 |
| marios | Tengu: but i don't get it. what does it mean about no attribute rc | 11:16 |
| Tengu | marios: you can debug: var=py2_docker_installed | 11:16 |
| marios | Tengu: hm but i added changed_when: false on that | 11:16 |
| chandankumar | mandre: hey | 11:16 |
| marios | Tengu: because of a linting issue i hit in another task earlier | 11:16 |
| Tengu | you'll see, in that case you won't get the "rc" in the dict, since it's "skipped" | 11:16 |
| chandankumar | mandre: please have a look at this patch https://review.openstack.org/#/c/652937/ little improvement in tempest container | 11:17 |
| Tengu | marios: or "rc in py2_docker_installed" then | 11:17 |
| *** cylopez has joined #tripleo | 11:17 | |
| marios | Tengu: k will try that thanks | 11:17 |
| Tengu | marios: and you'll need that for the other resource using that py2_docker_installed.rc | 11:17 |
| marios | Tengu: yeah was just thinking that | 11:17 |
| Tengu | marios: another way is to disable the calls in check mode, using "check_mode: no" | 11:18 |
| marios | Tengu: i mean its gonna be a bit verbose... | 11:18 |
| Tengu | that's as you want. | 11:18 |
| Tengu | both work well. | 11:18 |
| *** panda is now known as panda|lunch | 11:18 | |
| *** jchhatbar has joined #tripleo | 11:20 | |
| marios | Tengu: added note for now https://review.openstack.org/#/c/651789/5/roles/tripleo-docker-rm/tasks/main.yaml | 11:20 |
| marios | Tengu: to be honest I've never tried that with dry run before merging any other ansible tasks and wasn't aware somethign else is relying on that | 11:21 |
| marios | Tengu: i mean, if this breaks it, then probably lots of other things will too cos i dont think anyone checks that ! | 11:21 |
| marios | Tengu: i can add the check_mode: no there if you think its necessary (-1 it :D) otherwise i wont | 11:22 |
| *** janki has quit IRC | 11:22 | |
| *** holser|lunch is now known as holser_ | 11:26 | |
| *** rh-jelabarre has joined #tripleo | 11:26 | |
| Tengu | marios: hm, can't say for sure if we will hit an issue or not. maybe, maybe not. I'm even unable to tell how ppl are running that "check" for the generated playbooks. | 11:26 |
| Tengu | marios: you can see the issue here: https://bugzilla.redhat.com/show_bug.cgi?id=1697507 | 11:27 |
| openstack | bugzilla.redhat.com bug 1697507 in openstack-tripleo-heat-templates "Forward logging to haproxy.log file task failing in dry-run mode" [High,Modified] - Assigned to cjeanner | 11:27 |
| marios | Tengu: do you recall what/where is checking htat. and do you mean somehwere we actually run with dry-run before executing or something? | 11:27 |
| marios | Tengu: tx looking | 11:27 |
| Tengu | marios: downstream at least. and it's not the "--dry-run" apparently. | 11:27 |
| Tengu | there are some jenkins things running the --check, with plain "ansible-playbook <playbook> --check <other options>" | 11:28 |
| Tengu | can't say more :(. | 11:28 |
| marios | Tengu: ack thanks for pointer | 11:28 |
| marios | Tengu: i thought it was something worse like we are running it in the tripleo code or seomthing | 11:28 |
| Tengu | marios: well, it's actually some promotion blocker downstream | 11:28 |
| openstackgerrit | yatin proposed openstack-infra/tripleo-ci master: Allow to override kolla-build rpm config https://review.openstack.org/653408 | 11:29 |
| ykarel | panda|lunch, ^^ | 11:29 |
| marios | Tengu: the error he has there is the same one i saw .. dict has no attribute rc | 11:29 |
| marios | Tengu: but i don't get why that is a promotion blocker? | 11:29 |
| Tengu | marios: he's a QE working on rhosp-15 integration, and apparently there are QE tests with that kind of run in order to ensure things are working as expected. | 11:30 |
| Tengu | can't say more. you'd rather have a quick chat with Victor if you want details | 11:30 |
| marios | Tengu: ack ok thanks | 11:30 |
| EmilienM | marios++ | 11:30 |
| EmilienM | thanks | 11:31 |
| EmilienM | I'll look the patch shortly | 11:31 |
| marios | EmilienM: ack hoping the standalone-upgrade will be green there now lets see | 11:31 |
| Tengu | marios: added link to the BZ as an answer, for reference. | 11:31 |
| marios | Tengu: thank you | 11:32 |
| *** dsneddon has joined #tripleo | 11:32 | |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-quickstart-extras master: [DNM] Added quickstart-centos-ceph-nautilus repo https://review.openstack.org/652959 | 11:34 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: DNM: test build image in SA https://review.openstack.org/652720 | 11:34 |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-quickstart-extras master: Added quickstart-centos-ceph-nautilus repo https://review.openstack.org/652959 | 11:37 |
| *** dsneddon has quit IRC | 11:38 | |
| *** raildo has joined #tripleo | 11:48 | |
| *** ratailor has quit IRC | 11:48 | |
| *** shyamb has joined #tripleo | 11:48 | |
| *** gkadam-afk is now known as gkadam | 11:51 | |
| Tengu | beagles: heya! can you ping me once you're online? It's about https://review.openstack.org/652985 :). Thanks! | 11:51 |
| *** cylopez has quit IRC | 11:52 | |
| *** morazi has joined #tripleo | 11:52 | |
| beagles | Tengu: sure - I'm semi-nomadic while son is at appointment atm so I might get interrupted | 11:52 |
| beagles | Tengu: neutron-keepalived-state-change only runs as a daemon | 11:53 |
| Tengu | beagles: but without detaching to the background right? | 11:53 |
| beagles | Tengu: but that only means that it cannot run in its own container - so it needs to be exec'ed into something else is all | 11:53 |
| beagles | Tengu: yeah it detaches pretty sure | 11:53 |
| beagles | Tengu: I was just bringing it up to explain why were exec'ing it in the first place is all :) | 11:54 |
| Tengu | beagles: ok. so if it detaches, we don't really need the "--detach" option for podman. hopefully. | 11:54 |
| beagles | Tengu: instead of podman run | 11:54 |
| Tengu | yup | 11:54 |
| beagles | Tengu: yeah fingers crossed | 11:54 |
| Tengu | now I get it. so once I correct the CMD for podman, we'll be clean. | 11:54 |
| Tengu | and we'll be able to actually test that somewhere. | 11:54 |
| beagles | I think so - nice catch | 11:54 |
| Tengu | well, thank you for the review and for catching my own mistake ;)( | 11:55 |
| openstackgerrit | Cédric Jeanneret proposed openstack/puppet-tripleo master: Correct how podman exec is called for the neutron-keepalived-state-change https://review.openstack.org/652985 | 11:56 |
| Tengu | let's go then. | 11:56 |
| beagles | cool thanks for the catch and the fix! | 11:56 |
| Tengu | there are some differences between podman exec and docker exeec. | 11:56 |
| Tengu | hopefully those won't make the world burn. | 11:56 |
| Tengu | beagles: btw, I would love some discussions about "how to get some logs from the sidecars" launched by Neutron. At least the haproxy one doesn't output anything - I didn't check the others.... I guess you wouldn't be against some "-d" flag wherever it's possible right? since it's going to the container stdout... | 11:58 |
| Tengu | as soon as we get things to the stdout, that will be written in a dedicated file | 11:58 |
| beagles | Tengu: dnsmasq doesn't either afaict - logging to stdout might work best for debugging - and something is better than the nothing there is now | 11:59 |
| *** thrash|g0ne is now known as thrash | 11:59 | |
| Tengu | beagles: I'll try to get some "-d" | 12:00 |
| beagles | Tengu: ack | 12:00 |
| Tengu | just some mtg in the way, I'll work on that after it. | 12:01 |
| *** florianf has joined #tripleo | 12:02 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart master: Test install-deps.sh script using pytest/molecule https://review.openstack.org/652971 | 12:04 |
| *** dsneddon has joined #tripleo | 12:06 | |
| *** dtantsur is now known as dtantsur|brb | 12:07 | |
| *** hberaud|lunch is now known as hberaud | 12:08 | |
| *** dsneddon has quit IRC | 12:13 | |
| *** rlandy has joined #tripleo | 12:20 | |
| *** boazel has joined #tripleo | 12:27 | |
| *** rlandy is now known as rlandy|ruck | 12:27 | |
| *** boazel_ has quit IRC | 12:29 | |
| *** pcaruana has quit IRC | 12:30 | |
| *** panda|lunch is now known as panda | 12:32 | |
| *** shyamb has quit IRC | 12:37 | |
| bogdando | Tengu there is debug parameter may be a proper switching trigger for advanced stdout logs | 12:40 |
| *** artom has quit IRC | 12:42 | |
| ccamacho | mwhahaha o/ | 12:42 |
| ccamacho | :) quick question | 12:43 |
| ccamacho | about https://review.openstack.org/#/c/641516/ | 12:43 |
| ccamacho | we started to hit the issue downstream when testing 13->14 | 12:43 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart master: Run novajoin tempest tests in all TLS job using os_tempest https://review.openstack.org/639324 | 12:43 |
| ccamacho | do we have BZs for tracking the fix?! | 12:43 |
| ccamacho | I cant find any reference to a BZ there | 12:43 |
| ccamacho | lbezdick ^ | 12:44 |
| *** social has joined #tripleo | 12:45 | |
| Tengu | bogdando: for haproxy, yeah. but not for dnsmasq. that one is a monster =.= | 12:46 |
| bogdando | Tengu: yeah, just saying let's control that behavior to not make this excessive logging as a default behavior :) | 12:47 |
| *** mcornea has joined #tripleo | 12:47 | |
| Tengu | bogdando: hmm yeah. have to check how to get that into puppet-tripleo, since wrappers are generated there. | 12:48 |
| Tengu | THAT said... we might configure haproxy to actually push its thing to /dev/stdout | 12:49 |
| bogdando | Tengu: some examples https://review.openstack.org/#/c/651505/6/templates/neutron/dnsmasq.epp@50 | 12:49 |
| Tengu | so that we will get its standard logs without debug. | 12:49 |
| Tengu | bogdando: yeah, well, we can forget about dnsmasq being polite anyway :/ | 12:50 |
| bogdando | Tengu: yeah, especially given that removing podman run -a flag in units would render all syslog logging efforts for podman futile | 12:50 |
| bogdando | would it btw?.. | 12:50 |
| * bogdando thinks of disastrous UX regressions... | 12:50 | |
| Tengu | bogdando: hmmm well, stdout (aka old journald) is already written on the disk | 12:51 |
| Tengu | so it's not THAT bad | 12:51 |
| *** dsneddon has joined #tripleo | 12:51 | |
| Tengu | but indeed, journalctl -u tripleo_foo won't output much | 12:51 |
| bogdando | Tengu: yes, I was just suddenly stunned by that questions, would /dev/log we pass for all containers take any effect when executed via podman run w/o -a | 12:51 |
| Tengu | bogdando: no reasons it wouldn't. | 12:52 |
| bogdando | well. hope so | 12:52 |
| Tengu | the -a attaches the container stdout to the systemd process | 12:52 |
| Tengu | while /dev/log is a standard socket | 12:52 |
| bogdando | it also takes care of journald integration | 12:52 |
| Tengu | no real reasons the -a will change that. | 12:52 |
| Tengu | nope | 12:52 |
| Tengu | well, yes, but not with /dev/log | 12:53 |
| Tengu | same goal, different ways. | 12:53 |
| *** pcaruana has joined #tripleo | 12:53 | |
| bogdando | Tengu: ack | 12:54 |
| bogdando | then it's fine | 12:54 |
| Tengu | yup | 12:54 |
| Tengu | the -a is compensated by the --log-driver and stuff | 12:55 |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-heat-templates master: Avoid concurrent nova cell_v2 discovery instances https://review.openstack.org/652039 | 12:55 |
| *** dsneddon has quit IRC | 12:57 | |
| *** shyamb has joined #tripleo | 12:58 | |
| *** amoralej is now known as amoralej|lunch | 13:01 | |
| *** Goneri has joined #tripleo | 13:05 | |
| *** baha has joined #tripleo | 13:06 | |
| *** dtantsur|brb is now known as dtantsur | 13:07 | |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-heat-templates master: Avoid concurrent nova cell_v2 discovery instances https://review.openstack.org/652039 | 13:13 |
| mwhahaha | ccamacho: you are using the wrong containers | 13:19 |
| mwhahaha | ccamacho: see https://bugzilla.redhat.com/show_bug.cgi?id=1700096 | 13:20 |
| openstack | bugzilla.redhat.com bug 1700096 in openstack-tripleo-common "Mistral version mismatch between system libs and tripleo container" [High,Closed: notabug] - Assigned to apetrich | 13:20 |
| *** mjturek has joined #tripleo | 13:22 | |
| *** shardy has joined #tripleo | 13:23 | |
| *** dsneddon has joined #tripleo | 13:26 | |
| *** Vorrtex has joined #tripleo | 13:27 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-docs master: Clarify DCN requirements for provider networks https://review.openstack.org/652065 | 13:29 |
| *** tzumainn has joined #tripleo | 13:30 | |
| *** dsneddon has quit IRC | 13:32 | |
| ccamacho | thanks mwhahaha | 13:34 |
| *** ykarel is now known as ykarel|afk | 13:36 | |
| *** ykarel|afk has quit IRC | 13:41 | |
| *** vinaykns has joined #tripleo | 13:45 | |
| *** artom has joined #tripleo | 13:53 | |
| *** quiquell|rover has quit IRC | 14:00 | |
| *** quiquell has joined #tripleo | 14:00 | |
| *** quiquell is now known as quiquell|off | 14:00 | |
| *** amoralej|lunch is now known as amoralej | 14:06 | |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-common master: Use 'DEFAULT_VALIDATIONS_BASEDIR' variable from constants.py https://review.openstack.org/653363 | 14:07 |
| openstackgerrit | Marios Andreou proposed openstack-infra/tripleo-ci master: Add new base-standalone-upgrade for use in periodic https://review.openstack.org/653440 | 14:09 |
| *** boazel_ has joined #tripleo | 14:10 | |
| sshnaidm | rlandy|ruck, can we merge it? https://review.openstack.org/#/c/653174 039 job fails for different reason now, dns resolving is ok | 14:11 |
| sshnaidm | also it passed in previous run :( | 14:12 |
| *** aakarsh has quit IRC | 14:12 | |
| rlandy|ruck | sshnaidm; lol - was just goimg to say still failing | 14:12 |
| *** boazel has quit IRC | 14:12 | |
| rlandy|ruck | sshnaidm: k - let's merge this as it gets the job further at least | 14:14 |
| *** itlinux has joined #tripleo | 14:19 | |
| *** dsneddon has joined #tripleo | 14:22 | |
| *** itlinux has quit IRC | 14:22 | |
| *** ykarel|afk has joined #tripleo | 14:24 | |
| openstackgerrit | Martin Schuppert proposed openstack/tripleo-heat-templates master: Avoid concurrent nova cell_v2 discovery instances https://review.openstack.org/652039 | 14:25 |
| *** mjturek has quit IRC | 14:25 | |
| *** ykarel|afk is now known as ykarel | 14:25 | |
| *** dsneddon has quit IRC | 14:28 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add OS::TripleO::NovaAZConfig https://review.openstack.org/644903 | 14:29 |
| *** mjturek has joined #tripleo | 14:29 | |
| *** dsneddon has joined #tripleo | 14:30 | |
| *** aakarsh has joined #tripleo | 14:32 | |
| openstackgerrit | Merged openstack/tripleo-ha-utils master: Switch to {{ undercloud_user }} for rsync https://review.openstack.org/651509 | 14:34 |
| *** dsneddon has quit IRC | 14:36 | |
| *** iurygregory has quit IRC | 14:49 | |
| *** jchhatbar has quit IRC | 14:53 | |
| *** jchhatbar has joined #tripleo | 14:54 | |
| *** iurygregory has joined #tripleo | 14:54 | |
| *** weshay is now known as weshay|rover | 14:57 | |
| *** udesale has quit IRC | 14:57 | |
| *** mcornea has quit IRC | 14:59 | |
| *** udesale has joined #tripleo | 14:59 | |
| *** udesale has quit IRC | 14:59 | |
| Tengu | oh. ohooo | 14:59 |
| *** udesale has joined #tripleo | 14:59 | |
| Tengu | I just remember why we don't see the haproxy logs from the sidecars :D | 14:59 |
| Tengu | we do mount the host /dev/log inside the container - but the syslog configuration is set to ignore them. | 15:00 |
| Tengu | Guess we can do something nice and clean, like updating the local0 to something else in order to allow to get a dedicated file in the right location. | 15:00 |
| Tengu | that would already help a lot. | 15:00 |
| mwhahaha | pffft logs who needs them | 15:01 |
| Tengu | more over..... maybe some tweaks can be done in the haproxy config. | 15:01 |
| Tengu | mwhahaha: I know right ;). logs are like metrics: useless | 15:01 |
| Tengu | that's why we have to get them XD | 15:01 |
| mwhahaha | if you can't see the error, is it really and error? | 15:02 |
| Tengu | that's why I'm not happy when CI folks are removing my service validation thing (*stares at weshay|rover*) | 15:02 |
| Tengu | ostrich policy is bad ;) | 15:03 |
| Tengu | anyway. | 15:03 |
| *** mcornea has joined #tripleo | 15:03 | |
| Tengu | let's get something nice and neat. | 15:03 |
| *** udesale has quit IRC | 15:06 | |
| *** dsneddon has joined #tripleo | 15:07 | |
| *** dsneddon has quit IRC | 15:14 | |
| *** absubram has joined #tripleo | 15:17 | |
| *** absubram has quit IRC | 15:21 | |
| Tengu | hmm. | 15:23 |
| Tengu | beagles, dalvarez : quick question: what about allowing the neutron "haproxy" metadata agent get a new param, "log_facilty", instead of the hard-coded "local0" we have? That would allow a nice separation at syslog level, meaning "dedicated haproxy file" for that metadata agent. | 15:24 |
| Tengu | we can set that parameter default value to "local0" in order to avoid breaking things, but... | 15:25 |
| *** dsneddon has joined #tripleo | 15:25 | |
| Tengu | in tripleo, it would be neat to get a different value in order to differenciate that haproxy instance from the standard frontend. | 15:25 |
| *** absubram has joined #tripleo | 15:29 | |
| *** dsneddon has quit IRC | 15:32 | |
| *** jchhatbar has quit IRC | 15:33 | |
| *** jchhatbar has joined #tripleo | 15:33 | |
| *** itlinux has joined #tripleo | 15:35 | |
| *** jchhatbar has quit IRC | 15:36 | |
| *** jchhatbar has joined #tripleo | 15:36 | |
| *** jchhatbar has quit IRC | 15:40 | |
| *** sanjayu_ has quit IRC | 15:45 | |
| *** boazel has joined #tripleo | 15:53 | |
| *** iurygregory has quit IRC | 15:53 | |
| *** boazel_ has quit IRC | 15:56 | |
| *** bogdando has quit IRC | 16:01 | |
| *** holser_ has quit IRC | 16:05 | |
| *** ykarel is now known as ykarel|away | 16:05 | |
| openstackgerrit | Rabi Mishra proposed openstack/puppet-tripleo master: Set octavia provider_drivers config option correctly https://review.openstack.org/653375 | 16:05 |
| *** shyamb has quit IRC | 16:06 | |
| *** baha has quit IRC | 16:06 | |
| openstackgerrit | Marios Andreou proposed openstack-infra/tripleo-ci master: commenting emit-releases.py for maintainability https://review.openstack.org/652095 | 16:06 |
| *** suuuper has quit IRC | 16:09 | |
| *** lucasagomes has quit IRC | 16:09 | |
| beagles | Tengu: those config files are generated by neutron, correct? We'd have to either edit the cfg in place or submit a patch to neutron. | 16:11 |
| beagles | Tengu: I don't disagree though | 16:11 |
| beagles | Tengu: one thing we need think about is the lifecyles of networks/routers etc | 16:12 |
| beagles | Tengu: and when are the logs going to get cleaned up | 16:12 |
| *** ccamacho has quit IRC | 16:16 | |
| Tengu | beagles: yeah, generated within neutron, already checked where/how. | 16:17 |
| *** marios has quit IRC | 16:18 | |
| Tengu | beagles: regarding the logs: logrotate can take care of them eventually. Or we can tweak some tmpwatch call just for those logs. | 16:18 |
| beagles | Tengu: ack | 16:18 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add Etcd to DistributedCompute roles https://review.openstack.org/647494 | 16:19 |
| *** jpich has quit IRC | 16:20 | |
| *** rpittau is now known as rpittau|afk | 16:20 | |
| Tengu | beagles: I can try to provide a WIP patch tomorrow, but I'll need some help I think :). | 16:22 |
| Tengu | anyway. signing-off for real now :). See you tomorrow! | 16:23 |
| *** dtantsur is now known as dtantsur|afk | 16:24 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add CinderEtcdLocalConnect parameter https://review.openstack.org/647495 | 16:25 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Handle any network name in ServiceNetMap https://review.openstack.org/651401 | 16:25 |
| *** ramishra has quit IRC | 16:26 | |
| *** baha has joined #tripleo | 16:32 | |
| *** dsneddon has joined #tripleo | 16:32 | |
| *** florianf has quit IRC | 16:35 | |
| beagles | Tengu: cheers | 16:37 |
| *** dsneddon has quit IRC | 16:40 | |
| *** Chaserjim has quit IRC | 16:43 | |
| *** Chaserjim has joined #tripleo | 16:44 | |
| *** khyr0n has quit IRC | 16:47 | |
| *** gkadam has quit IRC | 16:57 | |
| *** hberaud is now known as hberaud|gone | 17:02 | |
| *** avivgt has quit IRC | 17:03 | |
| *** kopecmartin is now known as kopecmartin|off | 17:05 | |
| *** dsneddon has joined #tripleo | 17:08 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Remove ProcessTemplatesAction as base class https://review.openstack.org/599188 | 17:11 |
| *** dsneddon has quit IRC | 17:14 | |
| openstackgerrit | Merged openstack/puppet-tripleo master: Extend fencing to hosts using fence_rhevm agent. https://review.openstack.org/652019 | 17:14 |
| openstackgerrit | Merged openstack/tripleo-common master: Adding support for the new validation framework https://review.openstack.org/649288 | 17:15 |
| *** mcornea has quit IRC | 17:25 | |
| openstackgerrit | Piotr Kopec proposed openstack/tripleo-heat-templates stable/rocky: Allow NovaRbdPoolName to be role specific https://review.openstack.org/653496 | 17:26 |
| *** aakarsh has quit IRC | 17:27 | |
| *** krypto has joined #tripleo | 17:28 | |
| *** aakarsh has joined #tripleo | 17:33 | |
| krypto | hi all i am trying to add a new node to the stack,but its failing because of an existing node.I dont see any commands being executed on that node there is nothing on "/var/log/os-apply-config.log" ,while on other nodes i can see some changes in log files | 17:34 |
| *** dsneddon has joined #tripleo | 17:34 | |
| krypto | is there an agent in tripleo that runs on all nodes | 17:34 |
| krypto | last message during stack update after that it fails after timeout | 17:35 |
| krypto | 2019-04-17 17:14:48Z [ControllerIpListMap]: UPDATE_COMPLETE state changed | 17:35 |
| krypto | 2019-04-17 17:14:48Z [AllNodesValidationConfig]: UPDATE_COMPLETE state changed | 17:35 |
| *** boazel_ has joined #tripleo | 17:35 | |
| krypto | after posting here i noticed that os-collect-config is not running on old node | 17:37 |
| *** mcornea has joined #tripleo | 17:38 | |
| *** boazel has quit IRC | 17:38 | |
| *** khyr0n has joined #tripleo | 17:48 | |
| *** arxcruz is now known as arxcruz|off|23 | 17:53 | |
| openstackgerrit | Piotr Kopec proposed openstack/tripleo-heat-templates stable/queens: Allow NovaRbdPoolName to be role specific https://review.openstack.org/653502 | 17:54 |
| mwhahaha | krypto: what version | 18:02 |
| mwhahaha | and what task times out | 18:02 |
| *** amoralej is now known as amoralej|off | 18:03 | |
| *** Goneri has quit IRC | 18:06 | |
| *** morazi has quit IRC | 18:07 | |
| krypto | mwhahaha Newton ...this is a lab environment simulating production | 18:07 |
| mwhahaha | what does os-collect-config show in the journal | 18:08 |
| krypto | this is the last message | 18:08 |
| krypto | 2019-04-17 17:14:48Z [AllNodesValidationConfig]: UPDATE_COMPLETE state changed | 18:08 |
| krypto | Stack overcloud UPDATE_FAILED | 18:08 |
| krypto | Heat Stack update failed. | 18:08 |
| krypto | usr/bin/python /usr/bin/os-collect-config | 18:09 |
| krypto | Source [request] Unavailable. | 18:09 |
| krypto | No local metadata found (['/var/lib/os-collect-config/local-data']) | 18:09 |
| krypto | No auth_url configured. | 18:09 |
| krypto | { | 18:09 |
| openstackgerrit | Rafael Folco proposed openstack-infra/tripleo-ci master: DNM: Test push containers for buildah jobs https://review.openstack.org/652126 | 18:09 |
| mwhahaha | yea that message is always there | 18:09 |
| krypto | when i run it manually it exits with above messages | 18:09 |
| krypto | ok | 18:09 |
| mwhahaha | i think | 18:09 |
| mwhahaha | you might check the heat api logs | 18:09 |
| mwhahaha | make sure it's not erroring | 18:09 |
| krypto | on other nodes its running fine with this message"/var/lib/os-collect-config/local-data not found. Skipping | 18:10 |
| krypto | No local metadata found (['/var/lib/os-collect-config/local-data'])" | 18:10 |
| krypto | atleast the service is running fine | 18:10 |
| *** dmacpher_ has joined #tripleo | 18:11 | |
| *** psachin has quit IRC | 18:13 | |
| *** dmacpher has quit IRC | 18:13 | |
| *** saneax has joined #tripleo | 18:19 | |
| *** rfolco has quit IRC | 18:20 | |
| *** Goneri has joined #tripleo | 18:21 | |
| *** rfolco has joined #tripleo | 18:21 | |
| *** dtrainor has quit IRC | 18:51 | |
| openstackgerrit | Nagasai Vinaykumar Kapalavai proposed openstack/puppet-tripleo master: Configuration changes to support Qdr-mesh topology. https://review.openstack.org/647908 | 18:53 |
| *** saneax has quit IRC | 18:58 | |
| openstackgerrit | Nagasai Vinaykumar Kapalavai proposed openstack/tripleo-heat-templates master: Connect Qdrs to SAF (Server Side) https://review.openstack.org/647910 | 18:58 |
| *** ykarel|away has quit IRC | 19:02 | |
| *** raildo has quit IRC | 19:05 | |
| *** holser_ has joined #tripleo | 19:12 | |
| *** dtrainor has joined #tripleo | 19:16 | |
| *** boazel has joined #tripleo | 19:18 | |
| *** raildo has joined #tripleo | 19:20 | |
| *** boazel_ has quit IRC | 19:21 | |
| *** jcoufal has joined #tripleo | 19:21 | |
| *** Goneri has quit IRC | 19:21 | |
| *** artom has quit IRC | 19:22 | |
| *** khyr0n has quit IRC | 19:26 | |
| krypto | mwhahaha i dont see anything strange in heat api logs ,when i run python /usr/bin/os-collect-config it exits and doesnt stay up | 19:34 |
| *** Goneri has joined #tripleo | 19:34 | |
| krypto | Source [request] Unavailable. | 19:35 |
| mwhahaha | yea it runs periodically | 19:36 |
| krypto | this file is also empty /etc/os-collect-config.conf | 19:39 |
| krypto | i guess it happened because during stack update nova service on director was not responding and metadata was not reachable | 19:40 |
| *** rlandy|ruck is now known as rlandy|afk | 19:46 | |
| krypto | is there a way to regenerate /etc/os-net-config/config.json and /etc/os-collect-config.conf | 19:49 |
| *** dciabrin_ has joined #tripleo | 19:54 | |
| *** dciabrin has quit IRC | 19:55 | |
| *** raildo has quit IRC | 20:02 | |
| openstackgerrit | Martin Mágr proposed openstack/puppet-tripleo master: Switch metrics qdrs to edge mode https://review.openstack.org/653547 | 20:09 |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient master: WIP: Speed up plan operations by using a tarball in swift. https://review.openstack.org/581141 | 20:12 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: WIP: Use process-templates.py and cache templates to speed up plan ops. https://review.openstack.org/581153 | 20:18 |
| openstackgerrit | Rafael Folco proposed openstack-infra/tripleo-ci master: DNM: Test push containers for buildah jobs https://review.openstack.org/652126 | 20:20 |
| *** mjturek has quit IRC | 20:20 | |
| *** jcoufal has quit IRC | 20:22 | |
| openstackgerrit | Rafael Folco proposed openstack-infra/tripleo-ci master: DNM: Test push containers for buildah jobs https://review.openstack.org/652126 | 20:24 |
| *** jcoufal has joined #tripleo | 20:30 | |
| *** snecklifter has joined #tripleo | 20:30 | |
| snecklifter | launchpad broken for anyone else or just me? | 20:30 |
| *** absubram has quit IRC | 20:35 | |
| *** jcoufal has quit IRC | 20:39 | |
| *** Goneri has quit IRC | 20:40 | |
| *** raildo has joined #tripleo | 20:40 | |
| *** pcaruana has quit IRC | 20:43 | |
| *** xek has quit IRC | 20:43 | |
| *** shardy has quit IRC | 20:45 | |
| *** holser_ has quit IRC | 20:54 | |
| dsneddon | krypto, I'm not aware of any way to generate /etc/os-net-config/config.json and /etc/os-collect-config.conf without doing a stack update. | 20:55 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Evaluating ansible_check_mode as a boolean https://review.openstack.org/653554 | 20:55 |
| dsneddon | krypto, Do you see anything on the logs on the host about os-net-config? I wonder if the /etc/os-net-config/config.json was empty because of a failure. I would grep /var/log/messages for "os-net-config" or "network_config" and see if you have logs about whether the networking is at fault. | 20:57 |
| krypto | dsneddon heat stack-update was ignoring this node and there was no new logs in /var/log/os-apply-config.log probably because os-collect was not running.Finally i copied /etc/os-net-config/config.json from another node (changed IP) and copied /etc/os-collect-config.conf from another node (changed metdata values) and its now service is running | 20:58 |
| krypto | dsneddon you are right after reboot this node had only pxe nic ... in what case would os-collect remove interface configuration | 20:59 |
| *** boazel_ has joined #tripleo | 21:01 | |
| ade_lee | EmilienM, mwhahaha - hey guys, any tips on debugging an failing overcloud deploy where I'm getting a ValueError Value must be valid JSON: | 21:01 |
| *** snecklifter has quit IRC | 21:01 | |
| ade_lee | happening at the beginning of the deloyment after the pre-deploy validations | 21:01 |
| ade_lee | unfortunately, it doesn't tell me what json is broken .. | 21:02 |
| mwhahaha | would need the whole output | 21:02 |
| mwhahaha | that's a generic error | 21:02 |
| ade_lee | other than: ValueError: Value must be valid JSON: Extra data: line 1 column 6 - line 1 column 14 (char 5 - 13) | 21:02 |
| mwhahaha | need the surrounding lines, that doesn't mean anything other than some json is bad | 21:03 |
| ade_lee | mwhahaha, can give you a tmate if you can jump on and see .. | 21:03 |
| mwhahaha | sure | 21:03 |
| *** boazel has quit IRC | 21:04 | |
| *** rh-jelabarre has quit IRC | 21:09 | |
| *** rh-jelabarre has joined #tripleo | 21:09 | |
| *** slaweq has quit IRC | 21:17 | |
| *** slaweq has joined #tripleo | 21:20 | |
| *** slaweq has quit IRC | 21:24 | |
| *** Goneri has joined #tripleo | 21:29 | |
| *** mcornea has quit IRC | 21:35 | |
| *** Goneri has quit IRC | 21:36 | |
| *** Vorrtex has quit IRC | 21:38 | |
| *** absubram has joined #tripleo | 21:46 | |
| *** baha has quit IRC | 21:46 | |
| dsneddon | krypto, If os-net-config fails, it will fall back to a configuration where every NIC gets set to DHCP. And since only the PXE NIC probably has a DHCP server on the network, you only see that NIC active. | 21:50 |
| krypto | Thanks dsneddon that was the issue with that node | 21:51 |
| dsneddon | krypto, Which version are you deploying? You probably want to have "run-os-net-config.sh" calling os-net-config, not os-apply-config. If you see "os-apply-config" in your NIC config templates, I can guide you to make those changes. | 21:51 |
| krypto | newton | 21:53 |
| krypto | after i started os-net-collect it applied all the configuration and node is functional now ,however deployment failed | 21:54 |
| *** Goneri has joined #tripleo | 21:55 | |
| dsneddon | krypto, OK, I think you are doing the right thing for Newton, we still relied on os-collect-config to run the network configuration. | 21:57 |
| krypto | i am still learning triple-O ,the guy who deployed this lab left our company... | 21:58 |
| krypto | can you help me understand what this error means openstack stack failures list overcloud --long | 21:59 |
| krypto | overcloud.ComputeSriovDpdkAllNodesValidationDeployment.9: | 21:59 |
| krypto | resource_type: OS::Heat::StructuredDeployment | 21:59 |
| krypto | physical_resource_id: ff40d821-5439-43eb-a2e5-93d74db2d825 | 21:59 |
| krypto | status: CREATE_FAILED | 21:59 |
| krypto | status_reason: | | 21:59 |
| krypto | CREATE aborted | 21:59 |
| krypto | deploy_stdout: | | 21:59 |
| krypto | None | 21:59 |
| krypto | deploy_stderr: | | 21:59 |
| krypto | None | 21:59 |
| mwhahaha | sounds like validation issues | 21:59 |
| mwhahaha | i think that's where it does connectivity checks | 22:01 |
| krypto | is there a log i can look to catch these errors https://paste.ubuntu.com/p/v49CMNGBRz/ | 22:01 |
| openstackgerrit | Nagasai Vinaykumar Kapalavai proposed openstack/tripleo-heat-templates stable/rocky: Update to the ceilometer publisher list https://review.openstack.org/653144 | 22:01 |
| mwhahaha | the heat engine log might have something | 22:04 |
| mwhahaha | not completely sure | 22:04 |
| krypto | i havent enabled debug in heat possibly because of that ...i couldnt find anything in heat-engine.log | 22:06 |
| mwhahaha | in newton you could query heat for teh actual errors usually | 22:06 |
| *** rh-jelabarre has quit IRC | 22:08 | |
| *** boazel has joined #tripleo | 22:08 | |
| mwhahaha | krypto: https://docs.openstack.org/tripleo-docs/latest/install/troubleshooting/troubleshooting-overcloud.html#debugging-using-heat | 22:09 |
| *** boazel_ has quit IRC | 22:11 | |
| krypto | Thanks i wil check it | 22:21 |
| openstackgerrit | Merged openstack/python-tripleoclient stable/rocky: Allow 'container image prepare' to support drivers https://review.openstack.org/652934 | 22:26 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: TLS everywhere: switch Octavia to use DNS entries https://review.openstack.org/649333 | 22:26 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: Be able to know when we are running inside a minor update workflow https://review.openstack.org/651114 | 22:26 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: Do not restart bundles during a minor update https://review.openstack.org/651115 | 22:26 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: Run octavia-api under httpd https://review.openstack.org/651577 | 22:26 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: Bind mount undercloud.conf with ,z in mistral_executor https://review.openstack.org/651701 | 22:26 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: Increase default ulimit values for Neutron agents containers https://review.openstack.org/651709 | 22:26 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: ceilometer_agent_notification: disable-panko.yaml https://review.openstack.org/652704 | 22:26 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: ceilometer_agent_notification: disable-panko.yaml https://review.openstack.org/652702 | 22:26 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: Increase default ulimit values for Neutron agents containers https://review.openstack.org/651710 | 22:26 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: Be able to know when we are running inside a minor update workflow https://review.openstack.org/651181 | 22:26 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: Do not restart bundles during a minor update https://review.openstack.org/651182 | 22:26 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: Enable deep_compare of pcmk resources by default https://review.openstack.org/651110 | 22:26 |
| openstackgerrit | Merged openstack/puppet-tripleo stable/queens: Handle image_volume_cache_enabled in PureStorage Cinder backend. https://review.openstack.org/649159 | 22:26 |
| openstackgerrit | Merged openstack/tripleo-common master: Uncap jsonschema https://review.openstack.org/649805 | 22:26 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Fix CI ipv6 NIC config default route https://review.openstack.org/649524 | 22:27 |
| openstackgerrit | Nagasai Vinaykumar Kapalavai proposed openstack/tripleo-heat-templates stable/queens: Update to the ceilometer publisher list https://review.openstack.org/651630 | 22:30 |
| *** paramite has quit IRC | 22:30 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Scale-down tasks for RHSM https://review.openstack.org/651959 | 22:38 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: WIP - Introduce scale_tasks https://review.openstack.org/651507 | 22:38 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Scale-down tasks for RHSM https://review.openstack.org/651959 | 22:39 |
| *** Goneri has quit IRC | 22:40 | |
| openstackgerrit | Emilien Macchi proposed openstack/ansible-role-redhat-subscription master: unregister: use command to remove the subscription https://review.openstack.org/653620 | 22:42 |
| *** aakarsh has quit IRC | 22:49 | |
| *** tkajinam has joined #tripleo | 22:54 | |
| gouthamr | x-posting from #rdo: | 23:00 |
| gouthamr | hi folks, i'm looking for a stable/rocky backport to kolla to be available in a container image | 23:01 |
| gouthamr | backport in question (https://review.openstack.org/#/c/651368/) merged 3 days ago | 23:01 |
| gouthamr | i looked here: https://dashboards.rdoproject.org/rocky and if i understand this correctly, it will take longer for getting newer container images promoted? | 23:01 |
| *** vinaykns has quit IRC | 23:03 | |
| *** rlandy|afk is now known as rlandy|ruck | 23:03 | |
| gouthamr | unsure of the cadence here, any pointers appreciated :) | 23:05 |
| *** raildo has quit IRC | 23:07 | |
| EmilienM | weshay|rover: FYI https://review.openstack.org/#/c/652758/ | 23:07 |
| EmilienM | weshay|rover: we're going to branch, probably tomorrow | 23:07 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Remove puppet selinux management https://review.openstack.org/653625 | 23:11 |
| weshay|rover | EmilienM ah.. thanks | 23:12 |
| weshay|rover | rfolco rlandy|ruck HEADS UP | 23:12 |
| rlandy|ruck | k - thanks for the advanced warning | 23:12 |
| weshay|rover | it's like the first time :P | 23:13 |
| weshay|rover | only took EmilienM 10 yrs | 23:13 |
| * weshay|rover runs | 23:13 | |
| EmilienM | 18? wow I got an upgrade | 23:13 |
| openstackgerrit | Alex Schultz proposed openstack/puppet-tripleo master: Remove puppet selinux management https://review.openstack.org/653627 | 23:14 |
| * mwhahaha rm -rfs all thet hings | 23:14 | |
| EmilienM | YES to that | 23:15 |
| mwhahaha | EmilienM: btw placement is a Train thing | 23:16 |
| mwhahaha | nova is planning on supporting it for stein still i think | 23:16 |
| mwhahaha | if i recall the ML posts | 23:16 |
| EmilienM | mwhahaha: right but they plan to remove it after PTG I've heard | 23:16 |
| EmilienM | which will break our master jobs in promotion | 23:16 |
| EmilienM | as the placement integration isn't finished at all | 23:17 |
| EmilienM | still under review and failing heavily | 23:17 |
| EmilienM | so yeah kind of worried | 23:17 |
| mwhahaha | horible breakages in m1 | 23:17 |
| EmilienM | yep | 23:17 |
| EmilienM | anyway I'm out | 23:17 |
| mwhahaha | can't be worse than the other twelve times they've done it | 23:17 |
| * mwhahaha runs away | 23:17 | |
| EmilienM | lol that was a valid troll | 23:18 |
| weshay|rover | FUN | 23:19 |
| weshay|rover | we don't need no stinking jobs in nova | 23:20 |
| *** itlinux has quit IRC | 23:26 | |
| weshay|rover | EmilienM is this where we allow packages for f28 builds? https://github.com/rdo-infra/fedora-stable-config/blob/master/fedora_constraints | 23:37 |
| weshay|rover | rlandy|ruck ^ | 23:37 |
| weshay|rover | rlandy|ruck I see setuptools is there for python2/3 | 23:37 |
| *** tosky has quit IRC | 23:38 | |
| weshay|rover | rlandy|ruck ya.. so if the rpm is available | 23:39 |
| weshay|rover | perhaps it's not installed and we have to update a spec | 23:39 |
| * weshay|rover looks at the bug | 23:39 | |
| *** aakarsh has joined #tripleo | 23:41 | |
| weshay|rover | er.. rlandy|ruck https://github.com/rdo-packages/tripleo-common-distgit/blob/rpm-master/openstack-tripleo-common.spec#L63 | 23:42 |
| rlandy|ruck | weshay|rover; sec - fixing ci.centso failures- it was my fault :( | 23:42 |
| weshay|rover | IT"S ALL YOUR FAULT | 23:42 |
| *** khyr0n has joined #tripleo | 23:43 | |
| *** itlinux has joined #tripleo | 23:44 | |
| *** itlinux has quit IRC | 23:44 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Fix spacing when falor args are used https://review.openstack.org/653634 | 23:46 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Fix spacing when flavor args are used https://review.openstack.org/653634 | 23:47 |
| rlandy|ruck | weshay|rover: ^^ that should fix one set of problems | 23:47 |
| *** artom has joined #tripleo | 23:48 | |
| weshay|rover | k.. | 23:48 |
| *** absubram has quit IRC | 23:50 | |
| weshay|rover | rlandy|ruck afaict.. /me adding notes to the bug | 23:50 |
| weshay|rover | I don't see why we would get that import error yet | 23:50 |
| weshay|rover | rlandy|ruck so... this is a good one for the repro | 23:51 |
| weshay|rover | I'll fire one up | 23:51 |
| rlandy|ruck | we are only seeing that in promotions | 23:52 |
| weshay|rover | rlandy|ruck ya.. that is even weirder | 23:52 |
| weshay|rover | because it's all tripleo code | 23:52 |
| rlandy|ruck | also it's fedora | 23:52 |
| rlandy|ruck | we don't run that as commonly | 23:52 |
| *** boazel_ has joined #tripleo | 23:52 | |
| weshay|rover | it's running in check | 23:53 |
| weshay|rover | kicked a repro | 23:54 |
| rlandy|ruck | weshay|rover: thanks - can you vote on https://review.openstack.org/#/c/653634/ so we can hopefully clear that error | 23:55 |
| *** boazel has quit IRC | 23:55 | |
| *** khyr0n has quit IRC | 23:56 | |
| *** khyr0n has joined #tripleo | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!