| *** tosky has quit IRC | 00:00 | |
| *** tkajinam has quit IRC | 00:07 | |
| *** tkajinam has joined #tripleo | 00:07 | |
| *** bnemec has quit IRC | 00:14 | |
| *** saneax has quit IRC | 00:20 | |
| *** rlandy has quit IRC | 00:29 | |
| *** rodrigods has quit IRC | 00:39 | |
| openstackgerrit | Merged openstack/tripleo-ansible master: zuul: all functional jobs now depend on linter job first https://review.opendev.org/693355 | 00:47 |
|---|---|---|
| *** artom has quit IRC | 01:08 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates master: Deploy neutron metadata agent with nova metadata api https://review.opendev.org/699528 | 01:24 |
| tkajinam | hmmm... seems we need to disable relative_classname_inclusion for instack_undercloud to avoid the same error we observed for puppet modules... https://review.opendev.org/#/c/699401/ | 01:56 |
| openstackgerrit | Takashi Kajinami proposed openstack/puppet-tripleo master: Add support to configure virtlogd https://review.opendev.org/685349 | 01:59 |
| openstackgerrit | Takashi Kajinami proposed openstack/puppet-tripleo master: Add support to configure virtlogd https://review.opendev.org/685349 | 02:00 |
| *** apetrich has quit IRC | 03:09 | |
| *** gouthamr has quit IRC | 03:14 | |
| *** gouthamr has joined #tripleo | 03:24 | |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: Implement utils.copy_clouds_yaml https://review.opendev.org/699448 | 03:30 |
| *** psachin has joined #tripleo | 03:40 | |
| *** StevenK_ is now known as StevenK | 03:44 | |
| *** ykarel|away has joined #tripleo | 03:46 | |
| *** bhagyashris has joined #tripleo | 04:05 | |
| *** ykarel|away has quit IRC | 04:18 | |
| *** skramaja has joined #tripleo | 04:26 | |
| *** ykarel|away has joined #tripleo | 04:35 | |
| *** bhagyashris has quit IRC | 04:47 | |
| *** bhagyashris has joined #tripleo | 04:47 | |
| *** epoojad1 has joined #tripleo | 04:49 | |
| *** holser has joined #tripleo | 04:54 | |
| *** ykarel|away is now known as ykarel | 05:00 | |
| *** whoami-rajat__ has joined #tripleo | 05:03 | |
| openstackgerrit | yatin proposed openstack/tripleo-ci master: [DNM] Add option to build containers from package https://review.opendev.org/699549 | 05:18 |
| openstackgerrit | Jaganathan Palanisamy proposed openstack/tripleo-common stable/stein: Incorrectly derives NeutronPhysnetNUMANodesMapping https://review.opendev.org/699551 | 05:21 |
| openstackgerrit | Jaganathan Palanisamy proposed openstack/tripleo-common stable/rocky: OvS DPDK parameters failing in NIC Partitioning https://review.opendev.org/699552 | 05:22 |
| *** ramishra has joined #tripleo | 05:23 | |
| *** pcaruana has joined #tripleo | 05:24 | |
| openstackgerrit | yatin proposed openstack/tripleo-ci master: [DNM] Test container build with python3 on CentOS7 https://review.opendev.org/699555 | 05:28 |
| openstackgerrit | yogananth subramanian proposed openstack/tripleo-heat-templates master: Provide option to set reserved_huge_pages https://review.opendev.org/693994 | 05:31 |
| *** holser has quit IRC | 05:34 | |
| *** zaneb has quit IRC | 05:53 | |
| *** raukadah is now known as chkumar|rover | 05:58 | |
| *** surpatil has joined #tripleo | 05:59 | |
| *** bandini has joined #tripleo | 06:00 | |
| openstackgerrit | Suraj Patil proposed openstack/tripleo-quickstart-extras master: [queens] Removed passing Tests from skiplist https://review.opendev.org/699558 | 06:05 |
| *** whoami-rajat__ has quit IRC | 06:07 | |
| *** SurajPatil has joined #tripleo | 06:09 | |
| *** surpatil has quit IRC | 06:12 | |
| *** yprokule has joined #tripleo | 06:14 | |
| *** waleedm has joined #tripleo | 06:16 | |
| *** ksambor has quit IRC | 06:17 | |
| *** ksambor has joined #tripleo | 06:17 | |
| bandini | ugh gate is all clogged | 06:22 |
| *** saneax has joined #tripleo | 06:24 | |
| chkumar|rover | looks like last night so many patches got approved | 06:25 |
| *** SurajPatil has quit IRC | 06:25 | |
| *** surpatil has joined #tripleo | 06:26 | |
| *** lmiccini has joined #tripleo | 06:34 | |
| *** marios|ruck has joined #tripleo | 06:36 | |
| *** whoami-rajat__ has joined #tripleo | 06:37 | |
| *** shyamb has joined #tripleo | 06:40 | |
| shyamb | Hi owalsh: Created same bug in redhat bugzila: https://bugzilla.redhat.com/show_bug.cgi?id=1784706 | 06:41 |
| openstack | bugzilla.redhat.com bug 1784706 in tripleo-ansible "Duplicate horizon, keystone containers getting launched during overcloud deploy process" [Urgent,New] - Assigned to rhos-maint | 06:41 |
| *** ansmith_ has joined #tripleo | 06:42 | |
| *** ansmith has quit IRC | 06:43 | |
| *** janki has joined #tripleo | 06:48 | |
| *** jfrancoa has joined #tripleo | 06:52 | |
| *** rcernin has quit IRC | 06:56 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/stein: Revert "Add ntpdate to ensure hwclock sync" https://review.opendev.org/699033 | 07:01 |
| Tengu | chkumar|rover: apparently we did some unlocking in CI ? :) | 07:12 |
| Tengu | chkumar|rover: btw, any power on https://review.opendev.org/699438 ? | 07:12 |
| Tengu | would be good to get it merged so that we can close the LP | 07:13 |
| chkumar|rover | Tengu: thanks, will get it merged | 07:13 |
| Tengu | :) | 07:15 |
| Tengu | chkumar|rover: would you have a moment for some help on a CI issue? it keeps failing for a "tox -elinters" here: https://14d65eceddbce78ddf51-8bfb5d70b83a273fa97d15d51d14f1ae.ssl.cf1.rackcdn.com/698006/5/check/openstack-tox-linters/8233f64/job-output.txt | 07:17 |
| *** pkopec has joined #tripleo | 07:18 | |
| Tengu | chkumar|rover: I've tried to understand what's going on, I can reproduce it locally using a virtualenv with python3.6, but I'm unabe to find WHAT file is actually failint… apparently it's not always the same place. | 07:18 |
| chkumar|rover | Tengu: checking | 07:18 |
| Tengu | also, I don't really like the message about python version mismatch, especially when a TODO in the code shows a "raise for tox 4" | 07:19 |
| chkumar|rover | Tengu: at this file it failed https://review.opendev.org/#/c/698006/5/roles/validate-selinux/molecule/default/prepare.yml | 07:20 |
| Tengu | chkumar|rover: how did you find it?! | 07:21 |
| chkumar|rover | I am checking for last file order | 07:21 |
| chkumar|rover | reproducing it locally | 07:21 |
| Tengu | o_ | 07:21 |
| Tengu | here it changes every time | 07:21 |
| chkumar|rover | may be issue in ansible lint | 07:22 |
| Tengu | «again»... | 07:22 |
| Tengu | ? | 07:22 |
| Tengu | here, each run fails on a different file. | 07:22 |
| Tengu | now, it MIGHT indeed be related to that one since it's a new addition. Wondering how I can pin the exact issue though | 07:23 |
| *** TrevorV has joined #tripleo | 07:23 | |
| *** TrevorV has quit IRC | 07:23 | |
| chkumar|rover | Tengu: I also reproduced | 07:28 |
| Tengu | chkumar|rover: hmm. apparently, it's either ./molecule/default/playbook.yml or ./molecule/default/prepare.yml in the validate-selinux. Just called ansible-lint by hand, and one of those is failing. Thank for the investigations! | 07:28 |
| Tengu | yay! ./molecule/default/playbook.yml | 07:28 |
| Tengu | but absolutely NO clue as to why.... | 07:31 |
| *** holser has joined #tripleo | 07:31 | |
| *** mschuppert has joined #tripleo | 07:32 | |
| chkumar|rover | trying out pdb | 07:33 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/train: Add netcontrold service for DPDK roles https://review.opendev.org/698869 | 07:35 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/stein: Properly set loopback_users via rabbitmq::loopback_users https://review.opendev.org/698641 | 07:35 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/rocky: Fix OctaviaFlavorId type https://review.opendev.org/696847 | 07:35 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates stable/stein: Increase rabbitmq tcp backlog https://review.opendev.org/699573 | 07:36 |
| Tengu | chkumar|rover: apparently it has some issues finding the role path. At least, that's what I understand. | 07:37 |
| chkumar|rover | Tengu: yes | 07:37 |
| Tengu | non-sense :/ | 07:38 |
| *** ccamacho has joined #tripleo | 07:38 | |
| *** shyamb has quit IRC | 07:41 | |
| Tengu | chkumar|rover: found it. | 07:42 |
| chkumar|rover | is it include role? | 07:42 |
| Tengu | include_role dropped "role" param in favor of "name", but it's not even failing within ansible - ansible-lint on the other part seems to enforce it at some point. | 07:42 |
| chkumar|rover | runnign it rigjt now with that fix | 07:43 |
| chkumar|rover | Tengu: http://paste.openstack.org/show/787701/ | 07:43 |
| chkumar|rover | doe not work | 07:44 |
| Tengu | have to update allthe calls | 07:44 |
| chkumar|rover | ok | 07:45 |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/tripleo-validations master: New validation: validate-selinux https://review.opendev.org/698006 | 07:45 |
| Tengu | now if gerrit can... | 07:45 |
| Tengu | thank you gerrit. | 07:45 |
| Tengu | grumpf. | 07:45 |
| Tengu | that patch should be OK | 07:45 |
| openstackgerrit | Martin Schuppert proposed openstack/puppet-tripleo master: Make sure neutron [placement] config section is set https://review.opendev.org/698843 | 07:46 |
| *** ykarel is now known as ykarel|lunch | 07:47 | |
| *** whoami-rajat__ is now known as whoami-rajat|lun | 07:48 | |
| *** kozhukalov has joined #tripleo | 07:52 | |
| *** kopecmartin|off is now known as kopecmartin | 07:53 | |
| *** slaweq has joined #tripleo | 08:03 | |
| *** Chaserjim has quit IRC | 08:04 | |
| *** Chaserjim has joined #tripleo | 08:04 | |
| openstackgerrit | Daniel Bengtsson proposed openstack/tripleo-upgrade master: Clean up ansible variables. https://review.opendev.org/562957 | 08:13 |
| *** amoralej|off is now known as amoralej | 08:14 | |
| openstackgerrit | Daniel Bengtsson proposed openstack/tripleo-quickstart master: Use python -m pip instead pip command. https://review.opendev.org/693142 | 08:15 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-upgrade master: Fix bindeps https://review.opendev.org/699574 | 08:15 |
| *** tosky has joined #tripleo | 08:18 | |
| *** jpena|off is now known as jpena | 08:22 | |
| *** tmazur has joined #tripleo | 08:23 | |
| *** florianf has joined #tripleo | 08:23 | |
| chkumar|rover | Tengu: does linters passed? | 08:25 |
| *** hberaud|gone is now known as hberaud | 08:25 | |
| *** SurajPatil has joined #tripleo | 08:29 | |
| *** surpatil has quit IRC | 08:30 | |
| *** surpatil has joined #tripleo | 08:31 | |
| *** tosky has quit IRC | 08:32 | |
| *** tkajinam has quit IRC | 08:34 | |
| *** SurajPatil has quit IRC | 08:34 | |
| *** shyamb has joined #tripleo | 08:35 | |
| *** slaweq has quit IRC | 08:36 | |
| *** pbandark has joined #tripleo | 08:43 | |
| Tengu | chkumar|rover: failed, but on another issue - and one I didn't got locally.... | 08:46 |
| Tengu | ansible-lint is a nightmare. | 08:46 |
| *** slaweq has joined #tripleo | 08:50 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-quickstart-extras master: Add tempest.api.compute.images test_delete_saving_image to skip https://review.opendev.org/699624 | 08:52 |
| marios|ruck | chkumar|rover: ^ https://bugs.launchpad.net/tripleo/+bug/1856723/comments/3 | 08:53 |
| openstack | Launchpad bug 1856723 in tripleo "periodic-tripleo-ci-centos-7-standalone-full-tempest-master fails during tempest run ImagesTestJSON:test_delete_saving_image" [Critical,Triaged] | 08:53 |
| *** kozhukalov has quit IRC | 08:58 | |
| *** rpittau|afk is now known as rpittau | 08:59 | |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/tripleo-validations master: New validation: validate-selinux https://review.opendev.org/698006 | 09:00 |
| *** jpich has joined #tripleo | 09:01 | |
| *** lucasagomes has joined #tripleo | 09:05 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/python-tripleoclient master: Support standalone ansible.cfg addition and extra args https://review.opendev.org/699137 | 09:13 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: run ansible plugins on undercloud https://review.opendev.org/699314 | 09:13 |
| Tengu | chkumar|rover: w00t! linters just passed with latest patch! pfiouuu | 09:13 |
| *** pbandark has quit IRC | 09:14 | |
| *** tosky has joined #tripleo | 09:16 | |
| *** gfidente has joined #tripleo | 09:16 | |
| *** paramite|clone has joined #tripleo | 09:19 | |
| *** shyamb has quit IRC | 09:23 | |
| *** apetrich has joined #tripleo | 09:27 | |
| Tengu | gchamoul: I finally found the issue with my validate-selinux thing... ansible-lint being stricter than ansible itself regarding parameters for include_role... | 09:29 |
| Tengu | ansible does an undocumented alias name <-> role, while ansible-lint doesn't. yay | 09:31 |
| gchamoul | Tengu: ... | 09:35 |
| Tengu | yeah... well ;) | 09:36 |
| Tengu | so it should now pass CI, and be able to merge shortly. | 09:36 |
| Tengu | then, I'll have to corner weshay in order to activate it in some way :). | 09:36 |
| Tengu | and this MIGHT be the first call to the validation framework. yay! | 09:37 |
| *** derekh has joined #tripleo | 09:37 | |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-ansible stable/train: Create Validations logging directory https://review.opendev.org/699641 | 09:37 |
| *** bhagyashris has quit IRC | 09:41 | |
| *** ykarel|lunch is now known as ykarel | 09:44 | |
| *** whoami-rajat|lun is now known as whoami-rajat | 09:44 | |
| *** shyamb has joined #tripleo | 09:51 | |
| *** dpawlik has joined #tripleo | 09:51 | |
| *** leanderthal has joined #tripleo | 09:53 | |
| *** fmount has quit IRC | 09:53 | |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-validations master: Fix upper and lower constraints https://review.opendev.org/699499 | 09:55 |
| *** apetrich has quit IRC | 09:58 | |
| *** suuuper has joined #tripleo | 10:00 | |
| *** fmount has joined #tripleo | 10:02 | |
| *** tosky has quit IRC | 10:02 | |
| *** Chaserjim has quit IRC | 10:03 | |
| *** tosky has joined #tripleo | 10:03 | |
| *** Chaserjim has joined #tripleo | 10:03 | |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-validations master: Fix upper and lower constraints https://review.opendev.org/699499 | 10:13 |
| *** bhagyashris has joined #tripleo | 10:24 | |
| shyamb | Hi | 10:28 |
| shyamb | in tripleo-quickstart github repo, I don't find this ansible playbook- quickstart-extras-overcloud.yml | 10:28 |
| shyamb | this is getting referred in documentation | 10:28 |
| shyamb | Are we keep some playbooks in some other repository? | 10:29 |
| tosky | shyamb: it's in tripleo-quickstart-extras - and please clone the repositories from opendev.org, github is just a mirror | 10:29 |
| shyamb | I am looking at this install doc: https://docs.openstack.org/tripleo-quickstart/latest/getting-started.html#install-the-undercloud | 10:29 |
| shyamb | tosky | 10:29 |
| shyamb | got it | 10:29 |
| shyamb | so if I want to deploy openstack using tripleo-quickstart, I need to clone both repos? | 10:29 |
| shyamb | or just tripleo-quickstart? | 10:30 |
| shyamb | and internally it will clone the extras repo? | 10:30 |
| shyamb | tosky: install doc https://docs.openstack.org/tripleo-quickstart/latest/getting-started.html#install-the-undercloud | 10:30 |
| tosky | no | 10:31 |
| tosky | quickstart.sh clones the other repository | 10:31 |
| shyamb | okay | 10:31 |
| shyamb | got it | 10:31 |
| shyamb | tosky: thank you | 10:31 |
| tosky | I'd suggest trying to follow the instructions (until you get stuck - but hopefully you won't :) | 10:31 |
| shyamb | tosky: yes | 10:32 |
| leanderthal | shyamb, and then come ask questions here or #rdo | 10:32 |
| shyamb | leanderthal: I was going through the script and design of the project | 10:33 |
| leanderthal | cool cool | 10:33 |
| shyamb | wanted to understand the details | 10:33 |
| leanderthal | fair | 10:33 |
| shyamb | thank you | 10:34 |
| leanderthal | here and #rdo are great resources, but if no one's answering, also feel free to post to the mailing lists | 10:34 |
| shyamb | I want to deploy my product on openstack through tripleo-quickstart | 10:35 |
| leanderthal | https://lists.rdoproject.org/mailman/listinfo | 10:35 |
| leanderthal | cool! | 10:35 |
| *** janki has quit IRC | 10:35 | |
| shyamb | I need to write my ansible role? | 10:35 |
| openstackgerrit | Kamil Sambor proposed openstack/tripleo-heat-templates master: WIP: Change default dns domain in neutron https://review.opendev.org/699407 | 10:40 |
| chem_ | matbu: hey, around ? | 10:40 |
| *** shyamb has quit IRC | 10:42 | |
| *** holser has quit IRC | 10:50 | |
| *** holser__ has joined #tripleo | 10:50 | |
| matbu | chem_: yep | 10:51 |
| *** hberaud is now known as hberaud|school-r | 10:52 | |
| *** bhagyashris has quit IRC | 10:54 | |
| Tengu | folks, can anyone approve this patch? https://review.opendev.org/#/c/699438/ it would close a CIX :) | 10:56 |
| *** bhagyashris has joined #tripleo | 10:58 | |
| *** shyamb has joined #tripleo | 10:59 | |
| Tengu | or I can +A it.... ? since it's actually blocking promotion and things | 10:59 |
| *** tbarron_ has joined #tripleo | 11:00 | |
| *** Chaserjim has quit IRC | 11:00 | |
| *** Chaserjim has joined #tripleo | 11:00 | |
| *** pbandark has joined #tripleo | 11:01 | |
| sshnaidm | Tengu, done | 11:12 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-upgrade master: Run upgrade for previous hosts only in Controller role. https://review.opendev.org/699665 | 11:15 |
| *** ykarel is now known as ykarel|afk | 11:20 | |
| *** holser__ has quit IRC | 11:21 | |
| *** holser has joined #tripleo | 11:22 | |
| *** hberaud|school-r is now known as hberaud | 11:24 | |
| cgoncalves | chkumar|rover, marios|ruck: hi. stable/train tripleo-ci-centos-7-containerized-undercloud-upgrades has been failing consistently for the last 4 rechecks in https://review.opendev.org/#/c/698246/. I believe the patch does not touch code relevant to the job. are you aware of any issues with this job on stable/train? | 11:25 |
| marios|ruck | cgoncalves: checking ... we have a lp for that already i believe gimme min will find | 11:27 |
| marios|ruck | 2019-12-18 10:44:39.322627 | primary | "tripleo_ironic_neutron_agent_healthcheck.service loaded failed failed ironic_neutron_agent healthcheck" | 11:27 |
| *** shyamb has quit IRC | 11:27 | |
| marios|ruck | cgoncalves: yeah we hav https://bugs.launchpad.net/tripleo/+bug/1856288 | 11:28 |
| openstack | Launchpad bug 1856191 in tripleo "duplicate for #1856288 Ironic PXE healthcheck is broken in master" [Undecided,Fix released] - Assigned to Emilien Macchi (emilienm) | 11:28 |
| marios|ruck | cgoncalves: meant to be fixed by https://review.opendev.org/#/c/698580/ | 11:28 |
| marios|ruck | cgoncalves: but looks like its still an issue | 11:28 |
| marios|ruck | matbu: jfrancoa: fyi containerized undercloud-upgrades bug still ongoing (neutron healthcheck one0 ^^ | 11:29 |
| cgoncalves | marios|ruck, thanks. I see the same error in the failed job of my patch. | 11:30 |
| marios|ruck | cgoncalves: right i added comment in the bug just now | 11:30 |
| openstackgerrit | Daniel Bengtsson proposed openstack/tripleo-validations master: Remove the useless environment variable. https://review.opendev.org/699667 | 11:30 |
| *** ykarel|afk has quit IRC | 11:30 | |
| *** jfrancoa has quit IRC | 11:32 | |
| *** apetrich has joined #tripleo | 11:42 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/python-tripleoclient master: Support standalone ansible.cfg addition and extra args https://review.opendev.org/699137 | 11:44 |
| matbu | marios|ruck: i don't follow the review from Damien, fix it or not ? | 11:45 |
| *** jfrancoa has joined #tripleo | 11:47 | |
| surpatil | ls | 11:48 |
| *** bhagyashris has quit IRC | 11:52 | |
| marios|ruck | matbu: well there were two issues. dciabrin_ patch was for another healthcheck. for neutron agent healthcheck mwhahaha patch https://review.opendev.org/#/c/698580/ was meant to fix it | 11:52 |
| marios|ruck | matbu: but that is the failing healthcheck in https://bugs.launchpad.net/tripleo/+bug/1856288/comments/7 and from cgoncalves | 11:53 |
| openstack | Launchpad bug 1856191 in tripleo "duplicate for #1856288 Ironic PXE healthcheck is broken in master" [Undecided,Fix released] - Assigned to Emilien Macchi (emilienm) | 11:53 |
| *** lucasagomes has quit IRC | 11:57 | |
| marios|ruck | matbu: hmmm chkumar|rover wondering if this is just related to delayed consistent build | 11:57 |
| *** lucasagomes has joined #tripleo | 11:58 | |
| marios|ruck | chkumar|rover: so we don't have latest which includes fix from mwhahaha | 11:58 |
| marios|ruck | matbu: we should wait for next periodic run maybe will clear it | 11:58 |
| *** bhagyashris has joined #tripleo | 11:58 | |
| *** epoojad1 is now known as epoojad1|afk | 11:59 | |
| matbu | matbu: hmm oki | 12:00 |
| chkumar|rover | matbu: marios|ruck we need to wait for the promotion | 12:02 |
| matbu | then i can go to lunch :D | 12:03 |
| matbu | chkumar|rover: ack thx | 12:03 |
| marios|ruck | matbu: (this is the issue i mean https://bugs.launchpad.net/tripleo/+bug/1856830/comments/5 | 12:03 |
| openstack | Launchpad bug 1856830 in tripleo "master + train promote-consistent-to-tripleo-ci-testing times out blocks master+train pipeline" [Critical,Triaged] | 12:03 |
| *** sshnaidm has quit IRC | 12:04 | |
| *** epoojad1|afk has quit IRC | 12:04 | |
| *** raildo has joined #tripleo | 12:05 | |
| *** yog_ has joined #tripleo | 12:05 | |
| yog_ | mwhahaha, EmilienM .. it would be wonderful if you could give your review for the patch related to setting reserved_huge_pages ... https://review.opendev.org/#/c/693994/ | 12:09 |
| *** bhagyashris_ has joined #tripleo | 12:09 | |
| *** bhagyashris has quit IRC | 12:12 | |
| *** hjensas has quit IRC | 12:12 | |
| *** bhagyashris_ has quit IRC | 12:13 | |
| *** gfidente has quit IRC | 12:14 | |
| *** florianf has quit IRC | 12:18 | |
| *** ykarel|afk has joined #tripleo | 12:21 | |
| *** florianf has joined #tripleo | 12:25 | |
| *** gfidente has joined #tripleo | 12:29 | |
| *** jpena is now known as jpena|lunch | 12:41 | |
| *** ansmith_ has quit IRC | 12:53 | |
| *** morazi has joined #tripleo | 12:53 | |
| *** rlandy has joined #tripleo | 12:57 | |
| *** sshnaidm has joined #tripleo | 12:58 | |
| *** shyamb has joined #tripleo | 12:58 | |
| *** Goneri has quit IRC | 12:58 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: Properly set loopback_users via rabbitmq::loopback_users https://review.opendev.org/698655 | 12:58 |
| *** ekultails has joined #tripleo | 13:00 | |
| *** marios|ruck is now known as marios|ruck|call | 13:00 | |
| *** sshnaidm has quit IRC | 13:02 | |
| *** sshnaidm has joined #tripleo | 13:03 | |
| Tengu | folks, care to make that one merge? https://review.opendev.org/698006 would be great to get that new validation in the repo :) | 13:03 |
| chkumar|rover | Tengu: gates are in bad state, can we wait? | 13:04 |
| *** rh-jelabarre has joined #tripleo | 13:06 | |
| chkumar|rover | we have long queue time there | 13:06 |
| *** jainprasuk1996 has joined #tripleo | 13:07 | |
| slagle | fultonj: can you review https://review.opendev.org/#/c/697188/ plz? it's needed for minor update for multi-stack | 13:07 |
| fultonj | slagle: sure | 13:08 |
| slagle | thanks | 13:08 |
| Tengu | chkumar|rover: noooo we can't wait, never, EVER ! | 13:08 |
| Tengu | :] | 13:08 |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/tripleo-heat-templates master: Create a dedicated log file for healthchecks for collectd https://review.opendev.org/699226 | 13:08 |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: Implement utils.copy_clouds_yaml https://review.opendev.org/699448 | 13:08 |
| Tengu | chkumar|rover: (joke aside, of course we can wait, that one isn't high prio) | 13:09 |
| *** amoralej is now known as amoralej|lunch | 13:09 | |
| fultonj | slagle: +2 +w | 13:10 |
| *** udesale has joined #tripleo | 13:11 | |
| *** holser has quit IRC | 13:17 | |
| *** jbadiapa has joined #tripleo | 13:17 | |
| *** holser has joined #tripleo | 13:18 | |
| *** nkinder has joined #tripleo | 13:19 | |
| slagle | fultonj: cheers! | 13:19 |
| *** ssbarnea has quit IRC | 13:22 | |
| fultonj | slagle: would you mind reviewing https://review.opendev.org/#/c/696401 ? | 13:22 |
| *** jainprasuk1996 has quit IRC | 13:22 | |
| *** csmart has quit IRC | 13:23 | |
| slagle | fultonj: will take a look | 13:24 |
| fultonj | thks? | 13:24 |
| *** jainprasuk1996 has joined #tripleo | 13:28 | |
| cloudnull | mornings | 13:32 |
| openstackgerrit | Ronelle Landy proposed openstack/ansible-role-collect-logs master: WIP: Add functionality to sanitize collected logs https://review.opendev.org/691994 | 13:33 |
| *** ykarel|afk is now known as ykarel | 13:36 | |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-quickstart master: WIP: parameterize amphora download path https://review.opendev.org/692857 | 13:38 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates stable/queens: WIP/DNM Configure amphora for upload and run tempest https://review.opendev.org/692579 | 13:38 |
| EmilienM | chkumar|rover, marios|ruck|call : do we know why the gate is stacked up? | 13:42 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/queens: CiscoNexus: Update and Remove deprecated vars https://review.opendev.org/699420 | 13:44 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/queens: Remove incorrect mapped_data key from cisco ml2 hieradata https://review.opendev.org/699421 | 13:44 |
| *** jpena|lunch is now known as jpena | 13:45 | |
| *** Goneri has joined #tripleo | 13:46 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-ansible master: tripleo-keystone-resources: handle multiple roles per user https://review.opendev.org/699286 | 13:47 |
| chkumar|rover | EmilienM: we are seeing random times on multinode jobs, It is 9th hours from early morning, the patches in queue are high also | 13:48 |
| EmilienM | chkumar|rover: so it's timeout ? | 13:48 |
| chkumar|rover | yes | 13:48 |
| EmilienM | mhhh | 13:51 |
| *** amoralej|lunch is now known as amoralej | 13:51 | |
| EmilienM | chkumar|rover: are you able to tell since when we have these timeouts? | 13:51 |
| EmilienM | do you have graps? | 13:51 |
| EmilienM | graphs | 13:51 |
| chkumar|rover | EmilienM: http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-containers-undercloud-minion | 13:51 |
| EmilienM | we have merged a bunch of changes lately | 13:51 |
| EmilienM | chkumar|rover: is that only the minion? | 13:52 |
| chkumar|rover | grabing from the graphs | 13:52 |
| *** hjensas has joined #tripleo | 13:52 | |
| *** ansmith has joined #tripleo | 13:53 | |
| EmilienM | mwhahaha: ^ when you're up | 13:53 |
| *** fmount has quit IRC | 13:53 | |
| *** fmount has joined #tripleo | 13:54 | |
| EmilienM | it's indeed timeouting during the minion deployment | 13:54 |
| EmilienM | https://99b357a009d0a4edac99-8a8b5915810a5f7208ad3fdf123edf93.ssl.cf2.rackcdn.com/698936/7/check/tripleo-ci-centos-7-containers-undercloud-minion/f96a71f/logs/subnode-1/home/zuul/minion_install.log.txt.gz | 13:54 |
| EmilienM | it's like the timeouts are since yesterday or something | 13:55 |
| fultonj | Tengu: regarding https://review.opendev.org/#/c/694673/ what kind of unit test are you looking for? e.g. Molecule? are there existing tests in the repos I could use as an example? | 13:55 |
| marios|ruck|call | EmilienM: not aware of something it must be new as it wasn't an issue until earlier at least chkumar|rover ? | 13:56 |
| marios|ruck|call | EmilienM: in a call right now will try check in bit | 13:56 |
| EmilienM | i'm looking now | 13:58 |
| chkumar|rover | EmilienM:http://dashboard-ci.tripleo.org/d/si1tipHZk/jobs-exploration?orgId=1&var-influxdb_filter=result%7C%3D%7CTIMED_OUT&from=1576072854338&to=1576677654339&panelId=9&fullscreen | 14:01 |
| chkumar|rover | sorry wrong mapping | 14:01 |
| *** jainprasuk1996 has quit IRC | 14:02 | |
| *** sshnaidm has quit IRC | 14:02 | |
| *** udesale has quit IRC | 14:02 | |
| *** lbragstad_ is now known as lbragstad | 14:03 | |
| *** rh-jelabarre has quit IRC | 14:04 | |
| openstackgerrit | Riccardo Pittau proposed openstack/diskimage-builder master: Add python-yaml in bindep for ubuntu https://review.opendev.org/699681 | 14:05 |
| Tengu | fultonj: Molecule would be good, yes, but maybe "plain" unit test for the python lib itself would already be something good | 14:05 |
| chkumar|rover | marios|ruck|call: sawe few timeout on my patches | 14:06 |
| Tengu | fultonj: lemme dig some links | 14:06 |
| Tengu | fultonj: so python unit test would take place here: https://opendev.org/openstack/tripleo-validations/src/branch/master/tripleo_validations/tests/library | 14:07 |
| *** marios|ruck|call is now known as marios | 14:07 | |
| *** marios is now known as marios|call | 14:07 | |
| Tengu | fultonj: and you might have a look at https://review.opendev.org/#/c/698006/ for molecule tests (and we can work together if you want) | 14:07 |
| *** chkumar|rover is now known as chandankumar | 14:07 | |
| mwhahaha | EmilienM: Im PTO today. I bet it was the firewall change or something | 14:08 |
| fultonj | Tengu: should we have both? | 14:09 |
| EmilienM | mwhahaha: ah right, ttyl | 14:10 |
| Tengu | fultonj: python unit tests are testing the lib itself, method by method, while molecule is more like an "acceptance test" | 14:10 |
| *** bnemec has joined #tripleo | 14:10 | |
| *** ramishra has quit IRC | 14:13 | |
| mwhahaha | EmilienM: minion will hang if it can't MySQL to the undercloud because heat db sync hangs | 14:13 |
| Tengu | fultonj: and if we have to chose one, seeing the complexity, I'll ask for python unit tests in order to ensure we don't have regression at that level :). | 14:13 |
| EmilienM | cloudnull: i'm currently investigating the timeouts in tripleo gate and looking if it can be related to https://review.opendev.org/#/c/698620/ | 14:13 |
| openstackgerrit | Luke Short proposed openstack/tripleo-heat-templates stable/train: Use include_tasks instead of import_tasks https://review.opendev.org/699686 | 14:13 |
| EmilienM | mwhahaha: gotcha | 14:14 |
| cloudnull | EmilienM - are we seeing timouts across the board? | 14:14 |
| EmilienM | yes | 14:14 |
| EmilienM | gate is stacked up | 14:15 |
| Tengu | plugin is supposed to be faster :( | 14:15 |
| EmilienM | minion job is timing out (more than others apparently) | 14:15 |
| *** shyamb has quit IRC | 14:15 | |
| EmilienM | see what alex said ^ | 14:15 |
| cloudnull | yea, that plugin is quite a bit faster executing. | 14:16 |
| EmilienM | https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-containers-undercloud-minion | 14:16 |
| cloudnull | looking at the gate status page, and searching for tripleo, (http://zuul.openstack.org/status) it doesn't look like there are too many failures? | 14:17 |
| EmilienM | cloudnull: watch the queue | 14:18 |
| cloudnull | oh, so its just the minion job? | 14:18 |
| EmilienM | apparently not only, but with the most hits | 14:18 |
| cloudnull | in the current running jobs the only job with the status "timed_out" is a minion job | 14:20 |
| *** mcornea has joined #tripleo | 14:20 | |
| EmilienM | cloudnull: did we change logging for firewall tasks? | 14:20 |
| EmilienM | since yesterday I see: | 14:20 |
| EmilienM | tripleo-firewall : Manage firewall rules ------------------------------- 16.51s | 14:21 |
| EmilienM | check https://6965fd4aef581b711f1e-c46dc4d37fef1869c0d0d51dc6a4d27f.ssl.cf5.rackcdn.com/699505/1/check/tripleo-ci-centos-7-containers-undercloud-minion/eb9c291/logs/undercloud/home/zuul/undercloud_install.log.txt.gz | 14:21 |
| EmilienM | we didn't have it before: https://ac838a0a27e37f4efe0f-f8f923e4c1a8e99f5a61e3c81e6721bf.ssl.cf5.rackcdn.com/698211/6/check/tripleo-ci-centos-7-containers-undercloud-minion/0d87b18/logs/undercloud/home/zuul/undercloud_install.log.txt.gz | 14:21 |
| EmilienM | ah yeah we did https://review.opendev.org/#/c/698620/7/tripleo_ansible/roles/tripleo-firewall/tasks/main.yml | 14:21 |
| *** pbandark has quit IRC | 14:21 | |
| cloudnull | yea, its just the one task now | 14:21 |
| Tengu | 16 seconds, not a source of timeout, is it? | 14:22 |
| EmilienM | yeah and it takes 16.51s, not too bad | 14:22 |
| cloudnull | in verbose mode you see more output. | 14:22 |
| EmilienM | but I wonder if a rule would be missing or something | 14:22 |
| cloudnull | but without verbose its just a task return | 14:22 |
| EmilienM | i compared, I saw no issue | 14:22 |
| *** rh-jelabarre has joined #tripleo | 14:22 | |
| Tengu | EmilienM: we can check here maybe: https://6965fd4aef581b711f1e-c46dc4d37fef1869c0d0d51dc6a4d27f.ssl.cf5.rackcdn.com/699505/1/check/tripleo-ci-centos-7-containers-undercloud-minion/eb9c291/logs/undercloud/var/log/extra/network.txt.gz | 14:23 |
| EmilienM | yeah I did | 14:23 |
| EmilienM | I compared with a job from 3 days ago | 14:23 |
| EmilienM | and no change | 14:23 |
| Tengu | meh.. | 14:23 |
| Tengu | oh.... interesting..... my undercloud deploy seems to be stalled at step 3 | 14:23 |
| Tengu | ah. nope. it just unlocked -.- | 14:23 |
| mwhahaha | https://6965fd4aef581b711f1e-c46dc4d37fef1869c0d0d51dc6a4d27f.ssl.cf5.rackcdn.com/699505/1/check/tripleo-ci-centos-7-containers-undercloud-minion/eb9c291/logs/subnode-1/var/log/containers/stdouts/heat_engine_db_sync.log.txt.gz | 14:25 |
| mwhahaha | minion can't connect to the undercloud | 14:25 |
| Tengu | "No route to host" | 14:25 |
| mwhahaha | so it's hung on the db sync (like i said i twas) | 14:25 |
| EmilienM | interesting, I see one missing rule | 14:25 |
| Tengu | either missing firewall rule, or network issue | 14:25 |
| Tengu | EmilienM: so we probably have the cause then. what's that rule? | 14:26 |
| cloudnull | in the timout job it looks like the undercloud completed in ~1 hour | 14:26 |
| cloudnull | Start 2019-12-18 10:29:31, Stop 2019-12-18 11:30:15 | 14:26 |
| EmilienM | Tengu: https://www.diffchecker.com/GmEVf7pi | 14:27 |
| Tengu | Dec 18 13:05:56 undercloud.localdomain kernel: iptables dropped: IN=br-ex OUT= MAC=fe:ef:39:14:18:4e:fa:93:9d:fa:72:4d:08:00 SRC=192.168.24.4 DST=192.168.24.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=39564 DF PROTO=TCP SPT=56135 DPT=3306 WINDOW=29200 RES=0x00 SYN URGP=0 | 14:27 |
| EmilienM | nice catch | 14:28 |
| mwhahaha | like i said, probably the firewall change :D | 14:28 |
| Tengu | we're "just" missing rules allowing the minion to connect, indeed. and, hey, we have firewall logging :D | 14:28 |
| EmilienM | push ups ! | 14:28 |
| EmilienM | yeah is that new? | 14:28 |
| openstackgerrit | Ryan McCabe proposed openstack/tripleo-heat-templates master: Add SAF environment https://review.opendev.org/694623 | 14:28 |
| mwhahaha | the rewrite into python is | 14:28 |
| *** surpatil has quit IRC | 14:29 | |
| mwhahaha | https://review.opendev.org/#/c/698620/ | 14:29 |
| Tengu | err..... wait. "-A INPUT -m comment --comment "999 drop all ipv4" -m conntrack --ctstate NEW -j ACCEPT" accept, when it's saying "drop"? | 14:29 |
| EmilienM | yeah I know, I was wondering if the logging was new | 14:29 |
| Tengu | also, there are 2 rules after that one | 14:29 |
| EmilienM | mwhahaha: go back to PTO :P | 14:29 |
| * mwhahaha doesn't want to have to catch up on email tomorrow | 14:29 | |
| EmilienM | delete them all | 14:30 |
| Tengu | EmilienM: so logging was broken some months ago, I worked on that in order to make it back again - sooo yeah. | 14:30 |
| cloudnull | Tengu ++ that rule doesn't seem right. the comment says drop but the jump is accept | 14:30 |
| Tengu | cloudnull: yeah, and it should be at the far, far end I think. | 14:30 |
| cloudnull | ++ | 14:30 |
| Tengu | EmilienM: in fact, the two "missing" rules are present, but for other IPs I think. | 14:30 |
| EmilienM | wait, the thing on the left is what we had before | 14:30 |
| cloudnull | I would expect that to be "-A INPUT -m comment --comment "999 drop all ipv4" -m conntrack --ctstate NEW -j DROP" | 14:31 |
| EmilienM | the rules on the right is from a timeouted job | 14:31 |
| Tengu | cloudnull: yup. Still, that's a cosmetic detail. | 14:31 |
| mwhahaha | we shouldn't drop btw | 14:31 |
| mwhahaha | we used to reject | 14:31 |
| Tengu | EmilienM: care to check the IPs of the job matching the right one?= | 14:31 |
| mwhahaha | with icmp authorization prohibited or something | 14:31 |
| EmilienM | Tengu: https://6965fd4aef581b711f1e-c46dc4d37fef1869c0d0d51dc6a4d27f.ssl.cf5.rackcdn.com/699505/1/check/tripleo-ci-centos-7-containers-undercloud-minion/eb9c291/logs/subnode-1/var/log/extra/ is the job that timeouts | 14:31 |
| Tengu | EmilienM: since we apparently have 4 un-commented rules about 10.201.1 and 23.253 networks | 14:31 |
| EmilienM | for the working one: https://b0f2eef64be29a19651b-3de1975357a6c165386fdc5736e590dc.ssl.cf1.rackcdn.com/699425/1/check/tripleo-ci-centos-7-containers-undercloud-minion/226b09e/logs/subnode-1/var/log/extra/network.txt.gz | 14:32 |
| Tengu | inet 23.253.173.132/24 brd 23.253.173.255 scope global noprefixroute eth0 inet 10.210.1.132/19 brd 10.210.31.255 scope global noprefixroute eth1 | 14:32 |
| EmilienM | mwhahaha: we do reject | 14:32 |
| EmilienM | I see -A openstack-INPUT -j REJECT --reject-with icmp-host-prohibited | 14:32 |
| Tengu | EmilienM: they are allowed.... | 14:32 |
| Tengu | EmilienM: both minion IPs are allowed in the firewall :/ | 14:34 |
| Tengu | sooo..... that's maybe not the issue? | 14:34 |
| mwhahaha | https://b0f2eef64be29a19651b-3de1975357a6c165386fdc5736e590dc.ssl.cf1.rackcdn.com/699425/1/check/tripleo-ci-centos-7-containers-undercloud-minion/226b09e/logs/undercloud/var/log/extra/network.txt.gz | 14:34 |
| mwhahaha | we're missing 3306 | 14:34 |
| mwhahaha | that's teh bug | 14:34 |
| * mwhahaha would like to stop having to repeat himself | 14:34 | |
| Tengu | nah | 14:34 |
| Tengu | -A INPUT -s 23.253.173.92/32 -j ACCEPT <- accepts all from that IP | 14:35 |
| Tengu | same -A INPUT -s 10.210.1.132/32 -j ACCEPT | 14:35 |
| mwhahaha | you'd think | 14:35 |
| Tengu | those two are attached to the minion node (subnode-1) | 14:35 |
| Tengu | supposed to accept both TCP and UDP on all ports, yes. | 14:35 |
| mwhahaha | but the ip is 192.168.24.x | 14:35 |
| Tengu | hmm so the wrong IP is used|pushed. | 14:36 |
| mwhahaha | which we only allow snmp, ssh, memcache from | 14:36 |
| mwhahaha | plz compare https://1c854b43b98bfdb525a8-06ff0b77c4b7bc18f9f8408eaae15058.ssl.cf5.rackcdn.com/699019/2/check/tripleo-ci-centos-7-undercloud-containers/ab1a14b/logs/undercloud/var/log/extra/network.txt.gz | 14:36 |
| mwhahaha | 3306 is there | 14:36 |
| mwhahaha | it's missing from the rules | 14:37 |
| * mwhahaha broken record | 14:37 | |
| Tengu | yep, I see that - now the question is: why do we allow everything from the 2 other associated IPs? | 14:37 |
| mwhahaha | because tunnel | 14:37 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Change drop action https://review.opendev.org/699692 | 14:37 |
| mwhahaha | we setup a vxlan tunnel for 192.168.24 traffic on multinode | 14:37 |
| mwhahaha | it's a ci thing | 14:37 |
| Tengu | can't we allow all from that 192.168.24.x/24 ? | 14:37 |
| mwhahaha | not related tot his isue | 14:37 |
| cloudnull | mwhahaha Tengu that change will make it so we're adding the drop rules | 14:38 |
| mwhahaha | i'm sure we could, the point being that teh rules are way off | 14:38 |
| Tengu | right. Didn't realized it was using a 3rd subnet at some point -.-. | 14:38 |
| Tengu | pff. | 14:38 |
| Tengu | cloudnull: drop|reject I think... ? | 14:39 |
| cloudnull | it looks like before the action plugin it was using drop, but we can change that to reject ? | 14:40 |
| Tengu | cosmetics at that point. | 14:40 |
| EmilienM | mwhahaha: that was is from stable/stein | 14:40 |
| Tengu | fact is, we should understand why it's missing a rule. | 14:40 |
| EmilienM | mwhahaha: your "plz compare" URL | 14:40 |
| Tengu | and if we miss some others. | 14:40 |
| mwhahaha | EmilienM: yes we've dropped rules over the cycles (for reasons unknown) | 14:40 |
| cloudnull | https://github.com/openstack/tripleo-ansible/blob/master/tripleo_ansible/roles/tripleo-firewall/defaults/main.yml#L61-L63 | 14:40 |
| EmilienM | I looked at a job on master (before action plugin) and and there is no mysql | 14:40 |
| Tengu | EmilienM: I think it was working because 192.168.24.X/32 had full access, no limitation as today. | 14:41 |
| mwhahaha | anyway we need to audit the rules for the undercloud, in theory there should be a 3306 rule | 14:42 |
| Tengu | EmilienM: i.e. something is now injecting eth0 + eth1 IPs, but not the br-ex | 14:42 |
| Tengu | and, if we take back your diff, left one has only the br-ext IP, not eth0 nor eth1.... | 14:42 |
| Tengu | question would be "how are those IPs provided", and check if something in the comprehension of those IPs changed lately... | 14:43 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Change drop action https://review.opendev.org/699692 | 14:43 |
| cloudnull | checking a newly deployed undercloud from master I don't see a rule with port 3306 | 14:45 |
| *** marios|call is now known as marios | 14:45 | |
| EmilienM | cloudnull: right we had the rule in stein | 14:46 |
| EmilienM | let's see when it started to disappear | 14:46 |
| cloudnull | I think this rule "-A INPUT -m comment --comment "999 drop all ipv4" -m conntrack --ctstate NEW -j ACCEPT" was allowing it work. we were accepting ALL traffic, not dropping it. | 14:46 |
| mwhahaha | so you "fixed" it uncovering an older bug around missing rules :D | 14:47 |
| cloudnull | which was a bug in the way the ansible task driven firewall rules were being created | 14:47 |
| cloudnull | so does that mean I get negative pushups ? | 14:48 |
| cloudnull | :P | 14:48 |
| Tengu | cloudnull: makes sense as well | 14:48 |
| Tengu | cloudnull: call it squats ;) | 14:48 |
| cloudnull | hahaha | 14:48 |
| EmilienM | -1 on negative pushups | 14:48 |
| cloudnull | LOL | 14:48 |
| cloudnull | to get the gate working we could push a patch up to recreate that bug ? | 14:48 |
| cloudnull | IE add a rule that accepts all traffic | 14:49 |
| EmilienM | no, we need to figureo ut why mysql isn't added | 14:49 |
| cloudnull | that'd give us some time to go find / fix the real bug | 14:49 |
| cloudnull | ok | 14:49 |
| EmilienM | I'm playing mwhahaha's broken record | 14:49 |
| cloudnull | https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/database/mysql-base.yaml#L71-L79 | 14:51 |
| cloudnull | I see tht has the rule | 14:51 |
| Tengu | cloudnull: apparently it doesn't apply for the minion IP/case - else we'd see a lot more rules right? | 14:52 |
| EmilienM | I looked at a job from Nov 21, and we were missing mysql rule as well | 14:52 |
| *** sshnaidm has joined #tripleo | 14:53 | |
| * mwhahaha wanders off to PTO | 14:54 | |
| mwhahaha | good luck :D | 14:54 |
| EmilienM | slacker | 14:54 |
| EmilienM | It was there end of august | 14:54 |
| EmilienM | it disappeared somewhere in Sept/Oct | 14:54 |
| *** TrevorV has joined #tripleo | 14:55 | |
| Tengu | isn't it when things were moved to ansible for the firewall? | 14:55 |
| EmilienM | dciabrin_, bandini ^ by any chance, do you know why mysql iptables rule wouldn't be there on the undercloud anymore? | 14:55 |
| EmilienM | cloudnull, Tengu: when was the move again | 14:56 |
| dciabrin_ | EmilienM, meh no idea | 14:56 |
| bandini | EmilienM: erm not really no | 14:56 |
| bandini | did we migrate to the ansible for firewall and maybe something is lurking in there? | 14:56 |
| EmilienM | https://review.opendev.org/#/c/677237/ | 14:56 |
| Tengu | EmilienM: not really sure, I know there were "some" delay, but I think it was around october... cloudnull ? | 14:56 |
| EmilienM | was 2 weeks again | 14:57 |
| EmilienM | 2w ago* | 14:57 |
| Tengu | oh. craps. | 14:57 |
| Tengu | so not the same period right | 14:57 |
| Tengu | cloudnull: isn't there some kind of typo in the mysql-base maybe? for instance I don't see mention of port 4444 in the rules. | 15:00 |
| Tengu | cloudnull: or maybe some missing include, or whatever...? | 15:01 |
| yog_ | slagle, could kindly give your review again :) for the patch related to setting reserved_huge_pages ... https://review.opendev.org/#/c/693994/ .. had made minor code change regarding spacing/format in code based on saravanan's review | 15:01 |
| EmilienM | I wonder if it's because the rule doesn't have the service name in it | 15:02 |
| EmilienM | so it get ignored | 15:02 |
| Tengu | maybe? | 15:02 |
| Tengu | at least, none of the rules that are defined ONLY in this file are applied. | 15:02 |
| Tengu | there some others, but it's probably due to the fact they are defined in other templates. | 15:02 |
| Tengu | like port 873 | 15:03 |
| slagle | yog_: done. looks like it hit the CI timeout currently under investigation, so will need to hold off on +A | 15:04 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: WIP - use service_name for mysql firewall rules https://review.opendev.org/699699 | 15:04 |
| cloudnull | Tengu sorry was getting a coffee refill | 15:05 |
| * Tengu glues cloudnull to his chair until issue is sorted out | 15:05 | |
| Tengu | :) | 15:05 |
| EmilienM | my patch should happen IMHO | 15:05 |
| EmilienM | I didn't know the rule name was restrictive but I suspect it is right now | 15:06 |
| cloudnull | Tengu - https://github.com/openstack/tripleo-heat-templates/commit/50367fbe3563d34976deb377ed32b6f26aeca44f - looks like the initial move to the firewall role was nov 18 | 15:06 |
| Tengu | I think it might match the service_enabled list or something like that | 15:06 |
| Tengu | cloudnull: yep. in the meanwhile EmilienM might have found the issue. And it looks like something with some sens in it | 15:06 |
| EmilienM | yeah I think that's the bug | 15:06 |
| EmilienM | because I see some other rules missing | 15:07 |
| EmilienM | like ironic conductor TFTP | 15:07 |
| yog_ | slagle, thank you for the review ;) | 15:07 |
| Tengu | uhu | 15:07 |
| EmilienM | that's our bug | 15:07 |
| EmilienM | firewall rules that are applied are only the ones with a proper service_name matching hieradata | 15:07 |
| EmilienM | if the string isn't a service_name of hieradata on the node, the rule is skipped | 15:08 |
| Tengu | yup. sounds like something | 15:08 |
| *** jfrancoa has quit IRC | 15:08 | |
| Tengu | probably matchin [0-9]+\s<service_name> | 15:08 |
| cloudnull | where does that string matching happen ? | 15:08 |
| EmilienM | no idea yet but we'll find out | 15:08 |
| Tengu | heat template generation I suspect | 15:08 |
| * EmilienM digging THT | 15:08 | |
| EmilienM | btw | 15:11 |
| EmilienM | https://6965fd4aef581b711f1e-c46dc4d37fef1869c0d0d51dc6a4d27f.ssl.cf5.rackcdn.com/699505/1/check/tripleo-ci-centos-7-containers-undercloud-minion/eb9c291/logs/undercloud/home/zuul/undercloud-ansible-T0VOEe/Undercloud/host_prep_tasks.yaml.txt.gz | 15:11 |
| EmilienM | you can see that a lot of rules are missing in the tripleo_firewall_rules var | 15:11 |
| *** ykarel is now known as ykarel|away | 15:11 | |
| cloudnull | could it be something with the yaql https://github.com/openstack/tripleo-heat-templates/blob/master/common/services/role.role.j2.yaml#L344-L352 ? | 15:11 |
| EmilienM | it's likely around these lines: https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/common/services/role.role.j2.yaml#L344-L352 | 15:11 |
| Tengu | EmilienM: seems to match the "service_name isn't matching" thoughts. | 15:11 |
| Tengu | ouch... yaql shaving | 15:12 |
| *** ramishra has joined #tripleo | 15:12 | |
| *** waleedm has quit IRC | 15:13 | |
| Tengu | EmilienM: no rule with a non-service_name are applied indeed. comments in the applied rules are indeed showing only "<order> <service_name> <ip-version>" | 15:15 |
| Tengu | not that the ip-version is probably useless since we have two separated commands, iptables for inet and ip6tables for inet6... | 15:15 |
| EmilienM | i wonder if it's because the mysql rules are in a base template | 15:15 |
| Tengu | EmilienM: hmm... -A INPUT -p udp -m udp --dport 69 -m comment --comment "134 ironic conductor TFTP ipv4" -m conntrack --ctstate NEW -j ACCEPT | 15:16 |
| Tengu | it is actually present | 15:16 |
| Tengu | and this crashes the theory I think :/ | 15:17 |
| cloudnull | the FirewallRules yaql was modeled after AnsibleGroupVars so maybe both have similar issues ? | 15:17 |
| EmilienM | Tengu: I think my theory is now, for the rules in a base template | 15:17 |
| Tengu | EmilienM: ah. some kind of override? | 15:17 |
| *** udesale has joined #tripleo | 15:17 | |
| *** whoami-rajat has quit IRC | 15:19 | |
| cloudnull | EmilienM I see the rules in both the base and pacemaker templates - https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/database/mysql-pacemaker-puppet.yaml | 15:19 |
| cloudnull | https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/database/mysql-pacemaker-puppet.yaml#L111-L120 | 15:19 |
| cloudnull | could that be causing the issue ? | 15:19 |
| cloudnull | they're not in https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/database/mysql-container-puppet.yaml and https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/database/mysql-client.yaml | 15:20 |
| EmilienM | no | 15:20 |
| EmilienM | that file isn't used on the undercloud | 15:20 |
| EmilienM | the problem is that the yaql looks for firewall_rules in the imported templates | 15:20 |
| EmilienM | https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/common/services/role.role.j2.yaml#L344-L352 | 15:21 |
| openstackgerrit | Daniel Bengtsson proposed openstack/tripleo-upgrade master: Fix python command. https://review.opendev.org/699391 | 15:21 |
| cloudnull | that one is the same to what is being done in AnsibleGroupVars | 15:21 |
| EmilienM | (which doesn't mean AnsibleGroupVars is working :D) | 15:22 |
| EmilienM | I _think_ the problem is that we use the role_data | 15:22 |
| EmilienM | let me try to reproduce locally | 15:23 |
| cloudnull | and GlobalConfigSettings MonitoringSubscriptionsConfig too | 15:23 |
| cloudnull | all are using the same style yql query | 15:23 |
| *** jfrancoa has joined #tripleo | 15:24 | |
| *** csmart has joined #tripleo | 15:24 | |
| *** ssbarnea has joined #tripleo | 15:24 | |
| *** udesale has quit IRC | 15:25 | |
| *** udesale has joined #tripleo | 15:25 | |
| EmilienM | chandankumar: we need to clear the gate IMHO | 15:25 |
| EmilienM | now that we have identified the problem (at least we know where to look), we should not approve patches | 15:25 |
| chandankumar | EmilienM: yes, can we start abandoning then? | 15:26 |
| *** mwhahaha sets mode: +o EmilienM | 15:26 | |
| EmilienM | chandankumar: yes. Have oyu done it before? | 15:26 |
| EmilienM | mwhahaha: thx | 15:26 |
| chandankumar | EmilienM: nope, I donot have access to abandon button | 15:27 |
| *** EmilienM changes topic to "CI Status: RED | Issue has been identified with broken firewall rules - DO NOT RECHECK - DO NOT +W | community irc meeting Tues@1400 UTC - tripleo-ci-community meeting Tues@1330 UTC | https://docs.openstack.org/tripleo-docs" | 15:27 | |
| EmilienM | chandankumar: ok I'll do it asap | 15:27 |
| EmilienM | i'm also sending a note to the ML | 15:27 |
| chandankumar | https://review.opendev.org/#/c/699438/ https://review.opendev.org/#/c/697526/ https://review.opendev.org/#/c/697188/ https://review.opendev.org/#/c/699007/ https://review.opendev.org/#/c/696401/ https://review.opendev.org/#/c/698417/ https://review.opendev.org/#/c/698417/ | 15:28 |
| Tengu | EmilienM: I've cancelled my 2 patches in gate. | 15:28 |
| chandankumar | jobs are not started yet here | 15:28 |
| Tengu | https://review.opendev.org/#/c/699426/ https://review.opendev.org/#/c/699438/ are stopped just to be sure | 15:28 |
| EmilienM | chandankumar: I'll find them in zuul | 15:28 |
| EmilienM | no need to paste here | 15:28 |
| cloudnull | EmilienM its just the one job that's failing, the minion job. wouldn't it be better to make that one non-voting ? | 15:28 |
| EmilienM | meh | 15:29 |
| cloudnull | seems the regular multi-node jobs are passing ? | 15:29 |
| EmilienM | yeah because we are lucky | 15:29 |
| *** marios is now known as marios|call | 15:29 | |
| EmilienM | now let me file a bug, send an email and test a fix | 15:29 |
| cloudnull | for sure, something is broken that needs fixing | 15:29 |
| cloudnull | just didn't want to cause too much havoc :) | 15:30 |
| cloudnull | but you're right, meh. all the current jobs are likely going to fail | 15:31 |
| EmilienM | and if we make minion non voting alex will not be super happy :D | 15:31 |
| *** panda has quit IRC | 15:31 | |
| *** panda has joined #tripleo | 15:33 | |
| EmilienM | https://bugs.launchpad.net/tripleo/+bug/1856864 | 15:33 |
| openstack | Launchpad bug 1856864 in tripleo "Not all firewall rules are applied, leading to timeouts" [Critical,Triaged] - Assigned to Emilien Macchi (emilienm) | 15:33 |
| ramishra | EmilienM: Where do you see the minion logs? It was failing intermittently yesterday too, I could not find the logs then | 15:37 |
| EmilienM | ramishra: https://6965fd4aef581b711f1e-c46dc4d37fef1869c0d0d51dc6a4d27f.ssl.cf5.rackcdn.com/699505/1/check/tripleo-ci-centos-7-containers-undercloud-minion/eb9c291/logs/subnode-1/var/log/containers/stdouts/heat_engine_db_sync.log.txt.gz | 15:37 |
| *** holser has quit IRC | 15:40 | |
| *** holser has joined #tripleo | 15:40 | |
| ramishra | EmilienM: ok, I was looking in the wrong place, wrongly assumed that it's installed from the undercloud node;) | 15:40 |
| *** csmart has quit IRC | 15:46 | |
| *** ssbarnea has quit IRC | 15:46 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-upgrade master: Fix python3 compatibility during role adjustment. https://review.opendev.org/699709 | 15:48 |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/tripleo-heat-templates master: Create a dedicated log file for healthchecks for collectd https://review.opendev.org/699226 | 15:49 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: mysql: move firewall_rules to mysql-container-puppet https://review.opendev.org/699712 | 15:50 |
| EmilienM | ramishra: could you please look at ^ | 15:50 |
| EmilienM | ramishra: I'm wondering if we can do better, wrt the YAQL https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/common/services/role.role.j2.yaml#L344-L352 | 15:50 |
| *** yog_ has quit IRC | 15:52 | |
| *** marios|call is now known as marios | 15:54 | |
| cloudnull | EmilienM ramishra - If we can come up with a better query we should also update FirewallRules, AnsibleGroupVars, GlobalConfigSettings, MonitoringSubscriptionsConfig - they're all the same basic YAQL. | 15:55 |
| *** udesale has quit IRC | 15:56 | |
| EmilienM | I tested my patch locally, it works | 15:56 |
| cloudnull | is there a good YAQL debugger ? | 15:56 |
| EmilienM | slagle: do you mind looking at https://review.opendev.org/699712 please | 15:57 |
| slagle | cloudnull: there's a shell, where you can load json data from a file, and then run queries against it | 15:57 |
| slagle | cloudnull: best i've found. it's all mostly guess and check anyway | 15:58 |
| cloudnull | slagle ++ | 15:58 |
| slagle | EmilienM: looking | 15:58 |
| EmilienM | lol | 15:58 |
| slagle | EmilienM: it looks fine to me. they definitely need to be in the role_data for the service template. there's no "magic" inheritance | 16:00 |
| EmilienM | ok | 16:00 |
| EmilienM | then let's go with it | 16:00 |
| EmilienM | i'm purging the gate now | 16:00 |
| EmilienM | slagle++ thx for the double checking | 16:01 |
| slagle | you could however pull them off the base template, similr to what is done with service_name | 16:01 |
| slagle | in case they need to be shared, so you only have to write it once | 16:01 |
| ramishra | slagle: I commented the same:) | 16:01 |
| EmilienM | in the case of mysql, it doesn't make sense | 16:01 |
| EmilienM | because they are overriden in mysql-galera I think | 16:02 |
| EmilienM | with 104 mysql galera-bundle | 16:02 |
| *** jainprasuk1996 has joined #tripleo | 16:04 | |
| EmilienM | chandankumar: purging the gate now | 16:04 |
| chandankumar | EmilienM: thanks for taking care, | 16:05 |
| chandankumar | Tengu: cloudnull EmilienM: I will be going offline from 20th Dec to 05th Jan | 16:05 |
| cloudnull | chandankumar enjoy your time off :D | 16:07 |
| cloudnull | EmilienM slagle are there other base templates that have firewall rules within them ? | 16:07 |
| *** ksambor has quit IRC | 16:07 | |
| openstackgerrit | Ronelle Landy proposed openstack/ansible-role-collect-logs master: Add functionality to sanitize collected logs https://review.opendev.org/691994 | 16:07 |
| *** ksambor has joined #tripleo | 16:07 | |
| EmilienM | cloudnull: not afik | 16:07 |
| EmilienM | I looked and couldn't find any beside mysql | 16:07 |
| cloudnull | running `find deployment -type f -name '*base*' -exec grep -l firewall_rules {} \; ` I dont see any that match the "base" naming convention | 16:07 |
| cloudnull | but idk if that naming convention holds true across the stack | 16:08 |
| EmilienM | gate is cleared | 16:08 |
| *** leanderthal has quit IRC | 16:09 | |
| *** chandankumar is now known as raukadah | 16:10 | |
| *** leanderthal has joined #tripleo | 16:10 | |
| EmilienM | bandini: if you're still around, can you quickly look https://review.opendev.org/#/c/699712/ | 16:11 |
| EmilienM | I'll approve if it pass CI | 16:11 |
| *** lmiccini has quit IRC | 16:11 | |
| EmilienM | and restore the gate | 16:11 |
| openstackgerrit | Daniel Bengtsson proposed openstack/tripleo-upgrade master: Fix python command. https://review.opendev.org/699391 | 16:12 |
| bandini | EmilienM: on it | 16:12 |
| cloudnull | so i guess I'll go do some pushups: https://github.com/openstack/tripleo-heat-templates/blame/master/deployment/database/mysql-base.yaml#L71-L79 that came over with the firewall conversion. it used to be in "config_settings" which I guess does not have the same YAQL issues. | 16:12 |
| EmilienM | cloudnull: i told you | 16:12 |
| EmilienM | :D | 16:12 |
| EmilienM | raukadah: are you still around? | 16:13 |
| EmilienM | raukadah: unping | 16:13 |
| raukadah | EmilienM: yes, just lurking | 16:13 |
| * raukadah hides then | 16:13 | |
| *** marios is now known as marios|out | 16:15 | |
| openstackgerrit | Daniel Bengtsson proposed openstack/tripleo-upgrade master: Fix python command and fix python3 compatiblity. https://review.opendev.org/699391 | 16:16 |
| *** leanderthal has quit IRC | 16:16 | |
| ramishra | cloudnull: Did not see your msg about the debugger, you can use http://yaqluator.com/ too, I too will spend sometime tomorrow to see if it can be better | 16:16 |
| cloudnull | ramishra is there a way to capture the entire blob from a deployment to run queries against in that online validator ? | 16:18 |
| *** iurygregory has joined #tripleo | 16:20 | |
| ramishra | cloudnull: I mean you can load from a file or paste the json, but you need to paste what's passed in data to heat yaql function | 16:20 |
| ramishra | cloudnull: If I understand your question.. late for me too;) | 16:21 |
| cloudnull | is there a way to retrieve a json blob with all of that heat data ? | 16:22 |
| *** marios|out has quit IRC | 16:26 | |
| slagle | cloudnull: look at the stack outputs of the nested stacks that correspond to the service templates | 16:26 |
| slagle | identify where what you're looking for gets passed around in the templates, identify the nested stack that corresponds to those templates, openstack stack show on those nested stacks | 16:27 |
| openstackgerrit | Mohammed Naser proposed openstack/diskimage-builder master: modprobe.d: use $TMP_MOUNT_PATH https://review.opendev.org/699722 | 16:27 |
| *** waleedm has joined #tripleo | 16:33 | |
| openstackgerrit | Mohammed Naser proposed openstack/diskimage-builder master: modprobe.d: use $TMP_MOUNT_PATH https://review.opendev.org/699722 | 16:42 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Add tempest.api.compute.images test_delete_saving_image to skip https://review.opendev.org/699624 | 16:44 |
| *** ed_b has joined #tripleo | 16:46 | |
| *** csmart has joined #tripleo | 16:47 | |
| *** ssbarnea has joined #tripleo | 16:47 | |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: Implement utils.copy_clouds_yaml https://review.opendev.org/699448 | 16:49 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-upgrade master: Fix python3 compatibility during role adjustment. https://review.opendev.org/699709 | 16:49 |
| *** waleedm_ has joined #tripleo | 16:52 | |
| *** lucasagomes has quit IRC | 16:52 | |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: Implement utils.copy_clouds_yaml https://review.opendev.org/699448 | 16:53 |
| *** waleedm has quit IRC | 16:54 | |
| *** shardy has joined #tripleo | 16:55 | |
| *** kopecmartin is now known as kopecmartin|off | 16:58 | |
| *** dtrainor has quit IRC | 17:08 | |
| *** dtrainor has joined #tripleo | 17:08 | |
| m1p | on train/centos7 with custom prepare parameters and `push_destination: true`, deployment fails with docker on overcloud nodes not being able to pull some images. anybody seen this before? | 17:13 |
| m1p | https://www.irccloud.com/pastebin/DoKAIRw7/ | 17:13 |
| m1p | it happend with more images before, i deleted /var/lib/image-serve/v2/tripleotrain and recreated it with sudo openstack tripleo container image prepare, now it only happens for a few images | 17:15 |
| m1p | which is weird because I didn't modify anything, except using the undercloud registry | 17:15 |
| *** artom has joined #tripleo | 17:17 | |
| *** rlandy is now known as rlandy|brb | 17:17 | |
| *** suuuper has quit IRC | 17:20 | |
| *** yprokule has quit IRC | 17:21 | |
| *** bandini has quit IRC | 17:21 | |
| *** rpittau is now known as rpittau|afk | 17:21 | |
| *** Chaserjim has quit IRC | 17:22 | |
| *** Chaserjim has joined #tripleo | 17:23 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: run ansible plugins on undercloud https://review.opendev.org/699314 | 17:23 |
| *** gfidente is now known as gfidente|afk | 17:24 | |
| *** hberaud is now known as hberaud|gone | 17:26 | |
| *** ykarel|away has quit IRC | 17:26 | |
| openstackgerrit | Daniel Bengtsson proposed openstack/tripleo-upgrade master: Fix python command and fix python3 compatiblity. https://review.opendev.org/699391 | 17:33 |
| *** jpena is now known as jpena|off | 17:36 | |
| *** panda has quit IRC | 17:36 | |
| openstackgerrit | Damien Ciabrini proposed openstack/tripleo-heat-templates master: Limit concurrency in container-puppet to image pull https://review.opendev.org/699737 | 17:37 |
| *** panda has joined #tripleo | 17:39 | |
| *** psachin has quit IRC | 17:39 | |
| *** waleedm has joined #tripleo | 17:43 | |
| *** waleedm_ has quit IRC | 17:45 | |
| *** rlandy|brb is now known as rlandy | 17:46 | |
| openstackgerrit | Damien Ciabrini proposed openstack/tripleo-heat-templates master: Limit concurrency in container-puppet to image pull https://review.opendev.org/699737 | 17:51 |
| openstackgerrit | Daniel Bengtsson proposed openstack/tripleo-upgrade master: Fix python command and fix python3 compatiblity. https://review.opendev.org/699391 | 17:54 |
| *** chem_ has quit IRC | 17:55 | |
| *** rascasoft has quit IRC | 17:56 | |
| *** mcornea has quit IRC | 17:57 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: run ansible plugins on undercloud https://review.opendev.org/699314 | 17:58 |
| openstackgerrit | Daniel Bengtsson proposed openstack/tripleo-upgrade master: Fix python command and fix python3 compatiblity. https://review.opendev.org/699391 | 18:02 |
| *** derekh has quit IRC | 18:04 | |
| *** mcornea has joined #tripleo | 18:05 | |
| *** rfolco has quit IRC | 18:07 | |
| openstackgerrit | Daniel Bengtsson proposed openstack/tripleo-upgrade master: Fix python command and fix python3 compatiblity. https://review.opendev.org/699391 | 18:07 |
| *** Chaserjim has quit IRC | 18:12 | |
| *** Chaserjim has joined #tripleo | 18:12 | |
| openstackgerrit | Damien Ciabrini proposed openstack/tripleo-heat-templates master: Limit concurrency in container-puppet to image pull https://review.opendev.org/699737 | 18:17 |
| *** yolanda__ is now known as yolanda | 18:18 | |
| *** tosky has quit IRC | 18:23 | |
| *** Goneri has quit IRC | 18:28 | |
| openstackgerrit | Damien Ciabrini proposed openstack/tripleo-heat-templates master: container-puppet: create containers sequentially to avoid race https://review.opendev.org/699737 | 18:37 |
| *** tmazur has quit IRC | 18:38 | |
| openstackgerrit | Damien Ciabrini proposed openstack/tripleo-heat-templates master: Limit concurrency in container-puppet to image pull https://review.opendev.org/699737 | 18:40 |
| openstackgerrit | Damien Ciabrini proposed openstack/tripleo-heat-templates master: container-puppet: create containers sequentially to avoid race https://review.opendev.org/699803 | 18:40 |
| *** openstackgerrit has quit IRC | 18:43 | |
| *** amoralej is now known as amoralej|off | 18:44 | |
| *** rfolco has joined #tripleo | 18:52 | |
| *** waleedm has quit IRC | 18:56 | |
| *** waleedm has joined #tripleo | 18:57 | |
| *** sshnaidm is now known as sshnaidm|afk | 18:58 | |
| *** openstackgerrit has joined #tripleo | 18:58 | |
| openstackgerrit | Sergii Golovatiuk proposed openstack/tripleo-upgrade master: Fix python command and fix python3 compatiblity. https://review.opendev.org/699391 | 18:58 |
| *** jainprasuk1996 has quit IRC | 19:01 | |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: Implement utils.copy_clouds_yaml https://review.opendev.org/699448 | 19:01 |
| *** mburned has joined #tripleo | 19:03 | |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: Implement utils.copy_clouds_yaml https://review.opendev.org/699448 | 19:04 |
| *** mcornea has quit IRC | 19:12 | |
| *** chem_ has joined #tripleo | 19:12 | |
| *** chem_ has quit IRC | 19:18 | |
| *** mcornea has joined #tripleo | 19:20 | |
| *** chem_ has joined #tripleo | 19:21 | |
| openstackgerrit | Sergii Golovatiuk proposed openstack/tripleo-upgrade master: Fix python command and fix python3 compatiblity. https://review.opendev.org/699391 | 19:21 |
| *** holser has quit IRC | 19:24 | |
| *** chem_ has quit IRC | 19:26 | |
| *** chem_ has joined #tripleo | 19:27 | |
| *** jbadiapa has quit IRC | 19:31 | |
| *** Goneri has joined #tripleo | 19:37 | |
| *** waleedm_ has joined #tripleo | 19:52 | |
| *** chem_ has quit IRC | 19:52 | |
| *** waleedm has quit IRC | 19:52 | |
| *** chem_ has joined #tripleo | 19:57 | |
| *** pkopec has quit IRC | 19:58 | |
| *** jpich has quit IRC | 20:01 | |
| *** waleedm_ has quit IRC | 20:08 | |
| Tengu | EmilienM: is gate happy again? | 20:14 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Programatically generate ssh hosts lines https://review.opendev.org/699502 | 20:14 |
| EmilienM | Tengu: not yet | 20:15 |
| EmilienM | but soon | 20:15 |
| EmilienM | https://review.opendev.org/#/c/699712/ is in gate | 20:15 |
| Tengu | what was the issue? the yaql? | 20:15 |
| EmilienM | once it merges, we should be good | 20:15 |
| *** rf0lc0 has joined #tripleo | 20:15 | |
| Tengu | oh. damn. | 20:15 |
| Tengu | that was a tricky one :( | 20:16 |
| *** rfolco has quit IRC | 20:17 | |
| Tengu | anyway. will check the state of my patches tomorrow. One of them at least has been restored and will help gate/zuul with the /var/log/journal directory. | 20:18 |
| Tengu | 'night all ;) | 20:19 |
| openstackgerrit | Merged openstack/diskimage-builder master: modprobe.d: use $TMP_MOUNT_PATH https://review.opendev.org/699722 | 20:29 |
| *** ramishra has quit IRC | 20:31 | |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: Revert "Added for support for generating overcloud clouds.yaml" https://review.opendev.org/699329 | 20:31 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/train: mysql: move firewall_rules to mysql-container-puppet https://review.opendev.org/699866 | 20:32 |
| *** slaweq has quit IRC | 20:36 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates stable/train: Revert "Drop EC2MetadataIp parameter and its uses" https://review.opendev.org/699136 | 20:40 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates stable/train: Optional enable undercloud nova-metadata-api https://review.opendev.org/698593 | 20:40 |
| *** rf0lc0 has quit IRC | 20:41 | |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: Implement utils.copy_clouds_yaml https://review.opendev.org/699448 | 20:42 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: DO-NOT-MERGE Test new ceph-container builds https://review.opendev.org/562213 | 20:42 |
| openstackgerrit | Sagi Shnaidman proposed openstack/ansible-role-collect-logs master: WIP: render UC ara results https://review.opendev.org/699867 | 20:42 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates stable/train: DO-NOT-MERGE Test new ceph-container builds (OSP train) https://review.opendev.org/690036 | 20:42 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates stable/train: DO-NOT-MERGE Test new ceph-container builds (OSP train) https://review.opendev.org/690036 | 20:42 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: WIP: run ansible plugins on undercloud https://review.opendev.org/699314 | 20:43 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates stable/stein: DO-NOT-MERGE Test new ceph-container builds (OSP stein) https://review.opendev.org/656934 | 20:43 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: DO-NOT-MERGE Test new ceph-container builds https://review.opendev.org/562213 | 20:43 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates stable/train: DO-NOT-MERGE Test new ceph-container builds (OSP train) https://review.opendev.org/690036 | 20:43 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates stable/stein: DO-NOT-MERGE Test new ceph-container builds (OSP stein) https://review.opendev.org/656934 | 20:43 |
| *** slaweq has joined #tripleo | 20:45 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates stable/queens: DO-NOT-MERGE Test new ceph-container builds (OSP queens) https://review.opendev.org/562215 | 20:45 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates stable/queens: DO-NOT-MERGE Test new ceph-container builds (OSP queens) https://review.opendev.org/562215 | 20:45 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates stable/train: Revert "Drop EC2MetadataIp parameter and its uses" https://review.opendev.org/699136 | 20:47 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates stable/train: Optional enable undercloud nova-metadata-api https://review.opendev.org/698593 | 20:47 |
| *** iurygregory has quit IRC | 20:53 | |
| *** holser has joined #tripleo | 20:54 | |
| *** waleedm_ has joined #tripleo | 20:59 | |
| *** waleedm__ has joined #tripleo | 21:00 | |
| *** waleedm_ has quit IRC | 21:03 | |
| *** ayoung has joined #tripleo | 21:12 | |
| *** raildo has quit IRC | 21:16 | |
| openstackgerrit | Toure Dunnon proposed openstack/tripleo-ansible master: [WIP] Add service management support. https://review.opendev.org/681017 | 21:16 |
| *** rf0lc0 has joined #tripleo | 21:26 | |
| *** rf0lc0 has quit IRC | 21:26 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-heat-templates stable/train: Create /var/log/journal directory during step-0 https://review.opendev.org/699438 | 21:31 |
| *** thrash has quit IRC | 21:35 | |
| *** thrash has joined #tripleo | 21:37 | |
| *** ansmith has quit IRC | 21:38 | |
| *** Chaserjim has quit IRC | 21:40 | |
| *** Chaserjim has joined #tripleo | 21:41 | |
| *** pcaruana has quit IRC | 21:54 | |
| *** saneax has quit IRC | 21:55 | |
| *** mcornea has quit IRC | 21:59 | |
| *** morazi has quit IRC | 22:02 | |
| *** dpawlik has quit IRC | 22:08 | |
| *** TrevorV has quit IRC | 22:10 | |
| *** jfrancoa has quit IRC | 22:10 | |
| *** chem_ has quit IRC | 22:11 | |
| *** nkinder has quit IRC | 22:12 | |
| *** Chaserjim has quit IRC | 22:17 | |
| *** Chaserjim has joined #tripleo | 22:18 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-validations master: Introduce Ceph placement group validation https://review.opendev.org/694673 | 22:19 |
| *** haleyb has quit IRC | 22:22 | |
| openstackgerrit | Alan Bishop proposed openstack/tripleo-common master: Add container images for DCN/Edge deployments https://review.opendev.org/699879 | 22:25 |
| openstackgerrit | Alan Bishop proposed openstack/tripleo-heat-templates master: Add ability to deploy glance at DCN/Edge sites https://review.opendev.org/699880 | 22:25 |
| *** haleyb has joined #tripleo | 22:26 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates master: Revert "Optional enable undercloud nova-metadata-api" https://review.opendev.org/699881 | 22:36 |
| *** ansmith has joined #tripleo | 22:44 | |
| *** saneax has joined #tripleo | 22:54 | |
| cloudnull | anyone around able to give https://review.opendev.org/#/q/topic:fw-update+(status:open+OR+status:merged) a push | 23:02 |
| *** tkajinam has joined #tripleo | 23:05 | |
| *** holser has quit IRC | 23:09 | |
| *** rcernin has joined #tripleo | 23:10 | |
| *** rlandy is now known as rlandy|bbl | 23:11 | |
| *** ccamacho has quit IRC | 23:23 | |
| *** saneax has quit IRC | 23:24 | |
| fultonj | cloudnull: i'm reviewing them | 23:24 |
| cloudnull | tyvm | 23:25 |
| fultonj | cloudnull: i +2'd both | 23:27 |
| cloudnull | thanks! | 23:27 |
| * fultonj looking at Emilien's email regarding not +w'ing though i think this is related to fixing that right? | 23:27 | |
| cloudnull | this is not | 23:28 |
| cloudnull | fixing a different bug | 23:28 |
| fultonj | http://lists.openstack.org/pipermail/openstack-discuss/2019-December/011671.html | 23:28 |
| fultonj | cloudnull: i won't +w those two then as per ^ | 23:29 |
| fultonj | cloudnull: at least not tonight | 23:29 |
| cloudnull | ++ | 23:30 |
| cloudnull | no worries I appreciate the reviews | 23:30 |
| fultonj | np | 23:30 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!