| *** rlandy|brb is now known as rlandy | 00:01 | |
| weshay|rover | rlandy, it should merge in a minute | 00:16 |
|---|---|---|
| rlandy | yay - here we go now | 00:17 |
| rlandy | Change has been successfully merged by Zuul | 00:18 |
| *** hamzy has quit IRC | 00:18 | |
| rlandy | thank you zuul | 00:18 |
| *** hamzy has joined #oooq | 00:19 | |
| rlandy | pinged paul with the merge status | 00:20 |
| rlandy | weshay|rover: thank you for getting that through | 00:24 |
| *** rlandy is now known as rlandy|bbl | 00:34 | |
| *** ChanServ has quit IRC | 01:00 | |
| *** ChanServ has joined #oooq | 01:03 | |
| *** barjavel.freenode.net sets mode: +o ChanServ | 01:03 | |
| *** dtrainor has quit IRC | 01:07 | |
| *** dtrainor has joined #oooq | 01:08 | |
| *** jjoyce has quit IRC | 01:46 | |
| *** jjoyce has joined #oooq | 01:47 | |
| *** hamzy has quit IRC | 02:28 | |
| *** panda has quit IRC | 02:28 | |
| *** rlandy|bbl has quit IRC | 02:28 | |
| *** chandankumar has quit IRC | 02:28 | |
| *** sshnaidm|off has quit IRC | 02:28 | |
| *** hamzy has joined #oooq | 02:30 | |
| *** panda has joined #oooq | 02:30 | |
| *** rlandy|bbl has joined #oooq | 02:30 | |
| *** chandankumar has joined #oooq | 02:30 | |
| *** sshnaidm|off has joined #oooq | 02:30 | |
| *** openstack has joined #oooq | 13:19 | |
| *** ChanServ sets mode: +o openstack | 13:19 | |
| *** openstackstatus has joined #oooq | 13:21 | |
| *** ChanServ sets mode: +v openstackstatus | 13:21 | |
| *** openstackstatus has quit IRC | 13:22 | |
| *** openstackstatus has joined #oooq | 13:23 | |
| *** ChanServ sets mode: +v openstackstatus | 13:23 | |
| tosky | anyone else would like to add a +w to this review? https://review.openstack.org/#/c/509554/ | 13:32 |
| tosky | it has two +2 already | 13:32 |
| rlandy | tosky: looking | 13:32 |
| rlandy | done | 13:34 |
| tosky | thanks! | 13:35 |
| rlandy | rfolco: weshay|rover: should we live dangerously and merge https://review.rdoproject.org/r/#/c/15125 and https://review.rdoproject.org/r/#/c/15097? | 13:35 |
| rlandy | https://review.openstack.org/#/c/581488 is needed for reproducer | 13:36 |
| rlandy | but we can still test the job | 13:36 |
| *** amoralej|lunch is now known as amoralej | 13:39 | |
| weshay|rover | rlandy, you talk to paul? | 13:44 |
| rlandy | weshay|rover: no talking with nhicher | 13:44 |
| rlandy | on #sf-dfg | 13:44 |
| rlandy | I can wait for paul if you prefer | 13:44 |
| rlandy | he's the support person now afaict | 13:45 |
| weshay|rover | go 4 it | 13:45 |
| rlandy | weshay|rover: rfolco: ykarel: could I get some reviews on https://review.rdoproject.org/r/#/c/15125 and https://review.rdoproject.org/r/#/c/15097 (looking to merge) | 13:52 |
| rlandy | see discussion above and on #sf-dfg | 13:52 |
| rlandy | weshay|rover: ok - here we go - let's see what damage we do | 13:59 |
| bogdando | o/ | 14:01 |
| bogdando | should the oooq_* flavor names quickstart creates be mathching those generated in http://git.openstack.org/cgit/openstack/tripleo-quickstart/tree/roles/libvirt/setup/overcloud/templates/instackenv.json.j2#n38)? | 14:02 |
| ykarel | rlandy, sorry out of context for me, so u can try if earlier issues are fixed | 14:02 |
| bogdando | I think I have a mismatch, as my instack env sets profiles to compute and control and not oooq_compute/control | 14:02 |
| *** jaosorior has joined #oooq | 14:02 | |
| bogdando | right, I see config/nodes/1ctlr_1comp.yml uses node.flavor w/o that oooq_ prefix... | 14:03 |
| bogdando | still trying to add 2 + 2 to have something working with a simple command like quickstart.sh -N config/nodes/1ctlr_1comp.yml virthost :) | 14:04 |
| bogdando | but I have it 5 or 3 | 14:04 |
| bogdando | not 4 ;( | 14:04 |
| rascasoft | rlandy, if you have time for this https://code.engineering.redhat.com/gerrit/#/c/148001/ | 14:04 |
| rascasoft | thanks a ton :) | 14:04 |
| bogdando | shouldn't those in config/nodes refer to oooq_* flavors? | 14:05 |
| bogdando | what is the expected use case for those if not being used with the built in role to generate flavors? | 14:05 |
| bogdando | and what am I missing to go with node config as is, w/o overriding dances ? | 14:07 |
| bogdando | may be I just need | 14:08 |
| bogdando | flavor_map: | 14:08 |
| bogdando | control: control | 14:08 |
| bogdando | compute: compute? | 14:08 |
| rlandy | waiting for https://review.rdoproject.org/r/#/c/15125/ to merge ... | 14:09 |
| rlandy | rascasoft: your other review merged? | 14:10 |
| rascasoft | rlandy, yes milady | 14:10 |
| rlandy | awesoem | 14:10 |
| rlandy | https://code.engineering.redhat.com/gerrit/#/c/148001/ merged | 14:10 |
| rlandy | bogdando: reading back ... | 14:11 |
| bogdando | long story short, trying to deploy with | 14:12 |
| bogdando | flavor_map: | 14:12 |
| bogdando | control: control | 14:12 |
| bogdando | compute: compute | 14:12 |
| bogdando | step_overcloud_prep_flavors: false | 14:12 |
| rlandy | as long as the flavor can fit into what is defined in instackenv you are ok | 14:12 |
| bogdando | IIUC that's the only suitable set matching the flavor names in node/config templates? | 14:12 |
| bogdando | nice | 14:12 |
| rlandy | so .. do those flavors exist | 14:12 |
| bogdando | yes | 14:12 |
| rlandy | ie: openstack flavor list | 14:13 |
| bogdando | yeah | 14:13 |
| rlandy | what is returned? | 14:13 |
| bogdando | rlandy: https://pastebin.com/Ycqq70v3 | 14:13 |
| rlandy | if your nodes are not sufficient, you will get a deploy error | 14:13 |
| bogdando | right | 14:14 |
| rascasoft | rlandy++++ | 14:14 |
| bogdando | rlandy++ | 14:14 |
| bogdando | :) | 14:14 |
| * rlandy checks your instackenv - sec | 14:14 | |
| bogdando | so I was only thinking of the defaults to let oooq users to deploy w/o additional overrides | 14:14 |
| bogdando | just shoot -N and voila! | 14:14 |
| bogdando | so perhaps we need to rebind those to oooq_ ? generated ones? | 14:15 |
| rlandy | hol don | 14:15 |
| bogdando | kk | 14:15 |
| rlandy | bogdando: you may hit a problem with disk | 14:16 |
| rlandy | can you paste openstack baremetal node list | 14:16 |
| rlandy | and an ironic node-show on one node | 14:16 |
| rlandy | you need a bigger disk than the flavor specified | 14:17 |
| rlandy | even by 1GB is fine | 14:17 |
| rlandy | let's just check that | 14:17 |
| ssbarnea|ruck | weshay|rover sadly timeout seems to be consistent, even after recent changes, 100% failure rate http://zuul.openstack.org/builds.html?job_name=tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates --- not sure why this was not visible in dashboard. | 14:17 |
| rlandy | ugh - what do I need to get a merge on rdo-jobs??? | 14:20 |
| rlandy | bogdando: you with me? | 14:20 |
| bogdando | rlandy: yeah | 14:21 |
| ssbarnea|ruck | rlandy do you know how can I see older zuul logs? the default pagination does not include any successful build and I want to find one to compare with the failing one. | 14:21 |
| rlandy | ssbarnea|ruck: which job? | 14:21 |
| rlandy | I can find you one | 14:21 |
| ssbarnea|ruck | see prev link I posted | 14:21 |
| ssbarnea|ruck | the updates one that does timeout | 14:21 |
| rlandy | let me look | 14:21 |
| bogdando | | 0f8f277e-5ac7-47d6-b5e2-56b5e9f5c791 | control-0 | None | power off | clean failed | True | | 14:21 |
| bogdando | | properties | {u'memory_mb': u'4096', u'cpu_arch': u'x86_64', u'local_gb': u'19', u'cpus': u'2', u'capabilities': u'profile:control,cpu_vt:true,cpu_hugepages:true,boot_option:local,cpu_aes:true,cpu_hugepages_1g:true'} | 14:21 |
| bogdando | I thikn I have flavors mismatch :/ | 14:22 |
| rlandy | u'local_gb': u'19' | 14:22 |
| rlandy | ^^ too small | 14:22 |
| bogdando | so to fix the things, we prolly need to relink all node/config files to use the generated oooq_ flavors | 14:22 |
| bogdando | that worths a UX bug IMO | 14:22 |
| ssbarnea|ruck | well, i think I found a trick, I filtered by project and I ended up seeing one from 21st | 14:22 |
| rlandy | ssbarnea|ruck: you all set? | 14:22 |
| ssbarnea|ruck | rlandy: yeah, no need to bother. unless you know how to fix the timeout issue on updates ;) | 14:23 |
| rlandy | I see a successful job from 22 | 14:23 |
| rlandy | ssbarnea|ruck: not off hand:( | 14:24 |
| rlandy | bogdando: the rest looks ok - you just need a big enough node | 14:25 |
| rlandy | http://logs.openstack.org/63/593063/9/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/4640435/ | 14:25 |
| rlandy | ssbarnea|ruck: ^^ that is the last successful job I can see | 14:25 |
| rlandy | after that consistent failure | 14:25 |
| bogdando | rlandy: thanks! | 14:26 |
| * bogdando creates a bug | 14:26 | |
| *** jaosorior has quit IRC | 14:33 | |
| bogdando | rlandy: https://bugs.launchpad.net/tripleo/+bug/1788875 | 14:34 |
| openstack | Launchpad bug 1788875 in tripleo "quickstart creates oooq_foo flavors which do not match config/node templated names" [Undecided,New] | 14:34 |
| bogdando | WDYT? | 14:34 |
| bogdando | sounds fair? | 14:35 |
| bogdando | and an example for the path B https://github.com/bogdando/oooq-warp/commit/2ce6553f30873805cfa23efe21e3b9fb1b7582a2 | 14:36 |
| ssbarnea|ruck | rlandy : maybe you can give a hint here https://etherpad.openstack.org/p/ssbarnea | 14:38 |
| rlandy | sec - just helping rasca | 14:39 |
| ykarel | ssbarnea|ruck, weshay|rover fs018 failed in master and rocky twice, any bug for it? | 14:40 |
| ssbarnea|ruck | nope | 14:40 |
| ykarel | ack, good to have it as it's blocking both promotions now | 14:41 |
| weshay|rover | ykarel, http://dashboard-ci.tripleo.org/d/cEEjGFFmz/cockpit?orgId=1 | 14:46 |
| weshay|rover | has a list of promotion blockers | 14:46 |
| weshay|rover | and alerts | 14:46 |
| ykarel | weshay|rover, but i can't see fs018 there | 14:48 |
| weshay|rover | ssbarnea|ruck, ^ | 14:50 |
| weshay|rover | ssbarnea|ruck, please write up the bugs on these | 14:50 |
| *** ykarel is now known as ykarel|away | 14:55 | |
| rlandy | weshay|rover: do you want to 1-1 or too busy today? | 15:01 |
| weshay|rover | rlandy, need a few | 15:02 |
| rlandy | k - no rush | 15:02 |
| *** trown has joined #oooq | 15:02 | |
| ssbarnea|ruck | ykarel|away : can you give me a link to the fs018 failure? | 15:06 |
| *** ssbarnea|ruck has quit IRC | 15:07 | |
| rlandy | woohoo - we have a job trigger | 15:08 |
| weshay|rover | rlandy, ready in 3min | 15:08 |
| ykarel|away | ssbarnea|ruck, see last failure in: https://review.rdoproject.org/zuul/builds.html?job_name=legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset018-master and https://review.rdoproject.org/zuul/builds.html?job_name=legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset018-rocky and two more from current run(openstack-periodic):- https://review.rdoproject.org/zuul/status.html | 15:09 |
| weshay|rover | rlandy, k | 15:11 |
| weshay|rover | https://bluejeans.com/u/whayutin/ | 15:11 |
| bogdando | weshay|rover: hi! https://review.openstack.org/#/c/594133/2 looks gtg | 15:13 |
| bogdando | good to goooo | 15:13 |
| bogdando | also folks, WDYT of https://bugs.launchpad.net/tripleo/+bug/1788875 | 15:13 |
| openstack | Launchpad bug 1788875 in tripleo "quickstart creates oooq_foo flavors which do not match config/node templated names, that fails the overcloud deployment" [High,Triaged] | 15:13 |
| bogdando | did I describe it right? | 15:14 |
| *** dbecker has joined #oooq | 15:16 | |
| *** jfrancoa has quit IRC | 15:21 | |
| *** kopecmartin has quit IRC | 15:21 | |
| *** radez has joined #oooq | 15:23 | |
| rascasoft | ssbarnea, have you got 5 mins to discuss an issue I'm having? | 15:29 |
| *** ykarel|away has quit IRC | 15:30 | |
| weshay|rover | ssbarnea, need any help? | 15:32 |
| *** amoralej is now known as amoralej|off | 15:38 | |
| *** amoralej|off has quit IRC | 15:39 | |
| ssbarnea | wes, yep but let me finsh “late” breakfast first, see etherpad link | 15:44 |
| weshay|rover | ssbarnea, which this one? https://review.rdoproject.org/etherpad/p/ruckrover-sprint18 | 15:48 |
| *** dtantsur is now known as dtantsur|afk | 16:01 | |
| weshay|rover | rascasoft, you still around buddy? | 16:03 |
| bogdando | weshay|rover: ditto https://review.openstack.org/#/c/593103 | 16:04 |
| bogdando | good to go, imo | 16:04 |
| rascasoft | weshay|rover, here I am | 16:08 |
| rascasoft | fighting with pip, cache and cleanings | 16:08 |
| rascasoft | (loosing, of course) | 16:09 |
| *** bogdando has quit IRC | 16:11 | |
| weshay|rover | rascasoft, want to try battling code.eng again? also what's the status on https://bugs.launchpad.net/tripleo/+bug/1774653 | 16:11 |
| openstack | Launchpad bug 1774653 in tripleo "HA test failed because it is unable to create an instance due to block device mapping invalid (failed to get volume)" [Critical,Triaged] | 16:11 |
| weshay|rover | rascasoft, alex wants to remove promotion blocker | 16:12 |
| rascasoft | weshay|rover, about codeng we should be done | 16:13 |
| rascasoft | weshay|rover, about the bug: I'd agree if I had the opportunity to run HA validations consistently (which is not the case) | 16:13 |
| *** udesale has quit IRC | 16:19 | |
| weshay|rover | rascasoft, ok.. I'm happy to support what ever you think is right.. just let alex know one way or another | 16:20 |
| rascasoft | weshay|rover, you need me to comment the bug? | 16:21 |
| weshay|rover | rascasoft, and or ping alex | 16:22 |
| rascasoft | weshay|rover, first step is to have consistent runs for master | 16:22 |
| rascasoft | weshay|rover, I'll do it immediately, master | 16:22 |
| weshay|rover | rascasoft, sounds like it should be open but not a blocker | 16:22 |
| weshay|rover | imho | 16:22 |
| weshay|rover | depending on why it's inconsistent | 16:22 |
| rascasoft | weshay|rover, it's inconsistent because of all the CI integration problems I had to deal with in the last months, not for other reasons | 16:23 |
| rlandy | weshay|rover: job never started :( - checking on #sf-dfg | 16:26 |
| weshay|rover | ugh k | 16:26 |
| weshay|rover | rascasoft, so.. ya.. we need you in at the tripleo-ci layer | 16:27 |
| weshay|rover | getting there | 16:27 |
| weshay|rover | rascasoft, I would reraise it as a blocker once we're able to rebuild the issue consistently | 16:27 |
| weshay|rover | rascasoft, it can be a high priority bug w/o being a prod chain blocker | 16:28 |
| weshay|rover | or critical | 16:28 |
| rascasoft | weshay|rover, I obey my lord. | 16:28 |
| weshay|rover | lolz | 16:28 |
| rlandy | rfolco: any ideas?? pls see #sf-dfg | 16:29 |
| rascasoft | weshay|rover, commented the bug and removed the tag | 16:29 |
| rfolco | rlandy, looking | 16:32 |
| rfolco | rlandy, where did you define that the job runs in check pipeline for browbeat ? | 16:33 |
| rlandy | rfolco: <rlandy> the job is defined here: https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/zuul-v3-jobs.yaml | 16:34 |
| rfolco | rlandy, no, I mean, the pipeline | 16:34 |
| rfolco | check: <my_job> | 16:34 |
| rlandy | <rlandy> the job is defined here: https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/zuul-v3-jobs.yaml | 16:36 |
| rlandy | sorry | 16:36 |
| rlandy | https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/tripleo.yaml#L896 | 16:36 |
| rlandy | rfolco: ^^ | 16:37 |
| *** gkadam has quit IRC | 16:38 | |
| rfolco | rlandy, hmm ... I am checking this https://github.com/rdo-infra/review.rdoproject.org-config/blob/aa3cd2bc33c8a441542e72ac9d6fcbc6c49629fe/zuul.d/upstream.yaml#L17 | 16:42 |
| rfolco | trying to find another example that works... one project there that is in openstack-check pipeline | 16:43 |
| rlandy | rfolco: leagcy 053 s is only defined there as weel | 16:43 |
| rlandy | well | 16:43 |
| rfolco | my point is the full namespace git.openstack.org vs openstack.org on trigger: | 16:43 |
| rfolco | its just the same as legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset042-master-tht | 16:47 |
| rfolco | checking if this job triggers on tht | 16:48 |
| rfolco | hard to find file filter | 16:49 |
| rascasoft | rlandy, it failed while preparing the images https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/oooq-master-rdo_trunk-bmu-ha-lab-cygnus-float_nic_with_vlans/20/console | 16:49 |
| rascasoft | we'll never come to an end | 16:50 |
| rlandy | rascasoft: need to deal wit upstream now - will take a look later | 16:50 |
| rlandy | rfolco: the job is listed in zuul v3 | 16:50 |
| rlandy | it just never started | 16:50 |
| *** gkadam has joined #oooq | 16:52 | |
| weshay|rover | ssbarnea, http://logs.openstack.org/40/591540/8/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/fa9fe4f/logs/reproducer-quickstart.sh | 16:56 |
| rfolco | rlandy, ah you mean you see the job listed in zuul status page but it never runs or shows any progress ? | 16:57 |
| rlandy | yes - remains queued | 16:58 |
| rlandy | which usually means the definition is not found | 16:58 |
| rlandy | but defined https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/zuul-v3-jobs.yaml#L2 | 16:59 |
| rlandy | unless I missed something | 16:59 |
| *** holser_ has quit IRC | 16:59 | |
| rlandy | weshay|rover: ^^ ideas??? | 16:59 |
| weshay|rover | ssbarnea, https://trello.com/c/tFhFcqor/901-sprint-18-ruck-rover | 17:02 |
| *** dsneddon has quit IRC | 17:05 | |
| *** trown is now known as trown|lunch | 17:06 | |
| rfolco | rlandy, still trying to understand ... I see the job queued there... odd | 17:16 |
| weshay|rover | rlandy, /me reads | 17:16 |
| rlandy | rfolco: talking with nhicher on #sf-dfg | 17:17 |
| weshay|rover | rlandy, re: 596370 ? | 17:17 |
| rlandy | weshay|rover: yep - but talking with rdo infra now | 17:18 |
| rfolco | rlandy, do we have a card for reparenting or can I add your patch to the same card as moving extra playbook to job config ? if so, I add you ass member there | 17:35 |
| *** tosky has quit IRC | 17:36 | |
| rfolco | oh jeez | 17:37 |
| rfolco | sorry s/ass/add | 17:37 |
| rfolco | english can be embarrassing sometimes | 17:37 |
| rlandy | https://trello.com/c/TVsZ3Ut6/877-clean-up-rdo-sf-legacy-code-after-zuulv3-migration | 17:38 |
| rfolco | oh, I see... the thing is: your patch is a dependency for my card too, will add there just for reference | 17:38 |
| rfolco | thanks rlandy | 17:38 |
| rascasoft | rlandy, I think I've found where's the problem, for some reason in the config "download_overcloud_image" is not set to true, which causes the process of downloading the images not working and breaks the deploy | 17:39 |
| rlandy | rfolco: do you have +2 on https://review.rdoproject.org/r/#/c/15924/? | 17:40 |
| rascasoft | rlandy, I'm testing it right now by passing an extra var and that step (I'm testing locally on the jenkins workspace using tag overcloud-prep-images) seem to work | 17:40 |
| rascasoft | rlandy, it would be useful to understand why I didn't need it before | 17:40 |
| rfolco | rlandy, sorry did not follow discussion, why revert? | 17:40 |
| rascasoft | rlandy, shouldn't this be part of the usual master config? | 17:41 |
| rfolco | rlandy, I do not have +2 there | 17:41 |
| rlandy | rfolco: paul wants us to test with depens on | 17:41 |
| rlandy | shoot | 17:41 |
| rlandy | I am the only one who does :( | 17:41 |
| ssbarnea | weshay|rover: https://bugs.launchpad.net/tripleo/+bug/1788907 -- added fs018 failure info there - Failed running docker-puppet.py for designate, some hints would be needed. | 17:42 |
| openstack | Launchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Undecided,New] | 17:42 |
| ssbarnea | weshay|rover: please +W https://review.openstack.org/#/c/582501/ (verbose resize failures) as I am again slowed down by it while debugging other builds. | 17:48 |
| rascasoft | rlandy, https://code.engineering.redhat.com/gerrit/148025 added also that download part | 17:48 |
| rlandy | rascasoft: reading back | 17:50 |
| rascasoft | rlandy, in any case it failed again. Is there any other option for ./ironic-python-agent.initramfs ? | 17:51 |
| *** saneax has joined #oooq | 17:51 | |
| rascasoft | rlandy, I'm really out of energies and ideas | 17:52 |
| rlandy | sorry - we're just trying to migrate rdo to v3 | 17:53 |
| rascasoft | rlandy, I really don't know what is going on here | 17:55 |
| rascasoft | rlandy, I'm turning off, too late here, see you next week | 17:55 |
| rlandy | rascasoft: ok - I'll look at it more this afternoon | 17:55 |
| rascasoft | rlandy, ok if you find something and I should do something please comment the review, I'll work on it | 17:56 |
| *** dsneddon has joined #oooq | 18:00 | |
| ssbarnea | rlandy: for some reason on my machine (repro') virt-resize fails because files are owned by root, running same task as root works, but inside reproducer we do not have any sudo: yes. | 18:11 |
| ssbarnea | my personal take is that we should need root access but at the same time I wonder how these file endup owned by root. i wonder if this a libvirt misconfig on my machine. | 18:12 |
| rlandy | same thing happened to me yesterday | 18:13 |
| rlandy | I use a diff dir for images | 18:13 |
| rlandy | and set permissions on that dir | 18:13 |
| rlandy | and then pass that dir as an extra-var | 18:13 |
| *** trown|lunch is now known as trown | 18:16 | |
| ssbarnea | rlandy: this sounds like a serious bug in reproducer, one that impacts its sole purpose. time to dig a little bit on how to fix it. | 18:17 |
| ssbarnea | i need to find out who creates this files as root as i didn't see any become in fetch images code. | 18:18 |
| rlandy | ssbarnea: ok - just kind of busy with rdo v3 atm and then I have to help with rascasoft's work so whatever you think | 18:18 |
| ssbarnea | thanks, it really helped to get conf from you about this. at least I know is something to fix in repro. | 18:19 |
| *** gkadam has quit IRC | 18:30 | |
| *** rlandy is now known as rlandy|brb | 18:32 | |
| *** rlandy|brb is now known as rlandy | 18:50 | |
| *** saneax has quit IRC | 19:04 | |
| *** dsneddon has quit IRC | 19:06 | |
| *** dsneddon has joined #oooq | 19:07 | |
| weshay|rover | rlandy, need anything? | 19:09 |
| weshay|rover | ssbarnea, you still around/ | 19:11 |
| weshay|rover | ssbarnea, fyi.. https://bugs.launchpad.net/tripleo/+bug/1788907 is a promotion blocker, not an alert | 19:11 |
| openstack | Launchpad bug 1788907 in tripleo "Failed running docker-puppet.py for designate" [Undecided,New] | 19:11 |
| weshay|rover | because it's only impacting the promotion jobs | 19:11 |
| ssbarnea | ahh, ok. I added a line about this on your google doc. | 19:14 |
| rlandy | weshay|rover: just going through failure from test on removing legacy | 19:27 |
| *** apetrich has quit IRC | 19:32 | |
| weshay|rover | rlandy, so it finally kicked? | 19:33 |
| rlandy | weshay|rover: yep and failed ... | 19:33 |
| weshay|rover | ping for eyes | 19:33 |
| rlandy | see discussion on #sf-dfg | 19:33 |
| rlandy | <pabelanger> 2018-08-24 17:35:14,263 DEBUG zuul.AnsibleJob: [build: 5a7230d967cb4928bde619dfc92fa4b0] Ansible output: b"ERROR! the role 'legacy-copy-jenkins-scripts' was not found in | 19:33 |
| rlandy | <pabelanger> /tmp/tmp2yxuinem/5a7230d967cb4928bde619dfc92fa4b0/untrusted/project_3/git.openstack.org/openstack-infra/tripleo-ci/playbooks/openstack-zuul-jobs/legacy/roles:/tmp/tmp2yxuinem/5a7230d967cb4928bde619dfc92fa4b0/ansible/pre_playbook_5/role_1/rdo-jobs/roles:/tmp/tmp2yxuinem/5a7230d967cb4928bde619dfc92fa4b0/ansible/pre_playbook_5/role_3/zuul-jobs/roles:/tmp/tmp2yxuinem/5a7230d967cb4928bde619dfc92fa4 | 19:33 |
| rlandy | <pabelanger> b0/untrusted/project_3/git.openstack.org/openstack-infra/tripleo-ci/playbooks/openstack-zuul-jobs/legacy | 19:33 |
| rlandy | weshay|rover: see comment left here ... https://review.openstack.org/#/c/578432/8/zuul.d/base.yaml | 19:34 |
| rlandy | so I tried to remove it - not so simple - we get failures | 19:35 |
| rlandy | weshay|rover: tried this test patch ... https://review.openstack.org/#/c/596422 - no good | 19:36 |
| rlandy | will have to work on what is really needed there | 19:36 |
| weshay|rover | bah | 19:38 |
| rlandy | so that is where we are | 19:39 |
| rlandy | weshay|rover: last comment there was ... | 19:43 |
| rlandy | <pabelanger> rlandy: that info can we read from the inventory now | 19:43 |
| rlandy | so we would need to rewrite that role to use inventory | 19:44 |
| weshay|rover | rlandy, I don't think this worked https://review.rdoproject.org/r/#/c/15902/2/zuul.d/zuul-legacy-jobs.yaml | 20:05 |
| weshay|rover | rlandy, bah.. I added it to periodic :( | 20:07 |
| rlandy | lol | 20:08 |
| rlandy | weshay|rover: ok - judgment call here ... | 20:08 |
| weshay|rover | rlandy, https://review.rdoproject.org/r/#/c/15927/1/zuul.d/zuul-legacy-jobs.yaml | 20:09 |
| weshay|rover | rlandy, which role do we need to rewrite? | 20:09 |
| rlandy | weshay|rover: sec - back to your change ... | 20:09 |
| rlandy | since 3rd party does not vote anyways, why the voting: false change? | 20:10 |
| rlandy | we can merge with rdocloud -1 | 20:10 |
| weshay|rover | rlandy, right | 20:10 |
| weshay|rover | rlandy, to get folks to respect the -1 | 20:10 |
| weshay|rover | for humans | 20:11 |
| * rlandy does not have faith in human respect | 20:11 | |
| rlandy | I mean there is nothing wrong with the change | 20:11 |
| rlandy | but we should fix it | 20:11 |
| weshay|rover | rlandy, we are trying | 20:11 |
| weshay|rover | it's not easy | 20:11 |
| rlandy | ok - I +2'ed | 20:13 |
| rlandy | can you w+1 on this or only I can? | 20:13 |
| weshay|rover | rlandy, I think you have to | 20:14 |
| weshay|rover | thanks | 20:14 |
| rlandy | weshay|rover: done | 20:16 |
| rlandy | weshay|rover: ok - now back to my question ... | 20:16 |
| weshay|rover | k | 20:16 |
| rlandy | weshay|rover: I put in this change ... https://review.openstack.org/#/c/596422/2/playbooks/openstack-zuul-jobs/legacy/pre.yaml | 20:16 |
| rlandy | which so far is working ok | 20:16 |
| rlandy | isn't not a complete rewrite of legacy | 20:17 |
| rlandy | but it does deal with the eeror | 20:17 |
| rlandy | error | 20:17 |
| rlandy | your call if I should try rewrite that role | 20:17 |
| rlandy | or see if this is good enough to get us by | 20:17 |
| weshay|rover | rlandy, looks for it | 20:17 |
| rlandy | weshay|rover: see progress so far on http://zuul.openstack.org/ | 20:18 |
| rlandy | let me know what you think | 20:18 |
| weshay|rover | rlandy, why was that not | 20:18 |
| weshay|rover | role: legacy-copy-jenkins-scripts | 20:19 |
| weshay|rover | and where is the src for that | 20:19 |
| * weshay|rover looking | 20:19 | |
| rlandy | openstack-zuul-jobs | 20:19 |
| * weshay|rover sees it called here in the same way http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/playbooks/legacy/pre.yaml#n6 | 20:20 | |
| rlandy | that is the issue | 20:20 |
| weshay|rover | oh reallhy | 20:20 |
| weshay|rover | lolz | 20:20 |
| weshay|rover | fak | 20:20 |
| rlandy | causes a dep on untrusted/project_3/git.openstack.org/openstack-infra/tripleo-ci/playbooks/openstack-zuul-jobs/legacy | 20:21 |
| rlandy | role 'legacy-copy-jenkins-scripts' was not found in | 20:22 |
| weshay|rover | rlandy, looks like it's pasisng | 20:22 |
| * weshay|rover has a thought | 20:22 | |
| rlandy | I just removed that call to see what it impacted | 20:22 |
| rlandy | this is like whack-a-mole | 20:22 |
| rlandy | weshay|rover: it wasn't passing when I removed the whole playbook | 20:24 |
| rlandy | but this could work | 20:24 |
| weshay|rover | rlandy, ya.. we need pre | 20:25 |
| weshay|rover | if that's what you mean | 20:25 |
| rlandy | yep | 20:26 |
| weshay|rover | rlandy, another way to see if we need that.. https://review.openstack.org/596469 | 20:26 |
| weshay|rover | https://github.com/openstack-infra/openstack-zuul-jobs/commit/842f2416183778a4c0052edc338708fb180d1941 | 20:26 |
| rlandy | weshay|rover: not sure how that would help - we need to remove our call | 20:28 |
| rlandy | anyways - I'll have to ask paul if that is good enough a change or of he wants legacy gone altogether | 20:28 |
| weshay|rover | rlandy, ya.. just want to see what breaks | 20:31 |
| weshay|rover | lolz | 20:31 |
| rlandy | me too | 20:32 |
| rlandy | nothing like waiting for an explosion | 20:32 |
| rlandy | weshay|rover: were you tracking rascasoft's stuff at all? | 20:48 |
| weshay|rover | rlandy, I ff'd the internal branches for him | 20:48 |
| weshay|rover | rlandy, that is all I know | 20:48 |
| rlandy | weshay|rover: k - looks like he's getting as far as prep-images | 20:49 |
| weshay|rover | rlandy, which job? | 20:49 |
| rlandy | ttps://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/oooq-master-rdo_trunk-bmu-ha-lab-cygnus-float_nic_with_vlans/20/consoleFull | 20:49 |
| rlandy | with an h | 20:49 |
| rlandy | better than before - checking his error | 20:51 |
| weshay|rover | rlandy, hrm.. that worked on our jobs | 20:51 |
| weshay|rover | rlandy, https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rdo-promote-master-rdo_trunk/247/ | 20:51 |
| weshay|rover | why doesn't he have logs :( | 20:52 |
| weshay|rover | rascasoft, man.. stegatzi | 20:52 |
| weshay|rover | :) | 20:52 |
| rlandy | may be my fault :( | 20:53 |
| rlandy | I'll fix that | 20:53 |
| rlandy | jenkins, why are you sooooooo slow?? | 20:53 |
| weshay|rover | rlandy, his jobs were passing in your spot checks w/ fs001 right? | 20:54 |
| weshay|rover | could be his config fell behind again | 20:54 |
| weshay|rover | --config $WORKSPACE/tripleo-environments/hardware_environments/ha-lab-cygnus/network_configs/float_nic_with_vlans/config_files/config.yml \ | 20:54 |
| rlandy | I have no idea | 20:54 |
| rlandy | ok - will collect logs now | 20:55 |
| rlandy | retriggering | 20:55 |
| rlandy | lucky 21 | 20:56 |
| weshay|rover | rlandy, did the osp-13 job ever work for you? | 20:58 |
| rlandy | weshay|rover: died on overcloud-deploy - because if node cleaning | 20:59 |
| rlandy | I was at the point of removing node cleaning and running agin | 20:59 |
| rlandy | can trigger that in a few | 20:59 |
| rlandy | you see what I mean about dumping stuff?? | 20:59 |
| weshay|rover | rlandy, ya.. we have you on too much | 21:01 |
| weshay|rover | rlandy, again | 21:01 |
| weshay|rover | I think if we can close out zuulv3.. on rdo things will fall back into place | 21:01 |
| weshay|rover | the size of creating jobs for browbeat/ha-utils grew x5 after paul migrated zuul | 21:02 |
| weshay|rover | planning is hard.. when the physics you plan w/ change | 21:02 |
| rlandy | weshay|rover: sec - mom on phone | 21:06 |
| weshay|rover | rlandy, say hi and good shaboos | 21:06 |
| *** trown is now known as trown|outtypewww | 21:08 | |
| rlandy | weshay|rover: k - back ... not a big deal on rhos-13 - I have the change - will just trigger again | 21:18 |
| rlandy | rascasoft's job should collect logs if it fails again | 21:18 |
| weshay|rover | rlandy++ | 21:23 |
| weshay|rover | rlandy, I think ur good on https://review.openstack.org/#/c/596422/ | 21:30 |
| rlandy | weshay|rover: ack - going to ping paul about it on #sf-dfg in a sec - just clearing ws on downstream | 21:31 |
| rlandy | weshay|rover: https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/tq-gate-rhos-13-ci-rhos-ovb-featureset001/147/console - with no node cleaning | 21:35 |
| weshay|rover | rlandy, and we want that across all versions ? no | 21:36 |
| weshay|rover | now | 21:36 |
| rlandy | weshay|rover: remove cleaning - no | 21:36 |
| weshay|rover | rlandy, /me looking at https://review.openstack.org/#/c/577230/17/config/general_config/featureset001.yml | 21:37 |
| rlandy | just testing -the job is dnm | 21:37 |
| weshay|rover | k | 21:37 |
| rlandy | I can put that in the ci-rhos config if need be | 21:37 |
| rlandy | this will just give us the chance to test it | 21:37 |
| weshay|rover | the only change I see is ~/network.yml | 21:37 |
| weshay|rover | network-env.yml | 21:38 |
| rlandy | undercloud_clean_nodes: false | 21:38 |
| rlandy | plus the two changes we have in tqe | 21:38 |
| weshay|rover | ya.. but for queens, rocky, master | 21:38 |
| weshay|rover | that was off | 21:38 |
| weshay|rover | osp-13 is queens | 21:39 |
| rlandy | it was on | 21:39 |
| rlandy | I don;t plan to merge this | 21:39 |
| rlandy | just test with it | 21:39 |
| weshay|rover | https://github.com/openstack/tripleo-quickstart/blob/master/config/general_config/featureset001.yml#L43 | 21:39 |
| weshay|rover | rlandy, if the internal job is using fs001 | 21:40 |
| weshay|rover | it should be off the way I read it | 21:40 |
| weshay|rover | ? | 21:40 |
| rlandy | undercloud_clean_nodes should be true for queens | 21:40 |
| weshay|rover | OH | 21:40 |
| rlandy | in the merged version | 21:40 |
| rlandy | and false in my test version | 21:40 |
| rlandy | cleaning is failing | 21:40 |
| rlandy | and leaving the nodes in a bad state | 21:41 |
| * weshay|rover is confused | 21:41 | |
| weshay|rover | but ok | 21:41 |
| rlandy | causing 'no available node' errors | 21:41 |
| weshay|rover | so.. hrm | 21:41 |
| rlandy | I am trying to skip the cleaning | 21:41 |
| weshay|rover | k k | 21:41 |
| rlandy | and hope that leaves the nodes in a good state to deploy | 21:41 |
| rlandy | make sense? | 21:41 |
| weshay|rover | well.. from what you say yes | 21:41 |
| *** rfolco has quit IRC | 21:41 | |
| rlandy | the nodes get left in clean failed | 21:42 |
| weshay|rover | from your change no | 21:42 |
| weshay|rover | understood | 21:42 |
| rlandy | I need them in available to deploy | 21:42 |
| weshay|rover | undercloud_clean_nodes: >- | 21:42 |
| weshay|rover | {% if release not in ['newton','ocata','pike'] -%} | 21:42 |
| weshay|rover | true | 21:42 |
| weshay|rover | {%- else -%} | 21:42 |
| weshay|rover | false | 21:42 |
| weshay|rover | {%- endif -%} | 21:42 |
| weshay|rover | queens ^ is false | 21:42 |
| weshay|rover | oh crap | 21:42 |
| weshay|rover | I'm so wrong | 21:42 |
| rlandy | not really | 21:42 |
| weshay|rover | and tire | 21:42 |
| weshay|rover | d | 21:42 |
| * weshay|rover quits | 21:42 | |
| rlandy | weshay|rover: log off for shabbos early :) | 21:42 |
| weshay|rover | rlandy, ok.. that's the sign.. /me needs to walk way for a bit | 21:42 |
| weshay|rover | rlandy, early.. it's already 5.43 | 21:43 |
| weshay|rover | rlandy, have a good weekend.. shabat shalom :) | 21:43 |
| * weshay|rover out | 21:43 | |
| rlandy | I'm not logging off, saying you should | 21:43 |
| rlandy | shabbat shalom | 21:43 |
| *** dsneddon has quit IRC | 22:05 | |
| *** dsneddon has joined #oooq | 22:05 | |
| rlandy | we have logs now | 22:05 |
| rlandy | oh I see ... Required file "./ironic-python-agent.initramfs" does not exist. | 22:05 |
| *** chem has quit IRC | 22:16 | |
| *** apetrich has joined #oooq | 22:17 | |
| *** rlandy has quit IRC | 22:33 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!