| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000 (1 more message) | 00:17 |
|---|---|---|
| rlandy|rover | yeah - finally - good to have a reproducer env | 00:22 |
| *** rlandy|rover is now known as rlandy|rover|bbl | 00:23 | |
| *** rascasoft has joined #oooq | 00:24 | |
| *** rascasoft has quit IRC | 00:31 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000 (1 more message) | 02:17 |
| *** rlandy|rover|bbl is now known as rlandy|rover | 02:33 | |
| *** ykarel has joined #oooq | 02:57 | |
| *** ykarel has quit IRC | 03:04 | |
| *** ykarel has joined #oooq | 03:05 | |
| *** apetrich has quit IRC | 03:16 | |
| *** udesale has joined #oooq | 04:12 | |
| *** skramaja has joined #oooq | 04:13 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ (1 more message) | 04:17 |
| *** rfolco has quit IRC | 04:34 | |
| *** rfolco has joined #oooq | 04:34 | |
| *** rf0lc0 has joined #oooq | 04:37 | |
| *** rfolco has quit IRC | 04:39 | |
| *** rf0lc0 has quit IRC | 04:40 | |
| *** rfolco has joined #oooq | 04:41 | |
| *** ratailor has joined #oooq | 05:15 | |
| *** ykarel has quit IRC | 05:24 | |
| *** ykarel has joined #oooq | 05:51 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ (1 more message) | 06:17 |
| *** agopi__ has joined #oooq | 06:23 | |
| *** agopi__ has quit IRC | 06:33 | |
| *** agopi__ has joined #oooq | 06:34 | |
| *** agopi__ has quit IRC | 06:36 | |
| *** sshnaidm has quit IRC | 06:52 | |
| *** jfrancoa has joined #oooq | 07:06 | |
| *** quiquell|off is now known as quiquell | 07:09 | |
| marios | 09:29 < marios> needs another +2 please thanks https://review.openstack.org/#/c/628266/ | 07:29 |
| *** saneax has joined #oooq | 07:30 | |
| *** jbadiapa has joined #oooq | 07:43 | |
| *** quiquell is now known as quiquell|brb | 07:44 | |
| *** dtrainor has quit IRC | 07:45 | |
| *** dtrainor has joined #oooq | 07:46 | |
| *** gkadam has joined #oooq | 07:48 | |
| *** gkadam is now known as gkadam-afk | 07:50 | |
| *** ykarel is now known as ykarel|lunch | 07:51 | |
| *** saneax has quit IRC | 07:53 | |
| *** saneax has joined #oooq | 07:54 | |
| *** kopecmartin|off is now known as kopecmartin | 08:00 | |
| *** ccamacho has joined #oooq | 08:09 | |
| *** quiquell|brb is now known as quiquell | 08:11 | |
| *** ssbarnea has joined #oooq | 08:14 | |
| quiquell | marios: Scenario001 is all good now to get merged into projects ? | 08:16 |
| *** ssbarnea|bkp2 has quit IRC | 08:16 | |
| marios | quiquell: almost i think there is one left like panko/aodh or some | 08:17 |
| quiquell | marios: goignt to workflow puppet-tripleo | 08:17 |
| quiquell | marios: looks like Alex is ok with it | 08:17 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ (1 more message) | 08:17 |
| marios | quiquell: https://review.openstack.org/#/c/620304/ https://review.openstack.org/#/c/620301/ | 08:17 |
| marios | thanks quiquell | 08:17 |
| quiquell | marios: workflowed | 08:18 |
| *** bogdando has joined #oooq | 08:23 | |
| quiquell | marios: About periodics, maybe it's better to reuse the upstream jobs and overwrite the periodic part | 08:24 |
| quiquell | marios: wdyt ? | 08:24 |
| quiquell | marios: I see a lot of redundancy here https://review.rdoproject.org/r/#/c/18093 | 08:24 |
| *** gkadam-afk is now known as gkadam | 08:24 | |
| bogdando | rlandy|rover: hi. How was it going for https://bugs.launchpad.net/bugs/1810690 ? | 08:26 |
| openstack | Launchpad bug 1810690 in tripleo "periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master - which includes idempotency check - is timing out during introspection" [Critical,Triaged] | 08:26 |
| *** sshnaidm has joined #oooq | 08:31 | |
| quiquell | sshnaidm: o/ | 08:31 |
| sshnaidm | quiquell, hi | 08:31 |
| quiquell | sshnaidm: I saw some conversation with kforde about images, what was the conclusion ? | 08:31 |
| sshnaidm | quiquell, yeah, we talked about image sharing between projects, I'll send a needed flow | 08:32 |
| sshnaidm | quiquell, but it is still not exactly what we need | 08:33 |
| quiquell | sshnaidm: yep, I suppose this is not the final solution, it involves a lot of manual steps | 08:33 |
| quiquell | sshnaidm: just good enough for team | 08:33 |
| sshnaidm | quiquell, there are problem with images different names each time, their locking, etc ,etc | 08:33 |
| sshnaidm | quiquell, we can make a script that will download latest image, rename it, upload back and then will share it with the same name like "upstream-fedora-28" or kind of | 08:34 |
| sshnaidm | quiquell, but need to check if it works that way | 08:35 |
| quiquell | sshnaidm: We also need to check if we can ssh as zuul into shared images | 08:36 |
| *** tosky has joined #oooq | 08:37 | |
| sshnaidm | quiquell, when you boot from image you can set keys | 08:37 |
| quiquell | sshnaidm: Not always, only if it works with cloud-init | 08:37 |
| quiquell | sshnaidm: upstream images, does not work with "key-name" | 08:38 |
| quiquell | sshnaidm: didn't check on "RDO" upstream image | 08:38 |
| sshnaidm | quiquell, that's good point | 08:38 |
| quiquell | sshnaidm: that's what I want to test | 08:38 |
| quiquell | sshnaidm: If it does not work, sharing is not enough | 08:39 |
| sshnaidm | quiquell, but if we'll do as above, we could insert there keys I think | 08:39 |
| quiquell | sshnaidm: I think we can put in place the mechanism you said, even if sharing works, so we can modify whatever we need | 08:39 |
| quiquell | sshnaidm: we can create a periodic zuul job that do it | 08:40 |
| sshnaidm | yeah | 08:40 |
| quiquell | sshnaidm: https://paste.fedoraproject.org/paste/rahI-VnCqUYizKw3HzE7Zw | 08:41 |
| sshnaidm | quiquell, and seems like we can use it even if it's not accepted, so less things to do manually | 08:41 |
| marios | ssbarnea: o/ | 08:41 |
| ssbarnea | marios: o/ | 08:42 |
| quiquell | sshnaidm: Ahh that's cool | 08:42 |
| ssbarnea | can anyone recommend me a solution for SNMP monitoring? preferably one that also acts as a SMTP trap server? | 08:43 |
| ssbarnea | r/SMTP/SNMP/ | 08:44 |
| sshnaidm | quiquell, tried to do ci on centos, but it fails in post every time, not idea what's the problem: https://review.rdoproject.org/r/#/c/18087/ | 08:44 |
| marios | ssbarnea: planning on having a go at 2/3 in a bit as discussed yesterday but wanted to sync with you as we said we would? | 08:44 |
| marios | ssbarnea: is ther something you want/need for the qe stuff ? | 08:44 |
| marios | ssbarnea: otherwise we can just sync later | 08:44 |
| ssbarnea | marios: lets sync in ~30mins, just want to read email, do the morning rechecks :D | 08:45 |
| marios | ssbarnea: k | 08:45 |
| quiquell | sshnaidm: Humm, we are using the host as the noodepool provider, maybe it screw up the ssh keys so we cannot access at post | 08:46 |
| *** sshnaidm has quit IRC | 08:47 | |
| quiquell | sshnaidm: do you know if secrets can be shared between trusted projects ? | 08:47 |
| *** jpena|off is now known as jpena | 08:50 | |
| marios | reviews please thanks https://review.openstack.org/#/c/627965/ | 09:02 |
| *** holser_ has joined #oooq | 09:07 | |
| quiquell | marios: +2, they look good now, only scenario001 is missing stuff ? | 09:11 |
| *** dtantsur|afk is now known as dtantsur | 09:21 | |
| *** ykarel|lunch is now known as ykarel | 09:22 | |
| *** apetrich has joined #oooq | 09:26 | |
| marios | quiquell: thanks man yeah well 4 had some extra | 09:27 |
| marios | quiquell: see https://tree.taiga.io/project/tripleo-ci-board/task/541?kanban-status=1447276 and https://tree.taiga.io/project/tripleo-ci-board/task/448?kanban-status=1447276 | 09:27 |
| marios | quiquell: we didn't check 2/3 yet | 09:28 |
| quiquell | marios: ack | 09:28 |
| quiquell | marios: what is the proccess of theck this, just look at containers ? | 09:28 |
| marios | quiquell: well the process i used is in Test Req (1 compare containers 2 compare tempest) as i wrote there it is exactly what ykarel pointed out | 09:29 |
| marios | quiquell: and see attached .png (scroll down) | 09:29 |
| *** derekh has joined #oooq | 09:33 | |
| quiquell | marios: ack | 09:34 |
| quiquell | marios: We only have multinode-scenario002 for stable branchs not master :-( | 09:46 |
| marios | quiquell: right | 09:48 |
| marios | quiquell: "Since there is no more scenario001 multinode containers job on master, we have to use the latest which is ~ 14/15th December [1][2]." | 09:48 |
| marios | from https://tree.taiga.io/project/tripleo-ci-board/task/448?kanban-status=1447276 | 09:48 |
| marios | quiquell: so me folco and ssbarnea had call about this yesterday. i was going to help out folco on 2/3 today (after pushign 1/4 a bit as they are more ready/the promotions bit is the crucial part would be nice to finish ) | 09:49 |
| quiquell | marios: Ahh we have scenario002 job logs too | 09:49 |
| quiquell | marios: we have | 09:49 |
| quiquell | on master | 09:49 |
| marios | quiquell: sounds like you are looking there too? can we sync? | 09:49 |
| marios | quiquell: sync/rather i mean lets not duplicate | 09:49 |
| quiquell | marios: Ahh ok, didn't know | 09:50 |
| marios | so we can take one each? | 09:50 |
| marios | quiquell: yeah ssbarnea is qe on all of these | 09:50 |
| quiquell | marios: just trying to help, but if you are on it I will continue with other stuff | 09:50 |
| marios | quiquell: but i was going to push the comparison/any fixes for 23 | 09:50 |
| quiquell | marios: ack, | 09:50 |
| quiquell | marios: btw looks like there logs for multinode version for master barnch | 09:50 |
| marios | quiquell: well didn't start yet. so you can take one if you want. | 09:50 |
| marios | quiquell: so lets say you do 2 i do 3? cc ssbarnea | 09:51 |
| marios | quiquell: works for you? | 09:51 |
| marios | quiquell: or if you have other things i can do them both | 09:51 |
| marios | quiquell: just preventing duplicate work thats all | 09:52 |
| quiquell | marios: I will continue with the reproducer stuff, so we have good base for next sprint | 09:52 |
| marios | quiquell: ack sounds good ok thanks (sorry i only brought it up as you were asking questions/sounded like you were gearing up to push on 2/3) | 09:52 |
| quiquell | marios: Too much christmas disconnections, here in spain is bloody long arggg | 09:52 |
| chandankumar | marios: quiquell why we need to keep data files in a ansible role something like that https://github.com/openstack/ansible-role-tripleo-modify-image/blob/master/setup.cfg#L22 ? | 09:53 |
| marios | chandankumar: don't have context on that sorry (don't know) | 09:53 |
| *** sshnaidm has joined #oooq | 09:55 | |
| quiquell | chandankumar: They are part of the role, without them the role does not work | 09:58 |
| chandankumar | quiquell: marios ack | 10:04 |
| chandankumar | quiquell: do I need to put a ansible.cfg file also if I move the files under data_files? | 10:04 |
| marios | ssbarnea: so i just finished a round of harrasing folks (email and joined irc chans) so the topic branches should be more/less merging today (I mean like this https://review.openstack.org/#/q/topic:replace-scen4 1/2/3/4) | 10:06 |
| marios | ssbarnea: i was planning on attack 2/3 in a bit after i check the promotions stuff on 1/4. | 10:06 |
| quiquell | chandankumar: some context would help | 10:06 |
| marios | ssbarnea: are you OK/clear on the qe tasks? | 10:06 |
| chandankumar | quiquell: https://review.openstack.org/#/c/629127/ | 10:06 |
| marios | ssbarnea: otherwise let me know what/how i can help? if you are happy to do the qe and other stuff let me know if you want one of 2/3 | 10:06 |
| chandankumar | quiquell: I was checking this role https://github.com/openstack/ansible-role-tripleo-modify-image/blob/master/ansible.cfg | 10:07 |
| chandankumar | quiquell: so asked | 10:07 |
| quiquell | chandankumar: Not sure about ansible.cfg I suspect you don't have too | 10:08 |
| chandankumar | quiquell: ok, I am putting all the pieces together and let's see how it goes | 10:08 |
| chandankumar | quiquell: one more question I have this playbook https://review.openstack.org/#/c/628415/4/playbooks/os-tempest.yml once standalone finishes I want to run it on the same job for that what to do? | 10:10 |
| quiquell | chandankumar: well, right now we run tempest as part of the deploy | 10:12 |
| quiquell | chandankumar: instead of a playbook I suppose youi have to replace the current validate-tempest role call with os_tempest | 10:13 |
| *** holser_ has quit IRC | 10:14 | |
| *** holser_ has joined #oooq | 10:15 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-containerized-undercloud-upgrades @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ (1 more message) | 10:17 |
| marios | ssbarnea: ? | 10:25 |
| ssbarnea | marios: done *3, only two more wf to receive on them. | 10:26 |
| marios | ssbarnea: ack thanks i was just mid rebase but you posted v3 | 10:26 |
| marios | (tripleo-common) | 10:26 |
| ssbarnea | :D | 10:26 |
| ssbarnea | marios: nobody asked the reasons behind including the node config in scenario name? if we would not had multinode suffix we could have repurposed it without so many changes. | 10:28 |
| marios | ssbarnea: well standalone is an important part of the name no? | 10:32 |
| marios | ssbarnea: i mean we clearly want to distinguish from multinode | 10:33 |
| ssbarnea | marios: that aspect can be included in scenario definitions, anyway too late now but i raised the question for the future. | 10:33 |
| ssbarnea | marios: you know how the downstream jobs are named,.... right? :D | 10:34 |
| *** kopecmartin is now known as kopecmartin|afk | 10:35 | |
| marios | ssbarnea: where do you mean by scenario definitions | 10:36 |
| marios | ssbarnea: i would still prefer to keep in the name | 10:36 |
| marios | ssbarnea: because standalone is a distinct animal | 10:37 |
| marios | ssbarnea: ok just harrassed openstack/telemetry too | 10:37 |
| marios | lets see | 10:37 |
| marios | forgot barbican going | 10:39 |
| ssbarnea | gate queue should go up, but for a good reason at least. | 10:42 |
| chandankumar | quiquell: something like this https://review.openstack.org/#/c/628415/ | 10:48 |
| marios | quiquell: workflowed this https://review.openstack.org/#/c/627965/ fyi (fix scen1) | 10:50 |
| quiquell | marios: cool, I suppose we can workflow those projects if Alex is ok | 10:51 |
| marios | quiquell: yeah he is +2 am pushing as much as possible on these today/tomorrow end of sprint whatever we don't finish we carry | 10:52 |
| quiquell | chandankumar: remove playbook variables from the task call | 10:52 |
| quiquell | chandankumar: hosts, gather_facts, etc... | 10:52 |
| marios | quiquell: but why don't you want to workflow ? what do you mean by if alex is ok to workflow those projects? | 10:53 |
| marios | oh "ci core" | 10:53 |
| marios | ? | 10:53 |
| marios | k | 10:53 |
| marios | np | 10:53 |
| quiquell | chandankumar: also tasks, this is a very bad copy paste dude! | 10:53 |
| marios | forgot | 10:53 |
| chandankumar | quiquell: ok formatting it | 10:53 |
| marios | quiquell: still would have been fine if you +A imo anyway | 10:53 |
| marios | quiquell: if anyone asks me i'll just say "what i have no idea what came over him he just decided to +a it ) | 10:54 |
| quiquell | marios: I am very new core at tripleo, for non ci repos, I am don't want to break stuff | 10:54 |
| ssbarnea | does anyone have a script for removing all local gerrit branches that are merged? | 10:54 |
| quiquell | marios: ack, just some caution from my part, but it's a CI change so yep, I cam workflow it | 10:55 |
| marios | quiquell: anyway done now next one | 10:55 |
| quiquell | marios: Can we workflow anything else ? | 10:57 |
| *** udesale has quit IRC | 10:57 | |
| quiquell | marios: or we depend on non tripleo projects ? | 10:57 |
| marios | anyone know if there is some way to see this running/reporting without merging it? https://review.rdoproject.org/r/#/c/18079/ which is wired up in https://review.rdoproject.org/r/18091 | 10:58 |
| marios | quiquell: maybe scen2/3 will have fixes but otherwise the topic branches should be good now for the replacement for 1-4 been harrssing people by email/irc/fax since this morning | 10:59 |
| marios | so maybe we just need to merge these ? https://review.rdoproject.org/r/#/c/18091/ | 11:01 |
| quiquell | marios: We have a way to run periodic jobs at check | 11:01 |
| quiquell | marios: but report is going to be executed but fail since the DLRN secret is not there | 11:02 |
| quiquell | marios: That would be enough ? | 11:02 |
| marios | quiquell: well i guess it would help us to sanity check it isn't completely broken for some reason there | 11:02 |
| marios | quiquell: like deploy pass etc | 11:02 |
| quiquell | marios: This is what we did for the portin of periodic jobs to zuulv3 | 11:03 |
| quiquell | marios: https://review.rdoproject.org/r/#/c/16806/ | 11:03 |
| marios | quiquell: cool thanks | 11:03 |
| quiquell | marios: Importnat part is the Depends-On | 11:03 |
| *** ccamacho has quit IRC | 11:03 | |
| quiquell | marios: Let me know if it works | 11:04 |
| quiquell | marios: You can even try the new reproducer too | 11:04 |
| marios | quiquell: ack | 11:15 |
| quiquell | marios: Going to send email con f28 standalone job to get it voting | 11:24 |
| ssbarnea | marios: 10 more replace- changes to merge. | 11:26 |
| marios | quiquell: i am going to restore & use this (well about to see how much pain rebase is otherwise will post new and abandon again ) fyi https://review.openstack.org/#/c/608166/ | 11:27 |
| marios | ssbarnea: you mean there are new ones? | 11:27 |
| marios | ssbarnea: 13:26 < ssbarnea> marios: 10 more replace- changes to merge. | 11:27 |
| ssbarnea | marios: nope, i counted the not merged yet ones. | 11:27 |
| marios | ssbarnea: ack | 11:28 |
| quiquell | marios: ack, do whatever you need with it | 11:28 |
| marios | ssbarnea: they all should be on their way anyway i joined the openstack/chans too after that email | 11:28 |
| marios | (those emails) | 11:28 |
| *** ccamacho has joined #oooq | 11:33 | |
| *** skramaja has quit IRC | 11:40 | |
| *** holser_ is now known as holser|lunch | 11:53 | |
| *** holser|lunch has quit IRC | 11:55 | |
| panda | rlandy|rover: can you +2 https://review.rdoproject.org/api/18122 again ? fixed couple of things, should be more ok now | 11:57 |
| *** ratailor has quit IRC | 11:57 | |
| marios | quiquell: do i need th eontainers build dependency @ https://review.rdoproject.org/r/#/c/18079/4/zuul.d/projects.yaml (gives me error) also do i need to add ALL the jobs here or can i just have the one i want | 11:59 |
| marios | quiquell: (comparing with https://review.rdoproject.org/r/#/c/16806/4/zuul.d/projects.yaml ) | 11:59 |
| *** d0ugal has quit IRC | 12:01 | |
| quiquell | marios: Only the one you want, the former review have all them because we where testing all of them there | 12:01 |
| marios | quiquell: ok thans going afk for a bit will try again when back | 12:03 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224, (1 more message) | 12:17 |
| *** ccamacho has quit IRC | 12:20 | |
| *** d0ugal has joined #oooq | 12:22 | |
| ssbarnea | rfolco: marios : gate failed due to one tempest test failure (basic networking) https://review.openstack.org/#/c/628277/ | 12:24 |
| ssbarnea | i think the problem is persistent based on http://logstash.openstack.org/#/dashboard/file/logstash.json?query=message:%5C%22within%20the%20required%20time%20(500%20s).%20Current%20status:%20BUILD.%20Current%20task%20state:%20scheduling%5C%22 | 12:27 |
| ssbarnea | but new | 12:27 |
| ssbarnea | happened 8 times, but only today | 12:27 |
| ssbarnea | and worse, only with tripleo-ci-centos-7-scenario003-standalone | 12:27 |
| ssbarnea | which makes me believe this is a bug | 12:28 |
| *** gkadam has quit IRC | 12:29 | |
| *** jpena is now known as jpena|lunch | 12:37 | |
| chandankumar | quiquell: I need some help in this playbook https://review.openstack.org/#/c/628415/ | 12:43 |
| chandankumar | quiquell: http://logs.openstack.org/00/627500/26/check/tripleo-ci-centos-7-standalone-os-tempest/09d589b/job-output.txt.gz#_2019-01-08_12_06_34_643760 | 12:44 |
| *** bogdando_ has joined #oooq | 12:48 | |
| *** holser_ has joined #oooq | 12:49 | |
| *** bogdando has quit IRC | 12:51 | |
| *** bogdando_ is now known as bogdando | 12:51 | |
| *** ccamacho has joined #oooq | 12:53 | |
| *** ccamacho has quit IRC | 12:54 | |
| *** ccamacho has joined #oooq | 12:54 | |
| *** holser__ has joined #oooq | 12:54 | |
| *** holser_ has quit IRC | 12:54 | |
| ssbarnea | marios: rfolco that means that no scenario 3 change will merge until we fix this. | 13:06 |
| rfolco | ssbarnea, could be related to missing services ? marios ? | 13:07 |
| marios | rfolco: o/ | 13:10 |
| marios | rfolco: just back from afk ssbarnea what happened? | 13:10 |
| marios | ah tempest? | 13:10 |
| marios | rfolco: was selling all the things this morning and looking at the promotion jobs | 13:11 |
| marios | rfolco: planning on doing the 2/3 sanity checks this afternoon | 13:11 |
| marios | rfolco: K? | 13:11 |
| rfolco | marios, hands tied here with f28 containers | 13:11 |
| rfolco | marios, sounds good | 13:12 |
| rfolco | ssbarnea, ^ | 13:12 |
| marios | rfolco: ssbarnea http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-scenario003-standalone | 13:13 |
| marios | rfolco: ack | 13:13 |
| *** trown|outtypewww is now known as trown | 13:17 | |
| *** rascasoft has joined #oooq | 13:19 | |
| *** udesale has joined #oooq | 13:20 | |
| rfolco | panda, can you tmate ral quick to unblock me ? | 13:20 |
| rlandy|rover | panda: https://review.rdoproject.org/r/#/c/18122 minor comment - of you want to fix it - if not, I'll w+ | 13:21 |
| *** zul has joined #oooq | 13:21 | |
| rlandy|rover | bogdando - hi, wrt https://bugs.launchpad.net/bugs/1810690 - I have a reproducer env - left a note in the bug | 13:22 |
| openstack | Launchpad bug 1810690 in tripleo "periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master - which includes idempotency check - is timing out during introspection" [Critical,Triaged] | 13:22 |
| rlandy|rover | looks like some containers are 'exited' | 13:23 |
| *** agopi has joined #oooq | 13:24 | |
| panda | rlandy|rover: I'd fix, but I'd hate to lose the other +2, are you going to +w after or you want to wait again the other +2 | 13:25 |
| panda | ? | 13:25 |
| rlandy|rover | panda: np - will w+ - not thta big a deal | 13:25 |
| panda | rfolco: sure, what's the command ? | 13:26 |
| panda | rfolco: unblock() | 13:26 |
| rfolco | thank you | 13:26 |
| rlandy|rover | panda: w+'ed | 13:26 |
| rfolco | that really helped | 13:26 |
| panda | rfolco: kill -9 pidof block | 13:26 |
| rfolco | :) | 13:26 |
| marios | ssbarnea: can you/did you file a bug for that tempest issue yet? | 13:26 |
| panda | rlandy|rover: thanks | 13:26 |
| weshay|ruck | rlandy|rover, \0/ https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/b9871f8/logs/undercloud/home/zuul/tempest.log.txt.gz#_2019-01-08_10_18_02 | 13:26 |
| ssbarnea | marios: not yet, i was hoping ruck/rover to have a look at it first. | 13:27 |
| rlandy|rover | panda: here's hoping we didn't mess up | 13:27 |
| weshay|ruck | if we need to we can bug that and put in the skip list | 13:27 |
| weshay|ruck | that is pretty good | 13:27 |
| chandankumar | panda: sshnaidm is this change correct https://review.openstack.org/#/c/629127/ for installing the role as TQE extras in TQ? | 13:27 |
| panda | rlandy|rover: we didn't mess up *again* | 13:27 |
| marios | ssbarnea: d0ugal just replied on the email and it looks like it was indeed the tempest there (email re mistral patches http://logs.openstack.org/74/628274/3/check/tripleo-ci-centos-7-scenario003-standalone/9cb0cf0/logs/undercloud/home/zuul/tempest.log.txt.gz | 13:27 |
| rlandy|rover | weshay|ruck: well we are further along | 13:27 |
| rlandy|rover | weshay|ruck: I know what that problem is ... | 13:27 |
| rlandy|rover | I added back overcloud_ssl | 13:28 |
| rlandy|rover | we can disable it | 13:28 |
| rlandy|rover | see the fs change | 13:28 |
| weshay|ruck | rlandy|rover, fs030 passed too | 13:28 |
| weshay|ruck | but I'm sure no work has been done | 13:28 |
| d0ugal | marios: right - so I should recheck? | 13:29 |
| rlandy|rover | weshay|ruck: also - while we are changing the fs - disabled idempotency check for all releases, we could add rocky back - up to you | 13:29 |
| rlandy|rover | weshay|ruck: yeah - don't think that bug moved | 13:29 |
| weshay|ruck | rlandy|rover, it's ok .. we have an open bug.. we're ok | 13:29 |
| marios | d0ugal: not yet we didn't get to a fix | 13:29 |
| marios | d0ugal: thanks man | 13:29 |
| marios | d0ugal: but whatever it is its new if you see on same review earlier it was green | 13:29 |
| rlandy|rover | weshay|ruck: putting in a patch to remove overcloud_ssl | 13:30 |
| d0ugal | ah okay, I'll wait a bit :) | 13:30 |
| rlandy|rover | we never had it before | 13:30 |
| weshay|ruck | sshnaidm, you ready | 13:30 |
| sshnaidm | weshay|ruck, yep, on your bluej | 13:30 |
| quiquell | rlandy|rover: o/ do you have the link with the assigned hardware we have ? | 13:32 |
| rlandy|rover | quiquell, https://docs.google.com/spreadsheets/d/17WOeF-wiNZfB8SZrWBhu4SjWkq78XzeEG1wASBQaJTM/edit#gid=0 | 13:33 |
| quiquell | rlandy|rover: thanks! | 13:34 |
| *** jpena|lunch is now known as jpena | 13:34 | |
| *** ykarel is now known as ykarel|away | 13:35 | |
| marios | quiquell: do you remember why you had the dependency on container image build in https://review.rdoproject.org/r/#/c/16806/4/zuul.d/projects.yaml | 13:45 |
| rfolco | weshay|ruck, argument --base: Invalid String(choices=['centos', 'rhel', 'ubuntu', 'oraclelinux', 'debian']) value: fedora", | 13:46 |
| quiquell | marios: to test exactly how we want it | 13:46 |
| rfolco | weshay|ruck, will try to hack this but not sure we'll get any far | 13:46 |
| marios | quiquell: i removed it in https://review.rdoproject.org/r/#/c/18079/6/zuul.d/projects.yaml cos otherwise was complaining. waiting to see if it runs now https://review.rdoproject.org/zuul/status (queued) | 13:46 |
| quiquell | marios: we need the dependant job to finish | 13:46 |
| marios | quiquell: so we need it here | 13:46 |
| quiquell | marios: if you add the dependency you need to also put it in the list of jobs | 13:46 |
| marios | quiquell: Unable to freeze job graph: Job periodic-tripleo-ci-centos-7-scenario001-standalone depends on periodic-tripleo-centos-7-master-containers-build which was not run. | 13:47 |
| marios | quiquell: i get this when i added the dependency | 13:47 |
| *** agopi has quit IRC | 13:48 | |
| marios | quiquell: thanks i see it now | 13:49 |
| quiquell | marios: the freezie thing it's usualÃly related to secret, it means it cannot decrypt the secrets | 13:49 |
| marios | quiquell: you have to define it | 13:49 |
| marios | quiquell: thanks | 13:49 |
| marios | updating | 13:49 |
| quiquell | marios: maybe you just want to test without the dependency to run the job, if a container is there it will run | 13:50 |
| marios | quiquell: ok i'll wait for the job run then first | 13:50 |
| quiquell | marios: It's not going to break anything since you cannot push containers from those jobs at check | 13:50 |
| quiquell | marios: ack | 13:50 |
| quiquell | rlandy|rover: do you have a free brain cycle to question on nodepool-setup role ? | 13:51 |
| rlandy|rover | quiquell: ack - what's up? | 13:52 |
| quiquell | rlandy|rover: It has a part that it changes zuul_inventory.yaml | 13:52 |
| quiquell | rlandy|rover: but since I am not reproducing already run jobs I am just running jobs I don't have it | 13:53 |
| quiquell | rlandy|rover: make sense to continue execution if it does not exists ? | 13:53 |
| quiquell | rlandy|rover: talking about https://review.openstack.org/#/c/617855/ | 13:55 |
| rlandy|rover | quiquell: I used the old nodepool-setup role | 13:55 |
| rlandy|rover | and removed some stuff | 13:55 |
| rlandy|rover | not the new one | 13:55 |
| rlandy|rover | in fact I should abandon all those reviews | 13:55 |
| quiquell | rlandy|rover: I don't have to use https://review.openstack.org/#/c/617855/ to run zuul repro with libvirt ? | 13:56 |
| rlandy|rover | quiquell: no | 13:56 |
| rlandy|rover | quiquell: because all you need it for is to patch the images | 13:57 |
| rlandy|rover | that is all | 13:57 |
| rlandy|rover | I did make a change to the role | 13:57 |
| quiquell | rlandy|rover: ack, so just the libvirt roles are enough ? | 13:57 |
| rlandy|rover | yeah | 13:57 |
| quiquell | rlandy|rover: cool cool :-), was just following the doc | 13:57 |
| rlandy|rover | because the rest you don;t need - zuul provides it | 13:57 |
| rlandy|rover | quiquell: what did I say in the doc??? I should read that back | 13:58 |
| quiquell | rlandy|rover: no worries and ansibilizing it :-) | 13:58 |
| ykarel|away | quiquell, why containers will not be pushed when running in check? asking against <quiquell> marios: It's not going to break anything since you cannot push containers from those jobs at chec | 13:58 |
| quiquell | rlandy|rover: and this other one is needed ? https://review.openstack.org/#/c/623294/ | 13:58 |
| ykarel|away | quiquell, afaik there is no such check for pushing, can u share where this is checked | 13:58 |
| rlandy|rover | quiquell: yes - | 13:59 |
| quiquell | ykarel|away: yep that what I mean, no problem running them at check | 13:59 |
| rlandy|rover | you only get one ip | 13:59 |
| quiquell | rlandy|rover: ack thanks ! | 13:59 |
| ykarel|away | quiquell, okk so containers will be pushed, ur message confused me | 14:00 |
| ykarel|away | it's good to not run consistent-to tripleo-ci testing in check | 14:00 |
| ykarel|away | as that willl update the symlink and may have some impact on jobs running in periodic | 14:01 |
| rlandy|rover | quiquell: I think I made some minor modifications to the old nodepool-setup role bceause the steps were not needed - ping if you hit another issue | 14:01 |
| quiquell | rlandy|rover: but we don't need nodepool-setup for zuul repro with libvirt, isn't that correct ? | 14:02 |
| *** d0ugal has quit IRC | 14:02 | |
| rlandy|rover | quiquell: it's a matter of the images used | 14:02 |
| rlandy|rover | quiquell: for libvirt, you use stack OS images | 14:02 |
| rlandy|rover | so they miss stuff the nodepool images have | 14:02 |
| rlandy|rover | that is what that role is meant to do | 14:02 |
| rlandy|rover | the old one at least | 14:03 |
| ykarel|away | quiquell, is there some doc to use zuul reproducer, i want to try that | 14:03 |
| rlandy|rover | use that one as is and see how you go | 14:03 |
| ykarel|away | rlandy|rover, ^^ | 14:03 |
| quiquell | ykarel|away: Not yet | 14:03 |
| *** zul has quit IRC | 14:03 | |
| quiquell | rlandy|rover: so we need to call the nodepool-setup thing from the libvirt zuul repro to prepare | 14:04 |
| ykarel|away | quiquell, ack will wait :), the old reproducer is broken for periodic jobs | 14:04 |
| *** d0ugal has joined #oooq | 14:04 | |
| quiquell | rlandy|rover: but not from this review 200~https://review.openstack.org/#/c/617855/ | 14:04 |
| rlandy|rover | after you provision nodes | 14:04 |
| rlandy|rover | no | 14:04 |
| quiquell | rlandy|rover: just the want at tqe master | 14:05 |
| *** kopecmartin|afk is now known as kopecmartin | 14:06 | |
| quiquell | ykarel|away: I think it was not working pushing containers from hacked jobs at check... | 14:08 |
| ykarel|away | quiquell, what's not working | 14:09 |
| ykarel|away | any link? | 14:09 |
| quiquell | ykarel|away: marios was just trying to exercise periodic jobs at a review hacking ci | 14:10 |
| ykarel|away | link of patch | 14:10 |
| *** zul has joined #oooq | 14:11 | |
| ykarel|away | hmm found https://review.rdoproject.org/r/#/c/18093/6/zuul.d/projects.yaml | 14:11 |
| marios | ykarel|away: here https://review.rdoproject.org/r/#/c/18093/ and https://review.rdoproject.org/r/#/c/18079/ | 14:12 |
| ykarel|away | periodic-tripleo-centos-7-master-containers-build not mentioned | 14:12 |
| ykarel|away | marios, u can just run the standalone job | 14:12 |
| marios | quiquell: on scen 4 i tried with the dependencies ^ | 14:12 |
| marios | ykarel|away: on the second link that is what i did | 14:12 |
| ykarel|away | looking, but u need a depends on | 14:12 |
| *** d0ugal has quit IRC | 14:12 | |
| marios | ykarel|away: it has depends on | 14:12 |
| marios | ykarel|away: but for https://review.rdoproject.org/r/#/c/18079/6/zuul.d/projects.yaml the job is still in queued in zuul status https://review.rdoproject.org/zuul/status | 14:14 |
| marios | so not sure | 14:14 |
| rlandy|rover | marios: I have periodic jobs running with testproject if that helps you | 14:14 |
| ykarel|away | marios, yes that will run when nodes will be available | 14:14 |
| rlandy|rover | https://review.rdoproject.org/r/#/c/18058/ | 14:14 |
| ykarel|away | periodic-tripleo-ci-centos-7-scenario001-standalone is queud | 14:14 |
| marios | ykarel|away: thanks | 14:16 |
| marios | very much | 14:16 |
| * ykarel|away out | 14:16 | |
| ykarel|away | np :) | 14:16 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224, (1 more message) | 14:17 |
| *** ykarel|away has quit IRC | 14:21 | |
| ssbarnea | rfolco: http://zuul.openstack.org/builds?job_name=barbican-dogtag-devstack-functional-fedora-latest job seem broken for 3+ weeks, and is voting, so we cannot merge our change to barbican. | 14:22 |
| sshnaidm | ssbarnea, any news from Ben Nemec about moving repo? | 14:22 |
| ssbarnea | sshnaidm: yes, he replied on taiga. wait till Thursday when we will make the move. | 14:25 |
| sshnaidm | ok | 14:26 |
| ssbarnea | sshnaidm: mainly he wanted to give people one week to change branch (one week since he blogged + emailed about it) | 14:26 |
| sshnaidm | I see | 14:26 |
| weshay|ruck | thanks arxcruz | 14:28 |
| weshay|ruck | quiquell, so do I need access to the infra to debug http://dashboard-ci.tripleo.org/d/cEEjGFFmz/cockpit?orgId=1&panelId=154&fullscreen | 14:32 |
| weshay|ruck | quiquell, also wondering if you saw https://review.rdoproject.org/r/#/c/18103/ | 14:33 |
| quiquell | weshay|ruck: haven't check RDO reviews yet | 14:34 |
| quiquell | weshay|ruck: the alert bugs is weird, with your review merged is working fine locally | 14:34 |
| quiquell | weshay|ruck: maybe we have to dig in infra too see what's happening | 14:34 |
| weshay|ruck | ya.. I see alert bugs locally, but not promotion | 14:35 |
| quiquell | weshay|ruck: is very unsual to need to do that | 14:35 |
| quiquell | weshay|ruck: not promotion ? what do you mean ? | 14:35 |
| weshay|ruck | promotion-blocker | 14:35 |
| weshay|ruck | the other tag | 14:35 |
| quiquell | weshay|ruck: to to check your review about layout give me a sec | 14:35 |
| quiquell | weshay|ruck: maybe you can put in the review a snapshot from dashboard | 14:36 |
| quiquell | weshay|ruck: grafana has a tool to paste snapshots (they are just links) | 14:36 |
| weshay|ruck | not sure what you mean | 14:37 |
| weshay|ruck | you are not referring to the json export? | 14:37 |
| quiquell | weshay|ruck: in this review https://review.rdoproject.org/r/#/c/18103/3 | 14:37 |
| weshay|ruck | aye | 14:37 |
| quiquell | weshay|ruck: Since it's related to layout, it's good to share a screenshot in the git comment | 14:37 |
| quiquell | weshay|ruck: to do so you have a "share" option at grasfana that generate a link | 14:38 |
| weshay|ruck | oh.. fair point | 14:38 |
| quiquell | weshay|ruck: so people can review without run it | 14:38 |
| weshay|ruck | so I'll describe it | 14:38 |
| quiquell | just generate the share link paste it in the review | 14:38 |
| weshay|ruck | I removed the box "upstream check jobs (noop) | 14:39 |
| weshay|ruck | I removed the box ( RDO periodic jobs ) | 14:39 |
| weshay|ruck | so the top rows have 3 boxes | 14:39 |
| rfolco | reminder >> CI community meeting (office hours) starts now at https://bluejeans.com/4113567798 | 14:39 |
| weshay|ruck | and moved alert and promotion-blocker bugs up to the top two rows | 14:39 |
| weshay|ruck | quiquell, I can resetup and screenshot if you like, | 14:40 |
| rfolco | weshay|ruck, quiquell ssbarnea sshnaidm panda chandankumar arxcruz marios kopecmartin | 14:40 |
| rfolco | reminder >> CI community meeting (office hours) starts now at https://bluejeans.com/4113567798 | 14:40 |
| quiquell | weshay|ruck: running the review to see it | 14:41 |
| *** zul has quit IRC | 14:44 | |
| marios | rfolco: joined sorry forgot | 14:44 |
| marios | :) | 14:44 |
| weshay|ruck | https://tree.taiga.io/project/tripleo-ci-board/epic/327 | 14:44 |
| quiquell | weshay|ruck: added snapshot https://review.rdoproject.org/r/#/c/18103/ | 14:46 |
| quiquell | looks good to me | 14:46 |
| *** zul has joined #oooq | 14:47 | |
| bogdando | rlandy|rover: https://bugs.launchpad.net/tripleo/+bug/1810690/comments/12 | 14:50 |
| openstack | Launchpad bug 1810690 in tripleo "periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master - which includes idempotency check - is timing out during introspection" [Critical,Triaged] | 14:50 |
| bogdando | we should add that paunch logs to CI artifacts | 14:50 |
| bogdando | oh wait, not really, it is duplicated in install-undercloud.log | 14:51 |
| rlandy|rover | bogdando: adding logs is simple enough if we need them | 14:53 |
| bogdando | rlandy|rover: yeah, that's just a minor thing | 14:54 |
| bogdando | the issues are as I listed it | 14:54 |
| rlandy|rover | bogdando: reading - thanks | 14:55 |
| weshay|ruck | http://logs.openstack.org/88/615988/6/check/tripleo-ci-split-controlplane-standalone/f29bf53/logs/quickstart_files/ssh.config.ansible.txt.gz | 14:55 |
| *** ykarel|away has joined #oooq | 15:04 | |
| *** ykarel|away is now known as ykarel | 15:09 | |
| *** ykarel is now known as ykarel|away | 15:21 | |
| *** saneax has quit IRC | 15:22 | |
| ssbarnea | sshnaidm: can you please tell me if I missed to address any of your comments from Jan 3rd on https://review.openstack.org/#/c/613672/ ? | 15:31 |
| sshnaidm | ssbarnea, will look tomorrow | 15:31 |
| ssbarnea | sshnaidm: thanks. | 15:32 |
| ssbarnea | sshnaidm: btw, i fixed molecule master to work with rawhide, merged today. once is officially included in a release, I will also enable testing with rawhide. | 15:33 |
| sshnaidm | ssbarnea, is it released? | 15:34 |
| ssbarnea | sshnaidm: merged to master, not released yet. | 15:34 |
| sshnaidm | ssbarnea, ok, will try it | 15:34 |
| *** chem has quit IRC | 15:37 | |
| weshay|ruck | rlandy|rover, well.. so I guess pushing those through will be our top priority | 15:41 |
| sshnaidm | ssbarnea, btw, tried to provision vms in molecule, works pretty nice.. maybe we can test our libvirt stuff with it | 15:41 |
| marios | rfolco: you didn't propose seomthing to make 2 /3 voting yet right? | 15:41 |
| marios | rfolco: (just checking but looks like not on master anyway ) | 15:41 |
| rfolco | marios, I did :( | 15:42 |
| marios | rfolco: oh yeah | 15:42 |
| marios | rfolco: no worries | 15:42 |
| marios | thanks | 15:42 |
| rfolco | marios, you're fixing the mess for all? | 15:42 |
| marios | rfolco: yes | 15:42 |
| rfolco | afazekas, thx | 15:42 |
| rfolco | oops | 15:42 |
| marios | rfolco: easier do it in one set of reviews | 15:42 |
| rfolco | marios, ^ | 15:42 |
| rfolco | sorry afazekas ignore | 15:43 |
| rfolco | marios, yup | 15:43 |
| quiquell | marios: ready to +2+w | 15:43 |
| ssbarnea | sshnaidm: we could test a *lot* with it. But I propose to adopt its use in small steps. | 15:43 |
| weshay|ruck | sshnaidm, rlandy|rover thoughts on a new layout for cockpit https://snapshot.raintank.io/dashboard/snapshot/d3chF5rI89uSCTBuj0urA3UFh8KKb7Y0?orgId=2 | 15:43 |
| sshnaidm | ssbarnea, a lot, except docker, hehe | 15:43 |
| weshay|ruck | https://review.rdoproject.org/r/#/c/18103/ | 15:44 |
| chandankumar | ssbarnea: I have one query for ansible we can use molecule for testing stuff but there are other config stuffs can we use test-infra for that? | 15:44 |
| sshnaidm | weshay|ruck, mm.. I think we lost two important singles like upstream noop jobs and periodic ones | 15:45 |
| * rlandy|rover looks | 15:46 | |
| sshnaidm | weshay|ruck, I definitely would like to see upstream noop jobs failure there, because it can show you broken jobs | 15:46 |
| weshay|ruck | sshnaidm, ya.. for master only? | 15:46 |
| weshay|ruck | on the first or second row | 15:47 |
| weshay|ruck | then the others below it | 15:47 |
| sshnaidm | weshay|ruck, yeah | 15:47 |
| rlandy|rover | weshay|ruck: I would agree with sshnaidm - we're missing periodic | 15:47 |
| weshay|ruck | so master no-op is just below Upstream failing gate jobs FYI.. | 15:47 |
| weshay|ruck | but you can't tell from the screenshot | 15:47 |
| rlandy|rover | which always had the most failures | 15:47 |
| weshay|ruck | periodic and no-op are two diff things rlandy|rover | 15:48 |
| sshnaidm | weshay|ruck, it's in the table, but better to have "single" too | 15:48 |
| weshay|ruck | rlandy|rover, guys.. that screenshot is just the first 2.5 rows | 15:48 |
| weshay|ruck | sshnaidm, rlandy|rover scroll down on https://snapshot.raintank.io/dashboard/snapshot/d3chF5rI89uSCTBuj0urA3UFh8KKb7Y0?orgId=2 | 15:49 |
| weshay|ruck | master no-op and master periodic are right there below upstream gate jobs | 15:49 |
| rlandy|rover | I see | 15:49 |
| weshay|ruck | that's pretty high up there imho | 15:49 |
| sshnaidm | weshay|ruck, do you talk about tables? | 15:50 |
| weshay|ruck | sorry, not 100% sure what you mean sshnaidm | 15:50 |
| sshnaidm | weshay|ruck, in grafana you have tables - with rows, also "singles" - squares with one data type (as zuul time, upstream failures count) | 15:51 |
| weshay|ruck | sshnaidm, k.. yes | 15:51 |
| weshay|ruck | understood | 15:51 |
| sshnaidm | weshay|ruck, so I see upstream no-op jobs only in the table, but I don't see a single like we have now | 15:52 |
| weshay|ruck | oh.. you would like to raise the visibility of the master no-op jobs via a square | 15:53 |
| weshay|ruck | ok.. I think that is a diff patch, but ya I can see that .. not sure how you would capture that exactly in a square | 15:53 |
| weshay|ruck | noting we also have the alert in this channel that everyone ignores :) | 15:53 |
| weshay|ruck | lolz | 15:53 |
| sshnaidm | weshay|ruck, yeah, I'm trying to keep the most important info in the top and short | 15:54 |
| weshay|ruck | sshnaidm, ya.. I can try and do that and run it by you.. I need more experience here | 15:54 |
| weshay|ruck | for now I'd like to merge the current change | 15:54 |
| sshnaidm | weshay|ruck, for alerts we'll have something different soon.. | 15:54 |
| weshay|ruck | ok.. I need the bug list to start working | 15:55 |
| sshnaidm | weshay|ruck, not sure I understand your change.. maybe we'll discuss it with quiquell later too | 15:55 |
| quiquell | sshnaidm: ack | 15:55 |
| weshay|ruck | it's just getting rid of the squares at the top that are meaningless | 15:55 |
| weshay|ruck | like rdo periodic jobs | 15:56 |
| weshay|ruck | and upstream check jobs fails | 15:56 |
| weshay|ruck | and replacing those two squares w/ alert and promotion-blocker squares | 15:56 |
| sshnaidm | weshay|ruck, why are they meaningless?? | 15:56 |
| weshay|ruck | sshnaidm, is there any action we take on failing upstream check jobs? | 15:56 |
| sshnaidm | weshay|ruck, it's not "check" jobs | 15:57 |
| weshay|ruck | or the total count of failing periodic / promotion jobs | 15:57 |
| sshnaidm | weshay|ruck, it's no-op jobs that should always pass | 15:57 |
| weshay|ruck | ah.. no-op | 15:57 |
| weshay|ruck | hrm.. | 15:57 |
| weshay|ruck | I see ur point | 15:57 |
| weshay|ruck | ok... | 15:58 |
| sshnaidm | weshay|ruck, total count of periodic maybe is not so important, yeah, as we have them fail a lot :D | 15:58 |
| weshay|ruck | agree | 15:58 |
| weshay|ruck | but needs some adjustment | 15:58 |
| weshay|ruck | sshnaidm, I'll put it back | 15:58 |
| weshay|ruck | should be red though, with one failure | 15:58 |
| weshay|ruck | right? | 15:58 |
| sshnaidm | weshay|ruck, it's red, but color of number itself doesn't work for me, no idea why :( | 15:58 |
| weshay|ruck | lolz k | 15:59 |
| weshay|ruck | sshnaidm, I'll update the review | 15:59 |
| weshay|ruck | and yet another mtg | 16:00 |
| *** chem has joined #oooq | 16:00 | |
| weshay|ruck | :) | 16:00 |
| sshnaidm | oh yeah | 16:00 |
| sshnaidm | quiquell, moderator, please join :) | 16:00 |
| quiquell | sshnaidm, weshay|ruck sorry about that | 16:00 |
| *** udesale has quit IRC | 16:02 | |
| rlandy|rover | panda: can we close this out? https://bugs.launchpad.net/tripleo/+bug/1810879 | 16:03 |
| openstack | Launchpad bug 1810879 in tripleo "fedora tripleo-ci-testing link creation fails" [Critical,Triaged] | 16:03 |
| panda | rlandy|rover: no | 16:06 |
| panda | rlandy|rover: the module is failing even with my fixes | 16:06 |
| *** bogdando has quit IRC | 16:06 | |
| panda | rlandy|rover: jpena is helping me, but it's going to be tough | 16:06 |
| panda | rlandy|rover: the error doesnt' make any sense | 16:07 |
| panda | rlandy|rover: and I cannot reproduce it locally | 16:07 |
| panda | rlandy|rover: so I can try to run the periodic as checks, or we are going to see resutls of any fix only every 4 hours, and in any case, we need someone in rdo side to tell us what error is zuul outputting, because we are blind | 16:10 |
| rlandy|rover | panda: what is the problem with running periodic as check to test? | 16:11 |
| panda | rlandy|rover: it would help, but we need eyes on the zuul logs as well | 16:14 |
| panda | rlandy|rover: because the only thing we see is "MODULE FAILURE" | 16:14 |
| rlandy|rover | in meeting | 16:15 |
| rlandy|rover | will look in a bit | 16:15 |
| panda | rlandy|rover: and for the record, currently the error is "'An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ValueError: No closing quotation'" | 16:15 |
| *** ykarel|away has quit IRC | 16:17 | |
| *** ykarel|away has joined #oooq | 16:17 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224, (1 more message) | 16:17 |
| marios | quiquell: thanks it voted \o/ https://review.rdoproject.org/r/#/c/18093/ | 16:29 |
| marios | && https://review.rdoproject.org/r/#/c/18079/ | 16:30 |
| marios | hmm sucess 58 mins tho i should probably look into it a bit... | 16:30 |
| *** ccamacho has quit IRC | 16:38 | |
| *** kopecmartin is now known as kopecmartin|off | 16:40 | |
| ssbarnea | out of curiosity, why we do not cache the already resized overcloud image instead of original? we could save some precious time. | 16:53 |
| quiquell | weshay|ruck: Have to drop, do you create another meeting for the repro ? | 17:07 |
| marios | rfolco: scen2 fix at https://review.openstack.org/629264 (updated taiga with the sanity check missing some services & extra horizon notes at https://tree.taiga.io/project/tripleo-ci-board/task/543 | 17:11 |
| marios | rfolco: i'll check 3 tomorrow | 17:13 |
| * marios out time | 17:13 | |
| rfolco | marios, thanks man | 17:14 |
| marios | weshay|ruck: incase you missed it & from my status : * mwhahaha: Y U NO GATE standalone jobs?!?! (+ chat in community call) - first make 2/3 non voting post https://review.openstack.org/#/c/629246/ (not finished checking & 3 broken for https://bugs.launchpad.net/tripleo/+bug/1810970) & post fixup for 1/4 @ https://review.openstack.org/#/q/topic:add-standalone-gates (https://review.openstack.org/620304 /629252 /628266 /629251) | 17:14 |
| openstack | Launchpad bug 1810970 in tripleo "scenario003: fails with TestNetworkBasicOps:test_network_basic_ops on tempest" [High,Triaged] | 17:14 |
| marios | rfolco: have a good 1 | 17:15 |
| rfolco | u have good 1 2 | 17:15 |
| *** quiquell is now known as quiquell|off | 17:19 | |
| quiquell|off | rlandy|rover: Puting here what we need from nodepool-setup for the repro https://review.openstack.org/#/c/629247/ | 17:21 |
| rlandy|rover | k- thanks | 17:24 |
| *** fultonj has joined #oooq | 17:33 | |
| *** holser__ is now known as holser|eod | 17:35 | |
| rlandy|rover | rfolco: how do you move a taiga task from one user story to another? | 17:39 |
| rfolco | rlandy|rover, let me refresh my memory... | 17:40 |
| panda | in the sprint board, drag and drop | 17:40 |
| fultonj | i was able to make a change to have an artifact collected from my undercloud, now I want to make a similar change to have it collected from my overcloud. | 17:41 |
| fultonj | https://review.openstack.org/#/c/615988/6/toci-quickstart/config/collect-logs.yml@255 | 17:41 |
| rlandy|rover | the other user story is not on the same board | 17:42 |
| fultonj | ^ was the change that made my file get picked up on my undercloud, should the same change apply to subnode2 in a multinode job? | 17:42 |
| rfolco | rlandy|rover, as long they are in the same board/sprint | 17:42 |
| rfolco | rlandy|rover, drag and drop | 17:42 |
| rfolco | rlandy|rover, for stories in different sprints, you need to go to backlog, move the story to the target sprint where the new task should be, and then drag from story to story inside the sprint board | 17:43 |
| rfolco | panda, is this an indication that I need undercloud installed and not just the repos configured ? | 17:46 |
| rfolco | INFO:kolla.common.utils.aodh-base:chmod: cannot access '/var/www/cgi-bin/aodh': No such file or directory | 17:46 |
| rfolco | and many containers fail to build due to missing files... | 17:48 |
| rfolco | chmod: cannot operate on dangling symlink '/openstack/healthcheck' | 17:48 |
| rfolco | like this ^ | 17:48 |
| *** trown is now known as trown|lunch | 17:48 | |
| panda | rfolco: you're still building with rdo-kolla role ? | 17:51 |
| rfolco | panda, yes, should I stop ? | 17:51 |
| rfolco | panda, I've got dozens of containers built in f28 using that modified script + hacking kolla to accept fedora as base | 17:53 |
| rfolco | panda, weshay|ruck fyi https://tree.taiga.io/project/tripleo-ci-board/task/567?kanban-status=1447275 | 17:54 |
| rfolco | I just want to have an idea on why the rest fails, if its because I need undercloud or not... | 17:55 |
| panda | I don't think /var/www/cgi-bin/aodh would be in the undercloud | 17:56 |
| panda | and I don't know what /openstack/healthcheck is | 17:56 |
| rfolco | so apparently having the undercloud installed would not make this any better | 17:57 |
| rlandy|rover | rfolco: let's try a diff method - can I make a task into a user story? | 17:57 |
| rfolco | rlandy|rover, if you are talking about the new tripleo-buildocntainers role, I've started it too to compare | 17:58 |
| fultonj | panda: in the past you helped me make this change to have my job pick up this file from the undercloud: https://review.openstack.org/#/c/615988/6/toci-quickstart/config/collect-logs.yml | 17:58 |
| *** ykarel|away has quit IRC | 17:58 | |
| rlandy|rover | rfolco: no I need to move https://tree.taiga.io/project/tripleo-ci-board/task/465 | 17:59 |
| rlandy|rover | to | 17:59 |
| fultonj | i am also using subnode-2 in this job and making the same file but it's not getting picked up | 17:59 |
| rlandy|rover | https://tree.taiga.io/project/tripleo-ci-board/epic/5 | 17:59 |
| fultonj | panda: would you expect the same change to result in it getting picked up? | 17:59 |
| *** derekh has quit IRC | 18:00 | |
| rfolco | rlandy|rover, ah pff crossed chat :) | 18:00 |
| panda | rlandy|rover: you want to move a task into an epic ? | 18:00 |
| rfolco | task into a story I guess? | 18:00 |
| panda | fultonj: I think we use a different config file for collect logs in upstream | 18:01 |
| rlandy|rover | whatever - taiga and I do not get on | 18:01 |
| rfolco | panda, story-to-epic is a possible promotion | 18:01 |
| rfolco | but not task-to-story | 18:01 |
| rfolco | correct ? | 18:01 |
| panda | rfolco: nope | 18:01 |
| panda | fultonj: nevermind, that's the file | 18:02 |
| panda | fultonj: how big is this tarball ? | 18:02 |
| rfolco | rlandy|rover, I know we can promote, just trying to remember how | 18:03 |
| panda | rfolco: promote what ? | 18:03 |
| panda | rfolco: to what ? | 18:03 |
| fultonj | panda: 11k | 18:03 |
| rfolco | promote task-to-story | 18:03 |
| fultonj | panda: i see it picked up on undercloud | 18:03 |
| fultonj | http://logs.openstack.org/88/615988/6/check/tripleo-ci-split-controlplane-standalone/f29bf53/logs/undercloud/home/zuul/ | 18:03 |
| panda | fultonj: aaaaaaaahhhh, disk space murderer!!!! | 18:03 |
| fultonj | my playbook then moves it to the subnode-2 | 18:04 |
| fultonj | but i don't see it on subnode-2 | 18:04 |
| fultonj | http://logs.openstack.org/88/615988/6/check/tripleo-ci-split-controlplane-standalone/f29bf53/logs/subnode-2/home/zuul/ | 18:05 |
| panda | fultonj: I see the collect list is the same in both nodes | 18:08 |
| panda | fultonj: are you certain you're copying it to /home/zuul ? | 18:08 |
| rfolco | panda, rlandy|rover: https://github.com/taigaio/taiga-front/issues/1441 | 18:09 |
| panda | fultonj: that list is then passed to a rsync command, so if it's there, it should be gathered | 18:10 |
| fultonj | panda: right | 18:11 |
| panda | fultonj: is there a link to this copy ? | 18:11 |
| fultonj | https://review.openstack.org/#/c/617368/11/playbooks/split-controlplane-standalone.yml | 18:12 |
| fultonj | panda: i have evidence that this playbook suceeded at creating export_control_plane.tar.gz as i see it on undercloud /home/zuul/ | 18:13 |
| weshay|ruck | rfolco, can I see the kolla config? | 18:13 |
| fultonj | but i don't have evidence that line 53 worked | 18:13 |
| fultonj | https://review.openstack.org/#/c/617368/11/playbooks/split-controlplane-standalone.yml@53 | 18:13 |
| fultonj | as i don't see it on subnode-2 | 18:13 |
| fultonj | however, it's good you confirmed that it should be picked up | 18:14 |
| panda | fultonj: it didn't | 18:14 |
| panda | fultonj: http://logs.openstack.org/88/615988/6/check/tripleo-ci-split-controlplane-standalone/f29bf53/job-output.txt.gz#_2019-01-04_00_59_23_474402 | 18:14 |
| weshay|ruck | rfolco, clearly you don't need an undercloud | 18:14 |
| panda | fultonj: that part was skipped | 18:14 |
| *** rf0lc0 has joined #oooq | 18:14 | |
| *** rfolco has quit IRC | 18:15 | |
| weshay|ruck | rfolco, can you try a manual build as well with the openstack container build command? | 18:15 |
| panda | fultonj: have you passed the proper tags ? I see you require tags: extract | 18:15 |
| panda | rf0lc0: weshay|ruck | rfolco, clearly you don't need an undercloud | 18:16 |
| fultonj | thank you panda! | 18:16 |
| *** rf0lc0 is now known as rfolco | 18:16 | |
| panda | rf0lc0: weshay|ruck | rfolco, can you try a manual build as well with the openstack container build command? | 18:16 |
| panda | fultonj: nothing. Need to drop. See you tomorrow. | 18:16 |
| weshay|ruck | panda, ? | 18:17 |
| *** panda is now known as panda|off | 18:17 | |
| rfolco | yes yes that's what I was going to try before the power outage here due to a rainstorm | 18:17 |
| panda|off | weshay|ruck: I was forwarding to rfolco messages taht he probably missed | 18:17 |
| rfolco | weshay|ruck, panda|off will make it a new task | 18:17 |
| weshay|ruck | k | 18:17 |
| rfolco | panda|off, I got it thanks :) | 18:17 |
| panda|off | rlandy|rover: updated https://bugs.launchpad.net/tripleo/+bug/1810879 | 18:17 |
| openstack | Launchpad bug 1810879 in tripleo "fedora tripleo-ci-testing link creation fails" [Critical,Triaged] | 18:17 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224, (1 more message) | 18:17 |
| panda|off | rlandy|rover: it should work I hope | 18:18 |
| panda|off | rlandy|rover: https://review.rdoproject.org/r/18128 should have fixed that error, as crazy as it sounds | 18:20 |
| panda|off | I hope nothing else is broken in that patch ... | 18:20 |
| panda|off | oh wait | 18:20 |
| weshay|ruck | panda|off, it failed | 18:20 |
| weshay|ruck | 2019-01-08 14:47:26.345615 | TASK [Promote consistent to tripleo-ci-testing for fedora repo if an equivalent exists] | 18:20 |
| weshay|ruck | 2019-01-08 14:47:26.866107 | rdo-centos-7 | MODULE FAILURE | 18:20 |
| weshay|ruck | https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-promote-consistent-to-tripleo-ci-testing/0535278/job-output.txt.gz#_2019-01-08_14_47_26_345615 | 18:20 |
| weshay|ruck | or will that be fixed next run? | 18:20 |
| panda|off | weshay|ruck: yes, this run was before the fix | 18:20 |
| weshay|ruck | panda|off, k.. thanks :) | 18:21 |
| panda|off | but if I added any other apostrophes on any other comments in other tasks, thy could fail too | 18:21 |
| panda|off | no, there are none | 18:22 |
| * panda|off crosses fingers | 18:22 | |
| weshay|ruck | rfolco, nice.. thank you! https://tree.taiga.io/project/tripleo-ci-board/task/573 | 18:25 |
| rfolco | working on it, no results yet | 18:25 |
| rfolco | updated the results in the other one if you haven't seen yet... https://tree.taiga.io/project/tripleo-ci-board/task/567?kanban-status=1447275 | 18:26 |
| rfolco | weshay|ruck, ^ | 18:26 |
| weshay|ruck | rfolco, aye.. I saw it thank you! | 18:26 |
| weshay|ruck | rfolco, it's too bad that keystone doesn't build | 18:27 |
| rlandy|rover | panda|off: thank you | 18:28 |
| rfolco | weshay|ruck, who cares ? | 18:28 |
| * rfolco hides | 18:29 | |
| rlandy|rover | weshay|ruck: sis I understand correctly that https://tree.taiga.io/project/tripleo-ci-board/task/465 should move to https://tree.taiga.io/project/tripleo-ci-board/epic/5 | 18:29 |
| *** bogdando has joined #oooq | 18:29 | |
| rlandy|rover | task -> user story | 18:29 |
| *** jpena is now known as jpena|off | 18:30 | |
| *** dtantsur is now known as dtantsur|afk | 18:45 | |
| rlandy|rover | done now | 18:45 |
| weshay|ruck | rfolco, update the new user stories to "doing" in f28 epic please | 18:49 |
| weshay|ruck | rlandy|rover, /me looking | 18:49 |
| rfolco | weshay|ruck, ok | 18:50 |
| *** holser|eod has quit IRC | 18:54 | |
| *** trown|lunch is now known as trown | 18:55 | |
| rlandy|rover | weshay|ruck: at which step is your setup failing with the new reproducer? | 19:05 |
| weshay|ruck | rlandy|rover, the zuul and zuul scheduler containers fail to start due to a "key" error | 19:06 |
| weshay|ruck | rlandy|rover, have you seen the CI on the repro project? | 19:10 |
| rlandy|rover | weshay|ruck: not really | 19:10 |
| rlandy|rover | weshay|ruck: I did notice the start up changed | 19:10 |
| rlandy|rover | in the doc instructions | 19:11 |
| rlandy|rover | we used to run | 19:11 |
| rlandy|rover | pipenv run ansible-playbook playbooks/start-ci-reproducer.yaml -e ansible_python_interpreter="/usr/bin/python3" -e upstream_gerrit_user=<> | 19:11 |
| rlandy|rover | weshay|ruck: ^^ I assume that is where your error happens | 19:12 |
| weshay|ruck | rlandy|rover, https://logs.rdoproject.org/46/18046/39/gate/tripleo-ci-reproducer-fedora-28/7a65e89/ | 19:13 |
| weshay|ruck | rlandy|rover, https://github.com/rdo-infra/ansible-role-tripleo-ci-reproducer/tree/master/playbooks/tripleo-ci-reproducer | 19:13 |
| weshay|ruck | rlandy|rover, imho the goal would be to ensure the user setup is as close to the CI as possible | 19:14 |
| weshay|ruck | so that's what I was doing, although w/ slightly diff variables passed.. e.g. for the directory where the project is initially checked out etc | 19:14 |
| weshay|ruck | that way we ensure the user experience | 19:14 |
| weshay|ruck | rlandy|rover, wdyt? | 19:15 |
| rlandy|rover | weshay|ruck: in theory, I totally agree | 19:16 |
| weshay|ruck | rlandy|rover, he's working on libvirt I guess too https://review.rdoproject.org/r/#/c/18121/ | 19:16 |
| rlandy|rover | libvirt won't help you if you can't get the first steps to run | 19:17 |
| *** bogdando has quit IRC | 19:23 | |
| rlandy|rover | yay - rhos-14 passing again :) | 19:32 |
| rlandy|rover | arxcruz: still around? | 19:43 |
| rlandy|rover | still working on the rhos-13 failure | 19:43 |
| rlandy|rover | https://github.com/openstack/tripleo-quickstart-extras/commit/6e3c506408887c88e0df38a1fa0614b6279816d6 | 19:43 |
| rlandy|rover | ^^ have impacted? | 19:43 |
| rlandy|rover | 2018-12-31 04:12:06 | 2018-12-31 04:12:06.702 27430 INFO config_tempest.main [-] Adding options from deployer-input file '/home/stack/tempest-deployer-input.conf' | 19:46 |
| rlandy|rover | ^^ no longer runs | 19:46 |
| weshay|ruck | rlandy|rover, nice | 19:47 |
| *** ccamacho has joined #oooq | 19:54 | |
| *** ccamacho has quit IRC | 19:58 | |
| *** jfrancoa has quit IRC | 20:03 | |
| rlandy|rover | arxcruz: /home/stack/tempest/tempest-deployer-input.conf | 20:05 |
| rlandy|rover | vs | 20:05 |
| rlandy|rover | arxcruz: --deployer-input /home/stack/tempest-deployer-input.conf | 20:05 |
| rlandy|rover | ^^ intended change? | 20:06 |
| rlandy|rover | there is no such file ... /home/stack/tempest/tempest-deployer-input.conf | 20:06 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224, (1 more message) | 20:18 |
| weshay|ruck | rlandy|rover, fyi.. rdo-cloud went down | 20:24 |
| weshay|ruck | rlandy|rover, when it comes back, we'll probably have to clean up and hit up kieren | 20:24 |
| rlandy|rover | weshay|ruck: ack - watching rhos-ops | 20:25 |
| rlandy|rover | weshay|ruck: I should not have said things were fine at yesterday's meeting | 20:25 |
| weshay|ruck | lolz | 20:25 |
| weshay|ruck | apparently they are done | 20:25 |
| * weshay|ruck checks stacks | 20:25 | |
| weshay|ruck | rlandy|rover, hrm.. looks clean | 20:26 |
| rlandy|rover | weshay|ruck: yep - doesn't look bad | 20:26 |
| rlandy|rover | weshay|ruck: I'll clean up in a few hours if the last stacks don't go away | 20:27 |
| * rlandy|rover got lost in tempest conf | 20:27 | |
| weshay|ruck | rlandy|rover, if you are interested in seeing how to setup the ruck/rover cockpit locally | 20:28 |
| weshay|ruck | I'm quite good at it now | 20:28 |
| rlandy|rover | weshay|ruck: yes - would be good to see | 20:28 |
| weshay|ruck | rlandy|rover, ok.. about to do it again... | 20:29 |
| weshay|ruck | blue? | 20:29 |
| rlandy|rover | ok | 20:29 |
| weshay|ruck | rfolco, ^ | 20:30 |
| rfolco | weshay|ruck, bj? | 20:30 |
| weshay|ruck | rfolco, if you want to see how to setup cockpit | 20:30 |
| weshay|ruck | rfolco, in my blue | 20:32 |
| weshay|ruck | <hubbot1> FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ https://review.openstack.org/567224, (1 more message) | 20:39 |
| weshay|ruck | rlandy|rover, rfolco ^ | 20:39 |
| weshay|ruck | https://review.rdoproject.org/r/#/c/18102/ | 20:44 |
| *** tosky has quit IRC | 20:49 | |
| *** jbadiapa has quit IRC | 21:10 | |
| rlandy|rover | chandankumar: you around? | 21:14 |
| rlandy|rover | weshay|ruck: did you clean up? | 21:15 |
| weshay|ruck | rlandy|rover, no | 21:16 |
| weshay|ruck | last I looked though it was all create_complete | 21:16 |
| rlandy|rover | ok - then cleaned up all on it own :) | 21:16 |
| rlandy|rover | ok - then things are ok | 21:16 |
| weshay|ruck | sure enough | 21:16 |
| weshay|ruck | rlandy|rover, do you know what the fix is for standalone scenario03 | 21:27 |
| weshay|ruck | fak | 21:30 |
| weshay|ruck | https://bugs.launchpad.net/tripleo/+bug/1810898 | 21:30 |
| openstack | Launchpad bug 1810898 in tripleo "tempest.scenario.test_minimum_basic.TestMinimumBasicScenario failure in CI" [Critical,Triaged] | 21:30 |
| weshay|ruck | rfolco, do you know the fix for scenario03? | 21:31 |
| weshay|ruck | ssbarnea, ^ | 21:31 |
| rfolco | weshay|ruck, I suspect its the missing containers/services but not sure | 21:32 |
| weshay|ruck | marios, mentioned the issue this morning | 21:32 |
| weshay|ruck | rfolco, it was missing services, why is it just failing now? | 21:32 |
| weshay|ruck | rfolco, can you please put in a patch to make it non-voting | 21:33 |
| rfolco | marios did | 21:33 |
| rfolco | weshay|ruck, https://review.openstack.org/#/c/629246/ | 21:36 |
| rfolco | weshay|ruck, need to go, resume tomorrow | 21:36 |
| weshay|ruck | it's about to merge | 21:37 |
| weshay|ruck | good | 21:37 |
| rlandy|rover | weshay|ruck: is that the fix or just the avoidance? | 21:41 |
| weshay|ruck | avoidance | 21:42 |
| rlandy|rover | right - ok - should I work on this or marios and team has it? | 21:43 |
| rlandy|rover | weshay|ruck: is this still a bug? - passing now in gates | 21:46 |
| weshay|ruck | rlandy|rover, what is passing in the gates? | 21:48 |
| weshay|ruck | scen003? | 21:48 |
| rlandy|rover | http://logs.openstack.org/13/619713/5/gate/tripleo-ci-centos-7-standalone/2db9fb6/logs/undercloud/home/zuul/tempest.log.txt.gz | 21:49 |
| rlandy|rover | https://bugs.launchpad.net/tripleo/+bug/1810898 | 21:49 |
| openstack | Launchpad bug 1810898 in tripleo "tempest.scenario.test_minimum_basic.TestMinimumBasicScenario failure in CI" [Critical,Triaged] | 21:49 |
| weshay|ruck | rlandy|rover, ya.. I only see it now on scen003 | 21:50 |
| rlandy|rover | weshay|ruck: so either this goes to be a scen003 issue or we close it | 21:50 |
| rlandy|rover | not assigned to anyone currently | 21:51 |
| weshay|ruck | http://cistatus.tripleo.org/ | 21:51 |
| weshay|ruck | rlandy|rover, this started to fail at 1/08/ 12:34 | 21:51 |
| weshay|ruck | 24 | 21:51 |
| rlandy|rover | I must be looking at the wrong link | 21:53 |
| rlandy|rover | looks green to me | 21:53 |
| rlandy|rover | tripleo-ci-centos-7-standalone | 21:53 |
| rlandy|rover | in both check and gate | 21:53 |
| weshay|ruck | 2019-01-08 14:04:19 | "stderr: Error: unable to find resource 'galera-bundle'", | 21:53 |
| weshay|ruck | rlandy|rover, ugh | 21:54 |
| weshay|ruck | standalone-scenario003 | 21:54 |
| weshay|ruck | rlandy|rover, yes.. ur looking at the wrong link.. the original bug was opened on standalone | 21:54 |
| weshay|ruck | but that does look ok now | 21:54 |
| rlandy|rover | weshay|ruck: k- that is the one I am asking about | 21:55 |
| rlandy|rover | because a scen03 bug is diff, no> | 21:55 |
| rlandy|rover | or we need to edit this | 21:55 |
| weshay|ruck | rlandy|rover, multiple diff failures here | 21:55 |
| rlandy|rover | agreed scen03 is failing | 21:56 |
| weshay|ruck | ugh.. it's messed up, about to go non-voting though | 21:57 |
| weshay|ruck | rlandy|rover, let's leave it for rfolco and marios https://review.openstack.org/#/c/629246/1/zuul.d/standalone-jobs.yaml | 21:58 |
| rlandy|rover | weshay|ruck: ok - sorry to be a pain about this, but we can close https://bugs.launchpad.net/tripleo/+bug/1810898? | 21:58 |
| openstack | Launchpad bug 1810898 in tripleo "tempest.scenario.test_minimum_basic.TestMinimumBasicScenario failure in CI" [Critical,Triaged] | 21:58 |
| weshay|ruck | rlandy|rover, we do need to push these through https://review.openstack.org/#/q/topic:add-standalone-gates+(status:open+OR+status:merged) | 21:58 |
| weshay|ruck | rlandy|rover, no.. I see the issue in scen003 | 21:59 |
| weshay|ruck | it's still a bug | 21:59 |
| rlandy|rover | but the scen03 fail has a diff trace | 22:00 |
| weshay|ruck | rfolco, patch order or depends-on matters | 22:00 |
| rlandy|rover | I don;t mind opening a diff bug | 22:00 |
| weshay|ruck | rlandy|rover, k | 22:00 |
| weshay|ruck | rlandy|rover, do what you think is right | 22:00 |
| weshay|ruck | I'm rechecking these patches | 22:00 |
| * rlandy|rover looks at patches to push | 22:00 | |
| rlandy|rover | https://review.openstack.org/#/c/628266/ was just rechecked | 22:02 |
| rlandy|rover | they are all w+ | 22:02 |
| rlandy|rover | so ok | 22:02 |
| *** trown is now known as trown|outtypewww | 22:07 | |
| rlandy|rover | weshay|ruck; https://bugs.launchpad.net/tripleo/+bug/1811004 - more targeted bug scen003 | 22:15 |
| openstack | Launchpad bug 1811004 in tripleo "tripleo-ci-centos-7-scenario003-standalone fails tempest.scenario.test_minimum_basic.TestMinimumBasicScenario" [Undecided,New] | 22:16 |
| weshay|ruck | rlandy|rover, k.. I saw another issue w/ the deployment too | 22:17 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades @ (1 more message) | 22:18 |
| rlandy|rover | weshay|ruck: pls link or lp and I will investigate | 22:18 |
| weshay|ruck | k k | 22:20 |
| weshay|ruck | rlandy|rover, well. the last 9 failures are basic_ops :0 | 22:22 |
| * weshay|ruck just must have randomly found one | 22:22 | |
| weshay|ruck | last 10 failures | 22:23 |
| rlandy|rover | ok | 22:23 |
| weshay|ruck | rlandy|rover, /me marked your bug.. triaged, critical and ml2 | 22:23 |
| rlandy|rover | ty | 22:23 |
| weshay|ruck | rlandy|rover, and you closed mandre's bug ya? | 22:24 |
| rlandy|rover | weshay|ruck: yes marked it fixed released with copied traces of last successes | 22:24 |
| weshay|ruck | thanks | 22:24 |
| weshay|ruck | perfecto | 22:25 |
| weshay|ruck | rlandy|rover, https://review.openstack.org/#/c/628070/ | 22:46 |
| rlandy|rover | weshay|ruck: there are two comments in there - which I agree with | 22:47 |
| weshay|ruck | k.. | 22:47 |
| weshay|ruck | full links | 22:47 |
| weshay|ruck | hope it doesn't fail lint | 22:49 |
| rlandy|rover | if it does - put back the old one and I'll +2 | 22:50 |
| weshay|ruck | rlandy|rover, have a good night.. congrats on a good ruck/rover sprint :) | 22:53 |
| rlandy|rover | weshay|ruck: you too - good night | 22:53 |
| *** d0ugal has joined #oooq | 22:56 | |
| *** tosky has joined #oooq | 23:01 | |
| *** rascasoft has quit IRC | 23:16 | |
| *** rlandy|rover is now known as rlandy|rover|bbl | 23:16 | |
| *** rascasoft has joined #oooq | 23:28 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!