| *** aakarsh has quit IRC | 00:23 | |
| *** aakarsh has joined #oooq | 00:24 | |
| *** rlandy has quit IRC | 00:25 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ (2 more messages) | 00:30 |
|---|---|---|
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ (2 more messages) | 02:30 |
| *** gkadam has joined #oooq | 02:48 | |
| *** gkadam has quit IRC | 02:50 | |
| *** vinaykns has joined #oooq | 02:53 | |
| *** vinaykns has quit IRC | 03:08 | |
| *** ykarel|away has joined #oooq | 03:27 | |
| *** ykarel|away is now known as ykarel | 03:40 | |
| *** ykarel has quit IRC | 04:11 | |
| *** udesale has joined #oooq | 04:11 | |
| *** aakarsh|2 has joined #oooq | 04:18 | |
| *** aakarsh has quit IRC | 04:18 | |
| *** ykarel has joined #oooq | 04:30 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ (2 more messages) | 04:30 |
| *** ratailor has joined #oooq | 04:59 | |
| *** marios_|ruck has joined #oooq | 05:34 | |
| *** marios has joined #oooq | 05:35 | |
| *** ratailor has quit IRC | 05:45 | |
| *** quiquell|off is now known as quiquell | 05:49 | |
| quiquell | rfolco: stein promoter log path is wrong, that's why we cannot see the log, will change it | 05:49 |
| quiquell | sshnaidm|rover: o/ | 05:50 |
| quiquell | sshnaidm|rover: Looks like /dev/log disappear in the middle of standalone deployment | 05:50 |
| *** ratailor has joined #oooq | 05:50 | |
| quiquell | sshnaidm|rover: in the reproducer | 05:51 |
| *** jfrancoa has joined #oooq | 06:03 | |
| quiquell | marios_|ruck: first step for promoter for stein https://review.rdoproject.org/r/#/c/19335/ <- parameterize stuff | 06:19 |
| marios_|ruck | quiquell: ack will check | 06:22 |
| quiquell | marios_|ruck: is not a big refactor on promoter should be safe | 06:23 |
| quiquell | marios_|ruck: workflow it if you see it's ok | 06:23 |
| marios_|ruck | quiquell: i dont have core there | 06:26 |
| marios_|ruck | quiquell: so can't | 06:26 |
| quiquell | marios_|ruck: ack | 06:27 |
| *** jtomasek has joined #oooq | 06:29 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (1 more message) | 06:30 |
| *** skramaja has joined #oooq | 06:36 | |
| quiquell | sshnaidm|rover: promoter config for fedora/stein | 06:50 |
| quiquell | sshnaidm|rover: https://review.rdoproject.org/r/#/c/19888/ | 06:50 |
| *** kopecmartin|off is now known as kopecmartin | 07:02 | |
| *** tosky has joined #oooq | 07:09 | |
| *** ccamacho has joined #oooq | 07:24 | |
| *** marios_|ruck has quit IRC | 07:34 | |
| *** marios has quit IRC | 07:34 | |
| *** marios_|ruck has joined #oooq | 07:36 | |
| *** marios has joined #oooq | 07:36 | |
| *** ccamacho has quit IRC | 07:42 | |
| *** zul has quit IRC | 07:44 | |
| marios_|ruck | arxcruz: chandankumar kopecmartin folks can you check here when you have a minute please - is that right? comment #9 https://bugs.launchpad.net/tripleo/+bug/1821690 | 07:48 |
| openstack | Launchpad bug 1821690 in tripleo "periodic-ovb-1ctlr_1comp-featureset020-master fails tempest test_auto_allocate_network tempest.api.compute.images.test_images" [High,Triaged] - Assigned to Arx Cruz (arxcruz) | 07:48 |
| *** ccamacho has joined #oooq | 07:49 | |
| *** ykarel is now known as ykarel|lunch | 07:58 | |
| chandankumar | marios_|ruck: sure will take a look in sometime | 07:58 |
| *** jpena|off is now known as jpena | 07:58 | |
| marios_|ruck | chandankumar: thank | 07:59 |
| marios_|ruck | s | 07:59 |
| *** bogdando has joined #oooq | 07:59 | |
| marios_|ruck | sshnaidm|rover: looks like new queens undercloud-upgrade issue gonna file the bug in a sec added note on etherpad for now | 07:59 |
| chandankumar | sshnaidm|rover: i have changed fs039 tempest format to pkg | 08:00 |
| chandankumar | sshnaidm|rover: is there a way to reproduce it locally? | 08:00 |
| quiquell | marios_|ruck: fixed https://review.openstack.org/648902 | 08:00 |
| marios_|ruck | quiquell: undercloud upgrade? | 08:02 |
| marios_|ruck | quiquell: looking | 08:02 |
| quiquell | marios_|ruck: nop tripleo-repos role fix for stein | 08:02 |
| marios_|ruck | quiquell: k | 08:03 |
| quiquell | marios_|ruck: replacing release with ci_branch | 08:03 |
| sshnaidm|rover | marios_|ruck, o/ | 08:03 |
| sshnaidm|rover | chandankumar, not currently :( | 08:03 |
| marios_|ruck | sshnaidm|rover: never mind, its that we have bug alread https://bugs.launchpad.net/tripleo/+bug/1822080 | 08:04 |
| openstack | Launchpad bug 1822080 in tripleo "periodic centos-7-ovb-1ctlr_1comp-featureset020-queens failing tempest "Endpoint not found"" [Critical,Triaged] - Assigned to Martin Kopec (mkopec) | 08:04 |
| sshnaidm|rover | quiquell, /dev/log doesn't disappear in libvirt case, right? | 08:04 |
| marios_|ruck | sshnaidm|rover: https://bugs.launchpad.net/tripleo/+bug/1822120 that one sorry :D | 08:05 |
| openstack | Launchpad bug 1822120 in tripleo "[CI][Queens] undercloud-upgrade task fails due to rabbitmq" [Critical,Triaged] - Assigned to Cédric Jeanneret (cjeanner) | 08:05 |
| quiquell | sshnaidm|rover: have just tested RDO case | 08:06 |
| quiquell | sshnaidm|rover: it disappear in the middle of the run | 08:06 |
| quiquell | sshnaidm|rover: at the beginning is all ok | 08:07 |
| quiquell | :-/ | 08:07 |
| quiquell | maybe standalone job restart the node and cloud-init prevent a proper setup ? | 08:07 |
| quiquell | sshnaidm|rover: so to remove cloud init just package remove cloud-init is enough ? | 08:08 |
| *** gkadam has joined #oooq | 08:10 | |
| arxcruz | marios_|ruck: sorry, i'll take a look, it's been a busy week :/ | 08:11 |
| sshnaidm|rover | quiquell, let's try it: https://review.rdoproject.org/r/19893 | 08:13 |
| quiquell | sshnaidm|rover: I was doing it at pre.yaml | 08:14 |
| quiquell | sshnaidm|rover: But let's just try at cloud_config we will have to fix too for dnf if it works though | 08:14 |
| marios_|ruck | arxcruz: np thanks when you have time. just trying to understand if we're done there. it will come up again in tomorrow escalation call (related https://trello.com/c/im5WRPw4/926-cixlp1821690tripleo-ci-tempest-testautoallocatenetwork-tempestapicomputeimagestestimages#comment-5ca208dadf5d3508f45a22b6 ) | 08:16 |
| arxcruz | marios_|ruck: we are not yet | 08:16 |
| marios_|ruck | arxcruz: don't tell me. tell launchpad when you get a minute please thanks ;D | 08:16 |
| arxcruz | i didn't have success to create an environment to test what's going on | 08:16 |
| arxcruz | lol | 08:16 |
| arxcruz | okay | 08:16 |
| quiquell | sshnaidm|rover: can we merge this ? https://review.rdoproject.org/r/#/c/19335/ | 08:17 |
| arxcruz | marios_|ruck: done | 08:18 |
| marios_|ruck | ack thank you arxcruz | 08:18 |
| sshnaidm|rover | quiquell, done | 08:19 |
| sshnaidm|rover | quiquell, I'll try to reboot it after also, will see how zuul will handle it | 08:21 |
| sshnaidm|rover | quiquell, maybe there is some timeout to wait for host and we can use it.. | 08:22 |
| quiquell | sshnaidm|rover: Running your review now | 08:22 |
| quiquell | sshnaidm|rover: Let's see | 08:22 |
| *** zbr|pto is now known as zbr | 08:23 | |
| zbr | hi i am back, who needs help? | 08:25 |
| quiquell | zbr: hello there | 08:26 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (1 more message) | 08:30 |
| zbr | quiquell: i started to look at tasks part of 837 epic like https://tree.taiga.io/project/tripleo-ci-board/task/871 -- but i am not sure what needs to be done on it. both CR are merged and last comment from wes seems addressed. | 08:33 |
| zbr | rfolco: ^ ? | 08:34 |
| quiquell | zbr: There is stuff to review | 08:38 |
| *** derekh has joined #oooq | 08:41 | |
| zbr | out of curiosity, when is likely for ocata to reach EOL? | 08:49 |
| *** ykarel|lunch is now known as ykarel | 08:50 | |
| zbr | quiquell: i added few comments on https://review.rdoproject.org/r/#/c/19846/' | 08:52 |
| quiquell | zbr: looking thanks ! | 08:57 |
| quiquell | sshnaidm|rover: same error with removing cloud-init :-/ | 09:07 |
| zbr | quiquell: small nit on https://review.openstack.org/#/c/648902/3 -- if you want I can fix it myself. | 09:08 |
| sshnaidm|rover | quiquell, try to add "reboot" after "yum remove" command | 09:08 |
| quiquell | zbr: try not to add patchsets to reviews | 09:13 |
| quiquell | zbr: can be problematic | 09:13 |
| quiquell | zbr: will fix give me a sec | 09:13 |
| *** holser_ has joined #oooq | 09:13 | |
| zbr | quiquell: sure, just let me know if you want me to update one. today I am focused on reviews. | 09:14 |
| *** holser_ has quit IRC | 09:14 | |
| *** ccamacho has quit IRC | 09:14 | |
| *** holser_ has joined #oooq | 09:14 | |
| zbr | quiquell: any reason why https://review.openstack.org/#/c/648950/2 has a WIP ? | 09:25 |
| quiquell | zbr: It's going to be bigger than this like really documenting stuff and refactoring | 09:32 |
| quiquell | zbr: maybe next sprint | 09:32 |
| zbr | quiquell: limit scope, the original remark on the current sprint asked about this minor fix. lets merge it before, and let other refactorings for planning. | 09:33 |
| *** dtantsur|afk is now known as dtantsur | 09:33 | |
| zbr | if bigger work is needed, it deserves a card. but this is minor and I would have included in the original change (nothing wrong to do it outside). | 09:34 |
| chandankumar | zbr: Hello | 09:41 |
| chandankumar | zbr: http://logs.openstack.org/80/648680/5/check/tripleo-ci-centos-7-standalone-os-tempest/2fc7e12/logs/ara_oooq/ara-report/result/4125c788-594e-44c4-80e9-fb64ef3b6af0/ | 09:41 |
| chandankumar | zbr: I am hitting template error https://review.openstack.org/#/c/648678/4/playbooks/tempest.yml | 09:42 |
| chandankumar | please have a look when free! | 09:42 |
| zbr | looking now... | 09:42 |
| zbr | chandankumar: wondering if this a bug in older ansible, testing now. | 09:44 |
| zbr | chandankumar: first error: item in allowed_services -- no quotes | 09:46 |
| zbr | you must avoid using{{ }} | 09:46 |
| zbr | chandankumar: i fixed it, lets see how it runs now. | 09:49 |
| chandankumar | zbr: yes, cool, got it | 09:49 |
| chandankumar | look for var in var_list | 09:50 |
| chandankumar | normal syntex | 09:50 |
| zbr | i use ansible 2.7 locally so there is a risk of finding surprises on older versions. | 09:50 |
| quiquell | zbr: fixed https://review.rdoproject.org/r/#/c/19846/ | 09:55 |
| quiquell | thanks | 09:55 |
| quiquell | zbr: about https://review.openstack.org/#/c/648950/2 is already commented that we will doo the stein part at this sprint and that in the next, but I did start it | 09:55 |
| zbr | quiquell: ok. now lets get help to review/merge it,. marios_|ruck sshnaidm|rover please help on https://review.rdoproject.org/r/#/c/19846/ | 09:58 |
| zbr | quiquell: can I please fix https://review.openstack.org/#/c/648950/1 so we can merge it? | 09:58 |
| marios_|ruck | zbr: ack will check in bit but am not core there anyway | 09:59 |
| quiquell | zbr: ahh I forgot | 09:59 |
| quiquell | zbr: let me address that | 09:59 |
| *** ccamacho has joined #oooq | 10:00 | |
| quiquell | zbr: fixed https://review.openstack.org/648950 | 10:01 |
| zbr | quiquell: now one trick that you could do is to rebase your 647419 change on top of ^ -- so they will both merge quickly, otherwise you will have to rework it later. | 10:03 |
| quiquell | zbr: we don't want to merge that | 10:04 |
| quiquell | zbr: more refactor on emit releases will be done | 10:04 |
| quiquell | zbr: only the stein part is going to be merged (if possible) | 10:04 |
| quiquell | zbr: This is what I understood from yesterday | 10:04 |
| chandankumar | arxcruz: https://review.openstack.org/#/c/648110/ please rebase it so that we can start adding os_tempest periodic scenario jobs | 10:17 |
| zbr | quiquell: i think you need to rework https://review.rdoproject.org/r/#/c/18892/ a little bit | 10:24 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (1 more message) | 10:30 |
| quiquell | zbr: let rebase | 10:30 |
| *** yolanda has quit IRC | 10:39 | |
| arxcruz | chandankumar: done | 10:44 |
| *** amoralej is now known as amoralej|lunch | 10:47 | |
| zbr | marios_|ruck or quiquell : simple https://review.openstack.org/#/c/645166/ | 11:03 |
| chandankumar | oh no a small typo | 11:07 |
| *** udesale has quit IRC | 11:10 | |
| chandankumar | zbr: by the way http://logs.openstack.org/80/648680/5/check/tripleo-ci-centos-7-standalone-os-tempest/caf7638/logs/ara_oooq/ara-report/result/42dd0d18-f3ad-4e0b-9377-18b91fd44939/ is working | 11:12 |
| chandankumar | zbr: but due to typo in var it does not installed anything in job | 11:12 |
| chandankumar | fixed it right now, waiting for next run | 11:12 |
| chandankumar | thanks for the help :-) | 11:12 |
| zbr | cool | 11:12 |
| quiquell | zbr: arrived late at https://git.openstack.org/cgit/openstack/tripleo-common/plain/tripleo_common/image/builder/buildah.py?h=refs/changes/50/649250/3 | 11:15 |
| quiquell | zbr: I mean https://git.openstack.org/cgit/openstack/tripleo-common/plain/tripleo_common/image/builder/buildah.py?h=refs/changes/50/649250/3 | 11:15 |
| quiquell | sshnaidm|rover: standalone + repro + cloud-init fix working :-) | 11:20 |
| quiquell | sshnaidm|rover: it's at tempest now | 11:20 |
| sshnaidm|rover | quiquell, \o/ | 11:20 |
| quiquell | sshnaidm|rover: we have to fix it for fedora aka dnf | 11:20 |
| sshnaidm|rover | quiquell, yeah, will do | 11:22 |
| sshnaidm|rover | quiquell, what's good - with update we don't need to update images anymore | 11:23 |
| sshnaidm|rover | quiquell, that's fine just to run dnf/yum update and restart | 11:23 |
| sshnaidm|rover | quiquell, and we can use the same images, making them public | 11:23 |
| quiquell | sshnaidm|rover: well updates takes time... | 11:24 |
| sshnaidm|rover | quiquell, not much though | 11:24 |
| quiquell | sshnaidm|rover: I would still prefer to use upstream images | 11:24 |
| quiquell | sshnaidm|rover: we don't know what else could be there | 11:24 |
| quiquell | sshnaidm|rover: but at least it has less priority since ours are updated now | 11:24 |
| sshnaidm|rover | quiquell, it worked that way with old reproducer always | 11:24 |
| quiquell | sshnaidm|rover: then let's forbout about updte images for now | 11:25 |
| quiquell | sshnaidm|rover: and if we find issues let's do that | 11:25 |
| sshnaidm|rover | quiquell, if we find way to make image cloud-wide public I'm only for using upstream images | 11:25 |
| *** yolanda has joined #oooq | 11:26 | |
| quiquell | sshnaidm|rover: we are already using cloud-wide public images | 11:26 |
| sshnaidm|rover | quiquell, what do you mean? | 11:27 |
| *** jpena is now known as jpena|lunch | 11:29 | |
| *** amoralej|lunch is now known as amoralej | 11:29 | |
| quiquell | upstream-cloudinit-centos-7 image are public | 11:29 |
| quiquell | they make that public | 11:29 |
| quiquell | But don't know if we can make that automatic | 11:29 |
| sshnaidm|rover | quiquell, we need ask kforde every time to do it | 11:47 |
| sshnaidm|rover | quiquell, and I'm not sure these images are cloud public yet.. | 11:48 |
| quiquell | nah let's just forget for now | 11:49 |
| weshay | quiquell sshnaidm|rover you guys solve the issue w/ the repro? | 11:59 |
| quiquell | weshay: look like https://review.rdoproject.org/r/#/c/19893/ | 12:00 |
| quiquell | weshay: sshnaidm|rover put a patch to cleanup cloud-init and update images | 12:01 |
| quiquell | weshay: I have test it and it has pass standalone deploy | 12:01 |
| quiquell | Yep I have a SUCCESS here so it's working | 12:02 |
| weshay | \0/ | 12:04 |
| weshay | \0/ | 12:04 |
| weshay | wtf.. how does cloud-init fuck that up | 12:05 |
| weshay | that was crazy | 12:05 |
| *** matbu has quit IRC | 12:06 | |
| sshnaidm|rover | weshay, it could be update also.. | 12:06 |
| sshnaidm|rover | testing with OVB now, it may require additional fixes for it | 12:06 |
| rfolco | quiquell, promoter path, where is the patch ? | 12:11 |
| quiquell | rfolco: https://review.rdoproject.org/r/#/c/19888/ | 12:11 |
| quiquell | rfolco: now we can do QE at promoter it should beok now | 12:12 |
| quiquell | rfolco: also containers at fedora/stein are ther e | 12:12 |
| rfolco | quiquell, how did you verify? | 12:13 |
| quiquell | rfolco: well it's expecting that path | 12:14 |
| quiquell | rfolco: but didn't run a private promoter though :-/ | 12:14 |
| quiquell | weshay: what do we do whit stein emiter ? https://review.openstack.org/#/c/647419 | 12:14 |
| rfolco | quiquell, trunk.registry.rdoproject.org/... | 12:15 |
| weshay | ya.. I'm so locked down w/ bs meetings atm | 12:15 |
| weshay | having trouble getting to that | 12:15 |
| quiquell | rfolco: added my comment back to https://review.openstack.org/#/c/648485/ | 12:15 |
| quiquell | rfolco: maybe I am wrong and that is testing elsewhere ? | 12:15 |
| quiquell | rfolco: you mean verify containers | 12:16 |
| quiquell | rfolco: use skope, give me a sec will give you command | 12:16 |
| rfolco | quiquell, yes, there is no web like docker has | 12:16 |
| rfolco | dockerhub I mean | 12:17 |
| quiquell | rfolco: so first we go to the review https://review.rdoproject.org/r/#/c/19819/ | 12:17 |
| quiquell | rfolco: then in the job we take one of the pushed images http://logs.rdoproject.org/19/19819/9/check/periodic-tripleo-fedora-28-stein-containers-build-push/cbeb6ad/ | 12:17 |
| quiquell | rfolco: for example trunk.registry.rdoproject.org/tripleostein/fedora-binary-base:a3a7c66ff3e578384135f7adccdf0ca18ea09d34_d70f8b29 | 12:18 |
| quiquell | rfolco: skopeo inspect docker://trunk.registry.rdoproject.org/tripleostein/fedora-binary-base:a3a7c66ff3e578384135f7adccdf0ca18ea09d34_d70f8b29 | 12:19 |
| quiquell | rfolco: that's working fine | 12:19 |
| quiquell | rfolco: and now same thing ro dockerhub | 12:19 |
| *** aakarsh|2 has quit IRC | 12:19 | |
| quiquell | rfolco: skopeo inspect docker://docker.io/tripleostein/fedora-binary-base:a3a7c66ff3e578384135f7adccdf0ca18ea09d34_d70f8b29 | 12:20 |
| quiquell | rfolco: they are there | 12:20 |
| quiquell | rfolco: so I think the behaviour is changed here and something is missing, but maybe I am wrong https://review.openstack.org/#/c/648485/ | 12:20 |
| quiquell | rfolco: let's clarify to merge this | 12:21 |
| rfolco | quiquell, ok thanks | 12:21 |
| *** trown|outtypewww is now known as trown | 12:24 | |
| chandankumar | arxcruz: now we can install tempest plugins also in os_temepst easily http://logs.openstack.org/80/648680/5/check/tripleo-ci-centos-7-standalone-os-tempest/a218fe6/logs/ara_oooq/ara-report/result/7e6ec142-a6a6-4e54-a4d1-97ddc7c0d151/ | 12:26 |
| *** jpena|lunch is now known as jpena | 12:27 | |
| rfolco | quiquell, why you want to handle f29 case now? | 12:27 |
| rfolco | quiquell, the code is handling f28 and centos7 | 12:28 |
| quiquell | rfolco: This was just an example, previous version was going to fail with something like that current one is not | 12:28 |
| quiquell | rfolco: but also stuff different from fedora28 and centos7 is going to pass and it was not the case before | 12:28 |
| weshay | rfolco f29 for which bit? | 12:28 |
| rfolco | quiquell, https://review.openstack.org/#/c/648485/1/tripleo_repos/main.py | 12:28 |
| quiquell | rfolco: I mean fedora28 and centos7 with != master | 12:28 |
| rfolco | weshay, ^ | 12:28 |
| quiquell | rfolco: more than f28 is about passing something that is not fedora28 or centos7 stuff | 12:29 |
| rfolco | ok so if any other distro, only master is ok | 12:29 |
| quiquell | rfolco: before we were failing | 12:29 |
| quiquell | rfolco: yep | 12:29 |
| rfolco | k | 12:29 |
| quiquell | that's it | 12:29 |
| rfolco | quiquell, arriba muchacho | 12:29 |
| quiquell | hehehe | 12:30 |
| rfolco | quiquell, :) | 12:30 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (1 more message) | 12:30 |
| marios_|ruck | weshay: call? | 12:31 |
| weshay | marios_|ruck I sent an email update about that meeting | 12:31 |
| *** matbu has joined #oooq | 12:33 | |
| marios_|ruck | weshay: ok we me emilien jaosorior and jpena were on the call | 12:33 |
| marios_|ruck | dropping? | 12:33 |
| marios_|ruck | weshay: we just dropped | 12:34 |
| weshay | k.. man | 12:34 |
| weshay | I scheduled that meeting months ago | 12:34 |
| marios_|ruck | i went and washed my face for nothing man | 12:34 |
| weshay | marios_|ruck bathing is so overrated | 12:35 |
| *** rlandy has joined #oooq | 12:39 | |
| *** aakarsh|2 has joined #oooq | 12:40 | |
| rlandy | rfolco; branching meeting? | 12:42 |
| rfolco | rlandy, :O | 12:43 |
| rfolco | 1 min | 12:43 |
| rlandy | I joined and nobody was there so left | 12:43 |
| rfolco | rlandy, I forgot about this one... ok so no show? | 12:45 |
| rlandy | rfolco: it was no show from what I could tell | 12:46 |
| rfolco | k | 12:46 |
| rlandy | rfolco: sshnaidm|rover: could we get the final word on https://review.rdoproject.org/r/#/c/19580/? ditch the review or not? | 12:47 |
| sshnaidm|rover | rlandy, I left comments there | 12:48 |
| rlandy | sshnaidm|rover: iiuc, you think we need nothing of that review | 12:48 |
| rfolco | rlandy, the task seems to be a no-op change. I'd just comment in the task what are the jobs in 3rd party that will run for stein. Make sense? | 12:50 |
| jdennis | I'm trying to do a oooq deployment with a supplemental node but the network service on the supplementation is failing with this error: "Device eth1 does not seem to be present", the undercloud node seems to have deployed OK and has eth0 and eth1, any idea of what went wrong on the supplemental node and how to fix it? | 12:50 |
| rlandy | sshnaidm|rover: rfolco: the vexxhost job is legit | 12:51 |
| *** aakarsh|2 is now known as aakarsh | 12:52 | |
| rlandy | I will confirm with weshay | 12:52 |
| *** mjturek has joined #oooq | 12:52 | |
| sshnaidm|rover | rlandy, it's already there | 12:52 |
| rfolco | if not blocking stein in branches regex, we're covered | 12:53 |
| *** mjturek has quit IRC | 12:53 | |
| sshnaidm|rover | rlandy, https://github.com/rdo-infra//rdo-jobs/blob/master/zuul.d/ovb-jobs.yaml#L233 | 12:54 |
| *** mjturek has joined #oooq | 12:54 | |
| rfolco | quiquell, there is no need to add a else or elif | 12:55 |
| rfolco | quiquell, other distros won't be considered | 12:55 |
| quiquell | rfolco: nope ? were is the check ? | 12:55 |
| rfolco | quiquell, there is a check ... | 12:55 |
| rfolco | if distro not in SUPPORTED_DISTROS: | 12:55 |
| rfolco | so only f28 and centos7 are accepted | 12:56 |
| rfolco | centos7 works for non master | 12:56 |
| rfolco | f28 I added the condition, only stein | 12:56 |
| rfolco | quiquell, it looks good to me | 12:58 |
| quiquell | rfolco: then we are all good | 12:59 |
| quiquell | rfolco: will remove my -1 do you have a pointer to SUPPORTED_DISTROS ? | 12:59 |
| *** quiquell is now known as quiquell|lunch | 13:00 | |
| quiquell|lunch | rfolco++ unit test was the propert thing | 13:03 |
| hubbot1 | quiquell|lunch: rfolco's karma is now 4 | 13:03 |
| quiquell|lunch | rfolco: +w it | 13:03 |
| rfolco | quiquell|lunch, thank you for your patience :) | 13:03 |
| quiquell|lunch | rfolco: what pieces do we miss for the stein enchilada ? | 13:04 |
| quiquell|lunch | rfolco: don't know what to do with stein emit releases | 13:04 |
| quiquell|lunch | rfolco: https://review.openstack.org/#/c/647419 | 13:04 |
| quiquell|lunch | rfolco: apart from adding a proper commit message | 13:04 |
| quiquell|lunch | jaosorior: +A to be able to bypass push at buildah builder https://review.openstack.org/#/c/649250/ ? | 13:06 |
| rfolco | rlandy, https://tree.taiga.io/project/tripleo-ci-board/task/871 --> promoter path, quiquell|lunch fixed it, pls double check and close it | 13:11 |
| zbr | re emit releases, i would merge https://review.openstack.org/#/c/648950/ first <-- rlandy marios_|ruck | 13:13 |
| quiquell|lunch | zbr: that's was only part my idea is to do the sme at all unit tests | 13:15 |
| rfolco | rlandy, sshnaidm|rover marios_|ruck weshay zbr: review this quick one pls: https://review.openstack.org/#/c/648485/ | 13:17 |
| marios_|ruck | rfolco: quique beat you to it 15 mins ago apparently | 13:18 |
| marios_|ruck | or wrong link | 13:18 |
| rfolco | marios_|ruck, ah +w already | 13:19 |
| rfolco | sorry | 13:19 |
| rfolco | :) | 13:19 |
| rfolco | nm | 13:19 |
| marios_|ruck | np | 13:20 |
| quiquell|lunch | marios_|ruck, rfolco: I am brainless merger | 13:21 |
| rfolco | quiquell|lunch, +1 on your emit_releases patches... if you have one more update, please add some code comments so next update (train) this will be easier | 13:22 |
| rlandy | rfolco: looking | 13:23 |
| quiquell|lunch | rlandy: idea is to refactor this after stein so it's easier to modify | 13:24 |
| quiquell|lunch | This was for rfolco | 13:24 |
| *** quiquell|lunch is now known as quiquell | 13:24 | |
| marios_|ruck | sshnaidm|rover: almost there .... https://www.timeanddate.com/countdown/generic?iso=20190403T18&p0=680&font=cursive&csz=1 | 13:25 |
| marios_|ruck | also end sprint 7 ! ^ merge merge merge | 13:26 |
| marios_|ruck | (but please don't break any jobs thanks) | 13:26 |
| rlandy | rfolco: k - looking | 13:26 |
| sshnaidm|rover | marios_|ruck, lol | 13:27 |
| rfolco | rlandy, I just received invitation for branching sync mtg, but the time is in the past.... | 13:29 |
| sshnaidm|rover | marios_|ruck, are you drawing lines on your room wall? | 13:29 |
| rlandy | rfolco: don't worry about it - I think everyone thought it was a no show | 13:30 |
| *** vinaykns has joined #oooq | 13:30 | |
| marios_|ruck | sshnaidm|rover: of course ;) | 13:30 |
| marios_|ruck | sshnaidm|rover: man i was really hoping to end with queens promotion but not sure now... maybe | 13:31 |
| marios_|ruck | sshnaidm|rover: the queens bugs are now blocking each other. https://review.openstack.org/#/c/649084/ will fix the promotion blocker but is blocked on that bug https://bugs.launchpad.net/tripleo/+bug/1822120 | 13:31 |
| openstack | Launchpad bug 1822120 in tripleo "[CI][Queens] undercloud-upgrade task fails due to rabbitmq" [Critical,Triaged] - Assigned to Cédric Jeanneret (cjeanner) | 13:31 |
| sshnaidm|rover | marios_|ruck, yeah, we need to figure out how was it possible to break it in so many ways.. | 13:32 |
| *** ratailor has quit IRC | 13:32 | |
| zbr | rfolco: please block https://review.openstack.org/#/c/648485/1 before it merges. | 13:33 |
| zbr | we still need to be able to run fedora28 on master till we have centos8 | 13:34 |
| rfolco | zbr, re-read if statement | 13:35 |
| rfolco | it does NOT block master for f28 | 13:35 |
| rfolco | it just blocks stable branches BEFORE stein | 13:35 |
| zbr | rfolco: aaah.. | 13:35 |
| rfolco | :) | 13:36 |
| zbr | ok. mea culpa | 13:36 |
| rfolco | zbr, thanks anyway for reviewing it. More eyes, more chances to be doing the right thing. | 13:37 |
| zbr | true | 13:37 |
| sshnaidm|rover | zbr, do you where is the ansible-lint rule about "Variables should have spaces before and after:" ? | 13:39 |
| sshnaidm|rover | zbr, it should be fixed.. | 13:39 |
| zbr | sshnaidm|rover: what do youmean by it? you want to disable this rule? | 13:40 |
| sshnaidm|rover | zbr, I want to fix it | 13:41 |
| zbr | sshnaidm|rover: which repo, i can check. probably is not enabled everywhere. | 13:41 |
| sshnaidm|rover | zbr, http://logs.rdoproject.org/93/19893/3/check/tox-linters/aad9a4f/job-output.txt.gz | 13:42 |
| zbr | (me looking) | 13:44 |
| rlandy | quiquell: https://review.openstack.org/#/c/648950/ ready to go? | 13:46 |
| marios_|ruck | sshnaidm|rover: that should help at least https://review.openstack.org/649035 | 13:46 |
| zbr | sshnaidm|rover: that one is interesting, i am unable to reproduce that error locally. likely that it comes from code that is not in that change. | 13:46 |
| quiquell | rlandy: This is a WIP want to do the same for all the unit tests | 13:47 |
| *** chem has quit IRC | 13:47 | |
| *** chem has joined #oooq | 13:48 | |
| quiquell | rlandy: the important one for this sprint is this https://review.openstack.org/#/c/647419/ | 13:49 |
| quiquell | rlandy: but don't know if weshay want to -2 this one and continue with fixing the script in the other | 13:49 |
| rlandy | quiquell; ack - was going to ask about that | 13:50 |
| rlandy | -2 from weshay is unusual | 13:50 |
| quiquell | rlandy: another important one is this https://review.rdoproject.org/r/#/c/19846/ | 13:50 |
| zbr | rlandy: indeed. my guess was timing on that one. | 13:50 |
| quiquell | rlandy: yep he wanto to improve the script so it's easier to introduce changes | 13:50 |
| quiquell | rlandy: it has grow very badly | 13:50 |
| quiquell | have to fix stein there too | 13:51 |
| rlandy | quiquell: k - merging https://review.rdoproject.org/r/#/c/19846/ - lgtm | 13:52 |
| rlandy | marios_|ruck: ^^ sorry in advance of possible breakage | 13:52 |
| quiquell | rlandy: wait I am fixing stein there | 13:52 |
| quiquell | rlandy: ok fixed look at it now | 13:53 |
| rlandy | quiquell: k - going now | 13:53 |
| quiquell | do I activate fedora stein too ? | 13:53 |
| quiquell | in the dlrn-promoter.sh | 13:53 |
| *** Vorrtex has joined #oooq | 13:53 | |
| marios_|ruck | rlandy: ack (noooooooo) thanks | 13:54 |
| rlandy | marios_|ruck: from tomorrow it won't be your problem anymore :) | 13:54 |
| quiquell | rlandy: can you re-vote ? https://review.rdoproject.org/r/#/c/19846/ | 13:55 |
| quiquell | rlandy: I have activate fedora-stein too | 13:55 |
| rlandy | yep - after zuul votes - you done now with changes :)? | 13:56 |
| quiquell | rlandy: no more sorry, all yours now | 13:56 |
| rlandy | ok - here we go | 13:58 |
| quiquell | yey ! | 13:58 |
| rlandy | quiquell: damage is done | 13:58 |
| * rlandy QE's 871 now | 13:59 | |
| * quiquell prepares the helmet | 13:59 | |
| quiquell | Let's ensure tha promoter take the changes | 13:59 |
| quiquell | stein logs should be there already | 13:59 |
| rlandy | right I need to see stein promote before confirming task done | 14:00 |
| quiquell | rlandy: not all jobs are passing | 14:00 |
| quiquell | rlandy: we need the emit releases fix | 14:00 |
| rlandy | quiquell: which one? | 14:01 |
| quiquell | rlandy: previews one with -2 from wes | 14:01 |
| rfolco | https://review.openstack.org/648950 and https://review.openstack.org/648950 | 14:01 |
| rfolco | rlandy, ^ | 14:01 |
| rfolco | oops | 14:01 |
| rfolco | https://review.openstack.org/647419 | 14:02 |
| marios_|ruck | tripleo weekly go wave | 14:02 |
| quiquell | rfolco, rlandy: this one https://review.openstack.org/#/c/647419/ | 14:02 |
| rlandy | quiquell: rfolco: not sure we can override another core's -2 | 14:03 |
| rfolco | its just weshay ignore him :) | 14:03 |
| rfolco | I think his point was to get upgrade folks involved and write some comments / doc for better maintainability | 14:04 |
| rfolco | rlandy, quiquell ^ | 14:04 |
| rlandy | rfolco: think he wanted quiquell to add doc, no? | 14:04 |
| quiquell | rlandy, rfolco: jfrancoa has already verify it, the failing standalone upgrade is legit now | 14:05 |
| rlandy | quiquell: I +2'ed it fwiw | 14:05 |
| quiquell | rlandy: don't know if I have to add it here or in the refacotring review | 14:05 |
| rfolco | yes a small readme or code comments would help | 14:05 |
| zbr | sshnaidm|rover: try to rebase https://review.rdoproject.org/r/#/c/19893/ on top of 56323067a946d83e6246c5058bbe4f494c7d8820 -- somehow rdo project does not allow me to do this, even if this is enabled for anyone on upstream. | 14:05 |
| sshnaidm|rover | zbr I have variable {{ var -}} and this rule sure detects "-" and requires a space to be before }} | 14:05 |
| rlandy | quiquell: rfolco: other reviews to vote on/merge? | 14:05 |
| zbr | sshnaidm|rover: is likely an already fixed bug in linter, bumping should about it but double check with ^' | 14:06 |
| quiquell | rfolco: going to move to done the #871 taiga task | 14:07 |
| quiquell | rlandy: promoter config | 14:07 |
| rlandy | rfolco: quiquell: no stein logs yet on http://38.145.34.55/ | 14:07 |
| quiquell | rlandy: I don't have any | 14:07 |
| quiquell | rlandy: that's bad | 14:07 |
| quiquell | rlandy: moved https://tree.taiga.io/project/tripleo-ci-board/task/871?kanban-status=1538166 to QE | 14:07 |
| quiquell | rlandy: Until we see logs there | 14:07 |
| quiquell | Have to drop in a few though | 14:08 |
| rlandy | quiquell: ack - I will wait for logs to confirm | 14:08 |
| sshnaidm|rover | zbr, ok, trying | 14:09 |
| quiquell | rlandy: promoter is at master | 14:09 |
| rlandy | k, waiting on stein | 14:10 |
| zbr | sshnaidm|rover: on the funny side, i encountered the same bug myself few weeks back, i remember a bump worked. still what I fail to understand is why I cannot reproduce it locally. | 14:10 |
| quiquell | rlandy: stein is the last one it will take some time | 14:10 |
| quiquell | have to drop | 14:10 |
| quiquell | read you tomorrow guys | 14:10 |
| *** quiquell is now known as quiquell|off | 14:11 | |
| *** holser_ has quit IRC | 14:17 | |
| *** holser__ has joined #oooq | 14:17 | |
| zbr | sshnaidm|rover: i guess this PR is fixing the bug you found https://github.com/ansible/ansible-lint/pull/500#pullrequestreview-221687641 right? | 14:17 |
| sshnaidm|rover | zbr, yeah, but it's not merged yet, right? | 14:24 |
| zbr | yep, in this case solution is to temporary add exclude for it, with comment pointing to the bug. | 14:24 |
| zbr | sshnaidm|rover: i will update your change and add the exclude. ok? | 14:24 |
| zbr | or you can do it, just add # - '206' ## temporary until https://github.com/ansible/ansible-lint/pull/500 is released. | 14:25 |
| zbr | to .ansible-lint file, there is another one there already. | 14:25 |
| zbr | always add link to bug when you do, so we would know when we can undo them | 14:26 |
| zbr | i am now trying to wake the maintainer to merge it and make a release, usually he is very quick on stuff like this. | 14:26 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (1 more message) | 14:30 |
| sshnaidm|rover | zbr, better you'd do it | 14:34 |
| *** skramaja has quit IRC | 14:36 | |
| sshnaidm|rover | quiquell|off, it's fine for ovb also: https://review.rdoproject.org/r/#/c/19893/ | 14:37 |
| *** ccamacho has quit IRC | 14:41 | |
| marios_|ruck | scenario 7 ready for review please https://review.openstack.org/631496 (env file) https://review.openstack.org/631497 (job definition) https://tree.taiga.io/project/tripleo-ci-board/us/341 | 14:41 |
| marios_|ruck | thank you | 14:41 |
| rfolco | marios_|ruck, will review in a bit | 14:41 |
| marios_|ruck | rfolco: thx | 14:42 |
| *** chem has quit IRC | 14:48 | |
| *** chem has joined #oooq | 14:49 | |
| sshnaidm|rover | weshay, with that reproducer is great again: https://review.rdoproject.org/r/#/c/19893 | 14:50 |
| rlandy | community meeting? | 14:51 |
| weshay | please review https://review.rdoproject.org/r/#/c/19893/ | 14:53 |
| weshay | marios_|ruck anything I can do to help you guys unblock queens? | 14:54 |
| marios_|ruck | weshay: well we're waiting on that fix for undercloud-upgrades at the moment | 14:55 |
| *** yolanda has quit IRC | 14:55 | |
| marios_|ruck | weshay: otherwise not much we can do | 14:55 |
| weshay | k.. thanks | 14:55 |
| marios_|ruck | weshay: folks are on that afaics (dciabrin etc) | 14:55 |
| weshay | k k | 14:56 |
| weshay | marios_|ruck everything else looks good to me.. :) | 14:57 |
| rlandy | weshay; pls see if you agree that this card is now a no-op ... https://tree.taiga.io/project/tripleo-ci-board/task/885?kanban-status=1447276 | 14:57 |
| rlandy | see sshnaidm|rover and rfolco's comments | 14:57 |
| marios_|ruck | weshay: let's not dwell on that ! (master/rocky/pike promote again today) | 14:58 |
| weshay | ya man :) | 14:58 |
| marios_|ruck | https://www.timeanddate.com/countdown/generic?iso=20190403T18&p0=680&font=cursive&csz=1 | 14:58 |
| rlandy | lol | 14:58 |
| weshay | rlandy ya.. I think you are correct | 14:59 |
| weshay | at the same time.. diff task is ensuring tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-stein-branch | 14:59 |
| weshay | exists | 14:59 |
| rlandy | weshay: k - abandoning review - moving card to done | 15:00 |
| weshay | rfolco rlandy is there a change that creates that job.. I think there is | 15:00 |
| weshay | recalls seeing it last night | 15:00 |
| rlandy | this work was ignored and done elsewhere | 15:00 |
| weshay | rfolco I started the us-8 board | 15:00 |
| weshay | whayutin•RDO-CI/TRIPLEO-QUICKSTART/review.rdoproject.org-config ᐅ egrep -rn tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-stein-branch master✔ thinkdoe ⌚ 9:01:58 | 15:02 |
| weshay | whayutin•RDO-CI/TRIPLEO-QUICKSTART/review.rdoproject.org-config ᐅ cd ../rdo-jobs master✔ thinkdoe ⌚ 9:02:03 | 15:02 |
| weshay | whayutin•RDO-CI/TRIPLEO-QUICKSTART/rdo-jobs ᐅ egrep -rn tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-stein-branch master✔ thinkdoe ⌚ 9:02:06 | 15:02 |
| weshay | whayutin•RDO-CI/TRIPLEO-QUICKSTART/rdo-jobs ᐅ | 15:02 |
| weshay | rfolco rlandy is there a patch up for that job? | 15:02 |
| rlandy | the branchful jobs were another task | 15:03 |
| jdennis | any thoughts on how to fix the "Device eth1 does not seem to be present" error I posted above ^^^ | 15:03 |
| weshay | rlandy right.. but just want to confirm we have it.. no matter what task / epic it's in | 15:04 |
| weshay | jdennis can you please paste it, done't see it | 15:04 |
| rlandy | weshay: getting | 15:07 |
| weshay | sshnaidm|rover so the task I was reffering to was #670 run specific tempest tests on fs039 novajoin I think | 15:08 |
| rlandy | weshay: https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/ovb-jobs.yaml#L202 | 15:08 |
| weshay | appropriate for sprint 8 or still blocked | 15:08 |
| weshay | rlandy++ thanks | 15:09 |
| hubbot1 | weshay: rlandy's karma is now 51 | 15:09 |
| jdennis | weshay: oooq deploy with supplemental node, undercloud deploy was successful but on the supplemental node networking won't come up, journalctl on supplemental shows this error: "Device eth1 does not seem to be present", "Failed to start LSB: Bring up/down networking.". Note, the undercloud has both eth0 and eth1, but the supplemental seems to only have eth0 | 15:11 |
| weshay | sshnaidm|rover ^ is that image related? perhaps ovb has networking setup on the supp node and not sure how jdennis is trying it atm | 15:12 |
| zbr | rlandy or weshay: one liner https://review.openstack.org/#/c/648902/ | 15:12 |
| weshay | sshnaidm|rover perhaps the virt config is off for the supp node in config? | 15:12 |
| jdennis | weshay: I did capture the full output of quickstart.sh at -vvvv log level if you want me to search for something | 15:13 |
| *** yolanda has joined #oooq | 15:13 | |
| weshay | jdennis if you can post it.. that would be helpful.. also make sure we have the quickstart.sh command used | 15:13 |
| rlandy | zbr: w+1ed https://review.openstack.org/#/c/648902/ | 15:14 |
| zbr | rlandy: thanks. | 15:14 |
| jdennis | weshay: where would you like me to post it, it's too big for pastbin | 15:14 |
| *** ykarel is now known as ykarel|away | 15:18 | |
| zbr | was this expected? i do not see visible signs of the 10 retries at http://logs.openstack.org/26/645626/16/check/tripleo-ci-centos-7-containers-multinode-queens/1aca86a/job-output.txt.gz#_2019-04-02_12_10_13_879531 | 15:22 |
| weshay | jdennis I guess any where you can :) | 15:23 |
| *** ykarel|away has quit IRC | 15:23 | |
| chandankumar | sshnaidm|rover: zbr commented back on this review https://review.openstack.org/#/c/648678/8 | 15:36 |
| weshay | chandankumar arxcruz how we doing on the branching goal.. need anything from me? | 15:38 |
| chandankumar | weshay: branching is created for all tempest plugins | 15:39 |
| chandankumar | weshay: we have pinned most of the tempest plugins in rdoinfo | 15:39 |
| weshay | nice | 15:39 |
| chandankumar | weshay: just waiting for new taggs to come up then we are done with pinning also | 15:39 |
| weshay | chandankumar arxcruz rock on w/ your bad selves :) | 15:42 |
| *** ccamacho has joined #oooq | 15:46 | |
| *** yolanda has quit IRC | 15:46 | |
| *** yolanda has joined #oooq | 15:47 | |
| *** chem has quit IRC | 15:49 | |
| *** hamzy has quit IRC | 15:49 | |
| rfolco | marios_|ruck, rlandy sorry connection issues | 16:03 |
| rlandy | rfolco: marios_|ruck and I dropped | 16:04 |
| marios_|ruck | rfolco: ack we dropeed | 16:04 |
| rlandy | nobody else there | 16:04 |
| marios_|ruck | rfolco: i thought maybe i have the power to rant people off bluejeans thanks for spoiling it | 16:04 |
| rfolco | marios_|ruck, in my defense, we adjusted community mtg due to several conflicts, not my decision | 16:04 |
| marios_|ruck | rfolco: ack | 16:04 |
| rfolco | marios_|ruck, I'll add a retro card to vote for a time that everyone is happy with and no more confusion | 16:05 |
| marios_|ruck | rfolco: already have one | 16:05 |
| marios_|ruck | rfolco: ;) | 16:05 |
| marios_|ruck | rfolco: https://trello.com/c/mT1yMZnq/838-improve-our-community-call | 16:05 |
| rfolco | thanks marios_|ruck | 16:06 |
| rfolco | how your countdown | 16:07 |
| rfolco | is | 16:07 |
| marios_|ruck | https://www.timeanddate.com/countdown/generic?iso=20190403T18&p0=680&font=cursive&csz=1 | 16:07 |
| rfolco | :) | 16:07 |
| marios_|ruck | 22 52 41 | 16:07 |
| rfolco | almost there! be strong! | 16:07 |
| rfolco | brb lunch | 16:07 |
| marios_|ruck | :D | 16:07 |
| *** ykarel|away has joined #oooq | 16:13 | |
| *** yolanda has quit IRC | 16:19 | |
| *** bogdando has quit IRC | 16:20 | |
| * marios_|ruck out ttyl | 16:20 | |
| *** marios has quit IRC | 16:21 | |
| *** marios_|ruck has quit IRC | 16:21 | |
| *** dtantsur is now known as dtantsur|afk | 16:24 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (1 more message) | 16:31 |
| *** ccamacho has quit IRC | 16:32 | |
| *** trown is now known as trown|lunch | 16:50 | |
| rfolco | weshay, do you want me to create sprint 8 board or we need to chat first ? | 16:55 |
| rfolco | weshay, def set_expectation(sprint_board): | 16:55 |
| weshay | rfolco I have | 16:55 |
| weshay | start taking a look | 16:56 |
| rfolco | ok | 16:56 |
| *** gkadam has quit IRC | 17:02 | |
| rfolco | weshay, do you get free cycles to qe https://tree.taiga.io/project/tripleo-ci-board/task/877 or you need me to assign someone else? | 17:02 |
| *** hamzy has joined #oooq | 17:05 | |
| rfolco | weshay, we need https://review.openstack.org/#/c/647419/ merged to get stein work done... some of the cards (branchful jobs) depend on this emit_releases to work | 17:14 |
| rfolco | weshay, if a more formal doc is required we can work in a follow-up patch. | 17:15 |
| weshay | rfolco we just need to comb through it and code comment | 17:16 |
| rfolco | weshay, ok, will talk to quiquell|off tomorrow morning about it. Please leave your requirements in comments there. | 17:18 |
| rfolco | weshay, don't forget 877 depends-on emit_releases :) | 17:19 |
| weshay | rfolco I removed my -2, perhaps we make it a task in next sprint.. also verified stein branchful | 17:19 |
| weshay | ya | 17:19 |
| rfolco | ok cool | 17:19 |
| rfolco | sshnaidm|rover, if still around, https://review.openstack.org/#/c/647419/ | 17:26 |
| *** amoralej is now known as amoralej|off | 17:28 | |
| *** jpena is now known as jpena|off | 17:28 | |
| *** holser__ has quit IRC | 17:44 | |
| *** chem has joined #oooq | 17:50 | |
| *** trown|lunch is now known as trown | 17:55 | |
| *** kopecmartin is now known as kopecmartin|off | 18:04 | |
| *** rlandy is now known as rlandy|brb | 18:19 | |
| *** dtrainor has quit IRC | 18:29 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 @ (2 more messages) | 18:31 |
| *** rlandy|brb is now known as rlandy | 18:36 | |
| *** ykarel|away has quit IRC | 18:40 | |
| rlandy | hmmm ... still no stein jobs on http://38.145.34.55/ | 18:42 |
| rlandy | rfolco: ^^ | 18:42 |
| rfolco | rlandy, did stein jobs ran after quiquell|off patch ? | 18:42 |
| rfolco | run | 18:43 |
| rlandy | I think he kicked them | 18:43 |
| weshay | rfolco did you see my comments on stein promotion config | 18:43 |
| weshay | yup | 18:43 |
| rfolco | weshay, rlandy ok let me understand whats going on there | 18:44 |
| rfolco | weshay, quiquell|off patch addressed that... rlandy I am trying to understand why logs are empty | 18:47 |
| rfolco | https://review.rdoproject.org/r/#/c/19888/ | 18:47 |
| rfolco | weshay, ^ | 18:47 |
| rlandy | https://review.openstack.org/#/c/647419/ | 18:47 |
| rlandy | ^^ not merged yet | 18:47 |
| rlandy | rfolco: ^^ going to w+ that | 18:48 |
| rfolco | rlandy, thx | 18:48 |
| weshay | thanks! | 18:48 |
| rfolco | rlandy, but this is unrelated, right? | 18:49 |
| rlandy | wee need it anyways | 18:49 |
| rfolco | yeah, I am looking into logs issue now | 18:49 |
| *** jfrancoa has quit IRC | 18:50 | |
| *** chem has quit IRC | 18:51 | |
| *** chem has joined #oooq | 18:51 | |
| weshay | is gerrit slow for you guys? | 18:58 |
| rlandy | which gerrit? | 18:59 |
| *** dtrainor has joined #oooq | 18:59 | |
| *** dtrainor has quit IRC | 18:59 | |
| weshay | upstream | 18:59 |
| *** mjturek has quit IRC | 19:02 | |
| *** mjturek has joined #oooq | 19:03 | |
| sshnaidm|rover | rlandy, do you have problems with resolving internal boxes names? | 19:34 |
| rlandy | sshnaidm|rover: no - which boxes? | 19:38 |
| sshnaidm|rover | rlandy, see private | 19:38 |
| rlandy | sshnaidm|rover: that hardware is down this week | 19:39 |
| rlandy | due to the lab move | 19:39 |
| sshnaidm|rover | rlandy, so its dns records is wiped out? | 19:39 |
| rlandy | the name will change | 19:39 |
| rlandy | as will the ip | 19:39 |
| sshnaidm|rover | rlandy, I see, and how do I know the new name? | 19:40 |
| rlandy | atm those boxes are physically disconnected | 19:40 |
| rlandy | the name should be the same - just with rdu2 - afaik | 19:40 |
| rlandy | we only expect access back on Friday | 19:41 |
| sshnaidm|rover | rlandy, ack | 19:41 |
| rlandy | sshnaidm|rover: I will update the spreadsheet as soon as I get info from the lab move team | 19:41 |
| sshnaidm|rover | rlandy, great, thansk | 19:42 |
| *** chem has quit IRC | 19:51 | |
| rfolco | rlandy, how do I know if dlrn script is being called ? I see now dlrn promoter running for master.... with ps ax |grep promoter | 19:52 |
| rlandy | look in the stein periodic jobs | 19:53 |
| rlandy | are we reporting - step one | 19:53 |
| rlandy | then to see if the promoter is running, get on the server | 19:53 |
| rlandy | check the cron | 19:53 |
| rfolco | I am there | 19:54 |
| rlandy | is the stein section active? | 19:54 |
| rlandy | any errors you can see? | 19:54 |
| *** chem has joined #oooq | 19:54 | |
| rlandy | must be a reason for no logs | 19:54 |
| *** holser_ has joined #oooq | 20:01 | |
| rfolco | rlandy, is promoter sh script killed and reloaded with new version ? it runs in a loop with sleeps looks like. | 20:05 |
| *** Vorrtex has quit IRC | 20:05 | |
| rlandy | correct | 20:05 |
| rfolco | rlandy, we need to manually kill it and reload ? | 20:06 |
| rlandy | sec | 20:06 |
| rlandy | submitting change | 20:06 |
| rfolco | k np | 20:06 |
| rlandy | you can run manual | 20:07 |
| rlandy | just the stein part | 20:07 |
| rlandy | anf see what happens | 20:07 |
| *** holser_ has quit IRC | 20:08 | |
| rfolco | k | 20:08 |
| rfolco | but I think the script is in memory did not get updated with the current contents of /home/centos/ci-config/ci-scripts/dlrnapi_promoter/dlrn-promoter.sh | 20:09 |
| rfolco | so thats why it does not run | 20:09 |
| rfolco | rlandy, makes sense ? | 20:09 |
| rlandy | possibly - kill and restart | 20:09 |
| *** mjturek has quit IRC | 20:11 | |
| *** holser_ has joined #oooq | 20:16 | |
| rfolco | rlandy, ok, running manually - http://38.145.34.55/centos7_stein.log | 20:17 |
| rfolco | weshay, fyi ^ | 20:17 |
| *** holser_ has quit IRC | 20:18 | |
| rfolco | we need to improve this bash promoter very badly | 20:18 |
| weshay | it's not bash | 20:18 |
| rfolco | having to kill the current process isn't right | 20:18 |
| weshay | rfolco isn't it a system service? | 20:19 |
| rfolco | the bash that call all the python ones | 20:19 |
| weshay | rfolco ya.. but that's not what runs I think | 20:19 |
| * weshay double checks | 20:19 | |
| rfolco | ps ax |grep promoter | 20:19 |
| rfolco | this is not a cron | 20:19 |
| rfolco | its a bash in loop sleeping for 10 min | 20:20 |
| rfolco | weshay, at least this is what is running now there. | 20:20 |
| weshay | [Service] | 20:20 |
| weshay | ExecStart=/home/centos/ci-config/ci-scripts/dlrnapi_promoter/dlrn-promoter.sh | 20:20 |
| weshay | User=centos | 20:20 |
| weshay | Restart=always | 20:20 |
| weshay | [centos@promoter ~]$ | 20:20 |
| weshay | ah it does kick the bash file | 20:20 |
| weshay | but it's an installed service | 20:20 |
| rfolco | right so need to restart service | 20:20 |
| rfolco | ok | 20:20 |
| rfolco | my bad | 20:20 |
| rfolco | just restart the service | 20:20 |
| rfolco | but anyways, when we have an updated version, we need to restart service automatically | 20:21 |
| rfolco | weshay, agree? | 20:21 |
| weshay | isn't that just a systemd option | 20:21 |
| weshay | ? | 20:21 |
| rlandy | cool - checking | 20:21 |
| weshay | https://singlebrook.com/2017/10/23/auto-restart-crashed-service-systemd/ | 20:22 |
| *** mjturek has joined #oooq | 20:22 | |
| weshay | rfolco heh. it is set to restart=always | 20:23 |
| weshay | not sure if that's working | 20:23 |
| rfolco | weshay, restart when file is updated? | 20:23 |
| rfolco | weshay, nothing crashed or failed | 20:23 |
| weshay | https://www.freedesktop.org/software/systemd/man/systemd.service.html | 20:24 |
| weshay | rfolco the config is updated every time the shell script is launched | 20:24 |
| weshay | we do put it to sleep at the end https://www.freedesktop.org/software/systemd/man/systemd.service.html | 20:26 |
| weshay | oops | 20:26 |
| weshay | # Sleep 10 minutes | 20:26 |
| weshay | sleep 600 | 20:26 |
| weshay | I don't know what you found.. but everything you are saying.. I think is done | 20:26 |
| rfolco | the config is updated, but the service is not reloaded | 20:27 |
| rfolco | I don't see where this is happening | 20:27 |
| weshay | rfolco are you centos user? | 20:27 |
| rfolco | yes | 20:27 |
| weshay | rfolco tmux -a | 20:27 |
| weshay | rfolco sorry.. tmux a | 20:29 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 @ (2 more messages) | 20:31 |
| rfolco | weshay, reload not restart | 20:34 |
| weshay | or reload | 20:34 |
| weshay | ya | 20:34 |
| weshay | that's a good add | 20:34 |
| weshay | rfolco put up a review and we'll merge it | 20:34 |
| rfolco | weshay, not sure we can reload itself | 20:34 |
| rfolco | ok | 20:34 |
| rfolco | weshay, nice | 20:34 |
| rfolco | weshay, thanks for your patience :) | 20:34 |
| weshay | np.. it's how you get to the root of things :) | 20:35 |
| rfolco | c00l | 20:35 |
| *** mjturek has quit IRC | 20:37 | |
| weshay | sshnaidm++ | 20:40 |
| weshay | sshnaidm|rover the repro is working for me.. got passed standalone deploy | 20:41 |
| weshay | quiquell++ | 20:41 |
| *** hamzy has quit IRC | 20:41 | |
| rlandy | rfolco: ok - moving card to done | 20:46 |
| rlandy | http://38.145.34.55/fedora28_stein.log | 20:47 |
| rlandy | ^^ failing | 20:47 |
| rlandy | 2019-04-02 20:15:11,015 19661 ERROR promoter u'current-tripleo-rdo' | 20:48 |
| rlandy | FAIL", "ERROR ========== fedora-binary-rsyslog-base IS NOT BUILT! FIX THIS ASAP! ==========", "FAIL", "ERROR =========== container list is empty THIS IS A CRITICAL BUG in the container push playbook"]} | 20:48 |
| rlandy | rfolco: ^^ known? | 20:48 |
| weshay | rlandy ah.. so something new has been added to tripleo-common | 20:51 |
| weshay | and it's not built yet | 20:51 |
| weshay | let's see | 20:51 |
| rlandy | weshay: ok - so I am moving the promoter card to done | 20:51 |
| rlandy | as the basic promotion stuff is working | 20:51 |
| *** chem has quit IRC | 20:52 | |
| *** chem has joined #oooq | 20:52 | |
| weshay | rlandy rfolco https://github.com/openstack/tripleo-common/blame/master/container-images/overcloud_containers.yaml.j2#L904 | 20:53 |
| weshay | hrm.. suspect | 20:53 |
| weshay | we're missing a container | 20:53 |
| * rlandy looks more | 20:54 | |
| weshay | what happened here? https://console.registry.rdoproject.org/registry#/images/tripleostein | 20:55 |
| *** aakarsh has quit IRC | 20:55 | |
| weshay | https://hub.docker.com/r/tripleostein/fedora-binary-rsyslog-base | 20:55 |
| rlandy | https://hub.docker.com/r/tripleostein/fedora-binary-rsyslog-base/tags | 21:00 |
| rlandy | current-tripleo | 21:00 |
| weshay | rlandy ya.. so if the rdo registry is empty | 21:00 |
| weshay | we'll get this problme | 21:01 |
| weshay | problem | 21:01 |
| weshay | did someone clean out the rdo registry of f28 stein? | 21:01 |
| weshay | that is the issue afaict | 21:01 |
| weshay | rfolco ^ ? | 21:01 |
| *** dtrainor has joined #oooq | 21:02 | |
| rlandy | somebody from infra? | 21:03 |
| weshay | maybe | 21:03 |
| weshay | rlandy I don't think we need to though | 21:04 |
| weshay | rlandy we should check that the f28 stein container build job is working though | 21:04 |
| weshay | it will update that for us | 21:04 |
| *** trown is now known as trown|outtypewww | 21:04 | |
| rlandy | looking | 21:04 |
| weshay | rlandy https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic&job_name=periodic-tripleo-fedora-28-stein-containers-build-push | 21:05 |
| weshay | why is that happening | 21:05 |
| weshay | there must be something borked w/ the fedora image or the zuul config | 21:06 |
| rlandy | tripleo_repos.main.InvalidArguments: Branches only suppported with centos7 | 21:07 |
| rlandy | raise InvalidArguments('Branches only suppported with centos7') | 21:07 |
| rlandy | https://github.com/openstack-infra/tripleo-ci/commit/8a60923e841c080ba03f86eb8b80179b788c7b6e was supposed to fix release | 21:11 |
| rlandy | distro broken | 21:11 |
| weshay | rlandy ah I see | 21:12 |
| * weshay opens a bug | 21:12 | |
| weshay | https://bugs.launchpad.net/tripleo/+bug/1822888 | 21:17 |
| openstack | Launchpad bug 1822888 in tripleo "f28 container build broken by tripleo-repos InvalidArguments('Branches only suppported with centos7')" [Critical,Triaged] | 21:17 |
| *** derekh has quit IRC | 21:26 | |
| rlandy | weshay: wrt rhos-14 ... diff is here https://github.com/openstack/tripleo-quickstart-extras/blob/cac8555a17cc9d48bd31a163222d62552c4615f7/roles/standalone/tasks/main.yml#L38 | 21:38 |
| rlandy | note that only a few parameters are updated | 21:38 |
| rlandy | as opposed to the overcloud deploy (not standalone) where you can set multiple diff parameters | 21:39 |
| weshay | rlandy right.. for standalone | 21:40 |
| weshay | this is what we were looking at a few weeks agao | 21:40 |
| weshay | you get a template, then have to update it post creation | 21:40 |
| weshay | which is why there where all the backup files | 21:40 |
| weshay | the registry was updated though in the job you sent me | 21:40 |
| rlandy | weshay: so ... we would need something like https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/overcloud-prep-containers/templates/overcloud-prep-containers.sh.j2#L34 | 21:40 |
| rlandy | the registry update is fine | 21:41 |
| rlandy | it's the ceph stuff | 21:41 |
| weshay | ah | 21:41 |
| weshay | how did you verify it was ceph | 21:41 |
| weshay | rlandy we can additional routines to update parts of the template | 21:41 |
| weshay | perhaps.. if ceph registry is not docker.io | 21:42 |
| weshay | we update it all at once | 21:42 |
| * rlandy looks into doing that | 21:42 | |
| rlandy | adding a ceph piece | 21:42 |
| rlandy | we should be able to pass standalone_container_prep_options | 21:44 |
| rlandy | but unsure of syntax | 21:44 |
| rlandy | didn't like my first attempt :( | 21:45 |
| rlandy | and I dont see where that is actually picked up | 21:45 |
| *** chem has quit IRC | 21:54 | |
| *** chem has joined #oooq | 21:54 | |
| weshay | rlandy I have to take my kids to hebrew school.. I'll help tomorrow | 21:54 |
| rlandy | weshay: np - just experimenting | 21:55 |
| *** vinaykns has quit IRC | 22:26 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 @ (2 more messages) | 22:31 |
| *** holser_ has joined #oooq | 22:32 | |
| *** holser_ has quit IRC | 22:49 | |
| *** rlandy has quit IRC | 22:49 | |
| *** chem has quit IRC | 22:54 | |
| *** chem has joined #oooq | 22:55 | |
| *** aakarsh has joined #oooq | 23:21 | |
| *** tosky has quit IRC | 23:24 | |
| *** vinaykns has joined #oooq | 23:37 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!