*** tosky has quit IRC | 00:26 | |
*** ysandeep has joined #oooq | 01:24 | |
*** apetrich has quit IRC | 03:08 | |
*** ysandeep has quit IRC | 03:22 | |
*** ysandeep has joined #oooq | 03:34 | |
*** bhagyashris has joined #oooq | 04:04 | |
*** skramaja has joined #oooq | 04:15 | |
*** ykarel has joined #oooq | 04:20 | |
*** udesale has joined #oooq | 04:37 | |
*** surpatil has joined #oooq | 05:12 | |
*** soniya29 has joined #oooq | 05:17 | |
*** epoojad1 has joined #oooq | 05:22 | |
*** raukadah is now known as chkumar|rover | 05:54 | |
*** jtomasek has joined #oooq | 06:26 | |
*** marios has joined #oooq | 06:45 | |
*** marios is now known as marios|rover | 06:51 | |
*** chkumar|rover is now known as chkumar|ruck | 06:51 | |
chkumar|ruck | marios|rover: Good morning | 06:52 |
---|---|---|
chkumar|ruck | marios|rover: needs +2 and +w on this https://review.opendev.org/#/c/696811/ | 06:52 |
*** jfrancoa has joined #oooq | 06:52 | |
chkumar|ruck | thanks! when free | 06:52 |
marios|rover | o/ morning and happy monday \o/ :/ | 06:53 |
marios|rover | chkumar|ruck: sure will do in bit | 06:53 |
chkumar|ruck | zbr_: Hello, I have fixed all the podman related issues, | 06:54 |
*** ksambor has joined #oooq | 07:09 | |
*** skramaja_ has joined #oooq | 07:13 | |
*** skramaja has quit IRC | 07:14 | |
*** surpatil has quit IRC | 07:29 | |
chkumar|ruck | dtantsur|afk: marios|rover Anything we can do https://review.opendev.org/#/c/696532/ to this review merged? | 07:32 |
*** akahat has joined #oooq | 07:33 | |
*** surpatil has joined #oooq | 07:40 | |
marios|rover | chkumar|ruck: i don't have core there but just replied to harald and dmitry | 07:40 |
chkumar|ruck | marios|rover: thanks :-) | 07:41 |
*** ykarel is now known as ykarel|lunch | 07:43 | |
*** jbadiapa has joined #oooq | 07:47 | |
*** apetrich has joined #oooq | 07:52 | |
*** matbu has joined #oooq | 08:09 | |
*** tesseract has joined #oooq | 08:14 | |
arxcruz | jpena|off: ping when you're available :) | 08:15 |
*** ykarel|lunch is now known as ykarel | 08:20 | |
*** epoojad1 has quit IRC | 08:22 | |
*** amoralej|off is now known as amoralej | 08:29 | |
apetrich | arxcruz, almost had a heart attack this morning. Went to get my bike at the garage and it wasn't there. It slept locked to the fences in front of my building. I totally forgot to bring it in | 08:30 |
arxcruz | apetrich: you're playing with fire! | 08:31 |
apetrich | arxcruz, because I'm stupid | 08:31 |
arxcruz | apetrich: btw, the insurance cover mine, and give me 50% of the money back IF i buy a new one | 08:31 |
*** tosky has joined #oooq | 08:31 | |
apetrich | I didn't want to play with fire | 08:31 |
apetrich | Nice! | 08:32 |
arxcruz | I got the same model as before, but I'll keep in my balcony with some bike cover | 08:32 |
apetrich | crap | 08:33 |
apetrich | good but crap | 08:33 |
apetrich | btw what chair do you have? I have a crappy one but I bought new rollers for it and I'm amazed by it | 08:33 |
arxcruz | apetrich: well, since according german laws, the building has nothing to do with robbery inside the building... | 08:33 |
*** ccamacho has joined #oooq | 08:34 | |
arxcruz | apetrich: i got one, not the one i wanted but a good one | 08:34 |
arxcruz | let me check the model | 08:34 |
apetrich | Sure. why would it? Pff german laws | 08:34 |
arxcruz | https://www.amazon.de/gp/product/B079BSRV7P/ref=ppx_yo_dt_b_asin_title_o08_s00?ie=UTF8&psc=1 | 08:35 |
arxcruz | apetrich: after a very long research, this was the best for the price | 08:35 |
arxcruz | it's fully adjustable | 08:35 |
arxcruz | armrests goes up and down, front and back | 08:35 |
apetrich | Nice | 08:36 |
arxcruz | and I like the mesh fabric because I swet a lot | 08:36 |
apetrich | So do I | 08:36 |
arxcruz | apetrich: btw, confirmed on dec 8 | 08:37 |
arxcruz | I forgot to tell you lol | 08:37 |
apetrich | arxcruz, got these https://www.amazon.de/dp/B07DXVSJWH | 08:37 |
arxcruz | what liam likes as toys? | 08:37 |
apetrich | arxcruz, on nice! I was about to ask you about that | 08:38 |
arxcruz | cool, i also got one floor protection from ikea | 08:38 |
arxcruz | pretty good | 08:38 |
apetrich | arxcruz, I think a safe bet is a small playmobil set. We are starting to build a collection. matchbox cars are also a good bet, specially the ones with firetrucks or something very different | 08:40 |
apetrich | what else? | 08:40 |
arxcruz | nespresso | 08:40 |
arxcruz | playmobil it will be | 08:40 |
apetrich | :) | 08:40 |
arxcruz | when i was kid, i had a lot of playmobil | 08:41 |
arxcruz | then i discover lego | 08:41 |
arxcruz | then i discover lego is too expensive | 08:41 |
apetrich | Me too. but lego was way way expensive | 08:41 |
apetrich | lol | 08:41 |
apetrich | yeah | 08:41 |
apetrich | that's probably the reason why he has too much lego :) | 08:42 |
apetrich | I never did | 08:42 |
arxcruz | apetrich: same with me | 08:43 |
arxcruz | unfortunately valentina likes more dolls than lego | 08:44 |
arxcruz | although when I buy some for me, she likes to help me to mount | 08:44 |
*** jpena|off is now known as jpena | 08:48 | |
jpena | arxcruz: I'm in. How can I help you? | 08:49 |
arxcruz | jpena: https://github.com/ceph/ceph-ansible/pull/4791 | 08:49 |
arxcruz | if i do a check-rdo, the master and train are triggered | 08:49 |
arxcruz | but the tripleo job not | 08:50 |
jpena | aha, let me check the job definition | 08:50 |
jpena | the master job is expected, since we did not specify any branch restriction iirc | 08:50 |
arxcruz | jpena: https://review.rdoproject.org/r/#/c/23853/ and https://review.rdoproject.org/r/#/c/23852/ got merged | 08:50 |
arxcruz | jpena: i think for the master, i should ad branches stable-5 | 08:51 |
*** dtantsur|afk is now known as dtantsur | 08:52 | |
jpena | arxcruz: in https://review.rdoproject.org/r/#/c/23852/13/zuul.d/projects.yaml, why are you specifying "branches: master" | 08:52 |
jpena | ? | 08:52 |
arxcruz | jpena: that was indeed a mistake | 08:52 |
arxcruz | lol | 08:52 |
*** chem has joined #oooq | 08:53 | |
arxcruz | jpena: remove or specify train ? | 08:53 |
jpena | you're already specifying branches: ^stable-4.0$ in https://review.rdoproject.org/r/#/c/23852/13/zuul.d/ceph-ansible.yaml, I think you can just remove those lines | 08:54 |
arxcruz | jpena: anything else ? | 08:55 |
jpena | arxcruz: I think that should be it | 08:55 |
*** Tengu has quit IRC | 09:01 | |
*** Tengu has joined #oooq | 09:02 | |
*** SurajPatil has joined #oooq | 09:07 | |
arxcruz | jpena: didn't work, still the master is getting triggered and the tripleo job isn't | 09:07 |
chkumar|ruck | marios|rover: Do we want to close this https://bugs.launchpad.net/tripleo/+bug/1853856 and https://bugs.launchpad.net/tripleo/+bug/1854517 ? | 09:08 |
openstack | Launchpad bug 1853856 in tripleo "Tempest basic ops test failed with Timed out waiting for 192.168.24.107 to become reachable in ovs deployment" [Critical,Triaged] | 09:08 |
openstack | Launchpad bug 1854517 in tripleo "RDO jobs failing with POST_FAILURE" [Critical,Invalid] | 09:08 |
arxcruz | jpena: can you check the logs ? | 09:09 |
jpena | arxcruz: sure, what's the review? | 09:09 |
jpena | not the PR, I mean the change in review.rdo | 09:09 |
arxcruz | jpena: https://review.rdoproject.org/r/#/c/23896/ | 09:09 |
*** surpatil has quit IRC | 09:09 | |
marios|rover | chkumar|ruck: ack am going though the list and updating i already moved 517 to invalid a little while ago | 09:10 |
chkumar|ruck | ack thanks! | 09:12 |
*** epoojad1 has joined #oooq | 09:13 | |
*** bogdando has joined #oooq | 09:14 | |
*** matbu has quit IRC | 09:20 | |
*** matbu has joined #oooq | 09:20 | |
jpena | arxcruz: I found the issue. The new jobs have tripleo-ci-rhel-8-standalone-rdo as their parent, and that specifies master as the only branch, see https://github.com/rdo-infra/rdo-jobs/blob/ec62458a975937b173924ed3a85894958a4774bf/zuul.d/standalone-jobs.yaml#L490 | 09:26 |
jpena | we had a similar issue in the past, remember? We need to remove that from the parent job, and let child jobs limit the branches | 09:26 |
arxcruz | jpena: yeah... okay... | 09:26 |
arxcruz | let's do it | 09:27 |
arxcruz | jpena: what about the master job running ? | 09:27 |
jpena | arxcruz: you're not limiting branches in the definition of tripleo-ceph-integration-master, so it would run on every PR | 09:28 |
jpena | https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/ceph-ansible.yaml#L25 | 09:28 |
arxcruz | jpena: so branches: ^master$ fix the problem | 09:28 |
jpena | yes | 09:28 |
arxcruz | jpena: actually, stable-5.0 | 09:29 |
chkumar|ruck | marios|rover: I think we have some real issue on fs01 | 09:29 |
chkumar|ruck | it is coming on rdocloud and vexxhost | 09:29 |
chkumar|ruck | no valid host found during overcloud deploy | 09:29 |
chkumar|ruck | filing the bug soon | 09:31 |
arxcruz | jpena: ^(stable-5.0|master).*$ right? | 09:31 |
jpena | lgtm | 09:31 |
marios|rover | chkumar|ruck: so | 09:31 |
marios|rover | chkumar|ruck: i've seen that a few times it is intermittent | 09:31 |
marios|rover | * http://logs.rdoproject.org/05/695805/6/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/eb60e5d/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 09:32 |
marios|rover | chkumar|ruck: today there ^ | 09:32 |
chkumar|ruck | marios|rover: yes | 09:32 |
marios|rover | chkumar|ruck: but hadn't file a bug for it makes sense to do it | 09:32 |
chkumar|ruck | marios|rover: but seen the same on vexxhost | 09:32 |
chkumar|ruck | marios|rover: http://logs.rdoproject.org/80/23880/3/check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-train-vexxhost-nhw/5ddd3bc/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 09:32 |
marios|rover | chkumar|ruck: but maybe 1/2 day or less sometimes | 09:32 |
marios|rover | chkumar|ruck: ack | 09:32 |
chkumar|ruck | marios|rover: need to check train fs01 also | 09:32 |
*** derekh has joined #oooq | 09:35 | |
ykarel | chkumar|ruck, marios|rover is fs020-stein failure known? https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-stein ? | 09:44 |
ykarel | looks like other jobs in stein also have issues from last couple of days | 09:51 |
ykarel | ex. 027 | 09:52 |
arxcruz | jpena: didn't work | 09:53 |
ykarel | hmm 027 should be fixed with https://review.opendev.org/#/c/696811/ | 09:53 |
arxcruz | jpena: the master worked... but the tripleo job isn't running though | 09:53 |
marios|rover | ykarel: is on my todo list to check the stein promotion as it is few days old now there might be some issue but not aware of bug yet | 09:57 |
jpena | arxcruz: ok, this time the error is different -> https://softwarefactory-project.io/paste/show/1675/ | 10:01 |
jpena | we're configuring the job to only trigger when certain files are changed (see https://review.rdoproject.org/r/#/c/23896/3/zuul.d/projects.yaml), and none of them match | 10:02 |
arxcruz | jesus... okay, removing for this job | 10:03 |
arxcruz | jpena: okay, what about now ? what's the error? | 10:05 |
arxcruz | is there a way i can see the zuul logs, so i won't keep bothering you? | 10:06 |
marios|rover | chkumar|ruck: we have a bug for train image build? 2019-12-01 21:24:03.148 | > error: Status code: 403 for https://rhui-cds/pulp/repos/content/dist/rhel8/rhui/8/x86_64/codeready-builder/os/repodata/repomd.xml (https://rhui-cds/pulp/repos/content/dist/rhel8/rhui/8/x86_64/codeready-builder/os/repodata/repomd.xml). | 10:09 |
marios|rover | chkumar|ruck: http://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-rhel-8-buildimage-overcloud-full-train/e342ee2/build.log | 10:09 |
chkumar|ruck | marios|rover: yes | 10:09 |
chkumar|ruck | marios|rover: https://bugs.launchpad.net/tripleo/+bug/1854685 | 10:09 |
marios|rover | chkumar|ruck: cool which one please thanks | 10:09 |
marios|rover | thx | 10:09 |
openstack | Launchpad bug 1854685 in tripleo "RHEL-8 master Image build job failed with Failed to synchronize cache for repo 'rhui-codeready-builder-for-rhel-8-x86_64-rhui-rpms'" [Critical,Confirmed] | 10:09 |
chkumar|ruck | marios|rover: add train also | 10:09 |
marios|rover | chkumar|ruck: yah its from train i saw it is on etherpad? | 10:09 |
chkumar|ruck | marios|rover: apevec, jpena and me looking into that | 10:09 |
marios|rover | chkumar|ruck: thanks | 10:10 |
chkumar|ruck | marios|rover: stein fs027 tempest issue got fixed | 10:10 |
marios|rover | rhui issue looks like | 10:10 |
chkumar|ruck | marios|rover: yes it is | 10:10 |
marios|rover | chkumar|ruck: ykarel: ^^ | 10:10 |
marios|rover | ykarel: 12:10 < chkumar|ruck> marios|rover: stein fs027 tempest issue got fixed | 10:10 |
chkumar|ruck | ah it is fs020 one | 10:10 |
bogdando | hi. PTAL https://review.opendev.org/#/c/696306/ https://review.opendev.org/#/c/696292/ https://review.opendev.org/#/c/696091/ | 10:11 |
jpena | arxcruz: you need ssh access into the zuul infra to check the logs (they can contain passwords and other sensitive info). Let me check what's going on now | 10:11 |
bogdando | and https://review.opendev.org/#/c/696056/ | 10:11 |
ykarel | marios|rover, hmm i got that <ykarel> hmm 027 should be fixed with https://review.opendev.org/#/c/696811/ | 10:11 |
*** skramaja_ is now known as skramaja | 10:12 | |
*** ysandeep has quit IRC | 10:12 | |
*** ysandeep_ has joined #oooq | 10:12 | |
chkumar|ruck | brb | 10:16 |
*** ysandeep_ has quit IRC | 10:20 | |
*** ksambor has quit IRC | 10:21 | |
*** ksambor has joined #oooq | 10:25 | |
*** tosky_ has joined #oooq | 10:26 | |
*** tosky has quit IRC | 10:28 | |
*** gchamoul has joined #oooq | 10:30 | |
jpena | arxcruz: hm, this is strange. It is still complaining about the files (https://softwarefactory-project.io/paste/show/1676/), but I can't understand why | 10:35 |
arxcruz | jpena: maybe branches: ^(stable(/train|-4.0)).*$ ? | 10:36 |
arxcruz | jpena: or maybe remove the job from the projects.yaml file ? | 10:36 |
* jpena facepalms | 10:37 | |
arxcruz | :( | 10:38 |
jpena | arxcruz: we're defining the job for project rdo-jobs | 10:38 |
jpena | not for ceph-ansible | 10:38 |
*** tosky_ is now known as tosky | 10:38 | |
arxcruz | jpena: so i can remove those right ? | 10:38 |
arxcruz | but the check-rdo should work... | 10:38 |
arxcruz | why is not? | 10:39 |
arxcruz | jpena: https://review.rdoproject.org/r/#/c/23853/4/zuul.d/projects.yaml get merged, and these jobs should run | 10:39 |
arxcruz | correct? | 10:39 |
jpena | I think it should, yes | 10:40 |
arxcruz | then why isn't ? | 10:41 |
jpena | that's what I'm trying to understand | 10:41 |
bogdando | as there is dead silence at #zuul, I'all ask here as well: | 10:45 |
bogdando | 11:35:15 - bogdando: is there a way to live test by local zuul itself a change in zuul-jubs? e.g. https://review.opendev.org/#/c/696337/ | 10:45 |
bogdando | 11:35:35 - bogdando: This change looks never applied by zuul executor running my CI job ? | 10:45 |
bogdando | perchance anyone tried? | 10:45 |
chkumar|ruck | marios|rover: https://bugs.launchpad.net/tripleo/+bug/1854721 | 10:46 |
openstack | Launchpad bug 1854721 in tripleo "FS 01 master periodic job failed during overcloud deploy with Exceeded maximum number of retries and 503 status code" [Critical,Confirmed] | 10:46 |
chkumar|ruck | marios|rover: other vexxhost jobs hitted with same errro https://review.rdoproject.org/r/#/c/23880/ | 10:49 |
jpena | arxcruz: could it be that one of the parent jobs in the chain does not trigger when we are changing the README.rst file? | 10:50 |
arxcruz | jpena: maybe, let me try to change something else | 10:50 |
jpena | arxcruz: yes, that's it | 10:51 |
jpena | https://opendev.org/openstack/tripleo-ci/src/branch/master/zuul.d/base.yaml#L53 | 10:51 |
jpena | the tripleo-ci-base file is in the parent chain, and it ignores any changes to .rst files in the root dir | 10:51 |
jpena | arxcruz: aha, now we're getting a different output. We need to make the master integration jobs run only on master (just like the rpm building job) | 10:53 |
arxcruz | jpena: now i got this on github side: | 10:53 |
arxcruz | Unable to freeze job graph: Job tripleo-ceph-integration-rhel-8-scenario001-standalone depends on tripleo-ceph-integration-master which was not run. | 10:53 |
jpena | yes, that's it | 10:53 |
arxcruz | oh, yeah | 10:53 |
arxcruz | :) | 10:53 |
arxcruz | YES | 10:55 |
arxcruz | jpena: it worked!@ | 10:55 |
arxcruz | jpena++ | 10:55 |
jpena | finally :) | 10:55 |
*** epoojad1 is now known as epoojad1|afk | 10:56 | |
arxcruz | let's see if it will work :D | 10:58 |
chkumar|ruck | marios|rover: Now poking at fs020 stein failure | 10:59 |
arxcruz | chkumar|ruck: rhel-8 are suppose to support train right ? | 11:07 |
chkumar|ruck | arxcruz: yes | 11:07 |
chkumar|ruck | arxcruz: rhel-8 train and master | 11:08 |
arxcruz | chkumar|ruck: any chance you know why this image isn't found? http://logs.rdoproject.org/42/23642/7/check/tripleo-ceph-integration-rhel-8-scenario001-standalone/c87b7a6/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz | 11:08 |
arxcruz | is it because it's docker:// | 11:08 |
chkumar|ruck | arxcruz: we are waiting for the promotion | 11:08 |
chkumar|ruck | there is no docker images | 11:08 |
chkumar|ruck | can you try with vars: force_periodic: true in the job currently | 11:08 |
arxcruz | chkumar|ruck: i see... thanks | 11:09 |
chkumar|ruck | arxcruz: https://bugs.launchpad.net/tripleo/+bug/1854062 | 11:09 |
openstack | Launchpad bug 1854062 in tripleo "rhel8 jobs failing for missing containers " Exception: Not found image: docker://..."" [Critical,Triaged] | 11:09 |
arxcruz | chkumar|ruck: cool, so, it's a real failure :) | 11:11 |
marios|rover | chkumar|ruck: ack thanks | 11:12 |
marios|rover | arxcruz: last rhel promotion was bad so check jobs broken | 11:13 |
marios|rover | matbu: ccamacho: jfrancoa: o/ hello can you please check comment #28 @ https://bugs.launchpad.net/tripleo/+bug/1853812 can you add more info if you have it are we at green job there? where? thanks :D | 11:29 |
openstack | Launchpad bug 1853812 in tripleo "train containerized-undercloud-upgrades failing with "error creating container the container name is already in use "" [Critical,Fix released] - Assigned to Bogdan Dobrelya (bogdando) | 11:29 |
*** surpatil has joined #oooq | 11:29 | |
*** SurajPatil has quit IRC | 11:31 | |
bogdando | marios|rover: hi, https://review.opendev.org/#/c/695242/ has tripleo-ci-centos-7-containerized-undercloud-upgrades green, so we need all its deps merged | 11:32 |
bogdando | updated the bug, ccamacho | 11:32 |
marios|rover | thanks bogdando exactly what i was looking for but why doesn't it have the bug on it | 11:33 |
marios|rover | ccamacho: please add the bug there i left a comment very soft -1 but given the amount of depends on i think you'll likely update at least once :D | 11:34 |
soniya29 | Hi everyone, I want to export the cockpit data to json. While doing it, I am getting https://paste.fedoraproject.org/paste/w5qHOtzTRMy~Bwit-ejjzw. How can I export it properly? | 11:36 |
ccamacho | marios|rover Ill update the patch | 11:36 |
ccamacho | done ^ | 11:38 |
marios|rover | ccamacho: thanks revoted | 11:40 |
soniya29 | marios|rover, sshnaidm, arxcruz, kopecmartin, Can anyone of you help me over this:-https://paste.fedoraproject.org/paste/w5qHOtzTRMy~Bwit-ejjzw? | 11:47 |
marios|rover | soniya29: hmm i recall hitting that a few times did you maybe remove your key? alternatively try removing and clearing that but ask sshnaidm he will likely give better info | 11:49 |
marios|rover | soniya29: "removing and clearing " i mean re-generate but i vaguely recall sshnaidm saying not to do that :D | 11:50 |
sshnaidm | soniya29, did you generate a key with create-api-key.py ? | 11:51 |
soniya29 | marios|rover, I never removed my key actually. Before trying to export the data with ./export-grafana.py, I have created a key using ./create-api-key.py | 11:52 |
soniya29 | sshnaidm, yes | 11:52 |
sshnaidm | soniya29, is it saved in grafana.key file? | 11:52 |
sshnaidm | soniya29, https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/roles/rrcockpit/files/grafana/export-grafana.py#L20:L20 | 11:54 |
soniya29 | sshnaidm: Nope. It says, failed to add API key | 11:54 |
*** dtantsur is now known as dtantsur|afk | 11:54 | |
sshnaidm | soniya29, either pass it as argument or save it in file | 11:54 |
sshnaidm | soniya29, if you don't have it, need to generate a new one | 11:55 |
soniya29 | sshnaidm, I tried to generate, but as I said, It failed to save it. I will once again try to re-generate | 11:57 |
sshnaidm | soniya29, try new "--key-name" when generating | 11:58 |
soniya29 | sshnaidm, yeah | 11:58 |
soniya29 | sshnaidm, It worked. Thanks a lot | 11:59 |
sshnaidm | np | 12:00 |
*** holser has joined #oooq | 12:01 | |
*** rfolco has joined #oooq | 12:09 | |
chkumar|ruck | I need help in debugging overcloud image build failure | 12:13 |
chkumar|ruck | https://bugs.launchpad.net/tripleo/+bug/1854685 | 12:14 |
openstack | Launchpad bug 1854685 in tripleo "RHEL-8 master Image build job failed with Failed to synchronize cache for repo 'rhui-codeready-builder-for-rhel-8-x86_64-rhui-rpms'" [Critical,Confirmed] | 12:14 |
chkumar|ruck | we have the node hold | 12:14 |
soniya29 | weshay, arxcruz, I have updated the patch along with skiplist_dashboard.json file. Can you review the patch? | 12:16 |
*** jpena is now known as jpena|lunch | 12:26 | |
*** udesale has quit IRC | 12:36 | |
*** udesale has joined #oooq | 12:37 | |
*** SurajPatil has joined #oooq | 12:46 | |
*** surpatil has quit IRC | 12:48 | |
*** soniya29 has quit IRC | 12:52 | |
*** rlandy has joined #oooq | 12:57 | |
rfolco | scrum time | 13:00 |
rfolco | sshnaidm, zbr_ | 13:01 |
*** amoralej is now known as amoralej|lunch | 13:05 | |
*** ccamacho has quit IRC | 13:12 | |
*** ccamacho has joined #oooq | 13:12 | |
*** ccamacho has quit IRC | 13:13 | |
*** ccamacho has joined #oooq | 13:13 | |
*** epoojad1|afk is now known as epoojad1 | 13:15 | |
sshnaidm | marios|rover, ping me after a mtg, I suspect you hit same problems we hit for rhel8 containers | 13:17 |
*** jpena|lunch is now known as jpena | 13:24 | |
*** skramaja has quit IRC | 13:24 | |
*** surpatil has joined #oooq | 13:26 | |
*** SurajPatil has quit IRC | 13:28 | |
*** SurajPatil has joined #oooq | 13:30 | |
marios|rover | chkumar|ruck: you on the other call? (reminder() | 13:30 |
*** surpatil has quit IRC | 13:33 | |
*** soniya29 has joined #oooq | 13:45 | |
weshay | https://review.rdoproject.org/r/#/c/23880/ | 13:49 |
weshay | https://bugs.launchpad.net/tripleo/+bug/1854721 | 13:49 |
openstack | Launchpad bug 1854721 in tripleo "FS 01 master periodic job failed during overcloud deploy with Exceeded maximum number of retries and 503 status code" [Critical,Triaged] | 13:49 |
weshay | https://review.rdoproject.org/r/22870 | 13:52 |
*** soniya29 has quit IRC | 13:53 | |
*** surpatil has joined #oooq | 13:56 | |
*** SurajPatil has quit IRC | 13:56 | |
*** SurajPatil has joined #oooq | 13:58 | |
*** amoralej|lunch is now known as amoralej | 13:58 | |
rfolco | weshay, rlandy what about ... NOW? | 13:58 |
rfolco | :) | 13:58 |
rlandy | rfolco: Look slike weshay has 0 mins | 13:58 |
rlandy | 30 | 13:58 |
rlandy | up to him - I'm free all day | 13:59 |
rfolco | his next hour shows as free to me | 13:59 |
weshay | ok.. I'm free atm | 14:00 |
rfolco | rejoin same channel ? | 14:00 |
rfolco | rlandy, weshay ^ | 14:00 |
rlandy | fine | 14:00 |
*** surpatil has quit IRC | 14:01 | |
panda | I don't like being laughed at. | 14:01 |
weshay | marios|rover, we're discussing pipeline stuff | 14:02 |
weshay | marios|rover, mtg is over | 14:02 |
weshay | marios|rover, need anything / help ? | 14:03 |
*** surpatil has joined #oooq | 14:03 | |
marios|rover | ack weshay thanks will drop then | 14:03 |
marios|rover | panda: we have a call then? or want to skip for now? | 14:03 |
chkumar|ruck | marios|rover: rlandy https://review.opendev.org/#/c/696885/ | 14:04 |
panda | marios|rover: I need to move the test on production forward, with or without ruck/rover. | 14:05 |
panda | marios|rover: so whatever. You may skip. | 14:05 |
panda | no one else is joining. | 14:05 |
*** SurajPatil has quit IRC | 14:05 | |
marios|rover | panda: ack i can join if you need to chat or just ping us if you do. maybe cancel the meet? | 14:05 |
marios|rover | panda: i saw no-one there are you there? | 14:05 |
panda | I'll cancel also the other days. | 14:05 |
chkumar|ruck | marios|rover: rlandy will fix the RHEL image build issue | 14:05 |
marios|rover | rlandy++ | 14:06 |
marios|rover | thanks chkumar|ruck | 14:06 |
panda | I'll try to move forward until someone else better than me at testing in production takes over. | 14:06 |
chkumar|ruck | thanks to jpena and apevec for helping | 14:06 |
marios|rover | chkumar|ruck: ah i thought you meant rlandy fixed it 16:05 < chkumar|ruck> marios|rover: rlandy will fix the RHEL image build issue | 14:06 |
*** holser has quit IRC | 14:06 | |
marios|rover | then chkumar|ruck++ ;) | 14:06 |
chkumar|ruck | weshay: regarding this bug https://bugs.launchpad.net/tripleo/+bug/1854721, I have seen this issue in most of the vexxhost job also | 14:07 |
openstack | Launchpad bug 1854721 in tripleo "FS 01 master periodic job failed during overcloud deploy with Exceeded maximum number of retries and 503 status code" [Critical,Triaged] | 14:07 |
chkumar|ruck | which got failed | 14:08 |
weshay | chkumar|ruck, aye.. sshnaidm and rlandy can be a resource if needed | 14:09 |
chkumar|ruck | rlandy: sshnaidm more context here https://bugs.launchpad.net/tripleo/+bug/1854721 | 14:10 |
openstack | Launchpad bug 1854721 in tripleo "FS 01 master periodic job failed during overcloud deploy with Exceeded maximum number of retries and 503 status code" [Critical,Triaged] | 14:10 |
sshnaidm | chkumar|ruck, is it on vexxhost? | 14:11 |
chkumar|ruck | sshnaidm: it is coming on rdocloud as well as vexxhost | 14:11 |
chkumar|ruck | both | 14:11 |
sshnaidm | chkumar|ruck, so it's common for ovb | 14:11 |
chkumar|ruck | yup | 14:11 |
chkumar|ruck | marios|rover: heading home, will back in an hr or half. | 14:12 |
marios|rover | chkumar|ruck: ack | 14:13 |
*** TrevorV has joined #oooq | 14:14 | |
sshnaidm | chkumar|ruck, look in /var/log/extra/errors.txt for ironic and neutron errors | 14:14 |
rlandy | marios|rover: chkumar|ruck: in meeting | 14:14 |
*** rlandy is now known as rlandy|mtg | 14:14 | |
*** ykarel is now known as ykarel|afk | 14:17 | |
*** SurajPatil has joined #oooq | 14:18 | |
*** akahat has quit IRC | 14:18 | |
weshay | chkumar|ruck, show me where you see it in rdo | 14:18 |
*** bhagyashris has quit IRC | 14:20 | |
*** surpatil has quit IRC | 14:21 | |
weshay | rlandy|mtg, rfolco https://tree.taiga.io/project/tripleo-ci-board/task/1420?kanban-status=1447274 | 14:24 |
*** ysandeep has joined #oooq | 14:27 | |
marios|rover | https://review.opendev.org/#/q/topic:containerized-undercloud-upgrades-voting votes please | 14:36 |
marios|rover | thank you | 14:36 |
*** Goneri has joined #oooq | 14:41 | |
*** holser has joined #oooq | 14:42 | |
chkumar|ruck | weshay: this time it got failed at different place http://logs.rdoproject.org/02/23902/1/check/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/2449e84/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 14:50 |
chkumar|ruck | weshay: http://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/b216f86/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz is from periodic job | 14:51 |
*** holser has quit IRC | 15:06 | |
weshay | chkumar|ruck, "Message: No valid host was found. , Code: 500" can happen for just about any reason w/ nodes | 15:07 |
weshay | chkumar|ruck, I don't think you've provided enough evidence we're seeing the same things in rdo and vexx yet | 15:08 |
weshay | chkumar|ruck, better to go after the port_security setting mnaser found | 15:08 |
chkumar|ruck | weshay: I need to learn how to debug those issue | 15:12 |
weshay | chkumar|ruck, check the ironic logs | 15:12 |
rlandy|mtg | what rhel build issue am I fixing? | 15:13 |
weshay | chkumar|ruck, /me notes.. blueprint to fix that horrible error https://blueprints.launchpad.net/tripleo/+spec/nova-less-deploy | 15:14 |
rlandy|mtg | chkumar|ruck: Went to status ERROR due to "Message: No valid host was found. , Code: 500" is a resource issue | 15:16 |
rlandy|mtg | is it reproducible? | 15:16 |
rlandy|mtg | could be caused by not enough resources on cloud at the time | 15:16 |
rlandy|mtg | unless the flavor requirements changed, probably not reproducible | 15:17 |
rlandy|mtg | try and see if it is | 15:17 |
chkumar|ruck | rlandy|mtg: in last run it was not reproducable | 15:17 |
*** rlandy|mtg is now known as rlandy | 15:18 | |
rlandy | chkumar|ruck: yeah - then no use in debugging | 15:18 |
rlandy | cloud just sometimes overruns its resources | 15:18 |
marios|rover | rlandy: looking | 15:19 |
marios|rover | rlandy: (sorry was about your ping in sf-ops seens thanks) | 15:19 |
rlandy | chkumar|ruck: marios|rover: commented on https://bugs.launchpad.net/tripleo/+bug/1854721 | 15:23 |
openstack | Launchpad bug 1854721 in tripleo "FS 01 master periodic job failed during overcloud deploy with Exceeded maximum number of retries and 503 status code" [Critical,Triaged] | 15:23 |
marios|rover | thanks rlandy | 15:23 |
chkumar|ruck | rlandy: thanks@ | 15:23 |
*** SurajPatil has quit IRC | 15:24 | |
rlandy | chkumar|ruck: marios|rover: is there another ovb error you guys need help with or are we done here? | 15:24 |
chkumar|ruck | currently nope! | 15:25 |
rlandy | awesome | 15:25 |
marios|rover | chkumar|ruck: seen that before? timeout on build-images centos7 train fs1 http://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-train/696e7d4/job-output.txt | 15:29 |
marios|rover | chkumar|ruck: "RUN END RESULT_TIMED_OUT" | 15:30 |
marios|rover | chkumar|ruck: am looking at failed things in criteria to push train from https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-latest-released | 15:30 |
marios|rover | chkumar|ruck: but a bit concerned that i don't see any 'missing successful jobs:' lists in http://38.145.34.55/centos7_train.log it's all 2019-12-02 14:32:38,748 29024 DEBUG promoter No remaining hashes after removing ones older than the currently promoted | 15:31 |
chkumar|ruck | marios|rover: nope seeing for first time | 15:31 |
marios|rover | chkumar|ruck: hmm gonna post a testproject in a sec but i want to look at the other jobs failed there too and include them gimme few | 15:31 |
chkumar|ruck | ok | 15:32 |
chkumar|ruck | marios|rover: https://review.rdoproject.org/zuul/builds?result=POST_FAILURE more post_failure with out logs | 15:36 |
chkumar|ruck | poked internally | 15:36 |
marios|rover | chkumar|ruck: same there fyi http://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-train-upload/018a177/job-output.txt but different issue (undercloud install with no logs? ) at | 15:36 |
marios|rover | http://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-singlenode-featureset027-train/7565764/job-output.txt | 15:36 |
marios|rover | chkumar|ruck: and yeah fs27 has no logs? | 15:36 |
marios|rover | chkumar|ruck: even though it faild in undercloud install task | 15:36 |
marios|rover | chkumar|ruck: posting testproject with those 3 jobs now | 15:37 |
chkumar|ruck | marios|rover: https://review.rdoproject.org/zuul/builds?result=POST_FAILURE after ci escalation call it happened | 15:37 |
chkumar|ruck | that time it was 4 only | 15:37 |
marios|rover | chkumar|ruck: i checked current train criteria https://paste.fedoraproject.org/paste/zyueU8VcOor58OKw0Ilofg/raw fwiw i think just those 3 missing from last run | 15:37 |
marios|rover | chkumar|ruck: for centos 7 train | 15:37 |
marios|rover | chkumar|ruck: ack thx 17:37 < chkumar|ruck> marios|rover: https://review.rdoproject.org/zuul/builds?result=POST_FAILURE after ci escalation call it happened | 15:37 |
chkumar|ruck | yes | 15:38 |
*** holser has joined #oooq | 15:39 | |
*** ysandeep has quit IRC | 15:40 | |
chkumar|ruck | marios|rover: looks like network failure https://softwarefactory-project.io/paste/raw/1679/ | 15:44 |
chkumar|ruck | from sf-ops | 15:44 |
marios|rover | chkumar|ruck: thanks, i noted in transient for now | 15:45 |
marios|rover | https://etherpad.openstack.org/p/ruckroversprint19 | 15:45 |
marios|rover | chkumar|ruck: you know maybe we want to move to hackmd ? | 15:45 |
chkumar|ruck | marios|rover: yes | 15:46 |
*** aakarsh has joined #oooq | 15:46 | |
chkumar|ruck | marios|rover: let's do that tomorrow | 15:46 |
marios|rover | chkumar|ruck: ack | 15:46 |
*** ykarel|afk is now known as ykarel|away | 15:54 | |
*** bhagyashris has joined #oooq | 15:57 | |
*** holser has quit IRC | 15:59 | |
*** dtrainor has quit IRC | 16:00 | |
chkumar|ruck | marios|rover: it is a rdo cloud issue due to ceph disk replacement | 16:01 |
chkumar|ruck | it is still in progress | 16:02 |
*** ykarel|away has quit IRC | 16:02 | |
marios|rover | chkumar|ruck: ack thanks | 16:09 |
marios|rover | chkumar|ruck: maybe we get lucky on the testproject run | 16:09 |
chkumar|ruck | marios|rover: let's do that tomorrow morning, I can get it done when i wake up | 16:10 |
*** dtrainor has joined #oooq | 16:10 | |
marios|rover | chkumar|ruck: i postd it sec | 16:10 |
marios|rover | chkumar|ruck: (its on the etherpad) | 16:10 |
chkumar|ruck | ok | 16:10 |
marios|rover | https://review.rdoproject.org/r/23904 | 16:10 |
chkumar|ruck | thanks! | 16:10 |
marios|rover | links notes@ https://paste.fedoraproject.org/paste/bGdqqjsH-8WLBB9iT37sqg/raw posted testproject | 16:10 |
chkumar|ruck | zbr_: https://github.com/containers/libpod/pull/4599 need help on getting this merged | 16:12 |
* chkumar|ruck needs to start learning go now | 16:12 | |
*** holser has joined #oooq | 16:14 | |
chkumar|ruck | time to rest now | 16:17 |
chkumar|ruck | marios|rover: see ya tomorrow | 16:17 |
*** chkumar|ruck is now known as raukadah | 16:17 | |
marios|rover | chkumar|ruck: have a good one | 16:17 |
marios|rover | raukadah: o/ | 16:17 |
marios|rover | raukadah: are you here to take over from chkumar|ruck? | 16:17 |
marios|rover | ;) | 16:17 |
raukadah | marios|rover: rlandy and weshay can help :-) | 16:18 |
marios|rover | bai go learn go | 16:18 |
weshay | raukadah, marios|rover what's up? | 16:19 |
marios|rover | weshay: nothing was just messing around | 16:19 |
*** ykarel|away has joined #oooq | 16:35 | |
*** d0ugal has joined #oooq | 16:37 | |
*** d0ugal has joined #oooq | 16:39 | |
*** udesale has quit IRC | 16:41 | |
* marios|rover almost hometime | 16:41 | |
marios|rover | weshay: need something before you lose ruck|rover for a few hours ? | 16:41 |
marios|rover | weshay: latest things at https://etherpad.openstack.org/p/ruckroversprint19 - chasing train/stein promotions atm | 16:42 |
weshay | marios|rover, nope.. thanks.. just note.. rdo is backed up on current -> consistent.. which sucks for us a bit | 16:43 |
marios|rover | weshay: ack... for stein should be ok cos 24hr | 16:44 |
*** bhagyashris has quit IRC | 16:46 | |
*** bhagyashris has joined #oooq | 16:50 | |
*** marios|rover is now known as marios|out | 16:52 | |
*** marios|out has quit IRC | 17:01 | |
*** sshnaidm is now known as sshnaidm|afk | 17:13 | |
*** bogdando has quit IRC | 17:31 | |
*** ykarel has joined #oooq | 17:32 | |
*** ykarel|away has quit IRC | 17:34 | |
*** Goneri has quit IRC | 17:40 | |
*** Goneri has joined #oooq | 17:54 | |
*** bhagyashris has quit IRC | 17:55 | |
*** jpena is now known as jpena|off | 17:59 | |
*** derekh has quit IRC | 18:00 | |
rfolco | rlandy, can you sync ? | 18:48 |
rlandy | rfolco: I don;t have results to show yet | 18:48 |
rlandy | my run is on generating tempest | 18:48 |
rlandy | experimented with a few options | 18:49 |
rfolco | rlandy, I am stuck coz I don't know which tasks you are working on | 18:49 |
rlandy | rfolco: I ma not touching any promotion work | 18:49 |
rfolco | rlandy, what about add_repo patch you said you would be trying | 18:49 |
rlandy | I am looking into using job.add_repos to include component | 18:49 |
rlandy | that is the one that is running tempest right now | 18:50 |
rlandy | I need about 30 mins to discuss results | 18:50 |
rlandy | and see whether it is a good solution | 18:50 |
rlandy | rfolco: what do you want to work on? | 18:50 |
rfolco | rlandy, I don't know what to do because don't know what is in your patch, what is the solution, if job is defined, if release config file patch is needed, what else. | 18:51 |
*** holser has quit IRC | 18:51 | |
rfolco | rlandy, should I pretend your patch does not cover my tasks ? | 18:52 |
rfolco | rlandy, I am sure this is not true | 18:52 |
rfolco | #1408 and #1409 | 18:53 |
rfolco | release file and standalone job | 18:53 |
rlandy | rfolco: what was in the standalone job other than adding one var to the job definition? | 18:55 |
rlandy | rfolco: wrt release file, idk - depends if we decide to use the release file to pass the component repo set_up values | 18:56 |
rlandy | results of my current test run are uploading | 18:56 |
rlandy | give me few to see if this worked or not | 18:57 |
rfolco | rlandy, ok let me propose the patches again and then we decide. I take the risk of throwing it away. | 18:57 |
rlandy | rfolco: hang on - just s few more minutes | 18:58 |
rlandy | then we can sync | 18:58 |
rlandy | the release file may be an option idk yer | 18:58 |
rlandy | problem is the var precedence | 18:58 |
rlandy | the release file could solve that | 18:59 |
rfolco | rlandy, ok | 18:59 |
*** Tengu has quit IRC | 19:00 | |
rlandy | I'll explain in a few | 19:00 |
*** Tengu has joined #oooq | 19:01 | |
rlandy | ugh collect logs takes forever | 19:02 |
rlandy | http://logs.rdoproject.org/41/23841/12/check/periodic-tripleo-ci-centos-7-standalone-compute-master/1c08b7d/logs/undercloud/etc/yum.repos.d/component.repo.txt.gz | 19:03 |
rlandy | okie dokie | 19:03 |
rlandy | we have a solution to talk about | 19:03 |
rlandy | weshay: rfolco: want to sync briefly now? | 19:04 |
rlandy | we need to confirm one direction and go from there now | 19:04 |
weshay | ok | 19:04 |
rfolco | ok | 19:05 |
rlandy | https://meet.google.com/wyp-znwe-gvs | 19:05 |
rlandy | weshay: rfolco: ^^ | 19:05 |
*** ykarel has quit IRC | 19:07 | |
*** amoralej is now known as amoralej|off | 19:42 | |
*** jfrancoa has quit IRC | 19:49 | |
*** tesseract has quit IRC | 19:49 | |
*** d0ugal has quit IRC | 20:02 | |
*** brault has quit IRC | 20:02 | |
rfolco | weshay, who can I talk to about dlrnapi password ? jpena|off only? | 20:07 |
rfolco | weshay, component one | 20:07 |
weshay | internal? | 20:07 |
weshay | staging upstream? | 20:07 |
rfolco | staging | 20:07 |
weshay | rfolco, so.. for upstream.. I think we're going to assume we're using the perm dlrn server | 20:08 |
weshay | I think jpena just has some patches to land | 20:08 |
rfolco | k | 20:08 |
weshay | rfolco, usually we have a weekly to sync on this kind of stuff | 20:08 |
weshay | but I'm pto this friday | 20:08 |
rfolco | weshay, thats ok, I am going to talk to jpena tomorrow morning | 20:09 |
rfolco | will work on the promotion job now, the one that checks results and promotes repo | 20:09 |
weshay | https://review.rdoproject.org/r/#/c/23277/ | 20:10 |
weshay | https://review.rdoproject.org/r/#/c/22394/ | 20:11 |
weshay | rfolco, so.. I guess our calculus is this.. if he is not planning on merging the code for components for quite some time.. we'll need staging fully setup and passwds etc.. to meet your requirements | 20:13 |
rfolco | k | 20:13 |
rlandy | rfolco: weshay: 4 reviews in play now ... | 20:18 |
rlandy | https://review.opendev.org/#/c/696951/ Revert "Component promotion jobs should read a custom release cfg" | 20:18 |
rlandy | https://review.opendev.org/#/c/696933/ Include job.add_repos to repo_setup | 20:18 |
rlandy | https://review.rdoproject.org/r/#/c/23841/ Add standalone jobs for component promotion | 20:19 |
rlandy | https://review.rdoproject.org/r/#/c/23906/ Test component standalone | 20:19 |
rlandy | now updating pipeline | 20:22 |
rlandy | rfolco: weshay: https://review.rdoproject.org/r/23712 Add openstack-component-compute pipeline and layout | 20:33 |
rlandy | .... and it's snowing | 20:35 |
*** holser has joined #oooq | 21:06 | |
rlandy | https://review.rdoproject.org/r/#/c/23712/10/ci-scripts/tripleo-upstream/get-hash.sh | 21:07 |
rlandy | ^^ that could use a design review pls | 21:07 |
*** jtomasek has quit IRC | 21:15 | |
*** TrevorV has quit IRC | 21:29 | |
*** aakarsh has quit IRC | 21:41 | |
rlandy | rfolco: weshay: how do we know we have installed the right rpm using the component | 21:47 |
rlandy | http://logs.rdoproject.org/06/23906/1/check/periodic-tripleo-ci-centos-7-standalone-compute-master/3947799/logs/undercloud/var/log/extra/rpm-list.txt.gz | 21:47 |
rlandy | ^^ looking there | 21:48 |
rlandy | python2-novaclient.noarch 1:16.0.0-0.20191106091533.380fc08.el7 | 21:49 |
rlandy | @delorean | 21:49 |
rlandy | no openstack-nova | 21:50 |
rlandy | Dec 01 00:34:37 Installed: 1:openstack-nova-common-20.1.0-0.20191128234359.f138265.el7.noarch | 21:52 |
rlandy | Dec 01 00:42:47 Installed: 1:openstack-nova-api-20.1.0-0.20191128234359.f138265.el7.noarch | 21:52 |
rlandy | using https://trunk-staging.rdoproject.org/centos7/component/compute/f1/38/ | 21:53 |
rlandy | it should have highest priority | 21:54 |
rlandy | we have a problem | 21:54 |
*** jbadiapa has quit IRC | 22:14 | |
*** aakarsh has joined #oooq | 22:26 | |
rlandy | anyone? | 22:52 |
rlandy | https://code.engineering.redhat.com/gerrit/186764 Add first rhos-17 release file | 23:08 |
rlandy | merged that - whatever - need it for pipeline | 23:09 |
rlandy | https://code.engineering.redhat.com/gerrit/186766 WIP: Add component standalone jobs | 23:36 |
*** holser has quit IRC | 23:38 | |
*** tosky has quit IRC | 23:41 | |
rlandy | testing downstream | 23:43 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!