| *** d0ugal has quit IRC | 00:07 | |
| *** Goneri has quit IRC | 00:12 | |
| *** Goneri has joined #oooq | 00:14 | |
| *** d0ugal has joined #oooq | 00:17 | |
| *** dsneddon_ has joined #oooq | 00:23 | |
| *** dsneddon_ has quit IRC | 00:28 | |
| EmilienM | hey folks | 00:44 |
|---|---|---|
| EmilienM | tripleo-ci-centos-7-scenario009-multinode-oooq-container seems broken | 00:44 |
| EmilienM | https://bc76d54bd49bc13e4cf8-798b7640b46a8671e5fce2cc99f63f34.ssl.cf5.rackcdn.com/686784/2/check/tripleo-ci-centos-7-scenario009-multinode-oooq-container/58ae91c/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 00:44 |
| EmilienM | it was passing fine 5 days ago | 00:45 |
| *** rfolco|bbl has quit IRC | 00:59 | |
| *** dsneddon_ has joined #oooq | 01:00 | |
| *** dsneddon_ has quit IRC | 01:05 | |
| *** dsneddon_ has joined #oooq | 01:40 | |
| *** dsneddon_ has quit IRC | 01:44 | |
| *** dsneddon_ has joined #oooq | 02:16 | |
| *** dsneddon_ has quit IRC | 02:21 | |
| *** dsneddon_ has joined #oooq | 02:53 | |
| *** ykarel has joined #oooq | 02:56 | |
| *** dsneddon_ has quit IRC | 02:57 | |
| *** dsneddon_ has joined #oooq | 03:08 | |
| *** ykarel has quit IRC | 03:47 | |
| *** ykarel has joined #oooq | 04:07 | |
| *** udesale has joined #oooq | 04:08 | |
| *** surpatil has joined #oooq | 05:12 | |
| *** ratailor has joined #oooq | 05:17 | |
| *** marios has joined #oooq | 05:57 | |
| *** slaweq has joined #oooq | 06:11 | |
| Tengu | hmm... DNS issue somewhere apparently. | 06:18 |
| *** jbadiapa has joined #oooq | 06:20 | |
| chandankumar | marios: Good morning | 06:43 |
| chandankumar | marios: Does updates to promoter server are enabled now? | 06:43 |
| chandankumar | till last friday It was disabled | 06:43 |
| marios | chandankumar: i don't know. the code merged, but weshay and panda|off are running isolated code on the production promoter ... by the time i left on friday it was not enabled | 06:44 |
| marios | chandankumar: i think they would have emailed about it if they had though? | 06:44 |
| chandankumar | marios: ok, thanks, Dropping an email | 06:45 |
| ykarel | it seems not enabled as i don't see centos7_train.log | 06:45 |
| *** jfrancoa has joined #oooq | 06:49 | |
| *** akahat has joined #oooq | 06:53 | |
| *** kopecmartin|off is now known as kopecmartin | 06:53 | |
| *** apetrich has joined #oooq | 06:53 | |
| marios | ykarel: " it seems not enabled cos things are still working" :D | 06:55 |
| marios | ykarel: yeah i'd be suprised if they enabled it on friday to be honest | 06:56 |
| marios | ykarel: but soon though as the code is mostly merged | 06:56 |
| *** fmount|off is now known as fmount | 07:00 | |
| *** saneax has joined #oooq | 07:00 | |
| *** jaosorior has joined #oooq | 07:01 | |
| *** ccamacho has joined #oooq | 07:09 | |
| *** jtomasek has joined #oooq | 07:10 | |
| *** tosky has joined #oooq | 07:17 | |
| *** amoralej|off is now known as amoralej | 07:25 | |
| *** ykarel is now known as ykarel|lunch | 07:44 | |
| *** jpena|off is now known as jpena | 07:45 | |
| *** holser has joined #oooq | 08:12 | |
| panda|off | chandankumar: the new code is working in test, the promoter is now running and succeeding e2e, but we are missing test. AFAIU the next step is to spawn a new promoter server and test the code in real life, then we can reenable the updates. | 08:21 |
| *** chem has joined #oooq | 08:22 | |
| *** chem has quit IRC | 08:24 | |
| *** chem has joined #oooq | 08:24 | |
| *** skramaja has joined #oooq | 08:29 | |
| chandankumar | panda|off: ok | 08:30 |
| chandankumar | panda|off: When we are going to deploy the new promoter server? | 08:30 |
| *** ykarel|lunch is now known as ykarel | 08:32 | |
| *** derekh has joined #oooq | 08:40 | |
| panda|off | chandankumar: I'll create one as soon as I return from an errand, but I suspect wes wants to set up one and be really sure. I hope we can look at this today. | 08:58 |
| tosky | chandankumar: is this connected somehow to the error I'm getting here, https://review.opendev.org/#/c/685332/, which is: | 09:00 |
| tosky | https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_ec4/685332/1/check/sahara-tripleo-ci-centos-7-scenario003-standalone/ec4eca4/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz | 09:01 |
| tosky | ? | 09:01 |
| *** dsneddon_ has quit IRC | 09:03 | |
| *** bogdando has joined #oooq | 09:04 | |
| *** jaosorior has quit IRC | 09:15 | |
| chandankumar | tosky: checking | 09:18 |
| chandankumar | tosky: Do donot have a train promotion | 09:20 |
| chandankumar | tosky: *we donot | 09:20 |
| chandankumar | tosky: that's why it is failing, we are working on that | 09:21 |
| tosky | chandankumar: so it is connected | 09:22 |
| tosky | :) | 09:22 |
| tosky | chandankumar: how can I notice that a promotion happens? Will http://dashboards.rdoproject.org/train be populated when the promotion will happen? | 09:24 |
| chandankumar | tosky: currently pushing updates to promotion server is disabled, panda|off is working on it, and yes http://dashboards.rdoproject.org/train once promotion will happen, it will be populated | 09:28 |
| tosky | ack, I will wait | 09:28 |
| *** dsneddon_ has joined #oooq | 09:39 | |
| *** panda|off is now known as panda | 09:54 | |
| *** rascasoft has quit IRC | 10:04 | |
| *** rascasoft has joined #oooq | 10:05 | |
| *** epoojad1 has joined #oooq | 10:18 | |
| *** dsneddon_ has quit IRC | 10:20 | |
| *** derekh has quit IRC | 10:25 | |
| *** epoojad1 has quit IRC | 10:34 | |
| *** udesale has quit IRC | 10:36 | |
| *** udesale has joined #oooq | 10:36 | |
| *** dsneddon_ has joined #oooq | 10:38 | |
| *** ratailor has quit IRC | 10:39 | |
| *** dsneddon_ has quit IRC | 10:42 | |
| *** skramaja has quit IRC | 10:48 | |
| *** dsneddon_ has joined #oooq | 11:09 | |
| *** ykarel is now known as ykarel|afk | 11:10 | |
| *** udesale has quit IRC | 11:11 | |
| *** derekh has joined #oooq | 11:12 | |
| *** saneax has quit IRC | 11:29 | |
| *** ykarel|afk is now known as ykarel | 11:29 | |
| *** saneax has joined #oooq | 11:29 | |
| *** jpena is now known as jpena|lunch | 11:33 | |
| *** dsneddon_ has quit IRC | 11:49 | |
| *** Goneri has quit IRC | 11:58 | |
| *** dsneddon_ has joined #oooq | 11:59 | |
| *** dsneddon_ has quit IRC | 12:04 | |
| weshay | chandankumar, hey | 12:04 |
| weshay | arxcruz|rover, 0/ | 12:05 |
| arxcruz|rover | weshay: hi boss | 12:05 |
| weshay | need anything on the promoter? | 12:05 |
| arxcruz|rover | weshay: not for now, we had some issues on rdo this weekend, several stacks stucked in delete, so many timeouts | 12:06 |
| weshay | ya.. saw that | 12:06 |
| arxcruz|rover | weshay: i'll have doctor appointment in 1 and half hour, than german class, not sure where zbr is | 12:06 |
| *** rfolco|bbl has joined #oooq | 12:07 | |
| weshay | arxcruz|rover, he's pto today | 12:08 |
| *** weshay is now known as weshay|ruck | 12:08 | |
| arxcruz|rover | ok | 12:08 |
| weshay|ruck | chandankumar, how is pooja? :) | 12:08 |
| weshay|ruck | she needs a monitor I hear | 12:08 |
| chandankumar | weshay|ruck: yes | 12:11 |
| chandankumar | weshay|ruck: but before that, we need to sort out desk issue. | 12:11 |
| chandankumar | weshay|ruck: she got the account setup | 12:11 |
| weshay|ruck | k.. cool | 12:11 |
| weshay|ruck | thanks for helping | 12:12 |
| chandankumar | weshay|ruck: we need to fix the promoter server | 12:12 |
| chandankumar | weshay|ruck: upstream projects where tripleo jobs runs they need to merge stable/train branches patches | 12:12 |
| chandankumar | weshay|ruck: we need train promotion | 12:13 |
| * weshay|ruck needs to hear the details on that | 12:13 | |
| weshay|ruck | is master impacted? | 12:13 |
| chandankumar | weshay|ruck: https://review.opendev.org/#/c/685332/ | 12:13 |
| tosky | weshay|ruck: I am impacted :) | 12:13 |
| chandankumar | weshay|ruck: nope, mistral team was asking for the same also | 12:14 |
| *** rfolco|bbl is now known as rfolco | 12:14 | |
| weshay|ruck | tosky, because the scen003 job is not passing on train? | 12:14 |
| tosky | weshay|ruck: yes, correct; it's failing because it can't access the repository | 12:15 |
| tosky | (more or less) | 12:15 |
| *** amoralej is now known as amoralej|lunch | 12:18 | |
| weshay|ruck | chandankumar, tosky tripleo has not yet had a promotion in train | 12:18 |
| tosky | weshay|ruck: that's what chandankumar told me a few hours ago (and also that there is some work in progress for this) | 12:20 |
| *** epoojad1 has joined #oooq | 12:20 | |
| chandankumar | weshay|ruck: meet epoojad1 (pooja) | 12:21 |
| weshay|ruck | epoojad1, WELCOME!! | 12:21 |
| epoojad1 | thank you :) | 12:22 |
| ykarel | chandankumar, are u aware of 2019-10-07 11:57:34.080800 | primary | emit_releases_file.py: error: argument --stable-release: invalid choice: 'train' (choose from 'newton', 'ocata', 'pike', 'queens', 'rocky', 'stein', 'master')? | 12:22 |
| chandankumar | ykarel: where it is coming? | 12:22 |
| ykarel | chandankumar, train promotion pipeline | 12:22 |
| ykarel | https://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset010-train/2868af6/job-output.txt | 12:22 |
| ykarel | may be in other jobs too, /me noticed this only | 12:22 |
| weshay|ruck | chandankumar, arxcruz|rover we need to start writing up the train failures | 12:23 |
| weshay|ruck | https://trello.com/b/j4IcIomh/production-chain-escalation | 12:23 |
| weshay|ruck | last one I saw was timeouts in rdo-cloud | 12:23 |
| chandankumar | ykarel: https://review.opendev.org/#/c/684260/ | 12:24 |
| ykarel | chandankumar, ack but seems it's blocked, i see -2 | 12:24 |
| chandankumar | ykarel: it needs reviews from other cores | 12:25 |
| ykarel | mmm waiting since 24th september | 12:25 |
| ykarel | i see +2,+W from panda , but seems he removed +W due to weshay|ruck -2 | 12:26 |
| epoojad1 | weshay|ruck : Today I had induction sessions, now I am setting up the on boarding configurations :) | 12:27 |
| weshay|ruck | epoojad1, ok.. good | 12:27 |
| weshay|ruck | epoojad1, so I'll chat w/ you for a few after today's scrum | 12:28 |
| * weshay|ruck checks if you are on it | 12:28 | |
| weshay|ruck | yes.. you are :) | 12:28 |
| epoojad1 | weshay|ruck : sure | 12:28 |
| weshay|ruck | chandankumar, get.. marios, rlandy, sagi ( pto ) so meh etc | 12:29 |
| marios | weshay|ruck: /me reads | 12:30 |
| *** jpena|lunch is now known as jpena | 12:31 | |
| marios | weshay|ruck: not sure what the ai is? we talk on scrum in half? | 12:32 |
| *** rlandy has joined #oooq | 12:32 | |
| *** dsneddon_ has joined #oooq | 12:32 | |
| weshay|ruck | marios, for that gerrit change? just need a review | 12:32 |
| marios | weshay|ruck: 15:24 < chandankumar> ykarel: https://review.opendev.org/#/c/684260/ | 12:33 |
| marios | ? that one looking | 12:33 |
| arxcruz|rover | weshay|ruck: boss? | 12:34 |
| chandankumar | marios: yes | 12:34 |
| chandankumar | marios: thanks! | 12:34 |
| *** jaosorior has joined #oooq | 12:40 | |
| chandankumar | rlandy: Good morning | 12:40 |
| rlandy | chandankumar: hey | 12:40 |
| chandankumar | rlandy: please have a look at this https://review.rdoproject.org/r/#/c/22978/ , thanks , fix for wed pipeline | 12:40 |
| rlandy | chandankumar: that makes me laugh - | 12:42 |
| rlandy | chandankumar: I asked that question on zuul | 12:42 |
| chandankumar | rlandy: yes, I was also got confused | 12:42 |
| rlandy | and paul said no, the week start on day 1 | 12:42 |
| rlandy | I think if you look at the first version, I had it this way | 12:42 |
| chandankumar | rlandy: is something different configured in RDO zuul tenant? | 12:43 |
| ykarel | weshay|ruck, fyi, when i mentioned zbr patch, i meant https://review.opendev.org/#/c/686173/ | 12:43 |
| rlandy | chandankumar: actually no, the doc specifies it .. let me get that | 12:43 |
| rlandy | which is why I asked | 12:43 |
| chandankumar | rlandy: asked tristin to look into that patch. | 12:44 |
| rlandy | midnight. The first weekday is Monday. | 12:44 |
| rlandy | https://zuul-ci.org/docs/zuul/admin/drivers/timer.html | 12:44 |
| ykarel | yes so Monday will be 0 | 12:44 |
| rlandy | chandankumar: ^^ zuul actually says so | 12:45 |
| rlandy | but that is not how cron usually works | 12:45 |
| chandankumar | ok | 12:46 |
| rlandy | chandankumar: if it's not working correctly, then we know that it's wrong and we need to change it | 12:46 |
| rlandy | Day of weekYes0–6 or SUN–SAT | 12:46 |
| rlandy | cron: ^^ | 12:46 |
| rlandy | hence the confusion | 12:47 |
| rlandy | chandankumar: ykarel: see the diff here | 12:47 |
| rlandy | never the less, if it's not working right here, we go with the other solution | 12:47 |
| rlandy | but that is why I left the comment | 12:48 |
| chandankumar | rlandy: https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-wednesday-weekend | 12:48 |
| rlandy | so there would be a trace that I looked into it | 12:48 |
| *** aakarsh has quit IRC | 12:48 | |
| ykarel | rlandy, ack we need to go with what zuul understands | 12:48 |
| ykarel | so 0 being Monday | 12:48 |
| chandankumar | ykarel: check tristin comment https://review.rdoproject.org/r/#/c/22978/ | 12:49 |
| ykarel | chandankumar, comment conflicts with zuul docs | 12:49 |
| rlandy | tristan did what I did - cross checked cron doc | 12:50 |
| chandankumar | rlandy: I also checked the cron guru docs while verifying ronell patch | 12:50 |
| rlandy | chandankumar: ykarel: I am ready to w+ that | 12:51 |
| chandankumar | one min | 12:51 |
| rlandy | experience shows who is right here | 12:51 |
| chandankumar | checking zuul code | 12:51 |
| rlandy | personally, I think zuul is wrong for not following cron convention - but it is what it is | 12:53 |
| weshay|ruck | need reviews on https://review.opendev.org/#/c/686430/ | 12:59 |
| rlandy | I for sure checked with the gurus on #zuul before committing that patch | 13:00 |
| rlandy | weshay|ruck: I was looking at that for bm | 13:00 |
| rlandy | had some issues accessing the proxy | 13:01 |
| rfolco | ping scrum weshay|ruck rlandy chandankumar | 13:01 |
| rfolco | panda, ^ | 13:01 |
| weshay|ruck | be there in a sec | 13:01 |
| rfolco | https://meet.google.com/oiv-geho-mai | 13:02 |
| * arxcruz|rover brb | 13:05 | |
| *** Goneri has joined #oooq | 13:08 | |
| chandankumar | rfolco: you can continue | 13:09 |
| *** amoralej|lunch is now known as amoralej | 13:09 | |
| chandankumar | rfolco: I will catch in middle | 13:09 |
| *** epoojad1 has quit IRC | 13:10 | |
| *** dsneddon_ has quit IRC | 13:12 | |
| weshay|ruck | panda, fyi.. removed promotion blocker, but added comment https://bugs.launchpad.net/tripleo/+bug/1846662 | 13:15 |
| openstack | Launchpad bug 1846662 in tripleo "dlrn_promoter is updating links in dlrn_trunk prior to all the containers being pushed" [Critical,Triaged] | 13:15 |
| *** ratailor has joined #oooq | 13:23 | |
| *** Vorrtex has joined #oooq | 13:23 | |
| *** derekh has quit IRC | 13:24 | |
| *** aakarsh has joined #oooq | 13:35 | |
| weshay|ruck | chandankumar, https://review.opendev.org/#/c/686750/ | 13:37 |
| weshay|ruck | https://review.rdoproject.org/r/#/c/22943/ | 13:38 |
| weshay|ruck | chandankumar, http://logs.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-train/3ccf85e/ | 13:40 |
| chandankumar | weshay|ruck: great | 13:44 |
| *** dsneddon_ has joined #oooq | 13:46 | |
| chandankumar | marios: https://review.rdoproject.org/r/#/c/20993/ | 13:48 |
| weshay|ruck | rlandy, chandankumar the timeout change is working.. but.. good news.. train fs001 took 77min :) | 13:48 |
| chandankumar | cool | 13:48 |
| weshay|ruck | but lets leave the extended timeout for now | 13:48 |
| chandankumar | ykarel: ^^ | 13:48 |
| rlandy | lol | 13:48 |
| rlandy | yeah - leave it for now | 13:48 |
| weshay|ruck | arxcruz|rover, perhaps we can add a timer check in the cockpit on overcloud deployments as well | 13:49 |
| weshay|ruck | so we can build trends over time :) | 13:49 |
| ykarel | weshay|ruck, can u get timeout change for phase1 jobs too | 13:49 |
| weshay|ruck | chandankumar, https://meet.google.com/kct-wkoi-sus ykarel you can join too if you want | 13:52 |
| weshay|ruck | talking train promotion | 13:53 |
| chandankumar | rlandy: marios https://en.wikipedia.org/wiki/Vijayadashami | 13:53 |
| marios | ack chandankumar | 13:53 |
| *** ratailor has quit IRC | 13:54 | |
| weshay|ruck | chandankumar, https://code.engineering.redhat.com/gerrit/#/q/project:rhosp-dashboards | 13:56 |
| ykarel | weshay|ruck, chandankumar https://trunk-primary.rdoproject.org/api-centos-train/api/civotes_detail.html?commit_hash=72487d804ef4c64d16889b1b474252f6963a18c2&distro_hash=3f633a877cc44c0fa774e06dbf99c7db29419b2f | 13:59 |
| *** chem has quit IRC | 14:01 | |
| panda | rlandy: https://docs.docker.com/registry/spec/api/ "PULLING AN IMAGE MANIFEST" | 14:01 |
| *** chem has joined #oooq | 14:02 | |
| *** saneax has quit IRC | 14:08 | |
| chandankumar | weshay|ruck: do we want to review the ceph-ansible podman planning docs? | 14:09 |
| weshay|ruck | chandankumar, can we push that to tomorrow? | 14:09 |
| chandankumar | weshay|ruck: tomorrow is holiday , If it is ok, I can schedule a call tomorrow might be evening? | 14:10 |
| panda | rlandy: https://hub.docker.com/r/tripleomaster/centos-binary-rabbitmq/tags | 14:10 |
| chandankumar | or better sync on wednesday | 14:10 |
| ykarel | arxcruz|rover, Rocky is being promoted now | 14:11 |
| ykarel | weshay|ruck, fyi ^^ | 14:11 |
| chandankumar | weshay|ruck: invite changed to wed | 14:12 |
| panda | marios: can you rejoin 2 minutes please ? | 14:12 |
| *** akahat has quit IRC | 14:13 | |
| marios | panda: sure gimme sec | 14:13 |
| weshay|ruck | \0/ | 14:13 |
| *** saneax has joined #oooq | 14:16 | |
| *** saneax has quit IRC | 14:22 | |
| *** saneax has joined #oooq | 14:24 | |
| rfolco | panda, ping | 14:25 |
| rfolco | panda, check if dlrn api actually finished a promotion process >> this is still failing... do we have a patch fixing this ? | 14:25 |
| chandankumar | rlandy: need +w https://review.opendev.org/#/c/686149/ on this, thanks! | 14:27 |
| chandankumar | rebased the patch earlier | 14:27 |
| panda | rfolco: this passes | 14:29 |
| panda | rfolco: we have a e2e workflow | 14:29 |
| rfolco | panda, where? I rebased on master and still fails | 14:30 |
| rfolco | http://logs.rdoproject.org/88/22988/2/check/tripleo-ci-promotion-staging/9215071/job-output.txt | 14:30 |
| weshay|ruck | ykarel, we need https://trunk.rdoproject.org/rhel8-train/ | 14:34 |
| panda | rfolco: mmmmhhh | 14:35 |
| ykarel | weshay|ruck, i think jpena was working on it ^^ | 14:35 |
| jpena | weshay|ruck: https://tree.taiga.io/project/morucci-software-factory/us/3011?milestone=242579 | 14:36 |
| panda | rfolco: worked for me, it's working for ronelle | 14:38 |
| rfolco | panda, will try to rebase again | 14:39 |
| panda | rfolco: wait, rlandy is seeing the same thing | 14:43 |
| rfolco | aha | 14:43 |
| rfolco | I told you Mr G | 14:43 |
| *** ykarel is now known as ykarel|afk | 14:44 | |
| rfolco | panda, crap ... look this | 14:45 |
| rfolco | http://logs.rdoproject.org/88/22988/2/check/tripleo-ci-promotion-staging/9215071/logs/promoter_logs/centos7_master.log | 14:45 |
| rfolco | ProtocolError: ('Connection aborted.', error(104, 'Connection reset by peer')) | 14:46 |
| chandankumar | weshay|ruck: we are waiting for this https://review.rdoproject.org/r/20993 to get rhel8-train dlrn builder | 14:50 |
| panda | rfolco: that's not the worst part | 14:51 |
| panda | rfolco: the worst part is that http://logs.rdoproject.org/88/22988/2/check/tripleo-ci-promotion-staging/9215071/logs/dlrnapi.log is not showing any error | 14:52 |
| *** saneax has quit IRC | 14:52 | |
| *** jfrancoa has quit IRC | 14:54 | |
| weshay|ruck | chandankumar, rfolco we going to pre-plan today? | 14:55 |
| chandankumar | weshay|ruck: yes | 14:55 |
| rfolco | weshay|ruck, could be a quick one just to refresh our sprint goals | 14:55 |
| panda | the dlrnapi does not receive the POST, probably uwsgi crashes ... | 14:55 |
| chandankumar | weshay|ruck: rlandy needs https://review.opendev.org/#/c/673400/ +2 and +w as os_temepst is working fine | 14:57 |
| rlandy | looking | 14:58 |
| chandankumar | rfolco: do we want to use bjn or google meet for pre-planning? | 15:01 |
| rfolco | weshay|ruck, chandankumar hmm did not create a gmeet for pre-planning... | 15:01 |
| rfolco | lets use my bj | 15:01 |
| rfolco | https://bluejeans.com/5878458097 | 15:01 |
| rlandy | '| OSError: [Errno 2] No such file or directory: '/tmp/promoter-staging/overcloud_images/centos/master/rdo_trunk/tripleo-ci-staging-promoted' | 15:06 |
| rlandy | still not :( | 15:07 |
| *** jfrancoa has joined #oooq | 15:08 | |
| panda | rlandy: centos7 | 15:13 |
| panda | rlandy: so the link should be distro + distro_version | 15:14 |
| rlandy | yep - updating | 15:21 |
| arxcruz|rover | ykarel|afk: \o/ | 15:28 |
| chandankumar | rfolco: weshay|ruck https://tree.taiga.io/project/tripleo-ci-board/us/1318 | 15:32 |
| *** ykarel|afk is now known as ykarel|away | 15:38 | |
| rlandy | panda: pls see if you agree with lines 72-96 now ... https://review.rdoproject.org/r/#/c/22958/15/ci-scripts/dlrnapi_promoter/tests/staging-setup/promoter_test.py | 15:41 |
| rlandy | this run that containers test is still commented out | 15:41 |
| *** ykarel|away has quit IRC | 15:43 | |
| *** surpatil has quit IRC | 15:45 | |
| *** marios has quit IRC | 15:48 | |
| panda | rlandy: that's fine, as a readability improvement you may want to unpack the split and refer to the components directly. eg. name, tag = line.split(":"), then format(name, tag) | 15:55 |
| * rlandy loks | 15:56 | |
| rlandy | looks | 15:56 |
| chandankumar | see ya on wednesday or little earlier | 15:57 |
| *** chandankumar is now known as raukadah | 15:57 | |
| rlandy | raukadah: I will be out on wed - anything you want me to do wrt train stuff today? | 15:59 |
| rlandy | I will update criteria patch | 15:59 |
| *** ykarel has joined #oooq | 16:03 | |
| rfolco | panda, rlandy: do you know why dlrapi is crashing/not receiving post ? | 16:10 |
| *** apetrich has quit IRC | 16:12 | |
| panda | rfolco: I'm investigating, reproducing the problem in the new promoter server | 16:14 |
| panda | there are a few things to fix also in the general code | 16:14 |
| rfolco | panda, ok thanks, I will try to debug here as well | 16:14 |
| *** apetrich has joined #oooq | 16:17 | |
| *** ykarel is now known as ykarel|away | 16:27 | |
| rfolco | panda, fyi the issue is intermittent, passed that point without any direct changes in the code.... http://logs.rdoproject.org/88/22988/6/check/tripleo-ci-promotion-staging/9a6f310/job-output.txt | 16:38 |
| rfolco | now promoter_test failed | 16:38 |
| rfolco | AttributeError: 'Namespace' object has no attribute 'stage_info_path' | 16:38 |
| panda | rfolco: rlandy is working on that part | 16:39 |
| panda | rfolco: I know it's intermittent, we need to stabilize it | 16:39 |
| rlandy | panda: rfolco: sorry - on ovb problem atm | 16:39 |
| panda | if uwsgi at a certain point stops responding and serving dlrnapi we either 1)find a root cause, 2) use a different wsgi server, possibily apache (the only one in which dlrnapi is officially tested 3) ???? 4) profit | 16:40 |
| raukadah | rlandy: need comments on few reviews about missing check train jobs https://review.rdoproject.org/r/#/c/22977/ and https://review.opendev.org/#/c/687004/ | 16:42 |
| rlandy | ok | 16:45 |
| weshay|ruck | rlandy, http://logs.rdoproject.org/93/686293/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/71e3ffa/job-output.txt | 16:45 |
| raukadah | weshay|ruck: do you have the fixes for fs039 handy? | 16:45 |
| weshay|ruck | raukadah, note.. I was told on the cix call they have this working, but haven't seen it myself https://review.rdoproject.org/r/#/c/21672/ | 16:46 |
| raukadah | if it works, I will get the reproduce to get os_temepst stuff done | 16:46 |
| raukadah | retry limit | 16:47 |
| *** jfrancoa has quit IRC | 16:48 | |
| raukadah | panda: needs +w https://review.opendev.org/#/c/673400/ on this | 16:48 |
| weshay|ruck | arxcruz|rover, https://review.rdoproject.org/r/#/c/22993/ | 16:53 |
| weshay|ruck | any cores home? https://review.rdoproject.org/r/#/c/22993/ | 16:53 |
| raukadah | weshay|ruck: done | 16:55 |
| panda | raukadah: should this by tested in train also ? | 16:56 |
| raukadah | panda: yes | 16:56 |
| weshay|ruck | thanks | 16:56 |
| raukadah | panda: it will work for train | 16:57 |
| weshay|ruck | raukadah, fyi... the promoter has been busy all morning | 16:57 |
| raukadah | ok | 16:57 |
| weshay|ruck | I hope train is next | 16:57 |
| panda | raukadah: I''ll take yur word for it | 16:57 |
| raukadah | panda: :-) | 16:57 |
| raukadah | weshay|ruck: what about moving fs020 to os_tempest now? | 16:57 |
| weshay|ruck | if you have the bandwidth.. sure | 16:58 |
| weshay|ruck | raukadah, you could also consider taking a break | 16:58 |
| raukadah | weshay|ruck: after ptg, going on a long break | 16:59 |
| raukadah | may be 15 days | 16:59 |
| rlandy | http://logs.rdoproject.org/58/22958/17/check/tripleo-ci-promotion-staging/a6cc04d/job-output.txt ugh | 16:59 |
| *** jfrancoa has joined #oooq | 17:03 | |
| weshay|ruck | raukadah, https://review.rdoproject.org/zuul/stream/fba0ba5a1d534914a0c5ef92c0a99d66?logfile=console.log | 17:04 |
| weshay|ruck | fs039 | 17:04 |
| *** jpena is now known as jpena|off | 17:11 | |
| weshay|ruck | cores.. https://review.opendev.org/#/c/686430/ | 17:11 |
| panda | rlandy: I can't reproduce locally yet, every atempt succeeds | 17:12 |
| *** holser has quit IRC | 17:16 | |
| rlandy | panda: running again | 17:22 |
| rlandy | sporadic | 17:22 |
| rlandy | my best guess is networking | 17:22 |
| *** dsneddon_ has quit IRC | 17:25 | |
| panda | rlandy: it's localhost :( we could try on a different port, 8080 is usually overused | 17:30 |
| panda | rlandy: reproducing locally isn't helping< I'll see if I can improve the logging, and gather more info on how this happens, by relaunching tests in zuul. | 17:32 |
| rlandy | ok - just getting the link test right | 17:32 |
| panda | rlandy: Iterating on https://review.rdoproject.org/r/22994 on top of your patch. | 17:36 |
| panda | but I can't stay here long :/ | 17:37 |
| rlandy | panda: np - of you need to go | 17:39 |
| rlandy | I can finish my bit for your review tomorrow | 17:39 |
| rlandy | panda: can you point me to where I would add stage_info['promotions']['failed_attempt']? | 17:59 |
| *** dsneddon_ has joined #oooq | 18:00 | |
| panda | rlandy: in the fixture file, you'll need to add another commit, with a new id, then in the promotions section there, add a promotion name that reference that id. The new commit's timestamp should be consistent, I would imagine a timestamp between the previous promotion and the promotion candidate | 18:02 |
| panda | rlandy: also, for the builtins problem, in py27 the module is called __builtin__ so the string in mock should be "__builtin__.open". You can add a builtin_str near the imports, then in the mock part you can use builtin_str + ".mock" | 18:04 |
| rlandy | pnada: ok - the overcloud_images test passes now | 18:04 |
| panda | rlandy: \o/ | 18:04 |
| rlandy | testing the containers test now | 18:04 |
| rlandy | panda: "You can add a builtin_str near the imports" - why not just below: def test_parse(self):? | 18:07 |
| rlandy | the import is diff? | 18:07 |
| panda | rlandy: if you do it with the imports you don't need anything else to undertand if you're running on ptyhon 2.7 or python3.6 | 18:11 |
| panda | rlandy: yes, the import of mock is differnt if we are running on py27 or py36 | 18:11 |
| rlandy | k | 18:11 |
| *** chem has quit IRC | 18:31 | |
| *** chem has joined #oooq | 18:33 | |
| rlandy | panda: still around? | 19:10 |
| *** holser has joined #oooq | 19:17 | |
| *** amoralej is now known as amoralej|off | 19:21 | |
| *** holser__ has joined #oooq | 19:21 | |
| *** holser has quit IRC | 19:24 | |
| rfolco | rlandy, did you fix promotion tests so I can rebase my patch ? | 19:28 |
| rlandy | rfolco: getting there - one more to go | 19:28 |
| rlandy | three are passing | 19:28 |
| rfolco | rlandy, nice | 19:29 |
| rlandy | http://logs.rdoproject.org/58/22958/23/check/tripleo-ci-promotion-staging/eeac4dc/job-output.txt | 19:29 |
| rlandy | I need to add something to fixture | 19:29 |
| *** ykarel|away has quit IRC | 19:34 | |
| rlandy | rfolco: one failure left - same on all tests | 19:43 |
| rlandy | need a failing promotion in the fixtures | 19:43 |
| rfolco | rlandy, you also need to run tox -e py27,py36 for next ps | 19:49 |
| rlandy | rfolco: which next ps? | 19:56 |
| rfolco | rlandy, your next one :) | 19:56 |
| rfolco | rlandy, py2,py3 tox failing on your latest ps | 19:56 |
| rlandy | rfolco: it's all the same error | 19:59 |
| rlandy | > failed_attempt = stage_info['promotions']['failed_attempt'] | 20:01 |
| rlandy | E KeyError: 'failed_attempt' | 20:01 |
| *** rnoriega- has quit IRC | 20:03 | |
| *** holser__ has quit IRC | 20:08 | |
| panda | rlandy: rfolco 4 attempts and dlrnapi is working perfectly ... maybe it's really just the port ... | 20:11 |
| rlandy | panda; nice ... also tests are down to one error | 20:12 |
| panda | rlandy: failed_attempt needs also to be added manually to stating environemtn ... lines 502-512 | 20:12 |
| rlandy | panda: pls see current patch | 20:12 |
| rlandy | https://review.rdoproject.org/r/#/c/22958/27/ci-scripts/dlrnapi_promoter/tests/staging-setup/staging_environment.py | 20:13 |
| rlandy | panda: ^^ better? | 20:13 |
| rlandy | once all is passing - will do style clean up | 20:13 |
| panda | rlandy: yep | 20:14 |
| *** dsneddon_ has quit IRC | 20:20 | |
| *** dsneddon_ has joined #oooq | 20:21 | |
| *** jfrancoa has quit IRC | 20:21 | |
| rlandy | panda: we're getting there ... http://logs.rdoproject.org/58/22958/27/check/tripleo-ci-promotion-staging/2322250/job-output.txt | 20:22 |
| rlandy | and a tox failure | 20:22 |
| *** slaweq has quit IRC | 20:33 | |
| weshay|ruck | rlandy, can you help me for a sec | 20:41 |
| weshay|ruck | ? | 20:41 |
| weshay|ruck | rlandy, you guys changed the login to the registry again | 20:41 |
| rlandy | weshay|ruck: ack | 20:41 |
| rlandy | weshay|ruck: we did | 20:41 |
| rlandy | rfolco: ^^? | 20:41 |
| weshay|ruck | https://meet.google.com/wbg-zpys-wsa | 20:42 |
| weshay|ruck | be right there | 20:42 |
| rfolco | rlandy, I did not change login | 20:43 |
| *** chem has quit IRC | 20:45 | |
| *** chem has joined #oooq | 20:48 | |
| *** Goneri has quit IRC | 20:55 | |
| *** aakarsh has quit IRC | 21:08 | |
| rlandy | TASK [Tag and push images to docker hub | 21:24 |
| rlandy | woohoo | 21:24 |
| rlandy | panda: around? | 21:35 |
| *** jtomasek has quit IRC | 21:54 | |
| *** jtomasek has joined #oooq | 21:55 | |
| *** jtomasek has quit IRC | 21:55 | |
| *** slaweq has joined #oooq | 22:11 | |
| *** slaweq has quit IRC | 22:16 | |
| *** chem has quit IRC | 22:25 | |
| *** chem has joined #oooq | 22:27 | |
| *** saneax has joined #oooq | 22:58 | |
| *** Vorrtex has quit IRC | 23:13 | |
| *** tosky has quit IRC | 23:40 | |
| *** matbu has quit IRC | 23:51 | |
| *** matbu has joined #oooq | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!