| *** rlandy has quit IRC | 00:17 | |
| *** agopi is now known as agopi|off | 00:19 | |
| *** agopi|off has quit IRC | 00:24 | |
| *** agopi has joined #oooq | 00:59 | |
| hubbot` | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, tripleo-ci-centos-7-scenario008-multinode-oooq-container @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci- (1 more message) | 01:01 |
|---|---|---|
| *** atoth has quit IRC | 01:52 | |
| *** vinaykns has quit IRC | 02:38 | |
| *** ykarel|away has joined #oooq | 02:59 | |
| hubbot` | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, tripleo-ci-centos-7-scenario008-multinode-oooq-container @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci- (1 more message) | 03:01 |
| *** holser_ has joined #oooq | 03:12 | |
| *** skramaja has joined #oooq | 03:20 | |
| *** apetrich has quit IRC | 03:23 | |
| *** apetrich has joined #oooq | 03:24 | |
| *** udesale has joined #oooq | 03:35 | |
| *** ykarel|away has quit IRC | 03:50 | |
| *** jaganathan has joined #oooq | 03:56 | |
| *** jaganathan has quit IRC | 03:58 | |
| *** jaganathan has joined #oooq | 03:58 | |
| *** links has joined #oooq | 04:02 | |
| *** ykarel has joined #oooq | 04:28 | |
| *** holser_ has quit IRC | 04:30 | |
| *** holser_ has joined #oooq | 04:48 | |
| hubbot` | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata, legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, tripleo-ci-centos-7-scenario008-multinode-oooq-container @ https://review.openstack.org/560445, stable/queens: tripleo-ci- (1 more message) | 05:01 |
| *** holser_ has quit IRC | 05:05 | |
| *** rfolco has quit IRC | 05:30 | |
| *** ratailor has joined #oooq | 05:42 | |
| *** bogdando has joined #oooq | 05:56 | |
| *** rfolco has joined #oooq | 06:04 | |
| *** holser_ has joined #oooq | 06:04 | |
| *** quiquell|off is now known as quiquell | 06:05 | |
| *** ccamacho has joined #oooq | 06:12 | |
| *** ccamacho has quit IRC | 06:12 | |
| *** ccamacho has joined #oooq | 06:13 | |
| *** saneax has joined #oooq | 06:14 | |
| quiquell | sshnaidm|afk: Remove old RDO script from RR Cockpit https://review.rdoproject.org/r/#/c/14821 | 06:15 |
| *** quiquell is now known as quiquell|bbl | 06:15 | |
| *** jfrancoa has joined #oooq | 06:24 | |
| quiquell|bbl | chkumar|ruck: Removed old RDO jobs from t he RR Cockpit | 06:36 |
| quiquell|bbl | chkumar|ruck: It was adding noice to noop check jobs results | 06:36 |
| quiquell|bbl | chkumar|ruck: It will go to normal in a few | 06:36 |
| quiquell|bbl | chkumar|ruck: Checking the skipped promotion failing jobs | 06:36 |
| *** gvrangan has joined #oooq | 06:44 | |
| *** dtrainor has quit IRC | 06:46 | |
| *** dtrainor has joined #oooq | 06:46 | |
| quiquell|bbl | chkumar|ruck: Skipped promotion failing job, fixed | 06:49 |
| quiquell|bbl | chkumar|ruck: Now you have them | 06:49 |
| quiquell|bbl | Now I leave | 06:49 |
| *** holser_ has quit IRC | 06:53 | |
| chkumar|ruck | quiquell|bbl: ack, thanks :-) | 06:58 |
| hubbot` | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata, legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, tripleo-ci-centos-7-scenario008-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata-branch @ (1 more message) | 07:01 |
| *** gkadam has joined #oooq | 07:01 | |
| *** tesseract has joined #oooq | 07:12 | |
| *** holser_ has joined #oooq | 07:12 | |
| *** quiquell|bbl is now known as quiquell | 07:15 | |
| *** amoralej|off is now known as amoralej | 07:26 | |
| chkumar|ruck | quiquell: which FS is used here https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-promote-master-current-tripleo-delorean-minimal-370/console.txt.gz? | 07:27 |
| quiquell | chkumar|ruck: 2018-07-12 16:42:08.812 | + ansible-playbook -vv /home/jenkins/workspace/tripleo-quickstart-promote-master-current-tripleo-delorean-minimal/playbooks/noop.yml -e @/home/jenkins/workspace/tripleo-quickstart-promote-master-current-tripleo-delorean-minimal/config/release/queens.yml -e | 07:30 |
| quiquell | @/home/jenkins/workspace/tripleo-quickstart-promote-master-current-tripleo-delorean-minimal/config/nodes/1ctlr_1comp.yml -e @/home/jenkins/workspace/tripleo-quickstart-promote-master-current-tripleo-delorean-minimal/config/general_config/minimal.yml -e @/home/jenkins/workspace/tripleo-quickstart-promote-master-current-tripleo-delorean-minimal/config/environments/default_libvirt.yml -e | 07:30 |
| quiquell | local_working_dir=/home/jenkins/workspace/tripleo-quickstart-promote-master-current-tripleo-delorean-minimal -e virthost=localhost -t untagged,provision,environment,libvirt,undercloud-scripts,undercloud-inventory,overcloud-scripts,undercloud-setup,undercloud-install,undercloud-post-install,tripleoui-validate,teardown-nodes | 07:30 |
| quiquell | chkumar|ruck: @/home/jenkins/workspace/tripleo-quickstart-promote-master-current-tripleo-delorean-minimal/config/general_config/minimal.yml | 07:30 |
| quiquell | minimal.yaml | 07:30 |
| chkumar|ruck | quiquell: thanks :-) | 07:30 |
| quiquell | chkumar|ruck: The trick, it's search from ansible-playbooks execut ion | 07:31 |
| chkumar|ruck | cool! | 07:32 |
| quiquell | marios: I have added the toci_jobtype variables here https://review.openstack.org/#/c/582465/ | 07:46 |
| quiquell | marios: So we can have them in our patches | 07:46 |
| quiquell | marios: Parenting them with this | 07:47 |
| *** ykarel is now known as ykarel|lunch | 07:47 | |
| marios | quiquell: ack thanks | 07:52 |
| quiquell | marios: At the end rlandy is the QE of bootstrap | 07:54 |
| marios | quiquell: ok np | 07:55 |
| quiquell | marios: I was missing the question mark | 07:56 |
| quiquell | marios: Asking about | 07:56 |
| marios | quiquell: oh, well yeah i think thats what happened in the call right? we can check trello | 07:57 |
| quiquell | marios: Going to change the card then | 07:57 |
| *** kopecmartin has joined #oooq | 07:57 | |
| marios | quiquell: yeah rlandy is qe on it | 07:58 |
| *** holser_ has quit IRC | 07:58 | |
| quiquell | marios: ok changed | 07:58 |
| *** tosky has joined #oooq | 08:06 | |
| *** holser_ has joined #oooq | 08:25 | |
| arxcruz | panda|off: sshnaidm|afk https://review.openstack.org/#/c/572347/ when you guys have time | 08:31 |
| arxcruz | panda|off: also, just adding -e PYTHONWARNINGS didn't work | 08:32 |
| *** ykarel|lunch is now known as ykarel | 08:32 | |
| *** gvrangan has quit IRC | 08:33 | |
| *** florianf has joined #oooq | 08:41 | |
| *** gvrangan has joined #oooq | 08:53 | |
| *** sshnaidm|afk is now known as sshnaidm|off | 09:00 | |
| hubbot` | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata, legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, tripleo-ci-centos-7-scenario008-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata-branch @ (1 more message) | 09:01 |
| *** sshnaidm|off has quit IRC | 09:05 | |
| *** udesale has quit IRC | 09:27 | |
| *** udesale has joined #oooq | 09:27 | |
| amoralej | do i need to recheck https://review.openstack.org/#/c/582327/ ? | 09:55 |
| amoralej | are ocata jobs fixed about md5? | 09:55 |
| *** sshnaidm|off has joined #oooq | 09:55 | |
| chkumar|ruck | sshnaidm|off: Can we push undercloud image manually for ocata promotion? | 10:00 |
| *** panda|off is now known as panda | 10:06 | |
| panda | arxcruz: so docker run doesn't follow its own reference ? | 10:08 |
| arxcruz | panda: nope, i have a depends-on patch that fails, i update the patch and i'm waiting the results | 10:09 |
| quiquell | panda: what do you think about this ? https://review.openstack.org/#/c/582465/ | 10:23 |
| panda | quiquell: don't like the names | 10:23 |
| panda | quiquell: I would use environemt_type instead of deploy type, and nodes, or nodes_conf instead of topology | 10:25 |
| panda | quiquell: they are not "better" just consistent we what we have on other places | 10:25 |
| quiquell | panda: will change, but the idea of merge inoffensive variables can help ? | 10:34 |
| *** sanjayu_ has joined #oooq | 10:34 | |
| panda | quiquell: seems an excessively small step, | 10:36 |
| quiquell | panda: It will align the variable names at code | 10:36 |
| *** saneax has quit IRC | 10:38 | |
| ssbarnea | https://review.openstack.org/#/c/582501/ | 10:41 |
| quiquell | marios, panda: jinja templates are good to merge now, https://review.openstack.org/#/c/581331/ | 10:53 |
| *** dtrainor_ has joined #oooq | 10:55 | |
| *** dtrainor has quit IRC | 10:56 | |
| quiquell | chkumar|ruck: Do you know if this is related to any bug ? http://logs.openstack.org/53/579653/8/check/tripleo-ci-centos-7-scenario008-multinode-oooq-container/b143744/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2018-07-13_08_31_24 | 10:57 |
| chkumar|ruck | quiquell: it is new, not seen | 10:59 |
| chkumar|ruck | quiquell: it is failing right now in the last patchset | 11:01 |
| hubbot` | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata @ https://review.openstack.org/564291, master: legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, tripleo-ci-centos-7-scenario008-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata-branch @ (1 more message) | 11:01 |
| quiquell | chkumar|ruck: Maybe is a problem with my patch | 11:01 |
| chkumar|ruck | quiquell: i am not sure is it related to multi-nic part on the dependent patch?\ | 11:06 |
| marios | quiquell: ack added for next reviews run | 11:08 |
| * marios school pickup kid biab | 11:08 | |
| quiquell | marios: thanks, run run | 11:10 |
| *** arxcruz is now known as arxcruz|brb | 11:13 | |
| *** udesale has quit IRC | 11:22 | |
| chkumar|ruck | panda: can we get some help on this https://logs.rdoproject.org/98/582198/1/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata-branch/ca234e3/job-output.txt.gz#_2018-07-13_09_24_13_583054 | 11:26 |
| chkumar|ruck | ? | 11:26 |
| quiquell | chkumar|ruck: scenario008 is failing at noop http://logs.openstack.org/98/582198/1/check/tripleo-ci-centos-7-scenario008-multinode-oooq-container/21451d9/logs/undercloud/home/zuul/undercloud_install.log.txt.gz | 11:27 |
| quiquell | The patch is good, it's also a non voting | 11:27 |
| chkumar|ruck | quiquell: this one is failing from very long time\ | 11:29 |
| quiquell | chkumar|ruck: ok ok | 11:29 |
| quiquell | Then the patch is good | 11:29 |
| chkumar|ruck | quiquell: yup | 11:30 |
| quiquell | panda: Jinja2 template is good, this is the test patch https://review.openstack.org/#/c/579653/ | 11:32 |
| quiquell | panda: Do we want to merge ? | 11:32 |
| *** sanjayu_ has quit IRC | 11:48 | |
| *** sanjayu_ has joined #oooq | 11:49 | |
| panda | chkumar|ruck: you need help in root cause analisys or in fixing ? | 11:50 |
| *** atoth has joined #oooq | 11:51 | |
| *** jaganathan has quit IRC | 11:56 | |
| *** brault has joined #oooq | 12:03 | |
| *** brault has quit IRC | 12:03 | |
| *** brault has joined #oooq | 12:04 | |
| *** agopi is now known as agopi|brb | 12:08 | |
| *** ratailor has quit IRC | 12:10 | |
| ssbarnea | is it normal for reproducer-quickstart.sh to fail when run as non-root account? | 12:11 |
| ssbarnea | in fact is not the script that fails, but the task fails trying to resize the image. | 12:11 |
| ssbarnea | when i tried the same as root it it pass | 12:12 |
| quiquell | ssbarnea: Nope, it should work | 12:12 |
| quiquell | ssbarnea: Where are you running it ? | 12:12 |
| ssbarnea | on my own centos7machine, using LIBVIRT=1 | 12:13 |
| *** agopi|brb has quit IRC | 12:13 | |
| ssbarnea | i checked and the downloaded image was owned by root so resize failed, thus why passed when run as root. | 12:14 |
| ssbarnea | this is also the reason why i raised the other CR because resize failure was opaque, with no hints on why it failed. | 12:14 |
| *** brault has quit IRC | 12:16 | |
| *** brault has joined #oooq | 12:17 | |
| *** panda is now known as panda|lunch | 12:18 | |
| chkumar|ruck | panda|lunch: we need to push undercloud image manually | 12:19 |
| chkumar|ruck | for ocata to image server otherwise it will again fail | 12:20 |
| chkumar|ruck | panda|lunch: last night sshnaidm|off weshay pushed the image manually for queens and master | 12:20 |
| chkumar|ruck | we need the same for ocata otherwise checksum task will fail\ | 12:20 |
| quiquell | rfolco: new variable name as panda suggested, https://review.openstack.org/#/c/582465/ | 12:21 |
| rfolco | quiquell, this is part of toci_jobtype work, https://review.openstack.org/#/q/topic:toci_jobtype+(status:open+OR+status:merged) | 12:22 |
| weshay | chkumar|ruck, was hoping that ocata would just promote again | 12:23 |
| quiquell | rfolco: Have some idea, do you have some time for bj ? | 12:24 |
| weshay | chkumar|ruck, other than ocata | 12:25 |
| chkumar|ruck | weshay: https://bugs.launchpad.net/tripleo/+bug/1781541 | 12:25 |
| weshay | chkumar|ruck, why was scenario02 only failing part of the time if the url for the api was wrong | 12:25 |
| openstack | Launchpad bug 1781541 in tripleo "[master][promotion][RDO phase1] Creating overcloud Heat stack failed giving Error([('SSL routines', 'SSL23_GET_SERVER_HELLO', 'unknown protocol')]" [Critical,Triaged] | 12:25 |
| weshay | chkumar|ruck, ya.. I think https://review.openstack.org/#/c/582405/ | 12:26 |
| weshay | will fix that | 12:26 |
| rfolco | quiquell, yes one min, will get a coffee and switch rooms | 12:26 |
| chkumar|ruck | weshay: as per ykarel it might be the issue https://review.openstack.org/#/q/topic:microversion_mixin+(status:open+OR+status:merged) | 12:27 |
| quiquell | rfolco: Sure no problem. | 12:27 |
| chkumar|ruck | weshay: the question for scenario 02 how sometime it is getting the wrong url, I am tried reproducing locally but no able to reproduce | 12:28 |
| weshay | panda|lunch, https://review.openstack.org/#/c/582405/ | 12:28 |
| weshay | chkumar|ruck, that's wild | 12:28 |
| weshay | as christian is saying it's the url | 12:28 |
| chkumar|ruck | weshay: as per tempest quys they have never seeb 412 errors in ci | 12:29 |
| chkumar|ruck | weshay: i am not sure how we proceed here | 12:29 |
| rfolco | quiquell, https://bluejeans.com/5878458097 | 12:30 |
| *** sanjay__u has joined #oooq | 12:36 | |
| *** rlandy has joined #oooq | 12:38 | |
| rlandy | quiquell: hi- need your help with grafana | 12:39 |
| rlandy | we may need to add a dashboards for the pref team | 12:41 |
| rlandy | weshay: what is the status of rdo on rhel ... http://git.app.eng.bos.redhat.com/git/tripleo-environments.git/tree/config/release/ | 12:43 |
| rlandy | agopi pointed out that we had no queens-rhel file | 12:43 |
| weshay | rlandy, totally dead atm as there are no rdo containers built on rhel | 12:43 |
| weshay | rlandy, we need to get to that in internal sf | 12:43 |
| quiquell | rlandy: Tell me | 12:43 |
| rlandy | weshay:ok - I tell agopi when he comes on line | 12:44 |
| *** amoralej is now known as amoralej|lunch | 12:44 | |
| quiquell | rlandy: We can add it, no problem, we need to know the source of information and the like | 12:44 |
| rlandy | quiquell: hey ... so we are trying to get a perf/browbeat job running with ovb in rdocloud | 12:44 |
| weshay | rlandy, ya.. pike, queens, rocky rdo on rhel is dead atm | 12:44 |
| weshay | atleast w/ tripleo | 12:45 |
| quiquell | rlandy: zuulv3 ? | 12:45 |
| rlandy | quiquell: yep - we went thought the new fun and games to get a a project/job added - here is the thing ... | 12:45 |
| quiquell | rfolco: Before parenting, review that I didn't put any field wrong | 12:45 |
| rlandy | https://github.com/openstack/browbeat/blob/master/browbeat-config.yaml | 12:46 |
| rfolco | quiquell, reviewing thx | 12:46 |
| rlandy | browbeat need elastic search and grafan to log it's data | 12:46 |
| quiquell | rlandy: We are getting all the builds from tripleo projects, we can add new projects and all the info will be there | 12:46 |
| rlandy | quiquell: do we deal with the keys at all ( I forgot, been a while since I used it) | 12:47 |
| rlandy | or we get that for free with the infra | 12:47 |
| *** trown|outtypewww is now known as trown | 12:47 | |
| quiquell | rlandy: To inject new dashboard you need an API key with permissions | 12:48 |
| rlandy | quiquell: right now, they are logging everything to an internal server | 12:48 |
| rlandy | which iiuc, is not accessible from rdocloud | 12:48 |
| rlandy | which is public | 12:48 |
| quiquell | rlandy: We can generate one, and inject it at browbeat jobs | 12:48 |
| rlandy | so my thought was to get them on ours | 12:48 |
| quiquell | rlandy: But zuul encryption is kind of complicated stuff | 12:49 |
| quiquell | rlandy: I mean bureocratic complexity | 12:49 |
| quiquell | rlandy: bj ? | 12:49 |
| rlandy | quiquell: I know but somehow we are doing this, right? | 12:49 |
| rlandy | ok | 12:50 |
| rlandy | your bj? | 12:50 |
| quiquell | https://bluejeans.com/7891065232 | 12:51 |
| *** panda|lunch is now known as panda | 12:55 | |
| *** links has quit IRC | 12:59 | |
| hubbot` | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container @ https://review.openstack.org/567224, stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata @ https://review.openstack.org/564291, master: legacy-tripleo-ci- (1 more message) | 13:01 |
| chkumar|ruck | weshay: scenario 8 job (non-voting) failing constitly, do i need to report that? | 13:02 |
| weshay | ssbarnea, you have a link to Arie's dashboard for me? | 13:03 |
| ssbarnea | weshay: http://rhos-ci.eng.lab.tlv.redhat.com | 13:03 |
| weshay | thanks | 13:04 |
| panda | rlandy: have a question regarding ovb. do you know where we store the tripleo-cd-admins key so we can access overcloud nodes from the undercloud ? | 13:04 |
| rlandy | sec - in meeting | 13:05 |
| panda | rlandy: no rush | 13:05 |
| *** gvrangan has quit IRC | 13:12 | |
| ssbarnea | i am now trying to run reproducer using LIBVIRT=0 and with my rdo cloud credentials (rc3) but I get "ovb-manage-stack : Deploy stack" failing due to "keystoneauth1.exceptions.http.BadRequest: Expecting to find domain in project. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400)" | 13:16 |
| *** agopi|brb has joined #oooq | 13:16 | |
| *** agopi|brb is now known as agopi | 13:16 | |
| rfolco | quiquell, please see my comments at https://review.openstack.org/#/c/582465 | 13:17 |
| rfolco | quiquell, variables with leading 0, e.g. 010 needs to be '010' | 13:18 |
| quiquell | rfolco: ack, changing, vim was chaning the color for a reason :-) | 13:20 |
| *** brault has quit IRC | 13:28 | |
| quiquell | rfolco: done | 13:31 |
| rfolco | quiquell, just to prove what I said: https://paste.fedoraproject.org/paste/WpOFa6iTcfDISSh5w7q5nQ | 13:31 |
| quiquell | weshay, rlandy: jinja templates for toci scripts https://review.openstack.org/#/c/581331/ | 13:32 |
| quiquell | rfolco: the variable nodes are read at standalone ? | 13:32 |
| rfolco | quiquell, singlenode and standalone don't have nodes, it's correct | 13:33 |
| rfolco | quiquell, as long as I define default value for ansible var :) | 13:33 |
| quiquell | rfolco: Do they try to read NODES ? | 13:34 |
| rfolco | quiquell, if you look the original case loop, *) means all string that is not any of the previous ones will be handled as nodes... 1ctlr_1comp for example... | 13:35 |
| *** skramaja has quit IRC | 13:35 | |
| quiquell | rfolco: Yep but we do semething with if only if it's present | 13:35 |
| quiquell | rfolco: we can do this conditional with ansible templates too | 13:35 |
| rfolco | quiquell, I am handling this in my patch | 13:35 |
| quiquell | rfolco: no need for default | 13:35 |
| quiquell | rfolco: at singlenode, standalone NODES_FILE is not going to be set | 13:37 |
| quiquell | rfolco: This is the same as saying that ansible varibale nodes is not set | 13:37 |
| rfolco | quiquell, yes | 13:37 |
| quiquell | rfolco: We can work with that. | 13:37 |
| quiquell | rfolco: Going for lunch now | 13:37 |
| *** quiquell is now known as quiquell|lunch | 13:38 | |
| rfolco | enjoy | 13:38 |
| quiquell|lunch | rfolco: thanks | 13:38 |
| panda | "git pill" is a nice type. For when you have stressful day. | 13:48 |
| panda | typo | 13:48 |
| ssbarnea | apparently there is no code in reproducer for getting ZUUL_CHANGES for tripleo-quickstart(-extras) repos, it just clones the master, i do not see any cherry picking after. | 13:51 |
| quiquell|lunch | weshay: Can you merge this RR Cockpit patches ? https://review.rdoproject.org/r/#/q/topic:rr-cockpit | 13:54 |
| rlandy | panda: sorry - forgot to answer your question above ... | 13:55 |
| rlandy | there are two sets of secrets or keys to worry about - | 13:56 |
| rlandy | the ones for the tenant in /etc/nodepoolrc | 13:56 |
| rlandy | and then https://github.com/openstack-infra/tripleo-ci/blob/master/scripts/te-broker/create-env#L31 | 13:57 |
| rlandy | ^^ I think you are asking about that | 13:57 |
| rlandy | agopi: spoke to quiquell|lunch about grafana access - can take you through that discussion later ... also | 13:58 |
| agopi | thanks rlandy++ | 13:59 |
| hubbot` | agopi: rlandy's karma is now 12 | 13:59 |
| rlandy | agopi: wrt quuens-rhel file | 13:59 |
| rlandy | queens-rhel | 13:59 |
| rlandy | <weshay> rlandy, totally dead atm as there are no rdo containers built on rhel | 13:59 |
| rlandy | <weshay> rlandy, we need to get to that in internal sf | 13:59 |
| agopi | i am making changes os it wont be necessary rlandy, i am trying to keep perfci from diverging a lot. | 14:00 |
| rlandy | agopi: I have a meeting with weshay at 11 - where I want to discuss a few options wrt kicking the playbook in post maybe. I will ping you after that | 14:00 |
| panda | rlandy: clarification. After te-broker creates the test environment using the keypair specified in the configuration, ansible needs ssh access to the undercloud. Where is the private key for the keypair needed to access the undercloud that the te-broker has provisioned, injecting the public part ? | 14:00 |
| agopi | okay rlandy | 14:01 |
| chkumar|ruck | weshay: arxcruz|brb I am heading home , see ya on monday , happy weekend :-) | 14:01 |
| *** vinaykns has joined #oooq | 14:04 | |
| rlandy | panda: idk - off hand - looking | 14:05 |
| rlandy | I remember it somewhere :( | 14:05 |
| *** ykarel is now known as ykarel|away | 14:05 | |
| quiquell|lunch | weshay: Now they are rebased, need +1w https://review.rdoproject.org/r/#/q/topic:rr-cockpit | 14:06 |
| weshay | quiquell|lunch, you need to update your VI or editor to remove white space man | 14:10 |
| weshay | ;) | 14:10 |
| weshay | those reviews look terrible :) | 14:10 |
| quiquell|lunch | weshay: json from grafana, is generated by grafana | 14:11 |
| quiquell|lunch | weshay: I can change the python that output it, it's not a vim problem | 14:11 |
| quiquell|lunch | weshay: If you are talking about python, we need linting on them | 14:11 |
| quiquell|lunch | weshay: Will add it | 14:11 |
| *** sanjayu__ has joined #oooq | 14:16 | |
| *** sanjayu_ has quit IRC | 14:19 | |
| *** quiquell|lunch is now known as quiquell | 14:21 | |
| ssbarnea | i observe that there is no irc notification on new reviews, is this by purpose or a bug? | 14:22 |
| ssbarnea | i hope i will not nag you too much with lots of proposed changes to reproducer. | 14:24 |
| weshay | ssbarnea, that's all in #tripleo | 14:24 |
| *** bogdando has quit IRC | 14:25 | |
| weshay | quiquell, ah.. you were the meeting that I lost on my phone | 14:29 |
| weshay | quiquell, too late for you? | 14:29 |
| quiquell | weshay: What meeting ? | 14:29 |
| weshay | we had a 1-1 | 14:29 |
| quiquell | Ups... | 14:29 |
| ssbarnea | weshay: ok. so keeping spam there. btw, I am pleased to see your CR, this subshell was likely to cause confusions. | 14:29 |
| quiquell | weshay: have time | 14:29 |
| quiquell | Give me a min | 14:29 |
| weshay | k.. I'll go to my bat channel | 14:29 |
| weshay | ssbarnea, ya. that subshell sucked | 14:29 |
| ssbarnea | https://review.openstack.org/#/c/582599/ -- similar | 14:30 |
| ssbarnea | if anyone is curious about the magic of PIP_USER variable I am ready to share it. | 14:31 |
| quiquell | weshay: I am at your channel | 14:31 |
| ssbarnea | mainly it adds --user to pip install, which is useful if you are outside an virtualenv. | 14:31 |
| ssbarnea | there is one feature that I want to add to reproduce to make it optionally skip the repo removal and re-cloning on each run, based on an environ variable. very useful when you start to hack it and you don't want to loose your local changes on each run. | 14:34 |
| *** ykarel|away is now known as ykarel | 14:39 | |
| *** sanjayu__ has quit IRC | 14:48 | |
| rlandy | panda: I think I found it - look at the log on te-broker | 14:52 |
| rlandy | sed -i -e 's/key_name:.*/key_name: tripleo-cd-admins/' env-6347-base.yaml | 14:52 |
| *** amoralej|lunch is now known as amoralej | 14:52 | |
| *** brault has joined #oooq | 14:55 | |
| rlandy | marios; hi | 14:58 |
| marios | rlandy: o/ | 14:58 |
| rlandy | marios; wanted to chat with you about the investigation you did into the 3 node failure | 14:59 |
| rlandy | it's reproducible :( | 14:59 |
| rlandy | so probably a real problem | 14:59 |
| marios | rlandy: sure, but didn't really investigate much :) it was just yesterday and added the sum of my findings on the review comment | 14:59 |
| marios | rlandy: i didn't cycle back to that today to be honest | 14:59 |
| rlandy | I have my 1-on-1 meeting now - how much longer are you still on line? | 14:59 |
| marios | rlandy: ack, i can dig some more on monday. panda was also asking me about this | 14:59 |
| marios | rlandy: not long, i've got to leave a little early today (is 1800 here) | 15:00 |
| rlandy | np - I will pick it up after the meeting and see how far I get | 15:00 |
| marios | rlandy: k, can you maybe add comment on the review ? | 15:00 |
| rlandy | k | 15:00 |
| *** atoth has quit IRC | 15:00 | |
| marios | rlandy: if you find somethihg. or we should probably file the bug at some point but revieww orks for now. so i can check on monday | 15:00 |
| rlandy | marios: yeah - I have NO idea why this is happening | 15:01 |
| hubbot` | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container @ https://review.openstack.org/567224, stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata @ https://review.openstack.org/564291, master: legacy-tripleo-ci- (1 more message) | 15:01 |
| rlandy | the reparent hits overcloud deploy, really> | 15:01 |
| marios | rlandy: same... i mean grep got me as far as where it was happening but not closer to why yet | 15:01 |
| marios | rlandy: also the filename is coincidental as ther is no upgrade here | 15:02 |
| marios | rlandy: but not sure if it is somehow specific to that file? (was it the same on reproduced?) | 15:02 |
| *** quiquell is now known as quiquell|off | 15:08 | |
| *** vkapalav has joined #oooq | 15:08 | |
| *** brault has quit IRC | 15:09 | |
| *** brault has joined #oooq | 15:10 | |
| *** vkapalav has quit IRC | 15:10 | |
| *** vinaykns has quit IRC | 15:11 | |
| panda | rfolco: in internal sf, we have core access to the trusted project ? | 15:13 |
| rfolco | panda, core yes, admin no | 15:13 |
| *** brault has quit IRC | 15:14 | |
| rfolco | you can create a new repo and add you as admin of that repo to force merge when ci fails for example.. but this is for gerrit side | 15:15 |
| *** yolanda__ has joined #oooq | 15:16 | |
| panda | rfolco: no I need a project where I can use secrets, and I can merge my own patches without waiting too much to iterate on my code | 15:16 |
| panda | rlandy: ovb-delete-stack currently delete every stack, or a particular stack | 15:18 |
| panda | ? | 15:18 |
| rfolco | panda, I did not play much with secrets but this sounds perfectly doable | 15:18 |
| *** EmilienM is now known as EvilienM | 15:18 | |
| *** yolanda_ has quit IRC | 15:18 | |
| rfolco | merging own code --> thats what i did ... and thats how I broke zuul a few times | 15:19 |
| rfolco | :) | 15:19 |
| panda | rfolco: also I really need to test a base job, so no inheritance. I just need secrets, tq and tq | 15:19 |
| panda | rfolco: this is what I need to test https://review.rdoproject.org/r/14781 | 15:20 |
| *** tesseract has quit IRC | 15:21 | |
| panda | rfolco: what do I need to change to make it work there ? | 15:21 |
| panda | rfolco: well, besides the secrets, they are instance and project specific | 15:22 |
| ssbarnea | who is in charge of ci.centos.org? asking as i want to be able to browse the logs. | 15:22 |
| *** ykarel is now known as ykarel|away | 15:23 | |
| panda | ssbarnea: centos administrators, we have an account there and some jobs, but not much control | 15:23 |
| ssbarnea | panda: well an emaila address or issue tracker would be more than enough. | 15:24 |
| ssbarnea | i don't plan to ssh to the machine and correct the webserver config myself :D | 15:25 |
| *** arxcruz|brb is now known as arxcruz | 15:26 | |
| rfolco | panda, looking at the patch I believe we can just try it on SF and hopefully we don't need much iteration with sf dfg. I can try it if you want. I just have a doctor app. now, but would be happy to test it for you if not urgent. | 15:28 |
| panda | ssbarnea: you can ask in internal #centos | 15:28 |
| panda | rfolco: this is the core of my card to replace te-broker | 15:29 |
| panda | rfolco: I can't test it derectly in rdo because it needs to be merged before, and for every change I have to ask a review and a merge | 15:29 |
| panda | rfolco: you have the appointment now, or can spare some time pointing me at what to do ? | 15:30 |
| rfolco | panda, must be real quick I need to go in 10 min | 15:30 |
| panda | rfolco: also, I'll be here for another 2:30 hours probably | 15:30 |
| panda | rfolco: 5 min chat then | 15:31 |
| panda | rfolco: bj/gcerami | 15:31 |
| *** jfrancoa has quit IRC | 15:36 | |
| *** agopi is now known as agopi|brb | 15:40 | |
| rlandy | panda: ovb-delete-stack delets a particular stack | 15:40 |
| panda | rlandy: ok, thanks | 15:40 |
| rlandy | panda: the clean up option on the reproducer deletes all stacks | 15:41 |
| rlandy | ^^ is a script not one ansible call | 15:41 |
| weshay | panda, shall we merge? https://review.openstack.org/#/c/581331/15 | 15:42 |
| panda | weshay: yeah, let's merge this on a Friday evening | 15:44 |
| panda | what could possibly go wrong :) | 15:45 |
| weshay | panda, I'm working the weekend | 15:45 |
| weshay | I'll revert it if needed | 15:45 |
| panda | +1W! | 15:45 |
| weshay | rlandy, panda https://review.openstack.org/#/c/582581/ | 15:46 |
| weshay | need reviews on that asap | 15:47 |
| rlandy | looking | 15:48 |
| panda | weshay: are these mirrored anywhere near openstack infra ? | 15:48 |
| weshay | panda, we don't use images in openstack infra my man | 15:49 |
| *** kopecmartin has quit IRC | 15:49 | |
| weshay | panda, this is for things outside of openstack infra | 15:49 |
| weshay | panda, no there is no mirror | 15:49 |
| weshay | rh1 is dead | 15:49 |
| panda | weshay: yeah, my fingers are quicker than my brain | 15:50 |
| panda | whay were we still using that ... :/ | 15:50 |
| weshay | panda, fak if I know | 15:50 |
| rlandy | agopi|brb: spoke with weshay - decision is to leave uploading data to grafana for now | 15:52 |
| rlandy | make sure you can collect the data you need | 15:52 |
| rlandy | and that the role does not fail if grafana does not succeed | 15:52 |
| *** ykarel|away has quit IRC | 15:52 | |
| rlandy | if you actually want to up load the data, you can consider that with rdo-infra afterwards | 15:53 |
| panda | weshay: workflowed | 15:55 |
| weshay | panda, rock on | 15:58 |
| *** ccamacho has quit IRC | 16:05 | |
| *** atoth has joined #oooq | 16:07 | |
| *** holser_ has quit IRC | 16:24 | |
| *** amoralej is now known as amoralej|off | 16:30 | |
| panda | rfolco: if you take a look at the change when you come back, I keep getting an error that has nothing to do with my change, and I don't know how to fix it. | 16:38 |
| *** gkadam has quit IRC | 16:55 | |
| hubbot` | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata @ https://review.openstack.org/564291, master: legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, tripleo-ci-centos-7-scenario008-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata-branch @ (1 more message) | 17:01 |
| *** panda is now known as panda|off | 17:23 | |
| rlandy | panda|off: marios: this may be something: "lookup isn't honouring the become=yes value" permissions on the set up are probably diff | 17:34 |
| *** jaganathan has joined #oooq | 18:00 | |
| rlandy | panda|off: marios: ah ah - found it - the nodes are named differently | 18:02 |
| *** jaganathan has quit IRC | 18:03 | |
| *** trown is now known as trown|lunch | 18:06 | |
| *** agopi|brb is now known as agopi | 18:35 | |
| *** gkadam has joined #oooq | 18:50 | |
| rlandy | weshay: do we have a usual mistral contact? | 18:54 |
| rlandy | debugging https://trello.com/c/95AvKjpX/845-migrate-3-and-4-nodes-jobs-to-tripleo-ci-base-parent | 18:54 |
| rlandy | see last comment | 18:54 |
| weshay | rlandy, rlandy apetrich or toure | 18:54 |
| rlandy | thanks | 18:55 |
| hubbot` | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata @ https://review.openstack.org/564291, master: legacy-tripleo-ci-centos-7-container-to-container-upgrades-master, tripleo-ci-centos-7-scenario008-multinode-oooq-container, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata-branch @ (1 more message) | 19:01 |
| *** gkadam has quit IRC | 19:17 | |
| *** sshnaidm|off has quit IRC | 19:40 | |
| *** trown|lunch is now known as trown | 19:49 | |
| *** florianf has quit IRC | 20:47 | |
| rlandy | hmmm .. did logs just go down? | 20:55 |
| rlandy | http://logs.openstack.org unreachable? | 20:56 |
| hubbot` | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container @ https://review.openstack.org/567224, stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata @ https://review.openstack.org/564291, master: legacy-tripleo-ci- (1 more message) | 21:01 |
| rlandy | anyone out there??? | 21:01 |
| agopi | yes rlandy | 21:05 |
| agopi | doesnt work for me too | 21:05 |
| *** trown is now known as trown|outtypewww | 21:09 | |
| weshay | weee | 21:15 |
| *** rfolco_ has joined #oooq | 21:27 | |
| *** rfolco has quit IRC | 21:27 | |
| *** sshnaidm|off has joined #oooq | 21:27 | |
| *** agopi is now known as agopi|off | 21:41 | |
| *** agopi|off has quit IRC | 21:45 | |
| -openstackstatus- NOTICE: logs.openstack.org is offline, causing POST_FAILURE results from Zuul. Cause and resolution timeframe currently unknown. | 21:47 | |
| *** rlandy has quit IRC | 22:11 | |
| *** agopi has joined #oooq | 22:50 | |
| hubbot` | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container @ https://review.openstack.org/567224, stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata, legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-ocata @ https://review.openstack.org/564291, master: legacy-tripleo-ci- (1 more message) | 23:01 |
| -openstackstatus- NOTICE: logs.openstack.org is back on-line. Changes with "POST_FAILURE" job results should be rechecked. | 23:32 | |
| *** tosky has quit IRC | 23:41 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!