| *** dpawlik has quit IRC | 00:01 | |
| *** dpeacock has quit IRC | 00:03 | |
| rdogerrit | rdo-trunk proposed rdoinfo master: Bump rdoinfo stein-uc tags to latest upper-constraints https://review.rdoproject.org/r/16237 | 00:03 |
|---|---|---|
| *** dpeacock has joined #rdo | 00:07 | |
| *** openstackgerrit has quit IRC | 00:09 | |
| *** cyberworm54 has quit IRC | 00:23 | |
| *** cyberworm54 has joined #rdo | 00:31 | |
| *** cyberworm54 has quit IRC | 00:46 | |
| *** chasmo77 has joined #rdo | 00:55 | |
| *** cyberworm54 has joined #rdo | 01:12 | |
| *** cyberworm54 has quit IRC | 01:26 | |
| *** fpan has joined #rdo | 01:45 | |
| *** haleyb has quit IRC | 01:48 | |
| *** linuxaddicts has joined #rdo | 02:56 | |
| *** mrunge_ has joined #rdo | 02:57 | |
| *** mrunge has quit IRC | 02:57 | |
| *** Son_Goku has quit IRC | 03:02 | |
| *** ykarel has joined #rdo | 03:31 | |
| *** udesale has joined #rdo | 03:34 | |
| *** ramishra has joined #rdo | 03:40 | |
| *** Son_Goku has joined #rdo | 03:43 | |
| *** oanson has quit IRC | 03:44 | |
| rdogerrit | Yatin Karel proposed rdo-infra/weirdo master: Add support for scenario-py3 for puppet-openstack https://review.rdoproject.org/r/14921 | 03:46 |
| *** dpawlik has joined #rdo | 03:57 | |
| *** dpawlik has quit IRC | 04:02 | |
| *** udesale has quit IRC | 04:03 | |
| *** udesale has joined #rdo | 04:24 | |
| *** udesale has quit IRC | 04:25 | |
| *** udesale has joined #rdo | 04:25 | |
| *** khyr0n has joined #rdo | 04:29 | |
| *** Son_Goku has quit IRC | 04:51 | |
| *** ykarel has quit IRC | 05:22 | |
| *** jbadiapa has joined #rdo | 05:25 | |
| *** quique|rover|off is now known as quiquell|rover | 05:30 | |
| *** janki has joined #rdo | 05:32 | |
| *** jtomasek has joined #rdo | 05:34 | |
| *** ykarel has joined #rdo | 05:42 | |
| *** jtomasek has quit IRC | 05:44 | |
| *** jtomasek has joined #rdo | 05:45 | |
| *** sbonazzo has joined #rdo | 05:48 | |
| *** ksambor_ has joined #rdo | 05:53 | |
| *** quiquell|rover has quit IRC | 05:56 | |
| *** quiquell has joined #rdo | 05:56 | |
| *** quiquell is now known as quiquell|rover | 05:56 | |
| *** iranzo has joined #rdo | 05:58 | |
| *** quiquell|rover is now known as quique|rover | 06:01 | |
| *** quique|rover is now known as quiquell|rover | 06:01 | |
| *** holser|ptg has joined #rdo | 06:03 | |
| *** agurenko has joined #rdo | 06:04 | |
| *** holser|p_ has joined #rdo | 06:05 | |
| *** jfrancoa has joined #rdo | 06:05 | |
| *** holser|ptg has quit IRC | 06:05 | |
| *** dpawlik has joined #rdo | 06:07 | |
| chkumar|ruck | sileht: I have opened a bz for telemetry tests failure https://bugs.launchpad.net/tripleo/+bug/1792862 | 06:09 |
| openstack | Launchpad bug 1792862 in tripleo "[master] Telemetry Tempest integration tests failed giving Unable to complete operation on subnet b13c76ec-c851-48d0-91a9-245a2fdcad9b: One or more ports have an IP allocation from this subnet." [Critical,Triaged] - Assigned to Mehdi Abaakouk (sileht) | 06:09 |
| quiquell|rover | ykarel: How do we do the pinning of ceph-ansible ? | 06:16 |
| quiquell|rover | we need to use newer version | 06:16 |
| *** rnoriega has quit IRC | 06:17 | |
| *** cylopez has joined #rdo | 06:18 | |
| *** rdopiera has joined #rdo | 06:18 | |
| *** apetrich has joined #rdo | 06:18 | |
| *** rnoriega has joined #rdo | 06:20 | |
| ykarel | quiquell|rover, i think ceph-ansible is comming from ceph repo | 06:21 |
| quiquell|rover | ykarel: ack, will look at it thanks | 06:23 |
| *** aufi has joined #rdo | 06:28 | |
| quiquell|rover | ykarel: It's in centos https://cbs.centos.org/koji/packageinfo?packageID=3726 | 06:32 |
| quiquell|rover | ykarel: Why aren't we installing the new one ? | 06:33 |
| ykarel | quiquell|rover, see tags on the version u are looking for | 06:33 |
| ykarel | and the repo that is enabled | 06:33 |
| *** quiquell|rover is now known as quique|rover|brb | 06:37 | |
| *** udesale has quit IRC | 06:38 | |
| *** udesale has joined #rdo | 06:40 | |
| *** aufi has quit IRC | 06:41 | |
| *** udesale has quit IRC | 06:45 | |
| ykarel | https://ci.centos.org/job/weirdo-generic-puppet-openstack-scenario001/6286/ stuck for 2.5 days | 06:54 |
| ykarel | jpena|off, ^^ and on cloudslave04 :( and puppet ci not running due to it: https://ci.centos.org/view/rdo/view/weirdo-pipelines/view/weirdo-promote-puppet/job/weirdo-promote-test-puppet/ | 06:54 |
| egonzalez | ykarel hi, know when are the tempest-all and collectd-gnocchi packages promoted to main repository? | 07:00 |
| *** slaweq has joined #rdo | 07:00 | |
| ykarel | egonzalez, there is a patch for promotion: https://review.rdoproject.org/r/#/c/16218 | 07:01 |
| ykarel | once ^^ is merged, we can propose a patch for release | 07:02 |
| *** quique|rover|brb is now known as quiquell|rover | 07:07 | |
| *** rcernin has quit IRC | 07:11 | |
| *** amoralej has joined #rdo | 07:11 | |
| amoralej | good morning | 07:11 |
| ykarel | o/ | 07:11 |
| ykarel | Welcome Back amoralej :) | 07:12 |
| ykarel | hope u had good vacations | 07:12 |
| amoralej | hi ykarel, yeah, very good | 07:12 |
| ykarel | cool | 07:12 |
| amoralej | really enjoyed | 07:12 |
| egonzalez | ykarel: thanks | 07:12 |
| *** shardy has joined #rdo | 07:15 | |
| quiquell|rover | ykarel, amoralej: I see ceph-ansible 3.1.3 at centos repo http://mirror.centos.org/centos/7/storage/x86_64/ceph-luminous/ | 07:17 |
| quiquell|rover | ykarel, amoralej: We are using this http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org/centos/7/storage/x86_64/ceph-luminous/ | 07:17 |
| quiquell|rover | how do I check that they are in sync ? | 07:18 |
| quiquell|rover | we are just using yum install -y ceph-ansible | 07:18 |
| quiquell|rover | but it's installing and old version | 07:18 |
| amoralej | mirrors should automatically be in sync afaik | 07:19 |
| amoralej | quiquell|rover, that url is correct? | 07:19 |
| amoralej | i can't find it | 07:19 |
| quiquell|rover | amoralej: Token from here https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset019-master/952a6cb/logs/undercloud/home/zuul/repo_setup.log.txt.gz#_2018-09-17_00_45_25 | 07:20 |
| *** karimb has joined #rdo | 07:21 | |
| quiquell|rover | amoralej: This is the repo file https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset019-master/952a6cb/logs/undercloud/etc/yum.repos.d/quickstart-centos-ceph-luminous.repo.txt.gz | 07:21 |
| *** Darcidride has joined #rdo | 07:22 | |
| amoralej | i can't connect to it right now | 07:22 |
| *** Darcidride has quit IRC | 07:23 | |
| *** zoli is now known as zoli|wfh | 07:24 | |
| *** zoli|wfh is now known as zoli | 07:24 | |
| *** aufi has joined #rdo | 07:24 | |
| quiquell|rover | amoralej: Maybe it's only accesible at nodepool nodes | 07:24 |
| *** holser|p_ is now known as holser_ | 07:24 | |
| amoralej | maybe | 07:25 |
| *** khyr0n has quit IRC | 07:25 | |
| quiquell|rover | amoralej: The package is from 12th | 07:25 |
| quiquell|rover | amoralej: It's kinda new | 07:25 |
| ykarel | amoralej, hmm something is wrong: curl http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org/centos/7/storage/x86_64/ceph-luminous/|grep ceph-ansible | 07:26 |
| ykarel | old packages | 07:26 |
| amoralej | and i guess repo metadata is not in sync, right? | 07:26 |
| quiquell|rover | ykarel: you are able to access ? | 07:26 |
| quiquell|rover | amoralej: where is the repo meta ? | 07:26 |
| ykarel | quiquell|rover, yes with curl | 07:27 |
| amoralej | repodata folder | 07:27 |
| amoralej | with chrome works fine | 07:27 |
| amoralej | not in my firefox | 07:27 |
| quiquell|rover | amoralej: what metadata do you refer too ? | 07:28 |
| amoralej | http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org/centos/7/storage/x86_64/ceph-luminous/repodata/repomd.xml | 07:28 |
| amoralej | i'm comparing | 07:28 |
| amoralej | yeah, it's out of sync | 07:29 |
| quiquell|rover | amoralej: how does this thing sync ? | 07:30 |
| amoralej | i'm not sure | 07:30 |
| *** assassin has joined #rdo | 07:30 | |
| quiquell|rover | amoralej: who can I ask ? jpena|off ? | 07:31 |
| amoralej | jpena|off, ^ who could help us with that mirror | 07:31 |
| quiquell|rover | chkumar|ruck: fs019, one of our centos mirror is out of sync :-( | 07:33 |
| chkumar|ruck | :-( | 07:36 |
| *** akrivoka has joined #rdo | 07:39 | |
| quiquell|rover | amoralej: maybe kforde knows ? | 07:44 |
| quiquell|rover | kforde: Do you know how do we sync http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org/centos/7/storage/x86_64/ceph-luminous/repodata/repomd.xml | 07:44 |
| amoralej | don't thing so | 07:44 |
| quiquell|rover | ups ok :-( | 07:44 |
| *** ykarel is now known as ykarel|lunch | 07:45 | |
| *** jpena|off is now known as jpena | 07:50 | |
| jpena | o/ | 07:51 |
| *** kopecmartin has joined #rdo | 07:52 | |
| *** jpich has joined #rdo | 07:52 | |
| quiquell|rover | jpena: buenos dias | 07:53 |
| quiquell|rover | :-) | 07:53 |
| quiquell|rover | jpena: we have an centos repo out of sync | 07:53 |
| quiquell|rover | jpena: http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org/centos/7/storage/x86_64/ceph-luminous/repodata/repomd.xml | 07:54 |
| quiquell|rover | jpena: Were can I check the sync mechanism ? | 07:54 |
| jpena | quiquell|rover: there's no sync, we're mounting the openstack.org AFS file system | 07:55 |
| quiquell|rover | jpena: then openstack.org is the one that is out of sync ? | 07:56 |
| *** jlibosva has joined #rdo | 07:56 | |
| quiquell|rover | jpena: http://mirror.centos.org/centos/7/storage/x86_64/ceph-luminous/ has correct version of ceph-ansible | 07:56 |
| jpena | let me check another mirror. If it's on sync, it can be a local AFS client issue | 07:56 |
| quiquell|rover | jpena: the one I paste it | 07:56 |
| quiquell|rover | ceph-ansible is the package | 07:56 |
| jpena | quiquell|rover: no, I mean another AFS mirror | 07:57 |
| *** ktibi has joined #rdo | 07:57 | |
| quiquell|rover | jpena: no clue about AFS mirrors | 07:58 |
| jpena | quiquell|rover: for example, check http://mirror.dfw.rax.openstack.org/centos/7/storage/x86_64/ceph-luminous/repodata/?C=M;O=A | 07:58 |
| jpena | that mirror has the same data (last synced on Oct 1) | 07:59 |
| quiquell|rover | jpena: This is not suppose to be like http://mirror.centos.org/centos/7/storage/x86_64/ceph-luminous/ ? | 07:59 |
| jpena | yes, but it's out of sync | 07:59 |
| quiquell|rover | jpena: Were does it sync ? | 07:59 |
| jpena | so this looks like an upstream infra issue, I'm going to ask at #openstack-infra | 07:59 |
| quiquell|rover | jpena: Ahh ok, RDO just mount it, that's it ? | 08:00 |
| jpena | quiquell|rover: right | 08:00 |
| *** slaweq has quit IRC | 08:00 | |
| quiquell|rover | jpena: ok thanks, I can ask, I am the rover now | 08:00 |
| jpena | quiquell|rover: ok, as you prefer | 08:00 |
| quiquell|rover | jpena: who would be the expertise on it ? | 08:00 |
| jpena | quiquell|rover: anyone at #openstack-infra | 08:00 |
| *** paramite has joined #rdo | 08:01 | |
| quiquell|rover | jpena: thanks so much | 08:02 |
| jpena | ykarel|lunch: when you're back, what's the puppet job having trouble? | 08:02 |
| jpena | quiquell|rover: you're welcome ;) | 08:02 |
| *** florianf has quit IRC | 08:02 | |
| *** slaweq has joined #rdo | 08:03 | |
| *** florianf has joined #rdo | 08:03 | |
| *** rpittau has joined #rdo | 08:23 | |
| *** udesale has joined #rdo | 08:25 | |
| *** udesale has quit IRC | 08:27 | |
| *** udesale has joined #rdo | 08:28 | |
| *** udesale has quit IRC | 08:29 | |
| *** udesale has joined #rdo | 08:29 | |
| *** udesale has quit IRC | 08:29 | |
| *** udesale has joined #rdo | 08:30 | |
| quiquell|rover | kforde: Do we have any issue with node pool ? | 08:33 |
| kforde | quiquell|rover: what are you seeing? | 08:33 |
| quiquell|rover | kforde: some introspections are failing and I don¡t see the subnodes in the logs | 08:33 |
| kforde | must be this nested virt issue ... just looking at it now | 08:33 |
| quiquell|rover | kforde: Also our python script that collect info is not running there | 08:33 |
| quiquell|rover | kforde: https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky/4820d62/logs/ | 08:34 |
| *** tosky has joined #rdo | 08:41 | |
| *** udesale has quit IRC | 08:47 | |
| *** moguimar has joined #rdo | 08:47 | |
| *** moguimar has quit IRC | 08:50 | |
| kforde | quiquell|rover, I've disabled two computes that looked like they were mis-behaving. I'll look into it now, please let me know if jobs start passing (or failing) | 08:51 |
| quiquell|rover | kforde: ack, chkumar|ruck nodepool issues ^ | 08:53 |
| *** udesale has joined #rdo | 08:55 | |
| chkumar|ruck | quiquell|rover: testing here https://review.rdoproject.org/r/#/c/13943/ | 08:55 |
| chkumar|ruck | fs01 rocky job | 08:55 |
| rdogerrit | Félix Enrique Llorente Pastora created rdo-infra/ci-config master: Add influxdb stats to te-broker crontab https://review.rdoproject.org/r/16284 | 08:56 |
| quiquell|rover | chkumar|ruck: ack, let's see the result | 08:57 |
| quiquell|rover | chkumar|ruck: Let's do the same with queens | 08:57 |
| quiquell|rover | chkumar|ruck: Ahh no queens has pass already | 08:57 |
| *** openstackgerrit has joined #rdo | 08:59 | |
| *** ChanServ sets mode: +v openstackgerrit | 08:59 | |
| openstackgerrit | Javier Peña proposed openstack/packstack stable/rocky: Switch stable/rocky to use rocky resources https://review.openstack.org/594918 | 08:59 |
| *** udesale has quit IRC | 09:01 | |
| quiquell|rover | kforde: There were some stacks stuck there, I have put in place the script that report them | 09:08 |
| quiquell|rover | kforde: You will receive alarms again | 09:08 |
| kforde | quiquell|rover I see them. Should I wait for your cleanup to run ? | 09:09 |
| quiquell|rover | kforde: Crontab cleanup script was already there | 09:10 |
| quiquell|rover | kforde: it's kicking every 5 minutes | 09:11 |
| quiquell|rover | let's wait 15 | 09:11 |
| *** florianf has quit IRC | 09:11 | |
| kforde | quiquell|rover ack. ping me if you need me to clear them | 09:11 |
| *** florianf has joined #rdo | 09:11 | |
| *** ykarel|lunch is now known as ykarel | 09:12 | |
| ykarel | jpena, puppet-ci job were having issues | 09:13 |
| ykarel | jpena, https://ci.centos.org/view/rdo/view/weirdo-pipelines/view/weirdo-promote-puppet/ | 09:13 |
| quiquell|rover | kforde: sure, thanks | 09:14 |
| jpena | ykarel: ok. I've tried one potential fix for the pip issue (remove the pip cache), let's see if that helps | 09:14 |
| jpena | I'll cancel all pending jobs but one | 09:14 |
| ykarel | jpena, ack | 09:15 |
| *** moguimar has joined #rdo | 09:15 | |
| ykarel | jpena, is it the same pip cache issue that we faced few months back and it was workarounded somehow, let me grab the link | 09:16 |
| jpena | ykarel: in that case we used an environment variable, so the different workers did not collide with each other | 09:16 |
| jpena | that should be ok now | 09:16 |
| ykarel | hmm okk, so this is something different | 09:16 |
| *** marios has joined #rdo | 09:17 | |
| *** owalsh_ is now known as owalsh | 09:18 | |
| *** gfidente has joined #rdo | 09:28 | |
| *** udesale has joined #rdo | 09:38 | |
| *** oanson has joined #rdo | 10:05 | |
| arxcruz|ptg | ykarel: hey, did you find out why the packages are not getting into fedora for https://review.rdoproject.org/r/#/c/16216/ ? | 10:09 |
| ykarel | arxcruz|ptg, yes, we are pending some automation on that part, jpena ^^ | 10:10 |
| *** Son_Goku has joined #rdo | 10:10 | |
| arxcruz|ptg | ykarel: k | 10:10 |
| arxcruz|ptg | ykarel: is there something i can help? | 10:10 |
| jpena | ykarel: amoralej was working on that today | 10:11 |
| ykarel | jpena, cool | 10:11 |
| ykarel | arxcruz|ptg, we have right persons on it ^^ | 10:11 |
| *** arxcruz|ptg is now known as arxcruz | 10:11 | |
| amoralej | ykarel, i've synchronized it | 10:12 |
| arxcruz | ykarel: thanks :) | 10:12 |
| arxcruz | ykarel: let me know when I can recheck :) | 10:12 |
| ykarel | arxcruz, yes go with recheck now, packages are available now: https://trunk.rdoproject.org/fedora/stable-base/latest/Packages/python2-subunit2sql-1.8.0-6.fc28.noarch.rpm | 10:12 |
| ykarel | amoralej, Thanks | 10:12 |
| *** kopecmartin has quit IRC | 10:15 | |
| *** dtantsur|afk is now known as dtantsur | 10:18 | |
| *** quiquell|rover is now known as quique|rover|afk | 10:19 | |
| *** paramite|clone has joined #rdo | 10:27 | |
| *** paramite has quit IRC | 10:29 | |
| *** zoli is now known as zoli|lunch | 10:32 | |
| *** zoli|lunch is now known as zoli | 10:32 | |
| *** jfrancoa has quit IRC | 10:35 | |
| *** jfrancoa has joined #rdo | 10:36 | |
| *** rgogunskiy has joined #rdo | 10:42 | |
| *** rh-jelabarre has joined #rdo | 10:43 | |
| *** quique|rover|afk is now known as quiquell|rover | 10:49 | |
| *** linuxaddicts has quit IRC | 10:52 | |
| quiquell|rover | kforde: Did you si something similar to this https://centos.logs.rdoproject.org/weirdo-generic-packstack-scenario001/3295/weirdo-project/logs/libvirt/qemu/instance-00000001.log.txt.gz | 10:52 |
| quiquell|rover | kforde: At the problematic nodes ? | 10:53 |
| quiquell|rover | ykarel: ^ | 10:53 |
| ykarel | quiquell|rover, yes i shared those logs with him today morning | 10:54 |
| ykarel | post that he disable some nodes | 10:54 |
| kforde | quiquell|rover, ykarel on one of the nodes I haven't seen this exact error ... checking other compute now | 10:55 |
| quiquell|rover | ykarel: but the logs you paste me does not run at RDO ? | 10:56 |
| ykarel | quiquell|rover, nodes are on RDO infra atleast for phase 1 promotion jobs, but let me confirm same case is with puppet ci jobs | 10:58 |
| ykarel | i can see issue in promotion job as well so RDO cloud is involved: https://centos.logs.rdoproject.org/weirdo-queens-promote-packstack-scenario001/128/weirdo-project/logs/libvirt/qemu/instance-00000002.log.txt.gz | 10:59 |
| quiquell|rover | ykarel: Than can be the same problematic nodes | 11:00 |
| quiquell|rover | ykarel: Updating qemu version or the like ? | 11:00 |
| ykarel | yes checked puppet ci jobs also run on RDO, quiquell|rover that's related with kernel but don't have much details now | 11:03 |
| *** ktibi_ has joined #rdo | 11:05 | |
| *** ktibi has quit IRC | 11:09 | |
| *** ktibi has joined #rdo | 11:11 | |
| *** ktibi_ has quit IRC | 11:14 | |
| *** kopecmartin has joined #rdo | 11:15 | |
| *** kopecmartin has quit IRC | 11:19 | |
| *** jpena is now known as jpena|lunch | 11:20 | |
| *** fultonj has joined #rdo | 11:20 | |
| *** fultonj has quit IRC | 11:20 | |
| chkumar|ruck | amoralej: jpena|lunch anythign more needed on this review https://review.rdoproject.org/r/#/c/16107/ ? | 11:23 |
| *** fultonj has joined #rdo | 11:25 | |
| *** lon has joined #rdo | 11:27 | |
| quiquell|rover | ykarel, kforde: Added you to CIX cards regarding ironic issues | 11:30 |
| rdogerrit | Arx Cruz proposed openstack/stackviz-distgit rpm-master: Initial spec file for stackviz https://review.rdoproject.org/r/16216 | 11:30 |
| ykarel | quiquell|rover, checking the bug | 11:33 |
| *** kopecmartin has joined #rdo | 11:34 | |
| quiquell|rover | kforde, ykarel: We have re-run rocky failing job and it's working now | 11:34 |
| ykarel | quiquell|rover, while checking those ovb failures,: so looks like panda change to add debug logs for tebroker are lost after tebroker resetup | 11:37 |
| ykarel | see http://38.145.33.166/ | 11:38 |
| ykarel | http://38.145.33.166/console-logs/ empty and bmc logs are also not http://38.145.33.166/testenv-worker.log | 11:38 |
| quiquell|rover | ykarel: Do you have the changes around ? | 11:39 |
| ykarel | quiquell|rover, there were two changes, one https://review.openstack.org/#/c/588488/2 and other looking | 11:41 |
| ykarel | for other there was no patch, trying to recall that env variable | 11:41 |
| *** iranzo has quit IRC | 11:43 | |
| *** zoli is now known as zoli|wfh | 11:46 | |
| *** zoli|wfh is now known as zoli | 11:46 | |
| *** zul has joined #rdo | 11:46 | |
| *** amoralej is now known as amoralej|lunch | 11:47 | |
| ykarel | quiquell|rover, other was PYTHONIOENCODING='utf-8' with nova console-log | 11:47 |
| ykarel | but looks like some change in the way ovb jobs are running now | 11:48 |
| ykarel | as i can't see ascii error in logs | 11:49 |
| quiquell|rover | ykarel: chkumar|ruck forced a rocky run and it was passing now, so yep now they are working | 11:49 |
| quiquell|rover | ykarel: maybe kforde removing machines fixed all the issues | 11:50 |
| ykarel | quiquell|rover, but i am still seeing issues, so not sure if it's really fixed | 11:50 |
| chkumar|ruck | ykarel: where? | 11:50 |
| ykarel | quiquell|rover, so fixing the te broker logs would help atlest to have some logs | 11:50 |
| ykarel | chkumar|ruck, in rdo phase 1 and puppet ci | 11:50 |
| quiquell|rover | ykarel: Have to drop for half an hour or so, will do after | 11:51 |
| ykarel | chkumar|ruck, i am still not sure about the ironic issue as we don't have much logs there | 11:51 |
| quiquell|rover | ykarel: it's very delicate to touch te-broker will take some time | 11:51 |
| chkumar|ruck | ykarel: let's keep the bug open | 11:51 |
| *** kopecmartin has quit IRC | 11:51 | |
| chkumar|ruck | then | 11:51 |
| ykarel | chkumar|ruck, the ironic error u shared in bug are seen earlier also, so i think u can find similar error in previous bugs as well | 11:52 |
| ykarel | quiquell|rover, ack take ur time, last time panda applied it manually on te-broker | 11:53 |
| ykarel | that's what i can tell, rest is mystery :) | 11:53 |
| quiquell|rover | ykarel: searching for the rest of stuff he applied | 11:54 |
| ykarel | quiquell|rover, ack i only remember those too hacks, may be more? | 11:55 |
| *** dhill_ has joined #rdo | 11:57 | |
| *** dprince has joined #rdo | 12:00 | |
| *** cyberworm54 has joined #rdo | 12:00 | |
| *** dneary has joined #rdo | 12:03 | |
| *** dneary_ has joined #rdo | 12:05 | |
| *** trown|outtypewww is now known as trown | 12:06 | |
| *** dneary has quit IRC | 12:08 | |
| *** amuller has joined #rdo | 12:12 | |
| *** cyberworm54 has quit IRC | 12:15 | |
| *** hseipp has joined #rdo | 12:18 | |
| *** Sigyn has quit IRC | 12:19 | |
| *** Sigyn has joined #rdo | 12:20 | |
| *** jpena|lunch is now known as jpena | 12:22 | |
| *** janki has quit IRC | 12:25 | |
| *** athomas has joined #rdo | 12:29 | |
| *** rlandy has joined #rdo | 12:33 | |
| *** quiquell|rover is now known as quique|rover|lch | 12:33 | |
| *** dneary_ has quit IRC | 12:44 | |
| *** Son_Goku has quit IRC | 12:45 | |
| *** udesale has quit IRC | 12:54 | |
| *** kopecmartin has joined #rdo | 12:55 | |
| *** kopecmartin has quit IRC | 12:55 | |
| *** kopecmartin has joined #rdo | 12:55 | |
| *** kopecmartin has quit IRC | 12:56 | |
| *** kopecmartin has joined #rdo | 12:56 | |
| *** kgiusti has joined #rdo | 12:57 | |
| *** eggmaster has joined #rdo | 12:58 | |
| *** amoralej|lunch is now known as amoralej | 12:58 | |
| *** cyberworm54 has joined #rdo | 13:01 | |
| *** dustins has joined #rdo | 13:04 | |
| *** dneary has joined #rdo | 13:05 | |
| *** imcleod has joined #rdo | 13:05 | |
| *** mjturek has joined #rdo | 13:05 | |
| *** quique|rover|lch is now known as quiquell|rover | 13:05 | |
| *** morazi has joined #rdo | 13:06 | |
| *** athomas has quit IRC | 13:07 | |
| *** mjturek has quit IRC | 13:11 | |
| honza | EmilienM: what does $* mean? ... I feel like it's very obvious, and I might just need more coffee | 13:12 |
| EmilienM | honza: I did a mistake with my gertty | 13:12 |
| *** thrash|g0ne is now known as thrash | 13:12 | |
| honza | EmilienM: haha, no worries | 13:12 |
| EmilienM | honza: I didn't intend to comment $* | 13:12 |
| EmilienM | #mondaymorning | 13:13 |
| honza | yes, me too | 13:13 |
| * number80 brings some coffee | 13:13 | |
| *** cyberworm54 has quit IRC | 13:19 | |
| *** baha has joined #rdo | 13:19 | |
| quiquell|rover | ykarel: about the logs at tebroker | 13:22 |
| *** nhicher has joined #rdo | 13:22 | |
| quiquell|rover | ykarel: do you miss something here https://review.openstack.org/588488 ? | 13:27 |
| *** mcornea has joined #rdo | 13:30 | |
| *** cyberworm54 has joined #rdo | 13:32 | |
| ykarel | quiquell|rover, afair it should be fine, but good to test it before merging | 13:34 |
| ykarel | may be panda had something extra locally | 13:34 |
| quiquell|rover | ykarel yep, i am asking | 13:34 |
| ykarel | quiquell|rover, ack | 13:34 |
| *** jcoufal has joined #rdo | 13:35 | |
| *** cyberworm54 has quit IRC | 13:36 | |
| *** bdodd has joined #rdo | 13:42 | |
| *** Son_Goku has joined #rdo | 13:42 | |
| *** ramishra has quit IRC | 13:44 | |
| *** agopi is now known as agopi|afk|ncsu|c | 13:58 | |
| *** agopi|afk|ncsu|c is now known as agopi|afk|car-fa | 13:58 | |
| *** agopi|afk|car-fa is now known as agopi|afk | 13:58 | |
| quiquell|rover | ykarel: more introspection issues https://logs.rdoproject.org/94/602594/2/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/76736ee/ | 13:59 |
| sfbender | Jakub Ružička proposed rpmreq master: Add last_version to query existing packages https://softwarefactory-project.io/r/13641 | 14:00 |
| ykarel | quiquell|rover, that's 3 days old | 14:02 |
| quiquell|rover | ykarel: ups damn it | 14:02 |
| quiquell|rover | ykarel: the rocky run is passing to for the failing OVB job | 14:02 |
| ykarel | quiquell|rover, yes that can happen as that issue was intermittent | 14:03 |
| quiquell|rover | ykarel: I ahve apply the stuff to the te-broker | 14:03 |
| ykarel | quiquell|rover, good that would help atleast with logs | 14:04 |
| quiquell|rover | ykarel: Don't see them yet http://38.145.33.166/console-logs/ | 14:04 |
| ykarel | quiquell|rover, where CONSOLE_LOG_PATH defined? | 14:06 |
| quiquell|rover | ykarel: fixing | 14:07 |
| rdopiera | I wonder if I should be concerned about all the e-mails I am getting about removing Horizon's dependencies from Fedora 30? | 14:07 |
| rdopiera | it's just about the python2 versions, right? they will leave the python3 ones? | 14:08 |
| ykarel | quiquell|rover, also check if bmc logs are available at: /var/log/bmc-console-logs | 14:09 |
| *** mpeterson_ is now known as mpeterson | 14:14 | |
| quiquell|rover | ykarel: yep they are | 14:16 |
| quiquell|rover | ykarel: Now I see stuff here http://38.145.33.166/console-logs/http://38.145.33.166/console-logs/ | 14:19 |
| ykarel | quiquell|rover, no plan to publish ^^ logs? | 14:19 |
| ykarel | i mean /var/log/bmc-console-logs | 14:20 |
| quiquell|rover | You cannot see that ? | 14:20 |
| quiquell|rover | ykarel: this is not it ? http://38.145.33.166/console-logs/bmc-1260-console.log.gz | 14:20 |
| ykarel | quiquell|rover, okk seeing, then it's good | 14:21 |
| quiquell|rover | ykarel: let me know if you miss something | 14:21 |
| *** quiquell|rover is now known as quique|rover|off | 14:22 | |
| ykarel | ack | 14:22 |
| *** ccamposr has joined #rdo | 14:25 | |
| sfbender | Jakub Ružička proposed rpmreq master: Add last_version to query existing packages https://softwarefactory-project.io/r/13641 | 14:27 |
| *** dtrainor has joined #rdo | 14:30 | |
| *** ykarel is now known as ykarel|away | 14:33 | |
| *** ksambor_ has quit IRC | 14:34 | |
| *** aufi has quit IRC | 14:37 | |
| *** imcleod has quit IRC | 14:38 | |
| *** sbonazzo has quit IRC | 14:40 | |
| *** Son_Goku has quit IRC | 14:42 | |
| *** Son_Goku has joined #rdo | 14:43 | |
| *** artom has joined #rdo | 14:45 | |
| *** cyberworm54 has joined #rdo | 14:46 | |
| *** ykarel|away has quit IRC | 14:53 | |
| *** cyberworm54 has quit IRC | 14:57 | |
| *** janki has joined #rdo | 14:58 | |
| *** bnemec has joined #rdo | 14:59 | |
| *** Son_Goku has quit IRC | 14:59 | |
| *** agurenko has quit IRC | 15:00 | |
| *** aufi has joined #rdo | 15:01 | |
| *** Son_Goku has joined #rdo | 15:02 | |
| *** kforde has quit IRC | 15:03 | |
| *** mjturek has joined #rdo | 15:04 | |
| *** cyberworm54 has joined #rdo | 15:05 | |
| *** rgogunskiy has quit IRC | 15:06 | |
| *** cyberworm54 has quit IRC | 15:07 | |
| *** morazi has quit IRC | 15:15 | |
| *** thrash has quit IRC | 15:20 | |
| *** jcoufal has quit IRC | 15:21 | |
| *** thrash has joined #rdo | 15:22 | |
| sfbender | Merged DLRN master: Allow specifying a tag in run_tests.sh https://softwarefactory-project.io/r/12186 | 15:24 |
| *** cyberworm54 has joined #rdo | 15:27 | |
| *** cyberworm54 has quit IRC | 15:29 | |
| *** artom has quit IRC | 15:30 | |
| *** Son_Goku has quit IRC | 15:32 | |
| *** Son_Goku has joined #rdo | 15:35 | |
| *** ktibi has quit IRC | 15:36 | |
| *** jlibosva has quit IRC | 15:41 | |
| *** marrusl has quit IRC | 15:47 | |
| *** morazi has joined #rdo | 15:57 | |
| *** ccamposr has quit IRC | 15:57 | |
| *** ykarel has joined #rdo | 15:59 | |
| *** cyberworm54 has joined #rdo | 16:00 | |
| *** jpich has quit IRC | 16:01 | |
| *** aufi has quit IRC | 16:06 | |
| *** cyberworm54 has quit IRC | 16:06 | |
| *** thrash is now known as thrash|biab | 16:11 | |
| *** marios is now known as marios|hometime | 16:12 | |
| *** marios|hometime is now known as marios | 16:13 | |
| *** zoli is now known as zoli|gone | 16:13 | |
| *** zoli|gone is now known as zoli | 16:14 | |
| *** holser_ has quit IRC | 16:14 | |
| *** imcleod has joined #rdo | 16:17 | |
| *** artom has joined #rdo | 16:20 | |
| *** imcleod has quit IRC | 16:24 | |
| *** hseipp has quit IRC | 16:25 | |
| rdogerrit | Javier Peña created rdo-infra/rdo-infra-playbooks master: Add playbooks to setup the RDO Trunk replicated DB https://review.rdoproject.org/r/16285 | 16:27 |
| rdogerrit | Javier Peña proposed rdo-infra/rdo-infra-playbooks master: Add playbooks to setup the RDO Trunk replicated DB https://review.rdoproject.org/r/16285 | 16:28 |
| *** jfrancoa has quit IRC | 16:30 | |
| *** Son_Goku has quit IRC | 16:42 | |
| *** paramite|clone has quit IRC | 16:46 | |
| *** ykarel is now known as ykarel|away | 16:50 | |
| *** kopecmartin has quit IRC | 16:51 | |
| *** dtantsur is now known as dtantsur|afk | 16:54 | |
| *** rdopiera has quit IRC | 17:00 | |
| *** mrunge has joined #rdo | 17:01 | |
| *** mrunge_ has quit IRC | 17:03 | |
| *** artom has quit IRC | 17:04 | |
| *** artom has joined #rdo | 17:05 | |
| *** amoralej is now known as amoralej|off | 17:10 | |
| *** jpena is now known as jpena|off | 17:11 | |
| *** artom has quit IRC | 17:15 | |
| *** trown is now known as trown|lunch | 17:18 | |
| *** thrash|biab is now known as thrash | 17:21 | |
| rdogerrit | Ronelle Landy proposed rdo-jobs master: Add centos-7 nodeset to sync with upstream https://review.rdoproject.org/r/15784 | 17:28 |
| *** dsneddon has quit IRC | 17:30 | |
| *** dsneddon has joined #rdo | 17:30 | |
| rdogerrit | Ronelle Landy proposed config master: Add Browbeat jobs https://review.rdoproject.org/r/14772 | 17:30 |
| rdogerrit | User bfournie created openstack/ironic-distgit rocky-rdo: Use copytruncate when rotating logs https://review.rdoproject.org/r/16286 | 17:39 |
| rdogerrit | User bfournie created openstack/ironic-inspector-distgit rocky-rdo: Use copytruncate when rotating logs https://review.rdoproject.org/r/16287 | 17:39 |
| *** tbarron_ is now known as tbarron | 17:41 | |
| *** shardy has quit IRC | 17:46 | |
| *** puzzled has joined #rdo | 18:02 | |
| *** janki has quit IRC | 18:04 | |
| *** artom has joined #rdo | 18:06 | |
| *** artom has quit IRC | 18:07 | |
| *** artom has joined #rdo | 18:07 | |
| *** openstackgerrit has quit IRC | 18:10 | |
| *** cyberworm54 has joined #rdo | 18:11 | |
| *** imcleod has joined #rdo | 18:14 | |
| *** leanderthal has quit IRC | 18:17 | |
| *** trown|lunch is now known as trown | 18:32 | |
| *** Son_Goku has joined #rdo | 18:50 | |
| *** ykarel|away has quit IRC | 18:53 | |
| *** imcleod has quit IRC | 19:07 | |
| *** zul has quit IRC | 19:08 | |
| *** amuller has quit IRC | 19:12 | |
| *** jtomasek has quit IRC | 19:17 | |
| *** jtomasek has joined #rdo | 19:20 | |
| *** zul has joined #rdo | 19:26 | |
| *** assassin has quit IRC | 19:29 | |
| *** jtomasek has quit IRC | 19:37 | |
| *** imcleod has joined #rdo | 19:39 | |
| *** jtomasek has joined #rdo | 19:40 | |
| *** imcleod has quit IRC | 19:42 | |
| *** haleyb has joined #rdo | 19:45 | |
| *** mjturek has quit IRC | 19:49 | |
| *** mjturek has joined #rdo | 19:52 | |
| *** imcleod has joined #rdo | 19:55 | |
| *** sasha1 has joined #rdo | 19:56 | |
| *** Son_Goku has quit IRC | 19:57 | |
| *** Son_Goku has joined #rdo | 20:04 | |
| *** akrivoka has quit IRC | 20:04 | |
| *** rh-jelabarre has quit IRC | 20:18 | |
| *** jtomasek has quit IRC | 20:22 | |
| *** Son_Goku has quit IRC | 20:23 | |
| *** rh-jelabarre has joined #rdo | 20:24 | |
| *** dprince has quit IRC | 20:25 | |
| *** Son_Goku has joined #rdo | 20:26 | |
| *** mmercer has joined #rdo | 20:28 | |
| *** zul has quit IRC | 20:32 | |
| *** kgiusti has left #rdo | 20:42 | |
| *** artom has quit IRC | 20:48 | |
| *** baha has quit IRC | 21:06 | |
| *** agopi|afk is now known as agopi | 21:06 | |
| *** trown is now known as trown|outtypewww | 21:07 | |
| *** mjturek has quit IRC | 21:08 | |
| *** dneary has quit IRC | 21:23 | |
| *** cylopez has quit IRC | 21:25 | |
| *** puzzled has quit IRC | 21:30 | |
| *** Son_Goku has quit IRC | 21:36 | |
| *** karimb has quit IRC | 21:39 | |
| *** Son_Goku has joined #rdo | 21:44 | |
| *** gfidente has quit IRC | 21:57 | |
| *** bdodd has quit IRC | 22:00 | |
| *** rcernin has joined #rdo | 22:08 | |
| *** slaweq has quit IRC | 22:09 | |
| *** thrash is now known as thrash|g0ne | 22:14 | |
| *** dustins has quit IRC | 22:33 | |
| *** fpan has quit IRC | 22:36 | |
| *** fpan has joined #rdo | 22:39 | |
| *** masonjoyers has joined #rdo | 22:40 | |
| *** imcleod has quit IRC | 22:42 | |
| *** rlandy is now known as rlandy|bbl | 22:50 | |
| *** slaweq has joined #rdo | 23:11 | |
| *** slaweq has quit IRC | 23:16 | |
| *** mmercer has quit IRC | 23:19 | |
| *** tosky has quit IRC | 23:23 | |
| *** dsneddon has quit IRC | 23:52 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!