Thursday, 2018-06-21

openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO  https://review.openstack.org/56722400:00
openstackgerritRedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras  https://review.openstack.org/56044500:00
*** jbadiapa has joined #tripleo00:02
*** dmacpher has quit IRC00:05
*** atarlov has joined #tripleo00:06
*** lifeless_ has joined #tripleo00:06
*** lifeless has quit IRC00:06
*** ooolpbot has joined #tripleo00:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION00:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177775900:10
openstackLaunchpad bug 1777759 in tripleo "pike, volume failed to build in error status. list index out of range in cinder" [Critical,In progress] - Assigned to wes hayutin (weshayutin)00:10
*** ooolpbot has quit IRC00:10
*** atarlov has quit IRC00:10
*** slaweq has joined #tripleo00:11
*** slaweq has quit IRC00:16
*** zzzeek has quit IRC00:25
*** zzzeek has joined #tripleo00:27
*** atarlov has joined #tripleo00:37
*** dxiri has quit IRC00:40
*** lebauce_ has joined #tripleo00:41
*** atarlov has quit IRC00:41
*** dxiri has joined #tripleo00:43
*** rh-jelabarre has quit IRC00:46
*** mhenkel_ has joined #tripleo00:56
*** mhenkel_ has quit IRC01:00
*** atarlov has joined #tripleo01:08
*** ooolpbot has joined #tripleo01:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION01:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177775901:10
*** ooolpbot has quit IRC01:10
openstackLaunchpad bug 1777759 in tripleo "pike, volume failed to build in error status. list index out of range in cinder" [Critical,In progress] - Assigned to wes hayutin (weshayutin)01:10
*** atarlov has quit IRC01:12
*** mrsoul_ has joined #tripleo01:18
*** Petersingh has joined #tripleo01:20
*** dxiri has quit IRC01:20
*** bugzy_ is now known as bugzy01:20
*** mschuppert has quit IRC01:22
*** morazi has quit IRC01:23
*** Petersingh is now known as Petersingh|afk01:28
*** Petersingh|afk is now known as Petersingh01:32
*** links has joined #tripleo01:33
*** yamahata has left #tripleo01:34
*** atarlov has joined #tripleo01:39
*** atarlov has quit IRC01:44
*** fzdarsky_ has joined #tripleo01:46
*** fzdarsky has quit IRC01:49
*** yamahata has joined #tripleo01:56
*** lblanchard has quit IRC02:03
*** lebauce_ has quit IRC02:05
*** ooolpbot has joined #tripleo02:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION02:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177775902:10
*** ooolpbot has quit IRC02:10
openstackLaunchpad bug 1777759 in tripleo "pike, volume failed to build in error status. list index out of range in cinder" [Critical,In progress] - Assigned to wes hayutin (weshayutin)02:10
*** slaweq has joined #tripleo02:11
*** atarlov has joined #tripleo02:11
*** lebauce_ has joined #tripleo02:14
*** dxiri has joined #tripleo02:14
*** dxiri has quit IRC02:14
*** atarlov has quit IRC02:15
*** dxiri has joined #tripleo02:15
*** slaweq has quit IRC02:16
openstackgerritJill Rouleau proposed openstack/tripleo-heat-templates master: Enable Ansible error handling per role  https://review.openstack.org/57668502:16
openstackgerritJill Rouleau proposed openstack/tripleo-common master: Enable Ansible error handling per role  https://review.openstack.org/57668602:16
*** dxiri has quit IRC02:20
*** lifeless_ has quit IRC02:26
*** lblanchard has joined #tripleo02:30
*** dxiri has joined #tripleo02:41
*** lblanchard has quit IRC02:42
*** atarlov has joined #tripleo02:42
*** itlinux has joined #tripleo02:43
*** lifeless has joined #tripleo02:43
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: DNM: Run undercloud fs003 job only  https://review.openstack.org/57697902:44
*** dxiri has quit IRC02:45
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: DNM: Run multinode fs016 job only  https://review.openstack.org/57698002:46
*** psachin has joined #tripleo02:46
*** mhenkel_ has joined #tripleo02:47
*** atarlov has quit IRC02:47
*** mhenkel_ has quit IRC02:51
*** spsurya has joined #tripleo02:51
*** rcernin has quit IRC02:52
*** Petersingh is now known as Petersingh|afk02:57
*** vinaykns has joined #tripleo03:00
*** Petersingh|afk is now known as Petersingh03:03
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: DNM: [WIP] Native ZuulV3 TripleO CI job  https://review.openstack.org/57695603:05
*** ooolpbot has joined #tripleo03:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION03:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177775903:10
openstackLaunchpad bug 1777759 in tripleo "pike, volume failed to build in error status. list index out of range in cinder" [Critical,In progress] - Assigned to wes hayutin (weshayutin)03:10
*** ooolpbot has quit IRC03:10
*** rcernin has joined #tripleo03:10
*** mmethot has quit IRC03:11
*** slaweq has joined #tripleo03:11
*** yolanda has quit IRC03:12
*** liverpooler has quit IRC03:14
*** Petersingh is now known as Petersingh|afk03:14
*** slaweq has quit IRC03:16
*** skramaja has joined #tripleo03:21
*** mmethot has joined #tripleo03:22
*** nyechiel_ has quit IRC03:32
*** zshi has quit IRC03:36
*** mmethot has quit IRC03:36
*** ramishra has joined #tripleo03:37
*** vinaykns has quit IRC03:38
*** ykarel|away has joined #tripleo03:49
*** Petersingh|afk is now known as Petersingh03:52
*** udesale has joined #tripleo03:52
*** rcernin has quit IRC03:55
*** psahoo has joined #tripleo03:56
*** zshi has joined #tripleo03:57
*** zshi has quit IRC04:02
*** zshi has joined #tripleo04:02
*** lifeless has quit IRC04:08
*** ooolpbot has joined #tripleo04:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION04:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177775904:10
*** ooolpbot has quit IRC04:10
openstackLaunchpad bug 1777759 in tripleo "pike, volume failed to build in error status. list index out of range in cinder" [Critical,In progress] - Assigned to wes hayutin (weshayutin)04:10
*** slaweq has joined #tripleo04:11
*** rcernin has joined #tripleo04:14
*** lifeless has joined #tripleo04:14
*** slaweq has quit IRC04:15
*** jaganathan has joined #tripleo04:17
*** lifeless has quit IRC04:20
*** lifeless has joined #tripleo04:20
*** lifeless has quit IRC04:25
*** lifeless has joined #tripleo04:26
*** homeski has quit IRC04:28
*** StevenK has quit IRC04:29
*** dansmith has quit IRC04:29
*** jillr has quit IRC04:29
*** lifeless has quit IRC04:30
*** cshastri has joined #tripleo04:30
*** lifeless has joined #tripleo04:31
*** cshastri_ has joined #tripleo04:33
*** mhenkel_ has joined #tripleo04:35
*** lifeless has quit IRC04:36
*** cshastri has quit IRC04:36
*** cshastri_ has quit IRC04:36
*** lifeless has joined #tripleo04:37
*** cshastri has joined #tripleo04:39
*** mhenkel_ has quit IRC04:39
*** pgadiya has joined #tripleo04:40
*** lifeless has quit IRC04:41
*** lifeless has joined #tripleo04:43
*** ykarel|away is now known as ykarel04:44
*** lifeless has quit IRC04:47
*** lifeless has joined #tripleo04:48
*** lifeless has quit IRC04:52
*** lifeless has joined #tripleo04:54
*** lifeless has quit IRC04:58
*** lifeless has joined #tripleo04:59
*** mhenkel_ has joined #tripleo05:02
*** mhenkel_ has joined #tripleo05:03
*** yprokule has joined #tripleo05:04
*** lifeless has quit IRC05:04
*** lifeless has joined #tripleo05:04
*** owalsh_ has joined #tripleo05:06
*** owalsh- has joined #tripleo05:07
*** owalsh| has joined #tripleo05:08
*** pdeore has joined #tripleo05:09
*** janki has quit IRC05:09
*** lifeless has quit IRC05:09
*** skramaja_ has joined #tripleo05:10
*** owalsh has quit IRC05:10
*** ooolpbot has joined #tripleo05:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION05:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177775905:10
openstackLaunchpad bug 1777759 in tripleo "pike, volume failed to build in error status. list index out of range in cinder" [Critical,In progress] - Assigned to wes hayutin (weshayutin)05:10
*** ooolpbot has quit IRC05:10
*** lifeless has joined #tripleo05:10
*** skramaja has quit IRC05:10
*** owalsh_ has quit IRC05:11
*** slaweq has joined #tripleo05:11
*** owalsh- has quit IRC05:12
*** nyechiel_ has joined #tripleo05:12
openstackgerritMerged openstack/tripleo-common master: Use the standard messaging in the parameters workbook  https://review.openstack.org/57618705:13
*** mhenkel_ has quit IRC05:14
*** lifeless has quit IRC05:14
*** lifeless has joined #tripleo05:15
*** slaweq has quit IRC05:16
*** ratailor has joined #tripleo05:18
*** ccamacho has quit IRC05:21
*** ksambor has joined #tripleo05:22
openstackgerritPranali Deore proposed openstack/tripleo-heat-templates master: Moving glance mount nfs block to puppet glance-api template  https://review.openstack.org/57604305:24
*** dansmith has joined #tripleo05:31
*** links has quit IRC05:32
*** dansmith is now known as Guest8832005:32
*** yolanda has joined #tripleo05:34
*** pdeore_ has joined #tripleo05:35
*** udesale_ has joined #tripleo05:36
*** udesale has quit IRC05:37
*** dparkes has joined #tripleo05:38
*** cshastri has quit IRC05:38
*** pdeore has quit IRC05:38
*** psachin has quit IRC05:38
*** owalsh has joined #tripleo05:39
*** bogdando has joined #tripleo05:40
*** owalsh| has quit IRC05:42
*** slaweq has joined #tripleo05:44
*** dparkes has quit IRC05:44
openstackgerritMerged openstack/tripleo-common master: Pass DockerCephDaemonImage to CephMgr  https://review.openstack.org/57339905:44
*** pdeore_ has quit IRC05:46
*** agurenko has joined #tripleo05:48
*** slaweq has quit IRC05:48
*** agurenko has quit IRC05:52
*** quiquell|off is now known as quiquell|rover05:53
*** links has joined #tripleo05:56
openstackgerritMerged openstack/tripleo-heat-templates stable/queens: Activate memcached debug only when using MemcachedDebug param.  https://review.openstack.org/57682305:56
openstackgerritMerged openstack/tripleo-common master: Verify host_vars and group_vars are valid yaml  https://review.openstack.org/57379405:56
*** mhenkel_ has joined #tripleo05:59
*** janki has joined #tripleo06:00
*** cshastri has joined #tripleo06:01
*** mhenkel_ has quit IRC06:01
openstackgerritGiulio Fidente proposed openstack/tripleo-common stable/queens: Pass DockerCephDaemonImage to CephMgr  https://review.openstack.org/57707206:01
*** nyechiel has joined #tripleo06:02
*** nyechiel_ has quit IRC06:02
*** mrsoul_ is now known as mschuppert06:03
*** mhenkel_ has joined #tripleo06:04
*** StevenK has joined #tripleo06:07
openstackgerritRabi Mishra proposed openstack/puppet-tripleo master: WIP Add manifest for netowking_ansible ML2 plugin  https://review.openstack.org/57707406:08
*** ooolpbot has joined #tripleo06:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION06:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177775906:10
*** ooolpbot has quit IRC06:10
openstackLaunchpad bug 1777759 in tripleo "pike, volume failed to build in error status. list index out of range in cinder" [Critical,In progress] - Assigned to wes hayutin (weshayutin)06:10
*** psachin has joined #tripleo06:11
*** slaweq has joined #tripleo06:11
*** slaweq has quit IRC06:16
*** radek__ has joined #tripleo06:16
*** Petersingh is now known as Petersingh|afk06:23
*** cshastri has quit IRC06:23
*** ykarel is now known as ykarel|afk06:23
*** ccamacho has joined #tripleo06:24
*** lvdombrkr has joined #tripleo06:25
*** cylopez has joined #tripleo06:27
*** cshastri has joined #tripleo06:27
*** Petersingh|afk is now known as Petersingh06:27
*** avivgt has joined #tripleo06:29
*** cylopez has left #tripleo06:31
*** sanjayu__ has joined #tripleo06:31
*** pdeore_ has joined #tripleo06:32
*** ykarel|afk has quit IRC06:33
*** sanjayu_ has joined #tripleo06:33
*** ykarel|afk has joined #tripleo06:34
yolandahi, owalsh , around?06:34
yolandawould you need to disable noveau in all kind of images, and at the same time, load this element on all ? https://review.openstack.org/#/c/573223/06:35
*** matbu has quit IRC06:36
*** sanjayu__ has quit IRC06:36
*** dparkes has joined #tripleo06:40
bogdandoo/ PTAL https://review.openstack.org/#/c/576086/06:42
ccamachoquiquell|rover06:43
ccamachomooorning!06:44
ccamachoxD06:44
ccamachoman quick question06:44
ccamachothere is a py35 job failing in instack-undercloud06:44
ccamachofor a deps issue06:44
ccamachoquiquell|rover this is an example of the failed job https://review.openstack.org/#/c/576165/506:45
*** jcoufal has joined #tripleo06:45
*** radek__ has quit IRC06:45
*** ykarel|afk is now known as ykarel06:46
*** k3vV1n8 has joined #tripleo06:51
*** apetrich has quit IRC06:58
*** apetrich has joined #tripleo06:58
k3vV1n8hi all i have ceph-mon running on openstack controller running newton release.Does tripleO fast forward update impact cep-mon running on openstack controller?06:59
*** kopecmartin has joined #tripleo06:59
*** rfolco_ has joined #tripleo06:59
*** rpioso|afk has quit IRC06:59
*** rpioso|afk has joined #tripleo07:01
*** rfolco has quit IRC07:02
*** quiquell|rover is now known as quique|rover|afk07:03
*** waleedm has joined #tripleo07:03
*** gkadam has joined #tripleo07:05
*** apetrich has quit IRC07:06
*** rodolof has joined #tripleo07:06
*** jtomasek has joined #tripleo07:06
*** jtomasek has quit IRC07:06
*** jtomasek has joined #tripleo07:07
*** rpioso|afk has quit IRC07:07
*** skramaja_ has quit IRC07:08
*** rpioso|afk has joined #tripleo07:09
*** moshele has joined #tripleo07:10
*** ooolpbot has joined #tripleo07:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION07:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177775907:10
*** ooolpbot has quit IRC07:10
*** skramaja_ has joined #tripleo07:10
openstackLaunchpad bug 1777759 in tripleo "pike, volume failed to build in error status. list index out of range in cinder" [Critical,In progress] - Assigned to wes hayutin (weshayutin)07:10
*** matbu has joined #tripleo07:10
*** florianf has joined #tripleo07:11
*** rodolof has quit IRC07:11
*** tesseract has joined #tripleo07:11
*** rodolof has joined #tripleo07:11
*** gfidente has joined #tripleo07:13
*** aufi has joined #tripleo07:14
*** udesale__ has joined #tripleo07:14
*** slaweq has joined #tripleo07:15
*** skramaja has joined #tripleo07:15
*** udesale_ has quit IRC07:16
*** skramaja_ has quit IRC07:17
*** links has quit IRC07:21
*** links has joined #tripleo07:21
*** pcaruana has joined #tripleo07:22
*** ubijtsa has joined #tripleo07:23
*** ubijtsa is now known as assassin07:23
*** amoralej|off is now known as amoralej07:25
*** shardy has joined #tripleo07:29
*** tosky has joined #tripleo07:33
*** atarlov has joined #tripleo07:35
shardyd0ugal: Hey, good morning - when you have time I'd appreciate any help with https://review.openstack.org/#/c/576814/2/workbooks/plan_management.yaml07:35
shardyI can't get the where filter to work on the yaml_parse output, and not sure why, tried a few ways now and running out of ideas07:36
shardyit's tricky since yaml_parse doesn't work in the yaql interactive parser and the error out of mistral is fairly cryptic07:36
shardyI can share my steps to test if that helps07:37
*** atarlov has quit IRC07:39
*** ffiore has joined #tripleo07:39
*** atarlov has joined #tripleo07:39
*** quique|rover|afk is now known as quiquell|rover07:39
*** sshnaidm|off has quit IRC07:40
*** Petersingh is now known as Petersingh|lunch07:42
flaper87shardy: morning07:42
flaper87so, here's a question for ya07:42
*** jpena|off is now known as jpena07:42
*** links has quit IRC07:42
flaper87shardy: infra installs unbound on every node by default07:42
flaper87we need to disable it before we run https://review.openstack.org/#/c/574233/07:43
flaper87because origin-node needs 127.0.0.1:53 to bind its own dns cache thing07:43
flaper87not dns cache but skydns)07:43
flaper87question is: Do you know if it'd be possible to disable unbound before the openshift template starts from oooq?07:44
flaper87it's actually a question for anyone, not just shardy :P07:44
shardyflaper87: can we do it in host_prep_tasks, which will happen before we run osa?07:44
flaper87shardy: we could but this is soooo specific to CI that I'm a bit unsure about putting it there07:44
shardyand good morning :)07:44
*** rcernin has quit IRC07:47
shardyflaper87: we could copy ./ci/environments/multinode-core.yaml and override OS::TripleO::Services::Core for this job?07:48
flaper87lemme check that07:48
shardyand add the special host_prep_tasks to the copy of multinode-core.yaml07:48
flaper87shardy: I like that idea better07:49
shardyanother option would be to add a special "service" with only the disable unbound host_prep_tasks I guess07:49
flaper87thanks07:49
shardyand just enable that in the 009 scenario07:49
lvdombrkrfolks, who use ipv6 in production? how you distribute ipv6 addreses between tenants?07:50
shardyflaper87: Hey quick question for you re scale out - if I start testing scale out without CNS (which I think needs t-h-t changes), could you look at the idempotency issues around CNS?07:50
shardyflaper87: I was thinking that might be a good way to share the scale-out/update work?07:50
*** atarlov_ has joined #tripleo07:50
*** dciabrin has quit IRC07:50
*** pmannidi has quit IRC07:51
shardylvdombrkr: you might want to chat with dsneddon about it, but he probably won't be around until later07:51
*** agurenko has joined #tripleo07:51
flaper87shardy: yes, I actually wanted to ping you about the scale out work yday but I wasn't feeling well07:52
flaper87shardy: I'd like to see how you're testing it and start doing the same tests locally and apply fixes there07:52
shardyflaper87: np, I was still trying to get the plan management stuff working but was planning to look at the scale stuff today07:52
flaper87shardy: re CI, I'll copy the multinode-core template, I like the idea of this being specific to the job07:52
flaper87and evident07:52
shardyflaper87: ack - as a first step I think just re-running the ansible playbook after a CNS enabled deploy is enough, for me that failed07:52
shardyso we probably need to fix that first, so that re-running with no changes is OK07:53
*** atarlov has quit IRC07:53
*** links has joined #tripleo07:55
*** aufi_ has joined #tripleo07:56
*** jpich has joined #tripleo07:56
lvdombrkrshardy: thanks i will ping him07:57
*** aufi has quit IRC07:59
openstackgerritRedHat RDO CI proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO  https://review.openstack.org/56722408:00
*** ykarel is now known as ykarel|lunch08:02
*** dciabrin has joined #tripleo08:02
*** atarlov has joined #tripleo08:03
*** atarlov_ has quit IRC08:06
lvdombrkrdsneddon: hi08:08
*** rodolof has quit IRC08:10
*** ooolpbot has joined #tripleo08:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION08:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177775908:10
openstackLaunchpad bug 1777759 in tripleo "pike, volume failed to build in error status. list index out of range in cinder" [Critical,In progress] - Assigned to wes hayutin (weshayutin)08:10
*** ooolpbot has quit IRC08:10
*** rodolof has joined #tripleo08:10
*** atarlov has quit IRC08:14
*** atarlov has joined #tripleo08:14
owalshyolanda: hey08:18
*** sshnaidm has joined #tripleo08:18
*** slaweq has quit IRC08:18
yolandahi owalsh08:19
*** atarlov has quit IRC08:19
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: Only detach device if all partitions have been cleaned  https://review.openstack.org/57687608:19
owalshyolanda: yes, need to disable nouveau on overcloud-full08:19
yolandaowalsh, but you need to enable nvidia element also there?08:20
owalshyolanda: no08:20
owalshyolanda: that will be a custom image for vGPU08:20
yolandaowalsh, then why do you need nouveau disabled in the overcloud-full?08:20
openstackgerritCarlos Camacho proposed openstack/instack-undercloud master: test py35 deps issue  https://review.openstack.org/57708808:20
owalshyolanda: PCI passthrough of GPUs08:20
yolandathe reason i ask, is that we were thinking that you could join in a single element08:20
owalshnope08:20
d0ugalshardy: taking a look now.08:21
yolandamm, we are worried about having so many elements in dib, that do the same functionality08:21
*** suuuper has joined #tripleo08:21
d0ugalshardy: if you have the error etc. that'd be useful08:21
d0ugalor maybe I can dig it out of the CI logs...08:21
yolandabut if you cannot join the 2 elements, having that copy functionality that Ian proposed is not going to help08:21
yolandaianw ^08:21
shardyd0ugal: ack sec let me reproduce it08:23
owalshyolanda, ianw: could move it to tripleo elements, but it's not really tripleo specific08:23
owalshe.g disable nouveau and install GPU drivers on guest images where the guest has a GPU (via PCI passthrough or vGPU)08:24
yolandaowalsh, we were thinking on better ways to reuse the modprobe element, that already exists08:24
owalshdoens't seem better IMO, also it's a trivial element so not really getting the opposition to it08:24
*** slaweq has joined #tripleo08:26
owalshagree with adding the ability to set options in modprobe-blacklist, but that's a different discussion I think08:26
owalshI was expect far more friction for https://review.openstack.org/573223 :-)08:27
owalshs/expect/expecting/08:27
yolandaowalsh, is not opposition but trying to make things more reusable, i removed the -1 but i think that with a bit of work in the modprobe element, we could reuse it08:28
owalshyolanda: ah, ok... just read ianw's suggestion....08:29
yolandayep, but ian was thinking you could reuse the nvidia element for it, that doesn't seem to be the case08:29
yolandaif you don't need to add in all images08:29
*** apetrich has joined #tripleo08:30
owalshyea, ideally would have a distinct element for this. I could use envvars in the image build but that a bit messier IMO08:30
owalshbut the disable-nouveau element could just use modprobe-blacklist08:31
yolandamm, and what about adding this copy functionality for the modprobe element, and you move the disable-nouveau element to tripleo-image-elements, reusing modprobe?08:31
owalshcould, but it's not something tripleo specific e.g if you wanted to create a ubuntu guest with GPU drivers to pass through a device08:32
shardyd0ugal: Ok this is weird, it was failing like http://paste.openstack.org/show/723942/ but I just rebased my changes and re-tested and now it works...08:33
shardyd0ugal: thanks for the help, I'm not quite sure what changed, will try to figure it out!08:33
yolandaowalsh, and that's the only thing needed to be able to enable gpu? maybe you can rename, to show the functionality of it08:34
yolandait looked very specific to me08:34
d0ugalshardy: It's always reassuring when code just starts working :)08:34
shardyerr... ;)08:34
owalshyolanda: PCI passthrough of GPUs - disable nouveau on host, disable nouveau & install drivers on guest08:35
owalshyolanda: vGPU: disable nouveau and install vGPU drivers on host, disable nouveau and install drivers on guest08:35
*** derekh has joined #tripleo08:36
*** holser_ has joined #tripleo08:37
yolandamm, i think then the element should explain that a bit more, maybe change the naming , like "allow-nvidia-gpu-passthrough". disable-nouveau looked to me, as you were just creating an element to blacklist an specific module08:38
owalshwell, that's pretty much what it does, but it's a pre-req for NVIDIA drivers08:39
owalshand vGPU is not gpu passthrough08:39
yolandai still think it's a bit overkill. As we also use the modprobe-blacklist settings, for example in the hardened images08:40
yolandaif that wasn't by that specific "options nouveau modeset=0", it should be just an extra item there08:40
owalshyup, I did look at modprobe-blacklist first08:40
yolandalet's wait for Ian's suggestions as well, and see how can we improve it08:41
*** assassin has quit IRC08:41
yolandamaybe accepting an extra setting for options can be enough, or we just accept that element, but depending on modprobe08:41
*** ubijtsa has joined #tripleo08:42
*** ubijtsa is now known as assassin08:42
owalshyea, latter seems like a good approach I think08:42
owalshsure, one more element, but it will be even more trivial08:42
*** agurenko has quit IRC08:43
yolandaand maybe modify a bit the readme or the name, to make it clear what this element is useful for08:44
yolandaso people know when to add it, and don't try to do their own08:44
owalshyolanda: oh. BTW I couldn't build the hardened image. Looks like bootloaded now requires block-device-mbr08:47
yolandaowalsh, yes, but i added that on the elements08:48
*** Petersingh|lunch is now known as Petersingh08:48
openstackgerritCarlos Camacho proposed openstack/instack-undercloud master: test py35 deps issue  https://review.openstack.org/57708808:48
yolandaowalsh, see http://git.openstack.org/cgit/openstack/tripleo-image-elements/tree/elements/overcloud-secure/element-deps08:49
*** Haresh has joined #tripleo08:49
owalshyolanda: ack, expect I had an older tripleo-image-elements (queens maybe)08:50
*** Distel has quit IRC08:53
yolandayep, this changed some time ago, when some support for gpt/efi was added08:55
sri_bfournie: thanks :)09:00
*** pkovar has joined #tripleo09:05
openstackgerritMerged openstack/tripleo-quickstart master: set the ceph container for pike  https://review.openstack.org/57685109:07
openstackgerritMerged openstack/tripleo-heat-templates master: Migrate Octavia to external_deploy_task  https://review.openstack.org/55937409:07
flaper87shardy: what if I just add the role_data to disable unbound to the same multinode-scenario009 file that we have already? https://review.openstack.org/#/c/574233/16/ci/environments/scenario009-multinode.yaml09:08
flaper87mmh, well, that wouldn't be re-usable for future jobs09:08
flaper87nvm, sep file is better09:08
*** k3vV1n8 has quit IRC09:11
openstackgerritFlavio Percoco proposed openstack/tripleo-heat-templates master: Update for openshift 3.9  https://review.openstack.org/57423309:17
shardyflaper87: yeah either would work, I'd probably just update the resource_registry in the scenario with a comment explaining why it's needed09:17
flaper87shardy: https://review.openstack.org/#/c/574233/ done, let's see if that works09:18
openstackgerritJiri Stransky proposed openstack/tripleo-common stable/queens: Provide timestamps in Ansible output by default  https://review.openstack.org/57710309:20
*** salmankhan has joined #tripleo09:22
*** stendulker has joined #tripleo09:24
*** panda|off is now known as panda09:24
*** mhenkel_ has quit IRC09:25
*** sshnaidm is now known as sshnaidm|lnch09:26
openstackgerritCarlos Camacho proposed openstack/instack-undercloud master: test py35 deps issue  https://review.openstack.org/57708809:29
openstackgerritYurii Prokulevych proposed openstack/tripleo-upgrade master: Make service check output more verbose.  https://review.openstack.org/57710809:29
*** ykarel|lunch is now known as ykarel09:29
*** stendulker has quit IRC09:29
openstackgerritCarlos Camacho proposed openstack/instack-undercloud master: test py35 deps issue  https://review.openstack.org/57708809:30
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Add ability to set openshift container images  https://review.openstack.org/57644109:36
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP][DNM] test to gather zuul inventory informations  https://review.openstack.org/57687909:37
openstackgerritMerged openstack/tripleo-docs master: Add minor version update documentation for Queens (and beyond)  https://review.openstack.org/57367209:38
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP][DNM] test to gather zuul inventory informations  https://review.openstack.org/57687909:39
*** rcernin has joined #tripleo09:39
*** zoli is now known as zoli|lunch09:40
*** mhenkel_ has joined #tripleo09:40
openstackgerritNguyen Van Trung proposed openstack/diskimage-builder master: Add iscsi-boot element for CentOS images  https://review.openstack.org/54270809:41
*** dtantsur|afk is now known as dtantsur09:41
dtantsura question for containers experts (bogdando?): how do I update the undercloud with my version of tripleo-common?09:43
dtantsurideally we should have it documented somewhere (maybe we do)09:43
bogdandodtantsur: there is no upstream docs for installing patches into containers09:43
bogdandoand updating rpms runtime is the only option prolly, it is CI only09:44
dtantsurwe must have it before switching to containerized undercloud by default09:44
bogdandowill not be productized09:44
dtantsurokay, so what's the developer's workflow now?09:44
bogdandodtantsur: there is some downstream docs for patching containers...09:45
bogdandoand for rpms update, dtantsur, you'll need to make your change into the package first09:45
dtantsurwait09:46
openstackgerritRabi Mishra proposed openstack/puppet-tripleo master: Add manifest for netowking_ansible ML2 plugin  https://review.openstack.org/57707409:46
dtantsurplease09:46
dtantsurdo we expect developers to stop using tripleo after the switch to containerized undercloud?09:46
dtantsurbecause what you say means "yes"09:46
openstackgerritRabi Mishra proposed openstack/puppet-tripleo master: Add manifest for networking_ansible ML2 plugin  https://review.openstack.org/57707409:47
bogdandodtantsur, EmilienM_PTO: had been worked on that bug09:49
bogdandoworking*09:49
dtantsurlink handy?09:49
bogdandocan't find yet)09:50
bogdandodtantsur: https://bugs.launchpad.net/tripleo/+bug/176242209:51
openstackLaunchpad bug 1762422 in tripleo "ci: containerized undercloud does not test tripleo-common" [High,Fix released] - Assigned to Emilien Macchi (emilienm)09:51
dtantsurbogdando: okay, so this is CI. what about local development?09:51
bogdandodtantsur: according to the latest comment, stevebaker may know more09:52
shardydtantsur: you might want to look at https://github.com/openstack/ansible-role-tripleo-modify-image - IIRC that's what we use in CI, probably we need a wrapper that makes it easy to run a local docker registry and update it from locally built dlrn packages09:52
bogdandodtantsur: had yet been worked out, please ask Steve. Perhaps he has some docs for that to push out ?09:52
shardystevebaker may have something like that already09:52
* dtantsur is frustrated09:52
shardytbh I'm still using the non-container undercloud partly because of this09:53
dtantsurI'm afraid to ask, but you do realize that this all is insanely complex compared to both what we used to have and devstack?09:53
bogdandoshardy, dtantsur: it doesn't help much for local dev09:53
*** rcernin has quit IRC09:53
dtantsurwell, that's my plan for now - use non-containerized undercloud09:53
dtantsurbut rocky is coming...09:53
bogdandomaking your change into rpms then into some mirror, to end up with that ansible role intalling it seems too long way to me09:53
shardybogdando: well you can just build your local undercloud using a local docker registry09:53
shardywhich can have whatever images you need in it09:54
bogdandodtantsur: meh, containers solve everything, they are not insanely complex09:54
shardyAFAICS the missing part is to make it really easy to rebuild images locally with source changes added09:54
bogdandoindeed09:55
bogdandolet me catch that up on trello ...09:55
*** mhenkel_ has quit IRC09:55
dtantsurthe missing part is people being familiar with docker registries...09:55
* dtantsur old man yelling at cloud09:55
dtantsurbogdando++09:55
shardydtantsur: It's no more complex than a web server really (running a docker registry) but I do agree in general we added a lot of complexity to achieve the aim of adding containers in a completely backwards compatible way09:56
bogdandoeven with a locsal registry, you need to make your code into packages09:56
bogdandoAFAIK, https://github.com/openstack/ansible-role-tripleo-modify-image can only work with packages09:57
dtantsurshardy: right, but we did not have to manage a web server to update one line of code previously ;)09:57
shardybogdando: we already have tool to do that, it's very easy09:57
ccamachoshardy anyone with powers in openstack/requirements?09:57
shardytripleo.sh --delorean-build openstack/tripleo-common09:57
ccamachopy35 is breaking for instack-undercloud09:57
ccamachohttps://review.openstack.org/#/c/577106/09:57
ccamachoquiquell|rover09:57
ccamacho^09:57
shardyor run dlrn directly, or have quickstart do it09:58
ccamachoplease check the patch09:58
*** spsurya has quit IRC09:58
shardydtantsur: yeah, I agree, that's containers for you I guess09:58
ccamachomwhahaha ^09:58
* shardy shrugs09:58
shardyWith ansible and containers everything gets easier!09:58
shardyoh wait.09:58
shardy;)09:58
bogdandohttps://trello.com/c/FFsajzOT/66-switch-the-default-undercloud-to-be-containerized dtantsur, shardy here it is...09:58
*** atarlov has joined #tripleo09:58
chemccamacho: to add to the dulwich error ... we need max dulwich-0.19.2 Release 0.19.209:59
chemccamacho: and release the contraint when 0.19.4 is release10:00
chemccamacho: the setup.cfg was added in 0.19.310:00
dtantsurthanks bogdando10:01
bogdandodtantsur, shardy: technically, if you bind-mount source code as is, it overrides installed rpms, and should work10:01
bogdandoafter container restarted10:01
dtantsuryeah, that's an option. just keep in mind that not everyone here knows how to bind-mount something into a container..10:02
bogdandonot tested tho, and there is no an interface to custom container volumes in tht, just a few...10:02
* dtantsur has been happily ignorant so far10:02
bogdandodtantsur: oh, c'mon10:02
bogdando:D10:02
dtantsurikr? one can only learn so much at the same time. and right now I'm learning the guts of tripleo-common :)10:03
bogdandothere is docker docs fully cover those things10:03
ccamachochem so no need for the fix in openstack/requirements? THe thing is that it might hit other projects, right?10:03
*** stendulker has joined #tripleo10:03
chemccamacho: all project I guess10:03
chemccamacho: yolanda doing a launchpad bug10:03
ccamachoso better in requiremtens10:03
ccamachochem https://bugs.launchpad.net/tripleo/+bug/177800410:04
openstackLaunchpad bug 1778004 in tripleo "[instack-undercloud] openstack-tox-py35 fails with dependency issue" [Critical,Triaged]10:04
*** mhenkel_ has joined #tripleo10:08
*** assassin has quit IRC10:12
dtantsurshardy: mind doing a quick sanity check on the approach in https://review.openstack.org/576856 ?10:13
dtantsurbefore I dive into expanding / testing / fixing it10:13
ccamachodmellado do you have powers in openstack/requirements? Mind to check https://review.openstack.org/#/c/577106/ ?10:13
ccamachoquick fix10:13
dtantsurccamacho: you may ask on #openstack-requirements10:13
dmelladoccamacho: I just have +1 power but I can take a look10:14
dtantsurbut it may be early for the most people there10:14
ccamachothanks dtantsur10:14
*** Petersingh is now known as Petersingh|afk10:14
shardydtantsur: looks like a good start - as we discussed yesterday though it may end up easier to create the heat stack then use the outputs to define the hostnames etc, vs wrangling the parameters10:15
shardyI saw slagle confirm that should work wrt deployed server10:15
dtantsurshardy: yeah, I'm still digesting slagle's comment10:15
openstackgerritKarthik S proposed openstack/os-net-config master: WIP: Adding VLAN support and other configs for sriov_vf  https://review.openstack.org/54162010:15
* dtantsur ENEEDMORECOFFEE10:15
*** Petersingh|afk has quit IRC10:16
dtantsurI guess I can insert my code after creating the stack but before running config-download10:16
shardydtantsur: I guess one way to figure it out is to create a stack configured for deployed server, then work out what data you need from openstack stack output list/show to build the nodes and have them configured10:16
shardydtantsur: yeah10:16
dtantsurnow I'll have to figure out what exactly I can get from heat and how10:17
shardydtantsur: I suspect most of what you need is already in the outputs, but we could easily create a special new output with all the data you need in one map10:17
dtantsurshardy: something around https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/role.role.j2.yaml#L643 ?10:19
openstackgerritMartin André proposed openstack/ansible-role-tripleo-modify-image master: Remove dependency on repoquery  https://review.openstack.org/57682510:19
dtantsuralso are we okay to me HostnameMap mandatory for all deployments? or should we start providing some default (then how)?10:20
shardydtantsur: I guess we could - the default would just be a map of the heat/HostnameFormat generated names with all keys/values the same10:21
*** nyechiel has quit IRC10:22
shardywe may be able to fix that inside t-h-t I think10:22
*** atarlov has quit IRC10:22
*** stendulker_ has joined #tripleo10:23
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/overcloud.j2.yaml#L101910:23
*** atarlov has joined #tripleo10:23
shardydtantsur: a lot of those outputs from role.role.j2.yaml are already exposed in the overcloud.j2.yaml, but we can add any that are missing10:23
shardye.g ServerIdData gives you all the server IDs10:23
shardyand all the roles and the number of nodes10:24
shardywe could have a generic ServerData map with all the info you need, when you define what keys are needed10:24
dtantsurshardy: yeah, but it seems to depend on Nova10:25
shardydoesn't OS::Heat::DeployedServer also return some kind of ID?10:26
* dtantsur no idea10:26
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/deployed-server/deployed-server.yaml#L15210:26
dtantsurI also need stuff like image, flavor, etc10:26
dtantsur(I mean, flavor name, there won't be real flavors)10:27
*** stendulker has quit IRC10:27
shardydtantsur: yeah, we can add all that, if you give me a list of what you need I'll push a WIP t-h-t patch that exposes it for all roles10:27
*** atarlov has quit IRC10:27
*** radeks has joined #tripleo10:28
dtantsurlemme come with a list10:28
shardyhttps://github.com/openstack/heat/blob/master/heat/engine/resources/openstack/heat/deployed_server.py#L15910:28
shardydtantsur: you'll still get a uuid, it'll just come from heat instead of nova10:28
shardyone disadvantage of that is it requires a persistent heat stack vs a transient one like in the undercloud deploy10:29
shardybut it's probably fine as a starting point10:29
*** dtrainor has quit IRC10:30
*** mhenkel_ has quit IRC10:30
*** dtrainor has joined #tripleo10:31
*** ubijtsa has joined #tripleo10:32
*** ubijtsa is now known as assassin10:32
dtantsurshardy: https://etherpad.openstack.org/p/tripleo-nova-less-deploy10:32
dtantsurwell, we don't provision bare metals when deploying undercloud, right?10:32
*** mhenkel_ has joined #tripleo10:32
shardydtantsur: yeah, I was just thinking what if we wanted at some point to make the overcloud work in the same way, e.g a heat-ansible shim or whatever10:34
*** maufart__ has joined #tripleo10:34
shardygiven that we're going with mistral for this integration I guess keeping state in mistral and heat is fine10:34
d0ugalshardy: what state?10:34
*** radeks has quit IRC10:34
* d0ugal doesn't like keeping state in Mistral10:35
shardyd0ugal: well in this case we're discussing the server IDs generated by heat, which would change if, say, you ran a temporary heat in a container like we do for undercloud deploy10:35
shardyI guess there's less state to worry about in mistral since we now persist the plan status in swift10:35
openstackgerritDerek Higgins proposed openstack/tripleo-heat-templates master: Add scenario011 to install ironic in the overcloud  https://review.openstack.org/48526110:36
d0ugalRight, I'm happy for a workflow to manage the state but it would be better stored in swift.10:36
*** aufi_ has quit IRC10:36
shardyyeah, I just meant we need the DB in both cases I guess10:36
dtantsurwe're planning for a bit far future :)10:37
*** apetrich has quit IRC10:38
shardywell it's worth thinking about since we already have the stateless solution working well for the single-node case10:39
shardybut I agree maybe a bit premature to actually do :)10:39
dtantsurshardy: I suspect we may not have an easy access to the SSH key yet, because it's currently in nova, right?10:39
shardydtantsur: yeah we'll need to pass it as a heat parameter or put it somewhere else (swift?)10:39
dtantsuryeah10:40
dtantsureverything else should be available, but not entirely obvious how10:40
dtantsurmmm, we also need to figure out the default user name10:40
dtantsurI think config-download expects heat-admin10:40
d0ugalIs this ssh key different from the one we have in Mistral now?10:40
dtantsurd0ugal: I need any suitable to bootstrap the node10:41
openstackgerritMerged openstack/tripleo-quickstart master: update README to exclude Fedora from support  https://review.openstack.org/57657710:42
d0ugaldtantsur: there is one in "mistral environment-list" - I forget the name of it and my env is rebuilding. "mistral environment-get ssh_keys" maybe10:42
d0ugalbut I honestly forget what it is for :/10:42
d0ugalMaybe overcloud nodes.10:42
shardyyeah forgot about mistral, that should work10:42
*** bfournie has quit IRC10:43
dtantsurso is it available from within the actions?10:44
*** bfournie has joined #tripleo10:44
*** dhill_ has quit IRC10:45
shardydtantsur: yes we use it for config-download ansible runs I think, grep ssh_private_key in the workbooks10:46
dtantsurack10:46
d0ugalhttps://github.com/openstack/tripleo-common/blob/7ed48476c2abb06aee42d89e384aae376832102b/workbooks/validations.yaml#L203-L20710:46
dtantsurshardy: do you know where exactly heat generates the metadata it sends to nodes? I'm trying to find it in the heat sources10:46
shardydtantsur: yes, sec10:46
dtantsurI wonder how the heat-admin user is created10:46
shardydtantsur: that's created in t-h-t via cloud-init10:47
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/firstboot/userdata_heat_admin.yaml#L2910:47
shardywe just need to expose the user_data per role I think10:48
shardythat's not the same as the cloud-init data injected by heat, but I think we may not need that10:48
*** bfournie has quit IRC10:48
openstackgerritMartin Mágr proposed openstack/tripleo-heat-templates master: Enable collectd to connect to metrics QDR  https://review.openstack.org/57605710:50
shardyd0ugal: do you know of an easy way to do split() on a list, e.g split every item, in yaql?10:50
shardyselect($.split("/")) perhaps?10:51
* shardy tries10:51
*** avivgt has quit IRC10:51
*** derekh_ has joined #tripleo10:51
d0ugalshardy: that seems to work10:52
d0ugalshardy: http://paste.openstack.org/show/724016/10:52
shardyyay, I think I'm slowly getting better at yaql again :)10:52
shardyd0ugal: thanks!10:52
d0ugalshardy: np, .map() may make more sense.10:53
d0ugal["a/1", "b/2"].map($.split("/"))10:53
d0ugalUnless you want to verify the result of the split as truthy10:53
openstackgerritMartin Mágr proposed openstack/tripleo-heat-templates master: QDR for metrics collection purposes  https://review.openstack.org/57231210:54
shardyI just need to split e.g plan-samples/openshift/plan-environment.yaml to get "openshift", but I get a list of all the stuff in <plan>/plan-samples10:54
*** derekh has quit IRC10:54
shardyboth work :)10:55
hjensasdtantsur: looking at your etherpad. Remember that deployed-server does not create neutron ports, it creates fake neutron-ports. Just a heat stack that output what a port would.10:55
*** rcernin has joined #tripleo10:55
hjensasdtantsur: e.g no ml2 magic.10:55
*** zul has quit IRC10:56
*** zul has joined #tripleo10:57
*** zoli|lunch is now known as zoli10:57
dtantsurhjensas: ah, so we still need to create actual ports. well, fine, then I need only the network list. Or do we expect specific port UUIDs?10:57
dtantsurshardy: so, is https://github.com/openstack/tripleo-heat-templates/blob/master/firstboot/userdata_heat_admin.yaml#L29 passed to nova or somehow via heat? does itwork the same way with deployed servers (I guess no)?10:58
dtantsurfor deployed servers, which user name do we use to log in? and which ssh key?10:58
shardydtantsur: yeah heat just passes it to nova11:00
hjensasdtantsur: not sure. Maby deployed-server should just create neutron ports ... Or create ports and generate the DeployedServerPortMap?11:00
dtantsurhjensas: we can create ports when provisioning, I don't care much11:01
dtantsurshardy: okay, so it should go to configdrive11:01
* dtantsur creates a TODO item for metalsmith11:01
shardydtantsur: yeah, this is different to the normal deployed server, where the user_data is ignored11:02
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/deployed-server/deployed-server.yaml#L2011:02
*** quiquell|rover is now known as quique|rover|lch11:03
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: run only quickstart in the job w/o preparations  https://review.openstack.org/57691311:04
*** sshnaidm|lnch is now known as sshnaidm11:04
dtantsurthanks shardy, now I have something to think about (and a few things to implement in metalsmith)11:05
shardydtantsur: np, I'll try to push a patch later to expose the outputs from the etherpad11:06
dtantsurgreat!11:06
*** mhenkel_ has quit IRC11:10
*** bfournie has joined #tripleo11:13
quique|rover|lchHello11:14
*** asbishop has joined #tripleo11:15
quique|rover|lchRunning some noop changes we get the following https://bugs.launchpad.net/tripleo/+bug/177804011:15
openstackLaunchpad bug 1778040 in tripleo "Error at overcloud_prep_containers Package: qpid-dispatch-router-0.8.0-1.el7.x86_64 (@delorean-master-testing)", " Requires: libqpid-proton.so.10()(64bit)" [Critical,New] - Assigned to Quique Llorente (quiquell)11:15
*** pdeore_ has quit IRC11:18
openstackgerritChristian Schwede proposed openstack/tripleo-heat-templates stable/queens: Disable pid file usage in the swift_rsync container  https://review.openstack.org/57712611:19
openstackgerritSteven Hardy proposed openstack/tripleo-common master: filter available role by tags  https://review.openstack.org/57681411:19
openstackgerritSteven Hardy proposed openstack/tripleo-common master: Add workflow to get available plans  https://review.openstack.org/57712711:19
shardyjtomasek: ^^ FYI I've tested that with the UI and it works well to filter the available roles11:19
*** rcernin has quit IRC11:19
*** stendulker_ has quit IRC11:19
shardyjtomasek: the next step is add a workflow to choose one of the plan-samples, and to regenerate the roles_data.yaml if it's not the default plan11:20
shardyjtomasek: when you have time I'd welcome any review to ensure this fits with the "create plan" UI workflow we discussed11:20
jtomasekshardy: very cool, I am going to play with the patches today11:20
*** pradk has joined #tripleo11:20
*** udesale__ has quit IRC11:20
openstackgerritGabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP][DNM] test to gather zuul inventory informations  https://review.openstack.org/57687911:20
*** pchavva has joined #tripleo11:21
*** udesale__ has joined #tripleo11:21
shardyjtomasek: one workaround is needed atm - you have to create a roles data that only contains the OpenShift* roles as the default one won't work when the openshift plan is enabled11:21
jtomasekok11:22
shardyI'll work on doing that automatically, but probably we should only do it when plan tags are specified, otherwise we'll create a roles_data with every possible role in it ;)11:22
shardyalso we'll need to be a bit careful with the CLI workflow that allows user provided roles_data and plan-environment files11:22
*** lvdombrkr89 has joined #tripleo11:24
*** lvdombrkr has quit IRC11:25
quique|rover|lchlooks like we are missing installing one package that provides libqpid-proton.so11:26
quique|rover|lchqpid-proton-c11:27
*** udesale__ has quit IRC11:27
*** morazi has joined #tripleo11:29
*** ratailor has quit IRC11:30
flaper87shardy: can you spot something wrong here? https://review.openstack.org/#/c/574233/11:33
flaper87shardy: didn't seem to work :(11:33
flaper87shardy: https://review.openstack.org/#/c/574233/17/ci/environments/disable-unbound.yaml11:33
* flaper87 checks if it was actually executed11:34
flaper87shardy: mmh, http://logs.openstack.org/33/574233/17/experimental/tripleo-ci-centos-7-scenario009-multinode-oooq/ea2637e/logs/undercloud/var/lib/mistral/config-download-latest/Controller/host_prep_tasks.yaml.txt.gz11:37
flaper87I was expecting to see it there11:37
flaper87oh, damn, I know11:38
flaper87mmh, I think :P11:39
*** rodolof has quit IRC11:39
*** rodolof has joined #tripleo11:40
*** ansmith has quit IRC11:43
*** root__ has joined #tripleo11:48
*** atarlov has joined #tripleo11:50
quique|rover|lchamoralej: I see your name here https://cbs.centos.org/koji/buildinfo?buildID=2295911:50
quique|rover|lchamoralej: Did you update qpid-dispatch ?11:50
quique|rover|lchamoralej: Looks like we are getting the wrong one and not yours11:50
*** lvdombrkr89 has quit IRC11:51
amoralejquique|rover|lch, it's my name because of automation11:51
amoralejbut anyway11:51
amoralejthat build is not push11:51
amoralejto the repo11:51
*** quique|rover|lch is now known as quiquell|rover11:51
amoralejwhere are you getting the wrong one?11:51
quiquell|roveramoralej: https://bugs.launchpad.net/tripleo/+bug/177804011:51
openstackLaunchpad bug 1778040 in tripleo "Error at overcloud_prep_containers Package: qpid-dispatch-router-0.8.0-1.el7.x86_64 (@delorean-master-testing)", " Requires: libqpid-proton.so.10()(64bit)" [Critical,New] - Assigned to Quique Llorente (quiquell)11:51
*** rh-jelabarre has joined #tripleo11:51
quiquell|roveramoralej: Don't know yet if the problem is the wrong package version for qpid-proton-c11:52
quiquell|roveramoralej: Maybe other package install mor modern version of the -c and breaks the other11:52
quiquell|roverlibqpid-proton.so.10 is missing11:52
quiquell|roverbut libqpid-proton.so.11 is what get installed instead11:52
amoraleji'm in a call, i'll check11:53
amoralejnumber80 is updating qpid in rdo11:53
quiquell|roveramoralej: np, let me know when you are back11:53
quiquell|roveramoralej: Ok11:53
amoraleji think it's in some intermediate status now11:53
*** dhill_ has joined #tripleo11:54
*** nyechiel has joined #tripleo11:59
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: DNM: [WIP] Native ZuulV3 TripleO CI job  https://review.openstack.org/57695612:01
*** trown|outtypewww is now known as trown12:05
*** raildo has joined #tripleo12:05
amoralejquiquell|rover, there is a package "python2-qpid-proton-0.21.0-2.el7" that i don't know where is it coming from12:06
openstackgerritsunqingliang proposed openstack/tripleo-quickstart master: add a link to release notes in README file  https://review.openstack.org/57713312:09
*** ooolpbot has joined #tripleo12:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION12:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177804012:10
*** ooolpbot has quit IRC12:10
openstackLaunchpad bug 1778040 in tripleo "Error at overcloud_prep_containers Package: qpid-dispatch-router-0.8.0-1.el7.x86_64 (@delorean-master-testing)", " Requires: libqpid-proton.so.10()(64bit)" [Critical,New] - Assigned to Quique Llorente (quiquell)12:10
*** amoralej is now known as amoralej|lunch12:10
*** atoth has joined #tripleo12:11
*** apetrich has joined #tripleo12:12
*** rlandy has joined #tripleo12:14
root__dsneddon : ping12:19
openstackgerritMerged openstack/diskimage-builder master: Convert labels to upper case  https://review.openstack.org/57609812:20
*** ohochman has joined #tripleo12:20
openstackgerritsunqingliang proposed openstack/tripleo-heat-templates master: add a link to releasenotes in README file  https://review.openstack.org/57713812:21
*** liverpooler has joined #tripleo12:24
openstackgerritMartin André proposed openstack/ansible-role-tripleo-modify-image master: Do not fail when repoquery is not installed  https://review.openstack.org/57682512:26
*** mcornea has joined #tripleo12:27
*** root__ is now known as lvdombrkr12:28
openstackgerritsunqingliang proposed openstack/tripleo-quickstart master: fix typos  https://review.openstack.org/57714012:29
*** psahoo has quit IRC12:30
*** aufi_ has joined #tripleo12:33
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: reparent from multinode  https://review.openstack.org/57714312:34
*** maufart__ has quit IRC12:36
weshay|ruckccamacho++12:37
ccamachoweshay|ruck do you have superpowers? :P https://review.openstack.org/#/c/577106/12:38
weshay|ruckccamacho, I just pinged doug12:38
ccamachoyeeeeehiii! :) thanks!12:38
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Parallelize server pre and post steps  https://review.openstack.org/57447412:39
*** cshastri has quit IRC12:41
dmelladoccamacho: there you go! xD12:41
*** links has quit IRC12:43
ccamachoweshay|ruck dmellado https://staffino.com/blog/wp-content/uploads/2016/09/59349318.jpg12:44
weshay|ruck:)12:44
weshay|ruckI'll drink in the am, cheers12:44
dmelladoccamacho: we got super dhellmann xD12:44
*** cdearborn has joined #tripleo12:47
*** mhenkel_ has joined #tripleo12:49
*** leanderthal has joined #tripleo12:50
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Fix centos opstools repo deactivation  https://review.openstack.org/57714512:53
*** mhenkel_ has quit IRC12:54
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart-extras master: Fix centos opstools repo deactivation  https://review.openstack.org/57714612:54
*** mhenkel_ has joined #tripleo12:54
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Add Network Topology diagram  https://review.openstack.org/56912512:55
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Warn when some network resources are not available  https://review.openstack.org/57611312:55
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Update to React 16.4.1  https://review.openstack.org/57684412:55
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Highlight assigned networks and roles on hover  https://review.openstack.org/57684512:55
*** toure|gone is now known as toure12:56
*** edmondsw has joined #tripleo12:58
*** maufart__ has joined #tripleo13:01
openstackgerritBogdan Dobrelya proposed openstack/python-tripleoclient master: Leverage log_file option to capture more UC logs  https://review.openstack.org/57649813:03
bogdandoPTAL folks ^^13:03
*** salmankhan has quit IRC13:04
*** aufi_ has quit IRC13:04
*** salmankhan has joined #tripleo13:04
openstackgerritJon Thomas proposed openstack/tripleo-validations master: Adds heat-manage purge_deleted cron job validation  https://review.openstack.org/57715013:05
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: DNM: [WIP] Native ZuulV3 TripleO CI job  https://review.openstack.org/57695613:06
*** moshele has quit IRC13:07
*** myoung|off is now known as myoung13:09
*** ooolpbot has joined #tripleo13:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION13:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177804013:10
*** ooolpbot has quit IRC13:10
openstackLaunchpad bug 1778040 in tripleo "Error at overcloud_prep_containers Package: qpid-dispatch-router-0.8.0-1.el7.x86_64 (@delorean-master-testing)", " Requires: libqpid-proton.so.10()(64bit)" [Critical,In progress] - Assigned to Quique Llorente (quiquell)13:10
quiquell|roverjoin #rdo13:11
quiquell|roverups13:11
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: WIP: move cellv2 discovery from control plane services to compute services  https://review.openstack.org/57648113:11
openstackgerritFlavio Percoco proposed openstack/tripleo-heat-templates master: Update for openshift 3.9  https://review.openstack.org/57423313:13
openstackgerritDerek Higgins proposed openstack/tripleo-quickstart master: Add featureset044 for Ironic in the Overcloud  https://review.openstack.org/50982913:15
openstackgerritBogdan Dobrelya proposed openstack/python-tripleoclient master: Leverage log_file option to capture more UC logs  https://review.openstack.org/57649813:15
*** ansmith has joined #tripleo13:16
quiquell|roverNeed some reviews for a blocker https://review.openstack.org/#/c/577145/13:21
*** mmethot has joined #tripleo13:21
*** mmethot is now known as mmethot|hackfest13:22
*** mmethot|hackfest is now known as mmethot13:22
Tenguhello there :)13:29
weshay|ruckbogdando, do you have time for some questions related to container health on the undercloud?13:30
weshay|ruckseeing some containers marked as restarting, nova again13:30
weshay|ruckI think it's leading to some errors while importing the overcloud images13:30
weshay|ruckaeead3796578        192.168.24.1:8787/tripleomaster/centos-binary-ironic-conductor:1c741636c7cf2c1eb6eacd37e4b0bf811fd354a0_2b16a546-updated-20180621000309            "kolla_start"            24 minutes ago      Restarting (0) 10 minutes ago                          ironic_conductor                                             160 kB (virtual 1.02 GB)13:31
weshay|ruck7c9f07ee6791        192.168.24.1:8787/tripleomaster/centos-binary-ironic-neutron-agent:1c741636c7cf2c1eb6eacd37e4b0bf811fd354a0_2b16a546-updated-20180621000309        "kolla_start"            24 minutes ago      Restarting (1) 10 minutes ago                          ironic_neutron_agent13:31
bogdandoweshay|ruck: oh, not for ironic stuff prolly...13:32
bogdandobut I can try13:32
weshay|ruckerror is ocaldomain:5672 after None tries: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication me│overcloud-full.d13:33
weshay|ruckchanism AMQPLAIN. For details see the broker logfile.13:33
bogdandoweshay|ruck: IMO that's higly likely cuz of puppets magic13:33
bogdandothose deprecations removed13:33
weshay|ruckugh13:33
bogdandoweshay|ruck: https://bugs.launchpad.net/tripleo/+bug/177764313:35
openstackLaunchpad bug 1777643 in tripleo "t-h-t hiera rabbit_* overrides stopped working because of deprecated puppet params got removed" [Critical,New]13:35
bogdandobut needs to be confirmed from logs13:35
weshay|ruckbogdando, k.. so the odd thing about this.. is that it's working in other jobs on the same hash13:36
bogdandoI can see ironic-base and nova-base in the list, so it's a check!13:36
weshay|ruckbogdando, ya.. it does look like it doesn't it13:36
bogdandohttp://codesearch.openstack.org/?q=rabbit_&i=nope&files=puppet%5C%2Fservices&repos=tripleo-heat-templates , that list I mean13:36
bogdandoweshay|ruck: hm13:36
weshay|ruckbogdando, maybe there is diff in packaging in the diff jobs.. but you are right.. that certainly looks like the right rabbit hole :)13:37
bogdandoweshay|ruck: do you have logs for the job?13:37
weshay|ruckhttps://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-master-upload/2d572f1/undercloud/home/jenkins/overcloud_prep_images.log.txt.gz#_2018-06-21_01_09_5913:38
weshay|ruckhttps://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-master-upload/2d572f1/undercloud/var/log/extra/docker/docker_allinfo.log.txt.gz13:38
weshay|ruckhttps://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-master-upload/2d572f1/undercloud/var/log/extra/errors.txt.gz13:39
*** derekh_ is now known as derekh13:43
*** lblanchard has joined #tripleo13:43
*** amoralej|lunch is now known as amoralej13:44
*** spsurya has joined #tripleo13:44
atarlovhi guys, can somebody point to me how is yum_update.sh is consumed in heat-config-scripts13:46
*** jistr is now known as jistr|mtg13:49
mwhahahaatarlov: what version? cause i don't think we use it anymore13:52
*** Skyraven has joined #tripleo13:52
*** vinaykns has joined #tripleo13:53
atarlovnewton, but i still see it in queens13:54
*** ccamacho1 has joined #tripleo13:54
*** ccamacho has quit IRC13:54
*** skramaja has quit IRC13:54
bogdandoweshay|ruck: so there is no rabbit_user/pass defined with puppet in the container config13:54
bogdandoit means it tries auth with guest/guest defaults13:55
weshay|ruckwhich line is that13:55
bogdandoI think this confirms that bug I referred above13:55
mwhahahaatarlov: the file might still exist but i think the update flow now uses ansible for that logic13:55
mwhahahaatarlov: it was usually executed in the update workflow13:56
*** janki has quit IRC13:56
mwhahahaatarlov: it gets pulled in via https://github.com/openstack/tripleo-heat-templates/blob/stable/newton/overcloud-resource-registry-puppet.j2.yaml#L1313:59
mwhahahaatarlov: which gets pulled in in the role via https://github.com/openstack/tripleo-heat-templates/blob/stable/newton/puppet/controller-role.yaml#L376-L38814:00
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart master: [WIP] [DNM]: Create zuul-hosts file for pre.yaml network setup  https://review.openstack.org/57700714:01
openstackgerritQuique Llorente proposed openstack/tripleo-quickstart master: Upgrade ansible to 2.5.5  https://review.openstack.org/57716814:03
*** waleedm has quit IRC14:05
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart master: Create zuul-hosts file for pre.yaml network setup  https://review.openstack.org/57700714:08
*** quiquell|rover is now known as quiquell|off14:09
*** lblanchard has quit IRC14:09
*** ykarel is now known as ykarel|away14:12
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: WIP: Add functionality to copy modprobe files  https://review.openstack.org/57717014:12
toureweshay|ruck ping14:16
*** links has joined #tripleo14:17
weshay|rucktoure, aye?14:18
touredoes your patch for container variables work more or less14:18
weshay|ruckwhich patch?14:19
*** janki has joined #tripleo14:19
tourehttps://review.openstack.org/#/c/576904/14:19
tourejust hit this problem on a deployment14:19
weshay|rucktoure, I have not had a chance to test it locally14:20
toureack14:20
tourewill pull it and try it out14:20
*** links has quit IRC14:23
*** rpioso|afk is now known as rpioso14:26
*** itlinux has quit IRC14:31
*** dxiri has joined #tripleo14:37
*** agurenko has joined #tripleo14:42
openstackgerritAlex Schultz proposed openstack/instack-undercloud master: Exclude bad version of dulwich  https://review.openstack.org/57719314:44
openstackgerrityolanda.robla proposed openstack/diskimage-builder master: WIP: Add new modprobe element  https://review.openstack.org/57717014:52
*** pcaruana has quit IRC14:56
*** rodolof has quit IRC14:58
*** rodolof has joined #tripleo14:59
*** dmacpher has joined #tripleo15:00
*** vpickard is now known as vpickard_15:00
*** jaganathan has quit IRC15:02
*** apetrich has quit IRC15:05
*** mmethot has quit IRC15:06
*** bogdando has quit IRC15:07
*** ratailor has joined #tripleo15:08
*** ramishra has quit IRC15:09
*** fzdarsky_ is now known as fzdarsky15:11
*** gkadam has quit IRC15:17
*** dparkes has quit IRC15:17
*** itlinux has joined #tripleo15:19
*** jaganathan has joined #tripleo15:20
*** etingof has quit IRC15:20
openstackgerritAndrew Austin proposed openstack/tripleo-heat-templates master: Prevent neutron-l3-agent from running on computes when using OVN DVR.  https://review.openstack.org/57721915:21
*** jaganathan has quit IRC15:23
*** jaganathan has joined #tripleo15:23
larsksThe puppet module I'm editing uses <%= scope[...]-%>.  Other templates use <%= $varname %> while yet others use <%= @varname %>. Which is preferred in modern puppet?15:26
openstackgerritMatt Young proposed openstack/tripleo-quickstart master: Let's tempestconf tool handle swift related conf  https://review.openstack.org/57322015:28
mwhahahalarsks: I think @varname is what you want15:29
larsksSo many choices! :)  Thanks.15:29
*** yprokule has quit IRC15:30
*** Goneri has joined #tripleo15:30
*** dxiri has quit IRC15:30
*** dxiri has joined #tripleo15:31
*** mmethot has joined #tripleo15:33
*** rodolof has quit IRC15:34
*** etingof has joined #tripleo15:34
*** rodolof has joined #tripleo15:35
*** jcoufal has quit IRC15:36
*** jistr|mtg is now known as jistr15:36
openstackgerritRonelle Landy proposed openstack-infra/tripleo-ci master: DNM: Testing using zuul vxlan networking  https://review.openstack.org/57664515:39
*** Skyraven has quit IRC15:40
*** maufart__ has quit IRC15:41
*** jistr is now known as jistr|off15:47
*** ykarel|away has quit IRC15:47
*** ramishra has joined #tripleo15:47
weshay|rucktoure, did that work for you?15:50
tourestill working, lets see if it hits the problem15:53
*** bnemec has quit IRC15:53
*** assassin has quit IRC15:53
weshay|rucktoure, thanks15:54
toureno thank you :)15:55
weshay|ruckno thank you15:55
toureno thank you :)15:56
tourelol15:56
toure:)15:56
*** pkovar has quit IRC15:56
*** leanderthal has quit IRC15:56
*** Guest88320 is now known as dansmith15:59
*** bnemec has joined #tripleo15:59
openstackgerritMerged openstack/tripleo-upgrade stable/queens: Use up to date parameter name for cli options conversion  https://review.openstack.org/57692116:01
*** jpich has quit IRC16:03
*** lvdombrkr has quit IRC16:04
openstackgerritDerek Higgins proposed openstack/tripleo-quickstart-extras master: Setup Ironic in Overcloud  https://review.openstack.org/50972816:05
*** ramishra has quit IRC16:05
openstackgerritDerek Higgins proposed openstack/tripleo-quickstart-extras master: Setup Ironic in Overcloud  https://review.openstack.org/50972816:08
*** dparkes has joined #tripleo16:08
*** ratailor_ has joined #tripleo16:08
*** atarlov has quit IRC16:09
*** gfidente has quit IRC16:09
*** tesseract has quit IRC16:09
*** sanjayu_ has quit IRC16:09
openstackgerritMartin Kopec proposed openstack/tripleo-docs master: Added Tempest Documentation for Tripleo  https://review.openstack.org/56516116:10
*** atarlov has joined #tripleo16:10
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: featureset002 was lagging behind featureset001  https://review.openstack.org/57698716:11
*** ratailor has quit IRC16:12
*** jillr has joined #tripleo16:12
*** ratailor_ has quit IRC16:12
*** agurenko has quit IRC16:13
*** panda is now known as panda|off16:13
*** atarlov has quit IRC16:14
*** d0ugal has quit IRC16:23
sri_mwhahaha, Hi one newbie question, how do I enable External network in my overcloud, I have two nics second nic I am using for provisioning and nic1 is for External/Public16:23
mwhahahasri_: usually you have to create an external network in neutron that you pass in the appropriate config options for.16:24
mwhahahasri_: depends on how you've configured the external network and what ml2 stuff you're using16:24
mwhahahasri_: for the default ovs & tripleo case you would do something like: openstack network create --external --provider-physical-network datacentre --provider-network-type flat public16:25
mwhahahathat's what i used for the standalone case and it worked16:25
mwhahahawe don't document this well16:25
*** trown is now known as trown|lunch16:27
*** spsurya has quit IRC16:28
sri_mwhahaha, in my setup only first has default gateway, when i create simple overcloud all openstack public endpoints (dashboard ) created in provisioning network , so I cant access those from my desktop16:28
mwhahahasri_: oh do you mean to access the test overcloud?16:29
mwhahahasri_: so you could use something like sshuttle to forward the network on your host16:29
*** amoralej is now known as amoralej|off16:29
*** yamahata has quit IRC16:30
sri_mwhahaha, Yes, but if i configure the nic1 in controller with routable ip address then we can talk to those apis16:30
mwhahahaso that's a network config option to configure the secondary interfaces and put the public network on it16:31
mwhahahai thought you meant for the cloud instances16:31
*** vpickard_ is now known as vpickard16:32
*** zoli is now known as zoli|gone16:32
sri_mwhahaha, no, i mean compute and controller16:32
mwhahahai'd assume you'd just use something like https://github.com/openstack/tripleo-heat-templates/tree/master/network/config/multiple-nics but that's a question for like dsneddon / bfournie16:33
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart master: [DNM] Add local network-environment file for fs001  https://review.openstack.org/57723016:33
sri_mwhahaha, I was trying to do this https://docs.openstack.org/tripleo-docs/latest/install/environments/baremetal.html16:33
sri_https://docs.openstack.org/tripleo-docs/latest/_images/TripleO_Network_Diagram_.jpg16:34
mwhahahayea you're running into issues configuring the network part, so like https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/network_isolation.html16:35
*** arxcruz is now known as arxcruz|off16:36
*** ccamacho1 has quit IRC16:36
*** ffiore has quit IRC16:38
sri_mwhahaha, Yes,  in pike we have heat-template like compute,controller in /network/config/... I in queens we are generating those templates using jinja right ?16:39
mwhahahayea16:39
mwhahahathere's a new format too, dsneddon and bfournie are best for those questions16:40
dsneddonsri_, You can generate the NIC configs with a script, then modify them further if need be. The script is in the tripleo-heat-templates directory, in ./tools/process_templates.py.16:40
openstackgerritJill Rouleau proposed openstack/tripleo-heat-templates master: Enable Ansible error handling per role  https://review.openstack.org/57668516:40
dsneddonsri_, You can run process_templates.py --help to get more info, but you can pass a custom network_data.yaml and roles_data.yaml if you have one, as well as an output directory.16:41
dsneddonsri_, If you have old NIC config templates from a previous installation, there is also a conversion tool in that same directory.16:42
mwhahahasdoran: did they change how local ssh connections are handled between ansible 2.5.2 and 2.5.416:42
*** shardy has quit IRC16:42
mwhahahasdoran: context is https://bugzilla.redhat.com/show_bug.cgi?id=1593345 we don't get these failures upstream because we're still using 2.5.216:42
openstackbugzilla.redhat.com bug 1593345 in openstack-tripleo "Undercloud is not reachable by mistral: "Authentication or permission failure"" [High,New] - Assigned to jslagle16:42
sdoranNot to my knowledge.16:42
sdoranWhat are seeing?16:43
sdoran*you16:43
mwhahahasdoran: our fact gathering against localhost is trying ssh16:43
mwhahahasdoran: but we configure...16:43
mwhahahasdoran: https://logs.rdoproject.org/88/576888/8/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/Z65ca4b68986b4b47b97ea1436c4bf565/undercloud/var/lib/mistral/d0cd9086-7794-4473-ae31-cdde6ac1d3ec/tripleo-ansible-inventory.yaml.txt.gz16:43
*** holser_ has quit IRC16:43
mwhahahasdoran: so we set ansible_host to localhost and ansible_connection to local for undercloud16:43
slaglei think it's because it's trying to use /home/mistral as local_tmp16:43
sri_dsneddon, understood, one more question is network-environment.yaml and network_data.yaml do the same thing ?16:43
slaglei don't think ssh is involved16:44
mwhahahaslagle: hmm ok so the local temp changed?16:44
dsneddonsri_, No, network_data.yaml specifies the network parameters, and network_environment.yaml specify some additional local parameters.16:44
slaglein the mistral action, we set HOME in the env dict to work_dir (which is /var/lib/mistral)16:44
slaglethat must not be working downstream for some reason16:44
mwhahahaslagle: yea that's why i asked about changes in ansible cause that's the first difference i noticed16:45
dsneddonsri_, Behind the scenes, network_data.yaml is used by Jinja to create the templates, and network-environment.yaml is used to pass Heat parameters at deploy time.16:45
mwhahahaslagle: if i change remote_tmp it worked fine16:45
mwhahahaslagle: which mad me think it was sshing16:45
*** derekh has quit IRC16:45
dsneddonsri_, But if you run process_templates.py, it will create environments/network-environment.yaml for you to use as a starting point.16:45
slaglemwhahaha: oh, hmm. i didn't even know remote_tmp was an option.16:46
slagleundocumented16:46
mwhahahaslagle: https://davesnigier.com/changing-ansible-temporary-directory/16:46
sri_dsneddon, I see I will try generateing templates using process_templates.py.16:46
mwhahahaslagle: but would be a security issue, so i was looking into something else :D16:46
mwhahahacause i don't think we should change it completely16:47
sri_dsneddon, mwhahaha : thanks guys16:47
sdoranmwhahaha: It’s failing to create the tmp file. So maybe /home/mistral doesn’t exist.16:47
mwhahahasdoran: it doesn't16:47
mwhahahasdoran: but that's a change16:47
slaglemwhahaha: this is what should be handling it: https://github.com/openstack/tripleo-common/blob/master/tripleo_common/actions/ansible.py#L48416:47
mwhahahasdoran: we dont' see this in upstream16:47
sdoranI believe we did make some changes to temp file handing but I’m not sure.16:48
mwhahahaslagle: yea that's set, but not getting hornoed https://logs.rdoproject.org/88/576888/8/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/Z65ca4b68986b4b47b97ea1436c4bf565/undercloud/var/lib/mistral/d0cd9086-7794-4473-ae31-cdde6ac1d3ec/ansible-playbook-command.sh.txt.gz16:48
mwhahahaso there's our problem16:48
slaglechange in processutils perhaps?16:48
mwhahahaansible changed something :D16:48
sdoranThat might only be in 2.6/2.7.16:48
mwhahahawell we're using 2.5.4 downstream16:49
mwhahahaand 2.5.2 upstream but it's fine16:49
slaglemwhahaha: are there downstream CI logs? check what HOME is set to in ansible-playbook-command.sh16:49
sdoranWhere is it putting temp files in 2.5.2?16:49
mwhahahaslagle: in that bug, we have ci jobs16:50
slaglemwhahaha: ok16:50
mwhahahaslagle: see https://bugzilla.redhat.com/show_bug.cgi?id=159334516:50
openstackbugzilla.redhat.com bug 1593345 in openstack-tripleo "Undercloud is not reachable by mistral: "Authentication or permission failure"" [High,New] - Assigned to jslagle16:50
mwhahahasdoran: i assume in the $HOME we configure in the bash script16:50
*** raopajay has joined #tripleo16:50
*** florianf is now known as florianf|afk16:51
*** kopecmartin has quit IRC16:52
*** dtantsur is now known as dtantsur|afk16:52
slagle$HOME is set correct: http://paste.openstack.org/show/724047/16:52
mwhahahayea i have a host where i can reproduce this16:52
mwhahahaso now i'm just trying to track it down16:52
mwhahahait's specifically how ansible is hanlding something16:52
mwhahahaso i guess i need to go diff the packages16:52
slaglewell either that or oslo-processutils in it's environment handling16:53
mwhahahaslagle: i can repro in bash16:53
mwhahahaif login as the mistral user and bash run the script it fails16:53
slagleoh ok16:53
mwhahahaso maybe ansible isn't honoring $HOME anymore16:54
sdoranIt could be using the home from the user settings rather than env.16:56
sdoranI’m asking around.16:56
mwhahahawhich would be a regression16:56
mwhahahabecause we were levering the env previously for the local connection16:56
sdoranI can bisect it if I don’t get an answer from the team.16:57
mwhahahai'm pulling down the rpms to see what changed between 2.5.2 and 2.5.416:57
*** psachin has quit IRC16:58
*** yamahata has joined #tripleo16:59
*** ksambor has quit IRC17:03
mwhahahahttps://github.com/ansible/ansible/commit/864fd7c53e45703554bb6de608fe13a2200b6aa017:04
mwhahahaits likely that17:04
mwhahahasdoran: -^17:04
*** ansmith has quit IRC17:10
mwhahahasdoran: so the quesiton is how do we configure DEFAULT_LOCAL_TMP, i tried ANSIBLE_LOCAL_TEMP and neither are working17:10
*** jpena is now known as jpena|off17:12
*** ansmith has joined #tripleo17:14
*** cylopez has joined #tripleo17:14
*** cylopez has left #tripleo17:15
*** moshele has joined #tripleo17:17
*** trown|lunch is now known as trown17:31
*** salmankhan has quit IRC17:31
openstackgerritMarius Cornea proposed openstack/tripleo-upgrade master: WIP: Adjust roles_data during FFU  https://review.openstack.org/55808117:31
openstackgerritMarius Cornea proposed openstack/tripleo-upgrade master: WIP: Adjust roles_data during FFU  https://review.openstack.org/55808117:32
*** moshele has quit IRC17:32
*** amoralej|off is now known as amoralej17:33
*** suuuper has quit IRC17:34
*** mhenkel_ has quit IRC17:34
*** atarlov has joined #tripleo17:36
*** atarlov has quit IRC17:40
mcorneaweshay|ruck: hi we have a gate job in tripleo-upgrade that keeps failing during undercloud installation - http://logs.openstack.org/08/577108/1/check/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/1b02961/17:50
mcorneaweshay|ruck: is this a known issue?17:50
weshay|ruckdepends what the issue is17:51
weshay|ruckwhere is it failing17:51
mcorneaweshay|ruck: http://logs.openstack.org/46/574646/3/gate/tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates/bdd5eba/logs/undercloud/home/zuul/undercloud_install.log.txt.gz17:52
mwhahahalibqpid-proton.so.1017:52
mwhahahayea that's the opstools thing17:52
mcorneais it something that's being worked on? can I help somehow? currently this is blocking the tripleo-upgrade patches17:54
weshay|ruckhttps://bugs.launchpad.net/tripleo/+bug/177804017:56
openstackLaunchpad bug 1778040 in tripleo "Error at overcloud_prep_containers Package: qpid-dispatch-router-0.8.0-1.el7.x86_64 (@delorean-master-testing)", " Requires: libqpid-proton.so.10()(64bit)" [Critical,In progress] - Assigned to Quique Llorente (quiquell)17:56
openstackgerritJon Schlueter proposed openstack/tripleo-common master: overcloud_containers.yaml.j2: remove unused *-base entry  https://review.openstack.org/51103917:57
mcorneathanks17:57
openstackgerritJon Schlueter proposed openstack/tripleo-common master: overcloud_containers.yaml.j2: remove unused *-base entry  https://review.openstack.org/51103917:57
mwhahahasdoran: where to report ansible bugs downstream?17:58
sdoranGitHub.17:58
*** moshele has joined #tripleo17:59
sdoran`ANSIBLE_LOCAL_TEMP` should be the correct setting.18:01
mwhahahait doesn't work18:01
mwhahahai've got a really simple reproducer18:01
mwhahahacreate a user with a non-existant home dir and it doesn't work18:01
*** mhenkel_ has joined #tripleo18:06
*** mhenkel_ has quit IRC18:08
sdoranIf you write up the issue, I will work on it. Or I can write it up. Whatever works for you.18:08
*** mhenkel_ has joined #tripleo18:08
mwhahahasdoran: writing it up now18:08
sdoranMany thanks.18:08
*** ooolpbot has joined #tripleo18:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION18:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177804018:10
*** ooolpbot has quit IRC18:10
openstackLaunchpad bug 1778040 in tripleo "Error at overcloud_prep_containers Package: qpid-dispatch-router-0.8.0-1.el7.x86_64 (@delorean-master-testing)", " Requires: libqpid-proton.so.10()(64bit)" [Critical,In progress] - Assigned to Quique Llorente (quiquell)18:10
*** mhenkel_ has quit IRC18:10
*** mhenkel_ has joined #tripleo18:12
*** apetrich has joined #tripleo18:16
openstackgerritMerged openstack/instack-undercloud stable/queens: Fall back to puppet-ntp defaults  https://review.openstack.org/57645018:22
openstackgerritMerged openstack/tripleo-heat-templates master: Introduces NovaComputeCpuSharedSet parameter  https://review.openstack.org/57544918:22
mwhahahasdoran: https://github.com/ansible/ansible/issues/4180818:23
*** moshele has quit IRC18:23
sdoran👍18:24
mwhahahasdoran: if you can make it work witht eh ANSIBLE_LOCAL_TEMP that's cool too, we just need a way of specifying that stuff18:25
*** janki has quit IRC18:25
sdoranAnsible Runner could help a good bit.18:26
sdoranBut I need to look into this bug to see if that setting is even being honored properly.18:27
mwhahahasdoran: it sounds like it should work with ANSIBLE_LOCAL_TEMP if my referenced change was the one that affected it18:28
mwhahahasdoran: the problem is that this would be a regression in 13 when this lands which is a problem so we can't necessarily solve it by switching to ansible runner18:28
*** Goneri has quit IRC18:30
*** jaganathan has quit IRC18:32
openstackgerritKarthik S proposed openstack/os-net-config master: WIP: Adding VLAN support and other configs for sriov_vf  https://review.openstack.org/54162018:33
*** gkadam has joined #tripleo18:36
*** derekh has joined #tripleo18:36
openstackgerritAlex Schultz proposed openstack/instack-undercloud stable/pike: Always deploy NTP  https://review.openstack.org/57725218:42
openstackgerritAlex Schultz proposed openstack/instack-undercloud stable/pike: Fall back to puppet-ntp defaults  https://review.openstack.org/57725418:42
openstackgerritAlex Schultz proposed openstack/instack-undercloud stable/pike: Fall back to puppet-ntp defaults  https://review.openstack.org/57725418:42
weshay|ruckqpid issue is hitting the undercloud too18:42
weshay|ruckoh.. originally opened on that18:43
weshay|rucknvrmind18:43
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: Remove unnecessary code from toci_* scripts  https://review.openstack.org/57683418:46
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: add scen001 pike check to cover additional scenarios  https://review.openstack.org/57685818:49
openstackgerritwes hayutin proposed openstack-infra/tripleo-ci master: update branchful coverage for queens  https://review.openstack.org/57695718:51
openstackgerritDan Sneddon proposed openstack/tripleo-heat-templates master: Fix incorrect Redis VIP declaration in fixed-ip-vips-v6.yaml  https://review.openstack.org/57726018:53
*** rodolof has quit IRC18:55
*** gkadam has quit IRC18:55
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: WIP: use the tq ansible.cfg for the undercloud install  https://review.openstack.org/57553518:58
*** amoralej is now known as amoralej|off18:59
*** moshele has joined #tripleo19:02
openstackgerritHarald Jensås proposed openstack/tripleo-heat-templates master: Get ctlplane interface config from the subnet  https://review.openstack.org/56954719:03
*** holser_ has joined #tripleo19:03
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: WIP, fix container variables  https://review.openstack.org/57690419:06
*** ooolpbot has joined #tripleo19:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION19:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177804019:10
*** ooolpbot has quit IRC19:10
openstackLaunchpad bug 1778040 in tripleo "Error at overcloud_prep_containers Package: qpid-dispatch-router-0.8.0-1.el7.x86_64 (@delorean-master-testing)", " Requires: libqpid-proton.so.10()(64bit)" [Critical,In progress] - Assigned to Quique Llorente (quiquell)19:10
*** weshay|ruck is now known as weshay_PTO19:12
*** tcw has joined #tripleo19:13
*** tcw has quit IRC19:14
*** tcw has joined #tripleo19:14
*** tcw has quit IRC19:15
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Warn when some network resources are not available  https://review.openstack.org/57611319:17
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Update to React 16.4.1  https://review.openstack.org/57684419:17
openstackgerritJiri Tomasek proposed openstack/tripleo-ui master: Highlight assigned networks and roles on hover  https://review.openstack.org/57684519:17
*** moshele has quit IRC19:19
*** derekh has quit IRC19:19
*** tcw has joined #tripleo19:20
*** Haresh has quit IRC19:24
openstackgerritRajini Karthik proposed openstack/puppet-tripleo master: Update DellEMC Manila VNX driver  https://review.openstack.org/57727819:24
openstackgerritJon Schlueter proposed openstack/tripleo-common master: overcloud_containers.yaml.j2: remove unused *-base entry  https://review.openstack.org/51103919:26
openstackgerritRajini Karthik proposed openstack/puppet-tripleo master: Update DellEMC Manila VNX driver  https://review.openstack.org/57727819:27
*** mfedosin has quit IRC19:32
*** tcw1 has joined #tripleo19:35
*** tcw has quit IRC19:35
openstackgerritRajini Karthik proposed openstack/puppet-tripleo master: Update DellEMC Manila Unity driver  https://review.openstack.org/57728219:39
*** tcw1 has quit IRC19:41
*** tcw has joined #tripleo19:42
*** mfedosin has joined #tripleo19:44
*** toure is now known as toure|biab19:46
*** holser_ has quit IRC19:53
*** holser_ has joined #tripleo19:54
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: DNM: [WIP] Native ZuulV3 TripleO CI job  https://review.openstack.org/57695620:04
*** yolanda_ has joined #tripleo20:07
*** ooolpbot has joined #tripleo20:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION20:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177804020:10
*** ooolpbot has quit IRC20:10
openstackLaunchpad bug 1778040 in tripleo "Error at overcloud_prep_containers Package: qpid-dispatch-router-0.8.0-1.el7.x86_64 (@delorean-master-testing)", " Requires: libqpid-proton.so.10()(64bit)" [Critical,In progress] - Assigned to Quique Llorente (quiquell)20:10
*** yolanda has quit IRC20:11
*** dhill_ has quit IRC20:11
*** liverpooler has quit IRC20:12
*** dhill_ has joined #tripleo20:15
*** pchavva has quit IRC20:16
*** lblanchard has joined #tripleo20:19
*** morazi has quit IRC20:20
openstackgerritRafael Folco proposed openstack-infra/tripleo-ci master: DNM: [WIP] Native ZuulV3 TripleO CI job  https://review.openstack.org/57695620:20
*** pchavva has joined #tripleo20:23
*** toure|biab is now known as toure20:33
*** ohochman has quit IRC20:33
*** paramite has quit IRC20:39
*** ansmith has quit IRC20:44
*** lblanchard has quit IRC20:47
*** trown is now known as trown|outtypewww20:48
*** raildo has quit IRC20:55
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: standalone run a post config to generate a clouds.yaml  https://review.openstack.org/57559721:07
*** ooolpbot has joined #tripleo21:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION21:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177804021:10
*** ooolpbot has quit IRC21:10
openstackLaunchpad bug 1778040 in tripleo "Error at overcloud_prep_containers Package: qpid-dispatch-router-0.8.0-1.el7.x86_64 (@delorean-master-testing)", " Requires: libqpid-proton.so.10()(64bit)" [Critical,In progress] - Assigned to Quique Llorente (quiquell)21:10
*** asbishop has quit IRC21:12
*** ayoung has joined #tripleo21:15
*** liverpooler has joined #tripleo21:16
*** toure is now known as toure|gone21:25
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: standalone run a post config to generate a clouds.yaml  https://review.openstack.org/57559721:26
*** dxiri has quit IRC21:27
*** dxiri has joined #tripleo21:28
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: standalone run a post config to generate a clouds.yaml  https://review.openstack.org/57559721:29
*** myoung is now known as myoung|off21:30
*** waleedm has joined #tripleo21:32
*** ledo_ has joined #tripleo21:33
*** mjturek has quit IRC21:36
*** waleedm has quit IRC21:37
*** pchavva has quit IRC21:40
*** d0ugal_ has joined #tripleo21:44
*** holser_ has quit IRC21:49
*** mjturek has joined #tripleo21:50
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: standalone run a post config to generate a clouds.yaml  https://review.openstack.org/57559721:51
*** liverpooler has quit IRC21:52
*** ansmith has joined #tripleo21:55
*** mmethot has quit IRC21:55
*** vpickard is now known as vpickard_21:57
*** cdearborn has quit IRC22:00
*** vinaykns has quit IRC22:00
*** itlinux has quit IRC22:04
*** bfournie has quit IRC22:04
*** liverpooler has joined #tripleo22:06
openstackgerritMerged openstack/tripleo-heat-templates stable/pike: Set Ceph pools rule_name to replicated_rule  https://review.openstack.org/57578922:07
*** mcornea has quit IRC22:07
*** ooolpbot has joined #tripleo22:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION22:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177804022:10
*** ooolpbot has quit IRC22:10
openstackLaunchpad bug 1778040 in tripleo "Error at overcloud_prep_containers Package: qpid-dispatch-router-0.8.0-1.el7.x86_64 (@delorean-master-testing)", " Requires: libqpid-proton.so.10()(64bit)" [Critical,In progress] - Assigned to Quique Llorente (quiquell)22:10
*** ledo_ has quit IRC22:16
*** holser_ has joined #tripleo22:18
*** mhenkel_ has quit IRC22:21
openstackgerritAndrew Austin proposed openstack/puppet-tripleo master: Add mechanism for cookie-based sessions in endpoints.  https://review.openstack.org/57730922:23
openstackgerritAndrew Austin proposed openstack/puppet-tripleo master: [WIP] Add mechanism for cookie-based sessions in endpoints.  https://review.openstack.org/57730922:24
*** edmondsw has quit IRC22:24
*** rcernin has joined #tripleo22:26
*** edmondsw has joined #tripleo22:26
*** mmethot has joined #tripleo22:28
*** moshele has joined #tripleo22:31
*** edmondsw has quit IRC22:35
*** edmondsw has joined #tripleo22:35
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: standalone run a post config to generate a clouds.yaml  https://review.openstack.org/57559722:36
*** marbindrakon has joined #tripleo22:40
*** edmondsw has quit IRC22:40
openstackgerritJill Rouleau proposed openstack/tripleo-common master: Enable Ansible error handling per role  https://review.openstack.org/57668622:43
*** mjturek has quit IRC22:43
*** moshele has quit IRC22:45
*** mhenkel_ has joined #tripleo22:51
*** mhenkel_ has quit IRC22:55
*** marbindrakon has quit IRC22:57
*** holser_ has quit IRC22:58
*** dhill_ has quit IRC22:58
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: standalone run a post config to generate a clouds.yaml  https://review.openstack.org/57559722:59
*** rlandy has quit IRC23:09
*** ooolpbot has joined #tripleo23:10
ooolpbotURGENT TRIPLEO TASKS NEED ATTENTION23:10
ooolpbothttps://bugs.launchpad.net/tripleo/+bug/177804023:10
*** ooolpbot has quit IRC23:10
openstackLaunchpad bug 1778040 in tripleo "Error at overcloud_prep_containers Package: qpid-dispatch-router-0.8.0-1.el7.x86_64 (@delorean-master-testing)", " Requires: libqpid-proton.so.10()(64bit)" [Critical,In progress] - Assigned to Quique Llorente (quiquell)23:10
*** itlinux has joined #tripleo23:10
*** liverpooler has quit IRC23:17
*** pradk has quit IRC23:22
*** pmannidi has joined #tripleo23:25
*** dmacpher has quit IRC23:31
*** moshele has joined #tripleo23:34
*** tosky has quit IRC23:34
*** moshele has quit IRC23:47
*** mjturek has joined #tripleo23:48
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: standalone run a post config to generate a clouds.yaml  https://review.openstack.org/57559723:49
*** owalsh_ has joined #tripleo23:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!