Monday, 2017-04-03

*** yamahata has joined #tripleo00:02
*** jkilpatr has quit IRC00:04
*** gbarros has joined #tripleo00:18
*** artom has quit IRC00:42
*** artom has joined #tripleo00:42
*** dougbtv|laptop has joined #tripleo00:54
*** dougbtv|laptop has quit IRC01:00
*** yamahata has quit IRC01:03
*** gbarros has quit IRC01:09
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo master: Move horizon to step 3  https://review.openstack.org/45234901:16
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo master: Deploy WSGI apps at the same step (3)  https://review.openstack.org/43401601:21
openstackgerritEmilien Macchi proposed openstack/puppet-tripleo master: Deploy WSGI apps at the same step (3)  https://review.openstack.org/43401601:22
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: Include panko in the default dispatcher  https://review.openstack.org/45177501:26
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Fix updating images with repos  https://review.openstack.org/45254301:29
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Fix updating images with repos  https://review.openstack.org/45254301:30
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Use AFS mirrors for optstool, delorean and qemu  https://review.openstack.org/45193801:32
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs  https://review.openstack.org/35921501:33
*** dmacpher has joined #tripleo01:39
*** rwsu has quit IRC01:54
*** rcernin has joined #tripleo01:58
*** rwsu has joined #tripleo02:00
*** noslzzp has quit IRC02:08
*** lucas-afk has quit IRC02:23
*** lucasagomes has joined #tripleo02:25
*** rcernin has quit IRC02:45
*** brault|away has joined #tripleo02:46
*** brault|away has quit IRC02:51
*** gbarros has joined #tripleo02:57
openstackgerritMerged openstack/os-collect-config master: Add splay option to offset polling intervals  https://review.openstack.org/45147802:58
*** ramishra has joined #tripleo03:00
openstackgerritMerged openstack/os-refresh-config master: Fix coverage job  https://review.openstack.org/45227703:01
*** artom has quit IRC03:09
*** artom has joined #tripleo03:09
*** fragatina has quit IRC03:10
*** artom has quit IRC03:10
*** artom has joined #tripleo03:10
openstackgerritSimon Dodsley proposed openstack/puppet-tripleo master: Composable services support for Cinder Pure Storage FlashArray  https://review.openstack.org/45220503:31
*** psahoo has joined #tripleo03:34
openstackgerritSimon Dodsley proposed openstack/puppet-tripleo master: Composable services support for Cinder Pure Storage FlashArray  https://review.openstack.org/45220503:36
*** gbarros has quit IRC03:48
openstackgerritHironori Shiina proposed openstack/diskimage-builder master: Fix code-block in README of rhel7 element  https://review.openstack.org/45259103:57
*** fragatina has joined #tripleo04:01
*** gkadam has joined #tripleo04:03
*** fragatina has quit IRC04:04
*** fragatina has joined #tripleo04:04
*** garyk1 has joined #tripleo04:24
*** gbarros has joined #tripleo04:33
*** dparkes has quit IRC04:46
*** rcernin has joined #tripleo04:57
*** gbarros has quit IRC05:04
*** flepied has quit IRC05:06
*** pgadiya has joined #tripleo05:11
*** udesale has joined #tripleo05:11
*** ratailor has joined #tripleo05:12
*** ratailor has quit IRC05:13
*** udesale has quit IRC05:14
*** janki has joined #tripleo05:14
*** udesale has joined #tripleo05:15
*** rcernin has quit IRC05:18
*** ratailor has joined #tripleo05:18
*** jaganathan has joined #tripleo05:18
*** gkadam has quit IRC05:20
*** gkadam has joined #tripleo05:21
*** percevalbot has quit IRC05:23
*** jaganathan has quit IRC05:23
openstackgerritSimon Dodsley proposed openstack/puppet-tripleo master: Composable services support for Cinder Pure Storage FlashArray  https://review.openstack.org/45220505:25
*** udesale has quit IRC05:27
*** percevalbot has joined #tripleo05:27
*** udesale has joined #tripleo05:27
*** prateek has joined #tripleo05:29
*** rcernin has joined #tripleo05:31
*** ealcaniz has joined #tripleo05:44
*** dparkes has joined #tripleo05:48
*** IRCFrEAK has joined #tripleo05:50
*** IRCFrEAK has left #tripleo05:50
*** mdnadeem has joined #tripleo05:51
*** yamahata has joined #tripleo05:55
*** florianf has joined #tripleo05:57
*** leanderthal|afk is now known as leanderthal06:00
*** ratailor has quit IRC06:04
*** pgadiya has quit IRC06:04
*** skramaja has joined #tripleo06:05
*** dsariel has joined #tripleo06:07
*** ratailor has joined #tripleo06:07
openstackgerritSaravanan KR proposed openstack/puppet-tripleo stable/newton: Check rabbitmq user at step >= 2  https://review.openstack.org/45260906:22
*** jaganathan has joined #tripleo06:22
*** michapma_dsk has joined #tripleo06:22
*** jprovazn has joined #tripleo06:24
*** lmiccini has joined #tripleo06:29
*** lmiccini has quit IRC06:30
*** lmiccini has joined #tripleo06:31
*** tesseract has joined #tripleo06:34
*** itzikb has joined #tripleo06:35
sshnaidmcores, please review/merge this: https://review.openstack.org/#/c/452543/06:41
*** brault|away has joined #tripleo06:42
*** anshul has joined #tripleo06:42
*** chem_gon` is now known as chem06:45
*** chem has quit IRC06:45
*** chem has joined #tripleo06:45
*** jcoufal has joined #tripleo06:46
*** nyechiel has joined #tripleo06:48
*** flepied has joined #tripleo06:49
*** aufi has joined #tripleo06:55
*** pcaruana has joined #tripleo06:57
*** udesale has quit IRC06:57
*** jaosorior has joined #tripleo06:57
*** udesale has joined #tripleo06:58
*** yatin_away is now known as yatinkarel06:58
*** udesale has quit IRC06:58
*** udesale has joined #tripleo06:59
*** yprokule has joined #tripleo07:00
*** ccamacho has joined #tripleo07:00
*** jcoufal has quit IRC07:04
*** lmiccini has quit IRC07:06
*** shardy has joined #tripleo07:07
*** bogdando has joined #tripleo07:11
*** jlinkes has joined #tripleo07:12
*** mcornea has joined #tripleo07:12
bandinimorning07:14
itzikbmarios: hi,can you please review https://review.openstack.org/#/c/448831/?07:15
*** amoralej|off is now known as amoralej07:20
*** cylopez has joined #tripleo07:23
openstackgerritDougal Matthews proposed openstack/python-tripleoclient master: Support multiple messages from deploy_plan  https://review.openstack.org/44643707:28
openstackgerritDougal Matthews proposed openstack/python-tripleoclient master: Support multiple messages from package_update  https://review.openstack.org/44643507:29
sshnaidmpradk, ping07:30
*** lmiccini has joined #tripleo07:30
*** ffiore has joined #tripleo07:31
*** dsariel has quit IRC07:40
*** nyechiel has quit IRC07:50
*** fzdarsky has joined #tripleo07:50
*** sshnaidm is now known as sshnaidm|afk07:50
*** suuuper has joined #tripleo07:56
*** zzzeek has quit IRC08:00
*** zzzeek has joined #tripleo08:00
*** jpich has joined #tripleo08:00
*** flepied has quit IRC08:01
openstackgerritMerged openstack/diskimage-builder master: Fix code-block in README of rhel7 element  https://review.openstack.org/45259108:01
*** openstackgerrit has quit IRC08:03
*** nyechiel has joined #tripleo08:04
chemFYI puppet CI is broken, I'm currently fixing it.08:04
*** openstackgerrit has joined #tripleo08:06
openstackgerritJulie Pichon proposed openstack/tripleo-common stable/ocata: Handle re-created resources in StackUpdateManager  https://review.openstack.org/45262608:06
*** shardy has quit IRC08:09
*** dparkes has quit IRC08:11
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Allow installing proliant tools from an OS package  https://review.openstack.org/34421408:12
*** dparkes has joined #tripleo08:12
*** nyechiel has quit IRC08:13
openstackgerritIan Wienand proposed openstack/diskimage-builder master: Allow installing proliant tools from an OS package  https://review.openstack.org/34421408:14
openstackgerritOliver Walsh proposed openstack/puppet-tripleo master: Configure migration SSH tunnel  https://review.openstack.org/45082608:14
*** dsariel has joined #tripleo08:15
*** gchamoul has joined #tripleo08:16
jaosoriorowalsh: nice! Just checked out the commits. commented on the puppet-tripleo on08:18
jaosorior*one08:18
owalshjaosorior: thanks08:19
*** agurenko has joined #tripleo08:21
*** yprokule_ has joined #tripleo08:22
*** jtomasek has joined #tripleo08:22
jaosoriorowalsh: hey, sorry to poke you about this, but where is the cold-migration libvirt uri set up in the nova code-base?08:22
openstackgerritOliver Walsh proposed openstack/tripleo-common master: Add MigrationSshKey to generated passwords  https://review.openstack.org/44923908:22
owalshjaosorior: it's not libvirt, it's nova08:22
*** gfidente has joined #tripleo08:22
owalshjaosorior: so nothing to config in nova.conf08:23
jaosoriorI see08:24
jaosorioralright08:24
jaosoriorthat works for me08:24
owalshjaosorior: https://github.com/openstack/nova/blob/4f91ed3a547965ed96a22520edcfb783e7936e95/nova/virt/libvirt/volume/remotefs.py#L18008:24
*** nmathew has joined #tripleo08:24
*** yprokule has quit IRC08:25
*** yprokule_ is now known as yprokule08:25
leanderthalo/08:25
*** zoli|gone is now known as zoli08:26
jaosoriorowalsh: thanks; that's what I needed08:26
owalshjaosorior: there is also an rsync driver, but that's rsync over ssh08:27
jaosoriorowalsh: one more comment in the patch08:27
*** fragatina has quit IRC08:28
openstackgerritJulie Pichon proposed openstack/python-tripleoclient stable/ocata: Fix return code when failing before launching the stack  https://review.openstack.org/45263408:28
mariositzikb: ack08:30
*** radeks has joined #tripleo08:30
itzikbmarios: thanks!08:30
*** zoli is now known as zoli|brb08:30
openstackgerritMerged openstack/tripleo-quickstart master: Fix updating images with repos  https://review.openstack.org/45254308:30
*** arxcruz|pto is now known as arxcruz08:31
*** shardy has joined #tripleo08:31
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: SSH known_hosts config  https://review.openstack.org/44966008:32
gfidentemarios morning08:32
gfidentesounds like we need to finish up with the change ordering batched vs non-batched?08:33
*** openstackgerrit has quit IRC08:33
*** paramite has joined #tripleo08:33
*** mjblack has quit IRC08:33
mariosgfidente: o/ https://review.openstack.org/#/c/448602/ seems to be working OK locally, will test some more today08:36
gfidentemarios ok I am going to test it too08:38
*** mjblack has joined #tripleo08:38
*** openstackgerrit has joined #tripleo08:38
openstackgerritOliver Walsh proposed openstack/tripleo-heat-templates master: Add migration SSH tunneling support  https://review.openstack.org/45084208:38
shardymarios: hey, why did you have to remove the heat conditional?08:39
shardymarios: there was a heat fix associated with that, is it possible we're missing a backport there?08:40
yolandahi, good morning. I'm having troubles passing CI on stable/newton, for tripleoclient: http://logs.openstack.org/51/450651/4/check/gate-tripleo-buildimage-overcloud-full-centos-7/c0de9a8/console.html08:40
yolandalooks as some dependency problem with ceph, not related to my patch08:41
yolandahttp://logs.openstack.org/51/450651/4/check/gate-tripleo-buildimage-overcloud-full-centos-7/c0de9a8/console.html#_2017-04-03_08_38_25_62874508:41
yolandais that a known issue, shall i report a bug?08:41
*** derekh has joined #tripleo08:42
yolandalooks as a python-rados package is needed, older than the one provided08:42
*** jpena|off is now known as jpena08:42
mariosshardy: hey, good morning. so the problem is we aren't always defining the batch tasks, and i was getting a specified reference "CephStorageUpgradeBatch_Step2" (in CephStorageUpgrade_Step0) is incorrect.08:42
mariosshardy: there is some discussino at https://review.openstack.org/#/c/448602/11/puppet/major_upgrade_steps.j2.yaml08:43
*** shardy has quit IRC08:46
chembandini: hi, and good morning, FYI I -1 https://review.openstack.org/#/c/451828/2 because it's monday08:46
*** yamahata has quit IRC08:48
chandankumarsshnaidm|afk: hello08:48
*** shardy has joined #tripleo08:48
chandankumarsshnaidm|afk: the jenkins jobs related to this review https://review.openstack.org/#/c/451747/ is failing, please have a look, Thanks :-)08:49
openstackgerritJulie Pichon proposed openstack/tripleo-common stable/newton: Handle re-created resources in StackUpdateManager  https://review.openstack.org/45263908:49
*** owalsh is now known as owalsh_afk08:49
bandinichem: ahaha thanks for the review ;)08:50
openstackgerritMichele Baldessari proposed openstack/puppet-pacemaker master: Fix potential cluster setup race  https://review.openstack.org/45182808:50
*** b00tcat has quit IRC08:51
*** gkadam_ has joined #tripleo08:52
*** gkadam has quit IRC08:53
shardymarios: Hey, is it possible you were missing https://review.openstack.org/#/c/419588/ ?08:53
shardythat was the heat fix which AFAIK fixed the error you mention, it was added in response to a bug I raised when testing those conditions08:53
mariosshardy: thanks , i can check it (though its pretty old, i mean i was using latest OSP11 stuff)08:54
shardymarios: Ok, would be good to confirm as perhaps there's some other issue08:54
shardyalso perhaps we should backport that to newton08:54
mariosshardy: ack yes will do, i am going to have a call with ajo about https://review.openstack.org/#/c/445494/ in a bit and then will check that (i.e. i'll be running the upgrade locally again)08:56
shardymarios: ack sounds good, FYI I just cherry-picked the fix to newton08:56
shardymarios: FWIW the reason I wanted those conditions was to avoid the round-trip of an empty deployment in the case where there's no batch tasks08:57
gfidenteshardy I was going to test marios' change as well, the idea is that on a patched heat we should be able to complete the deployment without removing the conditionals08:57
gfidente?08:57
shardygfidente: yes, the conditionals should work IMO08:57
gfidenteack08:57
mariosshardy: weird thing is, it isn't clear we even need to keep the l3 agents up (just had a chat with him and we will talk some more... it seems like the ping remains if you kill all your agents... seems openvswitch is the significant service to keep up08:57
shardymarios: ack, that's interesting08:58
shardymarios: I guess we want to fix the ordering problem here regardless tho, let me know if I can help with the review at all08:58
mariosshardy: sure, i mean yes i understand why it is better to conditionally create the resources but that resource error was really expensive (had to reset/deploy - someone said something about messing with heat db but already had too much goingon) to recover from so on the next go as there were no other/better suggestions to fix that i just removed the conditional08:59
mariosshardy: (I also rememberred to mentioning that perhaps we should remove the conditions, and wasn't sure if it was because of that invalid reference issue)09:00
mariosshardy: and yeah we really should fix the ordering09:00
openstackgerritJuan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE: Testing TLS everywhere  https://review.openstack.org/45264509:00
mariosshardy: i mean gfidente has already confirmed that for ceph we really do want it that way09:00
mariosshardy: and for the l3 agents thing it  really made testing  https://review.openstack.org/#/c/445494/  just confusing (they were sometimes being executed before sometimes after and i didn't always notice, not until i realised there was a problem with the order)09:01
shardymarios: yeah, agree completely we should fix this, I'd just like to ensure we're not removing the conditionals for no reason09:02
*** zoli|brb is now known as zoli09:02
shardyI mean, if it's just clearer to always run the deployments then fair enough, but there is a performance hit associated with that09:02
shardyparticularly because the round-trip to do nothing is serialized over all nodes09:02
mariosshardy: ack09:03
*** itzikb has quit IRC09:03
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: Containerize etcd service  https://review.openstack.org/44762709:03
jistrramishra thanks for the heat-templates review09:04
jistrerr heat-agents09:04
ramishrajistr: np:)09:04
*** jprovazn has quit IRC09:05
*** imran_work has joined #tripleo09:05
bogdandooooq folks, a set of patches for your attention please https://review.openstack.org/#/c/447409/ https://review.openstack.org/#/c/448294/ (ready for merge, PTAL adarazs ^ ^ ) https://review.openstack.org/#/c/447000/ https://review.openstack.org/#/c/448543/09:06
bogdandoit's all we need to implement https://bugs.launchpad.net/tripleo/+bug/167637309:06
openstackLaunchpad bug 1676373 in tripleo "Allow users to run oooq from any host OS, from a container" [Medium,In progress] - Assigned to Bogdan Dobrelya (bogdando)09:06
*** yprokule_ has joined #tripleo09:08
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Ensure working dirs to exist  https://review.openstack.org/44899109:11
*** yprokule has quit IRC09:11
*** yprokule_ is now known as yprokule09:11
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Write openstack bash completion for undercloud  https://review.openstack.org/44860609:11
yolandashardy, around? i'm having problems with image building in newton. looks as epel repo is installed, although jpena and amoralej told me it shall be removed. Do you know why epel is installed on the image builds?09:12
openstackgerritNoam Angel proposed openstack/diskimage-builder master: Add option to update all packages early on  https://review.openstack.org/45265309:13
*** sshnaidm|afk is now known as sshnaidm09:14
sshnaidmchandankumar, hi09:14
sshnaidmchandankumar, we had a problem with ovb oooq jobs, now it's ok, I ran recheck there09:14
sshnaidmchandankumar, but be aware that now we don't run tempest on ovb oooq, you need to do a different patch above this to test tempest09:15
shardyyolanda: Hi, not sure tbh - there have been patches disabling epel to tripleoclient and tripleo-common, but AFAICS they did get backported to newton09:17
shardyyolanda: I guess it's a question of looking over the logs and hopefully seeing where it's enabled?09:18
yolandashardy, the problem is that ceph is installed from epel, and then it causes errors , ci is blocked09:18
shardyyolanda: ack, OK we should not be using epel so we need to figure out why it's getting enabled09:18
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: testing tempest for oooq  https://review.openstack.org/45265509:19
sshnaidmchandankumar, ^^09:19
chandankumarsshnaidm: thanks !09:19
chandankumarsshnaidm: do we have any job currently which runs tempest on each review?09:20
sshnaidmchandankumar, no, I don't think so09:21
sshnaidmchandankumar, I believe we'll create experimental one09:22
chandankumarsshnaidm: i want to create one, how can i create that?09:22
chandankumarsshnaidm: any doc link?09:24
*** akrivoka has joined #tripleo09:25
sshnaidmchandankumar, actually we have one that should be replaced by oooq: gate-tripleo-ci-centos-7-ovb-ha-tempest09:25
sshnaidmchandankumar, right now it runs with old code and doesn't have any value09:25
sshnaidmchandankumar, but I think it's better to do it after transition of OVB jobs to quickstart09:26
sshnaidmchandankumar, because it will change the code09:26
yolandashardy, ok, looks as higher versions of dib don't hardcode the epel dependency, so i'll try updating it09:26
*** tosky has joined #tripleo09:27
*** dsariel has quit IRC09:27
*** karimb has joined #tripleo09:27
*** b00tcat has joined #tripleo09:28
openstackgerritAdriano Petrich proposed openstack/tripleo-quickstart master: Adding the mistral packages to tripleo-ci-master  https://review.openstack.org/45146809:29
openstackgerritNoam Angel proposed openstack/diskimage-builder master: Add option to update all packages early on  https://review.openstack.org/45265309:30
*** jprovazn has joined #tripleo09:30
*** lmiccini has quit IRC09:30
openstackgerrityolanda.robla proposed openstack/python-tripleoclient stable/newton: Support whole disk images in TripleO  https://review.openstack.org/45065109:30
*** lmiccini has joined #tripleo09:31
chandankumarsshnaidm: what is the due date for transition?09:32
sshnaidmchandankumar, good question.. I don't think it will happen until 20 Apr09:33
*** prateek has quit IRC09:33
apetrichadarazs, morning. Sorry I thought I had fixed this https://review.openstack.org/#/c/451468/ last week but it was hanging waiting for publish. Do you think I need more info?09:36
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates stable/newton: Increase libvirt/qemu.conf max_files and max_processes  https://review.openstack.org/44812209:36
adarazsapetrich: hi! I'll check it out, thanks for updating it.09:37
openstackgerritNoam Angel proposed openstack/diskimage-builder master: Add option to update all packages early on  https://review.openstack.org/45265309:37
openstackgerritJuan Antonio Osorio Robles proposed openstack/tripleo-quickstart-extras master: Add option to enable/disable TripleO UI in the undercloud  https://review.openstack.org/45021409:40
adarazsapetrich: I'm not sure if this is a good idea to use the "current" version of mistral from DLRN. That means that it's not going through the promotion process that results in "current-tripleo" and a mistral change can break all our jobs. AFAIK we're not gating mistral directly. <- sshnaidm, shardy, what do you think? (re https://review.openstack.org/451468 )09:42
sshnaidmadarazs, yes, we don't gate it09:43
*** karimb has quit IRC09:43
adarazsapetrich: I think all the packages there are actually directly gated by a bunch of jobs that sort of guarantees that "current" is not going to get broken.09:43
apetrichadarazs, I see.09:44
sshnaidmapetrich, why do you want it from current-tripleo?09:44
apetrichsshnaidm, I just want as close to git master as possible. all others rpms that I found were slight older09:45
adarazsapetrich: yeah, there's a lag because of the promotion process. if we can get a full tripleo gating going on mistral then we can do this, but without it it's not really a good idea.09:45
sshnaidmapetrich, it depends on promotion I think, we have it now every day, so you can check again09:45
apetrichadarazs, sshnaidm ok. I will abandon that then.09:46
*** prateek has joined #tripleo09:47
*** prateek has quit IRC09:47
jaosoriormandre: hey dude, have you had a chance to work on getting the other httpd-based services to mount the /etc/httpd directory like in keystone?09:47
sshnaidmapetrich, actually we can start from adding tripleo jobs to mistral gates, if no objection from mistral project..09:47
jaosoriormandre: I can help out if needed.09:47
*** prateek has joined #tripleo09:47
apetrichsshnaidm, Cheers! We have a mistral meeting this afternoon. I will bring it up.09:49
*** karimb has joined #tripleo09:51
*** mrunge_ is now known as mrunge09:52
yolandashardy, jpena , can you review https://review.openstack.org/452659 ?09:55
*** prateek has quit IRC09:56
*** prateek_ has joined #tripleo09:56
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: Add special case upgrade from openvswitch 2.5.0-14  https://review.openstack.org/45252409:56
*** garyk2 has joined #tripleo09:59
*** prateek has joined #tripleo10:00
*** garyk1 has quit IRC10:01
*** owalsh_afk is now known as owalsh10:02
sshnaidmadarazs, jaosorior EmilienM can you take a look please in your time? https://review.openstack.org/#/c/452542/10:02
*** prateek_ has quit IRC10:03
adarazssshnaidm: you're running CI scripts locally? :)10:03
*** zoli is now known as zoli|lunch10:03
sshnaidmadarazs, yeah, using dev env for example: https://etherpad.openstack.org/p/tripleo-ci-devenvs10:04
adarazssshnaidm: cool, I'm going to have to take a look at that.10:05
adarazsthanks10:05
*** rcernin has quit IRC10:05
d0ugaladarazs: what does gating mistral directly mean? it does run scenario 310:06
d0ugalsshnaidm: what tripleo jobs would you want to add to Mistral?10:07
d0ugalapetrich: maybe you know?10:07
sshnaidmd0ugal, multinode-nonha-oooq10:07
adarazsd0ugal: and is it voting on it too? (I really don't know, I was partly asking what's going on around there)10:08
sshnaidmd0ugal, it's more or less main job in our gates, and it could start from non-voting for example10:09
sshnaidmd0ugal, wrt ovb jobs - we don't have capacity right now for running them more10:09
*** prateek has quit IRC10:09
*** michapma_dsk has quit IRC10:10
sshnaidmd0ugal, sorry, full name is gate-tripleo-ci-centos-7-nonha-multinode-ooo10:10
openstackgerritMerged openstack/tripleo-heat-templates master: Fixes port binding controller for OpenDaylight  https://review.openstack.org/44883110:11
adarazsd0ugal: in general I'd extend testing a bit before using it from master. I mean we run 5+ gating jobs on other repos that we do allow to be used directly from current, so running 1 is probably not covering everything. /o\10:11
*** garyk2 has quit IRC10:11
d0ugalsshnaidm, adarazs: right, we have gate-tripleo-ci-centos-7-scenario003-multinode-puppet voting now. Would we need both?10:11
*** garyk1 has joined #tripleo10:12
d0ugalThe only Mistral objection is how much we slow things down, but otherwise I think they are happy for TripleO to test Mistral.10:12
adarazsd0ugal: I'd bring this up on the TripleO meeting tomorrow, I can't really tell what would be deemed enough. EmilienM probably has some better idea.10:12
d0ugalokay, thanks.10:12
shardyd0ugal: What's the motivation behind running mistral from -current, vs just waiting for promotion?10:13
d0ugalshardy: I'm not sure, that was apetrich I think :)10:13
shardyd0ugal: Ok, sorry I've not fully read all the scrollback10:14
*** prateek has joined #tripleo10:15
*** mburned is now known as mburned_out10:18
gfidentemarios so my stack is in CREATE_COMPLETE, going to try the upgrade10:19
*** garyk1 has quit IRC10:20
*** rcernin has joined #tripleo10:24
*** salmankhan has joined #tripleo10:25
*** abishop has quit IRC10:25
openstackgerritMerged openstack/os-net-config master: Updated from global requirements  https://review.openstack.org/45227310:25
openstackgerritMerged openstack/os-net-config master: Fix coverage tox config  https://review.openstack.org/45227410:28
*** stendulker has joined #tripleo10:31
*** garyk1 has joined #tripleo10:32
openstackgerritMerged openstack/tripleo-heat-templates master: Change heat and mistral to use v3/ec2tokens url  https://review.openstack.org/45214910:34
openstackgerritMerged openstack/tripleo-quickstart master: Generate/inject instack.json if overcloud nodes  https://review.openstack.org/44829410:34
*** garyk1 has quit IRC10:34
*** zoli|lunch is now known as zoli10:38
openstackgerritDougal Matthews proposed openstack/python-tripleoclient master: Support multiple messages from deploy_plan  https://review.openstack.org/44643710:39
bogdandoshardy: oh gush, I wish I could understand changes like https://review.openstack.org/#/c/448602 soon10:40
bogdandocould you recommend anything to read on that topic to move me to that goal?..10:40
imran_workworked out why dhcp-all-interfaces.sh is giving me a ballache10:42
imran_workit seems to be timing out doing dhcp10:42
bogdandomarios: and could you please as well? ^ ^10:43
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates master: WIP: Add role specific information to the service template  https://review.openstack.org/43795610:43
openstackgerritDougal Matthews proposed openstack/tripleo-common master: Run introspection 20 nodes at a time  https://review.openstack.org/43956810:44
openstackgerritDougal Matthews proposed openstack/tripleo-common master: TESTING small batch  https://review.openstack.org/44921510:44
*** pkovar has joined #tripleo10:51
mariosbogdando: we don't really have any developer docs around the workflow am afraid... we did land some operator docs about running the upgrade. i guess tripleo-heat-templates would be a good start, for example, each service has/defines some ansible  upgrade_tasks, e.g. https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/neutron-l3.yaml#L87 or10:52
marioshttps://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/nova-api.yaml#L194 is more complex because workarounds10:52
*** ansmith has joined #tripleo10:52
mariosbogdando: those are collected at https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/services.yaml#L11210:53
bogdandomarios: thank you, I'd like to know more about the "depends-on" magic and those jinja templates' darkest magic as well10:53
*** mburned_out is now known as mburned10:54
mariosbogdando: we step through and apply that with the step number as tags with https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/major_upgrade_steps.j2.yaml10:54
bogdandomarios: I wonder if there is a heat tool to display the resulting graph?10:55
mariosbogdando: (and this is the actual ansible config definition from those https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/upgrade_config.yaml )10:55
bogdandolike heat stack-preview or so10:55
mariosbogdando: right, so shardy is the one to ask about stack-review (he has spoken about it before) but you *can* get the 'rendered' templates from swift10:56
mariosbogdando:or10:56
mariosbogdando: tox -e templates10:56
bogdandomarios: oh, that's nice to know, thank you10:56
mariosbogdando: also does it (ie. it gives you a major_upgrade_steps.yaml from the j2.yaml10:56
*** jkilpatr has joined #tripleo11:01
*** thrash|g0ne is now known as thrash11:02
*** arxcruz has quit IRC11:07
*** athomas has joined #tripleo11:08
*** adarazs is now known as adarazs_lunch11:10
shardybogdando: Heat does expose the dependency information, and several folks have written tools to interpret and visualize it, e.g dotstack from larsks11:11
shardyhttps://libraries.io/github/larsks/dotstack11:11
shardybogdando: there was discussion about adding something like that directly into python-heatclient, but I don't think it's actually happened yet11:11
bogdandoshardy: thank you!11:11
shardysimilarly AFAIK nobody has done the work to make that run cleanly via stack-preview11:12
shardybut that would be a nice idea if enough information is exposed, honestly I've not yet checked11:12
*** arxcruz has joined #tripleo11:12
*** dsariel has joined #tripleo11:13
shardymarios, bogando: you can also get the rendered templates by running tox -e templates in t-h-t11:13
bogdandoshardy: that would make deployment steps auto decomented11:17
bogdandodocumented*11:17
gfidenteshadower marios I actually get things rolling with the conditionals and the heat patch11:17
gfidenteshardy ^^ sorry shadower  :)11:17
shardybogdando: well, partly - we can inspect what happens for the upgrade steps, but for the puppet steps it's harder because what's run at each step is controlled by conditionals in the puppet-tripleo profiles11:18
gfidentemarios don't want to 'overwrite' your submission but would like to share the 'working version' that I have11:18
shardyhaving a tool which plots exactly what happens at each step would be useful, and I think it'll be easier with the container driven model11:18
gfidenteshall I make a different submission or pastebin the diff?11:18
shardyas the steps aren't hidden in puppet-tripleo anymore11:19
bogdandoshardy: right, although even having just a step name following another one , no details wrt specific tasks vs cases, would be awesome11:19
bogdandowhich is UML :)11:19
shardybogdando: there are some docs here:11:19
mariosshardy: ack thanks i'd mentioned the tox above dotstack looks really cool11:19
shardyhttps://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/README.rst11:19
mariosgfidente: hey , do you mean the upgrade batch tasks one?11:20
shardybogdando: that's pretty high level, but it does explain what sort of things we expect to happen at each step11:20
gfidentemarios yeah11:20
gfidentemaybe a pastebin works better11:20
*** lucasagomes is now known as lucas-hungry11:20
bogdandoshardy: thank you. Sadly, it is manually generated and became obsolete by the many recent changes, AFAICT11:21
mariosgfidente: sure, so does your update also add back the conditional? sorry i got distracted by calls/other things, i was going to update that11:21
mariosgfidente: and also check for the dependency shardy mentioned11:21
gfidentemarios yeah I am using the conditionals11:21
bogdandoand that's why I created https://bugs.launchpad.net/bugs/167175711:21
openstackLaunchpad bug 1671757 in tripleo "[doc] Document pre/post/upgrade/batch ordered and concurrent steps in the tht deployment steps guide" [High,Triaged]11:21
mariosgfidente: ok then go ahead and update it and we discuss there, if you think it looks good i can also test it in my run momentarily11:21
shardybogdando: yeah we have to try to keep it updated, but we don't have a way to fully automate it yet11:21
gfidentemarios yeah but I don't want to update the working version11:21
shardylike, we could extract a bunch of descriptions for the upgrade steps but currently we don't11:22
gfidentemarios sec11:22
shardybogdando: also, it's dependent on what services are enabled for a particular deployment11:22
shardybut perhaps we could generate some kind of per-step matrix in the docs11:22
shardythat would be useful11:22
*** stendulker has quit IRC11:23
bogdandoshardy: well, what I mean is to have something like this: "[node-by-node upgrade_batch_tasks{0..3} -> concurrent upgrade_tasks{0..6}] -> host_prep_tasks (docker only, https://review.openstack.org/#/c/443751/) -> puppet's PostUpgrade{1..6}-> puppet's PostDeploy{1..6}" auto generated from tht11:23
bogdandoand those prepended by [docker_puppet_tasks{1..6}]11:23
shardybogdando: ack, yeah that would be nice if we can automate it11:23
*** ccamacho is now known as ccamacho|lunch11:24
bogdandoso newcomers to not ask you folks same things again and again )11:24
bogdandoespecially having them change *fast* these days11:24
bogdandochanging*11:25
openstackgerritItzik Brown proposed openstack/tripleo-heat-templates stable/ocata: Fixes port binding controller for OpenDaylight  https://review.openstack.org/45271511:25
shardybogdando: yeah, I guess, I mean the puppet deploy sequence has been pretty stable for a number of releases, but composable upgrades and now containers are introducing some changes11:25
gfidentemarios wait it failed on the second upgrade run, as you mentioned11:25
bogdandoshardy: yeah, and it's really nice to auto follow them in dev docs...11:26
jaosoriorgfidente: could you check this out https://review.openstack.org/#/c/452150/11:26
bogdandoI'll take a look onto heat preview and dotstack, thank you11:26
*** Guest20391 has quit IRC11:27
*** Guest20391 has joined #tripleo11:27
gfidentejaosorior ack, thanks :)11:27
jaosoriorthanks11:28
bogdandolarsks, shardy: marios the dotstack looks great indeed11:28
bogdandoso a little change to build graphs not from tasks but from common steps and woila, a perfection :)11:29
apetrichshardy, sorry I missed the ping early. We needed an up to date mistral for testing something or other, but last week every other rpm that I found was old i.e.  openstack-mistral-api-4.0.0-0.2017031015565911:31
shardyapetrich: the best way to do that before a promotion is via a Depends-On to a patch for mistral I think11:31
shardye.g you can push a dummy noop change to mistral then we'll build effectively trunk mistral and test it11:32
shardyIMO there is value in the full promotion testing for all projects, even if there is partial coverage in the CI for the projects themselves (which is great! but it doesn't test as much as the promtion jobs)11:33
apetrichshardy, no. I want to use tripleo-quickstart to build me an env with an uptodate mistral so I can poke. the way that I'm able to do that is doing this patch https://review.openstack.org/#/c/451468/ locally and building an env11:33
openstackgerritDougal Matthews proposed openstack/tripleo-common master: Run introspection 20 nodes at a time  https://review.openstack.org/43956811:34
openstackgerritDougal Matthews proposed openstack/tripleo-common master: TESTING small batch  https://review.openstack.org/44921511:34
apetrichshardy, but to add those lines to that tripleo-ci-master means we have to be gating mistral and that was where all this talk started11:34
shardyapetrich: IMHO we should just add a variable to quickstart like "current_repo_whitelist"11:34
shardythen you can set whatever list of current things you want for local testing11:35
shardyor even append a current_repo_extras or something11:35
shardyIMO it's not something we'd want to do in CI by default11:35
shardyapetrich: also, FWIW, I often just update the whitelist manually in the local delorean-current repo file on the undercloud11:36
shardythat makes it very easy to update mistral, heat or whatever to the latest for local testing11:37
d0ugalThat sounds useful :)11:37
apetrichshardy, true. that's why I abandoned the change11:37
shardyapetrich: sure I was just sharing some ideas around alternative solutions11:38
*** shardy is now known as shardy_lunch11:38
openstackgerritDougal Matthews proposed openstack/python-tripleoclient master: Don't create tempest-deployer-input.conf in tests  https://review.openstack.org/44766511:41
*** dsariel has quit IRC11:46
*** dsariel has joined #tripleo11:47
pradkcan someone merge this backport please https://review.openstack.org/#/c/452541/11:47
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates master: Disable ceilometer API  https://review.openstack.org/45090311:47
EmilienMhappy Monday11:48
openstackgerritDougal Matthews proposed openstack/python-tripleoclient master: Allow users to specify the Introspection batch size  https://review.openstack.org/45122811:49
*** morazi has joined #tripleo11:54
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates master: Disable ceilometer API  https://review.openstack.org/45090311:54
*** abishop has joined #tripleo11:54
*** bfournie has quit IRC11:55
*** karimb has quit IRC11:55
ccamacho|lunchhey shardy, slagle, Im finishing to clean up the env files and testing the clean up for [Pre|Post]Puppet & [Pre|Post]Config Ill push the changes when finish the deployment11:56
*** ccamacho|lunch is now known as ccamacho11:56
*** jaganathan has quit IRC11:57
*** abishop has quit IRC12:00
*** dprince has joined #tripleo12:00
*** ratailor has quit IRC12:00
openstackgerritDan Radez proposed openstack/tripleo-heat-templates stable/ocata: Setting keystone region for tacker  https://review.openstack.org/45273112:01
imran_workwoop setting DIB_DHCP_TIMEOUT=60 fixed introspection for me12:02
*** abishop has joined #tripleo12:03
*** ansmith has quit IRC12:03
*** trown|outtypewww is now known as trown12:05
*** jayg|g0n3 is now known as jayg12:07
*** lucas-hungry is now known as lucasagomes12:07
*** adarazs_lunch is now known as adarazs12:08
*** dougbtv_ has quit IRC12:08
arxcruzsshnaidm: adarazs when you guys have time https://review.openstack.org/#/c/450177/12:09
*** prateek has quit IRC12:10
dtantsurfolks, could you please check https://review.openstack.org/#/c/408581/ ? it's part of "move everything to workflows" effort, and it's been without reviews for months already..12:11
*** jjung has joined #tripleo12:12
*** rhallisey has joined #tripleo12:13
*** lblanchard has joined #tripleo12:13
openstackgerritJames Slagle proposed openstack/tripleo-docs master: Link to tripleo-quickstart docs for virtual envs  https://review.openstack.org/45191212:17
*** deadnull has joined #tripleo12:17
*** lblanchard has quit IRC12:19
*** jcoufal has joined #tripleo12:19
*** jaganathan has joined #tripleo12:20
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: Merge pre|post puppet resources into pre|post config.  https://review.openstack.org/45273412:25
*** panda has joined #tripleo12:31
*** rlandy has joined #tripleo12:32
*** psahoo has quit IRC12:33
*** dougbtv_ has joined #tripleo12:33
jaosoriorEmilienM: the TLS everywhere gate broke; this puppet-nova commit will fix it up https://review.openstack.org/#/c/452651/ The problem is novajoin starts before the authtoken settings are set :/12:33
*** jaosorior is now known as jaosorior_brb12:36
*** dougbtv_ is now known as dougbtv12:36
*** limao has joined #tripleo12:36
EmilienMjaosorior_brb: ok, looking at it12:36
openstackgerritDmitry Tantsur proposed openstack/instack-undercloud master: Enable boot mode detection by ironic-inspector  https://review.openstack.org/44862412:37
weshayadarazs, fyi  arxcruz responded https://review.openstack.org/#/c/451335/12:38
sshnaidmadarazs, can we merge it? https://review.openstack.org/#/c/434226/12:38
* adarazs looks at links12:39
adarazsweshay, arxcruz: answered.12:41
arxcruzadarazs: thanks, you're right, updating :)12:43
*** jpena is now known as jpena|lunch12:43
adarazssshnaidm: submitted.12:44
*** limao has quit IRC12:44
*** shardy_lunch is now known as shardy12:45
*** limao has joined #tripleo12:46
openstackgerritArx Cruz proposed openstack/tripleo-quickstart-extras master: Fix stackviz data generation  https://review.openstack.org/45133512:46
florianfshardy: Hi! I have a question regarding the number of processes assigned to some services on the undercloud. If you have a minute...12:47
*** fultonj has joined #tripleo12:47
weshaysshnaidm, panda do you think you guys can find apetrich an example w/ logs w/ the deployment failing but returning 012:47
shardyflorianf: hi, sure12:48
florianfshardy: great. So there is a validation that checks the number of running processes for a number of services. It fails if that number is larger than 4: https://github.com/openstack/tripleo-validations/blob/master/validations/undercloud-process-count.yaml#L20-L3312:49
shardyflorianf: I don't really think we can have a sensible hard-coded limit like that12:49
florianfshardy: But I noticed that for instance the heat config has a comment that the number of processes is set to the number of cpus if that's larger than the number in the config12:50
shardyit depends a lot on the hardware and the number of nodes you want to deploy12:50
*** artom has quit IRC12:50
florianfshadower: exactly.12:50
*** artom has joined #tripleo12:50
shardyflorianf: Yeah, we hit heat particularly hard so we don't impose a low limit on the number of workers12:50
florianfshadower: sorry, wrong outocomplete there... ^12:50
shardyhttps://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/puppet-stack-config.yaml.template#L14312:50
shadowerflorianf: no worries, you're like the 100th person to do that :-)12:51
*** liverpooler has joined #tripleo12:51
chemmarios: he man, could you +1workflow this on https://review.openstack.org/#/c/452460/ ?12:51
openstackgerritSaravanan KR proposed openstack/tripleo-common master: Add an action to fetch and flatten the heat resource tree and parameters  https://review.openstack.org/45002112:51
florianfshadower: next time I better not add insult to injury and apologize? :-)12:51
shardyflorianf: I know that e.g rook|tower and others have done significant tuning to achieve large deployments, I guess we can/should consider the knowledge gained there as part of the validation12:51
shadowerflorianf: don't sweat it :-)12:51
*** limao has quit IRC12:52
florianfshardy: oh thanks, that's a good idea. I wonder if it might make sense to set the maximum number of processes equal to the number of CPUs in the meantime.12:52
skramajajtomasek: ptal - https://review.openstack.org/450021 - to confirm that it matches with UI flattening of parameters.12:52
*** mjblack has quit IRC12:52
shardyflorianf: yeah I guess any number larger than #cpus may be counter-productive12:53
*** limao has joined #tripleo12:53
jtomasekskramaja: yep, It is on my list, I need to finish something first, I'll try to get to it asap12:54
florianfshardy: OK, thanks a lot for the input!12:54
shardyskramaja: Hey, did you by any chance look at instead flattening it via yaql directly in the workflow?12:54
shardyI know there is a flatten() function and I wondered if we could use it here to avoid a custom action12:54
*** amoralej is now known as amoralej|lunch12:54
mandrejistr: hi!12:55
*** mjblack has joined #tripleo12:55
mandrejistr: it looks like we have different view on how we should set the permissions for the persistent data directories12:56
openstackgerritDougal Matthews proposed openstack/python-tripleoclient master: Add a new command to get the overcloudrc files  https://review.openstack.org/40435412:57
marioschem: no12:57
*** artom has quit IRC12:58
jistrmandre: hi12:58
jistrmandre: can you explain please :)12:58
*** artom has joined #tripleo12:58
jistri'm not sure what you're reffering to right now12:58
apetrichweshay, sshnaidm, panda we have the example in the bug, and some more from that day but we don't see them happening again after that. if there's a new one I would appreciate12:58
apetrichweshay, sshnaidm panda even more a live machine that keeps making those12:59
mandrejistr: I'm reading your comment in https://review.openstack.org/#/c/447627/ :)12:59
mandrejistr: I don't think for this particular image it's that bad to use kolla bootstrap13:00
mandreactually in most case it should be fine13:00
chemmarios: I take in as a affront and ask for revange :)13:00
jistrcc bogdando for this discussion ^ :)13:01
skramajashardy: yes, i did look at it. it doesn't work for our case, the way we need it. I will send the details in a mail with example.13:01
mandreIMO we may want to bypass the kolla boostrap if it tries to be fancy, but for this simple case I think we may be better with it13:01
skramajajtomasek: thanks.13:02
*** florianf has quit IRC13:02
*** tosky has quit IRC13:02
*** tosky has joined #tripleo13:02
openstackgerritFlorian Fuchs proposed openstack/tripleo-validations master: Change number of allowed undercloud processes  https://review.openstack.org/45274613:02
jistrmandre: KOLLA_BOOTSTRAP could be fine too i guess, but i think it's important to keep in mind though that we don't control what is added or removed there. So we may get broken if kolla bootstrap gets fancy at some point.13:03
*** karimb has joined #tripleo13:03
*** limao has quit IRC13:04
*** limao has joined #tripleo13:04
mandrejistr: alternatively, that's something we may want to control with config.json13:04
mandrecould be a better approach13:04
ansiwenmwhahaha, EmilienM: it looks like ec2api tempest tests are not running anymore  in puppet ci?13:05
ansiwenhttp://logs.openstack.org/58/450658/1/check/gate-puppet-openstack-integration-4-scenario002-tempest-centos-7/214a957/console.html13:05
*** vpickard_ is now known as vpickard_afk13:05
*** vpickard_afk is now known as vpickard13:05
jistrmandre: IMO the less we depend on Kolla potentially specific/opinionated parts, the better, because Kolla commits aren't gated against TripleO deployments, so they can break us without us knowing upfront or being able to prevent it.13:06
openstackgerritwes hayutin proposed openstack/tripleo-quickstart master: allow easy access to undercloud's horizon dashboard  https://review.openstack.org/44446413:06
*** ansmith has joined #tripleo13:06
EmilienMansiwen: probably because tempest plugin is not here: http://logs.openstack.org/58/450658/1/check/gate-puppet-openstack-integration-4-scenario002-tempest-centos-7/214a957/console.html#_2017-03-28_09_58_07_31862413:07
*** skramaja has quit IRC13:07
ansiwenEmilienM: yes, saw that as well... but I wonder why isn't it there anymore13:07
EmilienMthey are still there: https://github.com/openstack/ec2-api/tree/master/ec2api/tests/functional/api13:09
jistrmandre, bogdando: though i'm not strictly against kolla_bootstrap, especially in this case where the chance that Kolla will start doing something wild is quite low. So if we want to utilize kolla_bootstrap, i'm still fine with it and would +2 such commit.13:09
mandrejistr: that's a good point, indeed13:09
EmilienMansiwen: probably this line: https://github.com/openstack/puppet-openstack-integration/blob/master/run_tests.sh#L28413:09
ansiwenEmilienM: maybe someone removed them from the package?13:09
EmilienMansiwen: ah no13:09
EmilienMansiwen: python-ec2-api-tests is missing13:10
ansiwenEmilienM: it was never there, the tests were included in the normal package13:10
ansiwenEmilienM: but maybe that got changed13:10
EmilienMhttps://github.com/rdo-packages/ec2-api-distgit/blob/rpm-master/openstack-ec2-api.spec#L27113:10
EmilienMansiwen: https://github.com/rdo-packages/ec2-api-distgit/commit/37a4a61f4f4ec54d72888c16e4f2d6dd99a7996e13:11
*** atheurer has joined #tripleo13:11
EmilienMansiwen: let me fix it, in puppet-tempest13:12
ansiwenEmilienM: ok, I see... so how can we protect ourselves against this? suddenly the tests are not executed and I just find out by chance doesn't sound reliable... :-/13:12
ansiwenEmilienM: thanks for your help!13:12
*** florianf has joined #tripleo13:12
bogdando> 15:06:07 - jistr: mandre: IMO the less we depend on Kolla potentially specific/opinionated parts, the better, […]13:13
bogdando+113:13
karthiksEmilienM,   Saravanan has provided comments for https://review.openstack.org/#/c/449530/.  Can you please take a look13:14
*** rbrady has joined #tripleo13:14
*** rbrady has quit IRC13:14
*** rbrady has joined #tripleo13:14
*** pradk has quit IRC13:14
*** gkadam_ is now known as gkadam13:15
*** limao has quit IRC13:15
*** limao has joined #tripleo13:16
openstackgerritSagi Shnaidman proposed openstack/tripleo-common master: DONT REVIEW: used for testing only  https://review.openstack.org/44727613:17
*** gkadam has quit IRC13:17
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: Support built packages to be installed in image  https://review.openstack.org/45275513:18
*** eck`gone is now known as eck`13:18
*** rook|tower is now known as rook13:19
*** tzumainn has joined #tripleo13:21
*** limao has quit IRC13:25
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Use AFS mirrors for optstool, delorean and qemu  https://review.openstack.org/45193813:27
EmilienMsshnaidm: when you have time for review ^ - thanks13:27
*** limao has joined #tripleo13:29
*** ma9 has joined #tripleo13:30
*** beekneemech is now known as bnemec13:31
*** dsariel has quit IRC13:31
*** jaosorior_brb is now known as jaosorior13:33
openstackgerritBrad P. Crochet proposed openstack/python-tripleoclient master: Avoid race condition when setting up websocket  https://review.openstack.org/45162413:33
openstackgerritEmilien Macchi proposed openstack/tripleo-docs master: Remove os-cloud-config bits  https://review.openstack.org/45108813:35
*** limao has quit IRC13:37
jristmandre, shadower if you get a chance https://review.openstack.org/#/c/452147/113:37
*** ma91 has joined #tripleo13:37
gfidentemarios regarding cinder-api upgrade, I see it moved behind httpd but the upgrade validations are checking for the systemd service?13:37
*** ma9 has quit IRC13:39
*** lblanchard has joined #tripleo13:40
*** ipsecguy has quit IRC13:40
*** dtrainor has joined #tripleo13:41
*** janki has quit IRC13:42
EmilienMcschwede: have you already seen http://logs.openstack.org/16/434016/9/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq/21a5523/logs/subnode-2/var/log/messages.txt.gz#_Apr__3_02_36_02 ?13:42
*** dtantsur is now known as dtantsur|brb13:42
EmilienMcschwede: swift-recon: command not found13:42
EmilienMcschwede: and also "/etc/swift/*.ring.gz: Cannot stat: No such file or directory"13:43
d0ugalthrash: what does flatten do? https://review.openstack.org/#/c/429757/14..18/workbooks/validations.yaml,unified13:43
*** lblanchard has quit IRC13:43
*** rbowen has quit IRC13:43
*** fzdarsky has quit IRC13:43
thrashd0ugal: [['foo'], ['bar', 'baz']] -> ['foo', 'bar', 'baz']13:43
d0ugalaha, thanks13:44
cschwedeEmilienM: no, didn't saw it yet. this should be never missing - are the swift pkgs not installed?13:44
mariosgfidente: o/ sry the calls ate my monday. so i haven't hit any cinder/other service related validation failure (i mean i am not hitting such an issue)13:44
*** janki has joined #tripleo13:44
* d0ugal should probably read docs13:44
*** lblanchard has joined #tripleo13:44
thrashd0ugal: the yaql docs have improved tremendously as of late.13:44
EmilienMcschwede: probably not13:44
d0ugalthrash: so they have :) http://yaql.readthedocs.io/en/latest/standard_library.html?highlight=flatten#flatten13:44
cschwedeEmilienM: well, it's not nice, but the deploy_status_code is still 0, so it shouldn't fail because of this?13:45
gfidentemarios ack , will add you on review13:45
thrashd0ugal: I understand you asking... Those docs were rather sparse before.13:45
*** ipsecguy has joined #tripleo13:45
d0ugalthrash: I wonder when rbrady will notice you are getting close to 1k lines in validations.yaml ;)13:45
thrashhehe13:46
*** bfournie has joined #tripleo13:46
*** ffiore has quit IRC13:46
*** limao has joined #tripleo13:46
*** bfournie has quit IRC13:46
EmilienMcschwede: the other thing with ring is also weird13:47
openstackgerritMerged openstack/tripleo-docs master: Remove os-cloud-config bits  https://review.openstack.org/45108813:47
*** ffiore has joined #tripleo13:48
*** noslzzp has joined #tripleo13:48
*** ratailor has joined #tripleo13:49
*** jpena|lunch is now known as jpena13:50
*** jaganathan has quit IRC13:50
*** fzdarsky has joined #tripleo13:51
*** jcoufal has quit IRC13:52
*** limao has quit IRC13:55
*** limao has joined #tripleo13:56
openstackgerritHarry Rybacki proposed openstack/tripleo-quickstart master: [WIP] Add supplemental node provisioner  https://review.openstack.org/45152313:56
*** Goneri has joined #tripleo13:56
shardytrown: Hey, quick question, any idea how hard it'd be to make quickstart not do all the unprivileged libvirt things?13:57
shardyI personally would find it useful if it could just do exactly what instack-virt-setup did and not have to deal with that extra layer of complexity13:57
trownshardy: there are some patches up from OPNFV folks to enable that13:58
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: Merge pre|post puppet resources into pre|post config.  https://review.openstack.org/45273413:58
shardytrown: Oh, nice, I'll give them a go13:58
trownthey actually require OVS for the libvirt networks, so need full root13:58
*** rbowen has joined #tripleo13:59
*** fragatina has joined #tripleo14:00
arxcruzweshay: do we have a 1-1 now ?14:01
weshayarxcruz, aye14:01
openstackgerritMerged openstack/instack-undercloud stable/newton: Disable VIP validation when UI is enabled  https://review.openstack.org/45185814:01
trownshardy: https://review.openstack.org/#/q/topic:bp/quickstart-opnfv is the topic for all the patches, but I dont think you would need all of those to just use root14:02
*** fragatina has quit IRC14:02
*** jkilpatr has quit IRC14:02
*** nmathew has quit IRC14:02
trownshardy: in fact maybe only this one was required https://review.openstack.org/#/c/422103/  and it is merged. setting ssh_user=root might just work14:03
*** mrunge_ has joined #tripleo14:03
*** mnaser has left #tripleo14:03
*** jaganathan has joined #tripleo14:03
trownhmm... probably not actually it would still setup all the networking to work unprivledged, and put vms under  user session14:04
trownin any case... yes it is on the roadmap for the OPNFV use case14:04
shardytrown: ah, yeah that's basically what I want to avoid, e.g have virt-manager just work without having to switch users etc14:05
shardytrown: sounds like just setting ssh_user to the current login might be enough tho14:05
adarazssshnaidm: can you add me to the CI squad board? https://trello.com/b/U1ITy0cu/tripleo-ci-squad14:05
*** artom has quit IRC14:06
shardye.g set that to shardy instead of letting it go put everything under stack14:06
*** limao_ has joined #tripleo14:06
*** artom has joined #tripleo14:06
trownshardy: ya that could work, I havent tried it though14:07
shardytrown: cool, no worries, I'll give it a go now14:07
*** ansmith has quit IRC14:07
*** ratailor has quit IRC14:08
*** rbowen has quit IRC14:08
*** artom has quit IRC14:08
*** artom has joined #tripleo14:08
sshnaidmadarazs, addes14:08
adarazssshnaidm: thanks :)14:08
adarazssshnaidm, weshay, panda, trown: do you mind if I move our cards over to the new board?14:09
*** limao has quit IRC14:09
openstackgerritJohn Trowbridge proposed openstack/tripleo-quickstart master: Allow permissive access to non_root_user files  https://review.openstack.org/41819814:09
sshnaidmadarazs, transition related?14:09
adarazssshnaidm: everything that we had on the RDO board.14:09
*** florianf has quit IRC14:09
adarazsas discussed.14:10
*** artom has quit IRC14:10
sshnaidmadarazs, everything???14:10
trownadarazs: yes please... if it is not too much of a pain14:10
pandaadarazs: only if you can do it atomically14:10
*** artom has joined #tripleo14:10
trownsshnaidm: I would think everything CI related14:10
sshnaidmtrown, adarazs yeah, I think if it's CI board, then only CI related14:11
adarazspanda: what do you mean by that? you want a possibility to reverse it?14:11
adarazssshnaidm: well, what is ours on the RDO board is the CI related stuff :)14:11
weshayadarazs, ack14:11
*** rbowen has joined #tripleo14:11
sshnaidmadarazs, but will you move RDO CI stuff too?14:11
adarazssshnaidm: yeah, hopefully we'll be one and the same :)14:12
sshnaidmadarazs, yeah, but we are still not..14:13
adarazssshnaidm: also, there's not much thing for RDO CI14:13
sshnaidmadarazs, isn't it more convenient to keep RDO CI stuff on RDO board?14:13
*** artom has quit IRC14:14
*** artom has joined #tripleo14:14
openstackgerritBen Nemec proposed openstack/puppet-tripleo master: Use correct manage_firewall hieradata  https://review.openstack.org/45277714:15
adarazssshnaidm: I don't see anything RDO specific there. I don't want to move the RDO Infra cards, just the CI cards, which is pretty much Quickstart related now.14:15
adarazsso it's not really RDO or TripleO dependent.14:15
bogdandomandre, jistr: http://lists.openstack.org/pipermail/openstack-dev/2017-April/114894.html14:16
EmilienMbnemec: nice fix. I won't if we should have a CI job without firewall enabled .14:16
*** udesale has quit IRC14:17
bnemecEmilienM: Maybe, although if that job ended up being the only one testing a service we might miss other breakage.14:17
bnemecI'm not positive whether it would have caught this anyway.14:17
*** pradk has joined #tripleo14:18
mandrebogdando: thanks! slightly related discussion in https://review.openstack.org/#/c/452742/ :)14:18
bnemecIt was only some of the rules, and if the main firewall manifest didn't create the default drop rule we'd still have been allowing everything.14:18
mandrebogdando: and https://review.openstack.org/#/c/452200/ afaict14:19
mandreit's important we agree on something14:19
bogdandomandre: thanks! let's follow up and brainstorm in the mail thread14:19
bogdandomay be Kolla folks will help us to pick the best way as well14:20
*** amoralej|lunch is now known as amoralej14:20
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: Merge pre|post puppet resources into pre|post config.  https://review.openstack.org/45273414:20
*** mich_chap has joined #tripleo14:22
*** ansmith has joined #tripleo14:22
openstackgerritMerged openstack/puppet-tripleo stable/ocata: Check rabbitmq user at step >= 2  https://review.openstack.org/45173714:22
openstackgerritMerged openstack/puppet-tripleo stable/newton: Fixes issues with raising mysql file limit  https://review.openstack.org/45146614:23
openstackgerritMerged openstack/tripleo-ui stable/ocata: Imported Translations from Zanata  https://review.openstack.org/45252614:23
openstackgerritMerged openstack/tripleo-ui master: Imported Translations from Zanata  https://review.openstack.org/45252514:23
openstackgerritMerged openstack-infra/tripleo-ci master: Set REMAINING_TIME for local run  https://review.openstack.org/45254214:23
openstackgerritMerged openstack/tripleo-heat-templates master: Remove EC2 endpoint from EndpointMap  https://review.openstack.org/45215014:23
openstackgerritMerged openstack/tripleo-heat-templates stable/newton: Increase libvirt/qemu.conf max_files and max_processes  https://review.openstack.org/44812214:23
openstackgerritMerged openstack/instack-undercloud master: Add a pointer to the tripleo-quickstart project as a replacement  https://review.openstack.org/45187214:24
openstackgerritMerged openstack/tripleo-quickstart master: Install requirements with pip update  https://review.openstack.org/43422614:24
*** florianf has joined #tripleo14:24
adarazssshnaidm: we also need @benkero2 @gchamoul @trown on that board.14:25
openstackgerritMerged openstack/puppet-pacemaker master: Fix potential cluster setup race  https://review.openstack.org/45182814:26
openstackgerritSven Anderson proposed openstack/tripleo-heat-templates stable/ocata: Add missing ec2api::api::keystone_ec2_tokens_url config  https://review.openstack.org/45043514:27
ansiwenEmilienM: ^^^ merged on master now, so stable/ocata now14:28
sshnaidmadarazs, done14:29
ansiwenEmilienM: ah, wait, the commit ID is wrong of course... I'll fix it14:29
adarazsthanks!14:29
sshnaidmbnemec, do you want be added to https://trello.com/b/U1ITy0cu/tripleo-ci-squad ?14:29
openstackgerritSven Anderson proposed openstack/tripleo-heat-templates stable/ocata: Add missing ec2api::api::keystone_ec2_tokens_url config  https://review.openstack.org/45043514:32
ansiwenEmilienM: ^^^ now14:33
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates stable/ocata: Set auth flag so ceilometer auth is enabled  https://review.openstack.org/45254014:36
*** ealcaniz has quit IRC14:38
openstackgerritSagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: Support built packages to be installed in image  https://review.openstack.org/45275514:38
*** ansmith has quit IRC14:38
*** vpickard is now known as vpickard_14:41
*** salmankhan has quit IRC14:41
*** ipsecguy has quit IRC14:42
*** chlong has joined #tripleo14:45
adarazssshnaidm: is this card done? https://trello.com/c/kqAmrQkv14:46
sshnaidmadarazs, yep14:47
*** ipsecguy has joined #tripleo14:47
*** janki has quit IRC14:48
adarazssshnaidm: moved it. I'm not going to transfer complete cards from the RDO board btw, I don't see much point.14:50
sshnaidmadarazs, I think it could be moved to completed14:50
*** salmankhan has joined #tripleo14:50
sshnaidmadarazs, not sure who should do it - me or tc/ua?14:50
adarazssshnaidm: you can do it, but I did it. :)14:51
openstackgerritLuke Hinds proposed openstack/puppet-tripleo master: SSHD Service extensions  https://review.openstack.org/44311314:51
sshnaidmadarazs, ok :)14:51
adarazssshnaidm: I mean that specific card.14:51
*** ansmith has joined #tripleo14:52
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: Don't check haproxy if external load-balancer is used.  https://review.openstack.org/45246014:54
openstackgerritMerged openstack/tripleo-heat-templates master: Remove useless trailing '\n' in /etc/hosts file.  https://review.openstack.org/44815214:54
openstackgerritMerged openstack/tripleo-heat-templates master: Qpid dispatch router composable role  https://review.openstack.org/42332614:54
*** lmiccini has quit IRC14:54
*** dtantsur|brb is now known as dtantsur14:54
openstackgerritMerged openstack/puppet-tripleo master: Move horizon to step 3  https://review.openstack.org/45234914:54
dr_gogeta86gm14:58
dr_gogeta86who can help me with the overcloud pxe ?14:59
bogdandojistr, mandre: another option is to use kolla_bootstrap by default having an "uptream" conditional as well, like TLS for rabbit https://review.openstack.org/#/c/450135/2/puppet/services/rabbitmq.yaml14:59
bogdandowhat do you think?14:59
jistrhmm i don't know about the upstream images...15:00
jistrthe advantage of kolla images is that we can build them all with our extensions, like puppet for the config containers15:01
jistrand we build them all the same way15:01
d0ugalrbrady: thanks for the reviews!15:02
jistralso if the upstream etcd image wants to run etcd as root, then i'd say that's a bit worse that the kolla image in this sense15:02
jistrhowever, i believe the entrypoints are perhaps a separate issue, and in many cases i'd probably prefer rolling our own rather than using Kolla ones15:03
jistr(as we somehow touched that topic in the thread about retirement of os-cloud-config)15:04
openstackgerritAlex Schultz proposed openstack/puppet-tripleo stable/ocata: Move horizon to step 3  https://review.openstack.org/45278515:05
openstackgerritTim Rozet proposed openstack/puppet-tripleo master: Fixes missing neutron base in sriov  https://review.openstack.org/45278615:06
openstackgerritTim Rozet proposed openstack/puppet-tripleo master: Fixes missing neutron base in sriov  https://review.openstack.org/45278615:06
*** limao_ has quit IRC15:06
*** limao has joined #tripleo15:07
openstackgerritMarios Andreou proposed openstack/tripleo-heat-templates master: Enforce upgrade_batch_tasks before upgrade_tasks order  https://review.openstack.org/44860215:08
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates master: [WIP] Ensure non-batched tasks start after batched tasks  https://review.openstack.org/45278915:08
bogdandofolks, could we please land it https://review.openstack.org/#/c/419040/ ?15:09
mariosgfidente: o/ is that like mine but without removing the conditional?15:13
gfidentemarios yes but I don't want to merge it, I wanted to push the version which causes the issue in heat15:14
mariosgfidente: ok i left a comment there fyi15:16
gfidentemarios I think I found the issue in heat15:16
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-common stable/newton: Updated from global requirements  https://review.openstack.org/45279915:17
openstackgerritPradeep Kilambi proposed openstack/puppet-tripleo master: Restrict mongodb memory usage  https://review.openstack.org/41909015:17
openstackgerritSteven Hardy proposed openstack/tripleo-heat-templates master: Remove not-working all-in-one upgrade environment  https://review.openstack.org/45280015:20
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Use openstack deploy to install the undercloud  https://review.openstack.org/41904015:20
openstackgerritBogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Custom containerized undercloud deployments  https://review.openstack.org/45079215:20
EmilienMshardy: hey, if you have 1 min, can you please look this cleanup https://review.openstack.org/#/c/452170/ ?15:22
shardyEmilienM: sure will do15:22
EmilienMsshnaidm: this one is ready for review https://review.openstack.org/#/c/452164/15:22
sshnaidmEmilienM, will look15:23
bogdandothis chain of patches https://review.openstack.org/#/c/450792 allows to drastically shorten dev envs (containerized undercloud-only) deployment time. I'm testing it now, but the the base patch from Flavio is tired of being rebased :/15:23
openstackgerritBogdan Dobrelya proposed openstack/tripleo-heat-templates master: Mount individual hostpath logs on /var/log  https://review.openstack.org/44260315:25
EmilienMbogdando: lgtm, +215:26
bogdandoEmilienM: thank you!15:26
*** deadnull has quit IRC15:26
openstackgerritTim Rozet proposed openstack/tripleo-heat-templates stable/newton: Enables increasing mariadb open files for noha deployments  https://review.openstack.org/45280315:27
shardybogdando: thanks, looking15:27
*** dsariel has joined #tripleo15:27
openstackgerritTim Rozet proposed openstack/tripleo-heat-templates stable/ocata: Fixes port binding controller for OpenDaylight  https://review.openstack.org/45271515:28
EmilienMcschwede: re: swift issues - should I file 2 bugs? not sure they exist yet15:28
bnemecsshnaidm: That would be fine.15:28
bogdandoshardy: thanks! note, the follow up patch https://review.openstack.org/#/c/450792/3 is kinda a 1st baby step15:28
bogdandoeventually those sh.j2 snowflakes shall emerge as playbooks15:28
*** leanderthal is now known as leanderthal|afk15:29
bogdandounless then, it just allows devs to have undercloud with all of the possible dev branches checked in15:30
cschwedeEmilienM: i think they are related, and might be because the swiftringbuilder service is included in the role but neiter swiftproxy nor swiftstorage? in that case i would expect both errors15:30
bogdandoquick and dirty :/15:30
*** tesseract has quit IRC15:31
openstackgerritHarry Rybacki proposed openstack/tripleo-quickstart master: [WIP] Add supplemental node provisioner  https://review.openstack.org/45152315:32
shardybogdando: I think I spotted in issue on https://review.openstack.org/#/c/419040/, please can you check my comments?15:34
shardydisclaimer - I'm still learning oooq15:34
cschwedeEmilienM: ok, now i know what happened. one second15:35
EmilienMcschwede: i'm working on another issue, I'll hold the swift thing for later, but let me know15:35
pandaadarazs: EPIC move is a task item, or a headline ?15:36
adarazspanda: can't parse question.15:36
bogdandoshardy: thanks! this Q is better off being answered by the authors ;)15:37
shardybogdando: AFAICT we've broken backwards compatibility as the old path will be ignored15:37
bogdandodprince, weshay:  PTAL ^^15:37
shardyso we need to write both to the path in undercloud.conf15:37
gfidentemarios I guess the condition needs to move out of step > 0 for lines 128 and 200 ?15:37
gfidente(too)15:38
bogdandoshardy: IIRC from the early revisions review, it is handled as appropriate15:38
*** limao has quit IRC15:38
bogdandohere https://review.openstack.org/#/c/419040/38/roles/undercloud-deploy/tasks/create-scripts.yml15:39
*** dsariel has quit IRC15:39
bogdandoshardy: it looks your comment is right tho!15:39
*** panda is now known as panda|bbl15:40
chemmarios: he, the upgrade ordering problem.  I think I have an instance of it15:42
chemmarios: I have a custom role that is doing its upgrade at the same time that the controller15:43
*** jlinkes has quit IRC15:43
chemmarios: would it match the description of what you're seeing ?15:43
bogdandoneed some help with troubleshooting https://review.openstack.org/#/c/450792/315:45
bogdandohttp://paste.openstack.org/show/B9msV5RVYYiWQwhkDAPZ/15:45
*** agurenko has quit IRC15:45
bogdandoso it deploys etcd for me, but I'm stuck with next steps :/15:45
bogdandowhy it fails for the rest of the stack15:45
bogdandowell, actually it deploys all I asked, just can't figure out the root cause of the failure in the snippet15:46
*** mdnadeem has quit IRC15:47
bogdandodocker ps https://pastebin.com/DLa6x0Sj15:47
*** rbrady is now known as rbrady-afk15:48
bogdandomy custom undercloud dev script https://github.com/bogdando/tripleo-quickstart-extras/blob/dev/roles/undercloud-deploy/templates/undercloud-deploy-dev.sh.j215:49
openstackgerritSaravanan KR proposed openstack/tripleo-heat-templates master: Add role specific information to the service template  https://review.openstack.org/43795615:51
openstackgerritEmilien Macchi proposed openstack/tripleo-quickstart master: Use AFS mirrors for optstool, delorean and qemu  https://review.openstack.org/45193815:52
cschwedeEmilienM: no direct need for 1 or 2 more bugs on launchpad; the issues will be fixed by this one: https://bugs.launchpad.net/tripleo/+bug/1665641 I included a short note why this happens15:52
openstackLaunchpad bug 1665641 in tripleo "Swift ringbuilding logic coupled to Controller/ObjectStorage roles" [Medium,In progress] - Assigned to Christian Schwede (cschwede)15:52
*** pcaruana has quit IRC15:53
marioschem: well it depends does the custome role used the batch tasks? https://review.openstack.org/#/c/448602/ is specifically about enforcing that batch_upgrade_tasks before upgrade_tasks15:53
EmilienMcschwede: excellent. i'll review the puppet patch15:54
chemmarios: how do I know that15:54
EmilienMcschwede: oh it seems like it's already -115:54
cschwedeEmilienM: wait a bit with reviewing, i'm working on it15:54
*** cschwede has left #tripleo15:55
EmilienMok15:55
openstackgerritJason E. Rist proposed openstack/tripleo-heat-templates stable/ocata: FQDN validation  https://review.openstack.org/45280915:55
bogdandoand my custom t-h-t dev is https://github.com/bogdando/tripleo-heat-templates/blob/dev/environments/mini.yaml / https://github.com/bogdando/tripleo-heat-templates/blob/dev/environments/docker-mini.yaml ^ ^ so please help with tshoot something from the logs15:55
*** cschwede has joined #tripleo15:56
bogdandoam I missing heat and/or neutron to the undercloud mini layout?..15:56
*** mcornea has quit IRC15:57
chemmarios:  in the role.yam file ?15:57
marioschem: no it is defined in the service manifest, right now just for ceph services15:57
marioschem: sec15:57
*** ipsecguy has quit IRC15:58
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: Merge pre|post puppet resources into pre|post config.  https://review.openstack.org/45273415:58
marioschem: e.g. https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/ceph-osd.yaml#L6615:58
*** ipsecguy has joined #tripleo15:58
sshnaidmadarazs, trown did you see this error? ERROR! the file_name '/home/jenkins/workspace/gate-tripleo-ci-centos-7-ovb-ha-oooq-nv/.quickstart/config/general_config/pacemaker.yml' does not exist, or is not readable15:59
sshnaidmadarazs, trown I saw it a few times today, i.e. http://logs.openstack.org/55/452755/2/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-oooq-nv/ec39f71/logs/oooq/quickstart_collectlogs.txt.gz15:59
chemmarios: so how come we can have this https://bugzilla.redhat.com/show_bug.cgi?id=1427569#c716:00
openstackbugzilla.redhat.com bug 1427569 in rhosp-director "OSP10 -> OSP11 upgrade fails when Nova services are running on a standalone node" [Urgent,Assigned] - Assigned to sathlang16:00
jristbnemec, slagle do you guys mind +A on https://review.openstack.org/452809 (no conflicts)16:01
adarazssshnaidm: nope. I also don't know where that file is coming from and how is it in the job.16:01
marioschem: is it upgrading ceph? or some ohter service (BZ talks about nova?)16:02
chemmarios: it's a nova custom role that get updated at the same time than the controller16:02
chemmarios: my head hurts16:03
marioschem: so if we need some nova services to be upgraded one node at a atime, then we can consider adding that service to the batch upgrade tasks, but might not be so simple (e.g. packaging bugs and services being started when we dont want them to like we've seen for neutron)16:03
marioschem: the batch_upgrade_tasks are executed on node at a time. i think the issue you're pointing at is about different roles' upgrade_tasks being executed simultaneously.16:04
chemmarios: yep16:04
chemmarios: that shouldn't happen right ?16:04
*** yamahata has joined #tripleo16:04
slaglejrist: sure, just needs to pass CI16:04
jristslagle: boom thanks16:04
jristbnemec: fair point, I wasn't sure how that worked16:05
marioschem: in which case it is about the ordering in the https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/major_upgrade_steps.j2.yaml#L183 , but we don't enforce cross role depdendency (i.e. controller step 1 depend on controller step 0 but i don't hink it cares about cinder step0 for example or even compute if you have a custome role like in your case16:05
bnemecjrist: Just occurred to me that we should have a bug for the backport.16:05
jristbeat you to it16:05
jristhaha16:05
bnemecYeah, I don't think we strictly require it, but it's preferred.16:05
chemmarios: yep, I was just reading it.  And you're right it doesn't care :)16:06
adarazssshnaidm: I sort of see the problem, but not how it arrives there.16:06
chemmarios: oki, so we now have a problem :)16:06
*** anshul has quit IRC16:06
adarazssshnaidm: this gets run: http://logs.openstack.org/55/452755/2/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-oooq-nv/ec39f71/console.html#_2017-04-03_15_48_30_94243516:06
chemmarios: how can we garanty that step5 has db back if role don't sync after each step16:06
paramitegreetings guys, can we get following merged? https://review.openstack.org/#/c/410210 https://review.openstack.org/#/c/45196216:06
chemmarios: we just don't care actually, we guaranty that whithin each role each step will be done after the other nothing more.16:07
adarazssshnaidm: and the ha,yml file gets corrected to pacemaker.yml: https://github.com/openstack/tripleo-quickstart/blob/master/quickstart.sh#L37816:07
chemmarios: we just happen to not hit it16:07
chemmarios: which is amazing really16:08
chemmarios: so I must be missing something16:08
adarazssshnaidm: but upstream uses a direct link to the file, and not a relative one... so it gets updated to the wrong one (from the $OPT_WORKDIR/16:08
sshnaidmadarazs, yeah, seems like that's a problem..16:08
*** trown is now known as trown|lunch16:08
shardydprince: Hey, what's the latest approach to https://docs.openstack.org/developer/tripleo-docs/containers_deployment/overcloud.html#populate-local-docker-registry ?16:09
adarazssshnaidm: one of the solutions it to correct the call to pacemaker.yml directly.16:09
chemmarios: but actually it may be.  We have controller/ceph/compute at best16:09
chemmarios: ceph does its batch thing16:09
chemmarios: compute is upgraded after16:09
chemmarios: so controller only16:09
shardydprince: I saw stevebaker has some changes ongoing in this area, so wasn't sure if I need to pull them or if this should work16:10
chemmarios: you just start to notice it when adding custom role16:10
dprinceshardy: yes, pull his changes I think.16:10
dprinceshardy: we have a custom Kolla template_overrides fill in tripleo-common16:10
dprinceshardy: http://git.openstack.org/cgit/openstack/tripleo-common/tree/contrib/tripleo_kolla_template_overrides.j216:10
dprinceshardy: if you do what to build your own imges you can use that file16:10
chemmarios: sorry for the spam.  I think I'm saying nonsense.  Just /ignore16:11
dprinceshardy: the images on tripleoupstream have been getting build like this though: https://github.com/dprince/undercloud_containers/blob/master/build_kolla.sh16:11
shardydprince: Ok, I think for now I just want to populate the local registry - how can I see what images it contains?16:11
chemmarios: hum ... maybe not.  I think that we have to loop over all the roles here: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/major_upgrade_steps.j2.yaml#L19416:12
marioschem: getting late here i am getting ready to call it EOF16:13
adarazssshnaidm: I'll try to fix quickstart.sh to not mess up the path.16:13
marioschem: maybe we can pickup tomorrow morning?16:13
chemmarios: ack, will post a bad ass review :)16:13
chemmarios: thanks16:13
dprinceshardy: I don't think we have a tool to list the local registry16:13
marioschem: sure you will :) thanks ttyl16:13
dprinceshardy: we push to it, and it checks for know images IDS16:13
shardydprince: Ok, cool - seems like something which might be useful in future for pre-flight tests16:15
dprinceshardy: agree :) and welcome to the wild west of docker registry16:15
dprinceshardy: seriously, maybe someone else knows besides me. I don't likst my local registry yet... other than to push things to it16:16
shardydprince: haha, well I've got a fresh environment up and running now so hopefully will be able to try out some things related to minor updates this week16:16
shardyI was thinking it'd be good know know what versions are in the registry etc before doing updates, will look into if/how it may be possible16:16
*** ma91 has quit IRC16:17
openstackgerritJason E. Rist proposed openstack/tripleo-heat-templates stable/ocata: FQDN validation  https://review.openstack.org/45280916:17
jristbnemec: ^16:17
*** ma9 has joined #tripleo16:18
*** dparkes has quit IRC16:18
*** jkilpatr has joined #tripleo16:18
*** d0ugal has quit IRC16:19
bogdandoplease review a set of patches https://review.openstack.org/#/c/447409/ https://review.openstack.org/#/c/447000/ https://review.openstack.org/#/c/448543/16:21
bogdandoit's all we need to implement https://bugs.launchpad.net/tripleo/+bug/1676373 !16:21
openstackLaunchpad bug 1676373 in tripleo "Allow users to run oooq from any host OS, from a container" [Medium,In progress] - Assigned to Bogdan Dobrelya (bogdando)16:21
EmilienMjaosorior: do we have some progress on https://review.openstack.org/#/c/445802/ and https://review.openstack.org/#/c/443010/ and also https://review.openstack.org/#/c/443104/ ?16:22
*** ma91 has joined #tripleo16:22
*** ma9 has quit IRC16:22
bogdandoand makes poor guys running non rhel os family happy16:22
*** zoli is now known as zoli|gone16:22
jaosoriorEmilienM: so I asked for reviews and one guy checked it out. But never answered :/16:23
*** mcornea has joined #tripleo16:23
EmilienMcschwede: when you have some time, could you look why https://review.openstack.org/#/c/443104/ is blocked please?16:23
*** jpich has quit IRC16:24
openstackgerritMartin André proposed openstack/tripleo-heat-templates master: Remove kolla_config copy from services  https://review.openstack.org/45136616:25
*** aufi has quit IRC16:29
jristflorianf: looks like https://review.openstack.org/#/c/451815/ has a +116:29
jristflorianf: workflow16:29
jristwoot16:29
*** dtrainor_ has joined #tripleo16:30
*** dtrainor_ has quit IRC16:30
*** dtrainor_ has joined #tripleo16:30
slaglebandini: any hints on how to debug this? http://paste.openstack.org/show/605273/16:32
weshayshardy, are you able to join the upgrade mtg?16:32
slaglebandini: puppet is getting stuck at step1 running that command. notice the "No route to host"16:32
slaglebandini: however, i can ping those hostnames (controller-0, controller-2) fine16:32
shardyweshay: sure joining16:33
*** dtrainor has quit IRC16:33
*** agurenko has joined #tripleo16:35
*** derekh has quit IRC16:35
openstackgerritAttila Darazs proposed openstack/tripleo-quickstart master: Fix the way we override legacy configs  https://review.openstack.org/45282716:35
adarazssshnaidm, weshay, trown|lunch, panda|bbl: https://review.openstack.org/452827 -- please try to push this through, we have problems with the OVB jobs upstream failing if this is not fixed with16:36
adarazs[0;31mERROR! the file_name '/home/jenkins/workspace/gate-tripleo-ci-centos-7-ovb-ha-oooq-nv/.quickstart/config/general_config/pacemaker.yml' does not exist, or is not readable[0m16:36
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates master: WIP: Make sure we wait for every role at each step during upgrade.  https://review.openstack.org/45282816:36
adarazsor at least the log collection...16:36
*** anshul has joined #tripleo16:37
openstackgerritSaravanan KR proposed openstack/tripleo-common master: Add an action to fetch and flatten the heat resource tree and parameters  https://review.openstack.org/45002116:39
* adarazs off16:39
thrashcan I get some love on https://review.openstack.org/#/c/449241/ and https://review.openstack.org/#/c/449242/16:39
*** mich_chap has quit IRC16:39
slaglebandini: i guess it's probably this16:40
slaglehttps://bugs.launchpad.net/tripleo/+bug/165710816:40
openstackLaunchpad bug 1657108 in tripleo "tripleo::firewall does not work as intended if the image has prepopulated firewall rules" [High,Fix released] - Assigned to Michele Baldessari (michele)16:40
*** b00tcat has quit IRC16:41
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/ocata: WIP: Make sure we wait for every role at each step during upgrade.  https://review.openstack.org/45283016:42
dprincemandre: have you profiled the longest runnings steps for the CI job?16:43
openstackgerritDmitry Tantsur proposed openstack/puppet-tripleo master: Include ironic::drivers::interfaces in the ironic-conductor profile  https://review.openstack.org/45283116:43
dprincemandre: just wondering if we have identified the longest running tasks...16:43
openstackgerritMerged openstack/tripleo-validations master: Fix token auth in ansible inventory script  https://review.openstack.org/45214716:47
openstackgerritMatt Young proposed openstack/tripleo-quickstart master: Revert "Install requirements with pip update"  https://review.openstack.org/45283316:49
*** bogdando has quit IRC16:50
openstackgerritJames Slagle proposed openstack/tripleo-heat-templates master: Purge initial firewall for deployed-server's  https://review.openstack.org/45283616:52
openstackgerritDmitry Tantsur proposed openstack/tripleo-heat-templates master: Add support for "neutron" Ironic networking plugin  https://review.openstack.org/45283716:53
*** ffiore has quit IRC16:53
openstackgerritDmitry Tantsur proposed openstack/tripleo-heat-templates master: DO NOT MERGE: testing ironic ocata  https://review.openstack.org/42805316:54
*** ma91 has quit IRC16:55
openstackgerritOpenStack Proposal Bot proposed openstack/os-collect-config master: Updated from global requirements  https://review.openstack.org/45104816:55
*** lucasagomes is now known as lucas-afk16:56
*** dtrainor_ is now known as dtrainor16:56
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Add scenario005 job (4nodes)  https://review.openstack.org/41809816:56
openstackgerritEmilien Macchi proposed openstack/tripleo-heat-templates master: Add scenario005 scenario  https://review.openstack.org/41809916:57
*** yprokule has quit IRC16:57
myoungtrown, rlandy, sshnaidm, weshay: if we could land this revert asap, all jobs are being affected --> https://review.openstack.org/#/c/452833/17:01
rlandymyoung" +1 on that  - killing rdocloud load test17:01
sshnaidmtrown|lunch, ^^17:02
*** rlandy is now known as rlandy|biab17:03
openstackgerritAlex Schultz proposed openstack/puppet-tripleo stable/ocata: Move horizon to step 3  https://review.openstack.org/45278517:03
*** jkilpatr has quit IRC17:05
*** chlong has quit IRC17:05
chandankumarsshnaidm: tosky trown|lunch please have a look on this https://review.openstack.org/#/c/451747/17:06
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates master: [WIP] Ensure non-batched tasks start after batched tasks  https://review.openstack.org/45278917:12
*** akuznetsov has joined #tripleo17:12
toskychandankumar: I don't understand, I don't find the log of tempest for the job where it was failing before17:13
weshaychem, do you want to own the AI to create a new git repo upstream?17:14
sshnaidmtosky, it's tested here https://review.openstack.org/#/c/452655/17:15
chemweshay: yes, I was about to ask for it17:15
sshnaidmtosky, we don't run tempest on every patch now17:15
weshaythanks +117:15
*** d0ugal has joined #tripleo17:15
chemweshay: but then ... we need to agree on a name !17:15
weshaylolz17:15
chemweshay: oki, will try to involve people and have it done by ... what would be the "deadline", next meeting ?17:16
weshaychem, sounds good to me17:16
chemweshay: ack17:17
*** dprince has quit IRC17:17
toskysshnaidm: I'm confused: in https://review.openstack.org/#/c/451747/, if you see my comment to patch 1, I could see the error (typo) because a task was failing17:17
toskysshnaidm: so that command was executed, but I'm not sure where the log was saved, I can't find it in the logs/ directory17:17
*** vpickard_ is now known as vpickard17:18
toskysshnaidm: the logs directory of gate-tripleo-ci-centos-7-ovb-ha-oooq-nv of https://review.openstack.org/#/c/451747/17:18
toskysshnaidm: because if the command was failed, it means that it was executed, so where is the log?17:18
sshnaidmtosky, we test it in different patch: here https://review.openstack.org/#/c/452655/17:18
sshnaidmtosky, we don't run tempest on every patch now17:19
*** akuznetsov has quit IRC17:19
sshnaidmtosky, you can look here: http://logs.openstack.org/55/452655/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha-oooq-nv/6cc38d2/logs/oooq/undercloud/home/jenkins/17:19
toskysshnaidm: care to expand? Do you mean that when the patch 1 was tested, tempest was executed in gate-tripleo-ci-centos-7-ovb-ha-oooq-nv  and now, few days later, it is not?17:19
sshnaidmtosky, yep, exactly17:20
toskyoh17:20
tosky"now" is a bit generic17:20
*** chlong has joined #tripleo17:20
sshnaidmtosky, yeah, it's from Friday17:20
openstackgerritMerged openstack/tripleo-ui stable/ocata: Add "Save and Close" button to configuration modal dialog  https://review.openstack.org/45198717:21
*** mcornea has quit IRC17:22
*** tosky has quit IRC17:23
EmilienMsshnaidm: should we revert https://review.openstack.org/#/c/451779/ ?17:26
sshnaidmEmilienM, yeah, I think we can17:27
EmilienMmandre: how is container job doing?17:27
sshnaidmEmilienM, 3 passed jobs, looks not bad17:27
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Revert "Remove ovb updates job from promotion pipeline"  https://review.openstack.org/45286617:27
openstackgerritEmilien Macchi proposed openstack-infra/tripleo-ci master: Revert "Remove ovb updates job from promotion pipeline"  https://review.openstack.org/45286617:27
sshnaidmEmilienM, I'd say containers job works pretty well too, something was changed that both upgrades and containers don't have timeouts17:28
sshnaidmEmilienM, sorry, s/upgrades/updates/17:29
*** liverpooler has quit IRC17:29
*** jaosorior is now known as jaosorior_away17:29
*** karimb has quit IRC17:30
*** yamahata has quit IRC17:32
florianfjrist: merged \o/17:34
openstackgerritCarlos Camacho proposed openstack/tripleo-heat-templates master: Merge pre|post puppet resources into pre|post config.  https://review.openstack.org/45273417:35
*** jpena is now known as jpena|away17:37
*** rlandy|biab is now known as rlandy17:38
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-heat-templates stable/ocata: Updated from global requirements  https://review.openstack.org/44391817:40
openstackgerritOpenStack Proposal Bot proposed openstack/tripleo-validations stable/ocata: Updated from global requirements  https://review.openstack.org/45288117:40
EmilienMbandini: hey, do you know about recent changes in cib? https://bugs.launchpad.net/tripleo/+bug/167903417:41
openstackLaunchpad bug 1679034 in tripleo "CI HA master job fails to set up cluster" [Critical,Confirmed]17:41
*** chem is now known as chem_gone17:41
*** akrivoka has quit IRC17:43
*** salmankhan has quit IRC17:44
*** shardy is now known as shardy_afk17:47
openstackgerritMerged openstack/tripleo-quickstart master: the ocata images in buildlogs are not available  https://review.openstack.org/45195517:48
openstackgerritMerged openstack/tripleo-quickstart-extras master: Remove ipxe dir before download  https://review.openstack.org/45079117:48
*** akrivoka has joined #tripleo17:48
openstackgerritMerged openstack/tripleo-quickstart-extras master: Delete symlinks and other non-files when collect logs  https://review.openstack.org/44671117:48
*** jjung has quit IRC17:49
EmilienMshardy_afk: re: https://review.openstack.org/#/c/452800/ should we have a release note? I think our users might be surprised if they used this file, now?17:52
*** bfournie has joined #tripleo17:53
*** lhinds is now known as lhinds|afk17:53
*** trown|lunch is now known as trown17:56
*** amoralej is now known as amoralej|off17:57
*** anshul has quit IRC17:57
*** d0ugal has quit IRC17:59
openstackgerritTom Barron proposed openstack/tripleo-heat-templates master: Change Manila NetApp DHSS default to false  https://review.openstack.org/45069017:59
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates master: WIP: Make sure we wait for every role at each step during upgrade.  https://review.openstack.org/45282818:00
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/ocata: WIP: Make sure we wait for every role at each step during upgrade.  https://review.openstack.org/45283018:00
*** dprince has joined #tripleo18:02
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates master: WIP: Make sure we wait for every role at each step during upgrade.  https://review.openstack.org/45282818:02
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/ocata: WIP: Make sure we wait for every role at each step during upgrade.  https://review.openstack.org/45283018:03
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates master: WIP: Make sure we wait for every role at each step during upgrade.  https://review.openstack.org/45282818:04
*** jaganathan has quit IRC18:05
openstackgerritAthlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/ocata: WIP: Make sure we wait for every role at each step during upgrade.  https://review.openstack.org/45283018:05
openstackgerritSagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Use web repo with installed packages for image build  https://review.openstack.org/45193618:08
*** florianf has quit IRC18:12
*** akrivoka has quit IRC18:15
openstackgerritDan Prince proposed openstack/tripleo-heat-templates master: HA test harness: minimal undercloud "HA testing"  https://review.openstack.org/45288918:15
*** florianf has joined #tripleo18:15
*** dtantsur is now known as dtantsur|afk18:15
*** pkovar has quit IRC18:16
EmilienMdprince: hey, do we have good progress on removing nonha job?18:16
openstackgerritMerged openstack-infra/tripleo-ci master: Populate logs exclude list with infra requirements  https://review.openstack.org/44637418:18
dprinceEmilienM: lets discuss tomorrow in the meeting18:18
openstackgerritMerged openstack-infra/tripleo-ci master: Remove unused variables for Upgrades  https://review.openstack.org/45217018:19
dprinceEmilienM: the IRC meeting. I want concensus on what we are going to do rather than guess.18:19
dprinceEmilienM: https://review.openstack.org/#/c/449785/ the comments here are asking for details. But we can't spend too much time on this.18:19
dprinceEmilienM: I think we should rather just turn it off.18:19
EmilienMdprince: sounds good. Please add a topic on https://etherpad.openstack.org/p/tripleo-meeting-items18:20
dprinceEmilienM: THe containers job already covers some of what 'nonha' covered anyways18:20
dprinceEmilienM: and anything else, we can make it cover in short order I think18:20
dprinceEmilienM: been spending time on speeding up the containers job, and HA fwiw18:20
EmilienMcool18:21
*** d0ugal has joined #tripleo18:21
dprinceEmilienM: lots of small things I think. Nothing major yet18:21
EmilienMdprince: good, just make sure to add it to the agenda, so we don't forget about it. Thanks18:22
*** athomas has quit IRC18:22
EmilienMdprince: https://etherpad.openstack.org/p/tripleo-meeting-items18:22
dprinceEmilienM: done18:22
openstackgerritwes hayutin proposed openstack/tripleo-quickstart-extras master: Use openstack deploy to install the undercloud  https://review.openstack.org/41904018:25
EmilienMdprince: ++18:26
*** maeca1 has joined #tripleo18:28
*** liverpooler has joined #tripleo18:32
EmilienMsshnaidm: can you look https://review.openstack.org/#/c/451938 please? see my last comment18:34
openstackgerritDan Prince proposed openstack/tripleo-common master: kolla_template_overrides: pacemaker rabbit  https://review.openstack.org/45289018:34
EmilienMpabelanger: rdo mirror look good & stable for us, we're going to switch tripleo to use it too sometimes very soon if not today18:34
dprincebandini: putting this together18:35
dprincebandini: https://etherpad.openstack.org/p/tripleo-containers-ha18:36
dprincebandini: might you and the HA crew help fill in some details from the pacemaker side?18:36
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: Re-Add bigswitch agent support  https://review.openstack.org/45227818:36
openstackgerritMerged openstack/instack-undercloud master: Enable boot mode detection by ironic-inspector  https://review.openstack.org/44862418:37
*** artom has quit IRC18:40
*** abishop_ has joined #tripleo18:41
*** abishop has quit IRC18:41
*** artom has joined #tripleo18:41
*** athomas has joined #tripleo18:46
*** maeca1 has quit IRC18:52
openstackgerritFlorian Fuchs proposed openstack/tripleo-validations master: Blacklist setuptools versions 34.0.0-34.3.2  https://review.openstack.org/45289818:52
colonwqEmilienM, Patch 452197 passes CI except for the non-voting upgrade jobs. Back porting this patch should resolve this upgrade issue.18:52
openstackgerritTim Rozet proposed openstack/tripleo-heat-templates master: Adds service for OpenDaylight DPDK + OVS  https://review.openstack.org/41964818:55
openstackgerritTim Rozet proposed openstack/tripleo-heat-templates master: Adds service for OpenDaylight DPDK + OVS  https://review.openstack.org/41964818:57
*** agurenko has quit IRC18:58
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Parameterize OVB stack flavors  https://review.openstack.org/44516318:59
*** florianf has quit IRC19:07
pabelangerEmilienM: please don't switch just yet19:14
pabelangerwe are still rolling out things19:14
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Move OVB settings to tripleo-quickstart-extras  https://review.openstack.org/44880519:17
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Parameterize OVB stack flavors  https://review.openstack.org/44516319:18
*** dparkes has joined #tripleo19:20
*** salmankhan has joined #tripleo19:21
*** maeca1 has joined #tripleo19:21
EmilienMpabelanger: ok19:22
*** maeca1 has quit IRC19:22
*** d0ugal has quit IRC19:23
*** d0ugal has joined #tripleo19:25
*** salmankhan has quit IRC19:26
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Move OVB settings to tripleo-quickstart-extras  https://review.openstack.org/44880519:33
*** thrash is now known as thrash|brb19:33
*** jjung has joined #tripleo19:34
*** d0ugal has quit IRC19:36
*** trozet has quit IRC19:36
*** bogdando has joined #tripleo19:37
*** chlong has quit IRC19:37
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Move OVB settings to tripleo-quickstart-extras  https://review.openstack.org/44880519:37
EmilienMsshnaidm: does multinode oooq support 4nodes jobs?19:38
sshnaidmEmilienM, not atm19:38
EmilienMsshnaidm: yeah, I figured https://review.openstack.org/#/c/418099/19:38
EmilienMI miss logs to debug: http://logs.openstack.org/99/418099/29/experimental/gate-tripleo-ci-centos-7-scenario005-multinode-oooq-nv/d937c56/logs/19:38
*** yamahata has joined #tripleo19:40
EmilienMsshnaidm: what are the missing bits exactly?19:40
openstackgerritMerged openstack/tripleo-quickstart master: Revert "Install requirements with pip update"  https://review.openstack.org/45283319:41
sshnaidmEmilienM, ovb jobs didn't save logs due a bug that was introduced today and already fixed, need just to rerun them19:42
sshnaidmEmilienM, looking at patch..19:42
EmilienMsshnaidm: my stuff is not ovb19:42
sshnaidmEmilienM, which logs do you miss?19:42
EmilienMsshnaidm: all subnodes19:43
openstackgerritPradeep Kilambi proposed openstack/instack-undercloud stable/ocata: Remove obsolete flag on gnocchi-upgrade  https://review.openstack.org/45291219:43
sshnaidmEmilienM, in which job?19:43
sshnaidmEmilienM, do you have a link?19:44
EmilienMsshnaidm: http://logs.openstack.org/99/418099/29/experimental/gate-tripleo-ci-centos-7-scenario005-multinode-oooq-nv/d937c56/logs/19:44
sshnaidmEmilienM, subnode-2 is there19:45
EmilienMsshnaidm: it's a 4nodes job like I said :D19:46
sshnaidmEmilienM, hmm.. can't answer now.. need to check19:50
*** jprovazn has quit IRC19:50
sshnaidmEmilienM, looks like a bug in oooq, I'll try to run it with some debug19:50
*** chlong has joined #tripleo19:51
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Move OVB settings to tripleo-quickstart-extras  https://review.openstack.org/44880519:51
*** nyechiel has joined #tripleo19:51
*** karimb has joined #tripleo19:52
sshnaidmEmilienM, are you sure it's 4 nodes?19:52
sshnaidmEmilienM, http://logs.openstack.org/99/418099/29/experimental/gate-tripleo-ci-centos-7-scenario005-multinode-oooq-nv/d937c56/console.html#_2017-04-03_17_56_48_69460519:53
sshnaidmEmilienM, "cat /etc/nodepool/sub_nodes_private" gives one node only19:53
EmilienMsshnaidm: probably a project config thing, let me check19:54
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Parameterize OVB stack flavors  https://review.openstack.org/44516319:54
EmilienMsshnaidm: https://review.openstack.org/45291319:55
*** dtrainor has quit IRC19:56
EmilienMsshnaidm: have you seen anything wrong to do recheck on https://review.openstack.org/#/c/451938/ ?19:56
*** thrash|brb is now known as thrash19:56
*** toure is now known as toure|biab19:57
pradkcan i get another review on this please https://review.openstack.org/#/c/419090/19:57
sshnaidmEmilienM, yeah, I want to see how ovb works with it, it didn't have logs because of bug which was fixed already19:58
EmilienMsshnaidm: ah ok19:58
*** dtrainor has joined #tripleo19:59
EmilienMsshnaidm: sorry what bug? what is fix?19:59
pradkthx19:59
weshaytrown, any idea what is going on w/ the artifacts server on centos?19:59
sshnaidmpradk, hi, tried to reach you today, do you know maybe why gnocchi fails in nonha quickstart jobs? http://logs.openstack.org/94/447994/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha-oooq-nv/7a7bb81/logs/oooq/undercloud/home/jenkins/overcloud_deploy.log.txt.gz#_2017-04-03_14_53_3120:00
pradklooking20:00
trownweshay: I did not realize there was an issue... whats up?20:00
weshayjust poking around.. https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-gate-newton-delorean-quick-basic-718/20:01
weshay40420:01
pradksshnaidm, do you have more logs? in particular /var/log/gnocchi/*20:01
sshnaidmEmilienM, I broke a little bit quickstart :D And we reverted today https://review.openstack.org/#/c/452833/20:01
weshayseeing too many of them.. but this is active https://ci.centos.org/artifacts/rdo/?C=M;O=D20:01
*** shardy_afk has quit IRC20:02
weshaysshnaidm, how was that not caught by the check gates?20:02
sshnaidmpradk, of course http://logs.openstack.org/94/447994/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha-oooq-nv/7a7bb81/logs/oooq/undercloud/var/log/20:02
sshnaidmpradk, it's undercloud20:02
EmilienMsshnaidm: ok20:02
sshnaidmpradk, all logs http://logs.openstack.org/94/447994/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha-oooq-nv/7a7bb81/logs/oooq/20:02
pradksshnaidm, well as per those logs upgrde worked20:03
sshnaidmweshay, seems like there were another changes and my patch was pretty old20:03
pradksshnaidm, http://logs.openstack.org/94/447994/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha-oooq-nv/7a7bb81/logs/oooq/undercloud/var/log/gnocchi/gnocchi-upgrade.log.txt.gz20:03
pradksshnaidm, what backend is this btw? swift or ceph?20:04
*** shardy_afk has joined #tripleo20:04
sshnaidmpradk, I think it may be overcloud gnocchi fails, could it be?20:04
sshnaidmpradk, http://logs.openstack.org/94/447994/1/experimental-tripleo/gate-tripleo-ci-centos-7-ovb-nonha-oooq-nv/7a7bb81/logs/oooq/overcloud-controller-0/var/log/gnocchi/gnocchi-upgrade.log.txt.gz20:04
pradkah yep20:05
pradkhm hostname mismatch20:05
pradkSSLError: hostname '192.168.24.8' doesn't match u'10.0.0.5'20:05
pradkthis looks like a config issue20:05
sshnaidmpradk, I see.. thanks20:06
pradk Certificate did not match expected hostname: 192.168.24.8.20:06
pradkwrong CN in cert perhaps20:06
trownweshay: those 404's usually just mean the job failed before uploading logs... like it failed to get a node20:07
weshaytrown, thanks20:09
*** vpickard has quit IRC20:10
jristslagle, bnemec - passing CI https://review.openstack.org/#/c/452809/20:10
*** vpickard_afk has joined #tripleo20:10
*** vpickard_afk is now known as vpickard20:10
jristanother one that needs a +a ... https://review.openstack.org/#/c/451711/520:11
*** jrist has left #tripleo20:11
*** jrist has joined #tripleo20:11
*** Goneri has quit IRC20:11
*** rwsu has quit IRC20:11
*** trozet has joined #tripleo20:11
radezpradk: could you take a quick look at this one too?  https://review.openstack.org/#/c/450182/20:12
radezI say too because I filed it the same time as the one you gave workflow to earlier today :)20:12
pradkradez, done20:12
radezthx pradk!20:12
pradknp20:12
*** rwsu has joined #tripleo20:15
*** liverpooler has quit IRC20:16
*** jayg is now known as jayg|g0n320:16
openstackgerritJames Slagle proposed openstack/python-tripleoclient master: Add --skip-deploy-identifier  https://review.openstack.org/45222320:19
*** salmankhan has joined #tripleo20:19
openstackgerritJames Slagle proposed openstack/tripleo-common master: Add skip_deploy_identifier  https://review.openstack.org/45231420:19
weshaysshnaidm, EmilienM we need to cut down on the logging some more w/ the ovb jobs.. at least for containers20:20
sshnaidmweshay, is there still problem with size?20:21
weshaysshnaidm, time20:21
*** iranzo has quit IRC20:21
*** vpickard has quit IRC20:21
*** flepied has joined #tripleo20:22
weshaysshnaidm, mandre was just discussing it20:22
jristthanks slagle20:22
sshnaidmweshay, it's about 13 minutes as I see20:23
openstackgerritJames Slagle proposed openstack/instack-undercloud master: Remove instack-virt-setup  https://review.openstack.org/45191420:23
weshaysshnaidm, apparently took 4 w/ tripleo.sh20:23
sshnaidmweshay, containers?? they never ran with tripleo.sh20:24
EmilienMweshay: I noticed the time to get logs from container is quite important20:24
*** jkilpatr has joined #tripleo20:25
*** vpickard_afk has joined #tripleo20:25
sshnaidmweshay, ha oooq jobs take 7 minutes, ha non-oooq jobs take about 5 minutes, 2 minutes difference is not so important I think20:25
weshaysshnaidm, that is true20:25
*** vpickard_afk is now known as vpickard20:25
sshnaidmweshay, containers have 13 minutes because they fetching logs from containers is expensive..20:26
weshaymandre, ^20:26
EmilienM13 min wtf20:26
openstackgerritJames Slagle proposed openstack/python-tripleoclient master: Add --skip-deploy-identifier  https://review.openstack.org/45222320:26
EmilienMI remember a script from flaper8720:26
weshayjistr, that is awesome20:26
weshayjistr++20:26
EmilienMhttps://review.openstack.org/#/c/441050/16/scripts/get_docker_logs.sh20:27
EmilienMthis one I think20:27
mandresshnaidm: last time I checked it only spent 4 minutes collecting container logs out of these 13 minutes20:27
gfidenteEmilienM can you check https://review.openstack.org/#/c/452197/ ?20:27
sshnaidmmandre, I see about 5 minutes in last job, yes20:27
gfidenteEmilienM meant to fix an issue with upgrades from O for RGW20:27
gfidentewe'll need to backport it to the stable branch too20:27
mandresshnaidm: btw, I was just pointing out that there may be something to optimize there :)20:28
sshnaidmmandre, I think getting overcloud info take more time too20:28
mandresshnaidm: my guess is that there is a lot of back and forth caused by ansible20:28
sshnaidmmandre, what do you mean?20:29
EmilienMgfidente: approved20:29
mandresshnaidm: lot of small ansible tasks20:29
gfidenteEmilienM thanks20:29
sshnaidmmandre, fetching docker logs is one bash script20:30
mandresshnaidm: yeah, it means that fetching the non-containerized logs take 9 minutes, it's a lot of time20:31
sshnaidmmandre, in usual ha job it's about 7 minutes, vs 5 minutes with tripleo.sh20:32
sshnaidmmandre, but collecting logs is also fetching info from overcloud, it's not only copying files20:33
sshnaidmmandre, I think we spend a minute more for this in containers20:33
mandresshnaidm: hmm in my experience the tripleo.sh log collecting was taking about half the time of oooq, but I haven't looked to closely at it20:33
sshnaidmmandre, just now checked in status page, oooq ha 7 minutes, non-oooq ha 5 minutes20:34
sshnaidmmandre, weshay but is it really the problem? getting logs from containers can not be so fast as copying files locally, isn't it?20:35
mandresshnaidm: we're just trying to identify potential slowness in the containers so we can optimize20:36
openstackgerritLokesh Jain proposed openstack/tripleo-heat-templates master: Addition of firewall rules for Nuage  https://review.openstack.org/45293220:36
jristdtrainor: ping20:38
jristdtrainor: are you chasing reviews on https://review.openstack.org/#/c/452017/ ? i.e. is it ready?20:38
dtrainorjrist, pong20:38
dtrainorah yeah i am actually, real bad.  sorry i had been working on something else.  i'd really like reviews on that.20:39
* jrist wonders how busy EmilienM is today20:39
dtrainori don't know how he even manages to eat, sleep, family etc20:40
dtrainorbnemec, jaosorior_away, i apologize for asking directly but if you have a few free moments may i ask you to take a look at https://review.openstack.org/#/c/452017/ ?20:40
EmilienMjrist: shoot :D20:42
openstackgerritLokesh Jain proposed openstack/tripleo-heat-templates master: Addition of firewall rules for Nuage  https://review.openstack.org/45293220:44
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Move OVB settings to tripleo-quickstart-extras  https://review.openstack.org/44880520:48
*** rbowen has quit IRC20:49
dtrainorthanks!  bnemec, nit: noted.  thanks.20:49
*** ansmith has quit IRC20:49
openstackgerritLokesh Jain proposed openstack/tripleo-heat-templates master: Addition of firewall rules for Nuage  https://review.openstack.org/45293220:49
*** rbrady-afk is now known as rbrady20:54
pradkEmilienM, so is the updates job just doing an update on existing deploy without pulling in new tht hiera data20:54
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates master: [WIP] Ensure non-batched tasks start after batched tasks  https://review.openstack.org/45278920:54
jristdtrainor: woot +A20:54
pradkEmilienM, if so we just have to merge tht first?20:54
*** trown is now known as trown|outtypewww20:55
EmilienMpradk: which one20:55
pradkEmilienM, your -1 on https://review.openstack.org/#/c/452541/20:55
pabelangerEmilienM: okay, lets ramp up testing of RDO proxy. We are ready to increase testing20:55
pradkEmilienM, i think its catch 22 issue.. once tht merges this wont be an issue i believe20:55
*** trozet has quit IRC20:56
pradkEmilienM, for example in master with both merged it works fine20:56
EmilienMpabelanger: ack20:56
EmilienMpradk: let's see20:57
EmilienMpradk: the tht patch is in gate now20:57
EmilienMpabelanger: https://review.openstack.org/#/c/451938/ we want to see it green on ovb first20:59
pradkEmilienM, https://review.openstack.org/#/c/450903/ is the prefect test for this and it works20:59
*** dprince has quit IRC21:00
openstackgerritDmitry Tantsur proposed openstack/tripleo-heat-templates master: Add support for "neutron" Ironic networking plugin  https://review.openstack.org/45283721:01
*** d0ugal has joined #tripleo21:01
*** dprince has joined #tripleo21:01
openstackgerritDmitry Tantsur proposed openstack/tripleo-heat-templates master: DO NOT MERGE: testing ironic  https://review.openstack.org/42805321:02
*** athomas has quit IRC21:07
*** shardy_afk has quit IRC21:08
jristEmilienM: thanks21:10
*** lblanchard has quit IRC21:10
*** trozet has joined #tripleo21:11
*** nyechiel has quit IRC21:12
*** bogdando has quit IRC21:13
*** dprince has quit IRC21:14
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart master: Download overcloud_release rpm for mixed upgrade  https://review.openstack.org/44934921:17
hamzydoes anyone know how http://artifacts.ci.centos.org/rdo/images/mitaka/delorean/consistent/stable/ is built?  I need to see if I can build a ppc64le version...21:17
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart master: Download overcloud_release rpm for mixed upgrade  https://review.openstack.org/44934921:19
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: Allow complex upgrade deployment for N to O  https://review.openstack.org/43959821:20
*** abishop_ has quit IRC21:21
*** nyechiel has joined #tripleo21:24
openstackgerritRonelle Landy proposed openstack/tripleo-quickstart-extras master: DO NOT MERGE: Add test commit to check gate triggers  https://review.openstack.org/40381821:24
*** artom has quit IRC21:28
*** maeca1 has joined #tripleo21:28
*** maeca1 has quit IRC21:28
*** artom has joined #tripleo21:29
*** artom_ has joined #tripleo21:30
*** salmankhan has quit IRC21:31
*** snecklifter has quit IRC21:32
*** Goneri has joined #tripleo21:33
*** snecklifter has joined #tripleo21:34
*** artom has quit IRC21:34
*** trozet has quit IRC21:36
EmilienMbnemec, sshnaidm: https://review.openstack.org/#/c/452866/21:38
EmilienMsshnaidm: https://review.openstack.org/#/c/451938/ failed, we still don't have much logs..21:39
sshnaidmEmilienM, it ran a little before fix was merged, ran recheck again21:40
openstackgerritSimon Dodsley proposed openstack/puppet-tripleo master: Composable services support for Cinder Pure Storage FlashArray  https://review.openstack.org/45220521:41
EmilienMsshnaidm: ack, thx21:42
*** ramishra has quit IRC21:42
openstackgerritMerged openstack/puppet-tripleo stable/ocata: Move horizon to step 3  https://review.openstack.org/45278521:42
openstackgerritMerged openstack/puppet-tripleo master: Restrict mongodb memory usage  https://review.openstack.org/41909021:42
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates master: [WIP] Ensure non-batched tasks start after batched tasks  https://review.openstack.org/45278921:48
*** nyechiel has quit IRC21:53
*** rhallisey has quit IRC21:55
*** bfournie has quit IRC22:00
*** morazi has quit IRC22:04
openstackgerritGiulio Fidente proposed openstack/tripleo-heat-templates master: [WIP] Ensure non-batched tasks start after batched tasks  https://review.openstack.org/45278922:06
*** gfidente has quit IRC22:10
openstackgerritAlexander Chuzhoy proposed openstack/tripleo-docs master: Accessing the ramdisk - selinux note  https://review.openstack.org/45297422:12
*** rcernin has quit IRC22:12
*** Goneri has quit IRC22:15
*** trozet has joined #tripleo22:16
openstackgerritPradeep Kilambi proposed openstack/tripleo-heat-templates master: Add params to tweak memory limit on mongodb  https://review.openstack.org/45297522:17
openstackgerritAlex Schultz proposed openstack/puppet-tripleo stable/newton: Move horizon to step 3  https://review.openstack.org/45297722:19
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: Setting keystone region for congress  https://review.openstack.org/45018222:24
openstackgerritMerged openstack/tripleo-heat-templates stable/ocata: FQDN validation  https://review.openstack.org/45280922:24
openstackgerritMerged openstack/puppet-tripleo stable/newton: Add support to changing the Rabbitmq password on update  https://review.openstack.org/45174222:24
openstackgerritMerged openstack/puppet-tripleo stable/newton: Check rabbitmq user at step >= 2  https://review.openstack.org/45260922:25
openstackgerritMerged openstack/puppet-tripleo master: Add tunnel timeout for ui proxy container  https://review.openstack.org/45201722:25
openstackgerritMerged openstack/instack-undercloud stable/ocata: Remove obsolete flag on gnocchi-upgrade  https://review.openstack.org/45291222:25
*** inc0 has joined #tripleo22:30
inc0hello:) I need help from puppetmasters here...22:31
inc0can I have someone help us write puppet module for registry in infra? so we can push our images from built gates?22:31
inc0tripleo would totally benefit from it too22:31
inc0issue is, in our community we don't have any puppet whisperers22:32
inc0https://review.openstack.org/#/c/399221/2 <-22:32
*** chlong has quit IRC22:38
*** Goneri has joined #tripleo22:41
*** suuuper has quit IRC22:42
openstackgerritAlex Schultz proposed openstack/tripleo-heat-templates master: Add tags to roles  https://review.openstack.org/45156122:45
*** Goneri has quit IRC22:46
openstackgerritPradeep Kilambi proposed openstack/instack-undercloud stable/ocata: Replace hardcoded regions by hiera call  https://review.openstack.org/45298522:48
*** sshnaidm is now known as sshnaidm|off22:48
openstackgerritPradeep Kilambi proposed openstack/instack-undercloud stable/ocata: Set default domain for all keystone users  https://review.openstack.org/45298722:49
*** pradk has quit IRC22:51
*** bfournie has joined #tripleo22:52
*** bfournie has quit IRC22:53
openstackgerritMerged openstack-infra/tripleo-ci master: Revert "Remove ovb updates job from promotion pipeline"  https://review.openstack.org/45286622:56
*** cylopez has quit IRC23:06
*** arxcruz has quit IRC23:19
*** yamahata has quit IRC23:21
*** arxcruz has joined #tripleo23:21
*** trozet has quit IRC23:22
openstackgerritMerged openstack/puppet-tripleo master: Deploy WSGI apps at the same step (3)  https://review.openstack.org/43401623:23
*** nyechiel has joined #tripleo23:43
*** ansmith has joined #tripleo23:44
*** nyechiel has quit IRC23:50

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!