15:02:57 <lennyb> #startmeeting third-party
15:02:58 <openstack> Meeting started Mon Jun  5 15:02:57 2017 UTC and is due to finish in 60 minutes.  The chair is lennyb. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:02:59 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:03:02 <openstack> The meeting name has been set to 'third_party'
15:03:07 <lennyb> Hello evrybody
15:08:09 <mmedvede> hey lennyb
15:08:15 <mmedvede> sorry I am late
15:09:42 <lennyb> mmedvede, nice to 'see' you :)
15:09:55 <lennyb> anything you would like to discuss today?
15:10:33 <lennyb> I have a question
15:10:39 <mmedvede> just a note about addition of etcd to devstack base service list. It kind of broke non-x86 things
15:10:54 <mmedvede> we are currently using dirty hacks to make it work
15:11:15 <lennyb> mmedvede, thanks. We experienced a lot of failures with this, until we disabled it by default :)
15:12:10 <mmedvede> you are deferring your breakage till some service would start depending on it :)
15:13:07 <lennyb> I've noticed that our nodepool makes jobs run slower on the cloud. We have a 3 jobs running one after another, but after first one is finished, the related VM is being deleted
15:13:13 <lennyb> and nodepool adds a new one
15:13:34 <lennyb> during nodepool delete and building the running job are very slow
15:13:57 <lennyb> any tips/ideas ?
15:14:21 <mmedvede> did you check cpu use of nodepoold? I had problems with it (and I think I still do). Last time I investigated it, shade was doing something bad
15:14:45 <lennyb> sorry, what is 'shade' ?
15:14:46 <mmedvede> I remember it had a busy wait loop with no sleep, so it was just spinning there like crazy
15:15:06 <mmedvede> lennyb: shade is an abstraction library nodepool uses to talk to clouds
15:15:08 <mordred> it shouldn't be a 0-wait busy loop - that was a bug back in the day
15:15:26 <mordred> so if it is, please let me know and we'll get that fixed
15:15:30 <mmedvede> mordred: yeah, I think it was solved
15:15:48 <lennyb> mordred, hi, do you remember when ( what version ) it was solved in?
15:16:04 <mordred> cool. there's also a few config settings I can point to that can alleviate that as well - lemme point to link
15:16:05 <mmedvede> but I still have strange high system load when nodepool does its thing, did not have time to investigate
15:16:07 <lennyb> I am using 0.3.1
15:16:49 <mordred> first 5 lines here: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/templates/nodepool/clouds.yaml.erb
15:17:29 <mordred> if you put them in your clouds.yaml - they should control the pause in teh poll loop for those resoruces, which is most likely the issue if you're seeing tight busy loops
15:17:35 <lennyb> mordred, thanks. I will take a look.
15:18:19 <mmedvede> I have mine set at 2, would try to set to 5
15:18:23 <mmedvede> thanks mordred
15:19:10 <lennyb> mordred, BTW, could you, pls, review #link https://review.openstack.org/#/c/450296/
15:19:11 <patchbot> patch 450296 - openstack-infra/puppet-openstackci - added Single Use Slave plugin
15:19:14 <mordred> sure nuff - definitely let me know if you continue having problems with that - all of that code is there ot enable scaling, not to make it harder :)
15:19:53 <lennyb> anything else you would like to share/discuss/bring up ?
15:20:08 <mordred> lennyb: done!
15:20:19 <lennyb> mordred, 10x
15:20:38 <mmedvede> was trying to find the patch for devstack/etcd fix non-x86 arches would need, would paste shortly
15:22:02 <lennyb> mmedvede, I dont see nodepool 0.3.2 tag, unless you meant another version
15:22:57 <mmedvede> No, I meant 0.3.1, there is no 0.3.2
15:23:10 <lennyb> mmedvede, do you have CI on stable branches?
15:24:48 <mmedvede> found the review for allowing custom etcd download in d-g: https://review.openstack.org/#/c/467437/
15:24:49 <patchbot> patch 467437 - openstack-infra/devstack-gate - Allow overriding ETCD_DOWNLOAD_URL
15:25:01 <mmedvede> lennyb: yes, we do test on stable branches selectively
15:26:43 <lennyb> mmedvede, you are running stable/branches on clean VMs, right?
15:30:06 <mmedvede> lennyb: yes
15:30:20 <mmedvede> we currently do not reuse any VMs
15:30:52 <lennyb> mmedvede, OK. we are reusing physical servers and having some issues with cleaning.
15:31:15 <lennyb> mordred, mmedvede, any other questions/issues for today's meeting?
15:33:03 <mmedvede> lennyb: nothing from me, thanks
15:33:51 <mmedvede> just a note - patchbot is awesome
15:34:41 <lennyb> mordred, mmedvede, Thanks for the meeting. I will close now. See you next week.
15:34:58 <lennyb> #endmeeting