15:00:20 <lennyb_> #startmeeting third-party
15:00:27 <openstack> Meeting started Mon Feb 27 15:00:20 2017 UTC and is due to finish in 60 minutes.  The chair is lennyb_. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:28 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:31 <openstack> The meeting name has been set to 'third_party'
15:00:35 <lennyb_> Hello
15:00:36 <asselin> o/
15:00:46 <lennyb_> how are you asselin?
15:01:35 <asselin> i'm fine, yourself?
15:02:11 <lennyb> our CI is down for a few days due to #link refs/changes/80/437880/1
15:02:26 <lennyb> and we are upgrading our Ci from Ub14.04 to 16.04
15:02:55 <asselin> why the upgrade? did -infra upgrade already?
15:03:16 <mmedvede> o/
15:03:24 <lennyb> #link https://review.openstack.org/#/c/437880
15:04:02 <lennyb> seems that devstack was not 'checked' on Ub14.04 and either I need to FORCE=yes or upgrade the CI servers
15:04:20 <lennyb> asselin, I am upgrading Mellanox Internal CI
15:04:37 <lennyb> hi mmedvede
15:04:50 <mmedvede> we are using FORCE all the time, makes things easier
15:05:17 <asselin> I haven't seen issues....maybe we have it pinned somewhere
15:05:27 <lennyb> mmedvede...yeah, but we need to check newer OS somewhen :)
15:06:19 <lennyb> mmedvede, maybe I will take your proposal and upgrade things slower
15:06:42 <lennyb> anything else you would like to bring up/discuss/complain ?
15:06:52 <mmedvede> lennyb: we are using 16.04, FORCE by default just makes it easier in case we decide to use something non-standard.
15:07:11 <mmedvede> last week was not a good week for third-party CIs :)
15:07:22 <lennyb> did some of you saw 'No hosts found to map to cell, exiting.' issue?
15:08:04 <lennyb> mmedvede, I have this week 'not a good' one
15:08:11 <mptacekx> lennyb: hi, can't be related to placement api introduction from earlier ? is nova-compute running
15:08:20 <mptacekx> mmedvede: not good indeed
15:08:57 <asselin> only I have is a request to review https://review.openstack.org/#/c/263021/
15:09:03 <lennyb> mptacekx, 'InternalError: Nova requires QEMU version 2.1.0 or greater.'  :)
15:09:53 <lennyb> mptacekx, thanks.
15:10:09 <lennyb> thanks asselin.
15:11:05 <lennyb> anything else on those issues?
15:11:41 <mptacekx> lennyb: we dont face that, due to nfv issues being tested we are using newer qemu 2.5+
15:13:09 <lennyb> anything else to discuss?
15:14:07 <mptacekx> we're currently fighting with 2 issues: 1) local.conf handling change when https://review.openstack.org/#/c/326585/ got merged (need to adapt on that) 2) race condition on setuptools with https://bugs.launchpad.net/devstack/+bug/1667545 - WA mentioned inside that ticket with pinning on tempest currently works well
15:14:07 <openstack> Launchpad bug 1667545 in devstack "install_tempest failed at uninstalling appdirs 1.4.1" [Undecided,In progress]
15:15:49 <mmedvede> For some reason our CI (IBM PowerKVM) was not affected by the appdirs bug
15:16:23 <lennyb> thanks mptaceks, we are facing this issue as well
15:16:48 <mmedvede> we do not pin tempest, and we use rather standard dib build process similar to upstream infra
15:16:52 <mptacekx> mmedvede: also upstream jenkins was not affected by that, no idea why. just found small discrepancy in logs, failing logs are building cmd2 wheels, working upstream not
15:17:47 <mptacekx> mmedvede: can you please share the "standard dib build process way" ?
15:18:33 <mptacekx> mmedvede: how to prevent "bad" setuptools to be deployed on system w/o patching tempest ?
15:19:07 <mmedvede> mptacekx: by "standard" I mean the one that infra team uses for their ubuntu images, they have a bunch of dib elements. We mostly copied that
15:20:27 <mmedvede> mptacekx: I can share more details if you're interested after the meeting. But it could be too involved just to fix appdirs problem
15:21:23 <mptacekx> mmedvede: thanks, I would appreciate that. maybe link to the relevant part of elements from project-config repo or somewhere would be great
15:21:24 <lennyb> mptacekx, IMHO the bug should be fixed, since workarounds are not welcome
15:22:06 <mptacekx> lennyb: thanks, I would cross-check and remove the WA. Out of curiosity still interested in the "elements way"
15:22:51 <mmedvede> mptacekx: that's the problem, I do not know which part is relevant :) Infra-specific elements are here https://github.com/openstack-infra/project-config/tree/master/nodepool/elements
15:23:23 <mmedvede> and they also have part of image update in puppet, which is whole another level
15:23:39 <mmedvede> so just for working around appdir problem, it is not worth it
15:24:48 <mptacekx> mmedvede: IMHO newer setuptools is not installed because of something on cloud-image but beacuse of it's availability in repo and by using pip -U install setuptools and not restricted in requirements like for tempest it simply comes up
15:25:56 <mmedvede> mptacekx: fwiw, here is their image update log: http://nb01.openstack.org/dib.ubuntu-xenial.log
15:26:00 <mptacekx> mmedvede: I also have some indications that wa is now not needed (also from lennyb few rows back), maybe it's simply gone
15:27:25 <mptacekx> mmedved: thanks, there are installing setuptools-34.2.0-py2.py3-none-any.whl which should be also wrong as it's from 34.x.x line, no idea why they were not impacted
15:29:49 <lennyb> I have a nodepool question. is there a way to save /var/log/nodepool/image/image.log unique for each image? we have 3 images and logs are overqritten
15:30:32 <asselin> lennyb, yes there is
15:30:51 <asselin> there's a script that will do that...let me look it up
15:31:35 <asselin> #link nodepool log config tool http://git.openstack.org/cgit/openstack-infra/system-config/tree/tools/nodepool_log_config.py
15:32:36 <asselin> didn't make it to project-config....I don't remember why, but I there was a reason
15:32:47 <lennyb> asselin, thanks, I will check it
15:33:34 <asselin> ok yeah...the script is configured to look at project-config by default: http://git.openstack.org/cgit/openstack-infra/system-config/tree/tools/nodepool_log_config.py#n33
15:33:35 <lennyb> asselin, do I need to run it once ?
15:34:09 <asselin> yes
15:34:32 <lennyb> asselin, thanks, I will try it.
15:34:57 <asselin> I don't recallthe details, but it does work and very useful to have each image in their own log file.
15:35:31 <lennyb> I wander why it's not the default behaviour
15:36:29 <asselin> ++ probably waiting for someone to propose the patch.. I remember discussing this with jeblair and mordred a long time ago.
15:36:57 <mmedvede> I think because it would require more complexity from nodepool code, more code. This way it is just logging config, left as an exercise to the user.
15:37:36 <asselin> anyway, the script is simple enough, works, and new images aren't added frequently
15:37:57 <lennyb> thanks. any other issues to discuss?
15:39:52 <lennyb> in that case, thank you all for coming. I will end the meeting now
15:42:31 <lennyb> asselin, mmedvede, mptacekx thanks for showing up.
15:43:13 <lennyb> #endmeeting
15:44:34 <mmedvede> lennyb: thanks
15:44:55 <mmedvede> bot is asleep? :)
15:45:52 <lennyb> mmedvede, I've saw him flirting with a bot-girl :), probably got lucky
15:47:47 <lennyb> #endmeeting
15:50:59 <fungi> lennyb: your nick has changed since you started the meeting
15:51:14 <fungi> lennyb: try `/nick lenyb_` and then #endmeeting again
15:51:20 <fungi> er, lennyb_
15:51:35 <mmedvede> thanks fungi
15:51:41 <lennyb_> #endmeeting