16:14:58 <mnaser> #startmeeting openstack_ansible_meeting
16:14:58 <openstack> Meeting started Tue Aug 28 16:14:58 2018 UTC and is due to finish in 60 minutes.  The chair is mnaser. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:14:59 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:15:01 <d34dh0r53> o/
16:15:03 <openstack> The meeting name has been set to 'openstack_ansible_meeting'
16:15:05 <mnaser> #topic rollcall
16:15:14 <evrardjp> my bad for the timing
16:15:14 <mnaser> o/
16:15:17 <evrardjp> o/
16:15:23 <mnaser> evrardjp: ya no worries
16:15:25 <hwoarang> o/
16:15:35 <guilhermesp> o/
16:15:40 <mnaser> so this is a bit of an impromptu meeting but better catch up quick rather than drop it entirely
16:15:46 <spotz> o/
16:15:56 <mnaser> #topic Last week highlights
16:16:02 <mnaser> so in catcvhing up with those
16:16:13 <mnaser> i believe jrosser finished a lot of the work on bionic
16:16:20 <mnaser> i saw a patch pushing jobs to voting
16:16:32 <hwoarang> great news
16:16:50 <mnaser> and the backport here to stable/rocky
16:16:51 <mnaser> https://review.openstack.org/594927
16:17:00 <mnaser> oh shucks
16:17:03 <mnaser> https://review.openstack.org/594926 is in merge conflict
16:17:21 <mnaser> i cant rebase, and jrosser is out for the week
16:17:30 <mnaser> be cool if someone could take care of rebasing that at some point
16:18:02 <hwoarang> i can look into that later this week
16:18:15 <mnaser> hwoarang: cool, thank you!
16:18:27 <mnaser> centos jobs are mostly in, they're just about to merge soon
16:18:32 <mnaser> so centos support is in
16:18:34 <mnaser> related to that however
16:18:47 <mnaser> i would ask for eyes on https://review.openstack.org/#/q/topic:rocky-deploy+(status:open+OR+status:merged) if possible when anyone has time
16:18:51 <hwoarang> what jobs? distro jobs?
16:18:56 <mnaser> i deployed a new region with osa and those were all the failing things
16:19:08 <mnaser> hwoarang: right now, aio metal and basekit non-distro
16:19:19 <mnaser> distro is wip, guilhermesp was fixing up those last little things left
16:19:19 <hwoarang> ah great
16:19:47 <mnaser> the wip patch to track is https://review.openstack.org/#/c/593360/
16:19:53 <guilhermesp> anything on my side to finish centos stuff mnaser ?
16:20:17 <mnaser> guilhermesp: not sure, maybe throw a recheck on that patch and see where we stand? i know we fixed the rdo related stuff
16:20:21 <mnaser> with the different path for api-paste
16:20:30 <mnaser> so that should get us past neutron
16:20:47 <odyssey4me> yep, that need a backport IIRC
16:20:48 <Taseer> evrardjp: do you still control what goes into Blazar ansible role ?
16:21:25 <mnaser> odyssey4me: yep ++
16:21:37 <odyssey4me> guilhermesp: a backport for https://review.openstack.org/595796 would be good
16:21:47 <mnaser> ok so
16:21:55 <mnaser> hwoarang: affecting you, we dropped leap15 jobs
16:21:59 <mnaser> they were just always failing
16:22:03 <mnaser> because of galera and co
16:22:05 <evrardjp> Taseer: what do you mean control? I do not control anything :)
16:22:07 <mnaser> any progress on your side on that?
16:22:41 <Taseer> evrardjp: since the functional tests are not passing, I still see the commits increasing everyday
16:23:20 <Taseer> someone is letting them through
16:23:34 <evrardjp> I think we should discuss this after the meeting Taseer or during the open discussion
16:23:50 <mnaser> evrardjp, Taseer agreed
16:23:59 <Taseer> evrardjp: sorry didnt know about that
16:24:01 <hwoarang> mnaser: yeah ok :(
16:24:12 <hwoarang> sad story
16:24:27 <mnaser> hwoarang: yeah looks like a mix of unfortuntae things
16:24:35 <mnaser> we're a revert away from bringing it back, anytime we get those stuff figured out
16:24:36 <hwoarang> maybe in the future
16:24:40 <hwoarang> ok
16:24:58 <mnaser> rc2 is already out, right evrardjp ?
16:25:54 <odyssey4me> I think there might need to be another RC once all the upstream projects have released. Our roles will need a bump too.
16:26:24 <mnaser> odyssey4me: i agree, or maybe our first release will make sure that the tags are all the released versions?
16:26:38 <mnaser> maybe instead of a commit hash we can put in the actual tag :D
16:26:57 <odyssey4me> That'll be up to evrardjp's release tool capabilities.
16:27:24 <evrardjp> odyssey4me: fun story I removed that feature :p I will rewrite it.
16:27:27 <evrardjp> or revert
16:27:43 <mnaser> ahaha
16:27:49 <mnaser> okay so next order
16:27:49 <evrardjp> oh you mean for OpenStack Services?
16:27:52 <mnaser> yes ^
16:27:58 <mnaser> like defaults/openstack_services.yml
16:28:05 <evrardjp> I thought we wanted to discuss this at PTG.
16:28:11 <evrardjp> I am fine with doing it right away
16:28:12 <odyssey4me> It might be nice to add another tracker for tag versions to allow for each service - then we always just release tags, not SHA's.
16:28:17 <odyssey4me> But meh, either way.
16:28:37 <evrardjp> odyssey4me: could you clarify?
16:28:58 <mnaser> oh well we can just do that for the initial release
16:29:04 <mnaser> because its cool and pretty
16:29:11 <mnaser> i support tracking stable branches for our occaisonal releases
16:29:21 <odyssey4me> ie we have something like nova_git_constraints: >12,<13 - then it will find the latest tag within that constraint set and use it
16:29:34 <evrardjp> to answer you mnaser yes, rc2 is out :)
16:29:55 <odyssey4me> or nova_release_constraint - but you get the picture
16:29:55 <evrardjp> I will indeed create an rc3 when all the projects have tagged final, which should be this thursday
16:30:08 <odyssey4me> that might pave the way to using the upstream wheels when they're available
16:30:11 <evrardjp> mnaser: it's all faster to deploy, as shallow cloning would work
16:30:28 <mnaser> evrardjp: sounds good to me!
16:30:45 <evrardjp> odyssey4me: I see
16:30:56 <mnaser> cool, and the usual weekly reminder
16:30:59 <mnaser> https://etherpad.openstack.org/p/osa-bugtriage
16:31:00 <evrardjp> I have a few questions, we should have this after the meeting
16:31:13 <mnaser> FWIW: we have time for a our onboarding session
16:31:19 <evrardjp> my comemnt was for odyssey4me
16:31:21 <mnaser> and project update
16:31:37 <evrardjp> mnaser: you mean in Berlin?
16:31:40 <mnaser> yes
16:31:47 <evrardjp> great
16:31:50 <mnaser> i would like to express that if it's important for you to be listed as a speaker to make it to the summit because of $company reasons
16:31:53 <mnaser> please let me know
16:32:01 <evrardjp> mnaser: ;)
16:32:11 <mnaser> as in, if that's what makes or breaks your trip, let me know and i can figure out something :)
16:32:20 <mnaser> (in pm or my email if need-be)
16:32:33 <mnaser> and we still have our lovely ptg etherpad here -- https://etherpad.openstack.org/p/osa-stein-ptg
16:32:43 <openstackgerrit> Byron McCollum proposed openstack/openstack-ansible-nspawn_container_create stable/rocky: Update UPPER_CONSTRAINTS_FILE for stable/rocky  https://review.openstack.org/590874
16:32:46 <mnaser> so, add yourself if you haven't, and propose subjects if you'd like
16:32:56 <mnaser> i would like to ask if anyone has some good AV equipment to help out set it up
16:33:15 <mnaser> jrosser and other cores might not be there and i want to try and make that option available
16:33:20 <hwoarang> i wont be there either
16:33:29 <hwoarang> i would also love the idea of joining remotely
16:33:32 <mnaser> so i'll try to work with OSF, but if anyone knows a thing or two about setting AV stuff up
16:33:36 <mnaser> i would love some help
16:33:39 <hwoarang> so thank you for trying to arranve that
16:33:46 <hwoarang> *arrange that
16:33:50 <mnaser> no problem :D
16:34:16 <mnaser> uhoh, etherpad not loading
16:34:19 <mnaser> oh fiou
16:34:20 <mnaser> it loaded
16:34:32 <spotz> mnaser: You killed the etherpad! How could you!!
16:34:37 <mnaser> i blame wifi
16:34:49 <mnaser> anyways, anything else before we move qucikly to the 5 little bugs we got?
16:35:00 <mnaser> and then maybe some open discussion time?
16:35:14 <spotz> mnaser: Not I, I need to sneak pack without the dog seeing:)
16:35:30 <mnaser> lols
16:35:44 <mnaser> i think they're having PUPPIES at oss
16:36:15 <spotz> Yeah, but luckily no Dals so I'm safe:)
16:36:33 <mnaser> ok
16:36:34 <mnaser> ETIMEOUT
16:36:41 <mnaser> next up
16:36:42 <mnaser> #topic bug triage
16:36:47 <mnaser> #link https://bugs.launchpad.net/openstack-ansible/+bug/1788837
16:36:47 <openstack> Launchpad bug 1788837 in openstack-ansible "glance db sync always fails during upgrade due to OsloDBDeprecationWarning: EngineFacade is deprecated" [Undecided,New]
16:37:14 <evrardjp> I don't know for this one. The dbsync should fail on deploys I guess, I haven't seen that
16:37:17 <evrardjp> I am missing info
16:37:27 <mnaser> it still hasn't loaded for me
16:37:27 <mnaser> yay
16:38:17 <mnaser> ok good loaded
16:38:26 <openstackgerrit> Merged openstack/openstack-ansible stable/ocata: Update all SHAs for 15.1.27  https://review.openstack.org/595358
16:38:41 <mnaser> i agree with evrardjp i think this failure is from something else
16:38:49 <mnaser> oh
16:38:52 <mnaser> "db_sync always fails during Q upgrade, see logs below"
16:39:10 <mnaser> so this could be upgrade related
16:39:18 <mnaser> lets ask for /var/log/glance ?
16:39:34 <mnaser> as is it is incomplete, because that isn't something we see in our agtes
16:39:35 <mnaser> gates
16:39:38 <evrardjp> indeed
16:39:40 <mnaser> the deprecation is not a reason for the job to fail
16:40:04 <evrardjp> I saw it was for upgrades, but it's still not enough info, as db sync happens with the upper branch code eventually
16:40:09 <odyssey4me> yeah, that seems informational
16:40:21 <evrardjp> I agree with odyssey4me
16:40:48 <mnaser> can someone else set the status
16:40:50 <mnaser> my latency is really bad
16:40:55 <mnaser> i say.. incomplete?
16:40:55 <odyssey4me> It looks like there's a fail - but the tool doesn't output why it fails.
16:41:31 <evrardjp> odyssey4me: well or the stderr is a real error we think it's just a warning ;)
16:41:37 <odyssey4me> It'd perhaps be useful to understand a few things - like what backend is being used in the test and what the source pike OSA version is.
16:41:47 <evrardjp> we definitely lack some dimension into this
16:42:03 <odyssey4me> evrardjp: if that is the case, then I would expect to see a lot more screaming going on
16:42:11 <evrardjp> odyssey4me: that would be the most valuable input: I have the impression backend can change stuff in here
16:42:23 <evrardjp> odyssey4me: )
16:42:41 <evrardjp> odyssey4me: except if that's a not well used backend, or db wasn't cleaned up for a while
16:42:47 <evrardjp> I think it's worth asking all those questions
16:43:20 <mnaser> i'd apprecaite if someone can help adding those as comments
16:43:37 <mnaser> 5s latency is barely ok for irc
16:43:37 <mnaser> lol
16:44:40 <mnaser> can we add those commetns and set to incomplete ?
16:45:37 <evrardjp> yes
16:46:25 <mnaser> ok great
16:46:33 <mnaser> #link https://bugs.launchpad.net/openstack-ansible/+bug/1788646
16:46:33 <openstack> Launchpad bug 1788646 in openstack-ansible "zfs backing fails on host with extra disk" [Undecided,New]
16:46:56 <openstackgerrit> Merged openstack/openstack-ansible-os_neutron stable/rocky: Drop SELinux support for CentOS 7  https://review.openstack.org/596572
16:47:01 <mnaser> hmm
16:48:05 <odyssey4me> there was a patch to address something like this
16:48:13 <mnaser> looks like it's probably trying to run that in a non-rax vm?
16:48:13 <evrardjp> yeah IIRC
16:48:31 <odyssey4me> https://review.openstack.org/591304
16:49:01 * mnaser falls asleep as patchset loads
16:49:08 <evrardjp> jrosser: ?
16:49:16 <odyssey4me> not sure if it's actually targeting that bug though
16:49:27 <odyssey4me> jrosser is unfortunately on holiday this week
16:49:34 <odyssey4me> well, unfortunately for us - fortunately for hiim
16:49:41 <mnaser> hahaha
16:49:48 <mnaser> how about we leave it's status not updated
16:50:08 <hwoarang> yeah i think the patch is unrelated. we prob need to look at why there is no partition on /dev/xvde
16:50:22 <hwoarang> but seems jrosser is looking into it anyway
16:50:34 <hwoarang> better not touch it
16:51:12 <odyssey4me> perhaps assign to him
16:52:24 <mnaser> so given its jrosser and im pretty confident this is probahbly athing
16:52:34 <mnaser> confirmed/medium/assign to jrosser ?
16:53:22 <mnaser> (can someone do that please?)
16:53:24 <mnaser> plane wifi not happy
16:53:27 <evrardjp> k
16:53:34 <mnaser> thank you so much for help :x
16:53:44 <mnaser> #link https://bugs.launchpad.net/openstack-ansible/+bug/1788272
16:53:44 <openstack> Launchpad bug 1788272 in openstack-ansible "Setup-hosts.yml fails to find /boot/config-4.15.0-32-generic" [Undecided,New]
16:54:51 <evrardjp> this one is funny - a backport of things for someone that wants to run 16.04 embedded in 18.04 with lxc1
16:55:41 <evrardjp> the backport was, I guess, not clean
16:55:41 <mnaser> i dont know enough about the ubuntu side of things
16:55:44 <mnaser> but sounds liek a package is missing?
16:55:56 <hwoarang> tricky
16:56:11 <hwoarang> well the kernel of 16.04 is missing in 18.04 ;p
16:56:53 <evrardjp> I can't say -- this is unclear what's really done, at least for me. Can someone translate that for me?
16:57:01 <openstackgerrit> Merged openstack/openstack-ansible stable/rocky: Bump upstream Rocky repos  https://review.openstack.org/596679
16:57:24 <hwoarang> yes i guess they have 18.04 hosts and 16.04 containers
16:57:35 <hwoarang> not sure if this mixing is supposed to work
16:57:49 <mnaser> i wouldn't call this a supported method of deployment.
16:57:59 <evrardjp> well I am confused I am not sure that's the real problem, I think it's with separate versions of things
16:58:01 <mnaser> i dont think we have enough resources to start supporting that type of deployment
16:58:06 <evrardjp> mnaser: I don't think that's supported
16:58:09 <evrardjp> :p
16:58:13 <evrardjp> either
16:58:15 <mnaser> (looks like we're gonna get a bit over time but we started later)
16:58:15 <hwoarang> apparently we have code in openstack_hosts to grep the kernel config file and this fails if there is a mismatch between container and host
16:58:21 <mnaser> so hope we can borrow 15 minutes more
16:59:50 <mnaser> do we wanna come to terms that that might just be an invalid bug?
17:00:27 <hwoarang> maybe wontfix?
17:00:46 <mnaser> i agree
17:00:54 <hwoarang> i mean the bug is valid but the environment is not something we can support so maybe wontfix is better to state 'dont do that'
17:00:57 <mnaser> if no one opposes maybe hwoarang can help set it to that state
17:01:16 <hwoarang> done
17:01:24 <mnaser> merci
17:01:24 <mnaser> v
17:01:25 <mnaser> #link https://bugs.launchpad.net/openstack-ansible/+bug/1788269
17:01:25 <openstack> Launchpad bug 1788269 in openstack-ansible "ceph-ansible - ceph-config task takes long time." [Undecided,New]
17:01:51 <mnaser> first of all
17:01:58 <mnaser> user somehow has ceph-ansible deployed via.... rpm?
17:02:07 <mnaser> "Tripleo deployment. Queens, container bases."
17:02:08 <mnaser> lol
17:02:16 <mnaser> can someone change that bug to tripleo
17:02:47 <evrardjp> that was a fun bug too :)
17:03:02 <evrardjp> I am changing it
17:03:07 <mnaser> ahaha
17:03:10 <mnaser> thank you
17:03:23 <mnaser> #link https://bugs.launchpad.net/openstack-ansible/+bug/1638683
17:03:23 <openstack> Launchpad bug 1638683 in openstack-ansible "LXC container restart not staged and too slow" [Undecided,New]
17:03:48 <evrardjp> mnaser: tbh I wanted to let you see it, and consume your bandwidth :p
17:03:50 <evrardjp> hahaha
17:04:00 <evrardjp> that one is a confirmed wishlist I'd say
17:04:03 <mnaser> its more like consuming my pateince
17:04:03 <odyssey4me> looks like a wishlist item
17:04:08 <mnaser> with 5s latency lmao
17:04:13 <mnaser> okay cool, yeah, i agree on that being wishlist
17:04:21 <odyssey4me> also, it's almost 2 years old
17:04:32 <mnaser> oh i think someone recently reset it to new
17:04:37 <mnaser> thats why
17:04:45 <mnaser> #link https://bugs.launchpad.net/openstack-ansible/+bug/1782388
17:04:45 <openstack> Launchpad bug 1782388 in openstack-ansible "Installing Multipath But Not Enabling In Nova Causes Volume Attachment Failures" [Undecided,New]
17:04:47 <mnaser> this is trailing from last time
17:04:48 <odyssey4me> yes, it appears that bjoern stopped working on it or something
17:04:58 <mnaser> i'm not sure when/what we're gonna do about that
17:05:08 <evrardjp> leave it open
17:05:09 <mnaser> it feels like a nova/cinder thing that should be figured out imho
17:05:13 <mnaser> ok by me
17:05:25 <evrardjp> that's what we said last week until more time
17:05:33 <mnaser> i know we're over by 5 but we started late.  any things we want to have for some open discussion?
17:05:57 <evrardjp> nothing, I am refactoring the release tooling and I'd like some reviews on my pending work
17:06:16 <mnaser> feel free to ping me on that
17:06:23 <mnaser> i'm at OSS this week so expect me to be in and out
17:06:28 <mnaser> but highlight me anytime and i have irccloud
17:06:45 <evrardjp> https://review.openstack.org/#/q/project:%255Eopenstack/openstack-ansible.*+owner:%22Jean-Philippe+Evrard+%253Cjean-philippe%2540evrard.me%253E%22+is:open+is:mergeable
17:07:16 <mnaser> until then
17:07:19 <mnaser> anything else? :)
17:07:29 <openstackgerrit> Valentin Boucher proposed openstack/openstack-ansible-os_nova master: Compute: Enable vGPU in Nova if exist on the host  https://review.openstack.org/597183
17:07:52 <evrardjp> none
17:08:10 <dmsimard> ARA 0.16.0rc1 tagged, changelog: https://github.com/openstack/ara/releases/tag/0.16.0.0rc1
17:08:10 <kaiokmo> mnaser, can I add something real quick?
17:08:21 <evrardjp> odyssey4me: should we discuss the other way to track things after the meeting?
17:08:25 <mnaser> kaiokmo: sure
17:08:27 <evrardjp> dmsimard: thanks : )
17:08:32 <mnaser> thanks for the heads up
17:08:51 <evrardjp> dmsimard: did someone bump it for you already? We should if it's not done yet
17:09:07 <mnaser> if its an rc1 we probably don't want to do that yet i guess
17:09:16 <kaiokmo> I'm still working on monasca support for opensuse and centos. on opensuse, I'm blocked by a issue related to influxdb https://github.com/influxdata/influxdb/issues/9623
17:09:22 <dmsimard> evrardjp: bump it where ?
17:09:44 <openstackgerrit> Valentin Boucher proposed openstack/openstack-ansible-os_nova master: Compute: Enable vGPU in Nova if exist on the host  https://review.openstack.org/597183
17:10:06 <kaiokmo> so I will probably bring back the centos work https://review.openstack.org/#/c/588545/ until influx is resolved
17:10:08 <dmsimard> mnaser: I always do one or two RCs for new versions to avoid screwing up :)
17:10:31 <mnaser> kaiokmo: that sounds good, if you need any reviews or an extra pair of eyes, we're all here
17:10:41 <mnaser> dmsimard: you have a big list of consumers now ;)
17:11:04 <hwoarang> out of curiosity is our release process documented somewhere?
17:11:11 <hwoarang> like what scripts to run, when etc?
17:11:14 <kaiokmo> jrosser got it working on bionic, so there's only opensuse and centos to go
17:11:38 <kaiokmo> thanks mnaser
17:11:50 <mnaser> no problem :D
17:12:01 <dmsimard> mnaser: yeah no kidding, a bad release could literally break thousands of jobs :/
17:12:13 <odyssey4me> hwoarang: yes it is, except they're out of date because evrardjp is rewriting them all in secret ;)
17:12:14 <dmsimard> (and that's just openstack, not talking about the other consumers eh)
17:12:31 <hwoarang> oh ok
17:12:49 <evrardjp> odyssey4me: it's always on github.
17:13:00 <evrardjp> no secret, just nobody cares
17:13:12 <odyssey4me> https://docs.openstack.org/openstack-ansible/latest/contributor/periodic-work.html#releasing
17:13:16 <mnaser> given that we're back to normal discussion
17:13:27 <mnaser> i think it's okay to end the meeting at this point
17:13:27 <odyssey4me> evrardjp: nobody knows
17:13:51 <evrardjp> odyssey4me: that was v1: https://github.com/evrardjp/osa_toolkit
17:14:03 <mnaser> #endmeeting