14:00:22 <rosmaita> #startmeeting glance
14:00:23 <openstack> Meeting started Thu Feb 15 14:00:22 2018 UTC and is due to finish in 60 minutes.  The chair is rosmaita. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:24 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:26 <openstack> The meeting name has been set to 'glance'
14:00:32 <abhishekk> o/
14:00:33 <rosmaita> #topic roll call
14:00:46 <rosmaita> hi abhishekk, hope you are feeling better (or at least decent)
14:00:59 <jokke_> o/
14:01:02 <abhishekk> rosmaita, yes better
14:01:08 <McClymontS> o/
14:01:14 <rosmaita> hi scott
14:01:19 <McClymontS> hey all
14:01:20 <rosmaita> and erno
14:01:42 <rosmaita> i'll get started right away
14:02:01 <rosmaita> my plan is quick meeting, and then we can re-convene in glance channel to figure out final stuff for RC-2
14:02:06 <rosmaita> #topic updates
14:02:20 <rosmaita> point releases for ocata (14.0.1) and pike (15.0.1)
14:02:28 <rosmaita> those include the eventlet fix
14:02:50 <rosmaita> in other news, we lost diverse-affiliation tag again: "Glance no longer has diverse affiliation, 85% of core reviews  coming from Red Hat and Verizon (55% for Red Hat alone)"
14:03:02 <rosmaita> not much we can do about that
14:03:14 <rosmaita> but it does impact the postgres situation
14:03:27 <rosmaita> in that no one working on the project actually uses it
14:03:36 <rosmaita> but i will say more about that later
14:03:59 <rosmaita> ok, that's all for updates, let's talk about rc-2
14:04:04 <rosmaita> #topic RC-2
14:04:26 <rosmaita> oh yeah, i forgot to post the agenda link
14:04:29 <rosmaita> #link https://etherpad.openstack.org/p/glance-team-meeting-agenda
14:04:54 <rosmaita> ok, what we're trying to get done in RC-2 are the bugs for interoperable image import
14:05:04 <rosmaita> and some postgres fixes
14:05:18 <rosmaita> we're tracking the progress here:
14:05:28 <rosmaita> #link https://etherpad.openstack.org/p/glance-queens-rc1-patches
14:06:07 <rosmaita> like i said, let's go into the details after the meeting
14:06:15 <abhishekk> ok
14:06:18 <rosmaita> it would be good to get RC-2 released today
14:06:46 <rosmaita> but there is always tomorrow -- anyway, we can discuss later
14:06:51 <rosmaita> #topic RC-3
14:06:56 <rosmaita> there will be an RC-3
14:07:08 <rosmaita> what i'd like is to get the postgres situation figured out
14:07:25 <rosmaita> the situation is that there may be problems migrating an existing postgres database
14:07:49 <rosmaita> i filed a bug to track that, gimme a minute to find the link
14:08:28 <rosmaita> #link https://bugs.launchpad.net/glance/+bug/1749466
14:08:29 <openstack> Launchpad bug 1749466 in Glance "postgresql db migration broken in queens" [Critical,Triaged] - Assigned to Ameed Ashour (ameeda)
14:08:39 <rosmaita> and i see someone has already filed another one
14:08:51 <rosmaita> #link https://bugs.launchpad.net/glance/+bug/1749640
14:08:51 <openstack> Launchpad bug 1749640 in Glance "db sync fails for mysql while adding triggers" [High,In progress] - Assigned to Abhishek Kekane (abhishek-kekane)
14:09:25 <rosmaita> let's discuss a strategy for dealing with this after the meeting after discussing rc-2
14:09:36 <abhishekk> ok
14:09:50 <ShilpaD> ok
14:09:55 <rosmaita> i'd like to shoot for tuesday for RC-3
14:10:09 <abhishekk> sounds good
14:10:19 <rosmaita> the final release candidate needs to be done late next week
14:10:31 <jokke_> Monday would be great, Tue night GMT pretty much the latest or we'll be screwed
14:10:39 <jokke_> we need to release at Thu IIRC
14:10:51 <rosmaita> ok, good point
14:11:05 <rosmaita> we will have to timebox and aim for Monday
14:11:51 <rosmaita> we can discuss the parameters of the timebox after the meeting
14:12:03 <rosmaita> i don't want people going insane over the weekend
14:12:04 <abhishekk> ok'
14:12:10 <jokke_> ++
14:12:28 <rosmaita> ok, next item
14:12:35 <rosmaita> #topic the uWSGI situation
14:12:42 <rosmaita> this impacts the next topic, too
14:12:54 <rosmaita> ok, so our recommendation has been don't use glance with uWSGI
14:13:10 <rosmaita> we are going to continue that recommendation in Queens
14:13:16 <McClymontS> is there any more info on this one?
14:13:28 <jokke_> unfortunately as it was pointed out in #os-glance while ago we do not officially indicate that anywhere
14:13:54 <rosmaita> i think it says that in the running glance under httpd docs
14:13:58 <rosmaita> i will check
14:14:21 <rosmaita> i will also submit a patch removing our completion of uwsgi support from the community goals
14:14:25 <jokke_> IIRC that doc just eplains how to do it ... we never made the change to state there that it should not be done
14:14:42 <rosmaita> ok, will have to check
14:15:08 <rosmaita> anyway, the impact is that if you want to run tasks in devstack ...
14:15:21 <rosmaita> you need to set WSGI_MODE=mod_wsgi
14:15:25 <rosmaita> in your local.conf
14:15:35 <rosmaita> that starts glance in systemd
14:15:43 <rosmaita> but uses the "regular" glance wsgi server
14:15:56 <abhishekk> ok
14:16:02 <jokke_> is it Glance only or does it push all the services under systemd?
14:16:14 <rosmaita> well, it's a bit weird
14:16:18 <esberglu> Is there a glance specific solution?
14:16:28 <rosmaita> yes, do not use uwsgi
14:16:48 <jokke_> esberglu: as in do not deploy glance under Apache
14:16:51 <rosmaita> so what happens when you do that is that glance and keystone run slightly differently
14:16:52 <esberglu> Yeah I thought that WSGI_MODE affected more than glance, not super familiar though
14:17:21 <rosmaita> yeah, i am focused on glance, so i am not 100% sure what happens to the other services
14:17:29 <rosmaita> this is what i have observed:
14:17:30 <McClymontS> I am not 100% sure but I suspect that wsgi mode is more global as well
14:17:44 <rosmaita> (1) in default devstack:
14:17:44 <jokke_> rosmaita: ok, cool ... well that's good to know anyways ... makes life much easier for testing things
14:18:06 <jokke_> sry, go ahead
14:18:07 <rosmaita> everything runs under systemd AND in a uwsgi "container"
14:18:30 <rosmaita> (2) when you use WSGI_MODE=mod_wsgi
14:18:50 <rosmaita> everything except keystone (and maybe one or two other things) runs under systemd
14:18:54 <rosmaita> including glance
14:19:24 <rosmaita> the diff is that the glance systemd starts /path/glance --conf /etc/glance-api.conf
14:19:28 <rosmaita> something like that
14:19:36 <rosmaita> keystone is run out of apache
14:19:38 <jokke_> so we need to figure out how to change all our test jobs to do that as well
14:20:07 <rosmaita> if you do systemctrl status apache2.service you can see what httpd is running
14:20:40 <rosmaita> well, the glance functional tests don't use devstack
14:20:51 <rosmaita> but yeah, we do need to do something
14:21:11 <rosmaita> the mod_wsgi support is supposed to be pulled from devstack
14:21:39 <rosmaita> i will put up a patch to introduce a new var to decide whether glance runs under uwsgi or not
14:21:47 <jokke_> yeah, what I mean is that we need to have tempest jobs for glance doing that as well
14:22:14 <rosmaita> yeah, apparently tempest doesn't cover any tasks stuff, or we'd be seeing problems
14:22:36 <rosmaita> so, the key point for the next 2 weeks is
14:22:54 <rosmaita> put the WSGI_MODE thing in your local.conf so you can test out image import
14:23:50 <rosmaita> ok, so esberglu i have kind of put a damper on your next item
14:24:10 <rosmaita> mtreinish has been working on some fixes to get glance working better with uwsgi
14:24:24 <rosmaita> but the key thing is that we absolutely need tasks for image import
14:24:48 <rosmaita> and the uwsgi upload fixes have not been tested at scale
14:25:15 <rosmaita> so i think we need to declare the uwsgi stuff experimental
14:25:31 <abhishekk> agree
14:25:35 <rosmaita> and if people really want it, they can propose some specs
14:26:01 <rosmaita> and we can discuss what would really be needed to get it done correctly
14:26:43 <rosmaita> #topic pike backport for image-create
14:26:49 <rosmaita> esberglu that's you
14:26:58 <jokke_> unfortunately I totally agree ... running glance in uwsgi container has almost no priority to atm. specially looking our resources and backlog
14:27:15 <jokke_> "to me atm."
14:27:26 <esberglu> I was looking at this bug
14:27:27 <esberglu> https://bugs.launchpad.net/glance/+bug/1703856
14:27:28 <openstack> Launchpad bug 1703856 in Glance "502 Bad gateway error on image-create" [High,Fix released] - Assigned to Matthew Treinish (treinish)
14:27:45 <esberglu> And this backport
14:27:46 <esberglu> https://review.openstack.org/#/c/525994/
14:28:08 <esberglu> But if the recommendation is to not use uwsgi I'm guessing that won't get in?
14:30:03 <rosmaita> jokke_ you are still release czar atm
14:30:18 <jokke_> esberglu: I'm very reluctant to approve that backport and claim that "Now it's ok to run Glance Pike in uwsgi"
14:31:02 <rosmaita> found the doc: https://docs.openstack.org/releasenotes/glance/pike.html#known-issues
14:31:12 <jokke_> that's just really really bad idea and backporting that to Pike just to get devstack working out of box is kind of those things that has priority sowhere so far I can't see it ;)
14:31:46 <rosmaita> agree
14:32:13 <rosmaita> i will make sure that advice is repeated in the queens release notes
14:32:31 <rosmaita> and put up a patch to the admin docs
14:33:07 <rosmaita> which just say, by the way, *if* you want to run glance under httpd, we recommend uwsgi ... there's also a note saying that it may not work
14:33:07 <abhishekk> 1) Upstream reviews, Fix for glance deployment with postgresql
14:33:08 <abhishekk> 2) More upstream reviews, Fix release critical issues
14:33:08 <abhishekk> 3) None
14:33:16 <abhishekk> sorry ignore that
14:34:03 <jokke_> :)
14:34:04 <rosmaita> ok, guess we covered that
14:34:14 <jokke_> just one thing
14:34:17 <esberglu> Thanks for the info
14:34:17 <rosmaita> sorry esberglu but thank you for attending
14:34:20 <jokke_> as a bit of clarification
14:35:00 <jokke_> my biggest problem with that is that fixing such thing we as upstream community reinforce supporting such deployments and we really can't afford doing so atm.
14:35:32 <jokke_> it's not so much technical problem as it is principal/political
14:35:40 <McClymontS> agree
14:36:04 <rosmaita> yes, and a key aspect is that we must have more & better test coverage
14:36:04 <jokke_> so if you really need to have that patch in, carry it in your own deployment on your own risk that the next thing will break
14:36:19 <rosmaita> because it will entail separating out the task stuff into a separate service
14:36:22 <jokke_> the patch itself is not bad and it's in queens
14:36:34 <rosmaita> which as jokke_ says is not technically a problem
14:36:45 <rosmaita> but it has an enormous potential to affect stability
14:37:22 <rosmaita> so we will need QA resources if that is to happen
14:37:35 <rosmaita> and our community is too small atm
14:37:57 <rosmaita> ok, anything else?
14:37:58 <jokke_> I'm ready for next topic :)
14:38:01 <rosmaita> me too
14:38:14 <rosmaita> #topic PTG planning
14:38:23 <rosmaita> jokke_ this is you
14:38:38 <jokke_> Ok, so PTG is coming faster than anyone can see from the release rush!
14:39:22 <jokke_> which means I want to start nailing down our PTG planning at latest Monday
14:39:35 <rosmaita> i would like to propose that we combine the queens retrospective with a guiness brewery tour
14:39:43 <rosmaita> don't know whether the logistics on that are reasonable, though
14:39:55 <rosmaita> someone told me they have wi-fi at guiness
14:40:07 <jokke_> Please do have a look on the planning etherpad, make any additions you might want to bring up and mark your interest
14:40:14 <smcginnis> There's a nice pub area in there too. :)
14:40:48 <jokke_> rosmaita: my vote goes to Jamesons' ... it's better tour and we might need some stronger tools :P
14:41:02 <smcginnis> Hah
14:41:05 <rosmaita> jokke_ you are PTL for a reason!  good choice
14:41:24 <jokke_> So Wed morning that is! :P
14:41:48 <rosmaita> how early does it open?
14:41:55 <jokke_> I need to check
14:42:16 <jokke_> but I think we get Glance sessions well started if we lubricate it with Whiskey
14:42:21 <smcginnis> If Jameson hasn't changed much in the last 18 years, they had a nice pub area too.
14:42:38 <jokke_> smcginnis: oh they do and they still serve the best Irish Coffee ever
14:42:45 <rosmaita> ok, i will leave it up to you Dublin experts to pick the place
14:42:56 <abhishekk> exactly
14:43:01 <rosmaita> but seriously, we could do the retrospective there
14:43:07 <rosmaita> combine business & pleasure
14:43:13 <rosmaita> just a suggestion
14:43:13 <jokke_> I'll keep that in mind
14:43:16 <rosmaita> cool
14:43:21 <jokke_> put it into the etherpad :P
14:44:11 <jokke_> but that's all form me about that topic, just get your input in so I don't need to come out with my own agenda for the week
14:44:23 <jokke_> no-one will like that :D
14:44:36 <rosmaita> ok, so jokke_ what do you say that deadline for proposals is 23:59 utc tomorrow?
14:44:40 <jokke_> #deathByPowerpoint
14:44:49 <rosmaita> what deadline for getting votes onto the etherpad?
14:44:54 <jokke_> rosmaita: 0900 UTC Mon Morning
14:45:18 <jokke_> after that I start composing the schedule
14:45:32 <jokke_> not doing it over the weekend
14:45:35 <rosmaita> ok, i will include that on the priority email
14:45:47 <jokke_> tyty
14:46:06 <rosmaita> #topic open discussion
14:46:22 <rosmaita> anyone?
14:46:32 <jokke_> one quick one
14:47:05 <mbindlish> Hi, I am looking for a workaround of https://bugs.launchpad.net/glance/+bug/1703856. I have already applied patches https://review.openstack.org/#/c/493654/ (Glance) and https://review.openstack.org/492651 But still facing issue in my devstack Pike with bad gateway 502 error
14:47:07 <openstack> Launchpad bug 1703856 in Glance "502 Bad gateway error on image-create" [High,Fix released] - Assigned to Matthew Treinish (treinish)
14:47:18 <jokke_> request for all ... anyone in the audience who really really cares about stable releases
14:47:54 <jokke_> Please come forward ... our Stable-maint group basically consists of myself and rosmaita and that will get backporting stuff very difficult
14:48:05 <smcginnis> ++
14:49:08 <jokke_> for example 544801 is pending to find external reviewer as it's my patch, I technically shouldn't +2 it and Abhishek is not stable-core so Brian should not merge it in by himself
14:49:58 <jokke_> so we really could do some help there
14:50:24 <abhishekk> mbindlish, just go through the previous discussion which will help you to understand the situation of uwsgi
14:50:27 <rosmaita> mbindlish i don't have a workaround for that and we are in queens rc-time, don't think anyone here will be able to help
14:50:42 <rosmaita> also, what abhishekk said
14:50:43 <mbindlish> ok
14:51:41 <rosmaita> anything else?
14:52:04 <abhishekk> so should we shift to openstack-glance?
14:52:24 <rosmaita> yes, and i will try to make it quick because i know it is late for you
14:52:39 <rosmaita> ok, thanks everyone! don't forget jokke_ 's request about the PTG planning
14:52:43 <rosmaita> #endmeeting