14:00:22 <rosmaita> #startmeeting glance 14:00:23 <openstack> Meeting started Thu Feb 15 14:00:22 2018 UTC and is due to finish in 60 minutes. The chair is rosmaita. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:24 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:26 <openstack> The meeting name has been set to 'glance' 14:00:32 <abhishekk> o/ 14:00:33 <rosmaita> #topic roll call 14:00:46 <rosmaita> hi abhishekk, hope you are feeling better (or at least decent) 14:00:59 <jokke_> o/ 14:01:02 <abhishekk> rosmaita, yes better 14:01:08 <McClymontS> o/ 14:01:14 <rosmaita> hi scott 14:01:19 <McClymontS> hey all 14:01:20 <rosmaita> and erno 14:01:42 <rosmaita> i'll get started right away 14:02:01 <rosmaita> my plan is quick meeting, and then we can re-convene in glance channel to figure out final stuff for RC-2 14:02:06 <rosmaita> #topic updates 14:02:20 <rosmaita> point releases for ocata (14.0.1) and pike (15.0.1) 14:02:28 <rosmaita> those include the eventlet fix 14:02:50 <rosmaita> in other news, we lost diverse-affiliation tag again: "Glance no longer has diverse affiliation, 85% of core reviews coming from Red Hat and Verizon (55% for Red Hat alone)" 14:03:02 <rosmaita> not much we can do about that 14:03:14 <rosmaita> but it does impact the postgres situation 14:03:27 <rosmaita> in that no one working on the project actually uses it 14:03:36 <rosmaita> but i will say more about that later 14:03:59 <rosmaita> ok, that's all for updates, let's talk about rc-2 14:04:04 <rosmaita> #topic RC-2 14:04:26 <rosmaita> oh yeah, i forgot to post the agenda link 14:04:29 <rosmaita> #link https://etherpad.openstack.org/p/glance-team-meeting-agenda 14:04:54 <rosmaita> ok, what we're trying to get done in RC-2 are the bugs for interoperable image import 14:05:04 <rosmaita> and some postgres fixes 14:05:18 <rosmaita> we're tracking the progress here: 14:05:28 <rosmaita> #link https://etherpad.openstack.org/p/glance-queens-rc1-patches 14:06:07 <rosmaita> like i said, let's go into the details after the meeting 14:06:15 <abhishekk> ok 14:06:18 <rosmaita> it would be good to get RC-2 released today 14:06:46 <rosmaita> but there is always tomorrow -- anyway, we can discuss later 14:06:51 <rosmaita> #topic RC-3 14:06:56 <rosmaita> there will be an RC-3 14:07:08 <rosmaita> what i'd like is to get the postgres situation figured out 14:07:25 <rosmaita> the situation is that there may be problems migrating an existing postgres database 14:07:49 <rosmaita> i filed a bug to track that, gimme a minute to find the link 14:08:28 <rosmaita> #link https://bugs.launchpad.net/glance/+bug/1749466 14:08:29 <openstack> Launchpad bug 1749466 in Glance "postgresql db migration broken in queens" [Critical,Triaged] - Assigned to Ameed Ashour (ameeda) 14:08:39 <rosmaita> and i see someone has already filed another one 14:08:51 <rosmaita> #link https://bugs.launchpad.net/glance/+bug/1749640 14:08:51 <openstack> Launchpad bug 1749640 in Glance "db sync fails for mysql while adding triggers" [High,In progress] - Assigned to Abhishek Kekane (abhishek-kekane) 14:09:25 <rosmaita> let's discuss a strategy for dealing with this after the meeting after discussing rc-2 14:09:36 <abhishekk> ok 14:09:50 <ShilpaD> ok 14:09:55 <rosmaita> i'd like to shoot for tuesday for RC-3 14:10:09 <abhishekk> sounds good 14:10:19 <rosmaita> the final release candidate needs to be done late next week 14:10:31 <jokke_> Monday would be great, Tue night GMT pretty much the latest or we'll be screwed 14:10:39 <jokke_> we need to release at Thu IIRC 14:10:51 <rosmaita> ok, good point 14:11:05 <rosmaita> we will have to timebox and aim for Monday 14:11:51 <rosmaita> we can discuss the parameters of the timebox after the meeting 14:12:03 <rosmaita> i don't want people going insane over the weekend 14:12:04 <abhishekk> ok' 14:12:10 <jokke_> ++ 14:12:28 <rosmaita> ok, next item 14:12:35 <rosmaita> #topic the uWSGI situation 14:12:42 <rosmaita> this impacts the next topic, too 14:12:54 <rosmaita> ok, so our recommendation has been don't use glance with uWSGI 14:13:10 <rosmaita> we are going to continue that recommendation in Queens 14:13:16 <McClymontS> is there any more info on this one? 14:13:28 <jokke_> unfortunately as it was pointed out in #os-glance while ago we do not officially indicate that anywhere 14:13:54 <rosmaita> i think it says that in the running glance under httpd docs 14:13:58 <rosmaita> i will check 14:14:21 <rosmaita> i will also submit a patch removing our completion of uwsgi support from the community goals 14:14:25 <jokke_> IIRC that doc just eplains how to do it ... we never made the change to state there that it should not be done 14:14:42 <rosmaita> ok, will have to check 14:15:08 <rosmaita> anyway, the impact is that if you want to run tasks in devstack ... 14:15:21 <rosmaita> you need to set WSGI_MODE=mod_wsgi 14:15:25 <rosmaita> in your local.conf 14:15:35 <rosmaita> that starts glance in systemd 14:15:43 <rosmaita> but uses the "regular" glance wsgi server 14:15:56 <abhishekk> ok 14:16:02 <jokke_> is it Glance only or does it push all the services under systemd? 14:16:14 <rosmaita> well, it's a bit weird 14:16:18 <esberglu> Is there a glance specific solution? 14:16:28 <rosmaita> yes, do not use uwsgi 14:16:48 <jokke_> esberglu: as in do not deploy glance under Apache 14:16:51 <rosmaita> so what happens when you do that is that glance and keystone run slightly differently 14:16:52 <esberglu> Yeah I thought that WSGI_MODE affected more than glance, not super familiar though 14:17:21 <rosmaita> yeah, i am focused on glance, so i am not 100% sure what happens to the other services 14:17:29 <rosmaita> this is what i have observed: 14:17:30 <McClymontS> I am not 100% sure but I suspect that wsgi mode is more global as well 14:17:44 <rosmaita> (1) in default devstack: 14:17:44 <jokke_> rosmaita: ok, cool ... well that's good to know anyways ... makes life much easier for testing things 14:18:06 <jokke_> sry, go ahead 14:18:07 <rosmaita> everything runs under systemd AND in a uwsgi "container" 14:18:30 <rosmaita> (2) when you use WSGI_MODE=mod_wsgi 14:18:50 <rosmaita> everything except keystone (and maybe one or two other things) runs under systemd 14:18:54 <rosmaita> including glance 14:19:24 <rosmaita> the diff is that the glance systemd starts /path/glance --conf /etc/glance-api.conf 14:19:28 <rosmaita> something like that 14:19:36 <rosmaita> keystone is run out of apache 14:19:38 <jokke_> so we need to figure out how to change all our test jobs to do that as well 14:20:07 <rosmaita> if you do systemctrl status apache2.service you can see what httpd is running 14:20:40 <rosmaita> well, the glance functional tests don't use devstack 14:20:51 <rosmaita> but yeah, we do need to do something 14:21:11 <rosmaita> the mod_wsgi support is supposed to be pulled from devstack 14:21:39 <rosmaita> i will put up a patch to introduce a new var to decide whether glance runs under uwsgi or not 14:21:47 <jokke_> yeah, what I mean is that we need to have tempest jobs for glance doing that as well 14:22:14 <rosmaita> yeah, apparently tempest doesn't cover any tasks stuff, or we'd be seeing problems 14:22:36 <rosmaita> so, the key point for the next 2 weeks is 14:22:54 <rosmaita> put the WSGI_MODE thing in your local.conf so you can test out image import 14:23:50 <rosmaita> ok, so esberglu i have kind of put a damper on your next item 14:24:10 <rosmaita> mtreinish has been working on some fixes to get glance working better with uwsgi 14:24:24 <rosmaita> but the key thing is that we absolutely need tasks for image import 14:24:48 <rosmaita> and the uwsgi upload fixes have not been tested at scale 14:25:15 <rosmaita> so i think we need to declare the uwsgi stuff experimental 14:25:31 <abhishekk> agree 14:25:35 <rosmaita> and if people really want it, they can propose some specs 14:26:01 <rosmaita> and we can discuss what would really be needed to get it done correctly 14:26:43 <rosmaita> #topic pike backport for image-create 14:26:49 <rosmaita> esberglu that's you 14:26:58 <jokke_> unfortunately I totally agree ... running glance in uwsgi container has almost no priority to atm. specially looking our resources and backlog 14:27:15 <jokke_> "to me atm." 14:27:26 <esberglu> I was looking at this bug 14:27:27 <esberglu> https://bugs.launchpad.net/glance/+bug/1703856 14:27:28 <openstack> Launchpad bug 1703856 in Glance "502 Bad gateway error on image-create" [High,Fix released] - Assigned to Matthew Treinish (treinish) 14:27:45 <esberglu> And this backport 14:27:46 <esberglu> https://review.openstack.org/#/c/525994/ 14:28:08 <esberglu> But if the recommendation is to not use uwsgi I'm guessing that won't get in? 14:30:03 <rosmaita> jokke_ you are still release czar atm 14:30:18 <jokke_> esberglu: I'm very reluctant to approve that backport and claim that "Now it's ok to run Glance Pike in uwsgi" 14:31:02 <rosmaita> found the doc: https://docs.openstack.org/releasenotes/glance/pike.html#known-issues 14:31:12 <jokke_> that's just really really bad idea and backporting that to Pike just to get devstack working out of box is kind of those things that has priority sowhere so far I can't see it ;) 14:31:46 <rosmaita> agree 14:32:13 <rosmaita> i will make sure that advice is repeated in the queens release notes 14:32:31 <rosmaita> and put up a patch to the admin docs 14:33:07 <rosmaita> which just say, by the way, *if* you want to run glance under httpd, we recommend uwsgi ... there's also a note saying that it may not work 14:33:07 <abhishekk> 1) Upstream reviews, Fix for glance deployment with postgresql 14:33:08 <abhishekk> 2) More upstream reviews, Fix release critical issues 14:33:08 <abhishekk> 3) None 14:33:16 <abhishekk> sorry ignore that 14:34:03 <jokke_> :) 14:34:04 <rosmaita> ok, guess we covered that 14:34:14 <jokke_> just one thing 14:34:17 <esberglu> Thanks for the info 14:34:17 <rosmaita> sorry esberglu but thank you for attending 14:34:20 <jokke_> as a bit of clarification 14:35:00 <jokke_> my biggest problem with that is that fixing such thing we as upstream community reinforce supporting such deployments and we really can't afford doing so atm. 14:35:32 <jokke_> it's not so much technical problem as it is principal/political 14:35:40 <McClymontS> agree 14:36:04 <rosmaita> yes, and a key aspect is that we must have more & better test coverage 14:36:04 <jokke_> so if you really need to have that patch in, carry it in your own deployment on your own risk that the next thing will break 14:36:19 <rosmaita> because it will entail separating out the task stuff into a separate service 14:36:22 <jokke_> the patch itself is not bad and it's in queens 14:36:34 <rosmaita> which as jokke_ says is not technically a problem 14:36:45 <rosmaita> but it has an enormous potential to affect stability 14:37:22 <rosmaita> so we will need QA resources if that is to happen 14:37:35 <rosmaita> and our community is too small atm 14:37:57 <rosmaita> ok, anything else? 14:37:58 <jokke_> I'm ready for next topic :) 14:38:01 <rosmaita> me too 14:38:14 <rosmaita> #topic PTG planning 14:38:23 <rosmaita> jokke_ this is you 14:38:38 <jokke_> Ok, so PTG is coming faster than anyone can see from the release rush! 14:39:22 <jokke_> which means I want to start nailing down our PTG planning at latest Monday 14:39:35 <rosmaita> i would like to propose that we combine the queens retrospective with a guiness brewery tour 14:39:43 <rosmaita> don't know whether the logistics on that are reasonable, though 14:39:55 <rosmaita> someone told me they have wi-fi at guiness 14:40:07 <jokke_> Please do have a look on the planning etherpad, make any additions you might want to bring up and mark your interest 14:40:14 <smcginnis> There's a nice pub area in there too. :) 14:40:48 <jokke_> rosmaita: my vote goes to Jamesons' ... it's better tour and we might need some stronger tools :P 14:41:02 <smcginnis> Hah 14:41:05 <rosmaita> jokke_ you are PTL for a reason! good choice 14:41:24 <jokke_> So Wed morning that is! :P 14:41:48 <rosmaita> how early does it open? 14:41:55 <jokke_> I need to check 14:42:16 <jokke_> but I think we get Glance sessions well started if we lubricate it with Whiskey 14:42:21 <smcginnis> If Jameson hasn't changed much in the last 18 years, they had a nice pub area too. 14:42:38 <jokke_> smcginnis: oh they do and they still serve the best Irish Coffee ever 14:42:45 <rosmaita> ok, i will leave it up to you Dublin experts to pick the place 14:42:56 <abhishekk> exactly 14:43:01 <rosmaita> but seriously, we could do the retrospective there 14:43:07 <rosmaita> combine business & pleasure 14:43:13 <rosmaita> just a suggestion 14:43:13 <jokke_> I'll keep that in mind 14:43:16 <rosmaita> cool 14:43:21 <jokke_> put it into the etherpad :P 14:44:11 <jokke_> but that's all form me about that topic, just get your input in so I don't need to come out with my own agenda for the week 14:44:23 <jokke_> no-one will like that :D 14:44:36 <rosmaita> ok, so jokke_ what do you say that deadline for proposals is 23:59 utc tomorrow? 14:44:40 <jokke_> #deathByPowerpoint 14:44:49 <rosmaita> what deadline for getting votes onto the etherpad? 14:44:54 <jokke_> rosmaita: 0900 UTC Mon Morning 14:45:18 <jokke_> after that I start composing the schedule 14:45:32 <jokke_> not doing it over the weekend 14:45:35 <rosmaita> ok, i will include that on the priority email 14:45:47 <jokke_> tyty 14:46:06 <rosmaita> #topic open discussion 14:46:22 <rosmaita> anyone? 14:46:32 <jokke_> one quick one 14:47:05 <mbindlish> Hi, I am looking for a workaround of https://bugs.launchpad.net/glance/+bug/1703856. I have already applied patches https://review.openstack.org/#/c/493654/ (Glance) and https://review.openstack.org/492651 But still facing issue in my devstack Pike with bad gateway 502 error 14:47:07 <openstack> Launchpad bug 1703856 in Glance "502 Bad gateway error on image-create" [High,Fix released] - Assigned to Matthew Treinish (treinish) 14:47:18 <jokke_> request for all ... anyone in the audience who really really cares about stable releases 14:47:54 <jokke_> Please come forward ... our Stable-maint group basically consists of myself and rosmaita and that will get backporting stuff very difficult 14:48:05 <smcginnis> ++ 14:49:08 <jokke_> for example 544801 is pending to find external reviewer as it's my patch, I technically shouldn't +2 it and Abhishek is not stable-core so Brian should not merge it in by himself 14:49:58 <jokke_> so we really could do some help there 14:50:24 <abhishekk> mbindlish, just go through the previous discussion which will help you to understand the situation of uwsgi 14:50:27 <rosmaita> mbindlish i don't have a workaround for that and we are in queens rc-time, don't think anyone here will be able to help 14:50:42 <rosmaita> also, what abhishekk said 14:50:43 <mbindlish> ok 14:51:41 <rosmaita> anything else? 14:52:04 <abhishekk> so should we shift to openstack-glance? 14:52:24 <rosmaita> yes, and i will try to make it quick because i know it is late for you 14:52:39 <rosmaita> ok, thanks everyone! don't forget jokke_ 's request about the PTG planning 14:52:43 <rosmaita> #endmeeting