14:01:02 <abhishekk> #startmeeting glance
14:01:03 <openstack> Meeting started Thu Jul 23 14:01:02 2020 UTC and is due to finish in 60 minutes.  The chair is abhishekk. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:04 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:01:06 <openstack> The meeting name has been set to 'glance'
14:01:07 <abhishekk> #topic roll call
14:01:10 <jokke> o/
14:01:13 <abhishekk> #link https://etherpad.openstack.org/p/glance-team-meeting-agenda
14:01:15 <abhishekk> o/
14:01:31 <abhishekk> greetings if the day!!
14:01:33 <rosmaita> man, i am so jealous that you got openstack-meeting ... cinder is stuck meeting in openstack-meeting-alt
14:01:43 <abhishekk> :D
14:02:14 <abhishekk> lets start, we have lengthy agenda today
14:02:35 <abhishekk> #topic release/periodic jobs update
14:02:41 <abhishekk> V2 is next week
14:03:16 <abhishekk> I want to tag release on 29th July which is just a day before our next meeting
14:04:00 <abhishekk> TODO's are doc patch for dansmith's policy changes, release note patch and config and policy regenerate patch
14:04:18 <abhishekk> I will submit most of them by tomorrow EOD
14:04:46 <abhishekk> For store and client we didn't have major changes so no need to worry about them
14:05:28 <abhishekk> There are still couple of bug fixes which I want to be part of this release
14:05:40 <abhishekk> I will flag them in open discussion
14:06:12 <abhishekk> Regarding periodic jobs, functional-py3* jobs were broken due to dnspython version 2.0.0
14:06:24 <abhishekk> it is fixed and we are green now
14:06:48 <abhishekk> any questions about release plan?
14:07:11 <smcginnis> ++
14:07:13 <rosmaita> no, sounds good to me
14:07:24 <abhishekk> cool, moving ahead
14:07:41 <abhishekk> #topic Specs status
14:07:55 <abhishekk> We still have plenty of specs for reviews
14:07:57 <abhishekk> Make cinder driver compatible with multiple stores - https://review.opendev.org/695152 - Need reviewes
14:07:57 <abhishekk> Update proposal for duplication image download - https://review.opendev.org/734683 - Need reviews
14:07:57 <abhishekk> Optimize Ceph store network usage - https://review.opendev.org/#/c/740980/ - Need reviews
14:07:57 <abhishekk> Introspect import plugin to calculate virtual size of image - https://review.opendev.org/741121 - Need reviews
14:07:57 <patchbot> patch 695152 - glance-specs - Make cinder driver compatible with multiple stores - 10 patch sets
14:07:59 <patchbot> patch 734683 - glance-specs - Update proposal for duplication image download - 3 patch sets
14:07:59 <abhishekk> Cache API - https://review.opendev.org/#/c/665258/ - Need reviews
14:08:00 <patchbot> patch 740980 - glance-specs - Optimize Ceph store network usage - 2 patch sets
14:08:02 <patchbot> patch 741121 - glance-specs - Introspect import plugin to calculate virtual size... - 2 patch sets
14:08:03 <patchbot> patch 665258 - glance-specs - Spec for Glance cache API - 4 patch sets
14:08:23 <abhishekk> Kindly focus on reviews for the same
14:08:50 <abhishekk> Moving ahead
14:09:11 <abhishekk> #topic race-condition in copy-image operation
14:09:37 <abhishekk> This is important issue and we need to fix it as early as possible
14:10:04 <abhishekk> I saw the new approach proposed by jokke and objections around it
14:10:41 <abhishekk> after following the discussions on the patch I think that there is no stable solution at the moment for this issue
14:11:13 <abhishekk> So how we should fix this issue?
14:13:26 <abhishekk> One temporary solution comes to my mind is reject second request on same image id
14:13:33 <jokke> My biggest problem with the alternative is the much increased DOS exposure hardlocking the images with only way out being manual admin intervention. I'm really not comfortable to let that slip
14:14:45 <abhishekk> Can we have periodic job to check the progress and release the lock?
14:15:51 <rosmaita> i think the problem is defining what 'progress' means
14:16:05 <jokke> Like I told before that makes literally no difference to the solution I proposed apart from being another piece of code that is not coulped together with the rest doing tht for us
14:16:27 <abhishekk> rosmaita, we are setting task-id to image property
14:16:28 <jokke> *coupled*
14:16:44 <abhishekk> we can check the status of task on the basis of task id
14:17:07 <rosmaita> right, but the task that gets the lock first will got to 'processing', perhaps forever
14:17:24 <jokke> abhishekk: I think that's true only on the node where that task is running
14:18:32 <abhishekk> how will that make a difference?
14:18:48 <jokke> So the periodic job would have 2 different views to the situation depending if it's running on the node where the task is running or not
14:19:50 <jokke> periodic job on the node where the task is not running would go "I don't see this task, it must have failed, break the lock" while the on in the correct node would go "I see the task still running, lets keep the situation as it is"
14:20:30 <rosmaita> i think dansmith is in pacific time zone?
14:20:44 <jokke> IIRC the task has quite few updates onto the DB, pretty much, when it's queued, picked up, running, success/failure
14:20:53 <abhishekk> jokke, even there are different nodes we are going to pick task state from db
14:21:38 <abhishekk> rosmaita, dansmith also has conflict today
14:21:52 <rosmaita> i think it would be good to have a video meeting to talk this out and take advantage of his nova experience with race conditions
14:22:50 <abhishekk> Ok, I will try to catch him after the meeting and will ask the convenient time for the same
14:24:05 <abhishekk> jokke, rosmaita,  (smcginnis) what will be good time for you?
14:24:13 <abhishekk> Friday or Monday?
14:24:33 <rosmaita> abhishekk: just schedule and i will try to make it
14:24:40 <abhishekk> ack
14:24:52 <rosmaita> i think this is really something you erno and dan need to work out
14:25:15 <abhishekk> hmm
14:25:22 <abhishekk> ok, moving ahead
14:25:24 <jokke> abhishekk: don't push it too late Fri for both of our weekends sake
14:25:54 <abhishekk> jokke, ack
14:26:19 <abhishekk> I will schedule it mostly on Monday
14:26:46 <abhishekk> #topic Remove single store support for glance
14:26:50 <jokke> I don't mind doing some late night during the week, but I'd prefer getting onto my weekend before Sat morn :P
14:27:03 <abhishekk> me too :D
14:27:17 <abhishekk> Ok, So I have started putting efforts on this
14:27:42 <abhishekk> and looks like its much more difficult than I thought
14:28:04 <abhishekk> So I am following below approach for the same
14:28:39 <abhishekk> First add support in devstack to configure multiple stores
14:28:51 <abhishekk> then remove single stores support from glance
14:28:57 <abhishekk> remove it from store
14:29:04 <abhishekk> and then remove it form devstack
14:29:27 <abhishekk> And below is my progress so far
14:29:29 <abhishekk> Add support to configure multiple file stores using devstack - merged
14:29:29 <abhishekk> Configure swift for glance if multiple stores is enabled - https://review.opendev.org/741654
14:29:29 <abhishekk> Configure ceph for glance if multiple stores is enabled - https://review.opendev.org/741801
14:29:29 <abhishekk> Default Glance to configure multiple stores in devstack - https://review.opendev.org/741802
14:29:29 <abhishekk> Remove single store support from glance - In progress (most complicated :D)
14:29:30 <patchbot> patch 741654 - devstack - Configure swift for glance if multiple stores is e... - 3 patch sets
14:29:30 <abhishekk> Remove 'default_store', 'stores' config options from glance_store - ToDo
14:29:31 <patchbot> patch 741801 - devstack-plugin-ceph - Configure ceph for glance if multiple stores is en... - 2 patch sets
14:29:33 <patchbot> patch 741802 - devstack - Default Glance to configure multiple stores - 1 patch set
14:30:02 <rosmaita> don't you want to move devstack to multiple stores *before* removing single store from glance?
14:30:32 <abhishekk> rosmaita, I have already submitted patches for the same
14:30:38 <jokke> abhishekk: IIRC oslo_config had hidden parameter (so it doesn't show up in the configgen) might be worthy to look that route so announce it eol, hide the config options and work devstack out of that model, then get rid of the specific tests and convert rest, then start removing bit by bit and see what breaks
14:30:41 <abhishekk> and those are under review
14:30:48 <rosmaita> i mean patch 741802
14:30:48 <patchbot> https://review.opendev.org/#/c/741802/ - devstack - Default Glance to configure multiple stores - 1 patch set
14:31:22 <abhishekk> Just see above 3 patches of it
14:31:41 <abhishekk> jokke, cool
14:31:50 <abhishekk> I will try to follow this
14:32:37 <jokke> The registry and v1 api's had fortunately their config options well enough isolated that I could remove them, but that was the approach we took there
14:32:46 <abhishekk> rosmaita, currently devstack for glance configures swift, file and rbd store
14:32:57 <abhishekk> I have submitted changes for all of them
14:33:14 <jokke> basically make it non-trivial to enable and remove bit by bit seeing where stuff breaks and needs refactoring
14:33:25 <jokke> makes it much easier than trying to get it out in one bang
14:33:40 <abhishekk> yeah I know
14:33:54 <abhishekk> I removed it all in one go and now messing around tests
14:34:08 <jokke> :P
14:34:08 <abhishekk> But I will try to follow your approach
14:35:06 <abhishekk> I will try to manage this in next couple of weeks (providing devstack patches merged by then)
14:35:18 * jokke has had good amount of execise whaking out dead code lately :D
14:35:28 <abhishekk> haha
14:35:50 <abhishekk> I might need your help here as well
14:35:54 <abhishekk> Ok, moving ahead
14:36:04 <jokke> just ping, you know how to reach me
14:36:28 <abhishekk> yep :D
14:36:39 <abhishekk> #topic Glance under uwsgi - whats next???
14:36:59 <abhishekk> So for this topic we have discussion with TC in next 30 minutes
14:37:17 <jokke> TC-Glance meeting about the topin at the top of the hour, see ML for details
14:37:22 <jokke> topic
14:37:37 <jokke> bottom of the hour for abhishekk ;)
14:37:45 <abhishekk> :D
14:37:56 <abhishekk> will provide update about the same in next meeting
14:38:06 <abhishekk> Moving ahead
14:38:14 <abhishekk> #topic Open discussion
14:38:29 <abhishekk> I have couple of review requests
14:38:46 <abhishekk> #link https://review.opendev.org/742115
14:38:47 <patchbot> patch 742115 - glance - Fix broken glance-cache-manage utility - 3 patch sets
14:39:41 <abhishekk> #link https://review.opendev.org/742309
14:39:41 <patchbot> patch 742309 - glance - Make our import-workflow job also convert images t... - 1 patch set
14:40:23 <abhishekk> Apart from these, please provide feedback on open specs as well
14:41:03 <abhishekk> There are some open patches for stable branches as well
14:42:31 <abhishekk> that is it from me for today
14:43:22 <abhishekk> anything else from any one?
14:44:02 <Steap> This low-hanging fruit would be nice to pick https://review.opendev.org/#/c/713720/ :)
14:44:03 <patchbot> patch 713720 - glance (stable/train) - Fix reloading under PY3 (MERGED) - 3 patch sets
14:44:31 <jokke> Steap: as patchbot pointed out, it's merged and will be released next week :P
14:44:42 <abhishekk> :D
14:44:56 <Steap> oh I only looked at the number of °+2s :)
14:44:59 <tosky> just wanted to mention that I realized there is a last legacy job used by glance
14:45:20 <tosky> you can see that from the email I sent yesterday (or a few hours ago, depending on your timezone)
14:45:28 <tosky> the global goal email
14:45:34 <abhishekk> tosky, ack
14:46:45 <jokke> abhishekk: quick question. V2 is next week, Dan has been doing amazing job on the Nova copy side, the copy race and now also tackling the uwsgi tasks it seems. How we are doing with our priority list items. Do we have something in the list we should already start flagging that we will slip to W?
14:47:21 <abhishekk> jokke, good point
14:47:30 <jokke> Just warning, I'm planning to take couple of weeks PTO in near future so that will be two weeks I'm not working on anything nor reviewing code
14:47:52 <Steap> jokke: unless we decline your request to take PTO
14:47:59 <abhishekk> I will collect the updates form stake holders and we can sort it out next week
14:48:22 <jokke> Steap: Fortunately I live in deeply socialistic country that has massive amount of worker rights :P
14:48:34 <abhishekk> We might move cache work, cluster awareness and duplicate downloads to W
14:48:51 <Steap> jokke: *laughs in French*
14:49:36 <jokke> Steap: ohh, french you don't even need pto as you guys will just go to strike when it's beach weather :P
14:49:58 <Steap> jokke: fun fact, there are no strikes in the summer since people are on holidays
14:50:06 <jokke> We're not in that deep yet
14:50:07 <Steap> we know the next strikes will happen in September
14:50:21 <jokke> oh yeah, lol who wants to waste their holidays for strikes
14:50:33 <smcginnis> :)
14:54:20 <abhishekk> 5 minutes to go
14:55:02 <abhishekk> Lets wrap up
14:55:05 <abhishekk> thank you all
14:55:28 <jokke> thanks
14:55:35 <alistarle> \o
14:55:56 <abhishekk> alistarle, anything from you?
14:56:39 <abhishekk> if yes, then ping me on openstack-glance channel
14:56:43 <alistarle> I just lurked around the meeting to see if something moved
14:56:46 <alistarle> but seems ok
14:56:51 <abhishekk> cool
14:56:54 <abhishekk> thank you
14:57:02 <abhishekk> #endmeeting