14:01:02 #startmeeting glance 14:01:03 Meeting started Thu Jul 23 14:01:02 2020 UTC and is due to finish in 60 minutes. The chair is abhishekk. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:01:04 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:01:06 The meeting name has been set to 'glance' 14:01:07 #topic roll call 14:01:10 o/ 14:01:13 #link https://etherpad.openstack.org/p/glance-team-meeting-agenda 14:01:15 o/ 14:01:31 greetings if the day!! 14:01:33 man, i am so jealous that you got openstack-meeting ... cinder is stuck meeting in openstack-meeting-alt 14:01:43 :D 14:02:14 lets start, we have lengthy agenda today 14:02:35 #topic release/periodic jobs update 14:02:41 V2 is next week 14:03:16 I want to tag release on 29th July which is just a day before our next meeting 14:04:00 TODO's are doc patch for dansmith's policy changes, release note patch and config and policy regenerate patch 14:04:18 I will submit most of them by tomorrow EOD 14:04:46 For store and client we didn't have major changes so no need to worry about them 14:05:28 There are still couple of bug fixes which I want to be part of this release 14:05:40 I will flag them in open discussion 14:06:12 Regarding periodic jobs, functional-py3* jobs were broken due to dnspython version 2.0.0 14:06:24 it is fixed and we are green now 14:06:48 any questions about release plan? 14:07:11 ++ 14:07:13 no, sounds good to me 14:07:24 cool, moving ahead 14:07:41 #topic Specs status 14:07:55 We still have plenty of specs for reviews 14:07:57 Make cinder driver compatible with multiple stores - https://review.opendev.org/695152 - Need reviewes 14:07:57 Update proposal for duplication image download - https://review.opendev.org/734683 - Need reviews 14:07:57 Optimize Ceph store network usage - https://review.opendev.org/#/c/740980/ - Need reviews 14:07:57 Introspect import plugin to calculate virtual size of image - https://review.opendev.org/741121 - Need reviews 14:07:57 patch 695152 - glance-specs - Make cinder driver compatible with multiple stores - 10 patch sets 14:07:59 patch 734683 - glance-specs - Update proposal for duplication image download - 3 patch sets 14:07:59 Cache API - https://review.opendev.org/#/c/665258/ - Need reviews 14:08:00 patch 740980 - glance-specs - Optimize Ceph store network usage - 2 patch sets 14:08:02 patch 741121 - glance-specs - Introspect import plugin to calculate virtual size... - 2 patch sets 14:08:03 patch 665258 - glance-specs - Spec for Glance cache API - 4 patch sets 14:08:23 Kindly focus on reviews for the same 14:08:50 Moving ahead 14:09:11 #topic race-condition in copy-image operation 14:09:37 This is important issue and we need to fix it as early as possible 14:10:04 I saw the new approach proposed by jokke and objections around it 14:10:41 after following the discussions on the patch I think that there is no stable solution at the moment for this issue 14:11:13 So how we should fix this issue? 14:13:26 One temporary solution comes to my mind is reject second request on same image id 14:13:33 My biggest problem with the alternative is the much increased DOS exposure hardlocking the images with only way out being manual admin intervention. I'm really not comfortable to let that slip 14:14:45 Can we have periodic job to check the progress and release the lock? 14:15:51 i think the problem is defining what 'progress' means 14:16:05 Like I told before that makes literally no difference to the solution I proposed apart from being another piece of code that is not coulped together with the rest doing tht for us 14:16:27 rosmaita, we are setting task-id to image property 14:16:28 *coupled* 14:16:44 we can check the status of task on the basis of task id 14:17:07 right, but the task that gets the lock first will got to 'processing', perhaps forever 14:17:24 abhishekk: I think that's true only on the node where that task is running 14:18:32 how will that make a difference? 14:18:48 So the periodic job would have 2 different views to the situation depending if it's running on the node where the task is running or not 14:19:50 periodic job on the node where the task is not running would go "I don't see this task, it must have failed, break the lock" while the on in the correct node would go "I see the task still running, lets keep the situation as it is" 14:20:30 i think dansmith is in pacific time zone? 14:20:44 IIRC the task has quite few updates onto the DB, pretty much, when it's queued, picked up, running, success/failure 14:20:53 jokke, even there are different nodes we are going to pick task state from db 14:21:38 rosmaita, dansmith also has conflict today 14:21:52 i think it would be good to have a video meeting to talk this out and take advantage of his nova experience with race conditions 14:22:50 Ok, I will try to catch him after the meeting and will ask the convenient time for the same 14:24:05 jokke, rosmaita, (smcginnis) what will be good time for you? 14:24:13 Friday or Monday? 14:24:33 abhishekk: just schedule and i will try to make it 14:24:40 ack 14:24:52 i think this is really something you erno and dan need to work out 14:25:15 hmm 14:25:22 ok, moving ahead 14:25:24 abhishekk: don't push it too late Fri for both of our weekends sake 14:25:54 jokke, ack 14:26:19 I will schedule it mostly on Monday 14:26:46 #topic Remove single store support for glance 14:26:50 I don't mind doing some late night during the week, but I'd prefer getting onto my weekend before Sat morn :P 14:27:03 me too :D 14:27:17 Ok, So I have started putting efforts on this 14:27:42 and looks like its much more difficult than I thought 14:28:04 So I am following below approach for the same 14:28:39 First add support in devstack to configure multiple stores 14:28:51 then remove single stores support from glance 14:28:57 remove it from store 14:29:04 and then remove it form devstack 14:29:27 And below is my progress so far 14:29:29 Add support to configure multiple file stores using devstack - merged 14:29:29 Configure swift for glance if multiple stores is enabled - https://review.opendev.org/741654 14:29:29 Configure ceph for glance if multiple stores is enabled - https://review.opendev.org/741801 14:29:29 Default Glance to configure multiple stores in devstack - https://review.opendev.org/741802 14:29:29 Remove single store support from glance - In progress (most complicated :D) 14:29:30 patch 741654 - devstack - Configure swift for glance if multiple stores is e... - 3 patch sets 14:29:30 Remove 'default_store', 'stores' config options from glance_store - ToDo 14:29:31 patch 741801 - devstack-plugin-ceph - Configure ceph for glance if multiple stores is en... - 2 patch sets 14:29:33 patch 741802 - devstack - Default Glance to configure multiple stores - 1 patch set 14:30:02 don't you want to move devstack to multiple stores *before* removing single store from glance? 14:30:32 rosmaita, I have already submitted patches for the same 14:30:38 abhishekk: IIRC oslo_config had hidden parameter (so it doesn't show up in the configgen) might be worthy to look that route so announce it eol, hide the config options and work devstack out of that model, then get rid of the specific tests and convert rest, then start removing bit by bit and see what breaks 14:30:41 and those are under review 14:30:48 i mean patch 741802 14:30:48 https://review.opendev.org/#/c/741802/ - devstack - Default Glance to configure multiple stores - 1 patch set 14:31:22 Just see above 3 patches of it 14:31:41 jokke, cool 14:31:50 I will try to follow this 14:32:37 The registry and v1 api's had fortunately their config options well enough isolated that I could remove them, but that was the approach we took there 14:32:46 rosmaita, currently devstack for glance configures swift, file and rbd store 14:32:57 I have submitted changes for all of them 14:33:14 basically make it non-trivial to enable and remove bit by bit seeing where stuff breaks and needs refactoring 14:33:25 makes it much easier than trying to get it out in one bang 14:33:40 yeah I know 14:33:54 I removed it all in one go and now messing around tests 14:34:08 :P 14:34:08 But I will try to follow your approach 14:35:06 I will try to manage this in next couple of weeks (providing devstack patches merged by then) 14:35:18 * jokke has had good amount of execise whaking out dead code lately :D 14:35:28 haha 14:35:50 I might need your help here as well 14:35:54 Ok, moving ahead 14:36:04 just ping, you know how to reach me 14:36:28 yep :D 14:36:39 #topic Glance under uwsgi - whats next??? 14:36:59 So for this topic we have discussion with TC in next 30 minutes 14:37:17 TC-Glance meeting about the topin at the top of the hour, see ML for details 14:37:22 topic 14:37:37 bottom of the hour for abhishekk ;) 14:37:45 :D 14:37:56 will provide update about the same in next meeting 14:38:06 Moving ahead 14:38:14 #topic Open discussion 14:38:29 I have couple of review requests 14:38:46 #link https://review.opendev.org/742115 14:38:47 patch 742115 - glance - Fix broken glance-cache-manage utility - 3 patch sets 14:39:41 #link https://review.opendev.org/742309 14:39:41 patch 742309 - glance - Make our import-workflow job also convert images t... - 1 patch set 14:40:23 Apart from these, please provide feedback on open specs as well 14:41:03 There are some open patches for stable branches as well 14:42:31 that is it from me for today 14:43:22 anything else from any one? 14:44:02 This low-hanging fruit would be nice to pick https://review.opendev.org/#/c/713720/ :) 14:44:03 patch 713720 - glance (stable/train) - Fix reloading under PY3 (MERGED) - 3 patch sets 14:44:31 Steap: as patchbot pointed out, it's merged and will be released next week :P 14:44:42 :D 14:44:56 oh I only looked at the number of °+2s :) 14:44:59 just wanted to mention that I realized there is a last legacy job used by glance 14:45:20 you can see that from the email I sent yesterday (or a few hours ago, depending on your timezone) 14:45:28 the global goal email 14:45:34 tosky, ack 14:46:45 abhishekk: quick question. V2 is next week, Dan has been doing amazing job on the Nova copy side, the copy race and now also tackling the uwsgi tasks it seems. How we are doing with our priority list items. Do we have something in the list we should already start flagging that we will slip to W? 14:47:21 jokke, good point 14:47:30 Just warning, I'm planning to take couple of weeks PTO in near future so that will be two weeks I'm not working on anything nor reviewing code 14:47:52 jokke: unless we decline your request to take PTO 14:47:59 I will collect the updates form stake holders and we can sort it out next week 14:48:22 Steap: Fortunately I live in deeply socialistic country that has massive amount of worker rights :P 14:48:34 We might move cache work, cluster awareness and duplicate downloads to W 14:48:51 jokke: *laughs in French* 14:49:36 Steap: ohh, french you don't even need pto as you guys will just go to strike when it's beach weather :P 14:49:58 jokke: fun fact, there are no strikes in the summer since people are on holidays 14:50:06 We're not in that deep yet 14:50:07 we know the next strikes will happen in September 14:50:21 oh yeah, lol who wants to waste their holidays for strikes 14:50:33 :) 14:54:20 5 minutes to go 14:55:02 Lets wrap up 14:55:05 thank you all 14:55:28 thanks 14:55:35 \o 14:55:56 alistarle, anything from you? 14:56:39 if yes, then ping me on openstack-glance channel 14:56:43 I just lurked around the meeting to see if something moved 14:56:46 but seems ok 14:56:51 cool 14:56:54 thank you 14:57:02 #endmeeting