10:01:57 <rosmaita> #startmeeting glance_bs
10:01:58 <openstack> Meeting started Mon Jul  9 10:01:57 2018 UTC and is due to finish in 60 minutes.  The chair is rosmaita. Information about MeetBot at http://wiki.debian.org/MeetBot.
10:01:59 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
10:02:02 <openstack> The meeting name has been set to 'glance_bs'
10:02:18 <rosmaita> good morning, time for the glance bug squad meeting
10:02:20 <abhishekk|afk> o/
10:02:32 <abhishekk|afk> good morning
10:02:38 <rosmaita> o/
10:03:02 <rosmaita> i think let's concentrate on glance-store today
10:03:26 <abhishekk> sounds good
10:04:50 <rosmaita> you are reading my mind!
10:05:05 <rosmaita> #link https://etherpad.openstack.org/p/glance-bug-squad-meeting
10:05:06 <abhishekk> :D
10:05:41 <rosmaita> glad tomoki is already working on it
10:06:47 <abhishekk> https://review.openstack.org/#/c/579335/
10:07:05 <rosmaita> think we should target it for 0.25.0 ?
10:07:06 <abhishekk> this is the patch for resolving the issue
10:07:18 <rosmaita> it seems like a bad error
10:07:44 <abhishekk> yes, it requires otherwise cinder backend is broken
10:08:16 <rosmaita> we should ask smcginnis[m] to take a look
10:08:36 <abhishekk> I will add him as a reviewer
10:09:15 <rosmaita> thanks!
10:09:33 <rosmaita> ok, i marked it as critical for 0.25.0
10:09:43 <abhishekk> ok
10:09:55 <rosmaita> although i guess that's overstating it a bit since the cinder driver still says it's experimental
10:10:40 <jokke_> It might be good time to drop that statement as well ... I did not realize it's still stating so ;)
10:11:10 <rosmaita> jokke_ : hello!
10:11:26 <rosmaita> yes, i think it gets logged when the store is loaded
10:11:34 <abhishekk> jokke_, o/
10:12:09 <rosmaita> i'll file a bug for that ... do we want to do it before 0.25.0 ?
10:12:50 <abhishekk> I don't think so
10:13:02 <jokke_> I'd be ok with that
10:13:07 <rosmaita> also, i guess this one is unreleated? https://bugs.launchpad.net/glance-store/+bug/1764200
10:13:07 <openstack> Launchpad bug 1764200 in glance_store "Glance Cinder backed images & multiple regions" [High,Triaged]
10:13:12 <jokke_> if we get this bugfix in as well
10:13:41 <rosmaita> true, it would be bad to say it's no longer experimental AND it doesn't work!
10:14:13 <jokke_> indeed ;)
10:15:50 <rosmaita> here's the list of open bugs, newest first: https://bugs.launchpad.net/glance-store/?orderby=-id&start=0
10:16:09 <rosmaita> let's take a quick glance (ha!) at them and see if anything looks critical
10:16:30 <rosmaita> because we have abhishekk's multiple backends that def needs to get in
10:16:42 <rosmaita> and i have a new multihash patch up
10:18:19 <abhishekk> agree
10:19:06 <rosmaita> this one is a year old: https://bugs.launchpad.net/glance-store/+bug/1706405
10:19:07 <openstack> Launchpad bug 1706405 in glance_store "ceph jobs failing to upload images in pike due to "AttributeError: 'NoneType' object has no attribute 'Rados'" on py35" [Undecided,Confirmed]
10:19:08 <jokke_> is https://bugs.launchpad.net/glance-store/+bug/1732036 still valid bug?
10:19:08 <openstack> Launchpad bug 1732036 in glance_store "Glance cannot delete incomplete chunked upload to Swift backend if token expires" [Undecided,New]
10:19:16 <rosmaita> it's a ceph thing, have you guys seen it?
10:20:04 <rosmaita> jokke_ i don't know
10:20:16 <rosmaita> my note on it says its a multitenant store problem
10:20:58 <jokke_> I thought the trust implementation fixed that ...
10:21:13 <jokke_> it's just even that bug is from 2017
10:21:18 <rosmaita> me too, but dmitry says otherwise
10:21:26 <abhishekk> rosmaita, I haven't got this error
10:21:26 <rosmaita> yes, but november 2017
10:21:59 <abhishekk> I have done lots of testing for ceph backend and never encounter this issue
10:22:04 <rosmaita> abhishekk: maybe it's a packaging thing, then -- you mean https://bugs.launchpad.net/glance-store/+bug/1706405 ?
10:22:04 <openstack> Launchpad bug 1706405 in glance_store "ceph jobs failing to upload images in pike due to "AttributeError: 'NoneType' object has no attribute 'Rados'" on py35" [Undecided,Confirmed]
10:22:39 <rosmaita> jokke_ i'm not sure how well the trusts stuff is tested in multitenant environment
10:22:56 <abhishekk> rosmaita, yes
10:23:10 <rosmaita> abhishekk great, i will mark that as incomplete for now, then
10:23:24 <abhishekk> rosmaita, ok
10:23:42 <jokke_> indeed ... well I'd assume we would have got a lot more heat if those two were broken in general
10:24:40 <rosmaita> jokke_ agreed, i think we can back-burner them both
10:25:39 <abhishekk> again, I haven't faced this issue as well
10:25:46 <abhishekk> https://bugs.launchpad.net/glance-store/+bug/1706405
10:25:46 <openstack> Launchpad bug 1706405 in glance_store "ceph jobs failing to upload images in pike due to "AttributeError: 'NoneType' object has no attribute 'Rados'" on py35" [Undecided,Incomplete]
10:25:56 <abhishekk> sorry wrong link
10:26:15 <abhishekk> https://bugs.launchpad.net/glance-store/+bug/1673280
10:26:16 <openstack> Launchpad bug 1673280 in glance_store "BackendException: Failed to add object to Swift. - While trying to upload image size of around 400 MB" [Undecided,New]
10:27:27 <rosmaita> that one is pretty old
10:28:37 <abhishekk> yes
10:29:33 <rosmaita> looks like swift is failing on the first segment (or the error message is misleading, not sure about the '...' in it)
10:31:59 <rosmaita> that's weird though, because the large object size is 5120 MB, and this is a 400 MB image
10:32:18 <abhishekk> yes
10:32:49 <rosmaita> i think we ask for more information on this one
10:33:31 <abhishekk> ok, I have tried to upload around 2 gb image without any issue
10:34:50 <rosmaita> ok, back to https://bugs.launchpad.net/glance-store/+bug/1764200
10:34:50 <openstack> Launchpad bug 1764200 in glance_store "Glance Cinder backed images & multiple regions" [High,Triaged]
10:35:07 <rosmaita> jokke_ you marked that one as high
10:36:59 <jokke_> yes, I think that's pretty high importance issue if it's really broken
10:37:43 <jokke_> but looks like you were able to confirm it
10:37:54 <rosmaita> jokke_ maybe shoot tomoki an email ?
10:38:11 <rosmaita> no, i only marked it as triaged because you assigned it an importance!
10:38:15 <abhishekk> good idea
10:39:00 <jokke_> I left the status intentionally unmarked as I did not have means to verify it ;)
10:39:38 <rosmaita> jokke_ can you shoot tomoki an email?
10:39:59 <rosmaita> he would be able to confirm right away, and would know if the proposed fix makes sense
10:40:00 <jokke_> yeah
10:40:06 <jokke_> will do
10:40:17 <rosmaita> ok, cool
10:41:34 <rosmaita> also, should we remove Glance from that bug?
10:43:01 <abhishekk> yes
10:43:53 <rosmaita> ok, i removed glance, marked it back to incomplete, and left a note that if confirmed, we want to get it into 0.25.0
10:44:23 <abhishekk> ok
10:44:33 <rosmaita> jokke_ should i just go ahead and target it for 0.25.0 ?
10:47:00 <jokke_> yeah, feel free
10:47:14 <rosmaita> done
10:51:12 <rosmaita> ok, that's all for now, bug-wise
10:51:28 <rosmaita> thanks! but stick around, i have a question
10:51:32 <rosmaita> #endmeeting