Tuesday, 2017-12-05

*** masber has quit IRC00:33
*** btully has joined #openstack-glance00:40
*** masber has joined #openstack-glance00:41
*** danpawlik has joined #openstack-glance00:42
*** danpawlik_ has quit IRC00:44
*** btully has quit IRC00:45
*** masber has quit IRC00:47
*** gyee has quit IRC00:56
*** zhurong has joined #openstack-glance01:01
*** mvk has quit IRC01:57
*** mvk has joined #openstack-glance01:59
*** zhurong has quit IRC02:00
*** dalgaaf has quit IRC02:04
*** dalgaaf has joined #openstack-glance02:05
*** gcb has joined #openstack-glance02:06
*** pbourke_ has quit IRC02:23
*** pbourke_ has joined #openstack-glance02:25
*** zhurong has joined #openstack-glance02:30
openstackgerritTin Lam proposed openstack/glance master: Admin API policy enforcement contingent on is_admin_project  https://review.openstack.org/38465502:37
*** masber has joined #openstack-glance02:39
openstackgerritOpenStack Proposal Bot proposed openstack/glance master: Updated from global requirements  https://review.openstack.org/52537403:02
openstackgerritOpenStack Proposal Bot proposed openstack/glance_store master: Updated from global requirements  https://review.openstack.org/52500303:02
*** links has joined #openstack-glance03:07
*** MattMan has quit IRC03:09
*** MattMan has joined #openstack-glance03:10
*** abhishekk has joined #openstack-glance03:23
*** threestrands has joined #openstack-glance03:27
openstackgerritOpenStack Proposal Bot proposed openstack/python-glanceclient master: Updated from global requirements  https://review.openstack.org/51915103:31
*** rosmaita has quit IRC03:37
*** pdeore has joined #openstack-glance04:10
*** zhurong has quit IRC04:27
abhishekksmcginnis, by any chance you know updated url for http://docs.openstack.org/ops-guide/ops_user_facing_operations.html04:43
*** nicolasbock has quit IRC04:48
*** udesale has joined #openstack-glance04:53
*** zhurong has joined #openstack-glance04:54
*** ratailor has joined #openstack-glance04:56
*** pdeore_ has joined #openstack-glance05:01
*** pdeore has quit IRC05:03
*** threestrands has quit IRC05:25
*** pdeore has joined #openstack-glance05:59
*** pdeore_ has quit IRC06:01
*** pdeore has quit IRC06:18
*** alexchadin has joined #openstack-glance06:32
*** pdeore has joined #openstack-glance06:38
*** arcolife has joined #openstack-glance06:45
*** mine0901 has quit IRC06:57
*** udesale__ has joined #openstack-glance06:58
*** udesale has quit IRC06:58
*** udesale has joined #openstack-glance07:00
*** udesale__ has quit IRC07:02
*** zhurong has quit IRC07:16
*** pcaruana has joined #openstack-glance07:32
*** haibinhuang has joined #openstack-glance07:33
*** haibinhuang has quit IRC07:33
*** rcernin has quit IRC07:48
*** zhurong has joined #openstack-glance07:50
*** NostawRm has quit IRC08:10
*** NostawRm has joined #openstack-glance08:10
*** pdeore has quit IRC08:15
*** pdeore has joined #openstack-glance08:16
*** tesseract has joined #openstack-glance08:24
*** rcernin has joined #openstack-glance08:33
*** tshefi has joined #openstack-glance08:36
*** belmoreira has joined #openstack-glance08:37
*** pcaruana has quit IRC08:40
*** alexchadin has quit IRC08:42
*** e0ne has joined #openstack-glance09:39
*** btully has joined #openstack-glance09:45
*** namnh has joined #openstack-glance09:45
*** btully has quit IRC09:49
*** mvk has quit IRC09:54
*** arcolife has quit IRC10:02
amito Hi, our cinder CI is failing in the last couple of days. looked in the logs and it seems glance is failing consistently on "BackendException: Cannot find swift service endpoint : The request you have made requires authentication. (HTTP 401)". Any idea?10:04
kairatyou need to check glance_store settings10:06
kairatamito, was it broken recently?10:06
kairati pushed one patch to master recently but there was no release so..10:06
amitoDefine recently? It worked ok until early morning yeterday10:07
kairatamito, check glance_store settings and ensure you are using keystone v3 api10:07
kairati mean how long it has been broken?10:08
kairatamito, ^10:08
kairatah, got it10:08
amitoSince yesterday morning-noon (UTC+2). It's an automated Jenkins-zuul setup. brings up devstack and the builds are triggered by patch sets. About the usual.10:08
kairat check glance_store settings and ensure you are using keystone v3 api10:09
kairatthis errors says that there is no swift endpoint in service catalog10:10
amitoit should be under /etc/glance/glance-swift-store.conf ?10:10
kairatyep10:10
amitoauth_version = 310:10
amitoauth_address = http://172.16.82.212/identity/v310:10
kairatdo you have swift service in services&10:10
kairat?10:10
amitoI'll check, one sec10:10
kairatwhat is the service name10:11
kairatthe errors says you need to check service catalog10:11
kairatthen check glance_store settings10:11
kairatservice name, region, etc10:12
kairatand clarify why swift endpoint cannot be found10:12
amitowe have OVERRIDE_ENABLED_SERVICES, but I don't think the swift ones are listed there10:13
kairatit is your cloud specific config=)10:14
kairatso only keystone service list can help here10:14
*** arcolife has joined #openstack-glance10:14
kairatthe way it was broken says it is related to your conf10:15
kairatbecause there was no release recently for glance store10:15
kairatsoo10:15
kairatunless you are installing from master=)10:16
amitoI am bringing up my environment from devstack10:17
amitoI believe devstack takes from master...10:19
kairati don't think so10:19
kairatit is for glance_store10:20
kairatbot glance10:20
kairatIIUC it is release based10:20
kairatyou need to check your conf10:20
kairatsorry i have meeting10:20
kairathope i get right direction10:21
kairat*gave10:21
*** mvk has joined #openstack-glance10:22
*** namnh has quit IRC10:29
*** rcernin has quit IRC10:30
*** arcolife has quit IRC10:34
*** arcolife has joined #openstack-glance10:36
*** arcolife has quit IRC10:36
*** arcolife has joined #openstack-glance10:37
*** trungnv has quit IRC10:50
*** trungnv has joined #openstack-glance10:51
*** abhishekk has quit IRC10:56
*** udesale has quit IRC11:29
*** btully has joined #openstack-glance11:34
*** btully has quit IRC11:38
*** alexchadin has joined #openstack-glance11:42
bhagyashriskairat: Hi, Just want to disscuss regarding the patch https://review.openstack.org/#/c/524060/3/glance/api/v2/image_data.py11:47
kairatyep11:48
bhagyashrisAs There are two methods to create images using the current master:-11:48
bhagyashrisMethod A)11:48
bhagyashrisPOST /v2/images11:48
bhagyashrisPUT /v2/images/{image_id}/file11:48
bhagyashrisMethod B)11:48
bhagyashrisPOST /v2/images11:48
bhagyashrisPUT /v2/images/{image_id}/stage11:48
bhagyashrisPOST /v2/images/{image_id}/import11:48
bhagyashrisThe traditional image upload API (PUT /v2/images/{image_id}/file)11:48
bhagyashrisuses 'upload_image' policy which is same for11:48
bhagyashrisMethod B (POST /v2/images/{image_id}/import)11:48
bhagyashrisimage-create-via-import(new API for image create) API using the set_data() method https://github.com/openstack/glance/blob/master/glance/api/policy.py#L19311:48
kairatsorry11:49
kairati meant another thing11:49
bhagyashriskairat: so this set_data() method is common for both the import and upload case11:49
kairatif you look at the architecture of v211:49
bhagyashrisso that's why11:49
kairatyou can see it is layered11:50
bhagyashristhe policy is enforce at the controller11:50
kairatwith some pattern11:50
bhagyashrisyeah11:50
kairatso all policy checks better to have in policy layer11:50
kairateverytime i see different it breaks incapsulation IMO11:51
kairatis set_data is common11:52
bhagyashrisbut as the set_data() method where the policy is enforce is common for the cases import and upload11:52
bhagyashrisyup11:52
kairatit doesn't mean you need to use it in another layer11:52
kairatitis premature optimization11:52
kairatIMO11:52
bhagyashrisbut as in the set_data() method the policy name is hard coded and that is applied to the import case11:53
kairatwhat I really want is policy checks in policy proxy=)11:53
bhagyashrishttps://github.com/openstack/glance/blob/master/glance/api/policy.py#L19311:54
kairatsorry then you better split it somehow11:54
bhagyashriskairat: could you elaborate11:55
kairatif image feature implementers used the same method for import and upload11:56
kairatand we need different policies for them11:56
kairatwe need to use two different methods in proxies IMO11:57
kairatbecause it is different operation11:57
kairatOR use the same policy11:57
kairatit is just my opinion11:57
kairati am bit of context TBH11:57
kairat*Out of context11:57
bhagyashrisok. but  if we use two different methods in proxies then this will need lot of refactoring in the code because the set_data() is used in lot many places11:59
kairatbut in ideal world it would be better to have set_data() and set_data_async()11:59
bhagyashriskairat: so i have chose this approach to make it as simple12:00
kairatit all can be resolved12:00
kairati don't know what the others core says12:00
kairat*will say12:00
kairatbut this simplicity  will lead to big mess12:01
kairatIMO12:01
kairatinstead of layers we break encapsulation12:02
kairatif we use two different methods in proxies then this will need lot of refactoring in the code because the set_data() is used in lot many places12:02
bhagyashriskairat: yeah12:03
kairatthat ^ should have been done when we merged initial feature12:03
*** nicolasbock has joined #openstack-glance12:03
kairatbut as i said you can solve it for ewxample12:04
kairatwe can introduce set_data_sync and set_data_async12:04
kairatand one of this to be set_data12:05
kairatit is just my opinion, nevrermind=)12:05
kairatbut I feel tech debt here12:06
bhagyashriskairat: and also the set_data() is define in different modules like location.py, proxy.py, policy.py and notifier.py so this will also used so if create the  set_data_async() for the import case then in all this module we will need the implementation and that will the big change12:14
kairatso requires big change is not a reason for bad approach, yeah?12:15
kairatwe need trying to do the proper thing IMO12:16
bhagyashriskairat: yeah, i would also like to hear other cores opinion12:18
kairat++12:19
kairati am former core TBH12:19
kairatso nevermind =)12:19
bhagyashriskairat: ok. thank you for review and your inputs :)12:21
openstackgerritBhagyashri Shewale proposed openstack/glance master: Add 'import_image' policy for import image API  https://review.openstack.org/52406012:23
openstackgerritBhagyashri Shewale proposed openstack/glance master: Add 'stage_image' policy  https://review.openstack.org/52557812:23
*** alexchadin has quit IRC12:34
*** alexchadin has joined #openstack-glance12:35
*** zhurong has quit IRC12:36
*** alexchadin has quit IRC12:40
*** alexchadin has joined #openstack-glance12:41
*** ratailor has quit IRC12:43
*** mvenesio has joined #openstack-glance12:43
*** zhurong has joined #openstack-glance12:45
*** zhurong has quit IRC13:02
*** zhurong has joined #openstack-glance13:02
*** rosmaita has joined #openstack-glance13:03
mvenesioHi guys, i'm having some issues setting the glance-scrubber over SSL i'm getting this error "Can not get scrub jobs from queue: [SSL: UNKNOWN_PROTOCOL]"13:03
mvenesioanyone knows how to do it correctly ?13:04
*** alexchadin has quit IRC13:09
kairatmvenesio, afaik it just do not support ssl13:11
kairatwait a sec13:12
kairati will check the patch tht supposed to do it13:12
mvenesiokairat: thanks, thats weird because there're many options that seems to help to do it like https_ca_certificates_file = https_insecure or https_insecure = true13:14
mvenesiokeekz: but no one is working for me13:14
kairatsorry, i can't find this patch, i just remember that we needed port scrubber to requests library to support ssl13:19
kairatand it had not happened13:20
kairati can recommend to open a bug or find a similar one13:20
*** alexchadin has joined #openstack-glance13:21
mvenesiokairat: do you have some documentation, or bug report that inform this SSL issue with the scrubber ?13:21
kairatno13:21
*** links has quit IRC13:22
kairatit seems a good point to raise on glance meeting13:26
*** zhurong has quit IRC13:27
*** udesale has joined #openstack-glance13:32
jokke_mvenesio: there is lots of unrelated clutter in that scrubber config file due to automatic config generation :(13:36
jokke_I think kairat is right, it's not supported13:36
*** e0ne_ has joined #openstack-glance13:40
mvenesiojokke_: i understand, just that would be great to have some kind of bug report or blueprint to show to my client that is not supported.13:41
mvenesiojokke_: maybe i can create a new bug report and wait an answer13:41
kairati remember there was a patch13:42
kairatbut have no time to find it13:42
mvenesiokairat: no problem if you can do it later or tomorrow i'll be around here for a couple of days13:43
mvenesiokairat: thanks for your help13:43
*** e0ne has quit IRC13:43
jokke_mvenesio: yeah, makes sense ... the scrubber documentation is not really that helpful13:46
bhagyashrisHi all, can any help me How to define property protection roles for service user (like nova or cinder) in the property-protection-roles.conf? for specific property13:47
jokke_mvenesio: the scrubber is currently under quite a bit of refactoring ... not sure if the change merged yet13:47
bhagyashrisI mean I want to give the access to for example: xyz property to create, update and delete for admin user and service user13:48
rosmaitabhagyashris: does the service user have any particular roles in keystone? you could pick one of those13:49
rosmaitaor if it doesn't you could create one and assign it to the service user13:49
jokke_bhagyashris: the service user awareness has not been implemented into glance yet so unless oslo_policy can do it without us intervening/providing any extra info I think you can't13:50
*** arcolife has quit IRC13:52
rosmaitajokke_: i added a note about what's going on with python-glanceclient at the bottom of https://etherpad.openstack.org/p/glance-queens-Q213:53
jokke_rosmaita: oh damn13:59
rosmaitano kidding14:00
rosmaitai'm working on fixing the "legacy" tests now14:00
rosmaitabut i wasted a lot of time yesterday with the upstream json patch stuff14:01
jokke_and because the legacy is failing we can even block that from the requirements14:02
bhagyashrisrosmaita: ok. I have just check by creating the snapshot/image of the instance from nova and just checked the req.context coming from nova in the create image and it shows the req.context.roles as [u'anotherrole', u'member']14:02
bhagyashrisrosmaita: so is that mean another and member is service user roles14:03
rosmaitabhagyashris no, those are sample roles created in devstack14:05
rosmaitai think all the default users have them14:05
rosmaitaas a short term thing to do your testing, you could create a 'service_user' role in keystone and only assign it to the users you want14:06
rosmaitaor, you could ask in keystone channel what is the best way to recognize a service user14:06
bhagyashrisrosmaita: ohh ok.14:06
bhagyashrisrosmaita, jokke_: thank you :)14:07
jokke_rosmaita: do you have moment? trying to get some sense of this fecking mess again :P14:08
rosmaitawhich mess would that be? :)14:08
rosmaita(we have several)14:09
jokke_rosmaita: our favorite, onion layers14:09
jokke_so that's related to Abhishek's patch 52336614:11
jokke_I think that ImageProxy which our quota implements is just fundamentally broken14:14
rosmaitaok, give me a minute to pull up the patch14:15
*** abhishekk has joined #openstack-glance14:15
rosmaitafound it, give me a few min to look it over before hearing your concerns14:18
rosmaitajokke_ : ready to discuss14:24
jokke_so this is fecking horrible no matter how you look at it14:26
jokke_in our location code we implement one version of that set_data, that checks that our image data does not exceed the max configured image size14:27
jokke_with LimitingReader14:28
jokke_we do that same in our quota code to check that we don't exceed the data quota set14:28
jokke_these passes the exactly same ImageSizeLimitExceeded exception up the layers14:29
jokke_so if we go through that quota layer, currently we just get StorageQuotaFull exception which might be either the quota check or the ImageSizeCap, with Abhishek's patch we wil get the ImageSizeLimitExceeded to the upload call in image_data but we still don't know which one it was that failed14:31
rosmaitai think abhishekk 's patch will give us the correct exception though? is that right?14:33
jokke_nope14:33
rosmaitaoh14:33
jokke_that's my point14:33
rosmaitagive me a minute to look again14:33
jokke_if we go through the quota layer with that patch we will never get the quota full exception14:33
openstackgerritMerged openstack/glance_store master: Updated from global requirements  https://review.openstack.org/52500314:34
jokke_we will just get the image size limit exceeded exception but we still don't know if it came from quota or from the image size cap limiter14:34
abhishekkjokke_: if you check line #336 you will get QuotaFullException from their14:34
*** alexchadin has quit IRC14:35
jokke_abhishekk: that gets raised only if that limiting reader set on line 302 doesn barf first14:36
jokke_as in if we don't know the size of the data coming in14:36
abhishekkwhy we are catching ImageSizeLimitExceeded and throwing StorageQuotaFull which is quite confusing14:38
jokke_abhishekk: abhishekk because that is setting the limit to the quota left14:38
jokke_on the LimitingReader14:38
*** pdeore has quit IRC14:39
jokke_now the problem is that that LimitingReader is used twice once on that quota implementation and second time in our location code that checks the image size cap from configuration. Now regardless if we use the current or the proposed code we have no idea which one actually raised the exception14:40
*** udesale has quit IRC14:40
abhishekkohh14:41
jokke_because it's nesting the same function inside itself with two different limit values14:41
jokke_like said ... the shit is fundamentally broken there14:42
abhishekksorry, need to rush, please add your comments on the patch, I will have a look once reached home14:42
jokke_abhishekk: sure14:42
abhishekkjokke_: thank you14:42
*** abhishekk has quit IRC14:43
jokke_rosmaita: are you on board on that analysis?14:43
rosmaitai'm on board with the "shit is fundamentally broken" analysis14:43
rosmaitayeah, the limiting reader is designed to detecte image size limit exceeded, but it'14:44
rosmaitas being used to detect storage quota full14:44
rosmaitajeez14:45
rosmaitasomeone did a bad hack there, i hope it wasn't me14:45
jokke_rosmaita: yup ... you see, great java coding ... lets reuse and reimplement everything everywhere and not care about the consequencies14:45
rosmaitaso you are saying we should wrap the LimitingReader inside a QuotaLimitingReader to fix this?14:47
rosmaita:)14:47
jokke_rosmaita: so one way to fix this is that we pass the exception class to LimitingReader (and make sure those exceptions are initialized same way) so in the quota code we would do something like "data = utils.LimitingReader(data, remaining, exc=exception.StorageQuotaFull)"14:49
jokke_and the limiting reader would initialize and raise exc instead of hardcoded ImageSizeLimitExceeded14:49
*** udesale has joined #openstack-glance14:50
jokke_that way if we utilize it somewhere else we could define what we want it to throw at us when it barfs14:50
jokke_but yeah, multiple nested layers of the same object rewritten and no-one bothered to write functional tests for them to see that we ever get the excepton out we expect14:52
jokke_fecking proxyclasses14:52
*** gcb has quit IRC14:53
rosmaitaok, we need to think about this some more14:53
rosmaitabut yeah, there is bad mojo going on there14:53
*** gcb has joined #openstack-glance14:54
*** udesale has quit IRC14:59
jokke_rosmaita: there is couple of Abhishek's fix patches waiting for +2A if you have time15:07
rosmaitaok, will make that a priority for next few hours15:07
*** btully has joined #openstack-glance15:11
*** btully has quit IRC15:16
openstackgerritMerged openstack/glance master: Updated from global requirements  https://review.openstack.org/52537415:16
jokke_afk for ~20min15:16
*** udesale has joined #openstack-glance15:36
*** udesale has quit IRC15:41
*** udesale has joined #openstack-glance15:42
jokke_b15:47
*** openstackgerrit has quit IRC15:48
*** belmoreira has quit IRC16:04
*** udesale has quit IRC16:23
*** e0ne_ has quit IRC16:33
*** mvk has quit IRC16:42
*** tshefi has quit IRC17:10
*** mvk has joined #openstack-glance17:14
*** pcaruana has joined #openstack-glance17:22
*** pcaruana has quit IRC17:26
*** pcaruana has joined #openstack-glance17:27
*** pcaruana has quit IRC17:58
*** jose-phillips has quit IRC18:02
*** jose-phillips has joined #openstack-glance18:04
*** mvenesio has quit IRC18:16
*** mvenesio has joined #openstack-glance18:17
*** tesseract has quit IRC18:21
*** jose-phillips has quit IRC18:23
*** btully has joined #openstack-glance18:49
*** btully has quit IRC18:53
*** mvenesio has quit IRC19:10
*** jose-phillips has joined #openstack-glance19:20
*** jose-phillips has quit IRC19:24
*** jose-phillips has joined #openstack-glance19:27
*** kuzko has quit IRC19:54
*** kuzko has joined #openstack-glance19:56
*** kuzko has quit IRC20:04
*** kuzko has joined #openstack-glance20:14
*** e0ne has joined #openstack-glance20:25
*** gyee has joined #openstack-glance20:44
*** nicolasbock has quit IRC21:10
*** e0ne has quit IRC21:25
*** threestrands has joined #openstack-glance22:05
*** threestrands has quit IRC22:05
*** threestrands has joined #openstack-glance22:05
*** rcernin has joined #openstack-glance22:05
*** McClymontS has joined #openstack-glance22:32
*** McClymontS has quit IRC22:33

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!