Tuesday, 2015-12-01

*** takedakn has joined #openstack-glance00:01
*** jerrygb_ has joined #openstack-glance00:04
*** jerrygb has quit IRC00:05
*** belmoreira has quit IRC00:15
*** Kennan has joined #openstack-glance00:16
*** openstackstatus has quit IRC00:24
*** openstackstatus has joined #openstack-glance00:25
*** ChanServ sets mode: +v openstackstatus00:25
*** Kennan has quit IRC00:31
*** Kennan has joined #openstack-glance00:32
*** ducttape_ has quit IRC00:34
*** takedakn has quit IRC00:46
*** openstackgerrit has quit IRC01:22
*** openstackgerrit has joined #openstack-glance01:22
*** ducttape_ has joined #openstack-glance01:33
*** haomaiwang has joined #openstack-glance01:46
*** jerrygb_ has quit IRC01:51
*** jerrygb has joined #openstack-glance01:52
*** julim has joined #openstack-glance01:53
*** jerrygb has quit IRC01:55
*** haomaiwang has quit IRC02:01
*** haomaiwang has joined #openstack-glance02:01
*** mingdang1 has joined #openstack-glance02:25
*** mtanino has quit IRC02:30
*** haomaiwang has quit IRC02:33
*** houming has joined #openstack-glance03:04
*** terryyao has joined #openstack-glance03:05
*** lakshmiS has joined #openstack-glance03:19
*** wxy__ has joined #openstack-glance03:30
*** wxy has quit IRC03:33
*** houming has quit IRC03:35
*** ayoung has quit IRC03:35
*** Kennan has quit IRC03:39
*** links has joined #openstack-glance03:39
*** houming has joined #openstack-glance03:43
*** lakshmiS has quit IRC03:45
*** azbiswas has quit IRC03:46
*** srini has quit IRC04:02
*** kebray has joined #openstack-glance04:07
*** kebray has quit IRC04:09
*** kebray has joined #openstack-glance04:15
*** links has quit IRC04:17
*** kebray has quit IRC04:18
*** kebray has joined #openstack-glance04:20
openstackgerritMerged openstack/glance: Port _validate_time() to Python 3  https://review.openstack.org/25138504:20
*** ninag has quit IRC04:22
*** Kennan has joined #openstack-glance04:35
*** ianbrown has joined #openstack-glance04:41
*** houming has quit IRC04:42
*** azbiswas has joined #openstack-glance04:47
*** ducttape_ has quit IRC04:49
*** houming has joined #openstack-glance04:51
*** achanda has joined #openstack-glance04:51
*** azbiswas has quit IRC04:52
*** ducttape_ has joined #openstack-glance04:58
openstackgerritOpenStack Proposal Bot proposed openstack/glance: Updated from global requirements  https://review.openstack.org/25095305:00
*** gberginc has joined #openstack-glance05:02
*** azbiswas has joined #openstack-glance05:03
*** kebray_ has joined #openstack-glance05:03
*** kebray has quit IRC05:05
*** changbl has quit IRC05:07
*** haomaiwa_ has joined #openstack-glance05:08
*** kebray_ has quit IRC05:11
*** haomaiw__ has joined #openstack-glance05:11
*** kebray has joined #openstack-glance05:13
*** haomaiwa_ has quit IRC05:14
*** sgotliv_ has joined #openstack-glance05:24
*** haomaiw__ has quit IRC05:30
*** ianbrown has quit IRC05:32
*** jerrygb has joined #openstack-glance05:35
*** links has joined #openstack-glance05:35
*** houming has quit IRC05:39
*** gb21_ has joined #openstack-glance05:42
*** azbiswas has quit IRC05:47
*** azbiswas has joined #openstack-glance05:47
*** houming has joined #openstack-glance05:48
*** azbiswas has quit IRC05:52
*** ducttape_ has quit IRC05:52
*** jerrygb has quit IRC05:52
openstackgerritOpenStack Proposal Bot proposed openstack/glance_store: Updated from global requirements  https://review.openstack.org/25165106:05
*** dims has quit IRC06:09
openstackgerritOpenStack Proposal Bot proposed openstack/python-glanceclient: Updated from global requirements  https://review.openstack.org/25097006:09
tsufievkrotscheck: will take a look06:13
*** dims has joined #openstack-glance06:14
*** azbiswas has joined #openstack-glance06:17
*** abhishekk has joined #openstack-glance06:33
krotschecktsufiev: Thanks :)06:46
*** ducttape_ has joined #openstack-glance06:53
*** ducttape_ has quit IRC06:59
*** achanda_ has joined #openstack-glance07:00
*** achanda has quit IRC07:03
*** ducttape_ has joined #openstack-glance07:03
*** kebray has quit IRC07:15
*** ducttape_ has quit IRC07:20
*** azbiswas has quit IRC07:36
*** achanda_ has quit IRC07:36
*** azbiswas has joined #openstack-glance07:36
*** azbiswas has quit IRC07:41
openstackgerritKairat Kushaev proposed openstack/glance_store: Implement re-auth for swift driver  https://review.openstack.org/24929007:55
*** markus_z has joined #openstack-glance08:19
*** ducttape_ has joined #openstack-glance08:21
*** achanda has joined #openstack-glance08:23
*** MattMan has quit IRC08:23
*** MattMan has joined #openstack-glance08:24
*** ducttape_ has quit IRC08:25
openstackgerritleoyyoung proposed openstack/glance: Change the format of some inconsistent docstring  https://review.openstack.org/25022408:29
*** rcernin has joined #openstack-glance08:37
*** jerrygb has joined #openstack-glance08:52
*** belmoreira has joined #openstack-glance09:00
*** ducttape_ has joined #openstack-glance09:21
*** ducttape_ has quit IRC09:26
*** achanda has quit IRC09:38
openstackgerritMerged openstack/glance: Document contribution guidelines  https://review.openstack.org/23465309:42
*** houming has quit IRC09:50
*** jistr has joined #openstack-glance09:52
*** terryyao has quit IRC09:53
openstackgerritMerged openstack/python-glanceclient: Updated from global requirements  https://review.openstack.org/25097010:03
*** aix has joined #openstack-glance10:05
openstackgerritMerged openstack/glance_store: Updated from global requirements  https://review.openstack.org/25165110:16
*** ducttape_ has joined #openstack-glance10:22
*** ducttape_ has quit IRC10:28
*** jerrygb has quit IRC10:31
*** e0ne has joined #openstack-glance10:37
*** mdavidson has joined #openstack-glance10:48
*** mingdang1 has quit IRC10:48
*** e0ne has quit IRC10:51
*** azbiswas has joined #openstack-glance10:56
*** azbiswas has quit IRC11:01
*** marekd has joined #openstack-glance11:04
*** aix has quit IRC11:06
openstackgerritMerged openstack/glance: Updated from global requirements  https://review.openstack.org/25095311:16
*** jerrygb has joined #openstack-glance11:16
*** aix has joined #openstack-glance11:18
*** jerrygb has quit IRC11:19
*** flaper87 has quit IRC11:30
*** flaper87 has joined #openstack-glance11:30
tsufievkrotscheck, tested it, still one question (see in review)11:31
*** chlong has quit IRC11:35
*** chlong has joined #openstack-glance11:35
*** flaper87 has quit IRC11:39
*** cpallares has quit IRC11:40
*** cpallares has joined #openstack-glance11:46
*** flaper87 has joined #openstack-glance11:47
*** flaper87 has quit IRC11:47
*** flaper87 has joined #openstack-glance11:47
*** houming has joined #openstack-glance11:49
*** jdurgin has quit IRC11:53
*** jdurgin has joined #openstack-glance11:54
*** ninag has joined #openstack-glance11:56
*** sgotliv__ has joined #openstack-glance12:00
*** ninag has quit IRC12:01
*** ducttape_ has joined #openstack-glance12:01
*** sgotliv_ has quit IRC12:03
*** ducttape_ has quit IRC12:06
*** clayton has quit IRC12:06
*** zao_ has joined #openstack-glance12:07
*** zao has quit IRC12:08
*** zao_ is now known as zao12:08
*** achanda has joined #openstack-glance12:09
*** gb21_ has quit IRC12:11
*** clayton has joined #openstack-glance12:13
*** jerrygb has joined #openstack-glance12:20
openstackgerritMerged openstack/glance_store: Add list of supported stores to help  https://review.openstack.org/25145012:21
*** jaypipes has joined #openstack-glance12:22
*** kairat has quit IRC12:24
*** e0ne has joined #openstack-glance12:25
*** jerrygb has quit IRC12:27
*** kairat has joined #openstack-glance12:27
*** zao has quit IRC12:27
*** zao has joined #openstack-glance12:27
*** edmondsw has joined #openstack-glance12:29
*** achanda has quit IRC12:31
*** e0ne has quit IRC12:34
*** e0ne has joined #openstack-glance12:34
*** e0ne_ has joined #openstack-glance12:37
*** achanda has joined #openstack-glance12:38
*** e0ne has quit IRC12:39
*** abhishekk has quit IRC12:45
*** sileht has joined #openstack-glance12:47
openstackgerritKairat Kushaev proposed openstack/glance_store: Implement swift store connection manager  https://review.openstack.org/25185012:48
openstackgerritKairat Kushaev proposed openstack/glance_store: Implement re-authentication for swift driver  https://review.openstack.org/25185112:48
*** jaypipes has quit IRC12:49
openstackgerritMike Fedosin proposed openstack/glance: Implement trust support for api v2  https://review.openstack.org/24198612:49
*** jaypipes has joined #openstack-glance12:54
*** jerrygb has joined #openstack-glance13:01
*** julim has quit IRC13:01
*** achanda has quit IRC13:01
openstackgerritKairat Kushaev proposed openstack/glance_store: Implement re-authentication for swift driver  https://review.openstack.org/25185113:06
*** achanda has joined #openstack-glance13:06
*** ninag has joined #openstack-glance13:08
*** takedakn has joined #openstack-glance13:08
*** ducttape_ has joined #openstack-glance13:10
*** takedakn has quit IRC13:15
*** stevemar has quit IRC13:22
*** stevemar_znc has joined #openstack-glance13:23
*** takedakn has joined #openstack-glance13:26
*** takedakn has quit IRC13:29
*** ducttape_ has quit IRC13:33
zigoflaper87: Hi there! I saw you contributed to pyngus, which is a new global-requirement for Mitaka. Do you happen to remember what the runtime dependencies are for that one?13:41
zigoflaper87: Can I just do: python-qpid ?13:41
zigoOr python-qpid-proton is mandatory?13:41
flaper87zigo: it should be python-qpid-proton which is different from python-qpid13:41
zigoOk, thanks.13:42
flaper87zigo: if it gives you trouble you can ping me and/or kgiusti (he's on openstack-oslo)13:42
zigoWell, I got to upgrade qpid-proton in Sid, maintained by others, but it should be fine.13:43
flaper87zigo: awesome!13:43
*** links has quit IRC13:47
*** dshakhray has joined #openstack-glance13:48
*** mingdang1 has joined #openstack-glance13:55
*** e0ne_ has quit IRC13:55
*** e0ne has joined #openstack-glance13:55
*** achanda_ has joined #openstack-glance13:56
*** achanda has quit IRC13:59
buntingflaper87: ping14:01
flaper87bunting: brb, meeting14:01
flaper87bunting: shoot14:01
buntingJust to clarify with the deprecate execeptions, do you mean change the message?14:02
buntingthat is in the actual exceptions file14:02
*** ayoung has joined #openstack-glance14:05
*** Ctina has joined #openstack-glance14:25
*** Ctina is now known as ctina14:25
*** mclaren has joined #openstack-glance14:30
flaper87bunting: I meant, throw a warning14:30
mclarenrosmaita: ping14:30
rosmaitamclaren: pong14:31
mclarenI bet you'd like nothing more than to chat about image import?14:31
*** azbiswas has joined #openstack-glance14:32
flaper87mclaren: rosmaita whatever you discuss, please, report back on the spec.14:32
flaper87I'll try to participate but I have some backlog to go through now14:32
rosmaitaflaper87: will do14:33
rosmaitamclaren: did you get a chance to read through my comments (posted a few hours ago)14:33
mclarenlet me double check14:33
*** davideagnello has quit IRC14:34
*** haomaiwa_ has joined #openstack-glance14:35
*** azbiswas has quit IRC14:37
mclarenrosmaita: ok I've read them -- fairly quickly, so bear with me if I missed something14:38
mclarenquestion 1:14:39
mclaren> Image validation are very I/O and CPU intensiv14:39
openstackgerritMerged openstack/glance: Remove anyjson useless requirement  https://review.openstack.org/24540914:39
mclarenwhat validation is cpu intensive?14:39
rosmaitawell, we use xen14:39
rosmaitaactually may not be so bad14:39
rosmaitabut the packaging to compress the VHD into an OVA puts a load on the node14:40
mclarenOk, vhd to ova is conversion, not validation14:40
mclarenI've tried to make a distinction between when we modify the checksum and when we don't14:40
mclarenwhich operations which don't modify the checksum are I/O / cpu intensive?14:41
*** achanda_ has quit IRC14:41
rosmaitawell, i/o to get the thing to the node for processing, and then to get it off14:41
mclarenyou can tee the image into /var as you upload it into the backend, the opposite of what we do with caching14:42
mclarenyou only need to do that in the non nova-snapshot case14:42
mclarenis that a deal-breaker?14:42
rosmaitawell, i think we need to plan for the case where /var is full14:43
rosmaitai mean, you can have pretty big images14:43
rosmaitaand if you happen to get a few of those simultaneously, you're in trouble14:43
rosmaitaso i view the tee thing as an optimization, not the main implementation14:44
mclarenhmm, so you are saying put it in the backend so you can retry after /var is cleaned out?14:44
mclarenok, question 214:45
mclarenin the multi-tenant swift store how do you retry?14:45
rosmaitawell, if the image data is in the "normal" place in the backend, i'm not sure there's a problem?14:46
rosmaita(but i haven't much experience with multi-tenant store)14:46
mclarenis it a user initiated retry?14:46
rosmaitaok, gotcha14:46
rosmaitawell, what i see happening is this:14:47
rosmaita1 - user creates image record14:47
rosmaita2 - user uploads to /bikeshed14:47
mclarenin terms of big picture, I've been thinking of the no-swift case as economy, and when swift is there then you get fancy retries (both initial upload and post upload)14:47
rosmaita3 - (am going to add back the "import" call)14:47
rosmaita4 - behind the scenes, import call creates a task14:48
rosmaita5 - task is eventually processed when resources are available14:48
rosmaitaso not really a retry14:48
mclarenbut a user could repeat the import call to get a retry of that step14:49
*** mtanino has joined #openstack-glance14:50
rosmaitai guess, but what would the point of that be?14:50
rosmaita(it is morning here, have not had sufficient coffee yet)14:50
*** ducttape_ has joined #openstack-glance14:50
mclarenlol, its late here, not sure what my excuse is!14:50
rosmaitayour caffeine has worn off!14:50
mclarenwhat happens if the task fails?14:50
mclarensay the node it's running on dies14:51
mclarenhow does the user re-use the data from the backend?14:51
rosmaitaok, that's a good question14:52
rosmaitai think i have an answer14:52
rosmaitawith the swift-local method, no problem, it's not glance's data so not in the multi=tenant store14:53
mclarenI think in the swift is available case this all works naturally14:53
rosmaitawith the glance-local method, probably tough luck?14:53
rosmaitadata would disappear from multi-tenant store same as when a nova snapshot fails?14:53
rosmaitabecause the user didn't put the data into swift, they put it into glance14:54
rosmaitai need to think some more about this i guess14:54
mclarenBasically, I'm leaning towards keeping things simple in the glance-only case, and taking advantage of Swift when it's there to provide retries etc.14:54
mclarenI think in the glance only case we'll end up needing two blobs temporarily14:55
mclarenand I think that could turn out to be a headache14:55
mclareneg v1 updates14:55
mclarendisk usage quota14:56
mclarenexposing blob usage to users14:56
rosmaitai don't know that we need to expose blob usage to users -- what's the diff between it being in /var and being in backend as far as a user is concerned?14:57
mclarenshould it cound towards their disk usage quota?14:57
mclarenstorage usage14:57
rosmaitahow does that work now for multitenant store?14:59
mclarengood question. I guess we just add up the sizes we know about from the glance database15:00
openstackgerritKairat Kushaev proposed openstack/glance-specs: Prevention of Unauthorized errors in Swift driver  https://review.openstack.org/24868115:00
rosmaitahaving swift available would solve so many problems15:00
mclarenand I'm not sure if it's only active images we count15:00
rosmaitablob woudl be in user-space to begin with as an object, not an image15:00
openstackgerritDarja Shakhray proposed openstack/glance-specs: Add filters using an 'in' operator  https://review.openstack.org/24697115:01
*** mfedosin has joined #openstack-glance15:01
mclarenso you're saying just don't count it?15:02
rosmaitanot sure what i'm saying, actually15:03
mclarenyeah, it's a pain! It kind of feels like it would be an attempt to shoehorn two things into a space really only designed for one15:04
mclaren(there's only one size, checksum per image)15:04
*** achanda has joined #openstack-glance15:04
rosmaitamclaren: you're right15:04
rosmaitai don't have an answer ATM15:04
rosmaitathink you may have to re-raise this on the next patch set15:05
mclarenthat's fine. (Me neither)15:05
jokke_the bikeshed backend could be any supported (writeable) backend and the retry would be possible ... only case where the retry wouldn't be possible is when the bikeshed is file without shared fs between the node15:05
rosmaitahoping to get that up by wed evening so people have some time to look before thurs glance meeting15:05
*** mingdang1 has quit IRC15:05
mclarenI actually don't really care much about retry where is so hard to implement. You can retry by resending the request ... it's the cloud.15:06
jokke_and that implementation would tie the worker also to the api node15:06
rosmaitajokke_: i was hoping not to have a separate bikeshed backend, just use the same space in the backend where the image would be stored15:06
jokke_rosmaita: that's definitely one option, but how do you deal with quota?15:06
mclarenjokke I don't think reusing the same location will work.15:07
rosmaitajokke_: it shouldn't affect quota? (thinking)15:07
mclarenyou need two places temporarily15:07
jokke_Will you penalize customer who has 50gb of quota and trying to import 30gb image because the initial blob and the actual image will not fit to the quota15:07
*** dims has quit IRC15:08
rosmaitajokke_: no, what you have is: 1 - user uploads 30gb image and glance puts in backend, 2- when ready, glance gets it out of backend and processes image, 3- glance replaces initial blob wiht processed image15:08
jokke_that's why thinking that glance/deployer should "own" the bikeshed and customer the image15:08
rosmaitaso the user wouldn't have 60gb taken up, just 30 gb15:09
jokke_rosmaita: that will kill the retry on every case if something fails during that upload15:09
mclarenright, but if there's a failure while replacing you can't retry afterwords15:09
jokke_as the original blob nor the image is available for retry15:09
mclarenand then of course v1 needs to be made bikeshed aware for delete etc15:10
rosmaitaso it would be same situation as a nova snapshot ... if snapshot at time t fails, you have to start over15:10
mclarenand if someone comes along and imports from swift before a retry you need to handle the bikeshed in that case15:11
mclarenI just see the code being peppered with [now we need to handle the bikeshed in this case...]15:12
rosmaitayeah, that's why i want to avoid having an actual bikeshed15:12
mclarenAnd in the swift case that happens more naturally15:13
rosmaitayes, the swift case is quite nice15:13
rosmaitabut apparently not very popular15:13
mclarenI'm assuming we'll add something to allow import from swift as a call when swift is available?15:14
mclarenlike copy-from-esque15:14
mclarenso when swift is there you only upload your bits in once, so a retry is less costly.15:15
mclarenI'm don't feel we need to do huge contortions to have that behaviour when swift isn't there15:16
*** mfedosin has quit IRC15:16
rosmaitawell, maybe we allow limited copy-from, so someone with swift might allow copy-from only from their local swift?15:16
rosmaitaless fastidious people can allow copy-from from arbitrary urls15:17
mclarenright, the interface can be whatever we decide, but you get retries when Swift is there15:18
rosmaitaexcept the discussion at the summit was copy-from is a non-starter15:18
mclarenI'm talking about import from swift being an additional option15:18
rosmaitabecause it requires the importer to have a web server to host the blob15:19
mclarenie defcore compute = upload to /file, defcore platform = upload to /file, or via swift15:19
rosmaitaso basically, 'swift-local' ?15:19
mclarenthe problem we've been asked to solve isn't a retry problem15:20
mclaren(I need to go in a few mins)15:21
*** sabari has joined #openstack-glance15:21
rosmaitame too ... ok, what i will do is revise back to PS 5 but without the image resource15:21
rosmaitai think that still has glance-local and swift-local15:21
rosmaitai mean, without the import resource!15:22
rosmaitai think we should keep images15:22
mclarenI'm not sure we need swift-local in the spec, perhaps just something to say that we envisage a complimentary/additional option for defcore platform15:22
rosmaitai may leave it in just as an example of what we need to accommodate15:23
mclarenas you wish15:23
rosmaitaor something ... anyway, you can comment on the spec!15:23
rosmaitaand this channel is logged, so this convo won't be lost15:23
rosmaitaok, have a good evening15:23
mclarenone question15:24
mclarendo you think we need to preserve the checksum?15:24
mclarenwhere no conversion is being done15:24
mclaren(ie unless the user flags it's ok in some way)15:24
rosmaitanot sure about that15:25
mclarenok, how about breaking existing clients who'll get a 40315:25
mclarena big deal or not?15:25
mclaren(if we change /file)15:26
rosmaitai think we should not change /file15:26
rosmaitai think we will take a strong stand that we can't break backward compatibility15:26
rosmaitabut that doesn't answer your question15:27
rosmaitaend-users will get 403 if the deployer wants them to go through the validation chain15:27
mclarenok, Flavio says (possibly correctly on paper) that changing to /blob and disabling the old method is non-breaking15:28
rosmaitabut that could always happen even now, by policy?15:28
rosmaitai'm with Flavio on that one15:28
rosmaita403 is a possible response to everything, i'd think15:29
mclarenok but everyone uploading today will start getting 403s and needing a new glance client. If our goal is compatability...15:29
rosmaitaeverybody needs a new glance client!15:29
*** e0ne has quit IRC15:29
rosmaitayeah, but you'd need a new client to take advantage of any new API stuff15:30
mclarenuploading an unmodified image is arguably not new stuff15:30
rosmaitayes, but the ability to upload has always been policy based15:31
rosmaitaif hte deployer changes the policy, the end user must adapt15:31
rosmaitait's up to the deployer to make it a smooth transition15:31
rosmaitaanyway, you can bring this up on the spec15:32
rosmaitait's a complication i don't want to think about ATM!15:32
mclarenSure. I just think in practice a 'legal' breaking change can have the same impact in practice as a 'non-legal' one.15:33
mclarenok, thanks for your time Brian15:34
*** belmoreira has quit IRC15:35
*** ninag has quit IRC15:36
rosmaitamclaren: np ... you definitely have a point, i just don't want to deal with it!15:36
flaper87nikhil: rosmaita mclaren jokke_ hemanthm sabari PPL: I'm about to cut m-1 and I wanted to know if there's anything we should absolutely, totally, entirely, have15:45
*** achanda has quit IRC15:45
flaper87note that this is the first milestone release so, it's not really a problem if we miss one or two patches15:46
*** e0ne has joined #openstack-glance15:47
*** aix has quit IRC15:49
jokke_flaper87: not that I know at least15:52
flaper87jokke_: thanks15:53
*** dims has joined #openstack-glance15:54
hemanthmflaper87: nothing comes to my head15:56
*** vijendar has joined #openstack-glance15:56
flaper87hemanthm: thanks15:57
openstackgerritMerged openstack/python-glanceclient: Change man page examples to correlate default v2  https://review.openstack.org/24252315:59
openstackgerritMerged openstack/python-glanceclient: Update set of wanted commands in read-only test  https://review.openstack.org/25082915:59
openstackgerritKairat Kushaev proposed openstack/glance_store: Implement re-authentication for swift driver  https://review.openstack.org/25185116:02
*** achanda has joined #openstack-glance16:03
*** achanda has quit IRC16:03
*** avarner has joined #openstack-glance16:04
*** rcernin has quit IRC16:04
*** kebray has joined #openstack-glance16:05
*** gb21_ has joined #openstack-glance16:06
*** dave-mcnally has joined #openstack-glance16:07
*** dims_ has joined #openstack-glance16:08
*** changbl has joined #openstack-glance16:09
*** dims has quit IRC16:10
*** achanda has joined #openstack-glance16:11
*** ativelkov has quit IRC16:11
*** tsufiev has quit IRC16:11
*** davideagnello has joined #openstack-glance16:13
dave-mcnallynikhil: thanks for your comments on https://review.openstack.org/#/c/249282/, you mentioned adding release notes. I'm not sure of the formatting but I guess that change should be made to glance/releasenotes/source/unreleased.rst? Or am I better off adding a release notes impact comment to the commit and changing the release notes in a future patch?16:13
*** kebray has quit IRC16:16
*** avarner has quit IRC16:17
*** ativelkov has joined #openstack-glance16:20
*** azbiswas has joined #openstack-glance16:21
*** TravT has quit IRC16:22
*** azbiswas has quit IRC16:22
*** TravT has joined #openstack-glance16:25
*** changbl has quit IRC16:27
*** tsufiev has joined #openstack-glance16:27
*** r-daneel has joined #openstack-glance16:27
*** kebray has joined #openstack-glance16:28
*** kebray has quit IRC16:28
*** kebray has joined #openstack-glance16:28
*** spzala has joined #openstack-glance16:29
*** dims has joined #openstack-glance16:30
*** azbiswas has joined #openstack-glance16:31
*** dims_ has quit IRC16:33
openstackgerritDrew Varner proposed openstack/glance_store: Add reno for release notes management in glance_store  https://review.openstack.org/25156116:39
*** changbl has joined #openstack-glance16:42
nikhilflaper87: nothing from me16:44
flaper87nikhil: thanks16:45
openstackgerritdave-mcnally proposed openstack/glance: Add the Docker container format  https://review.openstack.org/24928216:47
nikhildave-mcnally: hi, I would say use this keystone guideline http://docs.openstack.org/developer/keystone/developing.html#release-notes . THe process is same in glance though we don't have the documentation, it's literally the same.16:47
stevemar_zncnikhil: we're porting most of those to the project team guide16:48
* stevemar_znc looks for link16:48
stevemar_zncnikhil: http://docs.openstack.org/project-team-guide/release-management.html#managing-release-notes16:50
nikhilstevemar_znc: wow, that's good documentation. thanks16:51
* flaper87 is proud of the work that has been done w/ PTG16:53
*** houming has quit IRC16:58
dave-mcnallynikhil, stevemar_znc: that's great thanks16:59
*** kebray has quit IRC17:00
*** haomaiwa_ has quit IRC17:00
*** dave-mcnally is now known as davemc_afk17:01
*** kebray has joined #openstack-glance17:05
*** links has joined #openstack-glance17:08
*** dshakhray has quit IRC17:10
*** stevemar_znc is now known as stevemar17:25
*** ayoung has quit IRC17:28
*** e0ne has quit IRC17:30
sabariflaper87 Just got back from vacation - go m1 from my side :)17:39
*** avarner has joined #openstack-glance17:39
flaper87sabari: awesome17:40
flaper87oh, btw, I have to add you to cores now17:40
*** achanda has quit IRC17:45
*** gb21_ has quit IRC17:51
*** achanda has joined #openstack-glance17:52
*** jistr has quit IRC17:54
*** ayoung has joined #openstack-glance18:01
*** kebray has quit IRC18:02
*** ayoung_ has joined #openstack-glance18:02
*** ayoung has quit IRC18:03
*** ayoung_ is now known as ayoung18:03
sabariflaper87 awesome :D18:08
*** mfedosin has joined #openstack-glance18:09
kragnizsabari: congrats!18:13
*** kebray has joined #openstack-glance18:13
sabarithanks kragniz!18:14
*** avarner has quit IRC18:17
*** kebray has quit IRC18:17
*** mfedosin has quit IRC18:21
*** links has quit IRC18:24
*** markus_z has quit IRC18:24
*** mancdaz has quit IRC18:26
*** azbiswas has quit IRC18:27
*** mancdaz has joined #openstack-glance18:27
*** dshakhray has joined #openstack-glance18:30
*** rcernin has joined #openstack-glance18:38
*** alex_xu has quit IRC18:48
*** kebray has joined #openstack-glance18:49
*** alex_xu has joined #openstack-glance18:52
*** kebray has quit IRC18:52
*** kebray has joined #openstack-glance18:54
*** kebray has quit IRC18:58
openstackgerritFlavio Percoco proposed openstack/glance: Remove version from setup.cfg  https://review.openstack.org/25202119:02
flaper87sabari: done, please check you have +2 powers19:03
openstackgerritMerged openstack/python-glanceclient: Add ordereddict requirement for py26 tests  https://review.openstack.org/25075119:08
*** vijendar has quit IRC19:08
sabariflaper87 Yey! Just checked.19:24
flaper87sabari: thank *you*19:25
flaper87now, don't #$@#@ this up </joke>19:25
sabariflaper87 hahaha - sure i won't :D19:25
*** avarner has joined #openstack-glance19:28
*** avarner_ has joined #openstack-glance19:35
*** avarner has quit IRC19:35
*** avarner_ has quit IRC19:35
*** avarner has joined #openstack-glance19:36
*** julim has joined #openstack-glance19:36
*** kebray has joined #openstack-glance19:38
*** kebray has quit IRC19:39
*** cdelatte has quit IRC19:44
*** cdelatte has joined #openstack-glance19:45
*** azbiswas has joined #openstack-glance19:45
*** burgerk has joined #openstack-glance19:46
*** kebray has joined #openstack-glance19:46
*** azbiswas_ has joined #openstack-glance19:46
*** kebray has quit IRC19:48
*** cdelatte has quit IRC19:49
*** cdelatte has joined #openstack-glance19:50
*** azbiswas has quit IRC19:50
*** vijendar has joined #openstack-glance20:18
*** spzala has quit IRC20:19
*** haypo has joined #openstack-glance20:27
haypolifeless: here. i'm not really interested to work on the random unit test failure since it's very difficult to reproduce it, so i bet that nobody will notice it in the CI20:28
lifelesshaypo: it triggers about 3% of the time20:28
lifelesshaypo: thats more than enough to cause havoc in CI20:29
haypolifeless: it's a non-zero reproduction rate :-/20:29
lifelesshaypo: anyhow, have you filed a bug # for it ?20:29
haypolifeless: my bet is it's race condition in the unit test. something mocks the HTTP server with a request handler catching any URL, but the mock is not reverted before the "nonexistent" test is run20:29
haypolifeless: no, since i almost have no information about it20:30
haypolifeless: oh by the way, i would be very interested to know if it's specific to python 3 or not20:30
haypolifeless: i guess that it does not depend on the python major version20:30
lifelesshaypo: I believe it is20:30
lifelesshaypo: I ran the reliable reproducer I have overnight with no failures on 2.720:31
lifelessI'm down to < 150 tests to reproduce20:31
haypolifeless: would you like to open a bug report?20:31
lifelessI didn't want to duplicate20:31
lifelessI presume I need the patch to trigger this? its not in master?20:32
lifelessor the porting work isn't in master or something?20:32
lifelesshaypo: basically I'm just using bisection20:32
lifelesshaypo: I'll write up a post describing it after I guess20:32
haypolifeless: https://review.openstack.org/#/c/250083/ was approved, but the new patch set has no vote yet20:33
haypolifeless: the change was not approved because another change introduced a new python 3 issue. i wrote a different patch to fix the new python 3 issue, this change is already merged20:37
lifelessflaper87: around? I'd like to ask about the lockutils-wrapper thing20:41
lifelesshaypo: https://bugs.launchpad.net/glance/+bug/152175620:42
openstackLaunchpad bug 1521756 in Glance "race/python3 issue" [Undecided,New]20:42
haypolifeless: thanks20:44
haypolifeless: you cannot remove a test of your list if the test suite doesn't fail20:49
haypolifeless: but you can use a smaller list if your reproduce the failure20:49
haypolifeless: for your bisection ;)20:49
haypolifeless: is testr able to handle such race condition?20:50
lifelesshaypo: analyze-isolation automates bisection, but you can't use it (today) with one-in-N failures20:50
lifelessso I'm just doing what analyze-isolation does by hand20:50
haypoah, you replied :-) no, it doesn't20:50
lifeless(because it doesn't understand probabalistic failures20:50
*** achanda has quit IRC20:50
lifelessa fairly simple --min-passes=N would solve it I think20:51
lifelesse.g. require 200 passes before considering it non-failing20:51
lifelessalternatively, it could try both branches alternately until one fails20:51
lifelessprobably faster to do that, but needs more code changes20:51
haypomy manual algorithm: pick a random set of  tests20:51
hayporun it20:51
haypoif it fails, bisect from this list20:51
haypootherwise, pick another set of random tests20:52
lifelesshaypo: ouch - thats random sort :)20:55
lifelesshaypo: given 300 tests, if its just one causing it, then you need a phenomenal number of probes to reduce20:55
lifeless[the starting point was 525 tests, trivially reduced to 306 by order of execution20:56
haypolifeless: in my experience, at the end, you only need two tests20:57
hayposo the problem was to find this test on 2600+ tests :)20:57
haypoi had technical issues. when i used  testtools.run directly, i got errors, whereas i didn't get these errors with testr20:58
haypoit's because testr loads all tests and _then_ executes them (i guess)20:58
lifelesshaypo: probably due to not using the lockutils-wrapper20:58
haypoi worked around the issue with an "import_all.py" script :)20:58
lifelesshaypo: no, if subunit.run can execute the tests, testtools.run can as well20:58
haypolifeless: nope. it was an issue on oslo.log20:58
lifelesssubunit.run is a subclass20:58
lifelesspossibly you switched from discover based loading to name based loading20:59
haypoit's a common issue. depending on the import order and execute order, you get different result :)20:59
lifelesssure, but subunit.run is a subclass of testtools.run20:59
lifelessits identical for those bits, given the same parameters20:59
lifelessI don't dispute you had an error21:00
lifelessjust giving you the data about where it could be happening21:01
lifelesshaypo: the very first time it happens you can extract only the worker specific tests in order to make your starting point much smaller21:01
lifelesshaypo: the recipe for that is on the testr wiki page21:01
*** changbl has quit IRC21:12
*** ctina_ has joined #openstack-glance21:22
*** ctina has quit IRC21:26
*** ctina_ has quit IRC21:27
*** harlowja has quit IRC21:27
haypolifeless: i reproduce the fail with your short list of 19 tests. cool21:27
*** harlowja has joined #openstack-glance21:28
haypolifeless: i will continue to bisect manually :)21:28
lifelessdown to 3 tests21:28
haypoyour bisection technic looks to be efficient21:29
haypolifeless: what are the 3 tests?21:29
lifelessrefresh the bug page // check your mail21:29
lifelessits running another step now21:29
*** mancdaz has quit IRC21:31
*** rcernin has quit IRC21:31
*** jerrygb has quit IRC21:32
*** mancdaz has joined #openstack-glance21:32
haypolifeless: FYI there are two "nonexistant" unit tests. once, i saw the two failed21:35
haypolifeless: once, only one failed. once, it was the other :)21:35
lifelessthere may be multiple other races21:35
lifelessin fact you pastebinned me a failure with both in it21:36
lifelessbut I figure we'll find one clear root cause in all probability21:36
*** avarner has quit IRC21:36
lifelessok so21:36
haypothe two unit test checks that an invalid HTTP url gives an error 40421:36
lifelessis enough to trigger it21:37
lifelessnow to try just the test on its own21:37
haypolifeless: how do you run tests?21:41
haypolockutils-wrapper python setup.py testr --slowest --testr-args='glance.tests.unit.async.flows.test_import.TestImportTask.test_import_flow_no_import_flows glance.tests.unit.v1.test_api.TestGlanceAPI.test_upload_image_http_nonexistent_location_url'21:41
lifelesstriggered the failure on its own21:41
haypo^^ i'm trying this21:41
lifelesstestr run --load-list current-set --until-failure21:42
lifelessI have moved lockutils-wrapper into .testr.conf21:42
haypolifeless: do you mean that you fail to reproduce the race without 22:42 < lifeless> I have moved lockutils-wrapper into .testr.conf21:43
haypolifeless: do you mean that you fail to reproduce the race without lockutils-wrapper?21:43
lifelessno, I moved it so I didn't have to have it in my way21:44
lifelessI need to talk to glance folk to understand why its where it is in tox.ini21:44
haypolifeless: 'python -m testtools.run  test1 test2' alone is unable to reproduce the failure21:51
haypo(i ran a loop in shell, run 6 times in parallel)21:51
haypo(during 5 minutes)21:51
*** kebray has joined #openstack-glance21:59
haypolifeless: ok, using testr run --parallel, i'm able to reproduce the failure with only the 2 tests22:00
lifelesshaypo: I can reproduce it with just the one test22:05
lifelesshaypo: you probably need to use discover or else things won't load well22:06
lifelesspython -m testtools.run discover -t ./ ./glance/tests  --load-list filenamehere22:06
* jokke_ is reading backlog22:06
haypolifeless: things load well22:07
lifelesshaypo: I'm pretty sure that in some situations they won't, because the loader won't do parameterisation and so on22:07
haypolifeless: hum....... at the end, i'm able to reproduce the failure with:22:07
haypotestr run --until-failure testtools.run glance.tests.unit.v1.test_api.TestGlanceAPI.test_upload_image_http_nonexistent_location_url22:07
lifelesshaypo: this is a upstream unittest behaviour22:07
haypowhen my system load is high22:07
lifeless20 runs and I reproduced it with testtools22:08
lifelesswhile true; do python -m testtools.run discover -t ./ ./glance/tests  --load-list list-file; done22:08
lifelesswhere list-file contains22:08
lifelessanyhow, you've reproduced it, cool22:08
lifelessso its not a cross-test interaction22:08
lifelessoh ugh, it uses tearDown22:10
lifelessthat explains the process leaks22:10
lifelessjokke_: hi22:10
lifelessjokke_: so this lockutils-wrapper22:10
lifelessjokke_: whats it for?22:10
jokke_lifeless: I got there ... I have absolutely no idea22:10
haypolifeless: does "testr run --until-failure" create a fresh process at each run?22:11
lifelessjokke_: ok, so I'll put up a patch to make it saner22:11
lifelesshaypo: yes22:11
haypolifeless: ok22:11
haypohigh system load is not enough to reproduce the issue22:12
jokke_lifeless: seems like sdague added that in January ... he might even remeber why22:12
*** dshakhray has quit IRC22:12
jokke_nope ... he just changed that line22:13
haypolifeless: "20 runs and I reproduced it with testtools" hum. i'm running 'python -m testtools.run glance.tests.unit.v1.test_api.TestGlanceAPI.test_upload_image_http_nonexistent_location_url' and i'm unable to reproduce the issue22:15
jokke_ok so ZhiYan added that there when moved to oslo concurrency ~ year ago22:15
haypoi'm running this command in shell loop, 7 shells in parallel. it doesn't fail22:15
jokke_and he is not around anymore so I doubt we will get light into the reasoning22:15
haypoi also have 6 python -c 'while 1: pass' to burn the cpu22:15
haypolifeless: it looks like testtools discover does something magic :-D22:15
lifelesshaypo: yes, it does22:16
jokke_lifeless: if you have good idea what it's doing and how to improve, go for it. Just please, don't save the words in the commit message. They are cheap22:16
lifelesshaypo: 'python -m unittest discover' and 'python -m unittest testname' do not do the same thing for loading a test22:16
openstackgerritlifeless proposed openstack/glance: Move lockutils-wrapper into .testr.conf  https://review.openstack.org/25208622:17
lifelesshaypo: that said I don't see any load_tests hooks, so it shouldn't be a substantial difference here22:17
*** burgerk has quit IRC22:18
lifelesshaypo: I think the difference is more likely our machines22:18
lifelesshaypo:  / python versions (I'm on 3.4 for instance)22:18
lifelesshaypo: I have no reason to think load is part of the story22:18
haypoin my experience, race condition behave differently depending on the system load22:19
haypoi'm running fedora 23 with python 3.4.322:19
lifelesshaypo: right, but thats an assumption, we don't know that its a race, we only know its intermittent22:19
*** harlowja has quit IRC22:19
lifelesshaypo: I'm not saying you are wrong, just that we don't know yet22:19
*** ayoung has quit IRC22:21
*** harlowja has joined #openstack-glance22:21
haypolifeless: hat is ./glance/tests in '-m testtools.run discover -t ./ ./glance/tests  --load-list tests' command?22:23
haypowhat* is22:24
lifelessits the starting path for discovery22:26
haypomy reliable scenario to reproduce the bug: run 'while true; do python -m testtools.run discover -t ./ ./glance/tests  --load-list tests || break; done' command in 6 shell22:27
lifelessthats progress22:27
*** kebray has quit IRC22:27
openstackgerritlifeless proposed openstack/glance: Stop using tearDown in v1/test_api.py.  https://review.openstack.org/25209122:29
haypopartial python traceback (i added the raised exception ;-)) http://paste.alacon.org/3899022:29
lifelessjokke_: https://review.openstack.org/252086 and https://review.openstack.org/252091 may be of interest22:29
*** kebray has joined #openstack-glance22:31
*** jerrygb has joined #openstack-glance22:32
*** jerrygb has quit IRC22:37
jokke_I'll have a look22:41
*** edmondsw has quit IRC22:44
*** davideagnello has quit IRC22:45
*** davideagnello has joined #openstack-glance22:45
*** davideagnello has quit IRC22:48
*** davideagnello has joined #openstack-glance22:48
*** davideag_ has joined #openstack-glance22:50
*** r-daneel has quit IRC22:52
*** kebray has quit IRC22:53
*** davideagnello has quit IRC22:53
haypolifeless: "except Exception: return 0". wait, what? :-p22:55
haypolifeless: don't worry, there is a TODO :-D https://github.com/openstack/glance_store/blob/master/glance_store/_drivers/http.py#L17622:56
*** harlowja has quit IRC22:57
haypolifeless: i guess that the client sends an HTTP request to the server which has just been started, but the server is not fully ready yet22:58
haypolifeless: ooook. i'm now able to reproduce the bug each time with a diff of 1 line :-p23:00
haypoi modified the HTTP server to exit immediatly when it gets a request23:01
*** harlowja has joined #openstack-glance23:01
haypolifeless: to be clear, it's completly unrelated to pytohn 323:02
haypoi'm probably just the first one which noticed this random failure23:02
haypobecause i ran tests too many times :-D23:02
haypolifeless: ok, thanks for motivating me to continue to analyze this bug :-p23:09
*** terryyao has joined #openstack-glance23:09
haypolifeless: even if the bisection was funny: from 2600 tests, the final list is a single test :-D23:09
haypoi just didn't try to run the single test alone in a loop23:09
*** TravT has quit IRC23:10
haypoi probably had issues to have a reliable scenario to reproduce the issue because it's a race condition. so it depends on the system load23:10
haypoit's a race betwen the client and the server ;)23:10
haypolifeless: i completed https://bugs.launchpad.net/glance/+bug/152175623:10
openstackLaunchpad bug 1521756 in Glance "race condition in test_upload_image_http_nonexistent_location_url: glance_store doesn't handle properly invalid HTTP reply" [Undecided,New]23:10
haypolifeless: good night & thanks again23:10
lifelesshaypo: good night!23:11
*** TravT has joined #openstack-glance23:12
*** TravT_ has joined #openstack-glance23:12
*** kebray has joined #openstack-glance23:16
*** TravT has quit IRC23:16
*** kebray has quit IRC23:24
*** david-lyle has quit IRC23:25
*** TravT_ is now known as TravT23:27
*** darrenc is now known as darrenc_afk23:31
*** e0ne has joined #openstack-glance23:34
*** kebray has joined #openstack-glance23:34
*** kebray has quit IRC23:35
*** kebray has joined #openstack-glance23:36
*** e0ne has quit IRC23:36
*** kebray has quit IRC23:36
*** e0ne has joined #openstack-glance23:36
*** ayoung has joined #openstack-glance23:37
*** e0ne_ has joined #openstack-glance23:39
*** e0ne has quit IRC23:41
*** e0ne has joined #openstack-glance23:49
*** e0ne_ has quit IRC23:51
*** darrenc_afk is now known as darrenc23:52
*** e0ne_ has joined #openstack-glance23:52
*** e0ne has quit IRC23:54
*** Kennan has quit IRC23:55
*** Kennan has joined #openstack-glance23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!