| *** takedakn has joined #openstack-glance | 00:01 | |
| *** jerrygb_ has joined #openstack-glance | 00:04 | |
| *** jerrygb has quit IRC | 00:05 | |
| *** belmoreira has quit IRC | 00:15 | |
| *** Kennan has joined #openstack-glance | 00:16 | |
| *** openstackstatus has quit IRC | 00:24 | |
| *** openstackstatus has joined #openstack-glance | 00:25 | |
| *** ChanServ sets mode: +v openstackstatus | 00:25 | |
| *** Kennan has quit IRC | 00:31 | |
| *** Kennan has joined #openstack-glance | 00:32 | |
| *** ducttape_ has quit IRC | 00:34 | |
| *** takedakn has quit IRC | 00:46 | |
| *** openstackgerrit has quit IRC | 01:22 | |
| *** openstackgerrit has joined #openstack-glance | 01:22 | |
| *** ducttape_ has joined #openstack-glance | 01:33 | |
| *** haomaiwang has joined #openstack-glance | 01:46 | |
| *** jerrygb_ has quit IRC | 01:51 | |
| *** jerrygb has joined #openstack-glance | 01:52 | |
| *** julim has joined #openstack-glance | 01:53 | |
| *** jerrygb has quit IRC | 01:55 | |
| *** haomaiwang has quit IRC | 02:01 | |
| *** haomaiwang has joined #openstack-glance | 02:01 | |
| *** mingdang1 has joined #openstack-glance | 02:25 | |
| *** mtanino has quit IRC | 02:30 | |
| *** haomaiwang has quit IRC | 02:33 | |
| *** houming has joined #openstack-glance | 03:04 | |
| *** terryyao has joined #openstack-glance | 03:05 | |
| *** lakshmiS has joined #openstack-glance | 03:19 | |
| *** wxy__ has joined #openstack-glance | 03:30 | |
| *** wxy has quit IRC | 03:33 | |
| *** houming has quit IRC | 03:35 | |
| *** ayoung has quit IRC | 03:35 | |
| *** Kennan has quit IRC | 03:39 | |
| *** links has joined #openstack-glance | 03:39 | |
| *** houming has joined #openstack-glance | 03:43 | |
| *** lakshmiS has quit IRC | 03:45 | |
| *** azbiswas has quit IRC | 03:46 | |
| *** srini has quit IRC | 04:02 | |
| *** kebray has joined #openstack-glance | 04:07 | |
| *** kebray has quit IRC | 04:09 | |
| *** kebray has joined #openstack-glance | 04:15 | |
| *** links has quit IRC | 04:17 | |
| *** kebray has quit IRC | 04:18 | |
| *** kebray has joined #openstack-glance | 04:20 | |
| openstackgerrit | Merged openstack/glance: Port _validate_time() to Python 3 https://review.openstack.org/251385 | 04:20 |
|---|---|---|
| *** ninag has quit IRC | 04:22 | |
| *** Kennan has joined #openstack-glance | 04:35 | |
| *** ianbrown has joined #openstack-glance | 04:41 | |
| *** houming has quit IRC | 04:42 | |
| *** azbiswas has joined #openstack-glance | 04:47 | |
| *** ducttape_ has quit IRC | 04:49 | |
| *** houming has joined #openstack-glance | 04:51 | |
| *** achanda has joined #openstack-glance | 04:51 | |
| *** azbiswas has quit IRC | 04:52 | |
| *** ducttape_ has joined #openstack-glance | 04:58 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/glance: Updated from global requirements https://review.openstack.org/250953 | 05:00 |
| *** gberginc has joined #openstack-glance | 05:02 | |
| *** azbiswas has joined #openstack-glance | 05:03 | |
| *** kebray_ has joined #openstack-glance | 05:03 | |
| *** kebray has quit IRC | 05:05 | |
| *** changbl has quit IRC | 05:07 | |
| *** haomaiwa_ has joined #openstack-glance | 05:08 | |
| *** kebray_ has quit IRC | 05:11 | |
| *** haomaiw__ has joined #openstack-glance | 05:11 | |
| *** kebray has joined #openstack-glance | 05:13 | |
| *** haomaiwa_ has quit IRC | 05:14 | |
| *** sgotliv_ has joined #openstack-glance | 05:24 | |
| *** haomaiw__ has quit IRC | 05:30 | |
| *** ianbrown has quit IRC | 05:32 | |
| *** jerrygb has joined #openstack-glance | 05:35 | |
| *** links has joined #openstack-glance | 05:35 | |
| *** houming has quit IRC | 05:39 | |
| *** gb21_ has joined #openstack-glance | 05:42 | |
| *** azbiswas has quit IRC | 05:47 | |
| *** azbiswas has joined #openstack-glance | 05:47 | |
| *** houming has joined #openstack-glance | 05:48 | |
| *** azbiswas has quit IRC | 05:52 | |
| *** ducttape_ has quit IRC | 05:52 | |
| *** jerrygb has quit IRC | 05:52 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/glance_store: Updated from global requirements https://review.openstack.org/251651 | 06:05 |
| *** dims has quit IRC | 06:09 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-glanceclient: Updated from global requirements https://review.openstack.org/250970 | 06:09 |
| tsufiev | krotscheck: will take a look | 06:13 |
| *** dims has joined #openstack-glance | 06:14 | |
| *** azbiswas has joined #openstack-glance | 06:17 | |
| *** abhishekk has joined #openstack-glance | 06:33 | |
| krotscheck | tsufiev: Thanks :) | 06:46 |
| *** ducttape_ has joined #openstack-glance | 06:53 | |
| *** ducttape_ has quit IRC | 06:59 | |
| *** achanda_ has joined #openstack-glance | 07:00 | |
| *** achanda has quit IRC | 07:03 | |
| *** ducttape_ has joined #openstack-glance | 07:03 | |
| *** kebray has quit IRC | 07:15 | |
| *** ducttape_ has quit IRC | 07:20 | |
| *** azbiswas has quit IRC | 07:36 | |
| *** achanda_ has quit IRC | 07:36 | |
| *** azbiswas has joined #openstack-glance | 07:36 | |
| *** azbiswas has quit IRC | 07:41 | |
| openstackgerrit | Kairat Kushaev proposed openstack/glance_store: Implement re-auth for swift driver https://review.openstack.org/249290 | 07:55 |
| *** markus_z has joined #openstack-glance | 08:19 | |
| *** ducttape_ has joined #openstack-glance | 08:21 | |
| *** achanda has joined #openstack-glance | 08:23 | |
| *** MattMan has quit IRC | 08:23 | |
| *** MattMan has joined #openstack-glance | 08:24 | |
| *** ducttape_ has quit IRC | 08:25 | |
| openstackgerrit | leoyyoung proposed openstack/glance: Change the format of some inconsistent docstring https://review.openstack.org/250224 | 08:29 |
| *** rcernin has joined #openstack-glance | 08:37 | |
| *** jerrygb has joined #openstack-glance | 08:52 | |
| *** belmoreira has joined #openstack-glance | 09:00 | |
| *** ducttape_ has joined #openstack-glance | 09:21 | |
| *** ducttape_ has quit IRC | 09:26 | |
| *** achanda has quit IRC | 09:38 | |
| openstackgerrit | Merged openstack/glance: Document contribution guidelines https://review.openstack.org/234653 | 09:42 |
| *** houming has quit IRC | 09:50 | |
| *** jistr has joined #openstack-glance | 09:52 | |
| *** terryyao has quit IRC | 09:53 | |
| openstackgerrit | Merged openstack/python-glanceclient: Updated from global requirements https://review.openstack.org/250970 | 10:03 |
| *** aix has joined #openstack-glance | 10:05 | |
| openstackgerrit | Merged openstack/glance_store: Updated from global requirements https://review.openstack.org/251651 | 10:16 |
| *** ducttape_ has joined #openstack-glance | 10:22 | |
| *** ducttape_ has quit IRC | 10:28 | |
| *** jerrygb has quit IRC | 10:31 | |
| *** e0ne has joined #openstack-glance | 10:37 | |
| *** mdavidson has joined #openstack-glance | 10:48 | |
| *** mingdang1 has quit IRC | 10:48 | |
| *** e0ne has quit IRC | 10:51 | |
| *** azbiswas has joined #openstack-glance | 10:56 | |
| *** azbiswas has quit IRC | 11:01 | |
| *** marekd has joined #openstack-glance | 11:04 | |
| *** aix has quit IRC | 11:06 | |
| openstackgerrit | Merged openstack/glance: Updated from global requirements https://review.openstack.org/250953 | 11:16 |
| *** jerrygb has joined #openstack-glance | 11:16 | |
| *** aix has joined #openstack-glance | 11:18 | |
| *** jerrygb has quit IRC | 11:19 | |
| *** flaper87 has quit IRC | 11:30 | |
| *** flaper87 has joined #openstack-glance | 11:30 | |
| tsufiev | krotscheck, tested it, still one question (see in review) | 11:31 |
| *** chlong has quit IRC | 11:35 | |
| *** chlong has joined #openstack-glance | 11:35 | |
| *** flaper87 has quit IRC | 11:39 | |
| *** cpallares has quit IRC | 11:40 | |
| *** cpallares has joined #openstack-glance | 11:46 | |
| *** flaper87 has joined #openstack-glance | 11:47 | |
| *** flaper87 has quit IRC | 11:47 | |
| *** flaper87 has joined #openstack-glance | 11:47 | |
| *** houming has joined #openstack-glance | 11:49 | |
| *** jdurgin has quit IRC | 11:53 | |
| *** jdurgin has joined #openstack-glance | 11:54 | |
| *** ninag has joined #openstack-glance | 11:56 | |
| *** sgotliv__ has joined #openstack-glance | 12:00 | |
| *** ninag has quit IRC | 12:01 | |
| *** ducttape_ has joined #openstack-glance | 12:01 | |
| *** sgotliv_ has quit IRC | 12:03 | |
| *** ducttape_ has quit IRC | 12:06 | |
| *** clayton has quit IRC | 12:06 | |
| *** zao_ has joined #openstack-glance | 12:07 | |
| *** zao has quit IRC | 12:08 | |
| *** zao_ is now known as zao | 12:08 | |
| *** achanda has joined #openstack-glance | 12:09 | |
| *** gb21_ has quit IRC | 12:11 | |
| *** clayton has joined #openstack-glance | 12:13 | |
| *** jerrygb has joined #openstack-glance | 12:20 | |
| openstackgerrit | Merged openstack/glance_store: Add list of supported stores to help https://review.openstack.org/251450 | 12:21 |
| *** jaypipes has joined #openstack-glance | 12:22 | |
| *** kairat has quit IRC | 12:24 | |
| *** e0ne has joined #openstack-glance | 12:25 | |
| *** jerrygb has quit IRC | 12:27 | |
| *** kairat has joined #openstack-glance | 12:27 | |
| *** zao has quit IRC | 12:27 | |
| *** zao has joined #openstack-glance | 12:27 | |
| *** edmondsw has joined #openstack-glance | 12:29 | |
| *** achanda has quit IRC | 12:31 | |
| *** e0ne has quit IRC | 12:34 | |
| *** e0ne has joined #openstack-glance | 12:34 | |
| *** e0ne_ has joined #openstack-glance | 12:37 | |
| *** achanda has joined #openstack-glance | 12:38 | |
| *** e0ne has quit IRC | 12:39 | |
| *** abhishekk has quit IRC | 12:45 | |
| *** sileht has joined #openstack-glance | 12:47 | |
| openstackgerrit | Kairat Kushaev proposed openstack/glance_store: Implement swift store connection manager https://review.openstack.org/251850 | 12:48 |
| openstackgerrit | Kairat Kushaev proposed openstack/glance_store: Implement re-authentication for swift driver https://review.openstack.org/251851 | 12:48 |
| *** jaypipes has quit IRC | 12:49 | |
| openstackgerrit | Mike Fedosin proposed openstack/glance: Implement trust support for api v2 https://review.openstack.org/241986 | 12:49 |
| *** jaypipes has joined #openstack-glance | 12:54 | |
| *** jerrygb has joined #openstack-glance | 13:01 | |
| *** julim has quit IRC | 13:01 | |
| *** achanda has quit IRC | 13:01 | |
| openstackgerrit | Kairat Kushaev proposed openstack/glance_store: Implement re-authentication for swift driver https://review.openstack.org/251851 | 13:06 |
| *** achanda has joined #openstack-glance | 13:06 | |
| *** ninag has joined #openstack-glance | 13:08 | |
| flaper87 | o/ | 13:08 |
| *** takedakn has joined #openstack-glance | 13:08 | |
| *** ducttape_ has joined #openstack-glance | 13:10 | |
| *** takedakn has quit IRC | 13:15 | |
| *** stevemar has quit IRC | 13:22 | |
| *** stevemar_znc has joined #openstack-glance | 13:23 | |
| *** takedakn has joined #openstack-glance | 13:26 | |
| *** takedakn has quit IRC | 13:29 | |
| *** ducttape_ has quit IRC | 13:33 | |
| zigo | flaper87: Hi there! I saw you contributed to pyngus, which is a new global-requirement for Mitaka. Do you happen to remember what the runtime dependencies are for that one? | 13:41 |
| zigo | flaper87: Can I just do: python-qpid ? | 13:41 |
| zigo | Or python-qpid-proton is mandatory? | 13:41 |
| flaper87 | zigo: it should be python-qpid-proton which is different from python-qpid | 13:41 |
| zigo | Ok, thanks. | 13:42 |
| flaper87 | np | 13:42 |
| flaper87 | zigo: if it gives you trouble you can ping me and/or kgiusti (he's on openstack-oslo) | 13:42 |
| zigo | Well, I got to upgrade qpid-proton in Sid, maintained by others, but it should be fine. | 13:43 |
| flaper87 | zigo: awesome! | 13:43 |
| *** links has quit IRC | 13:47 | |
| *** dshakhray has joined #openstack-glance | 13:48 | |
| *** mingdang1 has joined #openstack-glance | 13:55 | |
| *** e0ne_ has quit IRC | 13:55 | |
| *** e0ne has joined #openstack-glance | 13:55 | |
| *** achanda_ has joined #openstack-glance | 13:56 | |
| *** achanda has quit IRC | 13:59 | |
| bunting | flaper87: ping | 14:01 |
| flaper87 | bunting: brb, meeting | 14:01 |
| flaper87 | bunting: shoot | 14:01 |
| bunting | Just to clarify with the deprecate execeptions, do you mean change the message? | 14:02 |
| bunting | that is in the actual exceptions file | 14:02 |
| *** ayoung has joined #openstack-glance | 14:05 | |
| *** Ctina has joined #openstack-glance | 14:25 | |
| *** Ctina is now known as ctina | 14:25 | |
| *** mclaren has joined #openstack-glance | 14:30 | |
| flaper87 | bunting: I meant, throw a warning | 14:30 |
| mclaren | rosmaita: ping | 14:30 |
| rosmaita | mclaren: pong | 14:31 |
| mclaren | I bet you'd like nothing more than to chat about image import? | 14:31 |
| *** azbiswas has joined #openstack-glance | 14:32 | |
| flaper87 | mclaren: rosmaita whatever you discuss, please, report back on the spec. | 14:32 |
| flaper87 | I'll try to participate but I have some backlog to go through now | 14:32 |
| rosmaita | flaper87: will do | 14:33 |
| rosmaita | mclaren: did you get a chance to read through my comments (posted a few hours ago) | 14:33 |
| mclaren | let me double check | 14:33 |
| mclaren | so...many...comments | 14:33 |
| *** davideagnello has quit IRC | 14:34 | |
| *** haomaiwa_ has joined #openstack-glance | 14:35 | |
| mclaren | reading... | 14:37 |
| *** azbiswas has quit IRC | 14:37 | |
| mclaren | rosmaita: ok I've read them -- fairly quickly, so bear with me if I missed something | 14:38 |
| mclaren | question 1: | 14:39 |
| mclaren | > Image validation are very I/O and CPU intensiv | 14:39 |
| openstackgerrit | Merged openstack/glance: Remove anyjson useless requirement https://review.openstack.org/245409 | 14:39 |
| mclaren | what validation is cpu intensive? | 14:39 |
| rosmaita | well, we use xen | 14:39 |
| rosmaita | vhd-utils | 14:39 |
| rosmaita | actually may not be so bad | 14:39 |
| rosmaita | but the packaging to compress the VHD into an OVA puts a load on the node | 14:40 |
| mclaren | Ok, vhd to ova is conversion, not validation | 14:40 |
| mclaren | I've tried to make a distinction between when we modify the checksum and when we don't | 14:40 |
| mclaren | which operations which don't modify the checksum are I/O / cpu intensive? | 14:41 |
| rosmaita | gotcha | 14:41 |
| *** achanda_ has quit IRC | 14:41 | |
| rosmaita | well, i/o to get the thing to the node for processing, and then to get it off | 14:41 |
| mclaren | you can tee the image into /var as you upload it into the backend, the opposite of what we do with caching | 14:42 |
| mclaren | you only need to do that in the non nova-snapshot case | 14:42 |
| mclaren | is that a deal-breaker? | 14:42 |
| rosmaita | well, i think we need to plan for the case where /var is full | 14:43 |
| rosmaita | i mean, you can have pretty big images | 14:43 |
| rosmaita | and if you happen to get a few of those simultaneously, you're in trouble | 14:43 |
| rosmaita | so i view the tee thing as an optimization, not the main implementation | 14:44 |
| mclaren | hmm, so you are saying put it in the backend so you can retry after /var is cleaned out? | 14:44 |
| rosmaita | exactly | 14:45 |
| mclaren | ok, question 2 | 14:45 |
| mclaren | in the multi-tenant swift store how do you retry? | 14:45 |
| rosmaita | well, if the image data is in the "normal" place in the backend, i'm not sure there's a problem? | 14:46 |
| rosmaita | (but i haven't much experience with multi-tenant store) | 14:46 |
| mclaren | is it a user initiated retry? | 14:46 |
| rosmaita | ok, gotcha | 14:46 |
| rosmaita | well, what i see happening is this: | 14:47 |
| rosmaita | 1 - user creates image record | 14:47 |
| rosmaita | 2 - user uploads to /bikeshed | 14:47 |
| mclaren | in terms of big picture, I've been thinking of the no-swift case as economy, and when swift is there then you get fancy retries (both initial upload and post upload) | 14:47 |
| rosmaita | 3 - (am going to add back the "import" call) | 14:47 |
| rosmaita | 4 - behind the scenes, import call creates a task | 14:48 |
| rosmaita | 5 - task is eventually processed when resources are available | 14:48 |
| rosmaita | so not really a retry | 14:48 |
| mclaren | but a user could repeat the import call to get a retry of that step | 14:49 |
| mclaren | ? | 14:49 |
| *** mtanino has joined #openstack-glance | 14:50 | |
| rosmaita | i guess, but what would the point of that be? | 14:50 |
| rosmaita | (it is morning here, have not had sufficient coffee yet) | 14:50 |
| *** ducttape_ has joined #openstack-glance | 14:50 | |
| mclaren | lol, its late here, not sure what my excuse is! | 14:50 |
| rosmaita | your caffeine has worn off! | 14:50 |
| mclaren | what happens if the task fails? | 14:50 |
| mclaren | say the node it's running on dies | 14:51 |
| mclaren | lol | 14:51 |
| mclaren | how does the user re-use the data from the backend? | 14:51 |
| rosmaita | ok, that's a good question | 14:52 |
| rosmaita | i think i have an answer | 14:52 |
| rosmaita | with the swift-local method, no problem, it's not glance's data so not in the multi=tenant store | 14:53 |
| mclaren | I think in the swift is available case this all works naturally | 14:53 |
| rosmaita | with the glance-local method, probably tough luck? | 14:53 |
| rosmaita | data would disappear from multi-tenant store same as when a nova snapshot fails? | 14:53 |
| mclaren | potentially | 14:54 |
| rosmaita | because the user didn't put the data into swift, they put it into glance | 14:54 |
| rosmaita | i need to think some more about this i guess | 14:54 |
| mclaren | Basically, I'm leaning towards keeping things simple in the glance-only case, and taking advantage of Swift when it's there to provide retries etc. | 14:54 |
| mclaren | I think in the glance only case we'll end up needing two blobs temporarily | 14:55 |
| mclaren | and I think that could turn out to be a headache | 14:55 |
| mclaren | eg v1 updates | 14:55 |
| mclaren | disk usage quota | 14:56 |
| mclaren | exposing blob usage to users | 14:56 |
| rosmaita | i don't know that we need to expose blob usage to users -- what's the diff between it being in /var and being in backend as far as a user is concerned? | 14:57 |
| mclaren | should it cound towards their disk usage quota? | 14:57 |
| mclaren | count | 14:57 |
| mclaren | storage usage | 14:57 |
| rosmaita | how does that work now for multitenant store? | 14:59 |
| mclaren | good question. I guess we just add up the sizes we know about from the glance database | 15:00 |
| openstackgerrit | Kairat Kushaev proposed openstack/glance-specs: Prevention of Unauthorized errors in Swift driver https://review.openstack.org/248681 | 15:00 |
| rosmaita | having swift available would solve so many problems | 15:00 |
| mclaren | and I'm not sure if it's only active images we count | 15:00 |
| rosmaita | blob woudl be in user-space to begin with as an object, not an image | 15:00 |
| openstackgerrit | Darja Shakhray proposed openstack/glance-specs: Add filters using an 'in' operator https://review.openstack.org/246971 | 15:01 |
| *** mfedosin has joined #openstack-glance | 15:01 | |
| mclaren | so you're saying just don't count it? | 15:02 |
| rosmaita | not sure what i'm saying, actually | 15:03 |
| mclaren | yeah, it's a pain! It kind of feels like it would be an attempt to shoehorn two things into a space really only designed for one | 15:04 |
| mclaren | (there's only one size, checksum per image) | 15:04 |
| *** achanda has joined #openstack-glance | 15:04 | |
| rosmaita | mclaren: you're right | 15:04 |
| rosmaita | i don't have an answer ATM | 15:04 |
| rosmaita | think you may have to re-raise this on the next patch set | 15:05 |
| mclaren | that's fine. (Me neither) | 15:05 |
| jokke_ | the bikeshed backend could be any supported (writeable) backend and the retry would be possible ... only case where the retry wouldn't be possible is when the bikeshed is file without shared fs between the node | 15:05 |
| rosmaita | hoping to get that up by wed evening so people have some time to look before thurs glance meeting | 15:05 |
| *** mingdang1 has quit IRC | 15:05 | |
| mclaren | I actually don't really care much about retry where is so hard to implement. You can retry by resending the request ... it's the cloud. | 15:06 |
| jokke_ | and that implementation would tie the worker also to the api node | 15:06 |
| rosmaita | jokke_: i was hoping not to have a separate bikeshed backend, just use the same space in the backend where the image would be stored | 15:06 |
| jokke_ | rosmaita: that's definitely one option, but how do you deal with quota? | 15:06 |
| mclaren | jokke I don't think reusing the same location will work. | 15:07 |
| rosmaita | jokke_: it shouldn't affect quota? (thinking) | 15:07 |
| mclaren | you need two places temporarily | 15:07 |
| jokke_ | Will you penalize customer who has 50gb of quota and trying to import 30gb image because the initial blob and the actual image will not fit to the quota | 15:07 |
| *** dims has quit IRC | 15:08 | |
| rosmaita | jokke_: no, what you have is: 1 - user uploads 30gb image and glance puts in backend, 2- when ready, glance gets it out of backend and processes image, 3- glance replaces initial blob wiht processed image | 15:08 |
| jokke_ | that's why thinking that glance/deployer should "own" the bikeshed and customer the image | 15:08 |
| rosmaita | so the user wouldn't have 60gb taken up, just 30 gb | 15:09 |
| jokke_ | rosmaita: that will kill the retry on every case if something fails during that upload | 15:09 |
| mclaren | right, but if there's a failure while replacing you can't retry afterwords | 15:09 |
| jokke_ | as the original blob nor the image is available for retry | 15:09 |
| mclaren | and then of course v1 needs to be made bikeshed aware for delete etc | 15:10 |
| rosmaita | so it would be same situation as a nova snapshot ... if snapshot at time t fails, you have to start over | 15:10 |
| mclaren | and if someone comes along and imports from swift before a retry you need to handle the bikeshed in that case | 15:11 |
| mclaren | I just see the code being peppered with [now we need to handle the bikeshed in this case...] | 15:12 |
| rosmaita | yeah, that's why i want to avoid having an actual bikeshed | 15:12 |
| mclaren | And in the swift case that happens more naturally | 15:13 |
| rosmaita | yes, the swift case is quite nice | 15:13 |
| rosmaita | but apparently not very popular | 15:13 |
| mclaren | I'm assuming we'll add something to allow import from swift as a call when swift is available? | 15:14 |
| mclaren | like copy-from-esque | 15:14 |
| mclaren | so when swift is there you only upload your bits in once, so a retry is less costly. | 15:15 |
| mclaren | I'm don't feel we need to do huge contortions to have that behaviour when swift isn't there | 15:16 |
| *** mfedosin has quit IRC | 15:16 | |
| rosmaita | well, maybe we allow limited copy-from, so someone with swift might allow copy-from only from their local swift? | 15:16 |
| rosmaita | less fastidious people can allow copy-from from arbitrary urls | 15:17 |
| mclaren | right, the interface can be whatever we decide, but you get retries when Swift is there | 15:18 |
| rosmaita | except the discussion at the summit was copy-from is a non-starter | 15:18 |
| mclaren | I'm talking about import from swift being an additional option | 15:18 |
| rosmaita | because it requires the importer to have a web server to host the blob | 15:19 |
| mclaren | ie defcore compute = upload to /file, defcore platform = upload to /file, or via swift | 15:19 |
| rosmaita | so basically, 'swift-local' ? | 15:19 |
| mclaren | sure | 15:19 |
| mclaren | the problem we've been asked to solve isn't a retry problem | 15:20 |
| mclaren | (I need to go in a few mins) | 15:21 |
| *** sabari has joined #openstack-glance | 15:21 | |
| rosmaita | me too ... ok, what i will do is revise back to PS 5 but without the image resource | 15:21 |
| rosmaita | i think that still has glance-local and swift-local | 15:21 |
| rosmaita | i mean, without the import resource! | 15:22 |
| rosmaita | i think we should keep images | 15:22 |
| mclaren | I'm not sure we need swift-local in the spec, perhaps just something to say that we envisage a complimentary/additional option for defcore platform | 15:22 |
| rosmaita | i may leave it in just as an example of what we need to accommodate | 15:23 |
| mclaren | as you wish | 15:23 |
| rosmaita | or something ... anyway, you can comment on the spec! | 15:23 |
| rosmaita | and this channel is logged, so this convo won't be lost | 15:23 |
| rosmaita | ok, have a good evening | 15:23 |
| mclaren | one question | 15:24 |
| rosmaita | sure | 15:24 |
| mclaren | do you think we need to preserve the checksum? | 15:24 |
| mclaren | where no conversion is being done | 15:24 |
| mclaren | (ie unless the user flags it's ok in some way) | 15:24 |
| rosmaita | not sure about that | 15:25 |
| mclaren | ok, how about breaking existing clients who'll get a 403 | 15:25 |
| mclaren | a big deal or not? | 15:25 |
| mclaren | (if we change /file) | 15:26 |
| rosmaita | i think we should not change /file | 15:26 |
| rosmaita | i think we will take a strong stand that we can't break backward compatibility | 15:26 |
| rosmaita | but that doesn't answer your question | 15:27 |
| rosmaita | end-users will get 403 if the deployer wants them to go through the validation chain | 15:27 |
| mclaren | ok, Flavio says (possibly correctly on paper) that changing to /blob and disabling the old method is non-breaking | 15:28 |
| rosmaita | but that could always happen even now, by policy? | 15:28 |
| rosmaita | i'm with Flavio on that one | 15:28 |
| rosmaita | 403 is a possible response to everything, i'd think | 15:29 |
| mclaren | ok but everyone uploading today will start getting 403s and needing a new glance client. If our goal is compatability... | 15:29 |
| rosmaita | everybody needs a new glance client! | 15:29 |
| *** e0ne has quit IRC | 15:29 | |
| rosmaita | yeah, but you'd need a new client to take advantage of any new API stuff | 15:30 |
| mclaren | uploading an unmodified image is arguably not new stuff | 15:30 |
| rosmaita | yes, but the ability to upload has always been policy based | 15:31 |
| rosmaita | if hte deployer changes the policy, the end user must adapt | 15:31 |
| rosmaita | it's up to the deployer to make it a smooth transition | 15:31 |
| rosmaita | anyway, you can bring this up on the spec | 15:32 |
| rosmaita | it's a complication i don't want to think about ATM! | 15:32 |
| mclaren | Sure. I just think in practice a 'legal' breaking change can have the same impact in practice as a 'non-legal' one. | 15:33 |
| mclaren | ok, thanks for your time Brian | 15:34 |
| *** belmoreira has quit IRC | 15:35 | |
| *** ninag has quit IRC | 15:36 | |
| rosmaita | mclaren: np ... you definitely have a point, i just don't want to deal with it! | 15:36 |
| flaper87 | nikhil: rosmaita mclaren jokke_ hemanthm sabari PPL: I'm about to cut m-1 and I wanted to know if there's anything we should absolutely, totally, entirely, have | 15:45 |
| *** achanda has quit IRC | 15:45 | |
| flaper87 | note that this is the first milestone release so, it's not really a problem if we miss one or two patches | 15:46 |
| *** e0ne has joined #openstack-glance | 15:47 | |
| *** aix has quit IRC | 15:49 | |
| jokke_ | flaper87: not that I know at least | 15:52 |
| flaper87 | jokke_: thanks | 15:53 |
| *** dims has joined #openstack-glance | 15:54 | |
| hemanthm | flaper87: nothing comes to my head | 15:56 |
| *** vijendar has joined #openstack-glance | 15:56 | |
| flaper87 | hemanthm: thanks | 15:57 |
| openstackgerrit | Merged openstack/python-glanceclient: Change man page examples to correlate default v2 https://review.openstack.org/242523 | 15:59 |
| openstackgerrit | Merged openstack/python-glanceclient: Update set of wanted commands in read-only test https://review.openstack.org/250829 | 15:59 |
| openstackgerrit | Kairat Kushaev proposed openstack/glance_store: Implement re-authentication for swift driver https://review.openstack.org/251851 | 16:02 |
| *** achanda has joined #openstack-glance | 16:03 | |
| *** achanda has quit IRC | 16:03 | |
| *** avarner has joined #openstack-glance | 16:04 | |
| *** rcernin has quit IRC | 16:04 | |
| *** kebray has joined #openstack-glance | 16:05 | |
| *** gb21_ has joined #openstack-glance | 16:06 | |
| *** dave-mcnally has joined #openstack-glance | 16:07 | |
| *** dims_ has joined #openstack-glance | 16:08 | |
| *** changbl has joined #openstack-glance | 16:09 | |
| *** dims has quit IRC | 16:10 | |
| *** achanda has joined #openstack-glance | 16:11 | |
| *** ativelkov has quit IRC | 16:11 | |
| *** tsufiev has quit IRC | 16:11 | |
| *** davideagnello has joined #openstack-glance | 16:13 | |
| dave-mcnally | nikhil: thanks for your comments on https://review.openstack.org/#/c/249282/, you mentioned adding release notes. I'm not sure of the formatting but I guess that change should be made to glance/releasenotes/source/unreleased.rst? Or am I better off adding a release notes impact comment to the commit and changing the release notes in a future patch? | 16:13 |
| *** kebray has quit IRC | 16:16 | |
| *** avarner has quit IRC | 16:17 | |
| *** ativelkov has joined #openstack-glance | 16:20 | |
| *** azbiswas has joined #openstack-glance | 16:21 | |
| *** TravT has quit IRC | 16:22 | |
| *** azbiswas has quit IRC | 16:22 | |
| *** TravT has joined #openstack-glance | 16:25 | |
| *** changbl has quit IRC | 16:27 | |
| *** tsufiev has joined #openstack-glance | 16:27 | |
| *** r-daneel has joined #openstack-glance | 16:27 | |
| *** kebray has joined #openstack-glance | 16:28 | |
| *** kebray has quit IRC | 16:28 | |
| *** kebray has joined #openstack-glance | 16:28 | |
| *** spzala has joined #openstack-glance | 16:29 | |
| *** dims has joined #openstack-glance | 16:30 | |
| *** azbiswas has joined #openstack-glance | 16:31 | |
| *** dims_ has quit IRC | 16:33 | |
| openstackgerrit | Drew Varner proposed openstack/glance_store: Add reno for release notes management in glance_store https://review.openstack.org/251561 | 16:39 |
| *** changbl has joined #openstack-glance | 16:42 | |
| nikhil | flaper87: nothing from me | 16:44 |
| flaper87 | nikhil: thanks | 16:45 |
| openstackgerrit | dave-mcnally proposed openstack/glance: Add the Docker container format https://review.openstack.org/249282 | 16:47 |
| nikhil | dave-mcnally: hi, I would say use this keystone guideline http://docs.openstack.org/developer/keystone/developing.html#release-notes . THe process is same in glance though we don't have the documentation, it's literally the same. | 16:47 |
| stevemar_znc | nikhil: we're porting most of those to the project team guide | 16:48 |
| * stevemar_znc looks for link | 16:48 | |
| stevemar_znc | nikhil: http://docs.openstack.org/project-team-guide/release-management.html#managing-release-notes | 16:50 |
| nikhil | stevemar_znc: wow, that's good documentation. thanks | 16:51 |
| * flaper87 is proud of the work that has been done w/ PTG | 16:53 | |
| *** houming has quit IRC | 16:58 | |
| dave-mcnally | nikhil, stevemar_znc: that's great thanks | 16:59 |
| *** kebray has quit IRC | 17:00 | |
| *** haomaiwa_ has quit IRC | 17:00 | |
| *** dave-mcnally is now known as davemc_afk | 17:01 | |
| *** kebray has joined #openstack-glance | 17:05 | |
| *** links has joined #openstack-glance | 17:08 | |
| *** dshakhray has quit IRC | 17:10 | |
| *** stevemar_znc is now known as stevemar | 17:25 | |
| *** ayoung has quit IRC | 17:28 | |
| *** e0ne has quit IRC | 17:30 | |
| sabari | flaper87 Just got back from vacation - go m1 from my side :) | 17:39 |
| *** avarner has joined #openstack-glance | 17:39 | |
| flaper87 | sabari: awesome | 17:40 |
| flaper87 | oh, btw, I have to add you to cores now | 17:40 |
| *** achanda has quit IRC | 17:45 | |
| *** gb21_ has quit IRC | 17:51 | |
| *** achanda has joined #openstack-glance | 17:52 | |
| *** jistr has quit IRC | 17:54 | |
| *** ayoung has joined #openstack-glance | 18:01 | |
| *** kebray has quit IRC | 18:02 | |
| *** ayoung_ has joined #openstack-glance | 18:02 | |
| *** ayoung has quit IRC | 18:03 | |
| *** ayoung_ is now known as ayoung | 18:03 | |
| sabari | flaper87 awesome :D | 18:08 |
| *** mfedosin has joined #openstack-glance | 18:09 | |
| kragniz | sabari: congrats! | 18:13 |
| *** kebray has joined #openstack-glance | 18:13 | |
| sabari | thanks kragniz! | 18:14 |
| *** avarner has quit IRC | 18:17 | |
| *** kebray has quit IRC | 18:17 | |
| *** mfedosin has quit IRC | 18:21 | |
| *** links has quit IRC | 18:24 | |
| *** markus_z has quit IRC | 18:24 | |
| *** mancdaz has quit IRC | 18:26 | |
| *** azbiswas has quit IRC | 18:27 | |
| *** mancdaz has joined #openstack-glance | 18:27 | |
| *** dshakhray has joined #openstack-glance | 18:30 | |
| *** rcernin has joined #openstack-glance | 18:38 | |
| *** alex_xu has quit IRC | 18:48 | |
| *** kebray has joined #openstack-glance | 18:49 | |
| *** alex_xu has joined #openstack-glance | 18:52 | |
| *** kebray has quit IRC | 18:52 | |
| *** kebray has joined #openstack-glance | 18:54 | |
| *** kebray has quit IRC | 18:58 | |
| openstackgerrit | Flavio Percoco proposed openstack/glance: Remove version from setup.cfg https://review.openstack.org/252021 | 19:02 |
| flaper87 | sabari: done, please check you have +2 powers | 19:03 |
| openstackgerrit | Merged openstack/python-glanceclient: Add ordereddict requirement for py26 tests https://review.openstack.org/250751 | 19:08 |
| *** vijendar has quit IRC | 19:08 | |
| sabari | flaper87 Yey! Just checked. | 19:24 |
| sabari | thanks! | 19:24 |
| flaper87 | sabari: thank *you* | 19:25 |
| flaper87 | now, don't #$@#@ this up </joke> | 19:25 |
| sabari | flaper87 hahaha - sure i won't :D | 19:25 |
| *** avarner has joined #openstack-glance | 19:28 | |
| *** avarner_ has joined #openstack-glance | 19:35 | |
| *** avarner has quit IRC | 19:35 | |
| *** avarner_ has quit IRC | 19:35 | |
| *** avarner has joined #openstack-glance | 19:36 | |
| *** julim has joined #openstack-glance | 19:36 | |
| *** kebray has joined #openstack-glance | 19:38 | |
| *** kebray has quit IRC | 19:39 | |
| *** cdelatte has quit IRC | 19:44 | |
| *** cdelatte has joined #openstack-glance | 19:45 | |
| *** azbiswas has joined #openstack-glance | 19:45 | |
| *** burgerk has joined #openstack-glance | 19:46 | |
| *** kebray has joined #openstack-glance | 19:46 | |
| *** azbiswas_ has joined #openstack-glance | 19:46 | |
| *** kebray has quit IRC | 19:48 | |
| *** cdelatte has quit IRC | 19:49 | |
| *** cdelatte has joined #openstack-glance | 19:50 | |
| *** azbiswas has quit IRC | 19:50 | |
| *** vijendar has joined #openstack-glance | 20:18 | |
| *** spzala has quit IRC | 20:19 | |
| *** haypo has joined #openstack-glance | 20:27 | |
| haypo | lifeless: here. i'm not really interested to work on the random unit test failure since it's very difficult to reproduce it, so i bet that nobody will notice it in the CI | 20:28 |
| lifeless | haypo: it triggers about 3% of the time | 20:28 |
| lifeless | haypo: thats more than enough to cause havoc in CI | 20:29 |
| haypo | lifeless: it's a non-zero reproduction rate :-/ | 20:29 |
| lifeless | haypo: anyhow, have you filed a bug # for it ? | 20:29 |
| haypo | lifeless: my bet is it's race condition in the unit test. something mocks the HTTP server with a request handler catching any URL, but the mock is not reverted before the "nonexistent" test is run | 20:29 |
| haypo | lifeless: no, since i almost have no information about it | 20:30 |
| haypo | lifeless: oh by the way, i would be very interested to know if it's specific to python 3 or not | 20:30 |
| haypo | lifeless: i guess that it does not depend on the python major version | 20:30 |
| lifeless | haypo: I believe it is | 20:30 |
| lifeless | haypo: I ran the reliable reproducer I have overnight with no failures on 2.7 | 20:31 |
| lifeless | I'm down to < 150 tests to reproduce | 20:31 |
| haypo | lifeless: would you like to open a bug report? | 20:31 |
| lifeless | sure | 20:31 |
| lifeless | I didn't want to duplicate | 20:31 |
| lifeless | I presume I need the patch to trigger this? its not in master? | 20:32 |
| lifeless | or the porting work isn't in master or something? | 20:32 |
| lifeless | haypo: basically I'm just using bisection | 20:32 |
| lifeless | haypo: I'll write up a post describing it after I guess | 20:32 |
| haypo | lifeless: https://review.openstack.org/#/c/250083/ was approved, but the new patch set has no vote yet | 20:33 |
| haypo | lifeless: the change was not approved because another change introduced a new python 3 issue. i wrote a different patch to fix the new python 3 issue, this change is already merged | 20:37 |
| lifeless | flaper87: around? I'd like to ask about the lockutils-wrapper thing | 20:41 |
| lifeless | haypo: https://bugs.launchpad.net/glance/+bug/1521756 | 20:42 |
| openstack | Launchpad bug 1521756 in Glance "race/python3 issue" [Undecided,New] | 20:42 |
| haypo | lifeless: thanks | 20:44 |
| haypo | lifeless: you cannot remove a test of your list if the test suite doesn't fail | 20:49 |
| haypo | lifeless: but you can use a smaller list if your reproduce the failure | 20:49 |
| haypo | lifeless: for your bisection ;) | 20:49 |
| lifeless | right | 20:49 |
| haypo | lifeless: is testr able to handle such race condition? | 20:50 |
| lifeless | haypo: analyze-isolation automates bisection, but you can't use it (today) with one-in-N failures | 20:50 |
| lifeless | so I'm just doing what analyze-isolation does by hand | 20:50 |
| haypo | ah, you replied :-) no, it doesn't | 20:50 |
| lifeless | (because it doesn't understand probabalistic failures | 20:50 |
| *** achanda has quit IRC | 20:50 | |
| lifeless | a fairly simple --min-passes=N would solve it I think | 20:51 |
| lifeless | e.g. require 200 passes before considering it non-failing | 20:51 |
| lifeless | alternatively, it could try both branches alternately until one fails | 20:51 |
| lifeless | probably faster to do that, but needs more code changes | 20:51 |
| haypo | my manual algorithm: pick a random set of tests | 20:51 |
| haypo | run it | 20:51 |
| haypo | if it fails, bisect from this list | 20:51 |
| haypo | otherwise, pick another set of random tests | 20:52 |
| lifeless | haypo: ouch - thats random sort :) | 20:55 |
| lifeless | haypo: given 300 tests, if its just one causing it, then you need a phenomenal number of probes to reduce | 20:55 |
| lifeless | [the starting point was 525 tests, trivially reduced to 306 by order of execution | 20:56 |
| haypo | lifeless: in my experience, at the end, you only need two tests | 20:57 |
| haypo | so the problem was to find this test on 2600+ tests :) | 20:57 |
| haypo | i had technical issues. when i used testtools.run directly, i got errors, whereas i didn't get these errors with testr | 20:58 |
| haypo | it's because testr loads all tests and _then_ executes them (i guess) | 20:58 |
| lifeless | haypo: probably due to not using the lockutils-wrapper | 20:58 |
| haypo | i worked around the issue with an "import_all.py" script :) | 20:58 |
| lifeless | haypo: no, if subunit.run can execute the tests, testtools.run can as well | 20:58 |
| haypo | lifeless: nope. it was an issue on oslo.log | 20:58 |
| lifeless | subunit.run is a subclass | 20:58 |
| lifeless | possibly you switched from discover based loading to name based loading | 20:59 |
| haypo | it's a common issue. depending on the import order and execute order, you get different result :) | 20:59 |
| lifeless | sure, but subunit.run is a subclass of testtools.run | 20:59 |
| lifeless | its identical for those bits, given the same parameters | 20:59 |
| lifeless | I don't dispute you had an error | 21:00 |
| lifeless | just giving you the data about where it could be happening | 21:01 |
| lifeless | haypo: the very first time it happens you can extract only the worker specific tests in order to make your starting point much smaller | 21:01 |
| lifeless | haypo: the recipe for that is on the testr wiki page | 21:01 |
| *** changbl has quit IRC | 21:12 | |
| *** ctina_ has joined #openstack-glance | 21:22 | |
| *** ctina has quit IRC | 21:26 | |
| *** ctina_ has quit IRC | 21:27 | |
| *** harlowja has quit IRC | 21:27 | |
| haypo | lifeless: i reproduce the fail with your short list of 19 tests. cool | 21:27 |
| *** harlowja has joined #openstack-glance | 21:28 | |
| haypo | lifeless: i will continue to bisect manually :) | 21:28 |
| lifeless | down to 3 tests | 21:28 |
| haypo | ah? | 21:28 |
| haypo | your bisection technic looks to be efficient | 21:29 |
| lifeless | yes | 21:29 |
| haypo | lifeless: what are the 3 tests? | 21:29 |
| lifeless | refresh the bug page // check your mail | 21:29 |
| lifeless | its running another step now | 21:29 |
| *** mancdaz has quit IRC | 21:31 | |
| *** rcernin has quit IRC | 21:31 | |
| *** jerrygb has quit IRC | 21:32 | |
| *** mancdaz has joined #openstack-glance | 21:32 | |
| haypo | lifeless: FYI there are two "nonexistant" unit tests. once, i saw the two failed | 21:35 |
| haypo | lifeless: once, only one failed. once, it was the other :) | 21:35 |
| lifeless | sure | 21:35 |
| lifeless | there may be multiple other races | 21:35 |
| lifeless | in fact you pastebinned me a failure with both in it | 21:36 |
| lifeless | but I figure we'll find one clear root cause in all probability | 21:36 |
| haypo | sure | 21:36 |
| *** avarner has quit IRC | 21:36 | |
| lifeless | ok so | 21:36 |
| lifeless | glance.tests.unit.async.flows.test_import.TestImportTask.test_import_flow_no_import_flows | 21:36 |
| haypo | the two unit test checks that an invalid HTTP url gives an error 404 | 21:36 |
| lifeless | glance.tests.unit.v1.test_api.TestGlanceAPI.test_upload_image_http_nonexistent_location_url | 21:36 |
| lifeless | is enough to trigger it | 21:37 |
| lifeless | now to try just the test on its own | 21:37 |
| haypo | lifeless: how do you run tests? | 21:41 |
| haypo | lockutils-wrapper python setup.py testr --slowest --testr-args='glance.tests.unit.async.flows.test_import.TestImportTask.test_import_flow_no_import_flows glance.tests.unit.v1.test_api.TestGlanceAPI.test_upload_image_http_nonexistent_location_url' | 21:41 |
| lifeless | triggered the failure on its own | 21:41 |
| haypo | ^^ i'm trying this | 21:41 |
| lifeless | testr run --load-list current-set --until-failure | 21:42 |
| lifeless | I have moved lockutils-wrapper into .testr.conf | 21:42 |
| lifeless | http://paste.ubuntu.com/13606612/ | 21:42 |
| haypo | lifeless: do you mean that you fail to reproduce the race without 22:42 < lifeless> I have moved lockutils-wrapper into .testr.conf | 21:43 |
| haypo | oops | 21:43 |
| haypo | lifeless: do you mean that you fail to reproduce the race without lockutils-wrapper? | 21:43 |
| lifeless | no, I moved it so I didn't have to have it in my way | 21:44 |
| lifeless | I need to talk to glance folk to understand why its where it is in tox.ini | 21:44 |
| haypo | lifeless: 'python -m testtools.run test1 test2' alone is unable to reproduce the failure | 21:51 |
| haypo | (i ran a loop in shell, run 6 times in parallel) | 21:51 |
| haypo | (during 5 minutes) | 21:51 |
| *** kebray has joined #openstack-glance | 21:59 | |
| haypo | lifeless: ok, using testr run --parallel, i'm able to reproduce the failure with only the 2 tests | 22:00 |
| lifeless | haypo: I can reproduce it with just the one test | 22:05 |
| lifeless | haypo: you probably need to use discover or else things won't load well | 22:06 |
| lifeless | python -m testtools.run discover -t ./ ./glance/tests --load-list filenamehere | 22:06 |
| * jokke_ is reading backlog | 22:06 | |
| haypo | lifeless: things load well | 22:07 |
| lifeless | haypo: I'm pretty sure that in some situations they won't, because the loader won't do parameterisation and so on | 22:07 |
| haypo | lifeless: hum....... at the end, i'm able to reproduce the failure with: | 22:07 |
| haypo | testr run --until-failure testtools.run glance.tests.unit.v1.test_api.TestGlanceAPI.test_upload_image_http_nonexistent_location_url | 22:07 |
| lifeless | haypo: this is a upstream unittest behaviour | 22:07 |
| haypo | when my system load is high | 22:07 |
| lifeless | 20 runs and I reproduced it with testtools | 22:08 |
| lifeless | while true; do python -m testtools.run discover -t ./ ./glance/tests --load-list list-file; done | 22:08 |
| lifeless | where list-file contains | 22:08 |
| lifeless | glance.tests.unit.v1.test_api.TestGlanceAPI.test_upload_image_http_nonexistent_location_url | 22:08 |
| lifeless | anyhow, you've reproduced it, cool | 22:08 |
| lifeless | so its not a cross-test interaction | 22:08 |
| lifeless | oh ugh, it uses tearDown | 22:10 |
| lifeless | that explains the process leaks | 22:10 |
| lifeless | jokke_: hi | 22:10 |
| lifeless | jokke_: so this lockutils-wrapper | 22:10 |
| lifeless | jokke_: whats it for? | 22:10 |
| jokke_ | lifeless: I got there ... I have absolutely no idea | 22:10 |
| haypo | lifeless: does "testr run --until-failure" create a fresh process at each run? | 22:11 |
| lifeless | jokke_: ok, so I'll put up a patch to make it saner | 22:11 |
| lifeless | haypo: yes | 22:11 |
| haypo | lifeless: ok | 22:11 |
| haypo | high system load is not enough to reproduce the issue | 22:12 |
| jokke_ | lifeless: seems like sdague added that in January ... he might even remeber why | 22:12 |
| *** dshakhray has quit IRC | 22:12 | |
| jokke_ | nope ... he just changed that line | 22:13 |
| haypo | lifeless: "20 runs and I reproduced it with testtools" hum. i'm running 'python -m testtools.run glance.tests.unit.v1.test_api.TestGlanceAPI.test_upload_image_http_nonexistent_location_url' and i'm unable to reproduce the issue | 22:15 |
| jokke_ | ok so ZhiYan added that there when moved to oslo concurrency ~ year ago | 22:15 |
| haypo | i'm running this command in shell loop, 7 shells in parallel. it doesn't fail | 22:15 |
| jokke_ | and he is not around anymore so I doubt we will get light into the reasoning | 22:15 |
| haypo | i also have 6 python -c 'while 1: pass' to burn the cpu | 22:15 |
| haypo | lifeless: it looks like testtools discover does something magic :-D | 22:15 |
| lifeless | haypo: yes, it does | 22:16 |
| jokke_ | lifeless: if you have good idea what it's doing and how to improve, go for it. Just please, don't save the words in the commit message. They are cheap | 22:16 |
| lifeless | haypo: 'python -m unittest discover' and 'python -m unittest testname' do not do the same thing for loading a test | 22:16 |
| openstackgerrit | lifeless proposed openstack/glance: Move lockutils-wrapper into .testr.conf https://review.openstack.org/252086 | 22:17 |
| lifeless | haypo: that said I don't see any load_tests hooks, so it shouldn't be a substantial difference here | 22:17 |
| *** burgerk has quit IRC | 22:18 | |
| lifeless | haypo: I think the difference is more likely our machines | 22:18 |
| lifeless | haypo: / python versions (I'm on 3.4 for instance) | 22:18 |
| lifeless | haypo: I have no reason to think load is part of the story | 22:18 |
| haypo | in my experience, race condition behave differently depending on the system load | 22:19 |
| haypo | i'm running fedora 23 with python 3.4.3 | 22:19 |
| lifeless | haypo: right, but thats an assumption, we don't know that its a race, we only know its intermittent | 22:19 |
| *** harlowja has quit IRC | 22:19 | |
| lifeless | haypo: I'm not saying you are wrong, just that we don't know yet | 22:19 |
| *** ayoung has quit IRC | 22:21 | |
| *** harlowja has joined #openstack-glance | 22:21 | |
| haypo | lifeless: hat is ./glance/tests in '-m testtools.run discover -t ./ ./glance/tests --load-list tests' command? | 22:23 |
| haypo | what* is | 22:24 |
| lifeless | its the starting path for discovery | 22:26 |
| lifeless | https://docs.python.org/3.5/library/unittest.html#test-discovery | 22:27 |
| haypo | my reliable scenario to reproduce the bug: run 'while true; do python -m testtools.run discover -t ./ ./glance/tests --load-list tests || break; done' command in 6 shell | 22:27 |
| lifeless | cool | 22:27 |
| lifeless | thats progress | 22:27 |
| *** kebray has quit IRC | 22:27 | |
| openstackgerrit | lifeless proposed openstack/glance: Stop using tearDown in v1/test_api.py. https://review.openstack.org/252091 | 22:29 |
| haypo | partial python traceback (i added the raised exception ;-)) http://paste.alacon.org/38990 | 22:29 |
| lifeless | jokke_: https://review.openstack.org/252086 and https://review.openstack.org/252091 may be of interest | 22:29 |
| *** kebray has joined #openstack-glance | 22:31 | |
| *** jerrygb has joined #openstack-glance | 22:32 | |
| *** jerrygb has quit IRC | 22:37 | |
| jokke_ | I'll have a look | 22:41 |
| *** edmondsw has quit IRC | 22:44 | |
| *** davideagnello has quit IRC | 22:45 | |
| *** davideagnello has joined #openstack-glance | 22:45 | |
| *** davideagnello has quit IRC | 22:48 | |
| *** davideagnello has joined #openstack-glance | 22:48 | |
| *** davideag_ has joined #openstack-glance | 22:50 | |
| *** r-daneel has quit IRC | 22:52 | |
| *** kebray has quit IRC | 22:53 | |
| *** davideagnello has quit IRC | 22:53 | |
| haypo | lifeless: "except Exception: return 0". wait, what? :-p | 22:55 |
| lifeless | *blink* | 22:56 |
| haypo | lifeless: don't worry, there is a TODO :-D https://github.com/openstack/glance_store/blob/master/glance_store/_drivers/http.py#L176 | 22:56 |
| *** harlowja has quit IRC | 22:57 | |
| haypo | BadStatusLine("''",) | 22:57 |
| haypo | lifeless: i guess that the client sends an HTTP request to the server which has just been started, but the server is not fully ready yet | 22:58 |
| haypo | lifeless: ooook. i'm now able to reproduce the bug each time with a diff of 1 line :-p | 23:00 |
| haypo | i modified the HTTP server to exit immediatly when it gets a request | 23:01 |
| *** harlowja has joined #openstack-glance | 23:01 | |
| haypo | lifeless: to be clear, it's completly unrelated to pytohn 3 | 23:02 |
| haypo | i'm probably just the first one which noticed this random failure | 23:02 |
| haypo | because i ran tests too many times :-D | 23:02 |
| haypo | lifeless: ok, thanks for motivating me to continue to analyze this bug :-p | 23:09 |
| *** terryyao has joined #openstack-glance | 23:09 | |
| haypo | lifeless: even if the bisection was funny: from 2600 tests, the final list is a single test :-D | 23:09 |
| haypo | i just didn't try to run the single test alone in a loop | 23:09 |
| *** TravT has quit IRC | 23:10 | |
| haypo | i probably had issues to have a reliable scenario to reproduce the issue because it's a race condition. so it depends on the system load | 23:10 |
| haypo | it's a race betwen the client and the server ;) | 23:10 |
| haypo | lifeless: i completed https://bugs.launchpad.net/glance/+bug/1521756 | 23:10 |
| openstack | Launchpad bug 1521756 in Glance "race condition in test_upload_image_http_nonexistent_location_url: glance_store doesn't handle properly invalid HTTP reply" [Undecided,New] | 23:10 |
| haypo | lifeless: good night & thanks again | 23:10 |
| lifeless | haypo: good night! | 23:11 |
| *** TravT has joined #openstack-glance | 23:12 | |
| *** TravT_ has joined #openstack-glance | 23:12 | |
| *** kebray has joined #openstack-glance | 23:16 | |
| *** TravT has quit IRC | 23:16 | |
| *** kebray has quit IRC | 23:24 | |
| *** david-lyle has quit IRC | 23:25 | |
| *** TravT_ is now known as TravT | 23:27 | |
| *** darrenc is now known as darrenc_afk | 23:31 | |
| *** e0ne has joined #openstack-glance | 23:34 | |
| *** kebray has joined #openstack-glance | 23:34 | |
| *** kebray has quit IRC | 23:35 | |
| *** kebray has joined #openstack-glance | 23:36 | |
| *** e0ne has quit IRC | 23:36 | |
| *** kebray has quit IRC | 23:36 | |
| *** e0ne has joined #openstack-glance | 23:36 | |
| *** ayoung has joined #openstack-glance | 23:37 | |
| *** e0ne_ has joined #openstack-glance | 23:39 | |
| *** e0ne has quit IRC | 23:41 | |
| *** e0ne has joined #openstack-glance | 23:49 | |
| *** e0ne_ has quit IRC | 23:51 | |
| *** darrenc_afk is now known as darrenc | 23:52 | |
| *** e0ne_ has joined #openstack-glance | 23:52 | |
| *** e0ne has quit IRC | 23:54 | |
| *** Kennan has quit IRC | 23:55 | |
| *** Kennan has joined #openstack-glance | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!