Wednesday, 2015-03-11

Index Thursday, 2015-03-12 »
*** openstack has joined #openstack-nova16:55
dansmithmriedem: so I can see how this fixes something in a test where we pass by position, but why is it failing in the real world?16:55
*** andreykurilin_ has quit IRC16:56
mriedemhmm,16:57
*** takedakn has joined #openstack-nova16:57
mriedembuild_and_run_instance is called with kwargs and that calls _build_and_run_instance which is passed args,16:57
mriedemso it would fail on _build_and_run_instance, but *should* get handled with the decorator on build_and_run_instance16:58
*** n0ano has joined #openstack-nova16:58
*** cbader has quit IRC16:58
*** cbader has joined #openstack-nova16:59
mriedemdansmith: it's not clear to me which level of code the bug was reported against, it might be juno16:59
mriedembefore do_build_and_run_instance was there with the lock around it16:59
mriedemthat might not matter16:59
mriedemyou can see from the bug report though that it fails and the task_state isn't reverted17:00
mriedemif there is something else going on, hopefully i'll find it in the jenkins run with the warning log now17:00
dansmithoh, wait,17:00
mriedemmaybe we're getting InstanceNotFound or something, or UnexpectedTaskStateError somewhere?17:00
dansmith_do_build_and_run_instance() is always passed by position17:00
*** TobiasE has quit IRC17:00
mriedemyeah17:01
dansmithand so reverts_task_state would fail there17:01
mriedemright17:01
mriedembut...build_and_run_instance calls do_build_and_run_instance17:01
mriedemboth are decorated17:01
dansmithyeah, but the top level one won't catch it17:01
mriedemso i though the decorator on build_and_run_instance would be the final catch in this17:01
mriedemwhy is that?17:01
dansmithmriedem: because we swallow the exception and pass on the inner one?17:02
dansmithoh, but we save and reraise17:02
dansmithbut we've dispatched a thread17:02
*** Longgeek has quit IRC17:02
dansmithso the parent is gone17:02
dansmither, the caller I mean17:03
*** mwagner_lap has quit IRC17:03
*** iamjarvo has joined #openstack-nova17:03
dansmithmriedem: right?17:03
mriedemshrug17:03
dansmithI think that's the deal17:03
mriedemit's a pretty safe fix regardless of how we're getting here17:03
dansmithI think this was probably being handled on the outer layer until we made that change17:03
dansmithtotes, and I think it's probably real anyway17:03
*** salv-orlando has quit IRC17:03
mriedemand https://review.openstack.org/#/q/Ife712c43c5a61424bc68b2f5ab47cefdb46ac168,n,z17:04
jogoanyone want to review https://review.openstack.org/#/c/158356/16  part of cells v2 BP17:04
mriedemso we need that back to icehouse :)17:04
dansmithmriedem: really, was that fix in icehouse?17:05
mriedemdansmith: backported17:05
dansmithah17:05
mriedemihar loved it so gd much17:05
dansmithmriedem: then you're the winner of a trifecta17:05
mriedemf yeah17:05
*** dulek has quit IRC17:05
mriedemi don't think i get points for it though17:05
*** takedakn has quit IRC17:05
mriedemremember, no one gives a shit about stable since stackalytics doesn't track it :)17:06
*** garyk has quit IRC17:06
*** annashen has quit IRC17:07
*** pkoniszewski has quit IRC17:07
*** annashen has joined #openstack-nova17:07
*** rajesht has quit IRC17:08
dansmithheh17:08
openstackgerritPaul Murray proposed openstack/nova: Use compute_node consistently in ResourceTracker  https://review.openstack.org/15976517:08
openstackgerritPaul Murray proposed openstack/nova: Cleanups for pci stats in preparation for RT using ComputeNode  https://review.openstack.org/16184317:08
openstackgerritPaul Murray proposed openstack/nova: Convert RT compute_node to be a ComputeNode object  https://review.openstack.org/16094217:08
openstackgerritPaul Murray proposed openstack/nova: Move ComputeNode creation at init stage in ResourceTracker  https://review.openstack.org/14890417:08
*** pask81 has quit IRC17:08
*** takedakn has joined #openstack-nova17:09
jogodansmith: looking over your patch series https://review.openstack.org/#/c/160500/717:11
jogodansmith: why is passing context to remotable methods bad?17:12
dansmithheh, I really should write something up on this17:12
dansmithjogo: so there are a couple things. first of all, all remotables have to take a context as the first parameter, however, you don't have to call them with one17:12
dansmithand if you don't, the decorator inserts it for you17:13
dansmithit was a decision we made early on and have regretted ever since17:13
dansmithbecause it confuses people17:13
dansmithit looks like the call signature is different on the calling and receiving side17:13
dansmithand it's further confused because the object has an attached context with it17:13
dansmithso if you make a remotable call and pass the context, it overrides what is in the object17:14
*** harlowja_away is now known as harlowja_17:14
*** mgagne_PHL is now known as mgagne17:14
dansmiththis convention was stripped when this code was copied to ironic, and it is gone from oslo.versionedobjects17:14
jogoahh so the context is passed automatically17:14
dansmithso nova has to migrate away from it before we can drop it17:14
dansmithjogo: yes17:14
dansmithjogo: the next patch after this will be to remove the actual arg from all the call signatures, but I haven't gotten that far yet17:14
openstackgerrityunhong-jiang proposed openstack/nova: Use VirtCPUModel in compute resource tracker.  https://review.openstack.org/14813917:15
*** gszasz has quit IRC17:15
jogodansmith: ahh thanks17:15
dansmithbasically, that one bad decision early on has confused almost every new objects contributor, and now is confusing everyone that reviews the code to remove it17:16
dansmithso basically, I suck17:16
dansmithbut I'm on the good path to less sucking, which is all I can really offer :)17:16
dimsdansmith: mriedem: https://review.openstack.org/#/c/163426/ revved up - "VMware: Deprecation warning - map one nova-compute to one VC cluster"17:16
openstackgerritDavanum Srinivas (dims) proposed openstack/nova: VMware: Deprecation warning - map one nova-compute to one VC cluster  https://review.openstack.org/16342617:17
jogo:)17:17
mriedemdansmith: doesn't the context thing also somehow always awaken the ghost of comstud?17:18
*** matrohon has quit IRC17:18
dansmithmriedem: only to commiserate :)17:18
dansmithor congratulate, if it's about this patch set :)17:18
mriedembtw, how did rax not hit this earlier? https://review.openstack.org/#/c/163466/17:19
mriedemthat code merged in october17:19
dansmithmriedem: they're a tad behind I think17:19
dansmithmriedem: also, they do big bang updates last I heard17:20
mriedemops meetup said everyone does because neutron doesn't have rolling upgrades17:20
dansmithbut also not everyone is running neutron, so that didn't make sense to me17:21
mriedemrax is17:21
mriedemsome mutated form of it17:21
mriedemoh regarding the n-net others17:22
mriedemyeah idk17:22
dansmithyeah, but they don't roll new nova and neutron each time they roll I'm sure17:22
dansmithI think they do big bangs because of cells17:22
dansmithjogo: thanks for hitting those17:23
*** markus_z has quit IRC17:23
jogodansmith: np17:24
*** vmtrooper has joined #openstack-nova17:25
mriedemjogo: fyi https://review.openstack.org/#/c/163515/17:25
mriedemjogo: i know you've had some runaround with fixing this undeletable instance whack a mole bug17:26
*** ybathia has joined #openstack-nova17:27
jogomriedem: yeah those are the worst17:28
*** wendar_ is now known as wendar17:28
*** exploreshaifali has quit IRC17:29
*** vmtrooper has quit IRC17:31
jogodansmith: in return can you +W https://review.openstack.org/#/c/163102/17:32
jogothat makes the nova unit tests work again on my debian box17:32
dansmithugh17:32
melwittdo any api wizards know what's the deal with case sensitivity in our service urls? for example, if I create flavor named "BLah" I can query its details by a GET to flavors/blah or flavors/BLah but not flavors/Blah. does anyone know why?17:32
mriedemthis sounds like a seinfeld joke in the making17:32
mriedemmelwitt: v2 or v2.1?17:33
melwittmriedem: v2, current state17:33
melwittmriedem: I did some searching around online and I know case sensitivity depends on what web server you're running, but I couldn't find anything about ours. it's eventlet wsgi server if I understand correctly17:34
mriedemyeah we use eventlet17:34
alaskidansmith: mriedem yeah, we do big bang updates so we don't hit certain compatibility issues17:34
alaskiwell, not so that we don't hit them, we don't hit them because of that17:35
dansmithjogo: this has some process change stuff in service.py, did you look at that in detail?17:35
*** bkopilov has quit IRC17:35
*** kmartin has quit IRC17:37
*** Sukhdev has quit IRC17:37
*** takedakn has quit IRC17:37
jogodansmith: not really, being its oslo and all17:37
dansmithjogo: well, it changes some stuff17:38
dansmithjogo: like ratelimiting respawning children17:38
*** asselin has joined #openstack-nova17:38
dansmithI checked that we're not passing a different timeout there and only calling it from one place though17:38
dansmithso I guess it's okay, although dropping that delay seems strange17:38
dansmithdims: you know anything about that?17:38
*** bkopilov has joined #openstack-nova17:39
*** kmartin has joined #openstack-nova17:40
*** annashen has quit IRC17:41
dimsdansmith: there is some detail in the commit message. removing a sleep since we bumped eventlet min version17:41
*** annashen has joined #openstack-nova17:41
*** annegentle has joined #openstack-nova17:41
dimshttps://review.openstack.org/#/c/156345/17:41
*** igordcard_ has joined #openstack-nova17:42
*** devlaps has joined #openstack-nova17:42
dansmithdims: and we're delaying by hanging on the wait() call, I just figured out17:43
dansmitheven though this could still result in a tight loop, but it's not what I thought it was, so, that's fine17:43
*** nellysmitt has joined #openstack-nova17:46
*** mwagner_lap has joined #openstack-nova17:48
*** matrohon has joined #openstack-nova17:49
*** burt has quit IRC17:49
*** nellysmitt has quit IRC17:50
*** mtanino has joined #openstack-nova17:52
*** achanda has joined #openstack-nova17:54
openstackgerritDavid Moreau Simard proposed openstack/python-novaclient: Fix typo in socket attribute name  https://review.openstack.org/16354517:54
*** lpetrut has quit IRC17:55
*** bkopilov has quit IRC17:55
*** burt has joined #openstack-nova17:56
*** claudiub has quit IRC17:57
*** bkopilov has joined #openstack-nova17:57
*** otter768 has joined #openstack-nova18:01
*** exploreshaifali has joined #openstack-nova18:01
*** Longgeek has joined #openstack-nova18:02
*** kaufer has quit IRC18:03
*** abhishekk_ has joined #openstack-nova18:03
*** salv-orlando has joined #openstack-nova18:04
*** abhishekk_ has quit IRC18:04
*** vivekd has quit IRC18:05
*** abhishekk has joined #openstack-nova18:05
*** emagana has joined #openstack-nova18:05
*** otter768 has quit IRC18:06
mriedemjogo: i got a little shame jab in here at the end, just for you :) http://lists.openstack.org/pipermail/openstack-dev/2015-March/058839.html18:06
*** Longgeek has quit IRC18:09
jogomriedem: nice !18:09
*** Vek has joined #openstack-nova18:10
*** Maike has quit IRC18:11
*** pixelbeat has quit IRC18:12
*** takedakn has joined #openstack-nova18:12
*** kbyrne has quit IRC18:13
*** kbyrne has joined #openstack-nova18:14
*** salv-orlando has quit IRC18:14
jogoso for the quota out of sync stuff18:16
jogodo we know how to reproduce any issues18:16
*** BobBall is now known as BobBall_AWOL18:16
*** vigneshvar has joined #openstack-nova18:16
*** kaufer has joined #openstack-nova18:18
*** pkoniszewski has joined #openstack-nova18:20
jogomriedem: http://pypi.dfw.openstack.org/packages/source/p/pysaml2/pysaml2-2.3.0.tar.gz18:21
jogoderp18:21
*** aix has quit IRC18:21
jogomriedem: https://bugs.launchpad.net/nova/+bug/1186354 what happened to that one?18:21
openstackLaunchpad bug 1186354 in OpenStack Compute (nova) "Limits API doesn't work with Neutron" [Medium,Confirmed]18:21
jogodansmith: I am thinking of adding a quota tag to the bug list18:21
jogothoughts?18:21
dansmithjogo: in launchpad you mean?18:22
jogodansmith: yup18:23
jogoso its easier to track how many quota related issues we have etc18:24
jogoand make it more front and center18:24
*** dave-mccowan has joined #openstack-nova18:24
*** lucasagomes is now known as lucas-hungry18:26
*** lucas-hungry is now known as lucas-dinner18:26
openstackgerritEd Leafe proposed openstack/nova: Pass correct context to get_by_compute_node()  https://review.openstack.org/14561918:26
*** nellysmitt has joined #openstack-nova18:27
dansmithsure18:27
openstackgerritmelanie witt proposed openstack/python-novaclient: Add a test for the TCPKeepAliveAdapter  https://review.openstack.org/16355718:28
*** romainh has quit IRC18:29
*** lpetrut has joined #openstack-nova18:30
melwittjogo: +1 to quota bug tag18:31
*** jcoufal has quit IRC18:32
*** jaypipes is now known as jaypipes-afk18:34
*** vigneshvar has quit IRC18:35
*** abhishekk has left #openstack-nova18:37
jogojust added it as quotas18:37
jogoand starting to tag things18:39
melwitt\o/18:39
jogonext step is to re triage all those bugs18:44
jogoas there are duplicates and fixed things in there18:44
*** salv-orlando has joined #openstack-nova18:44
*** pkoniszewski has quit IRC18:45
*** jistr has quit IRC18:46
jogohttps://bugs.launchpad.net/nova/+bugs?field.tag=quotas18:46
melwittokay18:47
*** vigneshvar has joined #openstack-nova18:48
mriedemjogo: i was working on https://bugs.launchpad.net/nova/+bug/1186354 back in i think havana or icehouse and then dropped it because we needed to pull up the network API interface to a base class and have the API extensions talk to that rather than have the is_neutron() checks all over the API extensions18:48
openstackLaunchpad bug 1186354 in OpenStack Compute (nova) "Limits API doesn't work with Neutron" [Medium,Confirmed]18:48
mriedemjogo: having said that, https://bugs.launchpad.net/nova/+bug/1186354 is totally resurrectable18:48
mriedemjogo: maybe that's a smallish bp for L because i don't know if it needs to be a v2.1 microversion thing or not?18:49
mriedemit was being discussed kind of pre-v318:49
jogoI don't *think* it needs a new API does it?18:50
jogojust wire up the existing API?18:50
mriedemno it's not a new api extension18:50
mriedemyeah, we need a couple new network API methods under the covers18:50
mriedemand then have the API extensions call them18:50
mriedemnot too hard18:50
mriedemwould be a no-op for nova-network i think18:50
jogoanother option is to make sure this fails correctly when neutron is being used18:51
mriedemjogo: could18:51
jogoso make sure the error message is clear18:51
mriedemjogo: i seem to remember there also being things in tempest that needed to be changed to handle the neutron case or something18:51
jogohmm, well bbiab lunch time18:51
mriedemso in a nutshell, i can pick up https://review.openstack.org/#/c/43822/ again18:52
mriedemif desired, becaues it was controversial in havana b/c v3 was saying it wasn't going to proxy anything to neutron18:53
mriedemthat's no longer the case18:53
mriedemsmallish bp with a spec might be needed for an API change though18:55
*** patrickeast has joined #openstack-nova18:56
*** mtanino has quit IRC18:57
*** mtanino has joined #openstack-nova18:57
*** Marga_ has quit IRC18:58
*** nellysmitt has quit IRC18:59
*** nellysmitt has joined #openstack-nova18:59
*** marun has quit IRC19:00
yjiang5 mriedem : I have not much idea of the interactive with neutron, so will wait for baoli input for your question on https://review.openstack.org/#/c/98488/19:00
mriedemyjiang5: he said it seemed fine19:01
mriedemi also posted to the ML19:01
mriedemdeprecation patch coming soon19:01
yjiang5mriedem: got it.19:01
*** hartsocks has joined #openstack-nova19:02
*** exploreshaifali has quit IRC19:03
*** hartsocks has left #openstack-nova19:03
*** gokrokve_ has quit IRC19:04
*** rajesht has joined #openstack-nova19:05
*** rmoe has quit IRC19:06
*** sdake__ has joined #openstack-nova19:08
yjiang5dansmith: seems you always received question on your context remove patch. Does that mean your commit message is not clear enough?19:09
dansmithyjiang5: maybe, although I think it's because the major explanation was on earlier patches in the series, which have since been committed19:09
*** rajesht has quit IRC19:09
yjiang5dansmith: hmm, that makes sense.19:10
*** rushiagr is now known as rushiagr_away19:10
*** rmoe has joined #openstack-nova19:11
*** sdake_ has quit IRC19:12
*** romainh has joined #openstack-nova19:14
*** vmtrooper has joined #openstack-nova19:14
*** annashen has quit IRC19:15
*** romainh1 has joined #openstack-nova19:17
*** romainh has quit IRC19:17
*** romainh1 has quit IRC19:17
*** romainh has joined #openstack-nova19:17
*** vmtrooper has quit IRC19:19
*** romainh has left #openstack-nova19:21
*** romainh has joined #openstack-nova19:23
*** diegows has joined #openstack-nova19:23
*** achanda has quit IRC19:27
*** emagana has quit IRC19:29
*** sbfox has quit IRC19:29
*** annashen has joined #openstack-nova19:30
*** sbfox has joined #openstack-nova19:30
*** Nikolay_St has joined #openstack-nova19:34
openstackgerritMatt Riedemann proposed openstack/nova: neutron: deprecate 'allow_duplicate_networks' config option  https://review.openstack.org/16358119:35
mriedemyjiang5: russellb: jogo: ^19:35
mriedemijw: ^19:35
openstackgerritJoshua Harlow proposed openstack/nova: Reopen socket if its determined to be bad  https://review.openstack.org/16358219:36
ijw?19:36
ijwAh19:36
ijwYeah, I think that's there for backward compatibility (but I don't really like it)19:37
openstackgerritJoshua Harlow proposed openstack/nova: Reopen socket if its determined to be bad (WIP)  https://review.openstack.org/16358219:37
mriedembackwards compat didn't make sense though19:37
mriedemif you tried this before it failed19:37
mriedemso most people wouldn't write their apps to try it19:37
*** kmartin has quit IRC19:37
mriedemnow it's possible, but neutered by default19:37
openstackgerritJoshua Harlow proposed openstack/nova: Reopen socket if its determined to be bad (WIP)  https://review.openstack.org/16358219:38
mriedemand as far as i can tell, it's not related to anything in nova-network at all19:38
*** achanda has joined #openstack-nova19:38
ijwYeah - but that is strict backward compatibility - before you got an error, now you get an error unless you do something19:38
ijwI don't think it would be terrible to break that compatibility but that's the argument for it19:38
mriedemijw: our api change guidelines say it's OK to change something that used to result in an error to now return success19:39
ijwok - then your patch is clearly good ;)19:39
ijwNot sure that config item has made it to a release version either19:39
mriedemijw: it was in juno19:40
mriedemnever tested19:40
ijwdammit19:40
mriedemso i'm also working on that19:40
openstackgerritMatt Riedemann proposed openstack/nova: neutron: deprecate 'allow_duplicate_networks' config option  https://review.openstack.org/16358119:40
*** takedakn has quit IRC19:45
*** johnthetubaguy is now known as zz_johnthetubagu19:45
*** nellysmitt has quit IRC19:46
*** baoli has quit IRC19:46
*** annegentle has quit IRC19:46
*** annegentle has joined #openstack-nova19:47
*** irenab has joined #openstack-nova19:48
ijwmriedem: did you see my comments on multinic?19:49
ijwNot terribly positive comments but I was trying to mentally get the shape of the problem to begin with19:49
*** nelsnelson has quit IRC19:50
*** claudiub has joined #openstack-nova19:51
*** vigneshvar has quit IRC19:51
*** adelinatuvenie has quit IRC19:51
mriedemijw: that bug report? yeah, and rasha's19:52
*** annegentle has quit IRC19:52
mriedemcan't say i understand it all though19:53
mriedembaby steps i guess, i'd like to get tempest tests passing with creating a server with multiple ports from the same network19:53
mriedemthen work on the add/remove fixed_ips stuff19:53
ijwWell, the basic issues is 1: the multinic plugin has bugger all documentation - I particularly like that it's about multiple NICs and yet both its APIs are fixed IP ones (and adding a second fied IP, at that, which is neither a multinic concern nor actually terribly useful)19:53
*** Longgeek has joined #openstack-nova19:54
ijwand 2: it assumes one NIC per network because it's really a nova-network extension, and since that's no longer true of Neutron the answer may be just to make it inapplicable to Neutron19:54
*** irenab has quit IRC19:55
*** gokrokve has joined #openstack-nova19:55
ijwDunno.  I'm into Neutron so I guess I just find it odd that people would drive Neutron from a Nova API that is a thin shim over a Neutron one19:55
*** sahid has quit IRC19:57
*** sahid has joined #openstack-nova19:58
*** scottda has left #openstack-nova19:58
*** mpaolino has joined #openstack-nova19:59
*** mpaolino has quit IRC19:59
*** Longgeek has quit IRC20:00
*** otter768 has joined #openstack-nova20:01
mriedemijw: yeah...thought about just raising NotImplemented for neutron,20:02
mriedembut someone implemented for some reason, even if it's not useful20:02
mriedemchanging now would be a backwards incompatiable api breakage20:02
mriedemespecially considering the mulitnic api extension is only testing in tempest if you're using neutron20:03
mriedemso we're really actually testing the wrong thing20:03
*** exploreshaifali has joined #openstack-nova20:03
ijw'It must be there for a reason' is not an argument to keep code.  'It must be there for backward compatibility' I would accept...20:04
*** kashyap has quit IRC20:05
*** dboik_ has quit IRC20:06
*** otter768 has quit IRC20:06
*** mspreitz has joined #openstack-nova20:07
*** dboik has joined #openstack-nova20:07
ijwmriedem: you don't feel that one multinic bug would have been enough?20:07
*** kashyap has joined #openstack-nova20:07
*** thangp has quit IRC20:08
mriedemchecking git history20:08
*** annegentle has joined #openstack-nova20:08
mriedemhttps://github.com/openstack/nova/commit/1ff3afc67f97ff28c6ca1c3884a90b97ee1cf9dd20:09
mriedemgaryk, i should have known!20:09
jogodansmith: quick sanity check -- we still count all instances in error state against quotas20:09
mriedemwell, this is about as sparse a commit message as they come https://review.openstack.org/#/c/19627/20:09
mriedemjogo: yeah20:10
dansmithjogo: I believe that's correct20:10
jogoeven if its from no host available20:10
mriedemsomeone was in here awhile ago that disagreed with that20:10
mriedemjogo: not sure about that20:10
jogore-triaging https://bugs.launchpad.net/nova/+bug/1308613 as opinion20:10
openstackLaunchpad bug 1308613 in OpenStack Compute (nova) "Quota usage should be updated if there's no valid host to run an instance" [Undecided,Confirmed] - Assigned to Santiago Baldassin (santiago-b-baldassin)20:10
*** vigneshvar has joined #openstack-nova20:12
edleafejaypipes-afk: are you planning on re-submitting https://review.openstack.org/149923?20:14
ijwmriedem: you know, I'm fairly sure that ipv6 subnets add a new and horrifying dimension to your bug.  I'm not sure I can bring myself to unleash that upon the world.20:16
mriedemijw: fun20:17
ijwmriedem: also, bloody garyk20:17
ijwall his fault20:17
mriedemthe tempest test, btw, is pretty dumb20:17
mriedemit adds a fixed ip and then removes it20:17
mriedemit doesn't test anything about it in the interim20:17
mriedemor validate i should say20:18
mriedemas as long as the API doesn't explode all over itself, it passes20:18
*** Marga_ has joined #openstack-nova20:20
*** pkoniszewski has joined #openstack-nova20:21
ijwNo comments or anything on that commit.  Just two core reviewers and that's it20:23
ijw(the nova one)20:24
ijwI suspect it was back in the day when no-one really deployed Neutron20:24
ijwAnd it works when it works, anyway20:24
mriedemkrtaylor: is powerkvm ci supposed to be running on stable branches? https://review.openstack.org/#/c/163456/20:25
mriedemijw: yeah i suspect that was the case20:25
mriedemfeature parity is a win20:25
dansmithmriedem: dims did what you asked here, are you good now? https://review.openstack.org/#/c/163426/20:27
jogodims: want a patch to work on? https://review.openstack.org/#/c/79904/20:28
jogofixes a quota bug https://bugs.launchpad.net/nova/+bug/129119920:28
openstackLaunchpad bug 1291199 in OpenStack Compute (nova) "update quota multi-value in one request, half done half failed" [Low,Confirmed]20:28
jogoauthor walked away20:28
mriedemdansmith: looking20:29
*** kmartin has joined #openstack-nova20:29
*** mwagner_lap has quit IRC20:30
mriedemdansmith: +2, didn't +W though20:32
dansmiththat's fine20:33
*** jistr has joined #openstack-nova20:35
*** sahid has quit IRC20:36
*** ndipanov has quit IRC20:36
*** marun has joined #openstack-nova20:37
dimsjogo: ack20:37
jogodims: I am trying to get a handle on our quota based bugs https://bugs.launchpad.net/nova/+bugs?field.tag=quotas20:38
dimsjogo: can't see a restore button20:38
jogoadded that tag today20:38
dimsfeeback from the ops meetup i presume20:39
jogorestored20:39
jogodims: yup20:39
dimsthanks jogo20:39
jogodims: thank you!20:39
dimsjogo: wrapping up something pick it up in a little bit20:39
jogodims: cool no rush20:39
*** jwang_ has quit IRC20:40
jogodansmith: didn't we fix this on master https://bugs.launchpad.net/nova/+bug/142115520:40
openstackLaunchpad bug 1421155 in OpenStack Compute (nova) "project instance limit also applicable for compute node" [Undecided,Incomplete]20:40
claudiubmriedem: hello, thanks for the review on https://review.openstack.org/#/c/138934/ . I've responded to your comment. :)20:41
*** salv-orlando has quit IRC20:43
jogohmm that bug is super vague20:43
*** takedakn has joined #openstack-nova20:43
openstackgerritChris Friesen proposed openstack/nova: Fix nova-compute service spuriously marked as "up"  https://review.openstack.org/16306020:44
openstackgerritDave McCowan proposed openstack/nova: Websocket Proxy should verify Origin header  https://review.openstack.org/16360420:45
*** Mike_D_laptop has joined #openstack-nova20:46
openstackgerritAndrew Laski proposed openstack/nova: WIP(tests) Add CellMapping object  https://review.openstack.org/16186720:46
openstackgerritAndrew Laski proposed openstack/nova: WIP Split out deleted/deleted_at from NovaPersistentObject  https://review.openstack.org/16360520:46
alaskidansmith: I would appreciate your thoughts/comments on https://review.openstack.org/163605 when you get a chance20:46
jaypipes-afkedleafe: yeah.20:49
openstackgerritJay Pipes proposed openstack/nova: Fix up join() and leave() methods of servicegroup  https://review.openstack.org/14992320:49
*** irenab has joined #openstack-nova20:50
*** sdake__ has quit IRC20:51
*** sdake_ has joined #openstack-nova20:53
*** jwang_ has joined #openstack-nova20:55
*** vladikr has quit IRC20:55
*** achanda has quit IRC20:55
*** mtanino has quit IRC20:56
openstackgerritmelanie witt proposed openstack/python-novaclient: Add a test for the TCPKeepAliveAdapter  https://review.openstack.org/16355720:56
*** thomasem has quit IRC20:57
*** raildo has quit IRC20:58
*** mspreitz has quit IRC21:00
openstackgerritJoe Gordon proposed openstack/python-novaclient: Update help message for nova boot --file  https://review.openstack.org/16360921:01
*** dprince has quit IRC21:02
openstackgerritmelanie witt proposed openstack/python-novaclient: Add a test for the TCPKeepAliveAdapter  https://review.openstack.org/16355721:03
*** vmtrooper has joined #openstack-nova21:03
*** thomasem has joined #openstack-nova21:03
*** thomasem_ has joined #openstack-nova21:05
*** r-daneel has quit IRC21:05
krtaylormriedem, re: powerkvm - yes, I believe so, looking at failure now21:05
*** jcru has quit IRC21:06
*** irenab has quit IRC21:07
*** jcru has joined #openstack-nova21:07
openstackgerritDave McCowan proposed openstack/nova: Websocket Proxy should verify Origin header  https://review.openstack.org/16303321:08
*** jcru has quit IRC21:08
*** vmtrooper has quit IRC21:08
*** thomasem has quit IRC21:08
jogolifeless: ping21:09
lifelesspong21:09
jogoquota issues21:09
jogohttps://bugs.launchpad.net/nova/+bug/128442421:09
openstackLaunchpad bug 1284424 in tripleo "nova quota statistics can be incorrect" [High,Triaged]21:09
*** tellesnobrega_ has joined #openstack-nova21:09
jogolifeless: quota issues came up as one of the top issues at the ops meetup21:09
*** eharney has quit IRC21:10
jogothe catch is AFAIK no one has a way of reproducing the issues21:10
jogowas wondering if you knew of any ways21:10
lifelessrun a cloud :/21:10
*** jistr has quit IRC21:10
lifelessparticularly one with occasional fragility around in-cloud operations21:11
lifelesstimeouts, failed tasks etc will trigger it quickly IME21:11
openstackgerritjichenjc proposed openstack/nova: Fix a typo of devref document for api_plugin  https://review.openstack.org/16361221:11
*** tellesnobrega_ has quit IRC21:11
lifelessI think its a bit structural though21:11
jogohmm timeouts and failed tasks21:11
jogolifeless: right, so just trying to come up with a test case that will work21:11
jogoeither nova functional test or devstack based test21:11
lifelesswe don't really treat quotas as a resource that can skew in the same way we do e.g. neutron ports or hypervisor stats21:11
*** aysyd has quit IRC21:12
*** andreykurilin has quit IRC21:12
jogoright now I don't care about the why its bad just how to show its broken21:12
*** Mike_D_laptop has quit IRC21:12
*** ozamiatin has quit IRC21:13
*** kudryashova has quit IRC21:13
*** Mike_D_wk has quit IRC21:13
*** kudryashova has joined #openstack-nova21:14
lifelessjogo: sure21:14
*** ozamiatin has joined #openstack-nova21:14
*** Mike_D_wk has joined #openstack-nova21:14
*** matrohon has quit IRC21:16
openstackgerritRick Harris proposed openstack/nova: Fix infinite recursion caused by unnecessary stub  https://review.openstack.org/16361421:16
openstackgerritDave McCowan proposed openstack/nova: Websocket Proxy should verify Origin header  https://review.openstack.org/16303321:16
*** andreykurilin has joined #openstack-nova21:16
dansmithalaski: seems fine on the surface, but I'll take a closer look21:16
alaskidansmith: cool.  I think it's pretty straightforward, but wanted to run it past you in case it led to a "it would be much easier to just do this two line thing here..."21:17
dansmithalaski: well, I haven't looked, but I think you could do this without changing all the other objects21:19
dansmithtwo mixins that add deleted and the other bits, and then make NovaPersistentObject a mixin of those mixins21:19
alaskiI did consider that, but thought it might be odd if these new objects were db backed but didn't inherit from NovaPersistentobject21:22
jogolifeless: any ideas on  a good way to reproduce?21:22
*** neelashah has quit IRC21:22
lifelessjogo: run zuul against a 4-5 node cloud with a flaky control plane for a few weeks is the only reliable way I have21:23
jogoa more concise way21:24
jogoalthough that is interesting21:24
jogoso just boots and deletes are enough to do it21:24
dansmithalaski: well, I don't think it would be, but maybe we just need to rename persistent anyway..21:25
dansmithalaski: or just use the existing persistent object and keep the extra fields for "consistency" :)21:25
alaskidansmith: heh :)21:26
*** dkliban is now known as dkliban_afl21:27
*** dkliban_afl is now known as dkliban_afk21:27
*** achanda has joined #openstack-nova21:28
*** salv-orlando has joined #openstack-nova21:28
alaskidansmith: I like not changing all of the other objects so I'll split NovaPersistentObject into a combination of mixins for now and perhaps we can rename it later21:29
*** annashen has quit IRC21:32
*** boris-42 has quit IRC21:32
*** ijw has quit IRC21:34
*** angdraug has joined #openstack-nova21:34
*** penick has joined #openstack-nova21:34
*** mmedvede has quit IRC21:34
jogolifeless: first attempt http://paste.ubuntu.com/10582352/21:38
jogofailed, no issues21:38
lifelesshah no21:39
*** annashen has joined #openstack-nova21:39
lifelessyou're going to need several thousand spawn and delete events21:39
jogolifeless: ahh21:39
lifelessand the controller flake (e.g. rabbit and or db and or conductor) to reproduce I expect21:39
lifelessthe bugs are not in the happy path21:39
lifelessits all going to be in error and race handling21:39
jogoI'll swap out the fake virt driver21:40
lifelessoh also concurrency21:40
jogoswap in21:40
lifelessyou'll want 10's of concurrent spawn/deletes happening at once21:40
dansmithalaski: okay21:40
jogoso do you think the same basic pattern but with a lot more concurrency would do it?21:41
*** baoli has joined #openstack-nova21:42
lifelesswhat does absolute-limits do ?21:43
*** apuimedo has quit IRC21:43
jogoshow current quota usage21:43
jogoprinting it to track things a bit21:43
lifelessso, it *may* be that we have a trivial race, in which case the same pattern with more concurrency may well do it.21:43
lifelessI'd chat to pcrews who has an API fuzzer21:44
*** ociuhandu has quit IRC21:44
jogoif I ramp up concurrency enough I may be able to tirgger timeouts too21:44
*** baoli has quit IRC21:44
jogohmm I'll try this first and go from there21:44
lifelesswhich would let you write an assertion to probe for the failure at each step21:44
lifelesshe's found a bunch of races with it21:44
jogooh nice21:45
*** baoli has joined #openstack-nova21:45
lifelesse.g. https://bugs.launchpad.net/nova/+bug/139292321:45
openstackLaunchpad bug 1392923 in OpenStack Compute (nova) "Orphan floating ip's created via rapid delete/assign/remove operations" [High,In progress] - Assigned to Marian Horban (mhorban)21:45
*** sdake_ has quit IRC21:46
lifelessI think its probably well worth your time to poke at that tool (rannsaka)21:46
dimssdague: can you please clear the -W on this one? https://review.openstack.org/#/c/79904/21:46
*** sdake_ has joined #openstack-nova21:48
*** amotoki has quit IRC21:50
*** kaufer has quit IRC21:51
*** mriedem is now known as mriedem_away21:55
*** iamjarvo has quit IRC21:58
*** sdake_ has quit IRC22:00
openstackgerritRick Harris proposed openstack/nova: Fix infinite recursion caused by unnecessary stub  https://review.openstack.org/16361422:01
*** marun has quit IRC22:01
*** lpetrut has quit IRC22:02
*** otter768 has joined #openstack-nova22:02
*** exploreshaifali has quit IRC22:04
*** burt has quit IRC22:05
*** annegentle has quit IRC22:05
*** otter768 has quit IRC22:07
*** baoli has quit IRC22:08
*** Nikolay_St has quit IRC22:10
*** dboik has quit IRC22:11
*** dims has quit IRC22:11
*** baoli has joined #openstack-nova22:13
*** penick has quit IRC22:14
*** ijw has joined #openstack-nova22:15
*** ijw has quit IRC22:15
*** lucas-dinner has quit IRC22:16
*** ijw has joined #openstack-nova22:17
*** Nic has quit IRC22:19
*** oro_ has quit IRC22:20
*** mmedvede has joined #openstack-nova22:21
*** oro has quit IRC22:21
*** david8hu has joined #openstack-nova22:21
*** annegentle has joined #openstack-nova22:25
*** vladikr has joined #openstack-nova22:30
*** annashen has quit IRC22:30
*** asettle is now known as asettle-mtg22:30
*** annashen has joined #openstack-nova22:30
*** jaypipes-afk is now known as jaypipes22:31
*** exploreshaifali has joined #openstack-nova22:31
*** adalbas has quit IRC22:31
*** dboik has joined #openstack-nova22:34
*** annegentle has quit IRC22:34
*** dboik_ has joined #openstack-nova22:34
openstackgerritJamie Lennox proposed openstack/nova: Use session with neutronclient  https://review.openstack.org/13693122:35
*** jgrimm is now known as zz_jgrimm22:36
*** dboik has quit IRC22:38
*** tellesnobrega_ has joined #openstack-nova22:40
*** pkoniszewski has quit IRC22:41
*** tellesnobrega_ has quit IRC22:41
*** nikhil_k is now known as _pope22:41
openstackgerritEd Leafe proposed openstack/nova: Implement instance update logic in Scheduler  https://review.openstack.org/16051122:42
openstackgerritEd Leafe proposed openstack/nova: Add the instance update calls from Compute  https://review.openstack.org/16051322:42
openstackgerritEd Leafe proposed openstack/nova: Modify filters to get instance info from HostState  https://review.openstack.org/16051222:42
openstackgerritEd Leafe proposed openstack/nova: Add the RPC calls for instance updates.  https://review.openstack.org/16050722:42
openstackgerritEd Leafe proposed openstack/nova: Add InstanceList.get_all method  https://review.openstack.org/16050322:42
*** _pope is now known as nikhil_k22:43
*** pixelbeat has joined #openstack-nova22:43
*** thedodd has quit IRC22:44
*** Longgeek has joined #openstack-nova22:45
*** dims_ has joined #openstack-nova22:46
*** Nikolay_St has joined #openstack-nova22:48
*** amotoki has joined #openstack-nova22:50
*** jecarey has quit IRC22:51
*** annashen has quit IRC22:52
*** vmtrooper has joined #openstack-nova22:52
*** Longgeek has quit IRC22:55
openstackgerritRick Harris proposed openstack/nova: network: Fix another IPv6 test for Mac  https://review.openstack.org/16363622:55
*** sdake_ has joined #openstack-nova22:55
*** amotoki has quit IRC22:56
*** penick has joined #openstack-nova22:56
*** penick has quit IRC22:56
*** vmtrooper has quit IRC22:57
*** sdake__ has joined #openstack-nova22:57
*** baoli has quit IRC22:59
*** sdake_ has quit IRC23:00
*** vigneshvar has quit IRC23:03
*** apuimedo has joined #openstack-nova23:04
*** patrickeast_ has joined #openstack-nova23:05
*** sdake__ has quit IRC23:06
*** patrickeast has quit IRC23:06
*** patrickeast_ is now known as patrickeast23:06
*** sdake_ has joined #openstack-nova23:09
*** exploreshaifali has quit IRC23:10
*** annashen has joined #openstack-nova23:12
*** dave-mccowan has quit IRC23:14
*** sdake_ has quit IRC23:14
*** Nikolay_St has quit IRC23:16
*** Nikolay_St has joined #openstack-nova23:18
*** Nikolay_St has quit IRC23:22
*** annashen has quit IRC23:24
*** sdake_ has joined #openstack-nova23:26
*** jecarey has joined #openstack-nova23:28
*** Nikolay_St has joined #openstack-nova23:29
openstackgerritEd Leafe proposed openstack/nova: Remove useless get_one() method in SG API  https://review.openstack.org/14992423:30
openstackgerritEd Leafe proposed openstack/nova: Fix up join() and leave() methods of servicegroup  https://review.openstack.org/14992323:30
*** annegentle has joined #openstack-nova23:32
*** sbfox has quit IRC23:33
*** FL1SK has joined #openstack-nova23:33
*** IanGovett has quit IRC23:39
*** annegentle has quit IRC23:43
*** Nikolay_St has quit IRC23:43
*** penick has joined #openstack-nova23:48
*** romainh has left #openstack-nova23:50
*** mtanino has joined #openstack-nova23:52
*** vladikr has quit IRC23:55
*** patrickeast has quit IRC23:58
*** penick has quit IRC23:59
*** Nikolay_St has joined #openstack-nova23:59
Index Thursday, 2015-03-12 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!