| *** tjakobs has quit IRC | 00:02 | |
| *** chas has quit IRC | 00:09 | |
| *** chas has joined #openstack-powervm | 00:09 | |
| *** chas has quit IRC | 00:14 | |
| *** jwcroppe has joined #openstack-powervm | 00:21 | |
| *** jwcroppe has quit IRC | 00:49 | |
| *** jwcroppe has joined #openstack-powervm | 00:49 | |
| *** esberglu_ has quit IRC | 01:11 | |
| *** esberglu has joined #openstack-powervm | 01:11 | |
| *** esberglu has quit IRC | 01:16 | |
| *** svenkat has joined #openstack-powervm | 01:33 | |
| *** thorst_ has joined #openstack-powervm | 01:46 | |
| *** esberglu has joined #openstack-powervm | 01:56 | |
| *** esberglu has quit IRC | 02:30 | |
| *** esberglu has joined #openstack-powervm | 02:30 | |
| *** esberglu has quit IRC | 02:34 | |
| openstackgerrit | Sridhar Venkat proposed openstack/networking-powervm: ProvisionRequest does not distinguish event source https://review.openstack.org/396467 | 02:45 |
|---|---|---|
| *** seroyer has joined #openstack-powervm | 03:01 | |
| *** tlian has quit IRC | 03:18 | |
| *** thorst_ has quit IRC | 03:24 | |
| *** thorst_ has joined #openstack-powervm | 03:26 | |
| *** seroyer has quit IRC | 03:31 | |
| *** thorst_ has quit IRC | 03:32 | |
| *** svenkat has quit IRC | 03:35 | |
| *** jwcroppe has quit IRC | 03:54 | |
| *** jwcroppe has joined #openstack-powervm | 03:55 | |
| *** thorst_ has joined #openstack-powervm | 04:30 | |
| *** thorst_ has quit IRC | 04:38 | |
| *** kotra03 has joined #openstack-powervm | 05:16 | |
| *** thorst_ has joined #openstack-powervm | 05:36 | |
| *** thorst_ has quit IRC | 05:43 | |
| *** chas has joined #openstack-powervm | 05:54 | |
| *** chas has quit IRC | 06:01 | |
| *** chas has joined #openstack-powervm | 06:06 | |
| *** chas has quit IRC | 06:11 | |
| *** esberglu has joined #openstack-powervm | 06:25 | |
| *** esberglu has quit IRC | 06:29 | |
| *** esberglu has joined #openstack-powervm | 06:30 | |
| *** esberglu has quit IRC | 06:35 | |
| *** chas has joined #openstack-powervm | 06:38 | |
| *** thorst_ has joined #openstack-powervm | 06:42 | |
| *** chas has quit IRC | 06:43 | |
| *** thorst_ has quit IRC | 06:48 | |
| *** esberglu has joined #openstack-powervm | 07:20 | |
| *** esberglu has quit IRC | 07:24 | |
| *** thorst_ has joined #openstack-powervm | 07:45 | |
| *** thorst_ has quit IRC | 07:53 | |
| *** esberglu has joined #openstack-powervm | 08:19 | |
| *** esberglu has quit IRC | 08:24 | |
| *** thorst_ has joined #openstack-powervm | 08:51 | |
| *** thorst_ has quit IRC | 08:58 | |
| *** esberglu has joined #openstack-powervm | 09:14 | |
| openstackgerrit | Arun Mani proposed openstack/nova-powervm: Object upload using swift occasionally fails with 'TypeError' exception. https://review.openstack.org/398868 | 09:16 |
| *** esberglu has quit IRC | 09:18 | |
| *** YuYangWang has quit IRC | 09:33 | |
| *** openstackgerrit has quit IRC | 09:48 | |
| *** openstackgerrit has joined #openstack-powervm | 09:49 | |
| *** thorst_ has joined #openstack-powervm | 09:56 | |
| *** thorst_ has quit IRC | 10:02 | |
| *** esberglu has joined #openstack-powervm | 10:09 | |
| *** esberglu has quit IRC | 10:13 | |
| *** k0da has joined #openstack-powervm | 10:40 | |
| *** esberglu has joined #openstack-powervm | 10:59 | |
| *** thorst_ has joined #openstack-powervm | 11:01 | |
| *** esberglu has quit IRC | 11:03 | |
| *** thorst_ has quit IRC | 11:08 | |
| *** esberglu has joined #openstack-powervm | 11:59 | |
| *** esberglu has quit IRC | 12:04 | |
| *** thorst_ has joined #openstack-powervm | 12:06 | |
| *** thorst_ has quit IRC | 12:12 | |
| *** seroyer has joined #openstack-powervm | 12:23 | |
| *** edmondsw has joined #openstack-powervm | 12:48 | |
| *** thorst_ has joined #openstack-powervm | 12:51 | |
| *** esberglu has joined #openstack-powervm | 12:54 | |
| *** esberglu has quit IRC | 12:58 | |
| *** seroyer has quit IRC | 13:01 | |
| *** seroyer has joined #openstack-powervm | 13:03 | |
| *** svenkat has joined #openstack-powervm | 13:08 | |
| *** seroyer has quit IRC | 13:14 | |
| *** esberglu has joined #openstack-powervm | 13:27 | |
| *** wangqwsh has joined #openstack-powervm | 13:30 | |
| thorst_ | CI? | 13:30 |
| adreznec | What's a CI? Never heard of it | 13:31 |
| esberglu | #startmeeting ci_scrum | 13:31 |
| openstack | Meeting started Thu Nov 17 13:31:23 2016 UTC and is due to finish in 60 minutes. The chair is esberglu. Information about MeetBot at http://wiki.debian.org/MeetBot. | 13:31 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 13:31 |
| openstack | The meeting name has been set to 'ci_scrum' | 13:31 |
| esberglu | #topic status | 13:31 |
| adreznec | In the future we should do powervm ci scrum, then all the logs go under the same directory out on eavesdrop | 13:32 |
| efried | o/ | 13:32 |
| thorst_ | \o | 13:32 |
| adreznec | o | 13:32 |
| adreznec | Haven't had enough coffee to raise hand | 13:32 |
| esberglu | I looked at the dir online and it just says ci_scrum? | 13:33 |
| adreznec | Yeah, in the past I was doing powervm ci meeting | 13:34 |
| adreznec | So they were going to http://eavesdrop.openstack.org/meetings/powervm_ci_meeting/ | 13:34 |
| adreznec | If you keep giving it the same name each time they all go to the same directory | 13:34 |
| adreznec | Not a big deal | 13:34 |
| esberglu | Oh I had the wrong bookmark | 13:35 |
| esberglu | http://eavesdrop.openstack.org/meetings/ci_scrum/ | 13:35 |
| esberglu | We must have done it there once or something | 13:35 |
| esberglu | Anyways | 13:35 |
| thorst_ | Status - stop everything until CI is running? | 13:35 |
| esberglu | I did end up getting a run to go through last night | 13:35 |
| esberglu | But still seeing the same behavior | 13:36 |
| thorst_ | elaborate? The pypowervm issue certainly isn't fixed | 13:36 |
| esberglu | Yeah. But I put USE_CONSTRAINTS=false | 13:36 |
| thorst_ | that worked? | 13:36 |
| esberglu | Didn't we think that would work around? | 13:36 |
| esberglu | It did not work | 13:37 |
| thorst_ | we thought it might. | 13:37 |
| thorst_ | is everyone aware what the pypowervm issue is? | 13:37 |
| thorst_ | adreznec? I'm not sure if you were yet | 13:37 |
| adreznec | So we're still seeing constraints issues? | 13:37 |
| adreznec | Why here and not in the upstream gate? | 13:37 |
| thorst_ | devstack installs from the requirements file. Which then blows away our local install (with the local2remote patch) | 13:37 |
| adreznec | Hmm | 13:38 |
| thorst_ | before we dive into that (and I'd like to do a deep dive on that) | 13:39 |
| thorst_ | are there any other items we need to chase esberglu? | 13:39 |
| thorst_ | otherwise the meeting will get away on us with that rabbit hole | 13:39 |
| esberglu | I still need to do work on the post stack vm cleaner script | 13:39 |
| esberglu | And neo 6,7,8 stop working last night. Probably just need to restart vio_daemon | 13:40 |
| thorst_ | esberglu: was that just the lu clean up? | 13:41 |
| adreznec | Just... stopped working? | 13:41 |
| adreznec | without the CI running/doing anything? | 13:41 |
| thorst_ | adreznec: apearson is adding in a 'heartbeat' of sorts for that. | 13:41 |
| * adreznec sighs | 13:41 | |
| thorst_ | yuh | 13:41 |
| thorst_ | so manual restarts for now, update of novalink in future should help | 13:41 |
| esberglu | thorst_: It looked like the LU thing. It happened after I was clearing out the nodes so I could deploy with USE_CONSTRAINTS | 13:41 |
| esberglu | But it wouldn't let me delete the LUs or the VMs | 13:42 |
| esberglu | I didn't spend too much time looking at it, was more interested in getting a run through the CI | 13:44 |
| thorst_ | esberglu: alright. | 13:45 |
| thorst_ | #action thorst to follow up with apearson on vio_daemon thing | 13:45 |
| thorst_ | #action esberglu to work on the clean up script for lu. Possibly restart vio_daemon depending on how far along that is in NL | 13:45 |
| thorst_ | any other items? | 13:46 |
| *** seroyer has joined #openstack-powervm | 13:47 | |
| *** tlian has joined #openstack-powervm | 13:48 | |
| esberglu | Not from me | 13:48 |
| esberglu | You guys ready to talk about the pypowervm versioning then? | 13:49 |
| thorst_ | esberglu: do you have a link to a run that we saw this issue/ | 13:49 |
| adreznec | So if I understand correctly it's happening because pypowervm is now a global-req, which because our stuff now as a real requirement.txt requirement on pypowervm is causing pypowervm to be installed down the develop branch path I put in, erasing our pre-installed version of pypowervm | 13:50 |
| thorst_ | adreznec: well, it doesn't erase it | 13:50 |
| esberglu | 9.114.251.93:8080/job/nova-powervm-pvm-dsvm-tempest-full/304/consoleFull | 13:50 |
| thorst_ | but yeah, effectively. | 13:50 |
| esberglu | Crap not what I meant to paste | 13:51 |
| thorst_ | heh | 13:51 |
| esberglu | http://184.172.12.213/15/328315/20/check/nova-powervm-pvm-dsvm-tempest-full/84aad2b/ | 13:51 |
| adreznec | Does it not install over the top of it? | 13:52 |
| thorst_ | adreznec: Search for this timestamp: 2016-11-16 20:27:46.703 | 13:52 |
| efried | It would be relatively easy to fix this in our CI - our scripts download the requirements project first, then hack the pypowervm lines out of global-requirements.txt and upper-constraints.txt, then proceed with the rest of devstack - but fixing it for our devstack in general is going to be tougher. | 13:52 |
| efried | Here's one suggestion, not particularly pretty: | 13:52 |
| efried | We start by re-syncing 1.0.0.4 into 1.0.0.3.9. | 13:52 |
| efried | Then moving forward, we write any new pypowervm function into nova-powervm itself (or whichever out-of-tree driver needs it), while continuing to develop against 1.0.0.4. | 13:52 |
| efried | Then periodically, we release a new version of pypowervm to pip (1.0.0.3.9.1, 1.0.0.3.9.2, ... just kidding) and update the global requirements. | 13:52 |
| efried | Once that merges, we update nova-powervm accordingly, removing that functionality and repointing its usage from the temporary nova-powervm code to the released pypowervm code. | 13:52 |
| thorst_ | the challenge with that is that we don't want frequent updates to the global-requirements.txt. | 13:53 |
| efried | Right, so when I say "periodically" above, we're talking once or twice per release. | 13:53 |
| efried | (openstack release, that is) | 13:54 |
| thorst_ | then we lose the flexibility that we have now of getting fixes across multiple components (in a short period of time) for the out of tree driver | 13:54 |
| efried | Yup. | 13:54 |
| adreznec | Yeah, that's kind of a pain | 13:54 |
| efried | It's pretty rare that we need something from pypowervm in multiple out-of-tree drivers, neh? | 13:54 |
| thorst_ | I think that's 100% right for in-tree...in fact I'd like the in-tree to use 1.0.0.4 basically once that goes out, even after pypowervm goes to 1.0.0.5 | 13:54 |
| thorst_ | at least support it. | 13:55 |
| *** tblakes has joined #openstack-powervm | 13:57 | |
| thorst_ | I like the idea of having the out of tree potentially overwrite the req. | 13:59 |
| efried | Can we accept that it won't work for devstack in general? Only for CI? | 14:00 |
| thorst_ | no | 14:00 |
| thorst_ | we need a devstack'd solution | 14:00 |
| thorst_ | I'd accept that we don't have the right answer for now...and maybe the work around is that we have something in the nova-powervm (or c/n) devstack scripts tweak the upper-constraints | 14:00 |
| thorst_ | and that we need to dwell on this a bit. | 14:01 |
| efried | Would need to confirm this, but I believe the devstack scripts run really early, as soon as the project is collected. | 14:01 |
| efried | The problem there is that we don't know for a given devstack run whether nova-powervm is collected before or after the requirements project. | 14:02 |
| efried | It's not deterministic. | 14:02 |
| efried | Which is kind of the point of global requirements - to at least make the levels of everything deterministic regardless of the order they happen to come in. | 14:03 |
| thorst_ | so it only installs for projects that require it. Right now that's nova-powervm, networking-powervm, and ceilometer-powervm (and at some point nova) | 14:03 |
| thorst_ | so it would be called before that | 14:03 |
| thorst_ | but potentially, quickly changed. | 14:03 |
| thorst_ | because nova would be the first thing called. | 14:04 |
| thorst_ | so it's deterministic right now... | 14:04 |
| thorst_ | I think the 'hack' (and it is a hack) is two stages. | 14:04 |
| efried | We can't get away with having nova's devstack scripts muck with the requirements. | 14:04 |
| thorst_ | 1) our devstacks tweak the upper-limits | 14:05 |
| thorst_ | 2) once in nova, we know that nova will go before nova-powervm. So there nova-powervm would have to uninstall what nova put in, then let the other install go through. | 14:05 |
| thorst_ | the other install being what we developed in 1 | 14:05 |
| thorst_ | core nova won't muck with it. That's a no go. | 14:06 |
| thorst_ | nor should it. | 14:06 |
| thorst_ | that'd be...awful. | 14:06 |
| efried | thorst_, I think we can implement both pieces at the same time. | 14:08 |
| efried | Basically *-powervm uninstalls any pypowervm that happens to be there and installs the right version. | 14:08 |
| thorst_ | efried: possibly, but if we separate out and get one before the other...then I'd like to do what's quickest | 14:08 |
| thorst_ | (unless its like a couple hour difference) | 14:09 |
| adreznec | Uninstalling 'any pypowervm that happens to be there' basically assumes it was pip installed | 14:09 |
| adreznec | Unless we're going to get in the game of doing apt/yum removes, etc | 14:10 |
| adreznec | Which sounds gross | 14:10 |
| thorst_ | adreznec: well, we'd just pip uninstall | 14:11 |
| thorst_ | not apt-get uninstall | 14:11 |
| adreznec | I think part of this comes back to our pypowervm versioning being broken | 14:12 |
| adreznec | again | 14:12 |
| efried | yup | 14:12 |
| adreznec | Right, because if we had proper versioning this wouldn't be a real issue | 14:12 |
| adreznec | We could bump upper-constraints to something reasonable | 14:13 |
| adreznec | and not worry about it ripping this version out from under us | 14:13 |
| efried | adreznec, wouldn't we still have the moving-target problem? | 14:14 |
| efried | Assuming we can't update upper-constraints twice a week whenever we put something new in pypowervm | 14:14 |
| thorst_ | and wouldn't it be well know that version 2.0.x doesn't exist, so how can that realistically be an upper-constraint | 14:14 |
| adreznec | Sure, but so does everyone else in openstack... it makes things a bit less flexible, I agree | 14:15 |
| *** mdrabe has joined #openstack-powervm | 14:15 | |
| efried | Right, I have to assume you can't put a version number that doesn't exist into requirements. Or that'd be a snap. | 14:15 |
| thorst_ | unless you can? | 14:15 |
| adreznec | Even if you could, I'd imagine the reqs team would -1 that patchset if they caught it | 14:16 |
| adreznec | Otherwise it kind of defeats the entire purpose of upper-constraints, no? | 14:16 |
| efried | For sure. | 14:16 |
| efried | That's what we're trying to do. | 14:16 |
| adreznec | Look, our upper-constraint is version 9000! | 14:16 |
| efried | Defeat the entire purpose of upper-constraints. | 14:16 |
| adreznec | Right | 14:16 |
| efried | I'm still finding it tough to believe that nobody else runs into this on a regular basis. | 14:17 |
| adreznec | Our goals are very different from their goals here since this is basically a development branch of an out-of-tree driver | 14:17 |
| adreznec | Yeah, I looked at the compute-hyperv driver | 14:18 |
| thorst_ | back to the 'out-of-tree driver should isolate its weirdness in itself' | 14:18 |
| adreznec | But they just bump os-win | 14:18 |
| adreznec | and requirements | 14:18 |
| efried | How frequently? | 14:18 |
| adreznec | https://github.com/openstack/os-win/releases | 14:19 |
| adreznec | Not that frequently | 14:19 |
| efried | So thorst_, give it to me again why the lockstep approach wouldn't work? (Accumulating function in nova-powervm and then, a couple times a release, "moving" it over to pypowervm.) | 14:19 |
| thorst_ | efried: well that was my request earlier. Can we get more frequent releases of pypowervm without driving the whole train along with it. | 14:20 |
| thorst_ | which perhaps we can. | 14:21 |
| esberglu | I know this won't work for all cases. But for now in the CI couldn't we just update the pypowervm requirement everywhere we need to before devstack even starts | 14:21 |
| efried | esberglu, yes, but we need a solution that works for devstack in general. | 14:21 |
| thorst_ | esberglu: no, cause its too high a version, so upper constraints installs over it | 14:21 |
| thorst_ | esberglu: sorry, I misread. | 14:21 |
| esberglu | efried: Yeah I just meant for now. So we can get CI up | 14:21 |
| thorst_ | esberglu: yeah, lets I guess do it for now. | 14:22 |
| esberglu | #action esberglu: Change pypowervm requirements before CI runs | 14:23 |
| thorst_ | alright...lets do that for CI, cool off and figure out how the heck we do this moving forward. | 14:25 |
| esberglu | Sounds good. Anything else before I end the meeting? I know we are way long | 14:25 |
| efried | It looks like, if we wanted to do this in the *-powervm projects' devstack scripts, it would install over the top of 1.0.0.3.9. | 14:26 |
| efried | ...which gets pulled down *with* each *-powervm in a way we have no control over. | 14:26 |
| thorst_ | yep. Lets close up meeting here | 14:27 |
| thorst_ | we can still discuss | 14:27 |
| esberglu | #endmeeting | 14:27 |
| openstack | Meeting ended Thu Nov 17 14:27:28 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:27 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/ci_scrum/2016/ci_scrum.2016-11-17-13.31.html | 14:27 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/ci_scrum/2016/ci_scrum.2016-11-17-13.31.txt | 14:27 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/ci_scrum/2016/ci_scrum.2016-11-17-13.31.log.html | 14:27 |
| *** tblakes has quit IRC | 14:40 | |
| *** wangqwsh has quit IRC | 14:47 | |
| openstackgerrit | Arun Mani proposed openstack/nova-powervm: Object upload using swift occasionally fails with 'TypeError' exception. https://review.openstack.org/398868 | 14:48 |
| *** kotra03 has quit IRC | 14:55 | |
| *** arunman has joined #openstack-powervm | 14:57 | |
| *** apearson has joined #openstack-powervm | 14:59 | |
| *** seroyer has quit IRC | 15:06 | |
| *** tblakes has joined #openstack-powervm | 15:09 | |
| arunman | thorst_: I have a review out for the swift issue. Please take a look when you have time | 15:17 |
| arunman | this is just a hack in nova-powervm, till we fix it the right way from swift client side | 15:17 |
| thorst_ | arunman: why do we do it only on the store? Couldn't this happen on the get as well/ | 15:17 |
| arunman | this is only hit with _upload_object_call on the swift service.py module | 15:20 |
| *** seroyer has joined #openstack-powervm | 15:20 | |
| arunman | it's actually _upload_object_job | 15:20 |
| arunman | here, https://github.com/openstack/python-swiftclient/blob/master/swiftclient/service.py#L1863 | 15:21 |
| *** tjakobs has joined #openstack-powervm | 15:23 | |
| thorst_ | arunman: ahh, cool. | 15:36 |
| thorst_ | that makes the workaround better. | 15:36 |
| thorst_ | arunman: I did a -1 | 15:39 |
| thorst_ | see comments as to why and we can chat it out | 15:39 |
| arunman | thorst_: sure | 15:40 |
| arunman | On changing the retry attempt to 3, this is just to give it one more try in case if there is ClientException and TypeError. The ClientException will most probably go off in the first retry attempt. | 15:43 |
| arunman | On why we NOT fail deploy here... this is only in case if a TypeError is hit. After few attempts I've seen that the periodic task updates swift with the required data. | 15:45 |
| arunman | But yes... if RR is attempted before swift gets updated, RR will fail or the VM on destination host will not have recent data | 15:46 |
| thorst_ | arunman: and we can't let remote rebuild fail. We've seen hyper issues where the volume got deleted in that case | 15:49 |
| thorst_ | so its better to fail the deploy there. | 15:50 |
| thorst_ | does the TypeError not go away with a retry? | 15:50 |
| arunman | most cases yes. It takes a while to go away | 15:51 |
| arunman | I'll look at handling deploy case | 15:52 |
| thorst_ | arunman: can you define 'takes a while to go away' | 16:00 |
| thorst_ | does it block deploys for several minutes? | 16:00 |
| thorst_ | if so, maybe we increase the retry count much higher (and add a small delay) | 16:00 |
| arunman | it's like... we keep doing a call to swift for upload. Once TypeError is hit, you keep seeing this for sometime. But, this goes off on its own after a while and the subsequent uploads will work | 16:06 |
| *** arunman has quit IRC | 16:06 | |
| *** arunman has joined #openstack-powervm | 16:07 | |
| arunman | I'm not able to predict the behavior... even if we increase the retry count to a higher value, can't be sure it will work. | 16:08 |
| thorst_ | but we lower the probability | 16:09 |
| thorst_ | I guess I'm wondering if we need a new session, or a new swift client, or something else | 16:09 |
| thorst_ | or, it just disappears. | 16:09 |
| thorst_ | so I would propose we retry more often...and then if it does fail...well it fails. | 16:09 |
| thorst_ | that way we don't get into data integrity issues. | 16:10 |
| arunman | I'm of the opinion that, it just disappears | 16:11 |
| arunman | yeah.. we were failing it anyway. My fix was just to handle it gracefully and not to fail any operation and we expect the periodic task to update it. | 16:14 |
| arunman | But if we retry more often and then fail it... I'm still OK with that | 16:14 |
| thorst_ | yeah, that's my preference | 16:16 |
| thorst_ | I don't think I'll allow anything through that allows us to deploy without NVRAM (when you set the driver to *require* NVRAM be persisted to Swift) | 16:17 |
| arunman | The same applies for volume attach and detach. How would you suggest to handle those? | 16:18 |
| thorst_ | anything that changes the NVRAM, if we can't persist to NVRAM, we need to block that action | 16:18 |
| thorst_ | that's my view. | 16:18 |
| thorst_ | the contract is that the transaction completed wholly and fully...and the task flow should unwind it when it fails. | 16:19 |
| arunman | Got that | 16:19 |
| *** arunman has quit IRC | 16:25 | |
| *** mdrabe has quit IRC | 16:39 | |
| *** mdrabe has joined #openstack-powervm | 16:44 | |
| *** thorst_ has quit IRC | 17:59 | |
| *** thorst_ has joined #openstack-powervm | 18:00 | |
| svenkat | thorst_ , efried: want to discuss https://review.openstack.org/#/c/396467/4 here. with respect to default value of None for vif_type parameter in ProvisionRequest init. | 18:03 |
| svenkat | vif_type parameter is valid and consumed only in for_event path and not in for_wrapper path. heal_and_optimize is not impacted. | 18:04 |
| svenkat | for_event path checks vif_type in ProvisionRequest and matches it with agent’s vif_type. | 18:05 |
| efried | svenkat, Agree for now we should leave it defaulting to None like you've got it. If we discover that there's a deterministic way to correlate wrapper type to vif type, we may incorporate that later - but certainly not in this change set. thorst_, you agree? | 18:05 |
| openstackgerrit | Eric Berglund proposed openstack/nova-powervm: DNM: CI Check2 https://review.openstack.org/328317 | 18:09 |
| *** jwcroppe has quit IRC | 18:12 | |
| *** thorst_ has quit IRC | 18:26 | |
| *** thorst_ has joined #openstack-powervm | 18:26 | |
| *** thorst_ has quit IRC | 18:58 | |
| *** thorst_ has joined #openstack-powervm | 19:05 | |
| thorst_ | sorry - was AFK. The only deterministic way is via vSwitch and vNIC config | 19:09 |
| thorst_ | if they're attached to a specific vSwitch we kind of know what they are | 19:10 |
| thorst_ | efried: can we do another review of 4533 and 4504? | 19:16 |
| thorst_ | looks like we got jenkins going | 19:16 |
| efried | thorst_, okay. Where do you want to start? | 19:16 |
| thorst_ | 4504 | 19:17 |
| thorst_ | I think we've both touched...so should both probably re-review | 19:17 |
| efried | yuh | 19:17 |
| efried | thorst, okay, this lgtm. Shall we both +2? | 19:18 |
| efried | thorst_ ^^ I +2ed. You do same, then merge. | 19:19 |
| thorst_ | k | 19:19 |
| thorst_ | onto 4533...surprised it didn't have a merge conflict | 19:19 |
| thorst_ | efried: 4533 will have another pass. | 19:20 |
| efried | thorst_, ferwhat? | 19:21 |
| thorst_ | the vOpt changes are unrelated | 19:21 |
| thorst_ | so I want to undo that | 19:21 |
| efried | okay. I was just inspecting those. Seem harmless, but yeah, unrelated. | 19:22 |
| efried | thorst_, hold on a tick. | 19:22 |
| efried | I don't think the vio_file removal thing is relevant, really. | 19:22 |
| efried | You're basically trying to remove it again the same way we tried to remove it originally. | 19:22 |
| thorst_ | it just should always be None | 19:23 |
| efried | Do you have some reason to believe that'll work where it didn't before because the vdisk is gone? | 19:23 |
| thorst_ | because now we're deleting it | 19:23 |
| thorst_ | the intention from kyle earlier was you needed the file to know how to remove it yourself. | 19:23 |
| efried | Yeah. | 19:23 |
| thorst_ | and I don't really agree...we delete it now | 19:23 |
| efried | We always deleted it. | 19:23 |
| efried | Or tried, anyway. | 19:23 |
| efried | We returned it if it didn't get deleted for some reason. | 19:23 |
| efried | Which I contend we should still do. | 19:23 |
| efried | That piece of code is tangential to your change. | 19:24 |
| efried | Unless you assert that the deletion may fail while the vdisk exists, but succeed after we remove it. | 19:24 |
| thorst_ | yep, I think that could happen | 19:25 |
| thorst_ | maybe? | 19:25 |
| thorst_ | at least, I think that's what kriskend saw | 19:26 |
| efried | thorst_ So even then, I want to return the vio_file if _that_ deletion also fails. | 19:27 |
| thorst_ | o bleh | 19:27 |
| thorst_ | I *hate* that variable | 19:28 |
| thorst_ | no one in their right mind will use it | 19:28 |
| efried | Then it's harmless. | 19:28 |
| thorst_ | but I do see your point | 19:28 |
| thorst_ | consistency with a bad past there. | 19:28 |
| efried | shrug | 19:28 |
| thorst_ | sometimes I hate your correctness | 19:28 |
| efried | Sometimes I hate being correct. | 19:29 |
| efried | Usually not, though. | 19:29 |
| thorst_ | http://i.imgur.com/aSJGX.gif | 19:30 |
| *** tblakes has quit IRC | 19:34 | |
| thorst_ | efried: pushed another up | 19:38 |
| efried | thorst_, beautiful, +2 | 19:42 |
| efried | Cleaner, no? | 19:42 |
| *** tblakes_ has joined #openstack-powervm | 19:45 | |
| *** openstackgerrit has quit IRC | 19:48 | |
| *** tblakes_ is now known as tblakes | 19:48 | |
| *** openstackgerrit has joined #openstack-powervm | 19:49 | |
| *** k0da has quit IRC | 19:51 | |
| *** kylek3h has quit IRC | 19:57 | |
| thorst_ | efried: yep | 19:59 |
| thorst_ | and kriskend is happy now | 19:59 |
| *** tjakobs has quit IRC | 20:00 | |
| *** tjakobs_ has joined #openstack-powervm | 20:00 | |
| *** tblakes has quit IRC | 20:33 | |
| *** kriskend has joined #openstack-powervm | 20:48 | |
| kriskend | thorst_ If pypowervm was dead when I did a bunch of VM deletes from openstack, after fixing pypowervm, shouldn't the reaper clean up my instances? | 20:50 |
| thorst_ | assuming they got far enough to create instances...yes, eventually | 20:50 |
| kriskend | how often does it run? | 20:50 |
| thorst_ | I thought on boot | 20:51 |
| kriskend | so I had instances, I killed pypowervm by uninstalling it and not fixing the sym link | 20:51 |
| kriskend | then I deleted the instances | 20:51 |
| kriskend | well restarted nova | 20:51 |
| kriskend | then deleted | 20:51 |
| kriskend | of course nova didn't work | 20:52 |
| kriskend | then I fixed pypowervm and restarted nova | 20:52 |
| thorst_ | ahhh, nova probably convinced itself it deleted them already | 20:52 |
| kriskend | I thought my instances would get cleaned up at that point... but no | 20:52 |
| kriskend | they are gone in openstack but still on my compute | 20:52 |
| thorst_ | yeah | 20:53 |
| thorst_ | so you already reaped | 20:53 |
| thorst_ | poorly | 20:53 |
| kriskend | sigh, so manual cleanup it is | 20:53 |
| thorst_ | yep | 20:54 |
| *** kylek3h has joined #openstack-powervm | 21:02 | |
| *** k0da has joined #openstack-powervm | 21:17 | |
| openstackgerrit | Eric Fried proposed openstack/nova-powervm: devstack INSTALL_PYPOWERVM after nova-powervm https://review.openstack.org/399254 | 21:38 |
| kriskend | thorst_ so trying out your close the pipe patch, I deployed 10 vms | 21:45 |
| kriskend | 4 fails with checksum errors | 21:45 |
| kriskend | and they all rolled back except for the lv | 21:45 |
| kriskend | cuz I don't have that patch | 21:45 |
| kriskend | whoops I lied , 6 failed.. | 21:46 |
| kriskend | glance seems really unhappy on this system | 21:46 |
| openstackgerrit | Eric Fried proposed openstack/networking-powervm: devstack INSTALL_PYPOWERVM after networking-pwervm https://review.openstack.org/399258 | 21:49 |
| openstackgerrit | Eric Fried proposed openstack/ceilometer-powervm: devstack INSTALL_PYPOWERVM after ceilometer-pvm https://review.openstack.org/399260 | 21:50 |
| kriskend | if glance can really fail this much, should we consider doing retries? | 21:56 |
| *** svenkat has quit IRC | 22:00 | |
| *** edmondsw has quit IRC | 22:09 | |
| *** apearson has quit IRC | 22:17 | |
| thorst_ | kriskend: possibly. | 22:17 |
| *** thorst_ has quit IRC | 22:18 | |
| *** thorst_ has joined #openstack-powervm | 22:19 | |
| *** thorst_ has quit IRC | 22:24 | |
| *** thorst_ has joined #openstack-powervm | 23:02 | |
| *** mdrabe has quit IRC | 23:03 | |
| *** seroyer has quit IRC | 23:05 | |
| *** esberglu has quit IRC | 23:18 | |
| *** esberglu has joined #openstack-powervm | 23:19 | |
| *** kriskend has quit IRC | 23:19 | |
| *** esberglu has quit IRC | 23:23 | |
| *** tjakobs_ has quit IRC | 23:28 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
