| *** edmondsw has joined #openstack-powervm | 00:10 | |
| *** edmondsw has quit IRC | 00:14 | |
| *** thorst has joined #openstack-powervm | 00:17 | |
| *** thorst has quit IRC | 00:22 | |
| *** chas has joined #openstack-powervm | 01:02 | |
| *** thorst has joined #openstack-powervm | 01:04 | |
| *** thorst has quit IRC | 01:04 | |
| *** chas has quit IRC | 01:06 | |
| *** edmondsw has joined #openstack-powervm | 01:58 | |
| *** edmondsw has quit IRC | 02:02 | |
| *** thorst has joined #openstack-powervm | 02:05 | |
| *** thorst has quit IRC | 02:24 | |
| *** edmondsw has joined #openstack-powervm | 02:38 | |
| *** edmondsw has quit IRC | 02:50 | |
| *** edmondsw has joined #openstack-powervm | 02:58 | |
| *** chas has joined #openstack-powervm | 03:03 | |
| *** chas has quit IRC | 03:08 | |
| *** thorst has joined #openstack-powervm | 03:21 | |
| *** thorst has quit IRC | 03:25 | |
| *** edmondsw has quit IRC | 03:34 | |
| *** jwcroppe_ has quit IRC | 03:57 | |
| *** jwcroppe has joined #openstack-powervm | 03:58 | |
| *** jwcroppe has quit IRC | 04:03 | |
| *** thorst has joined #openstack-powervm | 04:22 | |
| *** thorst has quit IRC | 04:26 | |
| *** edmondsw has joined #openstack-powervm | 04:45 | |
| *** edmondsw has quit IRC | 04:50 | |
| *** chas has joined #openstack-powervm | 05:04 | |
| *** chas has quit IRC | 05:09 | |
| *** arunman has joined #openstack-powervm | 05:24 | |
| *** thorst has joined #openstack-powervm | 06:24 | |
| *** thorst has quit IRC | 06:28 | |
| *** k0da has joined #openstack-powervm | 06:29 | |
| *** edmondsw has joined #openstack-powervm | 06:34 | |
| *** edmondsw has quit IRC | 06:38 | |
| *** chhavi has joined #openstack-powervm | 07:00 | |
| *** k0da has quit IRC | 07:06 | |
| *** chas has joined #openstack-powervm | 07:47 | |
| *** chas has quit IRC | 07:52 | |
| *** chas has joined #openstack-powervm | 07:53 | |
| *** k0da has joined #openstack-powervm | 08:14 | |
| *** edmondsw has joined #openstack-powervm | 08:22 | |
| *** thorst has joined #openstack-powervm | 08:25 | |
| *** edmondsw has quit IRC | 08:26 | |
| *** thorst has quit IRC | 08:45 | |
| *** zerick has quit IRC | 09:27 | |
| *** adreznec has quit IRC | 09:27 | |
| *** AndyWojo has quit IRC | 09:27 | |
| *** zerick has joined #openstack-powervm | 09:49 | |
| *** edmondsw has joined #openstack-powervm | 10:10 | |
| *** adreznec has joined #openstack-powervm | 10:13 | |
| *** AndyWojo has joined #openstack-powervm | 10:13 | |
| *** edmondsw has quit IRC | 10:14 | |
| *** thorst has joined #openstack-powervm | 10:42 | |
| *** thorst has quit IRC | 10:46 | |
| *** smatzek_ has joined #openstack-powervm | 11:17 | |
| *** thorst has joined #openstack-powervm | 11:35 | |
| *** arunman has quit IRC | 11:40 | |
| chhavi | efried,thorst,edmondsw: please review http://morpheus.pok.stglabs.ibm.com/#/c/5267/ | 12:03 |
|---|---|---|
| thorst | chhavi: responded. Minor tweak. | 12:18 |
| *** edmondsw has joined #openstack-powervm | 12:19 | |
| *** chas has quit IRC | 12:44 | |
| *** chas has joined #openstack-powervm | 12:45 | |
| *** chhavi_ has joined #openstack-powervm | 12:49 | |
| *** chas has quit IRC | 12:49 | |
| *** chhavi has quit IRC | 12:51 | |
| *** esberglu has joined #openstack-powervm | 12:55 | |
| *** esberglu_ has joined #openstack-powervm | 12:58 | |
| esberglu_ | #startmeeting powervm_driver_meeting | 13:01 |
| openstack | Meeting started Tue May 30 13:01:36 2017 UTC and is due to finish in 60 minutes. The chair is esberglu_. Information about MeetBot at http://wiki.debian.org/MeetBot. | 13:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 13:01 |
| openstack | The meeting name has been set to 'powervm_driver_meeting' | 13:01 |
| *** esberglu has quit IRC | 13:02 | |
| edmondsw | o/ | 13:02 |
| esberglu_ | #topic In Tree Driver | 13:03 |
| esberglu_ | #link https://etherpad.openstack.org/p/powervm-in-tree-todos | 13:03 |
| esberglu_ | List of todo's is above | 13:04 |
| esberglu_ | With the merge of the full flavor spawn/delete and console last week I think we are moving forward again here | 13:05 |
| esberglu_ | efried: You around? | 13:06 |
| esberglu_ | #topic Out Of Tree Driver | 13:08 |
| *** mdrabe has joined #openstack-powervm | 13:08 | |
| esberglu_ | I'm still slowly working backports from IT to OOT as able, there are a few other working items found in the link above that are for OOT as well | 13:08 |
| thorst | let me ping efried on slack...see if he's around | 13:08 |
| *** dwayne has quit IRC | 13:10 | |
| esberglu_ | Anyone other topics OOT that need discussion? | 13:10 |
| esberglu_ | *any | 13:10 |
| thorst | efried: is joining | 13:10 |
| thorst | I think with OOT, we'll be getting a new iSCSI patch from chhavi_ soon | 13:11 |
| thorst | which will depend on a new pypowervm release | 13:11 |
| thorst | so we'll have to merge that in slowly | 13:11 |
| efried | o/ Sorry I'm late, guys. | 13:13 |
| esberglu_ | efried: np | 13:13 |
| esberglu_ | Talking OOT now, we can loop back to IT after if you have stuff | 13:13 |
| efried | The pypowervm release is branched, but as of last week, I didn't see it in pypi yet. | 13:13 |
| efried | Does it have what you need in it, thorst ? | 13:14 |
| thorst | efried: nope | 13:14 |
| thorst | chhavi_ posted the review | 13:14 |
| thorst | I +2'd it for pypowervm, but not in | 13:14 |
| thorst | I'd like you to review | 13:14 |
| efried | ack | 13:14 |
| efried | 5267? | 13:15 |
| thorst | yessir | 13:15 |
| esberglu_ | Cool. That it OOT? | 13:16 |
| thorst | yeah, the iSCSI is for OOT | 13:16 |
| thorst | but that's ongoing work | 13:16 |
| esberglu_ | *that it for OOT | 13:16 |
| thorst | lol, whoops :-) | 13:18 |
| thorst | I have nothing else | 13:18 |
| thorst | (I read your 'that it' wrong obviously) | 13:18 |
| efried | Was "merges" all we talked about for in-tree? | 13:19 |
| efried | Were we planning to come back to that? | 13:19 |
| esberglu_ | efried: Yeah I was gonna wait for you | 13:19 |
| efried | The wait is over! | 13:19 |
| chhavi_ | efried can u qickly review 5267 | 13:19 |
| efried | Not quickly, no. | 13:19 |
| chhavi_ | its long cid | 13:19 |
| efried | I'll do it first thing, though. | 13:19 |
| efried | We're going to need to cut 1.1.6 with that, I guess. | 13:20 |
| efried | What's the desired time frame on that? | 13:20 |
| chhavi_ | its iscsi which is for OOT | 13:21 |
| edmondsw | I thought they cut 1.1.6 last week? So we might need 1.1.7 for the iSCSI change | 13:21 |
| chhavi_ | thorst: can u clarify i am not aware of this plan | 13:22 |
| thorst | chhavi_: we have release branches for pypowervm that are independent of NL | 13:22 |
| thorst | we can't accept the nova_powervm change that will consume your iSCSI change until it is in a RELEASE of pypowervm | 13:22 |
| thorst | fortunately, we can turn around pypowervm releases kind of quickly | 13:23 |
| edmondsw | oh, nm my comment above... I think 1.1.5 is still the latest | 13:23 |
| edmondsw | I think it was NL that went to 1.0.0.6 | 13:23 |
| thorst | so once your pypowervm change is approved and merged, we then will need to make a release of pypowervm before we can consider merging the nova_powervm one | 13:23 |
| efried | Glad we changed that release numbering scheme to be less confusing. | 13:24 |
| edmondsw | chhavi_ that's what I was trying to say when I -1'd your nova_powervm change | 13:24 |
| edmondsw | (what thorst was saying) | 13:24 |
| chhavi_ | ok, yeah that change is dependent on pypowervm | 13:24 |
| edmondsw | and we normally bump the requirements version in a separate patch, right? Rather than in the nova_powervm change we want to make that will depend on it | 13:25 |
| edmondsw | efried ^ | 13:25 |
| edmondsw | since we want to do that in global_requirements, not just in nova_powervm, right? | 13:26 |
| efried | edmondsw Yes, we do that in g-r and let the bot propose the nova-powervm bump. | 13:26 |
| edmondsw | that was going to be next question, if we had the bot setup... cool | 13:26 |
| edmondsw | done with OOT then? | 13:27 |
| edmondsw | back to IT? | 13:27 |
| esberglu_ | #topic In Tree Driver | 13:28 |
| efried | let's do it | 13:28 |
| efried | So I'm working on get_inventory(). | 13:28 |
| esberglu_ | I wonder if that will merge the two topics | 13:28 |
| efried | https://review.openstack.org/#/c/468560/ | 13:28 |
| efried | There's a lot of open questions here. As it stands, the change set can probably be considered "good enough", algorithmically. But we will eventually want to work through it and answer all the pending stuff. | 13:29 |
| efried | Y'all want to discuss it in any more detail here? | 13:30 |
| thorst | so give that a thorough review? | 13:30 |
| thorst | I haven't looked at it yet. | 13:30 |
| efried | Yeah, I'm afraid a "thorough review" could entail quite a bit of background reading... | 13:30 |
| efried | Swhy I'm wondering whether it would be useful to do an overview discussion here. | 13:31 |
| edmondsw | I haven't looked either yet, but I will try to do that today once I get through a couple PowerVC defects that have to get fixed today | 13:31 |
| thorst | This is just the thing where... | 13:31 |
| thorst | don't duplicate the storage space | 13:31 |
| efried | Not yet. | 13:31 |
| thorst | two nodes using same storage should have a single amount of storage | 13:31 |
| thorst | and you'll be pulling that to OOT as well I assume ;-) | 13:31 |
| efried | So fare, this is just the thing where get_available_resource is being superseded by get_inventory, which (internally) uses the placement API to register how much CPU, memory, and disk the host has. | 13:32 |
| efried | so far* | 13:32 |
| efried | Yes, I'll pull it OOT once we have it settled. Not going to even try that until it's approved and mergeable. | 13:32 |
| efried | SSP-ness is one of the open questions, which I figured would probably be a future change set. | 13:33 |
| efried | The other thing is figuring out "reserved" properly for all the resources. | 13:33 |
| efried | And "allocation_ratio". | 13:33 |
| efried | The other drivers aren't doing any of that yet, so I feel comfortable doing it incrementally. | 13:34 |
| edmondsw | sounds good to me | 13:34 |
| efried | Point is, when you're reviewing this, at least take a look at the other drivers' get_inventory() methods (I think only libvirt and ironic are currently implemented) | 13:35 |
| efried | yeah | 13:35 |
| efried | unless proposed-but-not-merged. | 13:35 |
| thorst | neat | 13:35 |
| thorst | cool to see we're relatively bleeding edge here | 13:36 |
| thorst | :-) | 13:36 |
| efried | I'll add blueprint references to the etherpad, in case you want to go more in-depth. | 13:36 |
| efried | Other in-tree news, just the SSP change set is left on the nova team's perception of "complete" for the pike blueprint for the powervm driver. | 13:38 |
| *** chas has joined #openstack-powervm | 13:38 | |
| efried | It has sdague +2, so I'll pester mriedem at some point this week to do his pass. | 13:38 |
| efried | The get_inventory change depends on it; otherwise I would have to cut one version of that without disk. | 13:39 |
| *** chas_ has joined #openstack-powervm | 13:39 | |
| efried | ...and then another version on top of the SSP change to add disk. | 13:39 |
| efried | So I decided to do it this way - anyone have objections to that? | 13:39 |
| thorst | efried: so...that means we can deploy without networks | 13:40 |
| esberglu_ | efried: Makes sense to me | 13:40 |
| thorst | come pike? | 13:40 |
| edmondsw | do you think the get_inventory change is going to make pike this way? | 13:40 |
| edmondsw | and do we care if it slips to queens? | 13:40 |
| edmondsw | sounds like it would | 13:40 |
| efried | edmondsw Yes, I think it'll make pike, but no, it doesn't break the world if it slips to queens. | 13:40 |
| efried | They're not getting rid of get_available_resource yet. | 13:40 |
| thorst | yeah, queens is really where we expect to have something viable in-tree | 13:41 |
| efried | The consuming code tries get_inventory, and if it's NotImplemented, falls back to get_available_resource. | 13:41 |
| efried | I think they're going to deprecate in queens, remove in rocky. | 13:41 |
| efried | assuming they can make the whole thing work in the first place, which I'm not totally convinced of yet. | 13:42 |
| efried | One of the major changes is that get_inventory, unlike get_available_resource, is not responsible for telling you how much resource is used - just how much total the host has. | 13:42 |
| *** jwcroppe has joined #openstack-powervm | 13:43 | |
| efried | The theory being that nova is keeping track of how much is used based on the claims attached to the instances it has deployed. | 13:43 |
| *** chas has quit IRC | 13:43 | |
| efried | Which makes the 'reserved' business much more important - and more difficult - for us to account for. | 13:43 |
| efried | Perhaps we should set up a separate slot (or stick around after this meeting) to go into that stuff in more depth. | 13:45 |
| *** arunman has joined #openstack-powervm | 13:45 | |
| efried | <crickets> | 13:46 |
| thorst | <reading....> | 13:46 |
| edmondsw | efried I would suggest a separate slot | 13:46 |
| thorst | yeah. | 13:46 |
| thorst | but I would like to read the review first | 13:47 |
| thorst | bang my head on it | 13:47 |
| edmondsw | yeah | 13:47 |
| efried | Okay. I tried to leave appropriate TODOs for the stuff I've mentioned above, but that's not a lot of context. | 13:47 |
| efried | So just grab me on IRC when y'all are ready. | 13:47 |
| efried | Otherwise in-tree: There's a couple of major pieces of work we should be looking at in the near future. | 13:48 |
| efried | First priority, getting powervm into the support matrix. | 13:48 |
| efried | Anyone volunteering to take that on? | 13:48 |
| efried | This should be fairly straightforward. | 13:49 |
| esberglu_ | efried: I can do that. Is that just a docs change? | 13:49 |
| efried | esberglu_ Yes, more or less. (Code-as-doc) | 13:50 |
| *** chas_ has quit IRC | 13:50 | |
| efried | esberglu_ The links in the etherpad should get you pointed in the right direction. | 13:50 |
| efried | I'll put your name on it. Thank you. | 13:50 |
| esberglu_ | efried: Perfect | 13:50 |
| *** chas has joined #openstack-powervm | 13:51 | |
| efried | Related, but more involved, and possibly not destined to make pike, is the item called "Document flavor extra_specs" in the etherpad. | 13:51 |
| efried | Just documenting our extra_specs wouldn't be a huge deal... | 13:52 |
| efried | ...but mriedem is looking at a bigger picture here, I think. | 13:52 |
| efried | In terms of getting at least the framework for broader per-driver documentation into the tree. | 13:52 |
| *** chas_ has joined #openstack-powervm | 13:52 | |
| efried | If nobody feels like starting to look into that right away, we can put it off for a bit. | 13:53 |
| efried | The last major item, more fun, actual code, is "Deactivate compute service if pvm-rest is dead." | 13:54 |
| *** mdrabe has quit IRC | 13:54 | |
| *** mdrabe_ has joined #openstack-powervm | 13:54 | |
| *** chas has quit IRC | 13:55 | |
| efried | Anyone feel like digging into that in the near future? | 13:55 |
| *** mdrabe_ has quit IRC | 13:55 | |
| edmondsw | efried what does deactivate entail? | 13:55 |
| efried | Mark the compute host as down so nothing gets scheduled to it. | 13:56 |
| *** mdrabe_ has joined #openstack-powervm | 13:56 | |
| edmondsw | and then keep listening and bring it back up when pvm-rest comes back? | 13:56 |
| efried | Yeah. The links in the etherpad point to code we can crib from vcenter to do the deactivate/reactivate part. | 13:56 |
| edmondsw | cool | 13:56 |
| edmondsw | are we trying to do that for pike? | 13:56 |
| efried | So it would just be figuring out how (easy) and where (??) to listen for pvm-rest being dead. | 13:57 |
| efried | edmondsw That would be nice, but I don't think it's critical. | 13:57 |
| edmondsw | this or the extra_specs doc higher priority? | 13:57 |
| efried | nova took some pressure off of that recently by putting in a change that deactivates the compute service if it fails some number of deploys in a row (configurable, 10 by default, I think). | 13:58 |
| efried | edmondsw That's kind of an apples/oranges question. | 13:58 |
| efried | If you have the space for it and it sounds interesting to you, grab it. | 13:58 |
| edmondsw | efried I can't start on it today, but you can put my name there and I will start on it later this week | 13:59 |
| efried | edmondsw Cool man. Links in the etherpad should get things rolling; let me know if you want more pointers/background. | 13:59 |
| edmondsw | tx | 13:59 |
| edmondsw | time check... 1 minute | 14:00 |
| efried | Only other in-tree stuff is the small stuff, low-hanging fruit. I think esberglu_, you have your eye on picking that stuff off? | 14:00 |
| esberglu_ | efried: Yep, slowly picking it off when I'm not fighting CI | 14:01 |
| esberglu_ | #topic CI | 14:01 |
| efried | Cool. Just put your name on stuff and add review links as you go so nobody ends up duplicating. | 14:01 |
| esberglu_ | So master CI is back up and passing runs are going through | 14:01 |
| esberglu_ | But not a good percentage | 14:02 |
| esberglu_ | The stable branches were broken for everything except nova | 14:02 |
| esberglu_ | The patch application logic would try to apply the patches regardless of branch | 14:02 |
| esberglu_ | Which would merge fail for the stable branches | 14:02 |
| esberglu_ | Since all the patches we applied are now merged I updated the jenkins jobs | 14:02 |
| esberglu_ | So they should be working now, I have a newton and ocata CI check running now | 14:03 |
| esberglu_ | Long-term I can just add a branch check there | 14:03 |
| esberglu_ | But I'm going to be reworking that in other ways as well | 14:03 |
| thorst | esberglu_: so nova_powervm is still borked? | 14:03 |
| thorst | for stable/ocata? | 14:03 |
| thorst | I know that the PVC team is...very eager...for that to be working | 14:03 |
| esberglu_ | thorst: Nope it should be good now. Let me confirm once the CI checks finish | 14:03 |
| esberglu_ | But I posted a recheck on larese's patch | 14:04 |
| thorst | good good | 14:04 |
| thorst | I think he had a heart attack on Friday when I said it was down | 14:04 |
| edmondsw | tx for the quick fix esberglu_ | 14:05 |
| esberglu_ | edmondsw: np | 14:05 |
| *** chas has joined #openstack-powervm | 14:05 | |
| edmondsw | esberglu_ anything else on the CI? | 14:05 |
| thorst | +2 | 14:05 |
| esberglu_ | If anyone has extra time this week and feels like looking into the failing CI logs let me know | 14:05 |
| esberglu_ | That's all I had CI | 14:06 |
| efried | I'll be looking at my changes, anyway. | 14:06 |
| esberglu_ | #topic Driver Testing | 14:06 |
| efried | I don't see Jay or Nilesh. | 14:06 |
| esberglu_ | Any new status? | 14:06 |
| edmondsw | chhavi_ do you know anything there? | 14:06 |
| esberglu_ | Let's move on since we are running over time | 14:08 |
| esberglu_ | #topic Open Discussion | 14:08 |
| esberglu_ | Any final thoughts before I call it? | 14:08 |
| *** chas_ has quit IRC | 14:09 | |
| thorst | everyone's doing real good. | 14:09 |
| thorst | :-p | 14:09 |
| edmondsw | +1 | 14:09 |
| efried | :) | 14:09 |
| esberglu_ | :) | 14:10 |
| esberglu_ | Thanks for joining | 14:10 |
| esberglu_ | #endmeeting | 14:10 |
| openstack | Meeting ended Tue May 30 14:10:48 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:10 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-05-30-13.01.html | 14:10 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-05-30-13.01.txt | 14:10 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-05-30-13.01.log.html | 14:10 |
| *** mdrabe_ is now known as mdrabe | 14:16 | |
| *** esberglu_ is now known as esberglu | 14:17 | |
| efried | thorst chhavi_ Help me think through this... | 14:26 |
| efried | nova_powervm/virt/powervm/volume/iscsi.py exists in ocata. | 14:26 |
| efried | and calls discover_iscsi. | 14:26 |
| efried | Which means that we can't change the signature of discover_iscsi. | 14:26 |
| efried | Because ocata doesn't have a cap on the pypowervm version it can take. | 14:27 |
| thorst | efried: yeah...it should've been marked experimental | 14:27 |
| thorst | it was not...was it... | 14:27 |
| efried | thorst Where would it be thus marked? | 14:27 |
| thorst | docs | 14:27 |
| efried | thorst Can you put your finger on that? Else I'll ask tjakobs | 14:28 |
| thorst | I don't think we have it marked | 14:29 |
| thorst | so you're right | 14:29 |
| thorst | which sux | 14:29 |
| efried | Sigh. That does complicate matters. | 14:29 |
| thorst | but...I would assert no one is really using it yet. | 14:29 |
| thorst | so if we backport...I could be made happy | 14:30 |
| thorst | backport then nova_powervm firx | 14:30 |
| thorst | *fix | 14:30 |
| thorst | but yeah...it does complicate. | 14:30 |
| efried | Wait, backport what? | 14:30 |
| efried | I was gonna say we could propose a requirements.txt change in ocata to cap the pypowervm version. | 14:30 |
| efried | That would be the easiest thing. | 14:30 |
| efried | Do we have the bot set up for ocata? Would that conflict? | 14:31 |
| efried | thorst Backporting anything in pypowervm is a nonstarter, I would think. But we could backport https://review.openstack.org/#/c/467599/ - is that what you were suggesting? | 14:32 |
| *** dwayne has joined #openstack-powervm | 14:32 | |
| thorst | yeah | 14:34 |
| efried | thorst That would only work if we also bumped the ocata pypowervm requirement. | 14:37 |
| thorst | so I think the fair question back to chhavi is...do we make this a new API and deprecate the old signature | 14:38 |
| thorst | down in pypowervm | 14:38 |
| thorst | that may be easiest | 14:38 |
| *** mdrabe has quit IRC | 14:49 | |
| efried | thorst We're quitting translating in nova-powervm, right? | 14:49 |
| thorst | no | 14:49 |
| thorst | we just won't translate new log messages | 14:49 |
| efried | k | 14:50 |
| thorst | ravi is still pushing up translations | 14:50 |
| *** k0da has quit IRC | 14:50 | |
| *** mdrabe has joined #openstack-powervm | 14:56 | |
| efried | thorst chhavi_ Reviewed http://morpheus.pok.stglabs.ibm.com/#/c/5267/ | 15:06 |
| efried | Whoops. | 15:06 |
| efried | Thought I was reviewing public :( | 15:06 |
| thorst | heh | 15:06 |
| thorst | I just talked to chhavi about that | 15:06 |
| thorst | even efried makes mistakes | 15:06 |
| efried | Anywho... I forgot to mention the thing about changing signatures. | 15:07 |
| efried | okay, added that comment. | 15:07 |
| efried | thorst chhavi_ Sorry, I kinda tore this one apart. | 15:08 |
| *** jpasqualetto has joined #openstack-powervm | 15:20 | |
| esberglu | There are some CI runs going through that are very long-running (3 hrs) | 15:48 |
| esberglu | Looks like there are a bunch of 30 min timeouts waiting to close vterms | 15:48 |
| *** kriskend has joined #openstack-powervm | 15:48 | |
| esberglu | Any idea what may be causing this? | 15:48 |
| efried | esberglu Hmm, we recently changed the exception handling in the vterm stuff. Is this IT or OOT? | 15:50 |
| esberglu | I haven't seen it IT yet, only OOT, but I haven't looked for it IT yet either | 15:50 |
| esberglu | http://184.172.12.213/47/469047/1/check/nova-out-of-tree-pvm/64721a5/ | 15:50 |
| *** tjakobs has joined #openstack-powervm | 15:56 | |
| efried | esberglu Okay, couple of problems here. | 15:59 |
| efried | The CloseVterm Job is taking too long. It only actually timed out twice; but the other few times I'm seeing a log trace for it, it's taking minutes. | 16:00 |
| efried | The other problem is this: 'This task is only allowed when the partition is powered off' | 16:01 |
| esberglu | efried: Yeah I saw that, thought it was a side effect from the closevterm timeout | 16:02 |
| *** chas has quit IRC | 16:02 | |
| esberglu | But we should probably handle that somehow if that's the case | 16:03 |
| *** chas has joined #openstack-powervm | 16:03 | |
| efried | esberglu Was this on ocata or master? | 16:04 |
| efried | master | 16:04 |
| esberglu | Master | 16:04 |
| efried | esberglu I think we need thorst, our vterm expert, to look at why close_vterm is taking so long. | 16:07 |
| *** chas has quit IRC | 16:07 | |
| efried | We should also consider trying to delete anyway even if close_vterm raises. | 16:07 |
| efried | thorst esberglu Here: https://github.com/openstack/nova-powervm/blob/master/nova_powervm/virt/powervm/vm.py#L624 | 16:08 |
| efried | esberglu thorst One complication here (which is good/bad) is that closing vterm takes a different code path when non-local. | 16:11 |
| efried | Good because it may mean this is not a problem in a real env. | 16:11 |
| efried | Bad because... well, it's causing CI problems. | 16:11 |
| efried | aha | 16:13 |
| efried | I think ./ci-ansible/roles/ci-management/templates/scripts/local2remote.py needs to set local_api to False | 16:14 |
| efried | never mind. | 16:14 |
| efried | It's based on use_file_auth, which local2remote is indeed setting. | 16:15 |
| esberglu | efried: Yeah I saw a log message somewhere about it being a non-local vterm | 16:15 |
| *** arunman has quit IRC | 16:38 | |
| thorst | weird... | 16:39 |
| *** chas has joined #openstack-powervm | 16:43 | |
| *** chas has quit IRC | 16:48 | |
| efried | esberglu http://184.172.12.213/15/468515/2/check/nova-powervm-out-of-tree-pvm/74fb3ca/powervm_os_ci.html wtf is a security group, and why is it breaking? | 17:08 |
| thorst | lol | 17:09 |
| thorst | a security group is really a network thing | 17:09 |
| thorst | SEA's don't support it. OVS does | 17:09 |
| thorst | it defines what a port can (or can't) talk to | 17:10 |
| efried | thorst This is OOT/ocata, which is using SEA. Does this mean we should be disabling tests that involve security groups? | 17:11 |
| efried | ...when using SEA? | 17:11 |
| thorst | yep | 17:11 |
| efried | esberglu ^^ | 17:11 |
| efried | Are these new? | 17:11 |
| thorst | though | 17:12 |
| thorst | usually its just transparent | 17:12 |
| thorst | a lot of tests use security groups | 17:12 |
| thorst | but it doesn't really matter that we don't support them | 17:12 |
| thorst | all the traffic flows through | 17:12 |
| thorst | and its fine... | 17:12 |
| thorst | unless they're explicitly testing security groups | 17:12 |
| efried | thorst I can't answer that. All I can tell you is these tests are failing on security group garbage. | 17:19 |
| thorst | efried: where is the failure actually at? nova logs? | 17:22 |
| thorst | they all look like timeout exceptions to me | 17:23 |
| efried | thorst Dunno, didn't get that far. Just saw it in the tempest log. | 17:23 |
| efried | thorst No, starting at the second one from the top, the next half dozen or so are security groups. | 17:23 |
| efried | Then yes, after that, a ton of timeouts and a couple power failures. | 17:23 |
| efried | But one thing at a time ;) | 17:23 |
| thorst | point me to which security group test...they all look power related to me | 17:26 |
| thorst | this is a new one to me: [instance: e841c4c6-e8fe-45f1-b6d2-07b223cb32ed] Instance disappeared during build. | 17:26 |
| thorst | we had a 'pwr_vm' task there take 1226 seconds to complete | 17:27 |
| thorst | this smells like something is up with the rest server. | 17:27 |
| efried | thorst http://184.172.12.213/15/468515/2/check/nova-powervm-out-of-tree-pvm/74fb3ca/powervm_os_ci.html look at the second failure. | 17:28 |
| thorst | huh | 17:29 |
| thorst | probably due to that previous test taking forever...and poorly written tests | 17:29 |
| thorst | that previous test is the thing that took 1226 seconds to power on, then disappeared. | 17:29 |
| efried | thorst Just looked at ocata reqs again - still says 'pypowervm' with no version spec. We probably want to go back and fix that anyway, huh? | 17:45 |
| efried | So when chhavi_'s change is ready, we'll need to decide: either we backport the nova-powervm side back to ocata and put pypowervm>=1.1.6; or don't backport it and put pypowervm<1.1.6. | 17:47 |
| thorst | efried: agree... | 17:52 |
| thorst | just want to make sure whatever version we put back there... | 17:52 |
| thorst | is available in a proper pvm-novalink release | 17:52 |
| thorst | don't want to jump the gun on that as i know we have lots of teams building on ocata...don't want them to hit a req that is only in pypi...want them to also have it available in the apt-get repo | 17:53 |
| efried | thorst Then the safest thing to do is mark it as <1.1.6 right now. | 17:54 |
| thorst | well, it has some upper boundary | 17:54 |
| thorst | *lower | 17:54 |
| efried | thorst It *should*. It doesn't today. | 17:55 |
| thorst | and I would argue it doesn't *really* have an upper boundary | 17:55 |
| thorst | right. | 17:55 |
| thorst | so what was the lower boundary when we shipped? | 17:55 |
| efried | No idea. | 17:55 |
| thorst | I think NL 1.0.0.4...whatever pypowervm went out with that... | 17:55 |
| thorst | back to terrible version numbers. | 17:55 |
| efried | thorst Sigh, safest thing is to leave existing discover_iscsi alone and make a new method with the new sig. | 17:56 |
| efried | thorst Or leave the existing signature. | 17:57 |
| efried | thorst chhavi_ I think the only thing we would need to do is add the lun as a kwarg. | 17:58 |
| efried | Get rid of ISCSIInputData entirely. | 17:58 |
| chhavi_ | the problem which put me to use that is having 8 input parameters limit | 17:58 |
| chhavi_ | adding lunid to existing is failing the sonar rules | 17:59 |
| efried | chhavi_ Mm. Does it complain about that even if some are kwargs? | 17:59 |
| chhavi_ | earlier kwargs was not used | 17:59 |
| chhavi_ | so it was all positional arguments | 17:59 |
| efried | chhavi_ I can't remember if it enforces that rule for kwargs or not. We would have to add the lun as a kwarg to maintain backward compatibility anyway (and figure out how to code it so it would work if that wasn't specified - is that even possible?) | 18:00 |
| chhavi_ | if we use kwargs as well it will change the function definition | 18:00 |
| efried | chhavi_ Yes, but it would be backward compatible. | 18:00 |
| chhavi_ | kwargs shud be allowed | 18:00 |
| chhavi_ | we can have kwargs as default None | 18:00 |
| efried | chhavi_ Right, but can the Job handle it if the LUN isn't specified? | 18:00 |
| thorst | or if the user doesn't specify can we have a basic default value. | 18:01 |
| esberglu | thorst: efried: Just getting caught up above the security group topic. Typically when we see that it is a side effect of another test failure | 18:03 |
| esberglu | As seems to be the case above | 18:04 |
| chhavi_ | have to confirm from hsien if he kept a default value for that parameter | 18:04 |
| efried | esberglu Okay. | 18:04 |
| chhavi_ | let me check | 18:04 |
| esberglu | I was not aware that security groups were a OVS specific thing | 18:04 |
| esberglu | So I may be able to disable them in the tempest.conf or just add the security group tests to the skip list | 18:04 |
| esberglu | Yep there's a tempest.conf setting for it | 18:06 |
| thorst | cool | 18:06 |
| efried | esberglu Okay, that'll help some, I guess. | 18:06 |
| efried | So now we just need to figure out these weird timeouts, and vterm. | 18:07 |
| chhavi_ | We can have a default lun id passed as 0, when kwargs is not passed | 18:07 |
| chhavi_ | so are we agreed to make the change and use kwargs instead of ISCSIInputData ? | 18:08 |
| efried | chhavi_ If that's doable, I think that's going to be the cleanest path forward. | 18:09 |
| thorst | I'm good with it | 18:10 |
| thorst | is it going to be nameless kwargs though? | 18:10 |
| thorst | I kinda hate that... | 18:10 |
| efried | We should still go figure out the lower limit for pypowervm in ocata, but that would then be an independent effort. | 18:10 |
| thorst | are they named parameters? | 18:10 |
| efried | thorst "nameless kwargs"? | 18:10 |
| thorst | **kwargs | 18:10 |
| thorst | or | 18:10 |
| thorst | lunid=0 | 18:11 |
| efried | oh, no, named. | 18:11 |
| thorst | good deal | 18:11 |
| efried | Before: def discover_iscsi(adapter, host_ip, user, password, iqn, vios_uuid, transport_type=None): | 18:11 |
| efried | After: def discover_iscsi(adapter, host_ip, user, password, iqn, vios_uuid, transport_type=None, lunid=0): | 18:11 |
| efried | (chhavi_, be sure to put the new one last, to account for misbehaving consumers who forget to name kwargs) | 18:12 |
| chhavi_ | this does not work, it complains for sonar rule | 18:12 |
| efried | thorst Put an exception for this file into our local sonar rules? Or globally up the limit? Or go back to the signature changing debacle? | 18:13 |
| thorst | ugh... | 18:14 |
| thorst | local sonar rules | 18:14 |
| thorst | that'd be my vote | 18:14 |
| thorst | its a good rule | 18:14 |
| thorst | but after debate like this, I think its fine for an exception | 18:14 |
| chhavi_ | because of this rule i need to make so many change | 18:16 |
| chhavi_ | changes :) | 18:16 |
| efried | thorst chhavi_ To be clear, the smallest granularity we can get away with is the whole file, not the method. | 18:17 |
| chhavi_ | for backward compatibility i think its good to have an exception for this file | 18:17 |
| efried | I'm still okay with that - just making sure y'all are aware. | 18:17 |
| efried | So chhavi_ - the change will be in sonar-project.properties. Let me know if you need help with it. | 18:17 |
| chhavi_ | i am ok | 18:18 |
| chhavi_ | where that file lies | 18:18 |
| efried | Base directory. | 18:18 |
| efried | of pypowervm rep. | 18:18 |
| efried | repo | 18:18 |
| thorst | efried: yep... | 18:18 |
| thorst | I'm good with that | 18:18 |
| chhavi_ | k | 18:20 |
| efried | thorst esberglu edmondsw Regarding get_inventory from this morning, I just had a talk with jaypipes, and it sounds like we don't want to be implementing this until more of the infra is baked. So I'm going to leave that change set hanging out there, but quit working on it for the time being. | 18:20 |
| edmondsw | efried understood | 18:21 |
| chhavi_ | efried: i need your help to figure out what rule to add in sonar properties file to ignore _iscsi.py for method parameters count | 18:22 |
| efried | chhavi_ Okay, PM me the link to the sonar failure. | 18:22 |
| chhavi_ | those builds are deleted it seems | 18:23 |
| chhavi_ | Problem accessing /job/pypowervm-flow-patchsetCreated/2660/. Reason: | 18:23 |
| chhavi_ | 18:23 | |
| chhavi_ | have we done this for any of the file, i can refer | 18:24 |
| efried | chhavi_ Not for this specific rule, it looks like. You'll need the rule ID to make the change. | 18:24 |
| efried | So I guess, go ahead and submit a new patch set with the signature change, let sonar fail, and then we'll put up another patch with the rule. | 18:25 |
| chhavi_ | sure | 18:25 |
| chhavi_ | we need to wait for tomorrow | 18:25 |
| chhavi_ | not quick changes to do now, will do tomorrow | 18:25 |
| efried | Okay. | 18:25 |
| chhavi_ | just confirm with hsien, in neo-rest, lunid is optional job parameter, so we need not set as default as 0 and should be None | 18:27 |
| chhavi_ | if the user send it we will add the jobparameter else not | 18:27 |
| *** chhavi_ has quit IRC | 18:35 | |
| *** chas has joined #openstack-powervm | 19:34 | |
| *** k0da has joined #openstack-powervm | 19:51 | |
| *** tjakobs has quit IRC | 20:03 | |
| *** tjakobs has joined #openstack-powervm | 20:03 | |
| *** jpasqualetto_ has joined #openstack-powervm | 20:05 | |
| *** jpasqualetto_ has quit IRC | 20:05 | |
| *** openstackgerrit has joined #openstack-powervm | 20:09 | |
| openstackgerrit | Eric Berglund proposed openstack/nova-powervm master: DNM: ci check https://review.openstack.org/328315 | 20:09 |
| *** smatzek_ has quit IRC | 20:15 | |
| *** kriskend has quit IRC | 20:55 | |
| openstackgerrit | Eric Berglund proposed openstack/nova-powervm master: DNM: CI Check2 https://review.openstack.org/328317 | 21:02 |
| *** thorst has quit IRC | 21:03 | |
| *** jpasqualetto has quit IRC | 21:08 | |
| *** thorst has joined #openstack-powervm | 21:28 | |
| *** esberglu has quit IRC | 21:30 | |
| *** esberglu has joined #openstack-powervm | 21:30 | |
| *** thorst has quit IRC | 21:32 | |
| *** esberglu has quit IRC | 21:35 | |
| *** esberglu has joined #openstack-powervm | 21:43 | |
| *** thorst has joined #openstack-powervm | 21:47 | |
| *** esberglu has quit IRC | 21:47 | |
| *** jwcroppe has quit IRC | 21:54 | |
| *** jwcroppe has joined #openstack-powervm | 21:55 | |
| *** dwayne has quit IRC | 21:56 | |
| *** thorst has quit IRC | 21:58 | |
| *** mdrabe has quit IRC | 21:59 | |
| *** chas has quit IRC | 22:08 | |
| *** edmondsw has quit IRC | 22:08 | |
| *** edmondsw has joined #openstack-powervm | 22:09 | |
| *** edmondsw has quit IRC | 22:14 | |
| *** thorst has joined #openstack-powervm | 22:29 | |
| *** thorst has quit IRC | 22:36 | |
| *** tjakobs has quit IRC | 22:44 | |
| *** jwcroppe has quit IRC | 22:47 | |
| *** dwayne has joined #openstack-powervm | 23:04 | |
| *** thorst has joined #openstack-powervm | 23:08 | |
| *** thorst has quit IRC | 23:12 | |
| *** thorst has joined #openstack-powervm | 23:43 | |
| *** thorst has quit IRC | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!