| *** k0da has quit IRC | 01:01 | |
| *** nullsign has joined #openstack-powervm | 01:59 | |
| *** zerick has quit IRC | 03:14 | |
| *** zerick has joined #openstack-powervm | 03:18 | |
| *** zerick has quit IRC | 03:20 | |
| *** chhagarw has joined #openstack-powervm | 04:49 | |
| *** AlexeyAbashkin has joined #openstack-powervm | 08:04 | |
| *** k0da has joined #openstack-powervm | 08:21 | |
| *** k0da has quit IRC | 10:45 | |
| *** chhagarw has quit IRC | 11:01 | |
| *** chhagarw has joined #openstack-powervm | 12:07 | |
| *** edmondsw has joined #openstack-powervm | 12:17 | |
| *** AlexeyAbashkin has quit IRC | 12:48 | |
| *** AlexeyAbashkin has joined #openstack-powervm | 13:10 | |
| edmondsw | esberglu any idea why this still hasn't merged? https://review.openstack.org/#/c/553168/ | 13:16 |
|---|---|---|
| edmondsw | seems like the gate must have lost it... how do we get it to go again, remove and readd +@? | 13:16 |
| edmondsw | +W | 13:16 |
| efried | yeah | 13:17 |
| edmondsw | k | 13:17 |
| efried | Or I can try tacking on a +W | 13:17 |
| edmondsw | well we both did :) | 13:18 |
| *** openstackgerrit has joined #openstack-powervm | 13:29 | |
| openstackgerrit | Merged openstack/ceilometer-powervm master: Updated from global requirements https://review.openstack.org/553168 | 13:29 |
| efried | booya | 13:30 |
| edmondsw | :) | 13:31 |
| *** tjakobs has joined #openstack-powervm | 13:36 | |
| *** esberglu has quit IRC | 13:39 | |
| *** esberglu has joined #openstack-powervm | 13:42 | |
| edmondsw | #startmeeting PowerVM Driver Meeting | 14:00 |
| openstack | Meeting started Tue Mar 20 14:00:08 2018 UTC and is due to finish in 60 minutes. The chair is edmondsw. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| *** openstack changes topic to " (Meeting topic: PowerVM Driver Meeting)" | 14:00 | |
| openstack | The meeting name has been set to 'powervm_driver_meeting' | 14:00 |
| esberglu | \o | 14:00 |
| efried | @/ | 14:00 |
| edmondsw | agenda: https://etherpad.openstack.org/p/powervm_driver_meeting_agenda | 14:00 |
| edmondsw | #topic In-Tree Driver | 14:00 |
| *** openstack changes topic to "In-Tree Driver (Meeting topic: PowerVM Driver Meeting)" | 14:00 | |
| edmondsw | https://etherpad.openstack.org/p/powervm-in-tree-todos | 14:01 |
| edmondsw | esberglu I just gave you a few easy comments on the vscsi commit | 14:01 |
| edmondsw | then that should be good for efried to look at | 14:01 |
| efried | I'm still ploughing my way through my review backlog, and the in-tree patches are on it. | 14:02 |
| edmondsw | we're stacking things up for the nova cores to look at, with none of them actually looking for a while | 14:02 |
| edmondsw | +1 | 14:02 |
| esberglu | edmondsw: Saw that. Yep once I fix those it's ready for review | 14:02 |
| efried | We ought to have runways kicked off "soon" - like maybe this week | 14:02 |
| edmondsw | efried I would look at that one next of the IT patches | 14:02 |
| esberglu | I live tested attach/detach again which looks good and also got extend_volume working | 14:02 |
| edmondsw | next on my list is snapshot, then disk adapter | 14:02 |
| esberglu | edmondsw: Cool | 14:02 |
| edmondsw | esberglu anything else you want to say for IT? | 14:03 |
| efried | in which case we can queue up the couple we have ready | 14:03 |
| esberglu | I just had a question about microversions | 14:03 |
| edmondsw | efried sorry I didn't follow that | 14:03 |
| edmondsw | ask away | 14:03 |
| esberglu | When I was issuing commands using the cinder CLI it was defaulting to 3.0 | 14:04 |
| esberglu | In cinder/api/openstack/api_version_request.py | 14:04 |
| esberglu | I changed the _MIN_API_VERSION = "3.42" | 14:05 |
| esberglu | The level required for extending attached volumes | 14:05 |
| esberglu | Is that the proper way to go about changing the min? | 14:05 |
| edmondsw | no | 14:05 |
| efried | For the CLI? Yes. It isn't? | 14:05 |
| esberglu | I couldn't find a clear answer in my googling | 14:05 |
| edmondsw | wait... maybe I'm not following what you're trying to do | 14:05 |
| edmondsw | are you working on a cinder change? | 14:06 |
| efried | edmondsw: I got this. | 14:06 |
| esberglu | edmondsw: No I was testing extend_volume in the vSCSI change | 14:06 |
| efried | esberglu: Different CLIs are different. | 14:06 |
| edmondsw | then you just need to set an env var, not alter code | 14:06 |
| * edmondsw letting efried explain | 14:07 | |
| efried | Some default to minimum version. Some default to a specific version. Some negotiate latest version with the server per call. Clearly cinder is the first one. | 14:07 |
| efried | And yes, there's an env var and/or flag you can specify to the CLI to get a different version. | 14:07 |
| efried | AND the CLI has to know how to handle whatever operation you're trying to execute. | 14:07 |
| edmondsw | yep | 14:08 |
| efried | So sometimes it may not be sufficient just to increase the version. | 14:08 |
| edmondsw | I think in this case I checked and it should be, but that was last week so... :) | 14:08 |
| efried | But it sounds like in this case you determined that the CLI *does* support it, and that you need to use 3.42 to get that support switched on. | 14:08 |
| efried | So you're doing the right thing. | 14:08 |
| edmondsw | efried by altering code? why would you say that over using an env var? | 14:09 |
| efried | eh? I never said altering code. | 14:09 |
| edmondsw | esberglu said he changed _MIN_API_VERSION in cinder/api/openstack/api_version_request.py | 14:09 |
| efried | Oh, I didn't follow that esberglu was actually changing code. My bad. | 14:09 |
| edmondsw | or at least that's how I read it | 14:09 |
| esberglu | Yeah I should have used an env. var but the end result is the same | 14:09 |
| efried | esberglu: There'll be an env var and/or CLI option to set the microversion. | 14:09 |
| efried | Do that instead. | 14:09 |
| efried | But if you're just testing locally, meh. | 14:10 |
| edmondsw | sure | 14:10 |
| efried | edmondsw: Runways. In Nova. Basically a mechanism to promote more equitable distribution of review time. See https://etherpad.openstack.org/p/nova-runways-rocky | 14:10 |
| esberglu | Anyways, point is that extend_volume works for attached vSCSI volumes if using the right microversion | 14:10 |
| edmondsw | cool | 14:10 |
| esberglu | That's all I had | 14:11 |
| esberglu | Just need reviews on snapshot and vscsi | 14:11 |
| edmondsw | efried we need to get the spec approved... | 14:12 |
| esberglu | efried: Saw that you asked about fast approve there, hopefully someone will push it through | 14:12 |
| edmondsw | I pinged melwitt the other day and she said just to put it on https://etherpad.openstack.org/p/rocky-nova-priorities-tracking which it was, but no activity yet | 14:12 |
| *** k0da has joined #openstack-powervm | 14:13 | |
| efried | Yeah. Re spec approval, when we talked about it the other day, I mentioned that we shouldn't be too strict about the "whole approved blueprint" aspect of this. | 14:13 |
| efried | So like, I fully intend to put the spec down as part of the runway if it's not approved by the time we get this going. | 14:13 |
| efried | esberglu: btw, are you planning to go on vacation any time soon? | 14:14 |
| efried | or be otherwise unavailable to address review comments quickly? | 14:14 |
| esberglu | efried: Nothing more than a day or 2 until July | 14:14 |
| efried | cause that would be the main thing stopping us from queuing up a runway. | 14:14 |
| edmondsw | we won't want to wait for "The code for the blueprint must be 100% ready for review" | 14:15 |
| edmondsw | I read the "If a blueprint is too large" following sentence as saying it still has to be 100% ready, but it doesn't have to all be reviewed in the same runway... but I hope I'm wrong there | 14:16 |
| efried | jgwentworth was supposed to leave the discussion in a separate etherpad. | 14:17 |
| efried | oh. | 14:17 |
| efried | It's at the bottom. | 14:17 |
| edmondsw | cool, I'll read over that later | 14:18 |
| edmondsw | and add comments as appropriate | 14:18 |
| edmondsw | tx for the link | 14:18 |
| edmondsw | anything else for IT? | 14:18 |
| esberglu | nope | 14:18 |
| edmondsw | #topic Out-of-Tree Driver | 14:18 |
| *** openstack changes topic to "Out-of-Tree Driver (Meeting topic: PowerVM Driver Meeting)" | 14:18 | |
| edmondsw | https://etherpad.openstack.org/p/powervm-oot-todos | 14:18 |
| edmondsw | I did some more reviewing on the refactor... need to finish that and post the comments | 14:19 |
| efried | I did some reviewing there. Did I post comments? | 14:19 |
| efried | I can't remember whether I saved 'em. | 14:19 |
| efried | yeah | 14:20 |
| edmondsw | yep, I see 'em | 14:20 |
| efried | I'm still -0.5 until we get some live testing. Which is gonna suck. | 14:20 |
| edmondsw | yeah, I think we're all agreed this will need some good testing | 14:20 |
| edmondsw | just wanted to get comments addressed first, so we're not doing that multiple times | 14:21 |
| efried | yuh | 14:21 |
| edmondsw | and I uncovered something interesting while digging through this... we don't have iSCSI live migration implemented | 14:21 |
| edmondsw | so I added that to the TODO list. PowerVC wants it | 14:21 |
| edmondsw | the other big thing OOT is https://review.openstack.org/552172 | 14:22 |
| edmondsw | hope to see a pypowervm release today or tomorrow so we can propose a requirements bump and unblock that | 14:23 |
| edmondsw | anything else to discuss OOT? | 14:23 |
| efried | actually | 14:23 |
| efried | have we released pypowervm yet? Or tagged it or whatever? | 14:23 |
| edmondsw | no | 14:24 |
| esberglu | Nope | 14:24 |
| efried | Cause I had one comment in the refactor about something that we should be doing in pypowervm instead of nova-powervm. | 14:24 |
| edmondsw | I asked about that yesterday, and you and hsien both said sure, but it hasn't been done | 14:24 |
| efried | So if we could get that into the release, that would help. | 14:24 |
| edmondsw | absolutely | 14:24 |
| edmondsw | let's drive that today | 14:24 |
| esberglu | efried: edmondsw: IIRC that was a vscsi comment that will apply IT as well | 14:25 |
| edmondsw | https://review.openstack.org/#/c/530816/6/nova_powervm/virt/powervm/volume/fcpvvscsi.py@48 | 14:25 |
| edmondsw | ah, yeah, that would apply to the IT commit as well | 14:25 |
| efried | cool | 14:26 |
| edmondsw | anything else? | 14:26 |
| efried | Well, first we need someone to confirm that always caching it is The Right Thing. | 14:26 |
| efried | I don't know from WWPNs, so that someone ain't me. | 14:26 |
| edmondsw | certainly... you want to discuss that here or after the mtg? | 14:27 |
| edmondsw | I'd rather do after so I can stop and think about it | 14:27 |
| efried | ight | 14:27 |
| edmondsw | #topic Device Passthrough | 14:27 |
| *** openstack changes topic to "Device Passthrough (Meeting topic: PowerVM Driver Meeting)" | 14:27 | |
| edmondsw | efried and I went through use cases yesterday and took some notes | 14:28 |
| edmondsw | I need to work that up and get a mtg on the calendar with the NovaLink guys | 14:28 |
| edmondsw | efried you have the floor | 14:29 |
| efried | Nothing to add | 14:29 |
| edmondsw | k | 14:29 |
| edmondsw | #topic PowerVM CI | 14:29 |
| *** openstack changes topic to "PowerVM CI (Meeting topic: PowerVM Driver Meeting)" | 14:29 | |
| edmondsw | https://etherpad.openstack.org/p/powervm_ci_todos | 14:30 |
| edmondsw | esberglu ? | 14:30 |
| esberglu | Since the start of the weekend CI has not been looking great, a bunch of new failures and long runs | 14:30 |
| esberglu | I've been taking inventory of failures here https://etherpad.openstack.org/p/powervm_tempest_failures | 14:31 |
| esberglu | That's what I'll be working on today | 14:31 |
| edmondsw | esberglu sure... focus on that and hold off on the vscsi IT commit respin while we workout the question of whether to move that into pypowervm | 14:32 |
| esberglu | edmondsw: Yep I'm gonna just sit IT until we start getting some movement so I don't have to rebase the world | 14:32 |
| esberglu | Other than that the CI management upgrade is ongoing, facing some roadblocks upgrading nodepool | 14:32 |
| esberglu | We're currently nodepool 0.3.0 | 14:32 |
| esberglu | Starting in 0.4.0 they don't allow the flow of taking an image, doing some stuff, taking a snapshot of it and spawning from the snapshot | 14:33 |
| esberglu | And instead you have to use diskimage-builder | 14:33 |
| esberglu | Which from what I've read so far only supports 14.04 | 14:33 |
| esberglu | And I really don't want to go back to 14.04 from 16.04 | 14:33 |
| edmondsw | I hope that's not right... someone you can catch on IRC to talk about that? | 14:34 |
| esberglu | But there may be a way around that, I need to do some more recon | 14:34 |
| edmondsw | not sure who... maybe tonyb? | 14:34 |
| esberglu | edmondsw: I really haven't looked to much into, just saw a blurb about it. I'm sure there are people using new nodepool with 16.04 | 14:35 |
| esberglu | So there's got to be a solution | 14:35 |
| edmondsw | good | 14:35 |
| esberglu | The other thing I need to do is update the CI firmware | 14:36 |
| edmondsw | getting the CI stable again is obviously the priority, but you might want to shoot off a couple feelers so that you have suggestions ready to try when you can get back to this | 14:36 |
| esberglu | So I will need to find a good time to take the CI down for a day or so | 14:36 |
| edmondsw | that related to the undercloud moving to queens, or just normal need to apply security updates and such? | 14:37 |
| esberglu | Security updates | 14:37 |
| edmondsw | ok good | 14:37 |
| edmondsw | maybe do that on a Friday? | 14:37 |
| esberglu | edmondsw: Yeah that was my plan, probably not until next week though | 14:38 |
| esberglu | That's it for me | 14:38 |
| edmondsw | #topic Open Discussion | 14:38 |
| *** openstack changes topic to "Open Discussion (Meeting topic: PowerVM Driver Meeting)" | 14:38 | |
| esberglu | I'll try to time it so we don't have much in the pipeline getting blocked | 14:38 |
| edmondsw | +1 | 14:38 |
| edmondsw | I had one thing to bring up here | 14:38 |
| edmondsw | we talked a little about the logo the other day: http://eavesdrop.openstack.org/irclogs/%23openstack-powervm/%23openstack-powervm.2018-03-14.log.html#t2018-03-14T19:03:19 | 14:39 |
| edmondsw | but I don't think I saw esberglu chime in | 14:39 |
| edmondsw | any thoughts? | 14:39 |
| edmondsw | or if anyone else is lurking and wants to throw something out there... | 14:39 |
| esberglu | edmondsw: +1 on gorilla, thought that was a great idea | 14:39 |
| edmondsw | cool | 14:39 |
| edmondsw | then barring something unexpected, I'll ask for gorilla | 14:40 |
| edmondsw | that's it from me | 14:40 |
| esberglu | nothing else from me | 14:40 |
| efried | wwpns? | 14:40 |
| edmondsw | sure we can talk about that... let me go back to your comment | 14:40 |
| efried | def get_physical_wwpns(adapter): | 14:41 |
| efried | """Returns the active WWPNs of the FC ports across all VIOSes on system. | 14:41 |
| efried | :param adapter: pypowervm.adapter.Adapter for REST API communication. | 14:41 |
| efried | """ | 14:41 |
| efried | vios_feed = vios.VIOS.get(adapter, xag=[c.XAG.VIO_STOR]) | 14:41 |
| efried | wwpn_list = [] | 14:41 |
| efried | for vwrap in vios_feed: | 14:41 |
| efried | wwpn_list.extend(vwrap.get_active_pfc_wwpns()) | 14:41 |
| efried | return wwpn_list | 14:41 |
| efried | def get_active_pfc_wwpns(self): | 14:41 |
| efried | """Returns a set of Physical FC Adapter WWPNs of 'active' ports.""" | 14:41 |
| efried | # The logic to check for active ports is poor. Right now it only | 14:41 |
| efried | # checks if the port has NPIV connections available. If there is a | 14:41 |
| efried | # FC, non-NPIV card...then this logic fails. | 14:41 |
| efried | # | 14:41 |
| efried | # This will suffice until the backing API adds more granular logic. | 14:41 |
| efried | return [pfc.wwpn for pfc in self.pfc_ports if pfc.npiv_total_ports > 0] | 14:41 |
| efried | So the question is: does the list of pfc_ports, or their number of npiv_total_ports, never* change? (*without, like rebooting) | 14:42 |
| * efried has GOT to figure out how to turn off emojis in this IRC client) | 14:42 | |
| edmondsw | physical WWPNs shouldn't change unless you hotplug an adapter... but I think you can do that? | 14:42 |
| efried | no idea | 14:43 |
| edmondsw | so this may have been an oversight when the code was first written and we've just never hit an issue because folks don't hotplug FC adapters on a regular basis | 14:43 |
| efried | If you can, then the nova-powervm code is wrong. | 14:43 |
| efried | yeah, that. | 14:43 |
| edmondsw | we should have this conversation with seroyer | 14:43 |
| efried | ight. | 14:43 |
| edmondsw | let's move that to slack so we can pull him in | 14:44 |
| edmondsw | anything else before we close here? | 14:44 |
| efried | nothing from me. | 14:44 |
| edmondsw | #endmeeting | 14:44 |
| *** openstack changes topic to "This channel is for PowerVM-related development and discussion. For general OpenStack support, please use #openstack." | 14:44 | |
| openstack | Meeting ended Tue Mar 20 14:44:50 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:44 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2018/powervm_driver_meeting.2018-03-20-14.00.html | 14:44 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2018/powervm_driver_meeting.2018-03-20-14.00.txt | 14:44 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2018/powervm_driver_meeting.2018-03-20-14.00.log.html | 14:44 |
| efried | test: :) | 15:06 |
| efried | BOOOya! | 15:07 |
| efried | Man, that took some hacking. | 15:07 |
| edmondsw | what irc client? | 15:35 |
| efried | Thunderbird | 16:48 |
| efried | Same as I use for email (that isn't us.ibm.com) | 16:48 |
| *** AlexeyAbashkin has quit IRC | 16:53 | |
| *** AlexeyAbashkin has joined #openstack-powervm | 18:14 | |
| *** AlexeyAbashkin has quit IRC | 18:14 | |
| *** AlexeyAbashkin has joined #openstack-powervm | 18:15 | |
| *** AlexeyAbashkin has quit IRC | 18:44 | |
| *** chhagarw has quit IRC | 19:02 | |
| *** AlexeyAbashkin has joined #openstack-powervm | 19:03 | |
| esberglu | edmondsw: efried: I'm seeing an error in CI when trying to create lpars where there aren't enough available processor units to complete the request | 19:30 |
| esberglu | At first glance I thought we might just need to allocated more to our AIO tempest vms | 19:30 |
| esberglu | But this is only ever happening for in-tree CI runs, so I'm worried this is a side effect of something else | 19:31 |
| efried | This sounds vaguely familiar... | 19:31 |
| edmondsw | doesn't sound familiar to me, so I'll defer to efried... | 19:32 |
| esberglu | http://184.172.12.213/62/549862/11/check/nova-in-tree-pvm/6a4646d/logs/n-cpu.txt.gz | 19:32 |
| esberglu | Example run, I haven't looked into it much, thought I would float it out and see if you guys had any ideas first | 19:33 |
| esberglu | Looks like there are some differences in the VMBuilder code between IT and OOT | 19:39 |
| esberglu | IT doesn't have a few of the processor conf options that OOT does | 19:42 |
| esberglu | Could this be related to the missing proc_units_factor opt? | 19:43 |
| esberglu | I'm not 100% clear on what that does from the description | 19:44 |
| esberglu | "Factor used to calculate the processor units per vcpu. Valid values are: 0.05 - 1.0" | 19:44 |
| efried | Basically a multiplier for CPUs. Could definitely do it. | 19:45 |
| efried | For every one physical CPU we have, multiply by 1/that value to get the number of virtual CPUs we can supply. | 19:46 |
| esberglu | OOT is setting that to 0.1, where IT is using the pypowervm default of 0.5 | 19:49 |
| efried | 0.5 or 0.05? | 19:49 |
| esberglu | 0.5 | 19:49 |
| efried | So we'll have 5x less CPU in tree. | 19:49 |
| esberglu | Yep | 19:50 |
| efried | But has something changed recently that is pushing us over the line? | 19:50 |
| efried | How frequently are we running up against this, and since when? | 19:51 |
| *** AlexeyAbashkin has quit IRC | 19:51 | |
| efried | Does it go away if we bump down our concurrency level? | 19:51 |
| esberglu | efried: Last time we were seeing this error the pok network was on the fritz and CI was in a bad state in general | 19:53 |
| esberglu | The frequency has gone way up in the last week or so | 19:53 |
| efried | And we use the pypowervm default in tree because we decided not to expose that conf option? | 19:54 |
| efried | Or we expose it and just aren't setting it? | 19:54 |
| esberglu | efried: We didn't expose the conf option | 19:54 |
| esberglu | Haven't tried changing the concurrency level | 19:54 |
| efried | Concurrency level is not preferred solution because it'll slow down our runs. | 19:55 |
| esberglu | We're gonna have to carry an experimental patch forward in IT CI runs to even confirm this is the issue | 19:55 |
| efried | Let's just try to meld in a patch that hardcodes that to 0.1 | 19:55 |
| efried | yeah, that. | 19:55 |
| efried | You got that covered or want me to propose it? | 19:55 |
| esberglu | efried: I've can do it | 19:56 |
| efried | coo | 19:56 |
| tonyb | esberglu, edmondsw: diskimage-builder supports 16.04 ATM, and we're working on 18.04 support for "from image" builds, the ubuntu-minimal element should work for 18.04 today | 20:24 |
| esberglu | tonyb: Thanks for the info! | 20:25 |
| tonyb | esberglu: np | 20:25 |
| esberglu | That should make the CI mgmt upgrade much simpler | 20:25 |
| tonyb | \o/ | 20:25 |
| tonyb | esberglu: If you get stuck on diskimage-builder $stuff ping me | 20:26 |
| esberglu | tonyb: Will do tx | 20:27 |
| edmondsw | tonyb tx much! | 20:27 |
| *** k0da has quit IRC | 20:28 | |
| esberglu | edmondsw: efried: 6407 for experimental proc_units_factor patching | 20:30 |
| *** esberglu has quit IRC | 20:31 | |
| *** openstackgerrit has quit IRC | 20:33 | |
| *** esberglu has joined #openstack-powervm | 20:34 | |
| *** k0da has joined #openstack-powervm | 20:41 | |
| *** edmondsw has quit IRC | 20:51 | |
| *** AlexeyAbashkin has joined #openstack-powervm | 21:19 | |
| *** AlexeyAbashkin has quit IRC | 21:23 | |
| *** esberglu has quit IRC | 21:35 | |
| *** esberglu_ has joined #openstack-powervm | 21:40 | |
| *** esberglu_ has quit IRC | 21:45 | |
| *** tjakobs has quit IRC | 21:49 | |
| *** AlexeyAbashkin has joined #openstack-powervm | 22:19 | |
| *** AlexeyAbashkin has quit IRC | 22:23 | |
| *** k0da has quit IRC | 22:30 | |
| *** AlexeyAbashkin has joined #openstack-powervm | 23:19 | |
| *** AlexeyAbashkin has quit IRC | 23:23 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!