*** svenkat has joined #openstack-powervm | 00:30 | |
*** jwcroppe has joined #openstack-powervm | 00:32 | |
*** thorst has joined #openstack-powervm | 00:40 | |
*** thorst has quit IRC | 00:44 | |
*** thorst has joined #openstack-powervm | 01:07 | |
*** thorst has quit IRC | 01:07 | |
*** jwcroppe has quit IRC | 01:08 | |
*** svenkat has quit IRC | 01:15 | |
*** thorst has joined #openstack-powervm | 01:41 | |
*** thorst has quit IRC | 02:06 | |
*** mdrabe has quit IRC | 02:15 | |
*** thorst has joined #openstack-powervm | 02:26 | |
*** thorst has quit IRC | 02:26 | |
*** jwcroppe has joined #openstack-powervm | 02:29 | |
*** thorst has joined #openstack-powervm | 02:42 | |
*** thorst has quit IRC | 02:42 | |
*** thorst has joined #openstack-powervm | 02:43 | |
*** thorst has quit IRC | 02:47 | |
*** thorst has joined #openstack-powervm | 03:14 | |
*** chhavi has joined #openstack-powervm | 03:30 | |
*** thorst has quit IRC | 03:32 | |
*** edmondsw has joined #openstack-powervm | 04:41 | |
*** edmondsw has quit IRC | 04:46 | |
*** efried has quit IRC | 04:52 | |
*** efried has joined #openstack-powervm | 05:04 | |
*** thorst has joined #openstack-powervm | 05:29 | |
*** thorst has quit IRC | 05:34 | |
*** jwcroppe has quit IRC | 05:42 | |
*** jwcroppe has joined #openstack-powervm | 06:10 | |
*** thorst has joined #openstack-powervm | 06:30 | |
*** thorst has quit IRC | 06:35 | |
*** jwcroppe has quit IRC | 06:47 | |
*** jwcroppe has joined #openstack-powervm | 07:29 | |
*** thorst has joined #openstack-powervm | 07:31 | |
*** thorst has quit IRC | 07:35 | |
*** edmondsw has joined #openstack-powervm | 08:17 | |
*** edmondsw has quit IRC | 08:22 | |
*** thorst has joined #openstack-powervm | 08:32 | |
*** thorst has quit IRC | 08:51 | |
*** k0da has joined #openstack-powervm | 08:58 | |
*** jwcroppe has quit IRC | 09:33 | |
*** thorst has joined #openstack-powervm | 09:48 | |
*** thorst has quit IRC | 09:52 | |
*** svenkat has joined #openstack-powervm | 11:44 | |
*** thorst has joined #openstack-powervm | 11:54 | |
*** edmondsw has joined #openstack-powervm | 11:55 | |
*** jwcroppe has joined #openstack-powervm | 11:59 | |
*** kriskend has joined #openstack-powervm | 12:17 | |
*** mdrabe has joined #openstack-powervm | 13:02 | |
esberglu | #startmeeting powervm_driver_meeting | 13:02 |
---|---|---|
openstack | Meeting started Tue Jun 6 13:02:57 2017 UTC and is due to finish in 60 minutes. The chair is esberglu. Information about MeetBot at http://wiki.debian.org/MeetBot. | 13:02 |
openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 13:03 |
openstack | The meeting name has been set to 'powervm_driver_meeting' | 13:03 |
thorst | o/ | 13:03 |
esberglu | #topic In Tree Driver | 13:03 |
esberglu | #link https://etherpad.openstack.org/p/powervm-in-tree-todos | 13:03 |
efried | o/ | 13:04 |
edmondsw | o/ | 13:04 |
efried | "Fixing" the get_info business led alll the way down the rabbit hole. | 13:04 |
efried | https://review.openstack.org/471146 | 13:04 |
efried | In the end, mriedem said we should just remove all the unused fields from InstanceInfo, everywhere. | 13:05 |
efried | This will impact the OOT driver if/when it merges. | 13:06 |
edmondsw | k | 13:06 |
edmondsw | that's for instances... were we also talking about host stats the other day? Are there unused fields to remove there as well? | 13:07 |
thorst | efried: you should probably run that by mdrabe. | 13:07 |
efried | We weren't talking about host stats. | 13:07 |
edmondsw | k | 13:07 |
thorst | I suspect pvc would be impacted (and I bet other OS products) | 13:08 |
efried | thorst Yeah, I was thinking it will probably be a good idea to blast the dev ML on this one. | 13:08 |
thorst | yep. But just ping mdrabe on the side too. I'm not sure how much they view the ML | 13:08 |
thorst | I know I can't (don't) keep up | 13:08 |
edmondsw | +1 | 13:09 |
efried | esberglu Depending how mriedem's comment plays out, this might impact your support matrix change. | 13:09 |
efried | https://review.openstack.org/#/c/471146/2/doc/source/support-matrix.ini@249 | 13:09 |
efried | Not sure if he's gonna ask to remove that whole section. | 13:09 |
esberglu | efried: ack | 13:09 |
efried | I think that's it for me in tree. | 13:10 |
edmondsw | I wanted to ask an IT question | 13:10 |
efried | floor is yours | 13:10 |
edmondsw | so when we were looking at the support matrix it clicked for me that our SSP support that merged IT is only ephemeral | 13:10 |
edmondsw | when we've talked about 2H17 priorities we've talked about network, config_drive, and vSCSI | 13:11 |
edmondsw | is vSCSI there ephemeral or data or both? | 13:11 |
edmondsw | and is vSCSI the top priority for data disk attach/detach, not SSP? | 13:12 |
efried | I don't remember a vSCSI discussion. iSCSI maybe? | 13:12 |
thorst | cinder | 13:12 |
edmondsw | thorst had said vSCSI | 13:12 |
mdrabe | Do we want vSCSI IT? | 13:12 |
thorst | thorst said cinder (via vSCSI) | 13:12 |
mdrabe | (o/ btw) | 13:12 |
thorst | vSCSI is simply a way to connect storage to a VM | 13:12 |
edmondsw | mdrabe read up, there was something for you above | 13:12 |
efried | Gotcha. So the VSCSIVolumeAdapter. | 13:13 |
thorst | when we talk about it in terms of Cinder, we typically mean FC volumes to a VM | 13:13 |
mdrabe | ack | 13:13 |
thorst | in fact in PVC, we simplified vSCSI to just mean that | 13:13 |
thorst | but vSCSI is used for SSP, iSCSI, FC PV, etc... | 13:13 |
thorst | so I probably used the wrong language there | 13:13 |
thorst | I meant cinder support via vSCSI | 13:13 |
efried | Really, for FC? I thought we had a fibre channel mapping that was different from a VSCSI mapping. | 13:14 |
thorst | FC also has this fancy NPIV support | 13:14 |
efried | Anyway, separate discussion. | 13:14 |
thorst | which is like a SR-IOV like thing for FC...though, yeah, separate discussion | 13:14 |
efried | Point is, we're looking to support the VSCSIVolumeAdapter in tree. | 13:14 |
thorst | +1 | 13:14 |
edmondsw | thorst, in terms of the support matrix, what should we be trying to flip to partial/complete among the storage.block items? | 13:15 |
edmondsw | https://github.com/openstack/nova/blob/master/doc/source/support-matrix.ini#L945 | 13:15 |
edmondsw | https://github.com/openstack/nova/blob/master/doc/source/support-matrix.ini#L972 | 13:15 |
edmondsw | https://github.com/openstack/nova/blob/master/doc/source/support-matrix.ini#L993 | 13:15 |
edmondsw | etc. | 13:15 |
thorst | 945 - partial, 972 - complete (though we can add NPIV later), 993 - missing (for now? If we can tuck in awesome) | 13:16 |
edmondsw | I think you're saying L972 via cinder vSCSI | 13:16 |
edmondsw | k | 13:16 |
thorst | reality is that today, everyone is FC. So that's the hole we should fill first for IT. | 13:16 |
edmondsw | what about cinder via SSP? | 13:16 |
thorst | no cinder driver for SSP | 13:17 |
edmondsw | oh, really | 13:17 |
mdrabe | everyone is FC? you mean Power folks? | 13:17 |
thorst | we talked about making one...but it never came to fruition | 13:17 |
efried | https://review.openstack.org/#/c/372254/ | 13:17 |
thorst | PowerVM - everyone is FC | 13:17 |
efried | Still open ;-) | 13:17 |
thorst | rest of world...not so much | 13:17 |
efried | Last action in January | 13:17 |
thorst | efried: yeah... | 13:17 |
thorst | we were hoping that would then allow us to make a cinder driver | 13:17 |
thorst | I think people got pulled in other directions | 13:17 |
thorst | like iSCSI...and my other crazy volume connectors | 13:18 |
edmondsw | thorst so when PowerVC uses SSP for data volumes... how is it doing that without a cinder driver? | 13:18 |
thorst | edmondsw: they have a cinder driver, but it isn't upstreamed yet | 13:18 |
edmondsw | ah | 13:18 |
esberglu | Anything else IT? | 13:19 |
mdrabe | I've a question | 13:19 |
efried | Real quick, back to the get_info discussion, this also came out of it: https://review.openstack.org/#/c/471106/ | 13:19 |
efried | trivial | 13:19 |
mdrabe | Where does os-brick come in to play with volume connectors? | 13:19 |
thorst | mdrabe: good q...shyama was looking into that. Its a way to replace (I think?) the connection_info object (bdm) | 13:20 |
thorst | not super sure | 13:20 |
edmondsw | I've been working on deactivating the compute service when we can't get a pvm session or there are no VIOS ready, but not ready to put up for review quite yet | 13:20 |
mdrabe | Ok yea can discuss later | 13:21 |
esberglu | Alright moving on | 13:22 |
esberglu | #topic Out Of Tree Driver | 13:22 |
efried | Perf improvement change (https://review.openstack.org/469982) - I owe another patch set. | 13:22 |
efried | But the testing came back good on that, so once those fixups are in, I think we're good to go there. | 13:23 |
efried | Then I plan to look into the "don't need a whole instance for the NVRAM manager" thing. | 13:23 |
efried | which could also yield perf improvements... maybe. | 13:23 |
efried | Gotta do that quick before arnoldja moves on to bluer pastures. | 13:24 |
thorst | efried: I don't disagree with what you did | 13:24 |
thorst | I just feel dirty about it | 13:24 |
efried | heh | 13:24 |
thorst | 'lets just wait 15 seconds for everything' | 13:24 |
thorst | 'because this API vomits up events' | 13:24 |
efried | Well, anything PartitionState | 13:24 |
thorst | so I don't disagree...I just think its bleh | 13:24 |
efried | It's always that way with perf improvements. | 13:25 |
thorst | yep yep yep | 13:25 |
efried | Most of the time they make the code uglier. | 13:25 |
thorst | just letting my voice be heard. :-p | 13:25 |
esberglu | This week is the pike 2 milestone (thursday), so I will be tagging the repos accordingly. | 13:25 |
*** smatzek has joined #openstack-powervm | 13:25 | |
mdrabe | efried is there a LP bug for that? | 13:26 |
efried | for the perf thing? | 13:26 |
mdrabe | yea... I guess it's not technically a bug | 13:26 |
efried | https://launchpad.net/bugs/1694784 | 13:26 |
openstack | Launchpad bug 1694784 in nova-powervm "Reduce overhead for redundant PartitionState events" [Undecided,New] | 13:26 |
mdrabe | Ah neat thx | 13:27 |
efried | This might have been said last week, but the get_inventory thing is on hold pending further baking of the infrastructure. | 13:29 |
thorst | yep | 13:29 |
efried | https://review.openstack.org/468560 | 13:29 |
efried | They've hit a snag with the design of shared resource providers. | 13:29 |
efried | It's going around the ML at the moment. Not sure how that's gonna shake out. An elegant solution is not yet forthcoming. | 13:29 |
efried | Subject line, in case you want to follow along at home: [openstack-dev] [nova][scheduler][placement] Allocating Complex Resources | 13:30 |
efried | I guess that's an IT/OOT thing. | 13:30 |
efried | Oh, I wanted to bring up t9n | 13:31 |
efried | I saw an email a couple days ago that may affect our stance on how aggressive we become about removing translations from various places. | 13:31 |
thorst | t9n? | 13:32 |
thorst | what does that stand for? | 13:32 |
efried | translation | 13:32 |
edmondsw | what was the email? | 13:32 |
efried | Right now the policy we're following in nova-powervm is just not translating any new log messages, and removing from anything we happen to touch while doing mods. | 13:32 |
thorst | so the 9 stands for len('ranslatio') | 13:33 |
efried | networking-powervm is subject to a hacking rule that disallows *any* translation. | 13:33 |
efried | (thorst, yeah, like i18n, etc.) | 13:33 |
efried | and k8s ;-) | 13:33 |
thorst | (got it - finally understand i18n too) | 13:33 |
efried | ...and I'm not sure whether we've even talked about a policy for pypowervm. | 13:33 |
* thorst feels dumb | 13:33 | |
thorst | efried: well, pypowervm is consumed by more ways than OpenStack...we've got two or three other direct users. | 13:34 |
thorst | I think any change there would need to be run by them | 13:34 |
thorst | we'd probably want to ask clbush from a CLI perspective too. | 13:35 |
edmondsw | I assumed efried was going to talk about nova-powervm, not pypowervm | 13:35 |
edmondsw | oh, missed a linee | 13:35 |
* edmondsw feels dumb, joining thorst | 13:35 | |
efried | So okay, agree that discussion outside this group is needed for pypowervm. | 13:36 |
efried | What about nova-powervm? | 13:36 |
thorst | I dunno, I'm dragging my feet on that | 13:36 |
thorst | and I'll admit, its really because I know pvc likes those messages translated. | 13:36 |
efried | It's probably not worth going all out and removing everything. | 13:36 |
edmondsw | thorst that's not true... PowerVC doesn't want log message translated | 13:37 |
efried | thorst That's the email I was referring to, yeah. | 13:37 |
thorst | o, huh | 13:37 |
thorst | well, then yeah. I'm fine with either being proactive or lazy about it then | 13:37 |
edmondsw | thorst PowerVC wants consistency, it just doesn't want to spend the resources to scrub the translations it already has in place | 13:37 |
thorst | got it. | 13:37 |
edmondsw | but a note was sent just a couple days ago abount starting to scrub things if/when you can | 13:37 |
thorst | neat | 13:37 |
thorst | well, then ... same goes for ceiometer-powervm too | 13:38 |
thorst | that one is probably easier to do (and probably could benefit from a patch set done against it) | 13:38 |
edmondsw | I'd probably prioritize ceilometer-powervm above nova-powervm | 13:38 |
efried | Okay, upshot for nova-powervm and ceilometer-powervm is: no need to hold back if you feel like scrubbing out all the log t9n from those guys. | 13:38 |
efried | But it's not a high priority. | 13:38 |
thorst | yep | 13:39 |
edmondsw | +1 | 13:39 |
*** k0da has quit IRC | 13:39 | |
efried | I added it to the etherpad https://etherpad.openstack.org/p/powervm-in-tree-todos line 69 | 13:39 |
edmondsw | that it for OOT? | 13:40 |
efried | nothing else from me. | 13:40 |
esberglu | #topic PowerVM CI | 13:41 |
*** k0da has joined #openstack-powervm | 13:41 | |
esberglu | https://etherpad.openstack.org/p/powervm_ci_todos | 13:41 |
efried | esberglu Okay, so you moved the CI to-dos out to another etherpad. | 13:41 |
esberglu | efried: Yeah I linked it in the other one | 13:42 |
esberglu | I can move it back if that's what people prefer | 13:42 |
esberglu | But I wanted to track tempest failures there and it was becoming a lot of info | 13:42 |
efried | esberglu I'm fine with it as long as everything's cross-linked. I added a backpointer from the CI one to the original. | 13:42 |
esberglu | efried: Good call. | 13:43 |
efried | What's the difference between WORKING and CURRENT? | 13:43 |
esberglu | Stuff that I'm actually doing (in staging) vs stuff that's just on the list | 13:43 |
esberglu | We still need to figure out a way to get the VNC tests working | 13:44 |
esberglu | And check what tests (if any) can be enabled with SSP merged | 13:44 |
edmondsw | esberglu change "CURRENT" to "NEXT"? | 13:44 |
edmondsw | efried that clearer? | 13:44 |
*** tjakobs has joined #openstack-powervm | 13:44 | |
efried | Yeah, that would be fine. Not a big thang. | 13:45 |
esberglu | CI has been looking really good since the last couple fixes last week | 13:46 |
esberglu | Which should open up some time to start knocking this list out | 13:46 |
esberglu | I'm gonna go through and prioritize the list today | 13:46 |
esberglu | Been seeing way less of the timeout errors since I upped the time limit. Which to me points to slow over hanging. | 13:47 |
edmondsw | esberglu can you make looking at the tempest failures part of that prioritized list? | 13:47 |
*** thorst is now known as thorst_afk | 13:47 | |
esberglu | edmondsw: Yeah | 13:47 |
edmondsw | esberglu so you did merge that timeout bump? | 13:48 |
esberglu | edmondsw: I thought we were just putting it in temporarily for investigation purposes. But I can | 13:48 |
efried | Hope not. We need to have a lively discussion first. | 13:49 |
edmondsw | no, I just asked because you're seeing "way less" | 13:49 |
esberglu | edmondsw: I just live patched the jenkins jobs | 13:49 |
edmondsw | if it didn't merge, wouldn't it only be in effect on a one-by-one basis? | 13:49 |
edmondsw | oh | 13:49 |
efried | Basically, my stance on this is that our CI isn't just testing "does it work.... eventually?" It's also there to alert us to what I'll call "performance problems" for lack of a better term. | 13:50 |
efried | So if stuff is taking a long time, we need to figure out why it's taking a long time, not just increase the timeout. | 13:51 |
edmondsw | efried yep, I think we all agree there | 13:51 |
efried | I would even go so far as to say, if we had the space for it, we should be *decreasing* timeouts to highlight things that are taking longer than they ought. | 13:51 |
edmondsw | I'll even agree with that... once we get these current timeouts figured out / addressed | 13:52 |
efried | yup. | 13:52 |
esberglu | Should I remove the timeout increase now? It will be easier to find failing runs to investigate that way | 13:53 |
efried | esberglu When you've got the space to really start digging into them, yes. | 13:53 |
efried | Not necessary if it's just going to result in more failures but no action. | 13:54 |
edmondsw | +1 or when one of us pings you that we have that time | 13:54 |
esberglu | efried: Ok. I want to do a couple other things first (like get the neo logged) which should help for debugging | 13:54 |
efried | fo sho. | 13:54 |
edmondsw | esberglu I'm not seeing getting the neo logged on your list | 13:55 |
efried | let me know if you need help figuring out how to do that; I have a couple of ideas. | 13:55 |
esberglu | edmondsw: Yeah that list is a WIP | 13:55 |
esberglu | That's all I had for CI | 13:55 |
esberglu | #topic Driver Testing | 13:55 |
esberglu | Any progress here? | 13:56 |
efried | We don't have testers on. But thorst_afk added https://etherpad.openstack.org/p/powervm-in-tree-todos starting line 92 | 13:56 |
efried | ...pursuant to our call the other day. | 13:56 |
thorst_afk | efried: we're lining up the test resources still. I don't think any tangible change, just formulating plan | 13:56 |
esberglu | Any discussion needed here? Otherwise I'll move on, running close to time | 13:57 |
thorst_afk | don't think so | 13:58 |
esberglu | #topic Open Discussion | 13:58 |
esberglu | Any final thoughts before I call it? | 13:58 |
efried | It's really confusing in my HexChat interface that esberglu and edmondsw both start with 'e' and have the same number of letters. | 13:58 |
efried | My old IRC client had different colors for each user. Haven't figured out how to do that in HexChat. | 13:58 |
thorst_afk | efried: bringing the real problems to light | 13:59 |
edmondsw | :) | 13:59 |
efried | You can count on me. | 13:59 |
thorst_afk | I'd make a quip...but yeah, we do count on you | 13:59 |
mdrabe | I got a q actually | 13:59 |
thorst_afk | alright, I need to bail. Need to go spread the gospel of open vswitch | 14:00 |
mdrabe | For test, what's the desired deployment route, devstack or OSA? | 14:00 |
thorst_afk | mdrabe: for now, devstack due to simplicity of setup | 14:00 |
thorst_afk | which is not all that simple, until you compare to OSA. | 14:00 |
*** kriskend has quit IRC | 14:00 | |
efried | Hah, ironic considering OSA is supposed to be the thing that makes it simple. | 14:01 |
thorst_afk | efried: OSA is the thing to make OpenStack production grade | 14:01 |
mdrabe | Would this be an opportunity to iron out the OSA path then? | 14:01 |
efried | Yeah | 14:01 |
efried | Sorry, thorst_afk Yeah. mdrabe No. | 14:01 |
thorst_afk | mdrabe: kinda. Lets chat more when I'm off the phone | 14:01 |
mdrabe | I'd say wainot, but sounds like it's complicated | 14:02 |
efried | esberglu Think we're done here. | 14:05 |
esberglu | #endmeeting | 14:05 |
openstack | Meeting ended Tue Jun 6 14:05:31 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:05 |
openstack | Minutes: http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-06-06-13.02.html | 14:05 |
openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-06-06-13.02.txt | 14:05 |
openstack | Log: http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-06-06-13.02.log.html | 14:05 |
*** kriskend has joined #openstack-powervm | 14:09 | |
efried | Yeeeessssss. /set text_color_nicks on | 14:11 |
*** mdrabe has quit IRC | 14:11 | |
*** mdrabe has joined #openstack-powervm | 14:12 | |
esberglu | efried: What were your ideas for getting the hostname printed? | 14:18 |
esberglu | Is there a way to do that with pvmctl? | 14:29 |
efried | esberglu Oh, maybe. I was thinking you could use the RMC discovery business in local2remote and then do a DNS lookup. | 14:37 |
efried | or yeah, after local2remote is set up, you could ask pvmctl for the managed system's IP and then DNS that. | 14:38 |
efried | esberglu Looks like the managed system IP is the FSP (which makes sense). But generally looking that up is still gonna be useful. | 14:40 |
efried | esberglu The nvl doesn't seem to have an RMC IP set, which is annoying, but also makes sense I guess. | 14:43 |
esberglu | efried: I found an easier way | 14:43 |
efried | do tell | 14:43 |
esberglu | sudo /usr/sbin/rsct/bin/getRTAS | 14:43 |
esberglu | That's in local2remote.py | 14:43 |
esberglu | And it prints the hostname as part of it | 14:44 |
efried | Cool. | 14:44 |
esberglu | I can just run that at the start of the jenkins job | 14:44 |
*** mdrabe has quit IRC | 14:47 | |
efried | esberglu I haven't succeeded in getting just the neo hostname out of that output yet. | 14:54 |
efried | We don't really want the IPs in there. | 14:54 |
esberglu | efried: Yeah I'm trying to get a bash command to do it right now | 14:55 |
efried | Annoyingly, this prints the wrong hostame: host `python -c 'import local2remote; print local2remote._get_local_rmc_address()'` | 14:55 |
efried | Do we have a way to map from 'vhmccloudvm126' to 'neo40'? | 14:57 |
efried | Alternatively, this works, but prints some extra gorp: sudo /usr/sbin/rsct/bin/getRTAS | sed 's/.*HscHostName=\([^;]*\);.*/\1/' | 14:59 |
esberglu | Could just toss a grep neo on the end | 14:59 |
efried | If we can count on the neo's hostname starting with 'neo' | 15:00 |
efried | sudo /usr/sbin/rsct/bin/getRTAS | sed -n 's/.*HscHostName=\(neo[^;]*\);.*/\1/p' | 15:00 |
esberglu | efried: Yeah they all do in CI and that shouldn't change | 15:00 |
efried | Cool, then the above works. | 15:01 |
esberglu | efried: Sweet thanks for the assist. I'll toss it on staging quick to confirm it works | 15:01 |
efried | and is way faster than loading up local2remote and running _get_local_rmc_addr (which does things like pinging the host) | 15:01 |
efried | edmondsw Talk to me about https://review.openstack.org/#/c/469982/10/nova_powervm/tests/virt/powervm/test_event.py@64 | 15:02 |
edmondsw | efried give me 2 minutes | 15:03 |
efried | k | 15:03 |
edmondsw | efried, alright, I'm here | 15:05 |
efried | So I wrote the test to walk through each code branch in order. | 15:06 |
efried | Are you saying something is missed? | 15:06 |
edmondsw | efried not exactly... I think with the current implementation this probably tests everything | 15:07 |
edmondsw | but if someone changed things, it could pass when it shouldn't in a case I was trying to point out | 15:07 |
edmondsw | ```self.mock_driver.nvram_mgr = None``` shouldn't be set when you test [] and ['foo', 'bar', 'baz'], in case the existence of nvram_mgr somehow affects that | 15:08 |
edmondsw | too picky? | 15:09 |
edmondsw | probably too picky | 15:10 |
efried | Well, I'll argue that the foo/bar/baz case is validated in the next chunk via the foo/PartitionState/bar case. And the [] case is reductive but effectively the same. | 15:10 |
efried | I'll agree it's always possible to change code to be wrong but let existing tests pass ;-) | 15:11 |
edmondsw | I changed to +1 | 15:12 |
edmondsw | you said you owed another patch... was it just for these comments, or something else? | 15:12 |
efried | Just for these. I'm at least gonna fix the typo. And look at the other thing. | 15:12 |
*** mdrabe has joined #openstack-powervm | 15:13 | |
openstackgerrit | Eric Fried proposed openstack/nova-powervm master: Performance improvements for Lifecycle events https://review.openstack.org/469982 | 15:16 |
efried | edmondsw Done ^^ | 15:16 |
edmondsw | +2 | 15:18 |
efried | thorst_afk You happy with ^^ ? | 15:24 |
efried | esberglu https://review.openstack.org/#/c/470999/ still needs the commit message updated. | 15:29 |
esberglu | efried: Oh missed that comment. One sec | 15:30 |
esberglu | efried: Done | 15:30 |
efried | thx | 15:30 |
esberglu | efried: See 5398 for the neo hostname change | 15:31 |
efried | ack | 15:31 |
efried | esberglu +2 | 15:32 |
efried | esberglu You addressed one comment on the commit message, but not the other. | 15:36 |
esberglu | Wow I'm blind apparently | 15:37 |
efried | esberglu Copied it over to PS4 for your convenience ;-) | 15:37 |
esberglu | efried: Done. Unless there are other invisible comments | 15:42 |
*** k0da has quit IRC | 15:46 | |
mdrabe | efried for that get_info business is there any plan for this https://github.com/openstack/nova-powervm/blob/stable/liberty/nova_powervm/virt/powervm/vm.py#L100 or is it gonna stay as is? | 16:11 |
efried | mdrabe In stable/liberty specifically? No plans to change anything earlier than pike afaik. | 16:12 |
mdrabe | ^ That's liberty sorry but more or less the same for master | 16:12 |
efried | oh | 16:12 |
efried | mdrabe Yeah, that would be the idea. To change it to look like https://review.openstack.org/#/c/471146/2/nova/virt/powervm/vm.py | 16:13 |
efried | Do y'all use those other fields for something? | 16:13 |
mdrabe | Little bit yea | 16:13 |
mdrabe | I just need to watch out for it is all | 16:14 |
efried | The way we've got it implemented with the subclass, we could leave it alone OOT and it would still work. Which fields do you use? | 16:14 |
mdrabe | We use _get_property to do basically the same thing the state property does for that object | 16:15 |
efried | mdrabe You use that derived InstanceInfo directly? | 16:16 |
mdrabe | So we should be fine getting rid of everything else same as IT, just need to respond to the change when it comes | 16:16 |
mdrabe | mhm | 16:16 |
efried | mdrabe As opposed to just using get_vm_qp directly? | 16:17 |
mdrabe | well it's... lemme fin dit | 16:18 |
mdrabe | find it* | 16:18 |
efried | mdrabe Seems like you could swap over to using get_vm_qp directly - you could do that any time, and arguably should, since _get_property is private ;-) | 16:18 |
mdrabe | https://github.com/openstack/nova-powervm/blob/master/nova_powervm/virt/powervm/driver.py#L227-L240 | 16:19 |
mdrabe | We use that, that returns the extended object | 16:20 |
*** kriskend has quit IRC | 16:20 | |
mdrabe | But we can just as easily use .state | 16:20 |
mdrabe | From whatever object, I don't think it'd matter | 16:20 |
efried | mdrabe get_info is staying around, and its response is still an InstanceInfo (a nova.virt.hardware.InstanceInfo instead of our subclass), and it still has a .state. So if that's all you're doing, you're safe. | 16:22 |
efried | mdrabe If you're using nova_powervm.virt.powervm.vm.InstanceInfo directly, and/or referencing any fields other than .state from the get_info result, you'll be affected. | 16:23 |
mdrabe | Right that's what we're doing now | 16:23 |
efried | Cool. | 16:24 |
mdrabe | efried got a sec about https://bugs.launchpad.net/nova-powervm/+bug/1694784? | 16:50 |
openstack | Launchpad bug 1694784 in nova-powervm "Reduce overhead for redundant PartitionState events" [Undecided,New] | 16:50 |
efried | mdrabe Sho | 16:50 |
mdrabe | arnoldje noticed that the NVRAM manager, even though it's using the instance object throughout, only ever really needs the intance UUID | 16:51 |
efried | mdrabe Yuh, that's the next thing I was planning to look at. | 16:52 |
efried | Separate change, tho | 16:52 |
mdrabe | So for 2) under that bug, the part about getting the instance object before making nvram_mgr calls, I've been looking into getting rid of that call entirely | 16:52 |
mdrabe | ok so you're aware of this | 16:52 |
efried | mdrabe Yes, I'm aware. | 16:53 |
efried | However, given the fix that's out there, I'm actually not positive we're going to see a benefit from making the nvram mgr change. | 16:53 |
efried | wait, thinking again... maybe we would. | 16:53 |
mdrabe | It's not the nvram mgr change that sees the performance improvement, it's avoiding the get_instance call | 16:53 |
efried | Yeah, I know. | 16:54 |
efried | So yeah, cause today we're doing the instance lookup right away with nvram mgr on; I was thinking that we already get the benefit because we cache & send that instance object down to the PartitionState handler, which would be doing that instance lookup anyway. | 16:54 |
efried | But | 16:55 |
efried | many of those PartitionState events never happen, so their instance lookups would never happen. | 16:55 |
efried | So if we can avoid the lookups at the outset in the nvram branch, we would actually end up skipping a whole bunch. | 16:55 |
efried | AND we can get rid of that goofy cache. | 16:55 |
efried | which will make the code cleaner. | 16:55 |
mdrabe | Well we might still need a cache | 16:56 |
efried | So hell, even if there winds up being no perf improvement, I would support the change just for the code cleanup aspect. | 16:56 |
efried | No, we wouldn't. | 16:56 |
efried | We would have no way to populate it. | 16:56 |
efried | Anyway, are you looking into making this change? | 16:56 |
efried | So I shouldn't? | 16:56 |
mdrabe | For PVM UUID to OpenStack UUID | 16:56 |
mdrabe | Although the NVRAM mgr could just use the PVM UUID | 16:57 |
mdrabe | But that would be a problem for pvc | 16:57 |
mdrabe | I'll do it | 16:57 |
efried | mdrabe The OpenStack UUID lookup goes to the PVM REST server. The get_instance goes to the nova db. Not sure which one is more expensive, but avoiding the latter will certainly help us. | 16:58 |
efried | mdrabe Cool, looking forward to seeing that. | 16:58 |
mdrabe | I believe we already have the LPAR wrapper | 16:59 |
efried | mdrabe Nope. | 16:59 |
efried | The event just has the URI | 16:59 |
efried | Oh, sorry, hold on, I was really confused above. | 16:59 |
efried | In order to figure out the OpenStack UUID, we do indeed have to do get_instance - sometimes twice. It doesn't go to the PVM REST server. | 17:00 |
mdrabe | efried here right https://github.com/openstack/nova-powervm/blob/master/nova_powervm/virt/powervm/event.py#L98 ? | 17:02 |
mdrabe | That's arnoldje's area of concern at least | 17:03 |
efried | mdrabe You may as well start by looking at the code I'm just about to merge. | 17:03 |
efried | https://review.openstack.org/#/c/469982/ | 17:03 |
efried | Sorry, thought I had added you to ^^ | 17:03 |
efried | just did that. | 17:03 |
efried | You're gonna have to relearn it ;-) | 17:04 |
efried | pretty well refactored. | 17:04 |
efried | mdrabe So yeah, here: https://review.openstack.org/#/c/469982/11/nova_powervm/virt/powervm/event.py@98 | 17:04 |
mdrabe | yea still grabbing the instance | 17:05 |
mdrabe | And at a higher frequency if I understand this change correctly I think? | 17:06 |
efried | mdrabe No, purely taking NVRAM events into account, with this change, we're grabbing the instance with roughly the same frequency. | 17:07 |
efried | mdrabe Digging into NVRAMManager, I don't see anything there to preclude storing by PVM UUID. What's the reason that doesn't work for pvc? | 17:07 |
mdrabe | VMs not managed by PVC | 17:08 |
efried | Because you have to account for upgrade paths from previous versions that used the instance? | 17:08 |
mdrabe | We don't wanna hold NVRAM info for those VMs | 17:08 |
efried | I don't see how that's a problem ;-) | 17:08 |
efried | If no pvm UUID, you don't store anything, nah? | 17:09 |
mdrabe | Except we're processing events for all LPARs right? | 17:09 |
mdrabe | Some might not be managed | 17:10 |
mdrabe | So we get an NVRAM for an unmanaged LPAR and now we're storing NVRAM info for it | 17:10 |
mdrabe | The way to know would be to check if there's an instance object behind it | 17:10 |
efried | wait, sorry, I gotcha. LPARs not managed by pvc. Not instances that aren't LPARs. | 17:11 |
mdrabe | yea | 17:11 |
mdrabe | but we're trying to avoid looking up the instance object | 17:11 |
efried | mdrabe Yeah, and there's no way to see if there's an instance behind it without doing get_instance. | 17:11 |
mdrabe | mhm, but it's better with a cache | 17:11 |
efried | Oh, yeah, I see where you're going with that - but the nature of the cache would have to change entirely. | 17:11 |
efried | It would have to be *just* a UUID mapping cache. | 17:12 |
mdrabe | Yep | 17:12 |
efried | ...which you'd have to figure out a way to age and expire entries. | 17:12 |
efried | Cause you can't count on getting notified on deletion. | 17:12 |
mdrabe | Exciting | 17:12 |
efried | I deliberately didn't implement that part of #3 in the bug report for that reason. | 17:13 |
mdrabe | notified in what way though? | 17:13 |
mdrabe | via event? | 17:13 |
efried | yeah | 17:14 |
mdrabe | Why can't we count on that? | 17:14 |
efried | Well. | 17:14 |
efried | The instance can be deleted without telling pvm about it. Not sure what the notifications look like when you migrate. All bets are off if you get CACHE_CLEARED. Those are off the top of my head. | 17:15 |
*** k0da has joined #openstack-powervm | 17:15 | |
efried | You could account for most of those, but I can guarantee you'd still have leaks somehow. | 17:15 |
efried | And the more you try to close the holes, the bigger the code gets - and at some point you have to weigh that against the benefit you're really deriving from it. | 17:16 |
efried | Which, again, is why I didn't mess with it for PartitionState. | 17:16 |
efried | Even with aging/expiring, which would be easier in the long run. | 17:16 |
efried | I dug into the nova API, btw, because I would expect it would be way cheaper to do an 'exists' check in the db that just returns a bool, as opposed to pulling the whole Instance record and building an object and sending it across the wire. | 17:17 |
efried | But no such API exists. | 17:17 |
mdrabe | Oo yea that'd be nice | 17:17 |
mdrabe | Although nova doesn't really have this problem | 17:18 |
efried | You should propose one. There's probably a bunch of places in the nova codebase that could benefit from that. | 17:18 |
efried | And definitely from external consumers of the nova API. | 17:18 |
mdrabe | I'm gonna ask arnoldje about that | 17:19 |
mdrabe | I wonder how much faster an exists check would be vs returning the entire object | 17:20 |
efried | mdrabe Does pvc not maintain its own databases related to instances? | 17:20 |
mdrabe | It's still a query | 17:20 |
mdrabe | Nah it's all nova | 17:20 |
efried | The NVRAM and slot maps are the *only* things you store? | 17:20 |
mdrabe | For swift yea I think so | 17:21 |
efried | Not talking about swift. Though I guess I am. | 17:21 |
efried | So yeah, I see the issue. | 17:21 |
mdrabe | Yea it could be stored anyhow I suppose | 17:21 |
mdrabe | The Instance object table specifically is pure nova in pvc | 17:22 |
efried | I was gonna say it shouldn't be too much extra storage - but you really never know what the env is gonna look like. | 17:22 |
mdrabe | What shouldn't be extra storage? | 17:22 |
efried | Although I really wouldn't expect people to be hosting a whole bunch of non-openstack LPARs on a compute node running openstack. | 17:22 |
efried | Hypothetically storing NVRAM for all LPARs that send you notifications. | 17:23 |
mdrabe | Ah right right | 17:23 |
efried | Is the size of the 'unmanaged' set bounded/finite? | 17:23 |
efried | Well, it's both of those things, but is it reliably small? | 17:23 |
mdrabe | It's bounded to scale reqs | 17:24 |
efried | Ya know, to that point, how are we accounting for dropping records for deleted LPARs in the NVRAM & slot mgrs? | 17:24 |
mdrabe | But it's tricky to say what folks will or won't do, I've seen some weird stuff done | 17:24 |
mdrabe | Mmm that's a good q | 17:24 |
efried | I guess there's prolly a periodic sweep that does a full LPAR feed GET and scrubs whatever ain't there. | 17:25 |
mdrabe | Idk if we do any kind of scrubbing | 17:25 |
mdrabe | I would hope so | 17:25 |
mdrabe | but I'm not aware of that | 17:25 |
efried | But where? | 17:25 |
efried | I would have thought it would be... here. | 17:25 |
mdrabe | I think we just do it sychronously with delete | 17:25 |
efried | Oh, yeah, that'd do it. | 17:26 |
mdrabe | Still can get stales though | 17:26 |
efried | How? | 17:26 |
mdrabe | I'm thinking of negative cases primarily | 17:27 |
mdrabe | something fails in destroy, maybe the instance record is gone but the NVRAM / slot map objects remain stored wherever they are | 17:27 |
mdrabe | Actually idk if that can happen | 17:28 |
efried | That would be a bug, one would hope. | 17:29 |
efried | But hopefully rare enough to be a vanishingly small problem. | 17:30 |
mdrabe | yea, anyway I'm getting hungry, gonna grab some grub. I'll dive into this afterwards | 17:30 |
efried | rgr | 17:31 |
*** k0da has quit IRC | 17:43 | |
openstackgerrit | Merged openstack/nova-powervm master: Fix some reST field lists in docstrings https://review.openstack.org/467452 | 17:47 |
openstackgerrit | Merged openstack/nova-powervm master: Performance improvements for Lifecycle events https://review.openstack.org/469982 | 17:50 |
openstackgerrit | Merged openstack/networking-powervm master: Updated from global requirements https://review.openstack.org/465572 | 17:55 |
efried | mdrabe ^^ now merged, pull your master branch accordingly. | 17:55 |
*** chhavi has quit IRC | 17:55 | |
esberglu | https://review.openstack.org/#/c/416667/ | 17:58 |
esberglu | Can we abandon that? | 17:58 |
esberglu | thorst_afk: efried: adreznec: ^ | 17:58 |
thorst_afk | I defer to adreznec | 17:59 |
efried | esberglu Rereading. I can't remember why we were doing this, but we had a really good reason at the time. | 18:00 |
efried | esberglu To work with this, we would just have to add `enable_service pvm-q-sea-agt` to our local.confs, right? | 18:04 |
efried | ...which oughtta be harmless even if it's already there (i.e. if we put that in place before this merges)? | 18:05 |
*** k0da has joined #openstack-powervm | 18:05 | |
esberglu | efried: IIRC we already put a change into neo-os-ci that has that in preparation for the above merging | 18:05 |
esberglu | But I thought we didn't want to merge the above anymore for some reason | 18:05 |
efried | So we should be able to recheck and have it work? | 18:05 |
esberglu | Let me look in the code quick and make sure | 18:06 |
esberglu | efried: Yeah it's enabled in both local.conf files | 18:06 |
efried | So I think it's a good idea for us to do this anyway. We have OVS support OOT, so you theoretically wouldn't need to have either of these enabled (if you're a regular consumer). | 18:07 |
efried | Let's powervm:recheck this sucker and make sure it has legs, then merge it already. | 18:07 |
openstackgerrit | Eric Berglund proposed openstack/networking-powervm master: Switch to manual service enablement for devstack plugins https://review.openstack.org/416667 | 18:07 |
thorst_afk | efried: but you wouldn't install networking-powervm if you were using the OVS | 18:07 |
thorst_afk | that's my issue with it | 18:07 |
thorst_afk | if you're installing networking-powervm, you're using either SR-IOV or SEA | 18:07 |
efried | thorst_afk But rarely both. | 18:07 |
thorst_afk | why would you install it but not use anything from it. | 18:07 |
thorst_afk | sure, but at least one. | 18:07 |
efried | Okay, that's a fair point. But there would be no way to disable one of them. | 18:08 |
efried | I'm also thinking this is a pretty common model for external services to need enable_service in their local.confs | 18:09 |
esberglu | efried: You could disable_service the one you don't want if you only want to run one I think? Same thing essentially | 18:10 |
esberglu | Unless the enable_service here https://review.openstack.org/#/c/416667/3/devstack/settings | 18:10 |
esberglu | overrides that | 18:10 |
esberglu | I guess the enable_service way still seems better to me though | 18:11 |
efried | I agree, and I'd rather move forward with it than abandon it. | 18:12 |
efried | Or we can continue to let it languish. | 18:12 |
efried | But esberglu is in cleanup mode, clearly ;-) | 18:12 |
*** k0da has quit IRC | 18:12 | |
esberglu | efried: Gotta take advantage when CI isn't on fire | 18:16 |
efried | esberglu While you're not nailing down the timeouts, anyway. | 18:17 |
esberglu | efried: Yeah I'm gonna turn the timeout back down this afternoon once I finish up the last couple things | 18:18 |
adreznec | esberglu: efried Sorry, just finished reading backscroll. Are we going to try and get that merged in then? | 18:24 |
esberglu | adreznec: Yep that's the plan. Just waiting for CI results | 18:24 |
adreznec | I see esberglu rebased it | 18:24 |
adreznec | Cool | 18:24 |
efried | adreznec I'd like to, but I think thorst_afk is still on the nay side. | 18:24 |
adreznec | Well he's afk, so clearly we just need to merge it before he gets back to object | 18:25 |
adreznec | :) | 18:25 |
esberglu | efried: Test timeouts are back to the default for CI | 18:31 |
thorst_afk | I'm still of the mindset that if you're installing it, you likely want SEA enabled | 18:32 |
thorst_afk | but | 18:32 |
thorst_afk | I'll also admit that the only ones using devstack are right here | 18:33 |
thorst_afk | so... | 18:33 |
thorst_afk | if this group wants it shut off in devstack / networking-powervm...I'll just ignore the review :-) | 18:33 |
*** jpasqualetto has joined #openstack-powervm | 18:43 | |
*** thorst_afk has quit IRC | 18:54 | |
*** thorst_afk has joined #openstack-powervm | 18:56 | |
*** thorst_afk has quit IRC | 19:00 | |
*** thorst_afk has joined #openstack-powervm | 19:14 | |
*** thorst_afk has quit IRC | 19:15 | |
*** thorst_afk has joined #openstack-powervm | 19:16 | |
*** k0da has joined #openstack-powervm | 19:23 | |
edmondsw | thorst_afk sure, if you're installing it you likely want either SEA or SR-IOV agents enabled, but not necessarily both, right? | 19:27 |
efried | I would contend *usually* not both. | 19:28 |
*** mdrabe has quit IRC | 19:46 | |
openstackgerrit | Merged openstack/networking-powervm master: Switch to manual service enablement for devstack plugins https://review.openstack.org/416667 | 19:56 |
*** mdrabe has joined #openstack-powervm | 19:56 | |
thorst_afk | efried edmondsw: sure. And neutron has both the Linux Bridge and OVS agents. It by default now turns on the OVS and if you want to use LB you turn off OVS and enable LB... | 20:42 |
edmondsw | thorst_afk you suggesting that SEA be on by default but not SR-IOV? | 20:43 |
thorst_afk | edmondsw: yes. | 20:43 |
thorst_afk | brb | 20:43 |
edmondsw | that makes sense | 20:45 |
*** smatzek has quit IRC | 20:46 | |
*** svenkat has quit IRC | 21:16 | |
*** thorst_afk has quit IRC | 21:19 | |
*** thorst_afk has joined #openstack-powervm | 21:20 | |
esberglu | efried: Of course when I'm finally hoping to see a timeout there aren't any coming through... | 21:23 |
efried | Of course. | 21:24 |
*** thorst_afk has quit IRC | 21:24 | |
esberglu | efried: edmondsw: I put up 2 changes, 1 to neo-os-ci and 1 to powervm-ci | 21:35 |
edmondsw | esberglu looking now | 21:35 |
esberglu | This is moving prep_devstack.sh for the same reason we moved the local.conf files | 21:35 |
esberglu | We can merge the powervm-ci one with no impact to production | 21:35 |
edmondsw | esberglu tempest really doesn't have a new version for stable/ocata, still uses master there? | 21:35 |
esberglu | edmondsw: Technically you are supposed to be used tempest master for the last 3 releases | 21:36 |
esberglu | But it doesn't work for newton | 21:36 |
edmondsw | wow | 21:36 |
edmondsw | esberglu I was also going to ask about the newton block | 21:36 |
esberglu | At least that was my understanding based on | 21:36 |
esberglu | https://github.com/openstack/tempest/releases/tag/15.0.0 | 21:37 |
edmondsw | why just 13.0.0... should we be figuring out the latest 13.y.z? | 21:37 |
edmondsw | and if master is supposed to work with newton, why doesn't it? | 21:38 |
esberglu | So at some point we were checking out different tempest versions for every branch. I think due to misunderstanding how far back master supported | 21:39 |
esberglu | Moving ocata to master worked but newton had issue (something with our security config in tempest IIRC) | 21:40 |
esberglu | And it just fell into the backlog of stuff and I haven't had a chance to look again | 21:40 |
esberglu | It might be missing from the TODO list though let me check | 21:40 |
esberglu | edmondsw: And I don't think there is a 13.y.z release other than 13.0.0 | 21:42 |
edmondsw | esberglu no, there doesn't seem to be... I was just thinking they could decide to cut one to fix something | 21:42 |
edmondsw | esberglu fyi: https://github.com/openstack/tempest/blob/16.0.0/README.rst#release-versioning | 21:42 |
edmondsw | with 12.y.z there was a 12.0.0, 12.1.0, and 12.2.0, but they don't seem to have done anything like that since | 21:44 |
edmondsw | maybe we just adjust if/when they do that before we get newton using master, if that's what it should really be using | 21:44 |
*** mdrabe has quit IRC | 21:44 | |
*** tjakobs has quit IRC | 21:46 | |
esberglu | edmondsw: I added it to the todo list. I don't think there is really a huge rush to get newton on master unless you feel otherwise | 21:47 |
esberglu | Not sure exactly what you mean by that last line | 21:48 |
edmondsw | esberglu no, I don't think there's any rush | 21:48 |
edmondsw | esberglu I meant if there is a 13.0.1 to fix some bug, we could update this script at that point to use 13.0.1, and not necessarily worry about it now | 21:49 |
esberglu | They won't ever release a 13.0.1 | 21:49 |
esberglu | They only have a master branch so they can't add a 13.y.z tag anymore | 21:50 |
esberglu | Only 16.0.1 and up | 21:50 |
edmondsw | ah, true | 21:50 |
edmondsw | esberglu did you have to make any changes in prep_devstack.sh, or is it just a straight copy? | 21:52 |
esberglu | I moved the ssh key and git global configuration steps to the image template build | 21:53 |
esberglu | It also sets the pypowervm_repo using $(git remote get-url origin) | 21:54 |
esberglu | in line 185 | 21:54 |
esberglu | Because that was previously the internal morpheus | 21:54 |
esberglu | Now we clone from morpheus instead of github | 21:54 |
esberglu | In the image-template build | 21:54 |
esberglu | So that command will set it to the internal repo without us putting the internal url in the public repo | 21:55 |
esberglu | (at least that's the idea) | 21:55 |
esberglu | edmondsw: I'm heading out. Will address any concerns tomorrow | 21:57 |
edmondsw | esberglu good night | 21:57 |
esberglu | same to you | 21:57 |
*** k0da has quit IRC | 22:02 | |
*** jpasqualetto has quit IRC | 22:18 | |
*** thorst_afk has joined #openstack-powervm | 22:50 | |
*** jwcroppe has quit IRC | 23:07 | |
*** thorst_afk has quit IRC | 23:09 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/nova-powervm master: Updated from global requirements https://review.openstack.org/470121 | 23:22 |
*** svenkat has joined #openstack-powervm | 23:31 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!