Tuesday, 2018-09-18

*** efried_off is now known as efried13:05
efriededmondsw: I've got to take a kid to the dentist this morning, will miss the meeting.13:30
efriedSorry for the short notice.13:30
efriedI'm in solo parentis for a couple of days, and it's been pretty heinous13:30
efriedOne thing if it was just life-as-normal, but we chose days where multiple people have multiple special events. And right after getting back from a trip.13:31
edmondswefried sounds fun! no worries here. Take care of the fam13:31
*** efried is now known as efried_afak13:32
*** efried_afak is now known as efried_afk13:32
*** efried_afk is now known as efried13:47
efriededmondsw: Hah, the dentist called right as I was about to leave and gave me an out. So I'll be in the mtg.13:47
edmondsw:)13:48
*** mujahidali has joined #openstack-powervm13:52
edmondsw#startmeeting PowerVM Driver Meeting14:00
openstackMeeting started Tue Sep 18 14:00:48 2018 UTC and is due to finish in 60 minutes.  The chair is edmondsw. Information about MeetBot at http://wiki.debian.org/MeetBot.14:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
*** openstack changes topic to " (Meeting topic: PowerVM Driver Meeting)"14:00
openstackThe meeting name has been set to 'powervm_driver_meeting'14:00
edmondsw#link agenda: https://etherpad.openstack.org/p/powervm_driver_meeting_agenda14:00
edmondswefried mujahidali y'all here?14:01
mujahidali^14:02
efriedō/14:02
edmondsw#topic In-Tree Driver14:02
*** openstack changes topic to "In-Tree Driver (Meeting topic: PowerVM Driver Meeting)"14:02
edmondswI haven't had any time to look at this in a while, and probably won't for a while14:02
edmondswefried anything we should discuss here, particularly anything from the PTG?14:02
efriedum14:03
efriedno, I don't think so.14:03
edmondswcool14:03
edmondswmoving on14:03
edmondsw#topic Out-of-Tree Driver14:03
*** openstack changes topic to "Out-of-Tree Driver (Meeting topic: PowerVM Driver Meeting)"14:03
edmondswI've got a WIP patch up to update our devstack examples: https://review.openstack.org/#/c/601628/14:04
edmondswstill working on some things there, but the SEA changes work14:04
edmondswit's an ongoing, as-able type of thing14:04
edmondswmdrabe has also been working on secure boot: https://review.openstack.org/#/c/595877/14:06
edmondswthere was a bug in pypowervm 1.1.17 so we had to release 1.1.18 for this to work properly14:06
*** mdrabe has joined #openstack-powervm14:06
edmondswI believe the patch has been updated to use 1.1.18 now, but last I checked that wasn't in u-c yet so it wouldn't work14:06
edmondsw#action edmondsw check on u-c for pypowervm14:07
edmondswanything else to discuss for OOT?14:07
edmondswmdrabe how are things looking for you to test MSP support, time-wise?14:08
mdrabeI can spare a moment14:09
mdrabeDo we have a multinode env?14:09
edmondswI have one devstack aio up that you could use, but not a 2nd compute to go with it14:09
edmondswrather, I have a second system partly setup, but not devstacked14:09
edmondswI am tied up in other things atm, but if you're available to work on that I could give you a devstack local.conf that I think should work and you could try it / play with that to get things working14:10
efriedI can lend my machine if you need a second.14:11
mdrabeMaybe, that'll take me some time. Got some other stuff to get to first14:11
edmondswthanks, but it'll need to be the 2nd that I already identified, so they share an SSP14:11
edmondswrather, I already set it up to use the same SSP14:11
efriedight14:11
edmondswmdrabe ok, let me know when you can get to it14:12
edmondsw#topic Device Passthrough14:12
*** openstack changes topic to "Device Passthrough (Meeting topic: PowerVM Driver Meeting)"14:12
edmondswefried ^14:12
* efried totally unprepared to report on anything14:12
efriedSean Mooney and I are going to work through some of the yaml file format ideas14:13
efriedRahul is going to start working on a PoC based on what we've got at the moment.14:13
efriedCyborg just might have a spawn-only solution in Stein.14:14
efriedwell, a flavor vehicle, so technically I suppose you could hot attach with resize.14:14
efriednot sure if that's going to be a supported path.14:14
efriedclosing on nrp is top priority14:15
efriedThat's about all I've got for now, until I get my head organized again. Hopefully more coherent next week.14:15
efriedAny questions?14:15
edmondswmuch discussion at the PTG about this stuff?14:15
efriedyes, from several angles.14:16
efriedMainly cyborg14:16
efriedLet me summarize what that's going to look like for phase 114:16
efriedcyborg API/CLI to create a "device profile"14:16
efriedStuff a reference to that profile into the flavor14:16
efriedboot with the flavor14:16
edmondswis that device profile anything like what we've designed to put in our yaml?14:17
efriedcompute communicates with cyborg to turn that reference into an object to pass to virt. Virt does the attach.14:17
efriedIt's analogous to a neutron port.14:17
efriedSo you create it with e.g. a resource class and traits, and it's just hanging out there in the world.14:18
efriedThen you boot with it and there's a point where it gets "bound" - only then does it become part of an instance+host.14:18
efriedAnd then the bound thingy is what gets passed to the virt driver, which does the attach.14:18
efriedAs for the yaml file, we agreed we were going to try to make it a common format that could be read by cyborg and/or nova.14:20
edmondswI assume "boot with it" = "cyborg programming an FPGA"14:20
efriedno14:20
efriedphase 1 I believe we're just talking about non-programmable device passthrough.14:21
efriedI mean nova boot --flavor X14:21
efriedwhere flavor X contains a reference to the device profile thingy (VAR - Virtual Accelerator Reference - I believe)14:21
edmondswthen I have no idea what this means: "Then you boot with it and there's a point where it gets "bound" - only then does it become part of an instance+host."14:22
edmondswif it's not programming, then the thing exists on the host before any of this process has started, nothing is creating it14:22
efriedopenstack var create --resource-class GPU --traits CUSTOM_FOO CUSTOM_BAR14:24
efried==> $var_uuid14:24
efriedopenstack flavor update --flavor X --extra-specs hw:cyborg:var:$var_uuid14:24
efriednova boot --flavor X14:24
efriedor are you talking about inventorying and discovery?14:24
efriedactually maybe s/var_uuid/profile_name/ -- I don't think these are one-time-use gizmos, I think they're more like flavors themselves. So the "binding" step actually picks a specific device on the host.14:25
efriednow I can't remember14:25
efriedBut Sundar is going to put up a spec.14:25
edmondswI'll wait and read the spec14:25
edmondswping me when it's up?14:26
efriedhttps://etherpad.openstack.org/p/stein-ptg.cyborg-nova-new14:26
edmondswlot there14:27
edmondswis this being actively looked at, or are comments there a waste of time?14:27
edmondsws'why I prefer commenting in reviews14:27
edmondswplus they have more structure so easier to make sense of them14:27
efriedI don't know if Sundar is still consuming that to construct his spec.14:29
efriedIn any case, yeah, I'll let you know when I see that spec come through.14:29
edmondswtx14:29
efriedBut in the meantime, if you have questions/comments, I may be able to answer them14:29
efriedbecause they may have already been discussed14:29
efriedwe had several hours to get to this point.14:30
edmondswI wasn't able to follow what you meant above, so...14:30
efriedabove where?14:30
edmondswa lot of it14:30
edmondswlooking at the etherpad helped me understand 'bound14:31
efriedguess it helps to be familiar with the neutron flow14:31
edmondswI'll see if I can figure out the rest later if i get some time14:31
efriedwhich I *barely* am.14:31
edmondswmight, but I don't think that's my issue here14:32
edmondswanyway, moving on14:32
efriedk, well, let me know if you want to discuss further.14:32
efriedyeah14:32
edmondsw#topic PowerVM CI14:32
*** openstack changes topic to "PowerVM CI (Meeting topic: PowerVM Driver Meeting)"14:32
edmondswmujahidali how are things?14:32
edmondswyesterday you had identified a couple nodes that were always timing out.. that fixed?14:33
mujahidaliJobs running on Neo24 and ne08 are taking too long to complete and eventually failing after timeout and neo 6 was having some problem with pvmctl so restarted neo6 and redeployed neo,6,8,24.14:33
mujahidaliCI run looks better now.14:33
edmondswgreat, tx14:33
edmondswthat it?14:35
mujahidaliI have tried installing nodepool,zuul and jenkins  on my test env but things are a little complicated14:35
edmondswthat sounds like an understatement :)14:35
mujahidalinodepool and zuul are not launching14:35
edmondswdid you also install zookeeper?14:35
mujahidalino14:35
edmondswI remember esberglu saying that would be needed with the newer versions14:36
mujahidaliokay14:36
mujahidaliI first want to try it on dev env then stage and then prod.14:36
edmondswyou have a dev env other than the staging env?14:37
edmondswif so, great14:37
mujahidaliI am just using vms14:37
mujahidalito look how installation and config is working.14:37
edmondswk14:37
edmondswsure14:37
mujahidaliwe had a discussion for the blacklist and whitelist for the 700 test case with :efried14:38
mujahidaliI am not able to conclude from that discussion.14:39
edmondswI think we table that for the time being14:39
mujahidaliokay14:39
edmondswanything else?14:39
mujahidalithat's it from me.14:39
edmondswthanks14:40
mujahidali:)14:40
edmondsw#topic Open Discussion14:40
*** openstack changes topic to "Open Discussion (Meeting topic: PowerVM Driver Meeting)"14:40
edmondswnothing from me here14:41
edmondswanyone else?14:41
mujahidalino14:41
edmondswalright, thanks everyone14:42
edmondsw#endmeeting14:42
*** openstack changes topic to "This channel is for PowerVM-related development and discussion. For general OpenStack support, please use #openstack."14:42
openstackMeeting ended Tue Sep 18 14:42:05 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:42
openstackMinutes:        http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2018/powervm_driver_meeting.2018-09-18-14.00.html14:42
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2018/powervm_driver_meeting.2018-09-18-14.00.txt14:42
openstackLog:            http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2018/powervm_driver_meeting.2018-09-18-14.00.log.html14:42
*** mujahidali has quit IRC14:46
edmondswmdrabe looks like u-c has pypowervm 1.1.18 in it now, so we should be good on https://review.openstack.org/#/c/595877 once you address the RR thing14:48
mdrabeedmondsw: There shouldn't be any additional changes for RR14:50
edmondswsee the comment I made in the review14:50
edmondswmdrabe it's the same thing we were talking about the other day14:50
mdrabeOh you want to block it in the driver?14:51
edmondswwhere else?14:51
mdrabeFor pvc we do it in the scheduler14:51
edmondswI don't think that's an option for community, since we don't have any powervm logic in the scheduler there14:52
mdrabeMm okay, I can make the change to block it in the driver for nova-powervm, it'll be a functional difference between pvc. I guess that's okay14:53
edmondswwould just mean pulling out the last section of lines you added in vm.py, right?14:54
mdrabeI think there'd need to be an explicit exception raised in the driver14:55
edmondswmdrabe should raise an exception anyway... you just thinking the default exception might not have enough info to understand why it failed?14:58
edmondswif that's the case, then we'd have an issue with the non-RR failure case even without this update... and similarly for other capabilities. So I hope the default exception is good enough14:59
mdrabeYea we can count on that I suppose15:00
mdrabeI'll remove those lines in vm.py, the evacuate would go through but the spawn would fail on the hypervisor15:00
edmondswand then nova would roll it back, right?15:01
mdrabeI don't think there's evacuation rollback15:02
edmondswif it leaves the VM on a host that can't support it, how will we move it to a host that does support it?15:02
edmondswmaybe cold migration15:03
mdrabeOr well, I believe the nova-powervm driver will clean up the LPAR/adapters from the failed deploy15:03
mdrabeIf the exception that gets raised triggers a rebuild, then the scheduler will choose a different host to deploy to15:03
edmondswbetter test and see what happens15:04
edmondswso we know we're not getting things stuck15:04
mdrabeStuck how?15:04
mdrabeYou should still be able to trigger an evacuate to a different host15:04
mdrabeThe question of rollback just doesn't really apply to evacuation I don't think15:05
edmondswI thought you were saying that the LPAR would be gone from the original host we RR'ed from15:05
mdrabeNo15:05
edmondswok, then we're probably fine15:05
edmondswjust try another RR to a different host15:05
mdrabeYep15:05
*** openstackgerrit has joined #openstack-powervm15:08
openstackgerritMatt Rabe proposed openstack/nova-powervm master: Support secure_boot extra_spec in _format_flavor  https://review.openstack.org/59587715:08
*** efried has quit IRC20:44
*** efried has joined #openstack-powervm20:44

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!