Tuesday, 2017-09-19

*** edmondsw has quit IRC00:22
*** thorst has joined #openstack-powervm00:26
*** edmondsw has joined #openstack-powervm00:46
*** edmondsw has quit IRC00:50
*** chhavi has joined #openstack-powervm01:15
*** thorst has quit IRC01:27
*** chhavi has quit IRC02:03
-openstackstatus- NOTICE: Gerrit is being restarted to feed its insatiable memory appetite02:41
*** thorst has joined #openstack-powervm03:28
*** thorst has quit IRC03:33
*** chhavi has joined #openstack-powervm03:37
*** thorst has joined #openstack-powervm04:29
*** thorst has quit IRC04:34
*** thorst has joined #openstack-powervm05:30
*** thorst has quit IRC05:34
*** chhavi has quit IRC06:05
*** mdrabe has quit IRC06:05
*** chhavi has joined #openstack-powervm06:05
*** mdrabe has joined #openstack-powervm06:10
*** thorst has joined #openstack-powervm06:31
*** thorst has quit IRC06:35
-openstackstatus- NOTICE: Post jobs are not executed currently, do not tag any releases07:23
*** ChanServ changes topic to "Post jobs are not executed currently, do not tag any releases"07:23
*** thorst has joined #openstack-powervm07:32
*** thorst has quit IRC07:36
*** thorst has joined #openstack-powervm08:33
*** thorst has quit IRC08:37
*** thorst has joined #openstack-powervm09:33
*** thorst has quit IRC09:38
*** thorst has joined #openstack-powervm10:18
*** thorst has quit IRC10:19
*** thorst has joined #openstack-powervm11:19
*** thorst has quit IRC11:26
*** kylek3h_ has joined #openstack-powervm12:00
*** kylek3h has quit IRC12:03
*** kylek3h_ has quit IRC12:04
*** thorst has joined #openstack-powervm12:04
*** edmondsw has joined #openstack-powervm12:09
*** kylek3h has joined #openstack-powervm12:25
*** esberglu has joined #openstack-powervm12:52
esberglu#startmeeting powervm_driver_meeting13:00
openstackMeeting started Tue Sep 19 13:00:48 2017 UTC and is due to finish in 60 minutes.  The chair is esberglu. Information about MeetBot at http://wiki.debian.org/MeetBot.13:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.13:00
openstackThe meeting name has been set to 'powervm_driver_meeting'13:00
edmondswo/13:01
efriedo/13:01
mdrabeo/13:01
esberglu#topic In-Tree Driver13:01
edmondswtx to esberglu for getting the queens spec up13:02
efried++13:02
esbergluI was planning on starting either the OVS or SEA IT patch this week13:02
esberglunp13:02
edmondswwe talked over the basic gist of the plan the spec lays out at the PTG, and the only response (from mriedem) was "that seems fine"13:03
edmondswesberglu sounds good13:03
esbergluNot sure which yet, I think OVS has a patch started but CI isn't set up for it. SEA is set up for CI but no work has gone into so far afaik13:04
esbergluedmondsw: Cool13:04
edmondswmriedem did ask about whether the CI would use OVS or SEA, and efried said SEA13:05
edmondswmakes sense since that's what we do OOT13:05
efriedIt would be neat to figure out OVS for the CI, but they're not going to require it.13:05
edmondswyeah, if we could somehow test both that would be awesome13:06
efriedBecause they recognize it would mean we would need essentially two separate CIs13:06
efriedand they don't force others to do that.13:06
esbergluOkay so SEA first then13:06
esbergluAnd then OVS IT with whatever CI changes are needed if practical13:06
efriedBut yeah, it would be pretty cool if, like, we could have the CI alternate or something.13:06
edmondswconfig drive patch still needs reviews from our team13:08
esbergluefried: Eh I think that would be weird. Like if you do a recheck and suddenly its using different networking13:08
edmondswesberglu yeah, it's definitely not optimal... but neither is never testing one of them13:09
efriedesberglu I don't disagree it would be weird.13:09
efriedYeah, what edmondsw said.13:09
edmondswat least we would see intermittent errors if we were alternating13:09
efriedLike, maybe we test even-numbered change sets with one, odd-numbered with the other.13:09
edmondswnot saying we have to do that, just something to think about13:09
efriedThen at least rechecks would be consistent.13:09
efriedBut need a way to force one or the other, so we can specifically target changes that affect the networking code, or whatever...13:10
esbergluedmondsw: efried: We can have this discussion further once we get to OVS. But yeah I agree we should try to get both of them being tested somehow13:10
esbergluI'll keep it in the back of my mind13:10
efriedI think the major blocker there was us running local commands, right?  Which we're no longer doing.13:10
edmondswanother thing we need to think about IT is the docs.. we never finished our TODOs there13:12
esbergluI don't think that was the only blocker. I have some notes somewhere that I need to dig up13:12
edmondswwe added ourselves to the support matrix, but didn't add a hypervisors page for powervm to the config ref13:12
edmondswprobably a bigger deal for queens than for pike, since the pike IT driver isn't all that usable without networking, but still...13:13
esbergluYeah I think they had a freeze on adding docs there when we tried to do that during pike13:13
edmondswif we could get the framework up for those docs, then we could be adding to it in the patches we are submitting for queens instead of after the fact13:14
esbergluThey were moving around the location of certain docs and didn't want new ones at that point13:14
esbergluI can look into it13:14
edmondswyep, understood13:15
edmondswI think the location is locked down now13:15
edmondswI think that's all for IT?13:15
esberglu#action esberglu: Finish docs updates13:15
esbergluYep13:15
esberglu#topic Out-of-Tree Driver13:15
esbergluAnyone have discussion items here?13:16
edmondswwe had to spin pypowervm 1.1.8 to fix a max slots bug. The u-c update for that has merged now after some initial gate issues13:17
efriedCurse LPAR builder.13:18
edmondswtjakobs has a patch up for iSCSI that I need to look at13:18
edmondswif any others have time to check that out...13:18
edmondswwe're pulling the never-really-supported Linux Bridge code13:19
efriedDid we decide to pull the trigger on that?13:20
edmondswtjakobs also has a patch up for ceph volumes to review13:20
edmondswefried yeah, I think so... I checked with thorst and he said pull it13:20
efriedk, well, that change is ready for review then.13:20
efriedprivsep stuff <waves hands>13:21
edmondswmdrabe there's a merge conflict on https://review.openstack.org/#/c/497977/13:21
mdrabeyep13:21
esbergluefried: Yeah they renamed dac_admin so I just need to keep an eye for that merge13:22
efriedPerhaps we should consider implementing support for Depends-On in our OOT CI.13:23
esbergluefried: Yep it's on the TODO list13:23
efriedokay.13:23
edmondsw++13:24
esbergluReady to move on?13:24
edmondswyep, on to PCI13:24
esberglu#topic Device Passthrough13:25
efriedWe got buy-in at the PTG that the way Nova will handle device passthrough should be done in ways that won't paint non-libvirt into a corner.13:26
edmondsw++++++++++13:27
efriedI feel like we need three major pieces of framework before we can work a full solution:13:27
efried1) Nested resource providers (in placement).  This got prioritized for early Queens.13:27
efried2) Ability to request allocations with a richer syntax, to support a) multiple allocations of the *same* resource class from *different* resource providers (e.g. VFs on different physnets); and b) ensuring multiple resources can be grabbed from the *same* resource provider (e.g. claim the VF inventory and the bandwidth inventory from the same PF).  This is committed for Queens (jaypipes to write spec; I started some cont13:29
efriedent)13:29
efried3) Affinity/anti-affinity via placement aggregates.  This is *not* committed for Queens.13:29
efriedOnce all of those things are done, Nova will consider starting on whatever framework is necessary for generic device management, with an eye towards getting rid of the existing PCI management code entirely.13:30
efriedBut once all of those things are done, we may be able to get a jump start on converting over from...13:31
efriedThe Hack13:31
efriedSo for the Q cycle, we're going to move forward with https://review.openstack.org/494733 and https://review.openstack.org/49643413:31
edmondswdid they talk about how to get rid of hte existing PCI management code without breaking operators who are still using it? backward compat type stuff13:32
efrieddefer, defer, defer13:32
efriedfingers in ears13:32
efriedlalalala, not going to think about it yet.13:32
edmondswlol13:32
efriedBut it'll be a deprecation cycle.13:32
efriedI anticipate individual hypervisors being able to take the lead for themselves.13:33
efriedto some extent13:34
edmondswFYI all, mdrabe is going to be helping efried on this as able, and driving the corresponding work in PowerVC13:34
efriedbecause the ways of specifying resources (in general, not just devices) is going to have to cut over at some point.13:34
edmondswefried do we need a recheck on https://review.openstack.org/#/c/496434/ or are you going to upload a new ps?13:36
efriedOh, yeah, it needs UT and stuff.  It's not close to done.  That and the spec are going to be my main gig for the next couple of weeks.13:37
efriedAlso need to talk to consumer architects (Joe?) to be sure we understand the requirements.13:37
edmondswyeah, I need to set that up13:38
efriede.g. do we need to do more hackage to overcome the limitation on grouping by vendor/product ID?13:38
edmondsw#action edmondsw: schedule PCI attach mtg with stakeholders13:38
edmondswefried anything else here?13:39
efriedI don't think so.13:39
efriedactually13:40
efriedlemme just drop a couple more links here for reference...13:40
efriedhttps://etherpad.openstack.org/p/nova-ptg-queens-generic-device-management13:40
efriedhttps://etherpad.openstack.org/p/nova-multi-alloc-request-syntax-brainstorm  <== this is for #2 mentioned above13:40
efriedthat is all13:41
edmondsw#topic PowerVM CI13:41
esbergluSlowly knocking out tempest failures and CI todo backlog items13:42
esbergluWe still need to get console tests working for CI which was having problems due to remote13:43
esbergluHaven't looked into that for a while because other things have been breaking13:43
edmondswdid you and thorst ever come up with a plan for that?13:43
esbergluWe talked about it a little bit a couple times13:43
esbergluBut not a solid plan13:43
edmondswit's not my top priority13:44
edmondswI would work on OVS/SEA IT first, adding Depends-On support, obviously any CI breakages...13:44
esbergluedmondsw: Yeah it keeps getting bumped by higher priority stuff. IIRC there are only 2 tempest tests for it13:45
edmondswbut good to not forget it13:45
esbergluI also am still working on that openstack dashboard. I got the web UI up on a test system but it wasn't populating properly13:45
edmondswwhat dashboard?13:46
efriedYeah, what dashboard?13:46
esbergluOpenstack health dashboard13:47
esbergluIts the tempest failure stats UI that the community uses13:47
esbergluhttp://status.openstack.org/openstack-health/#/13:47
edmondswok, I think we did talk about that13:48
esbergluThat's all I had for CI13:49
efried*-powervm are on there13:49
efriedwhat do we need beyond that?13:49
esbergluThat's not results for PowerVM CI13:50
esbergluThat's results on *-powervm for the community CI13:50
efriedokay.13:50
edmondswoh, interesting...13:51
edmondswso if we get our CI added, how would that look? Munged together with what is already there for *-powervm?13:51
efriedor a separate dashboard that just shows our runs?13:51
esbergluIt would be a separate dashboard that just shows our runs13:52
esbergluThat dashboard only show the tests that you see under the "Jenkins check" header on reviews. It doesn't have any 3rd party13:53
edmondswlooks like you can click one of these, e.g. nova-powervm, and get a new page with more details about what jobs ran there13:53
edmondswesberglu oh, so you'd be setting up something totally separate?13:53
edmondswdoes any other 3rd party CI do that?13:54
esbergluedmondsw: Yeah but that still is only showing the jenkins jobs (py27, py35)13:54
esbergluIt would be something totally separate13:54
esbergluPowerKVM has one, I talked to them a little bit about what they had to do13:54
edmondswk13:55
edmondswsounds cool, but not critical13:55
efriedagree13:55
esbergluedmondsw: efried: The idea of it is that we can get rid of failure emails13:55
esbergluAnd just monitor the dashboard13:55
edmondswsure13:55
edmondswesberglu want to talk about zuulv313:56
efriedWe would still get emails for changes we're subscribed to, right?13:56
esbergluefried: Yeah that's all external to us13:56
efriedCause the likelihood of me actively and continuously "monitoring" the dashboard is nil.13:56
efriedBut I still need to know if a change I care about fails.13:56
esbergluefried: Yeah you would still get that I believe13:57
esbergluBut I would make sure before getting rid of the failures13:57
esbergluedmondsw: Did you hear much about it at PTG?13:57
esbergluI didn't see a whole lot on the mailing list13:57
edmondswesberglu not much, just that they were trying to switch over, hit an issue early in the week, and were going to come back to is13:57
edmondswit*13:58
edmondswlater in the week13:58
esbergluedmondsw: Yeah that was my understanding too13:58
edmondswI should probably say "keep working on it" rather than "come back to it"13:58
esbergluWe aren't necessarily in a hurry to do that13:58
esbergluIt would just be nice13:58
edmondswand my understanding is that their switch doesn't really effect us, just would be nice if/when we can follow suit13:58
edmondswlet them work out the kinks first, though13:58
esbergluedmondsw: Yeah I'll just try to stay in the loop on it13:59
edmondsw++13:59
edmondswthe other thing I was going to ask about is neutron/ceilometer ci13:59
esbergluedmondsw: You saw my latest email on that?13:59
edmondswyeah, your email said that we caught that ceilometer issue in tox rather than in CI... which surprised me14:00
esbergluWe would need to talk to both neutron and ceilometer separately if we want to publish14:00
edmondswesberglu yeah, and I think that would probably be a good discussion to have... let's add that to the TODO list14:00
edmondswnon-voting, of course14:01
esbergluYep14:01
esbergluedmondsw: I also will add a todo to look into what we are testing for ceilometer14:01
edmondswtx14:01
edmondswdone with CI?14:01
esbergluYep14:01
edmondsw#topic Driver Testing14:02
edmondswI just have one quick thing here14:02
edmondswbasically, we're just going to have UT and CI at this point14:02
edmondswI thought we might be able to get Ravi to do some tempest work, but we lost him as well as Jay14:03
edmondswso I will remove this from the agenda going forward14:03
edmondsw:(14:03
edmondswof course OOT driver is still used/tested by PowerVC14:03
edmondswand I'll be asking for more resources to help here in the next cycle14:04
esbergluokay14:04
edmondswbut that will probably be dev resources, not test... so get ready to be asked to do more in tempest dev and usage14:04
edmondswthat is all14:04
edmondsw#topic Open Discussion14:05
efriededmondsw Following up from PTG, can you please review this series: https://review.openstack.org/#/q/topic:bp/nova-validate-certificates+status:open14:05
esbergluAny last thoughts? Nothing else from me14:05
edmondswefried yep, on my list14:05
efriedThanks.14:05
esberglu#endmeeting14:06
openstackMeeting ended Tue Sep 19 14:06:44 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:06
openstackMinutes:        http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-09-19-13.00.html14:06
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-09-19-13.00.txt14:06
openstackLog:            http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-09-19-13.00.log.html14:06
*** esberglu has quit IRC14:06
*** esberglu has joined #openstack-powervm14:07
*** esberglu has quit IRC14:11
*** esberglu has joined #openstack-powervm14:20
*** esberglu has quit IRC14:39
*** esberglu has joined #openstack-powervm14:40
*** esberglu has quit IRC14:41
*** esberglu has joined #openstack-powervm14:41
esbergluefried: edmondsw: I think I'm actually gonna go OVS first. That patch is a lot farther along than I remembered it being15:37
efriedesberglu "first" before what?15:37
esbergluSEA15:37
efriedesberglu That's fine by me, except that it brings forward the need to have OVS in the CI.15:47
efriedI'm not aware of what all the challenges are there.15:48
openstackgerritMatt Rabe proposed openstack/nova-powervm master: Add PPT ratio value to supported flavor fields  https://review.openstack.org/49797715:48
efriedthorst Care to comment on whether SEA or OVS would be more "strategic" to have in tree first?15:48
mdrabe^^ edmondsw efried15:49
efriedmdrabe +2.  Will +W when CI passes.15:49
esbergluefried: If we do SEA first I will be redoing a lot of work that has already been done in the OVS patch15:50
efriedBTW, since the gerrit upgrade, it seems folks aren't getting emails anymore.  I know I'm not, and several folks have complained to the ML.  So until fixed, be vigilant about checking your dashboard manually.15:50
esbergluAny code that is shared by both SEA and OVS is already in the OVS patch15:50
efriedesberglu Oh, the common code paths?15:50
edmondswefried I believe in past conversations we've not really cared whether SEA or OVS merged first15:50
efriedright.15:51
efriedSfine by me.  If there's no push from thorst or edmondsw to do one or the other first, I guess it's up to you whether it's more work to shift the common code to a SEA change set versus figuring out OVS in the CI.15:51
efriedThe latter has to be done eventually anyway, whereas the former would be throwaway work.15:52
efriedSo yeah.  OVS.15:52
efriedGlad we talked that through.15:52
edmondswwell, we may or may not ever do OVS in the CI, right?15:52
edmondswwe'd like to look at that... but it's not a requirement to do OVS15:52
efriedYeah, fair15:53
efriedWe don't test localdisk driver in the CI either.15:53
efriedor NPIV15:53
edmondswright15:54
edmondswso esberglu I'm fine with you focusing on OVS first, and pretty confident thorst would say the same15:55
esbergluedmondsw: efried: Cool16:07
efriedbtw, I can't resist:16:21
efriedGetting networking in tree will truly represent a sea change for powervm.16:21
efriedRegardless of whether it's a SEA change.16:21
thorstsea *of* change?16:22
efriedhttps://en.wikipedia.org/wiki/Sea_change_(idiom)16:22
efried(NOW he responds)16:22
thorstheh, I haven't read all back and about to jump to yet another call16:29
thorst:-(16:29
*** chhavi has quit IRC17:58
openstackgerritEric Berglund proposed openstack/nova-powervm master: WIP: Add context to detach_interface params  https://review.openstack.org/50537419:06
esbergluefried: edmondsw: I noticed this error in n-cpu logs recently.19:27
esbergluhttp://paste.openstack.org/show/621471/19:27
esbergluIt's getting called from https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L691319:27
esbergluAnd is hitting our code here19:27
esbergluhttps://github.com/openstack/nova-powervm/blob/master/nova_powervm/virt/powervm/driver.py#L97919:27
esbergluDigging deeper I'm seeing the error on passing runs too19:28
esbergluBut I'm curious if this is what is underlying the network related tempest failures19:28
esbergluBut the change adding context to the detach_interface was put in forever ago19:29
esbergluhttps://github.com/openstack/nova/commit/383e795a19:29
esbergluSo I think we have a hole in our UT as well19:30
edmondswwow, that is a long time... how could we have missed that?19:30
esbergluedmondsw: I hadn't noticed it in any CI logs until recently19:30
esbergluBut it should have been throwing errors since it went in19:31
efriedesberglu You gonna propose the fix?19:33
esbergluefried: Yeah19:33
efriedcoo19:33
-openstackstatus- NOTICE: Zuul and Gerrit are being restarted to address issues discovered with the Gerrit 2.13 upgrade. review.openstack.org will be inaccessible for a few minutes while we make these changes. Currently running jobs will be restarted for you once Zuul and Gerrit are running again.20:27
*** esberglu has quit IRC21:00
*** esberglu has joined #openstack-powervm21:01
*** esberglu has quit IRC21:05
*** esberglu has joined #openstack-powervm21:12
*** thorst has quit IRC21:14
*** thorst has joined #openstack-powervm21:16
*** thorst has quit IRC21:20
openstackgerritEric Berglund proposed openstack/nova-powervm master: Add context to detach_interface params  https://review.openstack.org/50537421:38
openstackgerritMerged openstack/nova-powervm master: Add PPT ratio value to supported flavor fields  https://review.openstack.org/49797721:39
openstackgerritEric Berglund proposed openstack/nova-powervm master: Add context to detach_interface params  https://review.openstack.org/50537421:42
*** kylek3h has quit IRC21:54
*** esberglu has quit IRC21:57
-openstackstatus- NOTICE: Gerrit is being restarted to address some final issues, review.openstack.org will be inaccessible for a few minutes while we restart22:33
*** openstackgerrit has quit IRC22:47
*** edmondsw has quit IRC22:50
*** edmondsw has joined #openstack-powervm22:55
*** edmondsw has quit IRC22:59
*** edmondsw has joined #openstack-powervm23:23
*** edmondsw has quit IRC23:27
*** ChanServ changes topic to "This channel is for PowerVM-related development and discussion. For general OpenStack support, please use #openstack."23:40
-openstackstatus- NOTICE: Gerrit is once again part of normal puppet config management. Problems with Gerrit gitweb links and Zuul post jobs have been addressed. We currently cannot create new gerrit projects (fixes in progress) and email sending is slow (being debugged).23:40
*** chhavi has joined #openstack-powervm23:54
*** chhavi has quit IRC23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!