Thursday, 2017-06-01

*** esberglu has quit IRC01:13
*** esberglu has joined #openstack-powervm01:14
*** thorst has quit IRC01:17
*** esberglu has quit IRC01:19
*** esberglu has joined #openstack-powervm01:25
*** esberglu has quit IRC01:26
*** thorst has joined #openstack-powervm01:33
*** thorst has quit IRC01:33
*** chas has joined #openstack-powervm01:40
*** jwcroppe has joined #openstack-powervm01:41
*** chas has quit IRC01:45
*** lan has joined #openstack-powervm01:52
*** thorst has joined #openstack-powervm01:53
*** thorst has quit IRC01:53
*** dwayne has quit IRC01:55
*** lan1 has joined #openstack-powervm02:00
*** lan has quit IRC02:01
*** lan2 has joined #openstack-powervm02:03
*** lan1 has quit IRC02:06
*** thorst has joined #openstack-powervm02:09
*** thorst has quit IRC02:10
*** edmondsw has joined #openstack-powervm02:11
*** edmondsw has quit IRC02:16
*** thorst has joined #openstack-powervm02:34
*** thorst has quit IRC02:38
*** thorst has joined #openstack-powervm03:38
*** chas has joined #openstack-powervm03:41
*** chas has quit IRC03:46
*** lan2 has quit IRC03:49
*** thorst has quit IRC03:57
*** chhavi has joined #openstack-powervm04:07
*** thorst has joined #openstack-powervm04:55
*** thorst has quit IRC04:59
*** chas has joined #openstack-powervm05:42
*** arunman has joined #openstack-powervm05:43
*** edmondsw has joined #openstack-powervm05:47
*** chas has quit IRC05:49
*** edmondsw has quit IRC05:52
*** thorst has joined #openstack-powervm05:56
*** thorst has quit IRC06:00
*** chas has joined #openstack-powervm06:07
*** tjakobs has joined #openstack-powervm06:38
*** chas has quit IRC06:55
*** thorst has joined #openstack-powervm06:56
*** tjakobs has quit IRC06:59
*** thorst has quit IRC07:01
*** k0da has joined #openstack-powervm07:45
*** thorst has joined #openstack-powervm07:58
*** k0da has quit IRC08:09
*** thorst has quit IRC08:16
*** chas has joined #openstack-powervm08:46
*** chas has quit IRC08:51
*** chas has joined #openstack-powervm09:07
*** chas has quit IRC09:12
*** thorst has joined #openstack-powervm09:13
*** thorst has quit IRC09:17
*** edmondsw has joined #openstack-powervm09:23
*** edmondsw has quit IRC09:27
*** chas has joined #openstack-powervm09:28
*** chas has quit IRC09:32
*** openstackstatus has quit IRC09:36
*** openstackstatus has joined #openstack-powervm09:38
*** ChanServ sets mode: +v openstackstatus09:38
-openstackstatus- NOTICE: There is a known issue with setuptools 36.0.0 and errors about the "six" package. For current details see https://github.com/pypa/setuptools/issues/1042 and monitor #openstack-infra09:45
*** chas has joined #openstack-powervm09:49
*** chas has quit IRC09:53
*** chas has joined #openstack-powervm10:10
*** chas has quit IRC10:14
*** chas has joined #openstack-powervm10:21
*** chas has quit IRC10:26
*** smatzek has joined #openstack-powervm10:58
*** smatzek_ has joined #openstack-powervm10:59
*** smatzek has quit IRC11:03
*** k0da has joined #openstack-powervm11:10
*** thorst has joined #openstack-powervm11:15
*** thorst has quit IRC11:19
*** thorst has joined #openstack-powervm11:41
chhaviefried: sonar is complaining for the max argument issue11:46
*** arunman has quit IRC11:51
*** svenkat has joined #openstack-powervm12:00
*** chas has joined #openstack-powervm12:22
*** edmondsw has joined #openstack-powervm12:26
*** chas has quit IRC12:26
*** kylek3h has quit IRC12:32
*** kylek3h has joined #openstack-powervm12:58
*** arunman has joined #openstack-powervm12:59
*** AndyWojo has joined #openstack-powervm12:59
*** mdrabe has joined #openstack-powervm13:08
edmondswefried thinking about how we organize this etherpad as it continues to grow over time... how do you feel about adding "Completed in Pike", etc. sections at the bottom and moving things there when we complete them13:14
edmondswso they don't continue to clutter up the higher sections13:14
edmondswand could serve as a good reminder of what we've done as well, I guess13:14
*** jpasqualetto has joined #openstack-powervm13:20
*** smatzek_ has quit IRC13:31
chhaviedmondsw: how about separate etherpad per release and just maintaining the link in the parent one, the way its been used in PTG per discussion13:32
*** esberglu has joined #openstack-powervm13:33
edmondswchhavi that could work, but it may get tricky if we're not sure which release something is going to be done in... wouldn't know which etherpad to put it in13:36
chhaviYeah, i was just visualizing as a single etherpad, which has links: "Things to Do", "Items completed", "Issues"13:42
chhavisomething like this, its just a thought.13:42
*** jwcroppe has quit IRC13:48
*** smatzek_ has joined #openstack-powervm13:58
*** kriskend has joined #openstack-powervm13:58
*** jwcroppe has joined #openstack-powervm14:01
efriededmondsw Yeah, I'm good with moving completed items to the bottom; or just dropping them off after some time.14:01
efriedWould rather avoid having multiple etherpads, I think.  We should have a small enough set of to-dos that we can keep it to one.14:03
esbergluedmondsw: efried: thorst: Still working on CI stability. Updated the novalink packages on all of the hosts and was seeing some green last night before something broke the world14:05
esbergluhttp://ci-watch.tintri.com/project?project=nova14:05
efriedyikes.14:07
efriedInterestingly, UT for py27 is passing, but py35 isn't.14:07
esbergluefried: It was causing devstack to fail like 4 minutes in all of last night. Looks like whatever the issue was may have been resolved already. I'm seeing runs complete the stack now14:10
efriedokay.14:10
efriedDiscussing it in #openstack-meeting right now.14:11
efried"the setuptools issue" - getting link14:11
efriedesberglu https://github.com/pypa/setuptools/issues/104214:12
esbergluStupid setuptool14:13
esbergluSeems like there have been a lot of issues with it that break devstack14:14
edmondswesberglu maybe they're learning... https://github.com/pypa/setuptools/issues/104514:32
edmondswor at least people are complaining :)14:33
-openstackstatus- NOTICE: python-setuptools 36.0.1 has been released and now making its way into jobs. Feel free to 'recheck' your failures. If you have any problems, please join #openstack-infra14:34
*** openstackgerrit has quit IRC14:34
*** tjakobs has joined #openstack-powervm14:37
*** tjakobs has quit IRC14:43
*** chas has joined #openstack-powervm14:53
*** k0da has quit IRC14:56
*** chas has quit IRC14:58
esbergluthorst: Dang we had a successful run go through on https://review.openstack.org/#/c/469291/15:06
esbergluBut another recheck got posted15:06
esbergluHopefully that one is successful as well15:06
thorstheh15:06
thorstwell, we'll let Esha's go through I guess...15:06
thorstodd that Esha's check didn't cancel out yours?15:06
esbergluActually that's right15:07
esberglurecheck's don't cancel out other rechecks15:07
esbergluBut a new patch coming in will15:07
thorsto...I thought thye did15:07
thorstok15:07
thorstTIL  :-)15:07
esbergluSo her's didn't actually do anything15:07
thorstso it no-ops?15:07
thorstor it adds to the queue?15:07
esbergluno-ops15:07
thorstneat15:08
esbergluYep just checked the queue to confirm. Not there15:08
thorstefried: you good if I w+1 it?15:08
efriedthorst Yes.  esberglu does our CI trigger without the colon?15:09
esbergluYeah it actually triggers on a bunch of stuff15:09
esberglupowervm:recheck (with or without spaces in betwee)15:09
esberglupowervm recheck (with any whitespace)15:09
esberglupowervm-recheck15:10
efriedk, wasn't sure.15:10
efriedthorst's powervmci:recheck was ignored ;-)15:10
esbergluOn which patch? That shouldn't be the case15:11
*** tjakobs has joined #openstack-powervm15:12
efriedesberglu On https://review.openstack.org/#/c/469291/ (PS2, yesterday morning)15:14
efriedesberglu I posted my recheck that afternoon because nothing had happened and somebody asked me why.15:15
efriedesberglu So next question: I seem to have posted my recheck on PS1 - does that actually run the CI on PS1, or does it always choose latest?15:15
esbergluNah the results are posted at 8:28 from his check15:16
esbergluOh wait I was looking at the wrong recheck. Idk why it didn't get picked up15:16
*** openstackgerrit has joined #openstack-powervm15:16
openstackgerritMerged openstack/networking-powervm master: VIOS availability check during agent startup  https://review.openstack.org/46929115:16
efriedesberglu Because he spelled it powervm*ci*:recheck15:18
esberglulol yeah that would do it15:18
efriededmondsw thorst FYI, esberglu just patched drmgr on the CI nvls in a bid to fix the vterm issues we've been having.  So keep an eye out for CI failures caused by slow/timing-out vterm close.15:19
thorstwas that apearson's update?15:20
esbergluyeah15:20
efriedThe drmgr patch was provided by apearson; word is that the fix is proposed to be backported, but not yet there.15:20
edmondswthorst esberglu can one of you add me to the list of those notified when the CI fails?15:23
thorstesberglu: edmondsw is asking to be added to the powervmci failure e-mail list  :-)15:24
thorstedmondsw: brace yourself....15:24
* edmondsw ducks15:24
*** dwayne has joined #openstack-powervm15:27
esbergluedmondsw: I would highly recommend having a filter set up15:30
esbergluefried: Regarding your earlier question. Looks like the recheck on PS1 runs using PS1 code15:31
esbergluNot the latest15:32
efriedesberglu While that seems logical on some level, I wonder if it's really what we want.15:32
esbergluI don't think we have any control over that unless we want to patch zuul15:33
*** openstackgerrit has quit IRC15:33
efriedI guess it's useful if we want to see if an old patch set passes where a new one failed.  That could also be done by reproposing the old patch - but that's messier.15:33
efriedOkay, if it's hard to change, so be it.15:33
efriedthorst edmondsw Can you review my review on 5267?15:35
efriedthorst edmondsw Mainly looking for consensus on whether we should split it up.  More I think about it, more I think that's appropriate.15:36
edmondswefried will pull it up15:36
thorstefried: on it.15:38
efriededmondsw thorst But I'm way happier wrt the footprint of the change, especially the fact that the method sig change is backward compatible so we don't need to dork with pypowervm reqs.15:39
thorstefried: I dunno...feels like little value in pulling that out15:39
thorstas long as chhavi updates the commit message, I'm fine leaving it in personally15:40
*** mdrabe has quit IRC15:41
efriedthorst Okay.  My thought was to isolate the part that changes a method signature.  An argument can be made here that both things are changing the API.  But the two changes really are unrelated.15:41
efriedIt's a red flag when the commit message has a conjunction ("ISCSI: Support LUN ID *and* handle error conditions")15:42
*** chas has joined #openstack-powervm15:43
thorstyeah, that to me could be 'iSCSI robustness updates'15:43
edmondswefried the lun portion of this is just about 5 lines, right?15:46
efriededmondsw Yeah.15:46
efriedWell15:46
efriedWith the suggested additional tests, it would be a bit more.15:46
efriedJust thinking for future debuggability/maintainability, it would be nice to be able to point to a single change that alters the method sig.15:47
efriedLike, if that breaks something, we would be able to revert just that bit with one click, rather than having to unwind it manually.15:47
efriedkind of thing.15:47
edmondswefried sure, tests would be more lines15:47
edmondswI think I'd prefer to split that out into a separate change unless chhavi has a good reason it needs to be in this one that I'm missing15:48
edmondswseems like it should be easy to split that out when she's doing this other cleanup15:48
efriedagree the mechanics aren't tough15:49
efriedWe've probably spent more time discussing it than it would have taken to do it ;-)15:49
esbergluedmondsw: Just added you to the mailing list15:51
edmondswesberglu thank you? ;)15:51
efriedI haven't been seeing a whole lot of traffic on that list15:52
efriedesberglu Am I on it?15:52
efriedI would have thought so.  Didn't I create it?15:52
esbergluYeah you did and you are on it15:53
esbergluAre you not getting them? There has been TONS of traffic the last week15:54
efriedesberglu Example subject line?15:54
esbergluCheck of openstack/nova 403660,42 failed15:54
chhaviefried: just saw the review comments, I can update the commit message and Doc Strings.16:03
chhaviThere is one test case for ISCSIDiscoveryFailed, and handling with lunid as well16:04
efriedesberglu Aha, got those going into a folder I never look at.  7,048 unread messages in there.16:05
efried0 messages in there.16:05
esbergluYeah you definitely would know if they were going to your main inbox ;)16:05
efriedchhavi Thanks - but also see discussion on splitting this into two changes.16:06
edmondswchhavi I think efried was trying to say that you test for ISCSIDiscoveryFailed on discover_iscsi, but not for discover_initiator16:06
efriedI don't know if it's a democracy, but if it is, thorst is outvoted ;-)16:07
edmondswand you test the case that lunid is specified, but not the case that it isn't16:07
efriedchhavi Yes, what edmondsw said is correct.16:07
chhaviefried: I think we can have a single change set, and go with updating the change set.16:09
edmondswchhavi why don't you want to split it? shouldn't be difficult16:10
chhavinot with respect to diffiulty, but all this got discovered and build as a part of single issue.16:13
*** chas has quit IRC16:18
chhaviWe should not be seing any backward compatibility with method signature, considering its constant, and also its not a mandatory job paramter.16:18
*** mdrabe has joined #openstack-powervm16:18
chhaviif we all still feel that having separate will be a good option, i am ok to spend time in doing so.16:19
esbergluthorst: There aren't any more ocata changes that CI is blocking correct?16:19
esbergluhttps://review.openstack.org/#/c/468515/16:19
thorstdon't think so16:20
esberglu^ that's already included in the build?16:20
thorstjust svenkat's and erlarese's16:20
thorstand they've gotten their fix into PVC through other means, but a backport removes the tech debt16:20
thorstso we still need to get it back there.16:20
esbergluYep. The stable runs have been looking really good today so that last one should go through with the latest recheck16:21
thorstrockin16:23
thorst:-)16:23
efriedthorst Note that there's some question whether pvc will be respinning or absorbing the tech debt regardless.  Talk to svenkat if there's more stuff in the pipe that they care about.16:23
thorstI can.  The neat thing about PVC though is as soon as we merge it, it just disappears from their tech debt16:23
thorstI think due to how they cherry pick16:23
svenkatefried : there are no other changes needed at this timeā€¦16:23
chhaviefried, thorst, edmondsw: So whats the vote, shall we split or update commit msg.  based on that i can update the review.16:25
edmondswefried and I would prefer it to be split unless thorst wants to overrule that16:26
chhavilets split it then16:27
chhaviit will take less time then taking everyone votes :)16:27
efried+116:32
thorstsorry - sounds good.16:44
chhaviefried: got the first change set for the Return code17:07
efriedchhavi Reviewed.17:11
efriedthorst edmondsw 5267 is now just the return code handling.17:12
*** chas has joined #openstack-powervm17:18
chhaviefried: You have to review once again until i get +2 :)17:19
efriedchhavi Want to make sure thorst and edmondsw are happy first.17:19
thorstlooking17:19
thorstwe plan to merge these basically together...right?17:20
*** AndyWojo has quit IRC17:20
chhavionce this is merged i will post the lun id changes17:21
thorstok17:21
chhaviso that there will be no conflicts17:21
thorstI'm good with PS3017:21
thorstwaiting for Jenkins to come back happy17:21
efriedthorst https://github.com/openstack/nova-powervm/blob/master/nova_powervm/virt/powervm/event.py#L6317:30
thorstyes...17:30
efriedthorst It appears as though we would handle an NVRAM event even if the path was SomethingOtherThanLogicalPartiton/{pvm_uuid]17:31
thorstIs there anything else that ends in LogicalPartition?17:31
thorstare you saying it should be /LogicalPartition/ ?17:31
efriedHold on17:31
efriedNo, that's not what I was saying.  But I think I've gorped up my sandbox and confused myself.17:32
efriedIgnore me for now, will come back in a bit if I'm still confused.17:32
thorstcool17:32
chhavibuild done17:38
edmondswchhavi couple spacing issues, but otherwise LGTM17:39
chhavican't take anymore17:40
edmondswchhavi sorry, this should be it :(17:40
chhavi:)17:41
edmondswis trove not like this?17:41
chhavinever17:41
edmondswefried esberglu is there a reason we don't have a .gitreview file in pypowervm or powervm-ci, or is that just an oversight?17:42
efriededmondsw Because we don't want to publish the hostname of our internal gerrit repo.17:42
chhaviwe focus more on code then on syntax, basic coding guidelines should be followed.17:42
chhavibetter to have these spacing rules included in pep817:44
edmondswefried ok, duh... I was thinking those were public but I guess not totally17:44
edmondswchhavi yeah, I agree that they should have been caught in pep817:44
efriededmondsw Not at all public.  Agree it's a minor pain, but c'est la vie.17:44
edmondswefried source is public on github... but of course that doesn't mean dev has to be public17:45
efriededmondsw Right, the gerrit repo is private.17:46
*** openstackgerrit has joined #openstack-powervm17:54
openstackgerritOpenStack Proposal Bot proposed openstack/networking-powervm master: Updated from global requirements  https://review.openstack.org/46557217:54
*** arunmani_ has joined #openstack-powervm17:56
edmondswchhavi... you put those first 2 blank lines in the wrong place17:57
*** arunman has quit IRC17:58
*** arunmani_ has quit IRC18:05
edmondswefried I just added my +2 on 5267, please look before chhavi decides to strangle one of us18:08
efriededmondsw chhavi +218:08
*** arunmani_ has joined #openstack-powervm18:11
chhaviThanks all of you for doing so many levels of reviews, its tough time :)18:11
edmondswchhavi thanks for not strangling us ;)18:20
chhavibut i think if we can add some more rules in pep8 checks for blanklines and spacing it will help18:21
edmondswchhavi agreed... will add that to the TODO list :)18:22
esbergluthorst: You have a chance to give the remote console for CI any more thought?18:35
edmondswefried should I open a LP bug for the "deactivate compute service if pvm-rest is dead", or would we just fix that with no bug number?18:36
efriededmondsw I don't think we need a bug.  How do you feel, thorst ?18:36
efriedNot something we're going to backport, so we don't *need* it.  Beyond that, is it actually a bug?  I say nah.18:37
*** arunmani_ has quit IRC18:42
*** chas has quit IRC18:55
*** chas has joined #openstack-powervm18:55
*** chhavi has quit IRC18:56
*** k0da has joined #openstack-powervm19:01
*** chas has quit IRC19:03
openstackgerritEric Fried proposed openstack/nova-powervm master: WIP: Delay get_instance in event processing  https://review.openstack.org/46998219:20
edmondswefried should we also deactivate the compute service when we can't find an active VIOS?19:33
edmondswcurrently raises an exception from init_host in that case19:33
efriededmondsw Yeah, I think we should dig into (and possibly change) that validate_vios_ready business.19:34
edmondswyep19:34
efriedRight now I think it spins for quite a while waiting for the VIOS to be ready.19:34
edmondswyes19:34
efriedWe should instead include vios readiness in the criteria for marking the compute host up/down.19:34
efriedSo whenever our periodic task hits, compute_host_state = up if (pvm-rest && vios ready)19:35
efriedBut hold on.19:35
efriedthorst should weigh in here.19:35
efriedCause there's *some* stuff we can still do without VIOSes.19:35
efriedLike report LPAR states.19:35
efriedcertain resizes, probably...19:36
efriedpower on/off/reboot19:36
efriedWhat we can't do is schedule new instances, delete existing, map volumes, or (if SEA/SR-IOV) change networking.19:37
edmondswefried even if we want it to be up when there's no VIOS, we need changes19:37
efriedYeah19:37
efriedI don't fully know what it means for a compute host to be down.19:37
edmondswthorst, when you have a minute to discuss this...19:37
efriedDoes the conductor just not even try to ask the compute anything?19:38
edmondswidk, but it at least won't schedule new instances19:39
edmondswefried and we can get rid of conn_tries=60 entirely and let it use the default of 1, right?19:43
edmondswon the Session create19:44
edmondswor is there a reason to still try a handful of times?19:44
efriededmondsw That would be nice, but I think a little more investigation is needed as to how often we'll get to check.19:44
efriedIf we only get hit once every five minutes, that means it'll almost always be five minutes after compute startup before we're alive.19:45
efried(on a reboot)19:45
efriedIf we get hit once every 30s or something, not so bad.19:45
edmondswefried can you walk me through how you setup an env to try things19:53
efriededmondsw Okay.  You got a victim?19:53
edmondswvictim?19:53
efrieda neo you can use19:54
edmondswno. I've got one I can use with a proxy, but not directly, and if we're going to be restarting things...19:54
edmondswthat's part of my question :)19:54
efriedthorst How are we on hardware?19:55
efriedWe need to get edmondsw a neo one way or another.19:56
*** smatzek_ has quit IRC20:06
edmondswefried if we use get_available_resource as you suggested, that gets called every 60s by default20:16
edmondswit's configurable with CONF.update_resources_interval20:17
*** jwcroppe has quit IRC20:24
thorstedmondsw: I can get you a system tomorrow?20:31
edmondswthorst that's fine20:31
edmondswthorst can you get me wiki access today?20:31
thorstPM me the wiki link?20:31
edmondswit's in your slack20:31
thorstand for vios readiness...I don't think the host is ready unless you have VIOS ready20:32
thorstso I like that idea.20:32
*** svenkat has quit IRC20:33
*** edmondsw has quit IRC20:52
*** thorst has quit IRC20:57
*** kriskend has quit IRC20:59
*** thorst has joined #openstack-powervm21:02
*** chas has joined #openstack-powervm21:03
*** jwcroppe has joined #openstack-powervm21:04
*** jwcroppe has quit IRC21:04
*** thorst has quit IRC21:07
*** chas has quit IRC21:09
openstackgerritEric Fried proposed openstack/nova-powervm master: WIP: Delay get_instance in event processing  https://review.openstack.org/46998221:17
*** thorst has joined #openstack-powervm21:20
*** thorst has quit IRC21:24
*** jpasqualetto has quit IRC21:24
*** edmondsw has joined #openstack-powervm21:32
*** edmondsw has quit IRC21:37
openstackgerritEric Fried proposed openstack/nova-powervm master: WIP: Delay get_instance in event processing  https://review.openstack.org/46998221:42
*** k0da has quit IRC22:00
*** kylek3h has quit IRC22:06
*** mdrabe has quit IRC22:16
*** tjakobs has quit IRC22:38
*** svenkat has joined #openstack-powervm23:00
*** jwcroppe has joined #openstack-powervm23:48

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!