Tuesday, 2016-07-26

« Monday, 2016-07-25 Index Wednesday, 2016-07-27 »
*** svenkat has joined #openstack-powervm00:25
*** thorst has quit IRC00:36
*** thorst has joined #openstack-powervm00:37
*** arnoldje has quit IRC00:46
*** thorst has quit IRC00:46
*** jwcroppe has joined #openstack-powervm01:04
*** thorst has joined #openstack-powervm01:10
*** thorst has quit IRC01:12
*** edmondsw has quit IRC02:05
*** catintheroof has quit IRC02:06
*** thorst has joined #openstack-powervm02:07
*** thorst has quit IRC02:07
*** thorst has joined #openstack-powervm02:07
*** thorst has quit IRC02:12
*** arnoldje has joined #openstack-powervm02:41
*** esberglu has quit IRC02:49
*** kotra03 has joined #openstack-powervm03:23
*** svenkat has quit IRC03:36
*** Ashana has joined #openstack-powervm04:00
*** Ashana has quit IRC04:05
*** Ashana has joined #openstack-powervm04:06
*** Ashana has quit IRC04:11
*** Ashana has joined #openstack-powervm04:12
*** Ashana has quit IRC04:16
*** Ashana has joined #openstack-powervm04:18
*** Ashana has quit IRC04:22
*** Ashana has joined #openstack-powervm04:30
*** Ashana has quit IRC04:34
*** Ashana has joined #openstack-powervm04:35
*** Ashana has quit IRC04:40
*** Ashana has joined #openstack-powervm04:41
*** Ashana has quit IRC04:46
*** Ashana has joined #openstack-powervm04:47
*** Ashana has quit IRC04:52
*** Ashana has joined #openstack-powervm04:53
*** Ashana has quit IRC04:57
*** Ashana has joined #openstack-powervm04:59
*** Ashana_ has joined #openstack-powervm05:01
*** Ashana has quit IRC05:03
*** Ashana_ has quit IRC05:05
*** Ashana has joined #openstack-powervm05:13
*** Ashana has quit IRC05:17
*** Ashana has joined #openstack-powervm05:19
*** arnoldje has quit IRC05:22
*** Ashana has quit IRC05:23
*** Ashana has joined #openstack-powervm05:24
*** Ashana has quit IRC05:29
*** Ashana has joined #openstack-powervm05:30
*** Ashana has quit IRC05:35
*** Ashana has joined #openstack-powervm05:36
*** Ashana has quit IRC05:41
*** Ashana has joined #openstack-powervm05:42
*** Ashana has quit IRC05:46
*** Ashana has joined #openstack-powervm05:48
*** Ashana has quit IRC05:52
*** Ashana has joined #openstack-powervm05:54
*** Ashana_ has joined #openstack-powervm05:56
*** Ashana has quit IRC05:58
*** Ashana_ has quit IRC06:00
*** Ashana has joined #openstack-powervm06:02
*** Ashana has quit IRC06:07
*** Ashana has joined #openstack-powervm06:08
*** jwcroppe has quit IRC06:09
*** Ashana has quit IRC06:12
*** Ashana has joined #openstack-powervm06:13
*** Ashana has quit IRC06:18
*** Ashana has joined #openstack-powervm06:19
*** Ashana has quit IRC06:24
*** Ashana has joined #openstack-powervm06:25
*** Ashana has quit IRC06:30
*** Ashana has joined #openstack-powervm06:31
*** Ashana has quit IRC06:35
*** Ashana has joined #openstack-powervm06:37
*** Ashana has quit IRC06:41
*** Ashana has joined #openstack-powervm06:43
*** Ashana has quit IRC06:47
*** Ashana has joined #openstack-powervm06:49
*** Ashana has quit IRC06:53
*** Ashana has joined #openstack-powervm06:54
*** kotra03_ has joined #openstack-powervm06:57
*** kotra03 has quit IRC06:58
*** kotra03_ is now known as kotra0306:58
*** Ashana has quit IRC06:59
*** Ashana has joined #openstack-powervm07:00
*** Ashana has quit IRC07:05
*** Ashana has joined #openstack-powervm07:06
*** Ashana has quit IRC07:11
*** Ashana has joined #openstack-powervm07:12
*** Ashana has quit IRC07:16
*** Ashana has joined #openstack-powervm07:18
*** Ashana has quit IRC07:22
*** k0da has joined #openstack-powervm08:02
*** k0da has quit IRC09:01
*** k0da has joined #openstack-powervm09:05
*** thorst has joined #openstack-powervm11:11
*** svenkat has joined #openstack-powervm11:46
thorstsvenkat: https://review.openstack.org/#/c/341125/11:54
thorstyou'll need to give a +2 (although I also want the +1 from efried)11:54
thorstI can't +2 as I contributed to it11:54
svenkatthorst. sure. one moment11:56
svenkatdone.12:03
*** Ashana has joined #openstack-powervm12:05
*** wangqwsh has joined #openstack-powervm12:05
*** edmondsw has joined #openstack-powervm12:17
openstackgerritDrew Thorstensen proposed openstack/networking-powervm: Support VIOS CNAs  https://review.openstack.org/34437912:23
thorstsvenkat: ^^ as well12:23
Ashana@wangqwsh so for the work items that I have proposed changes for is Support pulling architecture-tagged venv artifacts12:29
thorstwangqwsh Ashana: Are we asking what's left for the OSA PowerVM work?12:31
*** mdrabe has joined #openstack-powervm12:31
wangqwshthorst: yes. if some items' done, I could use the code directly.12:36
wangqwshashana: could you share the review link with me?12:36
*** catintheroof has joined #openstack-powervm12:46
Ashanayes https://review.openstack.org/#/c/345986/12:55
AshanaThis is Adam's reviews https://review.openstack.org/#/c/346863/ and https://review.openstack.org/#/c/346863/12:56
*** seroyer has joined #openstack-powervm12:57
Ashanawangqwsh: ^^12:58
*** tblakeslee has joined #openstack-powervm13:01
thorstwangqwsh: All of the various reviews need to be done together.  What I think adreznec was thinking we should do next is a full run through.  That means I'll need to reconfigure neo4 to have the physical I/O.  It unfortunately means that Ashana I'll have to shut down .23213:03
thorstbut then we could do a full run through where our control plane is x86 and we have a PowerVM compute node.13:03
*** tblakeslee has quit IRC13:03
*** tblakeslee has joined #openstack-powervm13:04
wangqwshashana thorst: ok, thanks13:11
*** seroyer has quit IRC13:19
thorstAshana wangqwsh: so I think the next step is getting neo4 'ready'.  Are you both OK doing that?13:23
thorstadreznec: is there a specific amount of disk I need for the repo build that will go on neo4?13:23
Ashanayep that is fine13:28
*** Ashana has quit IRC13:28
*** Ashana has joined #openstack-powervm13:28
*** k0da has quit IRC13:30
*** k0da has joined #openstack-powervm13:34
*** seroyer has joined #openstack-powervm13:34
*** apearson has joined #openstack-powervm13:34
wangqwshthorst: yes,13:37
thorstAshana wangqwsh: I'll work on that today.13:37
thorstthen tomorrow we can try to get everything all patched into these environments13:37
thorstand rip through a full blown PowerVM w/ x86 controller deploy?13:37
*** seroyer has quit IRC13:43
*** lmtaylor has joined #openstack-powervm13:44
thorstAshana: are all the necessary patches applied to your x86 controller?13:46
*** esberglu has joined #openstack-powervm13:46
openstackgerritDrew Thorstensen proposed openstack/networking-powervm: Support VIOS CNAs  https://review.openstack.org/34437913:47
*** jwcroppe has joined #openstack-powervm13:48
Ashanathe patches that Adam proposed? how do I add them to the controller?13:51
*** efried has joined #openstack-powervm13:58
thorstAshana: I think the same way you've been pulling down your patches?13:58
thorstalso need wangqwsh's patches13:59
Ashanaok last time I just would just update the vm for the cinder pactch13:59
thorstmight need to do that for these other ones14:00
Ashanahow can I check to make sure the patches are there after I update the vm?14:01
thorstunsure.  You were doing manually?  Not Git?  So I think manual verification is needed.14:02
*** apearson has quit IRC14:03
Ashanano i just did apt-get update, i didnt know you could use git. Would it be a git pull origin master ?14:04
*** apearson has joined #openstack-powervm14:05
*** mdrabe has quit IRC14:09
*** Ashana_ has joined #openstack-powervm14:17
*** efried has quit IRC14:18
*** tjakobs has joined #openstack-powervm14:19
*** wangqwsh has quit IRC14:19
*** efried has joined #openstack-powervm14:19
*** Ashana has quit IRC14:20
*** Ashana has joined #openstack-powervm14:24
efriedthorst, sorry for the delay, one doc issue on https://review.openstack.org/#/c/341125/ if you please.14:25
*** Ashana_ has quit IRC14:27
*** efried has quit IRC14:29
*** efried has joined #openstack-powervm14:30
*** arnoldje has joined #openstack-powervm14:31
Ashana@thorst would I need to do a git pull origin master on my node?14:34
Ashanathorst: would I need to do a git pull origin master on my node?14:34
*** efried has quit IRC14:34
thorstAshana: I think that would get you back to clean...14:39
thorstI use this when I want to get my stuff clean in my git tree14:39
thorsthttp://stackoverflow.com/questions/1628088/reset-local-repository-branch-to-be-just-like-remote-repository-head14:39
*** seroyer has joined #openstack-powervm14:41
Ashanaawesome yep that worked i believe fetched the new patches. What were all the new patches again?14:49
*** mdrabe has joined #openstack-powervm14:57
Ashanathorst:  awesome yep that worked i believe fetched the new patches. What were all the new patches again?15:00
thorstAshana: I think you're on all of them on gerrit review15:10
Ashanaohh ok yep I am15:10
thorsthttps://review.openstack.org/34666615:10
thorstthat one too15:10
Ashanathorst: ok im checking my system and I don't see the patches were they merged? if not should i just add them manaually?15:16
thorstI think you can add them via a git patch15:16
*** Ashana has quit IRC15:21
*** Ashana has joined #openstack-powervm15:21
*** Ashana has quit IRC15:24
*** Ashana_ has joined #openstack-powervm15:24
Ashana_thorst: I havent used git patch before?15:24
*** k0da has quit IRC15:27
thorstAshana_: I'll have time in 25 min or so.15:35
thorstchat through it then?15:35
Ashana_awesome thank you15:35
openstackgerritDrew Thorstensen proposed openstack/networking-powervm: Support returning non-mgmt LPAR UUIDs  https://review.openstack.org/34112515:58
*** Ashana_ has quit IRC16:08
*** Ashana has joined #openstack-powervm16:17
*** Ashana has quit IRC16:22
*** Ashana has joined #openstack-powervm16:23
*** Ashana has quit IRC16:27
*** Ashana has joined #openstack-powervm16:34
*** Ashana has quit IRC16:38
*** efried has joined #openstack-powervm16:41
*** Ashana has joined #openstack-powervm16:46
*** Ashana has quit IRC16:50
*** Ashana has joined #openstack-powervm16:51
*** Ashana_ has joined #openstack-powervm16:55
*** Ashana has quit IRC16:56
*** Ashana_ has quit IRC17:00
*** Ashana has joined #openstack-powervm17:03
*** Ashana has quit IRC17:08
*** Ashana has joined #openstack-powervm17:09
*** Ashana has quit IRC17:10
*** Ashana has joined #openstack-powervm17:10
*** efried has quit IRC17:34
*** Ashana has quit IRC17:39
*** Ashana has joined #openstack-powervm17:54
*** Ashana has quit IRC17:54
*** Ashana has joined #openstack-powervm17:55
*** k0da has joined #openstack-powervm18:03
*** efried has joined #openstack-powervm18:07
*** k0da has quit IRC18:26
*** tsjakobs has joined #openstack-powervm18:29
*** tjakobs has quit IRC18:32
thorstAshana: your neo4 is ready18:40
thorstI'll send a note to Adam and Qing Wu and you about it18:40
AshanaGreat, you want me to set up the network?18:41
adreznecthorst: My IRC client was having connection issues so I know I missed a bunch of conversation18:43
adreznecWhat did you guys end up doing on that node?18:43
thorstAshana: Network I think is set up.  But I think qingwu had left some stuff on there18:49
thorstso I think you just need to update the local openstack ansible stuff on neo418:49
*** mdrabe has quit IRC19:22
thorstsvenkat: please rereview: https://review.openstack.org/#/c/341125/1119:22
*** mdrabe has joined #openstack-powervm19:25
*** kotra03 has quit IRC19:42
*** Ashana has quit IRC19:45
*** Ashana has joined #openstack-powervm19:49
*** k0da has joined #openstack-powervm20:05
mdrabeefried: see my comment on 346234, I think the change needs to be a little more involved20:07
adreznechttps://www.openstack.org/summit/barcelona-2016/vote-for-speakers/presentation/15607 <- thorst esberglu20:40
adreznecWow. "Challenges faced from deploying the first Microsoft Openstack CI to scaling to over 300 hardware nodes and over 10 projects."20:40
adreznecMakes our CI seem tiny20:41
*** catintheroof has quit IRC20:52
esbergluLol imagine some of the issues we are hitting scaled to 300 nodes20:54
thorstadreznec: but are they as beefy as ours?  :-)21:02
*** thorst has quit IRC21:16
efriedmdrabe: Ready to discuss if you want.21:21
mdrabeefried: Whatcha thinkin21:23
efriedI'm thinking including a ComprehensiveScrub in driver init is a good thing... but as you've observed, it's not going to fix this bug.21:23
efriedCorrect me if I'm wrong, but there's nothing in our code path that's broken.21:24
efriedWe're appropriately attempting to delete the appropriate mappings in the appropriate place under the auspices of destroy_evacuated, right?21:25
efriedIt's just that the mapping deletion is failing due to this goofy database sync issue on the VIOS.21:25
mdrabeNo goofy database sync at all21:25
mdrabeIt's the vscsi drivers fault21:25
mdrabe2 secs21:26
seroyermdrabe, you’ve got my attention.  What’s the driver’s fault?21:26
mdrabeseroyer: Sorry the nova-powervm vscsi driver21:26
seroyerWhew.21:26
mdrabeToo many drivers21:26
efriedOh, this is the thing where we're not deleting a mapping whose ITL has been ripped out21:26
mdrabeSi21:26
efriedI thought we decided it was safe to do that.21:27
efriedSo why aren't we doing that?21:27
mdrabeCuz I thought the ComprehensiveScrub would work21:27
mdrabeUntil I realized I didn't think about it enough21:27
mdrabeNow I know it won't work21:27
efriedFor that matter, why are we trying to be careful in the slightest about deleting mappings for an LPAR we're destroying?21:27
efriedWhy not just blindly delete any mapping associated with that LPAR ID?21:27
efriedIn our destroy flow, are we trying to do careful, piecemeal disconnects?21:28
efriedI bet we are.21:28
efriedI reckon that's unnecessary.21:28
efriedLet's be aggressive.21:28
mdrabeIt's21:28
efriedadd_lpar_scrub_tasks to the destroy flow's stg_ftsk and be done.=21:28
mdrabeThe vscsi driver disconnect never took into account stale mappings21:29
mdrabeThat's not really the fix either21:29
mdrabeB/c there's the warning message21:29
mdrabeThe correct fix would address the warning message so that we don't get it21:29
mdrabepulling up code...21:30
mdrabehttps://github.com/openstack/nova-powervm/blob/master/nova_powervm/virt/powervm/volume/vscsi.py#L371-L39121:30
mdrabeThe warning message I'm talking about is the one at the end of that highlight21:30
mdrabe_discover_volume_on_vios obviously isn't going to work!21:31
mdrabeWe ripped the heck out of the volume21:31
mdrabeThe disconnect volume in the vscsi driver actually does a discovery, in this case, before it issues the hdisk/mapping removal21:32
efriedThat's what I'm saying.21:32
efriedWe shouldn't bother with the graceful disconnect at all.21:32
efriedJust scrub all mappings for this LPAR.21:32
efriedIt'll be quicker as well as more thorough.21:32
mdrabeWell remember, this is the normal disconnect flow as well21:32
mdrabeLike when a VM is just normally deleted21:32
efriedOr is it because we need to be able to delete the hdisk as well?21:32
efriedYeah, that should also be fine, no?21:33
mdrabeThe hdisk should be gone in the case of deleting an evacuated VM21:33
efriedMaybe we want to wait and have this conversation with thorst, but I'm not seeing any reason to be graceful or careful about ripping out mappings for an LPAR we're deleting, no matter how we're deleting it.21:33
mdrabeI make the change21:34
mdrabeI know what needs to be done21:34
mdrabeI'm just saying, it's not as nice "oh do a scrub task"21:34
mdrabeI can* make the change21:34
mdrabeefried: Yea scrubbing won't address that warning message, because the volume disconnection subtasks come before the LPAR is deleted21:41
mdrabeTo my understanding the mappings are only scrubable after the LPAR is deleted, unless the scrubbing logic is changed21:42
*** svenkat has quit IRC21:43
efriedmdrabe, you're not getting me.21:43
efriedWhat I'm proposing will avoid that warning message because it'll avoid that code altogether.21:43
mdrabedeleting the LPAR makes the client adapter go away?21:43
efriedNot sure about that.  But I'm talking about using add_lpar_storage_scrub_tasks or whatever that guy is called.21:43
efriedAnd specifying the flag that says, "I know the LPAR exists - blaze the mappings anyway."21:44
efriedLemme find the exact call...21:44
mdrabeYes ok I remember that flag21:44
mdrabeThat could work then I think21:44
efriednova_powervm/virt/powervm/tasks/vm.py:9821:45
efriedis an example21:45
efriedpvm_stg.add_lpar_storage_scrub_tasks([wrap.id], self.stg_ftsk, lpars_exist=True)21:45
mdrabeSo that'll blow up all the mappings for the LPAR?21:46
efriedAnd - again, we should run this by thorst - suggesting we use that in the destroy flow, period.  No matter how the LPAR is being destroyed.21:46
mdrabeShould be okay for vSCSI, not sure about NPIV though21:47
*** lmtaylor has quit IRC21:54
*** esberglu has quit IRC21:55
*** tblakeslee has quit IRC21:56
openstackgerritTaylor Jakobson proposed openstack/nova-powervm: [WIP] Add iscsi support  https://review.openstack.org/34692022:02
*** openstackgerrit has quit IRC22:03
*** openstackgerrit has joined #openstack-powervm22:04
*** mdrabe has quit IRC22:05
*** efried has quit IRC22:05
*** arnoldje has quit IRC22:07
*** tsjakobs has quit IRC22:11
*** edmondsw has quit IRC22:24
*** seroyer has quit IRC22:24
*** thorst has joined #openstack-powervm22:26
*** svenkat has joined #openstack-powervm22:39
*** Ashana has quit IRC22:44
*** k0da has quit IRC22:45
*** openstackgerrit has quit IRC22:48
*** openstackgerrit has joined #openstack-powervm22:48
*** Ashana has joined #openstack-powervm22:51
openstackgerritMerged openstack/networking-powervm: Support returning non-mgmt LPAR UUIDs  https://review.openstack.org/34112522:55
*** Ashana has quit IRC22:55
*** Ashana has joined #openstack-powervm22:57
*** Ashana has quit IRC23:02
*** Ashana has joined #openstack-powervm23:18
*** esberglu has joined #openstack-powervm23:18
*** Ashana has quit IRC23:23
*** esberglu has quit IRC23:23
*** esberglu has joined #openstack-powervm23:25
*** thorst has quit IRC23:29
*** esberglu has quit IRC23:29
*** thorst has joined #openstack-powervm23:29
*** apearson has quit IRC23:30
*** thorst has quit IRC23:38
*** wangqwsh has joined #openstack-powervm23:53
*** wangqwsh_ has joined #openstack-powervm23:55
*** wangqwsh has quit IRC23:58
*** wangqwsh_ is now known as wangqwsh23:58
« Monday, 2016-07-25 Index Wednesday, 2016-07-27 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!