| *** kriskend has quit IRC | 00:03 | |
| *** svenkat has quit IRC | 00:16 | |
| *** svenkat has joined #openstack-powervm | 00:44 | |
| *** efried has quit IRC | 00:48 | |
| *** jwcroppe has quit IRC | 00:57 | |
| *** efried has joined #openstack-powervm | 01:02 | |
| *** thorst_afk has joined #openstack-powervm | 01:17 | |
| *** svenkat has quit IRC | 01:21 | |
| *** thorst_afk has quit IRC | 01:41 | |
| *** thorst_afk has joined #openstack-powervm | 01:56 | |
| *** thorst_afk has quit IRC | 02:50 | |
| *** thorst_afk has joined #openstack-powervm | 03:03 | |
| *** thorst_afk has quit IRC | 03:04 | |
| *** thorst_afk has joined #openstack-powervm | 03:35 | |
| *** thorst_afk has quit IRC | 03:35 | |
| *** mdrabe has joined #openstack-powervm | 03:57 | |
| *** seroyer has joined #openstack-powervm | 04:34 | |
| *** seroyer has quit IRC | 04:49 | |
| *** thorst_afk has joined #openstack-powervm | 04:57 | |
| *** thorst_afk has quit IRC | 05:02 | |
| *** mdrabe has quit IRC | 05:05 | |
| *** tjakobs has joined #openstack-powervm | 05:08 | |
| *** tlian has quit IRC | 05:32 | |
| *** thorst_afk has joined #openstack-powervm | 06:59 | |
| *** thorst_afk has quit IRC | 07:03 | |
| *** thorst_afk has joined #openstack-powervm | 08:59 | |
| *** k0da has joined #openstack-powervm | 09:03 | |
| *** thorst_afk has quit IRC | 09:04 | |
| *** tjakobs has quit IRC | 09:07 | |
| *** edmondsw has joined #openstack-powervm | 09:54 | |
| *** k0da has quit IRC | 09:56 | |
| *** edmondsw has quit IRC | 09:58 | |
| *** thorst_afk has joined #openstack-powervm | 11:01 | |
| *** thorst_afk has quit IRC | 11:05 | |
| *** k0da has joined #openstack-powervm | 11:14 | |
| *** smatzek_ has joined #openstack-powervm | 11:39 | |
| *** smatzek_ has quit IRC | 11:40 | |
| *** smatzek has joined #openstack-powervm | 11:41 | |
| *** viclarson has joined #openstack-powervm | 12:01 | |
| *** seroyer has joined #openstack-powervm | 12:38 | |
| *** thorst_afk has joined #openstack-powervm | 12:43 | |
| *** Esha has joined #openstack-powervm | 12:53 | |
| Esha | The stable/ocata PowerVM CI isue was fixed, Thanks Eric ! Now seeing an issue with stable/newton PowerVM CI in https://review.openstack.org/#/c/435108/ | 12:55 |
|---|---|---|
| *** tblakes has joined #openstack-powervm | 12:58 | |
| *** svenkat has joined #openstack-powervm | 13:03 | |
| *** edmondsw has joined #openstack-powervm | 13:07 | |
| *** jpasqualetto has joined #openstack-powervm | 13:16 | |
| *** jpasqualetto has quit IRC | 13:48 | |
| *** dwayne has joined #openstack-powervm | 13:48 | |
| *** Esha has quit IRC | 13:53 | |
| *** mdrabe has joined #openstack-powervm | 13:54 | |
| *** viclarson has quit IRC | 13:55 | |
| *** viclarson has joined #openstack-powervm | 13:55 | |
| thorst_afk | mdrabe: did you have a chance to look at that slot error from the other day? | 13:56 |
| *** thorst_afk is now known as thorst_ | 13:56 | |
| mdrabe | Yea it didn't look like a slot error | 13:56 |
| mdrabe | thorst_ It was that the disk couldn't be found | 13:56 |
| thorst_ | which to me indicates that the rebuild map was called before the hdisk was discovered. | 13:57 |
| thorst_ | the hdisk was probably there (IMO), it probably just didn't run the VIOS command yet to discover it | 13:57 |
| thorst_ | does that make sense? | 13:58 |
| mdrabe | yea | 13:58 |
| thorst_ | so my question is...how is this working in other envs | 13:58 |
| mdrabe | But there were two errors printed as debug statements | 13:58 |
| mdrabe | one for each VIOS, that volume X couldn't be found on this VIOS | 13:58 |
| thorst_ | right...because the slot map checks each vios | 13:59 |
| mdrabe | But the error isn't coming from the slot map | 13:59 |
| mdrabe | No ITLs for VIOS vio1-38w for volume 35d30504-6a77-4fb2-acab-286a9cd067ad. _discover_volume_on_vios /usr/lib/python2.7/dist-packages/nova_powervm/virt/powervm/volume/vscsi.py:214 | 14:00 |
| *** jpasqualetto has joined #openstack-powervm | 14:00 | |
| mdrabe | I don't think that should be happening at all, regardless of the slot map | 14:01 |
| *** apearson has joined #openstack-powervm | 14:01 | |
| thorst_ | http://pastebin.com/FQ5zZm0f | 14:01 |
| thorst_ | that's what you're working off of? | 14:01 |
| mdrabe | No from the LP bug | 14:01 |
| mdrabe | https://bugs.launchpad.net/nova-powervm/+bug/1665407 | 14:02 |
| openstack | Launchpad bug 1665407 in nova-powervm "unable to migrate vm with attached volumes" [Undecided,New] | 14:02 |
| *** svenkat has quit IRC | 14:04 | |
| thorst_ | yeah, see comment 2: vif.plug() -> slot_mgr.build_map() -> slot_mgt.init_recreate_map() -> slot_mgr._pv_vscsi_vol_to_vio() -> vol_drv.is_volume_on_vios() | 14:04 |
| thorst_ | o damn...that does the discover. | 14:05 |
| mdrabe | Oh boy the slot map is doing discover... | 14:05 |
| thorst_ | yeah, I knew we had to be doing that. | 14:05 |
| thorst_ | otherwise this would be borked to hell elsewhere... | 14:05 |
| thorst_ | now viclarson is running stable/mitaka. I wonder if there was a change missing or something back there. | 14:07 |
| openstackgerrit | Merged openstack/nova-powervm master: Add imagecache information to devref https://review.openstack.org/434358 | 14:07 |
| mdrabe | It is possible, a lot of that stuff went in right at the end of mitaka | 14:08 |
| thorst_ | yeah, but I thought we did get it all back ported. | 14:09 |
| thorst_ | alright...so my current theory is that maybe cinder is not attaching it to the new host in time. | 14:09 |
| mdrabe | this is also resize to a different host, which wasn't looked at as much I don't think | 14:10 |
| thorst_ | right. | 14:12 |
| thorst_ | our testing was primarily resize on same host or remote rebuild | 14:13 |
| thorst_ | viclarson: you around? | 14:13 |
| viclarson | here | 14:14 |
| thorst_ | so current theory on the bug you opened. | 14:14 |
| thorst_ | at least my current theory... | 14:14 |
| thorst_ | This is a resize to a different host. Which means that Cinder needs to attach the volume to the other host before that 'finish_resize' method is called. | 14:15 |
| thorst_ | can you do the test again, and make sure that during the resize op, the new host has the volume attached to it on the backing v7000 (is that right?) when it calls that finish_resize | 14:16 |
| thorst_ | ? | 14:16 |
| viclarson | i changed slot_mgt to | 14:16 |
| viclarson | other_slot_mgr(vol_drv_iter=None) for PlugVif and PlugManagmentVif | 14:17 |
| viclarson | now it hungs on connections | 14:17 |
| viclarson | i think it because vm stays in active state | 14:18 |
| thorst_ | well you won't get your cinder volumes really there... | 14:18 |
| viclarson | cinder mapped volumes to hosts in time | 14:18 |
| viclarson | once again attach, detach, resize with attached volumes works fine (when nvram_store is none) | 14:19 |
| viclarson | mb i'm wrong | 14:20 |
| viclarson | and it's just unstable behaviour | 14:20 |
| thorst_ | so what the error is telling us is that the other host doesn't have a path to the volumes | 14:20 |
| thorst_ | the VIOS can't find the volumes... | 14:20 |
| thorst_ | this generally indicates that something is up between the target host and the storage subsystem | 14:20 |
| thorst_ | maybe a FC zone is off (probably not the case?) or the cinder driver isn't mapping to it in time | 14:21 |
| thorst_ | we're trying to definitively rule that out... | 14:21 |
| *** burgerk has joined #openstack-powervm | 14:28 | |
| viclarson | 16:31:14 mkvdiskhostmap target host | 14:30 |
| viclarson | 16:31:15 Operation finish migration | 14:30 |
| *** dwayne has quit IRC | 14:35 | |
| viclarson | <thorst_> maybe a FC zone is off (probably not the case?) **** or the cinder driver isn't mapping to it in time **** | 14:37 |
| viclarson | what to check? | 14:37 |
| *** jwcroppe has joined #openstack-powervm | 14:43 | |
| *** esberglu has joined #openstack-powervm | 14:44 | |
| thorst_ | viclarson: so that can be an easy check...can you deploy new VMs on each host and attach new volumes? If so, then its not a zoning issue | 14:44 |
| *** tjakobs has joined #openstack-powervm | 14:45 | |
| viclarson | yes i can | 14:48 |
| thorst_ | hmm... | 14:48 |
| thorst_ | how many hosts do you have? | 14:48 |
| viclarson | 2 | 14:48 |
| thorst_ | can you put one into maintenance...then resize the VM | 14:49 |
| thorst_ | it should do a local resize | 14:49 |
| thorst_ | want to see if that flow works. | 14:49 |
| viclarson | i switched out local resize | 14:49 |
| viclarson | dont work | 14:49 |
| thorst_ | same error eh? | 14:49 |
| viclarson | the same behaviour as i have now for remote resize | 14:50 |
| viclarson | vm hangs in state finishing resize | 14:50 |
| thorst_ | interesting... | 14:52 |
| viclarson | the wrong word ... boring... | 14:53 |
| *** kriskend has joined #openstack-powervm | 14:53 | |
| *** tlian has joined #openstack-powervm | 14:53 | |
| thorst_ | heh, wish I could get in the env. :-) | 14:56 |
| thorst_ | viclarson: you going to be at the PTG next week? If so, I could dig in on it with you | 14:56 |
| *** seroyer has quit IRC | 14:58 | |
| viclarson | no im not going and unfortunately its impossible. isolated environment. the bank | 14:58 |
| *** k0da has quit IRC | 15:03 | |
| thorst_ | ahh. | 15:04 |
| thorst_ | let me see if I can work through this a bit more...I'll work off code inspection | 15:04 |
| thorst_ | what we're thinking is somehow the volume id is changing | 15:04 |
| thorst_ | which is super weird. | 15:04 |
| *** tjakobs has quit IRC | 15:11 | |
| *** dwayne has joined #openstack-powervm | 15:15 | |
| *** tblakes has quit IRC | 15:17 | |
| *** seroyer has joined #openstack-powervm | 15:19 | |
| *** tblakes has joined #openstack-powervm | 15:20 | |
| *** tjakobs has joined #openstack-powervm | 15:41 | |
| *** viclarson_ has joined #openstack-powervm | 15:52 | |
| *** viclarson has quit IRC | 15:57 | |
| *** kylek3h has quit IRC | 15:58 | |
| *** kylek3h has joined #openstack-powervm | 15:59 | |
| *** kylek3h has quit IRC | 16:01 | |
| *** kylek3h has joined #openstack-powervm | 16:01 | |
| *** tlian2 has joined #openstack-powervm | 16:10 | |
| *** jpasqualetto has quit IRC | 16:11 | |
| *** tlian has quit IRC | 16:12 | |
| *** tlian2 has quit IRC | 16:15 | |
| *** jpasqualetto has joined #openstack-powervm | 16:27 | |
| *** svenkat has joined #openstack-powervm | 16:30 | |
| efried | thorst_ adreznec esberglu First five in-tree change sets are ready for your reviews. | 16:47 |
| *** jpasqualetto has quit IRC | 16:54 | |
| *** kjw3 has joined #openstack-powervm | 16:58 | |
| *** tblakes has quit IRC | 17:03 | |
| *** tblakes has joined #openstack-powervm | 17:03 | |
| *** jpasqualetto has joined #openstack-powervm | 17:06 | |
| *** jpasqualetto has quit IRC | 17:23 | |
| *** mdrabe has quit IRC | 17:24 | |
| *** mdrabe has joined #openstack-powervm | 17:32 | |
| *** jpasqualetto has joined #openstack-powervm | 17:57 | |
| viclarson_ | finally i changed slot_mgr = slot.build_slot_mgr( | 18:02 |
| viclarson_ | instance, self.store_api, adapter=self.adapter) #without vol_dev_it | 18:02 |
| viclarson_ | vm appears with attached volumes ip and hostame | 18:03 |
| viclarson_ | what will broke? | 18:04 |
| *** tlian has joined #openstack-powervm | 18:18 | |
| *** viclarson_ has quit IRC | 18:25 | |
| thorst_ | o man...just missed viclarson | 18:51 |
| thorst_ | I'll have to catch him Monday | 18:51 |
| thorst_ | efried: if you see him in IRC while I'm at PTG - basically the volumes won't be rebuilt at the same LUA index. This may confuse some OSes. It is in line with other platforms, but the purpose of the volume iterator is so that we can put the volumes back in the identical location. | 18:52 |
| thorst_ | efried: working on your reviews now | 18:52 |
| efried | ack | 18:52 |
| efried | "confuse some OSes" by making the disk names come up different, right? | 18:53 |
| *** kylek3h is now known as kylek3h_away | 19:02 | |
| efried | thorst_ 4883 when you get a chance too. Little 'un. | 19:02 |
| thorst_ | review.o.o is a pain today | 19:03 |
| thorst_ | ;ppls ;ole tjeu | 19:03 |
| thorst_ | looks like they're restarting it... | 19:03 |
| thorst_ | oo, I've been waiting for 4883... | 19:03 |
| -openstackstatus- NOTICE: Restarting gerrit due to performance problems | 19:04 | |
| efried | Yeah, r.o.o has been choppy last few minutes anyway. | 19:04 |
| thorst_ | had one comment on 4883 | 19:05 |
| esberglu | thorst_: efried: adreznec: Are there any *-powervm changes we are trying to get in before I cut the RC at the end of the day? | 19:11 |
| adreznec | Nothing from me | 19:14 |
| thorst_ | Not on my side...I'm still working on master | 19:19 |
| *** k0da has joined #openstack-powervm | 19:23 | |
| *** k0da has quit IRC | 19:37 | |
| efried | esberglu Sorry, missed the above. I wouldn't mind https://review.openstack.org/435114 but certainly not urgent/critical. | 19:43 |
| esberglu | Okay. If you want it in you have to go through that cherry-pick to ocata business | 19:45 |
| thorst_ | efried: I think I'm done with reviews | 20:12 |
| *** smatzek has quit IRC | 20:13 | |
| efried | ack | 20:13 |
| *** jpasqualetto has quit IRC | 20:18 | |
| *** jpasqualetto has joined #openstack-powervm | 20:35 | |
| efried | thorst_ Take another look at https://review.openstack.org/#/c/409402/ please? | 20:43 |
| thorst_ | done | 20:47 |
| openstackgerrit | Eric Fried proposed openstack/nova-powervm master: In-tree backports: Task UT, Delete args https://review.openstack.org/435114 | 21:06 |
| *** jpasqualetto has quit IRC | 21:17 | |
| *** kriskend_ has joined #openstack-powervm | 21:25 | |
| *** kriskend has quit IRC | 21:28 | |
| *** burgerk has quit IRC | 21:34 | |
| *** kjw3 has quit IRC | 21:44 | |
| *** thorst_ has quit IRC | 21:53 | |
| *** svenkat has quit IRC | 22:01 | |
| *** tjakobs has quit IRC | 22:12 | |
| mdrabe | efried: Is the OpenStack instance UUID _always_ supposed to match the LPAR's UUID? | 22:15 |
| mdrabe | ignoring the greater bit, the rest of the UUID should be identical I mean | 22:16 |
| *** edmondsw has quit IRC | 22:23 | |
| *** kriskend_ has quit IRC | 22:26 | |
| *** esberglu has quit IRC | 22:29 | |
| *** tblakes has quit IRC | 22:32 | |
| *** seroyer has quit IRC | 22:34 | |
| *** jwcroppe has quit IRC | 22:40 | |
| *** esberglu has joined #openstack-powervm | 22:47 | |
| *** esberglu has quit IRC | 22:52 | |
| *** edmondsw has joined #openstack-powervm | 22:55 | |
| *** edmondsw has quit IRC | 22:59 | |
| *** dwayne has quit IRC | 23:05 | |
| *** jwcroppe has joined #openstack-powervm | 23:05 | |
| *** jwcroppe has quit IRC | 23:09 | |
| *** mdrabe has quit IRC | 23:10 | |
| *** apearson has quit IRC | 23:24 | |
| *** thorst has joined #openstack-powervm | 23:27 | |
| *** thorst has quit IRC | 23:32 | |
| *** k0da has joined #openstack-powervm | 23:47 | |
| *** smatzek has joined #openstack-powervm | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!