| *** zerick has joined #openstack-powervm | 00:00 | |
| *** mdrabe has joined #openstack-powervm | 00:00 | |
| *** dwayne has joined #openstack-powervm | 00:00 | |
| *** esberglu has joined #openstack-powervm | 00:00 | |
| *** kylek3h has joined #openstack-powervm | 00:00 | |
| *** efried_zzz has joined #openstack-powervm | 00:00 | |
| *** adi_____ has joined #openstack-powervm | 00:00 | |
| *** AndyWojo_ has joined #openstack-powervm | 00:00 | |
| *** tonyb has joined #openstack-powervm | 00:00 | |
| *** BorD_ has joined #openstack-powervm | 00:00 | |
| *** toan has joined #openstack-powervm | 00:00 | |
| *** ChanServ has joined #openstack-powervm | 00:00 | |
| *** tepper.freenode.net sets mode: +o ChanServ | 00:00 | |
| *** svenkat has joined #openstack-powervm | 00:14 | |
| *** thorst has joined #openstack-powervm | 00:16 | |
| *** svenkat has quit IRC | 01:35 | |
| *** thorst has quit IRC | 01:59 | |
| *** esberglu has quit IRC | 04:10 | |
| *** esberglu has joined #openstack-powervm | 04:54 | |
| *** esberglu has quit IRC | 04:59 | |
| *** chhavi has joined #openstack-powervm | 05:41 | |
| *** chhavi has quit IRC | 05:54 | |
| *** chhavi has joined #openstack-powervm | 05:55 | |
| *** k0da has joined #openstack-powervm | 06:46 | |
| *** thorst has joined #openstack-powervm | 07:03 | |
| *** thorst has quit IRC | 07:07 | |
| *** chhavi has quit IRC | 07:17 | |
| *** adreznec has quit IRC | 07:52 | |
| *** adreznec has joined #openstack-powervm | 07:58 | |
| *** esberglu has joined #openstack-powervm | 08:33 | |
| *** esberglu has quit IRC | 08:37 | |
| *** chhavi has joined #openstack-powervm | 08:51 | |
| *** k0da has quit IRC | 08:53 | |
| *** thorst has joined #openstack-powervm | 09:04 | |
| *** thorst has quit IRC | 09:08 | |
| *** thorst has joined #openstack-powervm | 10:05 | |
| *** edmondsw has joined #openstack-powervm | 10:06 | |
| *** thorst has quit IRC | 10:09 | |
| *** edmondsw has quit IRC | 10:10 | |
| *** esberglu has joined #openstack-powervm | 10:21 | |
| *** chas has joined #openstack-powervm | 10:22 | |
| *** k0da has joined #openstack-powervm | 10:23 | |
| *** esberglu has quit IRC | 10:26 | |
| *** k0da has quit IRC | 10:57 | |
| *** k0da has joined #openstack-powervm | 11:03 | |
| *** k0da has quit IRC | 11:24 | |
| *** thorst has joined #openstack-powervm | 11:25 | |
| *** thorst has quit IRC | 11:29 | |
| *** svenkat has joined #openstack-powervm | 12:00 | |
| *** thorst has joined #openstack-powervm | 12:03 | |
| *** svenkat_ has joined #openstack-powervm | 12:03 | |
| *** svenkat has quit IRC | 12:04 | |
| *** svenkat_ is now known as svenkat | 12:04 | |
| *** esberglu has joined #openstack-powervm | 12:09 | |
| *** esberglu has quit IRC | 12:14 | |
| *** edmondsw has joined #openstack-powervm | 12:46 | |
| *** apearson has joined #openstack-powervm | 12:49 | |
| *** esberglu has joined #openstack-powervm | 12:54 | |
| esberglu | #startmeeting powervm_driver_meeting | 13:00 |
|---|---|---|
| openstack | Meeting started Tue Aug 15 13:00:37 2017 UTC and is due to finish in 60 minutes. The chair is esberglu. Information about MeetBot at http://wiki.debian.org/MeetBot. | 13:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 13:00 |
| openstack | The meeting name has been set to 'powervm_driver_meeting' | 13:00 |
| edmondsw | o/ | 13:00 |
| mdrabe | o/ | 13:00 |
| thorst | \o | 13:00 |
| *** chas has quit IRC | 13:01 | |
| edmondsw | efried_zzz is not going to make it today | 13:02 |
| esberglu | #topic In Tree Driver | 13:02 |
| esberglu | Nothing going on IT still I believe | 13:03 |
| edmondsw | I wanted to bring up our governance here | 13:03 |
| edmondsw | and use of releases for the *-powervm projects | 13:03 |
| edmondsw | thorst, did you see my note there? | 13:03 |
| thorst | edmondsw: I did, and I started to respond but got distracted | 13:03 |
| edmondsw | do you want to discuss here or? | 13:03 |
| thorst | let me pull it back up | 13:04 |
| thorst | I think a lot of the assumptions we had when we started aren't reality now that we've grown and are going in tree | 13:04 |
| edmondsw | yup, that's what I'm hoping | 13:04 |
| thorst | we have a "don't rock the boat" approach, but if dims is recommending that, it's obviously worth investigating | 13:04 |
| edmondsw | esberglu mdrabe forwarded the note to you... | 13:04 |
| edmondsw | sorry for leaving off | 13:05 |
| esberglu | np | 13:05 |
| thorst | I think a pvm-stackers style group makes sense | 13:05 |
| thorst | assuming the community would be on board | 13:05 |
| edmondsw | cool, because I was thinking the same | 13:05 |
| edmondsw | and maybe even move ceilometer-pvm there instead of under telemetry | 13:05 |
| edmondsw | so they're all together? | 13:05 |
| thorst | we knew that if we came into the community and asked for something like that early on...it'd have been seen as some agenda we were trying to drive | 13:06 |
| thorst | where as our end goal is to just get powervm enablement in the respective projects | 13:06 |
| thorst | I think we've demonstrated that to the community now | 13:06 |
| edmondsw | #action edmondsw to investigate creation of pvm-stackers project under governance | 13:07 |
| edmondsw | that's all I had for IT | 13:07 |
| esberglu | #topic Out of Tree Driver | 13:08 |
| edmondsw | iSCSI is a mess... | 13:09 |
| edmondsw | I've dug into that code some more and have a much better understanding of what we've got ahead of us | 13:09 |
| thorst | edmondsw: I think seroyer has someone that was going to be able to dedicate some time as well to that effort | 13:10 |
| edmondsw | we've got a meeting later today out of which I hope we'll have direction on priority and can put together a timeline | 13:10 |
| edmondsw | thorst awesome | 13:10 |
| edmondsw | so probably a bigger update on this next week | 13:10 |
| esberglu | Cool. Any other OOT discussion items? | 13:11 |
| mdrabe | NVRAM event performance stuff testing yielded some issues. With evacuate itself not the change. I'm wading through them. | 13:12 |
| edmondsw | nothing else from my end | 13:12 |
| mdrabe | it from me | 13:12 |
| esberglu | #topic PCI Passthru | 13:13 |
| edmondsw | efried_zzz has started wading through the code there and noting issues | 13:13 |
| esberglu | Yep saw that note. Nothing to discuss for the time being? | 13:14 |
| edmondsw | yeah, not without him | 13:14 |
| esberglu | #topic PowerVM CI | 13:14 |
| esberglu | Gonna try to link up with hsien today to look into some rest logs to find out why the compute service is occasionally failing to come up | 13:15 |
| edmondsw | ++ | 13:15 |
| edmondsw | we've gotta figure that out | 13:16 |
| esberglu | Other than that I've currently trying to take everything defined in the jenkins jobs and move them to the external scripts | 13:16 |
| esberglu | Which should do 2 things | 13:16 |
| esberglu | 1) Allow changes to those tasks to take effect immediately | 13:16 |
| esberglu | 2) Ease the transition to zuul v3 (no more jenkins!) | 13:16 |
| edmondsw | both good things | 13:17 |
| esberglu | I want to start that transition to zuulv3 asap after it goes live | 13:17 |
| edmondsw | what is the outlook for that? | 13:17 |
| esberglu | mdrabe: It's not worth pursuing that jenkins upgrade anymore. We can just stay at the current version until we get rid of it | 13:18 |
| esberglu | edmondsw: They are releasing it in conjunction with the PTG | 13:18 |
| mdrabe | whew good to hear | 13:18 |
| esberglu | mdrabe: If you have any extra time I can always use extra eyes on tempest failures. Not sure how busy you are elsewhere | 13:19 |
| esberglu | That's all for CI | 13:20 |
| esberglu | #topic Driver Testing | 13:20 |
| esberglu | Doesn't look like jay is on | 13:20 |
| edmondsw | esberglu I think we learned since our last meeting that we were wrong about the progress we thought we were making on the timeouts issues, right? | 13:20 |
| esberglu | edmondsw: Well we made some progress on the REST timeouts, but not the server build timeouts | 13:21 |
| esberglu | I'm planning on being 100% tempest failure debug for the rest of the week | 13:21 |
| edmondsw | sounds good | 13:21 |
| esberglu | unless something else breaks which seems to happen every time I start really digging in | 13:22 |
| edmondsw | I think we can move on to the next topic | 13:22 |
| esberglu | #topic Open Discussion | 13:22 |
| esberglu | Nothing further from me | 13:23 |
| edmondsw | I should have raised this under IT, but we should start looking at adding network support there now that master=queens | 13:23 |
| edmondsw | and config driver, and vSCSI... once we can free someone up to look at those things | 13:24 |
| esberglu | Do we have an order that we would like to do that in? | 13:24 |
| edmondsw | so mdrabe, esberglu, if you have any cycles let me know | 13:24 |
| edmondsw | esberglu good question... not at the moment | 13:24 |
| *** k0da has joined #openstack-powervm | 13:25 | |
| esberglu | edmondsw: I would like to start getting my foot in the door on that. I can work some time in for it | 13:25 |
| edmondsw | I need to look at the changesets we already have up and see what they still need | 13:25 |
| edmondsw | yep, that's what I wanted to hear | 13:25 |
| edmondsw | I think network is probably the priority, but if config_drive is easier we might tackle it first | 13:25 |
| edmondsw | that's it | 13:26 |
| esberglu | edmondsw: We can touch base later to figure out a gameplan | 13:26 |
| edmondsw | ++ | 13:26 |
| esberglu | Nice and quick today. Thanks for joining | 13:26 |
| esberglu | #endmeeting | 13:26 |
| openstack | Meeting ended Tue Aug 15 13:26:58 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 13:27 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-08-15-13.00.html | 13:27 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-08-15-13.00.txt | 13:27 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2017/powervm_driver_meeting.2017-08-15-13.00.log.html | 13:27 |
| edmondsw | esberglu we should do better about using the action tag during meetings... help me with that next time? | 13:28 |
| esberglu | edmondsw: Will do | 13:28 |
| edmondsw | tx | 13:29 |
| esberglu | The question there is whether people will ever go back to the minutes to look at the action items | 13:29 |
| esberglu | Action items to add stuff to the todo lists perhaps | 13:29 |
| *** k0da has quit IRC | 13:31 | |
| *** k0da has joined #openstack-powervm | 14:07 | |
| esberglu | edmondsw: I'm looking at the server timeouts right now. In the test in question there is an initial spawn which works fine. Then a rebuild starts. Before the rebuild spawn completes a delete comes in. | 14:23 |
| esberglu | https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_disk_config.py#L81 | 14:23 |
| esberglu | So the vm never reaches active status here because something is deleting the vm before the test completes | 14:24 |
| esberglu | But I can't see where this delete is coming from | 14:24 |
| esberglu | http://paste.openstack.org/show/618409/ | 14:24 |
| edmondsw | esberglu looking | 14:35 |
| *** efried_zzz is now known as efried | 14:37 | |
| edmondsw | esberglu we probably need to look at how rebuild works | 14:39 |
| edmondsw | efried welcome back. you probably want to read the mtg minutes above and then maybe help us on this timeout issue if you have cycles | 14:39 |
| edmondsw | esberglu I've got another mtg in 20 min but will look til then | 14:39 |
| esberglu | edmondsw: I had to manually rebase 5703 | 14:41 |
| efried | edmondsw Yuh, catching up now. | 14:41 |
| esberglu | Can I get another +2 and then we can go live with those changes | 14:41 |
| edmondsw | esberglu +2 | 14:43 |
| efried | esberglu Consider the possibility that the delete is coming in as part of cleanup because the test timed out. | 14:43 |
| edmondsw | esberglu I think the delete is part of the rebuild | 14:43 |
| edmondsw | https://github.com/openstack/tempest/blob/master/tempest/api/compute/base.py#L331 | 14:44 |
| efried | Right, that delete should be happening before the spawn starts. | 14:46 |
| efried | ...the second spawn, that is. | 14:46 |
| esberglu | efried: That destroy flow completes before the second spawn flow starts | 14:47 |
| efried | esberglu Right. And the delete you're concerned about is coming in when? During the second spawn? | 14:47 |
| esberglu | Yep | 14:47 |
| esberglu | 2nd spawn: http://184.172.12.213/34/493834/1/check/nova-in-tree-pvm/18f9a8f/logs/n-cpu.txt.gz#_Aug_15_07_29_29_007036 | 14:48 |
| esberglu | Destroy flow starts: http://184.172.12.213/34/493834/1/check/nova-in-tree-pvm/18f9a8f/logs/n-cpu.txt.gz#_Aug_15_07_30_46_499989 | 14:48 |
| esberglu | Instance deleted while rebuilding: http://184.172.12.213/34/493834/1/check/nova-in-tree-pvm/18f9a8f/logs/n-cpu.txt.gz#_Aug_15_07_30_54_876019 | 14:49 |
| efried | esberglu And the ultimate error is a test timeout? | 14:50 |
| efried | So yeah, that second delete would be the test cleanup. It's a symptom, not a cause. | 14:51 |
| *** tjakobs has joined #openstack-powervm | 14:51 | |
| edmondsw | efried I don't think so | 14:51 |
| edmondsw | it seems to start too early | 14:52 |
| edmondsw | but maybe | 14:52 |
| esberglu | efried: Nor do I. The build timeout is 20 min. This whole event from start to finish is about 12 min. Unless there is some other timeout in play that I'm not aware of | 14:53 |
| *** k0da has quit IRC | 14:56 | |
| efried | Yeah, there's a per-build timeout, nah? | 14:56 |
| efried | 196s or something? | 14:56 |
| esberglu | Yeah the per build timeout is 1200 | 14:58 |
| esberglu | However there are some additional tempest timeouts that we don't set. A timeout to wait for an image to become available and a timeout for network operations to complete. Both default to 300 | 15:03 |
| esberglu | I take that back | 15:04 |
| esberglu | "Other services that do not define build_timeout will inherit this value" with this value being the compute build timeout | 15:04 |
| *** k0da has joined #openstack-powervm | 15:47 | |
| *** k0da has quit IRC | 15:53 | |
| esberglu | edmondsw: efried: I missed the updated log path variable name in process logs in my manual rebase. 5712 | 16:09 |
| edmondsw | esberglu are you intentionally removing the -e from bash? | 16:14 |
| esberglu | edmondsw: Yeah. Lets say we fail early in the stack. Some of the files won't exist and the script will fail out. Meaning nothing will get posted to the logserver. This way if something fails we at least have a chance that something will be copied | 16:15 |
| edmondsw | cool | 16:17 |
| *** k0da has joined #openstack-powervm | 16:21 | |
| *** k0da has quit IRC | 16:27 | |
| esberglu | efried: edmondsw: One final process_logs fix. 5713. I really should have tested this on staging but didn't feel like spending 4 hours redeploying | 16:35 |
| edmondsw | esberglu +2 but now you'll need to also update the name of the var for that in process_logs accordingly | 16:37 |
| esberglu | edmondsw: Yep will do | 16:38 |
| edmondsw | zuul_log_path instead of zuul_base_log_path | 16:38 |
| edmondsw | tx | 16:38 |
| *** k0da has joined #openstack-powervm | 16:40 | |
| *** esberglu is now known as esberglu_lunch | 16:50 | |
| *** edmondsw has quit IRC | 17:35 | |
| *** thorst has quit IRC | 17:35 | |
| *** thorst has joined #openstack-powervm | 17:37 | |
| *** thorst has quit IRC | 17:41 | |
| *** thorst has joined #openstack-powervm | 17:54 | |
| *** thorst has quit IRC | 17:58 | |
| *** thorst has joined #openstack-powervm | 17:59 | |
| *** svenkat has quit IRC | 18:02 | |
| *** edmondsw has joined #openstack-powervm | 18:05 | |
| *** edmondsw has quit IRC | 18:05 | |
| *** edmondsw has joined #openstack-powervm | 18:06 | |
| *** esberglu_lunch is now known as esberglu | 18:11 | |
| *** chhavi has quit IRC | 18:14 | |
| efried | esberglu I think config drive is easier - or at least the most reasonable next in sequence. | 18:17 |
| efried | In general, we're porting code from nova-powervm with an eye to making nova change sets as small as possible, factoring out anything that's unnecessary for the code as it appears in nova (even if we're going to have to add it back in later), breaking things into the smallest possible functional pieces. | 18:18 |
| esberglu | efried: I thought there was a WIP patch for that but I'm not seeing it | 18:19 |
| esberglu | Nvm it was abandoned found it | 18:19 |
| efried | Config drive already had one started (https://review.openstack.org/409404) but it's waaaay out of date. | 18:19 |
| efried | yeah, sdague's bot abandons stuff that's old & untouched. | 18:19 |
| efried | esberglu Hm, maybe it's not so out of date. It could probably be resurrected very easily. | 18:20 |
| *** jwcroppe has joined #openstack-powervm | 18:25 | |
| esberglu | efried: You seeing log coloring here? | 18:30 |
| esberglu | http://184.172.12.213/56/464756/3/check/nova-powervm-out-of-tree-pvm/d55ebd9/logs/n-cpu.txt.gz | 18:30 |
| efried | Yesss!!! | 18:31 |
| edmondsw | awesome! | 18:31 |
| *** dwayne has quit IRC | 18:31 | |
| efried | How'd you do it? | 18:31 |
| edmondsw | efried ++ on everything you were saying about config_drive | 18:31 |
| edmondsw | I have a todo to go find the abandoned commits and resurrect them if nobody beats me to it | 18:32 |
| esberglu | I did absolutely nothing | 18:32 |
| esberglu | Thought you might have | 18:32 |
| esberglu | The filters are working too | 18:32 |
| efried | edmondsw I have the list of abandoned commits we care about. | 18:33 |
| efried | edmondsw But no point resurrecting 'em until we're gonna do something with 'em. | 18:33 |
| edmondsw | efried ++ | 18:34 |
| efried | Actually edmondsw I can drop 'em in the etherpad | 18:34 |
| edmondsw | efried that would be excellent | 18:34 |
| efried | Most of 'em are there, actually, under "Rebase and/or update and/or repropose (because currently authored by wangqwsh, who's no longer with us) existing in-tree change sets." | 18:35 |
| *** apearson has quit IRC | 19:00 | |
| *** jwcroppe has quit IRC | 19:00 | |
| *** k0da has quit IRC | 19:03 | |
| *** tjakobs has quit IRC | 19:03 | |
| *** k0da has joined #openstack-powervm | 19:03 | |
| *** tjakobs has joined #openstack-powervm | 19:03 | |
| *** jwcroppe has joined #openstack-powervm | 19:29 | |
| *** apearson has joined #openstack-powervm | 19:30 | |
| *** jwcroppe has quit IRC | 20:06 | |
| *** jwcroppe has joined #openstack-powervm | 20:13 | |
| *** jwcroppe has quit IRC | 20:20 | |
| *** jwcroppe has joined #openstack-powervm | 20:24 | |
| *** apearson has quit IRC | 21:09 | |
| *** esberglu has quit IRC | 21:31 | |
| *** esberglu has joined #openstack-powervm | 21:32 | |
| *** esberglu has quit IRC | 21:36 | |
| *** esberglu has joined #openstack-powervm | 21:45 | |
| *** k0da has quit IRC | 21:45 | |
| *** esberglu has quit IRC | 21:45 | |
| *** esberglu has joined #openstack-powervm | 21:45 | |
| *** thorst has quit IRC | 22:12 | |
| *** edmondsw has quit IRC | 22:25 | |
| *** svenkat has joined #openstack-powervm | 22:32 | |
| *** thorst has joined #openstack-powervm | 23:13 | |
| *** thorst has quit IRC | 23:17 | |
| *** thorst has joined #openstack-powervm | 23:43 | |
| *** tjakobs has quit IRC | 23:47 | |
| *** thorst has quit IRC | 23:48 | |
| *** jwcroppe has quit IRC | 23:49 | |
| *** edmondsw has joined #openstack-powervm | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!