| *** thorst_ has joined #openstack-powervm | 00:00 | |
| *** k0da has quit IRC | 00:02 | |
| *** edmondsw has quit IRC | 00:04 | |
| *** thorst_ has quit IRC | 00:17 | |
| *** thorst_ has joined #openstack-powervm | 00:17 | |
| *** thorst_ has quit IRC | 00:22 | |
| *** efried has quit IRC | 00:34 | |
| *** jwcroppe has quit IRC | 00:41 | |
| *** thorst_ has joined #openstack-powervm | 00:42 | |
| *** jwcroppe has joined #openstack-powervm | 00:42 | |
| *** thorst_ has quit IRC | 00:44 | |
| *** jwcroppe has quit IRC | 00:47 | |
| *** efried has joined #openstack-powervm | 00:47 | |
| openstackgerrit | zhangyanxian proposed openstack/networking-powervm: Fix typos in sea_agent.py https://review.openstack.org/424394 | 00:54 |
|---|---|---|
| openstackgerrit | zhangyanxian proposed openstack/networking-powervm: Fix typos in sea_agent.py https://review.openstack.org/424394 | 00:54 |
| *** k0da has joined #openstack-powervm | 01:48 | |
| *** edmondsw has joined #openstack-powervm | 02:22 | |
| *** thorst_ has joined #openstack-powervm | 02:49 | |
| *** thorst_ has quit IRC | 02:49 | |
| *** edmondsw has quit IRC | 02:51 | |
| *** edmondsw has joined #openstack-powervm | 02:52 | |
| *** k0da has quit IRC | 02:53 | |
| *** edmondsw has quit IRC | 02:54 | |
| *** edmondsw has joined #openstack-powervm | 02:54 | |
| *** edmondsw has quit IRC | 02:59 | |
| *** thorst_ has joined #openstack-powervm | 03:33 | |
| *** thorst_ has quit IRC | 03:34 | |
| *** k0da has joined #openstack-powervm | 03:36 | |
| *** k0da has quit IRC | 03:47 | |
| *** jwcroppe has joined #openstack-powervm | 04:02 | |
| *** jwcroppe has quit IRC | 04:02 | |
| *** jwcroppe has joined #openstack-powervm | 04:02 | |
| *** tlian has quit IRC | 04:14 | |
| *** jwcroppe has quit IRC | 04:31 | |
| *** jwcroppe has joined #openstack-powervm | 05:26 | |
| *** thorst_ has joined #openstack-powervm | 05:35 | |
| *** thorst_ has quit IRC | 05:40 | |
| *** jwcroppe has quit IRC | 06:24 | |
| *** thorst_ has joined #openstack-powervm | 07:36 | |
| *** thorst_ has quit IRC | 07:40 | |
| *** openstackgerrit has quit IRC | 08:03 | |
| *** k0da has joined #openstack-powervm | 08:37 | |
| *** thorst_ has joined #openstack-powervm | 09:30 | |
| *** thorst_ has quit IRC | 09:35 | |
| *** k0da has quit IRC | 11:11 | |
| *** edmondsw has joined #openstack-powervm | 11:20 | |
| *** edmondsw has quit IRC | 11:24 | |
| *** thorst_ has joined #openstack-powervm | 11:31 | |
| *** thorst_ has quit IRC | 11:36 | |
| *** smatzek_ has joined #openstack-powervm | 11:45 | |
| *** BorD__ has quit IRC | 12:00 | |
| *** edmondsw has joined #openstack-powervm | 12:41 | |
| *** edmondsw has quit IRC | 12:45 | |
| *** thorst_ has joined #openstack-powervm | 12:47 | |
| *** BorD_ has joined #openstack-powervm | 12:55 | |
| *** AlexeyAbashkin has joined #openstack-powervm | 13:16 | |
| *** edmondsw has joined #openstack-powervm | 13:18 | |
| *** edmondsw has quit IRC | 13:20 | |
| *** edmondsw has joined #openstack-powervm | 13:20 | |
| *** nbante has joined #openstack-powervm | 13:33 | |
| *** kylek3h has joined #openstack-powervm | 13:37 | |
| *** openstackgerrit has joined #openstack-powervm | 13:45 | |
| openstackgerrit | Drew Thorstensen (thorst) proposed openstack/nova-powervm: Fix hostname & initiator for volume drivers https://review.openstack.org/424290 | 13:45 |
| thorst_ | efried: in our driver mtg, I'd like to talk for a few minutes on CI stability around VM power off. | 13:53 |
| efried | ack | 14:02 |
| efried | We starting that now? | 14:02 |
| efried | adreznec: thorst_: (wangqwsh esberglu) we having a meeting? | 14:04 |
| *** wangqwsh has joined #openstack-powervm | 14:05 | |
| adreznec | Still not seeing esberglu efried. Is he out or just running late? | 14:07 |
| efried | Not online. Slacked him. | 14:08 |
| *** mdrabe has joined #openstack-powervm | 14:08 | |
| *** apearson has joined #openstack-powervm | 14:08 | |
| adreznec | Hmm ok | 14:08 |
| *** tblakes has joined #openstack-powervm | 14:09 | |
| efried | adreznec: thorst_: wangqwsh: He's on his way in now. Forgot his badge. | 14:14 |
| adreznec | Ok | 14:14 |
| adreznec | Well we can start without him and back up to CI when he arrives | 14:15 |
| wangqwsh | ok | 14:15 |
| adreznec | #startmeeting powervm_drver_meeting | 14:15 |
| openstack | Meeting started Tue Jan 24 14:15:12 2017 UTC and is due to finish in 60 minutes. The chair is adreznec. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:15 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:15 |
| openstack | The meeting name has been set to 'powervm_drver_meeting' | 14:15 |
| adreznec | #topic In-tree driver status | 14:15 |
| adreznec | Lets start here. I'll turn things over to you efried | 14:15 |
| *** esberglu has joined #openstack-powervm | 14:16 | |
| esberglu | Hey guys sorry I'm late, forgot my badge | 14:17 |
| efried | I'm slowly working my way through the early in-tree change sets. | 14:17 |
| adreznec | np esberglu, just fired up the meeting. talking in-tree driver status first | 14:17 |
| efried | First one is for sure ready for wider review; we're just waiting for in-tree CI before we can put pressure on the cores to review. | 14:18 |
| efried | If we can get the CI up today or tomorrow, there's a chance we can get mriedem to do a review before the nova meeting on Thursday. | 14:18 |
| efried | What's the o-3 date? Is it Thursday or Friday? | 14:18 |
| esberglu | Thursday | 14:19 |
| adreznec | efried: It's up to projects a bit. Most will be Thursday, but Jan 23-27 is the official range | 14:19 |
| efried | Well, okay, this is nova we're talking about. | 14:19 |
| efried | Confirmed Thursday. | 14:20 |
| thorst_ | whoops | 14:20 |
| efried | Sooo... we pretty much miss ocata if we don't have the CI up today. Cause I don't think we get the change merged on the first pass. | 14:20 |
| thorst_ | sorry... | 14:20 |
| thorst_ | efried: I think we've missed Ocata. :-) | 14:20 |
| thorst_ | for the intree. | 14:20 |
| *** smatzek_ has quit IRC | 14:21 | |
| efried | Let's work as if there's still a chance. | 14:21 |
| thorst_ | agree. | 14:21 |
| efried | Remainder of driver status is piddly details. I can go more in depth if you want, but we should probably spend the time on more important stuff. | 14:21 |
| efried | *in-tree driver status | 14:22 |
| thorst_ | I have oot driver talk, but I suspect that's a different part of meeting | 14:22 |
| efried | yuh, suggest waiting til after we talk in-tree CI. | 14:22 |
| efried | Anything else before we move on to that? | 14:22 |
| adreznec | Nothing here | 14:22 |
| adreznec | #topic In-tree driver CI status | 14:23 |
| adreznec | esberglu, the floor is yours | 14:23 |
| esberglu | The in-tree driver is failing to stack. "n-cpu service is not running" for some reason | 14:23 |
| esberglu | Problem is that the logs are failing to copy | 14:23 |
| esberglu | So i'm running one manually | 14:23 |
| esberglu | To see what the deal is | 14:23 |
| adreznec | esberglu: That's weird. Is it just not getting far enough to transfer somehow? | 14:24 |
| adreznec | Or is there actually an scp failure? | 14:24 |
| *** apearson has quit IRC | 14:24 | |
| esberglu | Nah it's actually a SCP failure. Which is weird because I didn't change anything for SCP | 14:25 |
| esberglu | Trying to create /srv/static/logs/59 | 14:25 |
| esberglu | It fails trying to create thta | 14:25 |
| esberglu | the log server isn't full or anything though | 14:25 |
| esberglu | I will let you guys know the results of the manual run when it finishes | 14:26 |
| esberglu | The other thing I had | 14:26 |
| adreznec | Hmm odd. Nothing in that flow should have changed unless something is broken in the build variables somehow | 14:26 |
| esberglu | There's a "test connection" thing in the configure system, and it is connecting to the log server fine | 14:27 |
| esberglu | 3 scripts run as part of the CI setup | 14:27 |
| esberglu | 1) prepare_node_powervm | 14:27 |
| esberglu | 2) ready_node_powervm | 14:27 |
| esberglu | 3) prep_devstack | 14:27 |
| esberglu | Previously we would install the patched (develop) pypowervm in prepare_node_powervm | 14:27 |
| esberglu | Since we are now using 2 different versions of pypowervm (1.0.0.4 for in tree, develop for oot) I moved the installation to prep_devstack | 14:27 |
| esberglu | The problem is that we need the patched pypowervm to be installed for the ready_node script to work | 14:27 |
| esberglu | So I was thinking just install the patched develop in prepare_node_powervm | 14:28 |
| esberglu | Then if it is in tree, just overwrite with the patched 1.0.0.4 | 14:28 |
| esberglu | Thoughts? | 14:28 |
| efried | As long as 1.0.0.4 is in place before the nova compute process starts, it shouldn't matter when we do it. | 14:29 |
| efried | Heck, I would even be okay skipping that wrinkle for now and just continuing to use develop | 14:29 |
| adreznec | efried: I think it does | 14:29 |
| efried | They're not different enough that it's going to cause failures. | 14:29 |
| adreznec | Because I think we use pvmctl before that | 14:29 |
| adreznec | To do node setup | 14:29 |
| efried | and we can focus on getting things working first, then worry about that version switch later. | 14:29 |
| esberglu | Yeah that was the problem. We don't know if it's in or out of tree until (3) but we need pvmctl by (2) | 14:30 |
| adreznec | esberglu: we could just always install develop for step 2 | 14:30 |
| adreznec | then switch it out for the "right" version in 3 if needed? | 14:30 |
| adreznec | Fragments it a bit, but... meh | 14:30 |
| esberglu | Yeah. That's exactly how I have it set up right now. Seems to be working, just wanted to make sure I wasn't missing something | 14:30 |
| efried | oh, okay. | 14:31 |
| efried | thorst_ adreznec - can you think of an easy way to have e.g. Adapter() init log the pypowervm version? | 14:31 |
| adreznec | There are other options like bundling pypowervm/pvmctl into a venv and shipping that whole thing so pvmctl has its own pypowervm to use always or something | 14:31 |
| adreznec | But they're more work | 14:32 |
| efried | I don't know where the version numbers are stored. I'm sure it involves pbr or something. | 14:32 |
| thorst_ | efried: no idea... | 14:32 |
| adreznec | We'd have to make a call off to pbr's version_string method to get that | 14:34 |
| efried | In [12]: pbr.version.VersionInfo('pypowervm').release_string() | 14:34 |
| efried | Out[12]: '1.0.0.dev4' | 14:34 |
| efried | :) | 14:34 |
| thorst_ | could we just log the version at the end of the CI job? | 14:34 |
| adreznec | Yeah | 14:34 |
| thorst_ | and call it a day there? | 14:34 |
| adreznec | I wonder if we should stop using pbr for pypowervm though | 14:34 |
| adreznec | pbr only really works well with semver | 14:35 |
| efried | Well, I wanted to have a way to be sure the compute process was started with the correct version. | 14:35 |
| adreznec | As you can see there | 14:35 |
| adreznec | Since the version probably isn't really 1.0.0.dev4 | 14:35 |
| adreznec | but 1.0.0.4 or 1.0.0.5 | 14:35 |
| esberglu | We would have to log it before it gets patched | 14:35 |
| efried | wouldn't think so | 14:36 |
| esberglu | Once it's patched the version becomes 1.0.1devxxx | 14:36 |
| esberglu | I'm pretty sure | 14:36 |
| efried | If this is going to be more than a five-minute thing, then never mind; but it would be useful in the long run. | 14:38 |
| efried | For right now, like I say, I would be okay moving forward even if the compute process is still using develop. | 14:38 |
| efried | I got lost. What's the next step here? Seeing how the local run goes, and then nailing down the scp thing? | 14:39 |
| esberglu | Yep | 14:40 |
| esberglu | #action esberglu: Finish manual in tree run and update with results | 14:40 |
| esberglu | #action esberglu: Figure out why logs aren't being copied | 14:40 |
| efried | esberglu: would having another body help move this along any faster, or are we bottlenecked? | 14:41 |
| efried | I would be volunteering someone like adreznec who knows this stuff ;-) | 14:41 |
| esberglu | If someone wants to help with the SCP thing. Nothing to do for the manual run but wait | 14:41 |
| efried | k. thorst_ is that in your wheelhouse? | 14:41 |
| thorst_ | huh? | 14:42 |
| efried | Do you have the expertise and bandwidth to help esberglu figure out this SCP boggle? | 14:43 |
| thorst_ | ooo, I do not. | 14:43 |
| efried | adreznec? | 14:43 |
| thorst_ | survival mode atm | 14:43 |
| efried | Cause I know I don't. | 14:43 |
| adreznec | efried: Not sure yet, still bogged down right now | 14:43 |
| efried | Okay, if there's nobody with the technical chops, I'd be happy to be a sounding board and additional googler. | 14:44 |
| adreznec | Will depend on how things shake out with meetings really | 14:44 |
| efried | #action efried to help esberglu with SCP boggle, for whatever that's worth. | 14:44 |
| efried | esberglu: is there anything else you can see on the horizon that will need to be addressed? Something we might be able to get a head start on if we're stuck waiting for whatever? | 14:45 |
| efried | The big obvious thing is paring down the test list - but we don't really know where to start with that. However, setting up the infrastructure to use a whitelist? | 14:45 |
| *** k0da has joined #openstack-powervm | 14:46 | |
| esberglu | I already know how we should do that | 14:46 |
| esberglu | This is the conf we use for out of tree | 14:46 |
| esberglu | https://github.com/powervm/powervm-ci/blob/master/tempest/os_ci_tempest.conf#L26 | 14:46 |
| esberglu | We need to make a second conf for in tree | 14:46 |
| thorst_ | well, its going to be a whitelist | 14:47 |
| esberglu | And then we set the BASE_TEST_REGEX to include all the tests we want | 14:47 |
| thorst_ | so its only supposed to be the tests we want | 14:47 |
| thorst_ | ahh, nm...I see | 14:47 |
| esberglu | Yep. The BASE_TEST_REGEX for out of tree includes all the tests | 14:47 |
| efried | oh, so it's already whitelisting. It's just really inclusive. | 14:47 |
| adreznec | consensus! | 14:47 |
| thorst_ | that's rough. | 14:47 |
| esberglu | Yeah the "whitelist" for out of tree is all tests, then it gets reduced by the skip_list | 14:48 |
| thorst_ | I'm going to nope myself out of anything with regex | 14:48 |
| efried | So the BASE_TEST_REGEX is going to be a regex with (id|id|id|id.....) | 14:48 |
| thorst_ | I find regex to be an awful creation | 14:48 |
| efried | ahh, thorst_, you don't understand the beauty of regex. | 14:48 |
| efried | <patpat> | 14:48 |
| efried | Sokay, I'm your regex guy. | 14:48 |
| thorst_ | efried: you are correct, I find it flawed and awful | 14:49 |
| thorst_ | but that's my definition of 'beauty' | 14:49 |
| thorst_ | can't be awful | 14:49 |
| esberglu | efried: It we be easier to use test names, then we could include groups of tests with one regex. But it was recommended to use to use ID's before | 14:49 |
| efried | Yeah, however, I don't know that we really want to handle the whitelist with a regex like that. There's probably another (better) way to do it. | 14:49 |
| efried | So - let me take another look at the os_ci_tempest.sh and see what I can figure out. Unless esberglu has already done that? | 14:50 |
| efried | I forget, which project holds the real one of those? neo-os-ci or powervm-ci? | 14:50 |
| esberglu | powervm-ci | 14:50 |
| efried | #action efried to investigate whitelisting | 14:52 |
| *** smatzek has joined #openstack-powervm | 14:53 | |
| adreznec | All right | 14:53 |
| adreznec | Anything else on in-tree CI? | 14:53 |
| adreznec | Ok, I know thorst_ had discussion on out-of-tree | 14:53 |
| adreznec | #topic Out-of-tree driver discussion | 14:54 |
| thorst_ | yeah, so our oot CI is kinda flaking out again | 14:54 |
| thorst_ | I'm seeing several patches failing... | 14:54 |
| thorst_ | I've at least root caused one of em. | 14:54 |
| thorst_ | http://pastebin.com/uN5aB1Kk | 14:54 |
| thorst_ | that's the error. Basically it is a non-force immediate power off. | 14:54 |
| thorst_ | and its just hanging | 14:54 |
| thorst_ | I think we've hit this a few times now...so we should get it fixed. | 14:55 |
| thorst_ | I opened a bug a long time ago around this | 14:55 |
| thorst_ | https://bugs.launchpad.net/nova-powervm/+bug/1562117 | 14:55 |
| openstack | Launchpad bug 1562117 in nova-powervm "power-off times are not adhered to" [Low,New] - Assigned to Lauren Taylor (lmtaylor) | 14:55 |
| thorst_ | I think we need some attention put on it now. Anyone have cycles to explore that fix? | 14:55 |
| thorst_ | I can also check with lmtaylor on it, but she's had it for a while and hasn't updated it recently. | 14:56 |
| efried | squeaky wheel | 14:56 |
| thorst_ | alright. | 14:56 |
| thorst_ | well, I'll work on it as I free up | 14:56 |
| thorst_ | but it's impacting CI. Sigh | 14:56 |
| thorst_ | that was about it | 14:57 |
| thorst_ | I suspect we'll argue in the review | 14:57 |
| thorst_ | so awareness now, this one will be a weird review...so pay attention to the review. | 14:57 |
| efried | thorst_: is the problem that you think we should be timing out faster? | 14:58 |
| *** jwcroppe has joined #openstack-powervm | 15:00 | |
| thorst_ | OpenStack gives us values for timeout and retry | 15:00 |
| thorst_ | what those values mean...is open to interpretation | 15:00 |
| thorst_ | does a 0 mean immediately or wait forevs | 15:00 |
| thorst_ | I interpret it as 'immediate' | 15:00 |
| thorst_ | :-) | 15:00 |
| efried | I see. | 15:02 |
| efried | Should prolly go look at how libvirt et al interpret those values. | 15:02 |
| efried | libvirt agrees with thorst_ | 15:03 |
| thorst_ | well then, its easy | 15:03 |
| efried | timeout != 0 => "gracefully" | 15:03 |
| thorst_ | anyway, I'll get on it | 15:03 |
| thorst_ | soon enough, cause its blocking my other changes (kinda) | 15:03 |
| thorst_ | that's all I had for OOT. Big thanks for reviews on the fileio thing | 15:04 |
| thorst_ | not sure if we can get that into ocata...would've been nice | 15:04 |
| efried | k, if you find you don't have time, I can take over. | 15:04 |
| efried | #action thorst_ https://bugs.launchpad.net/nova-powervm/+bug/1562117 - efried to help if needed. | 15:04 |
| openstack | Launchpad bug 1562117 in nova-powervm "power-off times are not adhered to" [Low,New] - Assigned to Lauren Taylor (lmtaylor) | 15:04 |
| adreznec | Ok | 15:04 |
| *** tlian has joined #openstack-powervm | 15:04 | |
| adreznec | So I know we're over | 15:04 |
| adreznec | #topic Open floor | 15:05 |
| adreznec | Anything else? | 15:05 |
| * thorst_ dances on open floor | 15:05 | |
| efried | not from me, but esberglu stick around so we can talk about the whitelist. | 15:05 |
| adreznec | Okay | 15:05 |
| adreznec | #endmeeting | 15:05 |
| openstack | Meeting ended Tue Jan 24 15:05:40 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:05 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/powervm_drver_meeting/2017/powervm_drver_meeting.2017-01-24-14.15.html | 15:05 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/powervm_drver_meeting/2017/powervm_drver_meeting.2017-01-24-14.15.txt | 15:05 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/powervm_drver_meeting/2017/powervm_drver_meeting.2017-01-24-14.15.log.html | 15:05 |
| efried | esberglu: Do you have an env set up where I can run testr? | 15:06 |
| efried | Cause I think I'm close to having an easier way than using the regex. | 15:07 |
| efried | back in 5 | 15:08 |
| *** apearson has joined #openstack-powervm | 15:13 | |
| esberglu | efried: thorst_: adreznec: Turns out I was looking at the wrong run in jenkins (one of the other in tree patches) | 15:32 |
| esberglu | The actual issue seems to be a tempest conf thing | 15:32 |
| esberglu | I'll PM you guys the link to the run | 15:32 |
| efried | esberglu: Have we merged the thing that converts the skip list from name to ID yet? | 15:42 |
| efried | Looks like not. Has it been tested live yet? | 15:43 |
| efried | Cause I don't actually see how it could work, given the code I'm looking at in os_ci_tempest.sh | 15:43 |
| esberglu | No I haven't had a chance to test it yet. I would have to put changes into the CI scripts to use that patch | 15:44 |
| esberglu | And there are already enough changes in those scripts, I didn't want it to get too messy | 15:44 |
| efried | okay. So yeah, the way we parse the skip list is to remove the square-bracketed part and match up on the base name. | 15:45 |
| efried | So work needs to be done in os_ci_tempest.sh before that'll work. | 15:45 |
| esberglu | Could we just move the comment with the name to a separate line? | 15:45 |
| efried | doesn't help | 15:46 |
| *** tjakobs has joined #openstack-powervm | 15:46 | |
| *** burgerk has joined #openstack-powervm | 15:49 | |
| esberglu | I think I may have found the source of the stack issue for in tree | 15:49 |
| esberglu | The out of tree driver has this in local.conf | 15:49 |
| esberglu | IMAGE_URLS="file:///home/jenkins/vm_images/base_os.img" | 15:49 |
| esberglu | But the in-tree has | 15:49 |
| esberglu | IMAGE_URLS="file:///home/neo/images/Ubuntu_16-04-01_2g.img" | 15:50 |
| efried | Yah, cause you ripped it from my test system. | 15:50 |
| esberglu | Yep | 15:50 |
| *** burgerk_ has joined #openstack-powervm | 15:50 | |
| efried | Easy fix. scp still mysterious. Can you be trying another run while we bang our heads against scp some more? | 15:51 |
| *** burgerk has quit IRC | 15:53 | |
| esberglu | Yeah. I will just make the change and do a manual run | 15:53 |
| esberglu | Otherwise we would have to wait for a full image rebuild and all that jazz | 15:53 |
| efried | dig | 15:54 |
| esberglu | Hmm okay | 15:54 |
| esberglu | I might know what it is | 15:54 |
| esberglu | the /src/static/logs directory is owned by jenkins | 15:55 |
| esberglu | *srv | 15:55 |
| efried | is that different than in oot? | 15:55 |
| esberglu | but all of the /srv/static/logs/# directories are owned by powervmci (which is the username SCP uses) | 15:56 |
| esberglu | I wonder if we just need to change the ownership of the logs dir to powervmci | 15:56 |
| esberglu | I'll give it a try | 15:58 |
| *** mdrabe has quit IRC | 15:58 | |
| adreznec | esberglu: Maybe... but that wouldn't explain how it worked for anything else | 16:03 |
| *** apearson has quit IRC | 16:07 | |
| *** apearson has joined #openstack-powervm | 16:11 | |
| esberglu | efried: adreznec: Changing ownership of that dir worked | 16:12 |
| efried | whoah | 16:12 |
| efried | cool | 16:12 |
| adreznec | That's... bizarre | 16:12 |
| efried | but cool | 16:13 |
| efried | but bizarre | 16:13 |
| adreznec | Wonder how it was creating subdirectories fine before | 16:13 |
| esberglu | No idea | 16:13 |
| esberglu | I redeployed the log server so maybe that changed something. But idk why it would, there haven't been any changes to the log server playbook that I know of | 16:14 |
| adreznec | Magic | 16:14 |
| adreznec | Is this prod or staging? | 16:14 |
| esberglu | staging | 16:15 |
| adreznec | Maybe something to do with the 16.04 upgrade then? | 16:15 |
| adreznec | Idk | 16:15 |
| esberglu | Yeah maybe. It's also being used for the cache / dns stuff now. Again idk how that would affect it | 16:16 |
| esberglu | Who knows why, but at least it's working | 16:16 |
| *** mdrabe has joined #openstack-powervm | 16:30 | |
| esberglu | adreznec: thorst_: efried: Manual run got through the stack with the updated image url | 16:38 |
| esberglu | Just kicked off os_ci_tempest. Lets see what happens | 16:38 |
| efried | noyce. | 16:38 |
| esberglu | Just running the test list from OOT, it might take a while if there are a bunch of timeouts / failures | 16:41 |
| esberglu | Well mostly timeouts, we know there will be tons of failures | 16:41 |
| *** apearson has quit IRC | 16:59 | |
| *** apearson has joined #openstack-powervm | 17:04 | |
| esberglu | Results are in | 17:21 |
| esberglu | Total tests: 1146 | 17:21 |
| esberglu | Passed tests: 947 | 17:21 |
| esberglu | Failed tests: 16 | 17:21 |
| esberglu | Skipped tests: 183 | 17:21 |
| esberglu | Wtf it used the wrong local.conf | 17:24 |
| esberglu | I must have messed something up running it manually | 17:30 |
| thorst_ | lol, we still passed 947 tests? | 17:48 |
| esberglu | It used the wrong local.conf | 17:49 |
| esberglu | So it stacked with nova-powervm enabled | 17:49 |
| adreznec | Aww | 17:50 |
| adreznec | I thought our driver was just super awesome | 17:50 |
| *** tblakes has quit IRC | 17:55 | |
| thorst_ | lol | 17:56 |
| thorst_ | efried: per earlier discussion: https://review.openstack.org/#/c/424780/ | 17:59 |
| efried | thorst_: ack | 18:00 |
| thorst_ | gerrit seems to be mad and won't let me add any reviewers | 18:01 |
| adreznec | weird | 18:01 |
| *** tblakes has joined #openstack-powervm | 18:08 | |
| efried | thorst_: reviewed | 18:09 |
| thorst_ | k | 18:09 |
| efried | thorst_: it let me add reviewers. | 18:10 |
| thorst_ | of course you crap on it quick | 18:10 |
| efried | Must just be mad at you. | 18:10 |
| thorst_ | yeah, I can now too | 18:10 |
| efried | I didn't crap on it. | 18:10 |
| efried | Sprinkled a little fairy dust, that's all. | 18:10 |
| efried | thorst_: Not gonna implement retry yet? | 18:11 |
| thorst_ | read the retry | 18:13 |
| thorst_ | it isn't a retry | 18:13 |
| thorst_ | its a retry_interval | 18:13 |
| thorst_ | it is actually a polling period to say 'should i check if the power off completed yet, how about now' | 18:13 |
| thorst_ | it is essentially what our pypowervm job framework does. | 18:14 |
| thorst_ | I don't see much value in implementing that one | 18:14 |
| openstackgerrit | Drew Thorstensen (thorst) proposed openstack/nova-powervm: Fix power off timings https://review.openstack.org/424780 | 18:14 |
| efried | thorst_: dig | 18:15 |
| efried | thorst_: LGTM. Want more reviews before you push it? | 18:17 |
| thorst_ | eh | 18:21 |
| thorst_ | lets see what CI says at least | 18:21 |
| esberglu | Making progress. Stacked with the right local.conf this time. Missing the | 18:31 |
| esberglu | NEUTRON_CREATE_INITIAL_NETWORKS=False | 18:31 |
| esberglu | Config option in the local.conf which causes script failures after the stack | 18:31 |
| *** jwcroppe_ has joined #openstack-powervm | 18:34 | |
| thorst_ | efried: is one of your boxes SR-IOV? | 18:35 |
| efried | uhhh. I think so. Stand by. | 18:35 |
| efried | yeah | 18:36 |
| efried | 40 | 18:36 |
| thorst_ | if so, erlarese would probably give you a hug if he could swap our your system with his | 18:36 |
| thorst_ | he needs a SR-iov system to mix with an OVS install | 18:36 |
| thorst_ | if you're actively using it...then that's one thing | 18:36 |
| efried | I am actively using it, but I can probably recover fairly quickly. | 18:36 |
| efried | After all, devstack is easy. | 18:36 |
| efried | clbush has the other SR-IOV system, and he may not be using it yet (or what he's doing may not conflict with what erlarese wants to do). | 18:37 |
| efried | I'll contact them. | 18:37 |
| *** jwcroppe has quit IRC | 18:37 | |
| thorst_ | efried: thx | 18:38 |
| *** jwcroppe_ has quit IRC | 18:38 | |
| thorst_ | esberglu: we're still getting a lot of one off failures | 18:40 |
| thorst_ | that to me, don't look related to the actual change set | 18:40 |
| thorst_ | would you agree? | 18:40 |
| *** nbante has quit IRC | 18:40 | |
| esberglu | Yeah | 18:41 |
| thorst_ | alright, we will need some focus there soon to root out what our code bugs are | 18:42 |
| thorst_ | hopefully after this power off thing merges it goes down | 18:42 |
| thorst_ | but I bet we have a few latent bugs | 18:42 |
| thorst_ | hammering it with this volume has been awesom | 18:42 |
| *** jpasqualetto has joined #openstack-powervm | 18:58 | |
| *** smatzek has quit IRC | 19:40 | |
| *** dwayne__ has quit IRC | 19:57 | |
| thorst_ | efried: I'm going to push this thing through | 19:58 |
| thorst_ | https://review.openstack.org/#/c/424780/ | 19:58 |
| thorst_ | then propose back to newton | 19:58 |
| thorst_ | also, csky gave his +1 here that you wanted: https://review.openstack.org/#/c/424290/ | 19:59 |
| openstackgerrit | Merged openstack/nova-powervm: Fix power off timings https://review.openstack.org/424780 | 20:08 |
| *** jpasqualetto has left #openstack-powervm | 20:12 | |
| *** smatzek has joined #openstack-powervm | 20:15 | |
| *** dwayne has joined #openstack-powervm | 20:25 | |
| thorst_ | efried: https://review.openstack.org/#/c/424835/ | 20:37 |
| openstackgerrit | Merged openstack/nova-powervm: Fix hostname & initiator for volume drivers https://review.openstack.org/424290 | 20:45 |
| efried | adreznec: thorst_: esberglu: Where's a recent AIO local.conf? | 20:47 |
| efried | ...for OOT | 20:47 |
| esberglu | The one we use for CI is in neo-os-ci/ci-ansible/roles/ci-management/templates/scripts/local.conf.aio | 20:48 |
| efried | thanks. | 20:48 |
| *** Jay1 has joined #openstack-powervm | 20:48 | |
| *** thorst_ has quit IRC | 21:55 | |
| *** Jay1 has quit IRC | 22:07 | |
| *** tblakes has quit IRC | 22:09 | |
| *** smatzek has quit IRC | 22:16 | |
| *** apearson has quit IRC | 22:26 | |
| *** apearson has joined #openstack-powervm | 22:30 | |
| *** tblakes has joined #openstack-powervm | 22:30 | |
| *** apearson has quit IRC | 22:30 | |
| *** apearson has joined #openstack-powervm | 22:41 | |
| *** burgerk__ has joined #openstack-powervm | 22:48 | |
| *** esberglu has quit IRC | 22:48 | |
| *** esberglu has joined #openstack-powervm | 22:48 | |
| *** burgerk_ has quit IRC | 22:51 | |
| *** esberglu has quit IRC | 22:53 | |
| *** tblakes has quit IRC | 22:53 | |
| *** burgerk__ has quit IRC | 22:54 | |
| *** mdrabe has quit IRC | 22:54 | |
| *** edmondsw has quit IRC | 22:58 | |
| *** apearson has quit IRC | 22:59 | |
| *** esberglu has joined #openstack-powervm | 23:02 | |
| *** esberglu has quit IRC | 23:06 | |
| *** tjakobs has quit IRC | 23:14 | |
| *** edmondsw has joined #openstack-powervm | 23:34 | |
| *** k0da has quit IRC | 23:38 | |
| *** edmondsw has quit IRC | 23:38 | |
| *** thorst_ has joined #openstack-powervm | 23:57 | |
| *** thorst_ has quit IRC | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!