| openstackgerrit | lan proposed openstack/nova-powervm master: Fix html_last_updated_fmt for Python3 https://review.openstack.org/475972 | 01:20 |
|---|---|---|
| *** esberglu has joined #openstack-powervm | 02:02 | |
| *** esberglu has quit IRC | 02:07 | |
| *** thorst has joined #openstack-powervm | 03:15 | |
| *** esberglu has joined #openstack-powervm | 03:51 | |
| *** thorst has quit IRC | 03:54 | |
| *** esberglu has quit IRC | 03:55 | |
| *** jwcroppe has joined #openstack-powervm | 04:01 | |
| *** chhavi has joined #openstack-powervm | 04:03 | |
| *** adreznec has quit IRC | 05:08 | |
| *** esberglu has joined #openstack-powervm | 05:39 | |
| *** esberglu has quit IRC | 05:44 | |
| *** kylek3h has quit IRC | 05:58 | |
| *** adreznec has joined #openstack-powervm | 06:00 | |
| *** k0da has joined #openstack-powervm | 06:21 | |
| *** kylek3h has joined #openstack-powervm | 06:37 | |
| *** edmondsw has joined #openstack-powervm | 07:03 | |
| *** edmondsw has quit IRC | 07:08 | |
| *** esberglu has joined #openstack-powervm | 07:27 | |
| *** k0da has quit IRC | 07:28 | |
| *** esberglu has quit IRC | 07:31 | |
| *** esberglu has joined #openstack-powervm | 07:37 | |
| *** esberglu has quit IRC | 07:42 | |
| *** esberglu has joined #openstack-powervm | 08:23 | |
| *** chhavi has quit IRC | 08:28 | |
| *** esberglu has quit IRC | 08:28 | |
| *** k0da has joined #openstack-powervm | 08:40 | |
| *** edmondsw has joined #openstack-powervm | 08:51 | |
| *** edmondsw has quit IRC | 08:56 | |
| *** chhavi has joined #openstack-powervm | 09:21 | |
| *** esberglu has joined #openstack-powervm | 10:08 | |
| *** esberglu has quit IRC | 10:13 | |
| *** edmondsw has joined #openstack-powervm | 10:39 | |
| *** edmondsw has quit IRC | 10:45 | |
| *** chhavi has quit IRC | 10:47 | |
| *** smatzek has joined #openstack-powervm | 11:11 | |
| *** smatzek has quit IRC | 11:47 | |
| *** smatzek has joined #openstack-powervm | 11:52 | |
| *** kylek3h has quit IRC | 12:07 | |
| *** edmondsw has joined #openstack-powervm | 12:21 | |
| *** k0da has quit IRC | 12:32 | |
| *** efried has joined #openstack-powervm | 12:34 | |
| *** efried has quit IRC | 12:39 | |
| *** kylek3h has joined #openstack-powervm | 12:40 | |
| *** dwayne has quit IRC | 12:49 | |
| *** jwcroppe has quit IRC | 13:03 | |
| *** jwcroppe has joined #openstack-powervm | 13:03 | |
| *** jwcroppe has quit IRC | 13:03 | |
| *** jwcroppe has joined #openstack-powervm | 13:04 | |
| *** jpasqualetto has joined #openstack-powervm | 13:06 | |
| *** jwcroppe has quit IRC | 13:06 | |
| *** thorst has joined #openstack-powervm | 13:16 | |
| *** k0da has joined #openstack-powervm | 13:18 | |
| *** esberglu has joined #openstack-powervm | 13:19 | |
| *** thorst has quit IRC | 13:22 | |
| *** esberglu has quit IRC | 13:24 | |
| *** jwcroppe has joined #openstack-powervm | 13:30 | |
| *** thorst has joined #openstack-powervm | 13:32 | |
| *** esberglu has joined #openstack-powervm | 13:35 | |
| *** smatzek has quit IRC | 13:42 | |
| esberglu | edmondsw: 5491 | 13:59 |
| *** smatzek has joined #openstack-powervm | 13:59 | |
| esberglu | thorst: ^ | 13:59 |
| esberglu | Thinking it might take a little while for https://review.openstack.org/#/c/475926/ | 14:00 |
| esberglu | to merge. 5491 adds it to the IT patch list | 14:00 |
| *** mdrabe has joined #openstack-powervm | 14:06 | |
| edmondsw | esberglu yeah... shame we haven't already gotten 5468 merged. Will this complicate that? | 14:11 |
| openstackgerrit | Merged openstack/nova-powervm master: Updated from global requirements https://review.openstack.org/470121 | 14:13 |
| esberglu | edmondsw: Yeah a little bit. But not much. We might have a couple failed runs while everything is getting converted | 14:13 |
| esberglu | I would rather just get IT CI going by merging 5491 than waiting for 5468. I still need to update and test 5468 | 14:14 |
| edmondsw | yep, I +2'd | 14:16 |
| *** dwayne has joined #openstack-powervm | 14:55 | |
| *** thorst has quit IRC | 15:05 | |
| *** efried has joined #openstack-powervm | 15:06 | |
| *** jwcroppe has quit IRC | 15:07 | |
| *** thorst has joined #openstack-powervm | 15:12 | |
| *** thorst has quit IRC | 15:16 | |
| esberglu | edmondsw: efried: Reworked and tested the openstack patching logic. It works and is ready for reviews. I also added the neo-os-ci side of the change | 15:16 |
| esberglu | We were overly complicating the deployment of this change | 15:16 |
| esberglu | We can just merge the powervm-ci change and live update the jenkins-jobs on the management system at the same time | 15:17 |
| esberglu | No need to split up the powervm-ci change | 15:17 |
| esberglu | There might be a few runs that don't behave exactly right while this transition happens, but I'm okay with that | 15:18 |
| *** chhavi has joined #openstack-powervm | 15:18 | |
| *** thorst has joined #openstack-powervm | 15:36 | |
| edmondsw | esberglu I believe 5468 will break the CI until/unless 5492 is merged, because neo-os-ci will still be sending -n and that will be an "Invalid option" | 15:37 |
| esberglu | edmondsw: Yeah I'm saying I can just apply that change to the management node though. The merge of 5492 is independent of me applying it to the management node | 15:38 |
| esberglu | Since I will just be live updating the jenkins jobs | 15:38 |
| edmondsw | ahh | 15:38 |
| *** thorst has quit IRC | 15:40 | |
| esberglu | edmondsw: Going live with the external patching | 16:00 |
| *** mdrabe has quit IRC | 16:01 | |
| edmondsw | esberglu +1 | 16:02 |
| *** mdrabe has joined #openstack-powervm | 16:07 | |
| efried | esberglu Did https://review.openstack.org/#/c/475926/ pick up edmondsw's recheck this morning? | 16:10 |
| efried | Would've expected results by now. | 16:11 |
| esberglu | efried: I was just looking at that. One of the zuul merger processes died | 16:11 |
| esberglu | I restarted it, but you need another rechec | 16:11 |
| esberglu | recheck | 16:11 |
| efried | esberglu Well, the existing job was successful | 16:11 |
| efried | So I'd be happy just to leave it alone. | 16:11 |
| esberglu | efried: Yeah sure | 16:12 |
| efried | just didn't want to start asking for reviews and then have another result come in that wasn't as favorable. | 16:12 |
| esberglu | efried: It's no longer in the queue you should be good to go | 16:13 |
| efried | thanks | 16:13 |
| *** thorst has joined #openstack-powervm | 16:14 | |
| edmondsw | efried you going to ask mriedem to merge that? | 16:15 |
| edmondsw | well, review anyway | 16:15 |
| efried | edmondsw Already on it. | 16:16 |
| efried | And eating crow, as predicted. | 16:16 |
| efried | mriedem +2, expect quick merge from sdague once he sees it. | 16:17 |
| *** thorst has quit IRC | 16:19 | |
| *** thorst has joined #openstack-powervm | 16:27 | |
| *** thorst has quit IRC | 16:27 | |
| *** thorst has joined #openstack-powervm | 16:34 | |
| *** jwcroppe has joined #openstack-powervm | 16:35 | |
| edmondsw | efried tx | 16:37 |
| *** thorst has quit IRC | 16:39 | |
| *** jwcroppe has quit IRC | 16:44 | |
| *** thorst has joined #openstack-powervm | 16:47 | |
| *** thorst has quit IRC | 16:51 | |
| *** efried has quit IRC | 16:53 | |
| *** chhavi has quit IRC | 17:12 | |
| *** thorst has joined #openstack-powervm | 17:19 | |
| *** jwcroppe has joined #openstack-powervm | 17:21 | |
| *** thorst has quit IRC | 17:24 | |
| *** thorst has joined #openstack-powervm | 17:28 | |
| *** thorst has quit IRC | 17:34 | |
| *** thorst has joined #openstack-powervm | 17:38 | |
| *** thorst has quit IRC | 17:40 | |
| openstackgerrit | Matthew Edmonds proposed openstack/nova-powervm master: Use CONF.host for powervm nodename https://review.openstack.org/474799 | 17:41 |
| edmondsw | efried I added the test_driver test to https://review.openstack.org/#/c/474799/ | 17:42 |
| esberglu | edmondsw: efried: 5495. I didn't hit this in my manual testing because I was running with prep_devstack with -f doesn't checkout any patches | 17:42 |
| edmondsw | that's what I was saying seems pretty pointless, but I guess for consistency... | 17:42 |
| esberglu | Should have been more diligent looking checking the script | 17:43 |
| edmondsw | +2 | 17:44 |
| *** jwcroppe has quit IRC | 17:45 | |
| *** thorst has joined #openstack-powervm | 17:53 | |
| *** thorst has quit IRC | 17:57 | |
| *** thorst has joined #openstack-powervm | 18:00 | |
| edmondsw | esberglu efried fyi, using github last night didn't work any better than git.openstack.org, no matter how many times I tried both... just had to wait and it worked today | 18:02 |
| edmondsw | so it wasn't a goo vs. github issue | 18:02 |
| *** thorst has quit IRC | 18:05 | |
| esberglu | Did you try using a different protocol? https vs git? | 18:05 |
| esberglu | We've had poor performance using git in the past, but I thought it was just a temporary network issue | 18:05 |
| *** jwcroppe has joined #openstack-powervm | 18:07 | |
| edmondsw | esberglu used git for both | 18:24 |
| edmondsw | I think it was just a temporary network issue, since it worked today | 18:25 |
| edmondsw | today I'm hitting a different issue... | 18:25 |
| edmondsw | tee: /etc/neutron/neutron.conf: No such file or directory | 18:25 |
| esberglu | https://github.com/powervm/powervm-ci/blob/master/devstack/prep_devstack.sh#L191-L195 | 18:25 |
| esberglu | Not sure why devstack can't create those | 18:26 |
| edmondsw | ah, yeah, I remember that now that you mention it | 18:26 |
| esberglu | I think it sets up files in /etc/nova so it should be able to work in /etc/neutron as well | 18:26 |
| esberglu | Might be a devstack bug, might be a config thing | 18:26 |
| edmondsw | yeah, it definitely sets up files in /etc/nova | 18:27 |
| edmondsw | and /etc/cinder, etc. | 18:27 |
| esberglu | edmondsw: And prior to getting rid of neutron legacy it had access there | 18:28 |
| edmondsw | yeah, can't be an env issue... has to be either devstack bug or a problem with our local.conf | 18:28 |
| esberglu | edmondsw: Yep. I just haven't looked into it yet | 18:28 |
| esberglu | I'm guessing local.conf | 18:28 |
| edmondsw | where does stack.sh log its output to? | 18:28 |
| esberglu | Just because I would think others would hit it if it is a devstack thing | 18:29 |
| esberglu | /opt/stack/logs/stack.sh.log | 18:29 |
| esberglu | Well I guess that's just what I LOGFILE set to something in the CI local.confs | 18:30 |
| esberglu | But that's where it will be by default if you are stacking out of /opt/stack/devstack/ I believe | 18:31 |
| edmondsw | could it be because we're enabling plugins before we enable services? | 18:32 |
| edmondsw | esberglu ^ | 18:33 |
| edmondsw | or the order in which we enable plugins | 18:33 |
| edmondsw | well, not the order of plugin enablement... it's the last one that's failing | 18:37 |
| edmondsw | esberglu I think I did just find one issue.. | 18:47 |
| edmondsw | is_neutron_legacy_enabled checks for any services starting with q- | 18:47 |
| esberglu | Are you worried about pvm-q-sea-agt? | 18:48 |
| edmondsw | yeah | 18:49 |
| edmondsw | I think it may find things that contain, not necessarily start with... not sure on the syntax | 18:50 |
| esberglu | is_neutron_legacy_enabled was returning false when I was testing | 18:50 |
| edmondsw | but one way or the other, 2017-06-21 18:24:02.264 | ++lib/neutron:is_neutron_legacy_enabled:102 return 1 | 18:50 |
| *** jwcroppe has quit IRC | 18:50 | |
| esberglu | edmondsw: Which means it isn't enabled | 18:51 |
| esberglu | Right? | 18:51 |
| esberglu | edmondsw: Yeah that's what we would expect | 18:52 |
| esberglu | https://github.com/openstack-dev/devstack/blob/master/lib/neutron#L101-L105 | 18:52 |
| edmondsw | ok, my bad | 18:52 |
| esberglu | https://github.com/openstack-dev/devstack/blob/master/lib/neutron#L129 | 18:56 |
| esberglu | Here it is | 18:56 |
| esberglu | It's doing sudo install | 18:56 |
| esberglu | But no sudo cp, sudo mkdir, etc | 18:56 |
| esberglu | It should chown $NEUTRON_CONF_DIR to $STACK_USER after creating it | 18:58 |
| esberglu | edmondsw: If you want to give that a try quick | 18:58 |
| edmondsw | esberglu $NEUTRON_CONF_DIR isn't being created, so chown won't help | 18:59 |
| edmondsw | not on its own, anyway | 18:59 |
| esberglu | Oh I misunderstood what that install command was doing | 19:00 |
| *** smatzek_ has joined #openstack-powervm | 19:03 | |
| *** smatzek has quit IRC | 19:03 | |
| esberglu | edmondsw: Yeah that install command is creating the directory I think | 19:05 |
| esberglu | install -d [OPTION]... DIRECTORY... (3rd format) | 19:05 |
| esberglu | "In the third format, create all components of the given DIRECTORY(ies)." | 19:05 |
| esberglu | Well I guess ensuring it's created and transferring ownership to STACK_USER | 19:06 |
| edmondsw | esberglu I just lost connectivity to my neo... :( but I'm sure the directory didn't exist | 19:07 |
| edmondsw | yeah, I'd tried ll -d /etc/neutron earlier and got back ls: cannot access '/etc/neutron': No such file or directory | 19:07 |
| esberglu | edmondsw: You're hitting this during the ml2 plugin setup? | 19:08 |
| edmondsw | esberglu during enable_plugin neutron | 19:09 |
| esberglu | edmondsw: Did you ever figure out what was causing your network issues previously? I can' access a bunch of systems right now either | 19:11 |
| edmondsw | esberglu no, I'm trying to figure out what I can and can't reach right now | 19:12 |
| edmondsw | seems like pok is having network issues | 19:12 |
| esberglu | always | 19:12 |
| *** thorst has joined #openstack-powervm | 19:19 | |
| *** thorst has quit IRC | 19:27 | |
| *** jpasqualetto has quit IRC | 19:36 | |
| *** jpasqualetto has joined #openstack-powervm | 19:36 | |
| esberglu | edmondsw: Perhaps we just need to set STACK_USER=jenkins | 19:46 |
| edmondsw | esberglu why jenkins, not neo? | 19:50 |
| esberglu | Oh I guess neo for you | 19:51 |
| esberglu | jenkins within the CI | 19:51 |
| edmondsw | esberglu oh, right | 19:51 |
| edmondsw | esberglu I thought of that, but I don't see why that should be necessary for neutron and not for nova, etc. | 19:51 |
| edmondsw | I thought STACK_USER was just if you were running as root | 19:51 |
| *** efried has joined #openstack-powervm | 19:52 | |
| esberglu | edmondsw: Just spitballing ideas | 19:52 |
| esberglu | You could be right | 19:53 |
| efried | Thought jenkins on powervm-ci was instant | 19:53 |
| efried | Something going on there? | 19:53 |
| esberglu | What do you mean by that first sentence | 19:54 |
| esberglu | efried: Oh I see what you mean. Pok network issues | 19:55 |
| *** jwcroppe has joined #openstack-powervm | 20:00 | |
| *** openstackgerrit has quit IRC | 20:03 | |
| *** openstackgerrit has joined #openstack-powervm | 20:05 | |
| openstackgerrit | Merged openstack/nova-powervm master: Use CONF.host for powervm nodename https://review.openstack.org/474799 | 20:05 |
| *** jwcroppe_ has joined #openstack-powervm | 20:17 | |
| *** jwcroppe has quit IRC | 20:21 | |
| *** jpasqualetto has quit IRC | 20:34 | |
| *** smatzek_ has quit IRC | 20:39 | |
| *** mdrabe has quit IRC | 20:41 | |
| *** mdrabe has joined #openstack-powervm | 20:42 | |
| edmondsw | esberglu adding STACK_USER=neo in my local.conf didn't help | 20:53 |
| edmondsw | jfyi | 20:54 |
| esberglu | edmondsw: Okay. I'm spinning up some instances to try some stuff. I'll let you know if I find a fix | 20:55 |
| efried | esberglu Do you have an easy way to run IT/OOT CI with pypowervm 5453? | 21:05 |
| esberglu | Sure | 21:05 |
| esberglu | efried: I can either put in a manual run or just live patch it long enough for us to push a CI Check patch through | 21:07 |
| esberglu | The latter is less work | 21:08 |
| efried | esberglu Cool, just want to sanity/regression test, make sure it doesn't break the world. | 21:08 |
| openstackgerrit | Eric Berglund proposed openstack/nova-powervm master: DNM: ci check https://review.openstack.org/328315 | 21:09 |
| esberglu | efried: ^ running on the above | 21:10 |
| efried | esberglu Thanks! | 21:11 |
| *** smatzek_ has joined #openstack-powervm | 21:12 | |
| esberglu | efried: It failed to apply | 21:12 |
| efried | um | 21:12 |
| esberglu | I have to go manual | 21:12 |
| efried | Does it need a rebase? | 21:12 |
| esberglu | efried: I'm think it just doesn't apply to the the CI version of pyowervm (1.1.5) | 21:14 |
| esberglu | *I think | 21:14 |
| esberglu | But I can do it on a manual run | 21:14 |
| *** mdrabe has quit IRC | 21:15 | |
| esberglu | efried: I won't have time to push a full manual run through before I head out. I can run it in the background later tonight unless you don't mind waiting until tomorrow | 21:16 |
| efried | It can wait. | 21:17 |
| esberglu | efried: Okay. On the list for tomorrow | 21:17 |
| edmondsw | esberglu I think I figured out why /etc/neutron didn't exist yet when devstack tried to access it... trying to figure out the best way to fix it now | 21:17 |
| esberglu | Enlighten me | 21:17 |
| edmondsw | configure_neutron hadn't been run yet | 21:17 |
| edmondsw | that is what will create /etc/neutron | 21:17 |
| edmondsw | I'm doing a test run where we enable_services before enable_plugins | 21:18 |
| edmondsw | I *think* that's the right fix | 21:18 |
| esberglu | I thought you were hitting this in configure_neturon_new? Which is called by configure_neutron. Maybe I misunderstood earlier | 21:19 |
| edmondsw | esberglu well, enable_service before enable_plugin didn't fix it, anyway | 21:20 |
| esberglu | So it's running the neutron plugin config before the neutron config? | 21:20 |
| edmondsw | esberglu no, hitting this in neutron_service_plugin_class_add_new | 21:20 |
| edmondsw | called by eutron_service_plugin_class_add | 21:20 |
| edmondsw | called by configure_qos_service_plugin | 21:21 |
| esberglu | What do you mean by enable_services before enable_plugins? | 21:21 |
| edmondsw | (/opt/stack/neutron/devstack/lib/qos) | 21:21 |
| esberglu | In the local.conf? I don't think order matters there | 21:21 |
| edmondsw | I mean that I tried to move the enable_service lines before the enable_plugin lines in local.conf | 21:21 |
| esberglu | edmondsw: Yeah I don't think order matters in local.conf | 21:22 |
| edmondsw | esberglu, yeah, probably right | 21:22 |
| esberglu | I think the problem is that devstack is trying to configure the ml2 plugin before it configures neutron | 21:22 |
| esberglu | (I'm still waiting on my nodes to come up so I'm still just taking shots based on the code) | 21:22 |
| esberglu | edmondsw: I think this might go all the way back to stack.sh itself calling things in the wrong order | 21:24 |
| openstackgerrit | Eric Fried proposed openstack/nova-powervm master: WIP: Adopt new pypowervm power_off APIs https://review.openstack.org/476274 | 21:24 |
| *** mdrabe has joined #openstack-powervm | 21:25 | |
| edmondsw | esberglu yes, the problem is definitely that devstack is trying to configure plugins (I believe it's qos, not ml2) before it configures neutron | 21:26 |
| edmondsw | esberglu I'm tracking back through to figure out why | 21:26 |
| edmondsw | esberglu configure_neutron is on line 1177 of stack.sh, whereas run_phase | 21:29 |
| edmondsw | run_phase stack install is on line 941, and will trigger the plugin installs | 21:29 |
| esberglu | I wonder what the impacts of that reorder are... | 21:31 |
| esberglu | (says anyone trying to do anything with devstack) | 21:32 |
| edmondsw | esberglu I think I'm going to open a bug and see if someone from neutron will bite | 21:33 |
| edmondsw | at least can put in what I've figured out | 21:33 |
| esberglu | edmondsw: We have a super easy workaround for the time being. +1 | 21:33 |
| edmondsw | yep | 21:34 |
| *** smatzek_ has quit IRC | 21:38 | |
| efried | esberglu If there's a way to do this too, 5453 + https://review.openstack.org/476274 OOT | 21:38 |
| efried | Passes live sniff test, so I'm fairly confident. | 21:38 |
| efried | But what I don't want to do is merge 5453, cut a new release, funnel it through g-r, then find out it's busted. | 21:39 |
| esberglu | efried: Yep I can make that happen | 21:39 |
| efried | esberglu Cool man, thanks. | 21:40 |
| efried | Again, tomorrow is soon enough. | 21:40 |
| efried | We're still waiting on chhavi to sign off on the iscsi stuff before we can cut pypowervm anyway. | 21:40 |
| esberglu | efried: Might get a chance kick it off in the background tonight. Planning on wrapping up some stuff tonight since I'm out friday | 21:40 |
| efried | k | 21:41 |
| *** esberglu has quit IRC | 21:41 | |
| edmondsw | esberglu https://bugs.launchpad.net/devstack/+bug/1699609 | 21:45 |
| openstack | Launchpad bug 1699609 in devstack "stack fails unable to find /etc/neutron/neutron.conf" [Undecided,New] | 21:45 |
| edmondsw | eserglu I'll throw up a patch adding that bug to the TODO comment | 21:46 |
| *** efried has quit IRC | 21:48 | |
| *** mdrabe has quit IRC | 21:48 | |
| edmondsw | esberglu efried 5499 | 21:54 |
| *** esberglu has joined #openstack-powervm | 21:55 | |
| *** esberglu has quit IRC | 22:00 | |
| *** dwayne has quit IRC | 22:01 | |
| *** kylek3h has quit IRC | 22:08 | |
| *** jwcroppe has joined #openstack-powervm | 22:15 | |
| *** jwcroppe_ has quit IRC | 22:19 | |
| *** jwcroppe has quit IRC | 22:20 | |
| *** k0da has quit IRC | 22:23 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!