| *** chas_ has joined #openstack-powervm | 00:05 | |
| *** chas_ has quit IRC | 00:09 | |
| *** chas_ has joined #openstack-powervm | 00:24 | |
| *** chas_ has quit IRC | 00:28 | |
| *** jwcroppe has quit IRC | 00:44 | |
| *** jwcroppe has joined #openstack-powervm | 00:44 | |
| *** svenkat has joined #openstack-powervm | 00:59 | |
| *** jwcroppe has quit IRC | 01:13 | |
| *** jwcroppe has joined #openstack-powervm | 01:15 | |
| *** edmondsw has quit IRC | 01:21 | |
| *** smatzek has joined #openstack-powervm | 01:22 | |
| *** chas_ has joined #openstack-powervm | 01:24 | |
| *** chas_ has quit IRC | 01:29 | |
| openstackgerrit | Sridhar Venkat proposed openstack/networking-powervm: ProvisionRequest does not distinguish event source https://review.openstack.org/396467 | 02:11 |
|---|---|---|
| *** svenkat has quit IRC | 02:17 | |
| *** chas_ has joined #openstack-powervm | 02:25 | |
| *** chas_ has quit IRC | 02:30 | |
| *** tjakobs has joined #openstack-powervm | 02:39 | |
| *** tjakobs has quit IRC | 02:43 | |
| *** esberglu has joined #openstack-powervm | 03:07 | |
| *** smatzek has quit IRC | 03:09 | |
| *** esberglu has quit IRC | 03:11 | |
| *** tjakobs has joined #openstack-powervm | 04:03 | |
| *** jwcroppe has quit IRC | 04:13 | |
| *** jwcroppe has joined #openstack-powervm | 04:14 | |
| *** apearson has joined #openstack-powervm | 04:15 | |
| *** jwcroppe has quit IRC | 04:18 | |
| *** chas_ has joined #openstack-powervm | 04:26 | |
| *** chas_ has quit IRC | 04:31 | |
| *** jwcroppe has joined #openstack-powervm | 04:36 | |
| *** esberglu has joined #openstack-powervm | 04:56 | |
| *** esberglu has quit IRC | 05:00 | |
| *** tjakobs has quit IRC | 05:24 | |
| *** chas_ has joined #openstack-powervm | 05:28 | |
| *** kotra03 has joined #openstack-powervm | 05:28 | |
| *** chas_ has quit IRC | 05:32 | |
| *** tjakobs has joined #openstack-powervm | 05:39 | |
| *** tjakobs has quit IRC | 05:43 | |
| *** thorst_ has quit IRC | 05:49 | |
| *** thorst_ has joined #openstack-powervm | 05:50 | |
| *** thorst_ has quit IRC | 05:59 | |
| *** apearson has quit IRC | 06:04 | |
| *** apearson has joined #openstack-powervm | 06:04 | |
| *** apearson has quit IRC | 06:05 | |
| *** apearson has joined #openstack-powervm | 06:05 | |
| *** apearson has joined #openstack-powervm | 06:06 | |
| *** apearson has quit IRC | 06:07 | |
| *** apearson has joined #openstack-powervm | 06:07 | |
| *** apearson has quit IRC | 06:07 | |
| *** apearson has joined #openstack-powervm | 06:08 | |
| *** apearson has quit IRC | 06:08 | |
| *** apearson has joined #openstack-powervm | 06:08 | |
| *** apearson has quit IRC | 06:09 | |
| *** apearson has joined #openstack-powervm | 06:09 | |
| *** apearson has quit IRC | 06:10 | |
| *** apearson has joined #openstack-powervm | 06:10 | |
| *** apearson has quit IRC | 06:11 | |
| *** apearson has joined #openstack-powervm | 06:11 | |
| *** apearson has quit IRC | 06:11 | |
| *** apearson has joined #openstack-powervm | 06:12 | |
| *** apearson has quit IRC | 06:12 | |
| *** apearson has joined #openstack-powervm | 06:12 | |
| *** apearson has quit IRC | 06:13 | |
| *** esberglu has joined #openstack-powervm | 06:45 | |
| *** esberglu has quit IRC | 06:49 | |
| *** jwcroppe has quit IRC | 07:11 | |
| *** jwcroppe has joined #openstack-powervm | 07:12 | |
| *** jwcroppe has quit IRC | 07:16 | |
| *** thorst_ has joined #openstack-powervm | 08:01 | |
| *** thorst_ has quit IRC | 08:08 | |
| *** esberglu has joined #openstack-powervm | 08:34 | |
| *** esberglu has quit IRC | 08:38 | |
| *** thorst_ has joined #openstack-powervm | 09:07 | |
| *** adreznec has quit IRC | 09:09 | |
| *** adreznec has joined #openstack-powervm | 09:11 | |
| *** thorst_ has quit IRC | 09:14 | |
| *** kylek3h has quit IRC | 09:16 | |
| *** kylek3h has joined #openstack-powervm | 09:23 | |
| *** esberglu has joined #openstack-powervm | 09:34 | |
| *** esberglu has quit IRC | 09:38 | |
| *** openstackgerrit has quit IRC | 09:48 | |
| *** openstackgerrit has joined #openstack-powervm | 09:49 | |
| *** thorst_ has joined #openstack-powervm | 10:12 | |
| *** thorst_ has quit IRC | 10:18 | |
| *** esberglu has joined #openstack-powervm | 10:28 | |
| *** esberglu has quit IRC | 10:33 | |
| *** thorst_ has joined #openstack-powervm | 11:19 | |
| *** thorst_ has quit IRC | 11:24 | |
| *** esberglu has joined #openstack-powervm | 11:31 | |
| *** esberglu has quit IRC | 11:36 | |
| *** jwcroppe has joined #openstack-powervm | 11:42 | |
| *** jwcroppe has quit IRC | 11:47 | |
| *** smatzek has joined #openstack-powervm | 11:52 | |
| -openstackstatus- NOTICE: Our OpenStack CI systems are stuck and no new jobs are submitted. Please do not recheck - and do not approve changes until this is fixed. | 12:01 | |
| *** thorst has joined #openstack-powervm | 12:20 | |
| *** thorst has quit IRC | 12:29 | |
| *** chas_ has joined #openstack-powervm | 12:38 | |
| *** chas_ has quit IRC | 12:43 | |
| *** chas_ has joined #openstack-powervm | 12:49 | |
| -openstackstatus- NOTICE: Our OpenStack CI system is coming back online again. Thanks for your patience. | 12:59 | |
| *** thorst has joined #openstack-powervm | 13:01 | |
| *** thorst_ has joined #openstack-powervm | 13:03 | |
| *** thorst has quit IRC | 13:05 | |
| *** thorst_ is now known as thorst | 13:12 | |
| *** svenkat has joined #openstack-powervm | 13:16 | |
| *** edmondsw has joined #openstack-powervm | 13:16 | |
| *** jwcroppe has joined #openstack-powervm | 13:29 | |
| *** jwcroppe has quit IRC | 13:29 | |
| *** jwcroppe has joined #openstack-powervm | 13:30 | |
| *** seroyer has joined #openstack-powervm | 13:34 | |
| *** jwcroppe has quit IRC | 13:34 | |
| *** tblakes has joined #openstack-powervm | 14:11 | |
| *** esberglu has joined #openstack-powervm | 14:14 | |
| *** smatzek has quit IRC | 14:20 | |
| *** jwcroppe has joined #openstack-powervm | 14:28 | |
| *** seroyer has quit IRC | 14:30 | |
| *** seroyer has joined #openstack-powervm | 14:40 | |
| openstackgerrit | Sridhar Venkat proposed openstack/networking-powervm: ProvisionRequest does not distinguish event source https://review.openstack.org/396467 | 14:41 |
| thorst | svenkat: did you get a chance to connect with efried on the ones above? | 14:43 |
| svenkat | yes… i juste uploaded second patch set based on Eric’s comments | 14:44 |
| thorst | awesome | 14:44 |
| *** apearson has joined #openstack-powervm | 14:45 | |
| openstackgerrit | Merged openstack/ceilometer-powervm: Update the init of the collector to take in a conf https://review.openstack.org/395918 | 14:47 |
| openstackgerrit | Merged openstack/nova-powervm: Fix rollback op's search call https://review.openstack.org/393801 | 14:47 |
| *** kriskend has joined #openstack-powervm | 14:54 | |
| *** AlexeyAbashkin has quit IRC | 15:11 | |
| *** smatzek has joined #openstack-powervm | 15:14 | |
| *** apearson has quit IRC | 15:31 | |
| thorst | esberglu: I think we should probably rev the version of nova-powervm to 3.0.1. Thoughts? | 15:40 |
| thorst | adreznec efried: ^^ | 15:40 |
| efried | The master branch? | 15:41 |
| adreznec | Probably stable/newton | 15:41 |
| esberglu | No the stable/newton | 15:41 |
| thorst | stable/newton. | 15:41 |
| adreznec | The master branch would be 4.0.0-dev now | 15:41 |
| adreznec | I guess that seems fine to me | 15:41 |
| *** seroyer has quit IRC | 15:42 | |
| *** tjakobs has joined #openstack-powervm | 15:44 | |
| esberglu | thorst: adreznec: Is that a go? | 15:46 |
| thorst | k. I'm seeing some patch set changes from svenkat that I think will warrant nova-powervm to later jump to 3.0.2 and networking-powervm to go to 3.0.1 | 15:46 |
| thorst | yeah, I think that's a go for the 3.0.1 nova-powervm stable/mitaka version update | 15:46 |
| adreznec | newton | 15:47 |
| adreznec | But yes | 15:47 |
| adreznec | Fire away | 15:47 |
| thorst | man...I screw that up ALL THE TIME | 15:48 |
| adreznec | thorst: just really, really liked mitaka | 15:48 |
| thorst | guess so | 15:49 |
| esberglu | thorst: Donw | 15:50 |
| esberglu | Done | 15:50 |
| thorst | u rock | 15:50 |
| thorst | efried: If possible, i think you'd take interest in my comments here: https://review.openstack.org/#/c/396467/2 | 15:51 |
| thorst | I talked to svenkat about it too | 15:51 |
| svenkat | thorst: I am working to get event source from nova… | 15:52 |
| thorst | svenkat: I put the exact LoC where it happens | 15:52 |
| svenkat | sure. thanks. | 15:52 |
| thorst | the detail is where you'd add something. maybe a 'type=' | 15:53 |
| thorst | and good find/debug on that issue | 15:53 |
| efried | thorst, that rebase is done - take a look. | 15:53 |
| thorst | looking now | 15:53 |
| efried | thorst, your comment on svenkat's change set is the same one I made ;-) | 15:54 |
| efried | great minds, and all that. | 15:54 |
| *** seroyer has joined #openstack-powervm | 15:55 | |
| *** smatzek has quit IRC | 15:57 | |
| *** smatzek has joined #openstack-powervm | 15:58 | |
| thorst | efried: excellent | 15:58 |
| thorst | getting used to these pok addresses is fun. | 15:59 |
| thorst | ... | 15:59 |
| *** apearson has joined #openstack-powervm | 16:02 | |
| thorst | efried: I'm good with that change | 16:08 |
| efried | thorst, so do we get esberglu to yank it into the CI env at this point? | 16:08 |
| thorst | I think he already has. | 16:09 |
| thorst | and he's fighting devstack issues now | 16:09 |
| thorst | esberglu: can you confirm? | 16:09 |
| *** efried1 has joined #openstack-powervm | 16:20 | |
| *** efried has quit IRC | 16:22 | |
| *** kotra03 has quit IRC | 16:25 | |
| *** k0da has joined #openstack-powervm | 16:30 | |
| esberglu | thorst: Are we talking about the remote patch? | 16:36 |
| efried1 | thorst, you good with 4487 yet? | 16:38 |
| efried1 | If PM comes after me, I'm deflecting 'em to you. | 16:39 |
| esberglu | thorst: And yeah I'm fighting devstack | 16:40 |
| esberglu | Devstack is basically my nemesis at this point | 16:41 |
| adreznec | Undercloud or overcloud? | 16:44 |
| esberglu | AIO devstack master | 16:45 |
| *** k0da has quit IRC | 16:51 | |
| *** seroyer has quit IRC | 17:02 | |
| efried1 | adreznec, any idea why this keeps failing to verify? https://review.openstack.org/#/c/391288/ | 17:10 |
| efried1 | I looked into the errors and they're wildly unrelated even to compute. Like, in the cinder driver and stuff. | 17:10 |
| *** mdrabe has joined #openstack-powervm | 17:11 | |
| adreznec | efried1: Let me take a look here | 17:13 |
| thorst | efried1: yeah, good with 4487 | 17:17 |
| thorst | esberglu: Anything we need to help with for devstack/ | 17:17 |
| efried1 | thorst, am I ready to merge stuff at this point? Infra all set up for it? | 17:18 |
| thorst | efried1: unclear | 17:18 |
| thorst | that's a dom q | 17:18 |
| efried1 | k, asking dclain. | 17:18 |
| efried1 | thorst, is there any need for APINotLocal exception anymore? Any code paths we want to block thusly? | 17:20 |
| efried1 | thorst, also, 4475, I put up a patch showing how it would work with hand-parsing to avoid circular imports. Not urgent, but on your list somewhere. | 17:22 |
| thorst | efried1: lets not pull APINotLocal yet... | 17:23 |
| thorst | that would require a full on scour | 17:23 |
| thorst | (cause we may need it in places we missed) | 17:23 |
| efried1 | thorst, huh? It was *just* introduced | 17:23 |
| thorst | will look at 4475 | 17:24 |
| thorst | efried1: o, well, cut it | 17:24 |
| efried1 | The only place it lives right now is in the UT. If we need to reintroduce it, we can. But I'd rather do that than leave it hanging around unused and forget to purge it. | 17:24 |
| thorst | meh, we can cut it | 17:25 |
| thorst | I like 4475 more | 17:25 |
| efried1 | REALLY? | 17:27 |
| efried1 | okay. | 17:27 |
| thorst | lol - personal preference | 17:28 |
| *** seroyer has joined #openstack-powervm | 17:46 | |
| *** smatzek has quit IRC | 17:51 | |
| *** kriskend has quit IRC | 17:51 | |
| *** esberglu has quit IRC | 17:51 | |
| *** tjakobs has quit IRC | 17:51 | |
| *** smatzek has joined #openstack-powervm | 17:52 | |
| *** esberglu has joined #openstack-powervm | 17:52 | |
| *** kriskend has joined #openstack-powervm | 17:52 | |
| *** tjakobs has joined #openstack-powervm | 17:52 | |
| adreznec | So efried1 I looked at that patch. The intel NFV CI is just hitting a timeout at 3 hours... not sure if that's a known problem or not | 18:06 |
| adreznec | The xenserver logs were gone | 18:07 |
| adreznec | I retriggered that to look | 18:07 |
| efried1 | adreznec, been retrying it for a week-plus | 18:07 |
| adreznec | It wasn't getting retriggered in any of your other retries | 18:07 |
| efried1 | Yeah, the xenserver logs are never there by the time I go look. | 18:07 |
| efried1 | Oh. | 18:07 |
| adreznec | It was looking specifically for 'xenserver: recheck' I guess | 18:07 |
| efried1 | 'recheck' doesn't hit everything? | 18:07 |
| efried1 | Booooooh gus. | 18:07 |
| adreznec | Guess not for that one | 18:07 |
| *** chas_ has quit IRC | 18:30 | |
| *** chas_ has joined #openstack-powervm | 18:30 | |
| esberglu | thorst: adreznec: I found the source of the devstack issue. Some deprecated exceptions were retired. I think we just need to change the import from | 18:32 |
| esberglu | from neutron.common import exceptions | 18:32 |
| esberglu | to | 18:32 |
| esberglu | from neutron_lib import exceptions | 18:32 |
| thorst | esberglu: yeah, I noticed that in a change from svenkat | 18:33 |
| *** smatzek_ has joined #openstack-powervm | 18:33 | |
| thorst | I in fact noted we should do that in a separate change set | 18:33 |
| thorst | did you want to fire that off and I can +2 it? | 18:33 |
| esberglu | Yeah. I know which file its coming from but I will check and see if we are using the incorrect import anywhere else | 18:33 |
| svenkat | thorst: i will take it off of my change set. | 18:34 |
| *** chas_ has quit IRC | 18:34 | |
| *** smatzek has quit IRC | 18:36 | |
| openstackgerrit | Eric Berglund proposed openstack/networking-powervm: Import exceptions from neutron_lib https://review.openstack.org/396728 | 18:38 |
| esberglu | thorst: ^^ | 18:39 |
| thorst | esberglu: thx | 18:43 |
| * thorst that feeling when the nova cores tell others to 'do what nova-powervm does' | 18:43 | |
| esberglu | efried1: That remote change is good to go right? | 18:43 |
| efried1 | esberglu: Well, no. | 18:44 |
| efried1 | Hold on, let me get a new patch set up. | 18:44 |
| thorst | efried1 esberglu: good for CI, not yet ready to be in pypowervm | 18:44 |
| efried1 | It'll have UT incomplete, but it'll be missing a couple of bugs you would introduce otherwise... | 18:44 |
| thorst | finishing UT I think | 18:44 |
| esberglu | Yeah thats what I meant | 18:44 |
| efried1 | thorst, no, there's a bug in the vopt path. | 18:44 |
| thorst | ahh, then devstack would fail there | 18:45 |
| esberglu | So I need to hold off for now? | 18:45 |
| efried1 | esberglu, no, I just posted the update. thorst, you want to pull in the 'ensure close' bit too, right? | 18:46 |
| thorst | efried1: yep. | 18:46 |
| efried1 | So esberglu, pull 4504 and you'll get both. | 18:46 |
| esberglu | Cool. I will redeploy with that and hopefully by then the networking-powervm fix will be in and devstack will work | 18:46 |
| efried1 | This is in release branch, btw esberglu. | 18:47 |
| efried1 | not develop | 18:47 |
| efried1 | in pypowervm | 18:47 |
| openstackgerrit | Merged openstack/networking-powervm: Import exceptions from neutron_lib https://review.openstack.org/396728 | 18:55 |
| thorst | esberglu: you're merged :-) | 18:55 |
| openstackgerrit | Sridhar Venkat proposed openstack/nova-powervm: ProvisionRequest does not distinguish event source https://review.openstack.org/396739 | 19:10 |
| esberglu | thorst: adreznec: efried: Right now the CI is using the develop branch for pypowervm. Will there be impacts if I change it to release? | 19:16 |
| esberglu | Since the local2remote patch is on develop and that new patch from 4504 is release. | 19:16 |
| esberglu | Or does it even matter? Maybe I can apply anyways? | 19:18 |
| esberglu | Looks like it will apply anyways | 19:25 |
| thorst | esberglu: yep...should be able to just apply since branches are near identical since they branched recently | 19:26 |
| esberglu | thorst: Verified locally, management playbook is running right now | 19:31 |
| efried1 | thorst, did you get a chance to grok this subprocess._eintr_retry_call thing? | 19:46 |
| efried1 | kinda threw that over the fence in the middle of this kafuffle, but we didn't really talk about it. We should. | 19:46 |
| thorst | efried1: haven't looked yet. | 19:47 |
| efried1 | thorst, it won't be obvious what's going on by looking at it. | 19:47 |
| efried1 | We had one failure yesterday where basically open() got a signal, EINTR, and that blew up the process. | 19:47 |
| efried1 | I did some research: apparently those signals "just happen" from time to time, and we're supposed to ignore them and retry. | 19:49 |
| efried1 | They fixed this at the lowest level in py3 by doing exactly what _eintr_retry_call does | 19:49 |
| efried1 | But in py2, we're still subject to this behavior, so we're supposed to do that retry ourselves. | 19:49 |
| efried1 | It's a bit ugly, cause instead of just being able to make syscalls like open() and close() and read() and write(), we have to wrap 'em in this retry nonsense. | 19:50 |
| thorst | whoa...ok | 19:50 |
| efried1 | ...or we could just take the risk and accept these failures when they happen. But that doesn't seem very robust. | 19:50 |
| efried1 | Now, of course, sonar hates the fact that I'm using a private member of a foreign thingy (subprocess._eintr_retry_call) | 19:51 |
| efried1 | The alternative is to dup the code inline as a util or something. | 19:51 |
| efried1 | It's only like 7LOC or so. | 19:51 |
| efried1 | How do you feel about that? | 19:51 |
| thorst | I'm ok with that for now | 19:54 |
| thorst | assuming I don't change my mind after i see it | 19:55 |
| efried1 | rgr | 19:56 |
| efried1 | thorst, FYI: https://www.python.org/dev/peps/pep-0475/ | 19:57 |
| thorst | my word...so that could be our whole 'this is why glance is failing' thing? | 20:12 |
| kriskend | so these signals just randomly happen? Is this a version of chaos monkey? | 20:18 |
| *** tblakes has quit IRC | 20:21 | |
| esberglu | thorst: efried: Might be a while before runs go through with that patch | 20:25 |
| esberglu | 100% (115572/115572), 45.06 MiB | 96.00 KiB/s | 20:25 |
| thorst | yikes... | 20:30 |
| *** thorst has quit IRC | 20:31 | |
| *** thorst has joined #openstack-powervm | 20:32 | |
| *** thorst has quit IRC | 20:36 | |
| *** tblakes has joined #openstack-powervm | 20:37 | |
| *** smatzek_ has quit IRC | 20:41 | |
| *** jwcroppe has quit IRC | 20:44 | |
| *** jwcroppe has joined #openstack-powervm | 20:45 | |
| *** jwcroppe has quit IRC | 20:49 | |
| efried1 | thorst, kriskend, seroyer - yes, and may in fact make the following comment obsolete: | 20:50 |
| efried1 | # Very rarely (think one in a thousand) there appears to be a | 20:50 |
| efried1 | # hiccup in the upload processing. A simple retry mechanism | 20:50 |
| efried1 | # should be enough to push it through | 20:50 |
| efried1 | May even make _wrap_user_stream unnecessary. | 20:51 |
| *** thorst has joined #openstack-powervm | 20:54 | |
| *** thorst has quit IRC | 20:58 | |
| *** svenkat has quit IRC | 21:03 | |
| *** thorst has joined #openstack-powervm | 21:04 | |
| *** jwcroppe has joined #openstack-powervm | 21:07 | |
| *** jwcroppe has joined #openstack-powervm | 21:07 | |
| *** k0da has joined #openstack-powervm | 21:57 | |
| efried1 | adreznec, how do I find out what magic incantation I need to recheck these other failing CIs? | 22:00 |
| adreznec | It should say in the comments from the CI systems | 22:00 |
| adreznec | You might have to hit the toggle CI button | 22:00 |
| adreznec | To show the comments | 22:00 |
| efried1 | ah | 22:00 |
| *** seroyer_ has joined #openstack-powervm | 22:01 | |
| *** seroyer has quit IRC | 22:02 | |
| *** seroyer_ is now known as seroyer | 22:02 | |
| *** kriskend has quit IRC | 22:07 | |
| *** edmondsw has quit IRC | 22:12 | |
| *** tblakes has quit IRC | 22:34 | |
| *** tjakobs has quit IRC | 22:35 | |
| *** esberglu has quit IRC | 22:41 | |
| *** esberglu has joined #openstack-powervm | 22:41 | |
| *** seroyer has quit IRC | 22:44 | |
| *** esberglu has quit IRC | 22:46 | |
| *** mdrabe has quit IRC | 22:58 | |
| *** chas_ has joined #openstack-powervm | 23:09 | |
| *** chas_ has quit IRC | 23:13 | |
| *** apearson has quit IRC | 23:15 | |
| *** tjakobs has joined #openstack-powervm | 23:15 | |
| *** tjakobs has quit IRC | 23:24 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!