| *** chhagarw has joined #openstack-powervm | 04:13 | |
| *** chhavi__ has joined #openstack-powervm | 04:42 | |
| *** chhagarw has quit IRC | 04:46 | |
| openstackgerrit | Chhavi Agarwal proposed openstack/nova-powervm master: iSCSI Live Migration Support https://review.openstack.org/567575 | 06:04 |
|---|---|---|
| *** chhavi__ has quit IRC | 08:12 | |
| *** openstack has joined #openstack-powervm | 09:27 | |
| *** ChanServ sets mode: +o openstack | 09:27 | |
| *** openstackgerrit has joined #openstack-powervm | 10:34 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova-powervm stable/pike: Fix sphinx-docs job for stable branch https://review.openstack.org/567823 | 10:34 |
| openstackgerrit | Stephen Finucane proposed openstack/nova-powervm stable/ocata: Fix sphinx-docs job for stable branch https://review.openstack.org/567824 | 10:37 |
| *** edmondsw has joined #openstack-powervm | 12:08 | |
| *** efried is now known as fried_rice | 12:35 | |
| *** openstackstatus has joined #openstack-powervm | 13:12 | |
| *** ChanServ sets mode: +v openstackstatus | 13:12 | |
| -openstackstatus- NOTICE: Due to a Zuul outage, patches uploaded to Gerrit between 09:00UTC and 12:50UTC, were not properly added to Zuul. Please recheck any patches during this window and apologies for the inconvenience. | 13:15 | |
| *** esberglu has joined #openstack-powervm | 13:44 | |
| *** tjakobs has joined #openstack-powervm | 13:47 | |
| openstackgerrit | Eric Berglund proposed openstack/nova-powervm stable/pike: Stringify instance UUID https://review.openstack.org/567616 | 14:01 |
| esberglu | edmondsw: fried_rice: Wanna push through https://review.openstack.org/#/c/567823/? | 14:25 |
| esberglu | It's failing PowerVM CI because of circular dependency on https://review.openstack.org/#/c/567616/ | 14:26 |
| esberglu | But just a tox change so we're fine | 14:26 |
| edmondsw | esberglu do we need to cherry-pick 567616 back to ocata as well? | 14:31 |
| edmondsw | I'm guessing that's why https://review.openstack.org/#/c/567824/ CI failed | 14:31 |
| edmondsw | (without looking) | 14:31 |
| fried_rice | esberglu: was gonna ask same | 14:32 |
| edmondsw | or did we just merge a sphinx change that we shouldn't have... ? | 14:33 |
| esberglu | edmondsw: fried_rice: Huh that actually passes the tempest run, one sec | 14:33 |
| esberglu | edmondsw: fried_rice: No the sphinx change for pike is good | 14:33 |
| edmondsw | esberglu it looks like the tests on https://review.openstack.org/#/c/567824/ all passed... so why the -1 ? | 14:34 |
| esberglu | edmondsw: Yes exactly, I'm looking now | 14:35 |
| esberglu | edmondsw: Must be a problem with the post_stack_vm_cleaner.py on ocata | 14:38 |
| esberglu | edmondsw: Also looks like the very end of the jenkins console log is being cut off, so I can't see why | 14:38 |
| esberglu | I'll have to do a manual run, might not get to it today though | 14:39 |
| esberglu | Well I guess it makes sense that the jenkins console log is cut off, since process logs gets run before post_stack_vm_cleaner | 14:41 |
| openstackgerrit | Merged openstack/nova-powervm stable/pike: Fix sphinx-docs job for stable branch https://review.openstack.org/567823 | 14:45 |
| openstackgerrit | Merged openstack/nova-powervm stable/pike: Stringify instance UUID https://review.openstack.org/567616 | 14:45 |
| openstackgerrit | Eric Fried proposed openstack/nova-powervm stable/queens: Fix boot volume connectivity type discovery https://review.openstack.org/567418 | 14:49 |
| *** openstackgerrit has quit IRC | 14:49 | |
| *** openstackgerrit has joined #openstack-powervm | 14:50 | |
| openstackgerrit | Eric Berglund proposed openstack/nova-powervm stable/pike: DNM: CI Check - Pike https://review.openstack.org/516790 | 14:50 |
| esberglu | edmondsw: Oh missed your other question, we do not need this for ocata, the oslo.vo changes were only pike and later | 14:52 |
| edmondsw | esberglu ok good | 14:52 |
| edmondsw | esberglu CI passed on 567418 before the rebase, looks good to me... take a quick look and +2/+A ? | 14:55 |
| edmondsw | esberglu efried once that's merged, I will propose a new stable/queens release to include all these recent fixes | 14:56 |
| edmondsw | we should do the same for pike once we get things working and merged there | 14:57 |
| edmondsw | fried_rice I keep calling you efried... | 15:00 |
| edmondsw | fried_rice is https://bugs.launchpad.net/nova-powervm/+bug/1769648 an issue with pike? | 15:01 |
| openstack | Launchpad bug 1769648 in nova-powervm "Can't create IBMi VM with RBD backend storage" [Medium,Fix released] - Assigned to Eric Fried (efried) | 15:01 |
| fried_rice | edmondsw: I didn't think we supported RBD in pike | 15:02 |
| fried_rice | but let me see if that bad conditional exists there... | 15:03 |
| edmondsw | fried_rice yep, looks like rbd was added in queens | 15:04 |
| fried_rice | edmondsw: But yeah, theoretically the problem (uninitialized variable) still exists in pike: https://github.com/openstack/nova-powervm/blob/stable/queens/nova_powervm/virt/powervm/driver.py#L1825-L1833 | 15:05 |
| fried_rice | edmondsw: The fix certainly shouldn't hurt there. But not sure if the risk is justified. | 15:05 |
| edmondsw | fried_rice how well do we understand the case where this bug would appear? | 15:07 |
| fried_rice | whoops, hold on, I went after the wrong branch. | 15:07 |
| fried_rice | yeah, same: https://github.com/openstack/nova-powervm/blob/stable/pike/nova_powervm/virt/powervm/driver.py#L1808-L1816 | 15:08 |
| fried_rice | edmondsw: Unfortunately, we friggin don't. | 15:08 |
| fried_rice | edmondsw: It depends on the bdms that come at us, and NOBODOY but NOBODY seems to have any freaking clue what those can actually look like irl. | 15:08 |
| edmondsw | mdrabe any thoughts here? | 15:09 |
| edmondsw | talking about whether to backport fix for https://bugs.launchpad.net/nova-powervm/+bug/1769648 to pike | 15:09 |
| openstack | Launchpad bug 1769648 in nova-powervm "Can't create IBMi VM with RBD backend storage" [Medium,Fix released] - Assigned to Eric Fried (efried) | 15:09 |
| edmondsw | the fix wasn't specific to rbd (which wasn't in pike) as far as we know | 15:09 |
| fried_rice | As I say, the fix at https://review.openstack.org/#/c/566420/ *shouldn't* be able to hurt anything, as it's backward compatible afaict. | 15:09 |
| mdrabe | What the BDMs look like? | 15:10 |
| fried_rice | but then again, maybe we wait to fix it until it breaks somebody. | 15:10 |
| edmondsw | we should try to be more proactive than that | 15:10 |
| edmondsw | that's actually one of the requirements for follows:stable-policy (which we don't technically do yet...) | 15:10 |
| esberglu | fried_rice: edmondsw: 6552 when you get a second | 15:10 |
| edmondsw | mdrabe I think the basic question is can we determine what caused us to go down the code path where that bug occurred, so we can understand when/whether it's gonna happen on pike | 15:11 |
| fried_rice | esberglu: done | 15:12 |
| esberglu | fried_rice: tx | 15:12 |
| edmondsw | already merged when I got to it :) | 15:13 |
| edmondsw | I'm glad I'm not facing you guys in a wild west showdown... ;) | 15:14 |
| mdrabe | edmondsw: Looks like that bug was something that could've been caught it UT / review maybe no? | 15:16 |
| mdrabe | It's a bug in pike no less | 15:16 |
| mdrabe | Can a fix be backported? | 15:17 |
| edmondsw | mdrabe yep... question is whether it's worth the risk to backport it to pike or not | 15:17 |
| edmondsw | we already backported to queens | 15:17 |
| mdrabe | Hmm what's the risk we're concerned about? | 15:18 |
| edmondsw | breaking something? | 15:18 |
| mdrabe | Seems kinda broken already | 15:19 |
| edmondsw | I mean breaking something else as we fix this | 15:19 |
| edmondsw | I don't think that's overly likely... so if we think someone is gonna hit this on pike I would vote to backport it | 15:20 |
| edmondsw | but if we think nobody's really going to ever hit this on pike, there's little point | 15:20 |
| edmondsw | hence the question of whether we think that code path will be exercised in pike, and when? | 15:20 |
| openstackgerrit | Merged openstack/nova-powervm stable/queens: Fix boot volume connectivity type discovery https://review.openstack.org/567418 | 15:20 |
| edmondsw | I'm going to be tagging new pike and queens releases soon to pick up some other fixes. If we are going to need this for pike, I'd want to do it soon so I don't have to do that again | 15:21 |
| fried_rice | edmondsw: Looking at the IBMi stuff earlier, I noticed that, at least for IBMi, we expect that sucker to return 'npiv', not 'fibre_channel'. So at least the docstring is... questionable. | 15:21 |
| fried_rice | edmondsw: That's kind of unrelated though. | 15:22 |
| edmondsw | fried_rice lovely | 15:23 |
| edmondsw | fried_rice the docstring said 'npiv' before your change... why did you change it to 'fibre_channel' ? | 15:24 |
| fried_rice | shit, I think because of the unit test. | 15:25 |
| mdrabe | edmondsw: Tough to say if the path could happen in pike, but it is there. I'd say at least boot_conn_type could be predefined at defaulted. | 15:25 |
| mdrabe | and* | 15:25 |
| fried_rice | mdrabe: edmondsw: It breaks if the bdm['connection_info'][['data'] contains neither 'connection-type' nor 'driver_volume_type'. Which apparently turned out to be the case for RBD (why??) but again, no idea when/why/how it would be the case otherwise (in pike). | 15:25 |
| fried_rice | The fix essentially just makes us default to 'vscsi' when ^ happens, because that seemed to be the intent of the calling method. But even that doesn't really feel right. | 15:27 |
| fried_rice | I don't know where the BDM comes from in the first place. If it's in any part populated by data we get from REST, then I would be quick to posit that we're not receiving/transferring something from REST in the RBD case, which is why those fields wind up absent. | 15:28 |
| mdrabe | efried: Looks like the BDM is created in the API, but I think the majority of the connection_info comes from cinder | 15:42 |
| mdrabe | Also I think connection-type is a pvc thing, and driver_volume_type is community | 15:43 |
| mdrabe | edmondsw: Does pure nova-powervm work with vscsi? | 15:54 |
| edmondsw | mdrabe if connection-type is a PowerVC thing, it shouldn't be in nova-powervm code... | 15:54 |
| mdrabe | edmondsw: Only reason I say that is because I don't see it anywhere else | 15:57 |
| openstackgerrit | Eric Fried proposed openstack/nova-powervm stable/ocata: Fix sphinx-docs job for stable branch https://review.openstack.org/567824 | 16:21 |
| openstackgerrit | Eric Fried proposed openstack/nova-powervm stable/pike: Use "$*" instead of "$@" in tox_install.sh https://review.openstack.org/567889 | 16:24 |
| *** fried_rice is now known as fried_rolls | 16:31 | |
| openstackgerrit | Matthew Edmonds proposed openstack/nova-powervm stable/queens: Sanitize the config drive UUID https://review.openstack.org/567894 | 16:43 |
| openstackgerrit | Merged openstack/nova-powervm master: Refactor _vol_drv_iter to use bdm.is_volume https://review.openstack.org/566888 | 16:55 |
| openstackgerrit | Merged openstack/nova-powervm stable/pike: Use "$*" instead of "$@" in tox_install.sh https://review.openstack.org/567889 | 16:58 |
| *** openstackstatus has quit IRC | 17:00 | |
| *** openstack has joined #openstack-powervm | 17:03 | |
| *** ChanServ sets mode: +o openstack | 17:03 | |
| edmondsw | esberglu ocata CI just failed here similar to before... tests pass but -1 vote: https://review.openstack.org/#/c/567824/ | 17:31 |
| esberglu | edmondsw: Yep I know I will look into it when I have a chance to do a manual run. I'm pretty sure post_stack_vm_cleaner.py is failing, but the output gets cut off | 17:32 |
| esberglu | And I haven't redeployed CI to pick up the change to fix that yet | 17:33 |
| edmondsw | do you always have to redeploy to make changes? Seems like we should be able to ssh in and make a change manually, no? | 17:33 |
| esberglu | edmondsw: Yeah that's what I'm going to do for this one, I use the term redeploy loosely | 17:42 |
| edmondsw | esberglu just noticed https://review.openstack.org/#/c/560519 has been sitting a while... plz look when you get a minute | 18:07 |
| esberglu | edmondsw: ack | 18:08 |
| esberglu | edmondsw: Ah okay I see what's happening on ocata. The pypowervm version used in ocata is too low to be compatible with post_stack_vm_cleaner.py | 18:12 |
| esberglu | Wow this has been broken for a while then, we must not have done anything in ocata for a long time | 18:13 |
| edmondsw | why does post_stack_vm_cleaner.py need a newer version? | 18:13 |
| esberglu | It uses the newer power off code | 18:15 |
| esberglu | from pypowervm.tasks import power_opts | 18:15 |
| esberglu | power_opts doesn't exist in 1.0.0.4 | 18:16 |
| esberglu | We updated the script a while back to use the newer stuff, I missed that I didn't work on ocata | 18:17 |
| esberglu | edmondsw: I'm just gonna pull the latest pypowervm if on ocata before running it | 18:24 |
| esberglu | Gotta hop offline for a few, will put up a patch when I'm back | 18:25 |
| *** esberglu has quit IRC | 18:25 | |
| *** esberglu has joined #openstack-powervm | 18:26 | |
| *** esberglu has quit IRC | 18:31 | |
| *** esberglu has joined #openstack-powervm | 18:44 | |
| *** fried_rolls is now known as fried_rice | 19:03 | |
| esberglu | edmondsw: fried_rice: 6553 for ocata run fix | 19:10 |
| fried_rice | esberglu: reviewed, needs fix | 19:12 |
| esberglu | fried_rice: You care whether develop or 1.1.15? | 19:12 |
| esberglu | Ocata won't be around much longer, probably doesn't matter | 19:13 |
| fried_rice | no | 19:13 |
| esberglu | fried_rice: Done | 19:13 |
| fried_rice | esberglu: Is a pull necessary after checking out develop? | 19:13 |
| esberglu | fried_rice: I don't think so, it should be new enough from the image template build. If we want the latest develop yes, but if we're missing the last commit or two it won't matter | 19:16 |
| fried_rice | I just didn't know how the repo was cloned to begin with. If you say it's good, it's good. | 19:16 |
| esberglu | Okay, no pull then | 19:17 |
| fried_rice | +2 | 19:17 |
| openstackgerrit | Merged openstack/nova-powervm stable/ocata: Fix sphinx-docs job for stable branch https://review.openstack.org/567824 | 19:22 |
| openstackgerrit | Merged openstack/nova-powervm stable/queens: Sanitize the config drive UUID https://review.openstack.org/567894 | 19:22 |
| openstackgerrit | Merged openstack/networking-powervm master: Uncap eventlet and sync requirements https://review.openstack.org/560519 | 19:35 |
| edmondsw | esberglu using develop will probably be fine as long as we don't break something | 19:38 |
| esberglu | edmondsw: One more way to catch it if we do :) | 19:39 |
| edmondsw | :) | 19:39 |
| esberglu | edmondsw: They changed the EOL process right? | 19:40 |
| edmondsw | esberglu yeah, we live in an "extended maintenance" world now | 19:40 |
| esberglu | edmondsw: So when do we stop testing in CI then? | 19:41 |
| edmondsw | good question :) | 19:42 |
| edmondsw | fried_rice do we need a pip10 stable fix for queens? | 19:45 |
| edmondsw | we've merged patches only for pike and ocata | 19:45 |
| edmondsw | the sphinx change | 19:45 |
| fried_rice | It's possible they successfully blacklisted pip10 in queens. Stephen is who I would ask, if I didn't want to go trying to dig myself. And he may be gone for the evening. | 19:47 |
| edmondsw | oh, we don't have tox_install.sh in queens | 19:47 |
| fried_rice | ah | 19:47 |
| fried_rice | was gonna say, the fix wouldn't hurt us either way, but if the file doesn't exist, then n/a. | 19:48 |
| fried_rice | edmondsw: btw, I punt on the vnc thing - any ideas? | 19:48 |
| edmondsw | fried_rice none... I haven't ever looked at that stuff | 19:49 |
| edmondsw | was hoping you would know | 19:49 |
| edmondsw | or that thorst would answer | 19:49 |
| fried_rice | Yeah, I think thorst was the one to get that stuff working originally. | 19:49 |
| *** tjakobs has quit IRC | 21:30 | |
| *** edmondsw has quit IRC | 21:58 | |
| *** edmondsw has joined #openstack-powervm | 22:02 | |
| *** edmondsw has quit IRC | 22:06 | |
| *** fried_rice is now known as efried | 22:16 | |
| *** edmondsw has joined #openstack-powervm | 22:29 | |
| *** esberglu has quit IRC | 22:30 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!