| *** ataraday_ has quit IRC | 00:03 | |
| *** goldyfruit_ has quit IRC | 00:20 | |
| *** goldyfruit_ has joined #openstack-lbaas | 01:05 | |
| johnsom | Well, on the gate front, I +A'd the stable/train fix as it will pass. I also rechecked and poked for a review of the element fix in DIB. I still think our patches will beat a new DIB being available and UC updated, but we might as well attack it on both fronts. | 01:33 |
|---|---|---|
| *** ricolin has joined #openstack-lbaas | 01:46 | |
| *** armax has quit IRC | 01:57 | |
| *** armax has joined #openstack-lbaas | 02:31 | |
| *** ricolin has quit IRC | 03:03 | |
| rm_work | Yep | 03:21 |
| openstackgerrit | Merged openstack/octavia stable/train: Fix gate issue with DIB https://review.opendev.org/695982 | 03:25 |
| johnsom | Ha, well, there we go | 03:26 |
| johnsom | Now someone can recheck the master patch | 03:26 |
| rm_work | rechecked | 03:36 |
| rm_work | will +A when it's done | 03:36 |
| rm_work | johnsom: the DIB side did merge.... so err... | 03:40 |
| rm_work | johnsom: i forget if we need to wait for a release, or actually if we're just good already | 03:40 |
| johnsom | It has to release and upper-constraints updated | 03:41 |
| rm_work | k we'll see who wins the race | 03:46 |
| rm_work | if we merge this and then they merge that, do we just revert? | 03:46 |
| rm_work | i think it shouldn't break anything? | 03:47 |
| johnsom | Correct | 03:50 |
| *** hongbin has joined #openstack-lbaas | 04:00 | |
| *** gthiemonge has quit IRC | 04:05 | |
| *** gthiemon1e has joined #openstack-lbaas | 04:05 | |
| *** ricolin has joined #openstack-lbaas | 04:08 | |
| *** ricolin has quit IRC | 04:29 | |
| *** ccamposr__ has joined #openstack-lbaas | 04:33 | |
| *** ccamposr has quit IRC | 04:36 | |
| openstackgerrit | Sam Morrison proposed openstack/octavia-tempest-plugin master: Add new scenario test to create LB in specific AZ https://review.opendev.org/695349 | 04:46 |
| *** gcheresh_ has joined #openstack-lbaas | 05:25 | |
| *** gthiemon1e is now known as gthiemonge | 05:36 | |
| *** tkajinam has quit IRC | 05:37 | |
| *** tkajinam has joined #openstack-lbaas | 05:38 | |
| rm_work | johnsom: yeah grenade is still failing on this fix lol | 05:39 |
| rm_work | we need to combine it with the PyYAML fix? | 05:39 |
| rm_work | i forget what that was, all of these issues are blending together | 05:40 |
| *** gcheresh_ has quit IRC | 05:44 | |
| johnsom | Pyyaml was the disable novnc patch | 05:46 |
| rm_work | k so we need to merge the two | 05:47 |
| rm_work | into one CR | 05:47 |
| rm_work | i can do that | 05:47 |
| johnsom | Feel free to mess with those, I am just about to get some sleep | 05:47 |
| rm_work | k | 05:48 |
| openstackgerrit | Adam Harwell proposed openstack/octavia master: Fix gate issue with DIB and Nova console services https://review.opendev.org/695947 | 05:51 |
| rm_work | done | 05:51 |
| rm_work | hopefully THAT can pass | 05:51 |
| rm_work | I'll track it and if it passes I'm going to +A it | 05:51 |
| *** hongbin has quit IRC | 06:18 | |
| *** sapd1 has joined #openstack-lbaas | 06:28 | |
| *** pcaruana has joined #openstack-lbaas | 06:32 | |
| *** sapd1 has quit IRC | 06:57 | |
| *** gcheresh_ has joined #openstack-lbaas | 07:45 | |
| *** gcheresh has joined #openstack-lbaas | 07:49 | |
| *** gcheresh_ has quit IRC | 07:50 | |
| *** maciejjozefczyk has joined #openstack-lbaas | 08:02 | |
| *** tesseract has joined #openstack-lbaas | 08:16 | |
| *** tkajinam has quit IRC | 08:19 | |
| *** ccamposr__ has quit IRC | 08:41 | |
| *** ccamposr__ has joined #openstack-lbaas | 08:42 | |
| openstackgerrit | Gregory Thiemonge proposed openstack/octavia master: Use 2048-bits keys for devstack certificates https://review.opendev.org/696037 | 09:00 |
| openstackgerrit | Gregory Thiemonge proposed openstack/octavia master: Use 2048-bits keys for devstack certificates https://review.opendev.org/696037 | 09:36 |
| *** abaindur has quit IRC | 09:47 | |
| *** ccamposr__ has quit IRC | 10:16 | |
| *** ivve has joined #openstack-lbaas | 10:35 | |
| *** ccamposr has joined #openstack-lbaas | 10:47 | |
| *** takamatsu has quit IRC | 10:55 | |
| *** ccamposr has quit IRC | 11:08 | |
| *** rcernin has quit IRC | 11:40 | |
| *** goldyfruit_ has quit IRC | 13:08 | |
| rm_work | johnsom: it still didn't pass grenade T_T another issue maybe, looking | 13:29 |
| rm_work | same PyYAML issue | 13:36 |
| rm_work | removing the nova console stuff didn't actually fix it | 13:36 |
| rm_work | probably something else pulls it in too | 13:36 |
| *** tkajinam has joined #openstack-lbaas | 14:00 | |
| rm_work | johnsom: i'm about to disable voting on our grenade >_> | 14:03 |
| rm_work | this is stupid | 14:03 |
| openstackgerrit | Adam Harwell proposed openstack/octavia master: Fix gate issue with DIB and Migrate grenade to PY3 https://review.opendev.org/695947 | 14:31 |
| openstackgerrit | Brian Haley proposed openstack/octavia master: Make octavia-grenade job use python3 https://review.opendev.org/693486 | 14:34 |
| *** goldyfruit has joined #openstack-lbaas | 14:36 | |
| *** goldyfruit_ has joined #openstack-lbaas | 14:38 | |
| *** TrevorV has joined #openstack-lbaas | 14:40 | |
| *** goldyfruit has quit IRC | 14:41 | |
| rm_work | lol | 14:45 |
| rm_work | brian is already doing that but i need to do it earlier :D | 14:45 |
| rm_work | ugh but it looks like his change requires another one in the grenade repo that isn't merged yet, FML | 14:48 |
| rm_work | what is his IRC nick? | 14:49 |
| haleyb | rm_work: hi | 14:49 |
| rm_work | ahh there we go | 14:49 |
| rm_work | was trying a bunch of combinations and failing :D | 14:50 |
| haleyb | rm_work: that other change i did actually didn't fix the issue, let me find int | 14:50 |
| rm_work | yeah so we need to get this fixed because now we are sunk | 14:50 |
| haleyb | https://review.opendev.org/#/c/695466/ | 14:50 |
| rm_work | we have two active issues that are breaking our gates | 14:50 |
| rm_work | yeah i was reading that | 14:50 |
| rm_work | i am about 30 seconds away from proposing a patch to just *disable* our grenade until this shit gets sorted out | 14:51 |
| haleyb | i had a depends on but it didn't work, was going to ping gmann, but you seem to be hitting yet a different problem with pyaml | 14:51 |
| haleyb | rm_work: so what's your latest theory? | 14:52 |
| rm_work | the py3 thing | 14:52 |
| rm_work | but i guess doing that requires more work than just enabling py3 on the grenade? | 14:52 |
| rm_work | [06:24:30] frickler:rm_work: so in fact the old stack is being set up with use_python3=false, the new one with true, this breaks. | 14:52 |
| haleyb | oh, yuck | 14:53 |
| rm_work | soooo | 14:54 |
| rm_work | yeah ok, 30 seconds are about up, proposing a patch to turn grenade to non-voting until it's fixed | 14:54 |
| haleyb | is this affecting other repos as well? or we just have some set of libraries that aren't playing nice? | 14:55 |
| rm_work | unclear | 14:57 |
| rm_work | well, the pyyaml thing is affecting designate too maybe? | 14:57 |
| rm_work | [06:11:39] frickler:rm_work: I think we are seeing the same issue in designate, the reason for the issue, afaict, seems to be that capped pip is installed only for py2, not for py3, and then with the newer pip, some things fail | 14:58 |
| haleyb | i think we saw this with ovn as well perhaps | 14:58 |
| *** tkajinam has quit IRC | 15:19 | |
| openstackgerrit | Brian Haley proposed openstack/octavia master: Make octavia-grenade job non-voting https://review.opendev.org/696107 | 15:21 |
| haleyb | rm_work: big hammer ^^ | 15:22 |
| rm_work | yeah that's going to have to be combined with the fix for DIB or all the OTHER jobs will fail too | 15:22 |
| haleyb | perfect week for a train wreck | 15:23 |
| rm_work | i'm just waiting for the rest of the jobs to finish running on this CR so they record their status | 15:23 |
| rm_work | and then i can put it on that one | 15:23 |
| rm_work | yeah this started last week, we've been poking at it since thursday I think? i am getting super tired of it still being a problem | 15:24 |
| haleyb | yes, i was poking last week, taking friday off didn't help move it along... it's a bit of a mess now | 15:26 |
| rm_work | i bet you're going to take not just friday, but even *thursday* off this week too, aren't you? T_T | 15:31 |
| haleyb | rm_work: and part of wednesday if i can, but i'll work on this to get the gates working before i leave | 15:32 |
| rm_work | heh, whatever | 15:32 |
| rm_work | if the gates are still broken wednesday night, i'll just spend most of thursday drinking :D | 15:33 |
| rm_work | (i mean, i probably would anyway, but then i'd have a better excuse) | 15:33 |
| haleyb | eat turkey; drink; repeat | 15:37 |
| cgoncalves | had turkey and mac and cheese last weekend at an American friend's house. also pumpkin cookies and pie. experienced full Thanksgiving menu, I guess | 15:42 |
| rm_work | stuffing? cranberry sauce? | 15:43 |
| rm_work | baked yams with those tiny marshmallows for some reason? | 15:43 |
| cgoncalves | yes and yes | 15:43 |
| haleyb | all you need is an (american) football game to watch and you'd have the full Thanksgiving experience | 15:43 |
| rm_work | noice | 15:43 |
| cgoncalves | will Trump pardon a turkey this year? :D | 15:45 |
| rm_work | two apparently | 15:45 |
| rm_work | i watched the livestream unveiling of the two turkeys like an hour or two ago <_< | 15:45 |
| cgoncalves | oh Adam, you're so american... | 15:46 |
| rm_work | i don't even know WHY, it just happened | 15:46 |
| rm_work | yeah ok my change got to the point where it ran into the osc issue | 15:49 |
| *** ccamposr has joined #openstack-lbaas | 15:53 | |
| *** spatel has joined #openstack-lbaas | 15:54 | |
| dulek | Hi there! Is dib compatible with Python 3? | 15:57 |
| dulek | We're switched kuryr-kubernetes gates to it recently, but we use test builds of Amphora. | 15:57 |
| dulek | But on local DevStack deployment I'm getting ImportError: No module named yaml. | 15:58 |
| *** ivve has quit IRC | 16:02 | |
| johnsom | dulek Yes, DIB is fully compatible with Python3, we have been running gate tests with it for over a year. | 16:03 |
| dulek | Alright, so probably my fault. Now that I think of it, I | 16:03 |
| dulek | 'm not even sure I enabled Python 3 on my env… | 16:04 |
| johnsom | That might be an issue... | 16:04 |
| johnsom | Also, check the bindep | 16:05 |
| haleyb | rm_work: were the two turkeys his sons? | 16:05 |
| dulek | Damn, I'm running with 48b519b5c62be33388b76fd25eebb5673d1e2c18, which enables it by default in DevStack… | 16:06 |
| * johnsom queues up the comedy drum roll | 16:06 | |
| rm_work | haleyb: i figured paul manafort and roger stone... | 16:06 |
| haleyb | rm_work: so is the DIB change failing getting osc-placement info? | 16:07 |
| johnsom | I thought they fixed that osc-placement bug already | 16:09 |
| *** gcheresh has quit IRC | 16:10 | |
| haleyb | there are too many bugs, fixed in devstack? | 16:10 |
| johnsom | Yeah, it was devstack or grenade, I can't remember which. That was one of *last* week's problems... | 16:11 |
| rm_work | haleyb: it's once py3 is enabled for grenade | 16:11 |
| rm_work | this is the thing you tried to fix | 16:11 |
| rm_work | johnsom: i vote we just disable voting for grenade and wait for this stuff to resolve | 16:12 |
| rm_work | we're essentially wasting our time here because it's not our problem | 16:13 |
| johnsom | rm_work Yeah, how do you feel about giving me an hour to look at the state of stuff. Just getting going today. If I don't see what is wrong and has a fix, I will merge it. | 16:13 |
| johnsom | It truly isn't our problem, but I also don't want to miss something that *is* our problem..... | 16:13 |
| rm_work | yeah, but we have until the next release to fix it, so... | 16:14 |
| rm_work | i am assuming the grenade issues will work themselves out within a week or two max | 16:14 |
| rm_work | i'm waiting for this run to report in first before i push the final version of the thing i want anyway | 16:18 |
| haleyb | rm_work: tampermonkey shows the grenade job already failed on https://review.opendev.org/#/c/695947/ is that's what you're waiting for | 16:20 |
| rm_work | yeah i know but i want it to "report in" | 16:20 |
| rm_work | otherwise we lose the results | 16:20 |
| rm_work | i want it recorded | 16:21 |
| johnsom | rm_work Why is python3 enabled in my patch? | 16:21 |
| johnsom | That we know is broken | 16:22 |
| rm_work | because that is the fix for the yaml issue | 16:22 |
| gmann | haleyb: i see. can we move back the grenade job to py2->py2 (explicitly disable py3) and unblock the gate | 16:23 |
| rm_work | it's not novnc, it's that py2 pip and py3 pip are fighting | 16:23 |
| *** tesseract has quit IRC | 16:23 | |
| rm_work | py3 pip is capped and allows replacing the system yaml package, but py2 pip isn't capped and is too new (the newer versions block system package replacement) | 16:24 |
| rm_work | ^^ i believe that's the correct explanation, going off what frickler was saying earlier if i understood correctly | 16:25 |
| johnsom | The other thing you can do is uninstall the system pyaml and pip/pip3 install the latest. That way it won't try to mess with it | 16:25 |
| rm_work | yeah i guess we could do some systemlogic in there | 16:25 |
| rm_work | grenade only runs on ubuntu right? | 16:26 |
| rm_work | so we can just run apt | 16:27 |
| haleyb | gmann: i don't know, but i think rm_work still saw a failure in that case | 16:28 |
| rm_work | alright, i'm willing to try that | 16:28 |
| gmann | ok, n-v also work till we figure out the fix | 16:28 |
| rm_work | well, considering the patch I really want to merge once we get gates working is the "disable py2 gates" patch .... | 16:29 |
| rm_work | switching the grenade gate to py2 seems counter-productive | 16:29 |
| johnsom | Yeah, I think I would try apt-get purge python3-yaml | 16:35 |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Fix gate issue with DIB and Migrate grenade to PY3 https://review.opendev.org/695947 | 16:39 |
| rm_work | yeah i was just doing the `git review` for that but looks like you beat me to it :D | 16:40 |
| johnsom | bets anyone? lol | 16:41 |
| rm_work | if this doesn't work or there's another problem after this, i'm going with non-voting | 16:41 |
| rm_work | i give it 50/50 odds | 16:41 |
| johnsom | I see inap has a broken mirror too, so we are probably also playing roulette | 16:41 |
| rm_work | yeah | 16:41 |
| rm_work | that's supposed to be "fixed-ish" | 16:41 |
| rm_work | fungi promised :D | 16:42 |
| johnsom | I agree. At that point we will have given it a good attempt | 16:42 |
| rm_work | i was complaining about it earlier in infra | 16:42 |
| johnsom | Well, they re-sync'd them yesterday to fix suse, but ... | 16:42 |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Fix gate issue with DIB and Migrate grenade to PY3 https://review.opendev.org/695947 | 17:04 |
| johnsom | The mirror is still broken so a bunch of jobs failed, but I also forgot the sudo part... | 17:05 |
| *** ivve has joined #openstack-lbaas | 17:06 | |
| rm_work | heh | 17:12 |
| *** ataraday_ has joined #openstack-lbaas | 17:19 | |
| *** spatel has quit IRC | 17:28 | |
| openstackgerrit | Ajay Kumar proposed openstack/octavia master: Add install guide for Ubuntu https://review.opendev.org/672842 | 17:34 |
| rm_work | johnsom: ok well inap disable patch just merged so we can recheck now ... | 17:36 |
| openstackgerrit | Ajay Kumar proposed openstack/octavia master: [WIP] Add install guide for Ubuntu https://review.opendev.org/672842 | 17:36 |
| johnsom | Well, the grenade job is still running, so I think I will let it go if you don't mind | 17:36 |
| *** StefanNica has joined #openstack-lbaas | 17:44 | |
| rm_work | yeah k | 17:48 |
| *** openstackgerrit has quit IRC | 17:49 | |
| rm_work | johnsom: do you know what exactly caused the allow-hosts stuff to break in distutils config? was it a new version of something? | 18:06 |
| johnsom | rm_work https://pastebin.com/4vRmGpV5 | 18:10 |
| *** armax has quit IRC | 18:14 | |
| johnsom | That was a draft of an e-mail that went out to zuul-announce | 18:14 |
| johnsom | The patches didn't have a lot of detail sadly | 18:14 |
| rm_work | yeah | 18:15 |
| rm_work | we started having the exact same issue in our internal build system >_< | 18:15 |
| rm_work | trying to explain it now | 18:15 |
| johnsom | well, my uninstall trick didn't work. Something must be installing that package. so, non-voting it is | 18:16 |
| johnsom | Yeah, nova installs it | 18:17 |
| rm_work | yeah k | 18:23 |
| rm_work | i was figuring we just non-voting the check, and comment out the gate | 18:23 |
| rm_work | i'll check in later :D | 18:23 |
| johnsom | Just about to post a patch (easy revert patch) | 18:24 |
| *** openstackgerrit has joined #openstack-lbaas | 18:32 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Make grenade non-voting https://review.opendev.org/696158 | 18:32 |
| *** openstackgerrit has quit IRC | 18:35 | |
| haleyb | johnsom: my patch doing the same actually passed zuul - https://review.opendev.org/#/c/696107/ | 18:41 |
| *** goldyfruit___ has joined #openstack-lbaas | 18:41 | |
| haleyb | you caught me at lunch | 18:41 |
| johnsom | Ah, somehow I missed that patch.... | 18:42 |
| rm_work | how did it pass ... literally anything else tho | 18:42 |
| rm_work | you didn't do the DIB stuff | 18:42 |
| rm_work | maybe they did a release already? | 18:42 |
| johnsom | Nope | 18:42 |
| rm_work | oh do we still use master? | 18:43 |
| rm_work | I think we do | 18:43 |
| rm_work | specifically because of stuff like this | 18:43 |
| rm_work | so yeah, i'm fine with merging Brian's :D | 18:43 |
| *** goldyfruit_ has quit IRC | 18:43 | |
| rm_work | we can just revert that patch when it's better | 18:44 |
| rm_work | I'm +2 | 18:44 |
| johnsom | Oh, we do pull in master DIB I think... | 18:44 |
| rm_work | yep i did that a while back because i got tired of waiting for releases everytime something broke us | 18:45 |
| rm_work | ok bbiab | 18:45 |
| johnsom | Ok, so that is going in. I'm going to open a bug against grenade since no one is fixing it nor is there a bug yet | 18:45 |
| rm_work | T_T k | 18:46 |
| *** goldyfruit_ has joined #openstack-lbaas | 18:51 | |
| *** colin- has quit IRC | 18:52 | |
| *** openstackgerrit has joined #openstack-lbaas | 18:53 | |
| openstackgerrit | Ann Taraday proposed openstack/octavia master: Jobboard based controller https://review.opendev.org/647406 | 18:53 |
| *** goldyfruit___ has quit IRC | 18:54 | |
| *** gcheresh has joined #openstack-lbaas | 19:02 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Fix gate issue with DIB and Migrate grenade to PY3 https://review.opendev.org/695947 | 19:04 |
| johnsom | Ignore that, just trying to capture some logs for the bug | 19:05 |
| *** goldyfruit_ has quit IRC | 19:07 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia stable/train: Revert "Fix gate issue with DIB" https://review.opendev.org/696165 | 19:07 |
| johnsom | We need to sit on that, but I didn't want it to get lost | 19:07 |
| *** maciejjozefczyk has quit IRC | 19:08 | |
| *** goldyfruit has joined #openstack-lbaas | 19:09 | |
| *** goldyfruit_ has joined #openstack-lbaas | 19:19 | |
| *** goldyfruit has quit IRC | 19:22 | |
| *** abaindur has joined #openstack-lbaas | 19:39 | |
| *** abaindur has quit IRC | 19:40 | |
| *** abaindur has joined #openstack-lbaas | 19:40 | |
| *** numans has quit IRC | 20:04 | |
| *** goldyfruit___ has joined #openstack-lbaas | 20:17 | |
| *** goldyfruit_ has quit IRC | 20:20 | |
| *** goldyfruit_ has joined #openstack-lbaas | 20:21 | |
| *** goldyfruit___ has quit IRC | 20:24 | |
| *** gtsr has joined #openstack-lbaas | 20:25 | |
| *** gtsr has left #openstack-lbaas | 20:25 | |
| openstackgerrit | Merged openstack/octavia master: Make octavia-grenade job non-voting https://review.opendev.org/696107 | 20:27 |
| *** gcheresh has quit IRC | 20:44 | |
| *** gcheresh has joined #openstack-lbaas | 20:56 | |
| *** TrevorV has quit IRC | 20:59 | |
| *** armax has joined #openstack-lbaas | 21:03 | |
| *** threestrands has joined #openstack-lbaas | 21:27 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Fix gate issue with DIB and Migrate grenade to PY3 https://review.opendev.org/695947 | 21:44 |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Fix gate issue with DIB and Migrate grenade to PY3 https://review.opendev.org/695947 | 21:46 |
| *** gcheresh has quit IRC | 21:52 | |
| *** abaindur has quit IRC | 22:01 | |
| *** abaindur has joined #openstack-lbaas | 22:01 | |
| *** rcernin has joined #openstack-lbaas | 22:04 | |
| openstackgerrit | Ann Taraday proposed openstack/octavia master: Jobboard based controller https://review.opendev.org/647406 | 22:10 |
| *** pcaruana has quit IRC | 22:16 | |
| *** armax has quit IRC | 22:42 | |
| *** StefanNica has quit IRC | 22:43 | |
| *** tkajinam has joined #openstack-lbaas | 23:08 | |
| *** strobert1 has joined #openstack-lbaas | 23:13 | |
| *** goldyfruit___ has joined #openstack-lbaas | 23:13 | |
| *** goldyfruit_ has quit IRC | 23:16 | |
| *** goldyfruit_ has joined #openstack-lbaas | 23:17 | |
| *** goldyfruit___ has quit IRC | 23:20 | |
| *** armax has joined #openstack-lbaas | 23:38 | |
| *** goldyfruit_ has quit IRC | 23:49 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!