| *** achanda has quit IRC | 00:01 | |
| fungi | kfox1111: ubuntu 14.04 lts | 00:02 |
|---|---|---|
| *** MarkAtwood has quit IRC | 00:02 | |
| *** achanda_ has quit IRC | 00:02 | |
| clarkb | fungi: you see request to rename the UX/personas mailing list | 00:02 |
| *** dims has quit IRC | 00:03 | |
| clarkb | ? | 00:03 |
| clarkb | I am trying to remember if that is a thing we have done efore | 00:03 |
| fungi | clarkb: i did, just haven't gotten around to replying | 00:03 |
| fungi | we renamed the board ml | 00:03 |
| clarkb | it would require a coordinated change to puppet too to make sure puppet doesn't make a new one with the old name | 00:03 |
| fungi | i can probably find my notes | 00:03 |
| fungi | and yeah, we'd need to delete it from puppet i think | 00:03 |
| yushiro | armax: Thank you for your review. | 00:04 |
| clarkb | krotscheck: ^ I distinctly remember asking about personas :P | 00:04 |
| fungi | in the case of the board ml we renamed one existing list to a new name and then another existing list to the first name to replace it, but neither of them were in puppet afair | 00:04 |
| *** hdd has joined #openstack-infra | 00:04 | |
| *** mtanino has quit IRC | 00:05 | |
| *** puranamr has quit IRC | 00:05 | |
| *** puranamr has joined #openstack-infra | 00:06 | |
| *** baoli has joined #openstack-infra | 00:07 | |
| *** wenlock has joined #openstack-infra | 00:07 | |
| *** sdake_ has joined #openstack-infra | 00:08 | |
| fungi | ssh slaves plugin has been upgraded and jenkins service cleaned up and restarted on 06 and 07. now i have 03 and 05 in prepare for shutdown | 00:09 |
| *** mtanino has joined #openstack-infra | 00:09 | |
| fungi | looks like zuul finally caught up with the backlog while i was at dinner | 00:10 |
| *** tjones has joined #openstack-infra | 00:10 | |
| *** sdake has quit IRC | 00:11 | |
| *** wenlock has quit IRC | 00:11 | |
| clarkb | fungi: 04 continues to look great fwiw | 00:12 |
| *** Swami has joined #openstack-infra | 00:12 | |
| openstackgerrit | Merged openstack-infra/project-config: Add python-jobs into networking-fujitsu https://review.openstack.org/218067 | 00:13 |
| *** alexsyip has quit IRC | 00:15 | |
| *** shashank_hegde has quit IRC | 00:15 | |
| *** Piet has quit IRC | 00:16 | |
| yushiro | armax, clarkb: Thank you for your review. | 00:17 |
| *** tjones has left #openstack-infra | 00:17 | |
| *** sdake has joined #openstack-infra | 00:18 | |
| *** teran has joined #openstack-infra | 00:18 | |
| *** teran_ has joined #openstack-infra | 00:21 | |
| *** teran_ has quit IRC | 00:21 | |
| *** sdake_ has quit IRC | 00:21 | |
| openstackgerrit | Claudiu Belu proposed openstack/requirements: WIP: Removes posix-ipc from global-requirements https://review.openstack.org/217824 | 00:22 |
| clarkb | mriedem: still around? why does sqlalchemy migrate depend on keystonemiddleware? | 00:22 |
| clarkb | mriedem: I am trying to figure out how those things are related | 00:22 |
| *** gyee has quit IRC | 00:23 | |
| *** teran has quit IRC | 00:23 | |
| clarkb | nibalizer: https://review.openstack.org/#/c/219031/ is extra special sauce | 00:25 |
| *** sharikapongubala has quit IRC | 00:25 | |
| clarkb | nibalizer: comments welcome since I have no better ideas | 00:25 |
| *** yushiro has quit IRC | 00:26 | |
| nibalizer | clarkb: that looks legit | 00:27 |
| nibalizer | obviously a hack, but simple and I'm down | 00:27 |
| clarkb | cool | 00:27 |
| *** vivekd has joined #openstack-infra | 00:28 | |
| *** zeih has joined #openstack-infra | 00:29 | |
| *** mtanino has quit IRC | 00:29 | |
| *** hdd has quit IRC | 00:30 | |
| *** ybathia has quit IRC | 00:31 | |
| *** mtanino has joined #openstack-infra | 00:32 | |
| *** hdd has joined #openstack-infra | 00:33 | |
| *** zeih has quit IRC | 00:34 | |
| *** puranamr has quit IRC | 00:36 | |
| *** Piet has joined #openstack-infra | 00:37 | |
| *** Swami has quit IRC | 00:40 | |
| *** pvaneck has quit IRC | 00:41 | |
| *** thorst has joined #openstack-infra | 00:45 | |
| *** geoffarnold has quit IRC | 00:46 | |
| *** aysyd has quit IRC | 00:47 | |
| *** Sukhdev has joined #openstack-infra | 00:47 | |
| *** sdake_ has joined #openstack-infra | 00:47 | |
| *** tiswanso has joined #openstack-infra | 00:48 | |
| *** geoffarnold has joined #openstack-infra | 00:49 | |
| *** sdake__ has joined #openstack-infra | 00:49 | |
| openstackgerrit | Merged openstack/requirements: Block sqlalchemy-migrate 0.9.8, cap at 0.10.0 https://review.openstack.org/219027 | 00:49 |
| *** sdake has quit IRC | 00:50 | |
| *** tiswanso has quit IRC | 00:51 | |
| *** abdelwas has quit IRC | 00:51 | |
| *** tiswanso has joined #openstack-infra | 00:51 | |
| *** sdake_ has quit IRC | 00:53 | |
| *** luqas has quit IRC | 00:53 | |
| *** luqas has joined #openstack-infra | 00:53 | |
| anteaya | armax: can you remove your -1 please? https://review.openstack.org/#/c/217446/ | 00:54 |
| *** hdd has quit IRC | 00:54 | |
| *** sdake__ is now known as sdake | 00:54 | |
| armax | anteaya: woot? | 00:55 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: zuul/layout: make non-voting some trusty beaker jobs https://review.openstack.org/219078 | 00:55 |
| armax | most certainly | 00:55 |
| anteaya | armax: very much woot | 00:55 |
| anteaya | armax: thanks | 00:55 |
| anteaya | armax: who is around to send it in? | 00:55 |
| armax | a recheck should do it | 00:55 |
| * armax reckons | 00:55 | |
| anteaya | I don't think that will send it into the gate | 00:56 |
| anteaya | mtreinish: are you about? | 00:56 |
| armax | but can it go straight to the gate, unless someone with powerful powers can? | 00:56 |
| anteaya | it needs a +1 from jenkins | 00:56 |
| *** madhu_ak has quit IRC | 00:56 | |
| EmilienM | anteaya: I don't want to abuse your time at this time of the day but I have an outstanding request: https://review.openstack.org/219078 | 00:56 |
| anteaya | so yes do recheckk | 00:56 |
| armax | anteaya: fired away | 00:57 |
| cp16net | anyone with Canonical here could answer a question on cloud-images.ubuntu.com? | 00:57 |
| anteaya | armax: thanks, let's watch | 00:57 |
| armax | anteaya: with trepidation | 00:57 |
| anteaya | EmilienM: not sure you could abuse my time | 00:57 |
| EmilienM | cp16net: zul | 00:57 |
| anteaya | armax: even my cat is watching! | 00:57 |
| armax | oh boy, good thing cats have 7 lifes | 00:57 |
| EmilienM | damn | 00:58 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: zuul/layout: make non-voting some trusty beaker jobs https://review.openstack.org/219078 | 00:58 |
| anteaya | EmilienM: you write great commit messages | 00:58 |
| anteaya | EmilienM: thank you | 00:58 |
| anteaya | I get context from them and everything | 00:58 |
| EmilienM | :) | 00:58 |
| anteaya | :) | 00:58 |
| EmilienM | anteaya: good night and have a good trip to MTL :) | 00:59 |
| *** glauco_ has joined #openstack-infra | 00:59 | |
| EmilienM | I'm off | 00:59 |
| anteaya | EmilienM: see you wednesday | 00:59 |
| anteaya | I travel tomorrow | 00:59 |
| anteaya | night | 00:59 |
| cp16net | zul: we've seen a few times where the /current dir of https://cloud-images.ubuntu.com/trusty/current go missing. Our trove gate is dependent on this location to build an image for our integration tests. Do know why this would happen? Have any suggestions to mitgate this? | 00:59 |
| anteaya | EmilienM: come back | 01:00 |
| *** bdemers has quit IRC | 01:00 | |
| openstackgerrit | Merged openstack-infra/project-config: Add python34 to list of jobs of EC2-API project https://review.openstack.org/218554 | 01:00 |
| anteaya | EmilienM: you have one glance too many | 01:00 |
| anteaya | gate-puppet-glance-(glance|ironic|heat)-beaker-rspec-dsvm-trusty | 01:00 |
| EmilienM | oops | 01:00 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 01:00 | |
| anteaya | and you forgot -upgrade in there | 01:00 |
| vipul | cp16net: hmm.. we're seeing similar but different errors from cloud-images url http://logs.openstack.org/24/215824/19/check/gate-cue-integration-dsvm-rabbitmq/6016ee1/logs/devstacklog.txt.gz#_2015-09-01_00_16_27_608 | 01:00 |
| EmilienM | no upgrade are non voting already | 01:01 |
| anteaya | EmilienM: then change the commit message | 01:01 |
| *** bdemers has joined #openstack-infra | 01:01 | |
| cp16net | vipul: thats exactly the same error we see because the current dir is missing | 01:01 |
| cp16net | vipul: if you browse through there you will notice its missing | 01:02 |
| vipul | cp16net: oh yea :P | 01:02 |
| vipul | wtf | 01:02 |
| EmilienM | anteaya: I should not be here :P | 01:02 |
| cp16net | causing the forbidden | 01:02 |
| anteaya | vipul: let's have polite language please | 01:02 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: zuul/layout: make non-voting some trusty beaker jobs https://review.openstack.org/219078 | 01:02 |
| EmilienM | anteaya: done ^ thanks a lot | 01:02 |
| anteaya | EmilienM: :) | 01:02 |
| *** chlong has joined #openstack-infra | 01:02 | |
| *** bookwar has quit IRC | 01:02 | |
| anteaya | EmilienM: that looks better | 01:03 |
| *** calebb has quit IRC | 01:03 | |
| vipul | anteaya: whoops.. apologies | 01:03 |
| anteaya | thanks | 01:03 |
| *** skylerberg has quit IRC | 01:03 | |
| anteaya | vipul: thank you | 01:03 |
| *** vivekd_ has joined #openstack-infra | 01:03 | |
| *** glauco_ has quit IRC | 01:03 | |
| mmmpork | SpamapS: quick question, I saw you did a "recheck" comment and it looks like some jenkins things failed that passed before | 01:03 |
| cp16net | vipul: i thought we had images we used for cases like this but i guess not? | 01:03 |
| *** bodepd has quit IRC | 01:03 | |
| mmmpork | i'm looking at the console output and it's not obvious to me what broke | 01:03 |
| *** bodepd has joined #openstack-infra | 01:03 | |
| *** alivigni has joined #openstack-infra | 01:04 | |
| *** vivekd has quit IRC | 01:04 | |
| *** vivekd_ is now known as vivekd | 01:04 | |
| mriedem | clarkb: it doesn't | 01:04 |
| *** bookwar has joined #openstack-infra | 01:04 | |
| clarkb | mriedem how are tgey related tgen | 01:04 |
| mriedem | clarkb: but keystone depends on sqla-migrate and keystonemiddleware | 01:05 |
| mmmpork | i just see "Build failed for element: apt-sources, test-element: test-sources" which i didn't touch | 01:05 |
| vipul | cp16net: Trove may.. but we always generate one one the fly | 01:05 |
| anteaya | clarkb: you are holding a baby | 01:05 |
| mriedem | the conflict is in deps on pr | 01:05 |
| *** calebb has joined #openstack-infra | 01:05 | |
| mriedem | *pbr | 01:05 |
| clarkb | anteaya yes | 01:05 |
| mriedem | clarkb: keystonemiddleware and sqla-migrate were requiring conflicting versions of pbr in that gate bug | 01:05 |
| clarkb | mriedem but middleware depends on the old version | 01:05 |
| anteaya | clarkb: I can tell your typing changes when your arms are full of baby | 01:05 |
| clarkb | so it needs updating? | 01:05 |
| mriedem | clarkb: keystone? | 01:05 |
| *** rossella_s has quit IRC | 01:05 | |
| clarkb | ya keystonemiddleware says pbr < 1.0 according to the job log | 01:06 |
| *** rossella_s has joined #openstack-infra | 01:06 | |
| mriedem | clarkb: yeah, and migrate 0.9.8 requires pbr>=1.3 | 01:06 |
| mriedem | so kaboom | 01:06 |
| mriedem | anyway, https://review.openstack.org/#/c/213877/ | 01:06 |
| *** asettle is now known as asettle-afk | 01:07 | |
| clarkb | oh I thought >1.0 was required all over | 01:07 |
| clarkb | so we forcefully install latest but thatay be a bad assumption | 01:07 |
| *** sharikapongubala has joined #openstack-infra | 01:07 | |
| armax | anteaya: actually on second thoughts, me wonders if the https://review.openstack.org/#/c/219027/ needs to merge in neutron first? | 01:08 |
| *** ramashri has quit IRC | 01:08 | |
| *** ramashri_ has joined #openstack-infra | 01:08 | |
| mriedem | clarkb: it was just stable/kilo that i screwed up, but because of grenade it also wedges master | 01:08 |
| mriedem | so basically i ruined everyone's life today | 01:08 |
| *** alivigni has quit IRC | 01:08 | |
| *** baoli has quit IRC | 01:08 | |
| anteaya | armax: well it is merged | 01:10 |
| armax | right, but that wasn’t pulled into stable/neutron | 01:10 |
| anteaya | armax: is there a patch | 01:10 |
| armax | not that I can find | 01:10 |
| anteaya | I think proposal bot should be offering one up | 01:10 |
| *** fitoduarte has quit IRC | 01:10 | |
| anteaya | clarkb: should it not? | 01:11 |
| armax | anteaya: it should | 01:11 |
| armax | https://review.openstack.org/#/q/owner:%22OpenStack+Proposal+Bot%22+status:open+project:openstack/neutron,n,z | 01:11 |
| armax | but a new PS hasn’t popped up yet | 01:11 |
| anteaya | for stable/kilo? | 01:11 |
| armax | aye | 01:11 |
| anteaya | okay I will take your word for it, perhaps it needs a moment to collect itself | 01:12 |
| armax | I’ll watch it with the help of your cats | 01:12 |
| clarkb | ir should but runs in post iirc | 01:12 |
| clarkb | so may be slow | 01:12 |
| armax | clarkb: yeah…I am not worried | 01:13 |
| anteaya | https://jenkins.openstack.org/computer/proposal.slave.openstack.org/ | 01:13 |
| *** DericHorn-HP has joined #openstack-infra | 01:13 | |
| anteaya | clarkb: so something should be happening here, yes? | 01:13 |
| *** spzala has joined #openstack-infra | 01:13 | |
| armax | btw anteaya I have no merge rights on stable | 01:13 |
| anteaya | armax: thanks :) | 01:13 |
| armax | so you’ll have to reach out to other elves | 01:13 |
| anteaya | armax: ah okay, good to know | 01:13 |
| anteaya | armax: when we have something we will start the trace to find them | 01:13 |
| mmmpork | hrm i was wondering if anyone could help me understand why this passed before but just suddenly stopped passing - https://review.openstack.org/#/c/218965/ | 01:13 |
| anteaya | armax: first we need a patch to point them at | 01:13 |
| mmmpork | according to the error console, it's failing on something i didn't touch as part of the patch, so that's why i'm a bit confused | 01:14 |
| armax | we’’l find out soon enough | 01:14 |
| *** bdemers has quit IRC | 01:14 | |
| clarkb | oh ya thats special node so maybe not slow | 01:14 |
| clarkb | check build history of that slave via jenkins ui | 01:14 |
| *** bdemers has joined #openstack-infra | 01:14 | |
| mmmpork | ah it looks like SpamapS triggered a recheck to rerun the tests | 01:15 |
| mmmpork | i just saw the "recheck" and the failed tests | 01:15 |
| anteaya | mmmpork: http://logs.openstack.org/65/218965/4/check/gate-dib-dsvm-functests-devstack-trusty/ad72368/console.html#_2015-09-01_00_29_08_930 | 01:15 |
| mriedem | armax: i'm going through the stable/kilo g-r syncs right now | 01:15 |
| mriedem | and fast approving everything | 01:15 |
| armax | mriedem: great | 01:15 |
| mmmpork | anteaya: thanks i just glossed over that, just saw the failed on the things i didn't touch ;D | 01:15 |
| anteaya | mriedem: thank you | 01:15 |
| armax | mriedem: can’t see anything popping up for neutron yet though | 01:15 |
| fungi | armax: clarkb: anteaya: the proposal job for that failed http://logs.openstack.org/4c/4c08ed336fdb79a7e4aaa09c1f9b2a5cc3e4f940/post/propose-requirements-updates/e25835d/ | 01:16 |
| mriedem | armax: does neutron use sqla-migrate? i thought neutron was only alembic? | 01:16 |
| mmmpork | i just got an email that the triple NO tests failed (which is normal) so i realized it's rerunning right now :) | 01:16 |
| anteaya | mmmpork: many times the job fails for strange reasons, the host can have issues, new dependecies get released, image issues, on and on | 01:16 |
| mmmpork | right, typical jenkins stuff ;D | 01:16 |
| armax | mriedem: indeed it does use alembic | 01:16 |
| anteaya | fungi: thanks fungi | 01:17 |
| mmmpork | i just am not familiar enough with our logs yet ;D | 01:17 |
| armax | mriedem: but we’re still affected because keystonemiddleware | 01:17 |
| armax | *of | 01:17 |
| *** ramashri_ has quit IRC | 01:17 | |
| anteaya | mmmpork: well actually dismissing it is unhelpful and a poor dev habit which we work hard to educate folks not to fall into | 01:17 |
| fungi | but yeah, looks like neutron didn't need an update anyway | 01:17 |
| anteaya | as it results in devs expecting others to fix it for them | 01:17 |
| mriedem | armax: keystonemiddleware doesnw't matter | 01:17 |
| anteaya | so yes learning how to read the logs is very helpful | 01:17 |
| mriedem | keystonemiddleware doesn't require sqla-migrate | 01:17 |
| mriedem | keystone does | 01:17 |
| fungi | seems it failed attempting to clone stackforge/congress for some reason | 01:18 |
| anteaya | and starting by finding or asking for help to find the failure is agreat start | 01:18 |
| anteaya | so thank you for doing that | 01:18 |
| * mmmpork has ridden this rodeo before ;) | 01:18 | |
| *** puranamr has joined #openstack-infra | 01:18 | |
| fungi | and some others. but the job at least seems to skip projects which are able to be updated | 01:18 |
| anteaya | mmmpork: anytime I'm wasting your time let me know, I will move onto other things | 01:19 |
| *** btully has joined #openstack-infra | 01:19 | |
| *** iraw_ has joined #openstack-infra | 01:19 | |
| fungi | for example, it did propose an update to ceilo https://review.openstack.org/#/c/218022/2/requirements.txt | 01:19 |
| *** dims has joined #openstack-infra | 01:20 | |
| fungi | so looks like it proposed the things it was going to propose | 01:20 |
| anteaya | so are we agreed that neutron does not need a stable kilo patch for the -migrate cap? | 01:20 |
| *** jyuso1 has joined #openstack-infra | 01:20 | |
| fungi | i don't see it in requirements or test-requirements at http://git.openstack.org/cgit/openstack/neutron/tree/?h=stable/kilo | 01:21 |
| fungi | so seems like stable/kilo of neutron shouldn't have gotten an update proposed from that merge anyway | 01:21 |
| anteaya | fungi: thank you | 01:21 |
| anteaya | armax: what else stands between us and mtreinish's patch being merged? | 01:22 |
| anteaya | clarkb: you said look on the jenkins ui for the proposal job, where do I look? https://jenkins.openstack.org/ | 01:22 |
| clarkb | anteaya on the page for the slave on the left is a build history button | 01:23 |
| anteaya | armax: just 217446? | 01:23 |
| anteaya | clarkb: thanks, /me looks | 01:23 |
| anteaya | fungi: is this the same failed build you were looking at? https://jenkins.openstack.org/job/propose-requirements-updates/lastBuild/console | 01:25 |
| fungi | anteaya: probably, but i identified it from the git commit id | 01:25 |
| zul | cp16net: passing the info along | 01:25 |
| anteaya | fungi: thanks, I see the same error failure to clone congress | 01:26 |
| fungi | anteaya: yeah, i think there are a handful in there which changed from stackforge to openstack and the projects.txt in openstack/requirements was never updated to match | 01:27 |
| *** FallenPegasus has joined #openstack-infra | 01:27 | |
| cp16net | zul: thanks! :) i have a script pinging it every min to alert me when it is available again. and trying to see if there is another image endpoint we can use in the mean time. | 01:27 |
| anteaya | fungi: ah | 01:28 |
| anteaya | I can do that | 01:28 |
| anteaya | unless you already have a script for that | 01:28 |
| armax | anteaya: yes | 01:28 |
| fungi | anteaya: it would be a big help! just search for any clone failures like that one in the console log, check to see if that repo has the wrong namespace on it and then update them all in a batch | 01:29 |
| armax | 217446 is the only one | 01:29 |
| armax | which btw just passed | 01:29 |
| anteaya | armax: grand | 01:29 |
| anteaya | armax: yay | 01:29 |
| armax | for reasons that escape me | 01:29 |
| * anteaya goes to +A mtreinish's patch | 01:29 | |
| *** baoli has joined #openstack-infra | 01:29 | |
| anteaya | armax: I'm not asking questions right now | 01:29 |
| anteaya | :) | 01:29 |
| *** mtanino has quit IRC | 01:29 | |
| armax | I won’t dare | 01:29 |
| *** Piet has quit IRC | 01:30 | |
| anteaya | okay waiting on the gate | 01:30 |
| *** zeih has joined #openstack-infra | 01:30 | |
| anteaya | guard cat is still with me | 01:30 |
| *** puranamr has quit IRC | 01:30 | |
| anteaya | fungi: okey dokie | 01:31 |
| jyuso1 | anteaya: hi | 01:31 |
| openstackgerrit | Dan Prince proposed openstack-infra/tripleo-ci: Pin python-netaddr to 0.7.15 https://review.openstack.org/219090 | 01:32 |
| anteaya | jyuso1: hey there, how are you | 01:32 |
| anteaya | jyuso1: you had a runaway system today | 01:32 |
| fungi | okay, jenkins03 and 05 have the ssh slaves plugin upgraded, and have been cleaned up and restarted. putting 01 and 02 into prepare for shutdown now | 01:32 |
| anteaya | jyuso1: have you found the error? | 01:32 |
| anteaya | jyuso1: we had to shut you down, sorry about that | 01:33 |
| anteaya | fungi: ack | 01:33 |
| jyuso1 | anteaya: hi,yes,that's my problem.something wrong with one of my environment | 01:33 |
| jyuso1 | anteaya: I've found the reason | 01:33 |
| *** baoli has quit IRC | 01:34 | |
| anteaya | jyuso1: figured you would | 01:34 |
| *** baoli has joined #openstack-infra | 01:34 | |
| anteaya | jyuso1: what happened? | 01:34 |
| jyuso1 | anteaya: we are try auto-retrigger function in our CI. | 01:34 |
| *** DericHorn-HP has quit IRC | 01:34 | |
| anteaya | morgan: what did python-keystoneclient-federation become? | 01:34 |
| anteaya | jyuso1: ah, let's not do that | 01:34 |
| morgan | anteaya: uhmm... keystoneauth-saml2? i think | 01:35 |
| anteaya | jyuso1: other ci operators have gotten into problems with taht before | 01:35 |
| jyuso1 | anteaya: and some step are broken,that's a exception i've never thought about. | 01:35 |
| *** zeih has quit IRC | 01:35 | |
| anteaya | morgan: I'll go with that and hope you can review the patch once it is up | 01:35 |
| anteaya | jyuso1: okay glad you are thinking about it now | 01:35 |
| morgan | anteaya: if you give me a moment I can chase it down | 01:35 |
| anteaya | morgan: all the time you need | 01:36 |
| anteaya | I'll tell jyuso1 a story | 01:36 |
| morgan | anteaya: https://github.com/openstack/keystoneauth-saml2 | 01:36 |
| jyuso1 | anteaya: you mean we'd better not use auto-retrigger? | 01:36 |
| anteaya | jyuso1: one time about a year and a half ago | 01:36 |
| anteaya | jyuso1: yes don't use auto-trigger | 01:36 |
| anteaya | morgan: thank you | 01:36 |
| jyuso1 | anteaya:OK.. | 01:36 |
| morgan | np | 01:36 |
| anteaya | jyuso1: about a year and a half ago someone used an auto retrigger | 01:37 |
| jyuso1 | anteaya: I'm listening:) | 01:37 |
| anteaya | and retriggered about 600 patches at once | 01:37 |
| anteaya | our system was smaller then | 01:37 |
| anteaya | but it was enough of a load, plus we were busy anyway | 01:37 |
| openstackgerrit | Merged openstack-infra/project-config: copy_puppet_logs: copy dstat logs if exist https://review.openstack.org/218586 | 01:37 |
| anteaya | and it took out our system | 01:37 |
| anteaya | we had to restart our gerrit | 01:37 |
| anteaya | so we aren't in favour of folks using auto-retriggers | 01:38 |
| anteaya | as they rarely have the consequences the author intends | 01:38 |
| anteaya | and usually end up doing weird things that are very upsetting for devs | 01:38 |
| anteaya | so if you can avoid any kind of automated commenting or retriggering from your system | 01:38 |
| anteaya | that works out best for us | 01:39 |
| anteaya | jyuso1: make sense? | 01:39 |
| jyuso1 | anteaya: ok | 01:40 |
| jyuso1 | anteaya: we just want to avoid the wrong return information.some error happened caused by networking:) | 01:40 |
| anteaya | jyuso1: I understand | 01:41 |
| anteaya | fungi: I don't know why it is trying to clone python-keystoneclient-federation: http://logs.openstack.org/4c/4c08ed336fdb79a7e4aaa09c1f9b2a5cc3e4f940/post/propose-requirements-updates/e25835d/console.html#_2015-09-01_00_55_01_041 | 01:41 |
| anteaya | fungi: do you see it in projects? http://git.openstack.org/cgit/openstack/requirements/tree/projects.txt | 01:41 |
| *** larainema has joined #openstack-infra | 01:41 | |
| *** vivekd has quit IRC | 01:42 | |
| fungi | anteaya: oh, that job is on the stable/kilo branch specifically | 01:42 |
| anteaya | ah | 01:43 |
| fungi | anteaya: so looking at that log, the cleanup would be for stable/kilo branch requirements projects.txt file | 01:43 |
| anteaya | I was in the wrong branch | 01:43 |
| anteaya | right sorry | 01:43 |
| anteaya | how to do I clone the branch again | 01:43 |
| anteaya | I can look it up | 01:43 |
| *** vivekd has joined #openstack-infra | 01:44 | |
| openstackgerrit | Merged openstack-infra/project-config: Tripleo CI reshuffle https://review.openstack.org/205479 | 01:44 |
| *** mriedem is now known as mriedem_away | 01:44 | |
| jyuso1 | anteaya: another thing.It looks like our account is forbidden,could you help to resume it? | 01:44 |
| fungi | you can do something like `git checkout -B requirements-renames --no-track origin/stable/kilo` to create a local non-tracking topic branch named "requirements-updates" from the current state of teh stable/kilo branch on the origin remote | 01:44 |
| anteaya | jyuso1: well first reply to the thread in third-party-announce and give me a full report of what went wrong | 01:45 |
| jyuso1 | anteaya:OK | 01:45 |
| anteaya | jyuso1: as well as the steps you will take to avoid the problem in the future | 01:45 |
| anteaya | jyuso1: and I'll read what you have | 01:46 |
| anteaya | jyuso1: thanks | 01:46 |
| anteaya | fungi: thanks | 01:46 |
| *** shashank_hegde has joined #openstack-infra | 01:46 | |
| *** geoffarnold is now known as geoffarnoldX | 01:46 | |
| *** Zhongjun has quit IRC | 01:47 | |
| *** larainema has quit IRC | 01:47 | |
| openstackgerrit | Jerry Zhao proposed openstack-infra/project-config: Add networking-fortinet project to neutron https://review.openstack.org/219055 | 01:47 |
| *** larainema has joined #openstack-infra | 01:47 | |
| anteaya | fungi: that worked, thank you | 01:47 |
| openstackgerrit | Merged openstack-infra/project-config: Add docs and publish jobs for kolla https://review.openstack.org/218137 | 01:48 |
| *** wenlock has joined #openstack-infra | 01:49 | |
| *** stevemar has joined #openstack-infra | 01:50 | |
| *** Sukhdev has quit IRC | 01:51 | |
| openstackgerrit | Merged openstack-infra/project-config: copy_puppet_logs: copy tempest.conf in txt file https://review.openstack.org/218245 | 01:54 |
| *** stevemar has quit IRC | 01:55 | |
| *** bdemers has quit IRC | 01:55 | |
| *** bdemers has joined #openstack-infra | 01:55 | |
| openstackgerrit | Anita Kuno proposed openstack/requirements: Updates moved project paths https://review.openstack.org/219091 | 01:56 |
| *** sridhar_ram has quit IRC | 01:56 | |
| openstackgerrit | Merged openstack-infra/project-config: Kolla: Add build gate for ubuntu source https://review.openstack.org/213531 | 01:57 |
| anteaya | morgan: if you want in on the review: https://review.openstack.org/#/c/219091/1 | 01:57 |
| *** madong_ has joined #openstack-infra | 01:57 | |
| anteaya | armax: yay it passed | 01:58 |
| anteaya | armax: third in the gate, waiting on the two before it | 01:58 |
| anteaya | morgan: thank you | 01:59 |
| morgan | :) | 01:59 |
| anteaya | :) | 01:59 |
| * anteaya goes to make tea | 01:59 | |
| *** larainema has quit IRC | 02:01 | |
| *** sthillma has quit IRC | 02:04 | |
| *** sabeen has quit IRC | 02:04 | |
| *** larainema has joined #openstack-infra | 02:04 | |
| *** madong_ has quit IRC | 02:04 | |
| *** jyuso has joined #openstack-infra | 02:06 | |
| *** asettle-afk has quit IRC | 02:07 | |
| *** zz_dimtruck is now known as dimtruck | 02:08 | |
| *** jyuso1 has quit IRC | 02:09 | |
| *** spzala has quit IRC | 02:10 | |
| *** Piet has joined #openstack-infra | 02:12 | |
| anteaya | jyuso: I have to go to bed soon, once you post your reply to the third-party-announce mailing list thread it will need to be approved by me before you can see it at lists.openstack.org | 02:14 |
| openstackgerrit | Motohiro/Yuanying Otsuka proposed openstack-infra/project-config: Enable Barbican in Magnum job definition https://review.openstack.org/216933 | 02:14 |
| *** iraw_ has quit IRC | 02:14 | |
| anteaya | jyuso: I'll be back for the third party meeting at 0800 utc so we can talk more then | 02:14 |
| jyuso | anteaya: OK,good night.I'll send the mail later.see you. | 02:16 |
| anteaya | thanks see you later | 02:17 |
| *** woodster_ has quit IRC | 02:19 | |
| *** tlian2 has quit IRC | 02:19 | |
| *** geoffarnoldX is now known as geoffarnold | 02:20 | |
| anteaya | mtreinish: 212864 is +A'd finally and I'm going to bed | 02:20 |
| *** rkukura has joined #openstack-infra | 02:24 | |
| *** bknudson has quit IRC | 02:24 | |
| *** thorst has quit IRC | 02:29 | |
| *** baoli has quit IRC | 02:30 | |
| *** zeih has joined #openstack-infra | 02:31 | |
| openstackgerrit | Ryu Ishimoto proposed openstack-infra/project-config: Set DEVSTACK_GATE_SETTINGS for midonet gate job https://review.openstack.org/207731 | 02:33 |
| *** zeih has quit IRC | 02:35 | |
| *** baoli has joined #openstack-infra | 02:37 | |
| *** FallenPegasus has quit IRC | 02:41 | |
| *** BharatK has joined #openstack-infra | 02:45 | |
| *** fitoduarte has joined #openstack-infra | 02:46 | |
| fungi | and finally, jenkins01 and 02 have the ssh slave plugin upgraded, then cleaned up and restarted. that makes all the nodepool jenkins masters. i'll get the others (jenkins.o.o and -dev) dealt with tomorrow | 02:47 |
| *** iraw_ has joined #openstack-infra | 02:48 | |
| *** adduarte has joined #openstack-infra | 02:48 | |
| *** FallenPegasus has joined #openstack-infra | 02:49 | |
| *** yamahata has quit IRC | 02:50 | |
| *** fitoduarte has quit IRC | 02:50 | |
| *** baoli has quit IRC | 02:52 | |
| *** larainema has quit IRC | 02:55 | |
| *** baoli has joined #openstack-infra | 02:55 | |
| *** shashank_hegde has quit IRC | 02:57 | |
| *** asettle has joined #openstack-infra | 02:58 | |
| *** glauco has joined #openstack-infra | 03:00 | |
| *** shashank_hegde has joined #openstack-infra | 03:02 | |
| *** glauco has quit IRC | 03:04 | |
| *** achanda has joined #openstack-infra | 03:07 | |
| *** larainema has joined #openstack-infra | 03:07 | |
| *** baoli has quit IRC | 03:08 | |
| *** Zhongjun has joined #openstack-infra | 03:10 | |
| *** baoli has joined #openstack-infra | 03:10 | |
| *** chlong has quit IRC | 03:12 | |
| *** xyang1 has quit IRC | 03:13 | |
| *** baoli has quit IRC | 03:14 | |
| *** shashank_hegde has quit IRC | 03:20 | |
| *** Apsu is now known as JorgeSonata | 03:20 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for infra 403 during devstack setup bug 1490806 https://review.openstack.org/219109 | 03:21 |
| *** larainema has quit IRC | 03:21 | |
| openstack | bug 1490806 in OpenStack-Gate "devstack fails in gate with "Server returned an unexpected response code. [403]"" [Undecided,New] https://launchpad.net/bugs/1490806 | 03:21 |
| *** larainema has joined #openstack-infra | 03:21 | |
| *** JorgeSonata is now known as Apsu | 03:22 | |
| *** bdemers has quit IRC | 03:26 | |
| *** bdemers has joined #openstack-infra | 03:26 | |
| *** ddieterly has quit IRC | 03:26 | |
| *** mriedem_away has quit IRC | 03:27 | |
| *** baoli has joined #openstack-infra | 03:28 | |
| *** darrenc is now known as darrenc_afk | 03:28 | |
| *** gildub has joined #openstack-infra | 03:29 | |
| *** unicell has quit IRC | 03:31 | |
| *** zeih has joined #openstack-infra | 03:31 | |
| *** shashank_hegde has joined #openstack-infra | 03:33 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for infra 403 during devstack setup bug 1490806 https://review.openstack.org/219109 | 03:34 |
| openstack | bug 1490806 in OpenStack-Gate "devstack fails in gate with "Server returned an unexpected response code. [403]"" [Undecided,New] https://launchpad.net/bugs/1490806 | 03:34 |
| *** zeih has quit IRC | 03:36 | |
| *** chlong has joined #openstack-infra | 03:38 | |
| *** tiswanso has quit IRC | 03:40 | |
| *** larainema has quit IRC | 03:42 | |
| *** rlandy has joined #openstack-infra | 03:42 | |
| *** larainema has joined #openstack-infra | 03:43 | |
| *** vivekd has quit IRC | 03:44 | |
| *** FallenPegasus has quit IRC | 03:46 | |
| *** links has joined #openstack-infra | 03:46 | |
| *** FallenPegasus has joined #openstack-infra | 03:47 | |
| *** bdemers has quit IRC | 03:48 | |
| *** sigmavirus24 has quit IRC | 03:48 | |
| *** bdemers has joined #openstack-infra | 03:48 | |
| *** sigmavirus24 has joined #openstack-infra | 03:50 | |
| *** achanda has quit IRC | 03:51 | |
| *** stevemar has joined #openstack-infra | 03:56 | |
| *** bhunter71 has quit IRC | 03:57 | |
| *** iraw_ has quit IRC | 03:58 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 03:59 | |
| *** jeadre has quit IRC | 03:59 | |
| *** bdemers has quit IRC | 04:00 | |
| *** markvoelker has joined #openstack-infra | 04:00 | |
| *** HeOS_ has quit IRC | 04:00 | |
| *** HeOS_ has joined #openstack-infra | 04:00 | |
| openstackgerrit | Motohiro/Yuanying Otsuka proposed openstack-infra/project-config: Enable Barbican in Magnum job definition https://review.openstack.org/216933 | 04:00 |
| *** bdemers has joined #openstack-infra | 04:00 | |
| *** markvoelker_ has joined #openstack-infra | 04:02 | |
| *** chlong_ has joined #openstack-infra | 04:04 | |
| *** chlong has quit IRC | 04:04 | |
| *** markvoelker has quit IRC | 04:04 | |
| *** ildikov has joined #openstack-infra | 04:04 | |
| *** btully has quit IRC | 04:05 | |
| *** geoffarnold is now known as geoffarnoldX | 04:07 | |
| *** Sree has joined #openstack-infra | 04:08 | |
| *** Sree has quit IRC | 04:08 | |
| *** Sree has joined #openstack-infra | 04:09 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: Make gate-ceilometer-dsvm-functional-postgresql voting https://review.openstack.org/215485 | 04:10 |
| *** jerryz has quit IRC | 04:11 | |
| *** yfried has joined #openstack-infra | 04:13 | |
| *** jyuso1 has joined #openstack-infra | 04:13 | |
| *** bdemers has quit IRC | 04:14 | |
| *** jyuso has quit IRC | 04:16 | |
| *** amotoki has joined #openstack-infra | 04:17 | |
| openstackgerrit | Merged openstack/requirements: Changing the bounds for Django https://review.openstack.org/216469 | 04:19 |
| openstackgerrit | Merged openstack/requirements: Changing the bounds for Django https://review.openstack.org/216469 | 04:19 |
| *** mrmartin has joined #openstack-infra | 04:21 | |
| *** bdemers has joined #openstack-infra | 04:21 | |
| *** robcresswell has quit IRC | 04:21 | |
| *** Sree_ has joined #openstack-infra | 04:22 | |
| *** Sree_ is now known as Guest41206 | 04:22 | |
| *** robcresswell has joined #openstack-infra | 04:22 | |
| *** bdemers has quit IRC | 04:23 | |
| *** geoffarnoldX is now known as geoffarnold | 04:23 | |
| *** bdemers has joined #openstack-infra | 04:23 | |
| *** Sree has quit IRC | 04:24 | |
| nigelb | fungi: I ended up banning the use of + on our mailing lists. Got tired of the whack-a-mole with spammers. | 04:24 |
| *** FallenPegasus has quit IRC | 04:24 | |
| *** ddieterly has joined #openstack-infra | 04:27 | |
| *** ayoung has quit IRC | 04:29 | |
| *** dims has quit IRC | 04:30 | |
| *** ddieterly has quit IRC | 04:32 | |
| *** zeih has joined #openstack-infra | 04:32 | |
| *** bdemers has quit IRC | 04:34 | |
| *** baoli has quit IRC | 04:34 | |
| *** puranamr has joined #openstack-infra | 04:35 | |
| *** yfried has quit IRC | 04:36 | |
| *** shashank_hegde has quit IRC | 04:36 | |
| *** rlandy has quit IRC | 04:36 | |
| *** bdemers has joined #openstack-infra | 04:37 | |
| *** zeih has quit IRC | 04:37 | |
| *** watanabe_isao has joined #openstack-infra | 04:39 | |
| *** yonglihe has joined #openstack-infra | 04:41 | |
| *** jyuso has joined #openstack-infra | 04:44 | |
| *** achanda has joined #openstack-infra | 04:45 | |
| *** jyuso1 has quit IRC | 04:47 | |
| *** jerryz has joined #openstack-infra | 04:49 | |
| ianw | does anyone know any names of people that have to do with the ubuntu cloud mirror? | 04:52 |
| *** darrenc_afk is now known as darrenc | 04:54 | |
| *** sharikapongubala has quit IRC | 04:57 | |
| *** sharikapongubala has joined #openstack-infra | 04:57 | |
| StevenK | ianw: About /trusty/current not existing? I've bought it up in #ubuntu-devel, but no answer yet | 04:57 |
| StevenK | ianw: But https://launchpad.net/~ubuntu-cdimage is what you're looking for | 04:58 |
| *** puranamr has quit IRC | 04:58 | |
| ianw | StevenK: ok cool, yeah i pinged in ubuntu-mirrors but also nobody around | 04:58 |
| *** Sukhdev has joined #openstack-infra | 04:58 | |
| openstackgerrit | Merged openstack-infra/project-config: Make gate-ceilometer-dsvm-functional-postgresql voting https://review.openstack.org/215485 | 04:59 |
| *** sharikapongubala has quit IRC | 05:01 | |
| *** unicell has joined #openstack-infra | 05:02 | |
| *** puranamr has joined #openstack-infra | 05:04 | |
| *** vivekd has joined #openstack-infra | 05:05 | |
| *** Adri2000 has joined #openstack-infra | 05:09 | |
| *** sthillma has joined #openstack-infra | 05:10 | |
| *** geoffarnold has quit IRC | 05:11 | |
| *** geoffarnold has joined #openstack-infra | 05:11 | |
| *** ildikov has quit IRC | 05:12 | |
| *** FallenPegasus has joined #openstack-infra | 05:16 | |
| *** sthillma_ has joined #openstack-infra | 05:16 | |
| *** puranamr has quit IRC | 05:19 | |
| *** sthillma has quit IRC | 05:20 | |
| *** sthillma_ is now known as sthillma | 05:20 | |
| *** stevemar has quit IRC | 05:20 | |
| *** yfried has joined #openstack-infra | 05:21 | |
| *** jyuso1 has joined #openstack-infra | 05:23 | |
| *** yfried_ has joined #openstack-infra | 05:25 | |
| *** jyuso has quit IRC | 05:26 | |
| *** yfried has quit IRC | 05:26 | |
| *** ddieterly has joined #openstack-infra | 05:28 | |
| *** jerryz has quit IRC | 05:33 | |
| *** stevemar has joined #openstack-infra | 05:33 | |
| *** zeih has joined #openstack-infra | 05:33 | |
| *** ddieterly has quit IRC | 05:33 | |
| *** shashank_hegde has joined #openstack-infra | 05:34 | |
| *** larainema has quit IRC | 05:35 | |
| *** larainema has joined #openstack-infra | 05:36 | |
| *** zeih has quit IRC | 05:37 | |
| *** stevemar has quit IRC | 05:38 | |
| *** ildikov has joined #openstack-infra | 05:38 | |
| *** larainema has quit IRC | 05:41 | |
| *** larainema has joined #openstack-infra | 05:41 | |
| *** maishsk has joined #openstack-infra | 05:46 | |
| *** dimtruck is now known as zz_dimtruck | 05:46 | |
| *** mrmartin has quit IRC | 05:47 | |
| *** jyuso1 has quit IRC | 05:48 | |
| *** jyuso1 has joined #openstack-infra | 05:50 | |
| *** geoffarnold is now known as geoffarnoldX | 05:51 | |
| *** markvoelker_ has quit IRC | 05:53 | |
| *** yfried_ has quit IRC | 05:54 | |
| *** kushal has joined #openstack-infra | 05:55 | |
| *** mrmartin has joined #openstack-infra | 05:58 | |
| *** yfried has joined #openstack-infra | 05:58 | |
| *** flepied has quit IRC | 05:58 | |
| *** hdd has joined #openstack-infra | 05:58 | |
| *** dims has joined #openstack-infra | 05:59 | |
| *** FallenPegasus has quit IRC | 06:02 | |
| *** dims has quit IRC | 06:04 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/219148 | 06:04 |
| *** sdake_ has joined #openstack-infra | 06:05 | |
| *** kushal has quit IRC | 06:05 | |
| *** jtomasek has joined #openstack-infra | 06:08 | |
| *** sdake has quit IRC | 06:08 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/218737 | 06:09 |
| *** FallenPegasus has joined #openstack-infra | 06:11 | |
| *** sdake has joined #openstack-infra | 06:11 | |
| *** sdake has quit IRC | 06:11 | |
| *** sdake has joined #openstack-infra | 06:13 | |
| *** jerryz has joined #openstack-infra | 06:13 | |
| *** yfried is now known as yfried|afk | 06:13 | |
| *** mrunge has joined #openstack-infra | 06:13 | |
| *** unicell has quit IRC | 06:15 | |
| *** sdake_ has quit IRC | 06:15 | |
| *** unicell has joined #openstack-infra | 06:15 | |
| *** yfried|afk is now known as yfried | 06:15 | |
| *** abregman has joined #openstack-infra | 06:16 | |
| *** btully has joined #openstack-infra | 06:17 | |
| *** Zhongjun has quit IRC | 06:17 | |
| *** ParsectiX has joined #openstack-infra | 06:22 | |
| *** asettle has quit IRC | 06:26 | |
| *** BharatK has quit IRC | 06:27 | |
| yamamoto | hi | 06:27 |
| yamamoto | where can i find results of periodic jobs? | 06:28 |
| *** ddieterly has joined #openstack-infra | 06:29 | |
| *** flepied has joined #openstack-infra | 06:29 | |
| *** liam__ has joined #openstack-infra | 06:29 | |
| *** zeih has joined #openstack-infra | 06:34 | |
| *** deepakcs has joined #openstack-infra | 06:34 | |
| *** ddieterly has quit IRC | 06:34 | |
| *** kushal has joined #openstack-infra | 06:35 | |
| *** sharikapongubala has joined #openstack-infra | 06:36 | |
| *** afazekas__ has joined #openstack-infra | 06:36 | |
| *** armax has quit IRC | 06:37 | |
| *** zeih has quit IRC | 06:38 | |
| *** flepied1 has joined #openstack-infra | 06:39 | |
| *** sharikapongubala has quit IRC | 06:40 | |
| *** liam__ has quit IRC | 06:42 | |
| *** kiran-r has joined #openstack-infra | 06:43 | |
| *** flepied has quit IRC | 06:43 | |
| *** sshnaidm has joined #openstack-infra | 06:44 | |
| *** hdd has quit IRC | 06:44 | |
| *** kdas_ has joined #openstack-infra | 06:45 | |
| *** b10n1k_ has quit IRC | 06:46 | |
| *** Sukhdev has quit IRC | 06:46 | |
| *** kushal has quit IRC | 06:46 | |
| *** jyuso has joined #openstack-infra | 06:49 | |
| *** jyuso1 has quit IRC | 06:52 | |
| *** kiran-r has quit IRC | 06:53 | |
| *** k4n0 has joined #openstack-infra | 06:55 | |
| *** dtantsur|afk is now known as dtantsur | 06:57 | |
| *** peristeri has joined #openstack-infra | 06:58 | |
| *** glauco has joined #openstack-infra | 07:01 | |
| *** FallenPegasus has quit IRC | 07:03 | |
| *** markus_z has joined #openstack-infra | 07:03 | |
| *** glauco has quit IRC | 07:06 | |
| *** slagle has quit IRC | 07:06 | |
| *** fhubik has joined #openstack-infra | 07:08 | |
| *** achanda has quit IRC | 07:11 | |
| *** sirushti has quit IRC | 07:15 | |
| *** liam__ has joined #openstack-infra | 07:16 | |
| *** achanda has joined #openstack-infra | 07:17 | |
| *** deepakcs has quit IRC | 07:22 | |
| *** chlong_ has quit IRC | 07:24 | |
| *** achanda has quit IRC | 07:25 | |
| *** achanda has joined #openstack-infra | 07:26 | |
| *** liam__ has quit IRC | 07:28 | |
| *** aukhan has joined #openstack-infra | 07:29 | |
| *** sharikapongubala has joined #openstack-infra | 07:29 | |
| *** zeih has joined #openstack-infra | 07:29 | |
| *** achanda has quit IRC | 07:30 | |
| *** ddieterly has joined #openstack-infra | 07:30 | |
| *** liam__ has joined #openstack-infra | 07:30 | |
| *** liam__ has quit IRC | 07:30 | |
| *** Zhongjun_ has joined #openstack-infra | 07:33 | |
| *** Ala has joined #openstack-infra | 07:34 | |
| *** ifarkas has joined #openstack-infra | 07:34 | |
| *** ddieterly has quit IRC | 07:35 | |
| *** shardy has joined #openstack-infra | 07:35 | |
| *** Zhongjun_ is now known as Zhongjun | 07:38 | |
| *** kiran-r has joined #openstack-infra | 07:38 | |
| *** bharathm has quit IRC | 07:41 | |
| *** dizquierdo has joined #openstack-infra | 07:41 | |
| *** bexelbie_ has joined #openstack-infra | 07:43 | |
| *** sharikapongubala has quit IRC | 07:44 | |
| *** vivekd has quit IRC | 07:46 | |
| *** bexelbie_ is now known as bexelbie | 07:47 | |
| *** yfried is now known as yfried|afk | 07:47 | |
| *** kdas_ is now known as kushal | 07:48 | |
| *** kushal has joined #openstack-infra | 07:49 | |
| *** HeOS has quit IRC | 07:52 | |
| *** sharikapongubala has joined #openstack-infra | 07:52 | |
| *** jamespage has joined #openstack-infra | 07:53 | |
| *** jlanoux has joined #openstack-infra | 07:53 | |
| openstackgerrit | Evgeny Antyshev proposed openstack-infra/devstack-gate: Third-party rhel-based oses in devstack-gate https://review.openstack.org/215029 | 07:53 |
| *** yfried|afk is now known as yfried | 07:57 | |
| *** luqas has quit IRC | 07:58 | |
| *** arxcruz has joined #openstack-infra | 08:02 | |
| *** ihrachys has joined #openstack-infra | 08:03 | |
| *** sthillma has quit IRC | 08:04 | |
| *** gildub has quit IRC | 08:07 | |
| *** jistr has joined #openstack-infra | 08:08 | |
| *** lennyb has joined #openstack-infra | 08:11 | |
| *** snikitin has joined #openstack-infra | 08:11 | |
| *** fhubik has quit IRC | 08:13 | |
| *** fhubik has joined #openstack-infra | 08:13 | |
| *** e0ne has joined #openstack-infra | 08:13 | |
| snikitin | All nova patches are failing because of job gate-tempest-dsvm-ironic-pxe_ssh | 08:14 |
| ianw | snikitin: maybe try a rerun, it should be ok now. see https://bugs.launchpad.net/openstack-gate/+bug/1490806 | 08:16 |
| openstack | Launchpad bug 1490806 in OpenStack-Gate "https://cloud-images.ubuntu.com/trusty/current/ returning 403 ; various tests failing" [Undecided,New] | 08:16 |
| *** cdent has joined #openstack-infra | 08:17 | |
| snikitin | ianw: thanks, I'll try | 08:18 |
| openstackgerrit | Antoine Musso proposed openstack/diskimage-builder: Disable DIB_DEBUG_TRACE by default https://review.openstack.org/219176 | 08:23 |
| *** yamahata has joined #openstack-infra | 08:26 | |
| *** fedora has quit IRC | 08:26 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/project-config: Create job for infra ansible https://review.openstack.org/219181 | 08:30 |
| *** ddieterly has joined #openstack-infra | 08:31 | |
| *** maishsk has quit IRC | 08:31 | |
| *** sirushti has joined #openstack-infra | 08:32 | |
| *** HeOS has joined #openstack-infra | 08:33 | |
| *** jcoufal has joined #openstack-infra | 08:34 | |
| *** ddieterly has quit IRC | 08:35 | |
| *** e0ne has quit IRC | 08:36 | |
| *** bexelbie_ has joined #openstack-infra | 08:38 | |
| *** bexelbie is now known as Guest2378 | 08:38 | |
| *** bexelbie_ is now known as bexelbie | 08:38 | |
| *** jerryz has quit IRC | 08:40 | |
| *** bharathm has joined #openstack-infra | 08:41 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/project-config: Gate infra-ansible with gate-infra-ansible job https://review.openstack.org/219185 | 08:42 |
| *** dims has joined #openstack-infra | 08:42 | |
| *** hichihara has quit IRC | 08:45 | |
| *** bharathm has quit IRC | 08:46 | |
| *** katyafervent is now known as katyafervent_awa | 08:47 | |
| *** e0ne has joined #openstack-infra | 08:48 | |
| *** dims has quit IRC | 08:48 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/system-config: Move manage_projects logic to puppet-jeepyb https://review.openstack.org/171565 | 08:49 |
| *** derekh has joined #openstack-infra | 08:49 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/system-config: Remove graphite wrapper class https://review.openstack.org/216693 | 08:50 |
| anteaya | fungi: once you are up and about, can we get Intel PCI CI re-enabled please? gerrit account id 15751 | 08:50 |
| anteaya | or anyone else who cares to do so | 08:50 |
| *** katyafervent_awa is now known as katyafervent | 08:51 | |
| *** ociuhandu has quit IRC | 08:54 | |
| *** dingyichen has quit IRC | 08:57 | |
| *** kiran-r has quit IRC | 08:57 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/infra-specs: storyBoard: Worklists and Boards https://review.openstack.org/202989 | 09:00 |
| *** boris-42 has quit IRC | 09:00 | |
| *** jaosorior has joined #openstack-infra | 09:00 | |
| *** maishsk has joined #openstack-infra | 09:01 | |
| *** marzif has joined #openstack-infra | 09:01 | |
| *** yamahata has quit IRC | 09:01 | |
| *** kiran-r has joined #openstack-infra | 09:04 | |
| *** marzif has quit IRC | 09:04 | |
| *** marzif has joined #openstack-infra | 09:05 | |
| *** pelix has joined #openstack-infra | 09:05 | |
| *** sharikapongubala has quit IRC | 09:05 | |
| *** shashank_hegde has quit IRC | 09:06 | |
| *** sshnaidm has quit IRC | 09:07 | |
| *** sirushti has quit IRC | 09:08 | |
| *** yfried is now known as yfried|afk | 09:09 | |
| *** hashar has joined #openstack-infra | 09:10 | |
| *** sshnaidm has joined #openstack-infra | 09:10 | |
| *** sirushti has joined #openstack-infra | 09:11 | |
| *** ildikov has quit IRC | 09:13 | |
| *** ociuhandu has joined #openstack-infra | 09:16 | |
| *** abregman is now known as abregman|lunch | 09:19 | |
| *** fhubik has quit IRC | 09:19 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/diskimage-builder: Map ubuntu versions to match the SHA256SUMs https://review.openstack.org/219199 | 09:20 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/infra-specs: StoryBoard: Worklists and Boards https://review.openstack.org/202989 | 09:24 |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-jenkins: Don't configure memory swap bytes on cgroups for Trusty https://review.openstack.org/218227 | 09:26 |
| *** fhubik has joined #openstack-infra | 09:28 | |
| *** yfried|afk is now known as yfried | 09:29 | |
| *** ddieterly has joined #openstack-infra | 09:31 | |
| *** ddieterly has quit IRC | 09:36 | |
| *** john-davidge has joined #openstack-infra | 09:36 | |
| *** dims has joined #openstack-infra | 09:36 | |
| *** mrmartin has quit IRC | 09:36 | |
| snikitin | ianw: Now http://cloud-images.ubuntu.com doesn't return 403, but job gate-tempest-dsvm-ironic-pxe_ssh is still failing with | 09:38 |
| snikitin | sha256sum: standard input: no properly formatted SHA256 checksum lines found | 09:38 |
| *** yfried is now known as yfried|afk | 09:39 | |
| tsufiev | hey, folks! Where in openstack-infra/config can I find the details about the Devstack configuration being used for Horizon integration tests? | 09:39 |
| *** john-davidge_ has joined #openstack-infra | 09:40 | |
| rcarrillocruz | tsufiev: you want to look at openstack-infra/project-config | 09:40 |
| rcarrillocruz | that's where the project tests are kept | 09:40 |
| *** john-davidge has quit IRC | 09:40 | |
| *** john-davidge_ is now known as john-davidge | 09:40 | |
| *** dims has quit IRC | 09:41 | |
| *** dtantsur is now known as dtantsur|brb | 09:41 | |
| tsufiev | rcarrillocruz, thanks, that was indeed the wrong repo ) | 09:41 |
| *** yfried|afk is now known as yfried | 09:45 | |
| *** asettle has joined #openstack-infra | 09:48 | |
| *** ildikov has joined #openstack-infra | 09:48 | |
| openstackgerrit | Claudiu Belu proposed openstack/requirements: WIP: Removes posix-ipc from global-requirements https://review.openstack.org/217824 | 09:48 |
| *** claudiub has joined #openstack-infra | 09:56 | |
| *** maishsk has quit IRC | 09:56 | |
| *** coolsvap has quit IRC | 10:01 | |
| *** coolsvap has joined #openstack-infra | 10:02 | |
| *** Guest41206 has quit IRC | 10:03 | |
| *** rkukura_ has joined #openstack-infra | 10:03 | |
| *** yfried is now known as yfried|afk | 10:04 | |
| *** Sree has joined #openstack-infra | 10:04 | |
| *** maishsk has joined #openstack-infra | 10:05 | |
| *** maishsk has quit IRC | 10:05 | |
| *** sharikapongubala has joined #openstack-infra | 10:06 | |
| *** rkukura has quit IRC | 10:06 | |
| *** rkukura_ is now known as rkukura | 10:06 | |
| *** Zhongjun has quit IRC | 10:07 | |
| *** Sree has quit IRC | 10:08 | |
| *** maishsk has joined #openstack-infra | 10:08 | |
| *** yfried|afk is now known as yfried | 10:10 | |
| *** sharikapongubala has quit IRC | 10:10 | |
| *** mrmartin has joined #openstack-infra | 10:11 | |
| *** skolekonov has joined #openstack-infra | 10:13 | |
| *** jeadre has joined #openstack-infra | 10:17 | |
| *** dizquierdo has quit IRC | 10:18 | |
| *** claudiub has quit IRC | 10:18 | |
| openstackgerrit | Merged openstack-infra/groups: Remove local Leaflet library and use version 0.7.3 https://review.openstack.org/218603 | 10:20 |
| *** abregman|lunch is now known as abregman | 10:22 | |
| *** dtantsur|brb is now known as dtantsur | 10:23 | |
| *** marzif has quit IRC | 10:24 | |
| *** yfried is now known as yfried|afk | 10:25 | |
| openstackgerrit | Merged openstack-infra/groups: Use '#!/usr/bin/env bash' as shebang in script files https://review.openstack.org/218602 | 10:26 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/puppet-pip: Fix the puppet-pip module to install pip https://review.openstack.org/211016 | 10:29 |
| *** btully has quit IRC | 10:29 | |
| *** yfried|afk is now known as yfried | 10:29 | |
| *** hashar has quit IRC | 10:29 | |
| openstackgerrit | Merged openstack-infra/groups: Cleanup and convert the README file to RST https://review.openstack.org/218601 | 10:30 |
| *** Sree has joined #openstack-infra | 10:30 | |
| *** fhubik is now known as fhubik_brb | 10:30 | |
| *** dims has joined #openstack-infra | 10:30 | |
| *** Sree has quit IRC | 10:30 | |
| *** bharathm has joined #openstack-infra | 10:31 | |
| *** Sree has joined #openstack-infra | 10:31 | |
| openstackgerrit | Ekaterina Chernova proposed openstack/requirements: Add django-formtools to global requirements https://review.openstack.org/219222 | 10:31 |
| *** yamamoto has quit IRC | 10:32 | |
| *** ddieterly has joined #openstack-infra | 10:32 | |
| *** pc_m has joined #openstack-infra | 10:32 | |
| *** bharathm has quit IRC | 10:35 | |
| *** dims has quit IRC | 10:36 | |
| *** ddieterly has quit IRC | 10:37 | |
| *** maishsk has quit IRC | 10:39 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/puppet-pip: Fix the puppet-pip module to install pip https://review.openstack.org/211016 | 10:39 |
| *** fhubik_brb has quit IRC | 10:40 | |
| *** bexelbie has quit IRC | 10:42 | |
| *** bexelbie has joined #openstack-infra | 10:43 | |
| *** marzif has joined #openstack-infra | 10:44 | |
| *** ociuhandu_ has joined #openstack-infra | 10:44 | |
| *** ociuhandu has quit IRC | 10:45 | |
| *** ociuhandu_ is now known as ociuhandu | 10:45 | |
| *** ZZelle has joined #openstack-infra | 10:48 | |
| *** haypo has joined #openstack-infra | 10:53 | |
| *** ParsectiX has quit IRC | 10:53 | |
| haypo | hi. my change https://review.openstack.org/#/c/213817/ was approved but sdague changed the commit message to add "(Pulled from gate, cinder still needs unit test fix before this can pass)" | 10:54 |
| haypo | i'm not aware of cinder issues. what is the problem? | 10:54 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Clean any yum cached data https://review.openstack.org/219227 | 10:54 |
| sdague | haypo: https://github.com/openstack/cinder/commit/5b45dcf3a097efa307e31ab7c8c50719d4d76969 | 10:56 |
| sdague | it' | 10:56 |
| sdague | it's fixed now | 10:56 |
| sdague | so you can delete that part and get tests run on it | 10:57 |
| sdague | haypo: you will notice after I did that, pep8 and python27 failed your patch, right? | 10:57 |
| haypo | sdague: a cinder changed introduced a requirement to OpenSSL and it broke the gates? | 10:58 |
| sdague | nope | 10:58 |
| sdague | glanceclient used to require openssl | 10:58 |
| *** Sree has quit IRC | 10:58 | |
| sdague | the new release does not | 10:58 |
| *** Sree has joined #openstack-infra | 10:59 | |
| sdague | cinder introduced code that used openssl and didn't add the requirement, because it worked when glanceclient installed it for it | 10:59 |
| sdague | which then broke after the new glanceclient release | 10:59 |
| haypo | sdague: ah no, i didn't notice the pep8 and py27 failures. i will see if it's now fixed ;) | 10:59 |
| *** bexelbie is now known as bexelbie|lunch | 10:59 | |
| *** hrubi has quit IRC | 10:59 | |
| openstackgerrit | Marton Kiss proposed openstack-infra/groups: Remove duplicate declaration in groups ambassadors ask form. https://review.openstack.org/219229 | 11:00 |
| sdague | yeh, based on what's happening in check, I think it is | 11:00 |
| *** spzala has joined #openstack-infra | 11:01 | |
| *** marzif has quit IRC | 11:02 | |
| *** FallenPegasus has joined #openstack-infra | 11:02 | |
| *** aukhan has quit IRC | 11:03 | |
| *** Sree has quit IRC | 11:03 | |
| *** yfried is now known as yfried|afk | 11:04 | |
| *** vinay_yadhav has joined #openstack-infra | 11:06 | |
| *** FallenPegasus has quit IRC | 11:07 | |
| *** sharikapongubala has joined #openstack-infra | 11:07 | |
| *** btully has joined #openstack-infra | 11:07 | |
| openstackgerrit | Merged openstack-infra/groups: Remove duplicate declaration in groups ambassadors ask form. https://review.openstack.org/219229 | 11:08 |
| haypo | sdague: oh. pep8 failed locally, "Error importing module cinder.volume.drivers.emc.emc_vmax_common: No module named OpenSSL" | 11:09 |
| sdague | haypo: rebase on master | 11:10 |
| sdague | and rebuild your tox envs | 11:10 |
| *** vinay_yadhav has quit IRC | 11:10 | |
| haypo | sdague: oh ok :) it's that simple, cool. thanks | 11:11 |
| *** sharikapongubala has quit IRC | 11:12 | |
| *** slagle has joined #openstack-infra | 11:12 | |
| *** btully has quit IRC | 11:12 | |
| *** iraw_ has joined #openstack-infra | 11:13 | |
| openstackgerrit | John Trowbridge proposed openstack/diskimage-builder: [ironic-agent] Use svc-map for enabling agent https://review.openstack.org/218879 | 11:13 |
| *** claudiub has joined #openstack-infra | 11:14 | |
| *** jaypipes has joined #openstack-infra | 11:14 | |
| *** ParsectiX has joined #openstack-infra | 11:17 | |
| dtantsur | hey folks! could anyone check my suspection that gate-tempest-dsvm-ironic-pxe_ssh-dib runs on DIB from PyPI instead of a DIB patch in question? | 11:17 |
| *** maishsk has joined #openstack-infra | 11:17 | |
| dtantsur | I'm suspecting it because of https://review.openstack.org/#/c/219199 I don't see the results of the patch in the devstack logs | 11:17 |
| sdague | dtantsur: yes, that's intentional | 11:18 |
| dtantsur | oh? so it's not possible to fix DIB gate? | 11:18 |
| dtantsur | what was the reasoning behind it? | 11:19 |
| *** wenlock has quit IRC | 11:19 | |
| sdague | dtantsur: the reasoning was to decouple all of these things | 11:19 |
| *** asettle has quit IRC | 11:20 | |
| sdague | but apparently no one responsible for this all understands the gate architecture | 11:20 |
| *** wenlock has joined #openstack-infra | 11:20 | |
| dtantsur | sdague, so, do you have any hints how to unbreak DIB gate, if it does not pick the change that fixes it? | 11:20 |
| *** zeih__ has joined #openstack-infra | 11:21 | |
| *** lucasagomes has joined #openstack-infra | 11:21 | |
| dtantsur | lucasagomes, failure on https://review.openstack.org/#/c/219199/ looks like we're not testing DIB from source on DIB itself (unless I'm confusing things) | 11:22 |
| *** zeih has quit IRC | 11:22 | |
| lucasagomes | dtantsur, lol seriously? | 11:22 |
| dtantsur | that's what I suspect, I'm asking people to confirm :) | 11:23 |
| lucasagomes | ouch hmm right | 11:23 |
| * lucasagomes look at the logs | 11:23 | |
| *** hodos has joined #openstack-infra | 11:23 | |
| *** yfried|afk is now known as yfried | 11:25 | |
| *** dims has joined #openstack-infra | 11:25 | |
| openstackgerrit | Sean Dague proposed openstack-infra/project-config: make gate-tempest-dsvm-ironic-pxe_ssh non voting https://review.openstack.org/219240 | 11:25 |
| *** dizquierdo has joined #openstack-infra | 11:26 | |
| dtantsur | sdague, "no longer pulling from the open internet" is something that all ironic gates do, and probably it's not trivial to fix.. | 11:27 |
| sdague | dtantsur: I'm sure it's not trivial to fix, but if you want jobs to cross vote on other projects, you have to | 11:27 |
| sdague | we've gone through extradinary effort in building caching layers into nodepool for this reason | 11:28 |
| dtantsur | sdague, fair enough, not sure why DIB does not support it (or it does?) | 11:28 |
| sdague | no idea, someone else needs to figure out a sane architecture here, I just know what is currently done is not sane. | 11:29 |
| *** dims has quit IRC | 11:29 | |
| *** hashar has joined #openstack-infra | 11:29 | |
| sdague | sorry for being grumpy, but this is the 5th or 6th "100% fail breaks everything" change I've looked at in 3 days | 11:29 |
| sdague | so my patience is low | 11:29 |
| dtantsur | sdague, no problem, I hear you | 11:30 |
| dtantsur | I just don't know how to make DIB work with infra mirrors (?) | 11:30 |
| *** vinay_yadhav has joined #openstack-infra | 11:31 | |
| sdague | dtantsur: is there a reason you can't just build a static image perodically and use that? | 11:31 |
| sdague | does ironic actually have to build the image every time? | 11:31 |
| dtantsur | sdague, we'll probably do it with move to IPA (i.e. pretty soon). the problem is DIB itself: would be nice to have a job verifying that we can still build an ironic ramdisk | 11:32 |
| dtantsur | * IPA = ironic-python-agent | 11:32 |
| lucasagomes | sdague, can we change the base OS for that job to something else? | 11:32 |
| lucasagomes | until we fix stuff on ubuntu? | 11:32 |
| sdague | dtantsur: right, so that's a functional test job on dib | 11:32 |
| *** thorst has joined #openstack-infra | 11:33 | |
| sdague | not integration tests with tempest and nvoa | 11:33 |
| *** wenlock has quit IRC | 11:33 | |
| *** fhubik has joined #openstack-infra | 11:33 | |
| *** ddieterly has joined #openstack-infra | 11:33 | |
| dtantsur | sdague, probably tempest is an overkill, but we still have to make sure that image we just built works | 11:33 |
| sdague | dtantsur: then build just the test that does that, and run that in isolation. The current test architecture here is a big ball of mud. | 11:34 |
| *** yfried is now known as yfried|afk | 11:35 | |
| dtantsur | yeah, something like that... | 11:35 |
| dtantsur | lucasagomes, we need to rethink our testing with move to IPA | 11:35 |
| dtantsur | good topic for the summit, also | 11:35 |
| *** gordc has joined #openstack-infra | 11:36 | |
| sdague | lucasagomes: changing out the base os is probably no really helpful, plus, all the projects where that is changable are blocked by this | 11:36 |
| sdague | where things went really wrong here is the -dib job is supposed to be using dib from git | 11:36 |
| sdague | however no one implemented that | 11:36 |
| lucasagomes | sdague, yeah I hear you. Let's make it non-vote then | 11:37 |
| lucasagomes | I will update the patch I put for DIB to map the ubuntu names to versions | 11:37 |
| sdague | lucasagomes: yeh, that's https://review.openstack.org/219240 | 11:37 |
| sdague | fungi: you up for fast approve on https://review.openstack.org/219240 ? | 11:38 |
| lucasagomes | sdague, thanks | 11:38 |
| *** ddieterly has quit IRC | 11:38 | |
| sdague | lucasagomes: also, if you post up changes to devstack so that DIB supports libs from git, we can get the -dib job to do the right thing | 11:38 |
| *** yfried|afk is now known as yfried | 11:38 | |
| sdague | but the support isn't even in devstack at the moment | 11:38 |
| sdague | so there was a massive disconnect somewhere when I told folks about spliting up these jobs | 11:39 |
| *** bexelbie|lunch is now known as bexelbie | 11:39 | |
| sdague | because, the theory would be that the -dib job would be a version of the pxe_ssh job that used dib from git (which would be sensible). And in the rest of the jobs it's from the last release | 11:40 |
| sdague | so dib can test new changes and make sure they won't break folks | 11:40 |
| sdague | that's how we do most of the client libs now | 11:40 |
| sdague | except, apparently the message was lost intranslation, and only the job got renamed, without doing the other work | 11:40 |
| *** ParsectiX has quit IRC | 11:41 | |
| lucasagomes | sdague, yeah seems a ball of wax, but I will take a look at it | 11:41 |
| lucasagomes | 1) fix devstack to git clone and install dib 2) update the job to do it | 11:42 |
| lucasagomes | and ofc fix ubuntu | 11:42 |
| fungi | anteaya: jyuso: i've reactivated the intel pci ci account now | 11:42 |
| *** nadya has joined #openstack-infra | 11:42 | |
| *** glauco has joined #openstack-infra | 11:45 | |
| openstackgerrit | Joseph Lanoux proposed openstack-infra/project-config: Enable SSL in devstack for experimental checks. https://review.openstack.org/180968 | 11:45 |
| *** e0ne has quit IRC | 11:46 | |
| sdague | lucasagomes: the devstack patches aren't huge, look at how they were done for oslo libs or novaclient | 11:46 |
| sdague | lucasagomes: thanks for looking at it | 11:47 |
| lucasagomes | sdague, will do! thanks | 11:47 |
| mescanef | hi! folks, regarding stackforge migration and its wiki page: https://wiki.openstack.org/wiki/Stackforge_Namespace_Retirement - what in case of fuel-plugins? lots of them are already there, and there was a idea, that for keeping "consistency" these fuel-plugin-* projects would *stay* in stackforge namespace. is that possible? | 11:49 |
| *** jyuso has quit IRC | 11:49 | |
| *** jyuso1 has joined #openstack-infra | 11:49 | |
| mescanef | fungi: mordred ^^ | 11:49 |
| *** nadya has quit IRC | 11:50 | |
| fungi | mescanef: you probably misunderstood the consistency argument. we said it was okay to create new fuel plugins in the stackforge namespace to keep them together for now, and then we'll move them all at the same time together into the openstack namespace in a couple months | 11:50 |
| *** nadya has joined #openstack-infra | 11:50 | |
| *** pradk has joined #openstack-infra | 11:50 | |
| fungi | rather than creating new ones in the openstack namespace now while the old ones are still in the stackforge namespace for the next couple months | 11:51 |
| openstackgerrit | Merged openstack-infra/project-config: make gate-tempest-dsvm-ironic-pxe_ssh non voting https://review.openstack.org/219240 | 11:51 |
| anteaya | fungi: thank you | 11:53 |
| *** sergmelikyan has joined #openstack-infra | 11:54 | |
| *** jyuso has joined #openstack-infra | 11:54 | |
| mescanef | fungi: so every fuel-plugin-* we shall add to the wiki page, yes? | 11:54 |
| *** rfolco has joined #openstack-infra | 11:55 | |
| sdague | lucasagomes: you have a bug up for the dib / sha256 mismatch? | 11:55 |
| lucasagomes | sdague, yeah 1 sec, dtantsur opened one. | 11:55 |
| lucasagomes | sdague, https://bugs.launchpad.net/diskimage-builder/+bug/1490864 | 11:56 |
| openstack | Launchpad bug 1490864 in diskimage-builder "Ubuntu mirror file renaming broken "ubuntu" element" [Undecided,In progress] - Assigned to Lucas Alvares Gomes (lucasagomes) | 11:56 |
| *** mase_x200 has joined #openstack-infra | 11:56 | |
| sdague | lucasagomes: great, thanks, I wanted to get that signature into our system | 11:56 |
| lucasagomes | cool | 11:56 |
| *** jyuso1 has quit IRC | 11:58 | |
| *** jyuso has quit IRC | 11:58 | |
| openstackgerrit | Sean Dague proposed openstack-infra/elastic-recheck: add query for sha256 dib / ironic issue https://review.openstack.org/219246 | 11:58 |
| *** iraw_ has quit IRC | 12:01 | |
| *** nadya has quit IRC | 12:01 | |
| *** lucasagomes is now known as lucas-hungry | 12:01 | |
| * lucas-hungry brb's quick getting a sandwich | 12:02 | |
| *** iraw_ has joined #openstack-infra | 12:02 | |
| *** ParsectiX has joined #openstack-infra | 12:03 | |
| *** yfried is now known as yfried|afk | 12:03 | |
| *** iraw_ has quit IRC | 12:06 | |
| *** yfried|afk is now known as yfried | 12:07 | |
| *** mwagner_lap has quit IRC | 12:08 | |
| *** iraw_ has joined #openstack-infra | 12:08 | |
| *** chlong_ has joined #openstack-infra | 12:09 | |
| *** annegentle has joined #openstack-infra | 12:11 | |
| openstackgerrit | Joseph Lanoux proposed openstack-infra/project-config: Enable SSL in devstack for experimental checks. https://review.openstack.org/180968 | 12:13 |
| *** salv-orlando has quit IRC | 12:14 | |
| *** nicodemos has joined #openstack-infra | 12:15 | |
| EmilienM | good morning infra, I have an outstanding request for puppet CI: https://review.openstack.org/#/c/219078/ - thanks a lot :-) | 12:16 |
| *** jcoufal has quit IRC | 12:17 | |
| *** vinay_yadhav has quit IRC | 12:17 | |
| *** trown|outttypeww is now known as trown | 12:18 | |
| *** e0ne has joined #openstack-infra | 12:18 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/os-loganalyze: Add support for listing folder indexes https://review.openstack.org/214207 | 12:18 |
| *** vinay_yadhav has joined #openstack-infra | 12:18 | |
| *** bharathm has joined #openstack-infra | 12:20 | |
| *** bswartz has joined #openstack-infra | 12:21 | |
| fungi | mescanef: yes, if your repository is active (you don't want it retired) and it's in the stackforge namespace now, it should go on the list of repos to migrate in that wiki article | 12:21 |
| *** baoli has joined #openstack-infra | 12:21 | |
| *** salv-orlando has joined #openstack-infra | 12:22 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: add query for sha256 dib / ironic issue https://review.openstack.org/219246 | 12:23 |
| *** salv-orl_ has joined #openstack-infra | 12:23 | |
| openstackgerrit | Glauco Oliveira proposed openstack-infra/puppet-zuul: Add acceptance tests for puppet-zuul https://review.openstack.org/217732 | 12:23 |
| *** bharathm has quit IRC | 12:24 | |
| *** dizquierdo has quit IRC | 12:25 | |
| *** dkranz has joined #openstack-infra | 12:26 | |
| *** yfried has quit IRC | 12:26 | |
| *** salv-orlando has quit IRC | 12:26 | |
| sdague | fungi: any idea if the ironic non voting change has propogated yet? | 12:27 |
| *** abregman is now known as abregman|afk | 12:29 | |
| fungi | sdague: i'll check zuul's logs | 12:29 |
| mescanef | fungi: ok tnx | 12:29 |
| *** aysyd has joined #openstack-infra | 12:30 | |
| EmilienM | fungi: thx for +a | 12:30 |
| *** burgerk has joined #openstack-infra | 12:30 | |
| *** dprince has joined #openstack-infra | 12:31 | |
| fungi | sdague: yep, it showed up nonvoting on a zuul config reload about 26 minutes ago | 12:31 |
| *** cburroughs has joined #openstack-infra | 12:31 | |
| EmilienM | can we read zuul logs somewhere ? | 12:32 |
| *** dims has joined #openstack-infra | 12:33 | |
| *** salv-orlando has joined #openstack-infra | 12:33 | |
| *** ddieterly has joined #openstack-infra | 12:34 | |
| openstackgerrit | svasheka proposed openstack-infra/project-config: Change screenshots source destination for gate-horizon-dsvm-integration job https://review.openstack.org/219254 | 12:34 |
| sdague | fungi: thanks | 12:34 |
| openstackgerrit | Merged openstack-infra/project-config: zuul/layout: make non-voting some trusty beaker jobs https://review.openstack.org/219078 | 12:34 |
| *** woodster_ has joined #openstack-infra | 12:36 | |
| fungi | EmilienM: unfortunately no, i have to ssh into the server and pull up the logs manually. we have a general distrust of logs for applications which might leak credentials | 12:36 |
| *** salv-orl_ has quit IRC | 12:37 | |
| EmilienM | fungi: ok no problem :-) - I was just curious to know | 12:37 |
| *** ddieterly has quit IRC | 12:38 | |
| *** mase_x200 has quit IRC | 12:38 | |
| *** fhubik has quit IRC | 12:39 | |
| *** hrubi has joined #openstack-infra | 12:39 | |
| *** dkranz has quit IRC | 12:40 | |
| *** jcoufal has joined #openstack-infra | 12:40 | |
| *** lucas-hungry is now known as lucasagomes | 12:40 | |
| *** dkranz has joined #openstack-infra | 12:42 | |
| openstackgerrit | Paul Michali proposed openstack-infra/project-config: Create experimental jobs for VPN DevStack plugin https://review.openstack.org/209887 | 12:43 |
| *** Sree has joined #openstack-infra | 12:44 | |
| *** puranamr has joined #openstack-infra | 12:44 | |
| *** hashar_ has joined #openstack-infra | 12:45 | |
| *** fedora has joined #openstack-infra | 12:45 | |
| *** amuller has joined #openstack-infra | 12:46 | |
| *** hashar has quit IRC | 12:46 | |
| *** dprince has quit IRC | 12:46 | |
| *** dprince has joined #openstack-infra | 12:48 | |
| *** nadya has joined #openstack-infra | 12:48 | |
| *** mase_x200 has joined #openstack-infra | 12:49 | |
| *** ddieterly has joined #openstack-infra | 12:49 | |
| *** hashar has joined #openstack-infra | 12:49 | |
| *** thiagop has joined #openstack-infra | 12:50 | |
| *** FallenPegasus has joined #openstack-infra | 12:50 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 12:51 | |
| *** puranamr has quit IRC | 12:51 | |
| *** rlandy has joined #openstack-infra | 12:52 | |
| *** derekh has quit IRC | 12:52 | |
| *** hashar_ has quit IRC | 12:52 | |
| *** yamamoto has joined #openstack-infra | 12:54 | |
| *** FallenPegasus has quit IRC | 12:55 | |
| *** puranamr has joined #openstack-infra | 12:56 | |
| *** amotoki has quit IRC | 12:57 | |
| *** ddieterly has quit IRC | 12:58 | |
| *** iraw_ has quit IRC | 12:58 | |
| *** mase_x200 has quit IRC | 12:58 | |
| *** oomichi has quit IRC | 12:58 | |
| *** iraw_ has joined #openstack-infra | 12:58 | |
| *** fhubik has joined #openstack-infra | 12:58 | |
| *** tlian has joined #openstack-infra | 12:59 | |
| *** mase_x200 has joined #openstack-infra | 12:59 | |
| *** abregman|afk is now known as abregman | 13:00 | |
| *** bexelbie_ has joined #openstack-infra | 13:01 | |
| *** bexelbie has quit IRC | 13:02 | |
| *** oomichi has joined #openstack-infra | 13:03 | |
| *** mase_x200 has quit IRC | 13:03 | |
| *** salv-orlando has quit IRC | 13:04 | |
| *** samuelBartel has joined #openstack-infra | 13:04 | |
| *** k4n0 has quit IRC | 13:05 | |
| *** rossella_s has quit IRC | 13:05 | |
| *** rossella_s has joined #openstack-infra | 13:06 | |
| *** vinay_yadhav has quit IRC | 13:07 | |
| *** sergmelikyan has quit IRC | 13:07 | |
| *** mjturek1 has joined #openstack-infra | 13:07 | |
| *** vinay_yadhav has joined #openstack-infra | 13:07 | |
| *** miqui has joined #openstack-infra | 13:08 | |
| *** gordc has quit IRC | 13:09 | |
| *** sergmelikyan has joined #openstack-infra | 13:12 | |
| *** watanabe_isao has quit IRC | 13:13 | |
| *** nadya has quit IRC | 13:13 | |
| *** esker has quit IRC | 13:14 | |
| *** amitgandhinz has joined #openstack-infra | 13:16 | |
| *** sdake has quit IRC | 13:17 | |
| *** burgerk has quit IRC | 13:17 | |
| *** puranamr has quit IRC | 13:20 | |
| *** iraw__ has joined #openstack-infra | 13:20 | |
| *** iraw__ has quit IRC | 13:21 | |
| *** sdake has joined #openstack-infra | 13:21 | |
| *** julim has joined #openstack-infra | 13:22 | |
| *** iraw_ has quit IRC | 13:22 | |
| *** puranamr has joined #openstack-infra | 13:23 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 13:24 | |
| *** mriedem has joined #openstack-infra | 13:24 | |
| *** dkranz has quit IRC | 13:25 | |
| openstackgerrit | Thierry Carrez proposed openstack-infra/release-tools: Script to release cycle-with-intermediary https://review.openstack.org/219274 | 13:26 |
| *** tiswanso has joined #openstack-infra | 13:26 | |
| *** changbl has quit IRC | 13:28 | |
| *** dkranz has joined #openstack-infra | 13:28 | |
| *** puranamr has quit IRC | 13:28 | |
| *** geoffarnoldX is now known as geoffarnold | 13:29 | |
| *** bhunter71 has joined #openstack-infra | 13:29 | |
| *** mtanino has joined #openstack-infra | 13:31 | |
| *** dizquierdo has joined #openstack-infra | 13:31 | |
| mordred | fungi: I'm going to release shade - I do not think it should affect us negatively, but should allow us to unpin puppet on nodepool | 13:31 |
| fungi | mordred: please do | 13:32 |
| *** bdemers has quit IRC | 13:33 | |
| ttx | clarkb, fungi: we'll need to enable the requirements cross-check jobs this week, ideally today, so progress needed on https://review.openstack.org/#/c/217300 | 13:33 |
| *** alivigni has joined #openstack-infra | 13:33 | |
| fungi | ttx: yep, was mostly waiting to see if clarkb had replies to your comments on it | 13:34 |
| mordred | fungi: tag pushed | 13:34 |
| *** bexelbie_ is now known as bexelbie | 13:35 | |
| *** bdemers has joined #openstack-infra | 13:36 | |
| *** fhubik is now known as fhubik_brb | 13:38 | |
| *** mwagner_lap has joined #openstack-infra | 13:38 | |
| *** Piet has quit IRC | 13:38 | |
| *** ddieterly has joined #openstack-infra | 13:38 | |
| *** ayoung has joined #openstack-infra | 13:39 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 13:39 | |
| openstackgerrit | Evgeny Antyshev proposed openstack-infra/devstack-gate: Third-party rhel-based oses in devstack-gate https://review.openstack.org/215029 | 13:39 |
| *** kiran-r has quit IRC | 13:40 | |
| *** kiran-r has joined #openstack-infra | 13:41 | |
| *** kiran-r has quit IRC | 13:41 | |
| *** fhubik_brb is now known as fhubik | 13:42 | |
| *** eharney has joined #openstack-infra | 13:44 | |
| *** jsavak has joined #openstack-infra | 13:45 | |
| *** edmondsw has joined #openstack-infra | 13:47 | |
| clarkb | I didnt realize there were comments as its +1 I can address this morning | 13:48 |
| *** sergmelikyan has quit IRC | 13:48 | |
| clarkb | I didnt add to master because for other cross branch thing we typically only do check | 13:49 |
| *** annegentle has quit IRC | 13:49 | |
| clarkb | devstack gate and tempest for example | 13:49 |
| *** gordc has joined #openstack-infra | 13:52 | |
| ttx | fungi, clarkb: could anyone add maishsk@gmail.com to openstack-planet-core at https://review.openstack.org/#/admin/groups/93,members ? Been very helpful reviewing those lately | 13:53 |
| mordred | ttx: done | 13:54 |
| fungi | ttx: done | 13:54 |
| ttx | maishsk: ^ | 13:54 |
| ttx | welcome to the group and thanks again for helping there | 13:54 |
| *** sergmelikyan has joined #openstack-infra | 13:54 | |
| maishsk | ttx: fungi: Thanks. | 13:55 |
| maishsk | Happy to help out. | 13:55 |
| fungi | (seems adding someone to a gerrit group multiple times is a quietly successful no-op) | 13:55 |
| ttx | gerrit is reentrant | 13:55 |
| *** zul has quit IRC | 13:56 | |
| *** zul has joined #openstack-infra | 13:56 | |
| *** glauco has quit IRC | 14:01 | |
| *** mriedem is now known as mriedem_meetings | 14:01 | |
| *** glauco has joined #openstack-infra | 14:01 | |
| openstackgerrit | Paul Michali proposed openstack-infra/project-config: Create experimental jobs for VPN DevStack plugin https://review.openstack.org/209887 | 14:02 |
| *** Kennan has joined #openstack-infra | 14:03 | |
| openstackgerrit | Jim Rollenhagen proposed openstack-infra/project-config: Use pxe_ipa job in Nova https://review.openstack.org/219285 | 14:03 |
| *** Kennan2 has quit IRC | 14:03 | |
| *** ihrachys has quit IRC | 14:06 | |
| *** burgerk has joined #openstack-infra | 14:06 | |
| *** bharathm has joined #openstack-infra | 14:08 | |
| *** sharikapongubala has joined #openstack-infra | 14:09 | |
| openstackgerrit | Merged openstack-infra/release-tools: clarify duties in library release process https://review.openstack.org/219017 | 14:10 |
| *** peristeri has quit IRC | 14:13 | |
| *** zeih__ has quit IRC | 14:13 | |
| *** bharathm has quit IRC | 14:13 | |
| *** ParsectiX has quit IRC | 14:13 | |
| *** sharikapongubala has quit IRC | 14:13 | |
| *** geoffarnold has quit IRC | 14:14 | |
| *** angdraug has joined #openstack-infra | 14:15 | |
| fungi | ttx: we still have a summit.openstack.org server at 2001:4800:7810:512:3bc3:d7f6:ff04:94fc 166.78.0.101 even though that name in dns is now a cname to static.openstack.org | 14:18 |
| *** ihrachys has joined #openstack-infra | 14:18 | |
| fungi | ttx: was there still anything on the old summit.o.o server you needed, or can i delete it now? | 14:19 |
| fungi | ttx: this is _not_ the design-summit-prep.openstack.org server, but rather its predecessor | 14:20 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Pass timeout to session, not constructors https://review.openstack.org/219290 | 14:20 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Remove last vestige of glanceclient being different https://review.openstack.org/219291 | 14:20 |
| mordred | fungi, clarkb: ^^ not urgent - just two cleanups that came out of yesterday's real patch | 14:20 |
| fungi | mordred: do we need another release with those ^ before we can unleash puppet agent on nodepool again? | 14:20 |
| *** amotoki has joined #openstack-infra | 14:20 | |
| fungi | ahh, sounds like no | 14:21 |
| mordred | nope | 14:21 |
| mordred | they're just to clean up behind myself while I'm thinking about it still | 14:21 |
| fungi | great idea | 14:21 |
| mordred | fungi: while I have your attention ... https://review.openstack.org/#/c/212177/ | 14:21 |
| *** wenlock has joined #openstack-infra | 14:22 | |
| *** links has quit IRC | 14:23 | |
| *** annegentle has joined #openstack-infra | 14:25 | |
| ttx | fungi: I'm only using design-summit-prep now, so you can scrap the old summit.o.o server | 14:25 |
| ttx | (if those are different ;) | 14:25 |
| fungi | ttx: thanks! yeah, we're getting tight on quota there so every little bit counts | 14:25 |
| ttx | I thought it was dead already | 14:26 |
| fungi | ttx: looks like the last time you logged into it was april 23 | 14:28 |
| rcarrillocruz | fungi: heya, can you advise why https://review.openstack.org/#/c/211925/ wasn't merged by Jenkins? | 14:28 |
| rcarrillocruz | change it depends on already merged, dunno what happened | 14:28 |
| *** dkranz has quit IRC | 14:29 | |
| *** regXboi has joined #openstack-infra | 14:29 | |
| fungi | rcarrillocruz: the change it depends on doesn't share a change queue with it, and the depending change was approved before the dependency merged | 14:29 |
| *** vinay_yadhav has quit IRC | 14:29 | |
| openstackgerrit | Merged openstack-infra/glean: Create systemwide lock for glean https://review.openstack.org/212177 | 14:30 |
| fungi | 211923 didn't merge until 08-27 16:34 but 211925 was approved before that, at 08-27 16:10 | 14:30 |
| *** shardy_ has joined #openstack-infra | 14:30 | |
| fungi | rcarrillocruz: so you need ttx to remove and re-add his approval on 211925 | 14:30 |
| *** shardy has quit IRC | 14:32 | |
| rcarrillocruz | huh | 14:32 |
| rcarrillocruz | ok | 14:32 |
| rcarrillocruz | ttx: ^ | 14:32 |
| *** sharikapongubala has joined #openstack-infra | 14:32 | |
| rcarrillocruz | could you please have a look? | 14:32 |
| *** maishsk has quit IRC | 14:33 | |
| fungi | ttx: and yeah, this old summit.o.o server does seem to have been the same, consistent with my recollection. it has a /var/www/summitsched with a wsgi app in there | 14:33 |
| fungi | so deleting it now | 14:33 |
| *** shardy_ has quit IRC | 14:35 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Migrate neutron to the common client interface https://review.openstack.org/214243 | 14:36 |
| *** shardy has joined #openstack-infra | 14:36 | |
| *** zz_dimtruck is now known as dimtruck | 14:37 | |
| *** vinay_yadhav has joined #openstack-infra | 14:37 | |
| *** arxcruz has quit IRC | 14:41 | |
| *** afazekas__ has quit IRC | 14:41 | |
| *** boris-42 has joined #openstack-infra | 14:42 | |
| *** tiswanso has quit IRC | 14:44 | |
| *** vinay_yadhav has quit IRC | 14:44 | |
| fungi | clarkb: https://jenkins.openstack.org/pluginManager/ warns that the ssh slaves plugin 0.21 installed there uses different configuration than 1.10. did you need to make any adjustments when upgrading previously from really old versions of it? should we just leave the ancient version there? | 14:44 |
| *** tiswanso has joined #openstack-infra | 14:44 | |
| *** vinayyadhav has joined #openstack-infra | 14:45 | |
| *** shardy_ has joined #openstack-infra | 14:47 | |
| *** hdd has joined #openstack-infra | 14:47 | |
| *** shardy has quit IRC | 14:48 | |
| *** adduarte has quit IRC | 14:50 | |
| clarkb | fungi: I didn't that seems like a really old version though | 14:51 |
| fungi | yep, like really, really, really old | 14:51 |
| fungi | sort of thinking i won't actually upgrade it until things quiet down, maybe on a weekend | 14:51 |
| clarkb | sounds good | 14:51 |
| clarkb | especially since it doesn't have the problem with many adds and removes | 14:52 |
| *** shardy_ has quit IRC | 14:52 | |
| *** jlanoux has quit IRC | 14:52 | |
| *** sharikapongubala has quit IRC | 14:53 | |
| *** geoffarnold has joined #openstack-infra | 14:53 | |
| *** glauco has quit IRC | 14:53 | |
| *** shardy has joined #openstack-infra | 14:53 | |
| *** armax has joined #openstack-infra | 14:53 | |
| *** sdake_ has joined #openstack-infra | 14:53 | |
| clarkb | ttx: fungi I realize now I probably want to restrict those new stable/liberty jobs to the master branch in zuul | 14:53 |
| fungi | okay | 14:54 |
| fungi | that sounds like a good idea, agreed | 14:54 |
| clarkb | so making that change and the commit message updates and will push new patch in a minute | 14:54 |
| fungi | k | 14:55 |
| *** tonytan4ever has joined #openstack-infra | 14:56 | |
| *** kgiusti has joined #openstack-infra | 14:56 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config: Add stable/liberty crosscheck jobs for reqs https://review.openstack.org/217300 | 14:57 |
| clarkb | ttx: fungi ^ | 14:57 |
| *** links has joined #openstack-infra | 14:57 | |
| *** sdake has quit IRC | 14:58 | |
| *** jistr is now known as jistr|call | 14:58 | |
| *** angdraug_ has joined #openstack-infra | 14:58 | |
| *** erlon has quit IRC | 14:58 | |
| clarkb | StevenK: if you are still awake please let me know which bits of the scripting I can help with | 14:58 |
| clarkb | StevenK: either running scripts or debugging or writing new ones etc | 14:58 |
| clarkb | StevenK: the big ones I am curious about are the populate transifex from zanata script(s) and the daily update projects from zanata scripts | 14:59 |
| *** bknudson has joined #openstack-infra | 14:59 | |
| *** angdraug has quit IRC | 15:00 | |
| *** daemontool_ has quit IRC | 15:01 | |
| *** fhubik is now known as fhubik_brb | 15:01 | |
| *** flepied has joined #openstack-infra | 15:05 | |
| openstackgerrit | Marton Kiss proposed openstack-infra/groups: Enable message email notification https://review.openstack.org/219307 | 15:05 |
| *** sdake_ is now known as sdake | 15:06 | |
| *** flepied1 has quit IRC | 15:06 | |
| *** burgerk has quit IRC | 15:09 | |
| *** vivekd has joined #openstack-infra | 15:12 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: don't assume we will get an existing tag https://review.openstack.org/219308 | 15:12 |
| *** salv-orlando has joined #openstack-infra | 15:13 | |
| *** pblaho has quit IRC | 15:13 | |
| *** zeih has joined #openstack-infra | 15:13 | |
| *** yuanying has quit IRC | 15:13 | |
| *** cbader has joined #openstack-infra | 15:14 | |
| *** bexelbie has quit IRC | 15:14 | |
| *** gmann has quit IRC | 15:14 | |
| openstackgerrit | SHIGEMATSU Mitsuhiro proposed openstack-infra/project-config: Gate infra-ansible with gate-infra-ansible job https://review.openstack.org/219185 | 15:15 |
| *** yamahata has joined #openstack-infra | 15:16 | |
| *** links has quit IRC | 15:16 | |
| *** dims has quit IRC | 15:17 | |
| *** zeih has quit IRC | 15:18 | |
| *** salv-orlando has quit IRC | 15:18 | |
| *** vinayyadhav has quit IRC | 15:18 | |
| *** vinayyadhav has joined #openstack-infra | 15:19 | |
| *** salv-orlando has joined #openstack-infra | 15:19 | |
| *** mjturek1 has quit IRC | 15:22 | |
| *** krickwix has joined #openstack-infra | 15:23 | |
| *** annegentle has quit IRC | 15:23 | |
| *** annegentle has joined #openstack-infra | 15:23 | |
| *** fhubik_brb is now known as fhubik | 15:24 | |
| *** gmann has joined #openstack-infra | 15:24 | |
| *** sputnik13 has joined #openstack-infra | 15:25 | |
| openstackgerrit | Kurt Taylor proposed openstack-infra/infra-specs: Host an initial CI monitoring dashboard service https://review.openstack.org/194437 | 15:25 |
| openstackgerrit | Jim Rollenhagen proposed openstack-infra/project-config: Use Ironic pxe_ipa job in Nova https://review.openstack.org/219285 | 15:25 |
| *** salv-orlando has quit IRC | 15:26 | |
| jroll | sdague: ^ I think that's what I want, I'm still a bit unsure about project-config things | 15:26 |
| *** jlanoux has joined #openstack-infra | 15:27 | |
| sdague | jroll: ok, will look in a second | 15:27 |
| openstackgerrit | Michael Jeanson proposed openstack-infra/jenkins-job-builder: Add 'do-not-fingerprint' to 'copyartifact' builder https://review.openstack.org/219312 | 15:27 |
| jroll | yeah, no rush. thanks :) | 15:28 |
| *** flepied has quit IRC | 15:29 | |
| *** yuanying has joined #openstack-infra | 15:30 | |
| *** stevemar has joined #openstack-infra | 15:30 | |
| *** flepied has joined #openstack-infra | 15:31 | |
| sdague | jroll: yeh, that looks fine | 15:33 |
| jroll | woo | 15:34 |
| *** vinayyadhav has quit IRC | 15:34 | |
| *** bharathm has joined #openstack-infra | 15:35 | |
| *** ihrachys has quit IRC | 15:36 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Start using keystoneauth for keystone sessions https://review.openstack.org/183176 | 15:36 |
| *** puranamr has joined #openstack-infra | 15:36 | |
| *** flepied has quit IRC | 15:36 | |
| clarkb | fungi: are all but jenkins.o.o done now? | 15:37 |
| *** tiswanso has quit IRC | 15:37 | |
| fungi | clarkb: yep | 15:38 |
| clarkb | I am checking thread counts now, 04 still looks great only ~10 forremoting threads | 15:38 |
| *** puranamr has quit IRC | 15:38 | |
| *** salv-orlando has joined #openstack-infra | 15:38 | |
| clarkb | lifeless sdague: from this point forwrad we don't expect to see the duplicate node usage problem assuming the jenkins ssh slaves plugin upgrades fixed things | 15:39 |
| openstackgerrit | Dmitry Tantsur proposed openstack/diskimage-builder: ironic-agent: ensure dmidecode and ipmitool are installed https://review.openstack.org/211521 | 15:40 |
| sdague | clarkb: \o/ | 15:40 |
| clarkb | lifeless: sdague so let us know if you see that fail case again | 15:40 |
| *** dkranz has joined #openstack-infra | 15:40 | |
| clarkb | sdague: and you saw that the pbr thing was related to sqlalchemy migrate and keystonemiddleware and only affected jobs doing stuff with kilo? | 15:40 |
| sdague | clarkb: yep | 15:41 |
| sdague | I gave that to mriedem last night, he fixed it | 15:41 |
| *** tiswanso has joined #openstack-infra | 15:41 | |
| sdague | it was also his fault, so all's good now | 15:41 |
| *** burgerk has joined #openstack-infra | 15:41 | |
| sdague | I got to say - http://status.openstack.org//elastic-recheck/ is just one meteor strike after another recently | 15:41 |
| *** glauco has joined #openstack-infra | 15:42 | |
| *** sridhar_ram has joined #openstack-infra | 15:42 | |
| clarkb | thats what happens when the entire project is trying to push code through quickly in order to get into the rlease ... | 15:42 |
| clarkb | at least it seems like 3rd milestone timeframe is pretty typically crazyness | 15:42 |
| mtreinish | sdague: but there's no dust cloud :) | 15:42 |
| sdague | clarkb: yeh | 15:43 |
| clarkb | fungi: just checked thread counts across all but jenkins.o.o andthey look good | 15:43 |
| sdague | also the fact that these things exist in a fragile state for a long time, and no one fixes them during the good times | 15:43 |
| sridhar_ram | infra team - need your help - https://review.openstack.org/#/c/211405/ has been Approved yesterday but it hasn't merged and I couldn't find an entry on zuul as well | 15:44 |
| sridhar_ram | please help | 15:44 |
| clarkb | at least in the case of jenkins weirdness upstream ignored my bug for forever and appears someone may have fixed it independently after finding a different bug injira | 15:44 |
| clarkb | so we were tracking it there was just fail to communicate with upstream | 15:44 |
| *** dangers_away is now known as dangers | 15:45 | |
| clarkb | sridhar_ram: the depends on was merged after you approved that change, if they do not share a gate queue then it will not be automatically gated | 15:46 |
| fungi | sridhar_ram: that change and its dependency don't share a change queue, and you approved it before the one it depends on merged | 15:46 |
| *** vikram has joined #openstack-infra | 15:46 | |
| vikram | Hi there | 15:46 |
| fungi | sridhar_ram: someone needs to remove and reapply that approval vote | 15:46 |
| sridhar_ram | fungi: clarkb: thanks .. | 15:46 |
| vikram | Can someone please help me to understand the release process of a big tent project | 15:47 |
| sridhar_ram | will ask the reviewer to -A and +A | 15:47 |
| clarkb | vikram: I think you want to talk to dhellmann lifeless and or ttx | 15:47 |
| vikram | my project belongs to neutron big tent.. | 15:47 |
| vikram | clarkb: thanks for your guidance | 15:47 |
| sridhar_ram | fungi: for next time, what is the correct procedure ? I thought merging the dependency first is the correct order | 15:49 |
| *** samuelBartel has quit IRC | 15:49 | |
| *** j_king has left #openstack-infra | 15:50 | |
| *** bknudson has quit IRC | 15:50 | |
| *** abregman has quit IRC | 15:51 | |
| clarkb | sridhar_ram: it is, but you have to approve the child after the parent merges if they do not share a gate queue | 15:51 |
| *** puranamr has joined #openstack-infra | 15:51 | |
| clarkb | sridhar_ram: if you approve it before the dependency merges then zuul nops because the dependency has not merged yet | 15:51 |
| *** mriedem_meetings is now known as mriedem | 15:51 | |
| Swanson | Had been looking at Ramy's CI stuff. Saw anteaya mention puppet-openstackci as a common CI solution. Is there documentation for this? I see some pages marked as inaccurate... | 15:52 |
| sridhar_ram | clarkb: got it, zuul doesn't retry ? | 15:52 |
| clarkb | sridhar_ram: no | 15:52 |
| clarkb | its event based and that event happened and was processed | 15:52 |
| asselin_ | Swanson, updating docs is wip | 15:52 |
| *** spzala has quit IRC | 15:53 | |
| sridhar_ram | clarkb: I see. All is well, lesson learnt. thanks team! | 15:53 |
| *** fhubik has quit IRC | 15:53 | |
| *** sharikapongubala has joined #openstack-infra | 15:53 | |
| fungi | vikram: you should read http://docs.openstack.org/project-team-guide/ and also talk to the neutron team to see where you fit in | 15:53 |
| vikram | fungi: ok | 15:54 |
| *** Swami has joined #openstack-infra | 15:54 | |
| fungi | vikram: since it sounds like your project is related to neutron, and perhaps in similar ways to lots of others | 15:54 |
| *** yfried has joined #openstack-infra | 15:55 | |
| *** ivar-lazzaro has joined #openstack-infra | 15:55 | |
| vikram | fungi: Since this will my first release.. so checking out things :-) | 15:55 |
| *** sshnaidm has quit IRC | 15:55 | |
| *** ivar-lazzaro has quit IRC | 15:55 | |
| *** ivar-lazzaro has joined #openstack-infra | 15:56 | |
| *** alivigni has quit IRC | 15:57 | |
| *** flepied has joined #openstack-infra | 15:57 | |
| *** juggler has joined #openstack-infra | 15:57 | |
| *** sharikapongubala has quit IRC | 15:57 | |
| *** ociuhandu_ has joined #openstack-infra | 15:58 | |
| *** esker has joined #openstack-infra | 15:59 | |
| *** esker has quit IRC | 16:00 | |
| *** zhenguo has quit IRC | 16:00 | |
| *** skylerberg has joined #openstack-infra | 16:00 | |
| *** eharney has quit IRC | 16:01 | |
| *** esker has joined #openstack-infra | 16:01 | |
| openstackgerrit | Antoine Musso proposed openstack-infra/nodepool: doc: FLUSH PRIVILEGES; is unneeded https://review.openstack.org/219324 | 16:01 |
| *** puranamr has quit IRC | 16:02 | |
| *** sridhar_ram has quit IRC | 16:03 | |
| *** ajmiller has joined #openstack-infra | 16:04 | |
| *** sridhar_ram has joined #openstack-infra | 16:04 | |
| *** baoli has quit IRC | 16:04 | |
| *** hashar has quit IRC | 16:04 | |
| *** baoli has joined #openstack-infra | 16:05 | |
| *** hashar has joined #openstack-infra | 16:05 | |
| *** dims has joined #openstack-infra | 16:05 | |
| *** sdake_ has joined #openstack-infra | 16:05 | |
| *** e0ne has quit IRC | 16:05 | |
| *** jtomasek has quit IRC | 16:06 | |
| *** dtantsur is now known as dtantsur|afk | 16:06 | |
| *** bknudson has joined #openstack-infra | 16:06 | |
| *** ociuhandu_ has quit IRC | 16:07 | |
| *** sdake has quit IRC | 16:09 | |
| clarkb | any objections to me deleting the old git08 now? the centos7 machine has been doing well | 16:09 |
| clarkb | I would also like to replace git07 | 16:09 |
| *** changbl has joined #openstack-infra | 16:10 | |
| *** baoli has quit IRC | 16:10 | |
| clarkb | that will give me a chance to test my latest iptables module update on rax images | 16:10 |
| clarkb | need to go test it on hpcloud centos too | 16:10 |
| *** fhubik has joined #openstack-infra | 16:11 | |
| *** Sukhdev has joined #openstack-infra | 16:11 | |
| *** fhubik has quit IRC | 16:11 | |
| *** burgerk has quit IRC | 16:11 | |
| *** Piet has joined #openstack-infra | 16:12 | |
| openstackgerrit | Jim Rollenhagen proposed openstack-infra/project-config: Switch ironic full tempest to use IPA https://review.openstack.org/219330 | 16:13 |
| *** bcourt has quit IRC | 16:13 | |
| jroll | mtreinish: ^^ gets rid of the dib stuff there | 16:13 |
| mtreinish | jroll: cool, +1 | 16:13 |
| greghaynes | jroll: dib method is broken? | 16:13 |
| JayF | jroll: is that 'devstack-virtual-ironic' section new? | 16:13 |
| JayF | jroll: it looks 10x more slicker way to configure that than before | 16:13 |
| jroll | greghaynes: yes, upstream ubuntu broke the heck out of it | 16:14 |
| jroll | JayF: I don't remember how new | 16:14 |
| greghaynes | jroll: oh, you mean the 403 from cloud-images.ubuntu? | 16:14 |
| jroll | greghaynes: it's not dib, it's dib + ubuntu I think? lucasagomes is digging | 16:14 |
| jroll | maybe, idk | 16:14 |
| lucasagomes | greghaynes, couple of errors there... | 16:15 |
| lucasagomes | greghaynes, the ubuntu element is broken due: https://review.openstack.org/#/c/219199/ | 16:15 |
| greghaynes | I am +1 on removing the dib test btw, just surprised it was broken | 16:15 |
| lucasagomes | greghaynes, and the pxe_ssh-dib job wasn't testing DIB from source | 16:15 |
| *** jaypipes has quit IRC | 16:16 | |
| lucasagomes | so it actually never tested building disk from source to see if changes in dib broke ironic | 16:16 |
| greghaynes | nice | 16:16 |
| *** eharney has joined #openstack-infra | 16:16 | |
| lucasagomes | greghaynes, later on I found that "pip install -e <diskimage-builder repo>" is also broken, that leaves the DIB commands partially installed | 16:16 |
| *** bcourt has joined #openstack-infra | 16:16 | |
| greghaynes | yes, -e (documentedly) not usable | 16:17 |
| lucasagomes | https://bugs.launchpad.net/diskimage-builder/+bug/1491035 | 16:17 |
| openstack | Launchpad bug 1491035 in diskimage-builder "Can't install diskimage-builder via "pip install -e"" [Undecided,New] | 16:17 |
| lucasagomes | greghaynes, right, that's what the libs in devstack does when installing things from source | 16:17 |
| *** jlanoux has quit IRC | 16:17 | |
| greghaynes | aye | 16:17 |
| lucasagomes | so to fix devstack we also need to fix that pip install -e :-( | 16:17 |
| lucasagomes | so it's a ball of wax | 16:18 |
| greghaynes | fun | 16:18 |
| lucasagomes | I will put a workaround on devstack first and leave that pip install -e for later | 16:18 |
| greghaynes | Its not that hard to fix, I just didnt because im not really a -e fan anyhow | 16:18 |
| lucasagomes | greghaynes, right, well that's needed :-/ | 16:18 |
| greghaynes | yea, I might do that if someone doesnt beat me to it then | 16:18 |
| *** enikanorov__ has joined #openstack-infra | 16:18 | |
| lucasagomes | greghaynes, heh please do | 16:19 |
| *** shashank_hegde has joined #openstack-infra | 16:19 | |
| *** ifarkas has quit IRC | 16:20 | |
| lucasagomes | greghaynes, btw where it's documented that -e doesn't work for dib? | 16:20 |
| *** burgerk has joined #openstack-infra | 16:20 | |
| *** esker has quit IRC | 16:20 | |
| greghaynes | lucasagomes: trying to find that now... I just know we had a big deal about this and that was the resolution... | 16:20 |
| *** tjones has joined #openstack-infra | 16:20 | |
| *** dolphm has joined #openstack-infra | 16:20 | |
| lucasagomes | ack | 16:20 |
| clarkb | it is really hard to boot servers in hpcloud ... | 16:21 |
| *** enikanorov has quit IRC | 16:21 | |
| *** dizquierdo has quit IRC | 16:21 | |
| *** sdake_ is now known as sdake | 16:21 | |
| fungi | clarkb: i am wholly in favor of allowing centos 7 to take over more of the cluster | 16:21 |
| clarkb | you have to first use neutron client to get a network lits | 16:22 |
| clarkb | then go back over to nova client to boot a node passing in the network | 16:22 |
| clarkb | and when thats done use neutron client again to make a floating ip then associate it | 16:22 |
| clarkb | oh and nova boot says HTTP 500 unknown error :/ | 16:23 |
| greghaynes | lucasagomes: hrm, having a hard time locating, so sounds like we failed at documenting well ;) | 16:25 |
| lucasagomes | greghaynes, heh well I opened that bug | 16:25 |
| lucasagomes | somehow it's documented that it doesn't work now | 16:25 |
| greghaynes | hah | 16:25 |
| clarkb | sorry you hav eto use nova client for the floating IPs because neutron client thinks creat has too few args | 16:25 |
| *** jaypipes has joined #openstack-infra | 16:27 | |
| *** jistr|call has quit IRC | 16:28 | |
| *** tonytan4ever has quit IRC | 16:28 | |
| *** dannywilson has joined #openstack-infra | 16:28 | |
| openstackgerrit | Ryan Carey proposed openstack-infra/jenkins-job-builder: Add support for Artifactory plugin https://review.openstack.org/173932 | 16:29 |
| clarkb | nibalizer: as soon as centos7 is done updating in hpcloud I will be testing my modifications to the iptables module | 16:30 |
| *** dannywilson has quit IRC | 16:30 | |
| greghaynes | crinkle: Is your launchpad ssh key correct? | 16:31 |
| greghaynes | crinkle: I am about to have two boxes for you | 16:31 |
| *** jeadre has quit IRC | 16:32 | |
| *** haypo has left #openstack-infra | 16:32 | |
| crinkle | greghaynes: i don't have an ssh key in launchpad so whatever you're looking at is probably not correct | 16:33 |
| *** jeadre has joined #openstack-infra | 16:33 | |
| greghaynes | crinkle: heh, im not looking at anything yet, just asking before I go hunting | 16:33 |
| greghaynes | (you should totally add one to lp) | 16:34 |
| crinkle | greghaynes: okay https://launchpad.net/~krinkle/+sshkeys | 16:35 |
| greghaynes | crinkle: awesome | 16:35 |
| docaedo | yolanda: thanks for your feedback on https://review.openstack.org/#/c/218898 | 16:35 |
| docaedo | yolanda: have a question for you on that topic (and for rest of infra team to) | 16:35 |
| docaedo | We are debating whether the script that creates a JSON file full of the assets should live in infra repo, or app-catalog repo (ala https://review.openstack.org/#/c/218670) | 16:36 |
| clarkb | nibalizer: notify implies before right? | 16:36 |
| clarkb | docaedo: if its an app catalog utility regardless of config management system I would put it there | 16:37 |
| docaedo | This is not super long term, but until we have a more complete backend for app-catalog (i.e. something more than a flat yaml file), we want to have reasonable way to retrieve the assets from the horizon plugin | 16:37 |
| clarkb | docaedo: if it is related to some puppet bug or behavior it should go in the puppet module | 16:37 |
| *** trown is now known as trown|lunch | 16:37 | |
| docaedo | clarkb: cool, that makes it a very easy decision, thank you! | 16:38 |
| *** ajmiller has quit IRC | 16:38 | |
| *** pblaho has joined #openstack-infra | 16:39 | |
| nibalizer | clarkb: i think notify implies require | 16:40 |
| clarkb | nibalizer: I think we mean the same thing, it sets up an ordering | 16:41 |
| *** thedodd has joined #openstack-infra | 16:41 | |
| clarkb | such that the notifier happens before the notified resource | 16:41 |
| openstackgerrit | Kevin Fox proposed openstack-infra/puppet-apps_site: Enable CORS for new /api directory and gzip https://review.openstack.org/218898 | 16:41 |
| nibalizer | yes | 16:42 |
| zaro | morning | 16:43 |
| *** MarkAtwood has joined #openstack-infra | 16:45 | |
| *** ashleighfarnham has joined #openstack-infra | 16:45 | |
| openstackgerrit | Kevin Fox proposed openstack-infra/puppet-apps_site: Enable CORS for new /api directory and gzip https://review.openstack.org/218898 | 16:45 |
| openstackgerrit | Kaitlin Farr proposed openstack/requirements: Add Castellan to global requirements https://review.openstack.org/184874 | 16:45 |
| *** ashleighfarnham has quit IRC | 16:46 | |
| *** ashleighfarnham has joined #openstack-infra | 16:46 | |
| *** skylerberg has quit IRC | 16:46 | |
| *** sergmelikyan has quit IRC | 16:48 | |
| *** david-ly_ has joined #openstack-infra | 16:49 | |
| *** david-lyle has quit IRC | 16:49 | |
| *** amuller is now known as amuller_afk | 16:49 | |
| *** sergmelikyan has joined #openstack-infra | 16:50 | |
| *** tonytan4ever has joined #openstack-infra | 16:50 | |
| *** geoffarnold has quit IRC | 16:50 | |
| *** amotoki has quit IRC | 16:52 | |
| *** ajmiller has joined #openstack-infra | 16:52 | |
| akraminsky | clarkb, can we add in a block_migrate_cinder_iscsi compute feature in the tempest config? | 16:54 |
| *** tjones has quit IRC | 16:54 | |
| *** geoffarnold has joined #openstack-infra | 16:54 | |
| *** ajmiller has quit IRC | 16:54 | |
| clarkb | akraminsky: maybe? mtreinish is a better person to ask, I am not sure what tempests rules for feature flags are | 16:55 |
| mtreinish | clarkb: well that flag exists, I think it's a devstack question for setting it to true | 16:56 |
| mtreinish | but that would only be on the multi node job | 16:56 |
| clarkb | pabelanger: nibalizer ok I have more infos, the problem is systemd doesn't appear to be loading the iptables service config into itself (doing a systemctl daemon-reload) and that is why it fails | 16:56 |
| clarkb | pabelanger: I am not sure how to debug that furhter? maybe a package bug? | 16:56 |
| akraminsky | mtreinish, i don't think there is a flag for it yet | 16:56 |
| openstackgerrit | Lucas Alvares Gomes proposed openstack-infra/project-config: Make the ironic-pxe_ssh-dib job to build DIB from source https://review.openstack.org/219351 | 16:57 |
| lucasagomes | sdague, ^ and https://review.openstack.org/#/c/219349/ | 16:57 |
| lucasagomes | sdague, not sure the project-config patch is correct tho. I'm not familiar with the yaml files there | 16:57 |
| mtreinish | akraminsky: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/config.py#n361 | 16:57 |
| clarkb | pabelanger: but maybe also a bug in puppet, perhaps its systemd service provider should do a daemon-reload prior to each service state change? that is very shotgun approach though | 16:58 |
| akraminsky | mtreinish, oh I thought the flags get set in devstack/lib/tempest with the iniset call | 16:58 |
| sdague | lucasagomes: https://review.openstack.org/#/c/219349 looks right | 16:58 |
| lucasagomes | sdague, yeah that I tested locally. But the project config which depends on this one I'm not 100% sure | 16:59 |
| mtreinish | akraminsky: that's how they get set if the configuration differs from the default value of the option. Devstack is the configuration engine more or less | 16:59 |
| mtreinish | akraminsky: but the options are all defined in tempest | 17:00 |
| *** julim has quit IRC | 17:00 | |
| *** markus_z has quit IRC | 17:00 | |
| *** Sukhdev has quit IRC | 17:00 | |
| sdague | lucasagomes: yeh, I think project-config is good as well | 17:01 |
| lucasagomes | o wow | 17:01 |
| lucasagomes | sdague, ok thanks very much! | 17:01 |
| *** skylerberg has joined #openstack-infra | 17:01 | |
| sdague | clarkb: you want to take a look? | 17:01 |
| *** hashar has quit IRC | 17:01 | |
| lucasagomes | I will see if I can fix the pip install -e tomorrow and the ubuntu element for DIB | 17:01 |
| lucasagomes | that should fix it all | 17:02 |
| *** ZZelle_ has joined #openstack-infra | 17:02 | |
| akraminsky | mtreinish, yeh I think that's what I'd like to do is configure it, just so upstream's default isn't true.. | 17:02 |
| *** glauco has quit IRC | 17:02 | |
| *** puranamr has joined #openstack-infra | 17:02 | |
| *** julim has joined #openstack-infra | 17:03 | |
| *** glauco__ has joined #openstack-infra | 17:03 | |
| *** unicell has quit IRC | 17:03 | |
| *** w_ has joined #openstack-infra | 17:03 | |
| *** ajmiller has joined #openstack-infra | 17:03 | |
| *** rossella_s has quit IRC | 17:06 | |
| *** sdake_ has joined #openstack-infra | 17:06 | |
| *** rossella_s has joined #openstack-infra | 17:06 | |
| clarkb | lucasagomes: I think your -z condition is wrong | 17:06 |
| *** sdake has quit IRC | 17:06 | |
| clarkb | lucasagomes: should be -n ya? | 17:06 |
| lucasagomes | clarkb, my bash scripting is quite bad | 17:07 |
| *** sdake has joined #openstack-infra | 17:07 | |
| *** olaph has quit IRC | 17:07 | |
| lucasagomes | lemme do some tests locally | 17:07 |
| * lucasagomes thought -z was for non set env vars | 17:07 | |
| clarkb | I think it should be -n | 17:07 |
| clarkb | lucasagomes: it is | 17:07 |
| *** david-ly_ has quit IRC | 17:07 | |
| *** alivigni has joined #openstack-infra | 17:07 | |
| clarkb | lucasagomes: but the code in that block assumes it is set | 17:07 |
| clarkb | so you need to invert and use -n | 17:07 |
| lucasagomes | clarkb, ohh good catch | 17:07 |
| lucasagomes | clarkb, end of the day sorry... I will update the patch | 17:08 |
| lucasagomes | clarkb, that's correct yes! updating... | 17:09 |
| openstackgerrit | Ryan Carey proposed openstack-infra/jenkins-job-builder: Add support for Artifactory plugin https://review.openstack.org/173932 | 17:09 |
| openstackgerrit | Lucas Alvares Gomes proposed openstack-infra/project-config: Make the ironic-pxe_ssh-dib job to build DIB from source https://review.openstack.org/219351 | 17:10 |
| *** vivekd_ has joined #openstack-infra | 17:10 | |
| *** sdake_ has quit IRC | 17:10 | |
| *** vivekd has quit IRC | 17:11 | |
| *** vivekd_ is now known as vivekd | 17:11 | |
| *** puranamr has quit IRC | 17:12 | |
| *** sergmelikyan has quit IRC | 17:12 | |
| *** shashank_hegde has quit IRC | 17:12 | |
| *** krickwix has quit IRC | 17:13 | |
| *** lucasagomes is now known as lucas-dinner | 17:14 | |
| *** sergmelikyan has joined #openstack-infra | 17:14 | |
| glauco__ | I have a question about gerrit: one gerrit-initial-init shouldn't it be executing the reindex only if the secondary index type was set? | 17:14 |
| glauco__ | Because, reading gerrit's source code, it says it will fail if the index type is SQL | 17:15 |
| *** zeih has joined #openstack-infra | 17:15 | |
| *** w_ is now known as olaph | 17:15 | |
| glauco__ | https://gerrit.googlesource.com/gerrit/+/v2.8.4/gerrit-pgm/src/main/java/com/google/gerrit/pgm/Reindex.java | 17:15 |
| glauco__ | please, take a look on line 95 | 17:15 |
| mmmpork | hey folks i'm seeing more errors in the jenkins logs and i'm wondering if someone can help me to troubleshoot | 17:15 |
| mmmpork | for my particular patch | 17:15 |
| mmmpork | "sha256sum: standard input: no properly formatted SHA256 checksum lines found" | 17:15 |
| mmmpork | https://review.openstack.org/#/c/218965/ | 17:16 |
| *** jerryz has joined #openstack-infra | 17:16 | |
| anteaya | clarkb: fungi would like some comments here, did you want to respin then I will approve? https://review.openstack.org/#/c/217300/2/zuul/layout.yaml | 17:16 |
| mmmpork | it's a minor patch, these have passed before, but since last night there was some issue connecting to the ubuntu cloud | 17:16 |
| mmmpork | i've looked at the logs of all the things erroring out and they all have this same checksum error | 17:17 |
| *** ihrachys has joined #openstack-infra | 17:17 | |
| *** tjones has joined #openstack-infra | 17:17 | |
| greghaynes | mmmpork: looks like ubuntu broke us | 17:18 |
| *** tonytan4ever has quit IRC | 17:18 | |
| *** sergmelikyan has quit IRC | 17:19 | |
| mmmpork | did something change on the ubuntu boxes? like a system update or something? | 17:19 |
| mmmpork | "boxes" whatever they are ;) | 17:19 |
| greghaynes | mmmpork: I think the format of their sha256sums file changed | 17:19 |
| mmmpork | ah ok | 17:19 |
| fungi | anteaya: clarkb: i was mostly asking whether the comments would help, or if we were going to remember when to rip that back out | 17:20 |
| *** zeih has quit IRC | 17:20 | |
| openstackgerrit | Jim Rollenhagen proposed openstack-infra/project-config: Switch ironic full tempest to use IPA https://review.openstack.org/219330 | 17:20 |
| openstackgerrit | Sean Dague proposed openstack-infra/project-config: enable nova v20 compat testing jobs https://review.openstack.org/219370 | 17:20 |
| anteaya | fungi: depends on the definition of we | 17:20 |
| anteaya | you and clarkb probably will know when it should come out but not remember | 17:20 |
| jroll | sdague: 219330 there is a blocker to working on this tempest full thing | 17:20 |
| *** ybathia has joined #openstack-infra | 17:20 | |
| anteaya | AJeager will come the file and have to ask | 17:21 |
| mmmpork | greghaynes: i'm happy to look into it but i'm not quite sure where to start :) if you could point me in the right direction i'm happy to see if i can figure it out | 17:21 |
| *** bharathm has quit IRC | 17:21 | |
| anteaya | fungi: clarkb I can live with or without comments, just wanted clarkb to offer an opinion before I close the door | 17:21 |
| greghaynes | mmmpork: https://review.openstack.org/#/c/219199/ looks like a fix from lucas-dinner | 17:21 |
| anteaya | fungi: AJeager will comb the file | 17:22 |
| anteaya | I can't type | 17:22 |
| sdague | jroll: +2 | 17:22 |
| anteaya | it is like I have my arms full of baby | 17:22 |
| jroll | thanks | 17:22 |
| mmmpork | oh excellent | 17:22 |
| openstackgerrit | Clint Adams proposed openstack-infra/infra-specs: WIP - tracking of priority efforts in yaml https://review.openstack.org/219372 | 17:22 |
| mmmpork | hrm although looks like his build failed, i'll take a look | 17:22 |
| *** b10n1k_ has joined #openstack-infra | 17:23 | |
| greghaynes | mmmpork: yea, that is a second ironic fail that I think he was looking at | 17:23 |
| mmmpork | oh man | 17:23 |
| greghaynes | well, second dib + ironic fail | 17:23 |
| *** tjones has left #openstack-infra | 17:24 | |
| mmmpork | yeah.. looks like devstack couldn't launch so i was seeing if i could figure out anything more :) | 17:24 |
| mmmpork | i "love" how it says "the setup script failed" err ok | 17:24 |
| mmmpork | thanks? | 17:24 |
| mmmpork | haha | 17:24 |
| mmmpork | i still see the checksum fail in the devstack log | 17:25 |
| *** angdraug_ has quit IRC | 17:25 | |
| *** achanda has joined #openstack-infra | 17:25 | |
| mmmpork | oh wait but it's getting a new thingy ahh ok | 17:25 |
| mmmpork | i can read! | 17:25 |
| mmmpork | :D | 17:25 |
| clarkb | fungi anteaya I can propose revert today and wip it | 17:26 |
| fungi | clarkb: i like that solution even better than adding comments | 17:26 |
| mmmpork | hrm.. still seeing that error in the devstack log | 17:26 |
| anteaya | clarkb: hey that works | 17:26 |
| * anteaya +As | 17:27 | |
| *** tonytan4ever has joined #openstack-infra | 17:27 | |
| greghaynes | jroll: left review on https://review.openstack.org/#/c/219330/ | 17:27 |
| pc_m | anteaya: hi | 17:28 |
| clarkb | anteaya I have two babies currently. one rocked by a foot and the other being fed | 17:28 |
| clarkb | 3 things at once | 17:28 |
| anteaya | clarkb: you multitasking father you | 17:28 |
| anteaya | clarkb: and I'm the one who can't type | 17:28 |
| anteaya | pc_m: how are you today? | 17:29 |
| greghaynes | jroll: er, nvm, DIB doesnt do -full | 17:29 |
| anteaya | <-- train wifi for those who care | 17:29 |
| pc_m | anteaya: nice. And you? | 17:29 |
| jroll | greghaynes: heh yeah | 17:29 |
| anteaya | pc_m: good thanks, love the train | 17:29 |
| pc_m | anteaya: train? | 17:29 |
| anteaya | pc_m: to montreal, puppet sprint tomorrow | 17:29 |
| *** sergmelikyan has joined #openstack-infra | 17:30 | |
| *** sergmelikyan has quit IRC | 17:30 | |
| *** sharikapongubala has joined #openstack-infra | 17:30 | |
| pc_m | anteaya: cool! | 17:30 |
| pc_m | anteaya: Had a question for ya... | 17:30 |
| anteaya | pc_m: thought so | 17:30 |
| * anteaya listens | 17:30 | |
| pc_m | anteaya: Is there an IRC channel for neutronclient development questions? | 17:30 |
| *** unicell has joined #openstack-infra | 17:30 | |
| anteaya | not that I'm aware | 17:30 |
| anteaya | just -neutron | 17:30 |
| anteaya | you seem to be low on reasources there | 17:31 |
| sdague | greghaynes: so I think you folks should figure out how to break up the test matrix more, because it seems like dib should have some narrow test of "I built a thing that can boot". | 17:31 |
| pc_m | anteaya: OK. Will see if anyone responds there (I posted already). Thanks. | 17:31 |
| greghaynes | sdague: That review exists, the problem is were hitting lots of issues because of network fails because we only have upstream mirrors to use | 17:32 |
| anteaya | pc_m: sure, approvers for neutronclient are neutron core: http://git.openstack.org/cgit/openstack-infra/project-config/tree/gerrit/acls/openstack/python-neutronclient.config | 17:32 |
| greghaynes | sdague: so our ability to increase test coverage has kind of stalled | 17:33 |
| *** adduarte has joined #openstack-infra | 17:33 | |
| pc_m | anteaya: Seeing odd operation, so looking for someone with experience in that area. | 17:33 |
| *** maishsk has joined #openstack-infra | 17:33 | |
| *** zz_ja has quit IRC | 17:34 | |
| *** maurosr has quit IRC | 17:34 | |
| sdague | greghaynes: why can't you stash the base image in infra somewhere? | 17:34 |
| greghaynes | Its not just base images - package repo's seem to fail more | 17:35 |
| *** sharikapongubala has quit IRC | 17:35 | |
| *** puranamr has joined #openstack-infra | 17:35 | |
| clarkb | also our images are 16GB huge need to stash less not more on them | 17:35 |
| anteaya | pc_m: yup I support that but I have no idea whom that would be | 17:35 |
| greghaynes | oh yea, we definitely wouldnt put them in the test images | 17:36 |
| *** baoli has joined #openstack-infra | 17:36 | |
| zaro | ianw: ping | 17:36 |
| mordred | greghaynes, sdague: yeah - and local package mirrors are a TDL item, but are several things down on the current list | 17:36 |
| sdague | mordred: well, it being several items down the list means it wrecked a bunch of unrelated projects for 24 hours | 17:37 |
| sdague | so I would ask that it gets brought up the list | 17:37 |
| *** abitha has joined #openstack-infra | 17:37 | |
| greghaynes | Was there a bug relating to dib being unable to boot? | 17:37 |
| sdague | no, it was a bug in dib not being able to build images because of changes in ubuntu's repos | 17:38 |
| greghaynes | yea, so that we test and caught | 17:38 |
| sdague | http://status.openstack.org//elastic-recheck/#1490864 | 17:38 |
| sdague | http://status.openstack.org//elastic-recheck/#1490806 | 17:38 |
| anteaya | so I have this error twice in this patch, failed two runs: http://logs.openstack.org/64/212864/3/gate/gate-project-config-layout/af79132/console.html#_2015-09-01_02_22_22_384 | 17:38 |
| anteaya | logstash only has 5 incidents of it in the last 12 hours, so fairly isolated | 17:39 |
| anteaya | what would cause zuul to fail to traverse parents of a commit? | 17:39 |
| sdague | greghaynes: if by test and caught you mean blocked all nova patches from having valid test results for 24 hours, I think we have different definitions of test and caught :P | 17:39 |
| anteaya | the most recent failure hasn't posted yet, one job is still running | 17:39 |
| greghaynes | sdague: Ubuntu does not gate on us, so catching it does not mean preventing the change form going in | 17:40 |
| *** baoli has quit IRC | 17:40 | |
| *** sergmelikyan has joined #openstack-infra | 17:40 | |
| sdague | greghaynes: but not depending on the internet in tests would have | 17:40 |
| *** abregman has joined #openstack-infra | 17:40 | |
| greghaynes | sdague: Until our mirrors updated... | 17:40 |
| mordred | sdague: it's important - there are just a few other things it's blocking on. however, I'll bring it up ta infra meeting today | 17:41 |
| sdague | no, then you gate mirror updates on it passing | 17:41 |
| *** baoli has joined #openstack-infra | 17:41 | |
| greghaynes | or new images went out/etc | 17:41 |
| sdague | basically like the whole constraints updates model lifeless is building for pip | 17:41 |
| greghaynes | oh my | 17:41 |
| *** Sukhdev has joined #openstack-infra | 17:41 | |
| mordred | I do not think we're planning on gated mirror updates | 17:41 |
| mordred | the only thing we were planning on for mirror updates was making sure we always had a working mirror copy before publishing it | 17:41 |
| mordred | not gating the contents on openstack working | 17:42 |
| openstackgerrit | Claudiu Belu proposed openstack/requirements: WIP: Removes posix-ipc from global-requirements https://review.openstack.org/217824 | 17:42 |
| *** stevemar has quit IRC | 17:42 | |
| mordred | and honestly - just getting a mirror at all is probably a good first step in either case | 17:42 |
| *** pvaneck has joined #openstack-infra | 17:42 | |
| sdague | mordred: that's probably fine, however we should test isolation things there | 17:43 |
| sdague | so only dib should have to pass tests that use that mirror | 17:43 |
| mordred | clarkb, docaedo: https://pbs.twimg.com/media/CN1BDvPUkAE8eTY.jpg:large <-- btw, this happened | 17:43 |
| sdague | nothing else should | 17:43 |
| greghaynes | Being broken by upstream distro packages is a completely unrelated issue to anything dib - the tooling to prevent that hasnt even been theorized and that problem exists for all our projects | 17:43 |
| mordred | sdague: it's possible you and I are talking about different things, for which I'm sorry if that's the case | 17:43 |
| sdague | mordred: it's possible | 17:44 |
| mordred | sdague: what I'm talking about is that infra has been discussing general local package mirrors for resilience of access to package repositories - not any sort of further curated thing | 17:44 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: add launchpad-login command https://review.openstack.org/219377 | 17:44 |
| mordred | with the one exception being | 17:44 |
| sdague | mordred: sure, here is the issue | 17:44 |
| freerunner | Hi folks! I would like to ask you for quick review. https://review.openstack.org/#/c/219222/ . Murano gate is broken because of django.. | 17:44 |
| sdague | ubuntu changed a thing | 17:45 |
| mordred | that we want to solve for atomic updates, ecuase the normal package mirroring approaches have those holes where the index is out of sync with the contents | 17:45 |
| sdague | which impacts dib not working | 17:45 |
| mordred | sdague: right. which is then tied into a whole bunch of other things - isn't the solution there to untangle dib from the other things for that? | 17:45 |
| sdague | it 100% blocked nova, ironic, tempest and possibly other projects from getting any valid test results | 17:45 |
| *** pelix has quit IRC | 17:45 | |
| sdague | mordred: yes | 17:45 |
| mordred | (like, I feel like jroll sent a message to the list outlining a solution to this) | 17:45 |
| *** trown|lunch is now known as trown | 17:46 | |
| *** sharikapongubala has joined #openstack-infra | 17:46 | |
| jroll | what'd I do | 17:46 |
| greghaynes | I dont understand how that issue is any different than our other distro package deps. Say libvirt updating and breaking nova | 17:46 |
| *** sharikapongubala has quit IRC | 17:46 | |
| sdague | he did, and that's a big chunk of this, it just seemed like I was hearing there being no belief that there was a real issue | 17:46 |
| *** afazekas__ has joined #openstack-infra | 17:46 | |
| mordred | greghaynes: it's different because it's an interface change in the repo itself | 17:46 |
| *** sergmelikyan has quit IRC | 17:47 | |
| mordred | greghaynes: the sha sums are a thing dib specifically looks at | 17:47 |
| mordred | normal usage of apt repos is not affected by that | 17:47 |
| greghaynes | Yes, so its not a package problem, but the issue is that were a downstream of that repo | 17:47 |
| mordred | for the other thing | 17:47 |
| mordred | if libvirt breaks the world - we _do_ want to know about that | 17:47 |
| mordred | even if it's painful | 17:47 |
| *** sharikapongubala has joined #openstack-infra | 17:47 | |
| mordred | because it means the libvirt will be breaking people for real in the world | 17:47 |
| *** mmedvede has joined #openstack-infra | 17:47 | |
| clarkb | isnt that true of dib here too? | 17:48 |
| mordred | dib would be broken for people in the world | 17:48 |
| clarkb | dib stopped working | 17:48 |
| clarkb | yes | 17:48 |
| mordred | but nova need not be broken by dib being broken | 17:48 |
| jroll | nothing depends on dib at runtime | 17:48 |
| greghaynes | yes, so I agree with the point that nova shouldnt care that dib is broken | 17:48 |
| *** sergmelikyan has joined #openstack-infra | 17:48 | |
| jroll | (afaik) | 17:48 |
| mordred | so yeah - we want dib to notice that ubuntu broke them | 17:48 |
| mordred | so that they can fix it | 17:48 |
| anteaya | do we identify which zuul merger is involved in a given job in the log? | 17:48 |
| greghaynes | totally unrelated to the issue of us needing to solve pinning for distro packaging | 17:48 |
| sdague | right, the point jroll made, dib is not a runtime dep | 17:48 |
| mordred | yah | 17:48 |
| anteaya | zm02 looks odd compared to the others | 17:48 |
| clarkb | anteaya yes its the zuul url parameter | 17:49 |
| *** bharathm has joined #openstack-infra | 17:49 | |
| anteaya | great thanks | 17:49 |
| greghaynes | because I do still want to know that dib is broken | 17:49 |
| *** Sukhdev has quit IRC | 17:49 | |
| jroll | right, I don't think anyone disagrees with that | 17:49 |
| jroll | but dib being broken shouldn't break the world | 17:49 |
| greghaynes | now, does the IPA change actually fix that issue? Isnt that just moving us on to another upstream? | 17:50 |
| jroll | the upstream is now tarballs.o.o | 17:50 |
| jroll | which is published to by a post job on IPA | 17:50 |
| greghaynes | oh, so coreos-i-b posts artifacts to there? | 17:50 |
| *** FallenPegasus has joined #openstack-infra | 17:50 | |
| jroll | so while it could still be a problem, it's far more controllable | 17:51 |
| greghaynes | sweet :) | 17:51 |
| jroll | :) | 17:51 |
| mordred | \o/ | 17:51 |
| *** afazekas__ has quit IRC | 17:51 | |
| mordred | now, in parallel to that - we do want better local mirrors - which would not have solved this, but which would/might solve other things | 17:52 |
| anteaya | clarkb: I see zuul-0.log and zuul-1.log does that mean that zuul merger 1 and 2 were involved? | 17:52 |
| anteaya | I might not be looking in the right spot | 17:52 |
| docaedo | mordred: yes that did, that meetup (moderated by clarkb!) is going to be awesome :) | 17:52 |
| *** vivekd_ has joined #openstack-infra | 17:52 | |
| * mordred is SO SAD he can't be there for clarkb moderating that panel | 17:52 | |
| greghaynes | haha | 17:52 |
| greghaynes | It is going to be fun | 17:52 |
| *** puranamr has quit IRC | 17:52 | |
| *** vivekd has quit IRC | 17:53 | |
| *** vivekd_ is now known as vivekd | 17:53 | |
| *** amuller_afk is now known as amuller | 17:53 | |
| *** david-lyle has joined #openstack-infra | 17:54 | |
| anteaya | clarkb: found it: http://logs.openstack.org/64/212864/3/gate/gate-project-config-layout/af79132/console.html#_2015-09-01_02_22_14_923 | 17:54 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 17:54 | |
| anteaya | clarkb: I was in the wrong spot | 17:54 |
| *** afazekas__ has joined #openstack-infra | 17:54 | |
| sdague | mordred: right, better mirrors will help with other failures | 17:55 |
| clarkb | anteaya ok | 17:55 |
| *** shashank_hegde has joined #openstack-infra | 17:56 | |
| kfox1111 | docaedo: hows the cors patch looking? | 17:56 |
| sdague | also, in theory, the only artifacts posting to tarballs.o.o are ones we know work, right jroll ? | 17:56 |
| odyssey4me | with regards to the mirrors, would it not make sense to make use of cloud-specific mirror configurations instead of the generalised ubuntu archive and such? | 17:56 |
| jroll | sdague: yes, they're published on every merge to master, and the gate jobs verify those | 17:57 |
| *** sputnik13 has quit IRC | 17:57 | |
| sdague | jroll: great | 17:57 |
| jroll | sdague: in theory it's a different build, so MAYBE we could race a mirror update | 17:57 |
| *** sharikapongubala has quit IRC | 17:57 | |
| *** madhu_ak has joined #openstack-infra | 17:57 | |
| odyssey4me | gor ubuntu, as an example, hpcloud has http://az[1-3].clouds.archive.ubuntu.com as mirrors in their standard image | 17:57 |
| jroll | sdague: gate does build+tempest, post does build+upload | 17:57 |
| odyssey4me | *for | 17:57 |
| kfox1111 | I don't quite understand yolanda's ordering issue. Do you? | 17:57 |
| mordred | odyssey4me: well | 17:58 |
| *** sharikapongubala has joined #openstack-infra | 17:58 | |
| mordred | odyssey4me: you'd think | 17:58 |
| *** sputnik13 has joined #openstack-infra | 17:58 | |
| anteaya | clarkb: logstash tells you the build_node, I'm going with we don't have it set up to tell you the zm though | 17:58 |
| mordred | odyssey4me: except that we've found that doing so actually _increases_ our failure rate | 17:58 |
| sdague | jroll: yeh, honestly, that's pretty good validation | 17:58 |
| mordred | odyssey4me: due to operational concerns on those mirrors themselves | 17:58 |
| odyssey4me | mordred ugh | 17:58 |
| * odyssey4me shakes his fist at the sky | 17:59 | |
| mordred | odyssey4me: this is why we're looking at running per-cloud-region mirrors ourselves | 17:59 |
| jroll | sdague: that's by intention :) | 17:59 |
| *** gyee has joined #openstack-infra | 17:59 | |
| mordred | odyssey4me: and doing it in a way that we won't publish them when they're bad/broken | 17:59 |
| sdague | jroll: right, so that bits is good. Now just to use it :) | 17:59 |
| jroll | yep, just waiting on a +2+W | 17:59 |
| *** esmute_ has joined #openstack-infra | 18:00 | |
| *** david-lyle has quit IRC | 18:01 | |
| *** mpopow has joined #openstack-infra | 18:01 | |
| mordred | jroll: patch? | 18:02 |
| *** tiswanso has quit IRC | 18:02 | |
| jroll | mordred: https://review.openstack.org/219285 https://review.openstack.org/219330 | 18:02 |
| jroll | the former being more important | 18:02 |
| *** tjones has joined #openstack-infra | 18:03 | |
| *** tiswanso has joined #openstack-infra | 18:03 | |
| docaedo | kfox1111: CORS patch looks good to me except need to clarify what yolanda was saying about mod_http and needing to call notify http | 18:03 |
| anteaya | all 5 failures in the last 12 hours are from zuul merger 2 | 18:03 |
| anteaya | message:"fatal: Failed to traverse parents of commit" | 18:03 |
| kfox1111 | k. thx. | 18:04 |
| mordred | jroll: +3 | 18:04 |
| *** zeus- has joined #openstack-infra | 18:04 | |
| *** wznoinsk has quit IRC | 18:04 | |
| anteaya | as well as this one: https://jenkins07.openstack.org/job/gate-project-config-layout/1637/console | 18:04 |
| jroll | mordred: <3 | 18:04 |
| anteaya | which I'm sure isn't indexed yet | 18:05 |
| anteaya | so zuul merger 2, are we able to take it out of rotation safely? | 18:05 |
| *** puranamr has joined #openstack-infra | 18:05 | |
| clarkb | anteaya: we should index the zuul url too | 18:06 |
| anteaya | clarkb: we should | 18:06 |
| anteaya | clarkb: can I help to do that? | 18:06 |
| anteaya | clarkb: as in I have no idea how to do that and would like to learn | 18:07 |
| clarkb | anteaya: I think it may just be a simple change to the jenkins zmq client and gearman client | 18:07 |
| anteaya | I will see what I can grep there | 18:07 |
| *** HeOS has quit IRC | 18:07 | |
| anteaya | and toss up a place holder which you can tell me how to fix | 18:07 |
| clarkb | anteaya: add it to https://git.openstack.org/cgit/openstack-infra/puppet-log_processor/tree/files/log-gearman-client.py#n114' | 18:07 |
| anteaya | I will do that | 18:07 |
| * anteaya does that | 18:07 | |
| *** afazekas__ has quit IRC | 18:07 | |
| *** wznoinsk has joined #openstack-infra | 18:07 | |
| *** alex_xu has quit IRC | 18:08 | |
| *** adduarte has quit IRC | 18:09 | |
| *** fitoduarte has joined #openstack-infra | 18:09 | |
| *** alex_xu has joined #openstack-infra | 18:11 | |
| sdague | so how long do we expect the merge of the top of the queue to take after it's passed all the tests? | 18:11 |
| *** Sukhdev has joined #openstack-infra | 18:12 | |
| sdague | because 215658,1 is going on 5 minutes now | 18:13 |
| openstackgerrit | Anita Kuno proposed openstack-infra/puppet-log_processor: Add build_zm parameter https://review.openstack.org/219388 | 18:14 |
| *** Hal has joined #openstack-infra | 18:14 | |
| clarkb | sdague: however long it takes the job result event to get processed | 18:15 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Migrate neutron to the common client interface https://review.openstack.org/214243 | 18:15 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Pass timeout to session, not constructors https://review.openstack.org/219290 | 18:15 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Remove last vestige of glanceclient being different https://review.openstack.org/219291 | 18:15 |
| clarkb | looks like there are a few hundred in the queue | 18:15 |
| *** sharikapongubala has quit IRC | 18:15 | |
| *** sharikapongubala has joined #openstack-infra | 18:15 | |
| anteaya | clarkb: so I created a new variable (I think) ZUUL_MERGER | 18:15 |
| anteaya | clarkb: where do I need to set or get the value for that, to send it to the logs | 18:16 |
| *** maishsk_ has joined #openstack-infra | 18:16 | |
| *** puranamr has quit IRC | 18:17 | |
| clarkb | anteaya: just copy that line I linked and update the names basically | 18:18 |
| clarkb | anteaya: what that does is copy it from the jenkins paylod and convert it into the gearman payload that will get indexed | 18:18 |
| anteaya | clarkb: this ran on zm02 and failed for the same bug: https://review.openstack.org/#/c/217300/ | 18:18 |
| *** puranamr has joined #openstack-infra | 18:19 | |
| *** david-lyle has joined #openstack-infra | 18:19 | |
| *** maishsk has quit IRC | 18:19 | |
| *** maishsk_ is now known as maishsk | 18:19 | |
| *** w_ has joined #openstack-infra | 18:19 | |
| openstackgerrit | Merged openstack/requirements: Updated from generate-constraints https://review.openstack.org/218737 | 18:20 |
| anteaya | clarkb: I copied and updated | 18:20 |
| openstackgerrit | Glauco Oliveira proposed openstack-infra/puppet-gerrit: Trigger reindex only when secondary index is set. https://review.openstack.org/219391 | 18:20 |
| anteaya | what would it take to get zm02 out of rotation? | 18:20 |
| clarkb | anteaya: I think the variable is ZUUL_URL | 18:20 |
| anteaya | clarkb: I will fix | 18:20 |
| clarkb | anteaya: you can confirm by looking at a parameters list on a jenkins job in jenkins | 18:20 |
| clarkb | anteaya: go to a job page, login in via lp and on the left it should have a parameters link | 18:21 |
| *** olaph has quit IRC | 18:22 | |
| openstackgerrit | Merged openstack-infra/project-config: Use Ironic pxe_ipa job in Nova https://review.openstack.org/219285 | 18:22 |
| *** ajmiller has quit IRC | 18:23 | |
| *** puranamr has quit IRC | 18:23 | |
| *** thorst has quit IRC | 18:23 | |
| *** thorst has joined #openstack-infra | 18:23 | |
| sdague | clarkb: ok, I guess I was surprised that it took about 15 minutes to merge the 17 passing jobs at top of gate, and that the sliding window didn't adjust until they all were merged | 18:23 |
| openstackgerrit | Anita Kuno proposed openstack-infra/puppet-log_processor: Add build_zuul_url parameter https://review.openstack.org/219388 | 18:24 |
| anteaya | clarkb: ah okay, thanks | 18:24 |
| clarkb | I have delted te old git08 | 18:24 |
| clarkb | going to make a new git07 now | 18:25 |
| openstackgerrit | Dan Prince proposed openstack-infra/tripleo-ci: Switch overcloud-f21-nonha to instack https://review.openstack.org/185151 | 18:27 |
| *** abregman is now known as abregman|afk | 18:28 | |
| *** e0ne has joined #openstack-infra | 18:29 | |
| *** vikram has quit IRC | 18:30 | |
| *** tjones has quit IRC | 18:30 | |
| *** jtomasek has joined #openstack-infra | 18:32 | |
| *** aysyd has quit IRC | 18:32 | |
| *** aysyd has joined #openstack-infra | 18:33 | |
| *** w_ is now known as olaph | 18:34 | |
| *** esker has joined #openstack-infra | 18:35 | |
| *** esker has quit IRC | 18:35 | |
| *** MarkAtwood has quit IRC | 18:35 | |
| *** baoli has quit IRC | 18:35 | |
| anteaya | clarkb: it is ZUUL_URL and nets us: http://zm06.openstack.org/p | 18:35 |
| clarkb | anteaya: thats it | 18:35 |
| pabelanger | nice, finally got nodepool running under centos7 | 18:36 |
| anteaya | clarkb: wonderful, thank you | 18:36 |
| glauco__ | nibalizer: thanks for you review. I didn't understood what you meant by having an puppet if. | 18:36 |
| clarkb | pabelanger: did you see my notes about why iptables is broken on hpcloud centos7? | 18:36 |
| yolanda | hi docaedo, kfox1111, so my review is about the usage of httpd_mod , currently is buggy | 18:36 |
| pabelanger | clarkb: no, missed them sorry. Look in scroll back now | 18:37 |
| nibalizer | glauco__: if $secondary_index { exec... } | 18:37 |
| yolanda | the way your change was done, won't work, because it will not reboot httpd service after modules are installed | 18:37 |
| pabelanger | clarkb: I know ianw was looking into is also | 18:37 |
| *** baoli has joined #openstack-infra | 18:37 | |
| yolanda | and i was pointing you to some samples to fix it | 18:37 |
| glauco__ | nibalizer: would we be able to notify that exec when $secondary_index is false? | 18:37 |
| yolanda | we need to do a proper fix on httpd_mod, but it's still a TODO | 18:37 |
| anteaya | nibalizer: thank you | 18:38 |
| pabelanger | clarkb: okay, caught up. That sucks | 18:38 |
| glauco__ | nibalizer: that exec will be notified every time gerrit gets up and down, no matter if it has a secondary type or not. Don't it? | 18:38 |
| *** e0ne has quit IRC | 18:39 | |
| nibalizer | glauco__: the exec can subscribe to the other execs | 18:39 |
| *** puranamr has joined #openstack-infra | 18:40 | |
| pabelanger | clarkb: Ya, I'm not sure how to proceed either. But, does sound like a packaging bug | 18:40 |
| *** dims has quit IRC | 18:40 | |
| glauco__ | nibalizer: at first we tried using puppet if, but we noticed that the exec will not be defined when the flag is false, and subscribe will not find the resource | 18:40 |
| clarkb | ok new git07 is built, I am gonna take old git07 out of the haproxy rotation now, update dns, force a sync from gerrit then push a change to update the haproxy config | 18:40 |
| *** dims has joined #openstack-infra | 18:40 | |
| glauco__ | nibalizer: do you mean to inverse the dependency order and have the exec subscribe instead of being notified? | 18:41 |
| openstackgerrit | Dan Prince proposed openstack-infra/tripleo-ci: Clean any yum cached data https://review.openstack.org/219227 | 18:41 |
| anteaya | clarkb: ack | 18:41 |
| *** puranamr has quit IRC | 18:41 | |
| bdemers | to request a new release via the _new_ process, all i need to do is create a yaml file for a given project name, similar to what was done here: https://review.openstack.org/#/c/217766/ ? | 18:42 |
| nibalizer | glauco__: subscribe and notify are inverses | 18:42 |
| nibalizer | so subscribe in one direction is equivalent to notify in the other direction | 18:42 |
| bdemers | and are the releases cut automaticly once it gets merged ? | 18:43 |
| nibalizer | so you should be able to express the dependency you want without using onlyif= > | 18:43 |
| anteaya | bdemers: release questions are best answered by the release managers | 18:43 |
| anteaya | bdemers: who tend to be available in #openstack-relmgr-office | 18:44 |
| pabelanger | Looks like ubuntu trusty dibs are broken. Looks like a patch however: https://review.openstack.org/#/c/219199/ | 18:44 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Clean any yum cached data https://review.openstack.org/219227 | 18:44 |
| anteaya | that channel is often used for meetings though so when you join wait a bit in case a meeting is in progress | 18:44 |
| glauco__ | nibalizer: got it. I will fix that. | 18:45 |
| *** ivar-lazzaro has quit IRC | 18:45 | |
| bdemers | anteaya: i | 18:45 |
| *** dims has quit IRC | 18:45 | |
| glauco__ | nibalizer: thank you | 18:45 |
| pabelanger | http://paste.openstack.org/show/438357/ for ubuntu trusty error | 18:45 |
| bdemers | anteaya: I ask more about the process / infra around the releas vs. an actual release itself | 18:46 |
| *** puranamr has joined #openstack-infra | 18:46 | |
| *** sharikapongubala has quit IRC | 18:47 | |
| anteaya | the release managers know the current status best | 18:47 |
| anteaya | and my have questions for you about your individual project | 18:47 |
| anteaya | s/my/may | 18:47 |
| *** sharikapongubala has joined #openstack-infra | 18:48 | |
| *** puranamr has quit IRC | 18:48 | |
| clarkb | ok haproxy and dns are done, have to wait for dns to resolve properly on review.o.o then I will trigger replication | 18:48 |
| *** puranamr has joined #openstack-infra | 18:48 | |
| anteaya | wonderful | 18:48 |
| *** sharikapongubala has quit IRC | 18:49 | |
| *** sharikapongubala has joined #openstack-infra | 18:49 | |
| greghaynes | SpamapS: ok, just deployed one node for crinkle using the baked in bridge/vlan network config + dhcp and it seems to work fine | 18:50 |
| *** sridhar_ram has quit IRC | 18:50 | |
| openstackgerrit | Glauco Oliveira proposed openstack-infra/puppet-gerrit: Trigger reindex only when secondary index is set. https://review.openstack.org/219391 | 18:50 |
| greghaynes | SpamapS: deploying second node now... | 18:50 |
| *** sridhar_ram has joined #openstack-infra | 18:50 | |
| glauco__ | nibalizer: ^^^ | 18:50 |
| *** sharikap_ has joined #openstack-infra | 18:50 | |
| openstackgerrit | Glauco Oliveira proposed openstack-infra/puppet-gerrit: Setup preconditions for acceptance tests. https://review.openstack.org/219400 | 18:52 |
| *** FallenPe_ has joined #openstack-infra | 18:53 | |
| *** sergmelikyan has quit IRC | 18:53 | |
| *** eharney has quit IRC | 18:53 | |
| *** FallenPegasus has quit IRC | 18:53 | |
| mordred | greghaynes, SpamapS: I wanna cut a new glean - the changes look like bugfixes to me - but I could be convinced that the lock file is a new feature and should make it 1.5.0 - thoughts? | 18:53 |
| greghaynes | 1.5 due to the lock | 18:54 |
| *** sharikapongubala has quit IRC | 18:54 | |
| *** sergmelikyan has joined #openstack-infra | 18:54 | |
| openstackgerrit | Merged openstack-infra/infra-specs: Host Stackalytics Service https://review.openstack.org/187715 | 18:54 |
| greghaynes | doesnt hurt to err on that side | 18:54 |
| clarkb | looking at review.o.o error logs we appear to have broken the welcome message script | 18:54 |
| *** sharikapongubala has joined #openstack-infra | 18:54 | |
| greghaynes | I think | 18:54 |
| *** sridhar_ram has quit IRC | 18:54 | |
| greghaynes | well, actually, I really dont care either way, the change isnt that behaviroal | 18:54 |
| greghaynes | er, behavioral | 18:54 |
| clarkb | I think stevenks fix logging change may have done that? we are passing --verbose but it should be -v | 18:54 |
| greghaynes | mordred: flip a coin ;) | 18:54 |
| *** e0ne has joined #openstack-infra | 18:55 | |
| *** sergmelikyan has quit IRC | 18:55 | |
| mordred | jeblair: ^^ thoughts? | 18:55 |
| openstackgerrit | Glauco Oliveira proposed openstack-infra/puppet-zuul: Add acceptance tests for puppet-zuul https://review.openstack.org/217732 | 18:55 |
| mordred | jeblair: new glean release introduces a lockfile to serialize creation of common files - is that a 1.4.2 or a 1.5.0 to you? | 18:55 |
| *** zeus- is now known as zeus | 18:56 | |
| *** zeus has quit IRC | 18:56 | |
| *** zeus has joined #openstack-infra | 18:56 | |
| jeblair | mordred: 1.4.2 | 18:57 |
| *** salv-orlando has quit IRC | 18:57 | |
| mordred | jeblair: thanks | 18:57 |
| jeblair | mordred: np. i also supply losing lottery numbers. | 18:58 |
| mordred | rcarrillocruz, jeblair, greghaynes, SpamapS: new glean cut | 18:58 |
| *** sharikap_ has quit IRC | 18:58 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/jeepyb: Add --verbose flag as valid option for logging https://review.openstack.org/219403 | 18:59 |
| clarkb | there is the verbose flag fix | 18:59 |
| *** HeOS has joined #openstack-infra | 18:59 | |
| *** jhesketh has quit IRC | 18:59 | |
| *** john-davidge has quit IRC | 18:59 | |
| *** annegent_ has joined #openstack-infra | 19:00 | |
| jeblair | meeting time! | 19:00 |
| *** angela-s has joined #openstack-infra | 19:00 | |
| *** changbl has quit IRC | 19:02 | |
| anteaya | clarkb: I filed a story for it: https://storyboard.openstack.org/#!/story/2000341 | 19:03 |
| *** nicodemos has quit IRC | 19:03 | |
| *** marcusvrn_ has joined #openstack-infra | 19:03 | |
| *** annegentle has quit IRC | 19:03 | |
| *** jhesketh has joined #openstack-infra | 19:03 | |
| clarkb | anteaya: thanks I think it may be related to the performance trouble we saw yesterday | 19:03 |
| clarkb | possible noisy neighbor was one theory I heard | 19:03 |
| *** Piet has quit IRC | 19:04 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 19:04 | |
| *** afazekas has quit IRC | 19:04 | |
| anteaya | clarkb: fair enough, I need it fixed though as it keeps my approved patheces from merging | 19:05 |
| jhesketh | Morning | 19:05 |
| *** salv-orlando has joined #openstack-infra | 19:06 | |
| SpamapS | mordred: ACK | 19:06 |
| *** esker has joined #openstack-infra | 19:07 | |
| *** stevemar has joined #openstack-infra | 19:07 | |
| SpamapS | greghaynes: wanna have another call to get the ship moving again on the terrible cloud? I feel like we lost the thread a bit. | 19:08 |
| *** MarkAtwood has joined #openstack-infra | 19:08 | |
| *** annegent_ has quit IRC | 19:08 | |
| *** esmute_ has quit IRC | 19:08 | |
| *** dims has joined #openstack-infra | 19:09 | |
| *** watanabe_isao has joined #openstack-infra | 19:09 | |
| *** hashar has joined #openstack-infra | 19:09 | |
| greghaynes | crinkle: ok, you now have .132 available | 19:09 |
| crinkle | greghaynes: \o/ | 19:09 |
| greghaynes | SpamapS: Next big thing for me is to get https://review.openstack.org/#/c/219026/ and the two dhcp servers codified | 19:10 |
| sdague | clarkb / mordred - how you folks feeling about - https://review.openstack.org/#/c/218966 - (though I need to recheck it because of the migrate bug yesterday) - as I'm watching a bunch of the large-ops jobs fail under timeout preasure again, and that would get us tracking that | 19:10 |
| *** puranamr has quit IRC | 19:10 | |
| rcarrillocruz | mordred: \o/ | 19:10 |
| greghaynes | SpamapS: basically I have a lot of one-off work on the bastion I need to formulate into reviews, the one off work is what is letting us do the separate conductor and deploy networks | 19:10 |
| clarkb | sdague: seems fine | 19:10 |
| openstackgerrit | Clint Adams proposed openstack-infra/infra-specs: WIP - tracking of priority efforts in yaml https://review.openstack.org/219372 | 19:10 |
| clarkb | sdague: I really do want to trim the list of PROJECTS too | 19:10 |
| sdague | clarkb: sure | 19:11 |
| clarkb | will have to take that on once transifex to zanata migration is complete | 19:11 |
| sdague | honestly, it would be really nice if we could generate the project list per job | 19:11 |
| *** annegentle has joined #openstack-infra | 19:11 | |
| *** krtaylor has quit IRC | 19:11 | |
| clarkb | sdague: yes thats the idea | 19:11 |
| greghaynes | SpamapS: then from there I think were going to cycle on a node to get the bifrost vlan config-drive stuff working, then the upstart script to move interface to a bridge | 19:12 |
| sdague | clarkb: neat | 19:12 |
| clarkb | sdague: with a base set in d-g for simplicity | 19:12 |
| sdague | clarkb: sure | 19:12 |
| clarkb | sdague: I think base set in d-g will reflect default devstack list | 19:12 |
| clarkb | then everything else will be job specific | 19:12 |
| SpamapS | greghaynes: k. | 19:12 |
| *** sdake_ has joined #openstack-infra | 19:12 | |
| lifeless | clarkb: cool thanks | 19:13 |
| *** sdake has quit IRC | 19:13 | |
| *** dims has quit IRC | 19:13 | |
| *** watanabe_isao has quit IRC | 19:13 | |
| SpamapS | greghaynes: right, thats what I was going to do. Just wanted to make sure we're all on the same page. | 19:13 |
| *** sdake has joined #openstack-infra | 19:13 | |
| *** julim has quit IRC | 19:14 | |
| greghaynes | SpamapS: Sweet, I'll try to be careful and not break the bastion host while I move the chagnes there upstream | 19:14 |
| greghaynes | SpamapS: if you want to poke at that | 19:14 |
| greghaynes | (I want to evacuate the changes from my head ASAP before I forget them) | 19:15 |
| SpamapS | greghaynes: best way to do that is just to deploy a new baremetal (lets call them baremetals when we deploy them with automation. ;) | 19:15 |
| greghaynes | oh, works for me | 19:15 |
| *** jaosorior has quit IRC | 19:15 | |
| SpamapS | greghaynes: it becomes bastion when we screw with it for 2 weeks though. ;) | 19:15 |
| greghaynes | heh | 19:15 |
| clarkb | sdague: maybe a function wrapper for other functions to do timing is something we should add in if we are going to time lots of things? | 19:15 |
| greghaynes | SpamapS: new host or take the plunge and redeploy the original bastion? | 19:16 |
| *** zeih has joined #openstack-infra | 19:16 | |
| sdague | clarkb: we could do that down the road if we do more of this | 19:16 |
| sdague | 3 seemed still in the copy/paste realm | 19:16 |
| *** sdake_ has quit IRC | 19:18 | |
| sdague | mordred / jeblair - https://review.openstack.org/#/c/202494 commented back about the ansible config, assuming that working directory is stable during d-g run is not a valid assumption I don't think | 19:18 |
| *** dims has joined #openstack-infra | 19:18 | |
| greghaynes | clarkb: you were messing with git stuffs earlier right? https://review.openstack.org/#/c/219330/ exploded on a clone, curious if I should try and recheck now | 19:19 |
| *** esmute_ has joined #openstack-infra | 19:19 | |
| *** cdent has quit IRC | 19:20 | |
| *** sabeen1 has joined #openstack-infra | 19:20 | |
| clarkb | greghaynes: I did but that should be unrelated unfortunately there is almost no logging there | 19:20 |
| clarkb | greghaynes: I disabled an haproxy backend which should keep existing connections open iirc | 19:21 |
| *** krtaylor has joined #openstack-infra | 19:21 | |
| greghaynes | clarkb: hrm, didnt you also reboot that backend? | 19:21 |
| *** zeih has quit IRC | 19:21 | |
| clarkb | greghaynes: no | 19:21 |
| greghaynes | wierd | 19:21 |
| greghaynes | It does look like a socket that got axed | 19:21 |
| *** sharikapongubala has quit IRC | 19:22 | |
| clarkb | greghaynes: I can check nodepool logs for IP addrs and do a lookup in the haproxy log after the meteing | 19:22 |
| greghaynes | ok, ty | 19:22 |
| *** shashank_hegde has quit IRC | 19:23 | |
| *** esmute_ has quit IRC | 19:24 | |
| *** dims has quit IRC | 19:25 | |
| *** sharikapongubala has joined #openstack-infra | 19:25 | |
| openstackgerrit | Dan Prince proposed openstack-infra/tripleo-ci: Switch overcloud-f21-nonha to instack https://review.openstack.org/185151 | 19:25 |
| *** armax has quit IRC | 19:26 | |
| *** geoffarnold has quit IRC | 19:27 | |
| *** pvaneck has quit IRC | 19:29 | |
| mriedem | i want to applaud the people that wrote this doc http://docs.openstack.org/infra/zuul/gating.html | 19:30 |
| *** yamamoto has quit IRC | 19:30 | |
| *** sharikapongubala has quit IRC | 19:31 | |
| *** pcrews has quit IRC | 19:31 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: publish docs built from releases repository https://review.openstack.org/219420 | 19:31 |
| *** sharikapongubala has joined #openstack-infra | 19:31 | |
| *** pcrews has joined #openstack-infra | 19:31 | |
| dhellmann | anteaya: I would appreciate a thorough review of ^^, since I'm cargo-culting those changes | 19:32 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Make README.rst a bit more generic https://review.openstack.org/218043 | 19:32 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Add DIB_KEEP_PACKAGE_CACHE variable https://review.openstack.org/208321 | 19:32 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Add DIB_USE_PACKAGE_CACHE variable https://review.openstack.org/211434 | 19:32 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Add #!/bin/bash to library functions https://review.openstack.org/218100 | 19:32 |
| *** ivar-lazzaro has joined #openstack-infra | 19:32 | |
| *** pvaneck has joined #openstack-infra | 19:33 | |
| anteaya | dhellmann: you will get the best I can offer | 19:33 |
| *** krtaylor has quit IRC | 19:34 | |
| *** HeOS has quit IRC | 19:34 | |
| *** aysyd has quit IRC | 19:34 | |
| *** thorst has quit IRC | 19:34 | |
| *** bharathm has quit IRC | 19:34 | |
| *** claudiub has quit IRC | 19:34 | |
| *** ZZelle has quit IRC | 19:34 | |
| *** mrunge has quit IRC | 19:34 | |
| *** lifeless has quit IRC | 19:34 | |
| *** matbu has quit IRC | 19:34 | |
| *** mtreinish has quit IRC | 19:34 | |
| *** Mmike has quit IRC | 19:34 | |
| *** mrodden has quit IRC | 19:34 | |
| *** raginbajin has quit IRC | 19:34 | |
| *** harlowja has quit IRC | 19:34 | |
| *** goodygum has quit IRC | 19:34 | |
| *** nithyag__ has quit IRC | 19:34 | |
| *** sobersabre has quit IRC | 19:34 | |
| *** dtantsur|afk has quit IRC | 19:34 | |
| *** kfox1111 has quit IRC | 19:34 | |
| *** pabelanger has quit IRC | 19:34 | |
| *** Sam-I-Am has quit IRC | 19:34 | |
| *** cschwede has quit IRC | 19:34 | |
| *** wenlock has quit IRC | 19:34 | |
| *** cloudnull has quit IRC | 19:34 | |
| *** sc68cal has quit IRC | 19:34 | |
| *** Gareth has quit IRC | 19:34 | |
| *** sdague has quit IRC | 19:34 | |
| *** niska has quit IRC | 19:34 | |
| *** jhesketh has quit IRC | 19:34 | |
| *** Piet has joined #openstack-infra | 19:34 | |
| *** baoli has quit IRC | 19:34 | |
| *** mriedem is now known as mattykins | 19:34 | |
| *** wenlock has joined #openstack-infra | 19:34 | |
| *** cloudnull has joined #openstack-infra | 19:34 | |
| *** sc68cal has joined #openstack-infra | 19:34 | |
| *** Gareth has joined #openstack-infra | 19:34 | |
| *** sdague has joined #openstack-infra | 19:34 | |
| *** krtaylor has joined #openstack-infra | 19:35 | |
| *** HeOS has joined #openstack-infra | 19:35 | |
| *** aysyd has joined #openstack-infra | 19:35 | |
| *** thorst has joined #openstack-infra | 19:35 | |
| *** bharathm has joined #openstack-infra | 19:35 | |
| *** claudiub has joined #openstack-infra | 19:35 | |
| *** ZZelle has joined #openstack-infra | 19:35 | |
| *** mrunge has joined #openstack-infra | 19:35 | |
| *** lifeless has joined #openstack-infra | 19:35 | |
| *** matbu has joined #openstack-infra | 19:35 | |
| *** mtreinish has joined #openstack-infra | 19:35 | |
| *** kfox1111 has joined #openstack-infra | 19:35 | |
| *** Mmike has joined #openstack-infra | 19:35 | |
| *** mrodden has joined #openstack-infra | 19:35 | |
| *** raginbajin has joined #openstack-infra | 19:35 | |
| *** goodygum has joined #openstack-infra | 19:35 | |
| *** nithyag__ has joined #openstack-infra | 19:35 | |
| *** sobersabre has joined #openstack-infra | 19:35 | |
| *** dtantsur|afk has joined #openstack-infra | 19:35 | |
| *** pabelanger has joined #openstack-infra | 19:35 | |
| *** Sam-I-Am has joined #openstack-infra | 19:35 | |
| *** cschwede has joined #openstack-infra | 19:35 | |
| *** harlowja has joined #openstack-infra | 19:35 | |
| *** harlowja has quit IRC | 19:35 | |
| *** harlowja has joined #openstack-infra | 19:35 | |
| *** jhesketh has joined #openstack-infra | 19:35 | |
| *** dannywilson has joined #openstack-infra | 19:35 | |
| *** w_ has joined #openstack-infra | 19:35 | |
| *** baoli has joined #openstack-infra | 19:35 | |
| *** dims has joined #openstack-infra | 19:35 | |
| *** eharney has joined #openstack-infra | 19:35 | |
| clarkb | greghaynes: I see 3 connections that had a CD state which I think is closed disconnected | 19:35 |
| greghaynes | clarkb: hrm | 19:35 |
| clarkb | now I have to go look up docs to hve it tell me which side did the closing | 19:36 |
| *** tiswanso has quit IRC | 19:36 | |
| *** niska has joined #openstack-infra | 19:36 | |
| clarkb | greghaynes: but only one of them was to the backend I was touching and the timeframes don't line up for when I disabled that particular backend | 19:36 |
| clarkb | greghaynes: if I had to guess right now, I would pin this on hpcloud networking not the git farm | 19:36 |
| greghaynes | clarkb: ok. so might just be network blip | 19:36 |
| greghaynes | fun | 19:36 |
| greghaynes | recheck it is | 19:36 |
| *** abitha has quit IRC | 19:37 | |
| *** tiswanso has joined #openstack-infra | 19:38 | |
| *** unicell has quit IRC | 19:38 | |
| *** olaph has quit IRC | 19:38 | |
| openstackgerrit | Clint Adams proposed openstack-infra/infra-specs: WIP - tracking of priority efforts in yaml https://review.openstack.org/219372 | 19:39 |
| *** esmute_ has joined #openstack-infra | 19:39 | |
| *** w_ has quit IRC | 19:40 | |
| *** erlon has joined #openstack-infra | 19:44 | |
| *** sdake_ has joined #openstack-infra | 19:45 | |
| greghaynes | lucas-dinner: jroll So, trying to get up to speed on what the state of the DIB ironic pxe_ssh job is. | 19:46 |
| clarkb | git07 looks good from my local clone testing, will have a change up to put it in the ortation as soon as git remote update works on that repo | 19:46 |
| greghaynes | lucas-dinner: jroll Is there any path forward for fixing it that doesnt involve fixing dib pip install -e as well as devstack? that seems a bit overly complex to get done right now | 19:46 |
| fungi | mattykins: looks like hashar and jeblair are primarily to blame for that excellent document | 19:47 |
| *** mattykins is now known as mriedem | 19:47 | |
| jroll | greghaynes: I'm not really up to speed, lucas-dinner left me these to look at https://review.openstack.org/#/c/219351/ https://review.openstack.org/#/c/219349/ | 19:47 |
| mriedem | nice | 19:47 |
| jroll | greghaynes: not sure if that's all | 19:47 |
| anteaya | mattykins: I just wanted to tab complete this name for you | 19:47 |
| greghaynes | ah, that fixes from source | 19:47 |
| *** harlowja has quit IRC | 19:47 | |
| *** krtaylor has quit IRC | 19:47 | |
| *** HeOS has quit IRC | 19:47 | |
| *** aysyd has quit IRC | 19:47 | |
| *** thorst has quit IRC | 19:47 | |
| *** bharathm has quit IRC | 19:47 | |
| *** claudiub has quit IRC | 19:47 | |
| *** ZZelle has quit IRC | 19:47 | |
| *** mrunge has quit IRC | 19:47 | |
| *** lifeless has quit IRC | 19:47 | |
| *** matbu has quit IRC | 19:47 | |
| *** mtreinish has quit IRC | 19:47 | |
| *** Mmike has quit IRC | 19:47 | |
| *** mrodden has quit IRC | 19:47 | |
| *** raginbajin has quit IRC | 19:47 | |
| *** goodygum has quit IRC | 19:47 | |
| *** nithyag__ has quit IRC | 19:47 | |
| *** sobersabre has quit IRC | 19:47 | |
| *** dtantsur|afk has quit IRC | 19:47 | |
| *** kfox1111 has quit IRC | 19:47 | |
| *** pabelanger has quit IRC | 19:47 | |
| *** Sam-I-Am has quit IRC | 19:47 | |
| *** cschwede has quit IRC | 19:47 | |
| greghaynes | so then were still broke by pip install -e... | 19:48 |
| jroll | greghaynes: stepping away, I'll bbiab, ping me if I can help other than reviewing those | 19:48 |
| anteaya | mriedem: while I could | 19:48 |
| greghaynes | jroll: ok, ty | 19:48 |
| mriedem | anteaya: and it's not even friday | 19:48 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config: Swap out git07 to new centos7 node https://review.openstack.org/219427 | 19:48 |
| clarkb | and there it is, ready whenever you are | 19:48 |
| fungi | mriedem: you could declare it pre-friday | 19:49 |
| *** esmute_ has quit IRC | 19:49 | |
| *** sdake has quit IRC | 19:49 | |
| anteaya | mriedem: I knkow, stepping out | 19:49 |
| *** sdake has joined #openstack-infra | 19:49 | |
| fungi | all non-fridays are hereby designated pre-fridays | 19:49 |
| *** sputnik13 has quit IRC | 19:49 | |
| fungi | let the early fridaying commence | 19:49 |
| *** ashleighfarnham has quit IRC | 19:49 | |
| anteaya | oh I feel so much better now | 19:49 |
| *** ayoung has quit IRC | 19:50 | |
| nibalizer | clarkb: i am poking the archive module with a stick, hopefully will have something for you soon | 19:50 |
| clarkb | nibalizer: cool and thanks | 19:50 |
| *** davideagnello has quit IRC | 19:50 | |
| fungi | mordred: can i safely reenable puppet agent on nodepool.o.o now (or did you do it yourself after the shade release)? | 19:51 |
| *** sharikapongubala has quit IRC | 19:51 | |
| *** ashleighfarnham has joined #openstack-infra | 19:52 | |
| *** harlowja has joined #openstack-infra | 19:52 | |
| *** krtaylor has joined #openstack-infra | 19:52 | |
| *** HeOS has joined #openstack-infra | 19:52 | |
| *** aysyd has joined #openstack-infra | 19:52 | |
| *** thorst has joined #openstack-infra | 19:52 | |
| *** bharathm has joined #openstack-infra | 19:52 | |
| *** claudiub has joined #openstack-infra | 19:52 | |
| *** ZZelle has joined #openstack-infra | 19:52 | |
| *** mrunge has joined #openstack-infra | 19:52 | |
| *** lifeless has joined #openstack-infra | 19:52 | |
| *** matbu has joined #openstack-infra | 19:52 | |
| *** mtreinish has joined #openstack-infra | 19:52 | |
| *** kfox1111 has joined #openstack-infra | 19:52 | |
| *** Mmike has joined #openstack-infra | 19:52 | |
| *** mrodden has joined #openstack-infra | 19:52 | |
| *** raginbajin has joined #openstack-infra | 19:52 | |
| *** goodygum has joined #openstack-infra | 19:52 | |
| *** nithyag__ has joined #openstack-infra | 19:52 | |
| *** sobersabre has joined #openstack-infra | 19:52 | |
| *** dtantsur|afk has joined #openstack-infra | 19:52 | |
| *** pabelanger has joined #openstack-infra | 19:52 | |
| *** Sam-I-Am has joined #openstack-infra | 19:52 | |
| *** cschwede has joined #openstack-infra | 19:52 | |
| *** geoffarnold has joined #openstack-infra | 19:52 | |
| *** bgmccollum has joined #openstack-infra | 19:52 | |
| *** sharikapongubala has joined #openstack-infra | 19:52 | |
| mordred | fungi: I have not done it - but it should be safe | 19:52 |
| fungi | mordred: thanks, done | 19:53 |
| *** sdake_ has quit IRC | 19:53 | |
| *** MarkAtwood has quit IRC | 19:53 | |
| *** 64MADQRLY has joined #openstack-infra | 19:53 | |
| *** davideagnello has joined #openstack-infra | 19:53 | |
| *** olaph has joined #openstack-infra | 19:53 | |
| *** 64MADQRLY has quit IRC | 19:53 | |
| *** sputnik13 has joined #openstack-infra | 19:53 | |
| *** mrmartin has quit IRC | 19:53 | |
| *** e0ne has quit IRC | 19:54 | |
| *** zz_ja has joined #openstack-infra | 19:55 | |
| *** e0ne has joined #openstack-infra | 19:55 | |
| *** ociuhandu has quit IRC | 19:55 | |
| *** maurosr has joined #openstack-infra | 19:55 | |
| hashar | fungi: o/ | 19:56 |
| hashar | mriedem: ah thank you very much! | 19:56 |
| *** ayoung has joined #openstack-infra | 19:56 | |
| hashar | mriedem: took me a while to figure out while asking question in this channel and thought it would save others a bunch of time :} There is more that needs to be documented though :/ | 19:57 |
| *** glauco__ has quit IRC | 19:58 | |
| *** julim has joined #openstack-infra | 19:58 | |
| *** glauco has joined #openstack-infra | 19:58 | |
| *** ayoung has quit IRC | 19:58 | |
| clarkb | fungi: jenkins04 still looking good | 19:58 |
| *** jtomasek has quit IRC | 19:59 | |
| fungi | two days in a row with crazy load/io spikes on zm02, huh? | 19:59 |
| fungi | bad enough we stopped being able to poll snmp on it for 20-30 minutes at a stretch | 20:00 |
| *** ayoung has joined #openstack-infra | 20:00 | |
| anteaya | fungi: https://storyboard.openstack.org/#!/story/2000341 | 20:00 |
| *** boris-42 has quit IRC | 20:00 | |
| anteaya | can we pull it out of rotation? | 20:00 |
| *** MarkAtwood has joined #openstack-infra | 20:00 | |
| anteaya | mtreinish wants this patch in and it keeps hitting zm02: https://review.openstack.org/#/c/212864/ | 20:01 |
| *** krtaylor has quit IRC | 20:01 | |
| fungi | jeblair: ^ is it safe to just stop the zuul-merger service on that worker? i'll open a trouble ticket with rackspace about poor i/o performance | 20:01 |
| *** mrunge has quit IRC | 20:01 | |
| mtreinish | anteaya: I want many patches in :) | 20:02 |
| anteaya | mtreinish: :) | 20:02 |
| fungi | i want all patches in, except the ones i don't | 20:02 |
| *** krtaylor has joined #openstack-infra | 20:02 | |
| jeblair | fungi: yes | 20:02 |
| anteaya | yay \o/ | 20:02 |
| anteaya | thank you | 20:02 |
| lucas-dinner | greghaynes, hi there, yes the pip install -e is being worked around on the devstack patch | 20:03 |
| lucas-dinner | we can fix it later | 20:03 |
| clarkb | ya should be safe just dont stop apache | 20:03 |
| mtreinish | mordred: is the ansible run taking a ~40 min a known thing: http://logs.openstack.org/79/219379/1/check/gate-tempest-dsvm-large-ops/6f83839/console.html#_2015-09-01_18_07_50_884 | 20:03 |
| *** sergmelikyan has joined #openstack-infra | 20:03 | |
| hashar | fungi: maybe the merger git repos have too many references and needs some cleanup ? | 20:03 |
| hashar | fungi: there is no garbage collection of .git refs and with time that really slow down fetches operations | 20:04 |
| *** glauco__ has joined #openstack-infra | 20:04 | |
| *** sridhar_ram has joined #openstack-infra | 20:04 | |
| fungi | hashar: we checked yesterday and it didn't look like that was the issue (and we have 7 other merger servers not exhibiting this behavior) | 20:04 |
| hashar | fungi: just striking a potential slowness option :} | 20:05 |
| pabelanger | fungi: clarkb: any issues adding the previous maintainers of trystack.org into the trystack-site-core group? | 20:05 |
| *** ayoung has quit IRC | 20:05 | |
| *** armax has joined #openstack-infra | 20:05 | |
| *** unicell has joined #openstack-infra | 20:05 | |
| pabelanger | hashar: ohai! I'm checking out your systemd stuff for nodepool | 20:05 |
| pabelanger | trying to get it working locally too | 20:06 |
| hashar | pabelanger: happy you found it :} | 20:06 |
| hashar | pabelanger: I have a debian package for nodepool as well though I have yet to publish it anywhere :/ | 20:06 |
| *** glauco has quit IRC | 20:06 | |
| pabelanger | hashar: having some issues with dib right now, and sudo requiring tty. Trying to fix that now | 20:07 |
| pabelanger | hashar: you should import it here! | 20:07 |
| *** ayoung has joined #openstack-infra | 20:07 | |
| mordred | mtreinish: not to me - that seems excessive | 20:07 |
| mtreinish | mordred: well it trigger the jenkins timeout and killed it | 20:07 |
| mtreinish | clarkb: were you looking at that before, it seems familiar for some reason | 20:07 |
| hashar | !createrepo openstack-infra/zuul-packaging from https://gerrit.wikimedia.org/r/operations/debs/nodepool | 20:08 |
| openstack | hashar: Error: "createrepo" is not a valid command. | 20:08 |
| hashar | pabelanger: so there is a pending patch at https://gerrit.wikimedia.org/r/#/c/224390/1/debian/nodepool.systemd,unified | 20:09 |
| *** ociuhandu has joined #openstack-infra | 20:09 | |
| clarkb | mtreinish is it running workspace setup? | 20:09 |
| clarkb | anisble is less important and the command being run ia what is ibteresting | 20:09 |
| hashar | pabelanger: and another in puppet https://github.com/wikimedia/operations-puppet/blob/production/modules/nodepool/templates/initscripts/nodepool.systemd.erb | 20:09 |
| greghaynes | lucas-dinner: oh, sweet | 20:10 |
| lucas-dinner | greghaynes, yeah, one thing I need to do yet is to extend that patch mapping the ubuntu versions | 20:10 |
| hashar | pabelanger: the idea is to let systemd handle the daemon / pid file by itself. So nodepool is launched with -d (our puppet source) | 20:10 |
| lucas-dinner | to map the other versions | 20:10 |
| *** jcoufal has quit IRC | 20:11 | |
| lucas-dinner | greghaynes, that was just a quick test to see if it would fix the gate, but then we found out that dib wasn't being tested from source there :-( | 20:11 |
| *** shashank_hegde has joined #openstack-infra | 20:11 | |
| *** ihrachys has quit IRC | 20:11 | |
| mtreinish | clarkb: yeah it looks like setup workspace is the slow thing: http://logs.openstack.org/79/219379/1/check/gate-tempest-dsvm-large-ops/6f83839/logs/devstack-gate-setup-workspace-new.txt.gz | 20:11 |
| clarkb | I think thats zm02 hitting us which fungi is addressing | 20:11 |
| * lucas-dinner will do it tomorrow since it's quite late over here today (but feel free to extend it if you have some free time) | 20:11 | |
| hashar | pabelanger: and systemd does not wait after a graceful stop, so you need to ship a script that polls for the process while it stops : / | 20:11 |
| clarkb | you can confirm by checking the zuul url | 20:11 |
| mtreinish | clarkb: yep: http://logs.openstack.org/79/219379/1/check/gate-tempest-dsvm-large-ops/6f83839/logs/devstack-gate-setup-workspace-new.txt.gz#_2015-09-01_18_10_40_296 | 20:12 |
| *** Sukhdev has quit IRC | 20:12 | |
| greghaynes | lucas-dinner: got it. Trying to figure out how we can unravel the two breaks we have in DIB right now, so sounds like we need to wait for that devstack change to merge then the dib fix | 20:12 |
| lucas-dinner | greghaynes, yeah | 20:12 |
| *** vivekd has quit IRC | 20:13 | |
| lucas-dinner | greghaynes, we can merge 1) devstack 2) project config to use dib from source non-voting 3) fix dib ubuntu element 4) project-config to set it to vote on gate again | 20:13 |
| *** kdas_ has joined #openstack-infra | 20:14 | |
| lucas-dinner | 2) and 3) can be done in parallel, don't need to be in order | 20:14 |
| greghaynes | oh, we need to make it non-voting? I was thinking just merge devstack, project config to pull from source, then ubuntu element fix | 20:14 |
| *** abregman|afk is now known as abregman | 20:15 | |
| lucas-dinner | greghaynes, it's already non-voting | 20:15 |
| *** eharney has quit IRC | 20:15 | |
| lucas-dinner | all the pxe_ssh jobs were set to non-voting cause it was blocking not only ironic but also nova, dib etc... | 20:15 |
| *** jeadre has quit IRC | 20:15 | |
| *** kushal has quit IRC | 20:15 | |
| greghaynes | oh, so it is :) | 20:15 |
| greghaynes | ok, im going to merge the ubuntu element fix then | 20:16 |
| *** rlandy has quit IRC | 20:16 | |
| lucas-dinner | greghaynes, ack, thanks! But please add more versions to that map | 20:16 |
| *** jeadre has joined #openstack-infra | 20:16 | |
| lucas-dinner | since I only put trusty there because that was the one broken | 20:16 |
| lucas-dinner | I think all the others still using their names for version | 20:16 |
| *** rlandy has joined #openstack-infra | 20:17 | |
| lucas-dinner | precise = precise in the mapping (needs to check the SHA256SUMS file in the ubuntu website) | 20:17 |
| *** alivigni has quit IRC | 20:17 | |
| lucas-dinner | greghaynes, and thanks for the helP! | 20:17 |
| lucas-dinner | help* | 20:17 |
| greghaynes | lucas-dinner: np, thanks for the fixes :) | 20:17 |
| pabelanger | hashar: okay, checking out the reviews now. Just caching centos6 right now | 20:17 |
| lucas-dinner | it's all good | 20:18 |
| hashar | I am upstreaming the nodepool graceful stop script | 20:18 |
| fungi | clarkb: we have at least some logs where we fail to retrieve some refs (also zm02) not just slowness | 20:18 |
| *** shashank_hegde has quit IRC | 20:18 | |
| clarkb | fungi: gotcha so some are fails and others are slow | 20:19 |
| clarkb | and sometimes the slow ones are also fails due to timeouts | 20:19 |
| fungi | might all be timeouts, just at different layers | 20:19 |
| *** kgiusti has left #openstack-infra | 20:19 | |
| fungi | for example, the error in the previous round of your liberty cross-check going into the gate (i've rechecked since) | 20:20 |
| *** esker has quit IRC | 20:20 | |
| *** salv-orl_ has joined #openstack-infra | 20:20 | |
| fungi | "fatal: remote did not send all necessary objects" http://logs.openstack.org/00/217300/2/gate/gate-project-config-layout/219cba8/console.html#_2015-09-01_17_28_30_601 | 20:21 |
| *** pblaho has quit IRC | 20:21 | |
| *** eharney has joined #openstack-infra | 20:22 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack-infra/project-config: Make the ironic-pxe_ssh-dib job to build DIB from source https://review.openstack.org/219351 | 20:23 |
| *** esmute_ has joined #openstack-infra | 20:24 | |
| openstackgerrit | Antoine Musso proposed openstack-infra/nodepool: tools: shell script to gracefully stop and wait https://review.openstack.org/219442 | 20:24 |
| hashar | pabelanger: ^^^ | 20:24 |
| pabelanger | hashar: ack | 20:24 |
| *** salv-orlando has quit IRC | 20:24 | |
| anteaya | fungi: is the story I filed lacking in detail? https://storyboard.openstack.org/#!/story/2000341 | 20:25 |
| hashar | pabelanger: the version in our puppet is the one deployed ( https://github.com/wikimedia/operations-puppet/blob/production/modules/nodepool/templates/initscripts/nodepool.systemd.erb ) | 20:25 |
| hashar | pabelanger: it misses a few bits though such as the After=network.target and [Install] WantedBy=multi-user.target | 20:26 |
| *** jcoufal has joined #openstack-infra | 20:26 | |
| hashar | pabelanger: I am rebuilding our nodepool package this week so will have something more usable soonish | 20:26 |
| *** jeadre has quit IRC | 20:26 | |
| pabelanger | When an infra-core has time, if they could add wfoster@redhat.com / kambiz@redhat.com to trystack-site-core, that would be great. | 20:27 |
| clarkb | hashar: do you run as daemon or foreground and let systemd daemonize? | 20:27 |
| anteaya | yay Dorval! | 20:27 |
| clarkb | I am curious what the more systemdy way of doing that is | 20:27 |
| *** jeadre has joined #openstack-infra | 20:27 | |
| kambiz | pabelanger: thanks! | 20:27 |
| *** kdas_ has quit IRC | 20:28 | |
| *** olaph has quit IRC | 20:28 | |
| *** julim has quit IRC | 20:29 | |
| *** mrmartin has joined #openstack-infra | 20:29 | |
| *** vivekd has joined #openstack-infra | 20:30 | |
| hashar | clarkb: foreground and systemd handles everything | 20:30 |
| *** mrmartin has quit IRC | 20:30 | |
| hashar | clarkb: so we don't need to write a .pid file and could even remove the python-daemon dependency | 20:30 |
| clarkb | hashar only if we assume systemd is the only way | 20:31 |
| hashar | clarkb: from discussions we had with Wikimedia ops, they tend to prefer letting systemd handle the demonization / pid tracking | 20:31 |
| clarkb | which is bad assumption for now | 20:31 |
| clarkb | ya and thats supported | 20:31 |
| fungi | anteaya: nah, it's sufficient, it's just not likely a bug we're able to fix (unless you count opening a ticket with the service provider and waiting for them to act on it a "bug fix") | 20:31 |
| anteaya | fungi: ah okay, thanks | 20:32 |
| hashar | clarkb: yeah nobody is forced to use systemd and python-daemon should stick in Nodepool. That was more a rhetorical question : should dev bother reinventing the wheel of pid tracking / demonization ? :} | 20:32 |
| fungi | anteaya: pretty much "zm02 is performing terribly" is about all the detail we really need in that storyboard story (if we even need a storyboard story) | 20:32 |
| clarkb | hashar I wrote my first unit files last week which is why I am curious | 20:33 |
| anteaya | fungi: ah okay thanks, wasn't sure | 20:33 |
| anteaya | wanted to set a good example | 20:33 |
| hashar | clarkb: there is a bunch of glitches. Took me a while to figure out even the most basic but eventually ended up with that file. There is probably a lot more achievable with systemd though | 20:33 |
| *** baoli has quit IRC | 20:34 | |
| *** baoli has joined #openstack-infra | 20:34 | |
| *** baoli has quit IRC | 20:36 | |
| clarkb | ya I wrote them due to bugs in the centos7 unit files | 20:36 |
| clarkb | learned a lot in the process | 20:37 |
| *** abregman is now known as abregman|afk | 20:37 | |
| * anteaya packs up to prepare to disembark the train | 20:37 | |
| *** baoli has joined #openstack-infra | 20:37 | |
| *** dprince has quit IRC | 20:38 | |
| *** afranc202 has joined #openstack-infra | 20:39 | |
| *** julim has joined #openstack-infra | 20:42 | |
| *** olaph has joined #openstack-infra | 20:43 | |
| *** baoli has quit IRC | 20:43 | |
| *** bswartz has quit IRC | 20:44 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove the local copy of subunit2html https://review.openstack.org/212864 | 20:45 |
| *** olaph has quit IRC | 20:47 | |
| *** jeadre has quit IRC | 20:48 | |
| *** jeadre has joined #openstack-infra | 20:49 | |
| *** yamahata has quit IRC | 20:49 | |
| *** sdake has quit IRC | 20:53 | |
| *** thedodd has quit IRC | 20:53 | |
| *** thedodd has joined #openstack-infra | 20:54 | |
| *** achanda has quit IRC | 20:54 | |
| *** earlephilhower has quit IRC | 20:55 | |
| *** olaph has joined #openstack-infra | 20:55 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Start using keystoneauth for keystone sessions https://review.openstack.org/183176 | 20:56 |
| mordred | fungi: ^^ more code deleted | 20:56 |
| nibalizer | clarkb: okay I argued on the wildfly PR | 20:56 |
| fungi | mordred: i also saw a neutron one fly by earlier? | 20:56 |
| mordred | fungi: yeah. although I'm not sure that one works yet | 20:57 |
| mordred | jamielennox, morgan: ^^ there's the latest patch for using keystoneauth | 20:57 |
| * morgan looks. | 20:57 | |
| morgan | mordred: that looks pretty straight forward | 20:58 |
| openstackgerrit | Antoine Musso proposed openstack/diskimage-builder: dib-run-parts: make cp to target root more robust https://review.openstack.org/219453 | 20:58 |
| *** thedodd has quit IRC | 20:59 | |
| *** flepied has quit IRC | 20:59 | |
| *** trown is now known as trown|outttypeww | 20:59 | |
| mordred | morgan: it's not terrible | 20:59 |
| mordred | morgan: I, in fact, removed some tests which would have wound up being tests of "does this library work" | 21:00 |
| *** olaph has quit IRC | 21:00 | |
| morgan | hah | 21:00 |
| morgan | yeah | 21:00 |
| *** shashank_hegde has joined #openstack-infra | 21:00 | |
| mordred | morgan: if that passes tests, I think releasing ksa will have my blessing | 21:01 |
| openstackgerrit | Adam Harwell proposed openstack-infra/project-config: Update Barbican Gate to use new Devstack config https://review.openstack.org/217330 | 21:01 |
| jamielennox | mordred: cool, what i want to be sure of there is that none of the clients are doing anything stupid that will be different from ksc -> ksa | 21:01 |
| morgan | there are a few more lined up patches from jamielennox that would be good to get too | 21:01 |
| rm_work | ianw: fixed https://review.openstack.org/#/c/217330/ to depend on the project CR, and updated that CR to only ADD the new method, not delete the old one at the same time -- should be good now | 21:01 |
| mordred | ++ | 21:01 |
| rm_work | ianw: let me know if that looks OK when you have a moment? :) thanks for the feedback! this really is a safer way to do it, not sure why i didn't in the first place | 21:02 |
| *** changbl has joined #openstack-infra | 21:02 | |
| *** aysyd has quit IRC | 21:04 | |
| *** bgmccollum has left #openstack-infra | 21:04 | |
| *** rfolco has quit IRC | 21:05 | |
| *** rossella_s has quit IRC | 21:05 | |
| *** rossella_s has joined #openstack-infra | 21:06 | |
| jroll | looks like I'm having sporadic git issues here https://review.openstack.org/#/c/219330/ | 21:07 |
| *** sridhar_ram has quit IRC | 21:07 | |
| jroll | 2015-09-01 19:54:41.794 | fatal: Failed to traverse parents of commit dc28559b05ce38f1780b85359f14aeaf92115445 | 21:07 |
| jroll | 2015-09-01 19:54:41.795 | fatal: remote did not send all necessary objects | 21:07 |
| jroll | twice now :( | 21:07 |
| *** achanda has joined #openstack-infra | 21:07 | |
| fungi | jroll: is there a url for zm02 in the log? if so, it was struggling so i've taken it out of the cluster and opened a ticket with the service provider | 21:08 |
| jroll | lifeless: do I need to add things to upper-constraints manually? readme says it's generated but CI says not? https://review.openstack.org/#/c/219011/ | 21:08 |
| jroll | fungi: no, zm05 it seems | 21:08 |
| lifeless | jroll: its both, and sometimes. See the README and if tht doesn't clear it up we'll patch it until it does :) | 21:08 |
| * jroll reads again | 21:09 | |
| *** changbl has quit IRC | 21:09 | |
| fungi | jroll: hrm, that's unfortunate. i wonder if more than just zm02 are misbehaving now | 21:10 |
| jroll | fungi: yeah :( | 21:11 |
| jroll | I'll recheck for now | 21:11 |
| *** pradk has quit IRC | 21:11 | |
| jroll | lifeless: so adding a new library will require me to at a minimum add that library to upper-constraints, and I should also check the diff | 21:11 |
| *** sridhar_ram has joined #openstack-infra | 21:11 | |
| *** pradk has joined #openstack-infra | 21:12 | |
| *** changbl has joined #openstack-infra | 21:13 | |
| fungi | clarkb: jeblair: yikes, i'm finding this on several more zuul-merger servers now | 21:14 |
| *** abregman|afk has quit IRC | 21:14 | |
| fungi | http://logs.openstack.org/30/219330/2/gate/gate-project-config-layout/0b6afac/console.html | 21:14 |
| fungi | http://logs.openstack.org/00/217300/2/check/gate-project-config-layout/83add3c/console.html | 21:14 |
| fungi | http://logs.openstack.org/00/217300/2/gate/gate-project-config-layout/219cba8/console.html | 21:15 |
| jeblair | fungi: are we sure that's because of the merger? | 21:15 |
| fungi | across workers in multiple providers, and impacting more than one repo | 21:15 |
| lifeless | jroll: yes | 21:15 |
| *** e0ne has quit IRC | 21:15 | |
| *** cburroughs has quit IRC | 21:15 | |
| fungi | jeblair: i'm not certain yet | 21:15 |
| openstackgerrit | Glauco Oliveira proposed openstack-infra/puppet-jenkins: Fix puppet-lint errors https://review.openstack.org/219454 | 21:15 |
| *** e0ne has joined #openstack-infra | 21:16 | |
| fungi | jeblair: looks like in those examples we see it affecting a clone of jenkins-job-builder and a clone of zuul | 21:16 |
| jeblair | fungi, clarkb: the git mirror changed recently, right? | 21:16 |
| *** dkranz has quit IRC | 21:16 | |
| fungi | oh... | 21:16 |
| *** esmute_ has quit IRC | 21:16 | |
| *** ldnunes has quit IRC | 21:16 | |
| fungi | yep, i bet this is zuul-cloner updating from git.o.o | 21:16 |
| *** geoffarnold has quit IRC | 21:17 | |
| fungi | i'm going to take 07 out of the haproxy rotation right now | 21:17 |
| clarkb | its already out? | 21:18 |
| clarkb | I didnt see that my change merged | 21:18 |
| *** zeih has joined #openstack-infra | 21:18 | |
| fungi | hrm, yeah it's not merged yet | 21:18 |
| fungi | just 08 | 21:19 |
| fungi | trying to correlate to logs | 21:19 |
| clarkb | fungi https://review.openstack.org/#/c/219427/ is not merged and I took 07 out when the dns change was made | 21:19 |
| clarkb | to avoid it going stale | 21:19 |
| fungi | yep, makes sense | 21:19 |
| *** tiswanso has quit IRC | 21:19 | |
| clarkb | so 07 shouldnt affect anything right now unless loss of 1/8 capacity broke us | 21:20 |
| lifeless | fungi: when the crisis is over, I'd like to get you and clarkb to +2A https://review.openstack.org/#/c/207262/ | 21:20 |
| *** maishsk has quit IRC | 21:20 | |
| jeblair | clarkb: what's the status of 08? | 21:20 |
| clarkb | jeblair its in use and was happy last I checked | 21:21 |
| clarkb | I did delete old 08 earlier today | 21:21 |
| clarkb | I was running top on it this mroning to keep an eye on general memory and cpu use and it seemed fine | 21:22 |
| *** zeih has quit IRC | 21:22 | |
| *** esmute_ has joined #openstack-infra | 21:23 | |
| clarkb | has been in ise since early saturday utc | 21:23 |
| fungi | analyzing http://logs.openstack.org/00/217300/2/gate/gate-project-config-layout/219cba8/console.html the 4693760 node was 15.126.217.54 and at 17:28:30 haproxy directed it to balance_git_https/git08.openstack.org | 21:24 |
| *** ddieterly has quit IRC | 21:24 | |
| *** glauco__ has quit IRC | 21:24 | |
| *** maishsk has joined #openstack-infra | 21:24 | |
| fungi | though it was balanced to a couple other backends around that timeframe too, so it's not a slam dunk | 21:24 |
| clarkb | greghaynes had a job fail like that too hit a bunch of backends | 21:24 |
| *** julim has quit IRC | 21:25 | |
| clarkb | could it be hp networking? any rax fails? | 21:25 |
| clarkb | and haproxy put state in CD for three of the connectiona in that case and each was a different backend | 21:25 |
| fungi | we have similar failures with the same error from rax workers too | 21:26 |
| *** geoffarnold has joined #openstack-infra | 21:26 | |
| fungi | for example http://logs.openstack.org/00/217300/2/check/gate-project-config-layout/83add3c/console.html | 21:26 |
| openstackgerrit | Paul Van Eck proposed openstack-infra/puppet-refstack: Add param to specify app protocol https://review.openstack.org/219460 | 21:27 |
| openstackgerrit | Paul Van Eck proposed openstack-infra/puppet-refstack: Update refstack.conf template https://review.openstack.org/219462 | 21:27 |
| *** thorst has quit IRC | 21:29 | |
| *** hashar has quit IRC | 21:30 | |
| *** achanda has quit IRC | 21:30 | |
| *** tlian2 has joined #openstack-infra | 21:31 | |
| fungi | clarkb: i've correlated the ipv6 address in that one to a request which got directed to git08 as well | 21:32 |
| jeblair | fungi: which protocol was it using? | 21:32 |
| clarkb | git:// should syslog and http/https will got to apache log | 21:32 |
| fungi | Sep 1 20:51:41 localhost haproxy[34107]: 2001:4801:7828:101:be76:4eff:fe11:99bf:53916 [01/Sep/2015:20:51:41.011] balance_git_https balance_git_https/git08.openstack.org 1/0/449 211385 -- 61/45/45/6/0 0/0 | 21:33 |
| jeblair | fungi: ah, thanks; and i see you included that in your earlier statement too, sorry. | 21:33 |
| fungi | jeblair: well, this is a different one, but same protocol and backend | 21:34 |
| clarkb | fungi that doesnt look like an errored connection from haproxies perspective so likely failed above tcp | 21:34 |
| *** bpoulos has joined #openstack-infra | 21:34 | |
| *** pc_m has quit IRC | 21:34 | |
| clarkb | the two --s are where the error flags go | 21:34 |
| *** tlian has quit IRC | 21:34 | |
| jeblair | fungi: okay, which repo was that last request for? | 21:35 |
| fungi | jeblair: zuul | 21:35 |
| fungi | i don't see any time-correlated errors in git08's /var/log/httpd/git-error.log | 21:37 |
| clarkb | neither do I | 21:37 |
| fungi | 104.239.158.55 - - [01/Sep/2015:20:51:41 +0000] "POST /openstack-infra/zuul/git-upload-pack HTTP/1.1" 200 202928 "-" "git/1.9.1" | 21:38 |
| clarkb | my guess is that the cgi script successfully returned failure to the cleitn and didn't treat that as exceptional? | 21:38 |
| fungi | that's the only request i can find which correlates time-wise and was for the correct repo | 21:38 |
| *** puranamr has joined #openstack-infra | 21:38 | |
| clarkb | I can clone from git08 and git show 7ff604053aa220bd8466b326350c4bc419e038f7 succesffuly now | 21:39 |
| fungi | i can locally clone from the copy of the repo it's serving and checkout the commit it wanted | 21:39 |
| clarkb | and am using trusty git 1.9.1 locally | 21:39 |
| fungi | yep, that too | 21:40 |
| fungi | baffling | 21:40 |
| clarkb | most of the errors in the httpd logs appear related to repos that have been renamed so not even seeing any errors there that could be related at different times | 21:40 |
| clarkb | there are no OOMkillers in dmesg | 21:41 |
| clarkb | so not seeing that behavior again | 21:41 |
| *** miqui has quit IRC | 21:42 | |
| *** sergmelikyan has quit IRC | 21:42 | |
| clarkb | and messages doesn't show any system problems during that time | 21:43 |
| clarkb | what if git pushes from gerrit are not atomic ? | 21:43 |
| clarkb | could the child commit be in the repo before the parent and cause that error? | 21:44 |
| fungi | how recent are those objects? | 21:44 |
| clarkb | https://review.openstack.org/#/c/187268/ couple weeks old | 21:45 |
| clarkb | so likely not it | 21:45 |
| fungi | the ones i see in the current example are 2-week-old objects | 21:45 |
| clarkb | ya | 21:45 |
| fungi | yeah | 21:45 |
| *** mriedem is now known as mriedem_away | 21:46 | |
| clarkb | the other thing it could be is selinux maybe? let me look at the audit log | 21:47 |
| clarkb | thats mostly just successful git-daemon starts from systemd's socket activation inetd replacement | 21:48 |
| *** annegentle has quit IRC | 21:48 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: publish docs built from releases repository https://review.openstack.org/219420 | 21:49 |
| fungi | yeah, i'm not finding any obvious signs as to what might be going on there | 21:50 |
| jeblair | fungi, clarkb; oh these are shallow clones | 21:50 |
| clarkb | oh hrm :/ | 21:50 |
| fungi | gah | 21:50 |
| clarkb | let me test that then | 21:50 |
| fungi | this might explain why we're just seeing it on the layout job | 21:50 |
| clarkb | we did test it but maybe corner case | 21:50 |
| fungi | or at least that's the only place i've spotted it thus far | 21:50 |
| jeblair | yeah, both of the jjb and zuul clones in the layout job are shallow | 21:51 |
| greghaynes | My fail was also in the layout job | 21:51 |
| *** burgerk has quit IRC | 21:51 | |
| jeblair | they use https | 21:51 |
| jeblair | i am unable to repro locally though | 21:51 |
| clarkb | git clone http://git08.openstack.org:8080/openstack-infra/zuul --depth 1 zuul-shallow wfm | 21:52 |
| clarkb | let me try https | 21:52 |
| fungi | i'm a little less concerned about this being an openstack-wide emergency now if it's only impacting jobs that do shallow clones | 21:52 |
| *** yamamoto has joined #openstack-infra | 21:53 | |
| clarkb | agreed my client crashes when I set GIT_SSL_NO_VERIFY=1 I think I have it all set up proprely and when I tested I did test http, https and git protocl | 21:53 |
| jeblair | i had to set up an /etc/hosts entry | 21:54 |
| clarkb | oh maybe thats it let me try | 21:54 |
| clarkb | I was using ip addrs when I tested | 21:54 |
| jeblair | but after doing so, it wfm; i also did the same on a bare-trusty-hpcloud-b2 node and it also works | 21:54 |
| clarkb | yup IP addr works and is fine | 21:54 |
| *** PinkFreud has quit IRC | 21:55 | |
| jeblair | oh that's easier then :) | 21:55 |
| clarkb | any opposition to set -x'ing that script? | 21:56 |
| clarkb | would've made catching its shallow cloning slightly easier | 21:56 |
| jeblair | clarkb: that doesn't seem necessary :( | 21:56 |
| clarkb | definitely won't help debug the rest of this, this is so weird that it works locally | 21:57 |
| *** asettle has joined #openstack-infra | 21:57 | |
| jeblair | it may have been a transient thing that only happens under certain conditions | 21:57 |
| *** changbl has quit IRC | 21:57 | |
| *** yamamoto has quit IRC | 21:58 | |
| clarkb | /tmp is part of / and has many gigs available so don't thin kwe filled a scratch space | 21:59 |
| *** Hal has quit IRC | 21:59 | |
| *** stevemar has quit IRC | 21:59 | |
| clarkb | oh wait | 21:59 |
| clarkb | jeblair: fungi the error messages says it couldn't traverse parents of commit foo which makes sense since its a depth of one so only foo would exist | 22:00 |
| clarkb | jeblair: fungi I wonder if the server is fine but its something client side | 22:00 |
| *** bpoulos has left #openstack-infra | 22:00 | |
| *** Piet has quit IRC | 22:01 | |
| *** shardy has quit IRC | 22:01 | |
| lifeless | pbr and the new release notes stuff are somewhat incompatible with thin clones | 22:01 |
| lifeless | in that they look at history | 22:01 |
| lifeless | (perhaps irrelevant to the issue at hand - sorry if so) | 22:01 |
| clarkb | lifeless: good point | 22:01 |
| jeblair | clarkb, lifeless: it appears to be the git clone itself that's failing | 22:01 |
| clarkb | jeblair: hard to say without -x but I think you may be right | 22:01 |
| *** amitgandhinz has quit IRC | 22:02 | |
| jeblair | clarkb: we see "Cloning into 'zuul'" but not "Cloning into 'jenkins-job-builder'" in http://logs.openstack.org/00/217300/2/gate/gate-project-config-layout/219cba8/console.html | 22:02 |
| *** stevemar has joined #openstack-infra | 22:02 | |
| jeblair | clarkb: so we know that it's failing on the first of two clone commands that are adjacent | 22:02 |
| *** hdd has quit IRC | 22:02 | |
| *** stevemar has quit IRC | 22:02 | |
| jeblair | (see http://logs.openstack.org/30/219330/2/gate/gate-project-config-layout/0b6afac/console.html for a successful zuul clone followed by an unsuccessful jjb clone) | 22:02 |
| openstackgerrit | Dan Prince proposed openstack-infra/tripleo-ci: Switch overcloud-f21-nonha to instack https://review.openstack.org/185151 | 22:06 |
| *** rlandy has quit IRC | 22:08 | |
| *** puranamr has quit IRC | 22:08 | |
| *** jsavak has quit IRC | 22:08 | |
| *** jcoufal has quit IRC | 22:09 | |
| *** e0ne has quit IRC | 22:10 | |
| *** glauco_ has joined #openstack-infra | 22:10 | |
| *** mquin_ has joined #openstack-infra | 22:12 | |
| clarkb | the error there is one I would expect if we attempted to look at the parent commit since the parent commit does not exist | 22:12 |
| *** mquin has quit IRC | 22:13 | |
| clarkb | what is confusing is why anything would want to traverse to the parent during a shallow clone | 22:13 |
| *** e0ne has joined #openstack-infra | 22:14 | |
| *** Rockyg has joined #openstack-infra | 22:14 | |
| *** glauco_ has quit IRC | 22:15 | |
| openstackgerrit | Adam Harwell proposed openstack-infra/project-config: Update Barbican Gate to use new Devstack config https://review.openstack.org/217330 | 22:16 |
| clarkb | could it be something silly and git clone cares about the the order of args, maybe local side sees it as not shallow but remote sees it as shallow? (I am thinking out loud here don't really expect thta to be the issue) | 22:18 |
| openstackgerrit | Adam Harwell proposed openstack-infra/project-config: Update Barbican Gate to use new Devstack config https://review.openstack.org/217330 | 22:18 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 22:18 | |
| *** zeih has joined #openstack-infra | 22:19 | |
| rm_work | hey, any chance anyone can look at this CR soon? we're trying to get this stuff in before the freeze -- I've corrected the job updates to be non-breaking (per ianw's suggestion), so hopefully not a big deal? https://review.openstack.org/217330 | 22:20 |
| rm_work | I know it's super busy around here pretty much permanently -- if we don't make the freeze, it's not the end of the world | 22:20 |
| *** regXboi has quit IRC | 22:21 | |
| clarkb | `while git clone http://104.239.146.131:8080/openstack-infra/zuul --depth 1 ; do rm -rf zuul ; done` is now running | 22:21 |
| *** zeih has quit IRC | 22:23 | |
| fungi | jhesketh: around yet? was there a proposed implementation for creating parent tree indices for swift-hosted job logs? | 22:23 |
| clarkb | fungi: there is a first draft iirc | 22:24 |
| clarkb | https://review.openstack.org/#/c/214207/ | 22:24 |
| clarkb | my while loop has yet to fail | 22:24 |
| fungi | oh, cool. rbradfor is hacking on solutions to do coverage job analysis and since those are all uploaded to swift now we don't have any way to guess the urls (because of the embedded zuul uuid) | 22:24 |
| rm_work | dougwig: ^^ | 22:25 |
| clarkb | fungi: we made coverage go to the log server again because of that | 22:25 |
| rm_work | dougwig: err, https://review.openstack.org/217330 -- should be right now I think | 22:25 |
| clarkb | fungi: so it should be working now | 22:25 |
| *** mwagner_lap has quit IRC | 22:26 | |
| fungi | clarkb: when did we do that? it's still going to swift right now (as in looking at samples from the last few minutes) | 22:26 |
| *** gordc has quit IRC | 22:27 | |
| clarkb | fungi: https://review.openstack.org/#/c/210114/ | 22:27 |
| *** edmondsw has quit IRC | 22:28 | |
| clarkb | oh wait wrong one | 22:28 |
| clarkb | gah I think I had the thing inverted | 22:28 |
| clarkb | so I fixed one issue with that change but it created the issue you are talking about | 22:29 |
| clarkb | :/ | 22:29 |
| openstackgerrit | Alan Pevec proposed openstack/requirements: Allow updated pyScss and django-pyscss on stable/kilo https://review.openstack.org/219474 | 22:29 |
| clarkb | while loop still looping | 22:29 |
| *** thiagop has quit IRC | 22:30 | |
| openstackgerrit | Jim Rollenhagen proposed openstack/requirements: Add ironic-lib at initial release https://review.openstack.org/219011 | 22:31 |
| jroll | lifeless: I should have trusted my instincts :P | 22:31 |
| lifeless | jroll: ? | 22:31 |
| jroll | that requirements patch right there | 22:31 |
| jroll | lifeless: only needed the new library in upper-constraints, because that code is copied from ironic, duh :) | 22:32 |
| lifeless | sure | 22:32 |
| *** earlephilhower has joined #openstack-infra | 22:32 | |
| lifeless | ping me when its passed CI | 22:32 |
| jroll | I guess I learned how to do the thing, at least | 22:32 |
| jroll | sure | 22:32 |
| lifeless | jroll: also | 22:32 |
| lifeless | jroll: you *will* want requirements syncin for it | 22:32 |
| lifeless | jroll: so you need check-requirements configured in zuul, and you need to add it to projects.txt too | 22:33 |
| *** TheJulia has quit IRC | 22:33 | |
| jroll | lifeless: it's already syncing | 22:33 |
| jroll | just ready to be used now | 22:33 |
| lifeless | kk | 22:34 |
| *** e0ne has quit IRC | 22:34 | |
| clarkb | jeblair: fungi any other debugging ideas? should we take 08 out of the rotation and put old 07 back? | 22:35 |
| jroll | fungi: hitting zm04 now :/ http://logs.openstack.org/30/219330/2/gate/gate-project-config-layout/cf054de/console.html | 22:35 |
| clarkb | my while loop is still running so if it is intermittent I don't see it locally | 22:35 |
| rm_work | ianw: thanks for the feedback and the +2, this is a lot cleaner than it was originally | 22:39 |
| rm_work | ianw: not sure why doing it this way wasn't obvious to me before :P | 22:39 |
| rbradfor | fungi, I can now determine the directory via status.json so I believe I can build an index of directories dynamically. A challenge for tomorrow | 22:39 |
| *** ddieterly has joined #openstack-infra | 22:39 | |
| *** jaypipes has quit IRC | 22:40 | |
| *** annegentle has joined #openstack-infra | 22:40 | |
| *** esmute_ has quit IRC | 22:41 | |
| jeblair | jroll: it's not a zm problem, that's a git server problem (1 out of our 8 git servers is running centos7 and having problems with shallow clones) | 22:44 |
| jroll | jeblair: ah, fungi asked about zm earlier | 22:44 |
| jeblair | jroll: yeah, we, er, adjusted our hypothesis :) | 22:44 |
| jroll | jeblair: should I just keep rechecking for now? | 22:44 |
| jroll | it keeps flapping between verified and unverified | 22:45 |
| *** puranamr has joined #openstack-infra | 22:45 | |
| jeblair | jroll: yes, rechecking may produce a different outcome depending on load balancer rotation | 22:45 |
| jroll | k, thank you | 22:45 |
| *** asselin_ has quit IRC | 22:46 | |
| *** esmute_ has joined #openstack-infra | 22:46 | |
| *** baoli has joined #openstack-infra | 22:48 | |
| *** gildub has joined #openstack-infra | 22:49 | |
| jeblair | clarkb: neat i just got a local repo | 22:49 |
| jeblair | repro | 22:49 |
| clarkb | jeblair I really thibk it has to be a client issue | 22:49 |
| clarkb | oh how? just repeated or something different? | 22:50 |
| jeblair | clarkb: going through the load balancer | 22:50 |
| jeblair | took a couple of times... | 22:50 |
| jeblair | clarkb: but i'm wondering if the issue is due to git version mismatch across the cluster | 22:51 |
| clarkb | are we not using source hash balancing anymore? | 22:51 |
| jeblair | clarkb: we did switch to leastconn | 22:51 |
| clarkb | ah ok | 22:51 |
| jeblair | so maybe the solution is to rip the bandaid off. | 22:51 |
| clarkb | wfm | 22:52 |
| jeblair | (or, if we want to go more slowly, temp switch back to source hash) | 22:52 |
| clarkb | maybe do thst so we arent rushed, but still pull bandaid quickly | 22:52 |
| *** Rockyg has quit IRC | 22:53 | |
| jeblair | it may cause other problems (like load-related timeouts), though we have a few more servers now, so maybe not as severe. | 22:53 |
| openstackgerrit | Clint Adams proposed openstack-infra/infra-specs: WIP - tracking of priority efforts in YAML https://review.openstack.org/219372 | 22:53 |
| jeblair | fungi: fyi ^ possible root cause | 22:54 |
| *** achanda has joined #openstack-infra | 22:54 | |
| *** Sukhdev has joined #openstack-infra | 22:55 | |
| clarkb | spinning up new backends is easy now, all patches landed. the only two special steps have been stopping the backend in haprpxy when dns flips and copyibg the rsa host key over so gerrit doesnt see a mismatch | 22:55 |
| *** angela-s has quit IRC | 22:55 | |
| *** wenlock has quit IRC | 22:55 | |
| *** watanabe_isao has joined #openstack-infra | 22:55 | |
| clarkb | I can roll through them quickly if we need to | 22:56 |
| *** ddieterly has quit IRC | 22:57 | |
| *** boris-42 has joined #openstack-infra | 22:57 | |
| openstackgerrit | Merged openstack-infra/project-config: Switch ironic full tempest to use IPA https://review.openstack.org/219330 | 22:58 |
| clarkb | (if you don't copy the host key we would need to restart gerrit in order to get replication working) | 22:58 |
| *** achanda has quit IRC | 22:59 | |
| *** jogo has quit IRC | 22:59 | |
| openstackgerrit | Darragh Bailey proposed openstack-infra/jenkins-job-builder: Support default string values for variables https://review.openstack.org/180117 | 23:00 |
| *** vivekd has quit IRC | 23:00 | |
| *** bharathm has quit IRC | 23:01 | |
| *** vivekd has joined #openstack-infra | 23:01 | |
| clarkb | StevenK: awake yet? you have details for item 3 https://etherpad.openstack.org/p/zanata-deploy ? would be good to get those into the etherpad | 23:02 |
| *** dims has quit IRC | 23:03 | |
| *** jogo has joined #openstack-infra | 23:05 | |
| *** jogo has quit IRC | 23:05 | |
| *** jogo has joined #openstack-infra | 23:05 | |
| *** sridhar_ram has quit IRC | 23:05 | |
| *** asselin has joined #openstack-infra | 23:06 | |
| *** ivar-laz_ has joined #openstack-infra | 23:07 | |
| *** Piet has joined #openstack-infra | 23:08 | |
| openstackgerrit | Christopher Aedo proposed openstack-infra/puppet-apps_site: Enable CORS for new /api directory and gzip https://review.openstack.org/218898 | 23:10 |
| fungi | jroll: clarkb: yeah, the clone failures and the performance issues on zm02 seem to be unrelated, but zm02 has still been having other issues (for which i opened a ticket with rax) | 23:10 |
| *** ivar-lazzaro has quit IRC | 23:10 | |
| jroll | fungi: ew tickets | 23:11 |
| jroll | :P | 23:11 |
| *** legoktm is now known as egoktm | 23:11 | |
| *** jswarren has quit IRC | 23:11 | |
| fungi | clarkb: yeah, other potential challenge with spinning up the rest of them is we're very close to quota | 23:11 |
| clarkb | I think switching balance type will also confirm if thats the bug | 23:13 |
| *** yamamoto has joined #openstack-infra | 23:13 | |
| openstackgerrit | Christopher Aedo proposed openstack-infra/puppet-apps_site: Enable CORS for new /api directory and gzip https://review.openstack.org/218898 | 23:13 |
| *** asselin has quit IRC | 23:14 | |
| *** amuller has quit IRC | 23:14 | |
| *** egoktm is now known as legoktm | 23:14 | |
| *** asselin has joined #openstack-infra | 23:14 | |
| fungi | agreed | 23:16 |
| *** yamamoto has quit IRC | 23:16 | |
| fungi | easy enough to toggle | 23:16 |
| *** eantyshev has quit IRC | 23:17 | |
| *** eantyshev has joined #openstack-infra | 23:17 | |
| *** chlong_ is now known as chlong | 23:18 | |
| *** olaph has joined #openstack-infra | 23:18 | |
| openstackgerrit | Merged openstack/requirements: require SQLAlchemy >=0.9.9 https://review.openstack.org/216360 | 23:18 |
| *** puranamr has quit IRC | 23:18 | |
| *** puranamr has joined #openstack-infra | 23:19 | |
| *** zeih has joined #openstack-infra | 23:19 | |
| *** watanabe_isao has quit IRC | 23:20 | |
| *** shashank_hegde has quit IRC | 23:21 | |
| *** mdbooth has quit IRC | 23:21 | |
| docaedo | yolanda: took me all day to circle back to it, but I think I made 218898 do what you suggested - let us know how it looks now, thanks! | 23:21 |
| *** tonytan4ever has quit IRC | 23:22 | |
| *** ZZelle_ has quit IRC | 23:23 | |
| *** puranamr has quit IRC | 23:23 | |
| *** dims__ has joined #openstack-infra | 23:24 | |
| *** zeih has quit IRC | 23:25 | |
| *** MarkAtwood has quit IRC | 23:25 | |
| *** mdbooth has joined #openstack-infra | 23:26 | |
| openstackgerrit | Christopher Aedo proposed openstack-infra/puppet-apps_site: Enable CORS for new /api directory and gzip https://review.openstack.org/218898 | 23:29 |
| *** dannywil_ has joined #openstack-infra | 23:30 | |
| *** ivar-laz_ has quit IRC | 23:33 | |
| *** wenlock has joined #openstack-infra | 23:33 | |
| *** dannywilson has quit IRC | 23:33 | |
| *** FallenPe_ has quit IRC | 23:36 | |
| *** hodos has quit IRC | 23:38 | |
| *** puranamr has joined #openstack-infra | 23:39 | |
| jgriffith | fungi: clarkb either of you know how to fix a patch that refuses to get picked up by the verify stage? | 23:40 |
| jgriffith | https://review.openstack.org/#/c/215847/ | 23:40 |
| jgriffith | fungi: clarkb mtanino and I have tried a number of things, "reverify" just add another +2/A etc... even took it back through check | 23:40 |
| jgriffith | fungi: clarkb but it seems to be *stuck* | 23:41 |
| *** bharathm has joined #openstack-infra | 23:41 | |
| *** geoffarnold has quit IRC | 23:43 | |
| openstackgerrit | Darragh Bailey proposed openstack-infra/jenkins-job-builder: Support default string values for variables https://review.openstack.org/180117 | 23:43 |
| *** salv-orlando has joined #openstack-infra | 23:43 | |
| *** salv-orl_ has quit IRC | 23:47 | |
| *** puranamr has quit IRC | 23:48 | |
| jgriffith | mtanino: well I tried anyway... need to head out, now | 23:52 |
| jgriffith | mtanino: I'll check in later tonight if I can, or in the morning | 23:52 |
| *** eantyshev has quit IRC | 23:52 | |
| mtanino | jgriffith: yeah, I hope infra member will check that message. | 23:52 |
| mtanino | jgriffith: Thanks :) | 23:52 |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/zuul: Ignore noop jobs when combining ChangeQueues https://review.openstack.org/219514 | 23:53 |
| *** puranamr has joined #openstack-infra | 23:53 | |
| *** esker has joined #openstack-infra | 23:53 | |
| *** jhesketh has quit IRC | 23:54 | |
| *** jhesketh has joined #openstack-infra | 23:55 | |
| fungi | jgriffith: looking | 23:55 |
| openstackgerrit | Darragh Bailey proposed stackforge/python-jenkins: Migration to using requests https://review.openstack.org/205637 | 23:55 |
| *** dingyichen has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
