| cinerama | clarkb: should have /opt/zanata & symlink to zanata-cli in /usr/local, and not have broken other stuff on there | 00:00 |
|---|---|---|
| pleia2 | ah, thanks cinerama | 00:00 |
| cinerama | i'm handwaving the other stuff | 00:00 |
| pleia2 | hehe | 00:00 |
| clarkb | cinerama: confirmed that those two things exist and zanata-cli --help gives me a help message | 00:01 |
| clarkb | also I see that ~jenkins/.config/zanata.ini is present | 00:01 |
| clarkb | so lgtm | 00:01 |
| pleia2 | \o/ | 00:01 |
| *** ZZelle_ has quit IRC | 00:01 | |
| clarkb | cinerama: I am basically handwaving my way around linux networking too :) | 00:01 |
| *** otter768 has quit IRC | 00:03 | |
| pleia2 | cinerama: yay, +2ed https://review.openstack.org/#/c/169074/ | 00:03 |
| clarkb | armax: another fun thing I discovered doing all this is netcat in cirros is from busybox and doesn't know how to listen properly | 00:03 |
| clarkb | armax: at least I couldn't get it to work | 00:04 |
| cinerama | pleia2: oh cool. what happens next? | 00:04 |
| armax | clarkb: good to know | 00:04 |
| clarkb | cinerama: pleia2 I will review that right now | 00:04 |
| *** dannywilson has quit IRC | 00:04 | |
| timrc | clarkb, Do y'all ever see timeouts in zuul-merger due to networking issues? It looks like we're using the default poll time and interval and occasionally a Exception: Timeout waiting for fetching ref for merging will get thrown which results in a failed merge being posted back to the review by zuul. | 00:05 |
| clarkb | pleia2: cinerama jeblair mentioned trove, are we just going to use a local db instead? | 00:05 |
| pleia2 | clarkb: check which netcat it is? there are two versions (netcat-traditional vs netcat-openbsd) and they have different options | 00:05 |
| clarkb | timrc: no because we don't have to poll anything | 00:05 |
| clarkb | pleia2: its netcat bsd and even better its like the minimally built one for busybox | 00:05 |
| pleia2 | ah, k | 00:06 |
| clarkb | cirros is a great tool because it boots on qemu quickly | 00:06 |
| pleia2 | clarkb: re: trove, I don't know for the dev server | 00:06 |
| timrc | clarkb, Ah. | 00:06 |
| clarkb | but its sometimes very different :) | 00:06 |
| cinerama | pleia2, clarkb: that was the first i'd heard about that...it is not in the spec | 00:06 |
| clarkb | cinerama: it is a comment on that change | 00:06 |
| clarkb | timrc: polling is only necessary if you pull chnages from not gerrit | 00:07 |
| *** achanda has quit IRC | 00:07 | |
| pleia2 | clarkb: yeah, we never "check[ed] in as a group" | 00:07 |
| cinerama | pleia2, clarkb: right, i'm saying that was the first time i'd seen that suggested. do we want to do that before we do the dev server? | 00:07 |
| timrc | clarkb, Yeah, we have to do that because.. yeah. Don't want to get into that :) | 00:07 |
| clarkb | timrc: also latest gerrit does have a replicated event so zuul could be made smarter about it instead of polling but it requires very new gerrit. >=2.9 iirc | 00:07 |
| clarkb | timrc: I am familiar :) | 00:07 |
| timrc | clarkb, Okay. Well, that's a good insight to have. | 00:08 |
| timrc | clarkb, Thanks. | 00:08 |
| clarkb | cinerama: pleia2: its a dev server, I think we can convert to a trove db later if you want to just keep moving today | 00:08 |
| pleia2 | cinerama: it's more of a team "when we have mysql servers, we put them in trove" generalization, not necessarily something we'd add to every spec | 00:08 |
| clarkb | cinerama: pleia2 or I am happy to spin up a trove db too and put the info in hiera | 00:08 |
| clarkb | though maybe it should go in specs to avoid this confusion at the last minute :) | 00:08 |
| pleia2 | clarkb: well, I've never done that, so perhaps it makes sense for me to set up trove when I set up the server for the learning | 00:09 |
| cinerama | pleia2, clarkb: if it's going to Just Work i'm okay with trying it | 00:09 |
| clarkb | pleia2: that works too | 00:09 |
| pleia2 | I think setting up server has to happen tomorrow morning though, I've had a very long day and things to do tonight | 00:09 |
| clarkb | pleia2: note I have only ever GUI'd it though | 00:09 |
| clarkb | but ya if you log into the rackspace control panel its pretty simple to just copy what review-dev does | 00:10 |
| *** sarob has joined #openstack-infra | 00:10 | |
| clarkb | I think its a 2GB instance with a non default mysql config | 00:10 |
| pleia2 | clarkb: ok, want to sync up tomorrow morning to get it done? | 00:10 |
| clarkb | pleia2: sure | 00:10 |
| pleia2 | great | 00:10 |
| *** Sukhdev has joined #openstack-infra | 00:10 | |
| clarkb | cinerama: we can update the change to assume a trove db with creds in hiera | 00:10 |
| clarkb | cinerama: so htat it is ready for us tomorrow | 00:11 |
| *** achanda has joined #openstack-infra | 00:11 | |
| clarkb | cinerama: to do that we need the mysql_host and zanata_db_password values to be passed from hiera in site.pp into zanata_dev.pp | 00:12 |
| clarkb | cinerama: then we can remove zanata::mysql from that manifest | 00:12 |
| *** ashleighfarnham has quit IRC | 00:13 | |
| clarkb | cinerama: I would use hiera keys of zanata_dev_mysql_host and zanata_dev_db_password. Once pleia2 has that trove db created hiera can be populated with those keys | 00:13 |
| *** sarob has quit IRC | 00:15 | |
| fungi | though really the host key is the one you can't prepopulate since it's provided to you at creation time. but might as well not bother editing the hiera file until we're ready to put both in | 00:16 |
| *** achanda has quit IRC | 00:16 | |
| fungi | er, hostname (not host key) | 00:16 |
| * fungi has keys on the brane | 00:16 | |
| clarkb | mmm branes | 00:16 |
| *** ianw has quit IRC | 00:17 | |
| *** ianw has joined #openstack-infra | 00:18 | |
| openstackgerrit | Stephanie Miller proposed openstack-infra/system-config: Add infra setup for zanata https://review.openstack.org/169074 | 00:18 |
| * fungi is a byproduct of brane theory | 00:19 | |
| *** yamamoto has joined #openstack-infra | 00:19 | |
| bknudson | can anybody tell me where the 1.4.0 version string is coming from here: http://logs.openstack.org/38/166438/2/check/gate-tempest-dsvm-neutron-src-keystonemiddleware-juno/b53f816/logs/devstacklog.txt.gz#_2015-04-09_22_55_21_918 | 00:19 |
| *** sputnik13 has quit IRC | 00:19 | |
| bknudson | looks like it just installed /opt/stack/new/keystonemiddleware ... where does it get 1.4.0 from? | 00:20 |
| clarkb | cinerama: that looks 99% there, noticed one small thing | 00:20 |
| bknudson | on my dev system I get keystonemiddleware-1.5.0.post9 | 00:20 |
| clarkb | bknudson: global reqs has a cap of < 1.5.0 or something | 00:20 |
| clarkb | bknudson: so it installs that version you see then downgrades it to 1.4.0 for juno | 00:21 |
| fungi | try again | 00:21 |
| fungi | it has a stable branch now | 00:21 |
| clarkb | fungi: oh that happened? I completely missed it | 00:21 |
| fungi | and the branch tip is also what 1.4.0 tagged | 00:21 |
| bknudson | ok, will recheck | 00:21 |
| bknudson | thanks | 00:21 |
| fungi | so installing stable/juno of keystonemiddleware gets you a commit that pbr regognizes as 1.4.0 | 00:21 |
| openstackgerrit | Stephanie Miller proposed openstack-infra/system-config: Add infra setup for zanata https://review.openstack.org/169074 | 00:22 |
| fungi | bknudson: i meant try again on the answer (to clark). i'm not sure rechecking is going to help yet | 00:22 |
| clarkb | cinerama: +2 | 00:22 |
| * bknudson is also wondering if maybe there isn't a release of requests that's breaking keystoneclient... | 00:22 | |
| clarkb | fungi: ya, so we need to make that job only run against stable/juno now | 00:22 |
| fungi | https://git.openstack.org/cgit/openstack/keystonemiddleware/log/?h=stable/juno | 00:22 |
| fungi | branch tip == 1.4.0 | 00:23 |
| bknudson | ok, won't recheck | 00:23 |
| bknudson | g-r in stable/juno has < 1.4.0 | 00:23 |
| *** yamada-h has joined #openstack-infra | 00:23 | |
| fungi | clarkb: i think what needs to happen is dropping the gate-tempest-dsvm-neutron-src-keystonemiddleware-juno job completely | 00:23 |
| clarkb | fungi: oh right, the regular src job will do the right thing | 00:24 |
| fungi | bknudson: yep, i think morganfainberg said something about a change to openstack/requirements stable branches to increase that | 00:24 |
| fungi | not sure if that change is already in flight though | 00:24 |
| clarkb | what I don't get is how we merged code that made the job fail in the first place | 00:24 |
| fungi | clarkb: because changes to cap reqs in openstack/requirements stable branches don't run client src jobs | 00:25 |
| morganfainberg | Crap. Sorry should have been more careful | 00:25 |
| morganfainberg | Thought that was <= 1.4.0 | 00:26 |
| bknudson | https://github.com/openstack/requirements/commit/9ed0e60c15e177c398d4c483eb12cfa917426d96 | 00:26 |
| morganfainberg | Both ttx and I missed on that. | 00:26 |
| bknudson | that was a long time ago. | 00:26 |
| bknudson | Feb 10 | 00:26 |
| morganfainberg | When we created the middleware branch today. | 00:26 |
| clarkb | fungi: OH! gah | 00:26 |
| morganfainberg | Should have been 1.3.x | 00:26 |
| fungi | bknudson: the commit which is tagged/branched is from months ago, but the branch itself was created today | 00:26 |
| morganfainberg | Can we purge the branch and retry? | 00:27 |
| fungi | morganfainberg: oh? branches from the wrong tag? yeah we can retry | 00:27 |
| morganfainberg | Yep. | 00:27 |
| fungi | just a sec and i'll get it | 00:27 |
| bknudson | oh, you wanted to branch 1.3.1 rather than 1.4.0? | 00:27 |
| morganfainberg | Yeah misread the g-r for Juno. | 00:27 |
| *** yamada-h has quit IRC | 00:28 | |
| *** jamielennox|away is now known as jamielennox | 00:28 | |
| morganfainberg | And we need to recheck icehouse branch and probably keystone clients as well (stable branches) | 00:28 |
| morganfainberg | But ksc looks to be passing. | 00:28 |
| fungi | grr, there are open changes proposed against that branch now which i need to abandon too. they'll need to be reproposed to the new branch | 00:28 |
| fungi | morganfainberg: you want to take care of that with your ptl hammer? | 00:29 |
| morganfainberg | Abandoning? Yeah give me a moment. | 00:29 |
| bknudson | probably just the requirements update. | 00:29 |
| fungi | just query gerrit for project:openstack/keystonemiddleware branch:stable/juno status:open and bring the hammer down | 00:29 |
| *** bswartz has quit IRC | 00:30 | |
| fungi | and yep, it is indeed just a failing reqs update | 00:30 |
| fungi | bknudson: wins | 00:30 |
| *** SumitNaiksatam has joined #openstack-infra | 00:30 | |
| *** salv-orlando has quit IRC | 00:31 | |
| fungi | okay, old branch deleted now | 00:31 |
| morganfainberg | Ok should be abandoned. | 00:31 |
| *** bswartz has joined #openstack-infra | 00:31 | |
| fungi | 1.3.1 tagged 2014-12-22 is what you want branched as stable/juno, right? | 00:31 |
| morganfainberg | Checking. | 00:31 |
| morganfainberg | 1.3.1 is the one before 1.4.0 right? | 00:32 |
| fungi | yep | 00:32 |
| morganfainberg | We don't have a 1.3.2 hiding somewhere. | 00:32 |
| morganfainberg | K then 1.3.1 | 00:32 |
| fungi | not that i've found | 00:32 |
| morganfainberg | Is correct. | 00:33 |
| fungi | done | 00:33 |
| morganfainberg | Keystone client looks ok as it is passing checks | 00:33 |
| morganfainberg | And gate. | 00:33 |
| fungi | so for kclient you branched icehouse from 0.11.2 and juno from 1.1.0 | 00:33 |
| morganfainberg | Yeah ksc was <= 1.1 | 00:34 |
| *** yamamoto has quit IRC | 00:34 | |
| armax | clarkb: so here: https://review.openstack.org/#/c/172227/2/functions.sh the setup for flat_if_name should be optional for neutron, correct? | 00:34 |
| morganfainberg | And <=0.11.2 | 00:34 |
| *** teran has joined #openstack-infra | 00:34 | |
| morganfainberg | We're good. Thanks fungi | 00:34 |
| fungi | morganfainberg: hopefully capped at <1.0.0 on icehouse and <1.2.0 on juno but double-checking that assumption now | 00:34 |
| clarkb | armax: yup, need to make that function more like the ovs_gre_bridge function so that it only sets up a single bridge per call | 00:34 |
| armax | clarkb: agreed | 00:35 |
| clarkb | armax: and optionally sets IP addrs and optionally sets ovs bridge | 00:35 |
| morganfainberg | We need to fix the caps | 00:35 |
| armax | I’d say let’s leave the bridge setup to Devstack | 00:35 |
| clarkb | armax: that works too | 00:35 |
| morganfainberg | We can't backport as is. | 00:35 |
| morganfainberg | But they aren't broken at the moment. | 00:35 |
| armax | clarkb: I am going to try that and see how it works | 00:35 |
| clarkb | armax: we can use PUBLIC_INTERFACE in devstack as the port to add to br-ex | 00:35 |
| fungi | morganfainberg: you're right, it's python-keystoneclient>=0.7.0,<=0.11.2 and python-keystoneclient>=0.10.0,<=1.1.0 | 00:35 |
| bknudson | morganfainberg: can you restore the requirements update? | 00:35 |
| clarkb | cool | 00:35 |
| armax | clarkb: indeed! | 00:35 |
| bknudson | or get it reproposed? | 00:36 |
| morganfainberg | bknudson: will need repropose I think | 00:36 |
| armax | clarkb: so that changes to d-g-wrap.sh become pretty minimal | 00:36 |
| clarkb | armax: ++ | 00:36 |
| fungi | anyway, i'm calling it a night | 00:36 |
| clarkb | jhesketh: were you planning to write those changes? I am happy to now that the DVR stuff is starting to gel | 00:36 |
| fungi | catch you all tomorrow | 00:36 |
| *** SumitNaiksatam has quit IRC | 00:36 | |
| clarkb | jhesketh: in either case let me know so we don't duplicate work :) | 00:36 |
| armax | clarkb: I am already trying that…but I’ll leave the cleanup for functions.sh as follow-up step for now | 00:36 |
| clarkb | fungi: good night | 00:37 |
| *** SumitNaiksatam has joined #openstack-infra | 00:37 | |
| jhesketh | clarkb: yep, I was going to have a stab but happy either way | 00:39 |
| *** yamamoto has joined #openstack-infra | 00:39 | |
| *** alex_xu has quit IRC | 00:42 | |
| clarkb | jhesketh: go for it then, I will try to finish up dvr testing things | 00:43 |
| jhesketh | cool | 00:43 |
| openstackgerrit | Stephanie Miller proposed openstack-infra/puppet-zanata: Remove dependency on mysql database https://review.openstack.org/172247 | 00:43 |
| *** bradjones has quit IRC | 00:45 | |
| *** alex_xu has joined #openstack-infra | 00:45 | |
| openstackgerrit | Merged openstack-infra/system-config: Increase redis maxmemory to 512m on ask.o.o https://review.openstack.org/172105 | 00:48 |
| *** bradjones has joined #openstack-infra | 00:50 | |
| *** tqtran has quit IRC | 00:50 | |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/devstack-gate: Configure sub-nodes for Neutron DVR aiopcpu https://review.openstack.org/158525 | 00:50 |
| *** tiswanso has joined #openstack-infra | 00:50 | |
| armax | clarkb: current run is only looking half-good https://jenkins05.openstack.org/job/check-tempest-dsvm-neutron-aiopcpu-full/32/console | 00:50 |
| armax | clarkb: as far as neutron is concerne | 00:51 |
| armax | e | 00:51 |
| armax | d | 00:51 |
| *** jerryz has joined #openstack-infra | 00:51 | |
| openstackgerrit | Eric Brown proposed openstack/requirements: Add Bandit as a global test requirement https://review.openstack.org/167126 | 00:51 |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/devstack-gate: ***DO NOT MERGE*** Flip DVR bit to collect experimental run datapoints https://review.openstack.org/171809 | 00:51 |
| *** wenlock has joined #openstack-infra | 00:53 | |
| *** tiswanso has quit IRC | 00:53 | |
| *** shashankhegde has quit IRC | 00:53 | |
| *** tiswanso has joined #openstack-infra | 00:53 | |
| ianw | oh yeah, bringing up a vpn on a vm exposes it to the internet ... /me starts rebuilding images | 00:55 |
| ianw | so far it seems this rootkit doesn't have a name | 00:55 |
| *** shashankhegde has joined #openstack-infra | 00:56 | |
| bknudson | stable/juno requirements has requests<=2.2.1, but this is causing keystoneclient tests to not work. | 00:57 |
| *** shashankhegde has quit IRC | 00:57 | |
| bknudson | that's not a recent change either... https://github.com/openstack/requirements/commit/75bdf54e31f1ac4ea9e54fbad1352ec12f55626f | 00:58 |
| *** yamamoto has quit IRC | 00:58 | |
| fungi | bknudson: probably we need to do a reqs sync on both new kclient stable branches | 00:59 |
| bknudson | fungi: that's the job that's failing. https://review.openstack.org/#/c/172220/ | 00:59 |
| bknudson | also, I thought you were calling it a night. | 00:59 |
| fungi | i fail at life | 01:00 |
| fungi | bknudson: yeah, then i guess kclient 1.1.x uses newer requests features not available on 2.2.1? | 01:00 |
| bknudson | I'll try different versions of requests and see if anything works. | 01:00 |
| bknudson | the failure doesn't look like using new features, looks like requests is just busted. | 01:01 |
| fungi | it's possible we branched from a too-recent version on that repo | 01:01 |
| *** koolhead17 has joined #openstack-infra | 01:01 | |
| fungi | or requests 2.2.1 might really have been a bad release | 01:01 |
| bknudson | how do I see what versions are available... | 01:01 |
| fungi | https://pypi.python.org/simple/requests/ | 01:01 |
| bknudson | I just tried 2.3.0 and tests passed. | 01:02 |
| *** wenlock has quit IRC | 01:03 | |
| *** pblaho has quit IRC | 01:03 | |
| *** ddieterly has joined #openstack-infra | 01:06 | |
| *** otter768 has joined #openstack-infra | 01:07 | |
| *** liusheng has joined #openstack-infra | 01:07 | |
| *** tsg_ has joined #openstack-infra | 01:08 | |
| *** tsg has quit IRC | 01:11 | |
| bknudson | I'll be the gerrit bot: https://review.openstack.org/#/c/172252/ | 01:12 |
| bknudson | Brant Knudson proposed openstack/requirements: requests skip 2.2.1 https://review.openstack.org/#/c/172252 | 01:12 |
| fungi | it's like the early days of the requirements list when we did that by hand everywhere | 01:12 |
| fungi | oh, i see what you're saying. openstackgerrit doesn't currently report on stable branch changes | 01:13 |
| fungi | at least not for the requirements repo. we could ask it to start | 01:13 |
| bknudson | there might be something else weird going on here... that solved the problem for me but still... | 01:15 |
| *** mtanino has quit IRC | 01:18 | |
| bknudson | oh, no, I was just confused... the code existed in stable/juno but it was moved | 01:18 |
| *** sarob has joined #openstack-infra | 01:20 | |
| armax | clarkb: it looks like we might not be able to delay the setup of br-ex on top of pub_if | 01:22 |
| armax | clarkb: from here https://jenkins03.openstack.org/job/check-tempest-dsvm-neutron-aiopcpu-full/35/console | 01:22 |
| armax | clarkb: I sense we don’t have a route to host | 01:22 |
| armax | clarkb: oh well, I might have looked at it wrong…never mind | 01:24 |
| *** yamada-h has joined #openstack-infra | 01:24 | |
| * fungi senses a disturbance in the source | 01:25 | |
| *** wenlock has joined #openstack-infra | 01:25 | |
| *** yamada-h has quit IRC | 01:29 | |
| *** asselin has joined #openstack-infra | 01:29 | |
| *** salv-orlando has joined #openstack-infra | 01:31 | |
| *** otter768 has quit IRC | 01:32 | |
| *** baoli has joined #openstack-infra | 01:33 | |
| *** yamamoto has joined #openstack-infra | 01:37 | |
| *** salv-orlando has quit IRC | 01:37 | |
| *** wenlock has quit IRC | 01:38 | |
| *** dboik_ has quit IRC | 01:39 | |
| *** otter768 has joined #openstack-infra | 01:45 | |
| *** asselin has quit IRC | 01:46 | |
| bknudson | I can't wip https://review.openstack.org/#/c/172252/ | 01:47 |
| *** ashleighfarnham has joined #openstack-infra | 01:53 | |
| *** jyuso1 has joined #openstack-infra | 01:54 | |
| *** skraynev has quit IRC | 01:55 | |
| *** skraynev has joined #openstack-infra | 01:55 | |
| *** lnr has joined #openstack-infra | 01:57 | |
| *** lnr has left #openstack-infra | 01:57 | |
| *** wenlock has joined #openstack-infra | 02:01 | |
| *** spzala has quit IRC | 02:03 | |
| *** bhunter71 has joined #openstack-infra | 02:03 | |
| *** Sukhdev has quit IRC | 02:05 | |
| *** dims_ has quit IRC | 02:07 | |
| *** shashankhegde has joined #openstack-infra | 02:08 | |
| *** baoli has quit IRC | 02:12 | |
| *** baoli has joined #openstack-infra | 02:13 | |
| *** otter768 has quit IRC | 02:13 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: DO NOT MERGE testing overlay network options https://review.openstack.org/171714 | 02:14 |
| *** otter768 has joined #openstack-infra | 02:15 | |
| *** sarob has quit IRC | 02:15 | |
| clarkb | armax: ^ might as well give vxlan a go as well | 02:16 |
| armax | clarkb: I saw | 02:17 |
| armax | clarkb: can’t hurt can it? | 02:17 |
| clarkb | armax: nope, worst case it will still not work :) best case maybe it will do much better than gre + ovs | 02:17 |
| clarkb | I have a hunch that the problem is more fundamental to ovs and not the tunnel technology though | 02:18 |
| armax | clarkb: well at least in one case we’ve proven that the ovs-led setup works the way it should | 02:19 |
| clarkb | armax: I was thinking about that and was wondering if it is possible that the network would force us to essentially run an all in one install | 02:20 |
| clarkb | armax: basically the controller would notice the compute node isn't working and just shcedule everything on itself | 02:20 |
| clarkb | but that is something I haven't even tried to look into yet | 02:21 |
| armax | clarkb: when if nova-compute is unable to report to the scheduler the vms would always land on the controller | 02:21 |
| armax | clarkb: however in my local setup I did indeed succeeded running tests reliably, having forced VM’s only on the subnode | 02:22 |
| *** rmcall has quit IRC | 02:22 | |
| armax | clarkb: so I can tell for certain that using ovs with setup as done 158525/24 that Neutron is setup correctly | 02:23 |
| armax | clarkb: I did hit the MTU issue on the private network | 02:23 |
| armax | clarkb: but I was able to ssh fine | 02:23 |
| armax | later I am going to give the latest instructions a try | 02:23 |
| clarkb | gotcha | 02:23 |
| openstackgerrit | Julia Kreger proposed openstack-infra/shade: WIP: Add ironic maintenance state pass-through https://review.openstack.org/172283 | 02:24 |
| openstackgerrit | Julia Kreger proposed openstack-infra/shade: WIP: Add Ironic machine power state pass-through https://review.openstack.org/172284 | 02:24 |
| armax | so I am not sure where this leaves us | 02:24 |
| *** rmcall has joined #openstack-infra | 02:24 | |
| armax | clarkb: it seems that using simply ovs makes neutron happy and screw up n-net | 02:24 |
| armax | clarkb: the other way around makes n-net but makes neutron grumpy :) | 02:24 |
| armax | I mean n-net happy | 02:25 |
| clarkb | ya its possible we may have to call it quits on making them the same at least for now | 02:25 |
| *** mmedvede has quit IRC | 02:25 | |
| armax | clarkb: I am gonna give this another whirl, but if we give in, we can’t be blamed for lack of trying | 02:26 |
| *** harlowja is now known as harlowja_away | 02:26 | |
| clarkb | yup and thats mostly what I wanted to make sure we did, beacuse in theory this should just work (tm) | 02:26 |
| armax | clarkb: :) indeed | 02:26 |
| *** sdake has joined #openstack-infra | 02:26 | |
| clarkb | and it almost does grr | 02:26 |
| armax | clarkb: one thing is for certain though, we needed cooperation from devstack :) | 02:26 |
| armax | clarkb: and that’s what we learned through this trial and error effort | 02:27 |
| armax | clarkb: yeah, it’s annoying, I must admit, we’re so close and yet it escapes us | 02:27 |
| tchaypo | jeblair: so the remaining issue I have with Gertty is that it still takes many hours to sync. When I tail the logs, I see a lot of things like this: | 02:28 |
| *** jcoufal has quit IRC | 02:28 | |
| tchaypo | https://www.irccloud.com/pastebin/cvcHsCny | 02:28 |
| tchaypo | I really don’t know what could be done about that aside from going async and doing requests in parallel | 02:29 |
| *** yamahata has quit IRC | 02:29 | |
| *** fedexo has joined #openstack-infra | 02:29 | |
| *** erlon has quit IRC | 02:31 | |
| *** koolhead17 has quit IRC | 02:33 | |
| *** asselin has joined #openstack-infra | 02:33 | |
| clarkb | tchaypo: possibly lazy load them | 02:33 |
| clarkb | tchaypo: you don't need the review comments until you view the change right? | 02:34 |
| tchaypo | sure, but i might be on a plane by then | 02:34 |
| *** dkranz has joined #openstack-infra | 02:35 | |
| *** Longgeek has joined #openstack-infra | 02:36 | |
| *** ashleighfarnham has quit IRC | 02:38 | |
| lifeless | tchaypo: I had a patch set that did a bunch of bulk stuff | 02:39 |
| lifeless | tchaypo: its probably in the stuff that bit rotted | 02:39 |
| tchaypo | that was bulk git fetches, right? | 02:39 |
| lifeless | tchaypo: teaching gerrit about better bulk stuff would be good too | 02:39 |
| lifeless | tchaypo: no, I batched up all classes of things | 02:39 |
| tchaypo | ah, okay. | 02:39 |
| *** achanda has joined #openstack-infra | 02:40 | |
| lifeless | tchaypo: lots of separate patches | 02:40 |
| tchaypo | bulk fetching of the comments seems like it would make a big difference | 02:40 |
| *** asselin has quit IRC | 02:43 | |
| *** asselin has joined #openstack-infra | 02:43 | |
| *** achanda has quit IRC | 02:45 | |
| *** ashleighfarnham has joined #openstack-infra | 02:48 | |
| *** achanda has joined #openstack-infra | 02:50 | |
| *** xyang1 has quit IRC | 02:54 | |
| *** sdake_ has joined #openstack-infra | 02:54 | |
| *** sdake has quit IRC | 02:57 | |
| *** woodster_ has quit IRC | 03:00 | |
| *** tonytan4ever has quit IRC | 03:00 | |
| *** tonytan4ever has joined #openstack-infra | 03:01 | |
| *** zz_dimtruck is now known as dimtruck | 03:01 | |
| tchaypo | lifeless: 105037 definitely conflicts with 171457 but I haven’t looked at how big the conflict is | 03:02 |
| lifeless | tchaypo: feel free to steal ideas or code | 03:03 |
| lifeless | tchaypo: I have no cycles to hack on this atm | 03:03 |
| *** trey has quit IRC | 03:06 | |
| openstackgerrit | David Pursehouse proposed openstack-infra/lodgeit: Update README https://review.openstack.org/172293 | 03:08 |
| *** dims_ has joined #openstack-infra | 03:08 | |
| *** trey has joined #openstack-infra | 03:10 | |
| *** mmedvede has joined #openstack-infra | 03:13 | |
| *** dims_ has quit IRC | 03:13 | |
| *** yamamoto has quit IRC | 03:19 | |
| *** gyee has quit IRC | 03:19 | |
| *** achanda has quit IRC | 03:20 | |
| *** tonytan4ever has quit IRC | 03:23 | |
| *** yamada-h has joined #openstack-infra | 03:25 | |
| *** yamada-h has quit IRC | 03:30 | |
| *** tiswanso has quit IRC | 03:31 | |
| *** asettle is now known as asettle-afk | 03:32 | |
| *** Swami has quit IRC | 03:33 | |
| *** ddieterly has quit IRC | 03:33 | |
| *** markvoelker has quit IRC | 03:35 | |
| *** ashleighfarnham has quit IRC | 03:35 | |
| *** bhunter71 has quit IRC | 03:36 | |
| *** fedexo has quit IRC | 03:37 | |
| *** bradjones has quit IRC | 03:40 | |
| *** shashankhegde has quit IRC | 03:42 | |
| *** bradjones has joined #openstack-infra | 03:43 | |
| *** bradjones has quit IRC | 03:43 | |
| *** bradjones has joined #openstack-infra | 03:43 | |
| *** salv-orlando has joined #openstack-infra | 03:46 | |
| *** teran has quit IRC | 03:48 | |
| *** dimtruck is now known as zz_dimtruck | 03:49 | |
| *** salv-orlando has quit IRC | 03:53 | |
| mordred | clarkb: I miss anything good today? | 03:54 |
| greghaynes | mordred: you should ask how he feels about ovs | 03:55 |
| clarkb | mordred: still having trouble with l2 overlays | 03:55 |
| clarkb | mordred: I am -1 on using ovs for infra cloud | 03:55 |
| clarkb | mordred: it has decided to not pass my arp packets | 03:55 |
| clarkb | mordred: because the one thing an l2 network must do to be useful is have working arp | 03:55 |
| greghaynes | or static arp tables in all the places ;) | 03:56 |
| clarkb | mordred: but quite literally I will ping from controller to cpmpute host, arp who has for the address is set out the correct interface based on my routes, it then goes into ovs where ovs drops it with a flow | 03:57 |
| *** ayoung has quit IRC | 03:57 | |
| clarkb | I have no idea why it has decided that is appropriate | 03:57 |
| clarkb | it SHOULD forward the packet to all nodes on the same l2 broadcast domain, then the interface with that address can respond with a helpful "I Have this address please throw wet cats this way" | 03:58 |
| mordred | clarkb: that's fantastic | 03:58 |
| mordred | clarkb: so - I believe I'm also -1 on ovs for infra cloud | 03:58 |
| mordred | clarkb: last I checked, I think markmcclain said we don't need it and he also thinks we shouldnt' use it | 03:59 |
| lifeless | clarkb: how are you configuring your ovs db? neutron? by hand? ryu? | 03:59 |
| clarkb | lifeless: by hand | 04:00 |
| clarkb | mordred: we do need it | 04:00 |
| clarkb | well more specifically we need something like it | 04:00 |
| clarkb | you cannot test openstack on opesntack without some sort of l2 overlay | 04:00 |
| clarkb | lifeless: https://review.openstack.org/#/c/171714/ is where most of the frustration has gone | 04:00 |
| clarkb | the reason we need an l2 overlay is we have control of neither the l2 or l3 networks that our VMs run in | 04:01 |
| clarkb | even in hpcloud where we can manage networks we can't manage a network per test env. Can you imagine how slow that would make nodepool? | 04:01 |
| clarkb | it should be noted that gretap interfaces with a linux bridge work excellently | 04:02 |
| mordred | clarkb: I don't think so - I think the suggestion is to use "provider network" with linux bridge - which will do direct VM-to-network routing | 04:02 |
| clarkb | mordred: that won't work | 04:02 |
| mordred | clarkb: no? | 04:02 |
| mordred | why not? | 04:03 |
| clarkb | mordred: no because tempest has to be able to ssh to the VMs | 04:03 |
| *** yamada-h has joined #openstack-infra | 04:03 | |
| mordred | clarkb: I'm talking infra cloud | 04:03 |
| mordred | not what you're doing | 04:03 |
| clarkb | oh | 04:03 |
| mordred | sorry | 04:03 |
| mordred | I was confusing | 04:03 |
| mordred | I was trying to agree to the -1 on ovs for infra cloud | 04:03 |
| clarkb | gotcha | 04:03 |
| clarkb | but ya long story short ovs has basically no docs, they mostly come in the form of random blog post by poor soul like me that unlike me actually managed to make a thing work | 04:04 |
| clarkb | it refuses to arp properly | 04:04 |
| clarkb | and you can't tcpdump its interfaces | 04:04 |
| clarkb | https://github.com/openvswitch/ovs/blob/master/README.md is literally where you get pointed to for documentation | 04:06 |
| *** markvoelker has joined #openstack-infra | 04:06 | |
| *** stevemar has joined #openstack-infra | 04:07 | |
| lifeless | clarkb: so arp is generally fine. can you pastebin your flow db for me? | 04:07 |
| clarkb | lifeless: I deleted the nodes I had to free up the VMs | 04:07 |
| *** otter768 has quit IRC | 04:07 | |
| mordred | clarkb: btw ... russellb has been hacking on ovs recently - maybe you should also rant at hi | 04:08 |
| clarkb | lifeless: but I can find all the pastes I made for armax earlier today | 04:08 |
| mordred | him | 04:08 |
| clarkb | mordred: yes tried that but caught russellb at his EOD | 04:08 |
| lifeless | clarkb: IIRC the default handler is flood forwarding with STP | 04:08 |
| lifeless | clarkb: so I'm inclined to suspect something in the devstack code being bong | 04:08 |
| *** dtantsur|afk has quit IRC | 04:09 | |
| lifeless | clarkb: sure, please | 04:09 |
| *** yamamoto has joined #openstack-infra | 04:10 | |
| lifeless | clarkb: I presume you need OVS to test flat mode stuff; since neutron w/ gre doesn't need l2 connectivity | 04:10 |
| *** dtantsur has joined #openstack-infra | 04:10 | |
| lifeless | (and in fact you'll need to be careful to partition your test stuff off from what neutron plays with) | 04:10 |
| clarkb | neutron doesnt need l2 | 04:11 |
| clarkb | but to test it we do | 04:11 |
| clarkb | because tempest must be able to ssh into VMs | 04:11 |
| *** asettle-afk is now known as asettle | 04:11 | |
| lifeless | thats implies l3 | 04:12 |
| lifeless | not l2 | 04:12 |
| clarkb | anyways almost have pastes togwther and now local network is derping | 04:12 |
| clarkb | lifeless yes l2 is how we l3 | 04:12 |
| clarkb | since you put l3 on l2 | 04:12 |
| lifeless | but we have public ips for all the vms right ? so we have l3 anyway | 04:12 |
| *** markvoelker has quit IRC | 04:12 | |
| clarkb | except thats one IP | 04:12 |
| lifeless | wat | 04:12 |
| clarkb | we need a bunch for the inner VMs | 04:12 |
| clarkb | and those need to be routable | 04:13 |
| lifeless | I'm missing some context here | 04:13 |
| lifeless | 'inner VMs'? We're running nested KVM ? | 04:13 |
| clarkb | I am running a multinode openstack on top of rax and hpcloud | 04:13 |
| clarkb | its just devstack gate with 2 nodes. a controller and a compute host | 04:13 |
| lifeless | tempest running on the controller right? | 04:14 |
| clarkb | yes | 04:14 |
| clarkb | so we need ssh to VMs on compute node | 04:14 |
| lifeless | so your public ips there will be from whatever range you choose | 04:14 |
| clarkb | yes | 04:14 |
| lifeless | the actual ssh to the 'vms' on the compute node will be carried by your neutron provisioned overlay network | 04:14 |
| clarkb | no | 04:14 |
| lifeless | well, not for the ones where you're testing flat networking | 04:15 |
| clarkb | its not because neutron doesnt make an overlay for br-ex | 04:15 |
| lifeless | which I caveated out before | 04:15 |
| clarkb | we arent flat networking aiui | 04:15 |
| clarkb | its default ovs ml2 | 04:15 |
| lifeless | so what node is your network node | 04:15 |
| clarkb | but that does not overlay br-ex | 04:15 |
| lifeless | controller or compute | 04:15 |
| clarkb | lifeless both | 04:15 |
| clarkb | its dvr | 04:15 |
| lifeless | ok | 04:15 |
| clarkb | the non dvr controller only network node works fine | 04:16 |
| clarkb | because that does use overlay | 04:16 |
| lifeless | then yes, you need a second public ip on the compute node, or you need your own vpn-thing | 04:16 |
| clarkb | with br-ex on controller adjacent to tempest | 04:16 |
| lifeless | thanks for indulging my lack of knowledge | 04:16 |
| clarkb | ok desktop has network again pastes incoming shortly | 04:17 |
| clarkb | lifeless: ok for these pastes they are in numeric order. I took this info while pinging 172.24.4.1 to 172.24.4.2. That is controller to compute host interface on floating ip network range | 04:18 |
| clarkb | lifeless: some pastes have info from both nodes the others from a single node. The pastes from just a single node were all made on the controller | 04:19 |
| lifeless | to compute host | 04:19 |
| lifeless | or to a vm ? | 04:19 |
| clarkb | http://paste.openstack.org/show/201479/ http://paste.openstack.org/show/201480/ http://paste.openstack.org/show/201493/ http://paste.openstack.org/show/201496/ http://paste.openstack.org/show/201505/ http://paste.openstack.org/show/201518/ | 04:19 |
| clarkb | lifeless: this was controller to compute host, not to any VMs hosted by the compute host | 04:20 |
| lifeless | bah, I need to go pick up C from kindy | 04:20 |
| lifeless | one thing though - having hosts on the floating ip range is often problematic IME. The agents + vms should be the only things there. | 04:20 |
| lifeless | so I'd suggest pinging the DVR agent's port on the range instead, for instance. | 04:21 |
| lifeless | since thats a legitimate, modelled address | 04:21 |
| clarkb | lifeless: yes so what we do is 172.24.4.1/23 on controller 172.24.4.2/23 on compute host then floating is restricted to 172.24.5.0/24 | 04:21 |
| clarkb | because tempest isn't running on a nested VM so it needs to have a route from controller to compute host hosted VMs | 04:21 |
| lifeless | uhm, yeah thats likely the problem. Need to check routing/arp interaction but you can't arp across subnet widths. | 04:22 |
| clarkb | the easy way to do that is to just have l3 addrs on the l2 overlay that jive | 04:22 |
| clarkb | its not across a subnet width | 04:22 |
| lifeless | you have to make sure you're routing across subnet widths | 04:22 |
| clarkb | 172.24.4.0/23 encloses 172.24.5.0/24 | 04:22 |
| clarkb | and this works perfectly fine with linux bridge | 04:22 |
| clarkb | (and every other switch I have ever used) | 04:22 |
| lifeless | under special circumstances it will | 04:22 |
| lifeless | sorry, hate to be vague and run, but I must. | 04:23 |
| clarkb | regardless | 04:23 |
| lifeless | enclosing is fine | 04:23 |
| clarkb | I am pnging from /23 to /23 network | 04:23 |
| clarkb | and that is what arp is failing for | 04:23 |
| lifeless | will try to look later tonight, otherwise will see about syncing with you and helping debug on your monday | 04:23 |
| clarkb | ok | 04:23 |
| *** cody-somerville has joined #openstack-infra | 04:23 | |
| *** hrubi has quit IRC | 04:25 | |
| *** melwitt has quit IRC | 04:27 | |
| *** sdake_ has quit IRC | 04:27 | |
| *** baoli has quit IRC | 04:27 | |
| *** crc32 has joined #openstack-infra | 04:27 | |
| *** sarob has joined #openstack-infra | 04:27 | |
| *** hrubi has joined #openstack-infra | 04:28 | |
| armax | clarkb: ping | 04:29 |
| clarkb | armax: pong | 04:30 |
| armax | quick question on the brctl gre setup… | 04:30 |
| armax | clarkb: when I try this locally I can seem to ping 172.24.4.x fine either way | 04:30 |
| clarkb | armax: yup that should work great | 04:30 |
| armax | clarkb: as soon as I create br-ex and add it to pub_if | 04:30 |
| armax | then pub_if looses the ip | 04:31 |
| clarkb | oh fun | 04:31 |
| armax | and I loose connectivity | 04:31 |
| clarkb | so swami's comment about moving the ip to the br-ex is probably necessary | 04:31 |
| *** sarob has quit IRC | 04:32 | |
| armax | clarkb: um I could try that | 04:32 |
| clarkb | armax: go for it | 04:32 |
| *** baoli has joined #openstack-infra | 04:32 | |
| armax | clarkb: I’ll look into it | 04:33 |
| *** ddieterly has joined #openstack-infra | 04:34 | |
| *** mrmartin has joined #openstack-infra | 04:35 | |
| armax | clarkb: I wonder if NetworkManager fiddles with my stack though | 04:36 |
| *** achanda has joined #openstack-infra | 04:36 | |
| armax | clarkb: I see some weird messages in syslog | 04:36 |
| *** cody-somerville has quit IRC | 04:40 | |
| *** Alexandra_ has joined #openstack-infra | 04:40 | |
| *** unicell has joined #openstack-infra | 04:41 | |
| *** ddieterly has quit IRC | 04:41 | |
| tchaypo | clarkb: could this be kernel arp filtering? | 04:42 |
| clarkb | tchaypo: maybe, but http://paste.openstack.org/show/201479/ shows ovs do the drop so I think its ovs | 04:43 |
| tchaypo | That seems fairly conclusive | 04:49 |
| *** sputnik13 has joined #openstack-infra | 04:51 | |
| *** pal has joined #openstack-infra | 04:55 | |
| *** sputnik13 has quit IRC | 04:56 | |
| tchaypo | In other news, it’d be nice if some other people could look at my comments on 152290 and tell me if I’m being silly. | 04:56 |
| *** woodster_ has joined #openstack-infra | 04:57 | |
| *** yamada-h has joined #openstack-infra | 05:03 | |
| *** yamada-h has quit IRC | 05:09 | |
| *** bradjones has quit IRC | 05:12 | |
| *** sdake has joined #openstack-infra | 05:14 | |
| *** sdake has quit IRC | 05:14 | |
| *** sdake has joined #openstack-infra | 05:14 | |
| *** fedexo has joined #openstack-infra | 05:19 | |
| *** bradjones has joined #openstack-infra | 05:21 | |
| *** Longgeek has quit IRC | 05:23 | |
| *** Longgeek has joined #openstack-infra | 05:24 | |
| *** sdake_ has joined #openstack-infra | 05:25 | |
| *** bradjones has quit IRC | 05:26 | |
| *** sdake has quit IRC | 05:29 | |
| *** tsg_ has quit IRC | 05:29 | |
| *** bradjones has joined #openstack-infra | 05:30 | |
| *** stevemar has quit IRC | 05:32 | |
| *** crc32 has quit IRC | 05:33 | |
| *** shashankhegde has joined #openstack-infra | 05:36 | |
| *** afazekas has joined #openstack-infra | 05:36 | |
| *** ddieterly has joined #openstack-infra | 05:38 | |
| *** Sukhdev has joined #openstack-infra | 05:38 | |
| *** BharatK has quit IRC | 05:38 | |
| *** ildikov_afk has quit IRC | 05:41 | |
| *** yamada-h has joined #openstack-infra | 05:41 | |
| *** ddieterly has quit IRC | 05:42 | |
| *** baoli has quit IRC | 05:44 | |
| *** achanda has quit IRC | 05:51 | |
| *** achanda has joined #openstack-infra | 05:52 | |
| *** sdake_ has quit IRC | 05:55 | |
| *** BharatK has joined #openstack-infra | 05:57 | |
| *** YorikSar has quit IRC | 05:57 | |
| *** jcoufal has joined #openstack-infra | 05:57 | |
| *** fedexo has quit IRC | 05:58 | |
| *** dims_ has joined #openstack-infra | 06:00 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/171473 | 06:00 |
| *** dims_ has quit IRC | 06:08 | |
| *** otter768 has joined #openstack-infra | 06:08 | |
| *** markvoelker has joined #openstack-infra | 06:09 | |
| *** bradjones has quit IRC | 06:13 | |
| *** jyuso1 has quit IRC | 06:13 | |
| *** otter768 has quit IRC | 06:13 | |
| *** markvoelker has quit IRC | 06:14 | |
| *** unicell has quit IRC | 06:17 | |
| *** unicell has joined #openstack-infra | 06:18 | |
| *** bradjones has joined #openstack-infra | 06:18 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-pip: Add the ability to set pip.conf and configure it https://review.openstack.org/168309 | 06:18 |
| *** bradjones has quit IRC | 06:18 | |
| *** bradjones has joined #openstack-infra | 06:18 | |
| *** mrunge has joined #openstack-infra | 06:26 | |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/devstack-gate: Add Neutron DVR aiopcpu support https://review.openstack.org/158525 | 06:27 |
| *** mrunge has quit IRC | 06:27 | |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/devstack-gate: ***DO NOT MERGE*** Flip DVR bit to collect experimental run datapoints https://review.openstack.org/171809 | 06:27 |
| *** Sukhdev has quit IRC | 06:35 | |
| *** ddieterly has joined #openstack-infra | 06:38 | |
| *** koolhead17 has joined #openstack-infra | 06:40 | |
| *** mrunge has joined #openstack-infra | 06:41 | |
| *** soren has joined #openstack-infra | 06:42 | |
| *** yamada-h has joined #openstack-infra | 06:42 | |
| *** ddieterly has quit IRC | 06:43 | |
| *** Krinkle|detached is now known as Krinkle | 06:43 | |
| *** mpaolino has joined #openstack-infra | 06:43 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/nodepool: Different queues for slow tasks. https://review.openstack.org/170788 | 06:44 |
| *** koolhead17 has quit IRC | 06:46 | |
| *** yamada-h has quit IRC | 06:47 | |
| *** Ala has joined #openstack-infra | 06:47 | |
| *** shashankhegde has quit IRC | 06:49 | |
| *** Ala has quit IRC | 06:50 | |
| *** Ala has joined #openstack-infra | 06:51 | |
| *** e0ne has joined #openstack-infra | 06:56 | |
| *** vlaza has joined #openstack-infra | 06:56 | |
| *** yamada-h has joined #openstack-infra | 06:58 | |
| openstackgerrit | Mehdi Abaakouk proposed openstack/requirements: Add pyngus to requirements https://review.openstack.org/172315 | 06:59 |
| *** ildikov_afk has joined #openstack-infra | 06:59 | |
| *** woodster_ has quit IRC | 07:00 | |
| *** e0ne has quit IRC | 07:00 | |
| *** yamahata has joined #openstack-infra | 07:02 | |
| *** Alexandra_ has quit IRC | 07:02 | |
| *** hashar has joined #openstack-infra | 07:03 | |
| *** yamada-h has quit IRC | 07:03 | |
| *** e0ne has joined #openstack-infra | 07:04 | |
| *** e0ne has quit IRC | 07:06 | |
| *** mpavone has joined #openstack-infra | 07:08 | |
| *** luqas has joined #openstack-infra | 07:08 | |
| *** markus_z has joined #openstack-infra | 07:09 | |
| *** markvoelker has joined #openstack-infra | 07:10 | |
| *** yamahata has quit IRC | 07:13 | |
| *** mattt has left #openstack-infra | 07:13 | |
| *** markvoelker has quit IRC | 07:15 | |
| *** armax has quit IRC | 07:16 | |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/171473 | 07:18 |
| *** shardy has quit IRC | 07:19 | |
| *** salv-orlando has joined #openstack-infra | 07:20 | |
| *** ibiris_away is now known as ibiris | 07:20 | |
| *** jamielennox is now known as jamielennox|away | 07:20 | |
| *** achanda has quit IRC | 07:24 | |
| *** andreykurilin_ has joined #openstack-infra | 07:25 | |
| *** achanda has joined #openstack-infra | 07:25 | |
| *** djangodog has joined #openstack-infra | 07:28 | |
| *** sarob has joined #openstack-infra | 07:28 | |
| *** achanda has quit IRC | 07:30 | |
| *** tnovacik has joined #openstack-infra | 07:31 | |
| *** mrda is now known as mrda-weekend | 07:31 | |
| *** sarob has quit IRC | 07:32 | |
| *** chillisantiago has joined #openstack-infra | 07:32 | |
| *** jlanoux has joined #openstack-infra | 07:34 | |
| *** kevinbenton has quit IRC | 07:35 | |
| djangodog | \dcc send chillisantiago | 07:36 |
| *** jlibosva has joined #openstack-infra | 07:36 | |
| djangodog | \dcc send chillisantiago c:\test\code.txt | 07:37 |
| chillisantiago | \dcc get djangodog | 07:37 |
| *** doude has joined #openstack-infra | 07:38 | |
| djangodog | \dcc send chillisantiago c:\test\code.txt | 07:38 |
| chillisantiago | \dcc get | 07:38 |
| *** ddieterly has joined #openstack-infra | 07:39 | |
| *** kevinbenton has joined #openstack-infra | 07:41 | |
| djangodog | \cd | 07:43 |
| *** teran has joined #openstack-infra | 07:43 | |
| *** ddieterly has quit IRC | 07:43 | |
| *** ihrachyshka has joined #openstack-infra | 07:44 | |
| *** e0ne has joined #openstack-infra | 07:44 | |
| *** mpaolino has quit IRC | 07:45 | |
| *** YorikSar has joined #openstack-infra | 07:45 | |
| djangodog | \dcc chat chillisantiago | 07:46 |
| *** _nadya_ has joined #openstack-infra | 07:46 | |
| *** e0ne is now known as e0ne_ | 07:46 | |
| chillisantiago | \dcc chat djangodog | 07:47 |
| djangodog | \dcc send chillisantiago c:\test\code.txt | 07:49 |
| chillisantiago | \dcc get | 07:49 |
| *** [HeOS] has quit IRC | 07:51 | |
| *** e0ne_ has quit IRC | 07:51 | |
| *** arxcruz has joined #openstack-infra | 07:55 | |
| djangodog | \msg chillisantiago hello | 07:57 |
| *** chillisantiago has quit IRC | 07:58 | |
| *** salv-orlando has quit IRC | 07:58 | |
| *** e0ne has joined #openstack-infra | 07:58 | |
| *** markus_z has quit IRC | 07:59 | |
| *** SumitNaiksatam has left #openstack-infra | 08:01 | |
| *** salv-orlando has joined #openstack-infra | 08:03 | |
| *** yamada-h has joined #openstack-infra | 08:03 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 08:05 | |
| *** chillisantiago has joined #openstack-infra | 08:05 | |
| chillisantiago | \msg djangodog hi there | 08:05 |
| chillisantiago | \dcc send djangodog c:\home\irc\code.txt | 08:06 |
| djangodog | \dcc get djangodog | 08:06 |
| *** skolekonov has joined #openstack-infra | 08:06 | |
| djangodog | \dcc get | 08:06 |
| *** otter768 has joined #openstack-infra | 08:09 | |
| *** e0ne has quit IRC | 08:09 | |
| *** derekh has joined #openstack-infra | 08:10 | |
| *** jistr has joined #openstack-infra | 08:11 | |
| djangodog | \dcc chat chillisantigo | 08:11 |
| *** markvoelker has joined #openstack-infra | 08:11 | |
| *** win2er has joined #openstack-infra | 08:11 | |
| *** salv-orlando has quit IRC | 08:11 | |
| *** _nadya_ has quit IRC | 08:11 | |
| *** salv-orlando has joined #openstack-infra | 08:12 | |
| *** djangodog has quit IRC | 08:12 | |
| *** chillisantiago has quit IRC | 08:12 | |
| *** e0ne has joined #openstack-infra | 08:13 | |
| *** e0ne has quit IRC | 08:13 | |
| *** otter768 has quit IRC | 08:14 | |
| *** ibiris is now known as ibiris_away | 08:15 | |
| *** markvoelker has quit IRC | 08:16 | |
| *** _nadya_ has joined #openstack-infra | 08:17 | |
| *** win2er has quit IRC | 08:21 | |
| *** btully has joined #openstack-infra | 08:21 | |
| openstackgerrit | Darren Birkett proposed openstack-infra/jeepyb: Handle projects not using stable/ branch names https://review.openstack.org/153057 | 08:22 |
| *** dizquierdo has joined #openstack-infra | 08:23 | |
| *** Krinkle is now known as Krinkle|detached | 08:30 | |
| *** Krinkle|detached is now known as Krinkle | 08:31 | |
| *** _nadya_ has quit IRC | 08:31 | |
| *** ibiris_away is now known as ibiris | 08:32 | |
| openstackgerrit | Andrey Pavlov proposed openstack/requirements: Add botocore to requirements https://review.openstack.org/172335 | 08:35 |
| *** mwhudson has quit IRC | 08:35 | |
| *** btully has quit IRC | 08:36 | |
| *** salv-orlando has quit IRC | 08:37 | |
| *** ssam2 has joined #openstack-infra | 08:37 | |
| *** ddieterly has joined #openstack-infra | 08:40 | |
| *** mwhudson has joined #openstack-infra | 08:41 | |
| *** mwhudson has joined #openstack-infra | 08:41 | |
| *** ajo has quit IRC | 08:43 | |
| openstackgerrit | Victor Stinner proposed openstack-infra/jenkins-job-builder: Fix Builder.update_job() on Python 3 https://review.openstack.org/172238 | 08:43 |
| *** ajo has joined #openstack-infra | 08:43 | |
| *** _nadya_ has joined #openstack-infra | 08:44 | |
| *** yamada-h has quit IRC | 08:44 | |
| *** ddieterly has quit IRC | 08:44 | |
| *** django__ has quit IRC | 08:45 | |
| *** yamada-h has joined #openstack-infra | 08:45 | |
| *** dhritishikhar_ has joined #openstack-infra | 08:46 | |
| *** yamada-h has quit IRC | 08:50 | |
| *** dhritishikhar_ has quit IRC | 08:50 | |
| *** dhritishikhar_ has joined #openstack-infra | 08:50 | |
| *** teran has quit IRC | 08:52 | |
| *** u_glide has joined #openstack-infra | 08:54 | |
| *** zz_avozza is now known as avozza | 08:55 | |
| u_glide | Hello guys, have anyone already faced issue with broken neutron-client? | 08:56 |
| u_glide | example build: https://jenkins03.openstack.org/job/gate-manila-python27/534/console | 08:57 |
| u_glide | change: https://review.openstack.org/#/c/171941/ | 08:57 |
| *** e0ne has joined #openstack-infra | 08:59 | |
| u_glide | root cause of problem - https://bugs.launchpad.net/python-neutronclient/+bug/1442367 | 09:02 |
| openstack | Launchpad bug 1442367 in python-neutronclient "2.4.0 release destroys grenade since stable/juno isn't capped properly" [Critical,Confirmed] - Assigned to Akihiro Motoki (amotoki) | 09:02 |
| *** achanda has joined #openstack-infra | 09:05 | |
| *** dizquierdo has quit IRC | 09:05 | |
| *** HeOS has joined #openstack-infra | 09:09 | |
| *** markvoelker has joined #openstack-infra | 09:12 | |
| *** achanda has quit IRC | 09:12 | |
| *** yamamoto_ has joined #openstack-infra | 09:13 | |
| *** yamamoto has quit IRC | 09:13 | |
| *** markus_z has joined #openstack-infra | 09:14 | |
| *** markvoelker has quit IRC | 09:17 | |
| *** pelix has joined #openstack-infra | 09:24 | |
| *** hichihara has quit IRC | 09:26 | |
| *** hashar is now known as hasharAway | 09:28 | |
| *** MaxV has joined #openstack-infra | 09:30 | |
| *** vdo has joined #openstack-infra | 09:36 | |
| *** salv-orlando has joined #openstack-infra | 09:38 | |
| *** ddieterly has joined #openstack-infra | 09:41 | |
| *** dhritishikhar_ has quit IRC | 09:41 | |
| *** claudiub has joined #openstack-infra | 09:41 | |
| *** salv-orlando has quit IRC | 09:42 | |
| *** andreykurilin_ has quit IRC | 09:42 | |
| *** isviridov has joined #openstack-infra | 09:42 | |
| *** dims_ has joined #openstack-infra | 09:43 | |
| *** salv-orlando has joined #openstack-infra | 09:43 | |
| *** ZZelle has quit IRC | 09:43 | |
| *** pblaho has joined #openstack-infra | 09:43 | |
| *** ZZelle has joined #openstack-infra | 09:43 | |
| *** pblaho has quit IRC | 09:44 | |
| *** pblaho has joined #openstack-infra | 09:44 | |
| *** ddieterly has quit IRC | 09:45 | |
| *** yamada-h has joined #openstack-infra | 09:45 | |
| *** yamada-h has quit IRC | 09:50 | |
| *** dhritishikhar_ has joined #openstack-infra | 09:51 | |
| *** BharatK has quit IRC | 09:51 | |
| *** amotoki has joined #openstack-infra | 09:51 | |
| *** aswadr has joined #openstack-infra | 09:54 | |
| *** dizquierdo has joined #openstack-infra | 09:55 | |
| *** dguitarbite has joined #openstack-infra | 09:55 | |
| *** cdent has joined #openstack-infra | 09:56 | |
| *** dims_ has quit IRC | 09:56 | |
| openstackgerrit | Alexander Evseev proposed openstack-infra/puppet-etherpad_lite: [MOSI-103] Set monospace font by default in Etherpad. https://review.openstack.org/172360 | 09:56 |
| *** bradjones has quit IRC | 09:57 | |
| *** dims_ has joined #openstack-infra | 09:57 | |
| *** yamamoto_ has quit IRC | 09:57 | |
| *** bradjones has joined #openstack-infra | 10:02 | |
| *** bradjones has quit IRC | 10:02 | |
| *** bradjones has joined #openstack-infra | 10:02 | |
| *** dtantsur is now known as dtantsur|brb | 10:02 | |
| *** _nadya_ has quit IRC | 10:04 | |
| *** teran has joined #openstack-infra | 10:06 | |
| *** sputnik13 has joined #openstack-infra | 10:06 | |
| *** dims_ has quit IRC | 10:07 | |
| *** avozza is now known as zz_avozza | 10:08 | |
| *** achanda has joined #openstack-infra | 10:09 | |
| *** teran_ has joined #openstack-infra | 10:09 | |
| *** otter768 has joined #openstack-infra | 10:10 | |
| *** teran has quit IRC | 10:11 | |
| *** wenlock has quit IRC | 10:12 | |
| *** markvoelker has joined #openstack-infra | 10:13 | |
| *** achanda has quit IRC | 10:14 | |
| *** wenlock has joined #openstack-infra | 10:14 | |
| *** otter768 has quit IRC | 10:15 | |
| *** dhritishikhar_ has quit IRC | 10:16 | |
| *** markvoelker has quit IRC | 10:17 | |
| *** amotoki has quit IRC | 10:18 | |
| *** dims_ has joined #openstack-infra | 10:19 | |
| *** tobiash has joined #openstack-infra | 10:23 | |
| *** amotoki has joined #openstack-infra | 10:25 | |
| *** wenlock has quit IRC | 10:26 | |
| *** yamamoto has joined #openstack-infra | 10:29 | |
| *** amotoki_ has joined #openstack-infra | 10:31 | |
| *** amotoki has quit IRC | 10:31 | |
| *** amotoki has joined #openstack-infra | 10:31 | |
| *** pc_m has joined #openstack-infra | 10:31 | |
| *** amotoki_ has quit IRC | 10:35 | |
| *** mtreinish has quit IRC | 10:40 | |
| *** kiseok7_ has quit IRC | 10:40 | |
| *** ddieterly has joined #openstack-infra | 10:41 | |
| *** mtreinish has joined #openstack-infra | 10:45 | |
| *** ddieterly has quit IRC | 10:46 | |
| *** jamespage has quit IRC | 10:46 | |
| *** gnuoy has quit IRC | 10:46 | |
| *** gnuoy has joined #openstack-infra | 10:46 | |
| *** jamespage has joined #openstack-infra | 10:46 | |
| *** pal has quit IRC | 10:47 | |
| *** yamamoto has quit IRC | 10:49 | |
| *** dizquierdo has quit IRC | 10:52 | |
| *** salv-orlando has quit IRC | 10:56 | |
| *** yamada-h has joined #openstack-infra | 10:59 | |
| *** marcusvrn has joined #openstack-infra | 11:03 | |
| *** yamada-h has quit IRC | 11:05 | |
| *** dizquierdo has joined #openstack-infra | 11:07 | |
| *** e0ne is now known as e0ne_ | 11:07 | |
| *** yamamoto has joined #openstack-infra | 11:08 | |
| *** ujuc has joined #openstack-infra | 11:08 | |
| *** _nadya_ has joined #openstack-infra | 11:10 | |
| *** salv-orlando has joined #openstack-infra | 11:10 | |
| *** markvoelker has joined #openstack-infra | 11:13 | |
| *** wenlock has joined #openstack-infra | 11:14 | |
| *** yamamoto has quit IRC | 11:16 | |
| *** markvoelker has quit IRC | 11:18 | |
| *** e0ne_ has quit IRC | 11:18 | |
| *** e0ne has joined #openstack-infra | 11:19 | |
| *** mmedvede_ has joined #openstack-infra | 11:19 | |
| *** e0ne has quit IRC | 11:26 | |
| *** wenlock has quit IRC | 11:26 | |
| *** claudiub_ has joined #openstack-infra | 11:26 | |
| *** e0ne has joined #openstack-infra | 11:27 | |
| *** claudiub has quit IRC | 11:29 | |
| *** jogo has quit IRC | 11:29 | |
| *** jogo has joined #openstack-infra | 11:30 | |
| *** MaxV has quit IRC | 11:34 | |
| *** yamamoto has joined #openstack-infra | 11:37 | |
| *** MaxV has joined #openstack-infra | 11:38 | |
| *** hasharAway is now known as hashar | 11:38 | |
| *** ociuhandu has joined #openstack-infra | 11:39 | |
| *** fifieldt has quit IRC | 11:42 | |
| *** mwagner_lap has quit IRC | 11:42 | |
| *** ddieterly has joined #openstack-infra | 11:42 | |
| *** ddieterly has quit IRC | 11:46 | |
| *** ldnunes has joined #openstack-infra | 11:47 | |
| *** dtantsur|brb is now known as dtantsur | 11:49 | |
| *** isviridov has quit IRC | 11:51 | |
| *** isviridov has joined #openstack-infra | 11:51 | |
| *** pal has joined #openstack-infra | 11:52 | |
| openstackgerrit | Julia Kreger proposed openstack-infra/shade: Add update_machine method and operator unit test substrate https://review.openstack.org/171165 | 11:53 |
| *** yamamoto has quit IRC | 11:53 | |
| *** [HeOS] has joined #openstack-infra | 11:58 | |
| *** HeOS has quit IRC | 11:58 | |
| *** amotoki_ has joined #openstack-infra | 11:58 | |
| *** marun has quit IRC | 11:59 | |
| *** amotoki has quit IRC | 12:01 | |
| *** pc_m_ has joined #openstack-infra | 12:03 | |
| *** amotoki_ has quit IRC | 12:03 | |
| *** amotoki has joined #openstack-infra | 12:04 | |
| *** dims_ has quit IRC | 12:04 | |
| *** dims_ has joined #openstack-infra | 12:04 | |
| *** dims_ has quit IRC | 12:05 | |
| *** amotoki has quit IRC | 12:08 | |
| *** markvoelker has joined #openstack-infra | 12:11 | |
| *** otter768 has joined #openstack-infra | 12:11 | |
| *** dizquierdo has quit IRC | 12:13 | |
| *** otter768 has quit IRC | 12:16 | |
| *** baoli has joined #openstack-infra | 12:17 | |
| *** wenlock has joined #openstack-infra | 12:18 | |
| *** dprince has joined #openstack-infra | 12:19 | |
| *** rfolco has joined #openstack-infra | 12:20 | |
| *** baoli_ has joined #openstack-infra | 12:21 | |
| *** sarob has joined #openstack-infra | 12:22 | |
| *** wenlock has quit IRC | 12:22 | |
| *** krtaylor has quit IRC | 12:23 | |
| *** zz_dimtruck is now known as dimtruck | 12:23 | |
| *** sputnik13 has quit IRC | 12:24 | |
| *** [HeOS] is now known as HeOS | 12:24 | |
| *** baoli has quit IRC | 12:25 | |
| *** dizquierdo has joined #openstack-infra | 12:25 | |
| *** sarob has quit IRC | 12:26 | |
| *** aysyd has joined #openstack-infra | 12:26 | |
| *** bhunter71 has joined #openstack-infra | 12:28 | |
| *** radez_g0n3 is now known as radez | 12:29 | |
| *** mrmartin has quit IRC | 12:31 | |
| *** sdake has joined #openstack-infra | 12:31 | |
| *** sdake_ has joined #openstack-infra | 12:32 | |
| *** bswartz has quit IRC | 12:33 | |
| *** bknudson has quit IRC | 12:33 | |
| *** jcoufal has quit IRC | 12:34 | |
| *** sdake has quit IRC | 12:36 | |
| *** gordc has joined #openstack-infra | 12:37 | |
| *** woodster_ has joined #openstack-infra | 12:37 | |
| *** jcoufal has joined #openstack-infra | 12:38 | |
| *** vlaza has quit IRC | 12:39 | |
| *** annegentle has joined #openstack-infra | 12:41 | |
| *** pal has quit IRC | 12:42 | |
| *** markvoelker has quit IRC | 12:43 | |
| *** markvoelker has joined #openstack-infra | 12:44 | |
| *** ddieterly has joined #openstack-infra | 12:44 | |
| *** sdake_ has quit IRC | 12:47 | |
| *** ildikov_afk is now known as ildikov | 12:49 | |
| *** xyang1 has joined #openstack-infra | 12:50 | |
| *** krtaylor has joined #openstack-infra | 12:52 | |
| *** aderyugin has joined #openstack-infra | 12:55 | |
| *** bknudson has joined #openstack-infra | 12:56 | |
| *** Ala has quit IRC | 13:00 | |
| *** annegentle has quit IRC | 13:01 | |
| *** annegentle has joined #openstack-infra | 13:02 | |
| *** wenlock has joined #openstack-infra | 13:02 | |
| *** dboik has joined #openstack-infra | 13:02 | |
| *** ddieterly has quit IRC | 13:03 | |
| *** zz_avozza is now known as avozza | 13:04 | |
| *** oomichi has joined #openstack-infra | 13:05 | |
| *** AJaeger has joined #openstack-infra | 13:05 | |
| openstackgerrit | Matthew Gilliard proposed openstack-infra/elastic-recheck: Add query for bug 1442602 https://review.openstack.org/172408 | 13:05 |
| openstack | bug 1442602 in OpenStack Compute (nova) "live migration fails during destination host check" [Undecided,Confirmed] https://launchpad.net/bugs/1442602 | 13:05 |
| openstackgerrit | Matthew Gilliard proposed openstack-infra/elastic-recheck: Add query for bug 1442602 https://review.openstack.org/172408 | 13:06 |
| *** hdd has joined #openstack-infra | 13:07 | |
| *** hdd has quit IRC | 13:09 | |
| *** wenlock has quit IRC | 13:10 | |
| *** spzala has joined #openstack-infra | 13:12 | |
| *** alexpilotti has joined #openstack-infra | 13:15 | |
| *** tiswanso has joined #openstack-infra | 13:17 | |
| *** bswartz has joined #openstack-infra | 13:20 | |
| *** e0ne is now known as e0ne_ | 13:22 | |
| *** avozza is now known as zz_avozza | 13:25 | |
| *** alexpilotti has quit IRC | 13:25 | |
| *** mriedem_away is now known as mriedem | 13:26 | |
| openstackgerrit | Alexey Deryugin proposed openstack-infra/devstack-gate: [murano] Neutron devstack default config conflicts with devstack-gate defaults https://review.openstack.org/172419 | 13:27 |
| *** annegentle has quit IRC | 13:30 | |
| *** ddieterly has joined #openstack-infra | 13:30 | |
| jeblair | tchaypo: i don't think there is a way to bulk-fetch comments, but if you find one, that's great. i don't think we should paralellize tasks because i think we should be nice to the gerrit server. | 13:31 |
| *** annegentle has joined #openstack-infra | 13:32 | |
| *** dustins has joined #openstack-infra | 13:33 | |
| *** sabeen has joined #openstack-infra | 13:34 | |
| jeblair | tchaypo: i notice that most of your output relates to neutron, where changes are more or less being continually updated by third-party ci systems, so gertty is going to need to update its copies a lot. | 13:35 |
| *** oomichi has quit IRC | 13:35 | |
| jeblair | tchaypo, clarkb: but here are some things that may help mitigate it: first, the more often you sync, the shorter it should be (including running it continuously, which is the intent) | 13:37 |
| jeblair | tchaypo, clarkb: and second, if you hit the refresh key (ctrl-r), it triggers a high-priority refresh of the current change. so you don't have to wait for a background sync to finish to start work. | 13:38 |
| *** eharney has joined #openstack-infra | 13:39 | |
| *** _nadya_ has quit IRC | 13:39 | |
| *** sabeen3 has joined #openstack-infra | 13:41 | |
| *** Ala has joined #openstack-infra | 13:41 | |
| jeblair | tchaypo, clarkb: anyway, i'm sure there's more optimization we can do. i've subscribed to neutron to drive up my own gertty load some more. i'll see what i can think of, and let me know if anything comes to you. | 13:41 |
| *** sabeen has quit IRC | 13:41 | |
| dansmith | are we waiting on this https://review.openstack.org/#/c/170913/ until after everyone gets an rc, or can we go ahead and turn that on? | 13:42 |
| *** yamamoto has joined #openstack-infra | 13:42 | |
| *** mrmartin has joined #openstack-infra | 13:42 | |
| *** dmellado has quit IRC | 13:42 | |
| *** esker has joined #openstack-infra | 13:43 | |
| *** dmellado has joined #openstack-infra | 13:44 | |
| *** yamamoto has quit IRC | 13:47 | |
| *** dansmith is now known as superdan | 13:47 | |
| *** gilliard is now known as gillllliard | 13:47 | |
| openstackgerrit | Merged openstack/requirements: Add pyngus to requirements https://review.openstack.org/172315 | 13:48 |
| *** cdent_ has joined #openstack-infra | 13:49 | |
| *** wenlock has joined #openstack-infra | 13:49 | |
| *** cdent has quit IRC | 13:49 | |
| *** cdent_ is now known as cdent | 13:49 | |
| *** e0ne_ is now known as e0ne | 13:49 | |
| *** jistr has quit IRC | 13:49 | |
| sdague | superdan: so I think it's fine, though I don't see Jon Bernard in either -qa or -infra channels | 13:55 |
| *** doug-fish has joined #openstack-infra | 13:55 | |
| sdague | which is normally where breaks are seen first | 13:55 |
| *** jistr has joined #openstack-infra | 13:56 | |
| superdan | sdague: I think he's sufficiently discoverable, but I'll be glad to ask him to join :) | 13:56 |
| *** jbernard has joined #openstack-infra | 13:56 | |
| superdan | sdague: ^ :) | 13:56 |
| jbernard | o/ | 13:56 |
| *** ayoung has joined #openstack-infra | 13:56 | |
| sdague | jbernard: hey | 13:56 |
| jbernard | sdague: morning | 13:57 |
| *** cdent has quit IRC | 13:57 | |
| * mkj_wk wonders whom to beg for another +2 (or otherwise) on git-review proposed change https://review.openstack.org/#/c/158877/ | 13:57 | |
| jbernard | sdague: if it is, in fact, morning where you are :) | 13:57 |
| *** mwagner_lap has joined #openstack-infra | 13:57 | |
| sdague | ok, I'm +2 on the ceph voting as long as you hang out in -infra and -qa and keep an eye out for breaks in the ceph job | 13:57 |
| jbernard | no problem, count me in | 13:57 |
| sdague | ok, vote registered | 13:57 |
| superdan | sdague: thanks! | 13:57 |
| jbernard | sdague: thank you | 13:58 |
| *** bswartz has quit IRC | 13:59 | |
| *** wenlock has quit IRC | 13:59 | |
| *** pal has joined #openstack-infra | 14:00 | |
| *** duran has joined #openstack-infra | 14:01 | |
| mordred | mkj_wk: reading it ... | 14:01 |
| *** teran has joined #openstack-infra | 14:02 | |
| *** bswartz has joined #openstack-infra | 14:02 | |
| mkj_wk | mordred: cool, thanks! :) | 14:03 |
| *** cdent has joined #openstack-infra | 14:03 | |
| *** e0ne is now known as e0ne_ | 14:05 | |
| *** teran_ has quit IRC | 14:05 | |
| *** hdd has joined #openstack-infra | 14:06 | |
| *** pal_ has joined #openstack-infra | 14:08 | |
| *** pal has quit IRC | 14:08 | |
| *** e0ne_ is now known as e0ne | 14:08 | |
| *** koolhead17 has joined #openstack-infra | 14:09 | |
| ttx | sdague, fungi: I was wondering... will the proposal bot actually propose stable/kilo g-r changes for merging into proposed/kilo branches ? | 14:12 |
| fungi | ttx: i'll check | 14:12 |
| sdague | ttx: no idea | 14:12 |
| *** otter768 has joined #openstack-infra | 14:12 | |
| sdague | there is a bunch of logic which goes sideways on proposed/ because it doesn't exist most of the time | 14:12 |
| fungi | i assume the implication is that we would like it to if it doesn't already? | 14:12 |
| ttx | If not, maybe we should cut a proposed/kilo req branch instead | 14:13 |
| ttx | fungi: yes | 14:13 |
| *** mestery is now known as mestery_afk | 14:13 | |
| fungi | hrm... proposed reqs branch is also an interesting idea | 14:13 |
| ttx | at rc1 we cut proposed/kilo branches and after that in theory we cut a stable/kilo branch for requirements | 14:13 |
| *** ddieterly has quit IRC | 14:13 | |
| ttx | but then we push caps | 14:13 |
| fungi | i'll see what we've got so far | 14:13 |
| *** xyang0 has joined #openstack-infra | 14:13 | |
| *** ddieterly has joined #openstack-infra | 14:14 | |
| ttx | I guess we could emulate the bot if it doesn't do that. In Liberty we'll likely push the caps on master during freeze so that the RC1 includes them | 14:14 |
| ttx | so it shouldn't be a problem in the future | 14:14 |
| *** mtanino has joined #openstack-infra | 14:14 | |
| ttx | (and uncap on master after the stable branch cut | 14:14 |
| ttx | AJaeger: around? | 14:15 |
| AJaeger | ttx, yes | 14:15 |
| ttx | AJaeger: remember for Juno we cleaned up translations before issuing the last RCs | 14:15 |
| AJaeger | ttx, yes, we did. | 14:16 |
| AJaeger | I pushed translations for all teams a week or two ago... | 14:16 |
| AJaeger | Let me check current status | 14:16 |
| ttx | I was wondering if you would be around next week so that we do the same this time around | 14:16 |
| ttx | (we need to remove incomplete ones on the proposed/kilo branch if they don't reach the % threshold) | 14:16 |
| AJaeger | https://review.openstack.org/#/q/status:open++branch:master+topic:transifex/translations,n,z - only nova and manila have pending changes | 14:17 |
| *** otter768 has quit IRC | 14:17 | |
| AJaeger | ttx, there shouldn't be any really incomplete ones anymore - the tools propose removal if they are below 30 per cent | 14:17 |
| fungi | ttx: sdague: so at the moment http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/propose_update.sh has no branch fallback logic. it relies on strict branch matching | 14:17 |
| ttx | Yes, I made sure they all synced from relatively recent translations | 14:17 |
| AJaeger | ttx, cool | 14:17 |
| AJaeger | ttx, yes, I'm around, we can do this together... | 14:17 |
| *** alaski is now known as lascii | 14:17 | |
| ttx | AJaeger: the toold won't copy them if they are below 30% ? | 14:18 |
| openstackgerrit | Monty Taylor proposed openstack-infra/git-review: Use git push-url instead of a second remote https://review.openstack.org/109869 | 14:18 |
| ttx | So we don't need to clean anything up ? | 14:18 |
| sdague | yeh, I still think a lot of things would be simpler if we got rid of proposed/ and just did stable/ at this point :) | 14:18 |
| fungi | we could throw in a conditional where it checks whether the branch exists on a given project in the loop to also s,stable/,proposed/, and look for that | 14:18 |
| mordred | fungi, mkj_wk: although it reminds me that I've been running on a local fork of git-review with that ^^ applied - so I rebased it on top of mkj_wk's change | 14:18 |
| AJaeger | ttx: Our scripts propose not only add new files but also remove them again if they are below 30 % | 14:18 |
| ttx | sdague: that creates other issues though. Like crazy stable people approving non-RC fixes | 14:18 |
| fungi | mordred: oh, cool i should check that one out too | 14:18 |
| ttx | proposed/* lets us be in control | 14:19 |
| *** isviridov has quit IRC | 14:19 | |
| mordred | fungi: I have the setting turned on globally for myself, which means I have no gerrit remotes anyway | 14:19 |
| mordred | fungi: anywhere | 14:19 |
| ttx | alternative is to play ACL switching all the time | 14:19 |
| AJaeger | ttx, it'S 20 % - see http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/common_translation_update.sh#n344 | 14:19 |
| * mkj_wk reading 109869... | 14:19 | |
| ttx | AJaeger: do you think 20% is what we want in release ? | 14:19 |
| ttx | as a threshold ? | 14:19 |
| AJaeger | ttx, let's do some maths on what we have and go from there ;) | 14:20 |
| AJaeger | ttx, any specific day next week for this? | 14:20 |
| *** rkukura has quit IRC | 14:21 | |
| ttx | AJaeger: not really. I'd like the final translations to be in RC2s, and those will likely start appearing end of next week | 14:21 |
| *** dvorak is now known as clayton | 14:21 | |
| ttx | so it would be good to have the final tarnslations update proposed for proposed/kilo by then | 14:21 |
| mordred | mkj_wk: oh piddle - one of your unittests is going to break my unittests ... | 14:22 |
| mkj_wk | mordred: we're using gerrit as origin normally | 14:22 |
| AJaeger | ttx, see the removal in action: https://review.openstack.org/#/c/171941/ | 14:22 |
| mkj_wk | mordred: ah, I was still reading cmd.py... | 14:22 |
| mordred | mkj_wk: it's ok - it's a simple fix in my patch | 14:22 |
| AJaeger | ttx, ok, I'll put this on my agenda for early next week | 14:22 |
| ttx | who speaks German anyway, right | 14:22 |
| jeblair | mordred: does that make "git clone https://github.com/username/project; git review -s; git push" weird for people who might like to do that? | 14:22 |
| *** tnovacik has quit IRC | 14:23 | |
| mordred | jeblair: only if they turn on the optional feature | 14:23 |
| AJaeger | ttx: I do ;) | 14:23 |
| mkj_wk | mordred: oh, indeed | 14:23 |
| mordred | jeblair: I actually do a split push-url thing for github stuff too | 14:23 |
| AJaeger | ttx: But my default is English for OpenStack :) | 14:23 |
| mkj_wk | mordred: it will be easier to understand in the test suite that way, I think | 14:23 |
| jeblair | mordred: ok cool | 14:23 |
| mkj_wk | without the two remotes pointing the same place | 14:23 |
| mordred | jeblair: which is that I clone from, say, github.com/ansible/ansible, and then I set the push-url to github.com/emonty/ansible | 14:23 |
| mordred | so that I can fetch/pull from upstream, but then push to my branches | 14:24 |
| jeblair | the circle of patches | 14:24 |
| *** jlibosva has quit IRC | 14:24 | |
| mordred | jeblair: ++ | 14:24 |
| *** jlibosva has joined #openstack-infra | 14:25 | |
| ttx | AJaeger: I know you do :) | 14:26 |
| AJaeger | ;) | 14:26 |
| *** hdd has quit IRC | 14:26 | |
| ttx | fungi: sounds like a conditional would be nice. If stable/foo doesn't exist and proposed/foo does, sync your stable/foo g-r there | 14:28 |
| fungi | ttx: yep, that's what i'm writing now. need to refactor the code and replace a few variables to be able to do that just once per project | 14:28 |
| *** sarob has joined #openstack-infra | 14:29 | |
| *** Krinkle is now known as Krinkle|detached | 14:29 | |
| ttx | fungi: ping me on review | 14:29 |
| fungi | ttx: will do | 14:30 |
| AJaeger | Regarding requirements repository: sdague is always chasing me to remove the doc repositories from requirements. I remember som discussion here about separating projects in requirements. | 14:30 |
| jeblair | tchaypo, clarkb: i found a bug where gertty will sync a change multiple times unecessarily. i'm seeing it happen during my full sync of neutron | 14:31 |
| AJaeger | so, idea was to utilize the soft flag and allow some repositories to not have all requirements i nglobal-requirements. | 14:31 |
| *** hdd has joined #openstack-infra | 14:32 | |
| AJaeger | I have an idea on how to do this - do I need some spec for this? Just send a patch? Or what's the best way here? | 14:32 |
| jeblair | tchaypo, clarkb: when gertty syncs a change in order to ensure a parent commit is available, it may enqueue the same change sync task for each patchset that depends on that commit | 14:33 |
| ttx | guys, david-lyle is asking whether https://review.openstack.org/#/c/172123/1 is enough to disable py26 jobs on horizon | 14:37 |
| openstackgerrit | Alexey Deryugin proposed openstack-infra/project-config: [gate-murano-devstack-dsvm] Fix bug with neutron misconfiguration https://review.openstack.org/172447 | 14:37 |
| *** koolhead17 has quit IRC | 14:37 | |
| david-lyle | ttx: on django-openstack-auth | 14:37 |
| ttx | err. yes. that | 14:37 |
| AJaeger | ttx, david-lyle will check | 14:37 |
| *** kgiusti has joined #openstack-infra | 14:37 | |
| AJaeger | yes, that should be all | 14:38 |
| *** dboik_ has joined #openstack-infra | 14:38 | |
| *** wenlock has joined #openstack-infra | 14:38 | |
| jeblair | tchaypo, clarkb: based on a quick log grep, that could amount to 3x-4x extra syncs | 14:39 |
| *** dboik_ has quit IRC | 14:39 | |
| *** wenlock1 has joined #openstack-infra | 14:39 | |
| *** soren has quit IRC | 14:39 | |
| *** dboik_ has joined #openstack-infra | 14:39 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config: Change proposals fall back on proposed branches https://review.openstack.org/172451 | 14:40 |
| fungi | ttx: sdague: thinking something like that ^ | 14:40 |
| *** zz_zz_ja| has quit IRC | 14:41 | |
| *** dims_ has joined #openstack-infra | 14:41 | |
| *** dangers_away is now known as dangers | 14:41 | |
| *** maurosr has quit IRC | 14:42 | |
| *** jlibosva has quit IRC | 14:42 | |
| *** dboik has quit IRC | 14:42 | |
| *** maurosr has joined #openstack-infra | 14:43 | |
| *** wenlock_ has joined #openstack-infra | 14:44 | |
| *** marun has joined #openstack-infra | 14:44 | |
| *** ajmiller has joined #openstack-infra | 14:46 | |
| *** sdake has joined #openstack-infra | 14:46 | |
| *** esker has quit IRC | 14:47 | |
| *** sarob has quit IRC | 14:48 | |
| *** jgrimm has quit IRC | 14:48 | |
| *** sdake_ has joined #openstack-infra | 14:49 | |
| *** zz_ja has joined #openstack-infra | 14:50 | |
| *** sdake has quit IRC | 14:50 | |
| fungi | AJaeger: you've had a fair involvement with some of the earlier refactoring in that script too | 14:52 |
| fungi | so extra eyeballs appreciated | 14:52 |
| AJaeger | fungi: will do soon | 14:52 |
| *** mjturek1 has left #openstack-infra | 14:55 | |
| *** mjturek1 has joined #openstack-infra | 14:55 | |
| *** dboik_ has quit IRC | 14:56 | |
| *** mjturek1 has left #openstack-infra | 14:56 | |
| *** wenlock_ has quit IRC | 14:56 | |
| *** dims_ is now known as dimsum__ | 14:57 | |
| *** mmedvede_ has quit IRC | 14:57 | |
| *** sdake has joined #openstack-infra | 14:58 | |
| *** maurosr has quit IRC | 14:58 | |
| *** xyang0 has quit IRC | 14:58 | |
| *** zz_ja has quit IRC | 14:58 | |
| *** freerunner has quit IRC | 14:58 | |
| *** teran has quit IRC | 14:58 | |
| *** freerunner has joined #openstack-infra | 14:59 | |
| *** teran has joined #openstack-infra | 14:59 | |
| *** dustins_ has joined #openstack-infra | 14:59 | |
| superdan | clarkb: just wanted to say, the aiopcpu and live migration test is already helping us | 15:01 |
| superdan | clarkb: so thanks for that :) | 15:01 |
| *** yamamoto has joined #openstack-infra | 15:01 | |
| *** sdake_ has quit IRC | 15:02 | |
| *** dustins has quit IRC | 15:03 | |
| *** maurosr has joined #openstack-infra | 15:03 | |
| *** sarob has joined #openstack-infra | 15:05 | |
| *** mestery_afk is now known as mestery | 15:06 | |
| *** armax has joined #openstack-infra | 15:06 | |
| cloudnull | hey all, it looks like the gate is having a bad day. known issue? | 15:07 |
| *** esker has joined #openstack-infra | 15:08 | |
| *** maurosr has quit IRC | 15:08 | |
| AJaeger | fungi: looks fine | 15:09 |
| *** rwsu has quit IRC | 15:09 | |
| *** xyang0 has joined #openstack-infra | 15:10 | |
| *** dannywilson has joined #openstack-infra | 15:10 | |
| fungi | cloudnull: hrm, looks like maybe we lost our graphite server, if you're referring to the graphs. i'll see what happened to it | 15:12 |
| cloudnull | yea I was referring to the graphs. | 15:13 |
| cloudnull | but if there's nothing to see here, ill move along :) | 15:13 |
| *** wenlock_ has joined #openstack-infra | 15:13 | |
| fungi | cloudnull: ahh, yep. looks like it stopped doing things http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=25 | 15:13 |
| rcarrillocruz | hey folks, could i please get another +2 on https://review.openstack.org/#/c/168334/ ? | 15:13 |
| *** AJaeger has quit IRC | 15:13 | |
| *** e0ne is now known as e0ne_ | 15:13 | |
| rcarrillocruz | got already quite a bit of positive feedback and +2 from jeblair | 15:13 |
| fungi | guessing some service choked and died | 15:13 |
| *** yamahata has joined #openstack-infra | 15:14 | |
| *** AJaeger has joined #openstack-infra | 15:14 | |
| cloudnull | it happens to the best of the services . | 15:14 |
| fungi | indeed | 15:14 |
| fungi | nothing exciting in dmesg output... an apache segfault over two weeks ago | 15:14 |
| fungi | activity on the server died fairly abruptly at 02:00 utc. checking syslog for any clues arounf that time | 15:16 |
| *** ddieterly has quit IRC | 15:17 | |
| fungi | that's the time that our cleanup cron job kicks off to delete cruft whisper files | 15:18 |
| *** ddieterly has joined #openstack-infra | 15:18 | |
| *** armax has quit IRC | 15:19 | |
| *** e0ne_ is now known as e0ne | 15:20 | |
| *** dboik has joined #openstack-infra | 15:20 | |
| annegentle | the one and only time that "late library bumps" makes sense in an email | 15:20 |
| * fungi bets his librarian friends have used that phrase before | 15:21 | |
| * annegentle worked in university libraries all through college | 15:21 | |
| fungi | it's when you're reshelving books late at night and one of them falls back on your head | 15:22 |
| annegentle | I still have nightmares, very rarely, about turning off all the lights in the basement of the huge library and standing alone in the pitch black while waiting for the elevator to arrive to go back upstairs. | 15:22 |
| annegentle | Darkest dark ever, except for caving expeditions where they turn the lights out. | 15:22 |
| annegentle | bump in the night! | 15:22 |
| fungi | i participated in more than a few sensory deprivation studies back in my university days... that's pretty dark | 15:23 |
| *** wenlock_ has quit IRC | 15:23 | |
| *** tsg_ has joined #openstack-infra | 15:24 | |
| *** asselin has quit IRC | 15:24 | |
| fungi | cloudnull: apparently the carbon-cache service died at 2am. i've started it back now and am looking for any logs which might explain why | 15:25 |
| fungi | clarkb: thanks for the heads up! | 15:25 |
| fungi | er, cloudnull ^ | 15:25 |
| fungi | (sorry clarkb!) | 15:25 |
| *** pc_m_ has quit IRC | 15:25 | |
| cloudnull | fungi: thank you for fixing it! | 15:25 |
| fungi | well, i'm pretty sure i haven't fixed whatever caused it to die | 15:26 |
| odyssey4me | fungi does that mean that jobs are actually running in the background, we just can't see them in the status page? | 15:26 |
| *** annegentle has quit IRC | 15:26 | |
| *** sushilkm has joined #openstack-infra | 15:27 | |
| fungi | odyssey4me: you can see them running on the status page, the graphs on the status page are merely flat-lined | 15:27 |
| *** sushilkm has left #openstack-infra | 15:27 | |
| *** krtaylor has quit IRC | 15:27 | |
| fungi | but the rest of the status page is active and current | 15:27 |
| fungi | though it does rather look like a lot of people have taken the day off | 15:28 |
| *** krtaylor has joined #openstack-infra | 15:28 | |
| odyssey4me | fungi I'm not so sure about that, the check queue has 12 jobs showing and I do know that we've got several jobs waiting for attention | 15:28 |
| fungi | can't say i blame them | 15:28 |
| *** thedodd has joined #openstack-infra | 15:28 | |
| fungi | odyssey4me: hrm. a patch i proposed just a little while ago got check results | 15:28 |
| *** ivar-lazzaro has joined #openstack-infra | 15:29 | |
| odyssey4me | fungi in fact, I just rebased 16 patches within the last 15 mins | 15:29 |
| fungi | interesting. maybe we have more than one problem | 15:29 |
| odyssey4me | none of them are showing in the check queue | 15:29 |
| fungi | was it more recent than the glanceclient change which showed up in the check pipeline 9 minutes ago? | 15:29 |
| *** annegentle has joined #openstack-infra | 15:30 | |
| odyssey4me | fungi no, I don't think so - the last rebase was 21 mins ago | 15:31 |
| odyssey4me | all the rebases are in stackforge, if that affects the analysis | 15:31 |
| fungi | okay, then sounds like we've possibly got other issues | 15:31 |
| fungi | i'll see what i can turn up | 15:31 |
| odyssey4me | thanks, if there's any help I can give then holler | 15:32 |
| *** maurosr has joined #openstack-infra | 15:32 | |
| *** zz_jgrimm has joined #openstack-infra | 15:32 | |
| odyssey4me | would a review number to target help? | 15:32 |
| *** ivar-lazzaro has quit IRC | 15:32 | |
| fungi | sure, that'll make this a little faster than me triggering something | 15:32 |
| *** ivar-lazzaro has joined #openstack-infra | 15:32 | |
| odyssey4me | here's one: https://review.openstack.org/170558 | 15:33 |
| fungi | thanks@ | 15:33 |
| *** zz_ja has joined #openstack-infra | 15:34 | |
| fungi | yep, the last data zuul believes it received from gerrit's event stream was 15:20:09 utc | 15:34 |
| *** Ala has quit IRC | 15:35 | |
| ddieterly | is zuul dead or alive? | 15:35 |
| ddieterly | looks like the connection between gerrit and zuul is not working? | 15:36 |
| fungi | that's what i'm finding out. i suspect the problem is gerrit yet again | 15:36 |
| ddieterly | yeah, thanks | 15:36 |
| *** hdd has quit IRC | 15:38 | |
| xyang0 | hi, we are setting up a CI using CentOS/RHEL 7. does anyone see a problem with this? | 15:38 |
| openstackgerrit | yolanda.robla proposed openstack-infra/system-config: Move server outside of puppetmaster class https://review.openstack.org/172485 | 15:38 |
| fungi | yep, we have a bunch of stream-events tasks waiting in the gerrit queue since 15:20 utc | 15:38 |
| *** sarob has quit IRC | 15:39 | |
| *** maurosr has quit IRC | 15:39 | |
| *** afazekas has quit IRC | 15:40 | |
| mriedem | jeblair: mordred: hey just read http://lists.openstack.org/pipermail/openstack-infra/2015-February/002459.html - i was wondering if you had an idea of how many compute nodes were going to be running there? | 15:40 |
| mriedem | or if there were reqs documented somewhere for contributing compute resources to infra? | 15:41 |
| fungi | okay, restarting gerrit to get this back on track again | 15:41 |
| *** dboik has quit IRC | 15:41 | |
| *** alexpilotti has joined #openstack-infra | 15:41 | |
| *** zz_jgrimm has quit IRC | 15:41 | |
| claudiub_ | gerrit down? | 15:42 |
| *** maurosr has joined #openstack-infra | 15:42 | |
| fungi | see the line just above where you asked that | 15:42 |
| markvoelker | claudiub_: looks that way to me | 15:42 |
| claudiub_ | ah, cool :)) | 15:42 |
| markvoelker | heh | 15:42 |
| claudiub_ | works now | 15:42 |
| claudiub_ | i was going to vote a -1 on a commit. Error. I thought I was not allowed to vote -1 anymore. :D | 15:43 |
| fungi | #status notice gerrit has been restarted to address a hung event stream. change events between 15:00 and 15:43 utc which were lost will need to be rechecked or have approval workflow votes reapplied for zuul to act on them | 15:43 |
| openstackstatus | fungi: sending notice | 15:43 |
| -openstackstatus- NOTICE: gerrit has been restarted to address a hung event stream. change events between 15:00 and 15:43 utc which were lost will need to be rechecked or have approval workflow votes reapplied for zuul to act on them | 15:44 | |
| fungi | 15:00 utc is a conservative guess... i see zuul receiving some events from gerrit as late as 15:20 but odyssey4me provided an example from 15:09 which does not show a corresponding event in the zuul debug log | 15:44 |
| *** AJaeger has quit IRC | 15:45 | |
| openstackstatus | fungi: finished sending notice | 15:45 |
| ttx | fungi: hmm, I just branched glance propsoed/kilo, hope it didn't hang somewhere | 15:46 |
| ttx | Looks fine on cgit at least http://git.openstack.org/cgit/openstack/glance/log/ | 15:46 |
| fungi | ttx: if you see something missing associated with that, let me know and i can probably retrigger whatever needs retriggering | 15:46 |
| *** maurosr has quit IRC | 15:46 | |
| ttx | no, it might have left a stale branch-tarball buty I don't need that | 15:47 |
| fungi | ttx: it should have only affected zuul triggering jobs | 15:47 |
| *** zz_ja has quit IRC | 15:47 | |
| ttx | I can do a tag now, right | 15:47 |
| fungi | so yes, it might have missed an event to run the tarball job | 15:47 |
| fungi | yep, should be fine | 15:47 |
| ttx | alright, let's do this | 15:48 |
| *** gyee has joined #openstack-infra | 15:48 | |
| fungi | i see a glance-tarball pre-release job | 15:50 |
| fungi | for refs/tags/2015.1.0rc1 | 15:51 |
| *** alexpilotti has quit IRC | 15:51 | |
| *** rwsu has joined #openstack-infra | 15:51 | |
| *** alexpilotti has joined #openstack-infra | 15:51 | |
| *** zz_ja has joined #openstack-infra | 15:52 | |
| *** cody-somerville has joined #openstack-infra | 15:52 | |
| *** pc_m_ has joined #openstack-infra | 15:52 | |
| *** rkukura has joined #openstack-infra | 15:53 | |
| fungi | ttx: looks like it worked fine | 15:53 |
| ttx | yep, all fine | 15:53 |
| fungi | http://logs.openstack.org/af/aff0c4e5e356d52ece7cf522a845fc1c9b364a76/pre-release/glance-tarball/cb36370/console.html | 15:54 |
| *** pc_m__ has joined #openstack-infra | 15:54 | |
| fungi | also, it's logged to swift now! | 15:54 |
| *** pc_m__ has quit IRC | 15:54 | |
| odyssey4me | fungi it seems like it's back at work now | 15:54 |
| fungi | odyssey4me: yep. sorry for the trouble! | 15:54 |
| *** maurosr has joined #openstack-infra | 15:54 | |
| fungi | this is a known issue with gerrit we're trying to track down | 15:54 |
| odyssey4me | fungi no worries, I'm surprised at how well the lemmings manage sometimes :) | 15:55 |
| *** _nadya_ has joined #openstack-infra | 15:55 | |
| fungi | seems to have gotten worse when we redeployed it on ubuntu trusty. we hit it much less frequently on precise | 15:55 |
| *** enikanorov has quit IRC | 15:55 | |
| *** bnemec is now known as beekneemech | 15:55 | |
| fungi | also hoping the upgrade to gerrit 2.9 or 2.10 next month eliminates it | 15:55 |
| *** Swami has joined #openstack-infra | 15:56 | |
| *** YorikSar has quit IRC | 15:57 | |
| *** markus_z has quit IRC | 15:57 | |
| *** pc_m_ has quit IRC | 15:58 | |
| *** ildikov has quit IRC | 15:58 | |
| *** zz_ja has quit IRC | 15:58 | |
| pleia2 | good morning | 15:59 |
| cinerama | pleia2: hi | 16:00 |
| *** aysyd has quit IRC | 16:00 | |
| *** maurosr has quit IRC | 16:00 | |
| *** aysyd has joined #openstack-infra | 16:00 | |
| *** shashankhegde has joined #openstack-infra | 16:00 | |
| *** btully- has joined #openstack-infra | 16:01 | |
| *** tsg_ has quit IRC | 16:02 | |
| *** btully- is now known as btully | 16:04 | |
| *** reed has quit IRC | 16:04 | |
| *** zz_ja has joined #openstack-infra | 16:04 | |
| *** sarob has joined #openstack-infra | 16:04 | |
| *** btully is now known as Guest27138 | 16:05 | |
| *** dimsum__ has quit IRC | 16:05 | |
| *** jlanoux has quit IRC | 16:05 | |
| *** e0ne is now known as e0ne_ | 16:05 | |
| *** reed has joined #openstack-infra | 16:06 | |
| *** jistr has quit IRC | 16:06 | |
| *** dims_ has joined #openstack-infra | 16:07 | |
| *** hashar is now known as hasharAway | 16:08 | |
| *** jcoufal has quit IRC | 16:08 | |
| *** dims_ is now known as dimsum___ | 16:08 | |
| *** dimsum___ is now known as dimsum_ | 16:08 | |
| *** zz_ja has quit IRC | 16:09 | |
| openstackgerrit | Sergey Reshetnyak proposed openstack-infra/devstack-gate: Run sahara in distributed mode https://review.openstack.org/172494 | 16:09 |
| *** e0ne_ is now known as e0ne | 16:10 | |
| *** spzala has quit IRC | 16:10 | |
| *** hdd has joined #openstack-infra | 16:12 | |
| *** pblaho has quit IRC | 16:12 | |
| *** ashleighfarnham has joined #openstack-infra | 16:12 | |
| *** otter768 has joined #openstack-infra | 16:13 | |
| *** wenlock_ has joined #openstack-infra | 16:13 | |
| *** annegentle has quit IRC | 16:14 | |
| *** dboik has joined #openstack-infra | 16:14 | |
| *** dtantsur is now known as dtantsur|afk | 16:15 | |
| *** unicell has quit IRC | 16:16 | |
| *** sdake_ has joined #openstack-infra | 16:16 | |
| *** aricg has joined #openstack-infra | 16:17 | |
| *** otter768 has quit IRC | 16:17 | |
| *** zz_avozza is now known as avozza | 16:17 | |
| pleia2 | cinerama: hopefully with that change we're good, just left a note about starting to develop some usage instructions since the module is getting complicated (the TODO could use some love too) | 16:17 |
| fungi | clarkb: you might be interested in replying to http://lists.openstack.org/pipermail/openstack-dev/2015-April/061155.html as it asks for details on why using ovs as the default in devstack could pose a problem (especially for newcomers to the platform) | 16:18 |
| *** hdd has quit IRC | 16:18 | |
| cinerama | pleia2: yep, a readme is on my list in the etherpad of stuff to do | 16:18 |
| fungi | clarkb: also http://lists.openstack.org/pipermail/openstack-dev/2015-April/061214.html | 16:18 |
| pleia2 | cinerama: cool | 16:18 |
| clarkb | fungi: thanks | 16:18 |
| *** zz_jgrimm has joined #openstack-infra | 16:18 | |
| zaro | morning | 16:18 |
| pleia2 | g'day zaro | 16:19 |
| zaro | pleia2: yo, did you ever get your nexus 6 phone? | 16:19 |
| clarkb | pleia2: cinerama: I am likely going to take it easy today, but want to help with the zanata server if I can | 16:19 |
| cinerama | pleia2: so i suspect if we can get https://review.openstack.org/#/c/172247/ merged we should be ok | 16:19 |
| pleia2 | zaro: I did! | 16:19 |
| pleia2 | cinerama: that's my hope | 16:20 |
| zaro | pleia2: like it? | 16:20 |
| pleia2 | clarkb: https://review.openstack.org/#/c/172247/ should fix the jenkins fail for https://review.openstack.org/#/c/169074/ | 16:20 |
| *** sdake has quit IRC | 16:20 | |
| aricg | jjb question. I'm hoping to set nodes per installer, getting "yaml.scanner.ScannerError: mapping values are not allowed here" https://gist.github.com/Aricg/848ca07b2648a511424b | 16:20 |
| *** maurosr has joined #openstack-infra | 16:20 | |
| pleia2 | zaro: it's still feels a bit too big, but the battery life is great and it's a beautiful phone that I like to use | 16:20 |
| *** HeOS has quit IRC | 16:20 | |
| *** zz_ja has joined #openstack-infra | 16:20 | |
| *** yamamoto has quit IRC | 16:21 | |
| clarkb | pleia2: cinerama thats an easy approval, done | 16:21 |
| cinerama | thanks clarkb | 16:21 |
| *** rmcall has quit IRC | 16:22 | |
| *** rmcall has joined #openstack-infra | 16:23 | |
| zaro | fungi: ready to approve change to upgrade gerrit on review-dev? | 16:23 |
| *** zz_jgrimm has quit IRC | 16:23 | |
| *** amotoki has joined #openstack-infra | 16:23 | |
| zaro | pleia2: is there a mini? | 16:23 |
| *** dboik has quit IRC | 16:24 | |
| *** wenlock_ has quit IRC | 16:24 | |
| *** dboik has joined #openstack-infra | 16:25 | |
| *** maurosr has quit IRC | 16:25 | |
| *** armax has joined #openstack-infra | 16:26 | |
| pleia2 | zaro: heh, no :) | 16:26 |
| *** pkarikh has quit IRC | 16:27 | |
| pleia2 | zaro: pretty sure the good battery life thing is due to the phone being so big, big phone, big battery | 16:27 |
| *** yamahata has quit IRC | 16:28 | |
| *** MaxV has quit IRC | 16:30 | |
| pleia2 | btw, LISA doesn't conflict with the summit this year and it's in DC, CFP ends on the 17th (I'm submitting a talk on the tools we use for open source+collaborative sysadmining) https://www.usenix.org/conference/lisa15/call-for-participation | 16:31 |
| jesusaurus | clarkb: you mentioned before that you purposefully limit jenkins masters to 100 slaves, what issues were you seeing when you went over that? | 16:31 |
| *** maurosr has joined #openstack-infra | 16:31 | |
| zaro | jesusaurus: slaves drop off | 16:31 |
| fungi | zaro: since it's a system-config change i was hoping for a second +2 there | 16:31 |
| zaro | lose connection i mean | 16:31 |
| *** zz_jgrimm has joined #openstack-infra | 16:31 | |
| *** Guest27138 is now known as bttully | 16:32 | |
| zaro | fungi: np | 16:32 |
| clarkb | jesusaurus: ya what zaro says. Basically jenkins can't keep up with all the slave connections and they disconnect and jobs fail | 16:32 |
| jesusaurus | zaro: does an exception about the channel already being closed seem familiar? | 16:32 |
| fungi | zaro: though if you can entice pleia2 or clarkb or someone to approve it, i can help you with any associated tasks when i get back from lunch in a little while | 16:33 |
| zaro | clarkb, pleia2 : here if you would like to leap gerrit forward in time https://review.openstack.org/#/c/172226/ | 16:34 |
| *** zz_ja has quit IRC | 16:34 | |
| pleia2 | I can confirm the link in the change works, but I haven't watched how this kind of thing goes before, does it automatically swap out for new gerrit? | 16:35 |
| *** tqtran has joined #openstack-infra | 16:36 | |
| *** maurosr has quit IRC | 16:37 | |
| *** dprince has quit IRC | 16:37 | |
| fungi | clarkb: https://review.openstack.org/172451 would be good to have fairly early in the rc period | 16:37 |
| openstackgerrit | Merged openstack-infra/puppet-zanata: Remove dependency on mysql database https://review.openstack.org/172247 | 16:38 |
| fungi | i'm going to go grab lunch, back in a while | 16:38 |
| *** tonytan4ever has joined #openstack-infra | 16:38 | |
| zaro | pleia2: yes, it goes thru the entire process of upgrading old to new gerrit version. | 16:38 |
| *** patrickeast has joined #openstack-infra | 16:39 | |
| *** zz_jgrimm has quit IRC | 16:39 | |
| pleia2 | zaro: sounds fun, shall I approve? | 16:40 |
| pleia2 | I'm around all morning, so you can just shout if it breaks everything | 16:40 |
| zaro | jesusaurus: i don't remember but there should be an issue for it on the jenkins issue tracker. let me try to find it. | 16:40 |
| pleia2 | (and you can't fix it for some reason) | 16:40 |
| clarkb | fungi: I will look as soon as I am done writing this email | 16:41 |
| clarkb | fungi: its getting a bit novelish but I was asked to complain about OVS | 16:41 |
| zaro | pleia2: yes, approve. i'll monitor. i'm root on there so can fix if necessary | 16:41 |
| pleia2 | zaro: done, good luck :) | 16:41 |
| *** cody-somerville has quit IRC | 16:42 | |
| *** yamamoto has joined #openstack-infra | 16:43 | |
| cinerama | pleia2: merged; rechecking the things now | 16:43 |
| *** martingc has quit IRC | 16:43 | |
| *** ajmiller_ has joined #openstack-infra | 16:44 | |
| *** martingc has joined #openstack-infra | 16:44 | |
| *** unicell has joined #openstack-infra | 16:44 | |
| *** yamamoto has quit IRC | 16:44 | |
| *** jerryz has quit IRC | 16:44 | |
| *** ajmiller has quit IRC | 16:46 | |
| *** mrmartin has quit IRC | 16:46 | |
| *** armax has quit IRC | 16:46 | |
| *** derekh has quit IRC | 16:47 | |
| *** yamahata has joined #openstack-infra | 16:47 | |
| *** ajmiller_ is now known as ajmiller | 16:47 | |
| *** mrmartin has joined #openstack-infra | 16:47 | |
| *** ivar-lazzaro has quit IRC | 16:49 | |
| *** bradjones has quit IRC | 16:49 | |
| *** maurosr has joined #openstack-infra | 16:50 | |
| *** xyang0 has quit IRC | 16:50 | |
| openstackgerrit | Merged openstack-infra/system-config: upgrade review-dev to Gerrit ver 2.10 https://review.openstack.org/172226 | 16:50 |
| *** sputnik13 has joined #openstack-infra | 16:51 | |
| *** unicell has left #openstack-infra | 16:51 | |
| *** yamamoto has joined #openstack-infra | 16:52 | |
| *** _shaps_ has quit IRC | 16:52 | |
| *** bradjones has joined #openstack-infra | 16:52 | |
| *** e0ne has quit IRC | 16:53 | |
| cinerama | pleia2: recheck successful | 16:53 |
| *** _shaps___ has joined #openstack-infra | 16:53 | |
| *** ivar-lazzaro has joined #openstack-infra | 16:54 | |
| *** zz_ja has joined #openstack-infra | 16:54 | |
| pleia2 | cinerama: great, approving | 16:54 |
| *** xyang0 has joined #openstack-infra | 16:54 | |
| *** yamamoto has quit IRC | 16:55 | |
| *** zz_jgrimm has joined #openstack-infra | 16:55 | |
| zaro | jesusaurus: i think it was 'unexpected termination of channel' https://issues.jenkins-ci.org/browse/JENKINS-6817 | 16:55 |
| *** wenlock_ has joined #openstack-infra | 16:55 | |
| *** xyang0 has quit IRC | 16:56 | |
| *** xyang0 has joined #openstack-infra | 16:56 | |
| *** bttully is now known as btully | 16:58 | |
| *** dizquierdo has quit IRC | 16:58 | |
| *** armax has joined #openstack-infra | 16:59 | |
| *** shashankhegde has quit IRC | 16:59 | |
| *** dustins_ has quit IRC | 16:59 | |
| *** zz_ja has quit IRC | 17:00 | |
| *** maurosr has quit IRC | 17:00 | |
| *** _shaps___ has quit IRC | 17:00 | |
| *** packet has joined #openstack-infra | 17:00 | |
| *** zz_jgrimm has quit IRC | 17:00 | |
| *** dprince has joined #openstack-infra | 17:01 | |
| *** zz_ja has joined #openstack-infra | 17:01 | |
| jesusaurus | zaro: that looks pretty similar to the exceptions i'm seeing, thanks | 17:01 |
| *** mwagner_lap has quit IRC | 17:02 | |
| *** dustins has joined #openstack-infra | 17:02 | |
| *** btully has quit IRC | 17:02 | |
| openstackgerrit | Merged openstack-infra/system-config: Add infra setup for zanata https://review.openstack.org/169074 | 17:04 |
| *** tsg_ has joined #openstack-infra | 17:04 | |
| *** maurosr has joined #openstack-infra | 17:04 | |
| pleia2 | \o. | 17:05 |
| pleia2 | \o/ | 17:05 |
| pleia2 | cinerama: I'm just wrapping up a couple things, will be with you for server launching soon | 17:05 |
| cinerama | pleia2: ok great. that should be fun | 17:05 |
| *** zz_ja has quit IRC | 17:05 | |
| *** wenlock_ has quit IRC | 17:06 | |
| *** sushilkm has joined #openstack-infra | 17:06 | |
| *** zz_ja has joined #openstack-infra | 17:06 | |
| *** sushilkm has left #openstack-infra | 17:06 | |
| *** shashankhegde has joined #openstack-infra | 17:08 | |
| *** ssam2 has quit IRC | 17:08 | |
| *** maurosr has quit IRC | 17:10 | |
| *** mrmartin has quit IRC | 17:10 | |
| *** zz_ja has quit IRC | 17:10 | |
| *** tjones1 has joined #openstack-infra | 17:11 | |
| *** Guest96211 has quit IRC | 17:12 | |
| *** tjones1 has quit IRC | 17:12 | |
| *** tjones1 has joined #openstack-infra | 17:13 | |
| *** shashankhegde has quit IRC | 17:13 | |
| *** tjones1 has quit IRC | 17:14 | |
| *** tjones1 has joined #openstack-infra | 17:14 | |
| *** bswartz has quit IRC | 17:15 | |
| *** annegentle has joined #openstack-infra | 17:15 | |
| clarkb | ok email sent | 17:15 |
| *** sks has joined #openstack-infra | 17:15 | |
| clarkb | hopefully I don't get yelled at too much | 17:15 |
| *** zz_jgrimm has joined #openstack-infra | 17:16 | |
| *** achanda has joined #openstack-infra | 17:16 | |
| clarkb | fungi: http://lists.openstack.org/pipermail/openstack-dev/2015-April/061220.html | 17:17 |
| clarkb | fungi: reviewing the proposals for not yet stable branches change now | 17:17 |
| *** bswartz has joined #openstack-infra | 17:18 | |
| *** pal_ has quit IRC | 17:18 | |
| *** zz_ja has joined #openstack-infra | 17:18 | |
| mtreinish | clarkb: no one will yell at you about that. If they do just stand behind sc68cal :) | 17:19 |
| mtreinish | clarkb: fwiw I completely agree with your post there | 17:19 |
| jroll | clarkb: great email | 17:19 |
| *** mpaolino has joined #openstack-infra | 17:20 | |
| *** YorikSar has joined #openstack-infra | 17:20 | |
| *** achanda has quit IRC | 17:20 | |
| *** achanda has joined #openstack-infra | 17:21 | |
| *** mpaolino has quit IRC | 17:21 | |
| *** zz_jgrimm has quit IRC | 17:21 | |
| *** tjones1 has quit IRC | 17:21 | |
| *** tjones1 has joined #openstack-infra | 17:22 | |
| *** zz_ja has quit IRC | 17:23 | |
| *** arxcruz has quit IRC | 17:24 | |
| *** maurosr has joined #openstack-infra | 17:24 | |
| *** luqas has quit IRC | 17:25 | |
| *** pelix has quit IRC | 17:25 | |
| *** sarob has quit IRC | 17:25 | |
| *** ddieterly has quit IRC | 17:26 | |
| *** ddieterly has joined #openstack-infra | 17:26 | |
| *** erlon has joined #openstack-infra | 17:26 | |
| *** sarob has joined #openstack-infra | 17:26 | |
| * sc68cal discards his draft | 17:27 | |
| sc68cal | clarkb: good post | 17:27 |
| *** zz_jgrimm has joined #openstack-infra | 17:28 | |
| *** esker has quit IRC | 17:29 | |
| *** maurosr has quit IRC | 17:29 | |
| *** dboik_ has joined #openstack-infra | 17:31 | |
| *** sarob has quit IRC | 17:31 | |
| *** zz_jgrimm is now known as jgrimm | 17:32 | |
| *** ayoung has quit IRC | 17:32 | |
| *** adalbas has joined #openstack-infra | 17:32 | |
| *** Krinkle|detached is now known as Krinkle | 17:34 | |
| *** dboik has quit IRC | 17:35 | |
| *** shashankhegde has joined #openstack-infra | 17:35 | |
| *** amitgandhinz has joined #openstack-infra | 17:36 | |
| *** ayoung has joined #openstack-infra | 17:36 | |
| *** jgrimm has quit IRC | 17:36 | |
| clarkb | fungi: I approved the proposal script update change | 17:38 |
| *** _nadya_ has quit IRC | 17:38 | |
| *** e0ne has joined #openstack-infra | 17:40 | |
| *** johnthetubaguy is now known as zz_johnthetubagu | 17:40 | |
| *** zz_ja has joined #openstack-infra | 17:40 | |
| *** maurosr has joined #openstack-infra | 17:41 | |
| superdan | can I beg again for this? https://review.openstack.org/#/c/170913/ | 17:41 |
| superdan | while it's passing, we'd like to keep it that way :) | 17:42 |
| *** pal has joined #openstack-infra | 17:42 | |
| clarkb | superdan: I can review | 17:42 |
| *** shashankhegde has quit IRC | 17:43 | |
| *** mestery has quit IRC | 17:43 | |
| superdan | clarkb: thanks | 17:43 |
| clarkb | oh cool you got cinder on board | 17:43 |
| *** mestery has joined #openstack-infra | 17:43 | |
| superdan | yep | 17:43 |
| clarkb | superdan: also great to hear the multinode testing and migration stuff is useful. jogo did a lot of the owrk there too | 17:43 |
| superdan | clarkb: well, this was specifically the live migration test that was helping, but yes, thanks to all involved :) | 17:43 |
| *** tjones1 has quit IRC | 17:44 | |
| *** harlowja_away is now known as harlowja | 17:44 | |
| *** tjones1 has joined #openstack-infra | 17:44 | |
| *** tjones1 has quit IRC | 17:46 | |
| cinerama | pleia2: we're probably going to need to land another change so we can get admin access to the web UI but i want to confer with you about how to proceed | 17:46 |
| *** sks has quit IRC | 17:46 | |
| *** maurosr has quit IRC | 17:46 | |
| pleia2 | cinerama: the web UI for Wildfly or Zanata? | 17:46 |
| pleia2 | afaik we don't need to touch Wildfly, and I thought Zanata's admin was the same as standard logins | 17:46 |
| cinerama | pleia2: zanata. basically we either need to turn openid off or specify the usernames we'll register as with openid | 17:47 |
| clarkb | we can list openids for admins, we do that for storyoard iirc | 17:47 |
| *** ddieterly has quit IRC | 17:47 | |
| cinerama | pleia2: because there isn't currently a way to login with the internal stuff in parallel with openid | 17:47 |
| pleia2 | yeah, we do | 17:47 |
| *** tjones1 has joined #openstack-infra | 17:48 | |
| cinerama | at least on the web. we can try creating a role account for jenkins without openid and see if we can login successfully with the cli client | 17:48 |
| pleia2 | cinerama: "internal stuff"? | 17:48 |
| *** ddieterly has joined #openstack-infra | 17:48 | |
| *** photocyte has joined #openstack-infra | 17:49 | |
| cinerama | pleia2: the basic authentication is referred to as "internal" in the zanata docs etc | 17:49 |
| superdan | clarkb: thanks! | 17:50 |
| *** zz_jgrimm has joined #openstack-infra | 17:50 | |
| openstackgerrit | Merged openstack-infra/project-config: Change proposals fall back on proposed branches https://review.openstack.org/172451 | 17:50 |
| *** e0ne is now known as e0ne_ | 17:50 | |
| pleia2 | cinerama: hm, so is there a way to specify openids for admins like clarkb suggests? this is how we've done it in storyboard: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/files/storyboard/superusers.yaml | 17:51 |
| pleia2 | of course storyboard pulls in that .yaml file, we'd do it zanata's way | 17:51 |
| *** SumitNaiksatam has joined #openstack-infra | 17:51 | |
| cinerama | pleia2: there is, we just need to use the zanata_admin_users option in the main class & provide there a comma-separated list of users | 17:52 |
| *** Longgeek has quit IRC | 17:52 | |
| pleia2 | cinerama: that seems like an easy enough change to add after we deploy the server, how about you get to work on that change and I see about launching the server and we see where we end up? | 17:52 |
| cinerama | pleia2: presumably we should get the usernames from hiera? | 17:53 |
| pleia2 | cinerama: nah, I think we can do like we did with storyboard | 17:53 |
| pleia2 | storyboard wants email<>openid mapping, we just do username<>openid | 17:54 |
| *** boris-42 has joined #openstack-infra | 17:54 | |
| *** zz_jgrimm has quit IRC | 17:54 | |
| *** e0ne_ has quit IRC | 17:55 | |
| *** zz_ja has quit IRC | 17:55 | |
| openstackgerrit | Feodor Tersin proposed openstack-infra/project-config: Check global requirements before running of tempest-dsvm-neutron-src https://review.openstack.org/172522 | 17:57 |
| *** maurosr has joined #openstack-infra | 17:57 | |
| *** zaneb has left #openstack-infra | 17:57 | |
| krotscheck | pleia2: Note that username <> openid will only work if someone's logged into Launchpad. If you just arrive via ubuntu-one that's never created. | 17:57 |
| cinerama | oh i see what's happening with storyboard there. the admin users need to be specified in standalone.xml | 17:57 |
| *** e0ne has joined #openstack-infra | 17:57 | |
| *** ihrachyshka has quit IRC | 17:57 | |
| fungi | clarkb: thanks for reviewing | 17:58 |
| *** ashleighfarnham has quit IRC | 17:58 | |
| krotscheck | That's why we removed the username field, because we were encountering an annoyingly large number of users who'd never done that. | 17:58 |
| *** ivar-lazzaro has quit IRC | 17:58 | |
| pleia2 | cinerama: are we using launchpad or openstackid-dev? | 17:58 |
| krotscheck | Ultimately, email shouldn't really be necessary either. | 17:59 |
| cinerama | we would need to script up the database manipulation to do it in the way storyboard does | 17:59 |
| *** spzala has joined #openstack-infra | 17:59 | |
| cinerama | pleia2: we are using openstackid-dev | 17:59 |
| fungi | also i'll just leave this here http://www.oconnorbrewing.com/our-brews/el-guapo/ | 17:59 |
| fungi | amazing stugg | 17:59 |
| fungi | stuff | 17:59 |
| pleia2 | krotscheck: yeah, I was just using storyboard's file for an example, we're not planning on using launchpad | 17:59 |
| pleia2 | just as "these are openid links" | 17:59 |
| krotscheck | Gotcha | 17:59 |
| fungi | next time i head to the airport i'm planning to tour their brewery | 17:59 |
| pleia2 | krotscheck: launchpad/ubuntuone is such a mess though :( | 18:00 |
| pleia2 | I think my oustanding ticket about it with Canonical is having it's first birthday this month | 18:00 |
| krotscheck | pleia2: I love how canonical pays attention, eh? | 18:00 |
| fungi | pleia2: i have some very old ubuntu bugs if you want them | 18:01 |
| cinerama | pleia2: so the way zanata wants to work is that you list the usernames you want to have admin access in standalone.xml | 18:02 |
| pleia2 | krotscheck: https://help.ubuntu.com/community/WikiGuide/Registration is quite the amusing read for Friday with a case of beer | 18:02 |
| *** amotoki has quit IRC | 18:02 | |
| *** mwagner_lap has joined #openstack-infra | 18:02 | |
| pleia2 | cinerama: right, so how are they paired with openid? | 18:02 |
| krotscheck | pleia2: I... will pass on that. i'm currently digging through all the javascript things with similar effect. | 18:02 |
| cinerama | pleia2: then when you go to set up an account on zanata for the first time, you login to openstackid, select a username & password in the zanata UI | 18:02 |
| pleia2 | krotscheck: ouch, good luck with that :) | 18:02 |
| cinerama | pleia2: at that point zanata will save in the database the association between the username & the openid uri | 18:03 |
| pleia2 | cinerama: ah, so we just need to list some usernames, and make sure we register with the right username? | 18:03 |
| openstackgerrit | Merged openstack-infra/project-config: Make (check|gate)-tempest-dsvm-full-ceph voting for cinder/glance/nova https://review.openstack.org/170913 | 18:03 |
| cinerama | pleia2: that's what i'm saying. we can build a tool similar to what we're using with storyboard and its associated XML, but that is how zanata seems to want to do it | 18:03 |
| cinerama | pleia2: thus my impetus to put the admin names in hiera | 18:04 |
| fungi | tristanC: got my nova and glance ballots, thanks! looks like things are working | 18:04 |
| pleia2 | cinerama: I'd rather avoid hiera because it's not secret data and we like being transparent about admins | 18:04 |
| cinerama | pleia2: or just specify it in the site.pp | 18:05 |
| pleia2 | cinerama: site.pp wfm | 18:05 |
| *** tsg_ has quit IRC | 18:06 | |
| pleia2 | alright, what size vm do we want this running on? | 18:06 |
| tchaypo | Jeblair: ouch | 18:06 |
| fungi | pleia2: cinerama: worth noting, the only reason the "sysadmins" e-mail addresses list is in hiera is so that people redeploying our openstack_project module won't configure their servers to e-mail us all their cronspam | 18:06 |
| clarkb | tristanC: fungi I got a nova ballot too, now I need to go find out what chnage I made to nova | 18:07 |
| fungi | it's not really secret, just cutting down on annoyance | 18:07 |
| pleia2 | fungi: yeah, we don't have that issue here | 18:07 |
| fungi | clarkb: if you haven't made changes to all the official projects then something is wrong | 18:07 |
| clarkb | fungi: I think its telling I don't remember making a change :) its just normal to do so | 18:08 |
| *** dboik_ has quit IRC | 18:08 | |
| fungi | i'll let you have the next scripted mass change proposal | 18:08 |
| clarkb | I think it may have een some stable backports of gate fixes | 18:08 |
| fungi | likely. we do make those | 18:09 |
| *** armax has quit IRC | 18:10 | |
| tristanC | clarkb: this commit makes you part of the Nova electorate: http://git.openstack.org//cgit/openstack/nova/commit/?id=4f3e3cd1883ea5aca237bcc88b64128c743a6862 | 18:10 |
| clarkb | aha! | 18:10 |
| tristanC | fungi: you're welcome :) | 18:10 |
| openstackgerrit | Feodor Tersin proposed openstack-infra/project-config: Check global requirements before running of tempest-dsvm-neutron-src https://review.openstack.org/172522 | 18:11 |
| *** maurosr has quit IRC | 18:11 | |
| fungi | tristanC: pleia2: remember since you set the eligibility period for the ptl and tc elections to the same date range, the tc_roll.csv i provided should be usable for the tc election next week without further help from me, but i'm still here if you need anything | 18:12 |
| pleia2 | fungi: nods, thanks :) | 18:12 |
| *** MarkAtwood has quit IRC | 18:12 | |
| pleia2 | cinerama, fungi, clarkb - which flavor for zanata? http://paste.openstack.org/show/202160/ (cinerama based on your testing, fungi+clarkb based on "what we use for infra") | 18:12 |
| clarkb | pleia2: 8GB maybe? | 18:13 |
| fungi | pleia2: we usually go for "performance" flavors | 18:13 |
| fungi | so whichever of those fits your memory requirements | 18:13 |
| cinerama | pleia2: minimum of 4G | 18:13 |
| pleia2 | 8G performance it is | 18:14 |
| *** mrmartin has joined #openstack-infra | 18:14 | |
| *** otter768 has joined #openstack-infra | 18:14 | |
| clarkb | I think 8GB performance has become fairly standard as a starting point | 18:14 |
| cinerama | pleia2: i'm not sure how much disk we'll use when we use it in anger. the base install & associated files are around 1G | 18:14 |
| clarkb | then grow if necessary | 18:14 |
| * pleia2 nods | 18:14 | |
| fungi | performance1-4 fits what i deployed the new ask.openstack.org on. performance-1-8 is what we're using for nodepool workers | 18:14 |
| *** ibiris is now known as ibiris_away | 18:14 | |
| fungi | cinerama: pleia2: don't base it on disk. for that, we have cinder | 18:14 |
| pleia2 | yep, just add volumes as needed for data | 18:15 |
| fungi | ideally, put any local data on a cinder volume. makes redeployment easier too | 18:15 |
| *** cdent has quit IRC | 18:15 | |
| cinerama | just throwing the info out there | 18:15 |
| pleia2 | cinerama: yep, that's what we need :) | 18:16 |
| fungi | even for my personal servers in rax, i use performance1-1 but i attach a 100gb cinder volume (the smallest they'll allow) at /srv | 18:16 |
| openstackgerrit | Stephanie Miller proposed openstack-infra/system-config: Add admin users https://review.openstack.org/172528 | 18:16 |
| *** dustins has quit IRC | 18:16 | |
| clarkb | oh speaking of I need to find a new home for my server | 18:16 |
| clarkb | $50/month is a bit steep to get irc | 18:16 |
| harlowja | can i get someone to look at https://review.openstack.org/#/c/172179/ so that i can start fixing the repos branches :) | 18:17 |
| harlowja | pretty please, ha | 18:17 |
| fungi | yeah, i similarly should look at migrating to save myself some green | 18:17 |
| harlowja | ^ if its busted/wrong let me know ;) | 18:17 |
| pleia2 | clarkb: I'm still with linode | 18:17 |
| harlowja | ^ will allow me to get the others to stop using https://code.launchpad.net/cloud-init ;) | 18:17 |
| clarkb | harlowja: what you want is to add the create permission to refs/heads | 18:17 |
| clarkb | harlowja: you don't need that new block | 18:17 |
| harlowja | k | 18:18 |
| fungi | yeah, that looks entirely redundant | 18:18 |
| harlowja | so like 'exclusiveGroupPermissions = create label-Code-Review label-Workflow' in the existing session? | 18:18 |
| harlowja | *section not session | 18:18 |
| *** otter768 has quit IRC | 18:18 | |
| fungi | should inherit from the all-projects acl and the refs/heads/* access | 18:18 |
| clarkb | pleia2: I have been looking at runabove. they have a $2.50/month flavor | 18:18 |
| clarkb | harlowja: no create = group cloud-init-core | 18:19 |
| *** pal has quit IRC | 18:19 | |
| clarkb | harlowja: or something like that | 18:19 |
| harlowja | k | 18:19 |
| clarkb | harlowja: grep for create in those dirs | 18:19 |
| harlowja | ya, got one | 18:19 |
| clarkb | you should ifnd an example or 20 | 18:19 |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Allow access to 0.7.x branch(es) https://review.openstack.org/172179 | 18:19 |
| fungi | all my friends are on linode. so far runabove has been the only option i've been able to suggest to them for moving to openstack without paying lots more | 18:19 |
| harlowja | ok, hopefully ^ is the one | 18:19 |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Allow access to create branch(es) https://review.openstack.org/172179 | 18:19 |
| clarkb | fungi: its hard to argue with $2.50/month and not getting a container | 18:19 |
| clarkb | though it is oversubscribed. THeir non oversubscribed flavors are also affordable | 18:20 |
| fungi | harlowja: okay, so you want to be able to create branches. that version should work fine | 18:20 |
| harlowja | thx | 18:20 |
| clarkb | though should it be release or core? | 18:20 |
| harlowja | hmmm | 18:20 |
| fungi | harlowja: the only suggestion i have is that creating new branches and tagging releases is more of a release activity | 18:20 |
| clarkb | harlowja: keep in mind that deletion is something that infra has to do because gerrit acls aren't rich enough there | 18:20 |
| fungi | yeah, what clarkb said | 18:20 |
| *** tsg_ has joined #openstack-infra | 18:21 | |
| harlowja | k, let me just make it release group | 18:21 |
| fungi | thanks. the idea being the people who create branches and tags are aware that deletion is a rare (or impossible) thing and act with the necessary care | 18:21 |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Allow access to create branch(es) https://review.openstack.org/172179 | 18:21 |
| harlowja | fungi i'm hoping not to delete much, i just want to create a 0.7.x branch (so we have a place to continue that development if any) and then create tags that mirror the bzr tags | 18:22 |
| harlowja | and then stop using bzr (eventually) | 18:22 |
| fungi | makes total sense | 18:22 |
| harlowja | cool | 18:22 |
| harlowja | and then profit, ha | 18:22 |
| fungi | prophet | 18:23 |
| harlowja | ha | 18:23 |
| Swami | clarkb: Do you have the details of the route info configured on the nodes that would route traffic between 172.24.4.0/23 network and 172.24.5.0/24 network for the multinode setup | 18:23 |
| pleia2 | fungi, clarkb, I'm going through launch/README to create this server, and added translate_dev_mysql_password to hiera | 18:23 |
| clarkb | Swami: ya its just IPs assigned to interfaces. so when you assign 172.24.4.1/23 to an interface that interface is used as the route to 172.24.5.0/24 | 18:24 |
| * pleia2 about to launch_node.py | 18:24 | |
| clarkb | Swami: its a hack but it works great for nova net | 18:24 |
| fungi | pleia2: definitely let me know if you have any questions. i use that process all the time, so it's possible there are assumptions i failed to document in it | 18:24 |
| Swami | clarkb: thanks. | 18:24 |
| clarkb | Swami: it does mean that nested VMs can't ssh to the controller/compute host but I think we want that anyways | 18:24 |
| *** dimtruck is now known as zz_dimtruck | 18:24 | |
| clarkb | s/ssh/connect/ | 18:26 |
| clarkb | pleia2: gl | 18:26 |
| *** soren_ has joined #openstack-infra | 18:27 | |
| pleia2 | oh hey, let's delete the old translate-dev.openstack.org server first | 18:27 |
| pleia2 | any objections? | 18:27 |
| *** armax has joined #openstack-infra | 18:27 | |
| *** sarob has joined #openstack-infra | 18:27 | |
| fungi | pleia2: none from me, but i neither created nor tested it | 18:27 |
| pleia2 | I think it was pootle | 18:28 |
| fungi | pleia2: if you're launching a new server called translate-dev without first deleting the old one, just be forewarned that the dns commands printed at the end of the launch process will be duplicated and you want the first set | 18:28 |
| *** dboik has joined #openstack-infra | 18:29 | |
| pleia2 | fungi: nods, that happened with the backups server too | 18:29 |
| fungi | well, you _probably_ want the first set, but you should double check that | 18:29 |
| fungi | for me it's always been the first set so far | 18:29 |
| fungi | also, here's hoping rax eventually moves to designate for dns | 18:30 |
| *** melwitt has joined #openstack-infra | 18:31 | |
| pleia2 | Could not find any suitable endpoint. Correct region? | 18:31 |
| pleia2 | hm | 18:31 |
| clarkb | that may be the double compute response from rax keystone | 18:31 |
| clarkb | I thought we had fixed that in the env files but maybe not for that particular one? | 18:32 |
| fungi | pleia2: oh, take a look at the variable script | 18:32 |
| pleia2 | nova list and things work ok | 18:32 |
| fungi | just a sec | 18:32 |
| *** ildikov has joined #openstack-infra | 18:32 | |
| fungi | try exporting OS_SERVICE_NAME=cloudServersOpenStack | 18:32 |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Update bouncy castle libs for Gerrit 2.10 https://review.openstack.org/172534 | 18:32 |
| *** sarob has quit IRC | 18:32 | |
| clarkb | hogepodge: ^ this is the rax keystone issue | 18:32 |
| fungi | sometimes it seems to want NOVA_SERVICE_NAME=cloudServersOpenStack instead | 18:32 |
| zaro | clarkb, fungi ^ mind taking a look? | 18:32 |
| *** zz_jgrimm has joined #openstack-infra | 18:33 | |
| pleia2 | fungi: thanks, that's better | 18:33 |
| *** _nadya_ has joined #openstack-infra | 18:33 | |
| clarkb | zaro: hrm we intentionally use system packages | 18:33 |
| zaro | yeah, i know | 18:33 |
| fungi | clarkb: worth noting defcore testing won't hit that unless they have legacy non-nova compute too | 18:33 |
| *** crc32 has joined #openstack-infra | 18:33 | |
| fungi | since there will be only one option and i think it will just use it | 18:34 |
| clarkb | oh right | 18:34 |
| clarkb | you need an older account | 18:34 |
| pleia2 | tsk, this is launching a precise server | 18:34 |
| clarkb | pleia2: precise may be the default | 18:34 |
| clarkb | pleia2: you can specify the image though | 18:35 |
| pleia2 | yeah, I may have to do that | 18:35 |
| zaro | Gerrit 2.10 is running on review-dev now. | 18:35 |
| pleia2 | zaro: yay | 18:35 |
| fungi | pleia2: yeah, you can just ctrl-c and then set the image name. we do default to precise still | 18:35 |
| *** teran_ has joined #openstack-infra | 18:35 | |
| clarkb | zaro: is newer bouncy castle a hard requirement or does it just enable features? | 18:35 |
| fungi | pleia2: there's an example in the readme | 18:35 |
| zaro | clarkb: gerrit fails to start without it | 18:36 |
| clarkb | zaro: gotcha | 18:36 |
| pleia2 | fungi: thanks | 18:36 |
| * zaro takes a break, bbl | 18:37 | |
| pleia2 | go go gadget trusty | 18:38 |
| *** zz_jgrimm has quit IRC | 18:38 | |
| *** teran has quit IRC | 18:38 | |
| *** photocyte has quit IRC | 18:38 | |
| *** teran_ has quit IRC | 18:40 | |
| *** zz_ja has joined #openstack-infra | 18:41 | |
| openstackgerrit | Merged openstack-infra/project-config: Allow access to create branch(es) https://review.openstack.org/172179 | 18:41 |
| *** maurosr has joined #openstack-infra | 18:41 | |
| * fungi loves that don adams performed the voice of inspector gadget | 18:41 | |
| fungi | <- "get smart" fan | 18:42 |
| *** zz_jgrimm has joined #openstack-infra | 18:42 | |
| *** achanda has quit IRC | 18:43 | |
| mtreinish | pleia2: you should write your own openstack client where that's how you boot from image :) | 18:43 |
| mtreinish | I would totally use that | 18:43 |
| *** Sukhdev has joined #openstack-infra | 18:43 | |
| pleia2 | hahaha | 18:43 |
| fungi | it might shadow the go compiler | 18:43 |
| fungi | (and nothing of value was lost) | 18:43 |
| mtreinish | fungi: heh, just another reason not to have that on your system | 18:44 |
| *** yamamoto has joined #openstack-infra | 18:44 | |
| harlowja | fungi after https://review.openstack.org/#/c/172179/ merges is there a wait-period before i should try pushing a branch (configs cycle every X minutes?) | 18:44 |
| harlowja | *if u know | 18:44 |
| clarkb | harlowja: ya you need to wait for puppet to update gerrit. Usually about 45 minutes now | 18:45 |
| harlowja | k | 18:45 |
| fungi | harlowja: yeah, if you like watching paint peel you can just keep reloading https://review.openstack.org/#/admin/projects/stackforge/cloud-init,access | 18:45 |
| harlowja | haha | 18:45 |
| harlowja | i like watching grass grow more | 18:45 |
| fungi | or, you know, find more productive uses of your time | 18:45 |
| *** zz_ja has quit IRC | 18:45 | |
| harlowja | impassible | 18:45 |
| *** freerunner has quit IRC | 18:46 | |
| pleia2 | cinerama: now for the fun part http://paste.openstack.org/show/202171/ | 18:46 |
| pleia2 | cinerama: server is up at 104.130.243.78, alas zanata is not | 18:47 |
| clarkb | The giant tree behind the house is in full bloom and I was outside yesterday. Today my head is hating me. I think the grass/plants need to be watched | 18:47 |
| *** maurosr has quit IRC | 18:47 | |
| *** dustins has joined #openstack-infra | 18:48 | |
| *** yamamoto has quit IRC | 18:49 | |
| cinerama | pleia2: looks like a dependency issue? what's up with the ROOT.war20150410-23922-10jsxz1.lock | 18:49 |
| *** _nadya_ has quit IRC | 18:49 | |
| pleia2 | oh, I also forgot to set up trove :\ | 18:50 |
| SpamapS | clarkb: I seem to have developed an alergy to oak tree pollin, which sucks because I am surrounded by Oak trees here in Encino, CA, which means "Oak, CA" | 18:50 |
| pleia2 | cinerama: I'll get trove set up and run the puppety things again and see where we are | 18:50 |
| pleia2 | clarkb: care to do trove things? | 18:50 |
| SpamapS | luckily they seem to be done spraying their detritus all over my neighborhood and I can almost breathe again. | 18:50 |
| clarkb | pleia2: sure, | 18:51 |
| clarkb | pleia2: want me to set it up or walk you through it? | 18:51 |
| *** zz_jgrimm has quit IRC | 18:51 | |
| pleia2 | clarkb: I don't think I've ever logged into the UI, so starting with where it is is probably good | 18:52 |
| cinerama | pleia2: i'll have two changes to land once we get things going | 18:52 |
| *** achanda has joined #openstack-infra | 18:52 | |
| clarkb | pleia2: https://mycloud.rackspace.com/ | 18:52 |
| pleia2 | cinerama: great | 18:52 |
| *** pal has joined #openstack-infra | 18:53 | |
| fungi | fwiw this _can_ be done via troveclient without touching a web browser | 18:53 |
| *** zz_ja has joined #openstack-infra | 18:53 | |
| fungi | also i have not yet tried to do so :/ | 18:53 |
| clarkb | yes ^ I have just never done it that way | 18:53 |
| clarkb | and as a general rule using *client is usually an exercise in figuring out how to work around all the bugs first | 18:53 |
| pleia2 | clarkb: are our login creditials somewhere on the puppetmaster? | 18:53 |
| clarkb | pleia2: in the file you source to boot nodes | 18:53 |
| fungi | pleia2: the same file your sourced | 18:53 |
| *** dustins has quit IRC | 18:54 | |
| pleia2 | k | 18:54 |
| clarkb | fungi: have ou troved with troveclient? | 18:54 |
| pleia2 | alright, logged in | 18:54 |
| clarkb | now I have to log in to remember what your options are | 18:54 |
| fungi | i haven't familiarized myself with troveclient yet mainly because i've usually been under the gun and it was several seconds clicking on things in the dashboard vs 10+ minutes reading and testing examples from the manpage | 18:54 |
| fungi | also, i feel pretty bad about that | 18:55 |
| *** Sukhdev has quit IRC | 18:55 | |
| clarkb | I dunno between requests cert issues, broken keystone catalogs, etc its often simply easiset to gui | 18:56 |
| *** ivar-lazzaro has joined #openstack-infra | 18:56 | |
| clarkb | pleia2: across the top is a databases tab | 18:56 |
| fungi | we browsers provide a lot of chances for the provider to just plaster over their bugs for you | 18:56 |
| fungi | er, web browsers | 18:56 |
| clarkb | pleia2: click that and pick mysql | 18:56 |
| clarkb | fungi: its true | 18:56 |
| pleia2 | clarkb: gotcha, I'm thinking we want a translate-dev-MySQL? | 18:56 |
| clarkb | pleia2: yup | 18:57 |
| fungi | or just mash random keys and whatever comes out is good enough | 18:57 |
| openstackgerrit | Stephanie Miller proposed openstack-infra/system-config: Add Zanata AJP listener https://review.openstack.org/172539 | 18:57 |
| clarkb | pleia2: and 2GB looks pretty standard for us so I would use that size | 18:57 |
| * pleia2 puts it in DFW | 18:57 | |
| fungi | i'm not personally thrilled that we've been using mixed-case and redundant terms in the database names, but whatever | 18:57 |
| *** zz_ja has quit IRC | 18:57 | |
| *** hasharAway is now known as hasharMeeting | 18:57 | |
| *** ujuc has quit IRC | 18:57 | |
| clarkb | pleia2: ya use same region as server. Then the other thing to do is select the "sanity" configuration | 18:58 |
| clarkb | also I love that its called "sanity" ++ to whoever did that | 18:58 |
| fungi | i guess the -mysql extension there is for the eventual future wherein rackspace starts offering postgresql via trove (or mangodeebee?_ | 18:58 |
| clarkb | fungi: ya | 18:58 |
| fungi | sanity was my smart-ass custom configuration name | 18:58 |
| clarkb | or drizzle or mariadb | 18:58 |
| * pleia2 does not see sanity | 18:59 | |
| *** Sukhdev has joined #openstack-infra | 18:59 | |
| *** enikanorov_ has joined #openstack-infra | 18:59 | |
| clarkb | pleia2: you might have to make the server then go back to it | 18:59 |
| pleia2 | k | 18:59 |
| fungi | oh, we need a sanity for newer mysql | 18:59 |
| * fungi checks | 18:59 | |
| pleia2 | do we add the database here too? | 18:59 |
| clarkb | pleia2: yes | 18:59 |
| clarkb | pleia2: and the user | 19:00 |
| pleia2 | k | 19:00 |
| fungi | ahh, yep the sanity configuration that's there is for mysql 5.1. i'll add a newer one | 19:00 |
| clarkb | pleia2: you can then put the host and password in hiera | 19:00 |
| pleia2 | I put the password in hiera (forgot that the hostname wouldn't be localhost), is there a special format for trove? | 19:01 |
| clarkb | pleia2: if you click on the database host you get the details for the host one of them is the hostname | 19:02 |
| clarkb | just use that fqdn | 19:02 |
| pleia2 | k | 19:02 |
| fungi | okay, there's a mysql 5.6 version of the sanity config in dfw now too | 19:02 |
| fungi | you'll want to restart the instance after switching its configuration, btw | 19:03 |
| openstackgerrit | Marton Kiss proposed openstack-infra/puppet-askbot: Update email sender address https://review.openstack.org/172540 | 19:03 |
| fungi | also make sure zanata is configured however it should be configured to expect utf8 at the database end | 19:03 |
| fungi | because apparently databases are still a decade behind modernity in this regard | 19:04 |
| * pleia2 applies sanity | 19:05 | |
| *** zz_ja has joined #openstack-infra | 19:06 | |
| *** zz_jgrimm has joined #openstack-infra | 19:06 | |
| fungi | pleia2: no matter how hard you apply sanity, you always need to apply it harder | 19:06 |
| pleia2 | and yeah, zanata wants utf8 by default, so we're good | 19:06 |
| pleia2 | :) | 19:06 |
| *** mpavone has quit IRC | 19:06 | |
| *** wenlock_ has joined #openstack-infra | 19:07 | |
| pleia2 | alright, what's the best way to run puppet again manually? | 19:07 |
| *** maurosr has joined #openstack-infra | 19:07 | |
| *** mattfarina has joined #openstack-infra | 19:07 | |
| clarkb | pleia2: `puppet agent --test` | 19:07 |
| clarkb | on the node | 19:07 |
| fungi | pleia2: as root (yes, sigh) that thing | 19:07 |
| clarkb | oh ya as root though that shouldn't be necessary unless making local mysql DBs | 19:08 |
| fungi | the puppet module for mysql does (or did) strange things under sudo | 19:08 |
| fungi | but right, probably not relevant in this particular case | 19:08 |
| pleia2 | we're not using the puppet module for mysql | 19:08 |
| pleia2 | yeah | 19:08 |
| *** vinkman has joined #openstack-infra | 19:09 | |
| *** dustins has joined #openstack-infra | 19:09 | |
| fungi | clarkb: btw, you mentioned enjoying cards against humanity a while back... they've added a "science pack" recently (i just received mine in the post). "all profits go to the Cards Against Humanity & SMBC Science Ambassador Scholarship" | 19:09 |
| fungi | (you know, for science!) | 19:10 |
| *** wenlock_ has quit IRC | 19:11 | |
| clarkb | nice | 19:11 |
| Clint | hmm | 19:12 |
| fungi | science good | 19:12 |
| *** crc32 has quit IRC | 19:13 | |
| *** dustins has quit IRC | 19:13 | |
| *** hdd has joined #openstack-infra | 19:14 | |
| *** arosen has joined #openstack-infra | 19:15 | |
| cinerama | pleia2: let me know where we're at and i can investigate paste (but i may need --debug output) | 19:15 |
| *** zz_dimtruck is now known as dimtruck | 19:17 | |
| pleia2 | cinerama: just setting up dns, will get back to looking at puppet in a moment | 19:17 |
| clarkb | armax: latest patchset to ovs with neutron and linux bridge with nova lgtm. We'll have to see how the tests do though | 19:18 |
| *** wenlock_ has joined #openstack-infra | 19:19 | |
| *** photocyte has joined #openstack-infra | 19:19 | |
| clarkb | armax: also we should add that job to neutron experimental. Do you want to write that change or should I? | 19:20 |
| *** crc32 has joined #openstack-infra | 19:20 | |
| armax | clarkb: I have run the job a couple of times, it’s been fairly happy so far..not completely though | 19:21 |
| armax | clarkb: local testing proved that I do have a sane data path | 19:21 |
| armax | clarkb: the change to make the job experimental is already cooking | 19:21 |
| armax | clarkb: but it’s depedented on Swami's | 19:21 |
| armax | clarkb: https://review.openstack.org/#/c/158415/ | 19:21 |
| reed | fungi, when you have time, https://review.openstack.org/#/c/172540 | 19:22 |
| *** eharney has quit IRC | 19:22 | |
| clarkb | armax: I think we need to make that job run against neutron though, right now its just nova, devstack, tempest and devstack-gate iirc | 19:22 |
| armax | clarkb: the only thing I left out was the setting of the MTU | 19:22 |
| armax | clarkb: is it? hang on let me double check | 19:23 |
| clarkb | armax: I think thats ok since d-g appears to already do that for br-ex later | 19:23 |
| armax | clarkb: that’s what I thought | 19:23 |
| pleia2 | cinerama: https://translate-dev.openstack.org/ is up but not working yet, browsing logs | 19:23 |
| clarkb | armax: oh I take that back it does run against neutron experimental now | 19:24 |
| *** ivar-lazzaro has quit IRC | 19:24 | |
| clarkb | armax: so we should be just about set | 19:24 |
| *** dustins has joined #openstack-infra | 19:24 | |
| armax | clarkb: oh good | 19:24 |
| fungi | clarkb: second quick set of eyes on https://review.openstack.org/172540 will get new user signup for ask.o.o working again | 19:24 |
| fungi | exim doesn't like it trying to send from a nonexistent local address | 19:24 |
| clarkb | fungi: looking | 19:24 |
| armax | clarkb: then we should be good to go and go on the next phase of our troubleshooting journey | 19:24 |
| clarkb | armax: yup | 19:25 |
| clarkb | fungi: but noreply exists locally? | 19:26 |
| * clarkb isn't super familiar with all of our exim setup | 19:27 | |
| *** wenlock_ has quit IRC | 19:27 | |
| *** HeOS has joined #openstack-infra | 19:27 | |
| fungi | nah, but openstack.org isn't local either | 19:27 |
| clarkb | oh I see | 19:28 |
| clarkb | the domain hcnage is what gets it working again | 19:28 |
| clarkb | approved | 19:28 |
| cinerama | pleia2: so i think primarily we want to be looking at the console logs and then we'll sort out the apache bits. should be able to see stuff on the local machine on port 8080 iirc if console.log looks ok | 19:28 |
| fungi | exim assumes that since the sender is openstack@ask.openstack.org then there must necessarily be an "openstack" mailbox on ask.openstack.org wher eit's running | 19:28 |
| clarkb | ya | 19:28 |
| fungi | the bug link there in the commit message has details | 19:28 |
| cinerama | pleia2: also we need to land the change adding the ajp listener, i forgot that bit yesterday | 19:29 |
| cinerama | (for apache to work) | 19:29 |
| pleia2 | cinerama: yeah, console.log is complaining about mysql stuff, so I'm looking that up now | 19:29 |
| *** sarob has joined #openstack-infra | 19:29 | |
| cinerama | pleia2, ah OK | 19:29 |
| pleia2 | cinerama: ah, I guess that explains in the apache error log? | 19:29 |
| pleia2 | [Fri Apr 10 19:24:02.961723 2015] [proxy:error] [pid 3692:tid 139732994418432] AH00959: ap_proxy_connect_backend disabling worker for (127.0.0.1) for 0s | 19:29 |
| pleia2 | [Fri Apr 10 19:24:02.961733 2015] [proxy_ajp:error] [pid 3692:tid 139732994418432] [client 2601:9:4e82:7400:853c:dd15:8104:939f:44183] AH00896: failed to make connection to backend: 127.0.0.1 | 19:29 |
| cinerama | pleia2: yeah. no open ajp port = no apache funtimes | 19:29 |
| pleia2 | gotcha | 19:30 |
| * pleia2 reviews that change | 19:30 | |
| *** cdent has joined #openstack-infra | 19:30 | |
| clarkb | cinerama: pleia2 have a link to the change I should review? | 19:31 |
| pleia2 | clarkb: https://review.openstack.org/#/c/172539/ | 19:31 |
| clarkb | armax: looks like nova net failed but only in the migration test so not a bunch of ssh issues | 19:33 |
| cinerama | we should land https://review.openstack.org/#/c/172528/ too | 19:33 |
| clarkb | armax: so I am not worried about it | 19:33 |
| *** sarob has quit IRC | 19:33 | |
| armax | clarkb: um it looks like Neutron failed the same way | 19:35 |
| clarkb | armax: so that may be a new nova bug | 19:35 |
| clarkb | superdan: https://jenkins04.openstack.org/job/check-tempest-dsvm-aiopcpu-full/124/console | 19:35 |
| *** ociuhandu has quit IRC | 19:35 | |
| clarkb | superdan: we are seeing live migration fail like that, want to point it out to you all | 19:36 |
| superdan | clarkb: like, sporadically? | 19:36 |
| pleia2 | thanks clarkb & fungi | 19:36 |
| clarkb | superdan: not sure yet how consistent it is, we can rerun it a few times | 19:36 |
| superdan | clarkb: okay | 19:36 |
| *** dustins has quit IRC | 19:36 | |
| fungi | pleia2: cinerama: feel free to add more of us as admins once you have some documentation you can point us at for things we can help with on the dev server | 19:37 |
| superdan | clarkb: is that done such that I can get at the rest of the logs? | 19:37 |
| *** isviridov has joined #openstack-infra | 19:37 | |
| *** dustins has joined #openstack-infra | 19:37 | |
| openstackgerrit | Merged openstack-infra/puppet-askbot: Update email sender address https://review.openstack.org/172540 | 19:37 |
| *** e0ne has quit IRC | 19:37 | |
| *** SumitNaiksatam has quit IRC | 19:38 | |
| *** SumitNaiksatam has joined #openstack-infra | 19:38 | |
| *** dustins has quit IRC | 19:38 | |
| clarkb | superdan: yup, one sec | 19:38 |
| *** hdd has quit IRC | 19:38 | |
| pleia2 | cinerama: bit hard to debug without apache trying to serving up zanata, so I might wait for this ajp thing to land, and I should probably eat lunch anyway :) | 19:38 |
| clarkb | superdan: http://logs.openstack.org/09/171809/6/experimental/check-tempest-dsvm-aiopcpu-full/ea64ec3/ | 19:38 |
| superdan | clarkb: oh, we've got a fix up for that already | 19:39 |
| superdan | it's #2 in the gate | 19:39 |
| clarkb | superdan: awesome thanks | 19:39 |
| superdan | basically, I suck and it's fixed now :D | 19:39 |
| clarkb | armax: ^ | 19:39 |
| *** dboik has quit IRC | 19:40 | |
| openstackgerrit | Merged openstack-infra/system-config: Add Zanata AJP listener https://review.openstack.org/172539 | 19:40 |
| cinerama | pleia2: well, step #0 is to make sure the ascii art zanata banner is appearing in /var/log/wildfly/console.log | 19:40 |
| *** hdd has joined #openstack-infra | 19:41 | |
| pleia2 | cinerama: yeah, it's not, so there certainly is stuff to do here | 19:43 |
| cinerama | pleia2: we can also look at what's on the built-in http server | 19:43 |
| cinerama | pleia2: if you put up the console log and puppet output i can have a look while you're eating | 19:44 |
| pleia2 | yeah, I do need to eat though :) | 19:44 |
| pleia2 | which is away from desk, because sanity | 19:44 |
| clarkb | I should eat too | 19:44 |
| *** e0ne has joined #openstack-infra | 19:44 | |
| pleia2 | cinerama: the puppet stuff comes out clean now (the ROOT.war is in place), putting up console output somewhere | 19:44 |
| cinerama | pleia2: thank you very much. then eat!! | 19:45 |
| pleia2 | cinerama: sent you log | 19:45 |
| *** tsg has joined #openstack-infra | 19:47 | |
| *** crc32 has quit IRC | 19:47 | |
| *** dboik has joined #openstack-infra | 19:48 | |
| cinerama | hmm...database weirdness | 19:48 |
| markmcclain | clarkb: was reading back scroll from the conversation about OVS | 19:48 |
| markmcclain | clarkb: any reason you don't want to run linuxbridge+vxlan? | 19:49 |
| clarkb | markmcclain: well I think mordred confused two issues. One is me trying to make OVS work for neutron and nova net testing and the other is what to run in an actual cloud | 19:49 |
| markmcclain | ah | 19:50 |
| clarkb | markmcclain: I think linuxbridge for actual cloud is the way I would do it particularly since we don't really need any fancy networking. We just need VMs that can get to internet | 19:50 |
| markmcclain | right | 19:50 |
| *** hdd_ has joined #openstack-infra | 19:50 | |
| *** tsg_ has quit IRC | 19:50 | |
| markmcclain | now for 2nd issue.. are you actually trying to test a feature of OVS or need to tunnel L2? | 19:50 |
| clarkb | need to tunnel L2 | 19:51 |
| fungi | pleia2: cinerama: i love that one of the troubleshooting steps here is to look for ascii art | 19:51 |
| clarkb | markmcclain: basically we have two node test envs in rax and hpcloud that need an L2 overlay so that we can put the floating IP network on that making it routable from tempest to the VMs | 19:51 |
| markmcclain | ok.. so you can also use linuxbridge there too with vxlan | 19:51 |
| clarkb | markmcclain: since rax doesn't give us control of L2 or L3 we use an overlay. HPCloud may give us enough control but it would make things slower in nodepool | 19:52 |
| cinerama | fungi: well, there's a lot of java spray in the logs but if you see the ascii art you're all good :) | 19:52 |
| hogepodge | markmcclain: I've never had much luck with vxlan in a vm testing environment, but I may be doing it wrong | 19:52 |
| clarkb | markmcclain: yes exactly thats what we use for nova net testing today | 19:52 |
| clarkb | markmcclain: except with gre | 19:52 |
| clarkb | markmcclain: but if I attach br-ex for neutron to that neutron breaks | 19:52 |
| *** tsg has quit IRC | 19:52 | |
| *** hdd has quit IRC | 19:52 | |
| clarkb | markmcclain: so we gave OVS a go, but that breaks nova net (due to the arp issues I was complaining aout) | 19:52 |
| *** mrmartin has quit IRC | 19:53 | |
| *** gordc has quit IRC | 19:53 | |
| clarkb | markmcclain: in any case we are going to move forward with nova net using linuxbridge + gre and neutron using ovs + gre | 19:53 |
| markmcclain | hogepodge: it can have a few interesting issues too, but we've found it to work | 19:53 |
| clarkb | (as for why gre instead of vxlan, whenever I read docs tehy say you need multicast for logical broadcasts which I am learning may not be the case always) | 19:53 |
| *** gordc has joined #openstack-infra | 19:53 | |
| clarkb | hogepodge: markmcclain the most likely issue is related to MTUs | 19:54 |
| markmcclain | clarkb: right so with l2pop you can avoid that of run a replication service for BUM traffic | 19:54 |
| fungi | this is part of what makes me wonder what kinds of n-net to quantron migration strategy operators can really expect besides forklift | 19:54 |
| clarkb | markmcclain: what is l2pop? | 19:54 |
| clarkb | markmcclain: oh is that ya I am not using that sorry | 19:55 |
| *** bradjones has quit IRC | 19:55 | |
| markmcclain | clarkb: yeah the you'll need to reduce the guest MTU by 54 bytes to ensure you stay under the host nets MTU | 19:55 |
| markmcclain | layer 2 prepopuliation | 19:55 |
| clarkb | markmcclain: "A switch without ARP is not useful." I stand by that statement re l2pop | 19:55 |
| *** e0ne has quit IRC | 19:56 | |
| fungi | also bum traffic. how had i never encountered that acronym before? | 19:56 |
| clarkb | for the same reasons I wish rax used dhcp | 19:56 |
| clarkb | it turns out that network engineers from many years ago solved these problems and solved them pretty well | 19:56 |
| clarkb | we should not be reinventing the wheel here with tools that work less well | 19:56 |
| markmcclain | yeah.. except dhcp services are currently prone to races | 19:57 |
| clarkb | that sounds like an implementation problem | 19:57 |
| fungi | though i think the answer there is to write a different dhcp service, not write a new protocol | 19:57 |
| dstufft | clarkb: reinventing the wheel is fun though! | 19:57 |
| clarkb | fungi: yes exactly | 19:57 |
| fungi | that. i mean yes the async nature of dhcp is inherently racy, but should be able to work around that i would think | 19:58 |
| markmcclain | fungi: I've long threaten to spawn a project I've tenatively title ragedhcp | 19:58 |
| clarkb | fungi: yes you can work around it | 19:58 |
| clarkb | fungi: one way is to lookup mappings dhcp server side in a consistent store like etcd | 19:58 |
| fungi | for example, out-of-band discussion between the virt manager and the dhcp service to maintain a table of reservations. done | 19:59 |
| fungi | the client still thinks it's begging for an assignment, but in reality there's already one waiting for it to claim | 19:59 |
| *** ivar-lazzaro has joined #openstack-infra | 19:59 | |
| fungi | anyway, rabbit hole | 19:59 |
| clarkb | fungi: you don't even have to do a reservation before hand | 19:59 |
| markmcclain | fungi: that's part of the basis of what I want to do | 20:00 |
| clarkb | you just need a consistent store of the data | 20:00 |
| fungi | yeah | 20:00 |
| markmcclain | fungi: also want to share fates better better the host and dhcp service | 20:00 |
| markmcclain | s/better better/better between/ | 20:00 |
| openstackgerrit | Russell Bryant proposed openstack-infra/reviewstats: Make local cache more permanent. https://review.openstack.org/172550 | 20:00 |
| *** hasharMeeting is now known as hashar | 20:00 | |
| clarkb | markmcclain: but at this point I am not sure where to go next | 20:00 |
| fungi | sure, i think dhcp ends up being a facade on whatever you build, but a facade that has already existing clients that know how to speak it | 20:00 |
| clarkb | fungi: yes | 20:01 |
| clarkb | markmcclain: so doing the easy thing for now to get testing in | 20:01 |
| markmcclain | clarkb: if you get GRE working then vxlan should work too | 20:01 |
| clarkb | markmcclain: with nova net? | 20:01 |
| markmcclain | because the bridge has to respond to arp | 20:01 |
| clarkb | markmcclain: which does not l2pop | 20:01 |
| *** dboik has quit IRC | 20:01 | |
| *** bradjones has joined #openstack-infra | 20:01 | |
| clarkb | markmcclain: well see thats teh tricky thing, ovs bridge kills arp | 20:01 |
| fungi | i fell asleep partway into the vxlan experiments last night. what was the upshot? | 20:02 |
| *** dboik has joined #openstack-infra | 20:02 | |
| clarkb | fungi: there really wasnt one everything still failed | 20:02 |
| fungi | oh... nice!(?) | 20:02 |
| clarkb | fungi: the issue is in the ovs flows not the tunnel protocol | 20:02 |
| markmcclain | fungi: the upshot of vxlan is that it is udp, so since you're routing over public nets any ecmp are 100% likely to work over the 5 tuple | 20:02 |
| markmcclain | for GRE only some routers are aware of the GRE tunnel key | 20:03 |
| fungi | did you get a chance to wrangle a port mirror to a virtual interface so you could sniff it? though i guess yeah the flows were showing it explicitly dropped anyway | 20:03 |
| clarkb | markmcclain: but broadcast requires multicast? | 20:03 |
| markmcclain | *ecmp capable routers | 20:03 |
| clarkb | markmcclain: and neutrons workaround for that is l2pop? | 20:03 |
| clarkb | fungi: I didn't I basically refused | 20:03 |
| fungi | right, the vxlan spec says broadcast frames map to a multicast group between your switches | 20:03 |
| markmcclain | clarkb: no with either a replication daemon or l2 prepopulation you won't need multicast | 20:04 |
| fungi | an ip multicast group i mean | 20:04 |
| clarkb | markmcclain: ok but you need some external service to the tunnel | 20:04 |
| *** tsg has joined #openstack-infra | 20:04 | |
| markmcclain | the folks at cumulus/metacloud developed this: https://github.com/metacloud/vxfld | 20:05 |
| clarkb | fungi: but also http://paste.openstack.org/show/201479/ shows ovs drop arp | 20:05 |
| fungi | it's sort of unfortunate that ovs would explicitly drop arp who-has broadcasts but not provide some means of identifying why it did so | 20:05 |
| markmcclain | essentially a service node service that uses unicast | 20:05 |
| clarkb | fungi: yes, debugging ovs in general is hard | 20:06 |
| markmcclain | useful if you don't what addresses are being used on every node | 20:06 |
| fungi | http://openvswitch.org/pipermail/discuss/2014-August/014821.html | 20:06 |
| fungi | also bug 209168 is amusing | 20:07 |
| openstack | bug 209168 in log4cxx (Ubuntu) "Timezone is not taken into account" [Wishlist,Invalid] https://launchpad.net/bugs/209168 | 20:07 |
| fungi | er, that's not a bug number | 20:07 |
| fungi | https://answers.launchpad.net/neutron/+question/209168 | 20:07 |
| clarkb | markmcclain: why not just use gre then? It does work | 20:07 |
| openstackgerrit | Russell Bryant proposed openstack-infra/reviewstats: Make local cache more permanent. https://review.openstack.org/172550 | 20:07 |
| markmcclain | clarkb: gre is fine | 20:07 |
| clarkb | markmcclain: as someone making this work, my goal is reliability, debuggability, and simplicity | 20:07 |
| clarkb | markmcclain: running linux bridge + gre fits the criteria best compared to the other options | 20:08 |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Update bouncy castle libs for Gerrit 2.10 https://review.openstack.org/172534 | 20:08 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Update query 1311066 for new warning https://review.openstack.org/171869 | 20:08 |
| fungi | comment #19 on that question looks pertinent | 20:08 |
| clarkb | fungi: except there is no neutron involved here | 20:10 |
| clarkb | I think thats a big piece everyone is missing | 20:10 |
| fungi | right, manual ovs | 20:10 |
| clarkb | OVS + gre is breaking without any neutron involved | 20:10 |
| markmcclain | clarkb: right.. the real difference between GRE and vxlan is possibly performance, but it's also possible you won't see much difference | 20:10 |
| *** tsg has quit IRC | 20:11 | |
| fungi | tried connection_mode="out-of-band" ? | 20:11 |
| clarkb | everyone that jumps in wants to say "just run with this neutron option, or use l2pop, or $neutron related thing" but there is no neutron. Its a pair of bridges manually created on different nodes talking to each other via gre or vxlan | 20:11 |
| fungi | i see some anecdotes implying that in-band control gets in the way of arp frames | 20:12 |
| clarkb | fungi: I think the controllers are in band though | 20:12 |
| *** achanda has quit IRC | 20:12 | |
| fungi | hrm. k | 20:12 |
| clarkb | fungi: the flow rules are local to the box | 20:12 |
| clarkb | and being set locally | 20:12 |
| clarkb | fungi: we can certainly try that though | 20:13 |
| markmcclain | clarkb: ok so it is orchestrated bridges that is far simpler | 20:14 |
| fungi | also poking random ml threads i'm seeing other anecdotes that make me wonder if the kernel is doing some sort of return path filtering | 20:14 |
| *** otter768 has joined #openstack-infra | 20:15 | |
| clarkb | fungi: that would be odd since the packet is killed on its first interface while egressing | 20:15 |
| *** xyang0 has quit IRC | 20:15 | |
| fungi | which would be odd, but not surprising. i've seen urpf get in teh way of layer-2 things you wouldn't expect (since in theory it's a layer 3 concept) | 20:15 |
| clarkb | we have 172.24.4.1/23 br < -- gre -- > br 172.24.4.2/23 | 20:16 |
| markmcclain | clarkb: how many nodes will be sharing the overlay net? | 20:16 |
| clarkb | when 172.24.4.1 pings 172.24.4.2 the br interface on 172.24.4.1 kills the packet | 20:16 |
| clarkb | markmcclain: currently just 2, but the existing design can accomodate an arbitrary number up to where overlays fail. Basically the bridge on the controller node is a hub and all other nodes spoke off of that | 20:17 |
| *** spzala has quit IRC | 20:17 | |
| *** dprince has quit IRC | 20:17 | |
| *** teran has joined #openstack-infra | 20:18 | |
| *** otter768 has quit IRC | 20:19 | |
| *** annegentle has quit IRC | 20:20 | |
| hogepodge | Have you set IPv4 forwarding? | 20:20 |
| *** annegentle has joined #openstack-infra | 20:20 | |
| clarkb | hogepodge: I don't believe thats related since ovs is handling the packets | 20:20 |
| fungi | i expect so or else forwarding over the gre tunnel wouldn't have worked either? | 20:21 |
| clarkb | but yes the linuxbridge version works fine | 20:21 |
| clarkb | fungi: right | 20:21 |
| pleia2 | cinerama: back | 20:21 |
| *** mattfarina has quit IRC | 20:21 | |
| cinerama | pleia2: oh hi. ok | 20:21 |
| clarkb | why don't I kick off a job run to build the env then people can poke at it | 20:22 |
| cinerama | pleia2: so it's trying to go through all these changelogs that it has for the database and it's complaining about this: | 20:22 |
| cinerama | Error executing SQL ALTER TABLE zanata.HGlossaryTerm DROP COLUMN sourceRef: Can't DROP 'sourceRef'; check that column/key exists | 20:22 |
| cinerama | pleia2: so i'm wondering what the db looks like - did the right columns etc get created | 20:22 |
| cinerama | pleia2: er tables | 20:22 |
| pleia2 | k, I'll have a look | 20:23 |
| fungi | clarkb: plus a link to simple ascii art network diagram with the intended addresses/links for clarity | 20:23 |
| *** ddieterly has quit IRC | 20:23 | |
| pleia2 | (the apache thing worked btw, shows wildfly page now) | 20:23 |
| fungi | clarkb: that sounds like a good way to invite troubleshooting | 20:23 |
| hogepodge | clarkb I think the kernel may be killing the packets, I have it enabled for my ovs/gre | 20:23 |
| clarkb | hogepodge: its ovs http://paste.openstack.org/show/201479/ | 20:24 |
| *** mestery has quit IRC | 20:24 | |
| fungi | i'm trying to pin down whether ovs defaults to not forwarding broadcast frames | 20:24 |
| fungi | but given i've barely looked in its general direction before, it's taking a lot of reading just to nail down its conventions | 20:25 |
| *** Krinkle is now known as Krinkle|detached | 20:25 | |
| clarkb | fungi: are you finding documentaton that even explains that stuff? | 20:25 |
| clarkb | fungi: I wasn't reall able to find anything beyond dev docs, some cookbooks, the command man pages and blogs | 20:25 |
| fungi | do random vendor and user blog rants count as documentation? ;) | 20:26 |
| clarkb | no | 20:26 |
| clarkb | :P | 20:26 |
| fungi | and posts to alt.flame.ovs.die.die.die | 20:26 |
| *** armax has quit IRC | 20:27 | |
| clarkb | ok I just held 1815867 in nodepool. The job running tehre just started and its running with 171417,9 which I believe to have been the most "correct" of the patchsets on that change | 20:27 |
| hogepodge | fungi you need a grey beard | 20:27 |
| clarkb | going to eat lunch now and can hand out access to that node once done | 20:28 |
| fungi | hogepodge: i'm cultivating it on top of my head. give it a few more years, the grey is coming in nicely | 20:28 |
| clarkb | er once its done running the test | 20:28 |
| clarkb | (I don't want to interfere with tempest and amke something weirder happen) | 20:28 |
| pleia2 | cinerama: once I figure out how to have a look :D | 20:28 |
| cinerama | pleia2: so at least wildfly is coming up ok, and ajp thing is ok - that's good | 20:29 |
| pleia2 | figured I could just use a mysql command with the rackspace hostname from trove, but that server doesn't have the mysql client installed (uses libmysql-java) and I'm not sure I can connect to it from outside of rackspace | 20:30 |
| pleia2 | seems to just hang when I try | 20:30 |
| cinerama | ugh | 20:30 |
| pleia2 | no ugh, just need to figure out how I'm supposed to access it :) | 20:30 |
| *** armax has joined #openstack-infra | 20:30 | |
| pleia2 | clarkb, fungi? | 20:30 |
| fungi | pleia2: quick trick, ssh into another server in that tenant and region where the mysqlclient is already installed (for example review-dev) | 20:30 |
| pleia2 | k | 20:31 |
| dstufft | or use ssh forwarding :D | 20:31 |
| *** armax has quit IRC | 20:31 | |
| clarkb | pleia2: you can connect from inside rax | 20:31 |
| pleia2 | fungi: so the issue is connecting to trove from non-rax? | 20:31 |
| fungi | pleia2: barely less quick trick, apt-get install mysql-client and then either apt-get remove --purge it when you're done or add it to the puppet manifest to make life easier for those who follow | 20:31 |
| clarkb | pleia2: correct the trove instance is on a 10.0.0.0/8 address so no routable outside of the region I think | 20:31 |
| fungi | pleia2: yes, the trove ip address is rfc-1918 off the eth1 interface i believe | 20:31 |
| pleia2 | gotcha | 20:32 |
| pleia2 | mysql> | 20:32 |
| pleia2 | woo | 20:32 |
| pleia2 | cinerama: | 20:32 |
| pleia2 | mysql> select * from HGlossaryTerm; | 20:32 |
| *** ivar-lazzaro has quit IRC | 20:32 | |
| pleia2 | Empty set (0.01 sec) | 20:32 |
| *** cdent has quit IRC | 20:33 | |
| pleia2 | so the database and schema are there, just no data | 20:33 |
| *** mestery has joined #openstack-infra | 20:33 | |
| *** ddieterly has joined #openstack-infra | 20:33 | |
| cinerama | pleia2: do you get the columns for that table? | 20:33 |
| pleia2 | cinerama: http://paste.openstack.org/show/202231/ | 20:34 |
| mtreinish | pleia2: you could use the simpleproxy module I wrote to setup a proxy to connect to the trove instance externally (but for just debug it's not too useful) | 20:34 |
| pleia2 | mtreinish: yeah, lots of ways I could do it now that I know what the problem is :) | 20:35 |
| *** hdd_ has quit IRC | 20:35 | |
| pleia2 | cinerama: so yeah, no sourceRef | 20:36 |
| cinerama | pleia2: how strange. so that table looks just like mine | 20:36 |
| pleia2 | tsk | 20:36 |
| cinerama | pleia2: but in my console log it just shows all the changelogs running successfully | 20:37 |
| cinerama | pleia2: hmm. if you do a service wildfly restart now what happens | 20:37 |
| * pleia2 tries | 20:37 | |
| *** boris-42 has quit IRC | 20:38 | |
| *** marcusvrn has quit IRC | 20:38 | |
| pleia2 | cinerama: same thing | 20:38 |
| cinerama | okay weeeeird | 20:39 |
| *** aswadr has quit IRC | 20:39 | |
| *** dimtruck is now known as zz_dimtruck | 20:39 | |
| pleia2 | computers, bah | 20:40 |
| * mtreinish hands pleia2 a slide rule | 20:42 | |
| pleia2 | \o/ | 20:42 |
| *** zz_dimtruck is now known as dimtruck | 20:43 | |
| *** kgiusti has left #openstack-infra | 20:44 | |
| fungi | straight edge and compass ftw | 20:47 |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Update bouncy castle libs for Gerrit 2.10 https://review.openstack.org/172534 | 20:49 |
| cinerama | pleia2: what do the contents of the DATABASECHANGELOG table look like? | 20:50 |
| pleia2 | cinerama: http://paste.openstack.org/show/202235/ | 20:52 |
| *** tonytan4ever has quit IRC | 20:53 | |
| *** baoli_ has quit IRC | 20:53 | |
| cinerama | pleia2: can we try emptying the existing zanata thing in trove, removing contents of /home/wildfly & /opt/wildfly & repuppet? | 20:56 |
| openstackgerrit | Khai Do proposed openstack-infra/system-config: Upgrade Gerrit from 2.8 to ver 2.10 on review.o.o https://review.openstack.org/155463 | 20:57 |
| pleia2 | cinerama: sure, that'll be fun | 20:57 |
| *** marcusvrn has joined #openstack-infra | 20:57 | |
| *** Sukhdev has quit IRC | 20:59 | |
| *** achanda has joined #openstack-infra | 20:59 | |
| *** dratushnyy has joined #openstack-infra | 21:01 | |
| *** pc_m has quit IRC | 21:02 | |
| zaro | would somebody help me figure out why puppet lint fails here? http://logs.openstack.org/34/172534/3/check/gate-puppet-gerrit-puppet-lint/9455ad4/console.html | 21:02 |
| dratushnyy | Hi all | 21:02 |
| dratushnyy | plz review | 21:02 |
| dratushnyy | https://review.openstack.org/#/c/172147/ | 21:03 |
| *** rfolco has quit IRC | 21:04 | |
| fungi | zaro: the log suggests the linter is confused and broken | 21:04 |
| fungi | zaro: my guess is you can't have colons (:) in the exec resource names | 21:04 |
| clarkb | Or in any names because : class {foo:} | 21:05 |
| fungi | zaro: also since the names of those exec resources don't contain variables, you'll want to wrap them in ' quotes not " | 21:05 |
| *** tiswanso has quit IRC | 21:06 | |
| *** mestery has quit IRC | 21:07 | |
| pleia2 | cinerama: we're right back where we were | 21:07 |
| pleia2 | looking to see if we're missing plugins or anything | 21:08 |
| cinerama | pleia2: well huh. so exact same log? | 21:08 |
| *** mestery has joined #openstack-infra | 21:08 | |
| *** bswartz has quit IRC | 21:08 | |
| pleia2 | cinerama: actually, it's dying on a new mysql bit! | 21:08 |
| cinerama | pleia2: ok, would be interesting to see | 21:08 |
| pleia2 | Reason: liquibase.exception.DatabaseException: Error executing SQL ALTER TABLE zanata.HGlossaryEntry MODIFY id BIGINT AUTO_INCREMENT: Cannot change column 'id': used in a foreign key constraint 'UKglossaryterm_glossary_entry_id' of table 'zanata.HGlossaryTerm' | 21:08 |
| *** tonytan4ever has joined #openstack-infra | 21:11 | |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Update bouncy castle libs for Gerrit 2.10 https://review.openstack.org/172534 | 21:11 |
| zaro | fungi: thanks. those logs aren't very helpful. | 21:12 |
| *** tqtran is now known as tqtran_afk | 21:12 | |
| fungi | zaro: yeah, they basically say "your puppet isn't valid enough to parse" | 21:12 |
| fungi | so not very helpful in pinpointing the cause | 21:12 |
| clarkb | pleia2: that sounds like a broken DB migration | 21:13 |
| *** annegentle has quit IRC | 21:13 | |
| *** ldnunes has quit IRC | 21:13 | |
| pleia2 | clarkb: it was a different error in a similar place last time, something is definitely going weird regarding database population | 21:14 |
| cinerama | pleia2, clarkb: i'd kind of like to try it with just a local mysql db & see if we get the same breakage | 21:14 |
| dratushnyy | is there any bug or feature request for more informative message about that situations? | 21:15 |
| *** HeOS has quit IRC | 21:16 | |
| *** zz_jgrimm is now known as jgrimm | 21:16 | |
| *** jgrimm is now known as zz_jgrimm | 21:17 | |
| clarkb | dratushnyy: no, but thats a good improvement | 21:17 |
| clarkb | fungi: ^ should address the cla thing too | 21:17 |
| *** baoli has joined #openstack-infra | 21:18 | |
| dratushnyy | think we need to make bp for it | 21:18 |
| clarkb | dratushnyy: nope, its fine as is | 21:18 |
| *** miqui has quit IRC | 21:18 | |
| dratushnyy | why? | 21:18 |
| cinerama | pleia2: https://www.redhat.com/archives/zanata-users/2015-March/msg00023.html | 21:18 |
| pleia2 | cinerama: gah | 21:19 |
| pleia2 | cinerama: truth | 21:19 |
| dratushnyy | clarkb: this is not informative | 21:19 |
| fungi | clarkb: dratushnyy: i wouldn't bother with a blueprint. i also want to find time to pickaxe back and figure out what introduced that behavior in the first place. older versions actually did echo the cla/contact info errors on -s | 21:19 |
| pleia2 | cinerama: I'll see what our options are for mysql version... | 21:19 |
| clarkb | markmcclain: fungi et al 104.239.143.189 is controller and 23.253.68.151 is compute host | 21:19 |
| cinerama | pleia2: i do not know a ton about how trove works | 21:19 |
| pleia2 | cinerama: it's using 5.6 | 21:19 |
| cinerama | pleia2: winner winner chicken dinner | 21:19 |
| clarkb | markmcclain: if you are interested in looking point me at a ssh public key and I will set it up so you can ssh in as jenkins | 21:19 |
| cinerama | pleia2: can we...not do that? :) | 21:20 |
| pleia2 | cinerama: our options are 5.1 or 5.6 | 21:20 |
| mtreinish | pleia2: maybe switch to mariadb? | 21:20 |
| fungi | mtreinish: mariadb on trove. awesome idea! | 21:20 |
| pleia2 | no idea if mariadb is supported at all :) | 21:20 |
| *** sarob has joined #openstack-infra | 21:20 | |
| pleia2 | I don't really know how their version numbers sync up | 21:20 |
| *** dboik_ has joined #openstack-infra | 21:20 | |
| mtreinish | pleia2: it's the fedora default. Is this a redhat package? if so then it might work better :) | 21:21 |
| *** dboik has quit IRC | 21:21 | |
| cinerama | https://bugzilla.redhat.com/show_bug.cgi?id=1048293 | 21:21 |
| openstack | bugzilla.redhat.com bug 1048293 in DatabaseChange "RFE/TECH DEBT: support mysql 5.6 or mariadb 10.x database for Zanata" [High,On_qa] - Assigned to sflaniga | 21:21 |
| pleia2 | cinerama: it's mariadb 10, so looks like that's no good either | 21:21 |
| *** aysyd has quit IRC | 21:21 | |
| pleia2 | mtreinish: it's a java tarball | 21:21 |
| cinerama | pleia2: "Zanata 3.6 seems to work with MariaDB 10.0 on some of our Fedora 21 dev boxes" | 21:21 |
| pleia2 | cinerama: ah | 21:22 |
| cinerama | from thee bug. we could try it | 21:22 |
| clarkb | we could 5.1 it | 21:22 |
| markmcclain | clarkb: https://launchpad.net/%7Emarkmcclain/+sshkeys (cask is preferred) | 21:22 |
| pleia2 | clarkb: that's what I'm thinking, since that's what it's tested wth | 21:22 |
| pleia2 | with too | 21:22 |
| *** mestery_ has joined #openstack-infra | 21:22 | |
| *** HeOS has joined #openstack-infra | 21:23 | |
| clarkb | markmcclain: you should have access to both nodes sshing as jenkins now | 21:23 |
| cinerama | i will defer to you both. it looks like they have a patchset for this pending so it's possible they'll release that before we go live for real | 21:24 |
| markmcclain | clarkb: thanks | 21:24 |
| clarkb | pleia2 cinerama ya I would do that should be quick and easy | 21:24 |
| clarkb | rather than trying to mariadb on ubuntu | 21:24 |
| fungi | clarkb: they communicate with each other as 10.1.0.3 and 10.1.0.5? | 21:24 |
| *** mestery has quit IRC | 21:25 | |
| cinerama | standing by | 21:25 |
| clarkb | fungi: thats the VM flat network | 21:25 |
| fungi | that much seems fine, okay | 21:25 |
| clarkb | fungi: yup that shouldn't be a problem I don't think | 21:25 |
| clarkb | fungi: its the floating ip network br_pub that has trouble | 21:26 |
| fungi | right, just starting from foundations there | 21:26 |
| *** mestery_ is now known as mestery | 21:26 | |
| fungi | so 172.24.4.1 and 172.24.4.2 | 21:26 |
| clarkb | yes | 21:26 |
| clarkb | and from there to 172.24.5.0/24 which is what the nested VMs get as IPs | 21:27 |
| fungi | but ultimately 172.24.4.1 and 172.24.4.2 should be able to reach each other and even they can't | 21:27 |
| *** alexpilotti has quit IRC | 21:27 | |
| clarkb | correct | 21:27 |
| clarkb | the simplest case of connectivity over that tunnel is 172.24.4.1 to 172.24.4.2 and vice versa | 21:27 |
| clarkb | thats just bridge to bridge with only the tunnel betwnee them. So no libvirt etc | 21:28 |
| fungi | and those br_pub interfaces are using what driver? | 21:28 |
| clarkb | fungi: its an ovs bridge | 21:28 |
| clarkb | so if you do ovs-vsctl show ou will see details | 21:28 |
| fungi | i wonder if the addresses should be on something like a vif0 bound to that bridge? | 21:28 |
| clarkb | maybe? I modeled this after how neutron expects br-ex | 21:29 |
| clarkb | it puts the IP directly on the bridge itself | 21:29 |
| pleia2 | clarkb: so should I delete this trove instance entirely and create again with mysql 5.1? | 21:29 |
| *** sarob has quit IRC | 21:29 | |
| clarkb | pleia2: its worth a shot | 21:29 |
| fungi | when i get a sec, i'll see if adding vif interfaces to the bridge and binding some unused common subnet between then solves it | 21:29 |
| clarkb | fungi: ok | 21:29 |
| pleia2 | cinerama: if I replace it's database out from under it, will it do create stuff magic on a new one when I run the puppet stuff? | 21:30 |
| clarkb | fungi: I think markmcclain may take a look later, so may want to clean up after if not | 21:30 |
| fungi | absolutely, it would be entirely temporaru | 21:30 |
| fungi | temporary | 21:30 |
| cinerama | pleia2: oh like if you give it a new blank DB to play in? i think it just initializes on startup so it SHOULD work but do you want me to mock it up real quick | 21:30 |
| fungi | i also might not get to it this evening, seeing as how it's friday | 21:30 |
| *** photocyte has quit IRC | 21:30 | |
| pleia2 | cinerama: nah, I'll just give it a try here, easy enough to blow away and recreate stuff if it's a problem | 21:31 |
| fungi | and markmcclain almost certainly has a more solid grasp of this stuff than i do | 21:31 |
| clarkb | fungi: ya thats what markmcclain said too :) its all fine, we do have a path forward for now and we can work on this stuff with less urgency | 21:31 |
| fungi | but let's make sure not to accidentally recycle these nodes | 21:31 |
| fungi | and see where the weekend takes us all | 21:31 |
| *** amitgandhinz has quit IRC | 21:31 | |
| cinerama | pleia2: i do have my vagrant box at the ready here - i'll just try it while i wait | 21:31 |
| clarkb | fungi: they are held so should stick around | 21:32 |
| *** dangers is now known as dangers_away | 21:33 | |
| fungi | i have a gear assembly for a fairly hefty stand mixer i need to figure out how to recompress without a suitable vise | 21:34 |
| *** wenlock_ has joined #openstack-infra | 21:34 | |
| fungi | so this is going to be occupying a substantial chunk of my evening i think | 21:34 |
| clarkb | fungi: zaro before you all weekend | 21:34 |
| clarkb | fungi: zaro https://review.openstack.org/#/c/171048/ that change is part of an outreachy application | 21:35 |
| clarkb | getting that in would be awesome | 21:35 |
| clarkb | pleia2: ^ you have the aweomse approve powers too | 21:36 |
| fungi | agreed. i've not been regularly reviewing jjb changes so didn't want to go approving it without visibility of one of the regulars | 21:36 |
| clarkb | ya me either but for this I figure its worthwhile | 21:37 |
| fungi | giving it a closer look now as it's been sitting there a few days | 21:37 |
| *** dratushnyy has left #openstack-infra | 21:38 | |
| *** wenlock_ has quit IRC | 21:38 | |
| rcarrillocruz | fungi, clarkb, pleia2: hiya, any chance you could +2 https://review.openstack.org/#/c/168334/ pls? already got it from jeblair | 21:40 |
| pleia2 | rcarrillocruz: on a friday afternoon? (I'm out most of next week, so I probably shouldn't :)) | 21:41 |
| openstackgerrit | Merged openstack-infra/git-review: git review -sv gets more verbose https://review.openstack.org/172147 | 21:41 |
| clarkb | rcarrillocruz: the problem is if puppetdb breaks all of puppet does | 21:42 |
| pleia2 | cinerama: I see ascii art :D :D | 21:42 |
| clarkb | rcarrillocruz: I handily discovered this during heartbleed cleanup | 21:42 |
| pleia2 | https://translate-dev.openstack.org/ | 21:42 |
| pleia2 | winnar \o/ | 21:42 |
| cinerama | pleia2: excellent! | 21:42 |
| clarkb | rcarrillocruz: have we done any testing of the change? | 21:42 |
| pleia2 | cinerama: now that we got it this far, I may have to pass this off now, I have a ton to do before I leave for the conference at 7AM Monday morning :\ | 21:43 |
| pleia2 | like, write my talk | 21:43 |
| cinerama | pleia2: oh wow heh | 21:43 |
| cinerama | pleia2: i'm just going to see if i can finish setting up as admin | 21:43 |
| clarkb | cinerama: pleia2 woot! | 21:44 |
| clarkb | fwiw I can't make a new user yet, guessing additional setup is required | 21:44 |
| rcarrillocruz | i've done locally, but i don't know if any of the openstack-root/cores have done it on infra dev envs | 21:44 |
| pleia2 | cinerama: sure | 21:44 |
| clarkb | rcarrillocruz: and when you did it locally you did an upgrade? | 21:44 |
| cinerama | i'm just looking to see if i got the mail. checking spam... | 21:45 |
| *** dimtruck is now known as zz_dimtruck | 21:45 | |
| cinerama | great spam subject: "Stop The Battle... with Alcohol." | 21:45 |
| fungi | alcohol stops most of my battles | 21:45 |
| cinerama | best ellipsis ever | 21:45 |
| *** mriedem is now known as mriedem_away | 21:45 | |
| pleia2 | haha | 21:46 |
| clarkb | looks like fungi is taking the puppetdb plunge :) | 21:47 |
| fungi | i un-plunged when you reminded me in here that it also runs on the puppetmaster | 21:47 |
| cinerama | pleia2: okay, so i got redirected appropriately but i never got my registration email with the link to click to confirm. | 21:47 |
| clarkb | fungi: ah | 21:47 |
| *** armax has joined #openstack-infra | 21:48 | |
| clarkb | ya my only concern doing this is if puppetdb breaks it breaks the master | 21:48 |
| fungi | but am around if i need to switch gears and debug. i think having it tank the puppetmaster is also tractable to debug and shouldn't take down other services in its wake | 21:48 |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Update bouncy castle libs for Gerrit 2.10 https://review.openstack.org/172534 | 21:48 |
| clarkb | which means it needs to happen when I am not distracted by many other things | 21:48 |
| *** stevemar has joined #openstack-infra | 21:48 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Add support for Scoverage plugin https://review.openstack.org/171048 | 21:48 |
| clarkb | fungi: correct other services will stay up they just won't puppet | 21:48 |
| fungi | i kinda feel like giving it a shot and will take my chops if it means a good chunk of my evening to unwind it again | 21:48 |
| pleia2 | cinerama: hm, exim is installed and I don't see anything in the mail queue, might require some more digging | 21:49 |
| fungi | pleia2: check teh rejectlog | 21:49 |
| pleia2 | fungi: there's only a mainlog, and nothing there | 21:49 |
| fungi | ahh, yeah then it's not making it as far as exim | 21:49 |
| cinerama | pleia2: oh crap i bet i know what it is | 21:49 |
| pleia2 | cinerama: oh good, that means we can fix it! | 21:50 |
| cinerama | pleia2: we need a default from address set up | 21:50 |
| *** _nadya_ has joined #openstack-infra | 21:50 | |
| clarkb | fungi: ok, I can mostly be around to help as well | 21:50 |
| cinerama | pleia2: if you have one we can use or if i should point it at hiera let me know | 21:50 |
| clarkb | fungi: should I/you/we reapprove? | 21:50 |
| fungi | i just did | 21:51 |
| pleia2 | cinerama: we hardcode addresses in jenkins_dev.pp and stuff, should be ok | 21:51 |
| cinerama | pleia2: what address should i use? | 21:51 |
| pleia2 | but others can chime in :) | 21:51 |
| *** mriedem_away has quit IRC | 21:52 | |
| pleia2 | cinerama: maybe zanata@translate-dev.openstack.org ? (or whatever username@ tends to be default in zanata, if there is one) | 21:52 |
| clarkb | cinerama: pleia2 we just used norepl@openstack.org | 21:52 |
| *** wenlock_ has joined #openstack-infra | 21:52 | |
| pleia2 | that works too | 21:52 |
| clarkb | er used ^ on ask.o.o | 21:52 |
| pleia2 | noreply@ | 21:52 |
| fungi | yup | 21:52 |
| *** tonytan4ever has quit IRC | 21:53 | |
| openstackgerrit | Merged openstack-infra/system-config: Bump puppetdb version https://review.openstack.org/168334 | 21:53 |
| fungi | i've left myself ssh'd into puppetmaster.o.o on the extreme off-chance that ^ causes it to start blocking new ssh sessions for some reason | 21:53 |
| clarkb | fungi: I don't expect that its the agent <-> master interaction that fails when puppetdb goes down | 21:53 |
| clarkb | fungi: because the agent can't report its run logs to the db | 21:54 |
| clarkb | ssh should be fine | 21:54 |
| fungi | yeah, it's also upgrading the firewall module too, which i think is doing nothing in this case anyway, so just a precaution | 21:54 |
| clarkb | oh right that | 21:54 |
| *** _nadya_ has quit IRC | 21:54 | |
| fungi | not being able to get into the server to fix it when it goes sideways is generally my biggest fear | 21:55 |
| *** gordc has quit IRC | 21:55 | |
| fungi | as long as i can log in, i can generally stab blindly until it is coerced back into working order again | 21:55 |
| fungi | (accurate depiction of my 31337 troubleshooting sk1llz) | 21:56 |
| *** bradjones has quit IRC | 21:56 | |
| openstackgerrit | Stephanie Miller proposed openstack-infra/system-config: Add default from address https://review.openstack.org/172578 | 21:57 |
| cinerama | of course the annoying bit is that we might have to do surgery on my account in the database | 21:57 |
| *** harlowja is now known as harlowja_away | 21:58 | |
| *** soren_ has quit IRC | 21:59 | |
| *** wenlock_ has quit IRC | 22:01 | |
| cinerama | at least that's documented | 22:01 |
| *** mrunge has quit IRC | 22:01 | |
| pleia2 | small victories :) | 22:02 |
| *** bhunter71 has quit IRC | 22:02 | |
| *** isviridov has quit IRC | 22:04 | |
| *** bradjones has joined #openstack-infra | 22:04 | |
| *** tqtran_afk is now known as tqtran | 22:06 | |
| *** pal has quit IRC | 22:07 | |
| *** stevemar has quit IRC | 22:07 | |
| *** annegentle has joined #openstack-infra | 22:08 | |
| *** duran has quit IRC | 22:08 | |
| *** andreykurilin__ has joined #openstack-infra | 22:08 | |
| cinerama | anyway so that change I just put up needs to be merged for things to work | 22:08 |
| *** bknudson has quit IRC | 22:09 | |
| clarkb | ok will review shortly | 22:09 |
| *** wenlock_ has joined #openstack-infra | 22:10 | |
| *** bswartz has joined #openstack-infra | 22:10 | |
| openstackgerrit | Stephanie Miller proposed openstack-infra/system-config: Add admin users https://review.openstack.org/172528 | 22:11 |
| cinerama | i also rebased the admin users change but i'm wondering who the admin users should be | 22:12 |
| *** Swami has quit IRC | 22:12 | |
| *** armax has quit IRC | 22:12 | |
| *** wenlock_ has quit IRC | 22:14 | |
| *** thedodd has quit IRC | 22:15 | |
| *** otter768 has joined #openstack-infra | 22:16 | |
| jogo | pleia2: I never got an email to vote in the nova election | 22:16 |
| clarkb | jogo: check spam? | 22:16 |
| *** wenlock_ has joined #openstack-infra | 22:16 | |
| clarkb | cinerama: I approved with one +2 because its just a rebase | 22:17 |
| mtreinish | jogo: maybe you aren't eligable? :) | 22:17 |
| mtreinish | jogo: or maybe the joe gordon in pittsburg got it? | 22:18 |
| *** sambetts has quit IRC | 22:19 | |
| fungi | jogo: the roll i provided to the officials has your gmail address showing as your preferred email in gerrit | 22:20 |
| *** marun has quit IRC | 22:20 | |
| *** otter768 has quit IRC | 22:20 | |
| *** sambetts has joined #openstack-infra | 22:21 | |
| *** harlowja_away is now known as harlowja | 22:21 | |
| *** yamamoto has joined #openstack-infra | 22:22 | |
| *** wenlock_ has quit IRC | 22:25 | |
| jogo | clarkb: I did | 22:26 |
| *** yamamoto has quit IRC | 22:26 | |
| jogo | clarkb: I checked | 22:26 |
| jogo | I'll check again | 22:26 |
| jogo | nope | 22:27 |
| openstackgerrit | Merged openstack-infra/system-config: Add admin users https://review.openstack.org/172528 | 22:27 |
| pleia2 | jogo: tristan.cacqueray@enovance.com is the poll supervisor, so that's who the mail would come from | 22:27 |
| jogo | mtreinish: I am eligable I7618050b9ca28690b55c8c2424d04b40bd879442 hah! | 22:28 |
| pleia2 | I don't have access to the poll directly, so all I can do is check the address (as fungi already did) and forward to him | 22:28 |
| *** andreykurilin__ has quit IRC | 22:28 | |
| *** andreykurilin__ has joined #openstack-infra | 22:28 | |
| jogo | pleia2: I don't see any thing from him | 22:28 |
| fungi | tristanC might have some visibility into it (if he's still awake)? | 22:29 |
| mtreinish | jogo: fwiw, I got mine and voted already | 22:29 |
| fungi | i received (and used) mine already, so i know it's working | 22:29 |
| fungi | mine came in around 16:52 utc, and i'm assuming they were all sent at the same time | 22:30 |
| clarkb | fungi: http://puppetboard.openstack.org/report/ask.openstack.org/c3d1c339eefff96e3099992a33c604da524aff56 is that a known thing? | 22:30 |
| *** hashar has quit IRC | 22:31 | |
| *** zz_dimtruck is now known as dimtruck | 22:31 | |
| clarkb | and I think the next run for http://puppetboard.openstack.org/node/puppetdb.openstack.org should be with the upated module | 22:31 |
| clarkb | so its about 30 minute out | 22:31 |
| fungi | clarkb: yes, known issue. i believe there's a patch proposed to fix it | 22:31 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/shade: Test volume list caching https://review.openstack.org/172586 | 22:33 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/shade: Unsteady state in volume list should prevent cache https://review.openstack.org/172587 | 22:33 |
| jogo | hmm I will email tristanC | 22:33 |
| clarkb | fungi: I don't see any open puppet-askbot changes, do you know wher the fix is proposed? | 22:34 |
| fungi | clarkb: https://review.openstack.org/171600 | 22:35 |
| *** sabeen3 has quit IRC | 22:35 | |
| clarkb | oh its a system-config change | 22:35 |
| fungi | yeah | 22:36 |
| fungi | nibalizer had some input on an earlier patchset | 22:36 |
| *** baoli has quit IRC | 22:39 | |
| *** markvoelker has quit IRC | 22:40 | |
| clarkb | ok reviewed | 22:42 |
| *** amitgandhinz has joined #openstack-infra | 22:42 | |
| clarkb | file resources don't understand refreshes, so that change is basically a noop. I suggested a simple workaround to the problem though | 22:42 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/shade: Invalidate volume list cache when creating https://review.openstack.org/172595 | 22:44 |
| * SpamapS gets deeper into the weeds | 22:44 | |
| *** annegentle has quit IRC | 22:45 | |
| *** annegentle has joined #openstack-infra | 22:45 | |
| *** wenlock_ has joined #openstack-infra | 22:46 | |
| *** amitgandhinz has quit IRC | 22:46 | |
| *** dimtruck is now known as zz_dimtruck | 22:48 | |
| *** bradjones has quit IRC | 22:50 | |
| *** erlon has quit IRC | 22:51 | |
| *** EmilienM is now known as EmilienM|afk | 22:56 | |
| *** wenlock_ has quit IRC | 22:56 | |
| *** bradjones has joined #openstack-infra | 22:57 | |
| *** zz_dimtruck is now known as dimtruck | 23:01 | |
| *** sputnik13 has quit IRC | 23:02 | |
| *** Svedrin has quit IRC | 23:02 | |
| *** sputnik13 has joined #openstack-infra | 23:03 | |
| *** Svedrin has joined #openstack-infra | 23:04 | |
| *** annegentle has quit IRC | 23:04 | |
| clarkb | fungi: puppetdb module doesn't appear to have updated | 23:05 |
| clarkb | fungi: the firewall module did update though | 23:05 |
| clarkb | rcarrillocruz: if you are awake ^ an idea why that would happen? | 23:06 |
| clarkb | I am guessing that the puppet module install failed | 23:06 |
| *** dboik has joined #openstack-infra | 23:07 | |
| clarkb | we do update git before running install modules so I don't think its an order of ops thing | 23:08 |
| clarkb | I am going to run install_modules by hand as root | 23:09 |
| *** avozza is now known as zz_avozza | 23:09 | |
| *** zz_avozza is now known as avozza | 23:10 | |
| clarkb | http://paste.openstack.org/show/202297/ looks like a good thing to sort out on monday | 23:10 |
| clarkb | fungi: ^ | 23:10 |
| *** dboik_ has quit IRC | 23:10 | |
| clarkb | rcarrillocruz: ^ | 23:10 |
| fungi | ahh, yeah that' | 23:11 |
| fungi | s not great | 23:11 |
| fungi | why would it upgrade firewall but not puppetdb? | 23:12 |
| clarkb | fungi: I think firewall was already at the correct version | 23:12 |
| clarkb | via some transitive dep or something. | 23:12 |
| fungi | oh, that would explain it | 23:12 |
| clarkb | we likely want to resurrect my old change to manage all modules directly via git | 23:12 |
| *** dimtruck is now known as zz_dimtruck | 23:12 | |
| clarkb | and just curate the list since it is relatively small | 23:13 |
| fungi | agreed | 23:15 |
| clarkb | ok anything else worht looking at before we declare it the weekend? | 23:20 |
| clarkb | cinerama: ^ | 23:20 |
| *** ddieterly has quit IRC | 23:20 | |
| *** andreykurilin__ has quit IRC | 23:21 | |
| pleia2 | https://review.openstack.org/#/c/172578/ should be ok to go in once it's rebased | 23:23 |
| *** wenlock_ has joined #openstack-infra | 23:23 | |
| *** Svedrin has quit IRC | 23:25 | |
| *** zz_dimtruck is now known as dimtruck | 23:25 | |
| *** davideagnello has quit IRC | 23:26 | |
| *** gyee has quit IRC | 23:26 | |
| *** Svedrin has joined #openstack-infra | 23:28 | |
| *** wenlock_ has quit IRC | 23:28 | |
| *** wenlock_ has joined #openstack-infra | 23:29 | |
| *** avozza is now known as zz_avozza | 23:29 | |
| *** sarob has joined #openstack-infra | 23:32 | |
| *** sarob has quit IRC | 23:33 | |
| *** wenlock_ has quit IRC | 23:33 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/shade: Deprecate use of cache in list_volumes https://review.openstack.org/172599 | 23:33 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/shade: Add test for invalidation after delete https://review.openstack.org/172600 | 23:33 |
| SpamapS | Shrews: ^^ pretty sure those two are crazy sauce and I will end up WIP'ing them or abandoning .. but take a look at how I"m thinking. ;) | 23:34 |
| * SpamapS starts the weekend | 23:34 | |
| *** ajmiller has quit IRC | 23:34 | |
| *** claudiub_ has quit IRC | 23:36 | |
| *** achanda has quit IRC | 23:36 | |
| *** markvoelker has joined #openstack-infra | 23:39 | |
| *** zz_avozza is now known as avozza | 23:41 | |
| *** achanda has joined #openstack-infra | 23:42 | |
| *** jerryz has joined #openstack-infra | 23:43 | |
| *** dimtruck is now known as zz_dimtruck | 23:48 | |
| *** dboik has quit IRC | 23:49 | |
| fungi | less so for me. people seem to have picked tonight to report vulnerabilities in etherpad | 23:50 |
| fungi | though i guess i knew what i was getting into when i volunteered to help them with that, so i don't have much room to complain | 23:51 |
| mtreinish | fungi: bah, security w/ etherpad. People should just encrypt what they put in the pads if they care :) | 23:53 |
| fungi | touche! | 23:53 |
| fungi | this is more like security of the server where etherpad is running unfortunately | 23:53 |
| mtreinish | oh, ok. Yeah that's no good | 23:53 |
| JayF | fungi: are the fixes embargoed or do you have a link? | 23:54 |
| JayF | fungi: I'd like to notify whoever runs our internal etherpad here | 23:54 |
| mtreinish | fungi: although I guess you could say containers and wave your hands :) | 23:54 |
| mtreinish | frantically | 23:54 |
| fungi | JayF: http://www.openwall.com/lists/oss-security/2015/04/10/11 | 23:54 |
| fungi | if there were other embargoed reports, i wouldn't be able to say whether they existed | 23:55 |
| fungi | though i would suggest keeping an eye on their git repo | 23:55 |
| JayF | fungi: cool, ty for the link. | 23:55 |
| JayF | that's not true for everything though, right? | 23:55 |
| mtreinish | fungi: ooh, yeah that exploit looks fun | 23:55 |
| JayF | Xenserver admits when it has embargoed fixes even if not what they are | 23:56 |
| JayF | Which is nice since when we have to reboot the cloud, our customers being able to know why is a good thing | 23:56 |
| fungi | sort of. they give their downstreams a heads up before they announce more broadly that they're planning to release a serious security fix, i believe | 23:56 |
| fungi | for xen i mean | 23:56 |
| *** sputnik13 has quit IRC | 23:57 | |
| JayF | fungi: you're probably right; I'm not the person who would get the extra-early ping in that case :) | 23:57 |
| * JayF very glad to be working without a hypervisor | 23:57 | |
| fungi | i wouldn't be surprised if it's just the linux distro xen packagers/security teams that get the extra early warning | 23:57 |
| JayF | Yeah, I'm not sure, but I'm happy to not know | 23:58 |
| JayF | knowing is responsibility :) | 23:58 |
| fungi | it is at that | 23:59 |
| * persia suspects stuff gets posted to distros@lists.openwall.org | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!