| *** jamielennox|away is now known as jamielennox | 00:00 | |
| odyssey4me | stevelle ah, fair enough - so you're skipping the morning workout now? or starting even earlier? | 00:01 |
|---|---|---|
| stevelle | starting earlier, in the dark | 00:02 |
| stevelle | sudden shift in light cycles | 00:02 |
| odyssey4me | ugh, as a non-morning person I just cannot do that | 00:03 |
| odyssey4me | I have tried many times, and failed | 00:03 |
| odyssey4me | anyone else having trouble with gerrit - specifically POST ops | 00:03 |
| cloudnull | [19:03] <clarkb> amrith: daily db backups | 00:04 |
| cloudnull | likely a table lock | 00:04 |
| openstackgerrit | Steve Lewis (stevelle) proposed openstack/openstack-ansible-os_ceilometer: Serve the ceilometer-api using mod_wsgi https://review.openstack.org/370352 | 00:05 |
| stevelle | ^ that should fix the ceilometer role | 00:05 |
| stevelle | looks like I figured out my wsgi problem | 00:05 |
| stevelle | had to rtfm | 00:06 |
| * odyssey4me holds thumbs | 00:06 | |
| odyssey4me | I'm pretty much thinking of doing a role/SHA pin first thing in the morning, so I'd like to get any essential fixes for release done asap. | 00:07 |
| odyssey4me | the tests can be improved and backported, so I'm not too worried about those | 00:07 |
| *** schwicht has quit IRC | 00:07 | |
| odyssey4me | but functional brokenness is something we need to care about, and once we tag an RC only essential fixes can be backported into the branch | 00:08 |
| odyssey4me | at least until the first hotfix tag | 00:08 |
| stevelle | if I can get 37035 and 366931 merged for RC that would be ideal, and the latter requires 370415 before it's going to even gate | 00:10 |
| odyssey4me | hmm, it looks like backups are interrupting tests | 00:10 |
| odyssey4me | git failures and such | 00:10 |
| stevelle | yeah, Update py_pkgs died | 00:10 |
| odyssey4me | I have https://review.openstack.org/#/c/366931 marked | 00:11 |
| *** adrian_otto has quit IRC | 00:11 | |
| odyssey4me | and yeah, https://review.openstack.org/#/c/370415/ needs another review | 00:11 |
| stevelle | I'm running my own full test of 366931 with 370415 patched in. the last one was what I used to debug and ended up with an inconsistent state in haproxy | 00:12 |
| odyssey4me | 37035 appears to be the wrong # - what's the review you meant to refer to? | 00:12 |
| stevelle | 370352 | 00:12 |
| odyssey4me | ok, got those all marked for follow up | 00:14 |
| *** adrian_otto has joined #openstack-ansible | 00:14 | |
| *** thorst has joined #openstack-ansible | 00:17 | |
| odyssey4me | stevelle a few nits on the mod_wsgi one, but happy with it as-is | 00:21 |
| odyssey4me | dunno if you'd like to clean those up quickly? | 00:21 |
| stevelle | can | 00:21 |
| *** schwicht has joined #openstack-ansible | 00:23 | |
| openstackgerrit | Steve Lewis (stevelle) proposed openstack/openstack-ansible-os_ceilometer: Serve the ceilometer-api using mod_wsgi https://review.openstack.org/370352 | 00:24 |
| *** thorst has quit IRC | 00:24 | |
| openstackgerrit | Merged openstack/openstack-ansible-repo_build: Update py_pkgs output data for updated lookup plugin https://review.openstack.org/370432 | 00:26 |
| odyssey4me | https://review.openstack.org/370282 needs a review, any cores still around | 00:28 |
| cloudnull | odyssey4me: done | 00:30 |
| openstackgerrit | Merged openstack/openstack-ansible-plugins: Fix proprietary regression in py_pkgs.py https://review.openstack.org/370420 | 00:30 |
| stevelle | beat me to it | 00:30 |
| odyssey4me | cloud another one for you: https://review.openstack.org/370415 | 00:31 |
| odyssey4me | cloudnull | 00:31 |
| *** schwicht has quit IRC | 00:32 | |
| cloudnull | yes? | 00:33 |
| odyssey4me | needs review: https://review.openstack.org/370415 | 00:33 |
| cloudnull | done | 00:34 |
| odyssey4me | once that's in, then all we need is https://review.openstack.org/366931 to be rechecked to validate that it works as planned | 00:35 |
| *** hblah has joined #openstack-ansible | 00:36 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create: Change the container prep using more intelligent commands https://review.openstack.org/354120 | 00:36 |
| odyssey4me | so far only 4 projects have pushed RC1 reviews, and most are marked WIP | 00:36 |
| odyssey4me | the deadline is tomorrow, so I expect a rush of them | 00:37 |
| odyssey4me | well, the deadline is later today | 00:37 |
| cloudnull | odyssey4me: updated based on your feedback https://review.openstack.org/354120 | 00:37 |
| cloudnull | re the command bit | 00:39 |
| cloudnull | i can write a longer if statement | 00:39 |
| odyssey4me | cloudnull not essential, I just figured it made for a nice test | 00:40 |
| odyssey4me | but meh | 00:40 |
| *** schwicht has joined #openstack-ansible | 00:40 | |
| odyssey4me | LGTM waiting for gate results | 00:40 |
| odyssey4me | cloudnull do we definitely want that change in Newton? | 00:41 |
| *** jlvillal has quit IRC | 00:41 | |
| odyssey4me | I expect it may be a little faster and perhaps more reliable. | 00:41 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create: Change the container prep using more intelligent commands https://review.openstack.org/354120 | 00:41 |
| cloudnull | i for sure would like it | 00:41 |
| cloudnull | but its not essential | 00:42 |
| cloudnull | it rounds out our opertunistic execution story | 00:42 |
| cloudnull | which should mean that things are more reliable | 00:43 |
| cloudnull | in terms of deployment | 00:43 |
| odyssey4me | yup, it'd be nice to try and get hold of an OSIC cluster to test it all out on | 00:43 |
| odyssey4me | perhaps benchmark some deployment times - liberty, vs mitaka, vs newton | 00:43 |
| *** johnmilton has quit IRC | 00:44 | |
| cloudnull | we should apply for access | 00:44 |
| cloudnull | https://osic.org/clusters | 00:44 |
| stevelle | are we going to use a heat template to orchestrate it? :D | 00:45 |
| cloudnull | no. | 00:46 |
| cloudnull | ^ not funny | 00:46 |
| cloudnull | :) | 00:46 |
| openstackgerrit | Merged openstack/openstack-ansible-os_gnocchi: Install required packages in identity only mode https://review.openstack.org/370415 | 00:46 |
| *** jlvillal has joined #openstack-ansible | 00:46 | |
| *** jlvillal has quit IRC | 00:47 | |
| *** jlvillal has joined #openstack-ansible | 00:47 | |
| *** jlvillal has quit IRC | 00:47 | |
| odyssey4me | yeah, I meant to apply for one via irtermite, but forgot | 00:47 |
| *** jlvillal has joined #openstack-ansible | 00:47 | |
| odyssey4me | methinks I should apply tomorrow - book a cluster for both deployment benchmarking and for the upcoming rolling upgrade work | 00:48 |
| cloudnull | that'd be a good thing | 00:49 |
| cloudnull | and i know some folks that manage that env ;) so deployment should be fairly simple. | 00:49 |
| odyssey4me | yep, the sooner the better - I'll do an application in the morning | 00:49 |
| cloudnull | cool | 00:49 |
| stevelle | working on review for the lxc_container_create patch now | 00:50 |
| odyssey4me | ok, I think we're looking good for an RC assuming the telemetry stuff works | 00:51 |
| odyssey4me | I'm heading to bed - will pick up follow through on any more things in the morning. | 00:51 |
| odyssey4me | Thanks stevelle and cloudnull for persisting. I know we're all tired - we're on the last mile now. | 00:51 |
| cloudnull | np | 00:52 |
| cloudnull | stevelle jmccrory: any chance you have a moment to review https://review.openstack.org/#/c/354120 | 00:52 |
| stevelle | cloudnull: working on it now, said ^ | 00:52 |
| cloudnull | oh. sorry missed that. | 00:53 |
| cloudnull | thanks :) | 00:53 |
| stevelle | I think this is the last one in my queue for tonight, besides the telemetry ones | 00:53 |
| cloudnull | odyssey4me: the placement db is coming so that's a change we'll need to get into the RC cycle. | 00:53 |
| openstackgerrit | Adam Reznechek proposed openstack/openstack-ansible-os_nova: Fix novnc console include positioning https://review.openstack.org/370501 | 00:54 |
| cloudnull | related https://review.openstack.org/#/c/367749/ | 00:54 |
| cloudnull | wont work yet as its waiting on nova code to merge. | 00:54 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_neutron: Add functional testing for Calico https://review.openstack.org/368287 | 00:55 |
| adreznec | If 370501 could merge pre-RC, that'd be awesome... in my testing today novnc console install is broken without it because it's trying to use the nova venv before the validate/download step in nova_install | 00:56 |
| cloudnull | today was a good day for cloud1 https://cloud1.osic.org/grafana/dashboard/db/openstack-all-compute-aggregates | 00:56 |
| cloudnull | adreznec: already +2'd | 00:56 |
| adreznec | Awesome | 00:56 |
| adreznec | We'll be using novnc in our CI, so once that's up it should catch stuff down the novnc path in the future | 00:57 |
| cloudnull | ++ | 00:57 |
| * stevelle adds that to the queue | 00:57 | |
| *** johnmilton has joined #openstack-ansible | 00:57 | |
| cloudnull | i'd be nice to switch that to the default | 00:57 |
| cloudnull | seems spice has not progressed. | 00:58 |
| cloudnull | we use novnc in the OSIC cloud1 | 00:58 |
| openstackgerrit | Merged openstack/openstack-ansible-ops: Remove osa-differ https://review.openstack.org/370234 | 00:58 |
| adreznec | Ah ok, I wasn't sure if anyone else was really using novnc since it was broken in Newton | 00:59 |
| adreznec | Glad to hear we're not complete outliers :) | 00:59 |
| cloudnull | not at all. | 00:59 |
| cloudnull | cloud1 is still liberty | 01:00 |
| stevelle | cloudnull: you have any idea what the time delta is on the lxc-create role with this? | 01:00 |
| cloudnull | we'll be upgrading to mitaka in the next few weeks. | 01:00 |
| cloudnull | stevelle: hard to say really. the biggest slowdown is when ssh hangs which wont be an issue during the container setup process. | 01:01 |
| cloudnull | but if you had happy path on both patches I'd imagine that they'd be similar in execution time | 01:01 |
| cloudnull | however should ssh hang or time out to the contianer, like when there are lots of container builds happening on a single host, the original method would be far slower. | 01:02 |
| openstackgerrit | Merged openstack/openstack-ansible-tests: Fix container destroy host group https://review.openstack.org/370282 | 01:03 |
| cloudnull | so like i said, hard to say really. | 01:03 |
| stevelle | feels like an improvement for larger deploys then if I understand | 01:03 |
| cloudnull | not really larger, maybe more dense. | 01:03 |
| cloudnull | like in the aio when there are 32 containers being spun up, this should be faster. | 01:03 |
| adreznec | Yeah, I don't think it would have much impact on a larger-scale deployment across a larger number of nodes? | 01:04 |
| cloudnull | but if you had 100 hosts and spread the containers out nicely there may be no improvement. | 01:04 |
| stevelle | I don't assume stuff gets spread out much I guess | 01:04 |
| cloudnull | it'd be opertunistically better, but nothing major in that case. | 01:04 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_nova: Compress test execution logs https://review.openstack.org/367886 | 01:07 |
| openstackgerrit | Merged openstack/openstack-ansible-ceph_client: Support deployment of extra ceph.conf files https://review.openstack.org/370431 | 01:08 |
| *** itsuugo has quit IRC | 01:08 | |
| *** itsuugo has joined #openstack-ansible | 01:10 | |
| *** schwicht has quit IRC | 01:13 | |
| *** schwicht has joined #openstack-ansible | 01:14 | |
| *** itsuugo has quit IRC | 01:15 | |
| stevelle | cloudnull: one more Q in the review | 01:15 |
| *** schwicht has quit IRC | 01:15 | |
| *** itsuugo has joined #openstack-ansible | 01:16 | |
| *** adrian_otto has quit IRC | 01:17 | |
| *** thorst has joined #openstack-ansible | 01:21 | |
| *** itsuugo has quit IRC | 01:22 | |
| *** itsuugo has joined #openstack-ansible | 01:22 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_ceilometer: Serve the ceilometer-api using mod_wsgi https://review.openstack.org/370352 | 01:23 |
| *** itsuugo has quit IRC | 01:27 | |
| *** itsuugo has joined #openstack-ansible | 01:28 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron: Add functional testing for Calico https://review.openstack.org/368287 | 01:29 |
| *** thorst has quit IRC | 01:29 | |
| *** thorst has joined #openstack-ansible | 01:35 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create: Change the container prep using more intelligent commands https://review.openstack.org/354120 | 01:36 |
| cloudnull | stevelle: ^ updated to remove the retries. | 01:36 |
| stevelle | sweet | 01:37 |
| cloudnull | jmccrory: automagically: hughsaunders: andymccr: odyssey4me: mattt: evrardjp: it'd be great to get some core love on many of these inflight reviews. | 01:39 |
| cloudnull | if you're still around . | 01:39 |
| cloudnull | specifically https://review.openstack.org/#/q/starredby:cloudnull+status:open,n,z | 01:41 |
| *** schwicht has joined #openstack-ansible | 01:41 | |
| *** thorst has quit IRC | 01:45 | |
| *** itsuugo has quit IRC | 01:48 | |
| jmccrory | cloudnull : looks like they most have approvals, need help tracking down ci failures? | 01:49 |
| cloudnull | yup | 01:49 |
| jmccrory | ah ok, taking a look through | 01:50 |
| *** itsuugo has joined #openstack-ansible | 01:50 | |
| *** jwagner is now known as jwagner[away] | 01:50 | |
| stevelle | well, 366931 passed in my local env, may timeout the gate though | 01:51 |
| stevelle | digging into 370150 | 01:54 |
| *** itsuugo has quit IRC | 01:55 | |
| *** itsuugo has joined #openstack-ansible | 01:57 | |
| *** itsuugo has quit IRC | 02:08 | |
| *** itsuugo has joined #openstack-ansible | 02:09 | |
| stevelle | welp, no that one is harder to repro | 02:13 |
| *** jwagner[away] is now known as jwagner | 02:13 | |
| *** jwagner is now known as jwagner[away] | 02:14 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible: Cleanup/standardize usage of tags in plays https://review.openstack.org/340977 | 02:16 |
| *** itsuugo has quit IRC | 02:16 | |
| *** dragonmaster has joined #openstack-ansible | 02:17 | |
| *** itsuugo has joined #openstack-ansible | 02:17 | |
| cloudnull | jmccrory: if you have time mind giving this one a nudge https://review.openstack.org/#/c/354120/ | 02:17 |
| cloudnull | stevelle: https://review.openstack.org/#/c/366931/ is cutting it close | 02:21 |
| cloudnull | 133 runtime, and is doing smoke checks currently | 02:21 |
| *** itsuugo has quit IRC | 02:22 | |
| stevelle | ikr | 02:22 |
| stevelle | I'm anticipating it getting cut off | 02:22 |
| *** itsuugo has joined #openstack-ansible | 02:23 | |
| * cloudnull has fingers crossed. | 02:23 | |
| stevelle | zuul says it's done | 02:23 |
| stevelle | kicked again | 02:23 |
| *** poopcat has quit IRC | 02:23 | |
| cloudnull | looks like trusty timeed out | 02:27 |
| cloudnull | and xenial failed smoke checks | 02:27 |
| cloudnull | and because it took too long it didnt collect logs. thats kinda a bastard. | 02:28 |
| *** itsuugo has quit IRC | 02:29 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_nova: Add specific support for automatic compute RPC pinning https://review.openstack.org/367752 | 02:30 |
| jmccrory | cloudnull : question on https://review.openstack.org/#/c/354120/ ,might have the wrong var in the routes task | 02:30 |
| stevelle | nice catch jmccrory | 02:30 |
| *** itsuugo has joined #openstack-ansible | 02:30 | |
| *** dragonmaster has quit IRC | 02:30 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create: Change the container prep using more intelligent commands https://review.openstack.org/354120 | 02:34 |
| cloudnull | jmccrory: nice catch indeed. | 02:34 |
| cloudnull | ^ updated | 02:34 |
| *** itsuugo has quit IRC | 02:35 | |
| *** itsuugo has joined #openstack-ansible | 02:36 | |
| *** itsuugo has quit IRC | 02:43 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron: Add container logs to the build https://review.openstack.org/370150 | 02:43 |
| *** itsuugo has joined #openstack-ansible | 02:44 | |
| *** jperry has joined #openstack-ansible | 02:44 | |
| *** poopcat has joined #openstack-ansible | 02:46 | |
| cloudnull | i gotz to step away. | 02:46 |
| *** poopcat has quit IRC | 02:46 | |
| cloudnull | if folks can shepard the gate it'd be appreciated. | 02:46 |
| *** thorst has joined #openstack-ansible | 02:47 | |
| *** hughmFLEXin has quit IRC | 02:48 | |
| *** itsuugo has quit IRC | 02:48 | |
| *** poopcat has joined #openstack-ansible | 02:49 | |
| *** itsuugo has joined #openstack-ansible | 02:50 | |
| adreznec | Anyone have an idea why I'm getting nova.exception.InstanceInfoCacheNotFound error back from test_server_basic_ops for 370501? | 02:50 |
| adreznec | First it failed trusty and passed xenial, then after a recheck it passed trusty and failed xenial | 02:50 |
| *** hughmFLEXin has joined #openstack-ansible | 02:50 | |
| stevelle | I'm assuming this is part of an oslo.db-based instability at this point | 02:51 |
| *** hughmFLEXin has quit IRC | 02:51 | |
| stevelle | I haven't looked to see if that test has been giving nova gate problems | 02:52 |
| *** sdake has joined #openstack-ansible | 02:53 | |
| *** poopcat has quit IRC | 02:53 | |
| adreznec | Yeah, I was willing to chalk it up to gate flakiness the first time but multiple rechecks is usually a bad sign. Might poke through the nova history here to see if I come back with anything | 02:55 |
| *** poopcat has joined #openstack-ansible | 02:55 | |
| *** thorst has quit IRC | 02:56 | |
| stevelle | I kind of expect tempest to give us grief at RC time just because tempest hates us | 02:56 |
| *** sdake has quit IRC | 02:58 | |
| adreznec | Of course, tempest just knows we've hit a critical point and is trying to punish us for leaving patches to the last minute | 02:59 |
| *** itsuugo has quit IRC | 03:01 | |
| *** sdake has joined #openstack-ansible | 03:01 | |
| *** jwagner[away] is now known as jwagner | 03:02 | |
| *** jwagner is now known as jwagner[away] | 03:02 | |
| *** itsuugo has joined #openstack-ansible | 03:03 | |
| *** itsuugo has quit IRC | 03:08 | |
| *** itsuugo has joined #openstack-ansible | 03:10 | |
| stevelle | 366931 is 48 minutes in and only installing nova. feels like another recheck is gonna be needed | 03:11 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_container_create: Change the container prep using more intelligent commands https://review.openstack.org/354120 | 03:16 |
| *** itsuugo has quit IRC | 03:17 | |
| *** itsuugo has joined #openstack-ansible | 03:18 | |
| adreznec | Yikes, yeah... | 03:18 |
| *** gouthamr has quit IRC | 03:19 | |
| stevelle | internap... | 03:21 |
| *** itsuugo has quit IRC | 03:23 | |
| *** itsuugo has joined #openstack-ansible | 03:24 | |
| stevelle | in other news your novnc fix got +1 verified | 03:24 |
| *** dragonmaster has joined #openstack-ansible | 03:24 | |
| *** itsuugo has quit IRC | 03:33 | |
| *** itsuugo has joined #openstack-ansible | 03:36 | |
| adreznec | Some days I feel like passing the gate should feel a lot less like winning the lottery than it does. | 03:40 |
| *** itsuugo has quit IRC | 03:41 | |
| *** itsuugo has joined #openstack-ansible | 03:42 | |
| *** thorst has joined #openstack-ansible | 03:44 | |
| openstackgerrit | Steve Lewis (stevelle) proposed openstack/openstack-ansible-os_ceilometer: Limit where Apache is installed for Ceilometer https://review.openstack.org/370529 | 03:45 |
| *** thorst has quit IRC | 03:46 | |
| *** jwagner[away] is now known as jwagner | 03:48 | |
| openstackgerrit | Steve Lewis (stevelle) proposed openstack/openstack-ansible-os_ceilometer: Limit where Apache is installed for Ceilometer https://review.openstack.org/370529 | 03:49 |
| stevelle | m hayden has a tweet for those moments ^ ... https://twitter.com/majorhayden/status/659059395037409280/photo/1 | 03:51 |
| *** itsuugo has quit IRC | 03:51 | |
| *** itsuugo has joined #openstack-ansible | 03:52 | |
| adreznec | So accurate | 03:56 |
| *** itsuugo has quit IRC | 03:59 | |
| *** itsuugo has joined #openstack-ansible | 04:00 | |
| *** poopcat has quit IRC | 04:02 | |
| *** dragonmaster has quit IRC | 04:02 | |
| *** jperry has quit IRC | 04:04 | |
| stevelle | 366931 finished on interop in about 1:29 but just got sent to verify on ovh-gra1 | 04:06 |
| *** phschwartz has quit IRC | 04:07 | |
| *** itsuugo has quit IRC | 04:10 | |
| *** kjw3 has quit IRC | 04:11 | |
| *** phschwartz has joined #openstack-ansible | 04:11 | |
| *** itsuugo has joined #openstack-ansible | 04:12 | |
| *** hblah has quit IRC | 04:16 | |
| *** itsuugo has quit IRC | 04:16 | |
| *** dragonmaster has joined #openstack-ansible | 04:17 | |
| *** itsuugo has joined #openstack-ansible | 04:18 | |
| *** itsuugo has quit IRC | 04:23 | |
| *** itsuugo has joined #openstack-ansible | 04:24 | |
| *** kjw3 has joined #openstack-ansible | 04:27 | |
| *** itsuugo has quit IRC | 04:29 | |
| *** thorst has joined #openstack-ansible | 04:29 | |
| *** itsuugo has joined #openstack-ansible | 04:29 | |
| *** itsuugo has quit IRC | 04:34 | |
| *** itsuugo has joined #openstack-ansible | 04:35 | |
| *** galstrom_zzz is now known as galstrom | 04:35 | |
| *** thorst has quit IRC | 04:39 | |
| *** itsuugo has quit IRC | 04:40 | |
| *** itsuugo has joined #openstack-ansible | 04:41 | |
| *** shausy has joined #openstack-ansible | 04:45 | |
| *** thorst has joined #openstack-ansible | 04:45 | |
| *** itsuugo has quit IRC | 04:53 | |
| *** itsuugo has joined #openstack-ansible | 04:54 | |
| *** thorst has quit IRC | 04:58 | |
| *** thorst has joined #openstack-ansible | 04:58 | |
| *** Jeffrey4l__ has joined #openstack-ansible | 05:00 | |
| *** sdake has quit IRC | 05:03 | |
| *** itsuugo has quit IRC | 05:03 | |
| *** Jeffrey4l_ has quit IRC | 05:04 | |
| *** itsuugo has joined #openstack-ansible | 05:04 | |
| *** thorst has quit IRC | 05:07 | |
| *** thorst has joined #openstack-ansible | 05:09 | |
| *** itsuugo has quit IRC | 05:10 | |
| openstackgerrit | Merged openstack/openstack-ansible: Implement scenario capability for AIO https://review.openstack.org/369485 | 05:10 |
| *** itsuugo has joined #openstack-ansible | 05:10 | |
| *** thorst has quit IRC | 05:13 | |
| *** thorst has joined #openstack-ansible | 05:15 | |
| *** thorst has quit IRC | 05:19 | |
| *** itsuugo has quit IRC | 05:20 | |
| *** itsuugo has joined #openstack-ansible | 05:21 | |
| *** sdake has joined #openstack-ansible | 05:22 | |
| *** itsuugo has quit IRC | 05:26 | |
| *** itsuugo has joined #openstack-ansible | 05:28 | |
| *** gouthamr has joined #openstack-ansible | 05:31 | |
| *** thorst has joined #openstack-ansible | 05:34 | |
| *** itsuugo has quit IRC | 05:37 | |
| *** itsuugo has joined #openstack-ansible | 05:38 | |
| *** markvoelker has quit IRC | 05:41 | |
| *** thorst has quit IRC | 05:42 | |
| *** itsuugo has quit IRC | 05:43 | |
| *** itsuugo has joined #openstack-ansible | 05:43 | |
| *** thorst has joined #openstack-ansible | 05:44 | |
| *** McMurlock1 has joined #openstack-ansible | 05:45 | |
| *** galstrom is now known as galstrom_zzz | 05:48 | |
| *** thorst has quit IRC | 05:49 | |
| *** thorst has joined #openstack-ansible | 05:50 | |
| *** itsuugo has quit IRC | 05:51 | |
| *** itsuugo has joined #openstack-ansible | 05:51 | |
| odyssey4me | o/ | 05:55 |
| *** thorst has quit IRC | 05:55 | |
| *** itsuugo has quit IRC | 05:56 | |
| *** thorst has joined #openstack-ansible | 05:56 | |
| *** itsuugo has joined #openstack-ansible | 05:58 | |
| stevelle | morning | 06:00 |
| *** thorst has quit IRC | 06:01 | |
| odyssey4me | morning stevelle - any news on anything specific to follow up? | 06:01 |
| stevelle | odyssey4me: just fighting 2 things in gate: timeouts, and nova being unstable | 06:01 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_glance: Compress test execution logs https://review.openstack.org/367868 | 06:01 |
| odyssey4me | yeah, nova is definitely failing more than passing and has been for some time | 06:02 |
| *** thorst has joined #openstack-ansible | 06:02 | |
| stevelle | I did raise another change for ceilo but it doesn't seem critical to the RC to me: 370529 | 06:02 |
| *** itsuugo has quit IRC | 06:03 | |
| odyssey4me | yeah, we'll get that one in - otherwise we'll end up with apache on the compute hosts :p | 06:03 |
| *** itsuugo has joined #openstack-ansible | 06:03 | |
| odyssey4me | In my inspection yesterday I think I found that the inventory bug I discovered is something we're actually having with the ceilometer groups... I need to revisit that, but I think we may be depending on that bug slightly. | 06:04 |
| odyssey4me | just for ceilo though | 06:04 |
| *** pcaruana has joined #openstack-ansible | 06:04 | |
| *** d3n14l has joined #openstack-ansible | 06:05 | |
| stevelle | eh, there's always another RC ) | 06:06 |
| *** thorst has quit IRC | 06:07 | |
| *** thorst has joined #openstack-ansible | 06:08 | |
| *** v1k0d3n has quit IRC | 06:11 | |
| odyssey4me | the first 2 in the gate queue are on OSIC (they're ipv6) so there's a good chance of them passing and merging | 06:12 |
| odyssey4me | the next one is on rax-ord, so there's a fair chance it'll make it before timeout | 06:13 |
| *** pcaruana is now known as pcaruana|afk| | 06:13 | |
| d3n14l | Can someone tell me why I can not use ping from the neutron-agents-container? Fails with a permission denied on a shared library, but i do not know if this is intended behaviour or just buggy. | 06:14 |
| *** thorst has quit IRC | 06:17 | |
| *** janki has joined #openstack-ansible | 06:17 | |
| odyssey4me | d3n14l if you're using master, then that's a known issue | 06:17 |
| odyssey4me | if any cores are around, a review of https://review.openstack.org/367868 would be great | 06:18 |
| *** itsuugo has quit IRC | 06:18 | |
| odyssey4me | along with https://review.openstack.org/370529 | 06:18 |
| *** thorst has joined #openstack-ansible | 06:19 | |
| *** itsuugo has joined #openstack-ansible | 06:19 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Cleanup/standardize usage of tags in plays https://review.openstack.org/340977 | 06:21 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Cleanup/standardize usage of tags in plays https://review.openstack.org/340977 | 06:23 |
| *** thorst has quit IRC | 06:24 | |
| *** thorst has joined #openstack-ansible | 06:25 | |
| *** itsuugo has quit IRC | 06:28 | |
| odyssey4me | d3n14l master is using overlayfs, that issue is known when the neutron-agents containers uses overlayfs - there's aa registered bug for it | 06:28 |
| odyssey4me | overlayfs usage is experimental and testing only at this point | 06:28 |
| odyssey4me | it doesn't affect our gating, so we've left it in for the AIO | 06:28 |
| *** asettle has joined #openstack-ansible | 06:29 | |
| odyssey4me | https://bugs.launchpad.net/openstack-ansible/+bug/1612412 | 06:29 |
| openstack | Launchpad bug 1612412 in openstack-ansible "Xenial: neutron agents container unable to resolve names" [Low,Confirmed] - Assigned to Kevin Carter (kevin-carter) | 06:29 |
| *** thorst has quit IRC | 06:30 | |
| *** itsuugo has joined #openstack-ansible | 06:30 | |
| *** thorst has joined #openstack-ansible | 06:32 | |
| odyssey4me | heading for a shower, then relocating - bbiab | 06:33 |
| *** asettle has quit IRC | 06:34 | |
| d3n14l | @odyssey4me thanks for the explanation :) | 06:36 |
| *** sdake has quit IRC | 06:36 | |
| *** thorst has quit IRC | 06:36 | |
| *** itsuugo has quit IRC | 06:37 | |
| *** itsuugo has joined #openstack-ansible | 06:38 | |
| *** pcaruana|afk| is now known as pcaruana | 06:40 | |
| *** gouthamr has quit IRC | 06:41 | |
| *** haasn has quit IRC | 06:42 | |
| openstackgerrit | Merged openstack/openstack-ansible: [DOCS] Added HAProxy example to Production environment document. https://review.openstack.org/370187 | 06:43 |
| *** itsuugo has quit IRC | 06:44 | |
| *** thorst has joined #openstack-ansible | 06:44 | |
| *** jamielennox is now known as jamielennox|away | 06:45 | |
| *** itsuugo has joined #openstack-ansible | 06:45 | |
| *** berendt has joined #openstack-ansible | 06:45 | |
| openstackgerrit | Merged openstack/openstack-ansible: Gnocchi identities created before Swift playbook https://review.openstack.org/366931 | 06:45 |
| *** berendt has quit IRC | 06:45 | |
| *** chhavi has joined #openstack-ansible | 06:46 | |
| *** berendt has joined #openstack-ansible | 06:46 | |
| *** haasn has joined #openstack-ansible | 06:48 | |
| *** thorst has quit IRC | 06:52 | |
| *** thorst has joined #openstack-ansible | 06:54 | |
| *** thorst has quit IRC | 06:59 | |
| *** itsuugo has quit IRC | 07:00 | |
| *** thorst has joined #openstack-ansible | 07:00 | |
| *** itsuugo has joined #openstack-ansible | 07:02 | |
| *** thorst has quit IRC | 07:05 | |
| *** chhavi_ has joined #openstack-ansible | 07:06 | |
| openstackgerrit | Steve Lewis (stevelle) proposed openstack/openstack-ansible-os_swift: Require lookup of Gnocchi identity when needed https://review.openstack.org/366932 | 07:06 |
| *** drifterza has joined #openstack-ansible | 07:07 | |
| *** chhavi has quit IRC | 07:07 | |
| *** berendt has quit IRC | 07:08 | |
| *** chhavi_ has left #openstack-ansible | 07:08 | |
| *** chhavi_ has joined #openstack-ansible | 07:09 | |
| *** berendt has joined #openstack-ansible | 07:09 | |
| *** thorst has joined #openstack-ansible | 07:11 | |
| *** itsuugo has quit IRC | 07:17 | |
| *** admin0 has joined #openstack-ansible | 07:17 | |
| *** thorst has quit IRC | 07:17 | |
| *** thorst has joined #openstack-ansible | 07:18 | |
| *** itsuugo has joined #openstack-ansible | 07:18 | |
| *** thorst_ has joined #openstack-ansible | 07:21 | |
| *** thorst has quit IRC | 07:23 | |
| *** itsuugo has quit IRC | 07:23 | |
| *** itsuugo has joined #openstack-ansible | 07:24 | |
| *** thorst_ has quit IRC | 07:25 | |
| *** itsuugo has quit IRC | 07:29 | |
| *** itsuugo has joined #openstack-ansible | 07:30 | |
| *** itsuugo has quit IRC | 07:35 | |
| *** itsuugo has joined #openstack-ansible | 07:35 | |
| *** asettle has joined #openstack-ansible | 07:37 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova: Compress test execution logs https://review.openstack.org/367886 | 07:38 |
| *** asettle has quit IRC | 07:39 | |
| *** markvoelker has joined #openstack-ansible | 07:42 | |
| *** itsuugo has quit IRC | 07:47 | |
| *** markvoelker has quit IRC | 07:48 | |
| *** itsuugo has joined #openstack-ansible | 07:49 | |
| *** admin0 has quit IRC | 07:54 | |
| *** jamielennox|away is now known as jamielennox | 07:57 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Cleanup/standardize usage of tags in plays https://review.openstack.org/340977 | 08:02 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Remove assumption that the neutron_lbaas var is set https://review.openstack.org/368905 | 08:02 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova: Fix novnc console include positioning https://review.openstack.org/370501 | 08:02 |
| *** mpjetta has quit IRC | 08:02 | |
| *** basilAB has quit IRC | 08:03 | |
| *** basilAB has joined #openstack-ansible | 08:05 | |
| *** mpjetta has joined #openstack-ansible | 08:05 | |
| odyssey4me | hughsaunders andymccr mhayden evrardjp jmccrory automagically cloudnull d34dh0r53 stevelle mattt here're some key reviews which need completion for this morning | 08:06 |
| odyssey4me | https://review.openstack.org/370529 (submitted by stevelle) | 08:06 |
| odyssey4me | https://review.openstack.org/367868 (submitted by odyssey4me) | 08:06 |
| odyssey4me | https://review.openstack.org/370201 (submitted by automagically) | 08:06 |
| odyssey4me | https://review.openstack.org/340977 (submitted by cloudnull) | 08:07 |
| odyssey4me | andymccr it looks like https://review.openstack.org/366932 needs a revision to make it work | 08:07 |
| *** chhavi_ has quit IRC | 08:25 | |
| *** chhavi_ has joined #openstack-ansible | 08:26 | |
| *** karimb has joined #openstack-ansible | 08:28 | |
| drifterza | hello all | 08:30 |
| evrardjp | hello | 08:31 |
| *** itsuugo has quit IRC | 08:32 | |
| odyssey4me | o/ | 08:32 |
| vnogin | \o | 08:33 |
| *** itsuugo has joined #openstack-ansible | 08:34 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_ceilometer: Limit where Apache is installed for Ceilometer https://review.openstack.org/370529 | 08:34 |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance: Compress test execution logs https://review.openstack.org/367868 | 08:38 |
| *** electrofelix has joined #openstack-ansible | 08:43 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_rally: Remove duplicate container prep from role test https://review.openstack.org/370201 | 08:46 |
| *** itsuugo has quit IRC | 08:46 | |
| *** itsuugo has joined #openstack-ansible | 08:46 | |
| *** itsuugo has quit IRC | 08:51 | |
| *** itsuugo has joined #openstack-ansible | 08:52 | |
| *** asettle has joined #openstack-ansible | 08:52 | |
| Attilla | odyssey4me: started a new VM with vagrant (Ubuntu 14) and ran a completely original AIO install on it | 08:52 |
| Attilla | According to these instructions: http://docs.openstack.org/developer/openstack-ansible/developer-docs/quickstart-aio.html | 08:52 |
| odyssey4me | Attilla yep, and? | 08:52 |
| Attilla | Again same auth error | 08:53 |
| odyssey4me | that is very odd | 08:53 |
| odyssey4me | what's the vm platform you're using? | 08:53 |
| Attilla | Libvirt/KVM on an Ubuntu 16 host | 08:53 |
| odyssey4me | and what did you install on to? | 08:54 |
| Attilla | I can try to do the same in Virtualbox, but that shouldn't make a difference | 08:54 |
| Attilla | Ubuntu 14 | 08:54 |
| odyssey4me | nope, won't make a difference | 08:54 |
| Attilla | The standard box image | 08:54 |
| odyssey4me | what id 'the standard box image' ? | 08:54 |
| odyssey4me | *is | 08:54 |
| Attilla | Sorry, that comes form the hasicorp repos. Just a clean 1404 image afaik | 08:55 |
| odyssey4me | clean and hopefully minimal? | 08:56 |
| Attilla | That is usually the case | 08:56 |
| *** asettle has quit IRC | 08:57 | |
| *** asettle has joined #openstack-ansible | 08:58 | |
| *** karimb has quit IRC | 08:58 | |
| Attilla | It does make me feel good that it isn't really anything in my config though. ;-) | 08:59 |
| *** karimb has joined #openstack-ansible | 09:01 | |
| odyssey4me | Attilla ok, well at least it's a well known installation - debug is enabled, so go ahead and check the keystone logs to see if you can find the errors | 09:04 |
| *** itsuugo has quit IRC | 09:06 | |
| Attilla | I lost my irssi session yesterday and you explained which playbook to run seperately | 09:06 |
| Attilla | Could you repeat that? | 09:06 |
| odyssey4me | at this point you should just be able to look at the keystone logs for errors | 09:06 |
| odyssey4me | because they would have been started in debug mode | 09:06 |
| odyssey4me | /var/log/keystone.log IIRC, in the keystone container | 09:07 |
| odyssey4me | alternatively that should be in /openstack/log/<container name> on the host | 09:07 |
| *** itsuugo has joined #openstack-ansible | 09:08 | |
| Attilla | Well, I acutally didn't enable debug for this latest build | 09:10 |
| Attilla | I'll do that now | 09:10 |
| odyssey4me | Attilla a stock AIO already does | 09:10 |
| Attilla | oh, great | 09:10 |
| Attilla | one moment then | 09:11 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible: Cleanup/standardize usage of tags in plays https://review.openstack.org/340977 | 09:13 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible: Cleanup/standardize usage of tags in plays https://review.openstack.org/340977 | 09:17 |
| *** itsuugo has quit IRC | 09:18 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_neutron: [DO NOT MERGE] More logging for neutron https://review.openstack.org/370335 | 09:18 |
| *** itsuugo has joined #openstack-ansible | 09:19 | |
| *** pmannidi has quit IRC | 09:19 | |
| Attilla | odyssey4me: Going over the logs, but a lot of information (and errors there) | 09:19 |
| Attilla | Have to search for the exact time where it went wrong | 09:19 |
| *** DanyC has joined #openstack-ansible | 09:21 | |
| *** adrian_otto has joined #openstack-ansible | 09:22 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: [WIP] Add multiple scenarios for testing https://review.openstack.org/370638 | 09:28 |
| openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-os_swift: Add "ansible_host" to each host https://review.openstack.org/370644 | 09:34 |
| andymccr | ^ odyssey4me, evrardjp - that will fix it. I'm going to be working on moving Swift to use the ansible-tests repo, so will clean up the swift tests in general, but that'll get us moving again. | 09:35 |
| evrardjp | makes sense | 09:37 |
| odyssey4me | yip, it does | 09:37 |
| evrardjp | please use one variable and reference it :p it pains me to see the same ip written twoce | 09:38 |
| evrardjp | twice | 09:38 |
| evrardjp | in the rewrite I mean | 09:38 |
| *** thorst has joined #openstack-ansible | 09:38 | |
| odyssey4me | yeah, that'd obviously be better - but this will do as a stop-gap | 09:38 |
| evrardjp | agreed | 09:38 |
| evrardjp | just for the future :) | 09:38 |
| odyssey4me | in fact, automagically is already working on the transition in the tests repo: https://review.openstack.org/369697 | 09:38 |
| evrardjp | cool | 09:39 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: [WIP] Add multiple scenarios for testing https://review.openstack.org/370638 | 09:40 |
| *** markvoelker has joined #openstack-ansible | 09:44 | |
| *** chhavi_ has quit IRC | 09:45 | |
| *** pjm6 has joined #openstack-ansible | 09:46 | |
| *** Trident has joined #openstack-ansible | 09:48 | |
| *** adrian_otto has quit IRC | 09:49 | |
| *** markvoelker has quit IRC | 09:49 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Cleanup/standardize usage of tags in plays https://review.openstack.org/340977 | 09:55 |
| *** chhavi_ has joined #openstack-ansible | 09:58 | |
| *** itsuugo has quit IRC | 10:00 | |
| *** schwicht has quit IRC | 10:00 | |
| Attilla | odyssey4me: https://gist.github.com/packetninja/dffbb04d46f5616c98101b667f369fa4 | 10:00 |
| Attilla | I added the error and keystone log | 10:01 |
| Attilla | It seems that it is a keystone issue, but now a different module | 10:01 |
| *** itsuugo has joined #openstack-ansible | 10:01 | |
| Attilla | I think it is around 22:55, but I don't know how to pinpoint the exact time | 10:01 |
| Attilla | The output just changed there | 10:01 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Cleanup/standardize usage of tags in plays https://review.openstack.org/340977 | 10:08 |
| *** itsuugo has quit IRC | 10:11 | |
| odyssey4me | evrardjp andymccr ^ happy with that final version? | 10:11 |
| *** itsuugo has joined #openstack-ansible | 10:12 | |
| evrardjp | Thanks for fixing this. | 10:16 |
| odyssey4me | lol @ adreznec "gate roulette" | 10:16 |
| adreznec | odyssey4me: that's what it feels like at this point... spin the wheel, land on trusty success, xenial failure. spin again, lxd fails. | 10:17 |
| odyssey4me | adreznec yep, there's clearly a race of some sort | 10:18 |
| odyssey4me | I see some neutron agent issues, and we're busy working through a similar problem for the neutron gate. | 10:18 |
| odyssey4me | There is a little suspicion that we may be hitting a timing issue - either due to time drift, or due to the usual gate issue of doing things too rapidly for neutron... and we need some sleeps... | 10:19 |
| adreznec | Yeah, I looked a bit at the Nova gate just to see if it was something upstream, but didn't see anything similar happening there | 10:20 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_swift: Require lookup of Gnocchi identity when needed https://review.openstack.org/366932 | 10:20 |
| odyssey4me | it'd be great to understand the differences in our testing regimen versus devstack to see where we might be stretching things a little too far. | 10:21 |
| odyssey4me | IIRC devstack isn't testing the ssh connection to the instance... but that was when I last looked a very long time ago. | 10:21 |
| odyssey4me | we do, and that's often the failure cause | 10:21 |
| *** adrian_otto has joined #openstack-ansible | 10:23 | |
| *** itsuugo has quit IRC | 10:25 | |
| *** itsuugo has joined #openstack-ansible | 10:27 | |
| *** schwicht_at_work has joined #openstack-ansible | 10:30 | |
| adreznec | Yeah, that was still the case at least as of July for Nova. I remember a thread about how they were only doing the SSH validation in a non-voting job. | 10:31 |
| *** schwicht_at_work has quit IRC | 10:31 | |
| *** itsuugo has quit IRC | 10:32 | |
| odyssey4me | yep, I've seen many moans about how horrible the tempest ssh connection thing is | 10:32 |
| *** itsuugo has joined #openstack-ansible | 10:32 | |
| odyssey4me | but, we've been using the ssh test since juno with reasonable success | 10:32 |
| odyssey4me | but I would imagine that when it comes to nova testing, it's our most common failure | 10:33 |
| odyssey4me | we could perhaps opt out of that test in the role, and only exercise it in the integrated gate... | 10:33 |
| *** itsuugo has quit IRC | 10:39 | |
| *** itsuugo has joined #openstack-ansible | 10:41 | |
| *** itsuugo has quit IRC | 10:46 | |
| *** itsuugo has joined #openstack-ansible | 10:46 | |
| *** kylek3h has joined #openstack-ansible | 10:48 | |
| *** kylek3h has quit IRC | 10:48 | |
| *** kylek3h has joined #openstack-ansible | 10:48 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_neutron: Wait for net to be fully instantiated https://review.openstack.org/370689 | 10:55 |
| openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-tests: [WIP] Modularize networking https://review.openstack.org/370690 | 10:56 |
| openstackgerrit | Merged openstack/openstack-ansible-os_swift: Add "ansible_host" to each host https://review.openstack.org/370644 | 11:01 |
| *** electrofelix has quit IRC | 11:01 | |
| *** adrian_otto has quit IRC | 11:09 | |
| openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-tests: [WIP] Modularize networking https://review.openstack.org/370690 | 11:10 |
| odyssey4me | jmccrory automagically cloudnull d34dh0r53 stevelle mattt hughsaunders andymccr mhayden evrardjp asettle rromans this is the patch list we need to be in for rc1 https://review.openstack.org/#/q/project:%255Eopenstack/openstack-ansible.*+starredby:jesse-pretorius+status:open | 11:10 |
| *** itsuugo has quit IRC | 11:23 | |
| *** itsuugo has joined #openstack-ansible | 11:25 | |
| *** rgogunskiy has joined #openstack-ansible | 11:25 | |
| *** thorst has quit IRC | 11:27 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova: Add specific support for automatic compute RPC pinning https://review.openstack.org/367752 | 11:29 |
| *** asettle has quit IRC | 11:30 | |
| *** janki has quit IRC | 11:30 | |
| *** janki has joined #openstack-ansible | 11:32 | |
| *** retreved has joined #openstack-ansible | 11:33 | |
| *** schwicht has joined #openstack-ansible | 11:37 | |
| *** janki has quit IRC | 11:39 | |
| *** itsuugo has quit IRC | 11:43 | |
| *** itsuugo has joined #openstack-ansible | 11:44 | |
| *** markvoelker has joined #openstack-ansible | 11:45 | |
| *** schwicht has quit IRC | 11:50 | |
| *** markvoelker has quit IRC | 11:50 | |
| *** karimb has quit IRC | 11:56 | |
| mgariepy | good morning everyone ! | 12:02 |
| *** woodard has joined #openstack-ansible | 12:04 | |
| *** woodard has quit IRC | 12:05 | |
| *** woodard has joined #openstack-ansible | 12:06 | |
| *** psilvad has joined #openstack-ansible | 12:08 | |
| mhayden | morning | 12:09 |
| mhayden | odyssey4me: i'll get a look as soon as i am fully awake | 12:09 |
| *** itsuugo has quit IRC | 12:23 | |
| *** itsuugo has joined #openstack-ansible | 12:23 | |
| *** itsuugo has quit IRC | 12:28 | |
| *** itsuugo has joined #openstack-ansible | 12:29 | |
| Matias | morning | 12:30 |
| *** asettle has joined #openstack-ansible | 12:30 | |
| *** czunker has joined #openstack-ansible | 12:33 | |
| *** asettle has quit IRC | 12:36 | |
| mhayden | morning Matias | 12:37 |
| *** sdake has joined #openstack-ansible | 12:38 | |
| *** markvoelker has joined #openstack-ansible | 12:38 | |
| openstackgerrit | Merged openstack/openstack-ansible: Remove assumption that the neutron_lbaas var is set https://review.openstack.org/368905 | 12:42 |
| openstackgerrit | Merged openstack/openstack-ansible-os_swift: Require lookup of Gnocchi identity when needed https://review.openstack.org/366932 | 12:44 |
| *** itsuugo has quit IRC | 12:45 | |
| *** itsuugo has joined #openstack-ansible | 12:46 | |
| *** jamesdenton has joined #openstack-ansible | 12:48 | |
| *** electrofelix has joined #openstack-ansible | 12:53 | |
| *** v1k0d3n has joined #openstack-ansible | 13:00 | |
| *** askb has quit IRC | 13:00 | |
| *** itsuugo has quit IRC | 13:01 | |
| *** galstrom_zzz is now known as galstrom | 13:02 | |
| *** itsuugo has joined #openstack-ansible | 13:02 | |
| *** psilvad has quit IRC | 13:05 | |
| *** psilvad has joined #openstack-ansible | 13:05 | |
| *** dragonmaster has quit IRC | 13:06 | |
| *** kylek3h has quit IRC | 13:07 | |
| *** psilvad has quit IRC | 13:09 | |
| *** psilvad has joined #openstack-ansible | 13:09 | |
| *** sdake_ has joined #openstack-ansible | 13:14 | |
| openstackgerrit | Marc Gariépy proposed openstack/openstack-ansible-openstack_hosts: Remove requiretty for sudo on centos hosts. https://review.openstack.org/370807 | 13:14 |
| openstackgerrit | Marc Gariépy proposed openstack/openstack-ansible-lxc_hosts: Remove requiretty for sudo on centos base image. https://review.openstack.org/370808 | 13:15 |
| *** sdake has quit IRC | 13:16 | |
| *** psilvad has quit IRC | 13:17 | |
| *** psilvad has joined #openstack-ansible | 13:18 | |
| *** czunker has quit IRC | 13:22 | |
| *** gouthamr has joined #openstack-ansible | 13:24 | |
| *** KLevenstein has joined #openstack-ansible | 13:28 | |
| *** kylek3h has joined #openstack-ansible | 13:29 | |
| *** asettle has joined #openstack-ansible | 13:33 | |
| openstackgerrit | Travis Truman (automagically) proposed openstack/openstack-ansible-os_watcher: Ansible 2.1.1 role testing https://review.openstack.org/370828 | 13:34 |
| openstackgerrit | Marc Gariépy proposed openstack/openstack-ansible: fix centos bootstrap https://review.openstack.org/370831 | 13:35 |
| *** woodard has quit IRC | 13:35 | |
| *** BjoernT has joined #openstack-ansible | 13:36 | |
| *** d3n14l has left #openstack-ansible | 13:37 | |
| *** TxGirlGeek has joined #openstack-ansible | 13:37 | |
| *** asettle has quit IRC | 13:37 | |
| *** chhavi_ has quit IRC | 13:40 | |
| *** jperry has joined #openstack-ansible | 13:41 | |
| *** itsuugo has quit IRC | 13:46 | |
| *** itsuugo has joined #openstack-ansible | 13:48 | |
| openstackgerrit | Marc Gariépy proposed openstack/openstack-ansible-os_glance: Implement CentOS 7 support in os_glance https://review.openstack.org/320542 | 13:54 |
| *** rk4n has joined #openstack-ansible | 13:55 | |
| *** chhavi_ has joined #openstack-ansible | 13:56 | |
| *** galstrom is now known as galstrom_zzz | 13:58 | |
| *** michaelgugino has joined #openstack-ansible | 14:02 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible: Modify use of assertTrue(A in B) https://review.openstack.org/369803 | 14:05 |
| *** galstrom_zzz is now known as galstrom | 14:05 | |
| *** gouthamr has quit IRC | 14:07 | |
| *** drifterza has quit IRC | 14:07 | |
| *** rk4n has quit IRC | 14:07 | |
| *** asettle has joined #openstack-ansible | 14:09 | |
| *** spedione|AWAY is now known as spedione | 14:11 | |
| *** chhavi_ has quit IRC | 14:12 | |
| *** rk4n has joined #openstack-ansible | 14:15 | |
| *** asettle has quit IRC | 14:15 | |
| *** psilvad has quit IRC | 14:17 | |
| *** psilvad has joined #openstack-ansible | 14:17 | |
| cloudnull | mornings | 14:19 |
| openstackgerrit | Kyle L. Henderson proposed openstack/openstack-ansible-os_trove: Update role with fixes for AIO testing https://review.openstack.org/370880 | 14:25 |
| *** woodard has joined #openstack-ansible | 14:25 | |
| *** woodard has quit IRC | 14:26 | |
| *** jheroux has joined #openstack-ansible | 14:27 | |
| *** galstrom is now known as galstrom_zzz | 14:27 | |
| *** woodard has joined #openstack-ansible | 14:27 | |
| mhayden | mgariepy: i thought we were only turning off requiretty for root | 14:29 |
| mhayden | or did i miss a discussion later? | 14:29 |
| *** DanyC has quit IRC | 14:30 | |
| *** shausy has quit IRC | 14:31 | |
| cloudnull | +1 i think thats the way forward. | 14:33 |
| cloudnull | mgariepy: do you have time to work on that today? maybe mhayden and I can help out? | 14:33 |
| *** woodard has quit IRC | 14:34 | |
| *** woodard has joined #openstack-ansible | 14:34 | |
| openstackgerrit | Qin Wang (qwang) proposed openstack/openstack-ansible: [install-guide] complete commands in prepare target hosts Previously, apt-get update is not enforced before package installation, and some of the package install will fail due to outdates source list. https://review.openstack.org/370893 | 14:35 |
| *** Trident has quit IRC | 14:41 | |
| *** Trident has joined #openstack-ansible | 14:41 | |
| mgariepy | mhayden, ok i'll resubmit, give a minute or 2 :) do you prefer to have it in /etc/sudoers.d/openstack-ansible or directly in /etc/sudoers ? | 14:44 |
| *** rk4n has quit IRC | 14:44 | |
| mhayden | mgariepy: i think the .d directory might make it easier to manage | 14:45 |
| mhayden | especially if we update sudo | 14:45 |
| jamesdenton | anyone know if there were recent changes to nginx init script in repo container regarding log files? Reason I ask is during an upgrade nginx wouldn't start since it couldn't write to /var/log/nginx. The directory didn't exist | 14:47 |
| openstackgerrit | Marc Gariépy proposed openstack/openstack-ansible-openstack_hosts: Remove requiretty for sudo on centos hosts. https://review.openstack.org/370807 | 14:48 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_nova: Implement placement AUTH/DB https://review.openstack.org/367749 | 14:48 |
| *** rk4n has joined #openstack-ansible | 14:48 | |
| cloudnull | jamesdenton not that i'm aware of. | 14:49 |
| cloudnull | that was an upgrade from mitaka to newton ? | 14:49 |
| jamesdenton | this was a master from early august to today | 14:49 |
| cloudnull | ok. | 14:49 |
| cloudnull | i mean its quite possible . | 14:49 |
| openstackgerrit | Travis Truman (automagically) proposed openstack/openstack-ansible-rabbitmq_server: Ansible 2.1.1 role testing https://review.openstack.org/370904 | 14:50 |
| jamesdenton | yeah i'm not too worried. fixed and moving along now. thanks :) | 14:50 |
| cloudnull | i dont see anything that specifically creates the /var/log/nginx | 14:50 |
| cloudnull | dir within the repo_server role | 14:50 |
| cloudnull | so it may have been something triggered by the package? | 14:51 |
| openstackgerrit | Marc Gariépy proposed openstack/openstack-ansible-lxc_hosts: Remove requiretty for sudo on centos base image. https://review.openstack.org/370808 | 14:51 |
| cloudnull | its also something that we could add to the repo_server role to ensure it exists and that the perms are correct. | 14:51 |
| *** rk4n has quit IRC | 14:52 | |
| *** rk4n has joined #openstack-ansible | 14:52 | |
| *** weezS has joined #openstack-ansible | 14:53 | |
| *** rk4n has quit IRC | 14:53 | |
| jamesdenton | cloudnull http://paste.openstack.org | 14:53 |
| jamesdenton | ahh | 14:53 |
| jamesdenton | one sec | 14:53 |
| jamesdenton | http://paste.openstack.org/show/577459/ | 14:53 |
| cloudnull | ooohhhh. new paste | 14:53 |
| cloudnull | :) | 14:53 |
| cloudnull | interesting that in the rsyslog role for the repo -server | 14:54 |
| *** rk4n has joined #openstack-ansible | 14:54 | |
| cloudnull | looking for log files in a dir that doesn't exist. | 14:54 |
| cloudnull | # /var/log/nginx | 14:54 |
| cloudnull | https://github.com/openstack/openstack-ansible-rsyslog_client/blob/master/tasks/rsyslog_client_post_install.yml#L31 | 14:55 |
| *** jperry has quit IRC | 14:55 | |
| jamesdenton | you're welcome | 14:56 |
| openstackgerrit | Marc Gariépy proposed openstack/openstack-ansible: fix centos bootstrap https://review.openstack.org/370831 | 14:56 |
| mgariepy | michaelgugino, http://logs.openstack.org/97/370197/2/experimental/gate-openstack-ansible-openstack-ansible-aio-centos-7/5eec032/console.html#_2016-09-14_23_14_32_131930 | 14:57 |
| jamesdenton | nginx wouldn't start, either, without that dir in place. it's at the bottom of the paste | 14:58 |
| openstackgerrit | Nish Patwa(nishpatwa_) proposed openstack/openstack-ansible: [DOCS] Added missing level info for haproxy_hosts https://review.openstack.org/370914 | 15:00 |
| michaelgugino | mgariepy: indeed, looks like we're setting +e | 15:00 |
| *** rgogunskiy has quit IRC | 15:00 | |
| mgariepy | and what is the points of doing yum check-update ? | 15:01 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-rsyslog_client: Add check to ensure a target log dir exists https://review.openstack.org/370917 | 15:02 |
| mgariepy | since it's only listing the package that can be updated. | 15:02 |
| cloudnull | jamesdenton: ^ | 15:02 |
| jamesdenton | bueno | 15:02 |
| cloudnull | part 1/2 | 15:02 |
| michaelgugino | setting -e | 15:02 |
| *** berendt has quit IRC | 15:03 | |
| michaelgugino | mgariepy: My guess is it's a way to check if there are package updates. | 15:03 |
| openstackgerrit | Merged openstack/openstack-ansible: Cleanup/standardize usage of tags in plays https://review.openstack.org/340977 | 15:04 |
| openstackgerrit | Marc Gariépy proposed openstack/openstack-ansible-openstack_hosts: Remove requiretty for sudo on centos hosts. https://review.openstack.org/370807 | 15:04 |
| *** jperry has joined #openstack-ansible | 15:06 | |
| openstackgerrit | Marc Gariépy proposed openstack/openstack-ansible-lxc_hosts: Remove requiretty for sudo on centos base image. https://review.openstack.org/370808 | 15:08 |
| jamesdenton | cloudnull this failure may also interest you: http://paste.openstack.org/show/577462/ | 15:11 |
| *** asettle has joined #openstack-ansible | 15:11 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-repo_server: Ensure the repo server has a valid log directory https://review.openstack.org/370931 | 15:12 |
| cloudnull | jamesdenton: 2/2 | 15:12 |
| jamesdenton | thanks cloudnull! | 15:13 |
| *** gouthamr has joined #openstack-ansible | 15:15 | |
| *** asettle has quit IRC | 15:16 | |
| *** gouthamr has quit IRC | 15:17 | |
| *** gouthamr has joined #openstack-ansible | 15:17 | |
| *** pcaruana has quit IRC | 15:19 | |
| *** klamath has quit IRC | 15:19 | |
| *** hj-hpe has joined #openstack-ansible | 15:20 | |
| *** adrian_otto has joined #openstack-ansible | 15:20 | |
| *** klamath has joined #openstack-ansible | 15:22 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-repo_build: Add git cleanup to op-clone-script https://review.openstack.org/370940 | 15:23 |
| cloudnull | jamesdenton: ^ | 15:23 |
| luzC | morning, I opened bug https://bugs.launchpad.net/openstack-ansible/+bug/1623778 | 15:24 |
| openstack | Launchpad bug 1623778 in openstack-ansible "run_upgrade.sh mitaka to newton failing, missing nginx log folder" [Undecided,New] | 15:24 |
| luzC | I'm getting an error when upgrading deployment from stable/mitaka to master | 15:25 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-repo_server: Ensure the repo server has a valid log directory https://review.openstack.org/370931 | 15:25 |
| cloudnull | luzC: ^ | 15:26 |
| luzC | failure is on playbook openstack-ansible/playbooks/repo-server.yml due to nginx log folder does not exists... not sure how to fix it though... any idea? | 15:26 |
| cloudnull | luzC: ^ | 15:26 |
| stevelle | we have a bug we can attach to the commit msg now :) | 15:26 |
| cloudnull | done | 15:26 |
| luzC | cloudnull | 15:26 |
| luzC | I see the patch... thanks :) | 15:27 |
| luzC | awsome I'll test it and let you knwo | 15:27 |
| luzC | *know | 15:27 |
| cloudnull | tyvm | 15:28 |
| openstackgerrit | Nolan Brubaker proposed openstack/openstack-ansible: Add lxc_host dynamic group to inventory. https://review.openstack.org/369599 | 15:31 |
| palendae | cloudnull, ^ that changes your lxc_host_group change | 15:31 |
| palendae | Hopefully I'm not breaking other roles with that | 15:32 |
| cloudnull | nice . | 15:32 |
| cloudnull | that should be better being that the group will now simply be in inventory | 15:32 |
| palendae | Yeah | 15:32 |
| *** rk4n has quit IRC | 15:33 | |
| jamesdenton | cloudnull thanks for the quick turnarounds! | 15:33 |
| *** rk4n has joined #openstack-ansible | 15:33 | |
| mgariepy | mhayden, cloudnull https://review.openstack.org/#/c/370807/ and https://review.openstack.org/#/c/370808/ for the sudo stuff. | 15:33 |
| mhayden | thanks -- i'll get a look in a bit | 15:34 |
| mgariepy | mhayden, the glance role is deploying without errors, only the gate-openstack-ansible-os_glance-ansible-func-centos-7-nv isn't working. | 15:36 |
| mgariepy | https://review.openstack.org/#/c/320542/ ^^ | 15:36 |
| *** rk4n has quit IRC | 15:42 | |
| *** rk4n has joined #openstack-ansible | 15:42 | |
| *** will-hodor has joined #openstack-ansible | 15:43 | |
| openstackgerrit | Travis Truman (automagically) proposed openstack/openstack-ansible-tests: Address ansible_ssh_* var deprecation https://review.openstack.org/369697 | 15:47 |
| *** rk4n has quit IRC | 15:48 | |
| *** itsuugo has quit IRC | 15:48 | |
| *** scarlisle has joined #openstack-ansible | 15:48 | |
| *** itsuugo has joined #openstack-ansible | 15:49 | |
| *** castulo has joined #openstack-ansible | 15:58 | |
| mhayden | weekly meeting starting soon in #openstack-meeting-4! cloudnull, mattt, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae, Sam-I-Am, odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, mhayden, scarlisle, luckyinva, ntt, javeriak, automagically, spotz, vdo, jmccrory, alextricity25, jasondotstar, | 15:59 |
| mhayden | admin0, michaelgugino, ametts, v1k0d3n, severion, bgmccollum, darrenc, JRobinson__, asettle, colinmcnamara, thorst, adreznec, eil397 | 15:59 |
| *** drifterza has joined #openstack-ansible | 15:59 | |
| *** TxGirlGeek has quit IRC | 16:01 | |
| d34dh0r53 | o | 16:02 |
| *** gonzalo2kx has joined #openstack-ansible | 16:04 | |
| *** hughmFLEXin has joined #openstack-ansible | 16:04 | |
| castulo | hi guys, I'm trying to do an upgrade using openstack-ansible from stable/mitaka to master, and the run-upgrade.sh script is filing with this error message: "the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'osa_secrets_file_name' is undefined". "The error appears to have been in '/opt/openstack-ansible/scripts/upgrade-utilities/playbooks/user-secrets-adjustment.yml': line 25, column 7, | 16:04 |
| castulo | but may | 16:05 |
| castulo | be elsewhere in the file depending on the exact syntax problem." | 16:05 |
| *** rk4n has joined #openstack-ansible | 16:05 | |
| castulo | and I do see some changes were made to the user-secrets-adjustment.yml a couple days ago: https://github.com/openstack/openstack-ansible/commit/bceb1e15e61af8f996831f54477e840d123522f8 | 16:05 |
| *** adrian_otto has quit IRC | 16:06 | |
| castulo | so for some reason it seems like it cannot find the osa_secrets_file_name variable | 16:06 |
| *** rk4n has quit IRC | 16:08 | |
| cloudnull | I think you have to specify that file name | 16:12 |
| cloudnull | prometheanfire: ^ | 16:12 |
| prometheanfire | hi | 16:12 |
| *** asettle has joined #openstack-ansible | 16:12 | |
| cloudnull | oh opps scripts/upgrade-utilities/playbooks/defaults/main.yml:15:osa_secrets_file_name: | 16:12 |
| cloudnull | its defined there. | 16:12 |
| prometheanfire | ya, it's defined | 16:13 |
| prometheanfire | is that not loaded by default? | 16:13 |
| cloudnull | no that should be moved to playbooks/inventory/group_vars/all.yml | 16:13 |
| prometheanfire | castulo: you make a bug for that yet? | 16:14 |
| cloudnull | playbooks dont have "defaults" they have vars, var_files, etc. | 16:14 |
| castulo | no | 16:14 |
| prometheanfire | castulo: mind doing so while I do the commit? | 16:14 |
| castulo | sure, no problem | 16:15 |
| cloudnull | prometheanfire: can you wip that up today so we can try and get it in for RC1 | 16:15 |
| prometheanfire | yep | 16:15 |
| prometheanfire | we need to backport it too :( | 16:15 |
| *** rk4n has joined #openstack-ansible | 16:16 | |
| *** McMurlock1 has quit IRC | 16:16 | |
| *** asettle has quit IRC | 16:17 | |
| stevelle | one step before the other, seems like a simple fix but this gate... | 16:17 |
| openstackgerrit | Kyle L. Henderson proposed openstack/openstack-ansible-os_trove: Update role with fixes for AIO testing https://review.openstack.org/370880 | 16:17 |
| openstackgerrit | Travis Truman (automagically) proposed openstack/openstack-ansible-tests: Address ansible_ssh_* var deprecation https://review.openstack.org/369697 | 16:18 |
| prometheanfire | castulo: let me know the bug number and the review is ready | 16:18 |
| prometheanfire | git mv ftw | 16:18 |
| castulo | yep, one minute | 16:19 |
| castulo | prometheanfire: Ready, this is the bug: https://bugs.launchpad.net/openstack-ansible/+bug/1624029 | 16:24 |
| openstack | Launchpad bug 1624029 in openstack-ansible "Upgrade failing due to not finding the osa_secrets_file_name variable when running the user-secrets-adjustment.yml playbook" [Undecided,New] | 16:24 |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible: move variables file to the correct location https://review.openstack.org/370981 | 16:25 |
| *** woodard has quit IRC | 16:26 | |
| *** d3n14l has joined #openstack-ansible | 16:26 | |
| prometheanfire | castulo: can you confirm that the patch fixes it? | 16:26 |
| *** asettle has joined #openstack-ansible | 16:31 | |
| prometheanfire | castulo: I assume that the +1 means it works for you? | 16:32 |
| *** itsuugo has quit IRC | 16:32 | |
| *** adrian_otto has joined #openstack-ansible | 16:33 | |
| *** itsuugo has joined #openstack-ansible | 16:34 | |
| *** woodard has joined #openstack-ansible | 16:35 | |
| *** asettle has quit IRC | 16:36 | |
| *** adrian_otto has quit IRC | 16:37 | |
| *** drifterza has quit IRC | 16:38 | |
| *** adrian_otto has joined #openstack-ansible | 16:40 | |
| openstackgerrit | Travis Truman (automagically) proposed openstack/openstack-ansible-tests: Address ansible_ssh_* var deprecation https://review.openstack.org/369697 | 16:40 |
| cloudnull | jorgem mhayden: are there updated design docs for octavia? the current design docs from upstream are fairly old(http://docs.openstack.org/developer/octavia/design/version0.5/component-design.html#octavia-v0-5-component-design) but they state that the components are "not horizontally scalable". the wiki shows that "Fully scalable and HA command-and-control layer" was a milestone for mitaka but i'm not seeing any documentation on that. | 16:40 |
| openstackgerrit | Nolan Brubaker proposed openstack/openstack-ansible: Add lxc_host dynamic group to inventory. https://review.openstack.org/369599 | 16:40 |
| *** gouthamr has quit IRC | 16:40 | |
| cloudnull | maybe it just not been produced yet? | 16:40 |
| castulo | prometheanfire: yes, that seems to have fixed the probcd .. | 16:41 |
| castulo | ups, disregard that message | 16:41 |
| castulo | I'm still testing it | 16:41 |
| cloudnull | I'm asking becuae octavia is an OSIC deliverable and if you all are going to work on it I can update that story which is currently blocked due to that belief that Octavia can not be made HA. | 16:41 |
| prometheanfire | :D | 16:42 |
| *** gouthamr has joined #openstack-ansible | 16:42 | |
| mhayden | cloudnull: might need to ask jorgem to join our channel | 16:42 |
| *** adrian_otto has quit IRC | 16:43 | |
| *** janki has joined #openstack-ansible | 16:43 | |
| *** adrian_otto has joined #openstack-ansible | 16:46 | |
| *** adrian_otto has quit IRC | 16:46 | |
| castulo | prometheanfire: Hmmm, the upgrade failed with the same error msg: the field 'args' has an invalid value, which appears to include a variable that is undefined. The error was: 'osa_secrets_file_name' is undefined | 16:46 |
| prometheanfire | castulo: well, this sucks... | 16:47 |
| *** adrian_otto has joined #openstack-ansible | 16:48 | |
| prometheanfire | cloudnull: ^ | 16:48 |
| *** rk4n has quit IRC | 16:49 | |
| stevelle | how? | 16:49 |
| openstackgerrit | Travis Truman (automagically) proposed openstack/openstack-ansible-os_cinder: Ansible 2.1.1 update https://review.openstack.org/353569 | 16:50 |
| prometheanfire | now brown cow? | 16:50 |
| *** adrian_otto1 has joined #openstack-ansible | 16:53 | |
| *** d3n14l has quit IRC | 16:54 | |
| *** asettle has joined #openstack-ansible | 16:55 | |
| prometheanfire | castulo: I think it's because of the way we call each upgrade task, we only call it as a task, not a play | 16:55 |
| *** adrian_otto has quit IRC | 16:57 | |
| *** rk4n has joined #openstack-ansible | 16:57 | |
| openstackgerrit | Travis Truman (automagically) proposed openstack/openstack-ansible-tests: Address ansible_ssh_* var deprecation https://review.openstack.org/369697 | 16:58 |
| castulo | so maybe including that variable when calling the task? | 16:58 |
| *** McMurlock1 has joined #openstack-ansible | 16:59 | |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible: load variables file explicitly https://review.openstack.org/370981 | 17:01 |
| prometheanfire | castulo: ^ | 17:01 |
| *** asettle has quit IRC | 17:01 | |
| *** TxGirlGeek has joined #openstack-ansible | 17:03 | |
| *** adrian_otto1 has quit IRC | 17:04 | |
| *** adrian_otto has joined #openstack-ansible | 17:05 | |
| castulo | prometheanfire: I'm a complete noob in ansible, so I'm not sure what you mean :S | 17:06 |
| prometheanfire | castulo: try that :P | 17:07 |
| castulo | prometheanfire: I see you submitted a new patch, let me give it a try | 17:07 |
| prometheanfire | :D | 17:07 |
| jamesdenton | cloudnull For what it's worth, it took the 'git clean -d -f' and a 'git stash' to finish the repo build | 17:08 |
| prometheanfire | git gc is nice too, for long lived repos :D | 17:09 |
| cloudnull | so clean didn't do the needful? | 17:09 |
| jamesdenton | not by itself, no | 17:10 |
| *** d3n14l has joined #openstack-ansible | 17:10 | |
| cloudnull | that's a bummer | 17:10 |
| cloudnull | jamesdenton: i wonder, can you recreate that issue? | 17:11 |
| jamesdenton | hmmm, good question. | 17:12 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-repo_server: Ensure the repo server has a valid log directory https://review.openstack.org/370931 | 17:12 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-repo_build: Add git cleanup to op-clone-script https://review.openstack.org/370940 | 17:12 |
| cloudnull | as prometheanfire suggested i added git gc after clean | 17:13 |
| cloudnull | that "should" do the needful. | 17:13 |
| prometheanfire | cloudnull: git gc only makes git faster | 17:13 |
| prometheanfire | it doesn't remove changes like git stash does | 17:13 |
| cloudnull | it should remove unreachable object. | 17:13 |
| cloudnull | object(s) | 17:14 |
| prometheanfire | ya | 17:14 |
| prometheanfire | didn't know that was the problem | 17:14 |
| jamesdenton | gotcha | 17:14 |
| cloudnull | IDK if that's the specific problem | 17:14 |
| jamesdenton | so it wouldn't complain about this? error: Your local changes to the following files would be overwritten by checkout: | 17:14 |
| cloudnull | but i'm thinking it is. | 17:14 |
| cloudnull | See i think that someone force pushed, which is causing that | 17:15 |
| cloudnull | unless you changed thoes files. | 17:15 |
| cloudnull | ? | 17:15 |
| jamesdenton | no, i did not change those files | 17:15 |
| openstackgerrit | Kyle L. Henderson proposed openstack/openstack-ansible-os_trove: Update role with fixes for AIO testing https://review.openstack.org/370880 | 17:15 |
| castulo | prometheanfire: that patch did not work either: I'm getting "ERROR! 'include_vars' is not a valid attribute for a Play" | 17:15 |
| jamesdenton | this environment was a very basic install from master about a month ago or less, then attempting upgrade | 17:15 |
| jamesdenton | going through the playbook runs to upgrade | 17:16 |
| cloudnull | ok | 17:16 |
| *** McMurlock1 has quit IRC | 17:16 | |
| cloudnull | so yea, i think someone did something miserable to the git tree, you had a checkout from before, and then after the fetch things were foobar. | 17:16 |
| jamesdenton | possibly | 17:17 |
| prometheanfire | cloudnull: I moved that to the wrong location | 17:17 |
| prometheanfire | bah | 17:17 |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible: load variables file explicitly https://review.openstack.org/370981 | 17:17 |
| prometheanfire | castulo: ^ | 17:17 |
| cloudnull | well then move it to the right onw: ) | 17:17 |
| * cloudnull helper | 17:17 | |
| prometheanfire | castulo: you can test quickly by running openstack-ansible user-secrets-adjustment.yml | 17:18 |
| cloudnull | *one | 17:18 |
| prometheanfire | instead of running the script (that's what the script runs) | 17:18 |
| *** rk4n has quit IRC | 17:21 | |
| castulo | prometheanfire: ups I didn't see your message on time, I'm running the script again, but that last fix seems to be good, looks like the script is past the point where it was breaking. Seems to me like it is now running the setup-hosts.yml playbook | 17:28 |
| prometheanfire | cool | 17:29 |
| castulo | prometheanfire: thanks for looking into that :) | 17:31 |
| prometheanfire | well, I caused it | 17:31 |
| castulo | :P | 17:31 |
| *** electrofelix has quit IRC | 17:31 | |
| *** d3n14l has quit IRC | 17:36 | |
| *** adrian_otto has quit IRC | 17:36 | |
| *** KLevenstein has quit IRC | 17:36 | |
| odyssey4me | o/ | 17:37 |
| *** joseg has joined #openstack-ansible | 17:38 | |
| cloudnull | odyssey4me: go away, have fun :) | 17:38 |
| odyssey4me | Attilla apologies, I have been distracted today - Newton RC1 release and a team outing too | 17:38 |
| *** smatzek has joined #openstack-ansible | 17:39 | |
| *** janki has quit IRC | 17:40 | |
| openstackgerrit | Kyle L. Henderson proposed openstack/openstack-ansible-os_trove: Update role with fixes for AIO testing https://review.openstack.org/370880 | 17:42 |
| odyssey4me | cloudnull on a train right now - heading for the office to collect my car, then home | 17:44 |
| odyssey4me | it should be close to time to be able to pin the roles... but I see that nova role patch is still outstanding | 17:44 |
| cloudnull | yup | 17:44 |
| * odyssey4me is thinking of reducing test coverage to help get it through | 17:44 | |
| odyssey4me | we know it works, but there are bad races from upstream shenanigans | 17:45 |
| cloudnull | yup | 17:45 |
| cloudnull | its frustrating . | 17:45 |
| cloudnull | we're babysitting it | 17:45 |
| odyssey4me | so perhaps we should remove the ssh test or something from the tempest config? | 17:45 |
| odyssey4me | we've been babysitting it for over 24 hours | 17:45 |
| cloudnull | yup | 17:45 |
| odyssey4me | we need a pinned set of roles and upstream SHA's done in the integrated role in the next two hours, so how can we do this | 17:48 |
| odyssey4me | I need to have that merged and submit a release request today | 17:48 |
| hughmFLEXin | so i redeployed with br-ex on the controller nodes...added br-ex to the user_config...but now instances fail to spawn because they can't bind ports, neutron log throwing me this... | 17:50 |
| hughmFLEXin | ERROR neutron.plugins.ml2.drivers.linuxbridge.agent.linuxbridge_neutron_agent [-] Interface eth12 for physical network external does not exist. Agent terminated! | 17:50 |
| *** imp3tis has joined #openstack-ansible | 17:51 | |
| *** adrian_otto has joined #openstack-ansible | 17:52 | |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible: load variables as a simple var for upgrades https://review.openstack.org/370981 | 17:53 |
| *** adrian_otto1 has joined #openstack-ansible | 17:56 | |
| *** adrian_otto has quit IRC | 17:57 | |
| cloudnull | odyssey4me: well shit. its failed again on xenial due to the same stupid race. | 17:57 |
| *** adrian_otto1 has quit IRC | 17:57 | |
| cloudnull | hughmFLEXin: do you have eth12 on your compute nodes? | 17:58 |
| cloudnull | I suspect you have the flat network config in your openstack_user_config.yml | 17:58 |
| hughmFLEXin | no, but i also don't remember putting "host_bind_override: eth12" in the user_config eitehr | 17:58 |
| cloudnull | which points at eth12 | 17:58 |
| hughmFLEXin | but it is there | 17:58 |
| cloudnull | if you remove that entry and rerun ``openstack-ansible os-neutron-install.yml --tags neutron-config`` it'll be happy | 17:59 |
| cloudnull | or you can add eth12 to your compute nodes as a veth pair off the br-vlan interface | 17:59 |
| cloudnull | and it too will be happy | 17:59 |
| *** TxGirlGeek has quit IRC | 17:59 | |
| cloudnull | example https://github.com/openstack/openstack-ansible/blob/master/etc/network/interfaces.d/aio_interfaces.cfg#L52-L59 | 18:00 |
| hughmFLEXin | i love how you can reconfigure and re-run the plays for specific services | 18:01 |
| hughmFLEXin | its freakin awesome | 18:01 |
| hughmFLEXin | cloudnull: removing the entry and rerunning the play fixed it | 18:03 |
| *** KLevenstein has joined #openstack-ansible | 18:08 | |
| cloudnull | hughmFLEXin: awesome!! | 18:09 |
| hughmFLEXin | cloudnull: hmmm i think my network settings are still haywire somewhere, both ext and int networks are created with router in between, floating ip assigned, secgroups for icmp open. But still can't ping the floating ip | 18:10 |
| jamesdenton | hughmFLEXin did your instance boot and get its own IP? | 18:12 |
| hughmFLEXin | yea | 18:12 |
| jamesdenton | are you able to ping the instance from within the qrouter namespace? | 18:12 |
| *** gonzalo2kx has quit IRC | 18:12 | |
| hughmFLEXin | let me try | 18:12 |
| *** asettle has joined #openstack-ansible | 18:13 | |
| *** weezS has quit IRC | 18:15 | |
| odyssey4me | Attilla your log shows a ton of connectivity losses with the database... what's your kernel version? Which 14.04 are you using? | 18:15 |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible: load variables as a simple var for upgrades https://review.openstack.org/370981 | 18:16 |
| odyssey4me | jmccrory cloudnull the os_nova role is perpetually failing on the ssh connect test in at least one of the tests - I vote to remove that part of the test from the tempest config as we're covered in the integrated build test anyway | 18:20 |
| odyssey4me | we can't continue babysitting patches through like this | 18:21 |
| odyssey4me | can someone pick up a patch to do that? meanwhile of course we keep babysitting | 18:21 |
| *** TxGirlGeek has joined #openstack-ansible | 18:21 | |
| *** pcaruana has joined #openstack-ansible | 18:22 | |
| odyssey4me | michaelgugino can you please make the edits for https://review.openstack.org/365098 requested - I'd like that patch included in the release | 18:23 |
| *** will-hodor has quit IRC | 18:23 | |
| michaelgugino | odyssey4me: you got it | 18:23 |
| odyssey4me | cloudnull Matias I see that we have some work needed to be done on https://review.openstack.org/369729 and https://review.openstack.org/369745 - can we try and make that happen ASAP please? | 18:24 |
| odyssey4me | that seems important for the Newton release to me | 18:24 |
| odyssey4me | thanks michaelgugino - please ping rromans when you're done for a re-review | 18:24 |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible: load variables as a simple var for upgrades https://review.openstack.org/370981 | 18:25 |
| openstackgerrit | Matthew Thode proposed openstack/openstack-ansible: load variables as a simple var for upgrades https://review.openstack.org/370981 | 18:25 |
| odyssey4me | castulo ^ are you able to test that updated version please? | 18:26 |
| *** sdake_ is now known as sdake | 18:26 | |
| hughmFLEXin | jamesdenton: i changed internal network to vxlan, and from the qdhcp space i can ping it | 18:27 |
| cloudnull | odyssey4me: +1 | 18:28 |
| cloudnull | if that gets us through the gate, so be it | 18:28 |
| odyssey4me | cloudnull can you work out a patch to os_nova to make that happen? I need to relocate home and hope to have a solution in place by then | 18:29 |
| odyssey4me | I need to have the pinned roles set and the release request out in the next 90 mins or so, otherwise we miss the deadline. | 18:29 |
| openstackgerrit | Michael Gugino proposed openstack/openstack-ansible-os_nova: [DOCS] Update docs for mixed lxd hypervisors https://review.openstack.org/365098 | 18:30 |
| michaelgugino | rromans: ^^ | 18:30 |
| odyssey4me | I'll do the pinned role request when I get home. If the patch misses RC1 then so be it, we'll end up having to get it merged and backport it. | 18:30 |
| odyssey4me | bbiab - around 45 mins | 18:30 |
| cloudnull | ok | 18:30 |
| *** asettle has quit IRC | 18:34 | |
| jamesdenton | cloudnull looks like there is a similar problem with apt-cacher-ng log directory not existing in repo container, causing additional tasks to fail | 18:37 |
| jamesdenton | http://paste.openstack.org/show/577680/ | 18:37 |
| jamesdenton | hughmFLEXin progress! | 18:39 |
| cloudnull | jamesdenton: i think this will be a reoccuring issue if your newton(master) install was originall done before this commit | 18:40 |
| cloudnull | https://github.com/openstack/openstack-ansible/commit/992e616046a737357183394808ea5a0c48a49065 | 18:40 |
| jamesdenton | ok | 18:41 |
| *** itsuugo has quit IRC | 18:42 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-os_watcher: Use Ansible 2.1.1 for role testing https://review.openstack.org/370828 | 18:43 |
| *** itsuugo has joined #openstack-ansible | 18:44 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_nova: For tempest to run serial https://review.openstack.org/371055 | 18:46 |
| *** d3n14l has joined #openstack-ansible | 18:47 | |
| *** AnarchyAo has joined #openstack-ansible | 18:47 | |
| jamesdenton | cloudnull So the mount appears to be there | 18:48 |
| cloudnull | yea, so if you unmount all of those missing dirs will be back | 18:50 |
| castulo | odyssey4me: sorry I stepped away from the computer to go to a meeting, let me try the new patch now | 18:52 |
| jamesdenton | cloudnull yep, thats exactly what happened. | 18:56 |
| *** TxGirlGeek has quit IRC | 18:56 | |
| *** TxGirlGeek has joined #openstack-ansible | 18:57 | |
| cloudnull | so a mitaka to newton rc1 would not have these issues. but i we might want to do a bit of shuffling so we can find/handle edge cases. | 18:57 |
| cloudnull | -cc castulo luzC | 18:57 |
| jamesdenton | cloudnull what would handle copying the existing /var/log data from the container to the host then? | 18:57 |
| jamesdenton | or is that not an issue | 18:57 |
| hughmFLEXin | jamesdenton: so i can ping the vm, and when i ssh into the vm through the qrouter space, i can ping both ext and internal router interfaces along with the gateway. But from outside i still can't ping the vm floating ip... | 18:58 |
| cloudnull | jamesdenton: it might be an issue though i've not encountered that just yet | 18:59 |
| jamesdenton | fair enough | 18:59 |
| cloudnull | we can use the /var/backups as a staging point, copy all of the logs there, move them to the new mount point, and then carry on. | 18:59 |
| *** weezS has joined #openstack-ansible | 18:59 | |
| *** KLevenstein has quit IRC | 18:59 | |
| jamesdenton | hughmFLEXin When you say you can ping the gateway, you mean the gateway for the router on the flat network? | 19:00 |
| cloudnull | what would ensure we dont hit these kinds of issues. | 19:00 |
| cloudnull | but it'd be something that we'd have to do in a specific upgrade task. | 19:00 |
| jamesdenton | sure. i don't want to create any more work than necessary, but it may show up during an upgrade | 19:01 |
| hughmFLEXin | i can ping the vxlan gateway | 19:01 |
| cloudnull | which would still not help an installation that was done before that commit back in august. | 19:01 |
| *** hybridpollo has joined #openstack-ansible | 19:02 | |
| jamesdenton | hughmFLEXin Where are you pinging from? I would perform packet captures along all related interfaces. The physical server interface, br-vlan, the veth to the container, and eth11/eth12 and brq bridge inside the container | 19:03 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-os_watcher: Use Ansible 2.1.1 for role testing https://review.openstack.org/370828 | 19:03 |
| jamesdenton | you might also check the router has a default gateway. that IP should match the 'gateway_ip' of the external network | 19:04 |
| *** KLevenstein has joined #openstack-ansible | 19:04 | |
| cloudnull | jamesdenton: it might. I know luzC and castulo are testing upgrades (mitaka > newton) so it'd be great to see if we can catch the edge cases there. | 19:04 |
| prometheanfire | stevelle: gonna pull a neutron on me? | 19:08 |
| castulo | odyssey4me: before trying this patch again https://review.openstack.org/#/c/370981/7 I swich back to master to make sure if running the run-upgrade.sh from master would fail as it was failing before, but this time it worked. Could that be caused because I had already run that playbook successfully in the past? | 19:09 |
| cloudnull | if cores are around it'd be great to see this through https://review.openstack.org/#/c/370940/ | 19:11 |
| cloudnull | https://review.openstack.org/#/c/370931/ | 19:11 |
| cloudnull | https://review.openstack.org/#/c/370917/ | 19:12 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_nova: Force tempest to run serial and retry tests https://review.openstack.org/371055 | 19:14 |
| luzC | cloudnull, jamesdenton I kind of lost the conversation thread... what you are saying is that the log issue (which will apply for all log folders) is due to the patch merged on Aug hence bug will appear when upgrading mitaka to newton(after patch)... like in our case that are upgrading mitaka to newton-master... is that correct? | 19:14 |
| cloudnull | no. | 19:15 |
| cloudnull | but maybe | 19:15 |
| cloudnull | :) | 19:15 |
| luzC | hahaha | 19:15 |
| cloudnull | there was a patch merged in august for bind mounting the log dirs | 19:15 |
| cloudnull | in jamesdenton's case he had a newton deploy running before that patch was merged. | 19:15 |
| jamesdenton | well, given that the container will be bind mounting the log directory, and those dirs don't exist, i don't see how you WON'T run into it | 19:15 |
| cloudnull | so things were a little out of whack | 19:16 |
| jamesdenton | yeah maybe ::P | 19:16 |
| cloudnull | the log dirs will be recreated when the service role runs | 19:16 |
| cloudnull | which should mean that we wont this issue on upgrade. | 19:16 |
| jamesdenton | gotcha | 19:16 |
| cloudnull | and things like the apt-cacher-ng log files will be created because thats a new thing installed in newton | 19:17 |
| jamesdenton | leave it to me to be difficult | 19:17 |
| cloudnull | BUT if we do see these issues we should see about adding an upgrade step | 19:17 |
| cloudnull | and maybe we want to have an upgrade step anyway | 19:17 |
| cloudnull | idk | 19:17 |
| *** jimbaker has quit IRC | 19:17 | |
| *** jimbaker has joined #openstack-ansible | 19:17 | |
| jamesdenton | well if what you're saying is true, then maybe https://review.openstack.org/#/c/370931/ isn't necessary? | 19:18 |
| openstackgerrit | Merged openstack/openstack-ansible-rsyslog_client: Add check to ensure a target log dir exists https://review.openstack.org/370917 | 19:18 |
| cloudnull | i think it should be there, the role should've been doing that already | 19:19 |
| jamesdenton | k | 19:19 |
| cloudnull | so valid bug imo | 19:19 |
| cloudnull | anyway luzC castulo as you're testing things let us know how it goes. | 19:19 |
| cloudnull | and if somehting needs fixing lets get that on the map | 19:20 |
| *** gonzalo2kx has joined #openstack-ansible | 19:20 | |
| castulo | sure | 19:20 |
| luzC | ok, get it.. | 19:20 |
| cloudnull | it'd be really nice to see this all worked out before release. | 19:20 |
| cloudnull | so I really appreciate the cycles you all are putting into it -- luzC castulo jamesdenton :) | 19:21 |
| jamesdenton | no problem :) I forgot why I started this in the first place... | 19:22 |
| *** d3n14l has quit IRC | 19:23 | |
| odyssey4me | o/ | 19:24 |
| cloudnull | jamesdenton: because you're an amazing individual and an all around good netizen :) | 19:25 |
| jamesdenton | oh brother | 19:25 |
| cloudnull | hahaha | 19:25 |
| jamesdenton | upgrade by accident is more like it | 19:25 |
| odyssey4me | cloudnull figured out a solution for os_nova yet? | 19:26 |
| *** ametts has joined #openstack-ansible | 19:26 | |
| cloudnull | trying a thing or two before I remove the tests. | 19:26 |
| cloudnull | https://review.openstack.org/#/c/371055/ | 19:26 |
| odyssey4me | cloudnull I was just thinking to set run_validation to False: http://docs.openstack.org/developer/tempest/configuration.html#ssh-connection-configuration | 19:27 |
| odyssey4me | ie set the var tempest_compute_run_ssh to False | 19:28 |
| cloudnull | but then we may as well just comment out the tempest run all together. | 19:28 |
| cloudnull | it'd create a network , volume, flavor and be done. | 19:29 |
| odyssey4me | nope - it'll still build an instance and do other validations | 19:29 |
| odyssey4me | it just won't try to ssh to the instance | 19:29 |
| cloudnull | but an instance you cant use is worthless. | 19:29 |
| odyssey4me | agreed, so I'd rather avoid it - but we also need to merge patches, and our failsafe is the integrated repo which still tests that test | 19:30 |
| odyssey4me | it doesn't seem to be hitting the same issues at the os_nova role. | 19:30 |
| openstackgerrit | Merged openstack/openstack-ansible-repo_server: Ensure the repo server has a valid log directory https://review.openstack.org/370931 | 19:30 |
| cloudnull | its pinned to a specific version | 19:31 |
| cloudnull | where the nova role is pulling from master | 19:31 |
| cloudnull | https://github.com/openstack/openstack-ansible/blob/master/playbooks/defaults/repo_packages/openstack_services.yml#L147 | 19:32 |
| odyssey4me | so? | 19:32 |
| cloudnull | so it's likely that changes between then and now are causing us grief | 19:32 |
| *** pcaruana has quit IRC | 19:32 | |
| odyssey4me | and upstream does not do the run_validation test | 19:32 |
| cloudnull | which may not be specifically the nova role. | 19:33 |
| openstackgerrit | Marc Gariépy proposed openstack/openstack-ansible-galera_client: Centos: Remove dir before trying to create symlink https://review.openstack.org/371073 | 19:33 |
| cloudnull | it could be nova, neutron, tempest, etc | 19:33 |
| odyssey4me | sure, we already know that we're hitting a pretty bad race condition for neutron | 19:33 |
| odyssey4me | that's why we're similarly having issues merging patches there | 19:33 |
| odyssey4me | so we're stuck with not being able to merge fixes to our code because of shitty upstream code | 19:33 |
| cloudnull | sounds like it. | 19:34 |
| odyssey4me | I'd rather relax our upstream test validation to fix our code, then work with upstream to fix their stuff and restore our testing again. | 19:34 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova: [DOCS] Update docs for mixed lxd hypervisors https://review.openstack.org/365098 | 19:34 |
| cloudnull | but we'll see if adding a retry to the test makes things happier | 19:34 |
| odyssey4me | I definitely don't want to remove the tests - just the one part where we're failing regularly while we try to dig into why it's happening. | 19:34 |
| *** asettle has joined #openstack-ansible | 19:34 | |
| odyssey4me | IT's especially important because we don't see the same failure in the integrated build, or in local tests. | 19:35 |
| odyssey4me | sure, maybe a retry helps | 19:35 |
| odyssey4me | I like a retry more than removal. | 19:35 |
| odyssey4me | ugh, a ton of patches are merging into nova too | 19:36 |
| openstackgerrit | Merged openstack/openstack-ansible-repo_build: Add git cleanup to op-clone-script https://review.openstack.org/370940 | 19:36 |
| cloudnull | yup-- its a race to the bottom | 19:37 |
| *** jimbaker is now known as jimbaker_ | 19:38 | |
| odyssey4me | ok, I guess we need a pinning patch for the integrated build with whatever our current role states are - we'll have to backport whatever patches we feel must get into Newton that merge after the pins. | 19:38 |
| cloudnull | https://review.openstack.org/#/c/371055/ | 19:38 |
| *** jimbaker_ is now known as jimbaker` | 19:39 | |
| *** jimbaker` is now known as jimbaker | 19:39 | |
| cloudnull | if we merge that, and rebase the nova patches i think we're out of the woods | 19:39 |
| *** asettle has quit IRC | 19:40 | |
| *** david-lyle has quit IRC | 19:40 | |
| *** david-lyle has joined #openstack-ansible | 19:40 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_nova: Fix novnc console include positioning https://review.openstack.org/370501 | 19:41 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_nova: Fix novnc console include positioning https://review.openstack.org/370501 | 19:42 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_nova: Force tempest to run serial and retry tests https://review.openstack.org/371055 | 19:42 |
| odyssey4me | I actually think that running tempest in that way may be something we want to do everywhere. | 19:42 |
| cloudnull | its not a bad idea | 19:42 |
| cloudnull | given the amount failures we see from transient problems | 19:42 |
| odyssey4me | yep | 19:43 |
| odyssey4me | and we need to convert the neutron role to use tempest instead of our own home grown test | 19:43 |
| cloudnull | yup | 19:43 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-os_watcher: Use Ansible 2.1.1 for role testing https://review.openstack.org/370828 | 19:43 |
| mhayden | this ansible 2.1.1 business is making me lose hairs | 19:43 |
| cloudnull | hahaha | 19:44 |
| odyssey4me | interesting - all tests passed first time | 19:44 |
| cloudnull | odyssey4me: its possible they've fixed the bug | 19:44 |
| cloudnull | or we were lucky | 19:44 |
| odyssey4me | or the serial via CLI instead of env var is working better | 19:45 |
| cloudnull | that too | 19:45 |
| odyssey4me | a lesson to carry to the central tests repo | 19:45 |
| cloudnull | mhayden: for the 211 role | 19:46 |
| cloudnull | you'll need to add in the tox role link | 19:46 |
| cloudnull | https://review.openstack.org/#/c/353616/10/tox.ini@123 | 19:46 |
| odyssey4me | nishpatwa_ feedback in https://review.openstack.org/370914 | 19:46 |
| mhayden | cloudnull: OOOOOOOOOOOOORLY | 19:46 |
| cloudnull | odyssey4me: i think once everything is on the central testing repo life will be a lot better. | 19:46 |
| odyssey4me | lemme just edit it | 19:46 |
| cloudnull | we have A LOT of tests but having a single source of truth will ensure we dont hit strange edge cases. | 19:47 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: [DOCS] Added missing level info for haproxy_hosts https://review.openstack.org/370914 | 19:47 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-os_watcher: Use Ansible 2.1.1 for role testing https://review.openstack.org/370828 | 19:47 |
| odyssey4me | cloudnull yep, agreed | 19:47 |
| odyssey4me | mhayden ^ revisit of the haproxy_hosts would be great | 19:47 |
| mhayden | what type of revisiting are we talking about? :) | 19:48 |
| * mhayden has his hands into too many pies today | 19:48 | |
| cloudnull | pie?! | 19:48 |
| cloudnull | i love pie | 19:48 |
| *** sdake has quit IRC | 19:49 | |
| *** sdake_ has joined #openstack-ansible | 19:49 | |
| automagically | mhayden - 2.1.1 is driving me nuts too | 19:49 |
| openstackgerrit | Nish Patwa(nishpatwa_) proposed openstack/openstack-ansible: [DOCS] Added missing level info for haproxy_hosts https://review.openstack.org/370914 | 19:49 |
| automagically | No idea why I’m seeing such volatility on https://review.openstack.org/#/c/370904/ | 19:49 |
| mhayden | awww, nishpatwa_ you had two +2's and workflow :P | 19:50 |
| odyssey4me | nishpatwa_ I added a little extra to that last patch set which you've wiped out :) | 19:50 |
| nishpatwa_ | May be I uploaded the patch | 19:50 |
| nishpatwa_ | and it got conflct with yours | 19:51 |
| mhayden | i'll add odyssey4me's bit back in right quick, nishpatwa_ | 19:51 |
| cloudnull | automagically: looks like "ansible_hostname" is not getting set ? | 19:51 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible: [DOCS] Added missing level info for haproxy_hosts https://review.openstack.org/370914 | 19:52 |
| cloudnull | i wonder if you put that specifically in inventory if it'll have the same issue? | 19:52 |
| cloudnull | maybe a fact isn't getting set? | 19:52 |
| automagically | cloudnull: Yeah, but it only happens intermittently | 19:52 |
| odyssey4me | automagically always trusty? | 19:52 |
| automagically | And it seems like only on trusty | 19:52 |
| *** itsuugo has quit IRC | 19:53 | |
| cloudnull | interesting | 19:53 |
| nishpatwa_ | apology odyssey4me for the inconvenience | 19:53 |
| openstackgerrit | Marc Gariépy proposed openstack/openstack-ansible-os_cinder: use 2 spaces instead of 1 for pkg list. https://review.openstack.org/371080 | 19:53 |
| *** rk4n has joined #openstack-ansible | 19:53 | |
| cloudnull | i gotta run for a few, back in a bit | 19:55 |
| *** itsuugo has joined #openstack-ansible | 19:55 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-os_watcher: Use Ansible 2.1.1 for role testing https://review.openstack.org/370828 | 19:58 |
| *** asettle has joined #openstack-ansible | 19:58 | |
| openstackgerrit | Travis Truman (automagically) proposed openstack/openstack-ansible-tests: Address ansible_ssh_* var deprecation https://review.openstack.org/369697 | 19:59 |
| *** ametts has quit IRC | 20:01 | |
| *** TxGirlGeek has quit IRC | 20:04 | |
| *** Ravi_ has joined #openstack-ansible | 20:07 | |
| mhayden | i keep getting infra_eth1 doesn't exist, automagically | 20:07 |
| mhayden | http://logs.openstack.org/28/370828/6/check/gate-openstack-ansible-os_watcher-ansible-func-ubuntu-trusty/46c5433/console.html#_2016-09-15_20_05_19_890400 | 20:07 |
| mhayden | haven't figured out why yet | 20:07 |
| * automagically looking mhayden | 20:09 | |
| *** hughmFLEXin has quit IRC | 20:10 | |
| *** d3n14l has joined #openstack-ansible | 20:11 | |
| *** d3n14l has left #openstack-ansible | 20:11 | |
| *** Ravi_ has quit IRC | 20:12 | |
| *** hughmFLEXin has joined #openstack-ansible | 20:12 | |
| openstackgerrit | Marc Gariépy proposed openstack/openstack-ansible-os_cinder: [WIP] Add centos support https://review.openstack.org/371088 | 20:13 |
| prometheanfire | mhayden: do we hardcode interface names? | 20:13 |
| mhayden | in the tests dir, i think we do | 20:13 |
| *** ametts has joined #openstack-ansible | 20:13 | |
| prometheanfire | because that's gonna bite us in xenial | 20:13 |
| mhayden | i'm still a little green on some of the latest testing changes | 20:13 |
| prometheanfire | devstack had to change to dynamically set it | 20:14 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-openstack_hosts: Remove requiretty for sudo on centos hosts. https://review.openstack.org/370807 | 20:18 |
| odyssey4me | mgagne mhayden ^ minor tweak, otherwise ansible interprets the number differently | 20:18 |
| *** asettle has quit IRC | 20:18 | |
| automagically | mhayden: Looks like you need to create a version of https://github.com/openstack/openstack-ansible-tests/blob/master/network_config.cfg.j2.sample for watcher | 20:18 |
| mhayden | odyssey4me: ah good catch | 20:18 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-lxc_hosts: Remove requiretty for sudo on centos base image. https://review.openstack.org/370808 | 20:18 |
| mhayden | automagically: oh wow | 20:19 |
| mhayden | odyssey4me: nice catch on the bare integer too -- that woulda made ansible sad | 20:19 |
| odyssey4me | mhayden automagically yeah, I'd suggest checking the lxc_container_create role for inspiration | 20:19 |
| mhayden | "inspiration" | 20:20 |
| mhayden | more like perspiration right now | 20:20 |
| odyssey4me | mhayden automagically I'd appreciate it currently integrated roles take precedence in the Ansible 2.x work though... not sure if those are all done yet? | 20:20 |
| *** rk4n has quit IRC | 20:21 | |
| automagically | odyssey4me: Yeah, agreed. Been trying to work out the kinks in rabbitmq_server and openstack-ansible-tests first | 20:21 |
| mhayden | odyssey4me: good point -- i was hacking on watcher hoping it would be an easy one :P | 20:21 |
| automagically | Looks like some common failures that will effect everything using the tests repo like: http://logs.openstack.org/89/353589/7/check/gate-openstack-ansible-os_glance-ansible-func-ubuntu-trusty/81daf91/console.html#_2016-09-14_13_48_15_384552 | 20:22 |
| cloudnull | also mhayden if you dont have the bridge defined you will need that in your localhost host_vars file | 20:23 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible: Updated from global requirements https://review.openstack.org/371113 | 20:23 |
| cloudnull | like os https://review.openstack.org/#/c/353612/28/tests/host_vars/localhost.yml | 20:23 |
| cloudnull | *so | 20:23 |
| mhayden | i have that -- should i only have br-mgmt there? | 20:24 |
| cloudnull | yea | 20:24 |
| cloudnull | well... | 20:24 |
| cloudnull | idk does watcher use other networks? | 20:25 |
| odyssey4me | need another core review on https://review.openstack.org/371113 | 20:25 |
| odyssey4me | michaelgugino ^ re: watcher | 20:25 |
| michaelgugino | ? | 20:25 |
| automagically | later all, head sore from running into 2.1.1 walls all day | 20:26 |
| michaelgugino | odyssey4me: I'm not following | 20:27 |
| odyssey4me | cloudnull it looks like Matias is not around, can you update https://review.openstack.org/369729 & https://review.openstack.org/369745 accordingly? | 20:27 |
| odyssey4me | night automagically - thanks for pitching in | 20:27 |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_hosts: Remove requiretty for sudo on centos hosts. https://review.openstack.org/370807 | 20:27 |
| odyssey4me | michaelgugino mhayden and cloudnull were asking about watcher's network needs | 20:27 |
| Matias | odyssey4me: I'm updating it right now | 20:27 |
| odyssey4me | Matias ah, great thanks | 20:27 |
| openstackgerrit | Paulo Matias proposed openstack/openstack-ansible-memcached_server: Adjust file descriptor limit when systemd is used https://review.openstack.org/369745 | 20:27 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder: use 2 spaces instead of 1 for pkg list. https://review.openstack.org/371080 | 20:28 |
| michaelgugino | watcher doesn't have any specific network needs | 20:28 |
| Matias | but we will still need galera in order to close the bug | 20:28 |
| michaelgugino | watch primarily consumes / interacts with other openstack services. | 20:28 |
| michaelgugino | *watcher | 20:29 |
| odyssey4me | Matias what needs fixing in galera? | 20:29 |
| *** hughmFLEXin has quit IRC | 20:29 | |
| *** hughmFLEXin has joined #openstack-ansible | 20:31 | |
| palendae | odyssey4me, Is that a loaded question? :) | 20:31 |
| odyssey4me | heh :p | 20:32 |
| *** TxGirlGeek has joined #openstack-ansible | 20:32 | |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts: Remove requiretty for sudo on centos base image. https://review.openstack.org/370808 | 20:32 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-os_watcher: Use Ansible 2.1.1 for role testing https://review.openstack.org/370828 | 20:32 |
| odyssey4me | heh, and I thought this was a joke: https://github.com/webscalesql/webscalesql-5.6 | 20:32 |
| Matias | odyssey4me: fd limit too, it is also listed in the bug | 20:33 |
| openstackgerrit | Nolan Brubaker proposed openstack/openstack-ansible: Add test for Horizon group membership https://review.openstack.org/371140 | 20:33 |
| * odyssey4me submits a patch to convert all OSA DB's to http://www.mongodb-is-web-scale.com/ | 20:33 | |
| palendae | ZODB. | 20:34 |
| palendae | It's native python. | 20:34 |
| odyssey4me | Matias I expect that rabbitmq has the same issue - the fd limit default there is pretty low | 20:34 |
| odyssey4me | Matias typey typey :) | 20:34 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-os_watcher: Use Ansible 2.1.1 for role testing https://review.openstack.org/370828 | 20:34 |
| palendae | So odyssey4me, that review I just posted was a test with the environment you submitted in https://bugs.launchpad.net/openstack-ansible/+bug/1623584. The horizon_all hosts is empty like it should be though O.o | 20:35 |
| openstack | Launchpad bug 1623584 in openstack-ansible "Dynamic inventory clash between host group name and container group name" [Undecided,Confirmed] - Assigned to Nolan Brubaker (nolan-brubaker) | 20:35 |
| *** d3n14l has joined #openstack-ansible | 20:35 | |
| odyssey4me | palendae ? looking | 20:35 |
| *** d3n14l has left #openstack-ansible | 20:36 | |
| Matias | dude I'm suffering from sleep deprivation | 20:36 |
| Matias | please fix bugs for me lol | 20:36 |
| odyssey4me | Matias welcome to the club :p | 20:36 |
| Matias | I also want some patch for the gnocchi grafana plugin done by yesterday, please | 20:36 |
| Matias | lol | 20:36 |
| Matias | let me fix the rabbit PR | 20:37 |
| odyssey4me | palendae ah, you're missing the conf.d bits to activate it and make it show the issue | 20:38 |
| palendae | Oh, horizon_hosts | 20:38 |
| palendae | Ok | 20:38 |
| odyssey4me | palendae without the conf.d bits, no host is added to any groups | 20:38 |
| odyssey4me | all the env.d bits do is make the group exist | 20:38 |
| palendae | I spaced and though those were already in there | 20:38 |
| palendae | Right | 20:38 |
| odyssey4me | yeah, be aware the the current master has a host allocated to 'deshboard_hosts' | 20:39 |
| odyssey4me | do you may see a clash of content | 20:39 |
| Matias | odyssey4me: RC1 is today? | 20:39 |
| odyssey4me | Matias I need to pin all SHA's for roles and get it merged today | 20:39 |
| palendae | odyssey4me, If that happens I'll just have the dashboard_hosts removed | 20:39 |
| mhayden | prometheanfire: so that netaddr requirement went in after all? ;) | 20:40 |
| odyssey4me | with that I do the release request, and that become RC1 and the branch of stable/newton | 20:40 |
| odyssey4me | from then on, stable policy rules apply | 20:40 |
| prometheanfire | mhayden: yep | 20:40 |
| odyssey4me | palendae https://review.openstack.org/369485 added individual host groups for services... so it may interfere with duplicating the issue purely | 20:40 |
| *** hughmFLEXin has quit IRC | 20:40 | |
| Matias | so my hopes of getting gnocchi with ceph in stable/newton are now officially void :P | 20:40 |
| * mhayden reassures Matias :) | 20:41 | |
| prometheanfire | it's gonna get more fun in ocata, divergent requirements are coming | 20:41 |
| odyssey4me | Matias we're usually pretty open to backports up to milestone 1 in the next cycle - as that's when most people try things and find bugs, and also when some unfinished things often get finished | 20:41 |
| mhayden | brace yourselves | 20:42 |
| mhayden | divergent requirements are coming | 20:42 |
| * mhayden feels a meme coming on | 20:42 | |
| Matias | nice, thanks for restoring my hope :D | 20:42 |
| odyssey4me | Matias *but* they need to be pretty isolated, and opt-in only changes | 20:42 |
| odyssey4me | otherwise they risk destabilization | 20:42 |
| *** ametts has quit IRC | 20:42 | |
| prometheanfire | mhayden: uc will still work | 20:42 |
| prometheanfire | so we will be fine :D | 20:42 |
| *** hughmFLEXin has joined #openstack-ansible | 20:43 | |
| palendae | odyssey4me, Shouldn't; those conf.d entries weren't used by the tests last I knew. But we'll find out | 20:43 |
| odyssey4me | ie it gets harder to just get something in, and cores are reticent to merge things which aren't properly tested | 20:43 |
| *** askb has joined #openstack-ansible | 20:44 | |
| openstackgerrit | Nolan Brubaker proposed openstack/openstack-ansible: Add test for Horizon group membership https://review.openstack.org/371140 | 20:47 |
| *** itsuugo has quit IRC | 20:47 | |
| openstackgerrit | Nolan Brubaker proposed openstack/openstack-ansible: Add test for Horizon group membership https://review.openstack.org/371140 | 20:48 |
| openstackgerrit | Paulo Matias proposed openstack/openstack-ansible-rabbitmq_server: Adjust file descriptor limit when systemd is used https://review.openstack.org/369729 | 20:50 |
| *** itsuugo has joined #openstack-ansible | 20:50 | |
| Matias | odyssey4me: I need to go now, I can come back in 4 to 5 hours, galera still needs to receive the same change | 20:52 |
| Matias | cya | 20:52 |
| *** hughmFLEXin has quit IRC | 20:52 | |
| *** hughmFLEXin has joined #openstack-ansible | 20:53 | |
| *** gouthamr has quit IRC | 20:54 | |
| odyssey4me | Matias great, thanks :) | 20:54 |
| odyssey4me | cloudnull when you have a moment, reviews of https://review.openstack.org/369729 and https://review.openstack.org/369745 would be appreciated | 20:55 |
| odyssey4me | mhayden ditto on that - I'm not all that familiar with systemd | 20:55 |
| cloudnull | both lgtm | 20:56 |
| cloudnull | thanks Matias | 20:56 |
| cloudnull | sorry for being nitpicky | 20:56 |
| *** hughmFLEXin has quit IRC | 20:57 | |
| odyssey4me | cloudnull he mentioned galera needing the same thing? could you perhaps work up a patch for that if you agree? | 20:58 |
| cloudnull | i agree. | 20:58 |
| *** hughmFLEXin has joined #openstack-ansible | 21:00 | |
| Attilla | odyssey4me: I have built aio successfully | 21:00 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-os_watcher: Use Ansible 2.1.1 for role testing https://review.openstack.org/370828 | 21:00 |
| Attilla | Seems that it was something with the Vagrant image I used | 21:01 |
| odyssey4me | Attilla w00t! | 21:01 |
| odyssey4me | :) | 21:01 |
| Attilla | Don't know what, but there is a difference between boxcutter and original Ubuntu images | 21:01 |
| * odyssey4me suddenly understands why the systemd thingys are needed | 21:02 | |
| odyssey4me | patch reviews convey understanding | 21:02 |
| odyssey4me | clearly I am descending into supernerd status or something | 21:03 |
| *** weezS has quit IRC | 21:04 | |
| *** psilvad has quit IRC | 21:06 | |
| *** gouthamr has joined #openstack-ansible | 21:09 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova: Force tempest to run serial and retry tests https://review.openstack.org/371055 | 21:14 |
| *** catintheroof has joined #openstack-ansible | 21:15 | |
| *** weezS has joined #openstack-ansible | 21:15 | |
| *** jlockwood has left #openstack-ansible | 21:15 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-galera_server: Adjust file descriptor limit when systemd is used https://review.openstack.org/371160 | 21:17 |
| cloudnull | odyssey4me: ^ | 21:17 |
| *** asettle has joined #openstack-ansible | 21:19 | |
| *** gonzalo2kx has quit IRC | 21:19 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-os_nova: Add specific support for automatic compute RPC pinning https://review.openstack.org/367752 | 21:21 |
| openstackgerrit | Merged openstack/openstack-ansible-memcached_server: Adjust file descriptor limit when systemd is used https://review.openstack.org/369745 | 21:23 |
| odyssey4me | Matias see https://review.openstack.org/371160 :) | 21:24 |
| *** asettle has quit IRC | 21:24 | |
| *** ZuZuD has quit IRC | 21:25 | |
| *** woodard_ has joined #openstack-ansible | 21:25 | |
| odyssey4me | mhayden ^ when you're available... please review | 21:26 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-galera_server: Adjust file descriptor limit when systemd is used https://review.openstack.org/371160 | 21:26 |
| cloudnull | i fat fingered a upstart | 21:26 |
| cloudnull | bah. | 21:26 |
| cloudnull | i can't type | 21:26 |
| *** ZuZuD has joined #openstack-ansible | 21:27 | |
| cloudnull | :| | 21:27 |
| *** woodard has quit IRC | 21:28 | |
| *** woodard_ has quit IRC | 21:29 | |
| mrda | morning OSA | 21:29 |
| cloudnull | mornings | 21:30 |
| cloudnull | odyssey4me: OMFG... ubuntu-xenial-internap-mtl01 -- failed -- https://review.openstack.org/#/c/370501 | 21:31 |
| cloudnull | rechecking | 21:31 |
| cloudnull | it passed on internap twice on the previous check | 21:32 |
| cloudnull | then ... gremlins | 21:32 |
| stevelle | cloudnull: shipit | 21:33 |
| odyssey4me | grrr | 21:33 |
| cloudnull | :'( | 21:33 |
| * cloudnull goes for a beer | 21:33 | |
| mgagne | cloudnull: could I get more information about the failure you see with this provider? Why is it failing here and not elsewhere? | 21:34 |
| cloudnull | its not failing just there. | 21:35 |
| cloudnull | its just randomly failing | 21:35 |
| cloudnull | seems there's a race condition somewhere. | 21:35 |
| cloudnull | which looks like a neutron thing | 21:36 |
| cloudnull | but i've not pin pointed where. | 21:36 |
| odyssey4me | mgagne gremlins? | 21:36 |
| mgagne | ok, I got a bit concerned that internap was mentioned | 21:36 |
| cloudnull | no that was just the most recent failure | 21:37 |
| odyssey4me | :) there are so many factors in a gate check's success | 21:37 |
| odyssey4me | it could be any number of things | 21:37 |
| cloudnull | though we do see A LOT of timeouts when jobs run on OVH | 21:37 |
| odyssey4me | mgagne do you work for internap? | 21:37 |
| mgagne | yes I do | 21:37 |
| cloudnull | cool | 21:38 |
| odyssey4me | oh :) well, you'll be pleased to know that internap is one of the more reliable providers as far as I've seen | 21:38 |
| mgagne | =) | 21:38 |
| cloudnull | if we could get folks to make OVH faster life would be SO much better. | 21:38 |
| odyssey4me | we do see many more failures on OVH - it seems the combination of being more remote, mostly (they have EU nodepool regions) from some resources | 21:38 |
| stevelle | Our integrated gate often just *barely* finishes on internap. | 21:38 |
| stevelle | but can say the same for OSIC cloud too | 21:39 |
| odyssey4me | and also it would seem that they have very high contention for their compute hosts, so everything is a *lot* slower | 21:39 |
| mgagne | stevelle: any job I could monitor or compare? I think it would be a good thing to get a dashboard up so we can compare | 21:39 |
| stevelle | mgagne: I believe jmccrory may have some scripts that he used to collect some data about our timing results en masse | 21:40 |
| mgagne | stevelle: there is no more than 4 jobs on the same server, each getting dedicated cores. Would have to get more info/stats on resources usage to see where the bottleneck is | 21:40 |
| cloudnull | mgagne: theirs the OpenStack status page | 21:40 |
| odyssey4me | I would sincerely love to get some sort of dashboard together which tracks jobs across providers, specific to branches | 21:40 |
| odyssey4me | stevelle yeah, it was actually logan- | 21:40 |
| mgagne | there is http://status.openstack.org/openstack-health/#/ but I'm not sure how to use it to get the info I'm looking for | 21:41 |
| cloudnull | http://status.openstack.org/openstack-health/#/g/node_provider/osic-cloud1 | 21:41 |
| stevelle | sometimes my memory... | 21:41 |
| odyssey4me | this is a modified version of the original: https://gist.github.com/odyssey4me/21ba3ae59433ca6d88e69fa725334740 | 21:41 |
| odyssey4me | it works for master | 21:41 |
| odyssey4me | but yeah, we get crazy inconsistencies which is frustrating | 21:42 |
| odyssey4me | sometimes on OSIC we get an integrated build done in 1hr, then others it barely makes it within 1h30m | 21:42 |
| odyssey4me | on the same provider | 21:42 |
| cloudnull | http://status.openstack.org/openstack-health/#/g/node_provider/ovh-bhs1 http://status.openstack.org/openstack-health/#/g/node_provider/ovh-gra1 | 21:42 |
| odyssey4me | cloudnull I never seem to manage to get a working regex search right on those pages | 21:45 |
| *** mrda has quit IRC | 21:46 | |
| *** meteorfox has quit IRC | 21:46 | |
| *** jamielennox has quit IRC | 21:46 | |
| *** mhayden has quit IRC | 21:46 | |
| *** automagically has quit IRC | 21:46 | |
| *** jhesketh has quit IRC | 21:47 | |
| *** mattoliverau has quit IRC | 21:47 | |
| *** gus has quit IRC | 21:47 | |
| *** neillc has quit IRC | 21:47 | |
| *** d34dh0r53 has quit IRC | 21:47 | |
| *** mrhillsman has quit IRC | 21:47 | |
| *** matt6434 has joined #openstack-ansible | 21:47 | |
| *** neillc has joined #openstack-ansible | 21:47 | |
| *** gus has joined #openstack-ansible | 21:48 | |
| *** jhesketh has joined #openstack-ansible | 21:49 | |
| *** matt6434 is now known as mattoliverau | 21:49 | |
| mgagne | cool, you can create private dashboard on-premise with the public graphite of openstack.org =) | 21:50 |
| cloudnull | http://status.openstack.org/openstack-health/#/tests -- i cant find our tests. | 21:51 |
| cloudnull | so maybe they're not indexing them | 21:51 |
| *** meteorfox has joined #openstack-ansible | 21:51 | |
| *** automagically has joined #openstack-ansible | 21:51 | |
| *** d34dh0r53 has joined #openstack-ansible | 21:51 | |
| cloudnull | or maybe i'm not looking in the right place. | 21:52 |
| odyssey4me | cloudnull yep, that was my experience - I wonder if they're only indexing a subset | 21:54 |
| odyssey4me | mgagne do you use OSA, or just lurk here for fun? :) | 21:54 |
| stevelle | we are a pretty fun crowd | 21:54 |
| cloudnull | we are a delightful bunch | 21:54 |
| mgagne | I looked at OSA for Galera ~6 months ago but changed focus to something else =) | 21:55 |
| odyssey4me | cloudnull it would seem that https://review.openstack.org/367752 is a bad idea - we haven't had any success... so the theory doesn't match the practise | 21:55 |
| odyssey4me | that said, the nova role is full of nonsense at this point | 21:56 |
| cloudnull | yea nova folks say that's how you do rolling upgrades. though it looks like setting that breaks nova. | 21:56 |
| cloudnull | so :\ | 21:56 |
| odyssey4me | yep | 21:57 |
| *** jheroux has quit IRC | 21:57 | |
| cloudnull | mgagne: what did you end up using instead? | 21:57 |
| cloudnull | if you don | 21:57 |
| cloudnull | **don't mind me asking | 21:57 |
| *** spedione is now known as spedione|AWAY | 21:57 | |
| mgagne | nothing :D | 21:57 |
| mgagne | statu quo | 21:58 |
| mgagne | mysql + drbd | 21:58 |
| * odyssey4me hands mgagne a beer for his pain | 21:58 | |
| mgagne | haha, I'm not the one managing that thing so I'm happy =) | 21:59 |
| *** mrda has joined #openstack-ansible | 21:59 | |
| cloudnull | curious how that all works these | 21:59 |
| cloudnull | days | 21:59 |
| cloudnull | i used to do that a WHILE back | 22:00 |
| odyssey4me | cloudnull with difficulty, as always :) | 22:00 |
| *** mrhillsman has joined #openstack-ansible | 22:00 | |
| *** mhayden has joined #openstack-ansible | 22:00 | |
| cloudnull | it seemed to work fine | 22:00 |
| odyssey4me | alright, lemme go spend some time with my wife for a bit... bbl | 22:00 |
| cloudnull | ttyl odyssey4me | 22:00 |
| *** jamielennox has joined #openstack-ansible | 22:01 | |
| *** smatzek has quit IRC | 22:03 | |
| *** kylek3h has quit IRC | 22:03 | |
| *** retreved has quit IRC | 22:03 | |
| *** jperry has quit IRC | 22:04 | |
| *** KLevenstein has quit IRC | 22:05 | |
| *** TxGirlGeek has quit IRC | 22:06 | |
| *** TxGirlGeek has joined #openstack-ansible | 22:07 | |
| *** TxGirlGeek has quit IRC | 22:08 | |
| *** hughmFLEXin has quit IRC | 22:09 | |
| *** hughmFLEXin has joined #openstack-ansible | 22:11 | |
| *** itsuugo has quit IRC | 22:15 | |
| *** michaelgugino has quit IRC | 22:15 | |
| *** itsuugo has joined #openstack-ansible | 22:17 | |
| *** asettle has joined #openstack-ansible | 22:21 | |
| *** hughmFLEXin has quit IRC | 22:23 | |
| *** hughmFLEXin has joined #openstack-ansible | 22:24 | |
| *** weezS has quit IRC | 22:26 | |
| *** asettle has quit IRC | 22:26 | |
| *** scarlisle has quit IRC | 22:29 | |
| openstackgerrit | Merged openstack/openstack-ansible-rabbitmq_server: Adjust file descriptor limit when systemd is used https://review.openstack.org/369729 | 22:30 |
| cloudnull | WOOT! | 22:31 |
| *** joseg has quit IRC | 22:34 | |
| *** itsuugo has quit IRC | 22:34 | |
| *** itsuugo has joined #openstack-ansible | 22:37 | |
| *** itsuugo has quit IRC | 22:41 | |
| *** itsuugo has joined #openstack-ansible | 22:42 | |
| cloudnull | OMFG... ubuntu-xenial-infracloud-vanilla-4324909 -- failed -- https://review.openstack.org/#/c/370501 | 22:43 |
| cloudnull | nova is making me fairly sad | 22:43 |
| *** hughmFLE_ has joined #openstack-ansible | 22:43 | |
| cloudnull | its passing everything and then in the final gate job, after succeeding a dozen times, it fails... | 22:45 |
| cloudnull | :| | 22:45 |
| *** hughmFLEXin has quit IRC | 22:45 | |
| cloudnull | going to eat bbl | 22:45 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Aodh should inherit service_region https://review.openstack.org/370405 | 22:55 |
| *** jamesdenton has quit IRC | 22:56 | |
| Matias | cloudnull: do you think there is time to get your PR for yml override merged? | 23:02 |
| Matias | i can test it in a few hours | 23:03 |
| Matias | i'm in the mobile now, getting back to the desk in a few hours | 23:03 |
| *** itsuugo has quit IRC | 23:08 | |
| *** itsuugo has joined #openstack-ansible | 23:09 | |
| stevelle | Matias: which one? this? https://review.openstack.org/#/c/352054/ | 23:10 |
| odyssey4me | Matias a review of https://review.openstack.org/371160 would be useful | 23:14 |
| odyssey4me | stevelle I'm too tired to do it justice... :/ | 23:15 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: [DOCS] Added missing level info for haproxy_hosts https://review.openstack.org/370914 | 23:15 |
| *** galstrom_zzz is now known as galstrom | 23:17 | |
| *** pmannidi has joined #openstack-ansible | 23:17 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible: Add lxc_host dynamic group to inventory. https://review.openstack.org/369599 | 23:18 |
| *** catintheroof has quit IRC | 23:21 | |
| *** asettle has joined #openstack-ansible | 23:22 | |
| *** itsuugo has quit IRC | 23:23 | |
| *** itsuugo has joined #openstack-ansible | 23:24 | |
| odyssey4me | meh, I need to get to sleep | 23:25 |
| odyssey4me | I've arranged a grace period, so I'll pick up the release in the morning. | 23:26 |
| odyssey4me | please babysit the patches as far as you can | 23:26 |
| *** asettle has quit IRC | 23:27 | |
| *** itsuugo has quit IRC | 23:35 | |
| *** weezS has joined #openstack-ansible | 23:35 | |
| *** johnmilton has quit IRC | 23:37 | |
| *** itsuugo has joined #openstack-ansible | 23:37 | |
| *** schwicht has joined #openstack-ansible | 23:39 | |
| *** markvoelker has quit IRC | 23:39 | |
| *** Jeffrey4l__ is now known as Jeffrey4l | 23:40 | |
| *** AnarchyAo has quit IRC | 23:41 | |
| *** BjoernT has quit IRC | 23:44 | |
| *** galstrom is now known as galstrom_zzz | 23:45 | |
| *** hblah has joined #openstack-ansible | 23:46 | |
| *** itsuugo has quit IRC | 23:49 | |
| *** itsuugo has joined #openstack-ansible | 23:51 | |
| openstackgerrit | Nolan Brubaker proposed openstack/openstack-ansible: Add test for Horizon group membership https://review.openstack.org/371140 | 23:54 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!