| *** nadya has quit IRC | 00:01 | |
| *** sdague has quit IRC | 00:01 | |
| *** amitgandhinz has quit IRC | 00:01 | |
| *** jgriffith is now known as jgriffith_away | 00:04 | |
| openstackgerrit | Lin Yang proposed openstack-infra/project-config: Add gerritbot event for openstack/rsc project https://review.openstack.org/384739 | 00:04 |
|---|---|---|
| *** tiswanso has joined #openstack-infra | 00:04 | |
| *** tiswanso has quit IRC | 00:09 | |
| *** rderose has quit IRC | 00:12 | |
| *** sdake_ has quit IRC | 00:13 | |
| *** AlexxNica has joined #openstack-infra | 00:17 | |
| openstackgerrit | Merged openstack-infra/system-config: Update solr package repository url https://review.openstack.org/352267 | 00:21 |
| *** gouthamr has joined #openstack-infra | 00:25 | |
| *** jamielennox is now known as jamielennox|away | 00:30 | |
| *** Apoorva has quit IRC | 00:32 | |
| *** mriedem has quit IRC | 00:33 | |
| *** mtanino has quit IRC | 00:34 | |
| *** markvoelker has joined #openstack-infra | 00:36 | |
| *** EricGonczer_ has quit IRC | 00:40 | |
| *** markvoelker has quit IRC | 00:41 | |
| *** tuanluong has joined #openstack-infra | 00:45 | |
| *** arahal_ has joined #openstack-infra | 00:46 | |
| *** sdake has joined #openstack-infra | 00:47 | |
| *** jamielennox|away is now known as jamielennox | 00:48 | |
| *** apetrich has quit IRC | 00:48 | |
| *** apetrich has joined #openstack-infra | 00:49 | |
| *** arahal_ has quit IRC | 00:50 | |
| *** EricGonczer_ has joined #openstack-infra | 00:53 | |
| *** jerryz has quit IRC | 00:57 | |
| *** EricGonczer_ has quit IRC | 00:57 | |
| *** dimtruck is now known as zz_dimtruck | 00:57 | |
| *** amitgandhinz has joined #openstack-infra | 00:58 | |
| *** AlexxNica has quit IRC | 00:59 | |
| openstackgerrit | Ramy Asselin proposed openstack-infra/elastic-recheck: Include query results in graph https://review.openstack.org/260188 | 00:59 |
| *** vsaienko has joined #openstack-infra | 01:01 | |
| asselin_ | nibalizer, when you get a change I have a few puppet-bandersnatch changes proposed. Mostly upstreaming some from our fork. | 01:02 |
| *** amitgandhinz has quit IRC | 01:02 | |
| asselin_ | nibalizer, also is nikki around still? | 01:02 |
| *** aeng has quit IRC | 01:04 | |
| nibalizer | asselin_: ok I will look | 01:04 |
| nibalizer | I think nikki is at bluebox working on their CI stuff | 01:04 |
| nibalizer | (which isn't an infra) | 01:04 |
| *** alex_xu has quit IRC | 01:05 | |
| *** kaisers_ has joined #openstack-infra | 01:07 | |
| *** alex_xu has joined #openstack-infra | 01:08 | |
| asselin_ | ok, well I thought I'd ping her since she started the module and I'm upstreaming some of her changes. I don't see her nick though, so guess she's not on irc anymore. | 01:09 |
| *** kzaitsev_mb has quit IRC | 01:09 | |
| *** watersoul has quit IRC | 01:09 | |
| *** kzaitsev_mb has joined #openstack-infra | 01:09 | |
| *** watersoul_ has joined #openstack-infra | 01:10 | |
| *** kaisers_ has quit IRC | 01:11 | |
| *** AlexxNica has joined #openstack-infra | 01:11 | |
| nibalizer | :shrug: could always try twitter | 01:11 |
| *** markvoelker has joined #openstack-infra | 01:12 | |
| *** zhurong has joined #openstack-infra | 01:14 | |
| *** jamesdenton has quit IRC | 01:14 | |
| *** yanyanhu has joined #openstack-infra | 01:15 | |
| *** spzala has joined #openstack-infra | 01:17 | |
| *** aeng has joined #openstack-infra | 01:18 | |
| *** zz_dimtruck is now known as dimtruck | 01:19 | |
| *** vsaienko has quit IRC | 01:20 | |
| *** Jeffrey4l has joined #openstack-infra | 01:21 | |
| *** adrian_otto has quit IRC | 01:22 | |
| *** Jeffrey4l has quit IRC | 01:22 | |
| *** salv-orlando has joined #openstack-infra | 01:23 | |
| *** Jeffrey4l has joined #openstack-infra | 01:23 | |
| *** Jeffrey4l has quit IRC | 01:24 | |
| *** Jeffrey4l has joined #openstack-infra | 01:25 | |
| *** EricGonczer_ has joined #openstack-infra | 01:25 | |
| *** Jeffrey4l has quit IRC | 01:26 | |
| *** Jeffrey4l has joined #openstack-infra | 01:26 | |
| *** salv-orlando has quit IRC | 01:27 | |
| *** dimtruck is now known as zz_dimtruck | 01:28 | |
| *** sdake has quit IRC | 01:31 | |
| *** jamesdenton has joined #openstack-infra | 01:32 | |
| *** Julien-zte has joined #openstack-infra | 01:33 | |
| *** cody-somerville has joined #openstack-infra | 01:40 | |
| *** cody-somerville has joined #openstack-infra | 01:40 | |
| *** caowei has joined #openstack-infra | 01:43 | |
| *** jianghuaw has quit IRC | 01:43 | |
| *** jianghuaw has joined #openstack-infra | 01:44 | |
| *** Jeffrey4l_ has joined #openstack-infra | 01:54 | |
| *** sputnik13 has joined #openstack-infra | 01:55 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources: Fix on moderators https://review.openstack.org/384765 | 01:55 |
| *** AlexxNica has quit IRC | 01:56 | |
| openstackgerrit | Merged openstack-infra/openstackid-resources: Fix on moderators https://review.openstack.org/384765 | 01:57 |
| *** Jeffrey4l has quit IRC | 01:57 | |
| *** kzaitsev_mb has quit IRC | 01:57 | |
| *** EricGonczer_ has quit IRC | 01:58 | |
| *** amitgandhinz has joined #openstack-infra | 01:59 | |
| *** tlian has quit IRC | 01:59 | |
| *** dave-mccowan has quit IRC | 01:59 | |
| *** tqtran has quit IRC | 02:01 | |
| *** amitgandhinz has quit IRC | 02:03 | |
| *** yarkot has joined #openstack-infra | 02:03 | |
| *** sdake has joined #openstack-infra | 02:07 | |
| *** sdake has quit IRC | 02:07 | |
| *** sdake has joined #openstack-infra | 02:07 | |
| *** jcook has quit IRC | 02:08 | |
| *** nadya has joined #openstack-infra | 02:09 | |
| *** sdake has quit IRC | 02:09 | |
| jlvillal | Still seeing errors in the OSIC cloud 1 hour ago :( | 02:10 |
| *** markvoelker has quit IRC | 02:10 | |
| cloudnull | jlvillal: say again ? | 02:10 |
| *** markvoelker has joined #openstack-infra | 02:10 | |
| jlvillal | cloudnull, Near the end of: http://logs.openstack.org/51/384651/3/check/gate-grenade-dsvm-neutron-ubuntu-xenial/684531d/logs/grenade.sh.txt.gz#_2016-10-10_23_59_56_115 | 02:11 |
| jlvillal | cloudnull, Not able to install packages from PyPi | 02:12 |
| *** sdake has joined #openstack-infra | 02:12 | |
| cloudnull | ah. looks like 403 Client Error: Forbidden | 02:12 |
| jlvillal | cloudnull, Seems like it shouldn't be doing that. I'm thinking. | 02:13 |
| cloudnull | one would think | 02:13 |
| jlvillal | cloudnull, I saw that this morning (PDT) too. | 02:13 |
| cloudnull | maybe perms issues on the osic mirros? | 02:13 |
| *** jcook has joined #openstack-infra | 02:13 | |
| cloudnull | *mirrors | 02:13 |
| cloudnull | jhesketh: ^ | 02:13 |
| *** esikachev has joined #openstack-infra | 02:13 | |
| *** nadya has quit IRC | 02:14 | |
| jlvillal | cloudnull, I know jhesketh had proposed this earlier today: https://review.openstack.org/384737 | 02:14 |
| jhesketh | hmm, I haven't seen 403's before... it could be that it dropped out while half way through | 02:15 |
| *** woodster_ has quit IRC | 02:15 | |
| jlvillal | jhesketh, I saw 403 this morning and now this evening. | 02:15 |
| *** Liuqing has joined #openstack-infra | 02:16 | |
| jlvillal | Maybe early afternoon and not morning. | 02:16 |
| *** esikachev has quit IRC | 02:18 | |
| cloudnull | yah. idk if that solves that issue though. | 02:18 |
| cloudnull | it looks, from the logs, that it downloaded most of the packages you needed. | 02:19 |
| jhesketh | so it looks like servers are still dropping in and out of the afs cell.. | 02:19 |
| cloudnull | jhesketh: have you seen interuptions over the last 15 min or so | 02:19 |
| jhesketh | it's quite transient though.. I suspect some disconnects even go unnoticed due to retries | 02:19 |
| cloudnull | ? | 02:19 |
| jlvillal | cloudnull, jhesketh: Not sure if related. But my SSH connection to my OSIC VM kept dropping quite often today. | 02:20 |
| cloudnull | we just pushed an FDB fix out to the cluster which was causing some of our general networking issues due to our use of l2pop | 02:20 |
| jhesketh | the last one went down at around 01:44:57 and came back 01:45:01 | 02:20 |
| cloudnull | jlvillal: was it in the middle of the day | 02:20 |
| cloudnull | ? | 02:20 |
| jhesketh | cloudnull: oh cool, maybe it's okay then | 02:20 |
| jlvillal | cloudnull, This afternoon (PDT) at work. | 02:20 |
| cloudnull | because we did a major release upgrade this afternoon | 02:21 |
| cloudnull | so you likely ran into those issues. | 02:21 |
| jlvillal | cloudnull, Okay. Thanks. | 02:21 |
| cloudnull | jhesketh: by all means if it's still being unstable lets remove it from the cluster. | 02:21 |
| cloudnull | but i'd be curious to know if its still unstable after the FDB push | 02:21 |
| jhesketh | cloudnull: nothing in the last 30min, so lets wait to see if it is fixed and if not then we take it out of the pool | 02:22 |
| cloudnull | sounds good | 02:22 |
| *** salv-orlando has joined #openstack-infra | 02:24 | |
| sdake | possible problem with mirroring here: | 02:25 |
| sdake | http://logs.openstack.org/99/384599/2/check/gate-kolla-docs-ubuntu-xenial/8f60d32/console.html#_2016-10-11_01_11_11_222926 | 02:25 |
| sdake | anyone aware of this one? | 02:25 |
| jlvillal | sdake, I think cloudnull jhesketh are looking at similar thing ^^^^ | 02:26 |
| *** hongbin has joined #openstack-infra | 02:27 | |
| sdake | jlvillal thx | 02:27 |
| jhesketh | sdake: yeah, osic mirror has been having trouble... we're hoping it's stable now but will take it out of the pool if it continues further | 02:27 |
| *** tphummel has quit IRC | 02:27 | |
| sdake | jhesketh roger thanks :) | 02:28 |
| *** salv-orlando has quit IRC | 02:28 | |
| *** pfallenop has joined #openstack-infra | 02:30 | |
| *** Liuqing has quit IRC | 02:31 | |
| *** yamamoto_ has joined #openstack-infra | 02:32 | |
| *** spzala has quit IRC | 02:33 | |
| *** Liuqing has joined #openstack-infra | 02:33 | |
| *** spzala has joined #openstack-infra | 02:33 | |
| *** armax has quit IRC | 02:36 | |
| *** jamesdenton has quit IRC | 02:36 | |
| jhesketh | cloudnull: looks like the problem might have just gotten worse :-( | 02:38 |
| cloudnull | what are we seeing now ? | 02:38 |
| jhesketh | lots of disconnects as before | 02:38 |
| jhesketh | and lots of red in the gate | 02:39 |
| jhesketh | infra-root: anybody about to +w this https://review.openstack.org/#/c/384737/ | 02:39 |
| *** spzala has quit IRC | 02:40 | |
| cloudnull | do we still see flapping in the afs cells? | 02:41 |
| jhesketh | cloudnull: yes | 02:43 |
| *** thorst has quit IRC | 02:43 | |
| jhesketh | lots of it in the last 10min | 02:43 |
| jhesketh | cloudnull: any objections before I disable it? | 02:43 |
| cloudnull | none at all. | 02:43 |
| *** thorst has joined #openstack-infra | 02:44 | |
| * jhesketh self approves | 02:44 | |
| jhesketh | it just failed check because of hte error, so hopefully it goes through or otherwise I'll ninja it | 02:44 |
| *** Rockyg has quit IRC | 02:45 | |
| cloudnull | +1 on approving, not that my +1 matterss | 02:45 |
| cloudnull | :) | 02:45 |
| jhesketh | #status notice Jobs running on osic nodes are failing due to network issues with the mirror. We are temporarily disabling the cloud. | 02:46 |
| openstackstatus | jhesketh: sending notice | 02:46 |
| *** aviau has quit IRC | 02:46 | |
| *** aviau has joined #openstack-infra | 02:46 | |
| *** hongbin has quit IRC | 02:47 | |
| -openstackstatus- NOTICE: Jobs running on osic nodes are failing due to network issues with the mirror. We are temporarily disabling the cloud. | 02:47 | |
| *** hongbin has joined #openstack-infra | 02:48 | |
| *** spzala has joined #openstack-infra | 02:48 | |
| *** rlandy|afk is now known as rlandy | 02:49 | |
| *** rlandy has quit IRC | 02:49 | |
| openstackstatus | jhesketh: finished sending notice | 02:49 |
| jhesketh | cloudnull: the next question is how do we debug and confirm the problem is fixed? | 02:50 |
| cloudnull | so it looks like the v4 address is down. | 02:50 |
| cloudnull | v6 is up | 02:50 |
| cloudnull | i have a check going from multiple locations in and outside of the cloud to the mirror | 02:51 |
| cloudnull | so idk if this is something going on from within the instance | 02:51 |
| cloudnull | or is something going on from within the cloud | 02:51 |
| jhesketh | hmm okay | 02:51 |
| *** thorst has quit IRC | 02:52 | |
| cloudnull | jhesketh: do you have ssh access to the VM ? | 02:53 |
| jhesketh | cloudnull: yes | 02:53 |
| cloudnull | can you log in and do a ``ip a l`` ? | 02:53 |
| cloudnull | I just want to see what the instance is reporting its interfaces are. | 02:54 |
| jhesketh | cloudnull: http://paste.openstack.org/show/585263/ | 02:55 |
| *** yamahata has quit IRC | 02:55 | |
| *** kaisers_ has joined #openstack-infra | 02:56 | |
| *** sdake_ has joined #openstack-infra | 02:56 | |
| cloudnull | are you connected to the vm over "172.99.106.183" ? | 02:57 |
| openstackgerrit | Merged openstack-infra/project-config: Disable osic-cloud1 https://review.openstack.org/384737 | 02:57 |
| cloudnull | i get nothing to that IP from IAD, DFW, and PDX. | 02:58 |
| *** pradk has quit IRC | 02:59 | |
| *** sdake has quit IRC | 02:59 | |
| *** amitgandhinz has joined #openstack-infra | 02:59 | |
| *** kaisers_ has quit IRC | 03:00 | |
| *** amitgandhinz has quit IRC | 03:04 | |
| *** adrian_otto has joined #openstack-infra | 03:05 | |
| *** apetrich has quit IRC | 03:06 | |
| *** jascott1_ has quit IRC | 03:09 | |
| cloudnull | jhesketh: in the console log for this instance I see a lot of | 03:10 |
| cloudnull | http://cdn.pasteraw.com/jaw2b6glv0ickyy6qevd9jz4svvy0d6 | 03:10 |
| *** sputnik13 has quit IRC | 03:10 | |
| *** apetrich has joined #openstack-infra | 03:10 | |
| cloudnull | idk if there's something else going on with the node or if that's normal . | 03:10 |
| cloudnull | 's/./?/' | 03:10 |
| jhesketh | cloudnull: I think apache is having trouble because the symlink to the afs volume breaks mid-stream | 03:11 |
| jhesketh | so it's adding to the trouble, but I don't think it is the cause | 03:11 |
| cloudnull | ok. | 03:12 |
| cloudnull | do you mind if i migrate that vm to another compute node? | 03:12 |
| jhesketh | cloudnull: should be fine, but lets wait for the running osic jobs to clear out first | 03:13 |
| cloudnull | or if i power cycle it ? | 03:13 |
| cloudnull | kk | 03:13 |
| jhesketh | as they still may work if they are lucky | 03:13 |
| jhesketh | (or not unlucky) | 03:13 |
| cloudnull | fair point. | 03:13 |
| *** gouthamr has quit IRC | 03:13 | |
| cloudnull | ok . im relocating . bbl let me know when i can poke at it with a stick | 03:13 |
| mordred | 739137 | 03:14 |
| jhesketh | cloudnull: cool, thanks for your help | 03:15 |
| *** aeng has quit IRC | 03:16 | |
| *** zz_dimtruck is now known as dimtruck | 03:18 | |
| *** amotoki has quit IRC | 03:19 | |
| *** kzaitsev_mb has joined #openstack-infra | 03:24 | |
| *** salv-orlando has joined #openstack-infra | 03:24 | |
| *** sputnik13 has joined #openstack-infra | 03:27 | |
| *** salv-orlando has quit IRC | 03:29 | |
| *** kzaitsev_mb has quit IRC | 03:29 | |
| *** amotoki has joined #openstack-infra | 03:30 | |
| *** aeng has joined #openstack-infra | 03:32 | |
| prometheanfire | cloudnull: I'd be intrested in know what went wrong | 03:33 |
| cloudnull | that makes two of us :) | 03:33 |
| *** davidlenwell has quit IRC | 03:34 | |
| prometheanfire | since I was suposidly responsible for the neutron upgrade bits rpc side (of which there was maybe one thing) | 03:34 |
| prometheanfire | cloudnull: so, turn it off and back on again? | 03:34 |
| cloudnull | have to count to 10 first | 03:34 |
| prometheanfire | oh, right | 03:34 |
| *** vikrant has joined #openstack-infra | 03:35 | |
| *** amotoki has quit IRC | 03:35 | |
| jhesketh | prometheanfire, cloudnull: the nodepool change has applied so we're just waiting on the jobs to finish and nodepool should clear out the rest of the nodes... I'll let yo uknow when it's done | 03:36 |
| cloudnull | ok | 03:36 |
| *** jamesdenton has joined #openstack-infra | 03:39 | |
| *** psachin has joined #openstack-infra | 03:40 | |
| *** amotoki has joined #openstack-infra | 03:40 | |
| jhesketh | cloudnull: although of course the mirror hasn't had a blip in 30min :-s | 03:42 |
| *** adriant has quit IRC | 03:43 | |
| *** davidlenwell has joined #openstack-infra | 03:43 | |
| *** sputnik13 has quit IRC | 03:43 | |
| cloudnull | haha typical computers. | 03:44 |
| *** kaisers_ has joined #openstack-infra | 03:47 | |
| dasm | hmm.. disabling OSIC produced some unexpected errors (or maybe expected?) | 03:48 |
| dasm | http://paste.openstack.org/show/585264/ | 03:48 |
| dasm | affected review: https://review.openstack.org/#/c/384786/ with plenty of tests failing | 03:48 |
| cloudnull | dasm: likely related to the mirror failures. | 03:48 |
| cloudnull | its being looked into | 03:48 |
| dasm | cloudnull: ack | 03:48 |
| *** winggundamth has quit IRC | 03:48 | |
| jhesketh | dasm: disabling the mirror because of those failures... ie to prevent it | 03:50 |
| *** thorst has joined #openstack-infra | 03:50 | |
| jhesketh | err, disabling the cloud | 03:50 |
| jhesketh | the mirror is still there, but erratic | 03:50 |
| dasm | jhesketh: any tips to solve "being erratic"? recheck? or other solution, that needs to be provided by infra? | 03:51 |
| jhesketh | dasm: recheck should be okay as new jobs won't be in osic | 03:52 |
| dasm | jhesketh: ack | 03:52 |
| *** rfolco has quit IRC | 03:53 | |
| *** vern has quit IRC | 03:57 | |
| *** yuanying_ has joined #openstack-infra | 03:57 | |
| *** thorst has quit IRC | 03:58 | |
| *** amitgandhinz has joined #openstack-infra | 04:00 | |
| *** yuanying has quit IRC | 04:01 | |
| *** sputnik13 has joined #openstack-infra | 04:01 | |
| *** links has joined #openstack-infra | 04:01 | |
| *** amotoki has quit IRC | 04:01 | |
| *** yuanying_ has quit IRC | 04:02 | |
| *** hongbin has quit IRC | 04:02 | |
| *** amitgandhinz has quit IRC | 04:04 | |
| *** tiswanso has joined #openstack-infra | 04:06 | |
| *** amotoki has joined #openstack-infra | 04:07 | |
| *** tiswanso has quit IRC | 04:11 | |
| *** spzala has quit IRC | 04:13 | |
| *** spzala has joined #openstack-infra | 04:13 | |
| *** nadya has joined #openstack-infra | 04:16 | |
| *** vern has joined #openstack-infra | 04:17 | |
| *** spzala has quit IRC | 04:18 | |
| *** adrian_otto has quit IRC | 04:19 | |
| *** adrian_otto has joined #openstack-infra | 04:20 | |
| cloudnull | jhesketh: are we clear to stop / start that instance? | 04:22 |
| jhesketh | cloudnull: there are 41 nodes in osic still.. | 04:23 |
| *** robcresswell has quit IRC | 04:23 | |
| jhesketh | cloudnull: oh, that includes cloud8 | 04:24 |
| *** nadya has quit IRC | 04:24 | |
| jhesketh | there are only 2 in cloud1 | 04:24 |
| *** vern has quit IRC | 04:24 | |
| jhesketh | cloudnull: they are probably past using hte mirror... I'd say it'd be pretty safe, especially since it's super late for you | 04:24 |
| jhesketh | cloudnull: just let me check the last afs disconnect we had | 04:24 |
| cloudnull | ok | 04:25 |
| *** robcresswell has joined #openstack-infra | 04:25 | |
| *** salv-orlando has joined #openstack-infra | 04:25 | |
| *** kzaitsev_mb has joined #openstack-infra | 04:26 | |
| jhesketh | for the record it hasn't happened since 03:12:04 | 04:26 |
| jhesketh | it could be because of the load though | 04:26 |
| jhesketh | cloudnull: you should be good if you want to migrate the instance | 04:26 |
| cloudnull | ok | 04:27 |
| cloudnull | we're rebooting it first. | 04:27 |
| jhesketh | ack, but since we havne't seen disconnects in a while I'm not sure how to measure the effectivenss of a restart | 04:29 |
| jhesketh | (or move for that matter) | 04:29 |
| *** thingee has quit IRC | 04:29 | |
| jhesketh | oh interesting, it just had a couple of afs blips | 04:29 |
| *** salv-orlando has quit IRC | 04:30 | |
| *** thingee has joined #openstack-infra | 04:30 | |
| *** yamamoto_ has quit IRC | 04:31 | |
| cloudnull | it was just rebooted | 04:32 |
| cloudnull | L)( | 04:32 |
| cloudnull | :) | 04:32 |
| *** davidlenwell has quit IRC | 04:35 | |
| cloudnull | jhesketh: can you log in to that vm again | 04:39 |
| cloudnull | and get the routing table for us ? | 04:39 |
| cloudnull | we have a suspicion that that there's a race condition causing the gateway issues | 04:40 |
| jhesketh | cloudnull: what node did you reboot? | 04:42 |
| cloudnull | the mirror | 04:43 |
| cloudnull | 172.99.106.18 | 04:43 |
| cloudnull | ** 172.99.106.183 | 04:43 |
| *** aeng has quit IRC | 04:44 | |
| jhesketh | k | 04:45 |
| jhesketh | I didn't get disconnceted so I was curious but the uptime is right | 04:45 |
| cloudnull | hum . that's interesting | 04:46 |
| cloudnull | essentially we believe that the instnace is choosing the wrong interface for it's GW. | 04:47 |
| jhesketh | actually I may have ssh'd after you rebooted it but before you pinged me | 04:47 |
| cloudnull | which may or may not be an OpenStack race condition which we're seeing in mitaka that we were not seeing in liberty | 04:47 |
| jhesketh | cloudnull: http://paste.openstack.org/show/585269/ | 04:48 |
| jhesketh | cloudnull: http://paste.openstack.org/show/585270/ | 04:50 |
| *** adrian_otto has quit IRC | 04:50 | |
| *** amotoki has quit IRC | 04:51 | |
| jhesketh | cloudnull: looks like we had 1 afs disconnect since rebooting fyi | 04:51 |
| cloudnull | thanks jhesketh mind -- cat /etc/network/interfaces* /etc/network/interfaces.d/* | 04:52 |
| * cloudnull assuming its ubuntu | 04:53 | |
| jhesketh | cloudnull: http://paste.openstack.org/show/585271/ | 04:54 |
| *** Julien-zte has quit IRC | 04:54 | |
| jhesketh | ack | 04:54 |
| *** thorst has joined #openstack-infra | 04:55 | |
| *** salv-orlando has joined #openstack-infra | 04:56 | |
| *** aeng has joined #openstack-infra | 04:57 | |
| *** amotoki has joined #openstack-infra | 04:57 | |
| *** tqtran has joined #openstack-infra | 04:57 | |
| *** winggundamth has joined #openstack-infra | 04:58 | |
| *** e0ne has joined #openstack-infra | 05:00 | |
| *** amitgandhinz has joined #openstack-infra | 05:01 | |
| *** tqtran has quit IRC | 05:03 | |
| jhesketh | cloudnull: was cloud8 also upgraded? we might be seeing similar faults there | 05:03 |
| openstackgerrit | Karthik Prabhu Vinod proposed openstack-infra/devstack-gate: Adding other combinations to Cinder multinode grenade https://review.openstack.org/384812 | 05:03 |
| *** thorst has quit IRC | 05:03 | |
| *** kzaitsev_mb has quit IRC | 05:03 | |
| cloudnull | no they've been running mitaka for a while and are not providing v4 network. | 05:03 |
| cloudnull | they rely on the cloud1 mirror over v6 | 05:04 |
| cloudnull | as its their only public access. | 05:04 |
| *** e0ne has quit IRC | 05:04 | |
| *** salv-orl_ has joined #openstack-infra | 05:05 | |
| cloudnull | so we believe the issue is the fact that the instance is dual homed. jamesdenton, BjoernT, and i have been looking into it for a bit | 05:05 |
| jhesketh | cloudnull: http://logs.openstack.org/56/380056/3/check/gate-rally-dsvm-rally/b2b75ee/console.html#_2016-10-11_04_27_09_326424 | 05:05 |
| *** amitgandhinz has quit IRC | 05:06 | |
| *** salv-orl_ has quit IRC | 05:06 | |
| *** salv-orl_ has joined #openstack-infra | 05:06 | |
| *** salv-orlando has quit IRC | 05:07 | |
| cloudnull | jhesketh: i believe that mirror.regionone.osic-cloud8.openstack.org is just pointing to mirror.regionone.osic-cloud1.openstack.org because they have no v4 space. | 05:09 |
| cloudnull | so to fix the issue the interface configs will need to be static | 05:09 |
| cloudnull | we've kinda confirmed the behavior from old instances that i had around which have now been fiddled with which were dual homed and all of them have needed to be updated so the config was static. | 05:12 |
| cloudnull | which leads me to believe that maybe there was a performance problem, bug, or something in liberty which masked the issue from before. | 05:13 |
| *** markvoelker has quit IRC | 05:13 | |
| *** markvoelker has joined #openstack-infra | 05:15 | |
| *** markvoelker has quit IRC | 05:16 | |
| *** yamamoto_ has joined #openstack-infra | 05:17 | |
| *** markvoelker has joined #openstack-infra | 05:18 | |
| *** zhurong has quit IRC | 05:18 | |
| *** markvoelker has quit IRC | 05:18 | |
| *** markvoelker has joined #openstack-infra | 05:19 | |
| *** zhurong has joined #openstack-infra | 05:20 | |
| *** markvoelker has quit IRC | 05:20 | |
| *** HeOS has joined #openstack-infra | 05:22 | |
| *** markvoelker_ has joined #openstack-infra | 05:25 | |
| *** markvoelker_ has quit IRC | 05:25 | |
| jhesketh | cloudnull: (sorry was on the phone). Right you are, yes... should have also disabled cloud8 in that case | 05:25 |
| cloudnull | jhesketh: esentially to make this happy the interface file needs to look something more like so http://cdn.pasteraw.com/j8ct2yeuluisz6pf7lj8g5jtjwhhgh3 | 05:26 |
| *** jascott1 has joined #openstack-infra | 05:27 | |
| *** markvoelker has joined #openstack-infra | 05:27 | |
| *** markvoelker has quit IRC | 05:28 | |
| *** markvoelker has joined #openstack-infra | 05:28 | |
| *** arxcruz has quit IRC | 05:29 | |
| *** rods has quit IRC | 05:30 | |
| *** thingee has quit IRC | 05:31 | |
| * cloudnull assuming that v6 is running on eth0 | 05:31 | |
| *** korzen has joined #openstack-infra | 05:33 | |
| *** arxcruz has joined #openstack-infra | 05:34 | |
| jhesketh | cloudnull: so is this a temporary solution, or will it need to be permanent | 05:35 |
| cloudnull | it'll need to be perminent | 05:35 |
| cloudnull | since the upgrade dual homed instances seem to be prefering the 10.x network as their gateway when it should be the 172.99.x | 05:36 |
| cloudnull | this is what I had on one of my test nodes before and after http://paste.openstack.org/show/585272/ | 05:36 |
| cloudnull | note, good routing table has "default via 172.99.106.1 dev ens4" while the bad one has "default via 10.0.0.1 dev ens3 " | 05:37 |
| *** HeOS has quit IRC | 05:37 | |
| cloudnull | and that is causing the flapping | 05:37 |
| cloudnull | so that VMs case I fixed it like so: http://cdn.pasteraw.com/5eaj3raaxh22yor5hm0de8tcw73a5mt | 05:38 |
| *** adam_g` has quit IRC | 05:38 | |
| *** asselin_ has quit IRC | 05:38 | |
| cloudnull | but so long as the node it dual homed a solution like this will need to be perminent | 05:38 |
| *** hurgleburgler has quit IRC | 05:39 | |
| cloudnull | which seems like a mitaka-ism going forward which we did not expect. | 05:39 |
| cloudnull | but better to find it out now in this env IMHO :) | 05:40 |
| *** d0ugal has quit IRC | 05:40 | |
| *** d0ugal has joined #openstack-infra | 05:41 | |
| cloudnull | oh jhesketh based on this http://paste.openstack.org/show/585263/ your interface change would be this -- http://paste.openstack.org/show/585273/ | 05:42 |
| cloudnull | which i think should work regardless of ordering. | 05:43 |
| * cloudnull goes to try | 05:43 | |
| *** markvoelker has quit IRC | 05:44 | |
| *** markvoelker has joined #openstack-infra | 05:45 | |
| cloudnull | yea that seems to do the needful. | 05:45 |
| cloudnull | but if you want to be 100% sure writing a static config is the way to go | 05:46 |
| *** e0ne has joined #openstack-infra | 05:47 | |
| *** esikachev has joined #openstack-infra | 05:47 | |
| *** rods has joined #openstack-infra | 05:48 | |
| openstackgerrit | Adit Sarfaty proposed openstack-infra/project-config: Add vmware-nsx tests as non voting to the vmware-nsxlib project https://review.openstack.org/384821 | 05:49 |
| cloudnull | sadly it doesn't look like there's a way to make cloud-init do static interfaces correctly even on instances that have already been built -- https://bugs.launchpad.net/cloud-init/+bug/1225922 | 05:50 |
| openstack | Launchpad bug 1225922 in cloud-init "Support static network configuration even on already configured devices" [High,Confirmed] | 05:50 |
| *** yuanying has joined #openstack-infra | 05:50 | |
| *** gmann__ has joined #openstack-infra | 05:51 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/system-config: Fix issue in osic-cloud1 networking https://review.openstack.org/384822 | 05:51 |
| jhesketh | cloudnull: here is my first pass at a fix ^ | 05:51 |
| *** esikachev has quit IRC | 05:52 | |
| jhesketh | I'd want other infra-root's input though as I'm not sure if that's a good way to fix it | 05:52 |
| cloudnull | ++ | 05:52 |
| cloudnull | there has to be a better way imo | 05:52 |
| cloudnull | but im tired. and idk | 05:52 |
| *** markmcclain has quit IRC | 05:52 | |
| openstackgerrit | linzi proposed openstack-infra/project-config: modify zuul/layout.yaml and jenkins jobs https://review.openstack.org/384823 | 05:53 |
| *** aeng has quit IRC | 05:55 | |
| *** e0ne has quit IRC | 05:56 | |
| *** salv-orl_ has quit IRC | 06:00 | |
| *** kzaitsev_mb has joined #openstack-infra | 06:00 | |
| *** qwertyco has joined #openstack-infra | 06:00 | |
| *** sflanigan has quit IRC | 06:01 | |
| *** thorst has joined #openstack-infra | 06:01 | |
| *** pahuang has quit IRC | 06:01 | |
| *** qwertyco has quit IRC | 06:01 | |
| *** qwertyco has joined #openstack-infra | 06:02 | |
| *** amitgandhinz has joined #openstack-infra | 06:02 | |
| cloudnull | jhesketh: i added an alt method as a comment to the PR. | 06:03 |
| cloudnull | on that note, im going to pass out. | 06:03 |
| cloudnull | have a good one | 06:03 |
| *** esikachev has joined #openstack-infra | 06:03 | |
| jhesketh | cloudnull: awesome thanks... I'll leave that for other reviewers to look at | 06:03 |
| jhesketh | and yes, you need sleep | 06:04 |
| jhesketh | thanks heaps for your help! :-) | 06:04 |
| *** markmcclain has joined #openstack-infra | 06:04 | |
| cloudnull | sorry for the issues. | 06:04 |
| cloudnull | take care. | 06:04 |
| *** nadya has joined #openstack-infra | 06:06 | |
| *** amitgandhinz has quit IRC | 06:07 | |
| *** rcernin has joined #openstack-infra | 06:07 | |
| *** aeng has joined #openstack-infra | 06:07 | |
| *** thorst has quit IRC | 06:07 | |
| *** nadya has quit IRC | 06:08 | |
| *** esikachev has quit IRC | 06:08 | |
| *** nadya has joined #openstack-infra | 06:11 | |
| *** pahuang has joined #openstack-infra | 06:14 | |
| *** spzala has joined #openstack-infra | 06:14 | |
| *** sflanigan has joined #openstack-infra | 06:16 | |
| *** sflanigan has joined #openstack-infra | 06:16 | |
| openstackgerrit | linzi proposed openstack-infra/project-config: modify zuul/layout.yaml and jenkins jobs https://review.openstack.org/384825 | 06:16 |
| *** thingee has joined #openstack-infra | 06:17 | |
| *** vern has joined #openstack-infra | 06:18 | |
| *** adam_g has joined #openstack-infra | 06:18 | |
| *** jamespage has quit IRC | 06:18 | |
| *** adam_g has quit IRC | 06:18 | |
| *** adam_g has joined #openstack-infra | 06:18 | |
| *** jamespag` has joined #openstack-infra | 06:18 | |
| *** spzala has quit IRC | 06:19 | |
| *** markmcclain has quit IRC | 06:20 | |
| *** vsaienko has joined #openstack-infra | 06:20 | |
| *** markmcclain has joined #openstack-infra | 06:21 | |
| *** markmcclain has quit IRC | 06:26 | |
| *** vern has quit IRC | 06:26 | |
| *** tnovacik_ has joined #openstack-infra | 06:30 | |
| *** tnovacik has joined #openstack-infra | 06:30 | |
| openstackgerrit | Karthik Prabhu Vinod proposed openstack-infra/project-config: Adding two other combinations of cinder multinode grenade job https://review.openstack.org/384836 | 06:33 |
| odyssey4me | cloudnull long night? | 06:34 |
| *** nherciu has joined #openstack-infra | 06:34 | |
| *** esikachev has joined #openstack-infra | 06:35 | |
| *** pcaruana has joined #openstack-infra | 06:36 | |
| *** sputnik13 has quit IRC | 06:37 | |
| *** esikachev has quit IRC | 06:40 | |
| *** sputnik13 has joined #openstack-infra | 06:41 | |
| *** yuanying has quit IRC | 06:41 | |
| *** nadya has quit IRC | 06:42 | |
| *** markmcclain has joined #openstack-infra | 06:43 | |
| *** hashar has joined #openstack-infra | 06:43 | |
| *** dtardivel has joined #openstack-infra | 06:43 | |
| *** martinkopec has joined #openstack-infra | 06:44 | |
| *** gaelL_ has quit IRC | 06:44 | |
| *** gaelL has joined #openstack-infra | 06:44 | |
| *** davidlenwell has joined #openstack-infra | 06:51 | |
| *** vern has joined #openstack-infra | 06:53 | |
| *** kaisers_ has quit IRC | 06:54 | |
| *** sshnaidm|away has quit IRC | 06:54 | |
| *** andreas_s has joined #openstack-infra | 06:55 | |
| odyssey4me | jhesketh is cloud8 also being taken offline? Cannot initiate the connection to mirror.regionone.osic-cloud8.openstack.org:80 | 06:56 |
| odyssey4me | that's from http://logs.openstack.org/09/384709/1/check/gate-openstack-ansible-galera_server-ansible-func-ubuntu-xenial/697e59b/console.html#_2016-10-11_06_52_47_520780 | 06:56 |
| *** davidlenwell has quit IRC | 06:56 | |
| openstackgerrit | Adit Sarfaty proposed openstack-infra/project-config: Add vmware-nsx tests as non voting to the vmware-nsxlib project https://review.openstack.org/384821 | 06:57 |
| *** watanabe_isao has joined #openstack-infra | 07:01 | |
| *** AJaeger has quit IRC | 07:01 | |
| *** jaosorior has joined #openstack-infra | 07:03 | |
| *** rbrndt has quit IRC | 07:04 | |
| *** thorst has joined #openstack-infra | 07:05 | |
| *** salv-orlando has joined #openstack-infra | 07:06 | |
| *** aeng has quit IRC | 07:08 | |
| *** pgadiya has joined #openstack-infra | 07:09 | |
| *** matrohon has joined #openstack-infra | 07:09 | |
| *** pgadiya has quit IRC | 07:09 | |
| *** ccamacho has joined #openstack-infra | 07:10 | |
| *** sflanigan has quit IRC | 07:11 | |
| *** thorst has quit IRC | 07:13 | |
| yolanda | good morning | 07:15 |
| *** davidlenwell has joined #openstack-infra | 07:16 | |
| *** markmcclain has quit IRC | 07:16 | |
| *** markmcclain has joined #openstack-infra | 07:17 | |
| *** rbrndt has joined #openstack-infra | 07:18 | |
| *** flaper87 has quit IRC | 07:24 | |
| *** sflanigan has joined #openstack-infra | 07:24 | |
| *** aeng has joined #openstack-infra | 07:25 | |
| *** oanson has joined #openstack-infra | 07:26 | |
| *** abregman has joined #openstack-infra | 07:26 | |
| openstackgerrit | Adit Sarfaty proposed openstack-infra/project-config: Add vmware-nsx tests as non voting to the vmware-nsxlib project https://review.openstack.org/384821 | 07:26 |
| *** amotoki has quit IRC | 07:26 | |
| *** vsaienko has quit IRC | 07:27 | |
| *** csomerville has joined #openstack-infra | 07:28 | |
| *** cody-somerville has quit IRC | 07:28 | |
| *** abregman has quit IRC | 07:28 | |
| *** pilgrimstack has joined #openstack-infra | 07:29 | |
| *** vsaienko has joined #openstack-infra | 07:33 | |
| *** oanson has quit IRC | 07:34 | |
| *** pilgrimstack has quit IRC | 07:37 | |
| *** amoralej|off is now known as amoralej | 07:39 | |
| *** jpena|off is now known as jpena | 07:39 | |
| *** qwertyco has quit IRC | 07:39 | |
| *** rbrndt has quit IRC | 07:42 | |
| *** rbrndt has joined #openstack-infra | 07:42 | |
| odyssey4me | argh, gerrit seems to be very slow :/ | 07:43 |
| *** salv-orlando has quit IRC | 07:43 | |
| *** davidlenwell has quit IRC | 07:44 | |
| *** davidlenwell has joined #openstack-infra | 07:46 | |
| *** amotoki has joined #openstack-infra | 07:51 | |
| *** amotoki has quit IRC | 07:51 | |
| *** nadya has joined #openstack-infra | 07:51 | |
| *** hogepodge has quit IRC | 07:51 | |
| *** amotoki has joined #openstack-infra | 07:51 | |
| *** rbrndt has quit IRC | 07:52 | |
| *** jesusaur has quit IRC | 07:52 | |
| *** drifterza has joined #openstack-infra | 07:53 | |
| openstackgerrit | Valeriy Ponomaryov proposed openstack-infra/project-config: [Manila] Use only xenial nodes for newton+ branches https://review.openstack.org/384862 | 07:55 |
| *** aarefiev has quit IRC | 07:55 | |
| rcarrillocruz | morning | 07:55 |
| rcarrillocruz | fungi: when you get around, mind replying nibalizer on https://review.openstack.org/#/c/378996/4 , re: adding a cacti hiera group | 07:56 |
| *** aarefiev has joined #openstack-infra | 07:56 | |
| *** pc_m has quit IRC | 07:57 | |
| *** dizquierdo has joined #openstack-infra | 07:57 | |
| *** markvoelker has quit IRC | 07:57 | |
| *** hogepodge has joined #openstack-infra | 07:58 | |
| *** docaedo4 has quit IRC | 07:58 | |
| *** jogo has quit IRC | 07:59 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** zzzeek has joined #openstack-infra | 08:00 | |
| *** e0ne has joined #openstack-infra | 08:00 | |
| *** gmann__ has quit IRC | 08:02 | |
| *** amitgandhinz has joined #openstack-infra | 08:04 | |
| *** jesusaur has joined #openstack-infra | 08:05 | |
| *** salv-orlando has joined #openstack-infra | 08:07 | |
| *** amitgandhinz has quit IRC | 08:09 | |
| *** dragonmaster_ has joined #openstack-infra | 08:10 | |
| *** docaedo4 has joined #openstack-infra | 08:11 | |
| *** pc_m has joined #openstack-infra | 08:12 | |
| *** thorst has joined #openstack-infra | 08:12 | |
| *** davidlenwell has quit IRC | 08:14 | |
| *** vsaienko has quit IRC | 08:17 | |
| *** tuanluong_ has joined #openstack-infra | 08:17 | |
| *** thorst has quit IRC | 08:17 | |
| *** davidlenwell has joined #openstack-infra | 08:18 | |
| *** slaweq has quit IRC | 08:18 | |
| *** dingyichen has quit IRC | 08:20 | |
| *** tuanluong has quit IRC | 08:21 | |
| *** sdake_ has quit IRC | 08:21 | |
| *** tnovacik has quit IRC | 08:21 | |
| *** tnovacik_ has quit IRC | 08:21 | |
| *** watanabe_isao has quit IRC | 08:22 | |
| *** LiYuenan has joined #openstack-infra | 08:22 | |
| *** jascott1 has quit IRC | 08:22 | |
| *** jordanP has joined #openstack-infra | 08:22 | |
| *** _degorenko|afk is now known as degorenko | 08:23 | |
| openstackgerrit | Merged openstack-infra/project-config: ceilometer: Add no-dsvm functional tests https://review.openstack.org/384685 | 08:23 |
| openstackgerrit | Merged openstack-infra/project-config: ceilometer: cleanup identity v3 jobs https://review.openstack.org/384686 | 08:23 |
| *** yuanying has joined #openstack-infra | 08:27 | |
| *** tuanluong__ has joined #openstack-infra | 08:28 | |
| *** kzaitsev1mb has joined #openstack-infra | 08:31 | |
| *** jamespag` is now known as jamespage | 08:31 | |
| *** burgerk_ has joined #openstack-infra | 08:32 | |
| *** tuanluong_ has quit IRC | 08:32 | |
| *** xarses_ has joined #openstack-infra | 08:32 | |
| *** ianychoi_ has joined #openstack-infra | 08:32 | |
| *** claudiub|2 has joined #openstack-infra | 08:32 | |
| *** Naeil has joined #openstack-infra | 08:32 | |
| *** john-davidge has joined #openstack-infra | 08:33 | |
| *** abregman has joined #openstack-infra | 08:33 | |
| *** derekh has joined #openstack-infra | 08:34 | |
| *** BobBall_ has joined #openstack-infra | 08:36 | |
| *** sc`_ has joined #openstack-infra | 08:36 | |
| jhesketh | odyssey4me: (sorry was away) yes, we should.. I'll push up a patch now | 08:36 |
| *** jeblair_ has joined #openstack-infra | 08:36 | |
| *** dosaboy_ has joined #openstack-infra | 08:37 | |
| *** admcleod has joined #openstack-infra | 08:37 | |
| *** admcleod has joined #openstack-infra | 08:37 | |
| *** sc68cal has joined #openstack-infra | 08:38 | |
| *** tuanluong has joined #openstack-infra | 08:38 | |
| *** jogo has joined #openstack-infra | 08:38 | |
| *** jogo has joined #openstack-infra | 08:38 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/project-config: Also take osic-cloud8 offline https://review.openstack.org/384876 | 08:38 |
| *** benj has joined #openstack-infra | 08:39 | |
| *** Hal1 has joined #openstack-infra | 08:39 | |
| *** jascott1 has joined #openstack-infra | 08:39 | |
| jhesketh | infra-root: ^ | 08:39 |
| *** anthonyper_ has joined #openstack-infra | 08:39 | |
| *** benj is now known as Guest36113 | 08:39 | |
| *** andreykurilin_ has joined #openstack-infra | 08:39 | |
| *** oomichi_ has joined #openstack-infra | 08:39 | |
| *** amoralej_ has joined #openstack-infra | 08:40 | |
| *** raissa_ has joined #openstack-infra | 08:40 | |
| *** rustlebee has joined #openstack-infra | 08:40 | |
| *** mjblack- has joined #openstack-infra | 08:40 | |
| *** arif-ali_ has joined #openstack-infra | 08:40 | |
| *** filler_ has joined #openstack-infra | 08:40 | |
| *** vicky has joined #openstack-infra | 08:40 | |
| *** jraju has joined #openstack-infra | 08:41 | |
| *** davidlenwell has quit IRC | 08:41 | |
| *** pc_m has quit IRC | 08:41 | |
| *** docaedo4 has quit IRC | 08:41 | |
| *** nadya has quit IRC | 08:41 | |
| *** markmcclain has quit IRC | 08:41 | |
| *** vern has quit IRC | 08:41 | |
| *** adam_g has quit IRC | 08:41 | |
| *** kzaitsev_mb has quit IRC | 08:41 | |
| *** arxcruz has quit IRC | 08:41 | |
| *** links has quit IRC | 08:41 | |
| *** vikrant has quit IRC | 08:41 | |
| *** yarkot has quit IRC | 08:41 | |
| *** caowei has quit IRC | 08:41 | |
| *** xarses has quit IRC | 08:41 | |
| *** camunoz has quit IRC | 08:41 | |
| *** claudiub has quit IRC | 08:41 | |
| *** pots has quit IRC | 08:41 | |
| *** karthikp_ has quit IRC | 08:41 | |
| *** erlon has quit IRC | 08:41 | |
| *** burgerk has quit IRC | 08:41 | |
| *** admcleod_ has quit IRC | 08:41 | |
| *** khappone has quit IRC | 08:41 | |
| *** BobBall has quit IRC | 08:41 | |
| *** skraynev has quit IRC | 08:41 | |
| *** dteselkin has quit IRC | 08:41 | |
| *** paul_ has quit IRC | 08:41 | |
| *** mjblack has quit IRC | 08:41 | |
| *** amoralej has quit IRC | 08:41 | |
| *** coreyob_ has quit IRC | 08:41 | |
| *** oomichi has quit IRC | 08:41 | |
| *** peterlisak has quit IRC | 08:41 | |
| *** sc68cal_ has quit IRC | 08:41 | |
| *** makowals has quit IRC | 08:41 | |
| *** Hal has quit IRC | 08:41 | |
| *** askb has quit IRC | 08:41 | |
| *** mrtenio has quit IRC | 08:41 | |
| *** anthonyper has quit IRC | 08:41 | |
| *** ianychoi has quit IRC | 08:41 | |
| *** amit213 has quit IRC | 08:41 | |
| *** unrahul has quit IRC | 08:41 | |
| *** dham1 has quit IRC | 08:41 | |
| *** jpmaxman has quit IRC | 08:41 | |
| *** ogelbukh has quit IRC | 08:41 | |
| *** zxiiro|pto has quit IRC | 08:41 | |
| *** evgenyl has quit IRC | 08:41 | |
| *** madhuvishy has quit IRC | 08:41 | |
| *** electrical has quit IRC | 08:41 | |
| *** ediardo has quit IRC | 08:41 | |
| *** tdurakov has quit IRC | 08:41 | |
| *** andrewbogott has quit IRC | 08:41 | |
| *** mwhahaha has quit IRC | 08:41 | |
| *** hrybacki has quit IRC | 08:41 | |
| *** rockstar has quit IRC | 08:41 | |
| *** isviridov_away has quit IRC | 08:41 | |
| *** dtroyer has quit IRC | 08:41 | |
| *** vaidy has quit IRC | 08:41 | |
| *** knikolla has quit IRC | 08:41 | |
| *** irtermite has quit IRC | 08:41 | |
| *** benj_ has quit IRC | 08:41 | |
| *** sc` has quit IRC | 08:41 | |
| *** scottynomad has quit IRC | 08:41 | |
| *** fungi has quit IRC | 08:41 | |
| *** arif-ali has quit IRC | 08:41 | |
| *** rm_work has quit IRC | 08:41 | |
| *** mgagne has quit IRC | 08:41 | |
| *** gus has quit IRC | 08:41 | |
| *** ryanpetrello has quit IRC | 08:41 | |
| *** JayF has quit IRC | 08:41 | |
| *** odyssey4me has quit IRC | 08:41 | |
| *** dstanek has quit IRC | 08:41 | |
| *** mjb` has quit IRC | 08:41 | |
| *** hyakuhei has quit IRC | 08:41 | |
| *** craige has quit IRC | 08:41 | |
| *** timsim has quit IRC | 08:41 | |
| *** jbryce has quit IRC | 08:41 | |
| *** honza_ has quit IRC | 08:41 | |
| *** lewo has quit IRC | 08:41 | |
| *** wcriswell has quit IRC | 08:41 | |
| *** charz has quit IRC | 08:41 | |
| *** skipp has quit IRC | 08:41 | |
| *** tsufiev has quit IRC | 08:41 | |
| *** russellb has quit IRC | 08:41 | |
| *** jeblair has quit IRC | 08:41 | |
| *** dosaboy has quit IRC | 08:41 | |
| *** filler has quit IRC | 08:41 | |
| *** andreykurilin has quit IRC | 08:41 | |
| *** raissa has quit IRC | 08:41 | |
| *** onovy has quit IRC | 08:41 | |
| *** arif-ali_ is now known as arif-ali | 08:41 | |
| *** mjblack- is now known as mjblack | 08:41 | |
| *** filler_ is now known as filler | 08:41 | |
| *** oomichi_ is now known as oomichi | 08:41 | |
| *** tuanluong__ has quit IRC | 08:41 | |
| *** amoralej_ is now known as amoralej | 08:41 | |
| *** anthonyper_ is now known as anthonyper | 08:41 | |
| *** sc`_ is now known as sc` | 08:41 | |
| *** coreyob has joined #openstack-infra | 08:41 | |
| *** dteselkin has joined #openstack-infra | 08:41 | |
| *** charz has joined #openstack-infra | 08:41 | |
| *** berendt has joined #openstack-infra | 08:41 | |
| *** askb has joined #openstack-infra | 08:42 | |
| *** pc_m has joined #openstack-infra | 08:42 | |
| *** yarkot has joined #openstack-infra | 08:42 | |
| *** kaisers_ has joined #openstack-infra | 08:42 | |
| openstackgerrit | Merged openstack-infra/system-config: Fix deadlink in wiki doc https://review.openstack.org/378280 | 08:43 |
| *** dizquierdo has quit IRC | 08:43 | |
| *** salv-orl_ has joined #openstack-infra | 08:44 | |
| *** caowei has joined #openstack-infra | 08:44 | |
| *** paul_ has joined #openstack-infra | 08:44 | |
| *** skipp has joined #openstack-infra | 08:44 | |
| *** mhickey has joined #openstack-infra | 08:44 | |
| *** mrtenio has joined #openstack-infra | 08:44 | |
| *** tsufiev has joined #openstack-infra | 08:44 | |
| *** wcriswell has joined #openstack-infra | 08:44 | |
| *** jogo has quit IRC | 08:46 | |
| *** salv-orlando has quit IRC | 08:46 | |
| *** scottynomad has joined #openstack-infra | 08:46 | |
| *** kaisers_ has quit IRC | 08:47 | |
| openstackgerrit | Merged openstack-infra/system-config: hiera/common.yaml: Drop #openstack-astara channel https://review.openstack.org/384657 | 08:48 |
| openstackgerrit | Merged openstack-infra/project-config: Make swift tox-func-in-process-encryption job voting https://review.openstack.org/384408 | 08:49 |
| *** unrahul has joined #openstack-infra | 08:49 | |
| *** aunnam has joined #openstack-infra | 08:49 | |
| *** esikachev has joined #openstack-infra | 08:49 | |
| *** docaedo4 has joined #openstack-infra | 08:49 | |
| *** davidlenwell has joined #openstack-infra | 08:49 | |
| *** nadya has joined #openstack-infra | 08:49 | |
| *** markmcclain has joined #openstack-infra | 08:49 | |
| *** vern has joined #openstack-infra | 08:49 | |
| *** adam_g has joined #openstack-infra | 08:49 | |
| *** arxcruz has joined #openstack-infra | 08:49 | |
| *** karthikp_ has joined #openstack-infra | 08:49 | |
| *** erlon has joined #openstack-infra | 08:49 | |
| *** khappone has joined #openstack-infra | 08:49 | |
| *** skraynev has joined #openstack-infra | 08:49 | |
| *** peterlisak has joined #openstack-infra | 08:49 | |
| *** makowals has joined #openstack-infra | 08:49 | |
| *** amit213 has joined #openstack-infra | 08:49 | |
| *** dham1 has joined #openstack-infra | 08:49 | |
| *** jpmaxman has joined #openstack-infra | 08:49 | |
| *** ogelbukh has joined #openstack-infra | 08:49 | |
| *** zxiiro|pto has joined #openstack-infra | 08:49 | |
| *** evgenyl has joined #openstack-infra | 08:49 | |
| *** madhuvishy has joined #openstack-infra | 08:49 | |
| *** electrical has joined #openstack-infra | 08:49 | |
| *** ediardo has joined #openstack-infra | 08:49 | |
| *** tdurakov has joined #openstack-infra | 08:49 | |
| *** andrewbogott has joined #openstack-infra | 08:49 | |
| *** mwhahaha has joined #openstack-infra | 08:49 | |
| *** hrybacki has joined #openstack-infra | 08:49 | |
| *** rockstar has joined #openstack-infra | 08:49 | |
| *** isviridov_away has joined #openstack-infra | 08:49 | |
| *** dtroyer has joined #openstack-infra | 08:49 | |
| *** vaidy has joined #openstack-infra | 08:49 | |
| *** knikolla has joined #openstack-infra | 08:49 | |
| *** irtermite has joined #openstack-infra | 08:49 | |
| *** dstanek has joined #openstack-infra | 08:49 | |
| *** fungi has joined #openstack-infra | 08:49 | |
| *** rm_work has joined #openstack-infra | 08:49 | |
| *** JayF has joined #openstack-infra | 08:49 | |
| *** mgagne has joined #openstack-infra | 08:49 | |
| *** gus has joined #openstack-infra | 08:49 | |
| *** jbryce has joined #openstack-infra | 08:49 | |
| *** odyssey4me has joined #openstack-infra | 08:49 | |
| *** ryanpetrello has joined #openstack-infra | 08:49 | |
| *** timsim has joined #openstack-infra | 08:49 | |
| *** mjb` has joined #openstack-infra | 08:49 | |
| *** craige has joined #openstack-infra | 08:49 | |
| *** hyakuhei has joined #openstack-infra | 08:49 | |
| *** honza_ has joined #openstack-infra | 08:49 | |
| *** lewo has joined #openstack-infra | 08:49 | |
| *** onovy has joined #openstack-infra | 08:49 | |
| *** slaweq has joined #openstack-infra | 08:52 | |
| *** strigazi_AFK is now known as strigazi | 08:52 | |
| *** ogelbukh has quit IRC | 08:52 | |
| *** andrewbogott has quit IRC | 08:52 | |
| *** mwhahaha has quit IRC | 08:52 | |
| *** mwhahaha has joined #openstack-infra | 08:53 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/shade: Add description field to create_user method https://review.openstack.org/384404 | 08:53 |
| *** andrewbogott has joined #openstack-infra | 08:53 | |
| *** evgenyl has quit IRC | 08:53 | |
| *** docaedo4 has quit IRC | 08:54 | |
| *** salv-orl_ has quit IRC | 08:55 | |
| *** ogelbukh has joined #openstack-infra | 08:56 | |
| *** evgenyl has joined #openstack-infra | 08:57 | |
| *** markvoelker has joined #openstack-infra | 08:57 | |
| *** sambetts has joined #openstack-infra | 08:58 | |
| *** chandankumar has joined #openstack-infra | 09:00 | |
| *** r-mibu has quit IRC | 09:00 | |
| *** markvoelker has quit IRC | 09:03 | |
| *** HeOS has joined #openstack-infra | 09:03 | |
| *** panda has joined #openstack-infra | 09:05 | |
| *** adam_g has quit IRC | 09:06 | |
| *** mfedosin has joined #openstack-infra | 09:06 | |
| openstackgerrit | Merged openstack-infra/project-config: [magnum] Remove DIB elements from mesos build job https://review.openstack.org/384569 | 09:06 |
| *** adam_g has joined #openstack-infra | 09:07 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/shade: Add description field to create_user method https://review.openstack.org/384404 | 09:08 |
| *** adam_g has quit IRC | 09:08 | |
| *** adam_g has joined #openstack-infra | 09:08 | |
| openstackgerrit | Merged openstack-infra/project-config: Add releasenote job for python-senlinclient https://review.openstack.org/384365 | 09:13 |
| *** vsaienko has joined #openstack-infra | 09:14 | |
| *** spzala has joined #openstack-infra | 09:14 | |
| *** thorst has joined #openstack-infra | 09:15 | |
| *** r-mibu has joined #openstack-infra | 09:15 | |
| *** tmorin1 has joined #openstack-infra | 09:16 | |
| tmorin1 | hi everyone, I've got a question on how a "Depends-On" affects what is present in a test environment | 09:17 |
| *** nadya has quit IRC | 09:18 | |
| tmorin1 | my question is: assuming change Foo in project A has a Depends-On: change Bar in project B, does a test running on change Foo draws the branch of project B for change Bar ? and if so, how ? | 09:18 |
| *** nadya has joined #openstack-infra | 09:18 | |
| *** yamamoto_ has quit IRC | 09:18 | |
| *** spzala has quit IRC | 09:19 | |
| *** yamamoto has joined #openstack-infra | 09:20 | |
| *** tnovacik_ has joined #openstack-infra | 09:20 | |
| *** tnovacik has joined #openstack-infra | 09:20 | |
| frickler | tmorin1: yes, zuul does this, but I don't know the details. note that this only works if B is consumed from git, i.e. is not a library | 09:21 |
| *** docaedo4 has joined #openstack-infra | 09:21 | |
| openstackgerrit | Lee Yarwood proposed openstack/gertty: Match and replace SHA1 strings with an internal search https://review.openstack.org/384901 | 09:22 |
| *** thorst has quit IRC | 09:23 | |
| *** dtardivel has quit IRC | 09:24 | |
| *** davidlenwell has quit IRC | 09:24 | |
| *** vern has quit IRC | 09:24 | |
| *** salv-orlando has joined #openstack-infra | 09:25 | |
| *** markmcclain has quit IRC | 09:25 | |
| *** openstack has joined #openstack-infra | 09:30 | |
| *** peterlisak has quit IRC | 09:30 | |
| *** onovy has quit IRC | 09:30 | |
| *** kzaitsev1mb has quit IRC | 09:34 | |
| tobiash_ | tmorin1: zuul-cloner handles that automatically if you clone project A and B together in a single build job | 09:35 |
| tmorin1 | tobiash_: I've indeed seen an occurence where this happens (projectA == networking-bgpvpn, and project B==neutron) | 09:36 |
| tmorin1 | tobiash_: but I've also seen an occurence where this does not happen (projectA = networking-bgpvpn, and project B==networking-odl) | 09:36 |
| tmorin1 | tobiash_: for the working case projectA draws projectB with a script that indeed uses zuul-cloner | 09:37 |
| LiYuenan | Hi everyone | 09:37 |
| *** BobBall_ has quit IRC | 09:37 | |
| tmorin1 | tobiash_: in the non-working case, projectA simply draws project B with test-requirements.txt "-e git ..." | 09:38 |
| *** caowei has quit IRC | 09:38 | |
| *** onovy has joined #openstack-infra | 09:38 | |
| *** peterlisak has joined #openstack-infra | 09:38 | |
| *** gildub has quit IRC | 09:38 | |
| *** jaosorior has quit IRC | 09:39 | |
| tobiash_ | tmorin1: in that case the job must handle the zuul variables itself and take the correct source of project B. Otherwise depends-on has no effect other than scheduling of zuul | 09:39 |
| LiYuenan | I am interested in compass. It's a project in Openstack. I want to review my cord, but the log show that I don't have access rights. Does anyone can help me? Thanks! | 09:39 |
| *** jaosorior has joined #openstack-infra | 09:39 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci: Add "Failed creating OVB stack" to failure conditions in testenv-client https://review.openstack.org/384906 | 09:39 |
| LiYuenan | By the way, I am Foundation Member and I also submitted OpenStack Individual Contributor License Agreement. | 09:40 |
| *** karthikp_ has quit IRC | 09:41 | |
| *** jpich has joined #openstack-infra | 09:41 | |
| tmorin1 | tobiash_: ok, understood: the depends-on magic is done by zuul-cloner and not if a dependency is added via (test-)requirements.txt | 09:41 |
| tobiash_ | tmorin1: yes, depends-on haseffect on what zuul-cloner does and also on the scheduling of zuul in the dependent pipeline manager (gate) where it ensures that the changes are merged in the correct order | 09:43 |
| *** shardy has joined #openstack-infra | 09:47 | |
| tmorin1 | tobiash_: yes, I was aware of the second effect | 09:47 |
| tmorin1 | tobiash_: thanks! | 09:47 |
| *** dtardivel has joined #openstack-infra | 09:48 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci: Add "Failed creating OVB stack" to failure conditions in testenv-client https://review.openstack.org/384906 | 09:50 |
| *** tosky has joined #openstack-infra | 09:50 | |
| *** tuanluong has quit IRC | 09:57 | |
| *** dimtruck is now known as zz_dimtruck | 09:57 | |
| *** zz_dimtruck is now known as dimtruck | 09:57 | |
| *** markvoelker has joined #openstack-infra | 09:59 | |
| *** Naeil has quit IRC | 09:59 | |
| *** chem has quit IRC | 10:00 | |
| *** zhurong has quit IRC | 10:02 | |
| *** markvoelker has quit IRC | 10:04 | |
| *** pblaho has joined #openstack-infra | 10:04 | |
| *** rbrndt has joined #openstack-infra | 10:04 | |
| *** amitgandhinz has joined #openstack-infra | 10:06 | |
| *** dimtruck is now known as zz_dimtruck | 10:07 | |
| *** abregman is now known as abregman|afk | 10:07 | |
| *** kashyap has left #openstack-infra | 10:08 | |
| *** _vsaienko has joined #openstack-infra | 10:09 | |
| *** vsaienko has quit IRC | 10:09 | |
| *** amitgandhinz has quit IRC | 10:10 | |
| *** jraju has quit IRC | 10:15 | |
| *** jkilpatr has joined #openstack-infra | 10:16 | |
| *** ianychoi_ is now known as ianychoi | 10:17 | |
| vsaienk0 | ianw, pabelanger, clarkb: sorry for direct ping, could you please help to review/merge chain from 3 patches start from https://review.openstack.org/#/c/364830/ to unblock ironic team to land ironic code. Thank you! | 10:17 |
| *** sdague has joined #openstack-infra | 10:19 | |
| *** tqtran has joined #openstack-infra | 10:20 | |
| *** dtantsur|afk is now known as dtantsur | 10:20 | |
| *** thorst has joined #openstack-infra | 10:20 | |
| *** Liuqing has quit IRC | 10:22 | |
| *** sdake has joined #openstack-infra | 10:23 | |
| openstackgerrit | Proskurin Kirill proposed openstack-infra/project-config: Add fuel-ccp-ceph repository https://review.openstack.org/384913 | 10:23 |
| *** tqtran has quit IRC | 10:24 | |
| *** kmartin has quit IRC | 10:26 | |
| *** rossella_s has quit IRC | 10:27 | |
| *** zz_dimtruck is now known as dimtruck | 10:27 | |
| *** thorst has quit IRC | 10:28 | |
| *** rossella_s has joined #openstack-infra | 10:28 | |
| *** kzaitsev_mb has joined #openstack-infra | 10:30 | |
| *** Naeil has joined #openstack-infra | 10:30 | |
| *** kaisers_ has joined #openstack-infra | 10:31 | |
| *** jogo has joined #openstack-infra | 10:32 | |
| *** kaisers_ has quit IRC | 10:35 | |
| *** alex_xu has quit IRC | 10:36 | |
| *** dfflanders has quit IRC | 10:37 | |
| *** dimtruck is now known as zz_dimtruck | 10:37 | |
| *** alex_xu has joined #openstack-infra | 10:39 | |
| *** akorzeni__ has joined #openstack-infra | 10:40 | |
| *** yanyanhu has quit IRC | 10:41 | |
| *** korzen has quit IRC | 10:42 | |
| *** salv-orlando has quit IRC | 10:45 | |
| *** aspiers has quit IRC | 10:45 | |
| *** arif-ali has quit IRC | 10:47 | |
| *** aspiers has joined #openstack-infra | 10:47 | |
| *** BobBall has joined #openstack-infra | 10:47 | |
| *** markmcclain has joined #openstack-infra | 10:48 | |
| *** korzen has joined #openstack-infra | 10:48 | |
| *** akorzeni__ has quit IRC | 10:49 | |
| *** arif-ali has joined #openstack-infra | 10:50 | |
| *** mmedvede has joined #openstack-infra | 10:52 | |
| *** markmcclain has quit IRC | 10:53 | |
| *** smcginnis has quit IRC | 10:53 | |
| *** jkilpatr has quit IRC | 10:55 | |
| *** vsaienko has joined #openstack-infra | 10:59 | |
| *** markvoelker has joined #openstack-infra | 10:59 | |
| *** _vsaienko has quit IRC | 10:59 | |
| odyssey4me | is there any progress on disabling cloud8 in nodepool - I'm still seeing error like: Failed to fetch http://mirror.regionone.osic-cloud8.openstack.org in our tests | 10:59 |
| odyssey4me | jhesketh yolanda ^ | 11:00 |
| jhesketh | odyssey4me: hmm, I was hoping they were tranisent enough that I wouldn't need to push through https://review.openstack.org/#/c/384876/.. but if no other infra-roots are around I can do so | 11:01 |
| odyssey4me | jhesketh I'm seeing both apt and pip mirror failures fairly consistently for cloud8 :/ | 11:02 |
| jhesketh | :-( | 11:02 |
| odyssey4me | yeah, not ideal, but until we have a solution for cloud1 I don't see how we can continue using cloud8 | 11:02 |
| jhesketh | yep, I've +w | 11:03 |
| jhesketh | *w'd it | 11:03 |
| yolanda | hi odyssey4me , i was not aware of problems sorry | 11:03 |
| *** nadya has quit IRC | 11:03 | |
| *** yamamoto has quit IRC | 11:03 | |
| jhesketh | yolanda: osic-cloud1 is having network issues affecting the mirror | 11:03 |
| jhesketh | because cloud8 uses the same mirror it also experienced the problems | 11:03 |
| odyssey4me | yolanda cloud8 unfortunately uses cloud1's mirror | 11:03 |
| jhesketh | currently cloud1 is offline, I've just +w'd the change to offline cloud8 | 11:03 |
| odyssey4me | :) what jhesketh said | 11:03 |
| yolanda | k, thanks for the update | 11:04 |
| *** markvoelker has quit IRC | 11:04 | |
| jhesketh | yolanda: cloudnull was working through the issue which has to do with the default routes... I've started a change here but based off feedback it probably needs work: https://review.openstack.org/#/c/384822/1 | 11:05 |
| *** rods has quit IRC | 11:06 | |
| *** _vsaienko has joined #openstack-infra | 11:06 | |
| jhesketh | I'm also not sure if this is the best solution, so I'm leaving it for other infra-roots to take a look at | 11:06 |
| yolanda | checking | 11:06 |
| *** rhallisey has joined #openstack-infra | 11:06 | |
| *** vsaienko has quit IRC | 11:06 | |
| dragonmaster_ | hey all, anyone here seen experienced issues with zuul-launcher ? | 11:06 |
| yolanda | can we have control about dhcp servers, not giving default routes? | 11:06 |
| yolanda | i remember hitting some problems like that in infra-cloud, and i had to add an option to my dhcp server, to don't provide default routes by default | 11:07 |
| *** morgan has quit IRC | 11:07 | |
| dragonmaster_ | http://paste.openstack.org/show/585306/ | 11:07 |
| jhesketh | yolanda: something we can ask cloudnull/osic folk | 11:07 |
| *** amitgandhinz has joined #openstack-infra | 11:07 | |
| dragonmaster_ | class JJB(jenkins_jobs.builder.Builder): | 11:08 |
| dragonmaster_ | AttributeError: 'module' object has no attribute 'Builder' | 11:08 |
| dragonmaster_ | both zuul and JJB are built from latest github masters | 11:08 |
| *** rbrndt has quit IRC | 11:09 | |
| *** links has joined #openstack-infra | 11:09 | |
| *** rbrndt has joined #openstack-infra | 11:09 | |
| *** docaedo4 has quit IRC | 11:11 | |
| jhesketh | dragonmaster_: I think you might need to pin the jenkins version... our puppet-zuul module pins it to 1.6.1 | 11:12 |
| *** amitgandhinz has quit IRC | 11:12 | |
| jhesketh | because 2.5 isn't really intended for public use I don't think there are plans to support newer jjb | 11:12 |
| *** qwertyco has joined #openstack-infra | 11:12 | |
| dragonmaster_ | yeah, i see that gearman was removed between 2.1 and 2.5 | 11:13 |
| jhesketh | dragonmaster_: eh? removed from where | 11:13 |
| dragonmaster_ | removed from zuul | 11:13 |
| dragonmaster_ | https://github.com/openstack-infra/zuul/tree/2.1.0/zuul/launcher | 11:14 |
| *** sdake_ has joined #openstack-infra | 11:14 | |
| dragonmaster_ | or at least the ansible launch server code was added in 2.5 | 11:14 |
| *** _vsaienko has quit IRC | 11:15 | |
| *** vsaienko has joined #openstack-infra | 11:15 | |
| jhesketh | dragonmaster_: gear is still in there.. and yes, the ansible launcher is 2.5 | 11:15 |
| *** qwertyco has quit IRC | 11:15 | |
| *** sdake has quit IRC | 11:15 | |
| *** docaedo4 has joined #openstack-infra | 11:16 | |
| *** qwertyco has joined #openstack-infra | 11:16 | |
| *** jkilpatr has joined #openstack-infra | 11:17 | |
| openstackgerrit | Merged openstack-infra/project-config: Also take osic-cloud8 offline https://review.openstack.org/384876 | 11:17 |
| *** markmcclain has joined #openstack-infra | 11:18 | |
| dragonmaster_ | thanks jhesketh | 11:20 |
| jhesketh | anytime :-) | 11:21 |
| dragonmaster_ | i think i’ll need to build zuul from 2.1 possibly, the issue seems to come from the ansible lancher module (which we still need tons of work to get to internally ) | 11:22 |
| *** cgalan has joined #openstack-infra | 11:22 | |
| jhesketh | dragonmaster_: zuul master should still work with jenkins... what issue(s) are you seeing? | 11:24 |
| jhesketh | (you don't need to run the zuul-launcher if you're running jenkins) | 11:24 |
| *** AJaeger has joined #openstack-infra | 11:25 | |
| *** yolanda has quit IRC | 11:25 | |
| *** raissa_ is now known as raissa | 11:25 | |
| dragonmaster_ | well, gearman bitches about the jobs not existing there | 11:25 |
| jhesketh | dragonmaster_: are you using jenkins? | 11:26 |
| *** thorst has joined #openstack-infra | 11:26 | |
| dragonmaster_ | well, we are planning to implement it still setting everything up | 11:26 |
| jhesketh | right, so once you have jenkins done it'll register a CAN_DO with gearman for each of the jobs it knows about | 11:27 |
| jhesketh | and that'll solve any NOT_REGISTERED erros you may see | 11:27 |
| dragonmaster_ | ok, so if my job is a merge-check, that job needs to exists in jenkins | 11:28 |
| dragonmaster_ | i was hoping to get JJB to register the jobs, as we are using dynamic names | 11:29 |
| jhesketh | dragonmaster_: no, merges are a special case... you need to configure and run a zuul-merger for that | 11:29 |
| *** markmcclain has quit IRC | 11:29 | |
| dragonmaster_ | http://paste.openstack.org/show/585309/ | 11:29 |
| jhesketh | dragonmaster_: jjb will compile the jobs into jenkins xml, but it is jenkins that will register the jobs with zuul | 11:29 |
| dragonmaster_ | zuul-merger is configured (bar i based this off the project-configs and zuul config from the pupet modules) | 11:30 |
| *** rods has joined #openstack-infra | 11:30 | |
| *** markmcclain has joined #openstack-infra | 11:30 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Add #openstack-charms and associated projects https://review.openstack.org/329129 | 11:30 |
| *** abregman|afk is now known as abregman | 11:31 | |
| AJaeger | jhesketh, could you +2 the change above, please? I just rebased - I think we have a free channel and that's the oldest in the queue waiting for one... | 11:31 |
| dragonmaster_ | !paste 585309 is my logs | 11:31 |
| openstack | dragonmaster_: Error: "paste" is not a valid command. | 11:31 |
| *** rtheis has joined #openstack-infra | 11:31 | |
| jhesketh | AJaeger: if you're sure we have a free channel | 11:32 |
| *** thorst has quit IRC | 11:32 | |
| jhesketh | dragonmaster_: is configs-merge a job you intend on running? | 11:32 |
| dragonmaster_ | http://paste.openstack.org/show/585310/ < zuul config | 11:33 |
| *** abregman is now known as abregman|afk | 11:33 | |
| AJaeger | jhesketh: just double checking ;) - I approved once change to remove a channel | 11:33 |
| jhesketh | AJaeger: cool | 11:33 |
| jhesketh | dragonmaster_: could you please paste your layout.yaml | 11:33 |
| dragonmaster_ | sure, one sec | 11:33 |
| AJaeger | jhesketh: we test for the number of channels in the irc test, so should be all fine - but I'll rebased and wait for results | 11:34 |
| jhesketh | ack | 11:34 |
| *** nadya has joined #openstack-infra | 11:35 | |
| dragonmaster_ | http://paste.openstack.org/show/585311/ | 11:35 |
| dragonmaster_ | ^ layout.yaml | 11:35 |
| *** asselin has joined #openstack-infra | 11:36 | |
| *** jamielennox is now known as jamielennox|away | 11:36 | |
| jhesketh | dragonmaster_: okay, so you probably don't need all the pipelines there, but they don't hurt | 11:36 |
| jhesketh | dragonmaster_: the configs-merge part is the job name expected to be ran... you probably want to replace this with the names of the jobs you actually want and have then configured in jenkins (via jjb) | 11:37 |
| *** EricGonczer_ has joined #openstack-infra | 11:38 | |
| openstackgerrit | John Davidge proposed openstack-infra/elastic-recheck: Add query for neutron bug 1632290 https://review.openstack.org/384944 | 11:39 |
| openstack | bug 1632290 in neutron "RuntimeError: Metadata proxy didn't spawn" [Undecided,New] https://launchpad.net/bugs/1632290 | 11:39 |
| openstackgerrit | Merged openstack-infra/project-config: Drop bots from #openstack-astara https://review.openstack.org/383872 | 11:39 |
| *** askb has quit IRC | 11:39 | |
| *** timello has quit IRC | 11:39 | |
| dragonmaster_ | so, i was hoping i could do something like that hitting the job:: | 11:39 |
| dragonmaster_ | - name: ^.*-merge$ | 11:39 |
| dragonmaster_ | failure-message: Unable to merge change, please rebase and try again. | 11:39 |
| *** yolanda has joined #openstack-infra | 11:41 | |
| dragonmaster_ | alternatively, in the “gate” component call set_run_options, which is included in the python file at the top | 11:41 |
| jhesketh | dragonmaster_: zuul will post a merge failure message... take a look at http://docs.openstack.org/infra/zuul/zuul.html#pipelines (search for merge-failure) | 11:42 |
| jhesketh | dragonmaster_: openstack's layout.yaml is very complicated. I'd suggest taking a look at the docs | 11:43 |
| dragonmaster_ | i figured it’s complex :P | 11:43 |
| *** jpena is now known as jpena|lunch | 11:43 | |
| *** adam_g has quit IRC | 11:43 | |
| dragonmaster_ | partially why i’m here | 11:44 |
| jhesketh | :-) | 11:44 |
| *** yamamoto has joined #openstack-infra | 11:44 | |
| *** jamesdenton has quit IRC | 11:45 | |
| dragonmaster_ | also why i’m trying a simple job first | 11:45 |
| *** timello has joined #openstack-infra | 11:45 | |
| jhesketh | there's also some example configs in the zuul repo (in etc/) | 11:45 |
| *** yamamoto_ has joined #openstack-infra | 11:46 | |
| *** markmcclain has quit IRC | 11:46 | |
| jhesketh | although that's using the project-merge which isn't needed | 11:46 |
| *** thingee has quit IRC | 11:47 | |
| *** dprince has joined #openstack-infra | 11:47 | |
| openstackgerrit | Merged openstack-infra/project-config: Add #openstack-charms and associated projects https://review.openstack.org/329129 | 11:47 |
| dragonmaster_ | yeah, thats where i started | 11:47 |
| *** lucas-afk is now known as lucasagomes | 11:48 | |
| *** openstackgerrit has quit IRC | 11:48 | |
| jhesketh | heh, yeah it's a bit outdated sorry | 11:48 |
| dragonmaster_ | and got lost with openstack-infra/project-config/zuul | 11:48 |
| *** openstackgerrit has joined #openstack-infra | 11:48 | |
| jhesketh | hopefully the docs themselves aren't too far out of date... unfortunately they are difficult to read as a user story | 11:49 |
| *** thingee has joined #openstack-infra | 11:49 | |
| dragonmaster_ | basically drew up the confis between zuul etc, and the puppet template | 11:49 |
| jhesketh | it's quite late here, so I need to disappear sorry.. others will be around soon though who may be able to help you more | 11:49 |
| dragonmaster_ | yeah, true | 11:49 |
| jhesketh | yolanda: can you pass along the notes for osic-cloud please? | 11:49 |
| dragonmaster_ | deep end, i guess :D | 11:49 |
| yolanda | hi jhesketh , i added my comments on the review | 11:50 |
| jhesketh | dragonmaster_: best of luck :-). There's also the mailing list | 11:50 |
| *** yamamoto has quit IRC | 11:50 | |
| dragonmaster_ | btw, for any other people asking - this guy did a decent write up - http://abregman.com/2016/09/06/openstack-infra-how-to-deploy-zuul/ | 11:50 |
| jhesketh | yolanda: cool.. I meant pass along why it's disabled etc to other infra-roots | 11:50 |
| yolanda | ah ok, sure | 11:51 |
| jhesketh | anyway, I'm off | 11:51 |
| jhesketh | night all! :-) | 11:51 |
| yolanda | bye, have a good night | 11:51 |
| dragonmaster_ | thanks for the assist jhesketh | 11:52 |
| dragonmaster_ | cheers | 11:52 |
| *** number80 has quit IRC | 11:54 | |
| *** thorst has joined #openstack-infra | 11:54 | |
| *** arxcruz has quit IRC | 11:54 | |
| *** rfolco has joined #openstack-infra | 11:55 | |
| *** flaper87 has joined #openstack-infra | 11:57 | |
| *** EricGonc_ has joined #openstack-infra | 11:57 | |
| *** flaper87 is now known as Guest9738 | 11:58 | |
| *** EricGonczer_ has quit IRC | 11:59 | |
| *** markvoelker has joined #openstack-infra | 12:00 | |
| *** Guest9738 has quit IRC | 12:01 | |
| *** dargains has joined #openstack-infra | 12:01 | |
| *** kgiusti has joined #openstack-infra | 12:03 | |
| *** claudiub has joined #openstack-infra | 12:03 | |
| *** markvoelker has quit IRC | 12:05 | |
| *** zhurong has joined #openstack-infra | 12:06 | |
| *** claudiub|2 has quit IRC | 12:06 | |
| *** arxcruz has joined #openstack-infra | 12:06 | |
| *** honza_ is now known as honza | 12:07 | |
| *** honza is now known as Guest60959 | 12:08 | |
| *** amitgandhinz has joined #openstack-infra | 12:08 | |
| *** Liuqing has joined #openstack-infra | 12:09 | |
| *** pilgrimstack has joined #openstack-infra | 12:11 | |
| *** pilgrimstack has quit IRC | 12:11 | |
| openstackgerrit | Hiroyuki Eguchi proposed openstack-infra/project-config: Add openstack/hibana project https://review.openstack.org/384965 | 12:12 |
| *** dave-mccowan has joined #openstack-infra | 12:12 | |
| *** amitgandhinz has quit IRC | 12:13 | |
| *** edmondsw has joined #openstack-infra | 12:14 | |
| *** salv-orlando has joined #openstack-infra | 12:14 | |
| *** trown|outtypewww is now known as trown | 12:15 | |
| *** pradk has joined #openstack-infra | 12:15 | |
| *** openstackgerrit has quit IRC | 12:18 | |
| *** ccamacho is now known as ccamacho|lunch | 12:18 | |
| *** openstackgerrit has joined #openstack-infra | 12:18 | |
| *** tpsilva has joined #openstack-infra | 12:19 | |
| *** kaisers_ has joined #openstack-infra | 12:20 | |
| *** amoralej is now known as amoralej|lunch | 12:23 | |
| *** Liuqing has quit IRC | 12:24 | |
| *** kaisers_ has quit IRC | 12:25 | |
| openstackgerrit | Masayuki Igawa proposed openstack-infra/devstack-gate: SUPER WIP: Use new tempest run workflow https://review.openstack.org/355666 | 12:26 |
| *** EricGonc_ has quit IRC | 12:26 | |
| *** yamamoto_ has quit IRC | 12:27 | |
| *** Hal1 has quit IRC | 12:28 | |
| *** zz_dimtruck is now known as dimtruck | 12:28 | |
| *** jpich has quit IRC | 12:28 | |
| *** mat128 is now known as mat128|afk | 12:29 | |
| *** yamamoto has joined #openstack-infra | 12:30 | |
| *** jpena|lunch is now known as jpena | 12:37 | |
| *** gouthamr has joined #openstack-infra | 12:38 | |
| *** Naeil has quit IRC | 12:38 | |
| *** dimtruck is now known as zz_dimtruck | 12:38 | |
| *** dmsimard|pto is now known as dmsimard | 12:40 | |
| *** gouthamr has quit IRC | 12:41 | |
| *** korzen has quit IRC | 12:41 | |
| *** markvoelker has joined #openstack-infra | 12:43 | |
| *** gordc has joined #openstack-infra | 12:43 | |
| *** jcoufal has joined #openstack-infra | 12:44 | |
| *** jgriffith_away is now known as jgriffith | 12:44 | |
| *** Hal has joined #openstack-infra | 12:44 | |
| *** baoli has joined #openstack-infra | 12:45 | |
| *** amotoki has quit IRC | 12:45 | |
| *** baoli_ has joined #openstack-infra | 12:46 | |
| *** vicky has quit IRC | 12:47 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul: Add topic filter support https://review.openstack.org/384979 | 12:47 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul: Fix missing mutex release when aborting builds https://review.openstack.org/384980 | 12:48 |
| *** rlandy has joined #openstack-infra | 12:49 | |
| *** baoli has quit IRC | 12:50 | |
| tobiash_ | jhesketh: ^ could be important if openstack is using the mutex support of zuul | 12:50 |
| *** qwertyco has quit IRC | 12:50 | |
| *** flaper87 has joined #openstack-infra | 12:51 | |
| *** flaper87 has quit IRC | 12:51 | |
| *** flaper87 has joined #openstack-infra | 12:51 | |
| *** tlian has joined #openstack-infra | 12:51 | |
| *** docaedo4 has quit IRC | 12:52 | |
| *** moravec has quit IRC | 12:53 | |
| *** mat128|afk is now known as mat128 | 12:54 | |
| cloudnull | mornings | 12:56 |
| *** dragonmaster_ has quit IRC | 12:58 | |
| *** mdrabe has joined #openstack-infra | 12:58 | |
| *** jpich has joined #openstack-infra | 12:58 | |
| *** ihrachys has joined #openstack-infra | 12:59 | |
| *** xyang1 has joined #openstack-infra | 13:00 | |
| *** amitgandhinz has joined #openstack-infra | 13:02 | |
| *** jheroux has joined #openstack-infra | 13:02 | |
| ihrachys | AJaeger: I don't follow though, why would we create test databases for neutron-lib periodic unit test jobs? | 13:03 |
| *** chem has joined #openstack-infra | 13:03 | |
| *** flepied has quit IRC | 13:03 | |
| *** Guest60959 is now known as honza | 13:04 | |
| AJaeger | ihrachys: https://review.openstack.org/#/c/346271/ introduced that | 13:05 |
| *** fguillot has joined #openstack-infra | 13:05 | |
| *** dargains has quit IRC | 13:05 | |
| *** amotoki has joined #openstack-infra | 13:06 | |
| AJaeger | ihrachys: the periodic job needs to run on all neutron packages with the same setup - and there's at least one neutron repo that needs the databases | 13:06 |
| cloudnull | yolanda: o/ | 13:06 |
| ihrachys | AJaeger: couldn't we have those additional builders just where they are needed? in that particular repo? | 13:06 |
| cloudnull | are there notes that we should be looking into ? | 13:06 |
| AJaeger | ihrachys: not with current zuul | 13:07 |
| *** Goneri has joined #openstack-infra | 13:07 | |
| ihrachys | AJaeger: ack :) that's enough for me. :) | 13:07 |
| AJaeger | ihrachys: if you feel strongly, discuss with armax duplicating the periodic job - one version with database and one without. Not sure it's worth that... | 13:08 |
| ihrachys | AJaeger: I am fine. we could have a more explicit explanation of the rationale in the comments of the bindep.txt file maybe but I am fine anyhow. | 13:10 |
| *** tiswanso has joined #openstack-infra | 13:10 | |
| ihrachys | AJaeger: +2d | 13:10 |
| AJaeger | thanks, ihrachys. | 13:10 |
| *** hamzy has quit IRC | 13:10 | |
| *** dargains has joined #openstack-infra | 13:12 | |
| *** spzala has joined #openstack-infra | 13:14 | |
| *** flepied has joined #openstack-infra | 13:17 | |
| *** gouthamr has joined #openstack-infra | 13:18 | |
| yolanda | hi cloudnull ... so problems with osic | 13:19 |
| yolanda | <jhesketh> yolanda: osic-cloud1 is having network issues affecting the mirror | 13:19 |
| yolanda | <jhesketh> because cloud8 uses the same mirror it also experienced the problems | 13:19 |
| yolanda | <odyssey4me> yolanda cloud8 unfortunately uses cloud1's mirror | 13:19 |
| yolanda | <jhesketh> currently cloud1 is offline, I've just +w'd the change to offline cloud8 | 13:19 |
| *** docaedo4 has joined #openstack-infra | 13:19 | |
| yolanda | and also if you can take a look at https://review.openstack.org/#/c/384822 | 13:19 |
| cloudnull | ohai | 13:21 |
| *** ralonsoh has joined #openstack-infra | 13:21 | |
| cloudnull | I can get some data from dhcp services. it'll be a bit though. | 13:21 |
| *** adam_g has joined #openstack-infra | 13:22 | |
| cloudnull | I have to jump though some vpn hoops | 13:22 |
| * cloudnull working from home today | 13:22 | |
| *** adam_g has quit IRC | 13:22 | |
| *** adam_g has joined #openstack-infra | 13:22 | |
| yolanda | ok thanks | 13:22 |
| cloudnull | as for setting option 3 we dont want to disable the gw advertisement on one of the networks. its perfectly acceptable to use both nets on their own. the issue is only when an instance is multi-homed. | 13:24 |
| cloudnull | in liberty it seemed to function out of the box (which is likely a miracle or was caused by some performance issue) however in mitaka it doesn't seem to work without setting the interface post-up or by appending to the dhcpconf file. | 13:25 |
| *** ccamacho|lunch is now known as ccamacho | 13:26 | |
| *** jamesdenton has joined #openstack-infra | 13:26 | |
| rcarrillocruz | Shrews: good to have you around, i actually wanted to ask you if you think i should just log 'description is only for keystone v3' or error out | 13:27 |
| rcarrillocruz | a bit torn | 13:27 |
| rcarrillocruz | tho i lean towards logging it? (agreed on the INFO level btw) | 13:27 |
| Shrews | rcarrillocruz: i think it's fine to accept it, but log it for v2 | 13:29 |
| rcarrillocruz | ++ | 13:29 |
| Shrews | rcarrillocruz: i just added a second comment to clarify one of my statements | 13:30 |
| Shrews | rcarrillocruz: i.e., only log the message IF description is provided | 13:30 |
| *** mriedem has joined #openstack-infra | 13:31 | |
| Shrews | rcarrillocruz: y u no in #openstack-shade? | 13:31 |
| Shrews | :) | 13:31 |
| *** yamamoto has quit IRC | 13:31 | |
| *** yamamoto has joined #openstack-infra | 13:32 | |
| *** amotoki has quit IRC | 13:32 | |
| *** amoralej|lunch is now known as amoralej | 13:33 | |
| rcarrillocruz | hmm, i used to be | 13:33 |
| rcarrillocruz | thanks | 13:33 |
| rcarrillocruz | i have to add it to my weechat conf i guess to survive my bouncer reboots | 13:34 |
| *** EricGonczer_ has joined #openstack-infra | 13:37 | |
| *** claudiub|2 has joined #openstack-infra | 13:38 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 13:38 |
| *** esberglu has joined #openstack-infra | 13:39 | |
| *** dave-mccowan has quit IRC | 13:39 | |
| yolanda | cloudnull, so it will hit some races i guess? depending on the interface that is first configured, this will have the default route | 13:40 |
| cloudnull | it seems that way. | 13:40 |
| *** claudiub has quit IRC | 13:40 | |
| *** Naeil has joined #openstack-infra | 13:40 | |
| *** amotoki has joined #openstack-infra | 13:41 | |
| cloudnull | we're investigating defining a route metric. idk if that'll go. but that'd be easiest for everyone if we can do it server side. | 13:41 |
| cloudnull | any pointers neutron folks? :) | 13:42 |
| *** hichihara has joined #openstack-infra | 13:43 | |
| *** rustlebee is now known as russellb | 13:44 | |
| *** russellb has quit IRC | 13:44 | |
| *** russellb has joined #openstack-infra | 13:44 | |
| openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack-infra/project-config: ceilometer: use new functional jobs https://review.openstack.org/385008 | 13:45 |
| *** hurgleburgler has joined #openstack-infra | 13:45 | |
| *** dprince has quit IRC | 13:47 | |
| *** dprince has joined #openstack-infra | 13:47 | |
| *** panda is now known as panda|afk | 13:48 | |
| *** eharney has joined #openstack-infra | 13:48 | |
| rcarrillocruz | pabelanger: we just had to tweak ports, ram and cores for infracloud quotas, right? | 13:49 |
| rcarrillocruz | (morning!) | 13:49 |
| *** amotoki has quit IRC | 13:50 | |
| *** piet_ has joined #openstack-infra | 13:50 | |
| *** cardeois has joined #openstack-infra | 13:52 | |
| *** dargains has quit IRC | 13:53 | |
| *** adam_g has quit IRC | 13:54 | |
| *** jcoufal_ has joined #openstack-infra | 13:54 | |
| openstackgerrit | Maciej Relewicz proposed openstack-infra/puppet-apps_site: Glare support for app-catalog https://review.openstack.org/359029 | 13:55 |
| openstackgerrit | Merged openstack-infra/project-config: Add linters to the governance gate https://review.openstack.org/381425 | 13:56 |
| openstackgerrit | Maciej Relewicz proposed openstack-infra/puppet-apps_site: Glare support for app-catalog https://review.openstack.org/359029 | 13:56 |
| openstackgerrit | Merged openstack-infra/shade: Update simple_logging to not not log request ids by default https://review.openstack.org/384205 | 13:56 |
| *** dargains has joined #openstack-infra | 13:57 | |
| *** jcoufal has quit IRC | 13:57 | |
| *** moravec has joined #openstack-infra | 13:57 | |
| *** moravec has quit IRC | 13:58 | |
| *** claudiub|2 has quit IRC | 13:58 | |
| *** zxiiro|pto is now known as zxiiro | 13:58 | |
| *** bin_ has joined #openstack-infra | 13:59 | |
| *** pc_m has left #openstack-infra | 13:59 | |
| *** pc_m has joined #openstack-infra | 13:59 | |
| *** breton has left #openstack-infra | 13:59 | |
| *** medberry is now known as med_ | 13:59 | |
| *** yamamoto has quit IRC | 14:00 | |
| *** moravec has joined #openstack-infra | 14:00 | |
| *** yamahata has joined #openstack-infra | 14:01 | |
| *** dave-mccowan has joined #openstack-infra | 14:01 | |
| *** mfedosin has quit IRC | 14:02 | |
| *** amotoki has joined #openstack-infra | 14:02 | |
| *** e0ne has quit IRC | 14:03 | |
| openstackgerrit | Kirill Zaitsev proposed openstack-infra/project-config: Add non-voting npm-test job to murano-dashboard https://review.openstack.org/383007 | 14:03 |
| odyssey4me | Just got this from https://review.openstack.org/380325 - Err 503 http://mirror.mtl01.internap.openstack.org/ubuntu/ | 14:04 |
| *** mrunge is now known as mrunge__ | 14:05 | |
| *** e0ne has joined #openstack-infra | 14:06 | |
| AJaeger | odyssey4me: which log file exactly? | 14:06 |
| *** rodrigods has quit IRC | 14:06 | |
| *** rodrigods has joined #openstack-infra | 14:06 | |
| odyssey4me | AJaeger http://logs.openstack.org/25/380325/14/check/gate-openstack-ansible-openstack-ansible-aio-ubuntu-trusty/f3a8718/console.html#_2016-10-11_13_00_30_380006 | 14:07 |
| dave-mccowan | flaper87 ping | 14:07 |
| *** e0ne has quit IRC | 14:07 | |
| *** rcernin has quit IRC | 14:07 | |
| pabelanger | rcarrillocruz: yes | 14:07 |
| rcarrillocruz | k, i hope to hack on doing os_quota within the launcher this week | 14:08 |
| rcarrillocruz | not full-fledged, but at least the stuff we care about for now | 14:08 |
| AJaeger | odyssey4me: perhaps anybody else has an idea - looks temporary, I can acess everything now. | 14:09 |
| *** kaisers_ has joined #openstack-infra | 14:09 | |
| odyssey4me | AJaeger sure, thought it might be transitional, which is fine | 14:09 |
| odyssey4me | I just wanted to call it out, just in case | 14:09 |
| AJaeger | sure, odyssey4me . thanks | 14:10 |
| frickler | cloudnull: if you cannot disable default route, you can add routes for 0/1 and 128/1 on the net you want to prefer. more specific route will win in that case | 14:10 |
| *** Liuqing has joined #openstack-infra | 14:11 | |
| *** yamamoto has joined #openstack-infra | 14:11 | |
| *** yamamoto has quit IRC | 14:11 | |
| cloudnull | frickler: orly. | 14:12 |
| cloudnull | i'll give that a go and see how it shakes up | 14:12 |
| *** kaisers_ has quit IRC | 14:13 | |
| dave-mccowan | stable-maint-core folks: i need to be added to barbican-stable-maint. it looks like you all have permissions. can you help? https://review.openstack.org/#/admin/groups/1097,members | 14:15 |
| *** amitgandhinz has quit IRC | 14:16 | |
| *** dizquierdo has joined #openstack-infra | 14:16 | |
| *** amitgandhinz has joined #openstack-infra | 14:16 | |
| *** mfedosin has joined #openstack-infra | 14:17 | |
| *** panda|afk is now known as panda | 14:17 | |
| *** mriedem1 has joined #openstack-infra | 14:18 | |
| *** mriedem has quit IRC | 14:19 | |
| *** mriedem1 is now known as mriedem | 14:19 | |
| *** unrahul has quit IRC | 14:20 | |
| *** unrahul has joined #openstack-infra | 14:20 | |
| AJaeger | dave-mccowan: those groups are self-organized - ask one of the existing members to add you. | 14:21 |
| AJaeger | dave-mccowan: stable-maint-core can add you. | 14:22 |
| dave-mccowan | AJaeger yes... looking here for a stable-maint-core to core. flaper87 mriedem ^^ | 14:22 |
| mriedem | huh? | 14:23 |
| mriedem | i'm lacking context | 14:23 |
| dave-mccowan | i need to be added to barbican-stable-maint. it looks like you all have permissions. can you help? https://review.openstack.org/#/admin/groups/1097,members | 14:23 |
| odyssey4me | with https://review.openstack.org/371590 merged, please add me to https://review.openstack.org/#/admin/groups/1595,members so that I can adjust the core groups for the role, thanks | 14:23 |
| AJaeger | dave-mccowan: but this is the wrong channel for stable, go to openstack-stable | 14:23 |
| mriedem | dave-mccowan: yeah let's move to #openstack-stable | 14:24 |
| openstackgerrit | Maciej Relewicz proposed openstack-infra/puppet-apps_site: Glare support for app-catalog https://review.openstack.org/359029 | 14:24 |
| *** dargains has quit IRC | 14:26 | |
| *** eharney has quit IRC | 14:26 | |
| *** dargains has joined #openstack-infra | 14:27 | |
| *** tosky has quit IRC | 14:28 | |
| anteaya | sigmavirus: yesterday was Thanksgiving for me, it looks like clarkb got meetbot working again, thanks for reporting | 14:29 |
| *** zz_dimtruck is now known as dimtruck | 14:29 | |
| *** tosky has joined #openstack-infra | 14:29 | |
| *** hamzy has joined #openstack-infra | 14:30 | |
| openstackgerrit | Daniel Castellanos proposed openstack-infra/project-config: craton-dashboard set to non-voting py35 and JS jobs https://review.openstack.org/384610 | 14:31 |
| *** dargains has quit IRC | 14:32 | |
| sigmavirus | anteaya: ah, sorry about that. Thanks for following up though :) | 14:32 |
| anteaya | ah no worries | 14:32 |
| *** hongbin has joined #openstack-infra | 14:33 | |
| *** cardeois_ has joined #openstack-infra | 14:34 | |
| openstackgerrit | Isaku Yamahata proposed openstack-infra/project-config: networking-odl: use ubuntu-xenial https://review.openstack.org/385036 | 14:34 |
| *** yamamoto has joined #openstack-infra | 14:35 | |
| *** cardeois has quit IRC | 14:36 | |
| *** moravec has quit IRC | 14:38 | |
| *** gordc has quit IRC | 14:39 | |
| *** dimtruck is now known as zz_dimtruck | 14:39 | |
| *** berendt has quit IRC | 14:40 | |
| *** eharney has joined #openstack-infra | 14:40 | |
| openstackgerrit | Isaku Yamahata proposed openstack-infra/project-config: networking-odl: use ubuntu-xenial https://review.openstack.org/385036 | 14:40 |
| openstackgerrit | Isaku Yamahata proposed openstack-infra/project-config: networking-odl: use ubuntu-xenial https://review.openstack.org/385036 | 14:42 |
| *** sdake_ is now known as sdake | 14:42 | |
| *** smcginnis has joined #openstack-infra | 14:42 | |
| *** salv-orl_ has joined #openstack-infra | 14:43 | |
| *** john-davidge has quit IRC | 14:43 | |
| *** john-davidge has joined #openstack-infra | 14:44 | |
| *** armax has joined #openstack-infra | 14:44 | |
| *** salv-orlando has quit IRC | 14:46 | |
| *** rajinir has joined #openstack-infra | 14:46 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/devstack-gate: DNM: test live-migration + multinode + grenade https://review.openstack.org/380840 | 14:46 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/devstack-gate: Set LIVE_MIGRATE_BACK_AND_FORTH to True in LM+grenade job https://review.openstack.org/382456 | 14:46 |
| *** thcipriani|afk is now known as thcipriani | 14:47 | |
| *** john-davidge has quit IRC | 14:48 | |
| *** john-dav_ has joined #openstack-infra | 14:48 | |
| openstackgerrit | Jordan Pittier proposed openstack-infra/shade: Implement create/get/list/delete volume backups https://review.openstack.org/383769 | 14:50 |
| *** annegentle has joined #openstack-infra | 14:51 | |
| *** john-dav_ has quit IRC | 14:52 | |
| *** mfedosin has quit IRC | 14:52 | |
| *** hichihara has quit IRC | 14:53 | |
| *** Liuqing has quit IRC | 14:55 | |
| *** spzala has quit IRC | 14:55 | |
| *** spzala has joined #openstack-infra | 14:56 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/shade: Add description field to create_user method https://review.openstack.org/384404 | 14:56 |
| rcarrillocruz | ttyl, gotta pick my son | 14:56 |
| *** zhurong has quit IRC | 14:56 | |
| *** smcginnis has quit IRC | 14:58 | |
| *** smcginnis has joined #openstack-infra | 14:59 | |
| pleia2 | good morning | 15:00 |
| *** smcginnis has quit IRC | 15:00 | |
| *** spzala has quit IRC | 15:00 | |
| Zara | good morning-evening, pleia2! | 15:00 |
| pleia2 | o/ Zara | 15:01 |
| *** smcginnis has joined #openstack-infra | 15:01 | |
| AJaeger | good morning, pleia2 ! Good evening, Zara ! ;) | 15:01 |
| pleia2 | hey AJaeger | 15:01 |
| Zara | hi! :) (tbf, I'm not sure if 4pm counts as evening, but it didn't feel quite like afternoon) | 15:01 |
| Zara | now I'm overthinking it | 15:02 |
| pleia2 | hehe | 15:02 |
| SotK | feels like evening to me :) | 15:03 |
| AJaeger | Zara, I call it evening for me ;) - 5pm ;) | 15:04 |
| AJaeger | bbl | 15:04 |
| *** adrian_otto has joined #openstack-infra | 15:04 | |
| Zara | :) | 15:04 |
| *** mfedosin has joined #openstack-infra | 15:04 | |
| *** jtomasek is now known as jtomasek|bbl | 15:05 | |
| * pleia2 sips coffee | 15:05 | |
| openstackgerrit | Mike Fedosin proposed openstack-infra/system-config: Create apps-dev.openstack.org for testing community app-catalog https://review.openstack.org/382971 | 15:07 |
| *** makowals has quit IRC | 15:08 | |
| *** smcginnis_ has joined #openstack-infra | 15:08 | |
| *** smcginnis has quit IRC | 15:08 | |
| *** smcginnis_ is now known as smcginnis | 15:08 | |
| *** yamamoto has quit IRC | 15:12 | |
| *** psachin has quit IRC | 15:17 | |
| *** dragonmaster has joined #openstack-infra | 15:19 | |
| *** moravec has joined #openstack-infra | 15:19 | |
| *** panda is now known as panda|afk | 15:20 | |
| *** zz_dimtruck is now known as dimtruck | 15:20 | |
| cloudnull | yolanda: idk if you're still around. but we've added the routes and tested quite a few multi-homed instances and things seem to be working. -cc thanks frickler! for the suggestion :) | 15:23 |
| cloudnull | if we could dhclient on the mirror node, or just reboot it, to test things are working as expected it'd be appreciated. | 15:23 |
| *** jistr is now known as jistr|biab | 15:25 | |
| *** timello has quit IRC | 15:27 | |
| *** karthikp_ has joined #openstack-infra | 15:28 | |
| *** ilyashakhat_mobi has joined #openstack-infra | 15:29 | |
| *** timello has joined #openstack-infra | 15:30 | |
| *** pcaruana has quit IRC | 15:31 | |
| *** lucasagomes is now known as lucas-hungry | 15:32 | |
| *** flepied1 has joined #openstack-infra | 15:32 | |
| *** yamamoto has joined #openstack-infra | 15:33 | |
| *** gordc has joined #openstack-infra | 15:34 | |
| *** flepied has quit IRC | 15:35 | |
| *** claudiub|2 has joined #openstack-infra | 15:39 | |
| *** salv-orl_ has quit IRC | 15:39 | |
| *** vsaienko has quit IRC | 15:41 | |
| *** _vsaienko has joined #openstack-infra | 15:41 | |
| *** moravec has quit IRC | 15:42 | |
| *** moravec has joined #openstack-infra | 15:43 | |
| *** spzala has joined #openstack-infra | 15:45 | |
| ianychoi | pleia2, good morning! Can I briefly discuss on translation infrastructure for Zanata and translation checksite? | 15:46 |
| pleia2 | ianychoi: will you be around in a half hour or so? trying to work through a few things at the moment | 15:46 |
| ianychoi | pleia2, yep call me then thank you! | 15:47 |
| pleia2 | ianychoi: great, thanks | 15:47 |
| ianychoi | :) | 15:47 |
| *** tnovacik has quit IRC | 15:47 | |
| *** tnovacik_ has quit IRC | 15:47 | |
| *** korzen has joined #openstack-infra | 15:48 | |
| *** yankcrime has joined #openstack-infra | 15:50 | |
| openstackgerrit | Michael Johnson proposed openstack-infra/project-config: Add Octavia grafana dashboard https://review.openstack.org/385067 | 15:50 |
| *** rohanion has joined #openstack-infra | 15:51 | |
| ianychoi | AJaeger, I have one question on openstack-dev/sandbox with LP. After the merge: https://review.openstack.org/#/c/375834/ , I thought that openstack-infra will write a comment on https://bugs.launchpad.net/openstack-dev-sandbox/+bug/1631584 after I create a gerrit patch with Closes-Bug tag https://review.openstack.org/#/c/384071/ but strangely no comments were shown.. Could you help how to deep dive this situation? Or trying once more would be | 15:51 |
| ianychoi | better? | 15:51 |
| openstack | Launchpad bug 1631584 in openstack-dev-sandbox "Test with openstack-dev/sandbox integration is needed" [Undecided,New] | 15:51 |
| *** e0ne has joined #openstack-infra | 15:51 | |
| sdake | is openstack-inra suffering any sort of outage atm? | 15:53 |
| sdake | our gating of kolla seems not well | 15:53 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack-infra/project-config: Revert "Disable osic-cloud1" https://review.openstack.org/385070 | 15:53 |
| *** kzaitsev_mb has quit IRC | 15:53 | |
| anteaya | sdake: can you expand with details? | 15:54 |
| sdake | anteaya - Jeffrey4l_ can expand | 15:54 |
| *** links has quit IRC | 15:54 | |
| sdake | anteaya i just woke up :) | 15:54 |
| cloudnull | if infra has a moment to take a look at the mirror server and confirm it's more happy now, it'd be great to get back into the pool | 15:54 |
| anteaya | sdake: um, okay | 15:54 |
| Jeffrey4l_ | seem got lots of network issue today anteaya | 15:54 |
| anteaya | there is no outage of which I am aware | 15:54 |
| anteaya | Jeffrey4l_: can you expand with details? logs are really helpful | 15:54 |
| Jeffrey4l_ | OK. let me find it. 1 sed | 15:55 |
| Jeffrey4l_ | sec | 15:55 |
| *** timello has quit IRC | 15:55 | |
| anteaya | now we are not currently using osic cloud due to network issues | 15:55 |
| anteaya | so if your tests ran on osic cloud might I suggest you recheck them | 15:55 |
| irtermite | anteaya: network issue should have been resolved late last night. What are you seeing now? (cloudnull) | 15:56 |
| cloudnull | irtermite: cloud1/8 is disabled in infra right now due to net issues. | 15:56 |
| anteaya | irtermite: cloudnull are we currently using osic, or no? I skimmed backscroll but might have missed the latest | 15:56 |
| anteaya | cloudnull: ah thank you, yes I thought that was the latest | 15:57 |
| cloudnull | we're not seeing net issues any longer. | 15:57 |
| *** csomerville has quit IRC | 15:57 | |
| anteaya | cloudnull: but our nodepool is currently not using your cloud | 15:57 |
| cloudnull | but we'll see how things behave under load. | 15:57 |
| anteaya | cloudnull: let's wait for a few more core to arrive before we turn your cloud on again? | 15:57 |
| cloudnull | anteaya: correct. we have lots of other consumers on the cloud and from the lack of complaints this morning i believe we're good to go | 15:58 |
| cloudnull | :) | 15:58 |
| cloudnull | but only time will tell | 15:58 |
| *** kaisers_ has joined #openstack-infra | 15:58 | |
| *** acabot has quit IRC | 15:58 | |
| Jeffrey4l_ | anteaya, so before yesterday, there will be lots of network issue, right? | 15:59 |
| *** moravec has quit IRC | 15:59 | |
| *** hashar is now known as hasharAway | 16:01 | |
| *** salv-orlando has joined #openstack-infra | 16:01 | |
| anteaya | cloudnull: okay thanks, you okay if we wait for an infra root or two to be around? | 16:01 |
| *** timello has joined #openstack-infra | 16:01 | |
| cloudnull | sure am | 16:02 |
| cloudnull | im in no rush :) | 16:02 |
| anteaya | Jeffrey4l_: according to the backscroll, osic experienced network issues, I'm not sure when they began and then we turned off osic from nodepool several hours ago | 16:02 |
| *** ihrachys has quit IRC | 16:02 | |
| openstackgerrit | Nate Johnston proposed openstack-infra/project-config: Add grenade configuration for neutron-fwaas https://review.openstack.org/385076 | 16:02 |
| anteaya | cloudnull: great thank you | 16:02 |
| *** kaisers_ has quit IRC | 16:02 | |
| anteaya | cloudnull: do you want to offer a patch to turn your cloud(s) on again? | 16:03 |
| *** cody-somerville has joined #openstack-infra | 16:03 | |
| karthikp_ | sc68cal: Hi | 16:03 |
| *** yamahata has quit IRC | 16:03 | |
| Jeffrey4l_ | anteaya, ok cool. it seem i can not got the network issue in recent logs. will call u when i found it. thanks. | 16:03 |
| anteaya | Jeffrey4l_: well you don't have to call me, you can look in the logs, if they ran on osic, it is a known issue | 16:03 |
| anteaya | and I'm glad your recent logs have different results | 16:04 |
| *** esikachev has quit IRC | 16:04 | |
| Jeffrey4l_ | roger | 16:04 |
| *** tpsilva has quit IRC | 16:04 | |
| ianychoi | anteaya, good morning! Today I am calling several people with different topics.. :) | 16:04 |
| cloudnull | anteaya: https://review.openstack.org/#/c/385070/ | 16:05 |
| anteaya | ianychoi: morning on phone | 16:05 |
| anteaya | cloudnull: thank you | 16:05 |
| ianychoi | anteaya, I have a question on Wiki upgrades plan: https://etherpad.openstack.org/p/newton-infra-wiki-upgrades -> Is it still on-going? When I read http://lists.openstack.org/pipermail/openstack-dev/2016-September/103385.html , wiki is still heavily used. | 16:05 |
| cloudnull | thank you | 16:05 |
| sdake | anteaya sorry for afk -d ropping child at school | 16:05 |
| anteaya | sdake: okay | 16:06 |
| ianychoi | anteaya, I am asking this because of training-guides materials :) | 16:06 |
| *** jistr|biab is now known as jistr | 16:07 | |
| *** vhosakot has joined #openstack-infra | 16:07 | |
| irtermite | already submitted anteaya https://review.openstack.org/#/c/385070/ thanks | 16:07 |
| *** drifterza has quit IRC | 16:07 | |
| *** csomerville has joined #openstack-infra | 16:09 | |
| *** ggillies has quit IRC | 16:09 | |
| *** andreas_s has quit IRC | 16:09 | |
| *** mfedosin has quit IRC | 16:10 | |
| *** ggillies has joined #openstack-infra | 16:11 | |
| clarkb | cloudnull: did we change something on the mirror host to fix routing? | 16:11 |
| *** Naeil has quit IRC | 16:12 | |
| cloudnull | we're hoping that you dont have to | 16:12 |
| *** cody-somerville has quit IRC | 16:12 | |
| cloudnull | dhclient should work, worse case reboot it | 16:12 |
| cloudnull | the node should have a v4 routing table that looks lke this now http://cdn.pasteraw.com/tix9a51lxktmhxfh8qdqf2v2ohlh99e | 16:13 |
| clarkb | cloudnull: I still see default via 10.0.0.1 dev eth1 which is the wrong interface for default route if I raed scrollback properly | 16:13 |
| *** nherciu has quit IRC | 16:13 | |
| clarkb | cloudnull: there is no route for 0.0.0.0/1 | 16:13 |
| *** andrewbogott has quit IRC | 16:13 | |
| *** andrewbogott has joined #openstack-infra | 16:13 | |
| clarkb | also shouldn't it be for a /0? | 16:14 |
| *** cgalan has left #openstack-infra | 16:14 | |
| cloudnull | we added a host route to the gateway_net for 128/1 and 0/1 which should make it prefer that network gw over gateway-net-v6 | 16:15 |
| cloudnull | in tests it seems to be working without any new dhcp changes. | 16:15 |
| *** matrohon has quit IRC | 16:15 | |
| cloudnull | so we're hoping it'll all go without needing to do https://review.openstack.org/#/c/384822/ | 16:15 |
| clarkb | cloudnull: what is setting that route? dhcp? | 16:16 |
| karthikp_ | cd | 16:17 |
| pleia2 | ianychoi: hey, translations things now? | 16:17 |
| ianychoi | pleia2, yep :) | 16:18 |
| pleia2 | ianychoi: regarding zanata, we do have xenial working on long-lived hosts now, so our first step will be bringing up translate-dev on 16.04 with the new version of zanata | 16:18 |
| *** jascott1_ has joined #openstack-infra | 16:18 | |
| *** tnovacik_ has joined #openstack-infra | 16:19 | |
| *** tnovacik has joined #openstack-infra | 16:19 | |
| ianychoi | pleia2, wow...! it is working now? Great news ! | 16:19 |
| pleia2 | ianychoi: it'll probably be after summit though, and then we can work towards getting production upgraded | 16:19 |
| *** rohanion has quit IRC | 16:19 | |
| cloudnull | we defined a host route in neutron . | 16:19 |
| *** ilyashakhat_mobi has quit IRC | 16:19 | |
| *** vgadiraj has joined #openstack-infra | 16:20 | |
| jlvillal | Is there a command I can run that I give the name of a job (from project-config) and I can get the generated bash script from the job? | 16:20 |
| ianychoi | pleia2, Thanks a lot for that effort. I would like to follow up it: from translate-dev to translate.o.o | 16:20 |
| openstackgerrit | Carl Baldwin proposed openstack-infra/irc-meetings: Remove the routed networks meeting https://review.openstack.org/385082 | 16:20 |
| pleia2 | ianychoi: so my goal is for the ocata translations in a few months using the current upstream zanata | 16:20 |
| ianychoi | pleia2, to follow up, I bought your book: "Common OpenStack Deployments"! I need to have time to read puppet chapter later | 16:20 |
| cloudnull | clarkb: defined here http://cdn.pasteraw.com/ehgsj9t2iu1ngowdtwwodlx1e9qm7sc | 16:20 |
| *** HeOS has quit IRC | 16:21 | |
| pleia2 | ianychoi: oh how nice :) will you be at summit? mfisch and I can sign! | 16:21 |
| clarkb | cloudnull: looking at the leases file the renew/expire times are for tomorrow so we won't pick those up if they are set via dhcp without doing an explicit renew | 16:21 |
| ianychoi | pleia2, I also want to translate your book later ): | 16:21 |
| clarkb | cloudnull: I can try that on the mirror in a sec | 16:21 |
| mfisch | yes happy to meet users and discuss puppet ;) | 16:21 |
| pleia2 | ianychoi: we should talk to my editor about that, I'm not sure how that works | 16:21 |
| mfisch | s/users/readers/g | 16:21 |
| ianychoi | pleia2, I will be there but only in Friday afternoon | 16:21 |
| ianychoi | pleia2, your editor? | 16:22 |
| pleia2 | ianychoi: well, publisher | 16:22 |
| ianychoi | mfisch, Hi :) | 16:22 |
| pleia2 | ianychoi: ok, we'll make sure we sync up on Friday afternoon anyway for i18n stuff | 16:22 |
| mfisch | yes hi ;) | 16:22 |
| *** Ahharu has joined #openstack-infra | 16:22 | |
| pleia2 | ianychoi: as far as the translations checksite goes, I think that's still stalled on us not having a great environment to handle the refreshes | 16:22 |
| Ahharu | any issues with zuul and puppet-lint going on? | 16:22 |
| pleia2 | ianychoi: if someone wants to make time to propose a solution that we can evaluate that would be fine, but it's really hard for me to make the time | 16:23 |
| clarkb | of course RTNETLINK answers: file exists | 16:23 |
| ianychoi | pleia2, mfisch Aha yep. I want to translate your book in Korean. If you have time, could you talk with the publisher? | 16:23 |
| zaro | infra-root: i'm not seeing any data on cacti.o.o, are you aware of that? | 16:23 |
| pleia2 | ianychoi: a solution that they will actually implement | 16:23 |
| *** tmorin1 has quit IRC | 16:23 | |
| clarkb | cloudnull: but rerunning dhclient did populate those routes | 16:23 |
| ianychoi | pleia2, Thanks for translation checksite. I agree with Frank's suggestion would make install time shorter | 16:23 |
| pleia2 | ianychoi: I'm having trouble remembering what his latest proposal was, using containers? | 16:24 |
| ianychoi | pleia2, would you see http://lists.openstack.org/pipermail/openstack-i18n/2016-October/002484.html and http://lists.openstack.org/pipermail/openstack-i18n/2016-October/002513.html ? | 16:25 |
| clarkb | and I can hit the mirror from here via ipv4 | 16:25 |
| ianychoi | pleia2, Frank recently proposed using openstack-ansible :) | 16:25 |
| cloudnull | clarkb: nice | 16:25 |
| cloudnull | ianychoi: awesome ! | 16:26 |
| clarkb | cloudnull: did cloud8 get disabled? if so do we have a reenable change for that cloud too? | 16:26 |
| cloudnull | yes | 16:26 |
| clarkb | (sorry suffering from morning brain and lack of caffeine otherwise I would look at that stuff myself) | 16:26 |
| ianychoi | pleia2, hmm referring to https://etherpad.openstack.org/p/openstack-ansible-os_translation-checksite will be better | 16:26 |
| ianychoi | cloudnull, thanks :) | 16:26 |
| cloudnull | i figure once we're up and running w/ cloud1 again we can re-enable assuming the mirror remains stable | 16:26 |
| clarkb | cloudnull: ok | 16:26 |
| pleia2 | ianychoi: oh, I see | 16:27 |
| clarkb | I +2'd the reenable cloud1 change | 16:27 |
| openstackgerrit | Karthik Prabhu Vinod proposed openstack-infra/devstack-gate: Adding other combinations to Cinder multinode grenade https://review.openstack.org/384812 | 16:27 |
| pleia2 | ianychoi: I'll have to take a look at this, where/when did he propose this? | 16:27 |
| clarkb | maybe pleia2 anteaya AJaeger can review it too (based on cores for that projects that seem to be active here | 16:27 |
| *** jpich has quit IRC | 16:27 | |
| *** tongli has joined #openstack-infra | 16:27 | |
| *** morgan has joined #openstack-infra | 16:27 | |
| cloudnull | ianychoi: if i can help on the openstack-ansible front let me know . -cc odyssey4me andymccr | 16:28 |
| openstackgerrit | Carl Baldwin proposed openstack-infra/irc-meetings: Remove the routed networks meeting https://review.openstack.org/385082 | 16:28 |
| ianychoi | pleia2, about one week ago, Frank posted into i18n mailing list: http://lists.openstack.org/pipermail/openstack-i18n/2016-October/002481.html | 16:28 |
| *** chandankumar has quit IRC | 16:28 | |
| ianychoi | cloudnull, wow great thank you. I will tell such kind willing with other i18n members:) | 16:29 |
| clarkb | huh doesn't look like I got a new lease? just new static routes. DHCP is weird | 16:29 |
| pleia2 | ianychoi: thanks, haven't quite caught up with email yet :) (just returned from a conference last night) | 16:29 |
| clarkb | in any case the routes are there now after rerunning dhclient so I am happy | 16:29 |
| cloudnull | clarkb: works in dev... | 16:30 |
| cloudnull | :p | 16:30 |
| *** Swami has joined #openstack-infra | 16:30 | |
| pleia2 | ianychoi: tomorrow is a holiday for me, but I will make time this afternoon to look over these emails and proposal, maybe chat on thursday or friday? | 16:30 |
| ianychoi | pleia2, fine :) Recently, more translators are requesting such functionalities, and also as my role I would like to see current status and discuss with you for better solution :) | 16:30 |
| clarkb | cloudnull: ya the leases file didn't update, but it clearly did something when I ran it... | 16:30 |
| clarkb | cloudnull: that might just be a client oddity too | 16:31 |
| *** dtantsur is now known as dtantsur|afk | 16:31 | |
| pleia2 | ianychoi: good to know, it helps if we know how valuable it is to the community | 16:31 |
| ianychoi | pleia2, Sure! | 16:31 |
| zaro | clarkb, pleia2 : what's up with cacti.o.o? | 16:31 |
| ianychoi | pleia2, I really appreciate your kind help! | 16:31 |
| pleia2 | ianychoi: you're welcome! | 16:31 |
| pleia2 | zaro: let's see... | 16:32 |
| clarkb | zaro: pleia2 it doesn't have enough memory to handle all of the new infracloud hosts. rcarrillocruz was going to rebuild it last I heard | 16:32 |
| ianychoi | pleia2, thank you so much -:) | 16:32 |
| clarkb | (to make a bigger server with more memory) | 16:32 |
| pleia2 | thanks clarkb | 16:32 |
| *** degorenko is now known as degorenko|afk | 16:32 | |
| pleia2 | clarkb: should we remove them for now, and revert when we have the new server? | 16:32 |
| zaro | ok. thanks | 16:32 |
| clarkb | pleia2: I'm not sure. rcarrillocruz would be the best person to ask | 16:33 |
| *** martinkopec has quit IRC | 16:33 | |
| *** yamamoto has quit IRC | 16:33 | |
| zaro | or maybe just 'down for maintenace page'? | 16:33 |
| pleia2 | rcarrillocruz: if you're still about, ETA on new cacti? folks are missing the reporting :) | 16:33 |
| *** jaosorior has quit IRC | 16:33 | |
| *** yamamoto has joined #openstack-infra | 16:34 | |
| *** dizquierdo_ has joined #openstack-infra | 16:34 | |
| *** lucas-hungry is now known as lucasagomes | 16:34 | |
| *** dizquierdo has quit IRC | 16:34 | |
| rcarrillocruz | pleia2, clarkb : the changes are on my queue, i'm waiting for folks to approve and agree a maintenance window to do the volume detach/reattach | 16:35 |
| clarkb | oh /me wasn't aware any changes were needed | 16:35 |
| clarkb | typically its just make new host and edit dns | 16:35 |
| *** Ahharu has quit IRC | 16:36 | |
| pleia2 | rcarrillocruz: have a topic? | 16:37 |
| pleia2 | rcarrillocruz: also, want to add it to the meeting agenda? | 16:37 |
| rcarrillocruz | i can sure | 16:37 |
| pleia2 | (easy enough to chat informally about it, but we do have a meeting in a couple hours anyway) | 16:37 |
| pleia2 | thanks | 16:38 |
| *** baoli_ has quit IRC | 16:38 | |
| *** yamamoto has quit IRC | 16:38 | |
| *** kzaitsev_mb has joined #openstack-infra | 16:39 | |
| *** amotoki has quit IRC | 16:42 | |
| cloudnull | clarkb: sorry was afk for a sec i have some folks at that house. | 16:42 |
| cloudnull | looks like the mirror server is accessable from everywhere | 16:42 |
| cloudnull | which was not the case last night | 16:42 |
| cloudnull | so <fingers crossed> i think we're in the clear | 16:42 |
| *** amitgandhinz has quit IRC | 16:44 | |
| clarkb | cloudnull: ya it looks good to me so I +2'd the change to reenable | 16:45 |
| clarkb | pleia2: rcarrillocruz anteaya AJaeger can probably review too | 16:45 |
| *** rwsu has quit IRC | 16:46 | |
| *** xarses_ has quit IRC | 16:46 | |
| *** derekh has quit IRC | 16:46 | |
| anteaya | done | 16:46 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack-infra/project-config: Revert "Also take osic-cloud8 offline" https://review.openstack.org/385088 | 16:51 |
| cloudnull | that should get cloud8 back online once we have some reps to ensure cloud1 is acting right | 16:52 |
| *** ihrachys has joined #openstack-infra | 16:52 | |
| *** lezbar has quit IRC | 16:52 | |
| *** lezbar has joined #openstack-infra | 16:53 | |
| openstackgerrit | Jordan Pittier proposed openstack-infra/shade: Implement create/get/list/delete volume backups https://review.openstack.org/383769 | 16:53 |
| anteaya | cloudnull: thank you | 16:56 |
| *** lezbar has quit IRC | 16:56 | |
| zaro | asselin: have a question about https://git.openstack.org/cgit/openstack-infra/puppet-openstackci/tree/manifests/single_node_ci.pp | 16:59 |
| *** moravec has joined #openstack-infra | 17:00 | |
| *** tphummel has joined #openstack-infra | 17:00 | |
| asselin | zaro, sure what's your question? | 17:00 |
| zaro | asselin: was wondering if there's a puppet module that uses single_node_ci.pp somewhere? | 17:00 |
| asselin | zaro, not that I know of | 17:00 |
| zaro | so do you know whether it's up to date? | 17:01 |
| zaro | i'm interested in using it to setup a zuul-dev.o.o | 17:01 |
| zaro | without jenkins | 17:01 |
| asselin | zaro, it should be up-to-date. | 17:02 |
| *** eranrom has quit IRC | 17:02 | |
| zaro | i'm wondering wether it makes sense to refactor to decouple jenkins from that | 17:02 |
| asselin | zaro, only after zuulv3 is released.... | 17:02 |
| zaro | have a single_node_jenkins.pp and single_node_zuul.pp extend from single_node_base.pp | 17:03 |
| zaro | ? | 17:03 |
| zaro | not for 2.5? | 17:03 |
| asselin | zaro, no, infra said they don't want to support non-jenkins until zuulV3 | 17:03 |
| asselin | or rather, they don't want to support zuulv2.5 for non-infra | 17:03 |
| zaro | clarkb: ^ can you comment on that? what would be the best course for setting up zuul-dev.o.o? | 17:04 |
| *** tongli has quit IRC | 17:04 | |
| zaro | using puppet-openstackci or just going with custom module? | 17:05 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/gear: WIP: Handle new STATUS commands from protocol https://review.openstack.org/382043 | 17:05 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/gear: Make geard coalesce identical uniques https://review.openstack.org/385092 | 17:05 |
| asselin | zaro, you can use puppet-openstackci, but just not the single-node-ci. but it's simple enough to compose the pieces you need | 17:05 |
| zaro | asselin: make something similar to single-node-ci composed without jenkins? | 17:06 |
| clarkb | zaro: I would use the same pieces we already use for eg zuul | 17:07 |
| zaro | asselin: should that be checked in? | 17:07 |
| *** tnovacik_ has quit IRC | 17:07 | |
| clarkb | I think we use bits of it just not the default all in one like asselin says | 17:07 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/gear: Make geard coalesce identical uniques https://review.openstack.org/385092 | 17:07 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/gear: WIP: Handle new STATUS commands from protocol https://review.openstack.org/382043 | 17:07 |
| *** tnovacik has quit IRC | 17:07 | |
| *** xarses_ has joined #openstack-infra | 17:07 | |
| *** dizquierdo_ has quit IRC | 17:08 | |
| zaro | asselin: would you be able to point me to a project that uses puppet-openstackci? | 17:08 |
| asselin | zaro, yes, there is a plan to update single-node-ci to not have jenkins, but after zuulv3 is out. You can propose the patch to add it now, but I'll defer to other cores as to when that would get merged. | 17:09 |
| *** ihrachys has quit IRC | 17:10 | |
| *** sambetts is now known as sambetts|afk | 17:10 | |
| asselin | zaro, system-config uses it, and this page for setting up 3rd party ci references it: http://docs.openstack.org/infra/openstackci/ | 17:10 |
| *** tqtran has joined #openstack-infra | 17:11 | |
| *** esikachev has joined #openstack-infra | 17:11 | |
| zaro | asselin: well that change needs to be put somewhere for puppet to use it. i can't think of a better place besides puppet-openstackci | 17:11 |
| *** mhickey has quit IRC | 17:11 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources: Summit Event tags duplication https://review.openstack.org/385095 | 17:11 |
| *** ihrachys has joined #openstack-infra | 17:13 | |
| *** cgalan_ has joined #openstack-infra | 17:14 | |
| zaro | ahh yeah of course, system-config. thanks. | 17:14 |
| AJaeger | ianychoi: no idea about the sandbox bug problem, this looks strange indeed. But we had puppet disabled on gerrit for a few days, could you retry again *today*, please? | 17:14 |
| ianychoi | AJaeger, oh thanks for such information on puppet :) | 17:14 |
| ianychoi | AJaeger, I will try it | 17:14 |
| openstackgerrit | Merged openstack-infra/openstackid-resources: Summit Event tags duplication https://review.openstack.org/385095 | 17:15 |
| *** ricardoas has quit IRC | 17:15 | |
| *** clenimar has quit IRC | 17:16 | |
| *** mtanino has joined #openstack-infra | 17:16 | |
| *** mrtenio has quit IRC | 17:16 | |
| AJaeger | clarkb, pabelanger : Could you review a glean change for SUSE networking, please? https://review.openstack.org/378532 | 17:17 |
| asselin | zaro, so this class isn't using puppet-openstackci yet: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/zuul_launcher.pp I don't know if it makes sense to add that component now or wait for zuulv3. | 17:17 |
| *** cgalan_ has quit IRC | 17:17 | |
| jesusaur | asselin: zaro: what about adding it to openstackci, but with a warning() that it's not yet supported? | 17:18 |
| AJaeger | pleia2: Sorry, I won't be able to join the IRC meeting today. | 17:18 |
| *** amitgandhinz has joined #openstack-infra | 17:18 | |
| pleia2 | AJaeger: no problem, anything you'd like me to mention? | 17:19 |
| *** cgalan_ has joined #openstack-infra | 17:20 | |
| zaro | asselin: ok, looks i can just reuse modules from puppet-openstack without having to actually create a new module in there. i'm good. | 17:20 |
| AJaeger | pleia2: looking at the agenda: nothing to mention besides Newton is out ;) | 17:20 |
| pleia2 | AJaeger: hah, perhaps that's an announcement :) | 17:20 |
| zaro | asselin: it looks like nodepool.o.o in site.pp calls zookeeper now. i'm wondering if that's needed for just single nodepool? | 17:21 |
| AJaeger | I doubt anybody missed it ;) | 17:21 |
| pleia2 | "we survived another release" | 17:21 |
| * AJaeger is not feeling well and signs off now... | 17:21 | |
| pleia2 | AJaeger: feel better! | 17:21 |
| zaro | asselin: zookeeper is not in single_node_ci.pp | 17:21 |
| AJaeger | thanks, pleia2 ! Has nothing to do with the release;) | 17:21 |
| pleia2 | lol | 17:21 |
| anteaya | AJaeger: take care of yourself | 17:22 |
| anteaya | thanks for your help today | 17:22 |
| *** ihrachys has quit IRC | 17:22 | |
| asselin | jesusaur, it's a good question. My doubts are mostly related to how much will change b/w zuul v2.5 and v3 and the cost/benefit of doing it now vs later. | 17:22 |
| asselin | zaro, | 17:22 |
| *** cgalan_ has quit IRC | 17:23 | |
| zaro | asselin: are you referring to zookeeper? | 17:23 |
| asselin | zaro, I don't know what is zookeeper | 17:24 |
| *** esikachev has quit IRC | 17:24 | |
| *** dizquierdo_ has joined #openstack-infra | 17:25 | |
| *** dizquierdo_ has quit IRC | 17:25 | |
| *** clenimar has joined #openstack-infra | 17:25 | |
| *** dizquierdo has joined #openstack-infra | 17:25 | |
| *** AnarchyAo has joined #openstack-infra | 17:26 | |
| *** ihrachys has joined #openstack-infra | 17:26 | |
| zaro | asselin: it's used by nodepool, http://git.openstack.org/cgit/openstack-infra/system-config/tree/manifests/site.pp#n800 | 17:27 |
| *** mrtenio has joined #openstack-infra | 17:27 | |
| zaro | asselin: i'm pretty sure it's not need for single nodepool though. | 17:27 |
| asselin | zaro, yeah, seems not related | 17:28 |
| *** ralonsoh has quit IRC | 17:29 | |
| *** ricardoas has joined #openstack-infra | 17:29 | |
| *** tosky has quit IRC | 17:29 | |
| asselin | zaro, commit message doesn't say why to add it: https://review.openstack.org/#/c/324037/ | 17:30 |
| asselin | "Include will be friendlier to people once we start requiring this for both zuul and nodepool." | 17:31 |
| *** mkoderer has quit IRC | 17:32 | |
| *** amitgandhinz has quit IRC | 17:32 | |
| *** baoli has joined #openstack-infra | 17:33 | |
| *** amitgandhinz has joined #openstack-infra | 17:33 | |
| *** trown is now known as trown|lunch | 17:34 | |
| *** yamamoto has joined #openstack-infra | 17:36 | |
| *** tobiash has joined #openstack-infra | 17:37 | |
| *** tobiash has left #openstack-infra | 17:38 | |
| *** shardy has quit IRC | 17:38 | |
| *** pblaho has quit IRC | 17:39 | |
| *** ccamacho has left #openstack-infra | 17:39 | |
| *** tpsilva has joined #openstack-infra | 17:39 | |
| *** yamamoto has quit IRC | 17:41 | |
| openstackgerrit | matthew wagoner proposed openstack-infra/shade: Add test for os_project Ansible module https://review.openstack.org/384634 | 17:41 |
| *** esberglu has quit IRC | 17:41 | |
| *** amotoki has joined #openstack-infra | 17:42 | |
| *** esberglu_ has joined #openstack-infra | 17:46 | |
| clarkb | cloudnull: osic-cloud1 reenablement should merge shortly then we will have to wait for puppet, I will try to keep an eye on things once it is back in | 17:47 |
| *** amotoki has quit IRC | 17:47 | |
| cloudnull | awesome | 17:47 |
| cloudnull | i should be around so if it catches fire I can make smores. :) | 17:47 |
| openstackgerrit | Merged openstack-infra/project-config: Revert "Disable osic-cloud1" https://review.openstack.org/385070 | 17:48 |
| cloudnull | hold on to your butts. | 17:48 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources: Fix on IOC binding https://review.openstack.org/385109 | 17:49 |
| *** jpena is now known as jpena|off | 17:49 | |
| *** esberglu_ has left #openstack-infra | 17:50 | |
| openstackgerrit | Merged openstack-infra/openstackid-resources: Fix on IOC binding https://review.openstack.org/385109 | 17:50 |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Add new project hurricane https://review.openstack.org/385110 | 17:51 |
| *** esberglu_ has joined #openstack-infra | 17:51 | |
| *** esberglu_ has quit IRC | 17:52 | |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Add new project hurricane https://review.openstack.org/385110 | 17:52 |
| *** ihrachys has quit IRC | 17:54 | |
| *** amoralej is now known as amoralej|off | 17:59 | |
| *** jordanP has quit IRC | 17:59 | |
| *** pblaho has joined #openstack-infra | 18:01 | |
| *** nadya has quit IRC | 18:01 | |
| *** jkilpatr has quit IRC | 18:02 | |
| openstackgerrit | Ramy Asselin proposed openstack-infra/elastic-recheck: Add initial bug tracker for StoryBoard https://review.openstack.org/385112 | 18:02 |
| *** abregman|afk has quit IRC | 18:04 | |
| clarkb | cloudnull: ok bunch of instances building in cloud1 now, should start seeing them in use soon | 18:06 |
| *** korzen has quit IRC | 18:07 | |
| *** eranrom has joined #openstack-infra | 18:09 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Move qemu-img installation after repo setup https://review.openstack.org/384628 | 18:09 |
| *** tiswanso has quit IRC | 18:09 | |
| *** jamielennox|away is now known as jamielennox | 18:10 | |
| *** dizquierdo has quit IRC | 18:12 | |
| *** rbrndt has quit IRC | 18:12 | |
| *** drawsmcgraw has quit IRC | 18:12 | |
| openstackgerrit | Zara proposed openstack-infra/storyboard-webclient: Update Angular from 1.3.13 to 1.5.8 https://review.openstack.org/384532 | 18:13 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Try temprevert of keystone domain change on mitaka https://review.openstack.org/385114 | 18:13 |
| openstackgerrit | Zara proposed openstack-infra/storyboard-webclient: Update Bootstrap https://review.openstack.org/384179 | 18:14 |
| *** rbrndt has joined #openstack-infra | 18:14 | |
| *** rbrndt has quit IRC | 18:14 | |
| openstackgerrit | Zara proposed openstack-infra/storyboard-webclient: Update UI-Router https://review.openstack.org/384554 | 18:15 |
| *** jkilpatr has joined #openstack-infra | 18:15 | |
| *** ihrachys has joined #openstack-infra | 18:17 | |
| clarkb | cloudnull: I have a tail | grep running on zuul launcher logs for jobs finishing on osic. Will keep my eyes open for any failures | 18:18 |
| clarkb | cloudnull: so far just successes | 18:18 |
| *** salv-orlando has quit IRC | 18:19 | |
| SpamapS | To whom it may concern.. I'm trying to update gear so that it works with the latest gearman and so that it can be used for coalescing. Please give https://review.openstack.org/#/q/topic:thundering-herd a review if you can find some time to do so. Thanks! | 18:19 |
| *** claudiub has joined #openstack-infra | 18:20 | |
| *** claudiub|2 has quit IRC | 18:21 | |
| *** ihrachys has quit IRC | 18:22 | |
| cloudnull | woot! | 18:24 |
| *** chandankumar has joined #openstack-infra | 18:24 | |
| cloudnull | sorry was eating | 18:24 |
| *** cardeois_ has quit IRC | 18:24 | |
| cloudnull | clarkb: looks like a fairly high node build failure rate. | 18:24 |
| clarkb | cloudnull: I think because it thundering herded | 18:25 |
| cloudnull | maybe related to the thunderous herd | 18:25 |
| clarkb | cloudnull: ya | 18:25 |
| cloudnull | ya. | 18:25 |
| clarkb | cloudnull: lets see if that settles out and only worry about it if it persists | 18:25 |
| cloudnull | but it seems to have leveled off. | 18:25 |
| *** rossella_s has quit IRC | 18:27 | |
| *** rossella_s has joined #openstack-infra | 18:28 | |
| *** tiswanso has joined #openstack-infra | 18:29 | |
| *** tiswanso has quit IRC | 18:29 | |
| *** tiswanso has joined #openstack-infra | 18:29 | |
| anteaya | yay success | 18:30 |
| clarkb | ya still haven't seen any fails so I think we are good | 18:31 |
| clarkb | I am going to review the cloud8 change now | 18:31 |
| EmilienM | pabelanger: is there a mirror where I can download https://download.ceph.com/keys/release.asc ? | 18:31 |
| EmilienM | ceph.com has been down for more than 5 hours | 18:31 |
| clarkb | EmilienM: we vendor it in project-config or did | 18:32 |
| clarkb | oh sorry system-config for building our mirror | 18:32 |
| clarkb | cloudnull: you ok with me approving the cloud8 revert too? | 18:32 |
| clarkb | EmilienM: https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/files/reprepro/ceph-mirror-gpg-key.asc | 18:33 |
| EmilienM | clarkb: nice | 18:34 |
| EmilienM | clarkb: ++ | 18:34 |
| EmilienM | clarkb: so the file is not in the repo per-se? | 18:34 |
| clarkb | EmilienM: not with reprepro because it builds a mirror from scratch with its own index | 18:34 |
| clarkb | EmilienM: if you do a 1:1 mirror then that should be copied over | 18:35 |
| openstackgerrit | Michael Johnson proposed openstack-infra/project-config: Add Octavia grafana dashboard https://review.openstack.org/385067 | 18:35 |
| EmilienM | clarkb: I found it :) http://mirror.iad.rax.openstack.org/ceph-deb-jewel/lists/debian-ceph-jewel_xenial_Release.gpg | 18:36 |
| *** azvyagintsev_h has quit IRC | 18:36 | |
| EmilienM | ah no it's not the right file | 18:37 |
| clarkb | EmilienM: correct reprepro doesn't produce a 1:1 mirror. It ahs its own indexes so different keys | 18:37 |
| clarkb | EmilienM: we carry the key file in system-config so that we can instruct reprepro to trust that as an upstream but for the downstream side it isn't used | 18:38 |
| EmilienM | ok | 18:38 |
| clarkb | cloudnull: caught a fail but not related to the pypi mirror | 18:39 |
| cloudnull | clarkb: i think we're good to reenable cloud8 | 18:39 |
| clarkb | cloudnull: oh looks like we also have e-r rules for the fails http://status.openstack.org/elastic-recheck/ so we can just watch that for spikes | 18:39 |
| *** cardeois has joined #openstack-infra | 18:39 | |
| clarkb | cloudnull: done | 18:39 |
| cloudnull | cool I had no idea we had e-r rules for osic in place. | 18:40 |
| clarkb | cloudnull: not for osic specifically but for the pypi fails | 18:40 |
| cloudnull | ah, ok. that makes sense . | 18:41 |
| clarkb | the first three bugs/graphs there are for errors that happen when mirrors don't function | 18:41 |
| *** rbrndt has joined #openstack-infra | 18:42 | |
| *** amotoki has joined #openstack-infra | 18:43 | |
| *** kzaitsev_mb has quit IRC | 18:44 | |
| *** adam_g has joined #openstack-infra | 18:45 | |
| clarkb | cloudnull: thank you for all the help with that yesterday. I had to work on other stuff and also take advantage of first non rainy days in a while to do much needed house things | 18:45 |
| clarkb | jhesketh: you too | 18:45 |
| clarkb | (I had already spetn one afternoon getting completely drenched and do not recommend) | 18:45 |
| *** trown|lunch is now known as trown | 18:46 | |
| cloudnull | clarkb: thank you, and everyone else, for all the patience. | 18:46 |
| openstackgerrit | Merged openstack-infra/project-config: Revert "Also take osic-cloud8 offline" https://review.openstack.org/385088 | 18:47 |
| cloudnull | to be fair jamesdenton bjorneT did a lot of our OSIC upgrade lifting yesterday so most of the credit goes to them. | 18:47 |
| *** amotoki has quit IRC | 18:47 | |
| cloudnull | anyway happy that things are humming a long again, sorry for all of the interuptions | 18:48 |
| *** thorst has quit IRC | 18:48 | |
| *** chandankumar has quit IRC | 18:53 | |
| irtermite | what cloudnull said | 18:53 |
| *** pvaneck has joined #openstack-infra | 18:56 | |
| *** hasharAway is now known as hashar | 18:57 | |
| jhesketh | Morning | 18:58 |
| anteaya | morning jhesketh | 18:59 |
| clarkb | jhesketh: good morning. To fill you in the route hack for osic appears to have fixed the mirror (I had to manually run dhclient for the host to pick them up though as lease time is fairly long). And osic is back in action | 18:59 |
| cloudnull | mornings jhesketh | 18:59 |
| pleia2 | meeting time :) | 19:00 |
| jhesketh | clarkb: awesome, good to know :-) | 19:00 |
| *** karthikp has joined #openstack-infra | 19:00 | |
| anteaya | cloudnull: thanks for staying in contact, thank you jamesdenton and bjorneT for the osic upgrade | 19:00 |
| cloudnull | next stop newton :) | 19:00 |
| * cloudnull is joking | 19:01 | |
| clarkb | aw I was getting excited | 19:01 |
| cloudnull | we'll let it run for a couple of days before causing more pain | 19:02 |
| cloudnull | :p | 19:02 |
| cloudnull | we enabled a few new services this go too so it'll be interesting to see how those are used and how they hold up. | 19:03 |
| anteaya | cloudnull: what release are you on? | 19:03 |
| cloudnull | head of the stable/mitaka branch | 19:03 |
| anteaya | nice | 19:03 |
| *** amitgandhinz has quit IRC | 19:06 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool: Add new build number lock context manager https://review.openstack.org/385126 | 19:06 |
| *** nadya has joined #openstack-infra | 19:06 | |
| *** amitgandhinz has joined #openstack-infra | 19:07 | |
| *** nadya has quit IRC | 19:08 | |
| *** nadya has joined #openstack-infra | 19:08 | |
| *** spzala has quit IRC | 19:09 | |
| jhesketh | clarkb: did you manually apply the patch, or was there another fix? | 19:09 |
| *** spzala has joined #openstack-infra | 19:09 | |
| jhesketh | (Since https://review.openstack.org/#/c/384822/ hasn't merged) | 19:09 |
| clarkb | jhesketh: cloudnull et al applied a change to the neutron configs in the cloud itself whcih provides static routes via dhcp | 19:10 |
| *** Rockyg has joined #openstack-infra | 19:10 | |
| clarkb | jhesketh: so I didn't apply anything other than to kick dhclient so that the new data would be picked up from the dhcp server | 19:10 |
| jhesketh | Ah I see. Cool! | 19:10 |
| cloudnull | jhesketh: /me was tired last night sorry or I would've done that sooner. | 19:11 |
| jhesketh | I shall abandon that one then | 19:11 |
| jhesketh | cloudnull: all good, and thanks again for staying around so late and helping | 19:11 |
| *** berendt has joined #openstack-infra | 19:11 | |
| cloudnull | it all made sense after a few hours of sleep ... :) | 19:11 |
| cloudnull | The more you know! | 19:12 |
| jhesketh | :-) | 19:12 |
| *** thorst has joined #openstack-infra | 19:13 | |
| clarkb | I wonder if there is some other way to hint to linux that "this itnerface is more better for default route" | 19:14 |
| *** spzala has quit IRC | 19:14 | |
| clarkb | other than having explicit routes | 19:14 |
| clarkb | I would have to read through lots of man pages likely | 19:14 |
| anteaya | the more better config option | 19:14 |
| *** xarses_ is now known as xarses | 19:14 | |
| cloudnull | clarkb: we can set metrics which will give the routes weight | 19:16 |
| cloudnull | but i didn't find anything in neutron specifically to expose that | 19:16 |
| cloudnull | frickler had the idea of setting 0/1 and 128/1 on a given net that we want to prefer to essentially give us a "prefered" network | 19:17 |
| *** mdrabe has quit IRC | 19:17 | |
| *** mdrabe has joined #openstack-infra | 19:18 | |
| cloudnull | and that was how we finally made it go using host-routes in the GATEWAY_NET subnet | 19:18 |
| openstackgerrit | matthew wagoner proposed openstack-infra/shade: Add test for os_project Ansible module https://review.openstack.org/384634 | 19:20 |
| anteaya | EmilienM: you didn't get any turkey on the weekend? | 19:23 |
| anteaya | :( | 19:23 |
| *** dragonmaster has quit IRC | 19:23 | |
| *** dragonmaster has joined #openstack-infra | 19:24 | |
| bkero | Is turkey a thing for Canadian thanksgiving? | 19:24 |
| EmilienM | anteaya: I haven't! we did hiking all day in middle of nowhere | 19:24 |
| *** dragonmaster has quit IRC | 19:24 | |
| anteaya | bkero: yes | 19:24 |
| bkero | Cool | 19:24 |
| anteaya | EmilienM: oh hiking is nice | 19:24 |
| *** dragonmaster has joined #openstack-infra | 19:24 | |
| EmilienM | I'm a bad future canadian-citizen :( | 19:25 |
| anteaya | EmilienM: where did you go? | 19:25 |
| anteaya | EmilienM: for hiking? | 19:25 |
| *** moravec has quit IRC | 19:25 | |
| *** dragonmaster has quit IRC | 19:25 | |
| anteaya | EmilienM: I'm surprised immagration isn't chasing you to approve you for citizenship | 19:25 |
| *** dragonmaster has joined #openstack-infra | 19:25 | |
| EmilienM | anteaya: https://goo.gl/maps/1wDFMxRuwFE2 | 19:25 |
| *** Sukhdev has joined #openstack-infra | 19:25 | |
| *** dragonmaster has quit IRC | 19:26 | |
| *** dragonmaster has joined #openstack-infra | 19:26 | |
| anteaya | nice | 19:26 |
| anteaya | did you have fun? | 19:26 |
| *** dragonmaster has quit IRC | 19:26 | |
| EmilienM | anteaya: I always have fun here :) | 19:27 |
| *** dragonmaster has joined #openstack-infra | 19:27 | |
| anteaya | EmilienM: awesome, that looks like a great park | 19:27 |
| *** dragonmaster has quit IRC | 19:27 | |
| EmilienM | anteaya: I never did the same park since i've been there | 19:28 |
| *** dragonmaster has joined #openstack-infra | 19:28 | |
| *** dragonmaster has quit IRC | 19:28 | |
| anteaya | the park looks beautiful | 19:28 |
| openstackgerrit | Karthik Prabhu Vinod proposed openstack-infra/project-config: Adding two other combinations of cinder multinode grenade job https://review.openstack.org/384836 | 19:31 |
| * clarkb is going to have to -2 all multinode changes until neutron works on multinode :( | 19:32 | |
| clarkb | maybe I should email the list | 19:32 |
| *** asselin__ has quit IRC | 19:32 | |
| anteaya | good idea | 19:32 |
| *** _vsaienko has quit IRC | 19:32 | |
| *** _vsaienko has joined #openstack-infra | 19:34 | |
| *** kaisers_ has joined #openstack-infra | 19:35 | |
| *** moravec has joined #openstack-infra | 19:35 | |
| *** e0ne has quit IRC | 19:37 | |
| openstackgerrit | Chuck Short proposed openstack-infra/devstack-gate: Add stack to lxd group when using LXD https://review.openstack.org/385131 | 19:37 |
| clarkb | zul: why not do that in the lxd devstack plugin? | 19:37 |
| *** spzala has joined #openstack-infra | 19:38 | |
| zul | clarkb: yeah probably better...i was treating it the same as openvz | 19:38 |
| cloudnull | we gate on lxd within our os_nova role works really well. | 19:38 |
| clarkb | zul: ya I think there is probably a bunch of stuff like that that we can clean out of devstack-gate actually | 19:38 |
| *** askb has joined #openstack-infra | 19:39 | |
| cloudnull | sorry total squirl moment | 19:39 |
| zul | clarkb: ok | 19:39 |
| clarkb | zul: since we don't test openvz at all anywhere and we are trying to do more with devstack via plugins and less in devstack-gate | 19:39 |
| zul | clarkb: gotcha ill add it there | 19:39 |
| clarkb | zul: general rule of thumb is "if it is required for devstack to function on its own then ti should be in devstack" devstack-gate should be more "automate running of devstack in test env" | 19:40 |
| cloudnull | ok im going a away for a while. bbl. frickler clarkb yolanda anteaya jhesketh .*others-ive-forgotten.* thanks for help today. much appreciated. | 19:40 |
| anteaya | cloudnull: thank you | 19:40 |
| zul | clarkb: ack ok thanks | 19:40 |
| *** kaisers_ has quit IRC | 19:40 | |
| jhesketh | cloudnull: I owe you beer ;-) | 19:41 |
| *** e0ne has joined #openstack-infra | 19:41 | |
| *** Sukhdev has quit IRC | 19:41 | |
| *** salv-orlando has joined #openstack-infra | 19:42 | |
| *** mriedem has quit IRC | 19:43 | |
| *** moravec has quit IRC | 19:43 | |
| zaro | rcarrillocruz: what's the difference between vanilla and chocolate? | 19:44 |
| rcarrillocruz | we logically splitted in flavors | 19:44 |
| *** yamahata has joined #openstack-infra | 19:44 | |
| rcarrillocruz | but they're located in the same datacenter | 19:44 |
| rcarrillocruz | the idea is to have different openstack versions on each | 19:44 |
| rcarrillocruz | so apart from differing a bit from HW perspective, nothing much | 19:44 |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Add new project hurricane https://review.openstack.org/385110 | 19:45 |
| zaro | clarkb: which host should we use for nodepool on zuul-dev? infracloud? | 19:46 |
| *** bin_ has quit IRC | 19:47 | |
| clarkb | zaro: you mean where to run the nodepool service or what cloud to talk to? | 19:47 |
| zaro | which cloud to talk to? | 19:47 |
| clarkb | zaro: I think we don't really need the nodepool to test new gerrit, can just set a static host and boot it in whatever cloud (probably osic because lots of capacity) | 19:47 |
| anteaya | pleia2: can you also do a topic update in meeting-4? | 19:47 |
| clarkb | but if we want to talk to a cloud I would say osic due to the capacity there | 19:47 |
| *** moravec has joined #openstack-infra | 19:48 | |
| pleia2 | anteaya: sure | 19:48 |
| anteaya | thanks | 19:48 |
| *** dprince has quit IRC | 19:48 | |
| *** dave-mccowan has quit IRC | 19:48 | |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Add new project hurricane https://review.openstack.org/385110 | 19:49 |
| *** esikachev has joined #openstack-infra | 19:50 | |
| *** e0ne has quit IRC | 19:50 | |
| *** amitgandhinz has quit IRC | 19:50 | |
| *** amitgandhinz has joined #openstack-infra | 19:51 | |
| *** dimtruck is now known as zz_dimtruck | 19:51 | |
| *** nadya has quit IRC | 19:52 | |
| anteaya | pleia2: thank you | 19:52 |
| *** e0ne has joined #openstack-infra | 19:55 | |
| *** annegentle has quit IRC | 19:55 | |
| *** john-davidge has joined #openstack-infra | 19:56 | |
| *** annegentle has joined #openstack-infra | 19:56 | |
| *** e0ne has quit IRC | 19:58 | |
| *** Goneri has quit IRC | 19:58 | |
| *** john-davidge has quit IRC | 20:01 | |
| *** e0ne has joined #openstack-infra | 20:02 | |
| *** exploreshaifali has joined #openstack-infra | 20:02 | |
| *** _vsaienko has quit IRC | 20:06 | |
| *** moravec has quit IRC | 20:18 | |
| *** gyee has joined #openstack-infra | 20:19 | |
| *** pcrews has quit IRC | 20:20 | |
| *** lucasagomes is now known as lucas-afk | 20:21 | |
| *** adrian_otto has quit IRC | 20:22 | |
| *** smcginnis_ has joined #openstack-infra | 20:23 | |
| *** kgiusti has left #openstack-infra | 20:25 | |
| *** kzaitsev_mb has joined #openstack-infra | 20:25 | |
| *** cardeois_ has joined #openstack-infra | 20:27 | |
| *** cardeois has quit IRC | 20:27 | |
| *** hamzy has quit IRC | 20:28 | |
| *** inc0 has joined #openstack-infra | 20:28 | |
| *** denisra_ has joined #openstack-infra | 20:29 | |
| *** denisra_ has quit IRC | 20:29 | |
| *** denisra_ has joined #openstack-infra | 20:29 | |
| *** burgerk__ has joined #openstack-infra | 20:30 | |
| openstackgerrit | John L. Villalovos proposed openstack-infra/zuul: Add requirement for jenkins-job-builder https://review.openstack.org/385142 | 20:31 |
| *** denisra has quit IRC | 20:33 | |
| *** panda|afk is now known as panda | 20:33 | |
| *** dtardivel has quit IRC | 20:34 | |
| *** burgerk_ has quit IRC | 20:34 | |
| *** pcrews has joined #openstack-infra | 20:38 | |
| *** sdake_ has joined #openstack-infra | 20:39 | |
| *** sdake has quit IRC | 20:41 | |
| *** esikachev has quit IRC | 20:41 | |
| *** salv-orl_ has joined #openstack-infra | 20:43 | |
| *** adrian_otto has joined #openstack-infra | 20:44 | |
| *** eranrom has quit IRC | 20:44 | |
| *** amotoki has joined #openstack-infra | 20:45 | |
| *** salv-orlando has quit IRC | 20:46 | |
| *** inc0 has quit IRC | 20:47 | |
| *** burgerk__ has quit IRC | 20:48 | |
| *** jkilpatr has quit IRC | 20:50 | |
| *** amotoki has quit IRC | 20:50 | |
| *** trown is now known as trown|outtypewww | 20:51 | |
| *** Hal has quit IRC | 20:51 | |
| *** sdake has joined #openstack-infra | 20:51 | |
| *** jheroux has quit IRC | 20:51 | |
| *** nadya has joined #openstack-infra | 20:52 | |
| *** sdake_ has quit IRC | 20:53 | |
| *** gibi has quit IRC | 20:56 | |
| *** jcoufal_ has quit IRC | 20:56 | |
| *** rtheis has quit IRC | 20:56 | |
| *** mriedem has joined #openstack-infra | 20:57 | |
| *** nadya has quit IRC | 20:57 | |
| *** _vsaienko has joined #openstack-infra | 21:01 | |
| *** amitgandhinz has quit IRC | 21:01 | |
| *** karthikp_ has quit IRC | 21:01 | |
| *** jordanP has joined #openstack-infra | 21:01 | |
| *** exploreshaifali has quit IRC | 21:09 | |
| *** jkilpatr has joined #openstack-infra | 21:10 | |
| *** Hal has joined #openstack-infra | 21:11 | |
| *** hashar is now known as hasharAway | 21:11 | |
| *** Sukhdev has joined #openstack-infra | 21:15 | |
| *** rlandy has quit IRC | 21:17 | |
| *** armax has quit IRC | 21:18 | |
| *** salv-orl_ has quit IRC | 21:18 | |
| *** moravec has joined #openstack-infra | 21:19 | |
| *** rfolco has quit IRC | 21:20 | |
| *** kaisers_ has joined #openstack-infra | 21:25 | |
| *** yamahata has quit IRC | 21:26 | |
| irtermite | jhesketh: you can put me down for owing cloudnull a few more beers as well. together, we will make the rest of his week much less stressful | 21:26 |
| *** Sukhdev has quit IRC | 21:28 | |
| *** thorst has quit IRC | 21:29 | |
| *** kaisers_ has quit IRC | 21:29 | |
| *** jordanP has quit IRC | 21:29 | |
| jhesketh | irtermite: noted :-) | 21:29 |
| *** tdasilva has quit IRC | 21:30 | |
| *** Swami has quit IRC | 21:30 | |
| *** Swami has joined #openstack-infra | 21:31 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Update query for nova/neutron TLS timeout bug 1630664 https://review.openstack.org/385156 | 21:31 |
| openstack | bug 1630664 in OpenStack Compute (nova) "Intermittent failure in n-api connecting to neutron to list ports after TLS was enabled in CI" [Medium,Confirmed] https://launchpad.net/bugs/1630664 | 21:31 |
| *** Sukhdev has joined #openstack-infra | 21:31 | |
| *** piet_ has quit IRC | 21:32 | |
| *** tdasilva has joined #openstack-infra | 21:33 | |
| *** jamesdenton has quit IRC | 21:34 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Default $STABLE_RELEASE in bootstrap-overcloud-full.sh https://review.openstack.org/385159 | 21:34 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Add --setup-nodepool-files option https://review.openstack.org/385160 | 21:34 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Create dummy overcloud-full in --boostrap-subnodes https://review.openstack.org/385161 | 21:34 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Add configurable mtu to --mulitnode-setup https://review.openstack.org/385162 | 21:34 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Update query for nova/neutron TLS timeout bug 1630664 https://review.openstack.org/385156 | 21:36 |
| openstack | bug 1630664 in OpenStack Compute (nova) "Intermittent failure in n-api connecting to neutron to list ports after TLS was enabled in CI" [Medium,Confirmed] https://launchpad.net/bugs/1630664 | 21:36 |
| *** yamahata has joined #openstack-infra | 21:36 | |
| *** Sukhdev has quit IRC | 21:37 | |
| *** eharney has quit IRC | 21:38 | |
| *** Apoorva has joined #openstack-infra | 21:38 | |
| *** annegentle has quit IRC | 21:38 | |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/system-config: Fix logstash-filters test https://review.openstack.org/369575 | 21:41 |
| *** armax has joined #openstack-infra | 21:44 | |
| *** EricGonczer_ has quit IRC | 21:47 | |
| *** armax has quit IRC | 21:48 | |
| *** salv-orlando has joined #openstack-infra | 21:48 | |
| *** e0ne has quit IRC | 21:48 | |
| *** adriant has joined #openstack-infra | 21:49 | |
| *** e0ne has joined #openstack-infra | 21:49 | |
| *** e0ne has quit IRC | 21:50 | |
| *** e0ne has joined #openstack-infra | 21:50 | |
| *** e0ne has quit IRC | 21:50 | |
| *** smcginnis_ has quit IRC | 21:50 | |
| *** e0ne has joined #openstack-infra | 21:51 | |
| *** e0ne has quit IRC | 21:51 | |
| dmsimard | Are the network issues fixed ? Seeing a lot recently. | 21:52 |
| *** e0ne has joined #openstack-infra | 21:52 | |
| *** e0ne has quit IRC | 21:52 | |
| dmsimard | All OSIC afaict | 21:52 |
| *** e0ne has joined #openstack-infra | 21:52 | |
| *** e0ne has quit IRC | 21:53 | |
| *** e0ne has joined #openstack-infra | 21:53 | |
| *** e0ne has quit IRC | 21:54 | |
| *** claudiub has quit IRC | 21:54 | |
| *** e0ne has joined #openstack-infra | 21:54 | |
| *** amitgandhinz has joined #openstack-infra | 21:54 | |
| dmsimard | cloudnull: ^ ? | 21:54 |
| *** e0ne has quit IRC | 21:54 | |
| *** Swami has quit IRC | 21:55 | |
| *** annegentle has joined #openstack-infra | 21:56 | |
| clarkb | dmsimard: they were I was keeping an eye on the e-r graphs and things looked fine | 21:58 |
| clarkb | dmsimard: do you have specific examples? | 21:58 |
| dmsimard | clarkb: not going to link all of them but for example all failures here are network related and on OSIC nodes: https://review.openstack.org/#/c/377158/ | 21:59 |
| dmsimard | Have a couple reviews like that | 22:00 |
| clarkb | dmsimard: thats connection to not the mirror host so likely completely different | 22:01 |
| *** xyang1 has quit IRC | 22:01 | |
| clarkb | the earlier issue was very specifically related to routing on the mirror host | 22:01 |
| clarkb | dig says no ipv6 aaaa record for github.com | 22:02 |
| clarkb | so that will be going through the ipv4 nat | 22:02 |
| *** zigo has quit IRC | 22:02 | |
| irtermite | thank you clarkb for clarifying | 22:03 |
| clarkb | cloudnull: ^ fyi connectivity issues from osic test instance to github.com which should go via nat | 22:04 |
| *** zigo has joined #openstack-infra | 22:05 | |
| clarkb | so now to figure out why that connection would be reset | 22:05 |
| *** zigo is now known as Guest50630 | 22:05 | |
| irtermite | clarkb dmsimard: would you all mind? https://github.com/osic/osic-clouds/issues | 22:07 |
| irtermite | I believe cloudnull may already be on the road | 22:07 |
| *** annegentle has quit IRC | 22:08 | |
| *** zz_dimtruck is now known as dimtruck | 22:08 | |
| clarkb | irtermite: yacan do in a sec | 22:08 |
| *** clayg has joined #openstack-infra | 22:08 | |
| clarkb | I want to try reproducing independently | 22:08 |
| clayg | there's a thing I can search that can tell me if "Could not find a version that satisfies the requirement microversion-parse===0.1.4" is a thing right? | 22:08 |
| cloudnull | back . | 22:10 |
| * cloudnull reading | 22:11 | |
| clarkb | clayg: yes, you can check pypi.python.org and our mirror eg mirror.regionone.osic-cloud1.openstack.org/pypi/simple | 22:12 |
| cloudnull | clarkb: are we seeing mirror issues again ? | 22:12 |
| irtermite | cloudnull: I also just got an alert ticket. reading | 22:13 |
| clarkb | clayg: there were issues with the osic-cloud1 and osic-cloud8 mirrors yesterday and last night that might be related | 22:13 |
| clarkb | cloudnull: irtermite not sure if mirrors are also affected, dmsimard's specific example is for ipv4 connectivity to github | 22:13 |
| clarkb | I am working to try and investigate from my side | 22:13 |
| dmsimard | clarkb: yeah some puppet modules are hosted on github :( | 22:14 |
| *** panda is now known as panda|zZ | 22:14 | |
| irtermite | thank you | 22:14 |
| dmsimard | clarkb: I can provide some other examples as well, but it's most likely the same thing | 22:14 |
| cloudnull | dmsimard: trying to reproduce locally as well. | 22:14 |
| *** tpsilva has quit IRC | 22:14 | |
| clarkb | the mirror host at least still has its routes and can ping github | 22:15 |
| *** spzala has quit IRC | 22:15 | |
| zaro | clarkb: now that i'm trying to setup a nodepool for zuul-dev i'm wondering how we should handle job creation? | 22:16 |
| *** spzala has joined #openstack-infra | 22:16 | |
| zaro | clarkb: when it was jenkins connected to zuul-dev the jobs were expected to be created manually | 22:16 |
| clayg | i'm reading logs here -> http://logs.openstack.org/98/360298/5/check/gate-tempest-dsvm-neutron-src-python-swiftclient/8a39bd6/logs/devstacklog.txt.gz | 22:17 |
| clayg | the host listed doesn't seem to want to talk to me over https | 22:17 |
| clayg | mirror.regionone.osic-cloud1.openstack.org | 22:17 |
| clarkb | zaro: we should be able to use jjb and either use the dev config in project-config or use the same set of jobs then have jobs that only match for -dev | 22:17 |
| clayg | bbiab | 22:17 |
| clarkb | clayg: that is from the time period where that didn't work | 22:17 |
| clarkb | clayg: basically routing was wrong for the mirror host so you couldn't talk to it reliably | 22:18 |
| * zaro looks at the jobs | 22:18 | |
| clarkb | clayg: http://status.openstack.org/elastic-recheck is another good palce to track common issues affecting all CI | 22:19 |
| *** amitgandhinz has quit IRC | 22:19 | |
| *** smcginnis_ has joined #openstack-infra | 22:20 | |
| *** spzala has quit IRC | 22:20 | |
| cloudnull | clayg: those logs were from yesterday, that should be resolved now. | 22:20 |
| clarkb | cloudnull: irtermite fwiw I haven't been able to reproduce on another osic host | 22:21 |
| *** baoli has quit IRC | 22:21 | |
| cloudnull | dmsimard: building a new instance in the same AG on the same network I can't recreate the problem | 22:21 |
| clarkb | I have while git clone https://github.com/puppetlabs/puppetlabs-apache/ ; do rm -rf puppetlabs-apache ; done running | 22:21 |
| cloudnull | https://status.github.com/ shows all good | 22:21 |
| *** hasharAway has quit IRC | 22:21 | |
| cloudnull | clarkb: i have the same thing going in a while loop | 22:21 |
| *** Apoorva has quit IRC | 22:22 | |
| openstackgerrit | Merged openstack-infra/system-config: Fix syntax issue in Firehose websocket example https://review.openstack.org/384486 | 22:22 |
| *** gouthamr has quit IRC | 22:23 | |
| mtreinish | ^^^ heh, oops | 22:24 |
| clarkb | dmsimard: and your jobs aren't modifying networking before they hit that are they? | 22:24 |
| clarkb | looks like package installs from yum repos and gems | 22:24 |
| *** smcginnis_ has quit IRC | 22:24 | |
| mtreinish | although we are blocking websocket traffic still, so I'm kinda surprised someone caught it | 22:25 |
| clarkb | rubygems also does not have a AAAA record | 22:25 |
| *** smcginnis_ has joined #openstack-infra | 22:25 | |
| clarkb | so either somethign happens between then and the github clone or its intermittent | 22:26 |
| clarkb | cloudnull: I did just hit fatal: unable to access 'https://github.com/puppetlabs/puppetlabs-apache/': gnutls_handshake() failed: Error in the pull function | 22:26 |
| clarkb | and if I restart it goes back to happy so likely something intermittent | 22:26 |
| clarkb | with that /me files the bug/issue | 22:27 |
| cloudnull | -- fatal: unable to access 'https://github.com/puppetlabs/puppetlabs-apache/': gnutls_handshake() failed: Error in the pull function -- | 22:27 |
| cloudnull | yup same here. | 22:27 |
| cloudnull | maybe we're being disconnected by github | 22:27 |
| *** amitgandhinz has joined #openstack-infra | 22:27 | |
| openstackgerrit | Isaku Yamahata proposed openstack-infra/project-config: networking-odl: use ubuntu-xenial for newton+ https://review.openstack.org/385036 | 22:28 |
| openstackgerrit | Isaku Yamahata proposed openstack-infra/project-config: networking-odl: add periodic job for stable branches https://review.openstack.org/385181 | 22:28 |
| openstackgerrit | Isaku Yamahata proposed openstack-infra/project-config: networking-odl: add neutron-lib-master periodic job https://review.openstack.org/385182 | 22:28 |
| cloudnull | something like dos protection | 22:28 |
| cloudnull | ? | 22:28 |
| *** piet has joined #openstack-infra | 22:28 | |
| clarkb | maybe? I would be impressed if our connectionrate is high enough to do that | 22:28 |
| clarkb | cloudnull: are there any errors on the nat/router side? | 22:28 |
| cloudnull | its all NAT'd out of the same interface so , maybe? | 22:29 |
| * cloudnull looking into the hosts now. | 22:29 | |
| *** annegentle has joined #openstack-infra | 22:29 | |
| openstackgerrit | Isaku Yamahata proposed openstack-infra/project-config: networking-odl: use ubuntu-xenial for newton+ https://review.openstack.org/385036 | 22:30 |
| cloudnull | also maybe related https://bugs.launchpad.net/ubuntu/+source/git/+bug/1397685 | 22:30 |
| openstack | Launchpad bug 1397685 in gnutls28 (Ubuntu) "git gnutls_handshake() failed: A TLS packet with unexpected length was received." [Undecided,Confirmed] | 22:30 |
| *** pvaneck has quit IRC | 22:30 | |
| *** pvaneck has joined #openstack-infra | 22:31 | |
| cloudnull | gitlab also reported a similar issue https://gitlab.com/gitlab-org/gitlab-ce/issues/13909 | 22:31 |
| cloudnull | and while the cert is not a "self signed one" this could be something in a packages git as noted in the last comment. | 22:32 |
| * cloudnull still digging | 22:32 | |
| clarkb | cloudnull: irtermite https://github.com/osic/osic-clouds/issues/6 has been submitted with the data I have collected | 22:32 |
| cloudnull | cool thanks | 22:33 |
| irtermite | thank you clarkb | 22:33 |
| *** _vsaienko has quit IRC | 22:33 | |
| cloudnull | dmsimard: while not "ideal" is this something you can add a retry on failure to? | 22:35 |
| nibalizer | jesusaur: thanks for doing the work to make iptables more configurable | 22:35 |
| *** berendt has quit IRC | 22:35 | |
| *** rbrndt has quit IRC | 22:37 | |
| clarkb | cloudnull: irtermite looks like a different job failed on yum installation so likely not github specific | 22:38 |
| clarkb | http://logs.openstack.org/58/377158/13/check/gate-puppet-openstack-integration-4-scenario002-tempest-centos-7/494703d/console.html#_2016-10-11_21_32_40_033697 | 22:38 |
| clarkb | dmsimard: and http://logs.openstack.org/58/377158/13/check/gate-puppet-openstack-integration-4-scenario003-tempest-centos-7/5f8d8dd/console.html#_2016-10-11_21_45_46_724799 failed on real errors? can you confirm that? | 22:40 |
| *** erlon has quit IRC | 22:41 | |
| jesusaur | nibalizer: my pleasure, it doubles as a 'dont put upstream ips in downstream iptables rules' change | 22:46 |
| pleia2 | ok, fishbowls and workgroups have been added to the summit schedule https://www.openstack.org/summit/barcelona-2016/summit-schedule/global-search?t=Infrastructure%3A | 22:47 |
| pleia2 | just pleia2: me if there's anything I should add/update/adjust | 22:47 |
| *** amotoki has joined #openstack-infra | 22:47 | |
| karthikp | sc68cal: Hi Sean, I wanted to know if you are carrying any work in https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L224-L245 to move neutron services from primary to subnode | 22:48 |
| clarkb | the back to back stuff looks right to me | 22:49 |
| clarkb | karthikp: fwiw neutron multinode testing with devstack is currently broken and should be fixed (you'll see this manifest as errors with subnode communication due to routes being wrong) | 22:50 |
| clarkb | karthikp: just an fyi in case you are looking at multinode jobs | 22:50 |
| *** baoli has joined #openstack-infra | 22:51 | |
| karthikp | clarkb: Yes I am looking at multinode cinder though... I had a patch in place for cinder multinode testing ..but i heard Sean has some refactoring work planned for test-matrix portion. Correct me if am wrong | 22:52 |
| *** vhosakot has quit IRC | 22:52 | |
| nibalizer | yep | 22:52 |
| karthikp | clarkb: my work: https://review.openstack.org/#/c/384812/ | 22:52 |
| nibalizer | right before we re-ip the cacti server too | 22:52 |
| *** amotoki has quit IRC | 22:52 | |
| clarkb | karthikp: yes that prompted my message to the dev mailing list about "please fix devstack before doing anything else with multinode" | 22:53 |
| clarkb | karthikp: I do not know if sc68cal is working on the test matrixification | 22:53 |
| *** dangers_` is now known as dangers_away | 22:54 | |
| karthikp | clarkb: Ohh does that mean any new multinode changes are halted till devstack is fixed?? | 22:54 |
| clarkb | karthikp: that would be my preference, it is too difficult to review things when they are failing left and right | 22:54 |
| *** adrian_otto has quit IRC | 22:55 | |
| *** davidlenwell has joined #openstack-infra | 22:56 | |
| karthikp | clarkb: Hmm..makes sense. thanks | 22:56 |
| *** tiswanso has quit IRC | 22:58 | |
| *** tiswanso has joined #openstack-infra | 22:59 | |
| clarkb | I was really hoping it would get resolved back when it was reported but my proposed fix was rejected | 22:59 |
| *** jkilpatr has quit IRC | 23:00 | |
| *** jamesdenton has joined #openstack-infra | 23:01 | |
| *** hongbin has quit IRC | 23:03 | |
| *** rhallisey has quit IRC | 23:03 | |
| clayg | clarkb: full ack thanks | 23:04 |
| *** jamesdenton has quit IRC | 23:06 | |
| *** xarses has quit IRC | 23:10 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Add the groundwork for replacing python-*client https://review.openstack.org/384207 | 23:11 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Use REST for listing images https://review.openstack.org/384208 | 23:11 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Replace Image Creation v1 with direct REST calls https://review.openstack.org/385187 | 23:11 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Make munch_response smarter https://review.openstack.org/385188 | 23:11 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Replace Image Create/Delete v2 PUT with REST calls https://review.openstack.org/385189 | 23:11 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Move image tasks to REST https://review.openstack.org/385190 | 23:11 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Remove a few glance client mocks we missed https://review.openstack.org/385191 | 23:11 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Update image downloads to use direct REST https://review.openstack.org/385192 | 23:11 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Change image update to REST and remove glanceclient https://review.openstack.org/385193 | 23:11 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Remove glanceclient and warlock from shade https://review.openstack.org/385194 | 23:11 |
| mordred | jeblair_, pabelanger: ^^ the patch series I mentioned | 23:11 |
| anteaya | mordred: hi there | 23:11 |
| mordred | hey anteaya ! | 23:11 |
| anteaya | haven't seen you all day | 23:11 |
| anteaya | looks like you've been on a plane | 23:11 |
| mordred | anteaya: I was at ansiblefest yesterday and today | 23:11 |
| anteaya | ah | 23:11 |
| mordred | anteaya: but yes, planes have been involved :) | 23:11 |
| anteaya | how was ansiblefest? | 23:12 |
| *** amitgandhinz has quit IRC | 23:13 | |
| mordred | anteaya: it was very good! jeblair_, pabelanger and I had the opportunity to discuss zuul with some friendly humans from fedora, centos, ansible, kubernetes and other places - it was very productive | 23:13 |
| *** kaisers_ has joined #openstack-infra | 23:13 | |
| anteaya | oh I'm so glad | 23:14 |
| anteaya | that sounds like a great event | 23:14 |
| mordred | yah. it was rather nice | 23:14 |
| anteaya | awesome | 23:15 |
| anteaya | I have to reboot now, I would dearly love my xenial installation to be able to resolve dns, it appears that is an optional feature of this installation | 23:16 |
| *** sdake has quit IRC | 23:16 | |
| * anteaya crosses her fingers | 23:16 | |
| nibalizer | anteaya: ouch | 23:16 |
| *** AnarchyAo has quit IRC | 23:17 | |
| mordred | anteaya: good luck! | 23:18 |
| *** kaisers_ has quit IRC | 23:18 | |
| anteaya | nibalizer: yeah no kidding | 23:19 |
| anteaya | mordred: thanks, I haven't gotten it yet | 23:19 |
| anteaya | it can get some, fortunately it can find gerrit and our git farm | 23:19 |
| anteaya | but I have yet to be able to resolve speedtest.net regardless of how long I let it spin | 23:20 |
| cloudnull | dmsimard: clarkb: I just rescheduled the GATEWAY_NET_V6_ROUTER over to a single host, in the router agent hosting list it was showing up on two different agents and this is not an ha router. I have my while loop running again and well see if it runs for longer now. | 23:22 |
| clarkb | cloudnull: ok | 23:22 |
| clarkb | I have restarted mine too for extra data gathering | 23:22 |
| *** rajinir has quit IRC | 23:26 | |
| *** armax has joined #openstack-infra | 23:27 | |
| *** gordc has quit IRC | 23:28 | |
| *** spzala has joined #openstack-infra | 23:28 | |
| clarkb | cloudnull: its still going so things are looking good | 23:29 |
| cloudnull | yea im on 430 cycles still good | 23:30 |
| cloudnull | im letting it run for a while longer. just in case | 23:30 |
| clarkb | cloudnull: in that situation do the two routers fight over the ip address with arp then when it flips to other host you end up killing all those connections as they don't exist in the nat state tables? | 23:30 |
| *** edmondsw has quit IRC | 23:30 | |
| *** david-lyle has joined #openstack-infra | 23:31 | |
| cloudnull | clarkb: it looks that way, and the agent interval is 60 seconds so lots of connectsion will work many more will retry and some will die when it flips over. | 23:31 |
| *** spzala has quit IRC | 23:33 | |
| cloudnull | 645 still going strong | 23:33 |
| cloudnull | looks good to me now. | 23:34 |
| openstackgerrit | Ramy Asselin proposed openstack-infra/elastic-recheck: Make bug name, type, and url explicit https://review.openstack.org/375113 | 23:34 |
| openstackgerrit | Ramy Asselin proposed openstack-infra/elastic-recheck: Refactor launchpad code into bug_tracker https://review.openstack.org/385199 | 23:34 |
| *** xarses has joined #openstack-infra | 23:34 | |
| clarkb | cloudnull: I agree, thanks for looking into that | 23:34 |
| cloudnull | on to the next thing :) | 23:34 |
| *** david-lyle has quit IRC | 23:34 | |
| *** david-lyle has joined #openstack-infra | 23:35 | |
| *** salv-orlando has quit IRC | 23:35 | |
| *** kzaitsev_mb has quit IRC | 23:37 | |
| *** tiswanso has quit IRC | 23:38 | |
| *** david-lyle has quit IRC | 23:38 | |
| *** david-lyle has joined #openstack-infra | 23:39 | |
| openstackgerrit | Ramy Asselin proposed openstack-infra/elastic-recheck: Make bug name, type, and url explicit https://review.openstack.org/375113 | 23:41 |
| openstackgerrit | Ramy Asselin proposed openstack-infra/elastic-recheck: Fix test_queries regression https://review.openstack.org/381776 | 23:41 |
| openstackgerrit | Ramy Asselin proposed openstack-infra/elastic-recheck: Refactor launchpad code into bug_tracker https://review.openstack.org/385199 | 23:41 |
| *** zhurong has joined #openstack-infra | 23:45 | |
| *** whoops has quit IRC | 23:47 | |
| *** annegentle has quit IRC | 23:47 | |
| *** amotoki has joined #openstack-infra | 23:48 | |
| *** david-lyle has quit IRC | 23:49 | |
| *** sdake has joined #openstack-infra | 23:50 | |
| *** sdake_ has joined #openstack-infra | 23:51 | |
| *** amotoki has quit IRC | 23:53 | |
| *** hrybacki has quit IRC | 23:54 | |
| *** hrybacki has joined #openstack-infra | 23:55 | |
| *** sdake has quit IRC | 23:55 | |
| *** cardeois_ has quit IRC | 23:56 | |
| *** dham1 has quit IRC | 23:57 | |
| *** dham1 has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!