| *** david-lyle has quit IRC | 00:00 | |
| wenlock | anyone able to review some reviews that i think are close to +3, i'd like to try to wrap those up before weekend: https://review.openstack.org/104652, https://review.openstack.org/117892, https://review.openstack.org/119245 | 00:00 |
|---|---|---|
| sandywalsh | right, gotcha | 00:00 |
| anteaya | sandywalsh: you like the odd couple too? | 00:00 |
| sandywalsh | ? | 00:00 |
| *** derekh has joined #openstack-infra | 00:00 | |
| anteaya | klugman, quincy? | 00:00 |
| anteaya | odd couple | 00:00 |
| jeblair | fungi, clarkb, mordred: i'm out for the evening | 00:00 |
| mordred | kk | 00:01 |
| fungi | okay, so jenkins-dev has moved cleanly. we're deleting them from the etherpad or leaving them there? | 00:01 |
| jeblair | fungi: del from ep | 00:01 |
| *** ChuckC_ has quit IRC | 00:01 | |
| *** nelsnelson has joined #openstack-infra | 00:01 | |
| fungi | thanks jeblair! | 00:01 |
| jeblair | fungi, thank you! | 00:01 |
| fungi | i'm mostly out for the evening now as well. cookies, beer and star trek. three great tastes that go great together | 00:01 |
| anteaya | sandywalsh: https://en.wikipedia.org/wiki/Jack_Klugman | 00:01 |
| anteaya | fungi: enjoy | 00:02 |
| jeblair | nibalizer: puppet2 hostfile has 35 lines, puppet3 hostfile has 45 | 00:02 |
| fungi | i'll try to hit some of the low-hanging fruit servers in the morning if there are any left when i get online | 00:02 |
| jeblair | so we're >.5 done | 00:02 |
| jeblair | fungi: the xserves? ;) | 00:02 |
| sandywalsh | anteaya: ah, heh, Quincy ... forensic pathologic ... looking for the cause of death ... just like StackTach. | 00:02 |
| anteaya | sandywalsh: ah | 00:02 |
| anteaya | see it was klugman that made me wonder if you were branching out a bit | 00:02 |
| sandywalsh | pathologist | 00:03 |
| fungi | jeblair: those are indeed very low-hanging | 00:03 |
| anteaya | I remember that show | 00:03 |
| sandywalsh | quincy, quince and klugman | 00:03 |
| fungi | jeblair: but sort of an apples to oranges comparison | 00:03 |
| sandywalsh | api, api-implementation and cmdline tools :0 | 00:03 |
| sandywalsh | :) | 00:03 |
| anteaya | sandywalsh: yeah, I am wondering about the quincy, quince naming | 00:03 |
| anteaya | sandywalsh: feels too close to each other to me, but meh | 00:03 |
| *** emagana has quit IRC | 00:04 | |
| sandywalsh | cache invalidation and naming things ... two hardest problems | 00:04 |
| anteaya | yeah | 00:04 |
| *** praneshp has quit IRC | 00:05 | |
| *** oomichi_ has joined #openstack-infra | 00:05 | |
| *** praneshp has joined #openstack-infra | 00:05 | |
| *** oomichi has quit IRC | 00:08 | |
| *** _buzz_ has quit IRC | 00:08 | |
| *** dannywilson has quit IRC | 00:08 | |
| *** rfolco has joined #openstack-infra | 00:09 | |
| *** derekh has quit IRC | 00:09 | |
| *** dannywilson has joined #openstack-infra | 00:09 | |
| *** _nadya_ has joined #openstack-infra | 00:09 | |
| *** nelsnelson has quit IRC | 00:10 | |
| *** ZZelle_ has quit IRC | 00:10 | |
| jroll | sdague: if you have a quick sec, super easy devstack review: 120939 | 00:10 |
| jroll | err | 00:10 |
| jroll | https://review.openstack.org/#/c/120939/ | 00:10 |
| openstackgerrit | Spencer Krum proposed a change to openstack-infra/puppet-storyboard: Template was comparing to :undefined https://review.openstack.org/120944 | 00:10 |
| nibalizer | okay i think this ^^ fixes the problem morderd had | 00:10 |
| *** chuckC has quit IRC | 00:11 | |
| *** signed8bit is now known as signed8bit_ZZZzz | 00:11 | |
| nibalizer | which is less of a fix and more of making the behaviour be the same on puppet 2 and puppet 3 | 00:11 |
| *** signed8bit_ZZZzz has quit IRC | 00:12 | |
| *** dannywilson has quit IRC | 00:13 | |
| nibalizer | jeblair: sweetness, i'd say we met our goals for this week :P | 00:14 |
| *** tomoe has joined #openstack-infra | 00:14 | |
| *** _nadya_ has quit IRC | 00:14 | |
| *** hdd has quit IRC | 00:15 | |
| *** dannywilson has joined #openstack-infra | 00:16 | |
| *** yamamoto has joined #openstack-infra | 00:16 | |
| *** baoli has joined #openstack-infra | 00:16 | |
| *** wenlock has quit IRC | 00:17 | |
| *** dims__ has quit IRC | 00:17 | |
| *** xchu has joined #openstack-infra | 00:19 | |
| *** xchu has left #openstack-infra | 00:19 | |
| *** yamamoto has quit IRC | 00:20 | |
| clarkb | ok I am back. looks like we are winding down for the day. I probably shouldn't continue to move nodes just in case I trip over myself | 00:22 |
| mordred | clarkb: listen. here's the deal ... | 00:22 |
| mordred | drink a ton of beer | 00:22 |
| mordred | clarkb: THEN work on more nodes | 00:22 |
| clarkb | mordred: are you sure that is safe? I suppose I won't care at athat point :) | 00:23 |
| anteaya | don't drink and run commands | 00:23 |
| *** melwitt has quit IRC | 00:25 | |
| *** Vega has joined #openstack-infra | 00:25 | |
| *** Vega is now known as Guest51463 | 00:25 | |
| *** MaxV has joined #openstack-infra | 00:26 | |
| *** dannywilson has quit IRC | 00:26 | |
| *** baoli has quit IRC | 00:29 | |
| *** MaxV has quit IRC | 00:31 | |
| *** rhsu has quit IRC | 00:35 | |
| *** weshay has joined #openstack-infra | 00:35 | |
| *** jab has joined #openstack-infra | 00:38 | |
| *** jab has joined #openstack-infra | 00:38 | |
| *** hdd has joined #openstack-infra | 00:38 | |
| *** jab has quit IRC | 00:40 | |
| krotscheck | mordred: Where’s that migration script of yours? I want to do a test run | 00:40 |
| *** mriedem has joined #openstack-infra | 00:44 | |
| *** unicell has joined #openstack-infra | 00:44 | |
| *** MarkAtwood has quit IRC | 00:44 | |
| krotscheck | mordred: Durrr… did that merge already? | 00:45 |
| krotscheck | Oh it did. | 00:48 |
| *** david-lyle has joined #openstack-infra | 00:48 | |
| krotscheck | Alrightey, let;s see if this is a thing. | 00:48 |
| *** amcrn has quit IRC | 00:48 | |
| *** bhuvan_ has quit IRC | 00:50 | |
| mordred | krotscheck: yay! | 00:54 |
| krotscheck | mordred: I’m going to wrap a nice UX around this. | 00:54 |
| krotscheck | Like, you know, commandline options. | 00:55 |
| krotscheck | Do we have a convention out there somewhere? My gut is saying storyboard-import <source> <url> | 00:55 |
| krotscheck | where source is <launchpad|github|jira| etc> | 00:55 |
| *** mwagner_lap has quit IRC | 00:55 | |
| *** chuckC has joined #openstack-infra | 00:56 | |
| *** arosen has left #openstack-infra | 00:56 | |
| mordred | well... first stab, I'd say "storyboard-import project" (because I'm a simpleton) | 00:58 |
| mordred | and then --source= which defaults to launchpad | 00:58 |
| anteaya | krotscheck: I just -1'd https://review.openstack.org/#/c/111814/ not for anything you did in your patch, but we have an oppornity to make the file even better | 00:58 |
| *** hdd has quit IRC | 00:58 | |
| mordred | krotscheck: and, --url which is optional, but which can provide a url in case an import source is something like github or jira which might have a different url | 00:59 |
| mordred | krotscheck: so that the very simple invocation does the default thing that makes us happy - and that there is an increasing number of ways to say other things | 00:59 |
| krotscheck | OK: Do one thing, do it well. | 00:59 |
| krotscheck | anteaya: kk | 01:00 |
| mordred | krotscheck: there shoudl also be an option, --unicorn, which if given, wll cause a unicorn to appear | 01:01 |
| krotscheck | Hey now, let me learn argparse first. | 01:01 |
| *** baoli has joined #openstack-infra | 01:02 | |
| mordred | is argparse the one we want? | 01:03 |
| mordred | or is it the other one? | 01:03 |
| mordred | krotscheck: ^^ clarkb ^^ ? | 01:03 |
| krotscheck | optparse is the old one | 01:03 |
| mordred | ok. cool | 01:04 |
| mordred | I can never remember | 01:04 |
| krotscheck | There’s a 3.7 version called unicornparse that’ll replace it all at PID1 | 01:04 |
| mordred | krotscheck: SWEET! | 01:05 |
| clarkb | we want argparse | 01:05 |
| *** yamahata has joined #openstack-infra | 01:05 | |
| *** praneshp has quit IRC | 01:07 | |
| * krotscheck needs a non-reserved synonym for import | 01:12 | |
| clarkb | include | 01:13 |
| clarkb | bam! | 01:13 |
| *** shashankhegde has quit IRC | 01:13 | |
| *** mwagner_lap has joined #openstack-infra | 01:16 | |
| *** yamamoto has joined #openstack-infra | 01:16 | |
| anteaya | my thesaurus gives me nothing better than bam! | 01:16 |
| anteaya | I'm so disappointed | 01:17 |
| clarkb | I was channeling whats his name the chef | 01:17 |
| clarkb | when you throw spices into a dish you go "bam!" | 01:17 |
| anteaya | funnel? | 01:17 |
| anteaya | yeah, I know | 01:17 |
| anteaya | I can't allow my mental energy to follow the thought to think of him | 01:18 |
| anteaya | you can channel him | 01:18 |
| *** signed8bit has joined #openstack-infra | 01:21 | |
| *** yamamoto has quit IRC | 01:21 | |
| mordred | Emeril | 01:21 |
| *** signed8bit is now known as signed8bit_ZZZzz | 01:21 | |
| anteaya | no | 01:22 |
| anteaya | blarg | 01:22 |
| *** signed8bit_ZZZzz is now known as signed8bit | 01:23 | |
| *** hdd has joined #openstack-infra | 01:23 | |
| *** signed8bit has quit IRC | 01:23 | |
| *** Daisy has joined #openstack-infra | 01:26 | |
| *** MaxV has joined #openstack-infra | 01:26 | |
| *** dims_ has joined #openstack-infra | 01:28 | |
| jeblair | krotscheck: for fun, it should also be in http://en.wiktionary.org/wiki/Appendix:1000_basic_English_words maybe "introduce". "move_into". | 01:29 |
| *** MaxV has quit IRC | 01:31 | |
| krotscheck | jeblair: “Pink” works for me. | 01:31 |
| *** dims_ has quit IRC | 01:31 | |
| *** dims_ has joined #openstack-infra | 01:32 | |
| *** homeless has quit IRC | 01:33 | |
| anteaya | ha ha ha | 01:34 |
| *** garyh has quit IRC | 01:34 | |
| *** alaski has quit IRC | 01:36 | |
| *** sbalukoff has quit IRC | 01:38 | |
| *** pcrews has quit IRC | 01:39 | |
| *** dims_ has quit IRC | 01:39 | |
| *** marcoemorais has quit IRC | 01:40 | |
| *** jogo has quit IRC | 01:40 | |
| *** dims_ has joined #openstack-infra | 01:40 | |
| *** MaxV has joined #openstack-infra | 01:42 | |
| *** dims__ has joined #openstack-infra | 01:44 | |
| *** dims_ has quit IRC | 01:44 | |
| *** nosnos has joined #openstack-infra | 01:46 | |
| *** MaxV has quit IRC | 01:47 | |
| *** annegentle has joined #openstack-infra | 01:50 | |
| *** dims__ has quit IRC | 01:51 | |
| *** dims_ has joined #openstack-infra | 01:52 | |
| *** hdd has quit IRC | 01:54 | |
| *** baoli has quit IRC | 01:54 | |
| *** baoli has joined #openstack-infra | 01:55 | |
| *** dims_ has quit IRC | 01:57 | |
| *** annegentle has quit IRC | 01:57 | |
| *** patrickeast has quit IRC | 02:03 | |
| *** pcrews has joined #openstack-infra | 02:05 | |
| *** yaguang has joined #openstack-infra | 02:06 | |
| *** ramashri has quit IRC | 02:10 | |
| *** alexpilotti has quit IRC | 02:11 | |
| *** wenlock has joined #openstack-infra | 02:11 | |
| *** baoli has quit IRC | 02:13 | |
| *** baoli has joined #openstack-infra | 02:14 | |
| *** mriedem has quit IRC | 02:15 | |
| *** yamamoto has joined #openstack-infra | 02:16 | |
| *** pcrews has quit IRC | 02:16 | |
| *** annegentle has joined #openstack-infra | 02:18 | |
| *** praneshp has joined #openstack-infra | 02:21 | |
| *** yamamoto has quit IRC | 02:21 | |
| *** Ryan_Lane has quit IRC | 02:22 | |
| *** yamamoto has joined #openstack-infra | 02:23 | |
| *** sunrenjie6 has joined #openstack-infra | 02:26 | |
| *** ianw has quit IRC | 02:27 | |
| *** jyuso has joined #openstack-infra | 02:33 | |
| *** gokrokve has joined #openstack-infra | 02:36 | |
| *** weshay has quit IRC | 02:36 | |
| *** weshay has joined #openstack-infra | 02:37 | |
| *** stevemar has joined #openstack-infra | 02:41 | |
| *** yamamoto has quit IRC | 02:42 | |
| *** MaxV has joined #openstack-infra | 02:43 | |
| *** praneshp has quit IRC | 02:44 | |
| *** baoli has quit IRC | 02:45 | |
| *** baoli has joined #openstack-infra | 02:45 | |
| *** MaxV has quit IRC | 02:48 | |
| *** daya_k has joined #openstack-infra | 02:51 | |
| *** nosnos has quit IRC | 02:53 | |
| *** nosnos has joined #openstack-infra | 02:53 | |
| *** KanagarajM has joined #openstack-infra | 02:56 | |
| *** harlowja is now known as harlowja_away | 02:56 | |
| *** hdd has joined #openstack-infra | 02:56 | |
| *** nosnos has quit IRC | 02:58 | |
| *** rushiagr_away is now known as rushiagr | 03:07 | |
| *** wenlock has quit IRC | 03:09 | |
| *** praneshp has joined #openstack-infra | 03:12 | |
| *** adalbas has quit IRC | 03:14 | |
| *** stevemar has quit IRC | 03:16 | |
| *** stevemar2 has joined #openstack-infra | 03:16 | |
| *** gokrokve_ has joined #openstack-infra | 03:17 | |
| *** yamamoto has joined #openstack-infra | 03:18 | |
| *** gokrokve has quit IRC | 03:20 | |
| *** gokrokve_ has quit IRC | 03:21 | |
| *** praneshp_ has joined #openstack-infra | 03:22 | |
| *** armax has quit IRC | 03:25 | |
| *** armax has joined #openstack-infra | 03:26 | |
| *** armax has quit IRC | 03:26 | |
| *** praneshp has quit IRC | 03:26 | |
| *** praneshp_ is now known as praneshp | 03:26 | |
| *** rushiagr is now known as rushiagr_away | 03:28 | |
| *** rmcall has joined #openstack-infra | 03:29 | |
| *** praneshp has quit IRC | 03:36 | |
| *** praneshp has joined #openstack-infra | 03:38 | |
| *** _nadya_ has joined #openstack-infra | 03:39 | |
| *** praneshp_ has joined #openstack-infra | 03:41 | |
| *** praneshp has quit IRC | 03:42 | |
| *** praneshp_ is now known as praneshp | 03:42 | |
| *** MaxV has joined #openstack-infra | 03:44 | |
| *** MaxV has quit IRC | 03:48 | |
| *** baoli has quit IRC | 03:51 | |
| *** zz_jgrimm is now known as jgrimm | 03:51 | |
| *** amotoki has joined #openstack-infra | 03:55 | |
| *** ramashri has joined #openstack-infra | 03:57 | |
| *** koolhead17 has joined #openstack-infra | 03:59 | |
| *** ramashri has quit IRC | 04:02 | |
| *** rmcall has quit IRC | 04:03 | |
| *** rmcall has joined #openstack-infra | 04:04 | |
| *** AaronGreen has joined #openstack-infra | 04:07 | |
| *** gokrokve has joined #openstack-infra | 04:07 | |
| *** AaronGreen is now known as AaronGr | 04:07 | |
| *** rushiagr_away is now known as rushiagr | 04:12 | |
| *** ramashri has joined #openstack-infra | 04:12 | |
| *** jog0 has joined #openstack-infra | 04:13 | |
| *** nosnos has joined #openstack-infra | 04:13 | |
| *** jog0 is now known as jogo | 04:13 | |
| *** todd_dsm has quit IRC | 04:14 | |
| *** Daisy has quit IRC | 04:14 | |
| *** annegentle has quit IRC | 04:21 | |
| *** annegentle has joined #openstack-infra | 04:22 | |
| *** jogo has quit IRC | 04:23 | |
| *** jog0 has joined #openstack-infra | 04:23 | |
| *** ianw has joined #openstack-infra | 04:23 | |
| *** weshay has quit IRC | 04:24 | |
| *** jog0 has quit IRC | 04:25 | |
| *** jogo has joined #openstack-infra | 04:25 | |
| *** jogo is now known as Guest86139 | 04:25 | |
| *** annegentle has quit IRC | 04:29 | |
| *** Guest86139 has quit IRC | 04:30 | |
| *** jogo_ has joined #openstack-infra | 04:30 | |
| *** rfolco has quit IRC | 04:31 | |
| *** bhuvan has joined #openstack-infra | 04:31 | |
| *** wenlock has joined #openstack-infra | 04:42 | |
| *** Daisy has joined #openstack-infra | 04:44 | |
| *** MaxV has joined #openstack-infra | 04:45 | |
| *** jogo_ has quit IRC | 04:45 | |
| *** jogo_ has joined #openstack-infra | 04:45 | |
| *** jogo_ has quit IRC | 04:46 | |
| *** jog0 has joined #openstack-infra | 04:46 | |
| *** jog0 has quit IRC | 04:46 | |
| *** jogo_ has joined #openstack-infra | 04:47 | |
| *** MaxV has quit IRC | 04:49 | |
| *** rhsu has joined #openstack-infra | 04:49 | |
| *** gokrokve has quit IRC | 04:50 | |
| *** mestery has quit IRC | 04:52 | |
| *** jogo_ has quit IRC | 04:54 | |
| *** jogo_ has joined #openstack-infra | 04:54 | |
| *** HenryG is now known as HenryG_afk | 04:55 | |
| *** harlowja_at_home has joined #openstack-infra | 04:57 | |
| jogo_ | clarkb: I am playing with weechat and have a few questions if your around | 04:57 |
| clarkb | sure | 04:57 |
| jogo_ | clarkb: for some reason "/script" isn't working | 04:57 |
| *** tsufiev has quit IRC | 04:58 | |
| *** tsufiev_ has joined #openstack-infra | 04:58 | |
| jogo_ | Error: unknown command "script" (type /help for help) | 04:58 |
| jogo_ | clarkb: any ideas? | 04:59 |
| clarkb | try using specific language load commands like /python? | 04:59 |
| clarkb | but going to guess weechat was compiled without script support. I dont actually use any scripts | 05:00 |
| clarkb | my weechat doesnt have /script but has /python | 05:00 |
| clarkb | and ruby and perl | 05:01 |
| *** _nadya_ has quit IRC | 05:01 | |
| *** asettle has quit IRC | 05:04 | |
| jogo_ | clarkb: cool it worked | 05:04 |
| jogo_ | I wanted the buffers.pl script and its working | 05:04 |
| *** HenryG_afk has quit IRC | 05:04 | |
| *** alexandra has joined #openstack-infra | 05:05 | |
| *** alexandra is now known as asettle | 05:06 | |
| jogo_ | clarkb: any scripts you recommend? | 05:06 |
| clarkb | I dont use any. I do recommend autofilters though | 05:06 |
| jogo_ | autofilters? | 05:07 |
| clarkb | ya basically I only see joins parts and quits if I toggle a setting. they are all in my buffers but usually filtered out | 05:08 |
| clarkb | unlike an ignore they are still there if I need to see them | 05:09 |
| clarkb | gah smartfilters | 05:09 |
| clarkb | thats what they are really called | 05:09 |
| *** jogo_ is now known as jog0 | 05:10 | |
| *** jog0 is now known as Guest13875 | 05:10 | |
| clarkb | http://weechat.org/files/doc/weechat_faq.en.html#filter_irc_join_part_quit | 05:11 |
| clarkb | then use meta + = to toggle it on off | 05:11 |
| *** Ryan_Lane has joined #openstack-infra | 05:11 | |
| Guest13875 | oh neat I have it set that way too | 05:12 |
| Guest13875 | gah configuration | 05:12 |
| *** Guest13875 has quit IRC | 05:12 | |
| *** flashgordon has joined #openstack-infra | 05:14 | |
| *** Ryan_Lane has quit IRC | 05:14 | |
| *** flashgordon has quit IRC | 05:15 | |
| *** HeOS has quit IRC | 05:16 | |
| *** flashgordon has joined #openstack-infra | 05:18 | |
| *** reed has quit IRC | 05:18 | |
| *** flashgordon is now known as jogo | 05:19 | |
| *** jogo has quit IRC | 05:20 | |
| *** jogo has joined #openstack-infra | 05:22 | |
| jogo | clarkb: thanks, weechat is pretty nice | 05:22 |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/config: Add bare-f20 nodes https://review.openstack.org/117397 | 05:23 |
| *** HenryG has joined #openstack-infra | 05:23 | |
| *** HenryG is now known as HenryG_afk | 05:24 | |
| *** harlowja_at_home has quit IRC | 05:25 | |
| ianw | clarkb: ^ puppet-postgresql released so hopefully we can just merge the f20-bare node bring-up with no further changes | 05:25 |
| *** annegentle has joined #openstack-infra | 05:28 | |
| *** annegentle has quit IRC | 05:33 | |
| *** jogo has quit IRC | 05:33 | |
| *** julim has quit IRC | 05:33 | |
| *** jogo has joined #openstack-infra | 05:33 | |
| *** hdd has quit IRC | 05:35 | |
| *** loki184 has joined #openstack-infra | 05:35 | |
| *** ramashri has quit IRC | 05:36 | |
| *** jogo has quit IRC | 05:38 | |
| *** jogo has joined #openstack-infra | 05:38 | |
| *** Daisy has quit IRC | 05:39 | |
| *** HenryG has joined #openstack-infra | 05:42 | |
| *** vigneshvar has joined #openstack-infra | 05:42 | |
| *** HenryG has quit IRC | 05:43 | |
| *** MaxV has joined #openstack-infra | 05:45 | |
| *** rfolco has joined #openstack-infra | 05:50 | |
| *** MaxV has quit IRC | 05:50 | |
| *** Daisy has joined #openstack-infra | 05:50 | |
| *** morganfainberg is now known as stevemar3 | 05:50 | |
| *** stevemar3 is now known as morganfainberg | 05:51 | |
| *** emagana has joined #openstack-infra | 05:55 | |
| *** stevemar2 is now known as stevemar | 05:56 | |
| *** wenlock has quit IRC | 05:57 | |
| *** yfried has joined #openstack-infra | 05:58 | |
| *** emagana has quit IRC | 06:00 | |
| *** emagana has joined #openstack-infra | 06:01 | |
| *** emagana_ has joined #openstack-infra | 06:02 | |
| *** jgrimm is now known as zz_jgrimm | 06:03 | |
| *** sunrenjie6 has quit IRC | 06:04 | |
| *** shashankhegde has joined #openstack-infra | 06:05 | |
| *** emagana has quit IRC | 06:06 | |
| *** stevemar has quit IRC | 06:10 | |
| *** ildikov has quit IRC | 06:17 | |
| *** pkoniszewski has joined #openstack-infra | 06:17 | |
| *** flaper87|afk is now known as flaper87 | 06:18 | |
| *** shashankhegde has quit IRC | 06:18 | |
| *** emagana has joined #openstack-infra | 06:24 | |
| *** emagana_ has quit IRC | 06:27 | |
| *** annegentle has joined #openstack-infra | 06:28 | |
| *** jamespage_ has joined #openstack-infra | 06:29 | |
| *** emagana has quit IRC | 06:29 | |
| *** andreaf has joined #openstack-infra | 06:30 | |
| *** annegentle has quit IRC | 06:33 | |
| *** rmcall has quit IRC | 06:35 | |
| *** Guest51463 is now known as LuckyVega | 06:37 | |
| *** jcoufal has joined #openstack-infra | 06:39 | |
| *** HeOS has joined #openstack-infra | 06:42 | |
| *** k4n0 has joined #openstack-infra | 06:45 | |
| *** MaxV has joined #openstack-infra | 06:46 | |
| *** Longgeek has joined #openstack-infra | 06:47 | |
| *** sdake has joined #openstack-infra | 06:50 | |
| *** MaxV has quit IRC | 06:50 | |
| *** sdake has quit IRC | 06:52 | |
| openstackgerrit | A change was merged to openstack-dev/hacking: Updated from global requirements https://review.openstack.org/120628 | 07:00 |
| *** ildikov has joined #openstack-infra | 07:03 | |
| *** mrmartin has joined #openstack-infra | 07:03 | |
| *** mrmartin has quit IRC | 07:04 | |
| *** mrmartin has joined #openstack-infra | 07:05 | |
| *** mrmartin has quit IRC | 07:05 | |
| *** e0ne has joined #openstack-infra | 07:09 | |
| *** penguinRaider has joined #openstack-infra | 07:16 | |
| *** daya_k has quit IRC | 07:17 | |
| *** rhsu has quit IRC | 07:19 | |
| *** dtantsur|afk is now known as dtantsur | 07:20 | |
| *** e0ne has quit IRC | 07:20 | |
| *** arxcruz has joined #openstack-infra | 07:22 | |
| *** berendt has joined #openstack-infra | 07:23 | |
| *** yamamoto has quit IRC | 07:23 | |
| *** praneshp has quit IRC | 07:24 | |
| *** e0ne has joined #openstack-infra | 07:24 | |
| *** koolhead17 has quit IRC | 07:24 | |
| *** e0ne has quit IRC | 07:26 | |
| *** annegentle has joined #openstack-infra | 07:28 | |
| *** niska has quit IRC | 07:29 | |
| *** jp_at_hp has joined #openstack-infra | 07:29 | |
| *** Guest1245 has joined #openstack-infra | 07:30 | |
| *** annegentle has quit IRC | 07:33 | |
| *** niska has joined #openstack-infra | 07:34 | |
| *** ashaeron has joined #openstack-infra | 07:43 | |
| *** jistr has joined #openstack-infra | 07:47 | |
| *** che-arne has quit IRC | 07:49 | |
| *** amotoki has quit IRC | 07:51 | |
| *** amuller has joined #openstack-infra | 07:51 | |
| *** derekh has joined #openstack-infra | 07:57 | |
| *** ndipanov_gone is now known as ndipanoff | 07:58 | |
| *** mmaglana has quit IRC | 07:58 | |
| *** penguinRaider has quit IRC | 08:00 | |
| *** _nadya_ has joined #openstack-infra | 08:01 | |
| *** MaxV has joined #openstack-infra | 08:01 | |
| *** che-arne has joined #openstack-infra | 08:02 | |
| *** ihrachyshka has joined #openstack-infra | 08:03 | |
| *** jlibosva has joined #openstack-infra | 08:04 | |
| *** baoli has joined #openstack-infra | 08:05 | |
| *** _nadya_ has quit IRC | 08:06 | |
| *** lttrl has joined #openstack-infra | 08:11 | |
| *** baoli has quit IRC | 08:12 | |
| *** oomichi has joined #openstack-infra | 08:12 | |
| *** oomichi has quit IRC | 08:12 | |
| *** YorikSar_ has joined #openstack-infra | 08:13 | |
| *** jamespage_ has quit IRC | 08:13 | |
| *** dtantsur is now known as dtantsur|bbl | 08:15 | |
| *** oomichi_ has quit IRC | 08:16 | |
| *** YorikSar has quit IRC | 08:17 | |
| HeOS | fungi, when I need to rebase this change ? Once the root admins merge this change (https://review.openstack.org/#/c/120747/)? | 08:19 |
| *** andreykurilin has joined #openstack-infra | 08:19 | |
| *** e0ne has joined #openstack-infra | 08:20 | |
| *** yamahata__ has quit IRC | 08:22 | |
| *** yamahata__ has joined #openstack-infra | 08:23 | |
| *** yamahata has quit IRC | 08:26 | |
| *** annegentle has joined #openstack-infra | 08:28 | |
| *** e0ne has quit IRC | 08:30 | |
| *** annegentle has quit IRC | 08:33 | |
| *** jpich has quit IRC | 08:34 | |
| *** amuller has quit IRC | 08:35 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 08:52 | |
| *** luqas has quit IRC | 08:52 | |
| *** e0ne has joined #openstack-infra | 08:52 | |
| *** Dafna has joined #openstack-infra | 08:54 | |
| *** jab has joined #openstack-infra | 08:56 | |
| *** jab has joined #openstack-infra | 08:56 | |
| *** camunoz has quit IRC | 08:57 | |
| *** jab has quit IRC | 08:59 | |
| *** eglynn-officeafk is now known as eglynn-office | 09:00 | |
| *** rushiagr is now known as rushiagr_away | 09:01 | |
| *** dizquierdo has joined #openstack-infra | 09:12 | |
| *** rushiagr_away is now known as rushiagr | 09:13 | |
| *** Daisy has quit IRC | 09:14 | |
| *** daya_k has joined #openstack-infra | 09:16 | |
| *** marun has joined #openstack-infra | 09:16 | |
| *** jpeeler has quit IRC | 09:20 | |
| openstackgerrit | A change was merged to stackforge/dox: Add tests for tox_ini https://review.openstack.org/120765 | 09:24 |
| *** markmc has joined #openstack-infra | 09:26 | |
| *** pelix has joined #openstack-infra | 09:28 | |
| *** annegentle has joined #openstack-infra | 09:30 | |
| *** Azher has joined #openstack-infra | 09:32 | |
| *** annegentle has quit IRC | 09:35 | |
| *** luqas has joined #openstack-infra | 09:36 | |
| *** marun has quit IRC | 09:39 | |
| *** dizquierdo has quit IRC | 09:39 | |
| *** ociuhandu has quit IRC | 09:41 | |
| *** YorikSar_ has quit IRC | 09:45 | |
| *** YorikSar has joined #openstack-infra | 09:45 | |
| *** amotoki has joined #openstack-infra | 09:49 | |
| *** yamahata has joined #openstack-infra | 09:50 | |
| *** dizquierdo has joined #openstack-infra | 09:51 | |
| *** rlandy has joined #openstack-infra | 09:56 | |
| *** _nadya_ has joined #openstack-infra | 09:56 | |
| *** yaguang has quit IRC | 09:59 | |
| *** unicell has quit IRC | 10:01 | |
| *** cnesa9 has joined #openstack-infra | 10:01 | |
| *** dizquierdo has quit IRC | 10:02 | |
| *** jlibosva has quit IRC | 10:02 | |
| *** jlibosva has joined #openstack-infra | 10:03 | |
| *** yamamoto has joined #openstack-infra | 10:07 | |
| *** dims_ has joined #openstack-infra | 10:11 | |
| *** pbelanyi has quit IRC | 10:12 | |
| *** pcm_ has joined #openstack-infra | 10:13 | |
| *** Guest1245 has quit IRC | 10:13 | |
| *** dizquierdo has joined #openstack-infra | 10:14 | |
| *** yamamoto has quit IRC | 10:14 | |
| *** dims_ has quit IRC | 10:16 | |
| *** dims_ has joined #openstack-infra | 10:16 | |
| *** alexpilotti has joined #openstack-infra | 10:18 | |
| *** dims_ has quit IRC | 10:20 | |
| *** yamamoto has joined #openstack-infra | 10:21 | |
| *** andreykurilin has quit IRC | 10:21 | |
| *** jlibosva has quit IRC | 10:25 | |
| *** jlibosva has joined #openstack-infra | 10:26 | |
| *** jlibosva has quit IRC | 10:27 | |
| *** jlibosva has joined #openstack-infra | 10:27 | |
| *** annegentle has joined #openstack-infra | 10:28 | |
| *** YorikSar has quit IRC | 10:28 | |
| *** KanagarajM2 has joined #openstack-infra | 10:30 | |
| *** YorikSar has joined #openstack-infra | 10:30 | |
| *** KanagarajM has quit IRC | 10:31 | |
| *** annegentle has quit IRC | 10:33 | |
| *** yamamoto has quit IRC | 10:34 | |
| *** KanagarajM2 has quit IRC | 10:34 | |
| *** bhuvan has quit IRC | 10:44 | |
| *** kmartin has quit IRC | 10:44 | |
| *** nosnos has quit IRC | 10:44 | |
| *** nosnos has joined #openstack-infra | 10:45 | |
| *** yfried has quit IRC | 10:47 | |
| *** e0ne has quit IRC | 10:49 | |
| *** nosnos has quit IRC | 10:49 | |
| *** [HeOS] has joined #openstack-infra | 10:52 | |
| *** [HeOS] has quit IRC | 10:52 | |
| *** HeOS_ has joined #openstack-infra | 10:54 | |
| *** yfried has joined #openstack-infra | 10:54 | |
| *** rushiagr is now known as rushiagr_away | 10:56 | |
| *** HeOS_ has quit IRC | 10:58 | |
| *** _nadya_ has quit IRC | 10:59 | |
| *** HeOS_ has joined #openstack-infra | 10:59 | |
| *** jcoufal has quit IRC | 11:00 | |
| *** dims_ has joined #openstack-infra | 11:00 | |
| *** Azher has quit IRC | 11:00 | |
| *** yfried is now known as yfried__ | 11:04 | |
| *** yfried__ is now known as yfried | 11:04 | |
| *** andreaf has quit IRC | 11:05 | |
| *** andreaf has joined #openstack-infra | 11:05 | |
| *** yamamoto has joined #openstack-infra | 11:05 | |
| *** yamamoto_ has joined #openstack-infra | 11:08 | |
| *** yamamoto has quit IRC | 11:08 | |
| *** Clabbe has quit IRC | 11:10 | |
| *** Clabbe has joined #openstack-infra | 11:14 | |
| *** Dafna is now known as Dafna_lunch | 11:14 | |
| *** jyuso has quit IRC | 11:15 | |
| *** jyuso has joined #openstack-infra | 11:15 | |
| *** adalbas has joined #openstack-infra | 11:17 | |
| *** yfried has quit IRC | 11:17 | |
| *** yfried has joined #openstack-infra | 11:19 | |
| *** yfried has quit IRC | 11:22 | |
| *** adalbas has quit IRC | 11:22 | |
| *** ihrachyshka has quit IRC | 11:22 | |
| *** ihrachyshka has joined #openstack-infra | 11:22 | |
| *** ihrachyshka has quit IRC | 11:24 | |
| *** annegentle has joined #openstack-infra | 11:28 | |
| *** luqas has quit IRC | 11:32 | |
| *** annegentle has quit IRC | 11:34 | |
| *** yfried_ has joined #openstack-infra | 11:34 | |
| openstackgerrit | zimbatm proposed a change to stackforge/dox: Replace invalid project characters with '-' https://review.openstack.org/121056 | 11:35 |
| openstackgerrit | zimbatm proposed a change to stackforge/dox: Replace os.getlogin() by our own helper https://review.openstack.org/121057 | 11:35 |
| openstackgerrit | zimbatm proposed a change to stackforge/dox: Makes the project name configurable https://review.openstack.org/121058 | 11:35 |
| *** dtantsur|bbl is now known as dtantsur | 11:35 | |
| *** ociuhandu has joined #openstack-infra | 11:36 | |
| *** dalgaaf_ has joined #openstack-infra | 11:39 | |
| *** loki184 has quit IRC | 11:39 | |
| *** _nadya_ has joined #openstack-infra | 11:40 | |
| openstackgerrit | Davanum Srinivas (dims) proposed a change to openstack-infra/config: Switch py33 jobs for oslo.log and oslo.utils to vote https://review.openstack.org/119426 | 11:42 |
| *** lyxus has quit IRC | 11:44 | |
| *** marun has joined #openstack-infra | 11:44 | |
| *** Longgeek has quit IRC | 11:45 | |
| *** Longgeek has joined #openstack-infra | 11:46 | |
| *** Longgeek has quit IRC | 11:48 | |
| *** tty57 has joined #openstack-infra | 11:48 | |
| zaro | clarkb: i have someone next to me that cannot sign our ICLA agreement. error message is "cannot store contact info". do you know anything about that? | 11:50 |
| *** lyxus has joined #openstack-infra | 11:50 | |
| *** Longgeek has joined #openstack-infra | 11:51 | |
| *** Longgeek_ has joined #openstack-infra | 11:53 | |
| ZZelle | zaro, hi | 11:55 |
| ZZelle | zaro, did he register to https://www.openstack.org/join? | 11:56 |
| *** Longgeek has quit IRC | 11:56 | |
| *** david-lyle has quit IRC | 11:57 | |
| *** mwagner_lap has quit IRC | 11:58 | |
| *** e0ne has joined #openstack-infra | 11:58 | |
| *** rushiagr_away is now known as rushiagr | 11:58 | |
| zaro | no | 11:58 |
| *** yfried_ is now known as yfried|afk | 11:58 | |
| ZZelle | zaro, it's required | 11:59 |
| *** kgiusti has joined #openstack-infra | 11:59 | |
| zaro | ok. that was confusing. he actually started from storyboard and no where along the way did it provide info about this requirement. | 12:00 |
| *** rushiagr is now known as rushiagr_away | 12:05 | |
| *** aysyd has joined #openstack-infra | 12:05 | |
| *** Clabbe has quit IRC | 12:05 | |
| *** Clabbe has joined #openstack-infra | 12:08 | |
| *** marun has quit IRC | 12:08 | |
| *** russellb has joined #openstack-infra | 12:11 | |
| *** rushiagr_away is now known as rushiagr | 12:12 | |
| ZZelle | zaro, finally he must update its contact information | 12:12 |
| *** kgiusti has quit IRC | 12:14 | |
| *** kgiusti has joined #openstack-infra | 12:16 | |
| *** unicell has joined #openstack-infra | 12:18 | |
| *** _nadya_ has quit IRC | 12:18 | |
| *** baoli has joined #openstack-infra | 12:19 | |
| *** mbacchi has joined #openstack-infra | 12:19 | |
| *** vigneshvar has quit IRC | 12:20 | |
| *** dims_ has quit IRC | 12:21 | |
| *** dims_ has joined #openstack-infra | 12:22 | |
| *** dkranz has joined #openstack-infra | 12:24 | |
| *** e0ne has quit IRC | 12:25 | |
| *** kgiusti has quit IRC | 12:27 | |
| *** kgiusti has joined #openstack-infra | 12:28 | |
| *** annegentle has joined #openstack-infra | 12:28 | |
| *** kgiusti has left #openstack-infra | 12:28 | |
| *** hdd has joined #openstack-infra | 12:30 | |
| *** dizquierdo has quit IRC | 12:31 | |
| *** kgiusti has joined #openstack-infra | 12:31 | |
| *** dprince has joined #openstack-infra | 12:32 | |
| *** annegentle has quit IRC | 12:33 | |
| *** _nadya_ has joined #openstack-infra | 12:36 | |
| *** ihrachyshka has joined #openstack-infra | 12:37 | |
| *** bswartz has quit IRC | 12:38 | |
| *** Dafna_lunch is now known as Dafna | 12:39 | |
| *** tty57 has quit IRC | 12:39 | |
| *** _nadya_ has quit IRC | 12:39 | |
| *** Guest1245 has joined #openstack-infra | 12:43 | |
| *** tty57 has joined #openstack-infra | 12:44 | |
| *** julim has joined #openstack-infra | 12:44 | |
| *** luqas has joined #openstack-infra | 12:46 | |
| *** ling-yun has joined #openstack-infra | 12:46 | |
| ling-yun | hi stackers. Cinder project has a patch of https://review.openstack.org/#/c/120907/ to fix Jenkins failed. How could we do to expedite the patch in gate routine. | 12:48 |
| *** HeOS has quit IRC | 12:48 | |
| ling-yun | If could not expedite the patch https://review.openstack.org/#/c/120907/ to be run in gate, we may need wait more than 10 hours. | 12:49 |
| *** dmsimard_away is now known as dmsimard | 12:49 | |
| *** HenryG_afk is now known as HenryG | 12:51 | |
| openstackgerrit | Mike Heald proposed a change to openstack-dev/pbr: Retry the integration setup on connection error https://review.openstack.org/121093 | 12:53 |
| *** weshay has joined #openstack-infra | 12:53 | |
| *** yfried|afk is now known as yfried_ | 12:54 | |
| *** cdent has joined #openstack-infra | 12:55 | |
| flaper87 | fungi: I'm able to reproduce that pypy trace. However, the first time I executed it, I reproduced what I've seen in the pypy gate lately http://paste.openstack.org/show/110642/ | 13:02 |
| flaper87 | i believe they're related, though. | 13:02 |
| flaper87 | Alex_Gaynor: have you seen this? ^ | 13:02 |
| fungi | flaper87: do you have a link to one which was hit in the gate? | 13:03 |
| flaper87 | fungi: lemme try to find it | 13:03 |
| fungi | flaper87: because the supposition is that it was fixed after pypy 2.2.1 | 13:04 |
| *** tomoe has quit IRC | 13:04 | |
| fungi | flaper87: by https://bitbucket.org/pypy/pypy/commits/5ca1994 | 13:04 |
| *** tomoe has joined #openstack-infra | 13:05 | |
| *** tomoe has joined #openstack-infra | 13:05 | |
| *** ling-yun-1 has joined #openstack-infra | 13:05 | |
| fungi | flaper87: relevant details are in bug 1368418 and in the pypy bitbucket issues linked from it | 13:07 |
| uvirtbot | Launchpad bug 1368418 in pypy "Fatal RPython error: KeyError/Crash in JIT!" [Undecided,New] https://launchpad.net/bugs/1368418 | 13:07 |
| *** pbelanyi has joined #openstack-infra | 13:08 | |
| *** ling-yun has quit IRC | 13:08 | |
| *** ramashri has joined #openstack-infra | 13:08 | |
| *** tomoe has quit IRC | 13:09 | |
| openstackgerrit | Sandy Walsh proposed a change to openstack-infra/config: Timex - A time expressions library https://review.openstack.org/120852 | 13:10 |
| *** kgiusti has quit IRC | 13:11 | |
| *** mwagner_lap has joined #openstack-infra | 13:12 | |
| openstackgerrit | Sandy Walsh proposed a change to openstack-infra/config: StackDistiller - A notification extraction and transformation library. https://review.openstack.org/120858 | 13:12 |
| openstackgerrit | Sandy Walsh proposed a change to openstack-infra/config: Sandbox - DevStack-like dev environment for StackTach.v3 https://review.openstack.org/120844 | 13:13 |
| *** kgiusti has joined #openstack-infra | 13:13 | |
| flaper87 | fungi: mmh, can't find it. I got that traceback in the test I just ran | 13:14 |
| flaper87 | after that sqlite error, I got the same core dump you got | 13:14 |
| flaper87 | and the third time I ran the test it just failed with KeyError but without showing any output | 13:14 |
| flaper87 | I'm using the same ubuntu, tox and pypy as mentioned in the bug | 13:15 |
| fungi | flaper87: yeah it seems to be nondeterministic in behavior, possibly varying with different hash seed values | 13:15 |
| flaper87 | fungi: oh mmh, good point | 13:15 |
| flaper87 | I'll try re-using the same hash | 13:15 |
| fungi | if it's a new enough tox to randomize the hash seed, it will mention what hash seed it used | 13:15 |
| *** bswartz has joined #openstack-infra | 13:16 | |
| flaper87 | yeah, it did | 13:16 |
| flaper87 | I'll try to reproduce it with the same hash | 13:16 |
| fungi | though i have no idea whether pypy randomizes its seed by default like py3k or holds it at 0 like 2.x | 13:16 |
| fungi | Alex_Gaynor: ^ ? | 13:16 |
| fungi | but yeah, if tox is explicitly passing in a PYTHONHASHSEED envvar then that's a likely cause of variations in behavior | 13:17 |
| fungi | regardless of pypy's default behavior for hash seeds | 13:17 |
| openstackgerrit | Sandy Walsh proposed a change to openstack-infra/config: Simport - Simple python import library (not entry-point based) https://review.openstack.org/121108 | 13:17 |
| *** ramashri has quit IRC | 13:17 | |
| *** amuller has joined #openstack-infra | 13:19 | |
| *** rmcall has joined #openstack-infra | 13:19 | |
| *** stevemar has joined #openstack-infra | 13:20 | |
| *** ramashri has joined #openstack-infra | 13:21 | |
| *** mestery has joined #openstack-infra | 13:22 | |
| openstackgerrit | Sandy Walsh proposed a change to openstack-infra/config: notification_utils: utility library for working with OpenStack notifications. https://review.openstack.org/121109 | 13:22 |
| *** rmcall has quit IRC | 13:23 | |
| *** dims_ has quit IRC | 13:23 | |
| *** mfer has joined #openstack-infra | 13:23 | |
| *** dims_ has joined #openstack-infra | 13:23 | |
| *** garyh has joined #openstack-infra | 13:24 | |
| *** mriedem has joined #openstack-infra | 13:24 | |
| *** alaski has joined #openstack-infra | 13:25 | |
| *** dangers_away is now known as dangers | 13:27 | |
| *** amuller has quit IRC | 13:27 | |
| *** esker has joined #openstack-infra | 13:27 | |
| *** dims__ has joined #openstack-infra | 13:28 | |
| *** annegentle has joined #openstack-infra | 13:28 | |
| ZZelle | fungi, hi | 13:29 |
| *** esker has quit IRC | 13:29 | |
| *** dims_ has quit IRC | 13:29 | |
| *** adalbas has joined #openstack-infra | 13:32 | |
| *** annegentle has quit IRC | 13:33 | |
| *** dansmith is now known as superdan | 13:36 | |
| *** koolhead17 has joined #openstack-infra | 13:37 | |
| *** bknudson has joined #openstack-infra | 13:39 | |
| *** signed8bit has joined #openstack-infra | 13:41 | |
| *** nelsnelson has joined #openstack-infra | 13:42 | |
| *** annegentle has joined #openstack-infra | 13:44 | |
| fungi | what's up ZZelle? | 13:44 |
| ZZelle | fungi, if you have some time, could you have a look to git-review changes in gerrit? | 13:45 |
| fungi | ZZelle: it may end up waiting for the weekend or next week, but i will try to make it a priority | 13:47 |
| ZZelle | fungi, thnaks | 13:47 |
| ZZelle | fungi, thanks | 13:47 |
| *** wenlock has joined #openstack-infra | 13:51 | |
| *** e0ne has joined #openstack-infra | 13:53 | |
| *** annegentle has quit IRC | 13:54 | |
| *** doug-fish has left #openstack-infra | 13:54 | |
| *** eharney has joined #openstack-infra | 13:55 | |
| *** ihrachyshka has quit IRC | 13:56 | |
| *** rmcall has joined #openstack-infra | 13:59 | |
| *** hdd has quit IRC | 14:00 | |
| *** ihrachyshka has joined #openstack-infra | 14:02 | |
| *** yfried_ is now known as yfried|afk | 14:02 | |
| *** doug-fish has joined #openstack-infra | 14:03 | |
| *** jcoufal-mobile has joined #openstack-infra | 14:04 | |
| *** dims has joined #openstack-infra | 14:05 | |
| *** r-daneel has joined #openstack-infra | 14:05 | |
| sandywalsh | has anyone successfully integrated hosted travis with a stackforge project? | 14:06 |
| *** DuncanT is now known as DuncanT-away | 14:07 | |
| fungi | sandywalsh: no idea. we (openstack project infrastructure) generally haven't offered assistance for such endeavors since we already maintain and offer a massive continuous testing platform | 14:07 |
| openstackgerrit | Ian Cordasco proposed a change to openstack-infra/elastic-recheck: Add elastic-recheck query for bug 1368782 https://review.openstack.org/121126 | 14:07 |
| uvirtbot | Launchpad bug 1368782 in openstack-ci "Fixed IPs Test Build Errors Out" [Undecided,New] https://launchpad.net/bugs/1368782 | 14:07 |
| sandywalsh | fungi: gotcha, just thinking about something targeted (lean and mean) | 14:08 |
| fungi | i believe as of a couple months ago mordred asserted that we have more than twice the test capacity of travis-ci (just for a comparative data point on scale) | 14:08 |
| sandywalsh | good to know | 14:09 |
| *** jpeeler has joined #openstack-infra | 14:10 | |
| fungi | and also travis-ci doesn't use openstack-based services afaik, so we provide the benefit of dogfooding and generating demonstrative load on openstack service provider member companies too | 14:11 |
| anteaya | zaro: we don't expect storyboard as an entrance point: http://docs.openstack.org/infra/manual/developers.html#getting-started | 14:11 |
| fungi | i'm curious to know whether travis-ci is free software, or merely a hosted proprietary service like github | 14:11 |
| sandywalsh | both | 14:11 |
| fungi | oh, cool, so you can actually download and run your own travis instance? | 14:12 |
| sandywalsh | yep | 14:12 |
| anteaya | zaro: can you find out a bit more about how he decided storyboard was the entry point for openstack contribution? we need to add a patch to our docs then, both the manual and storyboard | 14:12 |
| sandywalsh | the "runs on openstack" thing is nice, but not a selection criteria. My concern is getting patches blocked due to higher priority projects. | 14:12 |
| *** eglynn-office is now known as eglynn-openstack | 14:13 | |
| sandywalsh | (when something breaks, what gets fixed first) | 14:13 |
| *** eglynn-openstack is now known as eglynn-officeafk | 14:13 | |
| fungi | sandywalsh: yep, totally understand. this time in the cycle we tend to be spread a lot more thin than usual | 14:13 |
| sandywalsh | fungi: yeah, you guys have a lot on your shoulders ... hate to pile more on there. | 14:13 |
| *** bnemec is now known as beekneemech | 14:13 | |
| fungi | playing in the same sandbox as integrated openstack projects is not without some drawbacks | 14:13 |
| sandywalsh | are there documented hooks we can tap into? | 14:14 |
| fungi | sandywalsh: at the moment, the only real event stream is the one gerrit provides, but it is well-documented yes | 14:14 |
| anteaya | sandywalsh: http://ci.openstack.org/third_party.html#reading-the-event-stream | 14:15 |
| sandywalsh | fungi: thanks ... I'll start there. Most likely we'll just stick with the norm. Just want to evaluate the options. | 14:15 |
| *** rmcall has quit IRC | 14:16 | |
| fungi | sandywalsh: yeah, what anteaya just linked. you could probably build some sort of bridge which could trigger travis and then report in on your changes (but the infra team would of course appreciate your development time a lot more if it were focused on easing our burden rather than just sidestepping it) | 14:16 |
| sandywalsh | fungi: ah, cool ... now I'm thinking about a gerrit -> stacktach event bridge. We could do some cool stuff with that feed. | 14:16 |
| sandywalsh | anteaya: thanks | 14:16 |
| anteaya | sandywalsh: yes we could | 14:16 |
| sandywalsh | fungi: fair point | 14:17 |
| anteaya | sandywalsh: if there is any support I can provide do ask | 14:17 |
| sandywalsh | will do | 14:17 |
| *** dims has quit IRC | 14:18 | |
| *** dims has joined #openstack-infra | 14:18 | |
| *** flaviof is now known as flaviof_zzz | 14:19 | |
| *** pcrews has joined #openstack-infra | 14:19 | |
| *** rmcall has joined #openstack-infra | 14:20 | |
| *** vigneshvar has joined #openstack-infra | 14:21 | |
| *** dims has quit IRC | 14:22 | |
| *** dims__ is now known as dimsum | 14:22 | |
| *** dimsum is now known as dimsum_ | 14:22 | |
| anteaya | sandywalsh: oops: https://review.openstack.org/#/c/121109/1/modules/openstack_project/files/zuul/layout.yaml | 14:23 |
| *** jcoufal-mobile has quit IRC | 14:23 | |
| *** dtroyer has joined #openstack-infra | 14:25 | |
| openstackgerrit | Sandy Walsh proposed a change to openstack-infra/config: notification_utils: utility library for working with OpenStack notifications. https://review.openstack.org/121109 | 14:25 |
| openstackgerrit | Sandy Walsh proposed a change to openstack-infra/config: notification_utils: utility library for OpenStack notifications. https://review.openstack.org/121109 | 14:25 |
| *** radez_g0n3 is now known as radez | 14:25 | |
| *** rlandy has quit IRC | 14:26 | |
| EmilienM | sandywalsh: fungi: afik, travis is not open-source: https://travis-ci.com/plans (the Open-Source plan, is the SaaS platform, travis-ci.org) | 14:27 |
| *** homeless has joined #openstack-infra | 14:28 | |
| *** rwsu has quit IRC | 14:28 | |
| anteaya | sandywalsh: you have python-jobs listed but you don't use them: https://review.openstack.org/#/c/121109/3/modules/openstack_project/files/jenkins_job_builder/config/projects.yaml | 14:28 |
| sandywalsh | EmilienM: https://github.com/travis-ci | 14:28 |
| sandywalsh | anteaya: is that not needed for the merge check? | 14:29 |
| anteaya | sandywalsh: no | 14:29 |
| anteaya | merge check is defined in zuul/layout.yaml | 14:29 |
| anteaya | you don't need python-jobs invoked in projects.yaml for merge check | 14:29 |
| sandywalsh | anteaya: does it hurt to have it? We will likely be turning on gate tests in the coming week(s) | 14:29 |
| fungi | sandywalsh: EmilienM: and in the readme "Distributed under the MIT license, the same as other Travis CI projects." so sounds like it's free software | 14:29 |
| anteaya | now setting up your node is great for later | 14:29 |
| *** mestery has quit IRC | 14:30 | |
| EmilienM | sandywalsh: fungi indeed, my bad | 14:30 |
| *** wenlock has quit IRC | 14:30 | |
| *** mestery has joined #openstack-infra | 14:30 | |
| anteaya | sandywalsh: while it doesn't really hurt there is a wave amounst reviewers to only have the tests in projects.yaml that you use in layout.yaml | 14:30 |
| *** adalbas has quit IRC | 14:30 | |
| anteaya | sandywalsh: so you can make a change to both files when you actually call the jobs in layout.yaml | 14:31 |
| anteaya | sandywalsh: I'm noticing your other repos don't have python-jobs in projects.yaml | 14:31 |
| *** zz_jgrimm is now known as jgrimm | 14:31 | |
| sandywalsh | anteaya: I guess my real question is "am I going to be -1'ed for that" :) | 14:31 |
| anteaya | sandywalsh: well I have -1'd two other patches for it | 14:31 |
| anteaya | since it was brought to my attention by someone I am mentoring in their reviews | 14:32 |
| anteaya | and he is correct | 14:32 |
| *** jungleboyj has joined #openstack-infra | 14:32 | |
| anteaya | so I would be inconsistent if I didn't | 14:32 |
| anteaya | and you might get dinged by someone else | 14:32 |
| sandywalsh | ok, stand by for a flurry | 14:32 |
| *** dtroyer has quit IRC | 14:32 | |
| *** dtroyer has joined #openstack-infra | 14:33 | |
| * anteaya stands by for a flurry of improved patches | 14:33 | |
| jungleboyj | A question for you infra experts ... Is there anyway we can move https://review.openstack.org/#/c/120907/ up the queue? It is making a mess of Cinder gate jobs right now. | 14:33 |
| *** shashankhegde has joined #openstack-infra | 14:33 | |
| *** koolhead17 has quit IRC | 14:33 | |
| *** david-lyle has joined #openstack-infra | 14:33 | |
| anteaya | jungleboyj: this patch is causing the problem or fixing the problem? | 14:33 |
| jungleboyj | anteaya: Fixing the problem. | 14:33 |
| anteaya | kk | 14:33 |
| anteaya | fungi: ^ | 14:34 |
| *** Sincler has joined #openstack-infra | 14:34 | |
| jungleboyj | anteaya: Good morning by the way. :-) It is the change to block sqlalchemy-migrate 0.9.2 | 14:34 |
| openstackgerrit | Sandy Walsh proposed a change to openstack-infra/config: notification_utils: utility library for OpenStack notifications. https://review.openstack.org/121109 | 14:35 |
| anteaya | jungleboyj: good morning | 14:35 |
| jungleboyj | anteaya: Been queued for 14 hours. | 14:35 |
| anteaya | jungleboyj: :( | 14:35 |
| anteaya | sandywalsh: so far we have a flurry of one | 14:36 |
| fungi | jungleboyj: i'll move it to the front of the gate shortly, on the grounds that there do seem to be multiple other cinder changes queued ahead of it which are going to cause gate resets and waste additional resources needlessly if i don't | 14:37 |
| sandywalsh | anteaya: seems so ... thought I had more. Not so bad. I'm getting good at flying through the repos now. | 14:37 |
| jungleboyj | fungi: You the man! Thank you! | 14:38 |
| anteaya | sandywalsh: yay | 14:38 |
| anteaya | he is the man | 14:38 |
| *** mestery has quit IRC | 14:39 | |
| *** Longgeek_ has quit IRC | 14:39 | |
| *** mestery has joined #openstack-infra | 14:39 | |
| *** lttrl has quit IRC | 14:40 | |
| anteaya | sandywalsh: I believe you have one source repo with two branches and one source repo with one branch and one tag | 14:40 |
| *** datsun180b has joined #openstack-infra | 14:40 | |
| sandywalsh | anteaya: thought they were all just 'master' ... lemme recheck | 14:41 |
| anteaya | sandywalsh: on those repos, and if there are any others with more than just one branch on the source repo, you can comment on those patches that this source repo status is intended | 14:41 |
| anteaya | sandywalsh: one has master and swift branches | 14:41 |
| sandywalsh | anteaya: k, sec | 14:41 |
| anteaya | sandywalsh: and this one has a tag: https://github.com/stacktach/stackdistiller | 14:41 |
| *** dizquierdo has joined #openstack-infra | 14:42 | |
| sandywalsh | anteaya: that's fine | 14:42 |
| sandywalsh | anteaya: I check the others too | 14:42 |
| anteaya | sandywalsh: can you comment on the patch that that is how you want to consume the source repo | 14:42 |
| anteaya | so reviewers know it isn't a mistake you will ask infra to fix after the repo is created? | 14:43 |
| *** emagana has joined #openstack-infra | 14:43 | |
| *** SumitNaiksatam has quit IRC | 14:43 | |
| anteaya | if memory serves, there were just two patches that had anything other than one branch | 14:43 |
| openstackgerrit | Mike Heald proposed a change to openstack-dev/pbr: Retry the integration setup on connection error https://review.openstack.org/121093 | 14:44 |
| sandywalsh | anteaya: nuked the sandbox swift branch | 14:44 |
| *** SumitNaiksatam has joined #openstack-infra | 14:44 | |
| *** hdd has joined #openstack-infra | 14:44 | |
| anteaya | sandywalsh: good call | 14:45 |
| *** adalbas has joined #openstack-infra | 14:45 | |
| anteaya | sandywalsh: add a comment to that patch to inform subsequent reviewers please | 14:47 |
| sandywalsh | anteaya: about the tag? | 14:47 |
| *** tonytan4ever has joined #openstack-infra | 14:47 | |
| anteaya | sandywalsh: yes and that you nuked teh swift branch | 14:47 |
| fungi | okay, all the changes at the front of the gate which were going to merge have, and the ceilo change is on its way to failing on gate-tempest-dsvm-neutron-full so i'm promoting the cinder change now | 14:48 |
| fungi | jungleboyj: 120907,1 is now next in line for the gate | 14:49 |
| coolsvap | hello all, i had a query, do we enable multiple cinder backends in the default devstack gate? or any specific gate? | 14:49 |
| jungleboyj | fungi: Thank you. | 14:50 |
| fungi | coolsvap: not sure. might be a more appropriate question for #openstack-qa | 14:50 |
| *** jaypipes is now known as leakypipes\ | 14:50 | |
| *** leakypipes\ is now known as leakypipes | 14:50 | |
| *** jp_at_hp has quit IRC | 14:50 | |
| jungleboyj | fungi: Just for my edification. We would have had cinder gate jobs failing without that change. Right? | 14:50 |
| fungi | jungleboyj: sounds like yes | 14:51 |
| jungleboyj | fungi: Other guys in the cinder channel weren't so sure. | 14:51 |
| *** wenlock has joined #openstack-infra | 14:51 | |
| fungi | jungleboyj: cinder unit tests are currently failing in the check pipeline, yes? | 14:51 |
| jungleboyj | fungi: Everything in check is failing that way right now. | 14:51 |
| jungleboyj | fungi: Yes. | 14:51 |
| fungi | jungleboyj: yeah, the ones in the gate ahead of that version pin would jave done as well | 14:51 |
| jungleboyj | fungi: Ok, that is what I thought. | 14:52 |
| fungi | coolsvap: you could also browse http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/jenkins_job_builder/config/devstack-gate.yaml and see what jumps out at you | 14:52 |
| *** armax has joined #openstack-infra | 14:52 | |
| tristanC | fungi: jungleboyj: I think we also need that fixing change for stable/havana: https://review.openstack.org/#/c/121063/ | 14:52 |
| fungi | coolsvap: but i don't see anything cinder-specific in our configuration there | 14:52 |
| jungleboyj | fungi: I think you are right. | 14:53 |
| jungleboyj | fungi: Let me approve that now. | 14:53 |
| *** reed_ has joined #openstack-infra | 14:53 | |
| *** morganfainberg is now known as morgan | 14:53 | |
| *** rwsu has joined #openstack-infra | 14:53 | |
| fungi | tristanC: if you want to play the vmt card, say that you have a security advisory which needs to go out and i'll happily promote that stable branch fix and whatever security-related change together | 14:54 |
| coolsvap | fungi, yeah, I'm looking at it | 14:55 |
| *** bhuvan has joined #openstack-infra | 14:55 | |
| openstackgerrit | Aidan McGinley proposed a change to openstack-infra/jenkins-job-builder: Adding support for the Powershell plugin. https://review.openstack.org/121142 | 14:56 |
| jeblair | time to start another day of moving things from one list to another list! | 14:56 |
| jungleboyj | fungi: https://review.openstack.org/#/c/120907/ is +A now. | 14:56 |
| *** zz_dimtruck is now known as dimtruck | 14:56 | |
| *** jp_at_hp has joined #openstack-infra | 14:57 | |
| *** hdd has quit IRC | 14:57 | |
| jeblair | also, i think the old puppetmaster is stuck, since i'm still seeing hosts that want to do the ssh key update | 14:57 |
| jeblair | fungi, clarkb, mordred: fyi ^ (but maybe we don't need to fix) | 14:57 |
| jeblair | (depending on how far we get today) | 14:58 |
| openstackgerrit | Aidan McGinley proposed a change to openstack-infra/jenkins-job-builder: Adding support for the Powershell plugin. https://review.openstack.org/121142 | 14:58 |
| *** hdd has joined #openstack-infra | 14:58 | |
| jungleboyj | fungi: Never mind on the stable ones. Looks like the global requirements haven't been updated. | 14:59 |
| *** emagana has quit IRC | 15:00 | |
| *** reed_ has quit IRC | 15:00 | |
| *** emagana has joined #openstack-infra | 15:00 | |
| *** reed_ has joined #openstack-infra | 15:00 | |
| *** vigneshvar has quit IRC | 15:00 | |
| *** emagana has quit IRC | 15:00 | |
| *** emagana has joined #openstack-infra | 15:01 | |
| *** yfried|afk is now known as yfried_ | 15:02 | |
| jeblair | on wiki.o.o, i'm getting this error: err: /Stage[main]/Elasticsearch/Package[elasticsearch]: Could not evaluate: Could not get latest version: Execution of '/usr/bin/dpkg-deb --show /tmp/elasticsearch-1.3.2.deb' returned 2: dpkg-deb: error: failed to read archive `/tmp/elasticsearch-1.3.2.deb': No such file or directory | 15:04 |
| jeblair | i'm assuming that's because it's just something that it can't do in noop mode | 15:05 |
| jeblair | so i think i'll just run it for real since i don't see any other errors | 15:05 |
| *** flaviof_zzz is now known as flaviof | 15:05 | |
| fungi | that would be my guess. --noop is going to refuse to download files which it will then proceed to want to pretend to do things with... catch-22 | 15:05 |
| jeblair | yep, and it's happy after running for real | 15:06 |
| *** kgiusti has quit IRC | 15:08 | |
| *** dannywilson has joined #openstack-infra | 15:08 | |
| *** Longgeek has joined #openstack-infra | 15:08 | |
| *** marun has joined #openstack-infra | 15:08 | |
| clarkb | ya that is one of the known limitations with noop and execs | 15:09 |
| *** dannywilson has quit IRC | 15:09 | |
| clarkb | and side effects you expect from the execs will probably fail | 15:09 |
| *** dannywilson has joined #openstack-infra | 15:09 | |
| *** jogo is now known as flashgordon | 15:11 | |
| *** mestery has quit IRC | 15:12 | |
| *** kgiusti has joined #openstack-infra | 15:12 | |
| *** kgiusti has quit IRC | 15:12 | |
| *** mestery has joined #openstack-infra | 15:12 | |
| *** kgiusti has joined #openstack-infra | 15:13 | |
| *** ramashri has quit IRC | 15:14 | |
| *** vigneshvar has joined #openstack-infra | 15:17 | |
| *** yfried_ is now known as yfried|afk | 15:20 | |
| *** ramashri has joined #openstack-infra | 15:20 | |
| *** reed_ is now known as reed | 15:21 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/infra-specs: Add spec for creating a test metric database https://review.openstack.org/106584 | 15:21 |
| *** reed has quit IRC | 15:21 | |
| *** reed has joined #openstack-infra | 15:21 | |
| *** kgiusti has quit IRC | 15:21 | |
| clarkb | note I think that logstash is finally digging into those very large nova unittest logs from yesterday which is why the queue exploded | 15:22 |
| *** e0ne has quit IRC | 15:22 | |
| jeblair | nice | 15:22 |
| *** dustins has joined #openstack-infra | 15:22 | |
| *** pabelanger has joined #openstack-infra | 15:23 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/infra-specs: Add spec for creating a test metric database https://review.openstack.org/106584 | 15:23 |
| clarkb | jeblair: fungi: for git.openstack.org the puppet changes are sufficient to end up with a diff in the haproxy config which will trigger an haproxy restart | 15:23 |
| *** armax has quit IRC | 15:24 | |
| *** r-daneel_ has joined #openstack-infra | 15:24 | |
| jeblair | clarkb: that will be a momentary outage, right? | 15:24 |
| clarkb | do we want to try and sneak that in today? or I can probably give it a go on sunday? I feel bad doing disruptive stuff while everyone is trying to get FFE stuff in | 15:24 |
| *** kgiusti has joined #openstack-infra | 15:24 | |
| clarkb | jeblair: yup any git connections that are live when we restart will be closed and those clients will error | 15:24 |
| *** Longgeek has quit IRC | 15:24 | |
| clarkb | something else we can try is turning off puppet, updating the config manually, performning a reload, then running puppet. But I don't think that has worked well in the past | 15:25 |
| *** r-daneel has quit IRC | 15:25 | |
| jeblair | clarkb: i think we should just take the hit. try not to do it right when the top of the gate is starting jobs :) | 15:25 |
| clarkb | ok | 15:26 |
| fungi | clarkb: well, we *could* replace the git.o.o a rr with 5 a rrs for the ip addresses of the backends... oh, except that they don't listen on the same ports | 15:26 |
| sdague | how much fail will it create? | 15:26 |
| *** r-daneel__ has joined #openstack-infra | 15:26 | |
| clarkb | fungi: ya that was my bad >_> | 15:26 |
| clarkb | fungi: lesson learned and I have a note somewhere to fix that at some point | 15:26 |
| fungi | no, it just dawned on me that wasn't going to work | 15:26 |
| clarkb | fungi: yeah, it should be made to work... but I derped | 15:27 |
| clarkb | sdague: if we time it right not much at all. It will only affect git fetches to git.o.o | 15:27 |
| fungi | perhaps we can neuter puppet's ability to restart that daemon? | 15:27 |
| fungi | just temporarily | 15:27 |
| clarkb | sdague: so if we time it when we have no nodes ready to go and other jobs are running we can mostly get away with it | 15:28 |
| *** ndipanoff has quit IRC | 15:28 | |
| clarkb | fungi: yeah I have tried in the past and it is hit and miss. I think the concat module "refreshes" in some cases even when the diff is nil | 15:28 |
| clarkb | fungi: as if it uses a cache or something | 15:28 |
| sdague | ok, just... you know, 25hr deep gate queue | 15:28 |
| fungi | bummer | 15:28 |
| clarkb | oh cool we caught up since yesterday | 15:29 |
| clarkb | it was ~32 hours last night | 15:29 |
| fungi | yeah, we've gained a few hours | 15:29 |
| *** koolhead17 has joined #openstack-infra | 15:29 | |
| fungi | i think it's powering through | 15:29 |
| *** rcarrillocruz has quit IRC | 15:29 | |
| fungi | but also having nova, cinder, et cetera broken for a good chunk of yesterday and today after the sqlam release probably also kept the queue from growing on the tail end as much as it would have otherwise | 15:30 |
| *** r-daneel_ has quit IRC | 15:30 | |
| *** emagana has quit IRC | 15:30 | |
| fungi | since fewer changes could get into the gate | 15:30 |
| *** koolhead_ has joined #openstack-infra | 15:30 | |
| flashgordon | sdague: so it looks like david-lyle patch https://review.openstack.org/#/c/120940/1 isn't ready yet | 15:31 |
| *** koolhead17 has quit IRC | 15:31 | |
| flashgordon | sdague: so we have the backup https://review.openstack.org/#/c/120938/ | 15:31 |
| *** ildikov has quit IRC | 15:31 | |
| *** ramashri has quit IRC | 15:31 | |
| *** emagana_ has joined #openstack-infra | 15:33 | |
| david-lyle | flashgordon: I think the skip patch is best for now, still a ways from a proper solution | 15:33 |
| *** amotoki_ has joined #openstack-infra | 15:33 | |
| flashgordon | david-lyle: kk | 15:33 |
| flashgordon | mtreinish: ^ tempest review | 15:33 |
| flashgordon | clarkb: btw thanks for the weechat help, its pretty nice | 15:34 |
| sdague | david-lyle: what's up with the patch? | 15:34 |
| clarkb | flashgordon: no problem | 15:34 |
| flashgordon | clarkb: the spell check support is better so that should be a big win for me :) | 15:34 |
| *** salv-orlando has joined #openstack-infra | 15:35 | |
| fungi | it *looks* like cacti.o.o is testing clean against the new master, but with all the execs it does, it's hard to be absolutely sure. no errors at least | 15:35 |
| *** mriedem has quit IRC | 15:35 | |
| *** rcarrillocruz has joined #openstack-infra | 15:35 | |
| fungi | any objections to me pressing forward with that one? | 15:35 |
| clarkb | fungi: jeblair: review.o.o and review-dev will need that patch from nibalizer to use the non deprecated variable access in erbs | 15:36 |
| clarkb | fungi: none here | 15:36 |
| *** johnthetubaguy is now known as zz_johnthetubagu | 15:36 | |
| sdague | david-lyle: so it looks like it's only touching a couple of files | 15:36 |
| openstackgerrit | A change was merged to openstack-infra/config: Fixing deprecation warnings in gerrit module https://review.openstack.org/107243 | 15:36 |
| sdague | david-lyle: http://paste.openstack.org/show/110695/ | 15:36 |
| openstackgerrit | A change was merged to openstack-infra/infra-specs: Add RSS feed https://review.openstack.org/120556 | 15:36 |
| clarkb | oh there it is. y'all are way ahead of me | 15:37 |
| *** mmaglana has joined #openstack-infra | 15:37 | |
| *** e0ne has joined #openstack-infra | 15:37 | |
| david-lyle | sdague: that patch is incomplete | 15:37 |
| sdague | david-lyle: ok. What else needs to change? | 15:37 |
| jeblair | clarkb: yeah, approved it 1st thing :) | 15:38 |
| david-lyle | I believe there's a collectstatic step that needs to happen as well | 15:38 |
| *** julim has quit IRC | 15:39 | |
| *** _nadya_ has joined #openstack-infra | 15:39 | |
| *** kmartin has joined #openstack-infra | 15:40 | |
| sdague | david-lyle: ok, do we know who the expert here might be? | 15:40 |
| *** dtantsur is now known as dtantsur|afk | 15:40 | |
| david-lyle | mrunge does the packaging for fedora, pinging him | 15:41 |
| *** julim has joined #openstack-infra | 15:42 | |
| *** dizquierdo has quit IRC | 15:42 | |
| *** dizquierdo has joined #openstack-infra | 15:43 | |
| *** lttrl has joined #openstack-infra | 15:44 | |
| *** _nadya_ has quit IRC | 15:44 | |
| flashgordon | sdague david-lyle: https://review.openstack.org/#/c/120938/ is now approved | 15:45 |
| flashgordon | so if we find a gate reset about to happen we can get that prioritized | 15:45 |
| flashgordon | sdague: thoughts? | 15:45 |
| *** todd_dsm has joined #openstack-infra | 15:46 | |
| flashgordon | sdague: in fact https://jenkins02.openstack.org/job/gate-tempest-dsvm-neutron-full/1961/console from top of gate is about to fail | 15:46 |
| *** todd_dsm has quit IRC | 15:46 | |
| fungi | flashgordon: that's unfortunate, since it's the sqlam pin to unblock cinder's unit tests | 15:49 |
| *** amotoki_ has quit IRC | 15:50 | |
| fungi | flashgordon: i can promote 120938 ahead of it i guess, to give it a second chance | 15:50 |
| flashgordon | fungi: haha ouch | 15:50 |
| flashgordon | fungi: any way to re-promote the cinder patch back to the top as well | 15:51 |
| *** rushiagr is now known as rushiagr_away | 15:52 | |
| fungi | flashgordon: yeah, just did | 15:52 |
| *** ZZelle has quit IRC | 15:53 | |
| *** mriedem has joined #openstack-infra | 15:53 | |
| *** harlowja_at_home has joined #openstack-infra | 15:53 | |
| *** ZZelle has joined #openstack-infra | 15:53 | |
| *** harlowja_home has joined #openstack-infra | 15:53 | |
| *** mriedem1 has joined #openstack-infra | 15:54 | |
| *** harlowja_home has quit IRC | 15:54 | |
| * krotscheck has a config naming conflict between a CLI parameter and a config setting :/ | 15:55 | |
| * krotscheck finds this tedious and annoying. | 15:55 | |
| sdague | krotscheck: you have old pyc files | 15:55 |
| sdague | probably | 15:55 |
| *** Guest1245 has quit IRC | 15:55 | |
| mordred | krotscheck: it's shocking that python does not have a normal library that does both CLI and config isn't it? | 15:56 |
| *** amotoki_ has joined #openstack-infra | 15:56 | |
| flashgordon | fungi: thank you | 15:56 |
| krotscheck | sdague: Naah, it’s actually us using a similar word for a different context. | 15:56 |
| krotscheck | s/similar/same/ | 15:56 |
| flashgordon | fungi: hopefully with those two patches we can help reduce the gate logjam | 15:56 |
| krotscheck | mordred: Oslo-config actually does that. | 15:56 |
| jgriffith | fungi: flashgordon :( | 15:56 |
| mordred | krotscheck: oh - you know ... we're already using oslo.config in storyboard - you could just use that instead of argparse directly | 15:56 |
| mordred | krotscheck: yes | 15:57 |
| mordred | krotscheck: sorry, you're one step ahead of me | 15:57 |
| * mordred shuts up and caffinates | 15:57 | |
| *** mriedem has quit IRC | 15:57 | |
| krotscheck | mordred: Downside is that we’re already using the “project” config term for something else" | 15:57 |
| *** arxcruz has quit IRC | 15:58 | |
| clarkb | fungi: jeblair: etherpad* are seeing the same ServerAdmin undefined diff that one of you saw yesterday. did we treat that as harmless or does a patch need writing? | 15:58 |
| *** dimtruck is now known as zz_dimtruck | 15:59 | |
| *** emagana_ has quit IRC | 15:59 | |
| jeblair | mordred: ^? | 15:59 |
| fungi | i don't remember encountering it | 15:59 |
| mordred | clarkb: the thought was a) it's harmless b) we should fix it anyway c) I don't know how to fix it | 15:59 |
| *** emagana has joined #openstack-infra | 15:59 | |
| clarkb | mordred: so I can go ahead and run it as not --noop? | 15:59 |
| *** zz_dimtruck is now known as dimtruck | 15:59 | |
| *** ashaeron has quit IRC | 16:00 | |
| jeblair | mordred: what's the deal with ipa.o.o? | 16:00 |
| clarkb | oh I see <%= scope.lookupvar("etherpad_lite::apache::serveradmin") %> is the lookup | 16:00 |
| clarkb | and that is coming back as empty string now but was 'undefined' before /me works out a patch | 16:01 |
| *** emagana has quit IRC | 16:01 | |
| *** emagana_ has joined #openstack-infra | 16:01 | |
| mordred | jeblair: sigh. I suck. it should diaf | 16:02 |
| mordred | I _keep_ meaning to delete it and keep forgetting | 16:02 |
| *** datsun180b has quit IRC | 16:02 | |
| *** morgan is now known as morganfainberg | 16:02 | |
| *** datsun180b has joined #openstack-infra | 16:02 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Add etherpadlite server admin variable https://review.openstack.org/121159 | 16:03 |
| clarkb | ok ^ that should fix it | 16:03 |
| *** hemna_ is now known as hemna | 16:04 | |
| krotscheck | mordred: Looking at this script of yours, all it does is dump a project from launchpad into a text file, yes? | 16:04 |
| jeblair | err: Could not retrieve catalog from remote server: Error 400 on SERVER: invalid byte sequence in US-ASCII at /opt/config/production/modules/openstack_project/manifests/lists.pp:1 on node lists.openstack.org | 16:04 |
| *** e0ne has quit IRC | 16:04 | |
| clarkb | did we put a snowman in that file? | 16:04 |
| jeblair | hrm. not on line 1, but i'm pretty sure there's utf8 in there | 16:05 |
| jeblair | or at least latin1 | 16:05 |
| *** marun has quit IRC | 16:05 | |
| mordred | krotscheck: no? | 16:05 |
| *** sabeen has quit IRC | 16:05 | |
| *** vigneshvar has quit IRC | 16:06 | |
| mordred | krotscheck: you're looking at launchpad_loader, right? | 16:06 |
| *** mriedem has joined #openstack-infra | 16:06 | |
| *** mriedem1 has quit IRC | 16:06 | |
| clarkb | yup 'español' | 16:06 |
| krotscheck | Right. I see main(), and do_load_models does the whole shebang, but I don’t see do_load_models getting invoked from anywhere. | 16:06 |
| ttx | hmm, the ratio of 'in use' vs. 'deleting' in nodepool is now suspicious | 16:07 |
| clarkb | jeblair: puppet bugs say that puppet will honor LANG of the env | 16:07 |
| jeblair | clarkb: i think that's latin1 | 16:07 |
| jeblair | clarkb: i think that character is latin1 | 16:07 |
| ttx | jeblair, clarkb: ^ | 16:07 |
| jeblair | clarkb: it says it's a server error, so i'm assuming that's the lang env that matters in this case? | 16:08 |
| clarkb | vim says '<ñ> 241, Hex 00f1, Octal 361' | 16:08 |
| mordred | krotscheck: oh, well that might just be me sucking | 16:08 |
| clarkb | jeblair: yup I think server side lang env is what matters | 16:08 |
| mordred | krotscheck: i was using the json dump to eyeball what it was pulling down | 16:09 |
| *** tty57 has quit IRC | 16:09 | |
| clarkb | jeblair: my lang is en_US.UTF-8 and I see the ñ properly | 16:09 |
| clarkb | though that may be vim being smart | 16:09 |
| krotscheck | mordred: Got it | 16:10 |
| jeblair | ttx: let's see if stays like that; i think there was a gate reset recently | 16:10 |
| *** pballand has joined #openstack-infra | 16:10 | |
| ttx | jeblair: ack | 16:10 |
| jeblair | clarkb: so we need to find out the lang of the apache process? | 16:11 |
| ttx | fwiw I witnessed 8 changes in the gate going at once, this morning | 16:11 |
| clarkb | jeblair: ya Ithink so. that error message implies it is US-ASCII | 16:11 |
| jeblair | ttx: woo | 16:12 |
| ttx | I almost cried | 16:12 |
| clarkb | jeblair: su - www-date then echo $LANG? | 16:12 |
| ttx | it was a beautiful thing to see | 16:12 |
| clarkb | jeblair: as an alternative we can probably use the utf8 codepoint for that character then set to utf8 instead of latin1 | 16:12 |
| *** rushiagr_away is now known as rushiagr | 16:12 | |
| jeblair | clarkb: i'm still in data-gathering mode, but the little voice in my head is already starting to say this is insane | 16:13 |
| clarkb | ya :/ | 16:13 |
| mordred | jeblair: the little voice in my head agrees with you | 16:13 |
| jeblair | it's not easy to su to that account | 16:13 |
| *** kgiusti has left #openstack-infra | 16:13 | |
| *** dmsimard is now known as dmsimard_away | 16:13 | |
| *** kgiusti has joined #openstack-infra | 16:13 | |
| *** mestery has quit IRC | 16:13 | |
| *** berendt has quit IRC | 16:14 | |
| clarkb | https://projects.puppetlabs.com/issues/11303 is the bug with the note btw | 16:14 |
| *** mestery has joined #openstack-infra | 16:14 | |
| sdague | hmmm... sphinx now always talks to the internet? - http://logs.openstack.org/92/109592/14/check/gate-nova-docs/a5d6d9b/console.html | 16:15 |
| mordred | looking | 16:15 |
| jeblair | that'll be the intersphinx link module | 16:15 |
| mordred | sdague: you know - we should turn off the intersphinx stuff | 16:15 |
| mordred | I don't think we really use it, do we? | 16:15 |
| *** sdague is now known as daguegoneit | 16:15 | |
| mordred | at least, we especially probably don't use the python reference | 16:16 |
| daguegoneit | would work for me | 16:16 |
| *** r-daneel__ has quit IRC | 16:16 | |
| daguegoneit | also, do you have an opinion on the pyc thing ? | 16:16 |
| daguegoneit | because I thought the last time it came up in irc turning it off didn't seem like a big deal | 16:16 |
| daguegoneit | but now folks are freaking out that I proposed it | 16:17 |
| clarkb | I think run_tests.sh has always deleted those files | 16:17 |
| jeblair | dhellmann: are you aware of any systemic intersphinx usage? (see ^) | 16:17 |
| clarkb | so at least in the past everyone was cool with it | 16:17 |
| jeblair | clarkb: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=723188 | 16:17 |
| uvirtbot | Debian bug 723188 in puppet-common "puppetmaster: Non-ASCII characters in manifests break catalogs" [Important,Open] | 16:18 |
| mordred | daguegoneit: what's the suggestion? just turning off pyc generation in the first place? | 16:18 |
| daguegoneit | mordred: yes | 16:18 |
| daguegoneit | in tox | 16:18 |
| mordred | daguegoneit: 100% in support of it | 16:18 |
| jeblair | clarkb: hrm. there's a suggested workaround and an indication it doesn't work :) | 16:18 |
| daguegoneit | mordred: then get in on the email thread :) | 16:18 |
| daguegoneit | http://lists.openstack.org/pipermail/openstack-dev/2014-September/045845.html | 16:19 |
| *** markmcclain has joined #openstack-infra | 16:19 | |
| clarkb | jeblair: could be they edited the wrong file | 16:19 |
| clarkb | jeblair: apache envvars vs /etc/default/puppet | 16:19 |
| *** shardy is now known as shardy_afk | 16:19 | |
| jeblair | seems like apache envars is what we'd want, yeah? | 16:19 |
| *** markmcclain has quit IRC | 16:20 | |
| clarkb | jeblair: yup | 16:20 |
| *** markmcclain has joined #openstack-infra | 16:20 | |
| clarkb | and restart apache | 16:20 |
| openstackgerrit | Mark Vanderwiel proposed a change to openstack-infra/config: Fix missing sudo on gem install https://review.openstack.org/121164 | 16:21 |
| *** patrickeast has joined #openstack-infra | 16:22 | |
| *** kgiusti has quit IRC | 16:22 | |
| *** koolhead_ has quit IRC | 16:23 | |
| *** kgiusti has joined #openstack-infra | 16:23 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 16:23 | |
| *** AJaeger has joined #openstack-infra | 16:24 | |
| jeblair | clarkb: okay i will give that a shot now | 16:24 |
| *** mestery has quit IRC | 16:24 | |
| AJaeger | fungi, clarkb: The translation changes from yesterday worked great and gave a few projects a chance to get merged! | 16:25 |
| *** mestery has joined #openstack-infra | 16:25 | |
| *** vigneshvar has joined #openstack-infra | 16:25 | |
| jeblair | clarkb: export LANG=en_US.UTF-8 | 16:25 |
| jeblair | made it happy | 16:25 |
| *** Longgeek has joined #openstack-infra | 16:25 | |
| clarkb | jeblair: awesome | 16:25 |
| jeblair | so... to puppet that | 16:25 |
| clarkb | jeblair: we should probably puppet that ;) | 16:25 |
| clarkb | does apache accept an envvars.d/ dir? that would probably be easiest | 16:26 |
| clarkb | then we don't have to worry about other things | 16:26 |
| *** jistr has quit IRC | 16:26 | |
| jeblair | clarkb: nope :/ | 16:27 |
| *** dhellman_ has joined #openstack-infra | 16:27 | |
| mordred | isn't there that apache mod_setenv ? | 16:27 |
| jeblair | mordred: that may be worth a try | 16:28 |
| *** r-daneel__ has joined #openstack-infra | 16:28 | |
| fungi | nodepool building ratio seems to only be getting worse not better... i found a bunch of nodes in a building state in dfw for more than an hour and have set them to delete successfully but there's probably something going on. i'll check the logs | 16:29 |
| *** marcoemorais has joined #openstack-infra | 16:29 | |
| jeblair | mordred: though probably as complicated, since the config file comes from the package | 16:29 |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/config: Install libxslt-devel on Fedora nodepool templates https://review.openstack.org/120480 | 16:29 |
| jeblair | the vhost config file | 16:29 |
| clarkb | fungi: I was just looking at that too. our error rate graphs aren't very bad looking though | 16:29 |
| *** Dafna has quit IRC | 16:29 | |
| * clarkb gets out of fungi's way | 16:29 | |
| *** Longgeek has quit IRC | 16:30 | |
| clarkb | we could puppet the entire envvars file | 16:30 |
| fungi | yeah, most of rax-dfw is in building state right now | 16:31 |
| *** MaxV has quit IRC | 16:32 | |
| fungi | we are still getting usable nodes out of dfw, just not particularly quickly | 16:33 |
| *** emagana_ has quit IRC | 16:33 | |
| *** ramashri has joined #openstack-infra | 16:34 | |
| jeblair | clarkb, mordred: did we not add apache2 to the puppetmaster manifest? | 16:34 |
| fungi | and there are plenty hanging out for an hour without completing boot, then get deleted by nodepool | 16:34 |
| mordred | jeblair: puppetmaster-passenger pulls it in | 16:34 |
| clarkb | fungi: that explains why they don't show up in error stats in the graphs. they go straight from building to delete | 16:35 |
| jeblair | mordred: i don't see that in the manifest | 16:35 |
| jeblair | https://review.openstack.org/#/c/120863/ | 16:36 |
| fungi | clarkb: yeah, if you look for LaunchStatusException in /var/log/nodepool/nodepool.log there are plenty | 16:36 |
| *** zehicle_at_dell has quit IRC | 16:36 | |
| *** tonytan4ever has quit IRC | 16:37 | |
| *** Ryan_Lane has joined #openstack-infra | 16:37 | |
| fungi | grep "NodeLauncher: LaunchStatusException" /var/log/nodepool/nodepool.log|cut -d' ' -f12|sort|uniq -c | 16:38 |
| *** salv-orlando has quit IRC | 16:38 | |
| *** derekh has quit IRC | 16:38 | |
| fungi | we've had 1561 launch exceptions today since the log rotated, and >99% were in rax-dfw | 16:39 |
| clarkb | I wonder if the thing roaert was talking about has gotten worse | 16:39 |
| *** k4n0 has quit IRC | 16:40 | |
| clarkb | maybe we should reduce max nodes in dfw for a bit? | 16:40 |
| *** dhellman_ has quit IRC | 16:40 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Add apache envvars file for puppetmaster https://review.openstack.org/121170 | 16:40 |
| clarkb | jeblair: I think a reasonable soluton is to splat a new complete envvars file over the top of what apache2 package installs with puppet | 16:40 |
| jeblair | clarkb: done ^ :) | 16:40 |
| fungi | i suspect that wreaks havoc with the nodepool demand calculations too, since it counts building as the same as ready, but a majority of them are dead weight which hang around for an hour and then get deleted without ever actually reaching a state where they can be put to use | 16:41 |
| clarkb | jeblair: reviewing | 16:41 |
| jeblair | fungi: well, they still count against quota | 16:41 |
| jeblair | so it's the right thing to do | 16:41 |
| fungi | yeah | 16:41 |
| jeblair | would reducing our usage help? | 16:41 |
| fungi | i'm just wondering whether we'd get better throughput by removing dfw until the problem there gets addressed | 16:42 |
| *** emagana has joined #openstack-infra | 16:42 | |
| *** tonytan4ever has joined #openstack-infra | 16:42 | |
| jeblair | what's the tldr on the thing roaert described? | 16:42 |
| clarkb | jeblair: they cannot recycle ipv4 addresses quickly enough for us | 16:42 |
| clarkb | jeblair: which is why trying to use ipv6 only was a potential workaround | 16:42 |
| fungi | something about we might see fewer errors if we only requested ipv6 addresses, except that openstack sort of assumes you will have an ipv4 address forced on you anyway | 16:43 |
| *** pkoniszewski has quit IRC | 16:43 | |
| clarkb | without knowing the entire situation reducing our churn in that region may help with ipv4 address recycling | 16:43 |
| fungi | and so there may be no good way to do that even if we did support v6-only providers | 16:43 |
| *** emagana has quit IRC | 16:43 | |
| *** dannywilson has quit IRC | 16:44 | |
| fungi | without some further modification to neutron and clients | 16:44 |
| jeblair | as long as we're over capacity, reducing usage in dfw won't significantly affect usage in other regions that are working; but if reducing usage of dfw makes it more successful, that's probably a net win | 16:44 |
| *** dannywilson has joined #openstack-infra | 16:44 | |
| clarkb | this is another thing rebuild vs delete may end up helping with | 16:45 |
| clarkb | should look into that as well as the ipv6 support | 16:45 |
| fungi | just wondering if having less-frequently-used node types tied up in building dead weight might be detrimentally impacting our ability to run some sorts of jobs | 16:45 |
| *** bhuvan has quit IRC | 16:45 | |
| jeblair | fungi: i think things will mostly average out as long as demand is high enough | 16:46 |
| jeblair | (unless dfw suddenly has a lot of capacity and the round-robin allocator puts everything in there) | 16:46 |
| *** dannywilson has quit IRC | 16:47 | |
| fungi | we have more bare-precise nodes in rax-dfw than all other providers combined, for example | 16:47 |
| *** dannywilson has joined #openstack-infra | 16:48 | |
| fungi | basically all in building and delete | 16:48 |
| jeblair | so we're currently at dfw=184/ord=55/iad=85 | 16:48 |
| jeblair | yay round robin allocations | 16:48 |
| fungi | though it did just manage to provide two bare-precise nodes into a used state now, so i guess it's not totally dead for those | 16:49 |
| jeblair | i'm trying to work out out actual quotas | 16:49 |
| fungi | py3k-precise seems to be fairly evenly distributed too, so i guess while dfw isn't really helping much, it's not especially hurting us more than not having it would | 16:49 |
| *** Hal has joined #openstack-infra | 16:50 | |
| clarkb | that is good news | 16:50 |
| *** harlowja_at_home has quit IRC | 16:50 | |
| *** Hal is now known as Guest45523 | 16:50 | |
| clarkb | so I think this boils down to whether or not dfw would be in a happier place with us requesting fewer things at once | 16:50 |
| clarkb | (for tweaking max server value) | 16:50 |
| openstackgerrit | A change was merged to openstack-infra/config: Install puppetmaster-passenger on puppetmasters https://review.openstack.org/120863 | 16:51 |
| jeblair | current: dfw=184/ord=55/iad=85 | 16:51 |
| jeblair | quota: dfw=192/ord=62/iad=93 | 16:51 |
| jeblair | so we don't actually have room to increase the other regions, we'd just be reducing | 16:52 |
| fungi | right, so the ratios probably mostly work out | 16:52 |
| jeblair | fungi: what do you mean? | 16:52 |
| jeblair | fungi: oh, about node type allocation. ok | 16:52 |
| fungi | seeing more nodes of a given type in dfw than ord and iad isn't entirely unexpected, as the quota is also much higher there | 16:52 |
| fungi | though hpcloud should still trump that by a wide margin | 16:53 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Drop 90 servers from rax-dfw https://review.openstack.org/121172 | 16:54 |
| jeblair | fungi: so most of dfw is build/delete, right? | 16:54 |
| dhellmann | jeblair: oslosphinx provides an intersphinx mapping for some openstack projects, but afaict it doesn't turn on the extension by itself | 16:54 |
| *** jlk has joined #openstack-infra | 16:55 | |
| *** vigneshvar has quit IRC | 16:55 | |
| jeblair | fungi: can you https://review.openstack.org/#/c/121170/ ? | 16:56 |
| fungi | jeblair: 63 used, 63 building, 43 delete, 5 ready | 16:56 |
| *** dannywilson has quit IRC | 16:56 | |
| fungi | jeblair: yeah, was already looking at it. is the envvars file one you lifted from the current server? | 16:56 |
| *** dannywilson has joined #openstack-infra | 16:56 | |
| clarkb | yup I checked the current file contents on the puppetmaster | 16:56 |
| fungi | agreed. sent it in now | 16:57 |
| jeblair | fungi: okay, so we're probably not going to loose anything by setting the max to 94 | 16:57 |
| fungi | jeblair: agreed | 16:57 |
| jeblair | in current state | 16:57 |
| fungi | yep, i believe that is probably the case | 16:57 |
| *** jp_at_hp has quit IRC | 16:58 | |
| fungi | if this means it can turn building servers over faster, then it may even help | 16:58 |
| jeblair | what's the deal with 111782,35 ? | 16:58 |
| jeblair | in the gate | 16:58 |
| jeblair | nova change with what looks like a deterministic py26 unit test failure that's been in queue for 1 hour | 16:59 |
| *** daya_k has quit IRC | 16:59 | |
| *** marcoemorais has quit IRC | 16:59 | |
| jeblair | is the time just wrong due to a bug in promote or something? | 16:59 |
| *** Hefeweizen has joined #openstack-infra | 17:00 | |
| *** marcoemorais has joined #openstack-infra | 17:00 | |
| jeblair | (since it's not at the top) | 17:00 |
| clarkb | jeblair: it was approved ~3 hours ago | 17:00 |
| clarkb | oh wait that aws yesterday nevermind | 17:00 |
| *** marcoemorais has quit IRC | 17:00 | |
| fungi | jeblair: yeah, there's been at least one promote since it entered the gate, and it was pulled in by a parent change which seems to cause zuul to not preserve the enqueued time | 17:00 |
| jeblair | ok, makes sense | 17:01 |
| fungi | we normally saw this back when we allowed approved parents to enqueue approved child changes | 17:01 |
| *** dannywilson has quit IRC | 17:01 | |
| fungi | and it's returned since we reenabled that behavior | 17:01 |
| *** dannywilson has joined #openstack-infra | 17:01 | |
| *** marcoemorais has joined #openstack-infra | 17:01 | |
| fungi | basically time in queue for the child changes gets reset | 17:02 |
| *** emagana has joined #openstack-infra | 17:02 | |
| fungi | i suspect there may also be a potential for a queue-hopping exploit (rebase your change on one near the top of the gate, get it approved, then wait for someone to promote an unrelated change while they're both still in the gate) but i doubt anyone would actually take advantage of that behavior | 17:03 |
| clarkb | rebasing like that means you need approval so it requires some oversight | 17:04 |
| fungi | yep, absolutely | 17:05 |
| *** ihrachyshka has quit IRC | 17:07 | |
| clarkb | fungi: when you are done debugging nodepool review on https://review.openstack.org/#/c/121159/ would be great | 17:07 |
| *** sabeen has joined #openstack-infra | 17:08 | |
| clarkb | ok /me does morning things back in a bit | 17:09 |
| *** marun has joined #openstack-infra | 17:09 | |
| *** e0ne has joined #openstack-infra | 17:10 | |
| *** bhuvan has joined #openstack-infra | 17:11 | |
| *** Adri2000 has quit IRC | 17:13 | |
| krotscheck | mordred: Did you ever decide what you were going to do with users that don’t have a valid openid provider? | 17:14 |
| *** tty57 has joined #openstack-infra | 17:14 | |
| fungi | jungleboyj: jgriffith: looks like the cinder pin made it into master now | 17:14 |
| mordred | krotscheck: can you say those same words but slightly differently? | 17:15 |
| krotscheck | mordred: DiscoveryFailure: No usable OpenID services found for https://launchpad.net/~janitor | 17:15 |
| mordred | krotscheck: yet there is a bug assigned to that person? | 17:15 |
| mordred | that's frustrating | 17:15 |
| mordred | krotscheck: so - for assignee, we can just leave those unassigned ... | 17:16 |
| mordred | krotscheck: for owner, maybe we allow owner/creator to be null? | 17:16 |
| fungi | krotscheck: mordred: might consider special-casing that one. ~janitor is a pseudo-account in lp | 17:16 |
| *** praneshp has joined #openstack-infra | 17:17 | |
| mordred | fungi: excellent point | 17:17 |
| fungi | it's not an actual user at all | 17:17 |
| clarkb | I should tell my custodian uncles that janitors are not actual people | 17:17 |
| *** melwitt has joined #openstack-infra | 17:17 | |
| clarkb | lp said so | 17:17 |
| krotscheck | Of course they’re not. They’re sanitary technicians | 17:18 |
| mordred | clarkb: we should make them a greeting card that says that and have all of openstack sign it | 17:18 |
| *** aviau has quit IRC | 17:18 | |
| krotscheck | I think we can allow nulls in all those fields. | 17:18 |
| *** grantbow has quit IRC | 17:18 | |
| *** torandu has quit IRC | 17:18 | |
| krotscheck | Are there any other pseudo accounts I need to know about? | 17:19 |
| mordred | krotscheck: what if ... (and bear with me for this being a STUPID idea)... | 17:19 |
| *** Guest55717 has quit IRC | 17:19 | |
| jeblair | we assign them all to mordred? | 17:19 |
| *** grantbow has joined #openstack-infra | 17:19 | |
| *** grantbow has joined #openstack-infra | 17:19 | |
| *** torandu has joined #openstack-infra | 17:19 | |
| fungi | krotscheck: you might ask lifeless or StevenK when they're around... i just happened to be aware of that one in particular | 17:19 |
| mordred | krotscheck: what if we had the import script alter the tables to allow nulls, then alter them back to put the constraint on when it's done | 17:19 |
| *** aviau has joined #openstack-infra | 17:19 | |
| * jeblair throws an unmaintainable database at mordred | 17:20 | |
| *** rhsu has joined #openstack-infra | 17:20 | |
| mordred | jeblair: yay! unmaintainable databases keep me in a job! | 17:20 |
| mordred | krotscheck: I retract my statement | 17:20 |
| fungi | flashgordon: daguegoneit: the tempest skip for that horizon compression concurrency issue is in now too | 17:20 |
| fungi | hopefully our gate stability will climb a bit | 17:20 |
| * krotscheck isn’t quite certain what just happened. | 17:20 | |
| * krotscheck goes back to reading about generators. | 17:20 | |
| flashgordon | fungi: yeah horizon compression was a big one so hoping that helps | 17:21 |
| jeblair | Detail: Could not find value for 'vhost_extra' at /opt/config/production/modules/meetbot/templates/vhost.erb:25 | 17:21 |
| *** dtantsur|afk has quit IRC | 17:22 | |
| jungleboyj | fungi: Thanks! | 17:22 |
| jeblair | clarkb: do i need to sprinkle '@' symbols everywhere in modules/meetbot/templates/vhost.erb ? | 17:22 |
| jungleboyj | Yeah, I had been watching that. | 17:22 |
| flashgordon | fungi: there is another gate bug fix in the gate queue btw: https://review.openstack.org/#/c/120843/ | 17:22 |
| flashgordon | fungi: although that one is much more infrequent | 17:23 |
| *** tty57 has quit IRC | 17:23 | |
| clarkb | jeblair: yup that should fix it | 17:23 |
| *** luqas has quit IRC | 17:23 | |
| fungi | flashgordon: the one you linked only impacts stable/icehouse anyway, so probably relatively infrequent given the percentage of changes it sees | 17:24 |
| flashgordon | fungi: it hits grenade actually | 17:24 |
| fungi | flashgordon: oh! | 17:24 |
| fungi | flashgordon: good point, then that would affect master changes as well | 17:24 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Add more @ signs to meetbot vhost https://review.openstack.org/121179 | 17:24 |
| flashgordon | but yes, 2 hits in 24 hours | 17:24 |
| *** sdake has joined #openstack-infra | 17:24 | |
| fungi | flashgordon: in the gate or overall? | 17:24 |
| flashgordon | fungi: overall | 17:24 |
| *** sdake has quit IRC | 17:24 | |
| *** sdake has joined #openstack-infra | 17:24 | |
| *** Adri2000 has joined #openstack-infra | 17:25 | |
| mordred | jeblair: @ symbols make it more like perl, which is what ruby wants to be anyway | 17:25 |
| fungi | ahh, if it gets worse i guess we could consider promoting it, otherwise it'll probably be applied in the next 24 hours anyway | 17:25 |
| jeblair | mordred: that i can understand | 17:25 |
| clarkb | jeblair: my only quetion about that is if we need @'s in the #{} string interpolation too | 17:25 |
| *** Adri2000 is now known as Guest24333 | 17:25 | |
| flashgordon | fungi: yeah no gate hits in over 24 hours for that one. | 17:25 |
| clarkb | jeblair: that might be a question for nibalizer | 17:25 |
| clarkb | nibalizer: ^ | 17:25 |
| *** lttrl has quit IRC | 17:25 | |
| sdake | morning | 17:26 |
| jeblair | clarkb: i looked at modules/drupal/templates/drupal.vhost.erb | 17:26 |
| *** tty57 has joined #openstack-infra | 17:26 | |
| jeblair | clarkb: and now that i look closer... i see we did not do it in one place and did it in another | 17:26 |
| clarkb | sdake: good morning | 17:27 |
| ekarlso- | phschwartz: sorry to nag, goes with the gh stuff ? | 17:27 |
| *** craigbr has joined #openstack-infra | 17:28 | |
| mordred | phschwartz: we want your codes!!! and also all of your chickens!!! and, perhaps, do you have a milk cow we could borrow for a while? | 17:30 |
| jeblair | clarkb: i'm testing that change in a local env | 17:30 |
| nibalizer | @thing in templates? | 17:30 |
| nibalizer | yea if you #{@foo} | 17:31 |
| nibalizer | do the @ thing | 17:31 |
| openstackgerrit | A change was merged to openstack-infra/config: Add etherpadlite server admin variable https://review.openstack.org/121159 | 17:31 |
| jeblair | #{@name} vs #{name} doesn't seem to make a difference according to noop. but i'll do the @ thing. | 17:31 |
| nibalizer | it makes it a 'global' variable in ruby i guess and the way the templating work is i guess the variables get injected as globals | 17:31 |
| crinkle | *instance variable | 17:32 |
| nibalizer | clarkb: ya something ruby mumble mumble | 17:32 |
| nibalizer | er crinkle ^ | 17:32 |
| crinkle | ya that | 17:33 |
| fungi | out of curiosity, was the "puppet agent --server puppetmaster.openstack.org" command (after removing from old list and before adding to new list) a typo? should that have had --test in it? doesn't seem to return, just ends up running as a daemon | 17:33 |
| *** harlowja_away is now known as harlowja | 17:34 | |
| *** jlibosva has quit IRC | 17:34 | |
| jeblair | nibalizer: wait a sec | 17:35 |
| fungi | hrm, looks like maybe it finally exited on jenkins-dev where i ran it last night, so maybe just takes time | 17:35 |
| jeblair | nibalizer: look at line 9 of https://review.openstack.org/#/c/121179/1/modules/meetbot/templates/vhost.erb | 17:35 |
| nibalizer | fungi: yea you likely wanted --test and --noop | 17:35 |
| nibalizer | jeblair: that looks right | 17:36 |
| fungi | nibalizer: the --test --noop was for testing it first i thought, but then after we takt it out of the old list we had noted to run without either of those | 17:36 |
| *** penguinRaider has joined #openstack-infra | 17:36 | |
| nibalizer | fungi: oh, hrm, yea you wanted --test on both sides i think | 17:36 |
| nibalizer | since we dont run puppet daemon anymore | 17:37 |
| fungi | okay, i wonder why that eventually exited on jenkins-dev then | 17:37 |
| jeblair | nibalizer: okay, we're getting conflicting information about #{name} | 17:37 |
| fungi | i assume everyone else was typing it from memory rather than cutting and pasting from the etherpad | 17:37 |
| jesusaurus | yeah --test doesnt actually mean 'test', it means 'no-daemon' | 17:37 |
| jeblair | nibalizer: name in that case is not passed into the template, it's a local var from the for loop | 17:37 |
| jeblair | nibalizer: in that case, should it have @ or not? | 17:37 |
| nibalizer | jeblair: in that case name should not have a @ | 17:38 |
| nibalizer | since its not being passed in from puppet, it was declared inside the template | 17:38 |
| jeblair | nibalizer: okay. but on line 11, we should put an @ in because it's passed in, right? | 17:38 |
| nibalizer | jeblair: yes line 11 should have @ | 17:39 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Add more @ signs to meetbot vhost https://review.openstack.org/121179 | 17:39 |
| *** ZZelle_ has joined #openstack-infra | 17:39 | |
| sandywalsh | craigbr: thanks for the eyeballs! | 17:40 |
| *** emagana has quit IRC | 17:41 | |
| *** emagana has joined #openstack-infra | 17:41 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Add more @ signs to logs-dev and status vhosts https://review.openstack.org/121181 | 17:41 |
| jeblair | clarkb: can you apply your serveradmin solution to storyboard.o.o? | 17:42 |
| jeblair | mordred: i'm stealing your hosts since you're busy :) | 17:43 |
| mordred | jeblair: please do | 17:43 |
| jeblair | clarkb: also, the etherpad changes have merged; do you want to pick those back up? | 17:44 |
| *** torandu has quit IRC | 17:44 | |
| jeblair | okay, i'll do the serveradmin thing for storyboard | 17:45 |
| *** rushiagr is now known as rushiagr_away | 17:45 | |
| craigbr | sandywalsh: np. Only one left to do. You left the repo name out of the commit for https://review.openstack.org/#/c/120864/. | 17:45 |
| *** emagana has quit IRC | 17:46 | |
| *** penguinRaider has quit IRC | 17:46 | |
| sandywalsh | craigbr: whoops, fixing | 17:46 |
| *** torandu has joined #openstack-infra | 17:46 | |
| nibalizer | jeblair: i put a review up for the serveradmin thing on storyboard | 17:46 |
| jeblair | nibalizer: link? | 17:46 |
| nibalizer | https://review.openstack.org/#/c/120944/ | 17:47 |
| openstackgerrit | Sandy Walsh proposed a change to openstack-infra/config: Winchester - A notification event processing library based on persistant streams. https://review.openstack.org/121182 | 17:47 |
| jeblair | nibalizer: i will paste it into the etherpad | 17:47 |
| openstackgerrit | Sandy Walsh proposed a change to openstack-infra/config: Shoebox: Binary data archiving library - supports uploading to object storage. https://review.openstack.org/120864 | 17:47 |
| *** SumitNaiksatam has quit IRC | 17:48 | |
| jeblair | nibalizer: can you explain why it's bad to compare to undefined? | 17:48 |
| openstackgerrit | Sandy Walsh proposed a change to openstack-infra/config: Shoebox: Binary data archiving library - with object storage upload. https://review.openstack.org/120864 | 17:48 |
| *** dizquierdo has quit IRC | 17:48 | |
| fungi | review-dev is way, way, way behind in puppetingness as puppet agent has been disabled on it for no idea how long | 17:48 |
| nibalizer | jeblair: :undef is a fine thing to compare to, but :undefined is a literally undefined variable, i.e. you could just as well compare to :lazors | 17:49 |
| fungi | zaro: this will presumably need to be reenabled for us to migrate it to the new puppet master. fair warning any local changes will probably be reverted | 17:49 |
| jeblair | nibalizer: but testing truthiness of a string is better? | 17:49 |
| openstackgerrit | Sandy Walsh proposed a change to openstack-infra/config: Shoebox: Binary data archiving library. https://review.openstack.org/120864 | 17:49 |
| nibalizer | changing it the way i did gives you the same resuluts on 2.7 and on 3 that you want, which is nothing to be printed out for serveradmin | 17:49 |
| jeblair | krotscheck: what's your preference for spelling server admin? "serveradmin" or "server_admin"? | 17:49 |
| *** marcoemorais has quit IRC | 17:49 | |
| sandywalsh | craigbr: sorry, had to do a few patches to get the subject line length to fit. | 17:50 |
| krotscheck | underscore | 17:50 |
| *** marcoemorais has joined #openstack-infra | 17:50 | |
| *** marcoemorais has quit IRC | 17:50 | |
| *** marcoemorais has joined #openstack-infra | 17:50 | |
| *** marcoemorais has quit IRC | 17:51 | |
| *** marcoemorais has joined #openstack-infra | 17:51 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/puppet-storyboard: Spell 'server_admin' consistently https://review.openstack.org/121186 | 17:51 |
| jeblair | krotscheck: nibalizer: let's do ^ too | 17:51 |
| jeblair | since that would also cause the error | 17:51 |
| krotscheck | Yay consistency! | 17:52 |
| jeblair | krotscheck: https://review.openstack.org/#/c/121186/ lgty too? | 17:52 |
| jeblair | krotscheck: oops, meant to ask about https://review.openstack.org/#/c/120944/ | 17:53 |
| clarkb | ok +2 on the storyboard server admin change | 17:53 |
| krotscheck | jeblair: Hrmmm… see, that one bit me a few weeks back, but I don’t recall enough about the circumstances. | 17:54 |
| jeblair | krotscheck: i can test out the stack real quick in noop and see if there's a diff | 17:54 |
| krotscheck | I think I was trying to make it all work on precise and trusty at the same time. | 17:54 |
| krotscheck | Without using the puppet ppa | 17:55 |
| jeblair | oh, we use the puppet ppa | 17:55 |
| krotscheck | Gothca. | 17:55 |
| krotscheck | So that should be good. | 17:55 |
| openstackgerrit | A change was merged to openstack-infra/config: Add apache envvars file for puppetmaster https://review.openstack.org/121170 | 17:56 |
| jeblair | fungi: can you aprv https://review.openstack.org/#/c/120944/ and https://review.openstack.org/#/c/121186/ ? | 17:56 |
| jeblair | nibalizer: if i want local modules for a test env, should i reverse the modulepath list order? | 17:57 |
| jeblair | nibalizer: eg, should it be /opt/config/corvus/modules:modules:$basemodulepath ? | 17:57 |
| *** shashankhegde has quit IRC | 17:57 | |
| rcarrillocruz | hi there | 17:58 |
| rcarrillocruz | could i get some more reviews on https://review.openstack.org/#/c/120381/ ? | 17:58 |
| rcarrillocruz | pls :-) | 17:58 |
| nibalizer | jeblair: im not sure, will require some fiddling i think | 17:58 |
| nibalizer | if you make a /etc/puppet/environments/corvus/environment.conf you can do anything you want | 17:59 |
| *** amotoki has quit IRC | 17:59 | |
| jeblair | nibalizer: yeah, that's what i'm talking about :) | 17:59 |
| nibalizer | so you might need to reverse it like you said, or just remove basemoudle path and manage all of corvus/modules by hand | 17:59 |
| jeblair | nibalizer: i'm just starting with the modulepath we're using for production, and it seems backwards for supporting local overrides | 17:59 |
| jeblair | nibalizer: however, it's probably good for production | 17:59 |
| nibalizer | im not sure if puppetmodules use the same kind of first-find-bail that other things do with PATH variables | 18:00 |
| jeblair | nibalizer: because it would mean that as we create new separate modules, they will override what we have in the config tree | 18:00 |
| jeblair | nibalizer: oh jeez. | 18:00 |
| nibalizer | i mean they might, but we shouldn't assume the puppet devs have used sanity | 18:00 |
| fungi | i certainly will never let myself be accused of assuming that | 18:00 |
| jeblair | nibalizer: you are right. it's probably a binary search starting from the middle because moustache. | 18:00 |
| *** SumitNaiksatam has joined #openstack-infra | 18:01 | |
| *** shashankhegde has joined #openstack-infra | 18:02 | |
| clarkb | jeblair: :{ | 18:02 |
| *** johnthetubaguy is now known as zz_johnthetubagu | 18:02 | |
| openstackgerrit | A change was merged to openstack-infra/config: Drop 90 servers from rax-dfw https://review.openstack.org/121172 | 18:03 |
| openstackgerrit | A change was merged to openstack-infra/puppet-storyboard: Template was comparing to :undefined https://review.openstack.org/120944 | 18:03 |
| jeblair | + ServerAdmin undef | 18:03 |
| jeblair | + | 18:03 |
| clarkb | jeblair: fwiw that nova change you were looking at with the python26 failure just passed python26 after a gate reset... so not very deterministic it seems | 18:04 |
| jeblair | clarkb: wow, that's bad. | 18:04 |
| *** emagana has joined #openstack-infra | 18:05 | |
| *** e0ne has quit IRC | 18:05 | |
| *** julim has quit IRC | 18:07 | |
| *** armax has joined #openstack-infra | 18:07 | |
| *** cdent has quit IRC | 18:07 | |
| jeblair | nibalizer, krotscheck: i think we want != :undef | 18:08 |
| krotscheck | Fall down go boom? | 18:08 |
| jeblair | + ServerAdmin undef | 18:08 |
| jeblair | the current state with those patches does that ^ | 18:08 |
| jeblair | i tested with != :undef and there's no diff from current | 18:09 |
| jeblair | with != :undefined it also adds ServerAdmin undef | 18:09 |
| nibalizer | jeblair: okay != :undef works for me | 18:09 |
| nibalizer | shall i update my review? | 18:09 |
| jeblair | maybe we could do ='' instead of =undef in the assignment | 18:10 |
| jeblair | nibalizer: no, your change has merged | 18:10 |
| *** e0ne has joined #openstack-infra | 18:10 | |
| openstackgerrit | Ian Cordasco proposed a change to openstack-infra/elastic-recheck: Add elastic-recheck query for bug 1368782 https://review.openstack.org/121126 | 18:10 |
| uvirtbot | Launchpad bug 1368782 in openstack-ci "Fixed IPs Test Build Errors Out" [Undecided,New] https://launchpad.net/bugs/1368782 | 18:10 |
| jeblair | nope, undef it is | 18:11 |
| *** sdake_ has joined #openstack-infra | 18:11 | |
| fungi | review-dev completed the transition to the new puppetmaster cleanly and gerrit is still running on it... should i go ahead and do review.o.o or save that for a quieter period? | 18:12 |
| *** sdake has quit IRC | 18:12 | |
| openstackgerrit | Spencer Krum proposed a change to openstack-infra/puppet-storyboard: Compare values to :undef to test for existence https://review.openstack.org/121192 | 18:13 |
| nibalizer | jeblair: like that? | 18:13 |
| jeblair | fungi: let's do it now | 18:13 |
| *** marun has quit IRC | 18:13 | |
| fungi | sounds good. starting it now | 18:13 |
| jeblair | nibalizer: please base that on my change | 18:14 |
| clarkb | I am going to do etherpad.o.o this will bounce apache (so you may need to refresh the puppet3 switch pad | 18:14 |
| nibalizer | jeblair: okay | 18:15 |
| *** Longgeek has joined #openstack-infra | 18:15 | |
| fungi | noted | 18:15 |
| *** pelix has quit IRC | 18:15 | |
| fungi | okay, review.o.o is done | 18:16 |
| clarkb | etherpad is done | 18:16 |
| fungi | gerrit on review.o.o is still running and responding normally | 18:17 |
| fungi | etherpad seems fine as well | 18:17 |
| *** flaviof is now known as flaviof_zzz | 18:17 | |
| openstackgerrit | Spencer Krum proposed a change to openstack-infra/puppet-storyboard: Compare values to :undef to test for existence https://review.openstack.org/121192 | 18:17 |
| *** dimtruck is now known as zz_dimtruck | 18:18 | |
| clarkb | nibalizer: see comment on that change | 18:18 |
| *** tonytan4ever has quit IRC | 18:18 | |
| nibalizer | gah! every time! | 18:19 |
| *** Longgeek has quit IRC | 18:19 | |
| * nibalizer ponders writing a pre-commit hook to test for that kind of thing | 18:19 | |
| *** shashankhegde has quit IRC | 18:19 | |
| openstackgerrit | Spencer Krum proposed a change to openstack-infra/puppet-storyboard: Compare values to :undef to test for existence https://review.openstack.org/121192 | 18:20 |
| *** ramashri has quit IRC | 18:20 | |
| *** _nadya_ has joined #openstack-infra | 18:21 | |
| *** jungleboyj has quit IRC | 18:23 | |
| jeblair | okay, those are all approved; the current storyboard state isn't any better on the old puppetmaster at this point, so i've gone ahead and moved it | 18:24 |
| *** tty57 has quit IRC | 18:24 | |
| clarkb | kk | 18:24 |
| clarkb | what about eavesdrop? | 18:24 |
| clarkb | nevermind I see the change | 18:24 |
| * krotscheck blinks | 18:24 | |
| clarkb | I missed hte consensus on @'s | 18:24 |
| krotscheck | Wait a sec, storyboard is down? Here I was eyeballs deep in the migration script. | 18:25 |
| clarkb | but sb has filled me in | 18:25 |
| clarkb | krotscheck: no, I don't think so | 18:25 |
| fungi | that list got plowed through surprisingly quickly | 18:25 |
| krotscheck | Oh, good | 18:25 |
| *** yamahata has quit IRC | 18:25 | |
| clarkb | krotscheck: just that hte puppet was equally broken with both masters | 18:25 |
| jeblair | it's actually slightly less broken on the new master right now, due to change sequencing | 18:25 |
| fungi | once we get these last few stragglers done, i suppose next wee we start going through and upgrading puppet on all the servers? | 18:25 |
| jeblair | though it may get worse then better again :) | 18:25 |
| fungi | s/wee/week/ | 18:25 |
| clarkb | fungi: yup | 18:26 |
| clarkb | though we might be able to start on that today? | 18:26 |
| jeblair | i was planning on it | 18:26 |
| fungi | we could probably make puppet upgrade puppet, but that strikes me as marginally dangerous | 18:26 |
| *** e0ne has quit IRC | 18:26 | |
| clarkb | jeblair: fungi: looking at the gate now do you think it is a good time to puppet git.o.o? | 18:26 |
| jeblair | clarkb: i think so | 18:26 |
| jeblair | (was just checking that myself) | 18:26 |
| clarkb | ok I will do that now | 18:26 |
| fungi | seems safe | 18:26 |
| clarkb | does someone else want to edit the hostfiles? | 18:26 |
| fungi | as long as it doesn't end up taking too long | 18:27 |
| fungi | sure, i'll edit | 18:27 |
| fungi | it's out of the old file now | 18:27 |
| clarkb | puppet is running now | 18:27 |
| fungi | assuming it's clear, i'll toss it into the new file | 18:28 |
| clarkb | all done | 18:28 |
| * clarkb tests git.o.o | 18:28 | |
| fungi | added to the new list | 18:28 |
| fungi | removed from the etherpad | 18:28 |
| *** AJaeger has quit IRC | 18:28 | |
| *** AJaeger has joined #openstack-infra | 18:29 | |
| clarkb | all three protocol worked cloning swift | 18:29 |
| clarkb | so I think we are good | 18:29 |
| fungi | so as soon as we have our liberal sprinkling of @ signs for eavesdrop... | 18:29 |
| jeblair | eta in 0 mins | 18:29 |
| openstackgerrit | A change was merged to openstack-infra/config: Add more @ signs to meetbot vhost https://review.openstack.org/121179 | 18:30 |
| fungi | moar @ signs | 18:30 |
| nibalizer | @fungi yes | 18:31 |
| jeblair | working on eavesdrop now | 18:31 |
| * fungi cringes | 18:31 | |
| *** ramashri has joined #openstack-infra | 18:31 | |
| fungi | i guess next i can add a copy of the new hostfile into the etherpad and we can divvy up pin edits/package upgrades? | 18:31 |
| jeblair | fungi: ++ | 18:32 |
| jeblair | eavesdrop is done | 18:32 |
| clarkb | fungi: sounds like a perfect idea | 18:32 |
| *** amotoki_ has quit IRC | 18:33 | |
| fungi | list is in there. grab what you like | 18:33 |
| jeblair | someone should write a sed command for the pin and paste it in there | 18:34 |
| jeblair | so we're all doing the same thing there | 18:34 |
| clarkb | do we haev an example for what it should look like? | 18:34 |
| clarkb | ++ to sed that is even better | 18:34 |
| fungi | adding now | 18:34 |
| *** IAX has quit IRC | 18:36 | |
| *** iax7 has quit IRC | 18:37 | |
| clarkb | fungi: jeblair is that sed sufficient? we pin other packages related to puppet too | 18:37 |
| clarkb | like facter | 18:37 |
| fungi | looks like new puppet is going to pull in "hiera libjson-ruby puppet-common ruby-json" | 18:37 |
| fungi | yeah, so what newer versions of facter et all do we want to be using? | 18:38 |
| fungi | nibalizer: ^ ? | 18:38 |
| nibalizer | oh hrm | 18:38 |
| nibalizer | is there a puppet-hiera package? | 18:38 |
| clarkb | I think facter should be 2.* right? | 18:38 |
| nibalizer | installed now? | 18:38 |
| nibalizer | need to nix that | 18:38 |
| fungi | puppetdb-terminus and facter are pinned to 1.* | 18:38 |
| nibalizer | you want puppetdb-terminus and facter to go to 2.* | 18:39 |
| nibalizer | you know | 18:39 |
| nibalizer | what you could do | 18:39 |
| nibalizer | well nevermind | 18:39 |
| nibalizer | i was gonna say comment out the whole file and let apt decide what to do | 18:39 |
| clarkb | and do we need to pin puppet to 3.6.*? | 18:39 |
| nibalizer | clarkb: i'd like to | 18:39 |
| *** shashankhegde has joined #openstack-infra | 18:40 | |
| dvorak | hrm, is there any way to have a default for a macro parameter in jenkins job builder? | 18:40 |
| nibalizer | technically we can go 3.7 but our master isn't 3.7 so thats not really advisable | 18:40 |
| clarkb | nibalizer: ya so we should pin to 3.6* | 18:40 |
| fungi | 3.6.* | 18:40 |
| fungi | so i've added that and the terminus/facter pin to 2.* | 18:41 |
| nibalizer | you might, instaed of complex sed, just heredoc in a new file | 18:41 |
| jeblair | what what? | 18:41 |
| jeblair | i thought we were pinning to 3.x | 18:42 |
| jeblair | why are we changing our minds? | 18:42 |
| *** ling-yun-1 has quit IRC | 18:42 | |
| *** ling-yun has joined #openstack-infra | 18:42 | |
| fungi | groupthink? but yes i explicitly remember yesterday the plan was 3.* | 18:42 |
| jeblair | these are the pins we agreed on earlier: https://review.openstack.org/#/c/120573/1 | 18:42 |
| clarkb | so the thing to be aware of is you don't want your master to be older than your agents | 18:43 |
| *** Sincler has quit IRC | 18:43 | |
| nibalizer | I remember that too(pinning to 3.x) | 18:43 |
| clarkb | if we are ok with the puppetmaster having that pin too so that it moves along with the clients then + | 18:43 |
| clarkb | er +1 | 18:43 |
| jeblair | clarkb: i'm pretty sure it will | 18:43 |
| jeblair | it's in base | 18:44 |
| nibalizer | do we manage the puppet package in puppet right now? | 18:44 |
| jeblair | oh, we can set it explicitly, but it matches in the proposed change. | 18:44 |
| nibalizer | what i can see happening is the puppetmaster never actually getting upgraded then a new node comes online with 3.7 | 18:44 |
| jeblair | nibalizer: that's an error | 18:44 |
| clarkb | we do autoupgrades with package pins | 18:44 |
| jeblair | nibalizer: the puppemaster should run nightly updates just like all the other machines | 18:45 |
| clarkb | so it should auto update both master and agent | 18:45 |
| clarkb | jeblair: yup | 18:45 |
| jeblair | and i _think_ we put puppetlabs in the list of repos to do that for | 18:45 |
| nibalizer | do we have to restart the puppetmaster service to pick up a new version taht was installed by apt? | 18:45 |
| clarkb | hrm actually it doesn't look like we d | 18:45 |
| jeblair | clarkb: i agree | 18:45 |
| flashgordon | we are starting to see some errors like this https://jenkins02.openstack.org/job/gate-nova-docs/21122/console | 18:46 |
| clarkb | I am good with adding ti as long as we have the pin to 3 and don't randomly get 4 :) | 18:46 |
| fungi | passenger needs apache restarted if it's upgraded i think, but can't recall if it does that on its own | 18:46 |
| *** Sincler has joined #openstack-infra | 18:46 | |
| flashgordon | getting a 502 from http://docs.python.org/objects.inv | 18:46 |
| *** dannywilson has quit IRC | 18:46 | |
| jeblair | flashgordon: stop using intersphinx | 18:46 |
| flashgordon | jeblair: what does intersphinx do anyway? | 18:46 |
| *** zz_dimtruck is now known as dimtruck | 18:47 | |
| notmyname | FYI I jsut added a -2 to something in the gate queue. it's been there for >24 hours, but I wanted to say something since the gate is so long and I'd hate for it to cause a gate flush because it has a -2 now. The patch is 120424 | 18:47 |
| *** dannywilson has joined #openstack-infra | 18:47 | |
| clarkb | notmyname: it won't flush. it will continue to be processed and if your -2 is still there when it should merge it won't merge | 18:47 |
| clarkb | notmyname: so should be fine and thank you | 18:47 |
| *** vigneshvar has joined #openstack-infra | 18:47 | |
| notmyname | oh ok | 18:47 |
| *** dannywilson has quit IRC | 18:47 | |
| *** dannywilson has joined #openstack-infra | 18:48 | |
| jeblair | flashgordon: links to other sphinx documents out on the internet | 18:48 |
| clarkb | jeblair: fungi nibalizer so should we write a change to allow auto updates of puppet? | 18:48 |
| jeblair | clarkb: probably so | 18:49 |
| * clarkb learns him an unattended upgrade config | 18:49 | |
| nibalizer | clarkb: you could just package { 'puppet': ensure => latest, } | 18:50 |
| clarkb | nibalizer: no | 18:50 |
| nibalizer | ok | 18:50 |
| clarkb | we don't do that for any apt packages because we want unattended upgrades to deal with it | 18:50 |
| clarkb | better to have consistency here | 18:50 |
| nibalizer | okay im with you on that | 18:50 |
| nibalizer | you can always win me over with a consistency argument | 18:50 |
| jeblair | fungi: the sed still needs something for puppetdb-terminus | 18:51 |
| jeblair | and facter | 18:51 |
| flashgordon | jeblair: thanks, https://review.openstack.org/121200 | 18:52 |
| jeblair | dhellmann: ^ | 18:52 |
| *** stevemar has quit IRC | 18:52 | |
| fungi | jeblair: should be okay now | 18:53 |
| *** ling-yun has quit IRC | 18:53 | |
| jeblair | fungi: lgtm | 18:53 |
| *** stevemar has joined #openstack-infra | 18:53 | |
| *** ling-yun has joined #openstack-infra | 18:53 | |
| fungi | also strangely, i see some hosts pinning puppetdb-terminus to 1.* in its own section, and others lumping it in with the puppet 2.7.* pin | 18:53 |
| *** MarkAtwood has joined #openstack-infra | 18:54 | |
| *** MarkAtwood has quit IRC | 18:54 | |
| jeblair | oh 3.7 is out now | 18:54 |
| fungi | compare paste.o.o and pypi.dfw.o.o | 18:54 |
| jeblair | so we'll be putting 3.7 on all the clients | 18:54 |
| *** nelsnelson has quit IRC | 18:54 | |
| fungi | perhaps we should upgrade the master again first? | 18:55 |
| *** amcrn has joined #openstack-infra | 18:55 | |
| markvan | fungi: clarkb: sorry for same song & dance on ruby patch, but I missed a sudo in there: https://review.openstack.org/#/c/121164/ thx! | 18:55 |
| jeblair | fungi: probably so | 18:55 |
| jeblair | nibalizer, clarkb: upgrade master to 3.7? | 18:55 |
| *** dmsimard_away is now known as dmsimard | 18:57 | |
| jeblair | fungi: maybe we should switch to the heredoc for consistency | 18:57 |
| clarkb | jeblair: we will need to have 3.7 master for 3.7 clients to reliably work | 18:57 |
| jeblair | fungi: the 3 tier one is what's in install_puppet | 18:57 |
| fungi | jeblair: probably | 18:57 |
| clarkb | jeblair: fungi: what do yo uthink about pinning to 3.6 today and working on 3.7 next week? (I don'treally care either way but that may reduce the number of variables) | 18:57 |
| fungi | i'll do that | 18:57 |
| fungi | (the heredoc thing) | 18:58 |
| fungi | do we think 3.7 is likely substantially different from 3.6? | 18:58 |
| jeblair | clarkb: i'm getting pretty tired of working on puppet. if you want to do that, i'll probably just switch to working on replacing puppet with ansible. | 18:58 |
| *** ling-yun has quit IRC | 18:59 | |
| jeblair | at that point, using puppet will have become _FAR_ more work than almost any other way i can think to manage systems, which is kind of the opposite of what it's supposed to be. | 18:59 |
| clarkb | fungi: no puppet is semver starting with version 3.0 | 18:59 |
| jeblair | so let's take them at their word and upgrade to 3.7 | 19:00 |
| clarkb | kk | 19:00 |
| fungi | i'm in favor | 19:00 |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/storyboard: Add 'task_origin' field to tasks table. https://review.openstack.org/121201 | 19:00 |
| jeblair | upgrading master now | 19:00 |
| *** emagana has quit IRC | 19:00 | |
| jeblair | upgrading puppetmaster-passenger did not restart apache | 19:01 |
| jeblair | i don't know if that's required or not. | 19:01 |
| jeblair | is there a way to ask the puppetmaster what version it's currently running? | 19:01 |
| jeblair | nibalizer: ^ any idea? | 19:02 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Allow puppet to be automatically upgraded https://review.openstack.org/121202 | 19:02 |
| *** tonytan4ever has joined #openstack-infra | 19:02 | |
| clarkb | I think that will do it for debuntu nodes | 19:02 |
| clarkb | looking at how we do autoupgrades on centos now | 19:02 |
| *** ling-yun has joined #openstack-infra | 19:03 | |
| *** ociuhandu has quit IRC | 19:03 | |
| jeblair | nothing in the apache error log that makes me thing it did anything | 19:03 |
| fungi | gah paste from etherpad is teh sux. i'll put it in a paste.o.o | 19:03 |
| *** ChuckC_ has joined #openstack-infra | 19:03 | |
| *** dkranz has quit IRC | 19:04 | |
| jeblair | restarted apache manually | 19:05 |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for sqla-migrate 0.9.2 bug 1368391 https://review.openstack.org/121203 | 19:05 |
| uvirtbot | Launchpad bug 1368391 in glance "sqlalchemy-migrate 0.9.2 is breaking nova unit tests" [Undecided,In progress] https://launchpad.net/bugs/1368391 | 19:05 |
| jeblair | i consider that a bug in the puppet-passenger package | 19:05 |
| *** emagana has joined #openstack-infra | 19:06 | |
| nibalizer | jeblair: puppet --versio | 19:07 |
| nibalizer | --version | 19:07 |
| jeblair | nibalizer: run that where? | 19:07 |
| nibalizer | on the master | 19:07 |
| openstackgerrit | A change was merged to openstack-infra/puppet-storyboard: Spell 'server_admin' consistently https://review.openstack.org/121186 | 19:07 |
| jeblair | nibalizer: and that will communicate with the master via apache to ask it the running version? | 19:08 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1368910 https://review.openstack.org/121204 | 19:08 |
| uvirtbot | Launchpad bug 1368910 in nova "intersphinx requires network access which sometimes fails" [Undecided,In progress] https://launchpad.net/bugs/1368910 | 19:08 |
| nibalizer | jeblair: no, i see what you're getting at now | 19:08 |
| *** ling-yun has quit IRC | 19:08 | |
| clarkb | hrm I don't see anything that indicates packagekit won't already update puppet for us | 19:08 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Add option for building Ironic deploy ramdisks https://review.openstack.org/119473 | 19:09 |
| openstackgerrit | A change was merged to openstack-infra/puppet-storyboard: Compare values to :undef to test for existence https://review.openstack.org/121192 | 19:09 |
| *** coolsvap has quit IRC | 19:09 | |
| jeblair | our manifests are going to cause some harmless comment churn on /etc/default/puppet. we should update that, but it's not critical | 19:09 |
| jeblair | i'm going to grab lunch now, but i think we're all set to proceed | 19:10 |
| fungi | okay, i'll start tackling a handful and raise in here if i find any exceptions | 19:10 |
| clarkb | we do indeed have latest puppet 2.7 on the git nodes | 19:10 |
| clarkb | so I think we may be ok there once we update the excludes in /etc/yum.repos.d/puppetlabs.repo | 19:11 |
| clarkb | I am going to grab lunch now back in a bit | 19:11 |
| fungi | yeah, unattended-upgrades does some magical whitelisting of specific repositories and won't upgrade packages from others by default without additional convincing | 19:11 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Update comments in puppet default https://review.openstack.org/121206 | 19:11 |
| fungi | hopefully 121202 passes in the right magic | 19:12 |
| superdan | So can we get 121200 promoted? | 19:12 |
| fungi | i honestly can't remember how to identify exactly what it needs without digging in apt repo release files to find the encoded suite name and all that | 19:12 |
| nibalizer | jeblair: this might work for getting puppetmaster status https://docs.puppetlabs.com/guides/rest_api.html#status | 19:13 |
| *** mjturek has left #openstack-infra | 19:13 | |
| superdan | fungi: ^ | 19:13 |
| clarkb | oh crud I just relized the archive of trusty may not be correcr in my auto updates change | 19:14 |
| clarkb | fungi ^ | 19:14 |
| *** e0ne has joined #openstack-infra | 19:15 | |
| nibalizer | ya so this, with paths replaced to be more openstacky, works to give version of the running puppetmaster | 19:15 |
| nibalizer | curl -k --cert /var/lib/puppet/ssl/certs/puppet.nibalizer.com.pem --key /var/lib/puppet/ssl/private_keys/puppet.nibalizer.com.pem -H "Accept: pson" https://localhost:8140/production/status/puppetclient | 19:15 |
| *** vigneshvar has quit IRC | 19:15 | |
| clarkb | jeblair: nibalizee subscribe an apache service resource to the passenger package | 19:16 |
| *** koolhead17 has joined #openstack-infra | 19:17 | |
| *** koolhead17 has joined #openstack-infra | 19:17 | |
| *** koolhead17 has quit IRC | 19:17 | |
| *** koolhead17 has joined #openstack-infra | 19:17 | |
| fungi | superdan: the bug linked from that change has a logstash query which indicates that this happened to four changes in the past week, most recently on tuesday | 19:18 |
| superdan | flashgordon: ? | 19:19 |
| superdan | fungi: thought it just took out like a ten-patch set of numa bits | 19:19 |
| fungi | unless it's got additional manifestations not covered by the query in the bug | 19:19 |
| *** homeless_ has joined #openstack-infra | 19:20 | |
| *** balles has quit IRC | 19:21 | |
| *** penguinRaider has joined #openstack-infra | 19:21 | |
| flashgordon | fungi: it happened moments ago in the gate | 19:21 |
| flashgordon | as well | 19:22 |
| fungi | nibalizer: what was the plan for dealing with the server=ci-puppetmaster.openstack.org in puppet.conf on all the servers? | 19:22 |
| flashgordon | fungi: if we are doing another batch of promotions here are some other patches that are ready | 19:22 |
| flashgordon | https://review.openstack.org/#/c/120793 | 19:22 |
| fungi | nibalizer: is that getting puppet-managed once we're done? | 19:22 |
| flashgordon | https://review.openstack.org/#/c/120843 | 19:22 |
| flashgordon | neither are very high on the gate bug list | 19:22 |
| flashgordon | bug they do hit occassionally | 19:23 |
| mtreinish | flashgordon: oh, good call on the second one. I had a patch bump on that one... | 19:23 |
| *** homeless has quit IRC | 19:23 | |
| superdan | do it do it! | 19:24 |
| flashgordon | figure for the next promote we do can do all the outstanding gate bug fixes that are approved | 19:24 |
| flashgordon | also it looks like logstash is pretty backed up | 19:25 |
| flashgordon | so we are running blind | 19:25 |
| *** cnesa9 has quit IRC | 19:26 | |
| flashgordon | not seeing many entries past 8:00 and its now 19:26 | 19:26 |
| fungi | yeah, its job queue is ~110k now | 19:27 |
| flashgordon | wow | 19:27 |
| fungi | seems to have spiked up shortly after the job volume did | 19:27 |
| fungi | which makes sense | 19:27 |
| *** dannywilson has quit IRC | 19:28 | |
| flashgordon | anyway time to find some food | 19:28 |
| *** dannywilson has joined #openstack-infra | 19:28 | |
| fungi | nibalizer: do we need to be worried about any of these new warnings? http://paste.openstack.org/show/110784/ | 19:29 |
| *** e0ne has quit IRC | 19:29 | |
| fungi | puppet agent likes to make them a nice bold red color | 19:29 |
| fungi | seems to think we should stop setting modulepath and manifest in puppet.conf, and manifestdir and templatedir at all | 19:30 |
| *** homeless_ has quit IRC | 19:31 | |
| fungi | hrm... i'm unconvinced our automated puppet runs are working... the change we merged to nodepool.yaml still hasn't been installed to the server, more than an hour later | 19:31 |
| *** Guest24333 is now known as Adri2000 | 19:32 | |
| *** Adri2000 has quit IRC | 19:32 | |
| *** Adri2000 has joined #openstack-infra | 19:32 | |
| *** chmouel- has joined #openstack-infra | 19:32 | |
| fungi | however, whatever the problem was in dfw seems to have lightened up again, so maybe we should revert that max-servers patch anyway | 19:32 |
| *** chmouel- has quit IRC | 19:33 | |
| *** dannywilson has quit IRC | 19:33 | |
| *** bhuvan_ has joined #openstack-infra | 19:33 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Remove query for bug 1347692 https://review.openstack.org/120916 | 19:34 |
| uvirtbot | Launchpad bug 1347692 in glance "gate failure with tempest.api.image.v1.test_images.CreateRegisterImagesTest.test_register_http_image (dup-of: 1345955)" [Undecided,Confirmed] https://launchpad.net/bugs/1347692 | 19:34 |
| uvirtbot | Launchpad bug 1345955 in horizon "KeyError: <SassRule <RuleAncestry ()>, 0 props> Failure" [Critical,Confirmed] https://launchpad.net/bugs/1345955 | 19:34 |
| *** dannywilson has joined #openstack-infra | 19:34 | |
| *** vigneshvar has joined #openstack-infra | 19:34 | |
| *** bhuvan has quit IRC | 19:36 | |
| *** sdake_ has quit IRC | 19:37 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/subunit2sql: Switch sample sqlalchemy url to use mysql https://review.openstack.org/115761 | 19:38 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/subunit2sql: Add get_failing_from_run db api method https://review.openstack.org/117860 | 19:38 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/subunit2sql: Add subunit2sql-db-manage utility https://review.openstack.org/118418 | 19:38 |
| *** e0ne has joined #openstack-infra | 19:39 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Revert "Drop 90 servers from rax-dfw" https://review.openstack.org/121210 | 19:39 |
| *** kgiusti has quit IRC | 19:39 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for sqla-migrate 0.9.2 bug 1368391 https://review.openstack.org/121203 | 19:41 |
| uvirtbot | Launchpad bug 1368391 in glance "sqlalchemy-migrate 0.9.2 is breaking nova unit tests" [Undecided,In progress] https://launchpad.net/bugs/1368391 | 19:41 |
| *** e0ne has quit IRC | 19:41 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add fingerprint for bug 1368910 https://review.openstack.org/121204 | 19:42 |
| uvirtbot | Launchpad bug 1368910 in nova "intersphinx requires network access which sometimes fails" [Medium,In progress] https://launchpad.net/bugs/1368910 | 19:42 |
| *** mriedem has quit IRC | 19:43 | |
| *** nelsnelson has joined #openstack-infra | 19:43 | |
| *** koolhead17 has quit IRC | 19:43 | |
| *** mriedem has joined #openstack-infra | 19:43 | |
| *** kgiusti has joined #openstack-infra | 19:44 | |
| *** sdake has joined #openstack-infra | 19:44 | |
| clarkb | flashgordon: fungi: it spiked due to nova console logs exploding yesterday | 19:45 |
| fungi | clarkb: oh, one other possible gotcha. i have a vague recollection that unattended-upgrades fails to interpret apt-cache policy and so doesn't work with pinned packages. looking now for confirmation | 19:46 |
| *** mriedem1 has joined #openstack-infra | 19:47 | |
| *** andreaf has quit IRC | 19:48 | |
| *** mriedem has quit IRC | 19:48 | |
| *** andreaf has joined #openstack-infra | 19:49 | |
| *** MaxV has joined #openstack-infra | 19:49 | |
| clarkb | fungi: oh fun | 19:51 |
| fungi | oh, right, also i think i remember looking into this before... the release files in the puppetlabs apt repo lack a suite field | 19:51 |
| fungi | http://apt.puppetlabs.com/dists/trusty/main/binary-all/Release | 19:52 |
| fungi | http://apt.puppetlabs.com/dists/precise/main/binary-all/Release | 19:52 |
| clarkb | fungi: is there a fallback value like 'main' or similar that apt will not blow up on? | 19:53 |
| clarkb | I am guessing not as this seems to be very conservative | 19:53 |
| fungi | oh, nevermind... http://apt.puppetlabs.com/dists/precise/Release | 19:53 |
| fungi | so "suite" is precise on precise, trusty on trusty | 19:53 |
| clarkb | gah ok so I need to use powers of facter to get that value | 19:53 |
| fungi | and origin is Puppetlabs (with a capital P) | 19:54 |
| clarkb | yup I got that bit | 19:54 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Allow puppet to be automatically upgraded https://review.openstack.org/121202 | 19:55 |
| clarkb | fungi: ^ I think that should pass jenkins testing and set the right suite | 19:55 |
| clarkb | but whether or not pins are respected is an open question | 19:56 |
| fungi | i did find surprisingly spot-on bug 1251228 but it's not yet been triaged so may be based on a fallacy | 19:56 |
| uvirtbot | Launchpad bug 1251228 in unattended-upgrades "unattended-upgrade doesn't upgrade packages pinned to version" [Undecided,New] https://launchpad.net/bugs/1251228 | 19:56 |
| fungi | nevertheless, if true, more dangerous than not upgrading pinned packages | 19:57 |
| clarkb | oh I see the issue with puppet I am going to fix it now | 19:57 |
| *** alexpilotti has quit IRC | 19:58 | |
| *** salv-orlando has joined #openstack-infra | 19:58 | |
| *** amcrn has quit IRC | 19:59 | |
| clarkb | I apparently missed that we had to manually run puppet on all the nodes for the ssh key thing | 19:59 |
| *** JayJ__ has joined #openstack-infra | 19:59 | |
| clarkb | I am swinging around and doing that on hosts that I skipped because they were like others... my bad | 19:59 |
| *** sdake has quit IRC | 20:00 | |
| *** nelsnelson has quit IRC | 20:00 | |
| fungi | i'm taking a break to cook dinner | 20:01 |
| fungi | biab | 20:01 |
| *** dannywilson has quit IRC | 20:01 | |
| *** dannywilson has joined #openstack-infra | 20:01 | |
| *** emagana has quit IRC | 20:02 | |
| *** alexpilotti has joined #openstack-infra | 20:02 | |
| dhellmann | jeblair: it looks like mriedem1 took care of approving that nova/intersphinx change | 20:03 |
| *** marcoemorais1 has joined #openstack-infra | 20:04 | |
| *** marcoemorais1 has quit IRC | 20:04 | |
| *** marcoemorais1 has joined #openstack-infra | 20:05 | |
| *** dannywilson has quit IRC | 20:05 | |
| *** dannywilson has joined #openstack-infra | 20:05 | |
| *** andreykurilin_ has joined #openstack-infra | 20:05 | |
| *** dmsimard is now known as dmsimard_away | 20:06 | |
| *** marcoemorais has quit IRC | 20:07 | |
| *** kgiusti has quit IRC | 20:08 | |
| clarkb | fungi: jeblair: did we decide to continue pinning facter to 1.*? | 20:09 |
| *** MaxV has quit IRC | 20:10 | |
| *** ramashri has quit IRC | 20:11 | |
| *** kgiusti has joined #openstack-infra | 20:11 | |
| *** nelsnelson has joined #openstack-infra | 20:15 | |
| nibalizer | fungi: yea we should stopdoing that | 20:16 |
| clarkb | judging by https://review.openstack.org/#/c/120573/1/modules/openstack_project/manifests/base.pp facter and puppetdb-termins should both be pinned to 2* | 20:17 |
| jeblair | clarkb: we did not decide to pin to 1. and this is the third time we've had this conversation. :) | 20:18 |
| clarkb | jeblair: I ask because the heredoc pasted on the etherpad pins that way... | 20:19 |
| jeblair | so http://paste.openstack.org/show/110791/ | 20:19 |
| jeblair | is an updated version | 20:19 |
| daguegoneit | fungi: we just took a gate reset, any chance we could promote this https://review.openstack.org/#/c/121200/ ? | 20:19 |
| clarkb | thanks | 20:19 |
| daguegoneit | should help on the nova docs thing killing FFE patches | 20:19 |
| jeblair | sigh, our centos pins are not equivalent to our ubuntu ones | 20:23 |
| *** Sincler has quit IRC | 20:24 | |
| *** andreaf has quit IRC | 20:24 | |
| *** rmcall has quit IRC | 20:25 | |
| *** andreaf has joined #openstack-infra | 20:25 | |
| clarkb | jeblair: ya they are inverted | 20:25 |
| clarkb | I am getting dpkg was interrupted errors | 20:25 |
| clarkb | and apparmore package corruption | 20:25 |
| * clarkb hacks with hacksaw | 20:25 | |
| flashgordon | daguegoneit: if we do a promote we should do https://review.openstack.org/#/c/120843 and https://review.openstack.org/#/c/120843 as well | 20:26 |
| superdan | oh, both of those? | 20:26 |
| superdan | I'd rather just 120843 | 20:26 |
| *** flaviof_zzz is now known as flaviof | 20:26 | |
| flashgordon | superdan: those are other fixes for known gate bugs | 20:26 |
| superdan | not 120843 though, just 120843 | 20:27 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Also stop managing yum puppet pins https://review.openstack.org/121220 | 20:27 |
| flashgordon | oh woops | 20:27 |
| jeblair | clarkb: https://review.openstack.org/121220 | 20:27 |
| * superdan is being a friday-inspired smartass | 20:27 | |
| flashgordon | https://review.openstack.org/#/c/120793 | 20:27 |
| flashgordon | daguegoneit: looks like fungi is AFK For a bit | 20:29 |
| openstackgerrit | Joshua Harlow proposed a change to openstack-infra/config: Ensure redis and memcached are on slaves (and running) https://review.openstack.org/121221 | 20:29 |
| *** lcheng_ has joined #openstack-infra | 20:29 | |
| *** r-daneel__ has quit IRC | 20:29 | |
| AJaeger | daguegoneit: I see that intersphinx is used by many other projects as well - should it be disabled everywhere? | 20:30 |
| daguegoneit | AJaeger: probably | 20:30 |
| flashgordon | AJaeger: at least on any project that fails on sphinx warnings | 20:31 |
| *** kgiusti has left #openstack-infra | 20:31 | |
| flashgordon | AJaeger: but I think everywhere makes sense | 20:31 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Revert "Stop managing puppet apt pins" https://review.openstack.org/120573 | 20:31 |
| AJaeger | flashgordon, daguegoneit is either of you doing some patches or shall I have a go? | 20:32 |
| daguegoneit | AJaeger: go for it | 20:32 |
| *** dustins has quit IRC | 20:32 | |
| flashgordon | AJaeger: go for it | 20:33 |
| *** emagana has joined #openstack-infra | 20:33 | |
| *** gyee has joined #openstack-infra | 20:34 | |
| *** leakypipes has quit IRC | 20:34 | |
| *** andreaf has quit IRC | 20:34 | |
| *** andreaf has joined #openstack-infra | 20:35 | |
| AJaeger | ok, will do some patches... | 20:35 |
| *** _nadya_ has quit IRC | 20:35 | |
| *** nelsnelson has quit IRC | 20:36 | |
| flashgordon | jeblair: oh I think I was supposed to remind you to look into why I can't do https://review.openstack.org/Documentation/rest-api-groups.html#group-members without being authenticated | 20:37 |
| *** nelsnelson has joined #openstack-infra | 20:37 | |
| *** emagana has quit IRC | 20:37 | |
| clarkb | oh I just remembered that the facter metadata stuff failing should be better in puppet3 iirc | 20:38 |
| clarkb | so this may fix a class of failure we hit \o/ | 20:38 |
| jeblair | flashgordon: i'm not seeing anything obvious; next step may involve examining the gerrit source to see if there's a concept of anonymous viewing of group members at all | 20:40 |
| jeblair | clarkb, fungi: etherpad links to pastes have been updated for ubuntu and centos | 20:40 |
| clarkb | jeblair: thanks | 20:41 |
| *** mmedvede has quit IRC | 20:41 | |
| *** ChuckC_ has quit IRC | 20:41 | |
| jeblair | i approved https://review.openstack.org/#/c/121220/ centos nodes will need to wait on that | 20:41 |
| *** nelsnelson has quit IRC | 20:42 | |
| flashgordon | jeblair: cool | 20:42 |
| *** radez is now known as radez_g0n3 | 20:42 | |
| AJaeger | flashgordon, daguegoneit: I'm reusing the change-id and appreciate reviews of https://review.openstack.org/#/q/I71e941e2a639641a662a163c682eb86d51de42fb,n,z | 20:43 |
| AJaeger | will do some more patches tomorrow | 20:43 |
| superdan | AJaeger: that just unapproved the one we had | 20:43 |
| superdan | was that intentional? | 20:43 |
| flashgordon | AJaeger: why did you upload a version of the nova one? | 20:44 |
| AJaeger | superdan, flashgordon: I screwed up, sorry ;(((( | 20:44 |
| AJaeger | I mixed up neutron and nova ;( | 20:45 |
| flashgordon | superdan: want to re-approve | 20:45 |
| superdan | did | 20:45 |
| superdan | is fungi the only one that can promote? | 20:45 |
| *** dimtruck is now known as zz_dimtruck | 20:46 | |
| flashgordon | superdan: no, clarkb can as well | 20:46 |
| superdan | okay | 20:46 |
| flashgordon | superdan: we need to wait for gate reset before doing it though | 20:46 |
| jeblair | any infra-root can | 20:46 |
| superdan | flashgordon: oh? | 20:46 |
| superdan | do we queue it for the next reset, or do we have to notice that there's a reset? | 20:47 |
| jeblair | we can do it at any time, it's just instantaneous, so it depends on what you're trying to accomplish | 20:47 |
| superdan | yeah, I figured that for the set of things we have, we'd just do it and take the obligatory reset | 20:47 |
| superdan | that was my assumption I mean | 20:48 |
| flashgordon | superdan: I think we should wait for a reset | 20:48 |
| flashgordon | as a few patches are close to landing | 20:48 |
| superdan | okay | 20:48 |
| flashgordon | or failing | 20:48 |
| superdan | sucks if you're the guy that gets kicked out because of this fail :) | 20:48 |
| *** mbacchi has quit IRC | 20:49 | |
| *** tsufiev_ is now known as tsufiev | 20:50 | |
| *** fifieldt_ has joined #openstack-infra | 20:50 | |
| *** craigbr has quit IRC | 20:50 | |
| AJaeger | 73 hits in various projects for the last 7 days searching with logstash... | 20:51 |
| *** marcoemorais1 has quit IRC | 20:52 | |
| *** marcoemorais has joined #openstack-infra | 20:53 | |
| flashgordon | AJaeger: not all are failures right? | 20:53 |
| AJaeger | flashgordon: correct, most don't have warnerrors set. | 20:53 |
| *** marcoemorais has quit IRC | 20:53 | |
| *** marcoemorais has joined #openstack-infra | 20:54 | |
| * AJaeger will update the commit message for the next patches | 20:54 | |
| *** fifieldt has quit IRC | 20:54 | |
| flashgordon | AJaeger: just be careful to stay away from the nova one :) | 20:54 |
| *** sdake has joined #openstack-infra | 20:54 | |
| *** dprince has quit IRC | 20:55 | |
| AJaeger | flashgordon: I'm calling it a day now, you should be safe from me now ;) | 20:55 |
| *** marcoemorais has quit IRC | 20:55 | |
| AJaeger | good night! | 20:55 |
| *** marcoemorais has joined #openstack-infra | 20:55 | |
| *** AJaeger has quit IRC | 20:55 | |
| *** ramashri has joined #openstack-infra | 20:56 | |
| *** dmsimard_away has quit IRC | 20:57 | |
| *** cnesa9 has joined #openstack-infra | 20:57 | |
| *** craigbr has joined #openstack-infra | 20:57 | |
| jeblair | found an error on wiki | 20:57 |
| jeblair | Error: Failed to apply catalog: Parameter name failed on Package[apache_dev_package]: Name must be a String not Array at /etc/puppet/modules/apache/manifests/dev.pp:20 | 20:57 |
| adam_g | daguegoneit, any chance of a peek at https://review.openstack.org/#/c/116761/ | 20:57 |
| *** mfer has quit IRC | 20:57 | |
| clarkb | I am just about done with logstash nodes and can help look at that in a few minutes | 20:58 |
| *** shashankhegde has quit IRC | 20:58 | |
| *** ChuckC_ has joined #openstack-infra | 20:58 | |
| daguegoneit | adam_g: looks fine | 20:58 |
| jeblair | that's the only manifest that requires 'apache::dev'; i wonder if we need to update apache module for puppet3 | 20:59 |
| *** dmsimard_away has joined #openstack-infra | 20:59 | |
| *** dmsimard_away is now known as dmsimard | 20:59 | |
| *** salv-orlando has quit IRC | 21:00 | |
| adam_g | daguegoneit, danke | 21:00 |
| jeblair | yeah, i think so | 21:01 |
| jeblair | nibalizer: thoughts on upgrading puppetlabs-apache? | 21:02 |
| jeblair | our version is like the oldest there is. | 21:03 |
| *** shashankhegde has joined #openstack-infra | 21:03 | |
| clarkb | jeblair: it requires a lot of work | 21:03 |
| jeblair | clarkb: more or less work than changing wiki.o.o to ansible? | 21:03 |
| clarkb | jeblair: because they made a lot of changes that were not backwrad compat and removed features but apparently they are back in now | 21:03 |
| jeblair | this is my new yardstick | 21:03 |
| clarkb | jeblair: well the tricky thing is that you can't update it just for wiki you have to do it for everything at once. so potentially a similar amount of work | 21:04 |
| mgagne | clarkb: except the custom vhost template =) | 21:04 |
| clarkb | mgagne: is that still missing? | 21:04 |
| mgagne | clarkb: yep | 21:04 |
| clarkb | :/ | 21:04 |
| * clarkb looks into what apache::dev does | 21:04 | |
| mgagne | clarkb: support matrix I did back then: https://docs.google.com/spreadsheet/ccc?key=0Avtl596mgKQ_dEhBV3pjMnBmYlJfMXVTOXNxeGxxTnc&usp=drive_web#gid=0 | 21:04 |
| jeblair | so for now we could drop using apache::dev and replace it with what that currently explicitly does | 21:04 |
| nibalizer | jeblair: i dont think we can upgrade our apache version without quite a bit of work | 21:05 |
| jeblair | because that is probably less work than ansibling things | 21:05 |
| clarkb | jeblair: yup | 21:05 |
| jeblair | clarkb: i think it's literally just package installs. i'll write the change. | 21:05 |
| clarkb | jeblair: ok | 21:05 |
| mgagne | clarkb: template support was dropped in 0.7.0 | 21:05 |
| *** _nadya_ has joined #openstack-infra | 21:06 | |
| nibalizer | you can custom_fragment, which is very close to a custom vhost | 21:06 |
| clarkb | jeblair: ya its trying to install all of https://github.com/puppetlabs/puppetlabs-apache/blob/0.0.4/manifests/params.pp#L50 in one go but that breaks on puppet3 | 21:06 |
| clarkb | jeblair: so that is a quick and easy work around to do what you are doing | 21:06 |
| nibalizer | it could also be possible for us to make openstack_apache::vhost that does the same vhost behavior as the 0.0.4 module and use a modern apache module for everything but the vhosts | 21:07 |
| *** pcm_ has quit IRC | 21:07 | |
| nibalizer | that would be a way forward i think | 21:07 |
| *** emagana has joined #openstack-infra | 21:07 | |
| flashgordon | daguegoneit: is there a devstack-gate patch for https://review.openstack.org/#/c/117517/7 ? | 21:07 |
| daguegoneit | not yet | 21:08 |
| devananda | daguegoneit: if you have another spare minute, I think https://review.openstack.org/#/c/112134/ would be great to land soon (consolidate redundant ironic tests) | 21:08 |
| daguegoneit | well, I don't have landing power in infra | 21:08 |
| devananda | gah, right | 21:08 |
| clarkb | mgagne: that is a handy matrix | 21:08 |
| flashgordon | daguegoneit: in that case I'll post one so once that lands we can try it out in d-g | 21:09 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Drop apache::dev in wiki https://review.openstack.org/121238 | 21:09 |
| jeblair | clarkb, nibalizer: ^ | 21:10 |
| clarkb | +2 | 21:10 |
| jeblair | clarkb: can you go ahead and aprv it? | 21:10 |
| clarkb | yup | 21:10 |
| clarkb | done | 21:10 |
| jeblair | it's just us right now and it's going to take a while | 21:10 |
| clarkb | rgr | 21:11 |
| *** marun has joined #openstack-infra | 21:11 | |
| *** ling-yun has joined #openstack-infra | 21:11 | |
| nibalizer | i found a tab so -1 | 21:11 |
| clarkb | nibalizer: will that fail lint? | 21:11 |
| nibalizer | should | 21:11 |
| clarkb | :( | 21:11 |
| devananda | clarkb: https://review.openstack.org/#/c/112134/ seems to need you to +2 it again | 21:11 |
| jeblair | lint should diea | 21:12 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Drop apache::dev in wiki https://review.openstack.org/121238 | 21:12 |
| *** coolsvap has joined #openstack-infra | 21:12 | |
| clarkb | reapproved | 21:13 |
| *** zz_dimtruck is now known as dimtruck | 21:14 | |
| devananda | jeblair, daguegoneit: assuming ironic graduates, at what point do we flip the switch and make an ironic test voting on nova (and, given the ongoing layering discussion, should we even do that?) | 21:14 |
| *** vigneshvar has quit IRC | 21:14 | |
| devananda | clarkb: ty | 21:14 |
| clarkb | devananda: oh I reapproved jeblair's thing | 21:14 |
| devananda | clarkb: oh lol | 21:15 |
| jeblair | devananda: whenever you and nova are ready | 21:15 |
| clarkb | devananda: will try to get to that other thing later but currently in puppet3 hackery mode | 21:15 |
| jeblair | devananda: and yeah, i think we should do it regardless | 21:15 |
| *** sabeen has quit IRC | 21:15 | |
| devananda | jeblair: cool | 21:16 |
| devananda | i like it when my questions have simple answers | 21:16 |
| jeblair | me too! | 21:16 |
| *** rfolco has quit IRC | 21:18 | |
| *** adalbas has quit IRC | 21:19 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/devstack-gate: Limit number of API workers used https://review.openstack.org/121242 | 21:19 |
| *** tonytan4ever has quit IRC | 21:21 | |
| *** dhellmann is now known as dhellmann_ | 21:22 | |
| *** jungleboyj has joined #openstack-infra | 21:22 | |
| harlowja | qq, how does one resolve a cache inconsistency between pypi.iad and pypi.ord ? | 21:23 |
| *** nelsnelson has joined #openstack-infra | 21:23 | |
| clarkb | harlowja: you don't... | 21:24 |
| clarkb | its not perfect | 21:24 |
| clarkb | and until we have afs is a hard thing to fix | 21:24 |
| jeblair | we might have a stuck bandersnatch on one though, right? | 21:24 |
| harlowja | clarkb will it eventually resync? | 21:24 |
| clarkb | jeblair: good point, we could have one that is stuck | 21:25 |
| clarkb | jeblair: usually I would expect all to get stuck though | 21:25 |
| jeblair | clarkb: i don't think bandersnatch works like that :( | 21:25 |
| jeblair | i think we need to run it in a timeout or something :( | 21:25 |
| clarkb | oh I see ya thats a good idea | 21:25 |
| clarkb | maybe a half hour timeout in the cron | 21:25 |
| clarkb | as updates shouldnt' | 21:26 |
| clarkb | take that long | 21:26 |
| *** andreykurilin_ has quit IRC | 21:26 | |
| *** shardy_afk is now known as shardy | 21:26 | |
| jeblair | clarkb: don't do review | 21:26 |
| jeblair | clarkb: please ack | 21:26 |
| clarkb | ack | 21:26 |
| *** vigneshvar has joined #openstack-infra | 21:26 | |
| *** bswartz has quit IRC | 21:26 | |
| jeblair | clarkb: we should start with review-dev first | 21:27 |
| clarkb | oh good point /me goes over to that node first | 21:27 |
| clarkb | oh you have it | 21:27 |
| * clarkb gets out of the way | 21:27 | |
| jeblair | yeah, just starting on that | 21:27 |
| flashgordon | superdan: btw if https://jenkins07.openstack.org/job/gate-tempest-dsvm-postgres-full/8695/console passes we merge 3 patches | 21:28 |
| flashgordon | superdan: which is why I want to wait for a reset | 21:28 |
| superdan | flashgordon: crossing all crossable appendages ..... now. | 21:28 |
| flashgordon | superdan: hehe | 21:28 |
| clarkb | jeblair: no reason to avoid proposal.slave.o.o right? | 21:29 |
| jeblair | clarkb: nope; but check out the output from the noop run on review-dev | 21:29 |
| jeblair | the gerrit scripts things are interesting | 21:29 |
| clarkb | we want those scripts | 21:30 |
| clarkb | ya | 21:30 |
| jeblair | file { '/usr/local/gerrit/scripts': | 21:30 |
| jeblair | ensure => absent, | 21:30 |
| jeblair | clarkb: we seem conflicted on the point. | 21:30 |
| clarkb | oh I was thinking these are the things in review_site/hooks | 21:30 |
| *** LuckyVega has quit IRC | 21:30 | |
| clarkb | jeblair: and the github scripts have been replaced by jeepyb? | 21:31 |
| jeblair | clarkb: i think maybe both of those dirs have | 21:31 |
| clarkb | that makes sense. odd that it hasn't been cleaned up already though | 21:31 |
| clarkb | maybe it isn't recursive and thus failing? | 21:31 |
| *** ramashri has quit IRC | 21:31 | |
| *** emagana has quit IRC | 21:31 | |
| jeblair | clarkb: i'm guessing so. those dirs do not exist on prod | 21:32 |
| jeblair | clarkb: so i think i'll just rm -rf them on dev and try again | 21:32 |
| *** emagana has joined #openstack-infra | 21:32 | |
| jeblair | sound good? | 21:32 |
| *** mwagner_lap has quit IRC | 21:32 | |
| clarkb | jeblair: yup | 21:32 |
| *** ramashri has joined #openstack-infra | 21:33 | |
| *** emagana has quit IRC | 21:33 | |
| openstackgerrit | A change was merged to openstack-infra/config: Also stop managing yum puppet pins https://review.openstack.org/121220 | 21:33 |
| flashgordon | superdan: your appendage crossing worked | 21:33 |
| clarkb | jeblair: doing pypi.slave now | 21:33 |
| jeblair | clarkb: looks good; did you want to do review.o.o? | 21:33 |
| clarkb | jeblair: you can do it | 21:33 |
| jeblair | will do | 21:33 |
| *** emagana has joined #openstack-infra | 21:33 | |
| superdan | flashgordon: os i nac ssorcna won ? | 21:34 |
| *** andreaf has quit IRC | 21:34 | |
| *** andreaf has joined #openstack-infra | 21:34 | |
| fungi | oh crappity. back now, fixing that heredoc | 21:35 |
| clarkb | fungi: its fine | 21:35 |
| clarkb | fungi: I think jeblair updated | 21:35 |
| clarkb | and we are almost done | 21:35 |
| fungi | ahh, somebody else already did. cool | 21:35 |
| *** flaper87 is now known as flaper87|afk | 21:36 | |
| *** dimsum_ has quit IRC | 21:36 | |
| *** dimsum_ has joined #openstack-infra | 21:36 | |
| *** sballe__ has joined #openstack-infra | 21:37 | |
| *** stevemar2 has joined #openstack-infra | 21:37 | |
| *** stevemar has quit IRC | 21:38 | |
| jeblair | i'm pretty sure i'm hitting the environment cache | 21:39 |
| jeblair | we should probably turn that off | 21:39 |
| *** jcooley_ has joined #openstack-infra | 21:39 | |
| clarkb | jeblair: ++ | 21:39 |
| *** jcooley has quit IRC | 21:39 | |
| *** jcooley_ is now known as jcooley | 21:39 | |
| clarkb | fungi: you getting the pypi mirrors? | 21:39 |
| * clarkb does hpcloud mirror | 21:39 | |
| *** jamielennox has quit IRC | 21:39 | |
| *** yfried has joined #openstack-infra | 21:40 | |
| *** jamielenz has joined #openstack-infra | 21:40 | |
| *** jamielenz is now known as jamielennox | 21:40 | |
| *** dimsum_ has quit IRC | 21:40 | |
| *** dmellado has quit IRC | 21:41 | |
| *** gnuoy has quit IRC | 21:41 | |
| *** sweston has quit IRC | 21:41 | |
| jeblair | http://yum.puppetlabs.com/el/6/products/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 404 Not Found" | 21:41 |
| jeblair | this is confusing me | 21:41 |
| *** winston-1 has joined #openstack-infra | 21:41 | |
| *** aysyd has quit IRC | 21:41 | |
| *** cnesa10 has joined #openstack-infra | 21:41 | |
| *** torandu_ has joined #openstack-infra | 21:41 | |
| jeblair | it should be trying on level down under x86_64 | 21:41 |
| *** aysyd has joined #openstack-infra | 21:41 | |
| *** Alex_Gaynor has quit IRC | 21:42 | |
| *** gnuoy has joined #openstack-infra | 21:42 | |
| jeblair | ooooh | 21:42 |
| *** sweston has joined #openstack-infra | 21:42 | |
| jeblair | $ escaping in bash | 21:42 |
| *** Mmike has quit IRC | 21:42 | |
| *** ZZelle has quit IRC | 21:43 | |
| *** cnesa9 has quit IRC | 21:43 | |
| *** torandu has quit IRC | 21:43 | |
| *** sballe_ has quit IRC | 21:43 | |
| *** winston-d has quit IRC | 21:43 | |
| *** yfried|afk has quit IRC | 21:43 | |
| *** michchap has quit IRC | 21:43 | |
| *** kmartin has quit IRC | 21:43 | |
| *** markvan has quit IRC | 21:43 | |
| jeblair | updated paste | 21:43 |
| *** ZZelle has joined #openstack-infra | 21:43 | |
| *** vladan has quit IRC | 21:43 | |
| *** stevemar2 has quit IRC | 21:43 | |
| fungi | okay, caught up with scrollback. i'll get the pypi mirrors, sure | 21:43 |
| *** markvan has joined #openstack-infra | 21:43 | |
| clarkb | fungi: I am going bottom to top you go top to bottom and we will meet | 21:43 |
| *** lcheng_ has quit IRC | 21:43 | |
| *** kmartin has joined #openstack-infra | 21:43 | |
| *** nimrodsun_ has quit IRC | 21:43 | |
| *** skraynev_ has joined #openstack-infra | 21:44 | |
| *** ubuntu has joined #openstack-infra | 21:44 | |
| fungi | k | 21:44 |
| *** ubuntu is now known as Guest20522 | 21:44 | |
| *** Alex_Gaynor has joined #openstack-infra | 21:44 | |
| *** vladan has joined #openstack-infra | 21:45 | |
| fungi | testing after puppet upgrade still involves passing --server puppetmaster.openstack.org it seems, as puppet.conf still sets server to the old master | 21:45 |
| *** ZZelle_ has quit IRC | 21:45 | |
| *** ubuntu3 has quit IRC | 21:45 | |
| *** skraynev has quit IRC | 21:45 | |
| fungi | presumably this is known | 21:45 |
| clarkb | fungi: yup | 21:45 |
| *** nimrodsun_ has joined #openstack-infra | 21:45 | |
| clarkb | we should do a final day change to remove that completely or update the value | 21:45 |
| *** simonmcc___ has quit IRC | 21:45 | |
| *** michchap has joined #openstack-infra | 21:45 | |
| clarkb | s/day/everything upgraded/ | 21:45 |
| jeblair | fungi: yeah, ansible supplies the arg explicitly | 21:46 |
| *** Mmike has joined #openstack-infra | 21:46 | |
| jeblair | but also what clarkb said | 21:46 |
| fungi | right | 21:46 |
| *** simonmcc____ has joined #openstack-infra | 21:47 | |
| fungi | are we continying to be paranoid about git.o.o? | 21:47 |
| fungi | continuing | 21:47 |
| jeblair | nope | 21:47 |
| clarkb | fungi: slightly. I was trying to get centos pasty to come up so I can do it | 21:47 |
| clarkb | but go ahead | 21:47 |
| *** dmellado has joined #openstack-infra | 21:47 | |
| fungi | on it | 21:47 |
| fungi | nevermind--jeblair claimed it while i was gabbing | 21:47 |
| jeblair | ya; i was holding until we all agreed i had that lock though; looks like we do | 21:48 |
| fungi | go for it | 21:48 |
| fungi | if nobody has done a pass checking for hung bandersnatch processes yet, i can do that now | 21:48 |
| flashgordon | mordred daguegoneit: https://review.openstack.org/121250 | 21:48 |
| flashgordon | mordred daguegoneit: run daguegoneit's script for openstack/openstack | 21:49 |
| flashgordon | hmm it didn't work | 21:49 |
| *** mriedem1 has quit IRC | 21:50 | |
| *** datsun180b has quit IRC | 21:51 | |
| fungi | right-o... bandersnatch is hung on ord since 00:50 utc today | 21:51 |
| fungi | killing | 21:51 |
| *** Longgeek has joined #openstack-infra | 21:52 | |
| clarkb | jeblair: do we want to give the wiki.o.o fix any special treatment? | 21:52 |
| clarkb | jeblair: or just swiing around on monday and get it done | 21:52 |
| flashgordon | daguegoneit: fixed https://review.openstack.org/121250 landing that should make our openstack/openstack numbers more accurate | 21:53 |
| jeblair | clarkb: i'm trying to figure out a problem with git01 that ansible sees that i did not see when running manually | 21:53 |
| fungi | seems bandersnatch hung on ord trying to sync https://pypi.python.org/pypi/tablets | 21:53 |
| clarkb | jeblair: kk | 21:54 |
| jeblair | Error: Could not set 'present' on ensure: Could not locate the pip command. at 74:/opt/config/production/modules/openstack_project/manifests/base.pp | 21:54 |
| clarkb | huh | 21:54 |
| * clarkb tries to reproduce on 05 | 21:54 | |
| flashgordon | fungi: ok just hit a gate reset | 21:54 |
| flashgordon | superdan: ^ | 21:54 |
| superdan | do eeet! | 21:54 |
| fungi | flashgordon: superdan: 120843,2 120793,1 121200,3 ? | 21:55 |
| jeblair | clarkb: ansible does not see the problem on 05 | 21:56 |
| flashgordon | fungi: yup! | 21:56 |
| clarkb | jeblair: oh right it runs them in parallel | 21:56 |
| clarkb | jeblair: looks like 03 may have the problem? | 21:56 |
| * clarkb looks at 03 | 21:56 | |
| daguegoneit | flashgordon: +2 | 21:56 |
| fungi | flashgordon: superdan: nevermind, seems to be 120843,1 120793,1 121200,4 but that last one needs an enqueue first since it got inadvertently bumped with a rebase | 21:56 |
| jeblair | clarkb: oh? i just ran it and only saw it on 01 | 21:56 |
| *** Longgeek has quit IRC | 21:56 | |
| clarkb | jeblair: ansible said 03 failed. could be something different | 21:56 |
| jeblair | clarkb: no, git03 was a run already in progress | 21:57 |
| flashgordon | fungi: sigh, it was a super trivial rebase | 21:57 |
| jeblair | i believe i collided with it | 21:57 |
| clarkb | jeblair: that explains it | 21:57 |
| flashgordon | fungi: and a mistaken one | 21:57 |
| fungi | yep, saw in scrollback | 21:57 |
| flashgordon | fungi: so we have to wait for it to get through check first? | 21:57 |
| *** carl_baldwin has joined #openstack-infra | 21:57 | |
| carl_baldwin | Hi, question on https://review.openstack.org/#/c/70090/ | 21:58 |
| *** hdd has quit IRC | 21:58 | |
| jeblair | clarkb: okay, noop does not show the error, but running it for real it's reproducible | 21:58 |
| carl_baldwin | It passed in the Jenkins check queue and went to the gate queue. | 21:58 |
| clarkb | jeblair: weird. and `which pip` shows me what I expect | 21:58 |
| fungi | flashgordon: no, i took care of it | 21:58 |
| carl_baldwin | Then, Kyle added recheck-opendaylight in a comment and the Jenkins check queue picked it up again. | 21:59 |
| flashgordon | fungi: awesome | 21:59 |
| flashgordon | fungi: btw want to review https://review.openstack.org/#/c/121250/ (its trivial) | 21:59 |
| clarkb | carl_baldwin: we match recheck* for jenkins | 21:59 |
| clarkb | carl_baldwin: so that will enqueue it into check | 21:59 |
| flashgordon | fungi: thanks | 21:59 |
| fungi | though this is an odd failure... seems to have failed to get refs from zm01 at 10:41 utc... https://jenkins01.openstack.org/job/gate-cinder-python26/7777/console | 21:59 |
| flashgordon | superdan: they are all in :) | 21:59 |
| carl_baldwin | clarkb: So, it seems there is not a way to retrigger 3rd party without retriggering Jenkins | 22:00 |
| fungi | er, 20:41 | 22:00 |
| fungi | was apache maybe restarting on zm01 at 20:41? | 22:00 |
| superdan | flashgordon: woot | 22:00 |
| clarkb | fungi: or maybe they restarted ti again? | 22:00 |
| clarkb | carl_baldwin: depends on the third party. however you shouldn't be rechecking changes that have bene approved I don't think that accomplishes much | 22:01 |
| clarkb | or vice versa don't approve changes that need rechecking | 22:01 |
| *** marcusvrn has quit IRC | 22:01 | |
| *** JayJ__ has quit IRC | 22:02 | |
| carl_baldwin | clarkb: I can’t always help that. | 22:02 |
| *** IAX has joined #openstack-infra | 22:02 | |
| carl_baldwin | If you look at the patch, you’ll see that I approved it, then opendaylight failed it, and then Kyle added recheck. We were both surprised to see it reenter the check queue. | 22:02 |
| *** david-lyle has quit IRC | 22:03 | |
| *** signed8bit has quit IRC | 22:03 | |
| anteaya | carl_baldwin: understood, but jenkins will recheck patches with recheck in the comment | 22:03 |
| fungi | i actually didn't realize zuul would enqueue a patchset into check which was already in the gate pipeline, but regardless it shouldn't hurt anything | 22:03 |
| jeblair | clarkb, nibalizer, fungi: any ideas on the pip thing? | 22:03 |
| *** signed8bit has joined #openstack-infra | 22:04 | |
| carl_baldwin | fungi: *shouldn’t* :) | 22:04 |
| clarkb | https://review.openstack.org/#/c/118623/4 looks like the current proposed syntax and jenkins will recheck on that | 22:04 |
| clarkb | jeblair: no, but let me look at it harder | 22:04 |
| jeblair | clarkb, fungi, nibalizer: pasted full error in etherpad | 22:04 |
| nibalizer | jeblair: at the bottom of infra-puppet3? | 22:05 |
| jeblair | t | 22:05 |
| fungi | jeblair: my kneejerk reaction would be to kiwf (rpm and pip-installed versions) and manually redance the get-pip magic and hope it sorted itself out | 22:05 |
| clarkb | it is trying to install virtualenv wiht the puppet pip provider | 22:05 |
| clarkb | iirc pip provider tries to find pip | 22:05 |
| nibalizer | i assume there is a which pip ya? | 22:05 |
| clarkb | nibalizer: yup | 22:05 |
| clarkb | and pip --version exits 0 | 22:06 |
| * clarkb reads puppet pip provider code | 22:06 | |
| fungi | it looks in a fairly arbitrary list of explicit paths | 22:06 |
| jeblair | virtualenv --version | 22:06 |
| jeblair | 1.11.4 | 22:06 |
| fungi | the provider i mean | 22:06 |
| jeblair | pip freeze|grep virtualenv | 22:06 |
| jeblair | virtualenv==1.11.4 | 22:06 |
| clarkb | https://github.com/puppetlabs/puppet/blob/3.7.0/lib/puppet/provider/package/pip.rb#L40 | 22:07 |
| *** signed8bit has quit IRC | 22:07 | |
| clarkb | I think that is the problem. now i don't know yet why only one node is affected | 22:07 |
| jeblair | yep | 22:07 |
| jeblair | /usr/bin/which: no pip-python in (/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin) | 22:07 |
| clarkb | 03 has pip-python | 22:07 |
| jeblair | exists on 02 | 22:07 |
| clarkb | iirc we did a symlink somewhere? so this may be an order of operations thing | 22:07 |
| jeblair | lrwxrwxrwx. 1 root root 3 Jan 30 2014 /usr/bin/pip-python -> pip | 22:07 |
| clarkb | we do it in install_puppet.sh | 22:08 |
| clarkb | maybe we should add it to the pip manifest too | 22:08 |
| clarkb | so that when you include pip you get the symlink | 22:08 |
| jeblair | how did git01 get into this state then? | 22:08 |
| clarkb | (still an order of ops thing though) | 22:08 |
| clarkb | jeblair: not sure | 22:08 |
| fungi | those are backward-compat symlinks because at one point rhel provided a pip-python (or wast it python-pip?) command and then a later fedora release switched that around and rhel (and thus centos) epel followed suit | 22:08 |
| nibalizer | weird | 22:09 |
| fungi | and there was a point at which the pip provider and the rpm conflicted and set up a symlink to a mutually-recursive pair of symlinks pointed at each other and no executable | 22:09 |
| nibalizer | is this because we're using the git-pip script instead of the package? | 22:10 |
| jeblair | i don't see any recent package installs that would have affected it | 22:10 |
| jeblair | nibalizer: yep | 22:10 |
| fungi | and so if you tried to run pip, you basically ran an infinite symlink loop causing your shell to barf | 22:10 |
| *** Guest45523 has quit IRC | 22:10 | |
| *** dalgaaf__ has joined #openstack-infra | 22:11 | |
| jeblair | fungi, clarkb: should we write this off as cosmic rays and shall i create the symlink? | 22:11 |
| clarkb | jeblair: I think so | 22:11 |
| fungi | yeah, i think so too | 22:11 |
| *** eglynn-regus has joined #openstack-infra | 22:11 | |
| fungi | i believe we manually dealt with this on all our centos servers at one point, but maybe missed one or reintroduced it somehow | 22:12 |
| *** eglynn-officeafk has quit IRC | 22:12 | |
| jeblair | i believe this is what has prevented puppet runs recently; perhaps the thing that changed is puppet's behavior around this. maybe this was a nonfatal error before. | 22:12 |
| fungi | i think we shouldn't run into it on new centos hosts we bootstrap, it was a relatively unresolvable upgrade issue | 22:13 |
| *** jgrimm_ has joined #openstack-infra | 22:13 | |
| *** aysyanne has joined #openstack-infra | 22:13 | |
| daguegoneit | clarkb: hey, so I dug into this - https://bugs.launchpad.net/nova/+bug/1093812 | 22:13 |
| uvirtbot | Launchpad bug 1093812 in openstack-ci "nova API version doc should point to Nova developer doc" [High,Confirmed] | 22:13 |
| daguegoneit | the urls in question aren't coming from nova docs tree | 22:13 |
| daguegoneit | they might have in the past | 22:13 |
| *** dmsimard_away has joined #openstack-infra | 22:14 | |
| daguegoneit | but they are all stale files that infra needs to delete | 22:14 |
| fungi | it cropped up when epel changed the package on the fly (which seems odd for a stable distro, but maybe epel is not assumed to be especially stable?) | 22:14 |
| *** cnesa11 has joined #openstack-infra | 22:14 | |
| daguegoneit | I have commented as such in the bug | 22:14 |
| nibalizer | so i think its sensible for puppet to try to use python-pip on redhat but fall back to pip on redhat if that file doesn't exist | 22:14 |
| jeblair | ok this time we made it past git* | 22:14 |
| *** Hefeweiz1n has joined #openstack-infra | 22:14 | |
| nibalizer | so i think I'm going to PR puppet and try to add that feature | 22:14 |
| nibalizer | which doesn't fix us in the here and now | 22:14 |
| clarkb | daguegoneit: the urls are in stale files? | 22:14 |
| jeblair | i accepted keys for dfw and ord mirrors | 22:14 |
| fungi | nibalizer: well, we also use a custom pip provider | 22:14 |
| *** ctracey has quit IRC | 22:15 | |
| *** willroberts has quit IRC | 22:15 | |
| daguegoneit | http://docs.openstack.org/developer/nova/object.model.html ... is not something that's currently generated | 22:15 |
| nibalizer | fungi: the error we're getting now though seems to come from the default provider | 22:15 |
| *** asadoughi has joined #openstack-infra | 22:15 | |
| clarkb | fungi: only for pip3 | 22:15 |
| *** serverascode has quit IRC | 22:15 | |
| nibalizer | we only have a custom provider for python 3 stuff | 22:15 |
| fungi | oh, doubly fun | 22:15 |
| daguegoneit | note the lack of theme consistency | 22:15 |
| *** jgrimm_ is now known as zz_jgrimm_ | 22:15 | |
| daguegoneit | it looks like there is a bunch of content there from an old docs process | 22:15 |
| fungi | nibalizer: ahh, maybe so. for some reason i thought we ended up replacing the actual pip provider too | 22:15 |
| clarkb | daguegoneit: so this is something we will have to fix with new docs hosting platform | 22:15 |
| *** unicell has quit IRC | 22:15 | |
| daguegoneit | that google still finds | 22:15 |
| clarkb | daguegoneit: because currently there is no safe way of deleting old stuff | 22:15 |
| *** tsufiev_ has joined #openstack-infra | 22:15 | |
| *** tsufiev has quit IRC | 22:15 | |
| daguegoneit | clarkb: ok | 22:15 |
| *** ctracey has joined #openstack-infra | 22:15 | |
| nibalizer | fungi: there is a long lived review to have our own custom pip2 and pip3 providers (from morderd i believe) | 22:16 |
| daguegoneit | well, anyway, I have commented on the bug | 22:16 |
| nibalizer | i dont think it ever landed | 22:16 |
| fungi | daguegoneit: current docs publication automation only adds or replaces files, it doesn't delete | 22:16 |
| daguegoneit | and marked it invalid for nova | 22:16 |
| *** jraim___ has joined #openstack-infra | 22:16 | |
| daguegoneit | right, that seems... suboptimal | 22:16 |
| *** sdake has quit IRC | 22:16 | |
| annegentle_ | daguegoneit: nice nick. Want me to just delete the object.model.html file on the server? | 22:16 |
| fungi | daguegoneit: it's an upshot of using rackspace cloudsites and the jenkins ftp publisher | 22:16 |
| *** StevenK_ has joined #openstack-infra | 22:16 | |
| daguegoneit | annegentle_: sure, I don't know what else needs to be purged | 22:16 |
| *** electrichead has joined #openstack-infra | 22:17 | |
| fungi | nibalizer: oh, right, i'm confusing that review with the production code. that's a danger when you're a sysadmin doing code review ;) | 22:17 |
| *** willroberts_ has joined #openstack-infra | 22:17 | |
| *** serverascode has joined #openstack-infra | 22:17 | |
| clarkb | oh wow we don't have a lot of nodes available? | 22:17 |
| *** winston-d has joined #openstack-infra | 22:17 | |
| clarkb | that is a giant blob of purple we seem to be getting past | 22:18 |
| *** dstufft has quit IRC | 22:18 | |
| anteaya | goodness no | 22:18 |
| *** kmartin has quit IRC | 22:18 | |
| *** torandu_ has quit IRC | 22:18 | |
| *** cnesa10 has quit IRC | 22:18 | |
| *** winston-1 has quit IRC | 22:18 | |
| *** Adri2000 has quit IRC | 22:18 | |
| *** weshay has quit IRC | 22:18 | |
| *** dalgaaf_ has quit IRC | 22:18 | |
| *** yamahata__ has quit IRC | 22:18 | |
| *** asadough1 has quit IRC | 22:18 | |
| *** StevenK has quit IRC | 22:18 | |
| *** rkukura_ has quit IRC | 22:18 | |
| *** rakhmerov has quit IRC | 22:18 | |
| *** jgrimm has quit IRC | 22:18 | |
| *** gpocentek has quit IRC | 22:18 | |
| *** Svedrin has quit IRC | 22:18 | |
| *** leifmadsen has quit IRC | 22:18 | |
| *** afazekas has quit IRC | 22:18 | |
| *** dhellmann_ has quit IRC | 22:18 | |
| *** jesusaurus has quit IRC | 22:18 | |
| *** msuriar has quit IRC | 22:18 | |
| *** jraim__ has quit IRC | 22:18 | |
| *** saper has quit IRC | 22:18 | |
| *** dstufft has joined #openstack-infra | 22:18 | |
| clarkb | dfw ord and iad are all having errors | 22:19 |
| *** dalgaaf__ is now known as dalgaaf_ | 22:19 | |
| clarkb | hpcloud is carrying us right now | 22:19 |
| *** s0nea_ has joined #openstack-infra | 22:19 | |
| *** mikap has joined #openstack-infra | 22:19 | |
| fungi | for a little while dfw was back to happiness | 22:19 |
| anteaya | for that thin skim of blue that are running nodes | 22:19 |
| fungi | ugh | 22:19 |
| *** devanand1 has joined #openstack-infra | 22:19 | |
| JayF | clarkb: Rackspace currently has an incident that is likely impacted y'all | 22:20 |
| *** jraim___ is now known as jraim__ | 22:20 | |
| clarkb | JayF: thanks /me checks status page | 22:20 |
| JayF | s/impacted/impacting/ | 22:20 |
| *** enikanorov__ has joined #openstack-infra | 22:20 | |
| fungi | JayF: i think i can confirm that, yeah | 22:20 |
| JayF | clarkb: off and on all day | 22:20 |
| *** leifmadsen has joined #openstack-infra | 22:20 | |
| JayF | fungi: the "likely" was more or less wishful thinking :) | 22:20 |
| jeblair | i wish we were less "lucky" | 22:21 |
| *** gpocentek has joined #openstack-infra | 22:21 | |
| fungi | hrm, status.r.c actually doesn't show any issues i would expect to be related | 22:21 |
| anteaya | cloudsites looks impacted too | 22:21 |
| *** nelsnels_ has joined #openstack-infra | 22:21 | |
| fungi | may be lagging behind reality though | 22:21 |
| clarkb | ya just cloudsites and panels | 22:21 |
| *** torandu has joined #openstack-infra | 22:21 | |
| *** rakhmerov has joined #openstack-infra | 22:21 | |
| *** boris-42_ has joined #openstack-infra | 22:22 | |
| fungi | though the writeup says "errors within the Control Panels as well as API's" | 22:22 |
| clarkb | oh that may explain it if it is api laer | 22:22 |
| fungi | so maybe they count "panels" to include api access to control instances | 22:22 |
| jeblair | minor detail | 22:22 |
| fungi | JayF: ^ any translation for rackspace terminology there? | 22:23 |
| *** schang_ has joined #openstack-infra | 22:23 | |
| *** MIDENN_ has joined #openstack-infra | 22:23 | |
| clarkb | I am going to take this as an opporunity to run home and pack for the wekend | 22:23 |
| clarkb | back in a bit | 22:23 |
| fungi | i would expect api access to control next-gen cloud servers to show up as impacting next-gen cloud servers. maybe this is a compromise for not having an "apis" impact line | 22:23 |
| * clarkb is barcamping then getting keys to new place hopefully | 22:24 | |
| fungi | clarkb: oh! have fun at barcamp. those are always a blast | 22:24 |
| jeblair | i thought mordred fixed the cacti puppet updates? | 22:24 |
| JayF | fungi: clarkb: It's control plane for most cloud things, I think your interpretation about panels == control plane is accurate | 22:24 |
| *** weshay has joined #openstack-infra | 22:24 | |
| jeblair | it's running and is very slow :/ | 22:25 |
| mordred | I thought I did too | 22:25 |
| *** Clabbe has quit IRC | 22:25 | |
| *** gondoi is now known as zz_gondoi | 22:25 | |
| anteaya | clarkb: safe travels | 22:25 |
| *** Adri2000 has joined #openstack-infra | 22:25 | |
| *** dannywil_ has joined #openstack-infra | 22:26 | |
| *** Adri2000 is now known as Guest54358 | 22:26 | |
| *** andreaf has quit IRC | 22:26 | |
| *** yamahata__ has joined #openstack-infra | 22:26 | |
| *** patrickeast has quit IRC | 22:27 | |
| *** stevemar has joined #openstack-infra | 22:27 | |
| *** patrickeast has joined #openstack-infra | 22:27 | |
| *** openstack has joined #openstack-infra | 23:37 | |
| *** bookwar is now known as 16WAAF8A5 | 23:42 | |
| *** 6JTAAFWQC has joined #openstack-infra | 23:42 | |
| *** YorikSar has joined #openstack-infra | 23:42 | |
| *** Mmike has joined #openstack-infra | 23:42 | |
| *** dkliban has joined #openstack-infra | 23:42 | |
| *** tristanC has joined #openstack-infra | 23:42 | |
| *** Svedrin has joined #openstack-infra | 23:42 | |
| *** coolsvap has joined #openstack-infra | 23:42 | |
| *** ianw has joined #openstack-infra | 23:42 | |
| *** NobodyCam has joined #openstack-infra | 23:42 | |
| *** skraynev has joined #openstack-infra | 23:42 | |
| *** bookwar has joined #openstack-infra | 23:42 | |
| *** dangers has joined #openstack-infra | 23:42 | |
| *** thingee has joined #openstack-infra | 23:42 | |
| *** yfried|afk has joined #openstack-infra | 23:42 | |
| *** pballand has joined #openstack-infra | 23:42 | |
| *** dimsum_ has joined #openstack-infra | 23:42 | |
| *** masayukig has joined #openstack-infra | 23:42 | |
| *** jgriffith has joined #openstack-infra | 23:42 | |
| *** marcoemorais has joined #openstack-infra | 23:42 | |
| *** Clabbe has joined #openstack-infra | 23:42 | |
| *** afazekas has joined #openstack-infra | 23:42 | |
| *** cnesa11 has joined #openstack-infra | 23:42 | |
| *** rakhmerov has joined #openstack-infra | 23:42 | |
| *** katyafervent_awa has joined #openstack-infra | 23:42 | |
| *** Hefeweizen has joined #openstack-infra | 23:42 | |
| *** jlk has joined #openstack-infra | 23:42 | |
| *** eglynn__ has joined #openstack-infra | 23:42 | |
| *** sweston has joined #openstack-infra | 23:42 | |
| *** zul has joined #openstack-infra | 23:42 | |
| *** dmellado has joined #openstack-infra | 23:42 | |
| *** ZZelle has joined #openstack-infra | 23:42 | |
| *** ling-yun-1 has joined #openstack-infra | 23:42 | |
| *** ekarlso- has joined #openstack-infra | 23:42 | |
| *** nikhil_k has joined #openstack-infra | 23:42 | |
| *** pcrews has joined #openstack-infra | 23:42 | |
| *** dkehnx1 has joined #openstack-infra | 23:42 | |
| *** mrodden has joined #openstack-infra | 23:42 | |
| *** dhellmann has joined #openstack-infra | 23:42 | |
| *** Daviey has joined #openstack-infra | 23:42 | |
| *** openstackgerrit has joined #openstack-infra | 23:42 | |
| *** weshay has joined #openstack-infra | 23:42 | |
| *** krotscheck has joined #openstack-infra | 23:42 | |
| *** zaro has joined #openstack-infra | 23:42 | |
| *** tchaypo has joined #openstack-infra | 23:42 | |
| *** reed has joined #openstack-infra | 23:42 | |
| *** saper_ has joined #openstack-infra | 23:42 | |
| *** schang has joined #openstack-infra | 23:42 | |
| *** mkoderer has joined #openstack-infra | 23:42 | |
| *** dmsimard has joined #openstack-infra | 23:42 | |
| *** dteselkin has joined #openstack-infra | 23:42 | |
| *** m1dev has joined #openstack-infra | 23:42 | |
| *** slagle has joined #openstack-infra | 23:42 | |
| *** jmeridth has joined #openstack-infra | 23:42 | |
| *** vipul has joined #openstack-infra | 23:42 | |
| *** asadoughi has joined #openstack-infra | 23:42 | |
| *** dstufft has joined #openstack-infra | 23:42 | |
| *** spiffxp_ has joined #openstack-infra | 23:42 | |
| *** msuriar has joined #openstack-infra | 23:42 | |
| *** pabelanger has joined #openstack-infra | 23:42 | |
| *** gnuoy` has joined #openstack-infra | 23:42 | |
| *** che-arne has joined #openstack-infra | 23:42 | |
| *** SpamapS has joined #openstack-infra | 23:42 | |
| *** mika has joined #openstack-infra | 23:42 | |
| *** winston-d has joined #openstack-infra | 23:42 | |
| *** AaronGr has joined #openstack-infra | 23:42 | |
| *** HenryG has joined #openstack-infra | 23:42 | |
| *** russellb has joined #openstack-infra | 23:42 | |
| *** blkperl has joined #openstack-infra | 23:42 | |
| *** _nadya__ has joined #openstack-infra | 23:42 | |
| *** Shrews has joined #openstack-infra | 23:42 | |
| *** gus has joined #openstack-infra | 23:42 | |
| *** pbelanyi has joined #openstack-infra | 23:42 | |
| *** iax7 has joined #openstack-infra | 23:42 | |
| *** mkerrin has joined #openstack-infra | 23:42 | |
| *** joa has joined #openstack-infra | 23:42 | |
| *** stevebaker has joined #openstack-infra | 23:42 | |
| *** davidlenwell has joined #openstack-infra | 23:42 | |
| *** medieval1_ has joined #openstack-infra | 23:42 | |
| *** marzif_ has joined #openstack-infra | 23:42 | |
| *** aysyanne has joined #openstack-infra | 23:42 | |
| *** markmc has joined #openstack-infra | 23:42 | |
| *** rhsu has joined #openstack-infra | 23:42 | |
| *** Guest74618 has joined #openstack-infra | 23:42 | |
| *** greghaynes has joined #openstack-infra | 23:42 | |
| *** chuckC has joined #openstack-infra | 23:42 | |
| *** jyuso012 has joined #openstack-infra | 23:42 | |
| *** doug-fish1 has joined #openstack-infra | 23:42 | |
| *** yamamoto has joined #openstack-infra | 23:42 | |
| *** Sukhdev has joined #openstack-infra | 23:42 | |
| *** baoli_ has joined #openstack-infra | 23:42 | |
| *** lbragstad has joined #openstack-infra | 23:42 | |
| *** jroll has joined #openstack-infra | 23:42 | |
| *** d34dh0r53 has joined #openstack-infra | 23:42 | |
| *** gondoi has joined #openstack-infra | 23:42 | |
| *** blitzrage has joined #openstack-infra | 23:42 | |
| *** adam_g has joined #openstack-infra | 23:42 | |
| *** gokrokve has joined #openstack-infra | 23:42 | |
| *** sc68cal has joined #openstack-infra | 23:42 | |
| *** dimtruck has joined #openstack-infra | 23:42 | |
| *** dolphm has joined #openstack-infra | 23:42 | |
| *** boris-42 has joined #openstack-infra | 23:42 | |
| *** nibalizer has joined #openstack-infra | 23:42 | |
| *** StevenK has joined #openstack-infra | 23:42 | |
| *** HeOS__ has joined #openstack-infra | 23:42 | |
| *** gpocente1 has joined #openstack-infra | 23:42 | |
| *** fungi has joined #openstack-infra | 23:42 | |
| *** BadCub_ has joined #openstack-infra | 23:42 | |
| *** persia__ has joined #openstack-infra | 23:42 | |
| *** QuinnyPi1 has joined #openstack-infra | 23:42 | |
| *** garyh has joined #openstack-infra | 23:42 | |
| *** kevinbenton has joined #openstack-infra | 23:42 | |
| *** harlowja_ has joined #openstack-infra | 23:42 | |
| *** juice has joined #openstack-infra | 23:42 | |
| *** gargola_ has joined #openstack-infra | 23:42 | |
| *** hemna has joined #openstack-infra | 23:42 | |
| *** patrickeast has joined #openstack-infra | 23:42 | |
| *** Guest54358 has joined #openstack-infra | 23:42 | |
| *** torandu has joined #openstack-infra | 23:42 | |
| *** enikanorov__ has joined #openstack-infra | 23:42 | |
| *** devananda has joined #openstack-infra | 23:42 | |
| *** s0nea has joined #openstack-infra | 23:42 | |
| *** serverascode has joined #openstack-infra | 23:42 | |
| *** willroberts_ has joined #openstack-infra | 23:42 | |
| *** electrichead has joined #openstack-infra | 23:42 | |
| *** jraim__ has joined #openstack-infra | 23:42 | |
| *** ctracey has joined #openstack-infra | 23:42 | |
| *** tsufiev_ has joined #openstack-infra | 23:42 | |
| *** zz_jgrimm_ has joined #openstack-infra | 23:42 | |
| *** simonmcc____ has joined #openstack-infra | 23:42 | |
| *** michchap has joined #openstack-infra | 23:42 | |
| *** nimrodsun_ has joined #openstack-infra | 23:42 | |
| *** vladan has joined #openstack-infra | 23:42 | |
| *** Alex_Gaynor has joined #openstack-infra | 23:42 | |
| *** Guest20522 has joined #openstack-infra | 23:42 | |
| *** markvan has joined #openstack-infra | 23:42 | |
| *** jamielennox has joined #openstack-infra | 23:42 | |
| *** jcooley has joined #openstack-infra | 23:42 | |
| *** sballe__ has joined #openstack-infra | 23:42 | |
| *** ramashri has joined #openstack-infra | 23:42 | |
| *** vigneshvar has joined #openstack-infra | 23:42 | |
| *** marun has joined #openstack-infra | 23:42 | |
| *** gyee has joined #openstack-infra | 23:42 | |
| *** bhuvan_ has joined #openstack-infra | 23:42 | |
| *** penguinRaider has joined #openstack-infra | 23:42 | |
| *** praneshp has joined #openstack-infra | 23:42 | |
| *** Ryan_Lane has joined #openstack-infra | 23:42 | |
| *** mestery has joined #openstack-infra | 23:42 | |
| *** markmcclain has joined #openstack-infra | 23:42 | |
| *** mmaglana has joined #openstack-infra | 23:42 | |
| *** rcarrillocruz has joined #openstack-infra | 23:42 | |
| *** rwsu has joined #openstack-infra | 23:42 | |
| *** wenlock has joined #openstack-infra | 23:42 | |
| *** dtroyer has joined #openstack-infra | 23:42 | |
| *** jpeeler has joined #openstack-infra | 23:42 | |
| *** niska has joined #openstack-infra | 23:42 | |
| *** flashgordon has joined #openstack-infra | 23:42 | |
| *** asettle has joined #openstack-infra | 23:42 | |
| *** ryanpetre has joined #openstack-infra | 23:42 | |
| *** LinuxJedi has joined #openstack-infra | 23:42 | |
| *** gothicmindfood has joined #openstack-infra | 23:42 | |
| *** JayF has joined #openstack-infra | 23:42 | |
| *** portante has joined #openstack-infra | 23:42 | |
| *** sandywalsh has joined #openstack-infra | 23:42 | |
| *** miqui has joined #openstack-infra | 23:42 | |
| *** sileht has joined #openstack-infra | 23:42 | |
| *** krtaylor has joined #openstack-infra | 23:42 | |
| *** dkehn has joined #openstack-infra | 23:42 | |
| *** _shaps_ has joined #openstack-infra | 23:42 | |
| *** asselin has joined #openstack-infra | 23:42 | |
| *** j^2 has joined #openstack-infra | 23:42 | |
| *** mattoliverau has joined #openstack-infra | 23:42 | |
| *** wendar has joined #openstack-infra | 23:42 | |
| *** dosaboy has joined #openstack-infra | 23:42 | |
| *** comstud has joined #openstack-infra | 23:42 | |
| *** odyi has joined #openstack-infra | 23:42 | |
| *** ttx has joined #openstack-infra | 23:42 | |
| *** flaper87|afk has joined #openstack-infra | 23:42 | |
| *** bodepd_ has joined #openstack-infra | 23:42 | |
| *** vponomaryov has joined #openstack-infra | 23:42 | |
| *** bogdando has joined #openstack-infra | 23:42 | |
| *** yongli has joined #openstack-infra | 23:42 | |
| *** sbadia has joined #openstack-infra | 23:42 | |
| *** morgan_remote has joined #openstack-infra | 23:42 | |
| *** pfallenop has joined #openstack-infra | 23:42 | |
| *** ilyashakhat has joined #openstack-infra | 23:42 | |
| *** Mithrandir has joined #openstack-infra | 23:42 | |
| *** imcsk8 has joined #openstack-infra | 23:42 | |
| *** toabctl has joined #openstack-infra | 23:42 | |
| *** bmwiedemann has joined #openstack-infra | 23:42 | |
| *** GheRivero_pto has joined #openstack-infra | 23:42 | |
| *** enikanorov has joined #openstack-infra | 23:42 | |
| *** shardy has joined #openstack-infra | 23:42 | |
| *** psedlak has joined #openstack-infra | 23:42 | |
| *** primeministerp has joined #openstack-infra | 23:42 | |
| *** jhesketh has joined #openstack-infra | 23:42 | |
| *** paul-- has joined #openstack-infra | 23:42 | |
| *** morgan has joined #openstack-infra | 23:42 | |
| *** crinkle has joined #openstack-infra | 23:42 | |
| *** mrda-vacation has joined #openstack-infra | 23:42 | |
| *** mmitchell has joined #openstack-infra | 23:42 | |
| *** Ng has joined #openstack-infra | 23:42 | |
| *** mflobo has joined #openstack-infra | 23:42 | |
| *** dougwig has joined #openstack-infra | 23:42 | |
| *** esmute has joined #openstack-infra | 23:42 | |
| *** dmitryme has joined #openstack-infra | 23:42 | |
| *** therve has joined #openstack-infra | 23:42 | |
| *** vishy has joined #openstack-infra | 23:42 | |
| *** zz_johnthetubagu has joined #openstack-infra | 23:42 | |
| *** gema has joined #openstack-infra | 23:42 | |
| *** rushiagr_away has joined #openstack-infra | 23:42 | |
| *** erw_ has joined #openstack-infra | 23:42 | |
| *** maurosr has joined #openstack-infra | 23:42 | |
| *** rossella_s has joined #openstack-infra | 23:42 | |
| *** cyeoh has joined #openstack-infra | 23:42 | |
| *** nijaba has joined #openstack-infra | 23:42 | |
| *** charz has joined #openstack-infra | 23:42 | |
| *** lxsli has joined #openstack-infra | 23:42 | |
| *** fabo has joined #openstack-infra | 23:42 | |
| *** Hunner has joined #openstack-infra | 23:42 | |
| *** EmilienM has joined #openstack-infra | 23:42 | |
| *** abramley has joined #openstack-infra | 23:42 | |
| *** niedbalski_ has joined #openstack-infra | 23:42 | |
| *** pleia2 has joined #openstack-infra | 23:42 | |
| *** jamespd_ has joined #openstack-infra | 23:42 | |
| *** zhiyan has joined #openstack-infra | 23:42 | |
| *** isviridov has joined #openstack-infra | 23:42 | |
| *** timrc has joined #openstack-infra | 23:42 | |
| *** Vek_ has joined #openstack-infra | 23:42 | |
| *** radez_g0n3 has joined #openstack-infra | 23:42 | |
| *** DuncanT-away has joined #openstack-infra | 23:42 | |
| *** mnaser has joined #openstack-infra | 23:42 | |
| *** phschwartz has joined #openstack-infra | 23:42 | |
| *** Vivek has joined #openstack-infra | 23:42 | |
| *** chmouel has joined #openstack-infra | 23:42 | |
| *** notmyname has joined #openstack-infra | 23:42 | |
| *** funzo has joined #openstack-infra | 23:42 | |
| *** soren has joined #openstack-infra | 23:42 | |
| *** bradm has joined #openstack-infra | 23:42 | |
| *** JoshNang has joined #openstack-infra | 23:42 | |
| *** kashyap has joined #openstack-infra | 23:42 | |
| *** timfreund has joined #openstack-infra | 23:42 | |
| *** gaelL has joined #openstack-infra | 23:42 | |
| *** SergeyLukjanov has joined #openstack-infra | 23:42 | |
| *** zz_naotok has joined #openstack-infra | 23:42 | |
| *** jvrbanac has joined #openstack-infra | 23:42 | |
| *** spiffxp has joined #openstack-infra | 23:42 | |
| *** sdague has joined #openstack-infra | 23:42 | |
| *** plars has joined #openstack-infra | 23:42 | |
| *** dvorak has joined #openstack-infra | 23:42 | |
| *** mikal has joined #openstack-infra | 23:42 | |
| *** mugsie has joined #openstack-infra | 23:42 | |
| *** uvirtbot has joined #openstack-infra | 23:42 | |
| *** wolsen has joined #openstack-infra | 23:42 | |
| *** DinaBelova has joined #openstack-infra | 23:42 | |
| *** gmurphy has joined #openstack-infra | 23:42 | |
| *** tteggel has joined #openstack-infra | 23:42 | |
| *** proffalk1n has joined #openstack-infra | 23:42 | |
| *** TheJulia has joined #openstack-infra | 23:42 | |
| *** jd__ has joined #openstack-infra | 23:42 | |
| *** NikitaKonovalov has joined #openstack-infra | 23:42 | |
| *** superdan has joined #openstack-infra | 23:42 | |
| *** ruhe has joined #openstack-infra | 23:42 | |
| *** dguerri has joined #openstack-infra | 23:42 | |
| *** froek has joined #openstack-infra | 23:42 | |
| *** clarkb has joined #openstack-infra | 23:42 | |
| *** mtreinish has joined #openstack-infra | 23:42 | |
| *** BobBall_AWOL has joined #openstack-infra | 23:42 | |
| *** clayg has joined #openstack-infra | 23:42 | |
| *** Kiall has joined #openstack-infra | 23:42 | |
| *** lifeless has joined #openstack-infra | 23:42 | |
| *** tcammann_ has joined #openstack-infra | 23:42 | |
| *** jeblair has joined #openstack-infra | 23:42 | |
| *** anteaya has joined #openstack-infra | 23:42 | |
| *** jbjohnso__ has joined #openstack-infra | 23:42 | |
| *** a2hill has joined #openstack-infra | 23:42 | |
| *** loquacities has joined #openstack-infra | 23:42 | |
| *** zigo has joined #openstack-infra | 23:42 | |
| *** gilliard has joined #openstack-infra | 23:42 | |
| *** cinerama has joined #openstack-infra | 23:42 | |
| *** mordred has joined #openstack-infra | 23:42 | |
| *** SlickNik has joined #openstack-infra | 23:42 | |
| *** jdsn has joined #openstack-infra | 23:42 | |
| *** morgabra has joined #openstack-infra | 23:42 | |
| *** d0ugal has joined #openstack-infra | 23:42 | |
| *** ChanServ has joined #openstack-infra | 23:42 | |
| *** dstanek has joined #openstack-infra | 23:42 | |
| *** sendak.freenode.net sets mode: +o ChanServ | 23:42 | |
| *** Steap_ has joined #openstack-infra | 23:42 | |
| fungi | nibalizer: curl: (60) SSL certificate problem: unable to get local issuer certificate | 23:42 |
| *** dimsum_ has quit IRC | 23:43 | |
| nibalizer | fungi: either add -k(insecure) or --cacert `puppet config print cacert` | 23:44 |
| *** hemna is now known as 18VAAXY46 | 23:44 | |
| fungi | (also the url needed quoting because of the &) | 23:44 |
| nibalizer | i would reccomend -k since it doesnt matter and im pretty sure the second idea is syntactically incorrect | 23:44 |
| *** grantbow has joined #openstack-infra | 23:44 | |
| *** bcrochet_ has joined #openstack-infra | 23:44 | |
| *** alaski has joined #openstack-infra | 23:44 | |
| *** harlowja has joined #openstack-infra | 23:44 | |
| *** annegentle_ has joined #openstack-infra | 23:44 | |
| *** hemna has joined #openstack-infra | 23:44 | |
| *** bcrochet has joined #openstack-infra | 23:44 | |
| *** harlowja has quit IRC | 23:44 | |
| *** annegentle_ has quit IRC | 23:44 | |
| *** hemna has quit IRC | 23:44 | |
| *** bcrochet has quit IRC | 23:44 | |
| *** bcrochet_ is now known as bcrochet | 23:44 | |
| *** annegentle has joined #openstack-infra | 23:45 | |
| *** aviau has joined #openstack-infra | 23:45 | |
| *** 18VAAXY46 is now known as hemna | 23:45 | |
| fungi | -k(insecure) also seems to be incorrect syntax, even after escaping the parens | 23:45 |
| *** gokrokve has quit IRC | 23:46 | |
| fungi | the --cacert option worked though | 23:46 |
| *** dimtruck is now known as zz_dimtruck | 23:46 | |
| nibalizer | no -k on its own | 23:47 |
| nibalizer | sorry the parens was my syntax | 23:47 |
| nibalizer | cool though | 23:47 |
| fungi | or generated a crapton of json as a response anyway | 23:47 |
| nibalizer | you got a response of some kind? | 23:47 |
| nibalizer | okay that says to me that the master has no cert problems interacting with puppetdb | 23:47 |
| fungi | er, maybe yaml | 23:47 |
| fungi | anyway some serialized response | 23:47 |
| nibalizer | yea if it didn't send you on your way you're probably fine | 23:47 |
| nibalizer | also sounds like puppetdb has made its api a little more sane | 23:48 |
| fungi | it's info on other machines which have reported in, looks like | 23:48 |
| nibalizer | yea, that was just a hello world test to check for certificate functionality | 23:48 |
| nibalizer | i probably should have said that at the beginning | 23:48 |
| fungi | i figured | 23:48 |
| *** wenlock has quit IRC | 23:49 | |
| nibalizer | if you apt-cache policy puppetdb on the puppetdb server, is it at latest? | 23:49 |
| fungi | not installed on the new puppet master | 23:49 |
| fungi | oh, on the puppetdb server | 23:49 |
| nibalizer | yea | 23:49 |
| fungi | nope, we never upgraded it. d'oh! | 23:50 |
| fungi | Installed: 1.6.3-1puppetlabs1 Candidate: 2.2.0-1puppetlabs1 | 23:50 |
| fungi | it's upgraded now | 23:50 |
| fungi | looks like that restarted the jvm it uses too | 23:51 |
| fungi | now we wait and drink? | 23:51 |
| nibalizer | uh | 23:52 |
| nibalizer | well we definitely do the second | 23:52 |
| nibalizer | want to run puppet on the puppet3master again | 23:52 |
| fungi | sure | 23:52 |
| *** pcrews has quit IRC | 23:52 | |
| nibalizer | that should tell us immedeatly if things are working | 23:52 |
| nibalizer | or worse, if we broke everything | 23:52 |
| fungi | completed with no errors | 23:52 |
| nibalizer | woot | 23:53 |
| fungi | but then again, it was doing that before | 23:53 |
| nibalizer | and report is in puppetdb! | 23:53 |
| nibalizer | http://puppetboard.openstack.org/node/puppetmaster.openstack.org | 23:53 |
| fungi | checking logs on the db | 23:53 |
| nibalizer | YES | 23:53 |
| clarkb | http://puppetboard.openstack.org/node/puppetmaster.openstack.org looks like it worked | 23:53 |
| *** bswartz has joined #openstack-infra | 23:53 | |
| fungi | oh, it was just lagging | 23:53 |
| clarkb | fungi: nibalizer: so what was the problem? | 23:53 |
| nibalizer | i think you need equal versions of puppetdb and puppetdb-terminus | 23:54 |
| clarkb | oh I see and us not explicitly updating puppetdb borked it | 23:54 |
| *** dimsum_ has joined #openstack-infra | 23:54 | |
| *** fifieldt has joined #openstack-infra | 23:54 | |
| *** lyxus has joined #openstack-infra | 23:54 | |
| *** andreaf has joined #openstack-infra | 23:54 | |
| *** changbl has joined #openstack-infra | 23:54 | |
| *** cody-somerville has joined #openstack-infra | 23:54 | |
| *** bnemec has joined #openstack-infra | 23:54 | |
| fungi | we might should pin puppetdb too then? | 23:54 |
| nibalizer | yea | 23:54 |
| nibalizer | yes | 23:54 |
| fungi | since it allowed us to upgrade puppetdb-terminus without forcing a puppetdb upgrade | 23:54 |
| nibalizer | im a bit leery of this 'auto upgrade puppet' business | 23:54 |
| *** dimsum_ has quit IRC | 23:54 | |
| nibalizer | i know we agreed to 3.x pin but pinning to a known version and upgrading on our terms is somtehing i would prefer | 23:55 |
| *** dimsum_ has joined #openstack-infra | 23:55 | |
| nibalizer | but thats a topic for next week or the meeting or the summit | 23:55 |
| clarkb | nibalizer: well jeblair said this or ansible :P | 23:55 |
| nibalizer | haha | 23:55 |
| fungi | yeah, the down side to "upgrading on our terms" is that if it's every minor release bump... that's potentially lots of these "log into all the servers" fire drills | 23:56 |
| nibalizer | nah, cus we could update the pin, and puppet + automatic upgrades would gently move us up | 23:56 |
| nibalizer | fungi: want to run one more check for me? | 23:56 |
| *** flaper87|afk has quit IRC | 23:56 | |
| *** flaper87|afk has joined #openstack-infra | 23:56 | |
| nibalizer | will you run 'install_modules.sh' manually and see if its all working good | 23:56 |
| nibalizer | there was a puppet 3.7 bug where they broke 'puppet module install' | 23:57 |
| fungi | nibalizer: on a random machine, or the master? | 23:57 |
| nibalizer | master | 23:57 |
| nibalizer | you can read roots crontab | 23:57 |
| nibalizer | to see what to do | 23:57 |
| *** pcrews has joined #openstack-infra | 23:57 | |
| *** zz_jgrimm_ has quit IRC | 23:58 | |
| *** zz_jgrimm_ has joined #openstack-infra | 23:58 | |
| *** ctracey has quit IRC | 23:58 | |
| *** ctracey has joined #openstack-infra | 23:58 | |
| *** jraim__ has quit IRC | 23:58 | |
| *** jraim__ has joined #openstack-infra | 23:58 | |
| *** serverascode has quit IRC | 23:58 | |
| *** serverascode has joined #openstack-infra | 23:58 | |
| *** willroberts_ has quit IRC | 23:58 | |
| *** willroberts_ has joined #openstack-infra | 23:58 | |
| *** hemna has quit IRC | 23:58 | |
| *** hemna has joined #openstack-infra | 23:58 | |
| *** gargola_ has quit IRC | 23:58 | |
| *** gargola_ has joined #openstack-infra | 23:58 | |
| *** harlowja_ has quit IRC | 23:58 | |
| *** harlowja_ has joined #openstack-infra | 23:58 | |
| fungi | nibalizer: http://paste.openstack.org/show/110856/ | 23:58 |
| *** baoli_ has quit IRC | 23:58 | |
| *** baoli_ has joined #openstack-infra | 23:58 | |
| *** marzif_ has quit IRC | 23:59 | |
| *** marzif_ has joined #openstack-infra | 23:59 | |
| nibalizer | uh thats fine | 23:59 |
| fungi | and 'echo $?' says 0 | 23:59 |
| *** mkerrin has quit IRC | 23:59 | |
| *** mkerrin has joined #openstack-infra | 23:59 | |
| *** iax7 has quit IRC | 23:59 | |
| *** iax7 has joined #openstack-infra | 23:59 | |
| *** pbelanyi has quit IRC | 23:59 | |
| *** pbelanyi has joined #openstack-infra | 23:59 | |
| nibalizer | weird that its trying to puppet module install prostgres | 23:59 |
| *** che-arne has quit IRC | 23:59 | |
| *** che-arne has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!