| *** ffrank has quit IRC | 00:01 | |
| *** ihrachys has joined #openstack-infra | 00:01 | |
| *** xarses has quit IRC | 00:05 | |
| *** vhosakot has quit IRC | 00:05 | |
| *** EricGonczer_ has quit IRC | 00:08 | |
| *** anteaya has quit IRC | 00:09 | |
| pabelanger | rcarrillocruz: looks like we have a DNS issue with baremetal00.vanilla, it is failing to resolve google DNS. Is there something local? | 00:09 |
|---|---|---|
| *** anteaya has joined #openstack-infra | 00:09 | |
| *** Julien-zte has joined #openstack-infra | 00:10 | |
| clarkb | pabelanger: we p robably dont want local as its been problematic in the past. we should make sure there are no firewall drop rules for our hosts | 00:11 |
| *** ddieterly has joined #openstack-infra | 00:14 | |
| openstackgerrit | C.J. Collier proposed openstack-infra/gerritbot: Improve gerritbot error handling https://review.openstack.org/287056 | 00:14 |
| *** claudiub has quit IRC | 00:15 | |
| zaro | are you able post to SB using rest it’s rest api? | 00:16 |
| zaro | anteaya: i mean without gerri plugin? | 00:16 |
| *** ihrachys has quit IRC | 00:17 | |
| *** sarob has quit IRC | 00:17 | |
| anteaya | zaro: good point, I should try that | 00:17 |
| openstackgerrit | Sachi King proposed openstack-infra/zuul: Support post jobs by supporting rev checkout https://review.openstack.org/293194 | 00:17 |
| pabelanger | clarkb: Agreed, will check firewalls. Was hoping something was local to testing networking | 00:18 |
| Nakato | jeblair: Newrev is up https://review.openstack.org/#/c/293194/14 | 00:18 |
| *** thorst has joined #openstack-infra | 00:19 | |
| pabelanger | clarkb: current firewall settings: http://paste.openstack.org/show/515864/ | 00:20 |
| *** pvaneck has quit IRC | 00:20 | |
| *** tqtran has quit IRC | 00:21 | |
| *** sarob has joined #openstack-infra | 00:21 | |
| clarkb | pabelanger: ya I am worried about hp firewalls not on host firewall | 00:21 |
| *** gyee has quit IRC | 00:21 | |
| pabelanger | clarkb: same | 00:21 |
| clarkb | not being able to talk ti 8.8.8.8 seems like hp firewall | 00:21 |
| pabelanger | right | 00:22 |
| pabelanger | I suspect that is the issue too | 00:22 |
| anteaya | pleia2: have you any sense of why I would get a 302 returned when I get the systeminfo on the api when the same url resolves using my browser? http://paste.openstack.org/show/515865/ | 00:22 |
| *** Jeffrey4l has joined #openstack-infra | 00:23 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Remove vexxhost from nodepool.o.o clouds.yaml https://review.openstack.org/329234 | 00:25 |
| *** denisra has quit IRC | 00:25 | |
| pabelanger | vexxhost properly cleaned up in nodepool.o.o now | 00:25 |
| pleia2 | anteaya: hm, that's a temporary redirect notice | 00:26 |
| *** piet_ has joined #openstack-infra | 00:26 | |
| *** denisra has joined #openstack-infra | 00:26 | |
| anteaya | pleia2: fair enough, why are we doing that? | 00:27 |
| *** yuanying has quit IRC | 00:27 | |
| *** lvdongbing has joined #openstack-infra | 00:27 | |
| pleia2 | anteaya: oh, I think it's because http > https | 00:27 |
| pleia2 | we're forcing use of https | 00:27 |
| *** yuanying has joined #openstack-infra | 00:27 | |
| pleia2 | so as far as I can tell, you're going to http and it's sending you to https | 00:27 |
| *** baoli has quit IRC | 00:28 | |
| pleia2 | anteaya: try: curl --insecure https://storyboard-dev.openstack.org/api/v1/systeminfo | 00:28 |
| *** baoli has joined #openstack-infra | 00:28 | |
| anteaya | pleia2: I'll try that | 00:28 |
| *** betherly has quit IRC | 00:28 | |
| *** betherly has joined #openstack-infra | 00:29 | |
| anteaya | thanks | 00:29 |
| anteaya | phone | 00:29 |
| pleia2 | sure | 00:29 |
| *** mtanin___ has joined #openstack-infra | 00:30 | |
| *** yamahata has quit IRC | 00:30 | |
| *** SotK has quit IRC | 00:31 | |
| *** mtanino has quit IRC | 00:31 | |
| *** SotK has joined #openstack-infra | 00:31 | |
| *** yamahata has joined #openstack-infra | 00:32 | |
| *** sflanigan has quit IRC | 00:34 | |
| *** sarob has quit IRC | 00:34 | |
| *** amitgandhinz has quit IRC | 00:35 | |
| *** kzaitsev_mb has quit IRC | 00:37 | |
| *** oanson has joined #openstack-infra | 00:39 | |
| *** jamesmcarthur has joined #openstack-infra | 00:40 | |
| *** yamada-h has joined #openstack-infra | 00:40 | |
| *** sarob has joined #openstack-infra | 00:40 | |
| *** thorst has quit IRC | 00:40 | |
| *** sarob has quit IRC | 00:40 | |
| *** bhavik has quit IRC | 00:40 | |
| *** zz_dimtruck is now known as dimtruck | 00:41 | |
| *** thorst has joined #openstack-infra | 00:41 | |
| *** jamesmcarthur has quit IRC | 00:44 | |
| *** yamada-h has quit IRC | 00:44 | |
| *** bhavik has joined #openstack-infra | 00:45 | |
| *** sputnik13_ has quit IRC | 00:46 | |
| *** sputnik13 has joined #openstack-infra | 00:47 | |
| *** Sukhdev has quit IRC | 00:49 | |
| *** thorst has quit IRC | 00:49 | |
| *** sflanigan has joined #openstack-infra | 00:51 | |
| anteaya | pleia2: sorry yes that was the issue when I tried this last week, thanks, I have no memory for anything anymore | 00:51 |
| *** baoli has quit IRC | 00:53 | |
| *** dimtruck is now known as zz_dimtruck | 00:57 | |
| *** bpokorny_ has joined #openstack-infra | 01:01 | |
| *** bpokorny has quit IRC | 01:04 | |
| *** zul has joined #openstack-infra | 01:04 | |
| *** Apoorva_ has quit IRC | 01:04 | |
| *** bpokorny_ has quit IRC | 01:05 | |
| *** amotoki has joined #openstack-infra | 01:06 | |
| *** bhavik has quit IRC | 01:07 | |
| *** bhavik has joined #openstack-infra | 01:11 | |
| *** Daisy has joined #openstack-infra | 01:12 | |
| *** baoli has joined #openstack-infra | 01:16 | |
| *** rhallisey has quit IRC | 01:16 | |
| *** amotoki has quit IRC | 01:21 | |
| *** hongbin has joined #openstack-infra | 01:24 | |
| *** baoli has quit IRC | 01:26 | |
| *** grassass has joined #openstack-infra | 01:26 | |
| *** ddieterly is now known as ddieterly[away] | 01:28 | |
| *** bhavik has quit IRC | 01:29 | |
| *** rwsu has quit IRC | 01:30 | |
| *** yanyanhu has joined #openstack-infra | 01:32 | |
| *** ddieterly[away] has quit IRC | 01:33 | |
| *** Goneri has joined #openstack-infra | 01:34 | |
| *** thorst has joined #openstack-infra | 01:34 | |
| *** Daisy has quit IRC | 01:34 | |
| *** Daisy has joined #openstack-infra | 01:35 | |
| *** Daisy_ has joined #openstack-infra | 01:35 | |
| *** amitgandhinz has joined #openstack-infra | 01:36 | |
| *** Daisy_ has quit IRC | 01:36 | |
| *** Daisy_ has joined #openstack-infra | 01:36 | |
| openstackgerrit | Numan Siddique proposed openstack-infra/project-config: Add networking-ovn in the 'PROJECTS' for dsvm-functional job https://review.openstack.org/329241 | 01:37 |
| *** Goneri has quit IRC | 01:38 | |
| *** piet_ has quit IRC | 01:39 | |
| *** Daisy has quit IRC | 01:39 | |
| *** amotoki has joined #openstack-infra | 01:39 | |
| *** tkelsey has joined #openstack-infra | 01:40 | |
| *** zz_dimtruck is now known as dimtruck | 01:41 | |
| *** zul has quit IRC | 01:41 | |
| *** amitgandhinz has quit IRC | 01:41 | |
| *** yamada-h has joined #openstack-infra | 01:43 | |
| *** tkelsey has quit IRC | 01:44 | |
| anteaya | pleia2: if you are still around I just tried `curl --insecure -i https://storyboard-dev.openstack.org/api/v1/stories/11/tasks/12 -X DELETE` which gave me a 500 but the way I read the documenation it is a valid command: http://docs.openstack.org/infra/storyboard/webapi/v1.html#delete--v1-stories--story_id--tasks | 01:45 |
| anteaya | it is possible I am not doing something I should know to do | 01:45 |
| anteaya | pleia2: I am curious what the storyboard-dev logs say in response to my DELETE attempt | 01:46 |
| *** yamada-h has quit IRC | 01:47 | |
| *** sflanigan has quit IRC | 01:48 | |
| *** bhavik has joined #openstack-infra | 01:48 | |
| pleia2 | anteaya: I am working on other things this evening and haven't looked at the storyboard API at all, but presumably you need some kind of authentication token in order to do any edit-type tasks | 01:48 |
| anteaya | okay thanks, sorry to bother you | 01:49 |
| anteaya | and yeah that would make sense | 01:49 |
| *** ddieterly has joined #openstack-infra | 01:49 | |
| pleia2 | anteaya: the straight curl you're doing doesn't pass auth info, so I think it's giving a server error since it doesn't know what to do if you're not "logged in" | 01:49 |
| anteaya | I'll try folks tomorrow to see what it is | 01:49 |
| anteaya | okay fair enough, thank you | 01:50 |
| *** zul has joined #openstack-infra | 01:54 | |
| *** bhavik has quit IRC | 01:54 | |
| *** xarses has joined #openstack-infra | 01:57 | |
| anteaya | I am able to query my user info: http://docs.openstack.org/infra/storyboard/webapi/v1.html#get--v1-users-%28user_id%29 | 01:57 |
| anteaya | I figured out my user_id | 01:57 |
| *** Daisy has joined #openstack-infra | 01:58 | |
| anteaya | I'm not sure what bits it wants me to offer it in a edit command since anyone has access to that info | 01:58 |
| *** ddieterly has quit IRC | 01:59 | |
| *** PollyZ has joined #openstack-infra | 01:59 | |
| *** sflanigan has joined #openstack-infra | 02:00 | |
| anteaya | this did give me a 401 unauthorized: curl --insecure -i https://storyboard-dev.openstack.org/api/v1/users/13/preferences -X GET | 02:00 |
| *** Daisy_ has quit IRC | 02:00 | |
| anteaya | so I wonder why I got a 500 earlier instead of a 401 | 02:01 |
| anteaya | anyway enough talking to myself, time to shut down for the night | 02:02 |
| anteaya | thanks for your help | 02:02 |
| anteaya | g'night | 02:02 |
| Qiming | infra team, pls help invite openstackgerrit back to #senlin channel, thank you!!! | 02:02 |
| *** PollyZ has quit IRC | 02:04 | |
| *** dimtruck is now known as zz_dimtruck | 02:06 | |
| *** erlon has quit IRC | 02:06 | |
| *** zz_dimtruck is now known as dimtruck | 02:06 | |
| *** amotoki has quit IRC | 02:07 | |
| *** piet_ has joined #openstack-infra | 02:07 | |
| *** amotoki has joined #openstack-infra | 02:07 | |
| *** Daisy has quit IRC | 02:11 | |
| *** Daisy has joined #openstack-infra | 02:11 | |
| *** yamahata has quit IRC | 02:12 | |
| clarkb | Qiming: http://docs.openstack.org/infra/system-config/irc.html you can propose the change but we also ask that the channel.be configured properly first | 02:13 |
| Qiming | thanks for the help clarkb, looking | 02:14 |
| Qiming | it was there, but we lost it for a few days now | 02:15 |
| *** mtanino has joined #openstack-infra | 02:16 | |
| *** sarob has joined #openstack-infra | 02:16 | |
| *** r-mibu has quit IRC | 02:17 | |
| *** ramishra has quit IRC | 02:17 | |
| *** mtanin___ has quit IRC | 02:18 | |
| *** ramishra has joined #openstack-infra | 02:18 | |
| *** ramishra has quit IRC | 02:18 | |
| *** ramishra has joined #openstack-infra | 02:19 | |
| *** ramishra has quit IRC | 02:19 | |
| *** ramishra has joined #openstack-infra | 02:19 | |
| *** ramishra has quit IRC | 02:19 | |
| *** bhavik has joined #openstack-infra | 02:19 | |
| *** Daisy_ has joined #openstack-infra | 02:20 | |
| *** sarob has quit IRC | 02:21 | |
| *** ramishra has joined #openstack-infra | 02:21 | |
| *** ramishra has quit IRC | 02:21 | |
| *** Daisy has quit IRC | 02:23 | |
| Qiming | clarkb, double checked system configuration, nothing has been changed, just that ... gerritbot is not in the #senlin channel, so we are missing all gerrit notifications recently | 02:24 |
| *** zhurong has joined #openstack-infra | 02:27 | |
| *** ramishra has joined #openstack-infra | 02:27 | |
| *** ramishra has quit IRC | 02:27 | |
| *** ramishra has joined #openstack-infra | 02:28 | |
| *** ramishra has quit IRC | 02:29 | |
| *** julim has joined #openstack-infra | 02:29 | |
| *** tphummel has quit IRC | 02:31 | |
| *** mtanino has quit IRC | 02:37 | |
| *** r-mibu has joined #openstack-infra | 02:37 | |
| *** amitgandhinz has joined #openstack-infra | 02:38 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Remove vexxhost from nodepool.o.o clouds.yaml https://review.openstack.org/329234 | 02:38 |
| *** gouthamr_ has quit IRC | 02:38 | |
| *** amotoki has quit IRC | 02:40 | |
| *** amitgandhinz has quit IRC | 02:42 | |
| *** zhurong has quit IRC | 02:44 | |
| *** ddieterly has joined #openstack-infra | 02:44 | |
| *** zhurong has joined #openstack-infra | 02:44 | |
| *** timothyb89 has quit IRC | 02:45 | |
| *** ddieterly has quit IRC | 02:46 | |
| *** timothyb89 has joined #openstack-infra | 02:47 | |
| *** woodster_ has quit IRC | 02:48 | |
| *** yuanying has quit IRC | 02:50 | |
| *** yamamoto_ has joined #openstack-infra | 02:50 | |
| *** yuanying has joined #openstack-infra | 02:52 | |
| openstackgerrit | Eli Qiao proposed openstack-infra/project-config: Zun: rename higgins to zun (1) https://review.openstack.org/326306 | 02:52 |
| openstackgerrit | Eli Qiao proposed openstack-infra/project-config: Zun: rename higgins to zun (2) https://review.openstack.org/329247 | 02:52 |
| *** amotoki has joined #openstack-infra | 02:54 | |
| *** amotoki has quit IRC | 02:57 | |
| *** thorst has quit IRC | 03:00 | |
| *** thorst has joined #openstack-infra | 03:00 | |
| *** yamamoto_ has quit IRC | 03:02 | |
| *** yamamoto_ has joined #openstack-infra | 03:05 | |
| *** links has joined #openstack-infra | 03:08 | |
| *** kushal has joined #openstack-infra | 03:09 | |
| *** thorst has quit IRC | 03:09 | |
| *** openstackgerrit has quit IRC | 03:11 | |
| *** openstackgerrit has joined #openstack-infra | 03:11 | |
| *** grassass has quit IRC | 03:12 | |
| pleia2 | Qiming: I restarted the bot to try to get it to return, but it seems like it's still not joining for some reason | 03:15 |
| Qiming | yes, noticed that, pleia2 | 03:15 |
| pleia2 | and my dinner has just arrived, so I'll have to pick this up in the AM | 03:15 |
| pleia2 | (unless someone else has a look before me) | 03:15 |
| Qiming | enjoy your dinner, pleia2, :) | 03:16 |
| *** piet_ has quit IRC | 03:16 | |
| *** dimtruck is now known as zz_dimtruck | 03:18 | |
| *** bhavik has quit IRC | 03:20 | |
| *** zhurong has quit IRC | 03:22 | |
| *** csomerville has quit IRC | 03:24 | |
| *** PollyZ has joined #openstack-infra | 03:24 | |
| openstackgerrit | Merged openstack-infra/irc-meetings: Added meeting time for the JavaScript SDK & Tooling group. https://review.openstack.org/329120 | 03:26 |
| *** binbincong has joined #openstack-infra | 03:26 | |
| *** afred312 has quit IRC | 03:26 | |
| openstackgerrit | Merged openstack-infra/irc-meetings: kuryr: Make wiki and ics match https://review.openstack.org/329023 | 03:27 |
| *** yamahata has joined #openstack-infra | 03:29 | |
| openstackgerrit | Merged openstack-infra/irc-meetings: Change meeting time to 1500 UTC https://review.openstack.org/329057 | 03:29 |
| *** yamamoto_ has quit IRC | 03:30 | |
| *** yuanying has quit IRC | 03:33 | |
| *** yuanying has joined #openstack-infra | 03:34 | |
| *** garyk has joined #openstack-infra | 03:34 | |
| *** yamamoto_ has joined #openstack-infra | 03:36 | |
| *** amitgandhinz has joined #openstack-infra | 03:39 | |
| *** asselin_ has quit IRC | 03:40 | |
| *** darvon has quit IRC | 03:40 | |
| *** darvon has joined #openstack-infra | 03:40 | |
| *** amitgandhinz has quit IRC | 03:43 | |
| *** sputnik13 has quit IRC | 03:44 | |
| *** ociuhandu has quit IRC | 03:45 | |
| *** PollyZ has quit IRC | 03:47 | |
| *** ociuhandu has joined #openstack-infra | 03:49 | |
| openstackgerrit | Thanh Ha proposed openstack-infra/jenkins-job-builder: Allow JJB to spoof plugin_info via yaml https://review.openstack.org/326722 | 03:50 |
| *** cody-somerville has joined #openstack-infra | 03:53 | |
| *** ayoung has quit IRC | 03:56 | |
| *** amotoki has joined #openstack-infra | 03:58 | |
| *** hongbin has quit IRC | 04:00 | |
| *** PollyZ has joined #openstack-infra | 04:01 | |
| *** yamada-h has joined #openstack-infra | 04:03 | |
| *** yuanying has quit IRC | 04:05 | |
| *** ociuhandu has quit IRC | 04:05 | |
| *** alekhya has joined #openstack-infra | 04:06 | |
| tonyb | Can someone with access to the logs work out why this https://review.openstack.org/#/c/329097/ isn't progressing to the gate pipeline? | 04:07 |
| *** thorst has joined #openstack-infra | 04:07 | |
| *** yuanying has joined #openstack-infra | 04:07 | |
| openstackgerrit | Eli Qiao proposed openstack-infra/project-config: Zun: rename higgins to zun (1) https://review.openstack.org/326306 | 04:11 |
| openstackgerrit | Eli Qiao proposed openstack-infra/project-config: Zun: rename higgins to zun (2) https://review.openstack.org/329247 | 04:11 |
| *** thorst has quit IRC | 04:14 | |
| *** xarses_ has joined #openstack-infra | 04:14 | |
| *** markvoelker has quit IRC | 04:16 | |
| *** andreww has joined #openstack-infra | 04:17 | |
| *** xarses has quit IRC | 04:17 | |
| *** andreww has quit IRC | 04:18 | |
| *** PollyZ has quit IRC | 04:18 | |
| *** andreww has joined #openstack-infra | 04:18 | |
| *** xarses_ has quit IRC | 04:20 | |
| *** cody-somerville_ has joined #openstack-infra | 04:23 | |
| openstackgerrit | Sachi King proposed openstack-dev/pbr: Restore warnerrors behavior https://review.openstack.org/229951 | 04:24 |
| *** alekhya has quit IRC | 04:24 | |
| *** grassass has joined #openstack-infra | 04:24 | |
| *** Daisy_ has quit IRC | 04:26 | |
| *** sree_ has joined #openstack-infra | 04:27 | |
| *** sree_ is now known as Guest94631 | 04:27 | |
| *** afred312 has joined #openstack-infra | 04:27 | |
| *** psachin has joined #openstack-infra | 04:32 | |
| *** hdaniel has joined #openstack-infra | 04:32 | |
| *** afred312 has quit IRC | 04:33 | |
| *** cody-somerville has quit IRC | 04:37 | |
| *** zhurong has joined #openstack-infra | 04:38 | |
| *** pcaruana has quit IRC | 04:39 | |
| *** amitgandhinz has joined #openstack-infra | 04:39 | |
| *** tkelsey has joined #openstack-infra | 04:42 | |
| *** amitgandhinz has quit IRC | 04:44 | |
| *** tkelsey has quit IRC | 04:47 | |
| *** tlian has joined #openstack-infra | 04:48 | |
| *** nadya has joined #openstack-infra | 04:51 | |
| *** abregman has joined #openstack-infra | 04:52 | |
| *** yamada-h has quit IRC | 04:53 | |
| *** yamada-h has joined #openstack-infra | 04:54 | |
| *** zhenguo_ has joined #openstack-infra | 04:54 | |
| *** mrmartin has joined #openstack-infra | 04:58 | |
| *** yamada-h has quit IRC | 04:59 | |
| *** yamada-h has joined #openstack-infra | 05:05 | |
| *** pgadiya has joined #openstack-infra | 05:06 | |
| clarkb | tonyb dont need logs it is because the verified +1 vote isnt changing so the gerrit event doesnt include that data due to no delta | 05:07 |
| clarkb | you can just reapprove it to get it in the gate | 05:07 |
| *** Daisy has joined #openstack-infra | 05:07 | |
| *** tphummel has joined #openstack-infra | 05:09 | |
| *** Daisy_ has joined #openstack-infra | 05:11 | |
| *** Daisy has quit IRC | 05:11 | |
| *** oanson has quit IRC | 05:11 | |
| *** jaosorior has joined #openstack-infra | 05:11 | |
| *** cody-somerville has joined #openstack-infra | 05:12 | |
| *** cody-somerville has joined #openstack-infra | 05:12 | |
| *** thorst has joined #openstack-infra | 05:12 | |
| *** cody-somerville has quit IRC | 05:16 | |
| *** cody-somerville_ has quit IRC | 05:16 | |
| *** markvoelker has joined #openstack-infra | 05:17 | |
| *** salv-orl_ has quit IRC | 05:17 | |
| *** salv-orlando has joined #openstack-infra | 05:18 | |
| *** thorst has quit IRC | 05:19 | |
| tonyb | clarkb: Thanks. Why didn't it automatically progress when the first Verified came in? (which looks to me after it had +W) | 05:20 |
| clarkb | tonyb: I think jeblair restarted zuul at some point | 05:21 |
| tonyb | clarkb: Ah okay | 05:21 |
| *** markvoelker has quit IRC | 05:22 | |
| tonyb | clarkb: Looks like it was betwn the +W and the +V on that review. | 05:23 |
| * tonyb will add that to the list of things to check | 05:23 | |
| *** mrmartin has quit IRC | 05:24 | |
| *** yamamoto_ has quit IRC | 05:28 | |
| *** salv-orl_ has joined #openstack-infra | 05:29 | |
| *** nadya has quit IRC | 05:29 | |
| *** salv-orlando has quit IRC | 05:31 | |
| *** salv-orl_ has quit IRC | 05:34 | |
| *** salv-orlando has joined #openstack-infra | 05:34 | |
| *** amitgandhinz has joined #openstack-infra | 05:40 | |
| *** nadya has joined #openstack-infra | 05:41 | |
| *** sarob has joined #openstack-infra | 05:41 | |
| *** amitgandhinz has quit IRC | 05:44 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: DONT MERGE TESTING https://review.openstack.org/316436 | 05:45 |
| *** sarob has quit IRC | 05:46 | |
| *** Guest94631 has quit IRC | 05:46 | |
| *** sree has joined #openstack-infra | 05:47 | |
| *** savihou has joined #openstack-infra | 05:49 | |
| *** sree has quit IRC | 05:49 | |
| *** sree has joined #openstack-infra | 05:50 | |
| openstackgerrit | YAMADA Hideki proposed openstack-infra/project-config: Add Sheepdog test job to glance_store https://review.openstack.org/328148 | 05:52 |
| *** nadya has quit IRC | 05:55 | |
| *** javeriak has joined #openstack-infra | 05:59 | |
| *** Sukhdev has joined #openstack-infra | 06:00 | |
| *** roxanaghe has joined #openstack-infra | 06:00 | |
| *** roxanaghe has quit IRC | 06:00 | |
| *** tlian has quit IRC | 06:00 | |
| *** savihou has quit IRC | 06:06 | |
| *** Daisy_ has quit IRC | 06:11 | |
| *** rcernin has joined #openstack-infra | 06:11 | |
| *** Daisy has joined #openstack-infra | 06:12 | |
| *** pfallenop has quit IRC | 06:12 | |
| *** salv-orlando has quit IRC | 06:13 | |
| *** salv-orlando has joined #openstack-infra | 06:14 | |
| *** tphummel has quit IRC | 06:14 | |
| *** pfallenop has joined #openstack-infra | 06:15 | |
| *** Daisy has quit IRC | 06:16 | |
| *** pcaruana has joined #openstack-infra | 06:16 | |
| *** thorst has joined #openstack-infra | 06:16 | |
| *** jbernard has quit IRC | 06:17 | |
| *** yamamoto_ has joined #openstack-infra | 06:17 | |
| *** jbernard has joined #openstack-infra | 06:19 | |
| openstackgerrit | Merged openstack-infra/zuul: Python 3 Fix: Divide operator returns float https://review.openstack.org/327303 | 06:19 |
| openstackgerrit | Merged openstack-infra/project-config: Add networking-ovn in the 'PROJECTS' for dsvm-functional job https://review.openstack.org/329241 | 06:20 |
| openstackgerrit | lvdongbing proposed openstack-infra/irc-meetings: Add the Bilean team meeting https://review.openstack.org/326271 | 06:21 |
| yolanda | good morning | 06:23 |
| *** yamahata has quit IRC | 06:23 | |
| openstackgerrit | Merged openstack-infra/zuul: Fix timeout debug print in tests https://review.openstack.org/327768 | 06:24 |
| *** thorst has quit IRC | 06:24 | |
| *** yamahata has joined #openstack-infra | 06:26 | |
| *** tphummel has joined #openstack-infra | 06:30 | |
| *** Daisy has joined #openstack-infra | 06:33 | |
| *** Sukhdev has quit IRC | 06:39 | |
| *** amitgandhinz has joined #openstack-infra | 06:41 | |
| *** ihrachys has joined #openstack-infra | 06:41 | |
| *** mrmartin has joined #openstack-infra | 06:42 | |
| *** andreas_s has joined #openstack-infra | 06:42 | |
| *** tkelsey has joined #openstack-infra | 06:44 | |
| openstackgerrit | Merged openstack-infra/project-config: Create new project karma-subunit-reporter https://review.openstack.org/327945 | 06:45 |
| *** amitgandhinz has quit IRC | 06:45 | |
| *** nmagnezi has joined #openstack-infra | 06:46 | |
| *** openstackgerrit has quit IRC | 06:48 | |
| *** openstackgerrit has joined #openstack-infra | 06:48 | |
| openstackgerrit | Merged openstack-infra/project-config: Add non-voting coverage check job for neutron-lib https://review.openstack.org/328445 | 06:48 |
| *** tkelsey has quit IRC | 06:49 | |
| AJaeger | morning, yolanda ! | 06:49 |
| *** aviau has quit IRC | 06:49 | |
| *** aviau has joined #openstack-infra | 06:50 | |
| *** ociuhandu has joined #openstack-infra | 06:50 | |
| openstackgerrit | Merged openstack-infra/project-config: Add trove period jobs with oslo master https://review.openstack.org/329086 | 06:55 |
| openstackgerrit | Merged openstack-infra/project-config: Add python version to wheel builders https://review.openstack.org/329036 | 06:55 |
| *** tesseract has joined #openstack-infra | 06:56 | |
| *** samP has joined #openstack-infra | 06:56 | |
| *** ccamacho has quit IRC | 06:56 | |
| openstackgerrit | Merged openstack-infra/project-config: Add gates for deploying kolla-kubernetes https://review.openstack.org/328814 | 06:57 |
| *** amoralej|off is now known as amoralej | 06:58 | |
| openstackgerrit | Merged openstack-infra/project-config: Add translation jobs to watcher https://review.openstack.org/329012 | 06:58 |
| openstackgerrit | Merged openstack-infra/project-config: Add release notes jobs for watcher https://review.openstack.org/329007 | 06:59 |
| *** mrmartin has quit IRC | 06:59 | |
| *** rossella_s has joined #openstack-infra | 07:00 | |
| openstackgerrit | Merged openstack-infra/project-config: [Manila] Add new experimental job https://review.openstack.org/328364 | 07:00 |
| openstackgerrit | Merged openstack-infra/project-config: Disable convergence for jobs with legacy engine https://review.openstack.org/327560 | 07:01 |
| *** grassass has quit IRC | 07:02 | |
| *** ccamacho has joined #openstack-infra | 07:06 | |
| *** zeih has joined #openstack-infra | 07:11 | |
| *** mrmartin has joined #openstack-infra | 07:12 | |
| *** xarses_ has joined #openstack-infra | 07:14 | |
| *** slaweq has joined #openstack-infra | 07:16 | |
| *** andreww has quit IRC | 07:17 | |
| *** markvoelker has joined #openstack-infra | 07:18 | |
| AJaeger | flaper87: could you review 328148, please? | 07:19 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Add new reference to puppet-certmonger to mirror list https://review.openstack.org/329025 | 07:19 |
| *** mikelk has joined #openstack-infra | 07:22 | |
| *** thorst has joined #openstack-infra | 07:22 | |
| *** markvoelker has quit IRC | 07:22 | |
| *** vincentll has joined #openstack-infra | 07:23 | |
| openstackgerrit | Merged openstack-infra/project-config: Add tempest-dsvm-ironic-pxe_ipa-full-nv to Ironic https://review.openstack.org/328245 | 07:26 |
| *** jlanoux has joined #openstack-infra | 07:26 | |
| *** arxcruz has joined #openstack-infra | 07:28 | |
| *** thorst has quit IRC | 07:29 | |
| *** abregman_ has joined #openstack-infra | 07:29 | |
| *** dizquierdo has joined #openstack-infra | 07:30 | |
| *** grassass has joined #openstack-infra | 07:30 | |
| *** dmk0202 has joined #openstack-infra | 07:32 | |
| *** ifarkas has joined #openstack-infra | 07:32 | |
| *** abregman has quit IRC | 07:33 | |
| *** sree has quit IRC | 07:33 | |
| *** sree has joined #openstack-infra | 07:34 | |
| *** jaosorior is now known as jaosorior_brb | 07:35 | |
| *** jlanoux has quit IRC | 07:37 | |
| *** tkelsey has joined #openstack-infra | 07:37 | |
| *** jlanoux has joined #openstack-infra | 07:38 | |
| *** sree has quit IRC | 07:38 | |
| *** jlanoux has quit IRC | 07:39 | |
| *** yamahata has quit IRC | 07:41 | |
| *** amitgandhinz has joined #openstack-infra | 07:41 | |
| *** nighttime has joined #openstack-infra | 07:41 | |
| *** sarob has joined #openstack-infra | 07:42 | |
| *** salv-orlando has quit IRC | 07:44 | |
| *** salv-orlando has joined #openstack-infra | 07:44 | |
| *** amitgandhinz has quit IRC | 07:46 | |
| *** sarob has quit IRC | 07:47 | |
| *** zeih has quit IRC | 07:47 | |
| *** Na3iL has joined #openstack-infra | 07:47 | |
| *** sree has joined #openstack-infra | 07:47 | |
| *** grazzazz has joined #openstack-infra | 07:49 | |
| *** grassass has quit IRC | 07:49 | |
| *** nadya has joined #openstack-infra | 07:50 | |
| *** nadya is now known as _nadya_ | 07:50 | |
| *** grazzazz has quit IRC | 07:50 | |
| openstackgerrit | Yuval Brik proposed openstack-infra/project-config: Use Smaug configuration from Smaug's repository https://review.openstack.org/329305 | 07:50 |
| *** grassass has joined #openstack-infra | 07:51 | |
| *** Na3iL has quit IRC | 07:54 | |
| *** samP has quit IRC | 07:56 | |
| *** sorantis has joined #openstack-infra | 07:56 | |
| *** abregman_ has quit IRC | 07:59 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** zzzeek has joined #openstack-infra | 08:00 | |
| *** Na3iL has joined #openstack-infra | 08:00 | |
| *** kushal has quit IRC | 08:01 | |
| *** jlanoux has joined #openstack-infra | 08:02 | |
| *** dtantsur|afk is now known as dtantsur | 08:04 | |
| *** eezhova has joined #openstack-infra | 08:04 | |
| *** shardy has joined #openstack-infra | 08:04 | |
| *** zeih has joined #openstack-infra | 08:05 | |
| *** e0ne has joined #openstack-infra | 08:05 | |
| *** zeih has quit IRC | 08:06 | |
| openstackgerrit | Di XiaoLi proposed openstack-infra/project-config: Add a heat job to use devstack plugin https://review.openstack.org/317811 | 08:06 |
| *** faizy has joined #openstack-infra | 08:08 | |
| *** zeih has joined #openstack-infra | 08:09 | |
| *** abregman_ has joined #openstack-infra | 08:11 | |
| openstackgerrit | Elena Ezhova proposed openstack-infra/project-config: Pass extra parameter to hooks in neutron-lbaas tempest jobs https://review.openstack.org/322811 | 08:17 |
| openstackgerrit | Di XiaoLi proposed openstack-infra/project-config: [WIP]Switch all heat jobs to use devstack plugin https://review.openstack.org/317817 | 08:18 |
| openstackgerrit | Elena Ezhova proposed openstack-infra/project-config: Add a functional job for neutron-lbaas https://review.openstack.org/322812 | 08:18 |
| *** Daisy has quit IRC | 08:19 | |
| *** Daisy has joined #openstack-infra | 08:19 | |
| *** mhickey has joined #openstack-infra | 08:20 | |
| *** faizy_ has joined #openstack-infra | 08:21 | |
| *** faizy has quit IRC | 08:22 | |
| *** sorantis_ has joined #openstack-infra | 08:22 | |
| *** sorantis has quit IRC | 08:25 | |
| *** sorantis_ is now known as sorantis | 08:25 | |
| *** pgadiya has quit IRC | 08:25 | |
| *** pgadiya has joined #openstack-infra | 08:26 | |
| *** thorst has joined #openstack-infra | 08:26 | |
| *** javeriak has quit IRC | 08:27 | |
| *** afred312 has joined #openstack-infra | 08:29 | |
| *** chem`` has joined #openstack-infra | 08:31 | |
| *** chem` has quit IRC | 08:32 | |
| *** javeriak has joined #openstack-infra | 08:33 | |
| *** afred312 has quit IRC | 08:34 | |
| *** thorst has quit IRC | 08:34 | |
| openstackgerrit | Di XiaoLi proposed openstack-infra/project-config: Switch all heat jobs to use devstack plugin https://review.openstack.org/317817 | 08:35 |
| *** sshnaidm has quit IRC | 08:39 | |
| *** jaosorior_brb has quit IRC | 08:39 | |
| *** tphummel has quit IRC | 08:39 | |
| *** jaosorior_brb has joined #openstack-infra | 08:40 | |
| *** derekh has joined #openstack-infra | 08:40 | |
| *** dingyichen has quit IRC | 08:41 | |
| *** amitgandhinz has joined #openstack-infra | 08:42 | |
| *** Qiming is now known as Qiming_afk | 08:43 | |
| *** kzaitsev_mb has joined #openstack-infra | 08:43 | |
| *** Julien-zte has quit IRC | 08:44 | |
| *** amitgandhinz has quit IRC | 08:47 | |
| *** kzaitsev_mb has quit IRC | 08:50 | |
| *** matrohon has joined #openstack-infra | 08:50 | |
| *** jaosorior_brb is now known as jaosorior | 08:53 | |
| *** dtardivel has joined #openstack-infra | 08:54 | |
| *** markusry has joined #openstack-infra | 08:54 | |
| *** javeriak has quit IRC | 08:56 | |
| *** acoles_ is now known as acoles | 08:58 | |
| *** hashar has joined #openstack-infra | 09:02 | |
| *** Daisy_ has joined #openstack-infra | 09:02 | |
| *** vgridnev has quit IRC | 09:03 | |
| *** Qiming_afk is now known as Qiming | 09:06 | |
| *** Daisy has quit IRC | 09:06 | |
| *** binbincong has quit IRC | 09:07 | |
| *** Daisy_ has quit IRC | 09:10 | |
| *** Daisy has joined #openstack-infra | 09:11 | |
| *** sambetts|afk is now known as sambetts | 09:14 | |
| *** jordanP has joined #openstack-infra | 09:14 | |
| *** zeih has quit IRC | 09:15 | |
| *** ams__ has joined #openstack-infra | 09:15 | |
| *** Daisy has quit IRC | 09:15 | |
| *** amotoki has quit IRC | 09:15 | |
| openstackgerrit | Chuck Short proposed openstack-infra/project-config: Add check-requirements check job for nova-lxd https://review.openstack.org/329331 | 09:18 |
| *** Na3iL has quit IRC | 09:19 | |
| openstackgerrit | Merged openstack-infra/irc-meetings: Add the Bilean team meeting https://review.openstack.org/326271 | 09:22 |
| *** electrofelix has joined #openstack-infra | 09:26 | |
| *** zeih has joined #openstack-infra | 09:27 | |
| *** binbincong has joined #openstack-infra | 09:30 | |
| openstackgerrit | Sam Betts proposed openstack-infra/elastic-recheck: Add query for nova resource tracker/scheduler race https://review.openstack.org/316364 | 09:30 |
| *** javeriak has joined #openstack-infra | 09:32 | |
| *** thorst has joined #openstack-infra | 09:32 | |
| *** Na3iL has joined #openstack-infra | 09:34 | |
| *** snarwade has quit IRC | 09:35 | |
| *** yamada-h has quit IRC | 09:35 | |
| *** pgadiya has quit IRC | 09:36 | |
| *** thorst has quit IRC | 09:39 | |
| *** _degorenko|afk is now known as degorenko | 09:39 | |
| *** eezhova has quit IRC | 09:40 | |
| wznoinsk | hi all | 09:41 |
| *** sshnaidm has joined #openstack-infra | 09:41 | |
| *** amitgandhinz has joined #openstack-infra | 09:43 | |
| *** sarob has joined #openstack-infra | 09:43 | |
| *** pgadiya has joined #openstack-infra | 09:43 | |
| openstackgerrit | Julien Danjou proposed openstack-infra/git-review: Set a default EXIT_CODE for GitReviewException https://review.openstack.org/323959 | 09:46 |
| openstackgerrit | Ales Komarek proposed openstack-infra/irc-meetings: Change opesntack-salt meeting chair https://review.openstack.org/329344 | 09:47 |
| *** amitgandhinz has quit IRC | 09:47 | |
| *** sarob has quit IRC | 09:48 | |
| openstackgerrit | Ales Komarek proposed openstack-infra/irc-meetings: Change openstack-salt meeting chair https://review.openstack.org/329344 | 09:48 |
| *** zeih has quit IRC | 09:48 | |
| *** eezhova has joined #openstack-infra | 09:50 | |
| *** ociuhandu has quit IRC | 09:51 | |
| *** snarwade has joined #openstack-infra | 09:51 | |
| frickler | I'm seeing random failures where it looks like the task has been run by zuul instead of jenkins, like e.g. http://logs.openstack.org/61/327761/2/check/gate-nova-docs/6899468/ although the exit code is 0 (= success I assume), this is logged as failure | 09:53 |
| *** javeriak has quit IRC | 09:56 | |
| openstackgerrit | Lukas Bednar proposed openstack-infra/git-review: Fix AttributeError when can not connect to Gerrit https://review.openstack.org/318181 | 09:56 |
| AJaeger | frickler: that looks to me like it should have been a success. Let's tell jeblair and pabelanger about your log file and have them look at it. | 09:58 |
| AJaeger | frickler: Yes, we're replacing jenkins by zuul-launcher and hit a few problems - and jeblair and pabelanger look into these. Thanks for telling us! | 09:58 |
| *** javeriak has joined #openstack-infra | 09:59 | |
| *** claudiub has joined #openstack-infra | 10:02 | |
| *** kzaitsev_mb has joined #openstack-infra | 10:02 | |
| *** lvdongbing has quit IRC | 10:03 | |
| *** amotoki has joined #openstack-infra | 10:04 | |
| *** tosky has joined #openstack-infra | 10:04 | |
| *** sree has quit IRC | 10:05 | |
| *** links has quit IRC | 10:07 | |
| *** dmk0202 has quit IRC | 10:08 | |
| openstackgerrit | Madhuri Kumari proposed openstack-infra/project-config: Add new project Zunclient to openstack repo https://review.openstack.org/317699 | 10:09 |
| *** kushal has joined #openstack-infra | 10:11 | |
| *** oanson has joined #openstack-infra | 10:13 | |
| *** ociuhandu has joined #openstack-infra | 10:13 | |
| openstackgerrit | Mateusz Matuszkowiak proposed openstack-infra/project-config: Added new repo for fuel-plugin-datera-cinder https://review.openstack.org/315651 | 10:13 |
| *** javeriak has quit IRC | 10:14 | |
| *** snarwade has quit IRC | 10:20 | |
| *** links has joined #openstack-infra | 10:20 | |
| *** javeriak has joined #openstack-infra | 10:22 | |
| *** zeih has joined #openstack-infra | 10:23 | |
| *** yanyanhu has quit IRC | 10:24 | |
| *** zhurong has quit IRC | 10:24 | |
| *** jlanoux_ has joined #openstack-infra | 10:25 | |
| *** pgadiya has quit IRC | 10:26 | |
| *** pgadiya has joined #openstack-infra | 10:26 | |
| *** zeih has quit IRC | 10:27 | |
| *** jlanoux has quit IRC | 10:28 | |
| *** zeih has joined #openstack-infra | 10:28 | |
| *** afred312 has joined #openstack-infra | 10:31 | |
| *** sdague has joined #openstack-infra | 10:31 | |
| *** dizquierdo is now known as dizquierdo_afk | 10:32 | |
| *** amotoki_ has joined #openstack-infra | 10:34 | |
| *** afred312 has quit IRC | 10:35 | |
| *** thorst has joined #openstack-infra | 10:35 | |
| *** yamada-h has joined #openstack-infra | 10:36 | |
| *** snarwade has joined #openstack-infra | 10:36 | |
| *** amotoki_ has quit IRC | 10:37 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Fix tempest configuration https://review.openstack.org/326055 | 10:38 |
| *** amotoki_ has joined #openstack-infra | 10:38 | |
| *** eezhova has quit IRC | 10:38 | |
| *** salv-orlando has quit IRC | 10:38 | |
| *** salv-orlando has joined #openstack-infra | 10:39 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: DONT MERGE TESTING https://review.openstack.org/316436 | 10:39 |
| *** Srinivas__ has joined #openstack-infra | 10:39 | |
| Srinivas__ | Hai. Fc passthrough will work only on ubuntu 14.04.1 only or it can be used in any OS. Please help me. i am using FC passthrough on ubuntu 14.04.04 but it is failing while running this command check this link http://paste.openstack.org/show/515937/ | 10:39 |
| *** yamada-h has quit IRC | 10:41 | |
| *** amotoki has quit IRC | 10:42 | |
| *** amitgandhinz has joined #openstack-infra | 10:44 | |
| *** thorst has quit IRC | 10:44 | |
| *** dtantsur is now known as dtantsur|brb | 10:45 | |
| *** jlanoux has joined #openstack-infra | 10:46 | |
| *** jlanoux_ has quit IRC | 10:48 | |
| *** amitgandhinz has quit IRC | 10:48 | |
| *** amotoki_ has quit IRC | 10:52 | |
| *** amotoki has joined #openstack-infra | 10:52 | |
| *** esikachev has joined #openstack-infra | 10:54 | |
| Srinivas__ | <esikachev>Hai. Fc passthrough will work only on ubuntu 14.04.1 only or it can be used in any OS. Please help me. i am using FC passthrough on ubuntu 14.04.04 but it is failing while running this command check this link http://paste.openstack.org/show/515937/ | 10:54 |
| *** salv-orlando has quit IRC | 10:58 | |
| *** salv-orlando has joined #openstack-infra | 10:58 | |
| *** zeih_ has joined #openstack-infra | 11:02 | |
| *** zeih has quit IRC | 11:02 | |
| openstackgerrit | Hisashi Osanai proposed openstack-infra/project-config: Add jobs for Monasca-Analytics https://review.openstack.org/329376 | 11:07 |
| *** permalac has quit IRC | 11:09 | |
| AJaeger | Srinivas__: did you see http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-06-13.log.html#t2016-06-13T16:17:10 ? | 11:09 |
| *** oanson has quit IRC | 11:10 | |
| AJaeger | Srinivas__: and http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-06-13.log.html#t2016-06-13T16:41:22 ? | 11:10 |
| *** Srinivas__ has left #openstack-infra | 11:14 | |
| *** ccamacho is now known as ccamacho|lunch | 11:19 | |
| *** eezhova has joined #openstack-infra | 11:20 | |
| *** ociuhandu has quit IRC | 11:23 | |
| *** ociuhandu has joined #openstack-infra | 11:23 | |
| *** ociuhandu has quit IRC | 11:26 | |
| *** ociuhandu has joined #openstack-infra | 11:26 | |
| *** ldnunes has joined #openstack-infra | 11:28 | |
| *** rhallisey has joined #openstack-infra | 11:28 | |
| *** salv-orl_ has joined #openstack-infra | 11:28 | |
| *** sree has joined #openstack-infra | 11:29 | |
| *** gordc has joined #openstack-infra | 11:30 | |
| *** weshay has joined #openstack-infra | 11:30 | |
| *** salv-orlando has quit IRC | 11:32 | |
| *** zeih_ has quit IRC | 11:32 | |
| *** sree has quit IRC | 11:34 | |
| *** rodrigods has quit IRC | 11:35 | |
| *** rodrigods has joined #openstack-infra | 11:35 | |
| *** kushal has quit IRC | 11:36 | |
| *** kushal has joined #openstack-infra | 11:37 | |
| *** dizquierdo_afk is now known as dizquierdo | 11:37 | |
| *** thorst has joined #openstack-infra | 11:41 | |
| *** thorst_ has joined #openstack-infra | 11:43 | |
| *** sarob has joined #openstack-infra | 11:44 | |
| *** piet has joined #openstack-infra | 11:44 | |
| *** exploreshaifali has joined #openstack-infra | 11:44 | |
| *** amitgandhinz has joined #openstack-infra | 11:44 | |
| *** jed56 has quit IRC | 11:45 | |
| *** thorst has quit IRC | 11:47 | |
| *** amotoki has quit IRC | 11:48 | |
| *** sarob has quit IRC | 11:48 | |
| *** thorst_ is now known as thorst | 11:49 | |
| *** ddieterly has joined #openstack-infra | 11:49 | |
| *** amitgandhinz has quit IRC | 11:49 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/system-config: Rename expand-groups to ansible-expand-groups https://review.openstack.org/326465 | 11:50 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/system-config: Tidy up ansible directories https://review.openstack.org/326464 | 11:50 |
| *** baoli has joined #openstack-infra | 11:51 | |
| *** baoli_ has joined #openstack-infra | 11:52 | |
| *** raildo-afk is now known as raildo | 11:54 | |
| *** baoli has quit IRC | 11:56 | |
| *** thiagop has joined #openstack-infra | 11:57 | |
| *** aysyd has joined #openstack-infra | 11:57 | |
| *** tpsilva has joined #openstack-infra | 11:57 | |
| *** dtantsur|brb is now known as dtantsur | 11:58 | |
| *** piet has quit IRC | 11:58 | |
| *** exploreshaifali has quit IRC | 11:58 | |
| mordred | jhesketh: do we care about deleting the old expand-groups script? | 12:00 |
| mordred | jhesketh: (I don't care a ton) | 12:00 |
| jhesketh | mordred: I was just rerolling some old patches... | 12:01 |
| jhesketh | mordred: is the plan to remove it? | 12:01 |
| *** jed56 has joined #openstack-infra | 12:02 | |
| *** sree has joined #openstack-infra | 12:04 | |
| *** zeih has joined #openstack-infra | 12:06 | |
| *** esikachev has quit IRC | 12:06 | |
| *** yamada-h has joined #openstack-infra | 12:06 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Start fleshing out ansible-launcher tests https://review.openstack.org/317482 | 12:07 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Clean up results returned from ansible-launcher https://review.openstack.org/317481 | 12:07 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Add jjb and zmq to requirements https://review.openstack.org/316752 | 12:07 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul: Prepare files for testing zuulv25 https://review.openstack.org/316753 | 12:07 |
| *** faizy_ has quit IRC | 12:07 | |
| *** markusry has quit IRC | 12:09 | |
| mordred | jhesketh: no - I just mean, the patch renames it to ansible-expand-groups - but nothing cleans up after it | 12:09 |
| jhesketh | oh right, I see what you mean... yeah I'll tidy that up | 12:10 |
| *** yamada-h has quit IRC | 12:10 | |
| *** esikachev has joined #openstack-infra | 12:11 | |
| *** yamamoto_ has quit IRC | 12:11 | |
| *** daemontool has joined #openstack-infra | 12:11 | |
| *** pradk has joined #openstack-infra | 12:12 | |
| *** markvoelker has joined #openstack-infra | 12:13 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/system-config: Rename expand-groups to ansible-expand-groups https://review.openstack.org/326465 | 12:14 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/system-config: Remove cleanup directive https://review.openstack.org/329410 | 12:14 |
| jhesketh | mordred: ^ | 12:14 |
| *** kgiusti has joined #openstack-infra | 12:15 | |
| *** esikachev has quit IRC | 12:15 | |
| *** matrohon has quit IRC | 12:15 | |
| *** ddieterly has quit IRC | 12:16 | |
| *** psilvad has joined #openstack-infra | 12:16 | |
| *** esikachev has joined #openstack-infra | 12:16 | |
| mordred | jhesketh: +2 | 12:19 |
| jhesketh | ta :-) | 12:20 |
| *** psilvad has quit IRC | 12:20 | |
| *** zeih has quit IRC | 12:21 | |
| *** esikachev has quit IRC | 12:21 | |
| *** dizquierdo has quit IRC | 12:21 | |
| *** eezhova has quit IRC | 12:23 | |
| *** ccamacho|lunch is now known as ccamacho | 12:23 | |
| *** matrohon has joined #openstack-infra | 12:25 | |
| *** Srinu has joined #openstack-infra | 12:26 | |
| Srinu | Hai. i am using this https://github.com/openstack/third-party-ci-tools/blob/master/provisioning_scripts/fibre_channel/invoke_fc_passthrough.sh | 12:26 |
| *** psilvad has joined #openstack-infra | 12:26 | |
| *** kushal has quit IRC | 12:26 | |
| Srinu | when running the virsh attach-device command in the script it is failing with the error http://paste.openstack.org/show/515937/ | 12:27 |
| Srinu | we are using Ubuntu 14.04.4 with QLOgic FC | 12:27 |
| Srinu | we are using devstack master | 12:28 |
| *** eezhova has joined #openstack-infra | 12:28 | |
| Srinu | Did anyone face the issue like this | 12:28 |
| *** kaisers1 has quit IRC | 12:29 | |
| Srinu | we are setting up CI for our FC driver and we need this script to be run to enable passthrough | 12:29 |
| pabelanger | frickler: AJaeger: I suspect that is a know issue we fixed yesterday. A recheck would be in order | 12:31 |
| *** tlian has joined #openstack-infra | 12:31 | |
| *** psilvad has quit IRC | 12:31 | |
| *** afred312 has joined #openstack-infra | 12:32 | |
| mordred | pabelanger: how's the 2.5 going? | 12:33 |
| *** salv-orl_ has quit IRC | 12:33 | |
| *** salv-orlando has joined #openstack-infra | 12:34 | |
| *** rlandy has joined #openstack-infra | 12:34 | |
| pabelanger | mordred: Excellent I believe. jeblair has 5 zuul-launchers online (currently using 4 of them). Finding some issues, but nothing major at this point | 12:34 |
| mordred | pabelanger: \o/ | 12:34 |
| *** psilvad has joined #openstack-infra | 12:35 | |
| pabelanger | mordred: jeblair: for example, http://paste.openstack.org/show/515966/ we need to see about disabling error handling in rsync. We are currently failing to synchronize logs if we cannot ready them, | 12:35 |
| *** esikachev has joined #openstack-infra | 12:35 | |
| pabelanger | POST_FAILURE is a new error we added, to replace UNSTABLE :) | 12:35 |
| *** zeih has joined #openstack-infra | 12:36 | |
| *** afred312 has quit IRC | 12:37 | |
| *** ddieterly has joined #openstack-infra | 12:37 | |
| *** trown|outtypewww is now known as trown | 12:37 | |
| mordred | ooh. I like that as a better error | 12:38 |
| *** nikeshm has joined #openstack-infra | 12:38 | |
| nikeshm | hi | 12:38 |
| nikeshm | we are setting up CI for our kaminario FC driver | 12:39 |
| *** psilvad has quit IRC | 12:39 | |
| nikeshm | we are following the steps required for FC passthrough | 12:40 |
| AJaeger | nikeshm, Srinu did you see http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-06-13.log.html#t2016-06-13T16:41:22 ? | 12:41 |
| *** sarob has joined #openstack-infra | 12:42 | |
| AJaeger | and also http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-06-13.log.html#t2016-06-13T16:17:10 ? | 12:42 |
| AJaeger | pabelanger: thanks. I see a recheck was done earlier today and worked. | 12:45 |
| *** amitgandhinz has joined #openstack-infra | 12:45 | |
| *** javeriak has quit IRC | 12:46 | |
| *** ddieterly is now known as ddieterly[away] | 12:46 | |
| *** exploreshaifali has joined #openstack-infra | 12:46 | |
| *** sarob has quit IRC | 12:47 | |
| *** psilvad has joined #openstack-infra | 12:47 | |
| *** pgadiya has quit IRC | 12:48 | |
| *** amoralej is now known as amoralej|lunch | 12:48 | |
| nikeshm | AJaeger: thanks, since infra channel helps us in configuring and understanding openstack CI for third party vendors, so people might be have faced such issue so they could help better | 12:48 |
| yolanda | hi pabelanger , mordred, care reviewing https://review.openstack.org/326605 and https://review.openstack.org/327707 ? | 12:48 |
| *** piet has joined #openstack-infra | 12:49 | |
| pabelanger | AJaeger: We also are bootstrapping our logs correctly now: http://logs.openstack.org/91/328391/7/check/gate-manila-tempest-dsvm-postgres-generic-singlebackend/4985dfa/console.html See first 2 lines | 12:49 |
| *** mrmartin has quit IRC | 12:49 | |
| pabelanger | jeblair: when we abort a job, should we also abort the post_playbook tasks? http://paste.openstack.org/show/515971/ I noticed this is passing this morning | 12:49 |
| AJaeger | nikeshm: but that question is wrong here. | 12:49 |
| AJaeger | pabelanger: great! | 12:49 |
| pabelanger | yolanda: looking | 12:49 |
| *** amitgandhinz has quit IRC | 12:50 | |
| openstackgerrit | Dmitry Tantsur proposed openstack-infra/project-config: Add jobs based on ipmitool drivers to ironic-python-agent https://review.openstack.org/329428 | 12:51 |
| *** piet has quit IRC | 12:51 | |
| openstackgerrit | Dmitry Tantsur proposed openstack-infra/project-config: Make ironic-{pxe,agent}_ipmitool jobs voting https://review.openstack.org/312596 | 12:52 |
| openstackgerrit | Dmitry Tantsur proposed openstack-infra/project-config: Make ironic-{pxe,agent}_ipmitool jobs voting https://review.openstack.org/312596 | 12:52 |
| rcarrillocruz | pabelanger: sorry, was off yesterday and today i'll be around on and off | 12:53 |
| *** piet has joined #openstack-infra | 12:53 | |
| rcarrillocruz | i will check it out with the lab, cos not able to reach 8.8.8.8 or whatever is just wrong | 12:53 |
| *** yamamoto has joined #openstack-infra | 12:53 | |
| *** matt-borland has joined #openstack-infra | 12:53 | |
| pabelanger | rcarrillocruz: thanks, I haven't tested this morning | 12:53 |
| *** zeih has quit IRC | 12:54 | |
| *** rossella_s has quit IRC | 12:54 | |
| rcarrillocruz | i also need to continue consolidating new ips and metadata for other servers, some are unreachable or had wrong creds, need to check it out with the ticket owner | 12:54 |
| rcarrillocruz | pabelanger: in other news, are we now on ansible 2.1 on puppetmaster or still pinned on 2.0.x ? | 12:55 |
| rcarrillocruz | i'd like to move forward the cloud launcher replacement for launch-node changes | 12:55 |
| yolanda | thx for reviews | 12:56 |
| *** markusry has joined #openstack-infra | 12:57 | |
| openstackgerrit | Merged openstack-infra/puppet-nodepool: Fix duplicated package for build-essential https://review.openstack.org/326605 | 12:57 |
| *** Hal has joined #openstack-infra | 12:58 | |
| *** yujunz has joined #openstack-infra | 12:58 | |
| *** Hal is now known as Guest16087 | 12:58 | |
| pabelanger | rcarrillocruz: still 2.0.x IIRC | 12:59 |
| rcarrillocruz | k | 12:59 |
| *** zeih has joined #openstack-infra | 12:59 | |
| pabelanger | rcarrillocruz: I think the issue is related to the change of the default type for AnsibleModules | 12:59 |
| rcarrillocruz | cos i've tested ansible latest from pypi (2.1.0) on one of my vms for other ansible-puppet stuff and haven't hit issues, so unsure if it's just i've been lucky or the fixes have already merged | 12:59 |
| *** salv-orlando has quit IRC | 13:00 | |
| *** salv-orlando has joined #openstack-infra | 13:00 | |
| *** zeih has quit IRC | 13:01 | |
| *** jcoufal has joined #openstack-infra | 13:02 | |
| *** xyang1 has joined #openstack-infra | 13:04 | |
| *** Goneri has joined #openstack-infra | 13:06 | |
| *** yamamoto has quit IRC | 13:07 | |
| *** yamamoto has joined #openstack-infra | 13:07 | |
| *** rossella_s has joined #openstack-infra | 13:08 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/ansible-role-puppet: Make hostvars[inventory_hostname].group_names is a list https://review.openstack.org/329431 | 13:09 |
| pabelanger | mordred: Shrews: rcarrillocruz: ^ I think that will help with ansible 2.1.0.0 | 13:09 |
| *** mdrabe has joined #openstack-infra | 13:10 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/ansible-role-puppet: Make puppet_get_hiera_file_list.groups a list https://review.openstack.org/329431 | 13:10 |
| rcarrillocruz | ah nice, typing is good... Shrews did a great job cleaning up a lot of params on the openstack modules that lacked typing | 13:10 |
| *** yamamoto has quit IRC | 13:12 | |
| pabelanger | Ya, I cannot remember our exact error, but know it was related to ansible-role-puppet | 13:13 |
| *** afred312 has joined #openstack-infra | 13:14 | |
| *** zhurong has joined #openstack-infra | 13:15 | |
| *** mrmartin has joined #openstack-infra | 13:16 | |
| *** asselin_ has joined #openstack-infra | 13:16 | |
| *** zhurong has quit IRC | 13:18 | |
| Srinu | asselin_: hi.i am using fc pass through in ubuntu 14.04.4 for our FC driver in in CI using this link https://github.com/openstack/third-party-ci-tools/blob/master/provisioning_scripts/fibre_channel/invoke_fc_passthrough.sh but it is failing at the commnad virsh attach-device giving error as http://paste.openstack.org/show/515937/ | 13:18 |
| *** zhurong has joined #openstack-infra | 13:19 | |
| *** pilgrimstack has joined #openstack-infra | 13:20 | |
| Shrews | pabelanger: yeah, you really need to be explicit with types | 13:20 |
| Shrews | pabelanger: you should make sure that the code that uses those params doesn't treat them as anything other than a list | 13:21 |
| *** gouthamr has joined #openstack-infra | 13:21 | |
| *** piet has quit IRC | 13:22 | |
| pabelanger | Shrews: ya, we are pulling in the variable directly from ansible, group_names | 13:23 |
| *** _ari_ has joined #openstack-infra | 13:23 | |
| *** pilgrimstack has quit IRC | 13:24 | |
| *** amotoki has joined #openstack-infra | 13:24 | |
| openstackgerrit | Omar Shykhkerimov proposed openstack-infra/project-config: Add murano periodic jobs with oslo master https://review.openstack.org/329440 | 13:25 |
| Srinu | amotoki: i am using fc pass through in ubuntu 14.04.4 for our FC driver in in CI using this link https://github.com/openstack/third-party-ci-tools/blob/master/provisioning_scripts/fibre_channel/invoke_fc_passthrough.sh but it is failing at the commnad virsh attach-device giving error as http://paste.openstack.org/show/515937/ | 13:27 |
| asselin_ | Srinu, hi. Best to ask this in #openstack-cinder | 13:27 |
| Srinu | asselin_: Ok. Thank you | 13:28 |
| amotoki | Srinu: i think so. I wonder you find me.. | 13:28 |
| *** ddieterly has joined #openstack-infra | 13:28 | |
| asselin_ | patrickeast and hemna might be able to help | 13:29 |
| *** ericksonsantos has joined #openstack-infra | 13:29 | |
| Srinu | asselin_: Did you faced this issue anytime | 13:31 |
| asselin_ | Srinu, I probably did, but don't remember. check that the drivers are loaded? | 13:32 |
| fungi | Srinu: i plugged the error you provided into the most advanced diagnostic system available... a web search engine... and it indicated that you might need to make allowances in apparmor for that | 13:33 |
| *** rfolco has joined #openstack-infra | 13:33 | |
| *** woodster_ has joined #openstack-infra | 13:33 | |
| Srinu | asselin_: which drivers i have to check | 13:33 |
| *** mriedem_away is now known as mriedem | 13:33 | |
| asselin_ | https://github.com/openstack/third-party-ci-tools/blob/master/provisioning_scripts/fibre_channel/install_fc_drivers.sh | 13:34 |
| *** annegentle has joined #openstack-infra | 13:34 | |
| *** erlon has joined #openstack-infra | 13:35 | |
| Srinu | asselin_: i installed drivers correctly | 13:35 |
| openstackgerrit | Raildo Mascena proposed openstack-infra/project-config: Run Heat functional tests in Identity v3-only https://review.openstack.org/314643 | 13:35 |
| fungi | Srinu: check dmesg output for any apparmor="DENIED" lines | 13:36 |
| asselin_ | check dmesg and syslog for more errors. I don't know where the virsh logs are. double check the fcoe and make sure it's correct for your fc environment | 13:36 |
| *** amoralej|lunch is now known as amoralej | 13:37 | |
| *** piet has joined #openstack-infra | 13:39 | |
| fungi | Srinu: nikeshm: also, as i and others have indicated, troubleshooting fc passthrough is pretty off-topic in this channel. you're better off seeking help from the developers of the software which is providing that error (virsh, qemu, nova, et cetera) | 13:40 |
| *** rbrndt has joined #openstack-infra | 13:40 | |
| *** abregman_ has quit IRC | 13:40 | |
| *** edmondsw has joined #openstack-infra | 13:41 | |
| rook | pabelanger another noob question re-infra... so, I have jenkins +1 but I am still unable to push ? https://review.openstack.org/#/c/328991/14 | 13:41 |
| *** PollyZ has joined #openstack-infra | 13:41 | |
| fungi | rook: you added your workflow +1 before there were any code-review +2 votes | 13:42 |
| Srinu | asselin_: the error in logs is http://paste.openstack.org/show/515992/ | 13:42 |
| fungi | rook: set workflow back to 0 and then to +1 again and it should trigger enqueuing into the gate | 13:42 |
| *** mtanino has joined #openstack-infra | 13:43 | |
| *** faizy_ has joined #openstack-infra | 13:44 | |
| rook | fungi ok - just did that. | 13:44 |
| *** xarses_ has quit IRC | 13:45 | |
| *** eharney has joined #openstack-infra | 13:45 | |
| asselin_ | Srinu, i don't see you in #openstack-cinder. I suggest you ask there because all the users of that script are cinder folks. | 13:45 |
| *** xarses_ has joined #openstack-infra | 13:45 | |
| *** amitgandhinz has joined #openstack-infra | 13:46 | |
| fungi | rook: note the automated comment which just appeared on that change. now search for 328991 on http://status.openstack.org/zuul/ to see its progress | 13:46 |
| *** fifieldt has joined #openstack-infra | 13:47 | |
| *** ddieterly is now known as ddieterly[away] | 13:47 | |
| fungi | jeblair: mordred: olaph: oh, look, that change (328991) came from someone with an ncsu.edu address! | 13:47 |
| *** dizquierdo has joined #openstack-infra | 13:47 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Allow the Subscribe directive to work without a list of subscriptions https://review.openstack.org/324549 | 13:47 |
| olaph | ncsu ftw! | 13:49 |
| rook | fungi: yup! looks good!! | 13:49 |
| rook | fungi so... is there a way to remove jenkins from pushing the change? | 13:49 |
| rook | and allow the *-core group to do it | 13:50 |
| rook | olaph you talking about Sai? | 13:50 |
| rook | olaph he is a NCSU Grad! | 13:50 |
| fungi | rook: jenkins doesn't "push" the change, it merely creates a merge commit for it (if needed) | 13:50 |
| olaph | rook: as am I! | 13:50 |
| rook | fungi - "Change has been successfully merged into the git repository by Jenkins" | 13:50 |
| rook | olaph++ | 13:50 |
| *** rbrndt has quit IRC | 13:51 | |
| rook | olaph: my wife was a UNC grad... watch out :P | 13:51 |
| rook | olaph: and she is always right! | 13:51 |
| *** amitgandhinz has quit IRC | 13:51 | |
| fungi | half my friends are unc grads and the other half ncsu grads. it's fun when they all come over for parties | 13:51 |
| *** PollyZ has quit IRC | 13:52 | |
| rook | fungi nice! | 13:52 |
| *** mrmartin has quit IRC | 13:52 | |
| fungi | rook: look at the commit log at http://git.openstack.org/cgit/openstack/browbeat/log/ and you'll see it's the original commit | 13:52 |
| * olaph assumes 'fun' is the polite term... | 13:52 | |
| fungi | olaph: actually, they all get along fine | 13:53 |
| Shrews | fungi: since i fall into neither of those groups, i now know where i stand | 13:53 |
| fungi | Shrews: you should come over more (or at all) | 13:53 |
| Shrews | fungi: last time i came over, i had to move furniture | 13:53 |
| *** mrmartin has joined #openstack-infra | 13:54 | |
| Shrews | so... fool me once, etc etc | 13:54 |
| fungi | oh, oops! | 13:54 |
| fungi | rook: the automation we have is basically taking your workflow +1 as an indication the change should merge, but defers merging it until it passes a final battery of testst (and until its prerequisites are merged, if there are any) | 13:56 |
| *** PollyZ has joined #openstack-infra | 13:56 | |
| *** yamamoto has joined #openstack-infra | 13:56 | |
| fungi | rook: however, it preserves the original commit, and merely adds a merge commit if required to stitch it into the history | 13:57 |
| *** ddieterly[away] is now known as ddieterly | 13:58 | |
| *** rbrndt has joined #openstack-infra | 13:59 | |
| *** xarses_ has quit IRC | 14:00 | |
| *** hongbin has joined #openstack-infra | 14:01 | |
| rook | i see fungi - so dont +1 until we want to jenkins to push | 14:01 |
| phschwartz | clarkb: another review is comming for the launch_node.py that changes how we are using paramiko. Under certain situations the transport will throw a timeout socker.error exception that fails to be caught and is the cause of the issues I have been having. Cleaning up the catch all exception for the SSHException finally exposed the issue. | 14:01 |
| rook | fungi thanks... /me is used to the gerrithub.io workflow where I manually pushed | 14:01 |
| fungi | rook: workflow +1, right | 14:01 |
| Qiming | hi, infra team, really appreciate it if you can help diagnose a problem at #senlin channel, gerritbot has disappeared for some days now ... | 14:01 |
| thiagop | Whatever you folks did to make devstack runs that fast, thank you so much! | 14:01 |
| *** mhickey has quit IRC | 14:02 | |
| *** Julien-zte has joined #openstack-infra | 14:02 | |
| *** burgerk has joined #openstack-infra | 14:02 | |
| fungi | rook: there's a brief slide presentation at http://docs.openstack.org/infra/publications/zuul/ which covers the highlights/benefits of this model | 14:03 |
| *** Julien-z_ has joined #openstack-infra | 14:03 | |
| *** zz_dimtruck is now known as dimtruck | 14:03 | |
| *** kushal has joined #openstack-infra | 14:04 | |
| *** pgadiya has joined #openstack-infra | 14:04 | |
| fungi | Qiming: i saw in scrollback where pleia2 was looking at it with you. skimming the logs for the bot, strangely the last two channels it _claims_ to join are #senlin and #openstack-searchlight but it never actually joins either of them that i can find (though it also doesn't mention any error) | 14:04 |
| fungi | Qiming: pleia2: as far as i can tell the first 242 channels it joins, it shows up in, but those last 2 it doesn't | 14:05 |
| *** jaosorior has quit IRC | 14:05 | |
| *** pgadiya has quit IRC | 14:05 | |
| *** yujunz has quit IRC | 14:05 | |
| Qiming | maybe it was getting tired knocking at so many doors ... | 14:05 |
| fungi | i wonder if we're hitting the chanlimit | 14:06 |
| *** mrmartin has quit IRC | 14:06 | |
| openstackgerrit | Raildo Mascena proposed openstack-infra/project-config: Run cinderclient functional tests in Identity v3-only https://review.openstack.org/313562 | 14:06 |
| fungi | maybe not. `/quote version` claims "CHANLIMIT=#:120" | 14:06 |
| *** Julien-zte has quit IRC | 14:07 | |
| Qiming | could it be some channel flags changed? | 14:07 |
| *** xarses_ has joined #openstack-infra | 14:08 | |
| *** xarses_ has quit IRC | 14:08 | |
| *** xarses_ has joined #openstack-infra | 14:09 | |
| *** yujunz has joined #openstack-infra | 14:10 | |
| *** amitgandhinz has joined #openstack-infra | 14:11 | |
| *** amitgandhinz has quit IRC | 14:11 | |
| *** cody-somerville has joined #openstack-infra | 14:11 | |
| *** cody-somerville has joined #openstack-infra | 14:11 | |
| *** amitgandhinz has joined #openstack-infra | 14:12 | |
| fungi | also the debug log isn't showing a ERRTOOMANYCHANNELS or anything along those lines that i can find | 14:12 |
| *** PollyZ has quit IRC | 14:13 | |
| *** PollyZ_ has joined #openstack-infra | 14:13 | |
| *** links has quit IRC | 14:14 | |
| *** cody-somerville has quit IRC | 14:16 | |
| *** mhickey has joined #openstack-infra | 14:17 | |
| *** faizy_ has quit IRC | 14:17 | |
| *** exploreshaifali has quit IRC | 14:18 | |
| fungi | on the other hand, gerritbot doesn't seem to log anything it receives from irc, so it may be that it's gettign an error back and we just don't know what | 14:19 |
| openstackgerrit | John Trowbridge proposed openstack-infra/tripleo-ci: [NO MERGY] Test a fake periodic job https://review.openstack.org/229789 | 14:20 |
| fungi | probably needs to be patched to log server messages at a minimum | 14:20 |
| fungi | as a short-term fix, we could try pruning some unused channels from its configuration and see if it starts joining those last two | 14:21 |
| *** xarses_ has quit IRC | 14:21 | |
| fungi | those are both pretty low-hanging-fruit options if anyone has time to work on them | 14:22 |
| fungi | (patching gerritbot to record server messages, checking channels in our gerritbot config to identify some unused ones) | 14:23 |
| yamamoto | pabelanger: ping | 14:24 |
| openstackgerrit | Darragh Bailey proposed openstack-infra/project-config: Enable python3 jobs for git-upstream https://review.openstack.org/329475 | 14:25 |
| *** matt-borland has quit IRC | 14:27 | |
| *** nmagnezi has quit IRC | 14:29 | |
| pabelanger | yamamoto: hello | 14:30 |
| yamamoto | pabelanger: hi | 14:30 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: WIP: Remove pagination for timeline events https://review.openstack.org/327682 | 14:30 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Rework timeline events into a timeline https://review.openstack.org/327231 | 14:30 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Add a way to make time-moment not use am-time-ago https://review.openstack.org/329478 | 14:30 |
| openstackgerrit | Raildo Mascena proposed openstack-infra/project-config: Run glanceclient functional tests in Identity v3-only https://review.openstack.org/313572 | 14:30 |
| yamamoto | pabelanger: can you explain you comment on https://review.openstack.org/#/c/315033/ ? | 14:30 |
| *** vhosakot has joined #openstack-infra | 14:30 | |
| yamamoto | your | 14:31 |
| *** mriedem1 has joined #openstack-infra | 14:31 | |
| yamamoto | does it imply some changes in grafyaml? | 14:31 |
| pabelanger | yamamoto: Right, lets create a new sub directory called openstack. For now, we can add midonet into that, and do a follow up patch for tempest and neutron | 14:31 |
| pabelanger | I don't believe we need to update grafyaml | 14:32 |
| *** _nadya_ has quit IRC | 14:32 | |
| *** _nadya_ has joined #openstack-infra | 14:33 | |
| sdague | mordred / jeblair - could one you upgrade mriedem & I to admin here - https://launchpad.net/~hacking-drivers/+members | 14:33 |
| *** mriedem has quit IRC | 14:33 | |
| *** kaisers has joined #openstack-infra | 14:33 | |
| *** sree has quit IRC | 14:35 | |
| yamamoto | pabelanger: at least tools/grafyaml-validate.sh needs a change to visit the subdirectory, right? | 14:35 |
| pabelanger | yamamoto: let me check | 14:35 |
| *** _nadya_ has quit IRC | 14:35 | |
| *** jamesmcarthur has joined #openstack-infra | 14:35 | |
| *** mhickey has quit IRC | 14:35 | |
| pabelanger | yamamoto: let me move tempest first | 14:35 |
| pabelanger | yamamoto: then you can rebase onto it | 14:36 |
| *** ayoung has joined #openstack-infra | 14:36 | |
| yamamoto | pabelanger: thank you! | 14:36 |
| *** _nadya_ has joined #openstack-infra | 14:37 | |
| *** rfolco has quit IRC | 14:37 | |
| pabelanger | yolanda: okay, that doesn't work, I'll add it to my list to fix. In the mean time, lets see if AJaeger can help +A 315033 | 14:41 |
| *** _nadya_ has quit IRC | 14:41 | |
| * AJaeger checks | 14:41 | |
| *** mdrabe has quit IRC | 14:42 | |
| pabelanger | yamamoto: ^ sorry yolanda | 14:42 |
| yolanda | pabelanger, ah, i was reading backlog trying to figure what was the message about :) | 14:42 |
| *** nmagnezi has joined #openstack-infra | 14:42 | |
| AJaeger | pabelanger: is span 4 really right - or should it be 3? | 14:43 |
| *** EricGonczer_ has joined #openstack-infra | 14:43 | |
| fungi | sdague: mriedem1: done. use your new-found superpowers only for good | 14:43 |
| pabelanger | AJaeger: should be, means panels are wider by will span 2 row. There are 5 of them | 14:44 |
| sdague | fungi: thanks | 14:44 |
| mriedem1 | fungi: what do i have now and why? | 14:44 |
| AJaeger | pabelanger: ok | 14:44 |
| *** _ari_ has quit IRC | 14:44 | |
| *** xarses_ has joined #openstack-infra | 14:44 | |
| *** _ari_ has joined #openstack-infra | 14:44 | |
| *** garyk has quit IRC | 14:44 | |
| fungi | mriedem1: sdague volunteered you to administer the hacking-drivers team on lp | 14:44 |
| *** ajmiller has quit IRC | 14:45 | |
| AJaeger | sdague: could you review https://review.openstack.org/313588 , please? | 14:45 |
| mriedem1 | oh i see | 14:45 |
| fungi | mriedem1: perhaps because he knows you don't have nearly enough to do already? ;) | 14:45 |
| * mriedem1 needs to update his resume then | 14:45 | |
| *** ajmiller has joined #openstack-infra | 14:45 | |
| *** eharney has quit IRC | 14:45 | |
| fungi | mriedem1: i hope your resume has a section called "thankless tasks" | 14:46 |
| *** rfolco has joined #openstack-infra | 14:46 | |
| clarkb | pabelanger: rcarrillocruz did we get anywhere with infra cloud firewall issues? | 14:46 |
| *** inc0 has joined #openstack-infra | 14:46 | |
| pabelanger | clarkb: I have not, but I also don't know who to contact other then rcarrillocruz | 14:46 |
| *** julim has quit IRC | 14:47 | |
| *** mhickey has joined #openstack-infra | 14:47 | |
| *** reduxio has joined #openstack-infra | 14:48 | |
| Qiming | fungi, following your suggestion, just digged through the channel list, cross verified the git repo status | 14:50 |
| fungi | Qiming: excellent. find a few we should clean up? | 14:50 |
| Qiming | it seems to me, the following two channels can be removed: openstack-nova-solver-scheduler, openstack-magnetodb | 14:50 |
| pabelanger | clarkb: who is our contact at OSIC? I wanted to follow up with them about our SSH timeout issues on a set of public IPs | 14:50 |
| Qiming | both channels are empty | 14:50 |
| *** xarses_ is now known as xarses | 14:50 | |
| clarkb | pabelanger: cloudnull hangs out here in channel | 14:50 |
| Qiming | both projects seemingly dead | 14:50 |
| pabelanger | clarkb: cloudnull helped in the past, but he is currently away for a while longer | 14:51 |
| Qiming | fungi, as for "openstack-vahana", not so sure, it is a new thing, not commit for 6 months, channel empty | 14:51 |
| *** zhurong has quit IRC | 14:52 | |
| jeblair | pabelanger: i'm not entirely sure i want to ignore rsync errors. i think it would be good to make the ansible logs available to users so they can diagnose that output themselves. but really, i think this is exposing an actual malfunction in the job. | 14:52 |
| clarkb | pabelanger: JayF is also at OSIC I think and James Thorne is the person that set up the cluster for us | 14:52 |
| reduxio | Hello. My company is developing a cinder driver, and I am currently in the process of setting up the third party CI. I understand that part of the process is to register as a third party company in order to be allowed to comment to gerrit. can anyone help with how i do this? | 14:52 |
| JayF | clarkb: pabelanger: I've got nothing to do with that OSIC cluster, but I can point people in your direction if needed. | 14:53 |
| jeblair | pabelanger: like -- someone said "copy a file from /var/log" but didn't correct the permissions to do that. that's a bug in the job. | 14:53 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard: Support subscription to worklists in the API https://review.openstack.org/324493 | 14:53 |
| JayF | reduxio: I believe that's all fairly well documented here: http://docs.openstack.org/infra/system-config/third_party.html | 14:53 |
| jeblair | clarkb, pabelanger: what if we told zuul-launcher to put the ansible output the same place as the console log? basically just overloaded the 'copy console log' publisher to also mean 'this is where you should copy the ansible log output' | 14:54 |
| jeblair | clarkb, pabelanger: we could either concatenate it to the console log, or make a separate file.... | 14:55 |
| * jeblair leans toward separate file | 14:55 | |
| clarkb | +1 for separte file | 14:55 |
| *** amotoki has quit IRC | 14:55 | |
| *** eharney has joined #openstack-infra | 14:56 | |
| fungi | Qiming: yeah, openstack/nova-solver-scheduler does look sort of dead (no commits this year, core reviewer team of 2, empty channel except for chanserv and openstackgerrit) | 14:56 |
| fungi | Qiming: and i believe magnetodb was completely dissolved? double-checking | 14:57 |
| reduxio | Jayf: In the link you sent, it says i need to join the weekly meeting and points me to the wiki. In the wiki, it states that if you want to participate in the meeting i need to add the issue to the meeting agenda. In order to add to the meeting agenda, i need to the login to the wiki. This fails with http error 500. | 14:57 |
| AJaeger | fungi: megnetodb is not in gerritbot setup AFAIK | 14:57 |
| fungi | AJaeger: Qiming: yeah, i don't see #openstack-magnetodb in the current config | 14:58 |
| openstackgerrit | Alina Nesterova proposed openstack-infra/project-config: Migrated sahara-client-functional-job https://review.openstack.org/329497 | 14:58 |
| JayF | reduxio: Wiki logins were disabled due to spam. You're 100% right. If you ask in here (I don't have access, I just idle here :P), someone may be willing to enable wiki logins for you to get a user "created" with your openid. Alternatively, I bet anteaya would be willing to add you to the agenda as iirc she runs those meetings. | 14:58 |
| pabelanger | jeblair: I was actually thinking about add log_path to our ansible.cfg (per job), that would give us an output log per job run. The figure out the upload part | 14:58 |
| *** mdrabe has joined #openstack-infra | 14:58 | |
| *** _ody has joined #openstack-infra | 14:58 | |
| fungi | reduxio: try logging in again. i've temporarily reenabled wiki account creation | 14:58 |
| pabelanger | jeblair: then upload zuul-launcher logs first, then job logs | 14:59 |
| pabelanger | should make debugging easier for the user | 14:59 |
| clarkb | fungi: how do you temporarily enable it? edit the config.php and restart apache? | 14:59 |
| openstackgerrit | Raildo Mascena proposed openstack-infra/project-config: Run Magnum functional tests in Identity v3-only https://review.openstack.org/315118 | 14:59 |
| reduxio | fungi: thanks! logged in now. | 14:59 |
| fungi | clarkb: simpler still. comment out line 149 (currently) of /srv/mediawiki/Settings.php | 14:59 |
| fungi | clarkb: #$wgGroupPermissions['*']['createaccount'] = false; | 14:59 |
| clarkb | fungi: no service restarts required? | 14:59 |
| fungi | clarkb: no restart needed, it's automatically reread on each connection apparently | 15:00 |
| jeblair | pabelanger: we still need the logs in the zuul-launcher logs. but as long as we can do that. | 15:00 |
| clarkb | huh | 15:00 |
| openstackgerrit | Merged openstack-infra/project-config: Add networking-midonet grafana dashboard https://review.openstack.org/315033 | 15:00 |
| fungi | reduxio: awesome, let us know if you need anything else | 15:00 |
| *** jamesmcarthur has quit IRC | 15:01 | |
| *** bhavik has joined #openstack-infra | 15:01 | |
| pabelanger | jeblair: agreed. 326006 was my first attempt to do our uploads from zuul-launcher, but not sure I like it as it creates an empty publisher in jenkins | 15:01 |
| pabelanger | I believe 326006,1 over 326006,2 | 15:02 |
| jeblair | pabelanger: yeah, i'm thinking maybe we just interpret the definition of a console log liberally to include also ansible logs and maybe ansible config if we want | 15:02 |
| pabelanger | works getter | 15:02 |
| pabelanger | jeblair: okay | 15:03 |
| jeblair | pabelanger: can we have ansible run every post task regardless of whether previous ones have succeeded or failed, but then still exit with a failure code if any have failed? | 15:03 |
| jeblair | Shrews: ^ | 15:03 |
| openstackgerrit | Alina Nesterova proposed openstack-infra/project-config: Migrated sahara-client-functional-job https://review.openstack.org/329497 | 15:03 |
| *** jamesmcarthur has joined #openstack-infra | 15:03 | |
| *** Swanson has joined #openstack-infra | 15:03 | |
| pabelanger | jeblair: I would need to check | 15:03 |
| clarkb | pabelanger: are ssh timeouts the source of the fairly consistent node boot errors in osic? | 15:04 |
| Shrews | jeblair: couldn't zuul handle that? you run each separately | 15:05 |
| clarkb | pabelanger: its possible we just need to bump up that timeout because cloud can't reliably do it in the allocated time | 15:05 |
| pabelanger | jeblair: I think we can use the always flag for block tasks | 15:05 |
| pabelanger | clarkb: Ya, it was a list of specific IPs. Let me see if I can find it again | 15:05 |
| Shrews | jeblair: oh, i misunderstood. you mean isolated to the post playbook only | 15:05 |
| jeblair | ya | 15:05 |
| *** yamahata has joined #openstack-infra | 15:05 | |
| jeblair | Shrews: because post will have several upload steps, and if one breaks, we still may want others to run, especially if the others tell us why the one broke | 15:06 |
| Shrews | jeblair: i think there is an ignore_errors thing | 15:06 |
| *** wc2912 has joined #openstack-infra | 15:06 | |
| jeblair | Shrews: (but i still want to know, at the end, if any failed with the exit code) | 15:06 |
| *** abregman has joined #openstack-infra | 15:07 | |
| *** mhickey has quit IRC | 15:07 | |
| *** Jeffrey4l has quit IRC | 15:07 | |
| jeblair | yeah, i think ignore_errors may be what we want as long as we still get the exit code (the docs are unspecific on that point) | 15:08 |
| rcarrillocruz | yup, passing ignore_errors true to a task makes it run regardless | 15:08 |
| Shrews | jeblair: maybe with some sortof handler. i'd have to investigate | 15:08 |
| rcarrillocruz | but that's a task only thing | 15:08 |
| *** fifieldt has quit IRC | 15:08 | |
| clarkb | rcarrillocruz: did you see there are firewall issues with the infracloud host(s) | 15:08 |
| rcarrillocruz | clarkb: i'll check | 15:08 |
| Shrews | jeblair: i don't know what ignore_errors does for the playbook exit code. | 15:09 |
| *** pradk has quit IRC | 15:11 | |
| *** rcernin has quit IRC | 15:11 | |
| rcarrillocruz | i don't think it's a firewall issue on google dns nameservers, but rather an outbound gw issue, i can't ping anything | 15:11 |
| rcarrillocruz | i'll check with the net folks | 15:11 |
| clarkb | rcarrillocruz: its likely the same firewall also blocks icmp :) | 15:12 |
| pabelanger | clarkb: http://paste.openstack.org/show/507754/ that was the list of IPs we cannot SSH into | 15:13 |
| clarkb | pabelanger: so its a consistent list of IPs that fail? fun | 15:13 |
| pabelanger | clarkb: Yup, I am hoping some ACL on osic side | 15:13 |
| *** julim has joined #openstack-infra | 15:14 | |
| *** lxsli has left #openstack-infra | 15:14 | |
| openstackgerrit | Merged openstack-infra/project-config: Pass extra parameter to hooks in neutron-lbaas tempest jobs https://review.openstack.org/322811 | 15:15 |
| openstackgerrit | Merged openstack-infra/project-config: Add Sheepdog test job to glance_store https://review.openstack.org/328148 | 15:15 |
| *** jlanoux has quit IRC | 15:18 | |
| *** croelandt has joined #openstack-infra | 15:18 | |
| *** psachin has quit IRC | 15:18 | |
| *** pilgrimstack has joined #openstack-infra | 15:18 | |
| rcarrillocruz | clarkb: not just icmp, but everything then, cos http does not work either | 15:19 |
| rcarrillocruz | which is i why i beleive there's an issue with the gw | 15:19 |
| rcarrillocruz | checking now with some folk | 15:19 |
| zul | mind someone +1ing this please? https://review.openstack.org/#/c/329331/ | 15:20 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Refactor local_action SCP task into function https://review.openstack.org/325601 | 15:22 |
| *** pilgrimstack has quit IRC | 15:24 | |
| Shrews | jeblair: a bit cumbersome with the registered vars, but this works: http://paste.openstack.org/show/516014/ | 15:24 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Add infrstucture scripts to prepare rh2 https://review.openstack.org/295243 | 15:24 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: [WIP] Add support for OVB based CI https://review.openstack.org/329521 | 15:24 |
| Shrews | jeblair: sort of a brute-force way which is kind of meh | 15:25 |
| *** xarses_ has joined #openstack-infra | 15:25 | |
| *** salv-orlando has quit IRC | 15:25 | |
| jeblair | Shrews: yeah, our autogenerated playbook will definitely look like an autogenerated playbook. :) | 15:25 |
| jeblair | Shrews: i just confirmed ignore_errors exits 0 :( | 15:25 |
| *** xarses has quit IRC | 15:26 | |
| Shrews | jeblair: yeah, i did too, which led me to that paste | 15:26 |
| *** salv-orlando has joined #openstack-infra | 15:26 | |
| Shrews | jeblair: gotta run out for a bit. bbl | 15:26 |
| jeblair | Shrews: thanks! | 15:27 |
| openstackgerrit | Thanh Ha proposed openstack-infra/jenkins-job-builder: Allow JJB to spoof plugin_info via yaml https://review.openstack.org/326722 | 15:27 |
| *** Julien-z_ has quit IRC | 15:28 | |
| openstackgerrit | Thanh Ha proposed openstack-infra/jenkins-job-builder: Allow update to provide plugin_info via yaml https://review.openstack.org/326722 | 15:28 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Add log_path to ansible.cfg https://review.openstack.org/329525 | 15:29 |
| *** javeriak has joined #openstack-infra | 15:29 | |
| *** salv-orlando has quit IRC | 15:30 | |
| *** salv-orlando has joined #openstack-infra | 15:30 | |
| *** tongli has joined #openstack-infra | 15:31 | |
| dhellmann | we're seeing lots of failures for wheel-build-ubuntu-trusty-amd64-python2 and wheel-build-centos-7-amd64-python2 from releases this morning; those just update our cache, right? so we don't need to worry that they failed? | 15:31 |
| *** matt-borland has joined #openstack-infra | 15:31 | |
| dhellmann | the error is "The executable {python} (from --python={python}) does not exist" -- http://logs.openstack.org/f5/f59009760a073b2125abdeeea703b2cc09cf17c9/release/wheel-build-centos-7-amd64-python2/363c6ea/console.html | 15:31 |
| dhellmann | looks like some sort of templating issue | 15:32 |
| pabelanger | dhellmann: checking | 15:32 |
| *** zeih has joined #openstack-infra | 15:32 | |
| *** zeih has quit IRC | 15:32 | |
| *** zeih has joined #openstack-infra | 15:32 | |
| pabelanger | dhellmann: yes, I see the issue. Let me propose a fix. | 15:33 |
| dhellmann | pabelanger : great, thanks! do we need to requeue those jobs? | 15:34 |
| *** jtomasek_ has joined #openstack-infra | 15:34 | |
| pabelanger | dhellmann: I'd have to see, but we can if needed | 15:34 |
| croelandt | Hey! I've been looking at bug https://bugs.launchpad.net/python-glanceclient/+bug/1546258 , and it seems that the tarball for glanceclient 1.2.0 is missing from PyPI (see https://pypi.python.org/pypi/python-glanceclient/json) | 15:35 |
| openstack | Launchpad bug 1546258 in python-glanceclient "python-glanceclient==1.2.0 not found by setuptools" [Undecided,New] | 15:35 |
| croelandt | Is there anything that can be done about this? | 15:35 |
| dhellmann | pabelanger : ok, http://lists.openstack.org/pipermail/release-job-failures/2016-June/thread.html#0 is a good source for a list if we need to | 15:35 |
| *** abregman is now known as abregman|afk | 15:35 | |
| jeblair | clarkb, pabelanger: how about we modify 325602 so that instead of looking for specific jjb configuration, we add an extra task to the console-logs publisher that copies the logs and ansible config into the staging area so that it gets uploaded with the console log upload. | 15:35 |
| *** Na3iL has quit IRC | 15:36 | |
| jamesmcarthur | clarkb: fungi: mordred: Per our conversation in Prague, can we move forward with patch# 285641 and then we will work with jpmaxman on next steps? | 15:36 |
| *** sorantis has quit IRC | 15:37 | |
| openstackgerrit | John L. Villalovos proposed openstack-infra/devstack-gate: WIP: Remove 'partial' support for Grenade https://review.openstack.org/329532 | 15:37 |
| *** zeih has quit IRC | 15:38 | |
| *** permalac has joined #openstack-infra | 15:38 | |
| thiagop | Guys, do zuul developers chat on this channel too? | 15:38 |
| pabelanger | jeblair: clarkb: We can start with that, as it gets our logs uploaded for now. But ideally, I would like to have publishing task specific to zuul-launcher, before we attempt to fetch logs from the remote node. That way, if that console task fails, we still have our logs published. | 15:39 |
| *** eezhova has quit IRC | 15:39 | |
| jeblair | pabelanger: i think we will... here, let me etherpad it | 15:39 |
| clarkb | jamesmcarthur: we should definitely keep moving on the things. I think fungi had some previously unknown info that maybe he wanted to share? (I think he and pleia2 were talking about doing the wiki upgrade real soon now like right now maybe?) | 15:39 |
| clarkb | thiagop: they do | 15:40 |
| scottda | Hi Infra. I'm looking for the place where I set a devstack variable. for example, L#240 in devstack-vm-gate.sh: | 15:40 |
| scottda | https://www.irccloud.com/pastebin/v2PLzHIQ/ | 15:40 |
| *** nmagnezi has quit IRC | 15:40 | |
| fungi | jamesmcarthur: jpmaxman: sorry, been meaning to reply to the points in the e-mail thread you started with me, fifieldt, mordred, clarkb and ttx. apparently the discussion in prague was based on some misconceptions (pleia2 and i have picked up the wiki cleanup and upgrade work since it turns out pabelanger and nibalizer can't work on it after all), the patrolling feature fifieldt was asking to have | 15:40 |
| thiagop | clarkb: I'm suspecting canceled jobs aren't releasing mutexes, but my knowledge of the codebase is a little shallow | 15:40 |
| fungi | turned on is and was already enabled, we've been working around the account lockdown by assisting new users in the interim, et cetera | 15:40 |
| scottda | I assume it is in some jenkins job? | 15:40 |
| *** yujunz has quit IRC | 15:40 | |
| scottda | I cannot find that variable in jenkins-job-builder repo. | 15:41 |
| scottda | Is there another repo for jenkins jobs? | 15:41 |
| *** matt-borland has quit IRC | 15:41 | |
| scottda | This is for Tempest testing in the merge queue | 15:41 |
| fungi | jamesmcarthur: jpmaxman: also most of the work we need help with on the wiki doesn't require ssh access at all. for example the thousands of pages/uploads i deleted and hundreds of users i blocked were done entirely through the web interface and mediawiki api | 15:41 |
| *** roxanaghe has joined #openstack-infra | 15:41 | |
| clarkb | scottda: you set it in the environment calling devstack gate | 15:42 |
| jamesmcarthur | fungi: but the wiki is still locked down to new users and not usable by the community | 15:42 |
| openstackgerrit | Thanh Ha proposed openstack-infra/jenkins-job-builder: Allow update to provide plugin_info via yaml https://review.openstack.org/326722 | 15:42 |
| jpmaxman | fungi I think the key question is how the spam was getting into the wiki in the first place | 15:42 |
| jpmaxman | did you figure that out? | 15:42 |
| fungi | jpmaxman: that has never been a question? | 15:42 |
| scottda | clarkb: And isn't that a jenkins job running tempest tests? | 15:42 |
| clarkb | scottda: I think that may be old code for older branches and current branches use a devstack plugin though | 15:42 |
| clarkb | scottda: which is why you don't see the var set in any of the jenkins jobs. Instead the ceph devstack plugin is enabled | 15:42 |
| fungi | jpmaxman: i last week i even found an actual openstack community member whose lp sso account was compromised and used to add spam pages | 15:42 |
| fungi | i've temporarily blocked them in the wiki until we can get up with them to reset their account on launchpad | 15:43 |
| *** yamamoto has quit IRC | 15:43 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Revert "Add python version to wheel builders" https://review.openstack.org/329536 | 15:43 |
| jpmaxman | in so far as I recall we had put captcha requests on edits and new page creations | 15:44 |
| jpmaxman | yet spam was still getting in | 15:44 |
| scottda | clarkb: OK, thanks. That makes sense. If I'm adding a new variable for DEVSTACK_CINDER_ENABLE_MULTI_BACKEND it won't be in a plugin. Where would I set that? | 15:44 |
| *** pradk has joined #openstack-infra | 15:44 | |
| jpmaxman | so the question was is the captcha not good enough | 15:44 |
| fungi | jpmaxman: oh, you're asking whether they're solving the captchas vs bypassing them by using page edit and create api calls? | 15:44 |
| jpmaxman | or was there a way they were getting around | 15:44 |
| pabelanger | fungi: dhellmann: I've proposed a revert for now, so not to block things | 15:44 |
| pabelanger | ^ | 15:44 |
| *** zeih has joined #openstack-infra | 15:44 | |
| clarkb | scottda: you would set that in the calling context of devstack-gate, likely a jenkins job | 15:44 |
| jamesmcarthur | jpmaxman: fungi: the issue is that the wiki is still locked down. we proposed to attempt best practices and try different plugins to ensure that even with access turned on again, we could prevent spam. | 15:44 |
| *** yamamoto has joined #openstack-infra | 15:44 | |
| jpmaxman | what I was wanting to figure out is exactly what the attack vectors were so we could then more knowledgeably move forward | 15:45 |
| jpmaxman | but if you want to start by upgrading everything and seeing where we're at | 15:45 |
| clarkb | jpmaxman: the attack vector is valid accounts were making edits and the rudimentary captcha on old mediawiki doesn't prevent those valid accounts from making edits | 15:45 |
| jpmaxman | that's reasonable too | 15:45 |
| jamesmcarthur | fungi: once we, in conjunction with infra, agreeed on a set of plugins that could be tested and work, we would commit back to the repo so that they would go in gerrit | 15:45 |
| *** roxanaghe has quit IRC | 15:46 | |
| clarkb | jpmaxman: at least that was my understanding, and part of the goal behind mediawiki upgrade was to use better captcha system available in newer mediawiki? | 15:46 |
| jpmaxman | calrkb is that true? If so I think it is a setting - you can set ALL accounts to be presented with captcha when making edits | 15:46 |
| *** eezhova has joined #openstack-infra | 15:46 | |
| clarkb | jpmaxman: we did set that and it didn't stop the spam | 15:46 |
| pabelanger | I would rather we spend the effort on upgrade our puppet manifest, and operating system, then reenable users on wiki.o.o. | 15:47 |
| jpmaxman | so there's the problem | 15:47 |
| scottda | clarkb: OK, thanks for the help. | 15:47 |
| pabelanger | then iterate forward on taht | 15:47 |
| jpmaxman | why didn't it stop the spam? | 15:47 |
| clarkb | jpmaxman: its too rudimentary iwth a tiny problem set | 15:47 |
| jpmaxman | because they were able to bypass captcha? | 15:47 |
| jamesmcarthur | fungi: clarkb: mordred: I'm a little flumoxed here. I feel like we came up with a pretty solid plan. | 15:47 |
| jpmaxman | or because the captcha was too easy? | 15:47 |
| clarkb | jpmaxman: yes that | 15:47 |
| *** yamamoto has quit IRC | 15:47 | |
| jpmaxman | if I recall one thing I suggested was to set the captch as an impossible to answer question | 15:47 |
| fungi | well, more that the current state of the spam mitigation extensions are driven by the spam handled by the wikipedia community, and since wikipedia always runs latest mediawiki those extensions are not well-supported on older releases | 15:47 |
| *** bpokorny has joined #openstack-infra | 15:48 | |
| jpmaxman | to see if it stopped it | 15:48 |
| jeblair | clarkb, pabelanger, Shrews: https://etherpad.openstack.org/p/p25pzG6OSr | 15:48 |
| jpmaxman | then you'd know | 15:48 |
| clarkb | jpmaxman: yes that stopped it | 15:48 |
| *** yamamoto has joined #openstack-infra | 15:48 | |
| jpmaxman | oh was that done? I thought it never was... | 15:48 |
| jamesmcarthur | fungi: clarkb: mordred: Can we confirm that the plan is no longer to approve jpmaxman's patch? | 15:48 |
| clarkb | jpmaxman: it was, we set it to a random many byte string and it killed the spam but using the small set of easier to answer things that humans could know didn't | 15:48 |
| clarkb | jamesmcarthur: I am happy to continue with the old plan but didn't realize that othe rpeople had volunteered separately to do the work already. So just wanted that info to be added in | 15:49 |
| fungi | also we originally had the captcha set only for password-based account creation and all our accounts were being created through openid until we made the captcha answerable, then we started getting all sorts of accounts showing up created without openid | 15:49 |
| jpmaxman | clarkb ok - in that case we could try: https://www.google.com/recaptcha/intro/index.html which may be the quickest fix | 15:49 |
| *** yamamoto has quit IRC | 15:50 | |
| jamesmcarthur | fungi: clarkb: again, the issue here is attempting to figure out which set of plugins are going to work best | 15:50 |
| jpmaxman | fungi: there was also the mobile view vector - both of those should be disabled | 15:50 |
| fungi | jpmaxman: right, the mobile view account creation was password-based, so it was unusable for creating new accounts up until we made the captcha something other than secret gibberish | 15:51 |
| jamesmcarthur | fungi: there was also a discussion of trying to get other people involved and contributing in positive ways to infra. Basically, the state of the wiki is "so bad we can't possibly do any worse" as ttx said. | 15:51 |
| pabelanger | jamesmcarthur: my objection is not to finding out which plugins work, it is the fact we are testing them on our production server. When in fact we should be setting up wiki-dev.o.o | 15:51 |
| jpmaxman | fungi: right also the mobile view login wasn't redirecting to open id | 15:52 |
| *** jlanoux has joined #openstack-infra | 15:52 | |
| jamesmcarthur | pabelanger: traffic doesn't go to wiki-dev.o.o. Right now you have a completely unusable wiki. | 15:52 |
| pabelanger | which, to get wiki-dev.o.o, requires us to migrate to ubuntu-trusty and update our puppet manifests | 15:52 |
| jamesmcarthur | pabelenager: no, we do not need to migrate to ubuntu-trusty | 15:52 |
| pabelanger | yes we do | 15:52 |
| jamesmcarthur | we are talking about utilizing existing plugins | 15:52 |
| jamesmcarthur | to help stop the spam as a first priority | 15:52 |
| jamesmcarthur | pabelanger: this circular reasoning is the reason we have been without a working wiki for 8 months | 15:53 |
| jpmaxman | pabelanger: the production server is what is under attack - so to quickly know what stops the spam it is necessary to run some quick tests there as opposed to dev | 15:53 |
| *** jamesmcarthur has quit IRC | 15:53 | |
| *** andreas_s has quit IRC | 15:53 | |
| jpmaxman | anyway when I last looked at it | 15:53 |
| fungi | the confirmedit extension seems to support recaptcha and works with mw 1.25, which we could enable in the near term https://www.mediawiki.org/wiki/Extension:ConfirmEdit | 15:53 |
| jpmaxman | there was a unified anit-spam plugin (which is installed already) | 15:53 |
| pabelanger | regardless of the spam being stopped or not, we need to migrate wiki.o.o to ubuntu-trusty | 15:53 |
| jpmaxman | and what we actually need to do is tweak that to use the right config of that plugin to be just hard enough to stop the spam but easy enough to let actual users publish | 15:53 |
| clarkb | fungi: right thoes types of steps are what I thin kwe were interested in doing in the very near term | 15:54 |
| fungi | jamesmcarthur: i take exception to the claim that we've been without a working wiki at all, much less for 8 months. people are using it constantly | 15:54 |
| clarkb | and jpmaxman/jamesmcarthur were willing to do the work when no one else was stepping up | 15:54 |
| clarkb | also potentially using the patrol feature to better curate edits to automate spam cleanup as well | 15:54 |
| *** jtomasek_ has quit IRC | 15:54 | |
| clarkb | pabelanger: we do but thats independent of stopping spam | 15:55 |
| clarkb | pabelanger: and unfortunately tying them together has killed any progress in the space | 15:55 |
| *** yamamoto has joined #openstack-infra | 15:55 | |
| fungi | clarkb: patrol is already enabled, as i mentioned earlier. i really think that what we need _first_ are people willing to monitor edits and contributions from new accounts. that's basically all patrolling is anyway | 15:55 |
| *** jamesmcarthur has joined #openstack-infra | 15:56 | |
| clarkb | fungi: yup, we were mistaken that it wasn't enabled. So next step there would be recruiting wiki admins to do the patroling | 15:56 |
| fungi | everyone seems to think that this is something we can solve with a few patches and then turn it loose again, but if this is going to work at all it needs a bunch of people keeping an eye on the wiki. that doesn't involve ssh access at all, just wiki curators | 15:56 |
| fungi | it's time spent staring at a web browser | 15:57 |
| *** brucet has joined #openstack-infra | 15:57 | |
| fungi | so far very few people seem to want to do _that_ part | 15:57 |
| jpmaxman | fungi: i prefer tailing logs ;) | 15:57 |
| fungi | i spent two days last week doing it, and removed thousands of pages of spam and blocked hundreds of users | 15:57 |
| bkero | fungi: did you get through all of it? I'm willing to monitor the change list for incoming spam | 15:58 |
| *** sshnaidm has quit IRC | 15:58 | |
| bkero | but that back log is scary | 15:58 |
| jpmaxman | anyway all I was going to do is refine the anti spam settings and test config changes wile watching the logs for the spam attacks | 15:58 |
| jpmaxman | as I said earlier if you want to upgrade everything first (mediawiki version, plugins, trusty, etc) | 15:58 |
| jpmaxman | and then see where you're at - I think that's valid too | 15:58 |
| jamesmcarthur | fungi: jpmaxman: right, but htat's been "the plan" for months now | 15:59 |
| fungi | bkero: as far as i know it is all cleaned up. so far a few people have gone through and done random searches for the sorts of keywords we saw in spam previously and have found no new hits so i think it's 100% but if you spot anything i missed please tell me | 15:59 |
| jpmaxman | If you need some help on the devops side, I'm happy to jump in. If you got it, that's cool too. | 16:00 |
| *** e0ne has quit IRC | 16:00 | |
| bkero | fungi: I'll see if I can spot anything new that pops up | 16:00 |
| fungi | jpmaxman: jamesmcarthur: anyway, "best practices" is not a plan. "install and configure the confirmedit plugin to use recaptcha for page edits, reenable new account creation and then monitor activity to see what slips through" is a plan i can get behind | 16:00 |
| jpmaxman | confirmedit is already installed | 16:00 |
| *** _nadya_ has joined #openstack-infra | 16:01 | |
| clarkb | I think we need to decouple upgrade from spam handling. We know there are plugins and options available to us on current versions of things that we can use. We also know that we need people to curate wiki edits more directly (sounds like bkero is willing to help there and hopeflly the existing wiki admins too) | 16:01 |
| jpmaxman | it just needs to be tweaked to use the right settings that block the spam assuming what clarkb said is correct that a complete hardening stopped it | 16:01 |
| openstackgerrit | Mikhail S Medvedev proposed openstack-infra/puppet-openstackci: Move subunit_worker.pp from system-config https://review.openstack.org/328438 | 16:01 |
| clarkb | then separately some group of people can work on upgrading to newer ubuntu as the platform to run on | 16:01 |
| *** thorst is now known as thorst_afk | 16:01 | |
| fungi | jpmaxman: configure confirmedit to use recaptcha, right. i stand corrected | 16:01 |
| openstackgerrit | Hisashi Osanai proposed openstack-infra/project-config: Add jobs for Monasca-Analytics https://review.openstack.org/329376 | 16:01 |
| jamesmcarthur | fungi: that's a bit frustrating. You know what the proposal was because we outlined it in an email after a discussion with mordred, clarkb, fifeldt, and ttx. | 16:01 |
| *** yamamoto has quit IRC | 16:01 | |
| *** _nadya_ has quit IRC | 16:02 | |
| clarkb | and the third thing that needs to happen is to properly get the wiki into config mgmt | 16:02 |
| *** markusry has quit IRC | 16:02 | |
| fungi | jamesmcarthur: no, you didn't. you wrote "Figure out best plugins for spam prevention/policing" in your plan | 16:02 |
| jamesmcarthur | fungi: right. There are a ton of options. | 16:02 |
| clarkb | the second and third items may be related depending on how difficult an upgrade is without config mgmt | 16:02 |
| jamesmcarthur | We have to see which set of plugins work together to solve the problem. | 16:02 |
| bkero | getting it into config management sounds like a lot of reverse engineering since we don't exactly know how it is set up | 16:03 |
| *** jcoufal_ has joined #openstack-infra | 16:03 | |
| fungi | jamesmcarthur: that's fine. and i like the initial suggestion of "let's see if the recaptcha implementation in confirmedit deters spammers better than the simple question/answer mechanism it has" | 16:03 |
| jamesmcarthur | fungi: if you want to know why people are having a hard time dealing with the infra team, this is a premier example. We came up with a very valid plan, working specifically with members of the infra team. Literally the phrase was "how can it get any worse" | 16:03 |
| jamesmcarthur | fungi: only to have it walked back two days later. | 16:03 |
| jpmaxman | there's also some cloud based solutions | 16:03 |
| *** matrohon has quit IRC | 16:04 | |
| jpmaxman | https://www.mediawiki.org/wiki/Extension:Antispam | 16:04 |
| jpmaxman | $8 / yr | 16:04 |
| *** arxcruz has quit IRC | 16:04 | |
| jpmaxman | but they ostensibly manage keeping up with the attack tactics | 16:04 |
| *** hashar is now known as hasharOut | 16:04 | |
| ttx | jamesmcarthur: The situation is not exactly what I thought it was when we had that discussion. But I would still support direct experimentation under the "can't really be worse" argument. That said I'm not infra :) | 16:04 |
| fungi | jamesmcarthur: jpmaxman: i don't get what's being walked back. i'm in favor of adding ssh access for you to test out recaptcha support in confirmedit and see if it helps. that's likely to get done sooner if you have time to focus on it than if i'm stuck having to work it into my week | 16:05 |
| *** gouthamr has quit IRC | 16:05 | |
| *** mikelk has quit IRC | 16:06 | |
| *** jcoufal has quit IRC | 16:06 | |
| fungi | i just wanted to follow up to your proposed plan to say that it should get a little more specific about which exact solutions were worth trying | 16:06 |
| ttx | It feels like we are still following the spirit of that plan, from where I'm standing | 16:07 |
| jamesmcarthur | fungi: the point is that you have to try out different plugins while the wiki is active and determine which are working to stop spam and which arent' | 16:07 |
| *** bpokorny has quit IRC | 16:07 | |
| jamesmcarthur | well step 1 in the plan was"Approve JP’s patch for root access (patch# 285641), addressing jeblair's comments. | 16:07 |
| jpmaxman | fungi: I think the plan was to rapidly try several and watch the logs to see what stopped the spam then to identify those and present them as an official patch | 16:07 |
| fungi | jamesmcarthur: yep, i get that. and the plugins/settings you wanted to try are what i was asking | 16:07 |
| jpmaxman | fungi: but the entire idea is that in the time it has taken to have this conversation we could have already tried several variations of that plugin setting | 16:08 |
| fungi | a couple of good suggestions (one free and one paid) have been mentioned in here now, and that's cool with me | 16:08 |
| jamesmcarthur | fungi: TBD once we were able to do it. There are a whole set of them. | 16:08 |
| jpmaxman | the last I checked the spam was coming in at such a rapid pace you'd know within minutes if what you did was effective | 16:08 |
| jamesmcarthur | Again, and I quote from clarkb: "Communicate to Infra that you're physically working on box. Communuication is more important than the technical steps." | 16:08 |
| jamesmcarthur | The point isn't outlining exactly which ones, just htat we were working on it. | 16:08 |
| fungi | jpmaxman: jamesmcarthur: i get that communicating with a community is tiresome for you, but please let's have conversations about what you want to do rather than "just do it" | 16:08 |
| jpmaxman | in the months we've been talking about it we could have written our own wiki :P | 16:09 |
| jamesmcarthur | fungi: that's a load of crap | 16:09 |
| anteaya | jamesmcarthur: please watch your language | 16:09 |
| jpmaxman | it's not tiresome, it's just talking in circles | 16:09 |
| *** ddieterly is now known as ddieterly[away] | 16:09 | |
| jpmaxman | because you're not actually trying anything | 16:09 |
| jamesmcarthur | fungi: we have been talking for 8 months | 16:09 |
| jpmaxman | so it's all conjecture and assumptions | 16:10 |
| jamesmcarthur | meanwhile we have no working wiki | 16:10 |
| *** yamamoto has joined #openstack-infra | 16:10 | |
| *** xyang1 has quit IRC | 16:10 | |
| jamesmcarthur | fungi: i sat down for 2 hours in Prague and we all agreed that the wiki is a mess. It's broken. Nobody can use it. | 16:10 |
| jamesmcarthur | fungi: if you want to tell me that us sitting here having a conversation with you is us not participating in the community, I think you should take a long look at your definition of community | 16:11 |
| fungi | i find the repeated assertions about "no working wiki for 8 months," "talking in circles" and "not actually trying anything" tiresome, but i'm willing to continue discussing | 16:11 |
| electrofelix | zxiiro: you happy with https://review.openstack.org/#/c/314900/ mutliple SCM fix? | 16:11 |
| *** zeih has quit IRC | 16:11 | |
| *** yamamoto_ has joined #openstack-infra | 16:11 | |
| *** yamamoto has quit IRC | 16:12 | |
| *** ddieterly[away] is now known as ddieterly | 16:12 | |
| mtreinish | jeblair, fungi: This failed the gate with a POST_FAILURE: https://review.openstack.org/#/c/328467/ I've not seen that before | 16:12 |
| mtreinish | I assume it's something related to all the recent zuul changes | 16:13 |
| jpmaxman | fungi: you're right - so as I said moments ago and months ago, if you'd like my help I'm happy to get into the server and figure out the various spam attack vectors and what does the best job of stopping them then propose a solution to the community. | 16:13 |
| *** xyang1 has joined #openstack-infra | 16:13 | |
| jpmaxman | if you don't, that's fine too, I got plenty to do :) | 16:13 |
| jpmaxman | jamesmcarthur had come back from Prague hopeful that we could all work together | 16:13 |
| ttx | jamesmcarthur: I think communicating about what we are trying is still in the spirit of the plan. I remember clarkb suggesting just dropping a line on IRC | 16:13 |
| jpmaxman | and had asked me to jump back in | 16:13 |
| jeblair | mtreinish: yeah, we're replacing UNSTABLE (cause 'what?') with that somewhat more specific error -- it means a publisher (like, log copying) failed. we're working today on making those logs more visible so you can self-diagnose the problem. let me look them up for you for that failure. | 16:14 |
| ttx | Also "the plan" was just a suggested way forward, still had to be discussed with the rest of the infra team | 16:14 |
| ttx | I personally think it's acceptable, but I'm not an active infra team member | 16:14 |
| *** yamahata has quit IRC | 16:14 | |
| fungi | jamesmcarthur: jpmaxman: yes, to get back to the root of the discussion, i'm in favor of turning new account creation back on if you are going to be tracking new accounts/edits for signs of spam and working on putting a better captcha in confirmedit, which will need ssh access i agree | 16:15 |
| *** abregman|afk is now known as abregman | 16:15 | |
| jpmaxman | cheers guys - I've got to turn my attention elsewhere. Let me know if you need anything. | 16:15 |
| *** mriedem has joined #openstack-infra | 16:15 | |
| ttx | I don't feel like requiring to drop a line on irc explaining what you're trying is an unreasonable requirement | 16:15 |
| *** yamamoto_ has quit IRC | 16:15 | |
| fungi | jamesmcarthur: jpmaxman: it's a good next step as far as i'm concerned | 16:16 |
| EmilienM | 50% packet loss on review.o.o | 16:16 |
| *** sergmelikyan has joined #openstack-infra | 16:16 | |
| EmilienM | just fyi | 16:16 |
| *** permalac has quit IRC | 16:16 | |
| jeblair | EmilienM: v4 or v6? | 16:16 |
| *** sergmelikyan has quit IRC | 16:16 | |
| EmilienM | v4 | 16:16 |
| fungi | jamesmcarthur: jpmaxman: since jeblair was the most outspoken about the original ssh access patch, i'd like to get him to weigh in so we can hopefully get on the same page there | 16:16 |
| *** Apoorva has joined #openstack-infra | 16:16 | |
| clarkb | EmilienM: I am not seeing loss from my location | 16:16 |
| *** kevinshort has quit IRC | 16:17 | |
| jeblair | clarkb, EmilienM: nor i | 16:17 |
| EmilienM | you guys in west coas iirc | 16:17 |
| *** zeih has joined #openstack-infra | 16:17 | |
| EmilienM | my remote is 104.130.246.91 | 16:17 |
| *** Apoorva has quit IRC | 16:17 | |
| EmilienM | fungi: you on east coast, can you try please? | 16:17 |
| *** bhavik has quit IRC | 16:17 | |
| *** yamahata has joined #openstack-infra | 16:18 | |
| fungi | jamesmcarthur: jpmaxman: i simply wanted to have the discussion in a venue where the team can participate. those sorts of decisions aren't made in a private discussion with two infra sysadmins at a table in prague | 16:18 |
| mtreinish | EmilienM: fwiw, I did a ping and got 46% loss with 15 packets sent | 16:18 |
| *** Apoorva has joined #openstack-infra | 16:18 | |
| *** mriedem1 has quit IRC | 16:18 | |
| EmilienM | paste.o.o -> same | 16:18 |
| clarkb | it sounds like people are mostly in violent agreement | 16:18 |
| fungi | EmilienM: ipv4 or ipv6? | 16:18 |
| EmilienM | v4 | 16:18 |
| *** vincentll has quit IRC | 16:18 | |
| mtreinish | heh, yeah I can't even get paste to load | 16:19 |
| fungi | EmilienM: so far no packet loss for me via ipv4 (broadband cable provider "charter communications" at my house) | 16:19 |
| clarkb | there are a set of things we can do to the wiki that will help the situation. Doing them likely requires root, there is a small number of people volunteering to do the work | 16:19 |
| EmilienM | pings: http://paste.openstack.org/show/pXLQkNkGqeXUS1MjgIzC/ | 16:19 |
| *** kevinshort has joined #openstack-infra | 16:19 | |
| *** ifarkas has quit IRC | 16:19 | |
| fungi | 50 packets transmitted, 50 received, 0% packet loss, time 49063ms, rtt min/avg/max/mdev = 55.669/58.229/62.906/1.432 ms | 16:19 |
| EmilienM | mhh ok | 16:19 |
| clarkb | EmilienM: try mtr | 16:19 |
| *** javeriak_ has joined #openstack-infra | 16:20 | |
| clarkb | EmilienM: you might find the hole more easily that way | 16:20 |
| fungi | yeah, this is probably a problem peering somewhere between providers i'm not traversing | 16:20 |
| *** javeriak has quit IRC | 16:20 | |
| jeblair | fungi: thank you. as someone who wasn't at the table or on the email thread, i appreciate that. however, it was pretty clear that in this conversation, suggestions deviating from what was decided at the table are not welcome, so i judged that my participation would not be helpful. | 16:20 |
| *** bpokorny has joined #openstack-infra | 16:21 | |
| AJaeger | scottda: did you use http://codesearch.openstack.org/ ? | 16:21 |
| EmilienM | clarkb: indeed | 16:21 |
| jeblair | fungi, jamesmcarthur, jpmaxman: i have no objections to adding jpmaxman's key to wiki. jpmaxman, could you propose the change to add your own key? | 16:22 |
| *** zeih has quit IRC | 16:22 | |
| fungi | jeblair: i also wasn't at that table, got briefed mostly via a private e-mail from jamesmcarthur, so was asking for more details on the plan | 16:22 |
| scottda | AJaeger: No, but thanks for the reminder! | 16:22 |
| mtreinish | EmilienM: I'm seeing a ~65% loss rate on: dls-b22-link.telia.net looking at mtr | 16:23 |
| EmilienM | clarkb: rackspace-ic-302090-dls-bb1.c.telia.net is loosing a lot of packets | 16:23 |
| EmilienM | yeah | 16:23 |
| zaro | morning | 16:23 |
| fungi | EmilienM: mtreinish: do you get consistent loss for hops after that one as well? | 16:23 |
| scottda | AJaeger: Unfortunately, it doesn't show up there: http://codesearch.openstack.org/?q=DEVSTACK_GATE_CEPH&i=nope&files=&repos= | 16:23 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/project-config: Remove gate-grenade-dsvm-partial-ironic https://review.openstack.org/329553 | 16:24 |
| mtreinish | fungi: yeah | 16:24 |
| jpmaxman | @jeblair it is proposed here patch# 285641 | 16:24 |
| mtreinish | it's a bit less in the percentages but still >50% | 16:24 |
| fungi | EmilienM: mtreinish: the name looks like the peering between telia and rackspace | 16:24 |
| jeblair | jpmaxman: that is mrmarton adding your key, which is somewhat unusual -- is it possible for you to propose adding your own key? | 16:24 |
| openstackgerrit | Evgeny Sikachev proposed openstack-infra/project-config: Migrated sahara-client-functional-job https://review.openstack.org/329497 | 16:25 |
| *** pradk has quit IRC | 16:26 | |
| fungi | EmilienM: mtreinish: my traceroute seems to go into rackspace via a peering between my internet service provider and rackspace in the dfw equinix facility, so i guess i'm lucky ;) | 16:26 |
| pleia2 | btw, I have the wiki upgrade spec drafted here: https://review.openstack.org/#/c/328455/ | 16:26 |
| jpmaxman | jeblair, he was just being helpful. If it makes a difference to you sure | 16:26 |
| pleia2 | only anteaya has looked at it so far, it needs more input | 16:26 |
| pabelanger | jpmaxman: looking at etherpad now | 16:26 |
| jpmaxman | link? | 16:26 |
| *** rwsu has joined #openstack-infra | 16:27 | |
| anteaya | morning zaro | 16:27 |
| *** gouthamr has joined #openstack-infra | 16:27 | |
| jeblair | jpmaxman, pabelanger: i think that might have been meant fro me | 16:27 |
| *** matt-borland has joined #openstack-infra | 16:27 | |
| fungi | jpmaxman: i too prefer the ssh key being added by the person claiming ownership of that key, from a standpoint of making sure we're adding the right key and tat the person proposing the change agrees about our access requirements | 16:27 |
| jpmaxman | "We are definitely okay with granting ssh access to production systems to folks with specialization in that area. There are several precedents. | 16:27 |
| jpmaxman | However, one of the things we consider most important is that operation of our systems is principally via manipulation of git repos with direct access and modification to be exceptions to the rules. It would be good to see some history of collaboration in git first." | 16:27 |
| pabelanger | jpmaxman: jeblair: yes, tabbing fail | 16:27 |
| EmilienM | fungi: we should all move to your island | 16:27 |
| EmilienM | fungi: do you have place? | 16:28 |
| jeblair | jpmaxman: yes, it seems we have decided to waive that on the promise of future collaboration. :) | 16:28 |
| jpmaxman | to address your concern - I don't have a lot of commits but have helped the foundation with their website and infrastructure for a number of years :) Anyway, I'm happy to provide my own patch | 16:28 |
| fungi | EmilienM: i can probably fit 8 guests comfortably. everyone else will have to sleep on the dock | 16:28 |
| *** daemontool has quit IRC | 16:28 | |
| jeblair | jpmaxman: thanks! :) | 16:28 |
| anteaya | jpmaxman: it is one small mechanism we use to ensure someone is acting with their own awareness and consent | 16:29 |
| *** daemontool has joined #openstack-infra | 16:29 | |
| anteaya | jpmaxman: with thousands of developers we don't always know who has given permission for what actions | 16:29 |
| AJaeger | scottda: if it does not show up, then I expect nobody sets it ... | 16:29 |
| *** njohnston|afk is now known as njohnston | 16:30 | |
| zxiiro | electrofelix: taking a look | 16:30 |
| *** Sukhdev has joined #openstack-infra | 16:31 | |
| jpmaxman | I'm also happy to not do anything - I only really jumped i here because tom fifield seemed frustrated and having enjoyed some frosty cold adult beverages with him thought I'd help | 16:31 |
| EmilienM | fungi: coming | 16:31 |
| jpmaxman | anteaya it's understandable, I hadn't met Jim until very briefly in Austin. Many of you I've enjoyed drinks with at many an openstack event :) | 16:32 |
| pleia2 | fungi: re: gerritbot, there is a patch up here to add some error handling, I'm not sure if it adds enough logging for us, but it may be good place to start WRT our channel joining mystery https://review.openstack.org/#/c/287056/ | 16:33 |
| anteaya | jpmaxman: oh sure, and it is always a pleasure to meet in person :) | 16:33 |
| anteaya | jpmaxman: one thing we have to be aware of in every action we take is that our decisons are available for evaulation by all our peers, including our peers who weren't there for drinks | 16:33 |
| *** johnny_ has joined #openstack-infra | 16:34 | |
| pleia2 | fungi: I think the standard chanlimit is 120 as you mention, but stopping at 240 seems strangely suspicious as a number that may be a "bigger chanlimit that we have an exception to get" | 16:34 |
| fungi | jpmaxman: i personally very much appreciate that you're willing to help us get this under control and more manageable, thanks! i mostly just want to make sure we don't sacrifice longer-term manageability in the process, which is part of why protracted discussions and more detailed planning become necessary | 16:34 |
| anteaya | jpmaxman: which is why things like clear descriptive commit messages may seem pedantic until you have had to find a commit from 6 months prior and the only commit message is remove foo, without any motivation for the change | 16:34 |
| *** gyee has joined #openstack-infra | 16:35 | |
| *** jlanoux has quit IRC | 16:36 | |
| *** yamamoto has joined #openstack-infra | 16:36 | |
| jpmaxman | @fungi of course. it is a unique situation because the production server is under active attack. So fixes have to be tested in production to know if they work | 16:36 |
| *** matt-borland_ has joined #openstack-infra | 16:36 | |
| pabelanger | fungi: pleia2: do you mind helping revert 329536. Our wheel-builds are currently failing | 16:37 |
| jeblair | mtreinish: rsync: send_files failed to open \"/home/jenkins/workspace/gate-puppet-openstack_health-puppet-beaker-rspec-ubuntu-trusty/logs/kern_log.txt\": Permission denied (13) | 16:37 |
| pabelanger | working on a patch, but talking longer then expected | 16:37 |
| jpmaxman | so sometimes you have to be able to suspend normal operating procedures in order to re-enforce your processes | 16:37 |
| anteaya | jpmaxman: as a note, you don't need to use @ in front of someone's name in irc, that is just for twitter (and perhaps hipchat?) | 16:37 |
| fungi | pleia2: if we can get some clearer debugging from gerribot to confirm that hypothesis, there are ways we can work around it (for example, since it's an informational bot we could shard by channel) | 16:37 |
| pleia2 | fungi: nods | 16:37 |
| pleia2 | fungi: care to look at the debugging patch? | 16:37 |
| anteaya | jpmaxman: well actually suspending normal operating procedures causes far more problems than it solves | 16:38 |
| mtreinish | jeblair: heh, I guess the kernel log is owner read only | 16:38 |
| pabelanger | jeblair: Yup, that is the rsync issue with misconfigured permissions for logs | 16:38 |
| *** matt-borland has quit IRC | 16:38 | |
| *** sergmelikyan has joined #openstack-infra | 16:38 | |
| sergmelikyan | jeblair, any chance you know if it's possible to use GitHub Pages with our repos given that they are mirrored to the GitHub? | 16:38 |
| anteaya | jpmaxman: with as many different services running as we have and as many different people in different timezones that we do, follow our process is often the only thing we can depend on | 16:38 |
| anteaya | jpmaxman: we always have some service in crisis of one form or another | 16:39 |
| mordred | sergmelikyan: I do not believe it is a thing we desire to support | 16:39 |
| *** sergmelikyan has quit IRC | 16:39 | |
| *** tongli has quit IRC | 16:39 | |
| mordred | sergmelikyan: we have the ability to publish web content to locations, what sort of content are you looking at? | 16:40 |
| fungi | pleia2: taking a look, thanks for the pointer | 16:40 |
| *** eezhova has quit IRC | 16:40 | |
| jeblair | pabelanger: i didn't really intend to have the conversation in the etherpad... | 16:41 |
| jeblair | pabelanger: so -- console-log last: i think it should be last so that when we use it to upload the ansible logs, we get as much as possible | 16:41 |
| jpmaxman | @anteaya sure - but you also have to be able to assign priority and set aside process to fix something that is urgent. It's kind of the age old adage that while we were following the process the village was pillaged. | 16:41 |
| pabelanger | jeblair: sure, removed | 16:42 |
| anteaya | jpmaxman: umm, it is been a while since I've worked on a pillage scenario so I might be out of my element on this one | 16:42 |
| pabelanger | jeblair: yes to last | 16:42 |
| jeblair | pabelanger: okay, what else we got? | 16:42 |
| anteaya | jpmaxman: in any case, it is nice to have your help here, and I'm willing to help you learn about our workflow if that is useful to you | 16:42 |
| *** daemontool has quit IRC | 16:43 | |
| *** tesseract has quit IRC | 16:43 | |
| jpmaxman | anteaya: :D | 16:43 |
| anteaya | :) | 16:43 |
| *** e0ne has joined #openstack-infra | 16:43 | |
| *** markusry has joined #openstack-infra | 16:43 | |
| jpmaxman | thank you! :) | 16:43 |
| anteaya | you're welcome | 16:44 |
| anteaya | jpmaxman: as we get to learn how to work with each other, please ask any question even if you think it is silly, it probably isn't | 16:44 |
| *** e0ne has quit IRC | 16:44 | |
| *** brucet has quit IRC | 16:44 | |
| anteaya | jpmaxman: and I'll do my best to keep an eye out for supportive suggestions, which you can accept or discard at your discresion | 16:45 |
| anteaya | also I'm a horrible speller | 16:45 |
| pabelanger | jeblair: to upload or in general? | 16:45 |
| *** sergmelikyan has joined #openstack-infra | 16:45 | |
| jeblair | pabelanger: was that the only issue with the plan in the etherpad? | 16:45 |
| fungi | pleia2: that's a great patch. i'm noticing we also need to update it for newer pbr and get rid of teh ancient <1.0 cap, but hopefully that can be done trivially in a separate change | 16:46 |
| sergmelikyan | mordred: sorry got disconnected, I was looking for hosting murano.io | 16:46 |
| pleia2 | fungi: yeah, glad I fished it out of oblivion while I was catching up on my queue the other week | 16:47 |
| pabelanger | jeblair: I think so. It should be a good start to get zuul-launcher logs into logs.o.o | 16:47 |
| anteaya | sergmelikyan: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-06-14.log.html#t2016-06-14T16:40:18 | 16:47 |
| *** tkelsey has quit IRC | 16:47 | |
| sergmelikyan | murano.io hostname is used in murano in the same way org.java.* is used in Java | 16:47 |
| jeblair | clarkb: do you have any feedback? | 16:47 |
| pleia2 | sergmelikyan: I think mordred was asking what kind of content (published docs, html pages, etc) you were looking to publish, not the domain :) | 16:48 |
| clarkb | jeblair: I don't know what is being discussed or what ehterpad is | 16:48 |
| jeblair | clarkb: https://etherpad.openstack.org/p/p25pzG6OSr | 16:48 |
| sergmelikyan | ah it's just 1 html page | 16:48 |
| mordred | pleia2: ++ | 16:48 |
| mordred | sergmelikyan: yah - so, publishing an html is definitely a thing that should be done via normal publishing mechanisms here | 16:49 |
| mordred | however, we've resisted per-project websites thusfar | 16:49 |
| fungi | pleia2: it would be more reviewable as about half a dozen separate changes, but i think we can probably merge it. i'd like to actually give it a try on review.o.o first and see if it gets us any better debugging info. can take it for a spin right now | 16:49 |
| *** salv-orlando has quit IRC | 16:49 | |
| mordred | so it is also important to determine if we'd like to have such a thing - I think it's a worthy question for the TC | 16:49 |
| *** salv-orlando has joined #openstack-infra | 16:50 | |
| mordred | which I know may sound like it's a lot of effort for one html page - but nothing in openstack is ever just one html page once you multiply by the number of projects ;) | 16:50 |
| sergmelikyan | Understand that :) | 16:50 |
| fungi | sergmelikyan: in fact, we've condensed a lot of old per-project vanity domains in recent years, turning them into deprecated redirects to official documentation sites | 16:50 |
| *** sputnik13 has joined #openstack-infra | 16:50 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Fix wheel-build macro issue https://review.openstack.org/329536 | 16:51 |
| fungi | e.g., http://nova.openstack.org/ and http://devstack.org/ | 16:51 |
| pabelanger | fungi: dhellmann: AJaeger: ^ updated the revert with the actual fix | 16:51 |
| clarkb | jeblair: pabelanger the set of steps on the etherpad seems reasonable to me | 16:51 |
| sergmelikyan | we are not looking to host documentation or anything significant there, we need some simple landing page | 16:51 |
| clarkb | basically copy all the things and fail gracefully but still record and note failure | 16:51 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Fix Multiple SCMs Plugin child XML format https://review.openstack.org/314900 | 16:51 |
| sergmelikyan | idea is that we use this domain in the app definition language, same way as in Java org.java.* used | 16:52 |
| *** Daisy has joined #openstack-infra | 16:52 | |
| sergmelikyan | and we don't want to loose domain | 16:52 |
| jeblair | pabelanger: do you want to update your changes to do that? | 16:52 |
| sergmelikyan | and it make sense to have some simple landing page there instead of 404 | 16:52 |
| pabelanger | jeblair: sure | 16:52 |
| dhellmann | pabelanger : thanks | 16:52 |
| fungi | sergmelikyan: oh, you're needing to register an application namespace? that's a somewhat different use case than typical vanity domain splash pages | 16:53 |
| *** hdaniel has quit IRC | 16:53 | |
| *** jordanP has quit IRC | 16:53 | |
| openstackgerrit | Khai Do proposed openstack-infra/system-config: run git gc instead of repack for git repos https://review.openstack.org/329566 | 16:53 |
| *** sshnaidm has joined #openstack-infra | 16:54 | |
| notmorgan | clarkb: i'll have the log capture thing respun shortly. thanks for the comments yesterday | 16:54 |
| sergmelikyan | fungi: exactly, right now (and previous 3 years) it was hosted on my private hosting and domain is controlled by Mirantis, we want to move that to the community | 16:54 |
| fungi | sergmelikyan: so why not appname.murano.openstack.org or something? | 16:54 |
| fungi | sergmelikyan: legacy pre-openstack namespace i guess? | 16:54 |
| sergmelikyan | fungi, too long, https://github.com/openstack/murano/blob/master/meta/io.murano/Classes/resources/Instance.yaml#L13 | 16:55 |
| sergmelikyan | example ^ | 16:55 |
| sergmelikyan | fungi: I would not say that it's exactly legacy, io.murano seems right and short name to use. Previously it was hosted this way because of exactly pre-openstack namespace rules | 16:56 |
| fungi | that doesn't seem to have stopped java and go communities from using insanely long package namespaces | 16:56 |
| *** Daisy has quit IRC | 16:56 | |
| sergmelikyan | fungi: that's why we need to have some short start like io.murano.* because people will end up with something like: io.murano.networking.neutron.loadbalancing | 16:57 |
| sergmelikyan | fungi: I am not looking for namespace for specific app | 16:57 |
| sergmelikyan | I am talking about base namespace for internal murano classes | 16:58 |
| sergmelikyan | used by other apps | 16:58 |
| fungi | sergmelikyan: so anyway, the foundation does continue to maintain domains used by projects that later officially joined openstack, so this seems like a pretty similar request | 16:59 |
| *** acabot has quit IRC | 16:59 | |
| fungi | e.g. devstack.org, refstack.org, trystack.org, et cetera | 16:59 |
| sergmelikyan | fungi: so should I add agenda item for TC meeting to approve moving murano.io to openstack-infra? | 17:00 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Capture Logs when Timeout is hit https://review.openstack.org/328516 | 17:00 |
| notmorgan | mordred, jeblair, pabelanger, clarkb: ^ | 17:01 |
| *** trown is now known as trown|lunch | 17:01 | |
| fungi | sergmelikyan: no, if you simply want the foundation to take over ongoing registration of the murano.io domain i _think_ we can just do that. if jbryce is around hopefully he can confirm that's reasonable given past precedent for similar domains | 17:02 |
| openstackgerrit | Nate Johnston proposed openstack-infra/system-config: Add #openstack-fwaas to accessbot channels https://review.openstack.org/329570 | 17:02 |
| openstackgerrit | Khai Do proposed openstack-infra/system-config: run git gc instead of repack for git repos https://review.openstack.org/329566 | 17:02 |
| fungi | sergmelikyan: that's more just an openstack foundation decision and not really infra team/tc | 17:03 |
| rcarrillocruz | ok | 17:03 |
| rcarrillocruz | pabelanger: | 17:03 |
| rcarrillocruz | clarkb: | 17:03 |
| rcarrillocruz | can you try now | 17:03 |
| rcarrillocruz | i think the issue is solved | 17:03 |
| sergmelikyan | fungi: thank you for the help! | 17:03 |
| *** acabot has joined #openstack-infra | 17:03 | |
| *** kushal has quit IRC | 17:04 | |
| fungi | sergmelikyan: as to whether there is web content which should be published for that url and whether it should be a redirect to a more standard openstack site, that's something worth discussing with us and the tc | 17:04 |
| sergmelikyan | fungi: I believe there is no specific requirements from the Murano team side regarding content. I think generic openstack site would be much better | 17:04 |
| fungi | but taking over the domain registration should be pretty straightforward | 17:05 |
| pabelanger | rcarrillocruz: sure | 17:05 |
| pabelanger | rcarrillocruz: ++ | 17:05 |
| pabelanger | works | 17:05 |
| sergmelikyan | Summary: I need confirmation from TC about taking over murano.io, and than discuss with TC what we should show on that domain. | 17:06 |
| *** PollyZ_ has quit IRC | 17:06 | |
| sergmelikyan | I will write e-mail to jbryce, is any better way to contact him? or should I use foundation@openstack.org | 17:07 |
| rcarrillocruz | k, let me know if there are further issues | 17:07 |
| openstackgerrit | Lucas Dutra Nunes proposed openstack-infra/jenkins-job-builder: SCM repo: add ignore-projects option https://review.openstack.org/293472 | 17:07 |
| fungi | sergmelikyan: i'm trying to raise him in some other places now | 17:07 |
| *** thorst_afk is now known as thorst | 17:07 | |
| rcarrillocruz | the gw was blocking outbound access as I expected | 17:07 |
| rcarrillocruz | btw, i asked the guy about the shutdown of the current datacenter and tells me it is expected to happen end of next month | 17:08 |
| rcarrillocruz | but anyway, i'll pass along info when i hear something official | 17:08 |
| clarkb | rcarrillocruz: did all hosts get fixed or just the one? | 17:08 |
| jbryce | sergmelikyan: jonathan@openstack.org | 17:09 |
| *** sergmelikyan has quit IRC | 17:10 | |
| *** sergmelikyan has joined #openstack-infra | 17:10 | |
| *** rossella_s has quit IRC | 17:11 | |
| rcarrillocruz | the fix has been put on the vlan | 17:11 |
| *** PollyZ has joined #openstack-infra | 17:11 | |
| *** e0ne has joined #openstack-infra | 17:12 | |
| *** ramishra has joined #openstack-infra | 17:12 | |
| *** ramishra has quit IRC | 17:12 | |
| *** ramishra has joined #openstack-infra | 17:13 | |
| *** ramishra has quit IRC | 17:13 | |
| *** markus__ has joined #openstack-infra | 17:15 | |
| *** ddieterly is now known as ddieterly[away] | 17:15 | |
| jeblair | pabelanger: aside from logs, what things from ansible are we interested in uploading? just the playbooks? or also the inventory and config files? | 17:15 |
| jeblair | pabelanger: and what about the scripts? (the shell snippets that we upload and run with zuul_runner) | 17:16 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: Remove unused plugins dir https://review.openstack.org/329581 | 17:16 |
| *** PollyZ has quit IRC | 17:16 | |
| rcarrillocruz | but i can't check on other hosts, unless i reboot them via ilo and get into recovery mode. The other hosts have old IP configs on their NICs, thus are not reachable | 17:16 |
| *** syjulian has joined #openstack-infra | 17:16 | |
| *** markusry has quit IRC | 17:16 | |
| clarkb | rcarrillocruz: at least we know what the problem is if we run into it again | 17:17 |
| * rcarrillocruz goes back to grunt work on checking statuses of different Jira issues | 17:17 | |
| *** pradk has joined #openstack-infra | 17:17 | |
| *** ramishra has joined #openstack-infra | 17:17 | |
| rcarrillocruz | clarkb: yup, cos the NET guy i've been working to sort this out was like 'omg, you are really really really opening these machines to the outside bad Internet' | 17:17 |
| rcarrillocruz | so if it happens again , i can point to a ticket | 17:18 |
| clarkb | jlk: you around? I remember yo umentioned we would need to rebuild out the bluebox cloud config, do you know if the cloud credentials changed too? wondering how some of us can get started on that work | 17:18 |
| *** qb_ has quit IRC | 17:18 | |
| rcarrillocruz | i hope there's no security guy capping that at some point in future | 17:18 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Upload ansible playbook directory to logs server https://review.openstack.org/325602 | 17:19 |
| *** ramishra has quit IRC | 17:19 | |
| pabelanger | jeblair: clarkb: ^ Want to confirm that is what you expected? | 17:19 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Upload ansible playbook directory to logs server https://review.openstack.org/325602 | 17:20 |
| *** dtantsur is now known as dtantsur|afk | 17:21 | |
| pabelanger | jeblair: we still need to add the ignore_failure logic | 17:21 |
| *** qb_ has joined #openstack-infra | 17:21 | |
| jeblair | pabelanger: let's do that in its own change | 17:21 |
| jeblair | pabelanger: you'r chatting in the etherpad again | 17:22 |
| *** alex_xu has quit IRC | 17:22 | |
| pabelanger | jeblair: I think we need to change line 15 in etherpad, since it contains plain text passords | 17:23 |
| jlk | clarkb: the credentials might have changed, true. I think mordred has access to the Blue Box support system where credentials are stored. | 17:23 |
| jeblair | pabelanger: i had some questions in here about what you want to upload | 17:23 |
| jeblair | pabelanger: aside from logs, what things from ansible are we interested in uploading? just the playbooks? or also the inventory and config files? | 17:23 |
| jeblair | pabelanger: and what about the scripts? (the shell snippets that we upload and run with zuul_runner) | 17:23 |
| jlk | It'd be reduced down to just cloud_admin user account to start with. | 17:23 |
| clarkb | mordred: ^ maybe you want to bootstrap things for us? | 17:23 |
| pabelanger | jeblair: at first glance, if we do the whole ansible folder, however I have some concerns about the script sub directory | 17:24 |
| jeblair | pabelanger: so that's *yes* to inventory and ansible.cfg files? | 17:25 |
| *** alex_xu has joined #openstack-infra | 17:25 | |
| pabelanger | jeblair: yes, inverentory, both playbooks | 17:25 |
| pabelanger | I am not sure what is in plugins currently | 17:25 |
| jeblair | pabelanger: it's empty, i just pushed a change to delete it | 17:26 |
| pabelanger | ack | 17:26 |
| *** salv-orlando has quit IRC | 17:26 | |
| jeblair | pabelanger: okay, what credential information could end up it scripts? | 17:26 |
| *** yamamoto has quit IRC | 17:26 | |
| *** salv-orlando has joined #openstack-infra | 17:26 | |
| jeblair | pabelanger: scripts just holds the shell snippets from jjb | 17:26 |
| pabelanger | jeblair: don't we store our FTP credentials there? | 17:26 |
| jeblair | pabelanger: no, we write those into the ftpdir that we create when we create an ftp task | 17:26 |
| pabelanger | okay | 17:26 |
| jeblair | pabelanger: that *is* under ansible_root currently | 17:26 |
| jeblair | pabelanger: so based on your earlier answers, i think we can do the following: | 17:27 |
| pabelanger | Ya, my patch above uploads the whole directory | 17:27 |
| jeblair | pabelanger: move logs under ansible/, and move ftp/scp staging directories *out* of ansible/ | 17:27 |
| jeblair | pabelanger: then we can upload the whole ansible dir | 17:27 |
| jeblair | pabelanger: so it will be like the 'future layout' section at the bottom of the etherpad | 17:28 |
| *** tphummel has joined #openstack-infra | 17:28 | |
| *** hongbin has quit IRC | 17:29 | |
| pabelanger | jeblair: yes, I like that | 17:29 |
| *** salv-orl_ has joined #openstack-infra | 17:29 | |
| jeblair | pabelanger: 602 wasn't really what i had in mind -- that will make the log upload task be the first task on the first scp publisher, which may or may not be the correct target... i think i see how this should all fit together now; do you mind if i push up another revision of that patch? | 17:30 |
| pabelanger | jeblair: please do | 17:30 |
| jeblair | pabelanger: cool, i'll have something in a few minutes i think | 17:30 |
| openstackgerrit | Luigi Toscano proposed openstack-infra/project-config: Enable Tempest test job also for sahara-tests https://review.openstack.org/329590 | 17:30 |
| notmorgan | jlk: yay cloud_admin pattern ;) | 17:31 |
| jlk | well | 17:31 |
| jlk | not the way you'd like to see it | 17:31 |
| jlk | sadly I think we were using the "cloud_admin" thing without looking at what Keystone was doing :( | 17:31 |
| jlk | for us "cloud_admin" means "the customer account who can do (some) admin things on this cloud". Not all admin things though, that's left to just the "admin" account which we don't hand out. | 17:32 |
| *** salv-orlando has quit IRC | 17:32 | |
| *** tqtran has joined #openstack-infra | 17:32 | |
| pabelanger | jeblair: when we abort a job, exit code -9, should be continue onto post_playbook to upload logs files? | 17:36 |
| *** mriedem1 has joined #openstack-infra | 17:38 | |
| *** PollyZ has joined #openstack-infra | 17:39 | |
| *** zz_ja is now known as zz_zz_ja | 17:39 | |
| jeblair | pabelanger: a zuul abort should probably not do that. a timeout should. | 17:40 |
| *** mriedem has quit IRC | 17:40 | |
| *** eezhova has joined #openstack-infra | 17:40 | |
| pabelanger | jeblair: I agree, let me get a patch | 17:40 |
| *** e0ne has quit IRC | 17:41 | |
| *** tonytan4ever has joined #openstack-infra | 17:41 | |
| *** electrofelix has quit IRC | 17:41 | |
| pabelanger | rcarrillocruz: incoming PM | 17:41 |
| *** dizquierdo has quit IRC | 17:41 | |
| *** mriedem1 is now known as mriedem | 17:42 | |
| *** sambetts is now known as sambetts|afk | 17:42 | |
| rcarrillocruz | k | 17:42 |
| *** _nadya_ has joined #openstack-infra | 17:42 | |
| openstackgerrit | Raildo Mascena proposed openstack-infra/project-config: Run Ceilometer functional tests in Identity v3-only https://review.openstack.org/315077 | 17:42 |
| *** roxanagh_ has joined #openstack-infra | 17:43 | |
| *** PollyZ has quit IRC | 17:43 | |
| *** tkelsey has joined #openstack-infra | 17:43 | |
| *** asselin_ has quit IRC | 17:44 | |
| *** ociuhandu has quit IRC | 17:45 | |
| clarkb | jeblair: pabelanger fwiw jenkins aborts did result in logs but that was probable less desireable. | 17:45 |
| clarkb | since an abort indicates an invalid state we shouldn't care about storing and indexing that info | 17:45 |
| *** ihrachys has quit IRC | 17:46 | |
| AJaeger | pabelanger fixed the wheel build with https://review.openstack.org/#/c/329536/2 - I'd like to have another project-config review on this one to merge quickly since wheel build is currently broken. | 17:47 |
| *** roxanagh_ has quit IRC | 17:47 | |
| *** tkelsey has quit IRC | 17:47 | |
| *** jesusaur has quit IRC | 17:48 | |
| openstackgerrit | John L. Villalovos proposed openstack-infra/devstack-gate: Remove 'partial' support for Grenade jobs https://review.openstack.org/329532 | 17:48 |
| *** jesusaur has joined #openstack-infra | 17:49 | |
| *** pvaneck has joined #openstack-infra | 17:49 | |
| jeblair | clarkb: ++ | 17:50 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Add compute004.uswest metadata for better tracking https://review.openstack.org/329602 | 17:50 |
| *** mtanino has quit IRC | 17:51 | |
| *** bpokorny has quit IRC | 17:51 | |
| *** tosky has quit IRC | 17:54 | |
| *** mhickey has joined #openstack-infra | 17:54 | |
| pabelanger | clarkb: jeblair: Ya, not sure it is a bad thing. If we want to make the improvement, I can submit the patch | 17:57 |
| *** Rockyg has joined #openstack-infra | 17:58 | |
| *** ajmiller_ has joined #openstack-infra | 17:58 | |
| *** ajmiller has quit IRC | 18:01 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: Remove unused plugins dir https://review.openstack.org/329581 | 18:01 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Upload ansible playbook directory to logs server https://review.openstack.org/325602 | 18:01 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Add log_path to ansible.cfg https://review.openstack.org/329525 | 18:01 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: move ftp/scp staging out of ansible dir https://review.openstack.org/329607 | 18:01 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: ensure that console/ansible publishing is last https://review.openstack.org/329608 | 18:01 |
| jeblair | pabelanger: yeah, i think we should improve it | 18:02 |
| *** sergmelikyan has quit IRC | 18:02 | |
| jeblair | pabelanger, clarkb, Shrews: ^ that's everything except the ignore_errors change. it's all untested, but wanted to push that up for early feedback. i'm going to write the ignore errors change, then test the whole stack. | 18:02 |
| pabelanger | jeblair: looking | 18:03 |
| *** nighttime has quit IRC | 18:04 | |
| *** kushal has joined #openstack-infra | 18:05 | |
| *** robot has joined #openstack-infra | 18:05 | |
| *** bpokorny has joined #openstack-infra | 18:05 | |
| *** bhavik has joined #openstack-infra | 18:06 | |
| *** jamesmcarthur has quit IRC | 18:07 | |
| *** _nadya_ has quit IRC | 18:07 | |
| *** ddieterly[away] is now known as ddieterly | 18:08 | |
| *** mtanino has joined #openstack-infra | 18:08 | |
| *** dmk0202 has joined #openstack-infra | 18:09 | |
| Shrews | jeblair: if EVERY task in the post-playbook will have ignore_errors=true, then that block:always: wouldn't be necessary | 18:09 |
| Shrews | jeblair: just the final task to check rc | 18:09 |
| *** acoles is now known as acoles_ | 18:10 | |
| *** salv-orl_ has quit IRC | 18:10 | |
| jeblair | Shrews: so skip the block construct, just have a list of tasks, all with ignore_errors, then at the end, one final task to check the registered rcs? | 18:11 |
| *** salv-orlando has joined #openstack-infra | 18:11 | |
| clarkb | jeblair: I notice that zl03 has a much larger number of used nodes than the other two zls, is that going o largely be driven by gearman response times? | 18:11 |
| Shrews | jeblair: aye | 18:11 |
| *** trown|lunch is now known as trown | 18:11 | |
| *** salv-orlando has quit IRC | 18:11 | |
| *** salv-orlando has joined #openstack-infra | 18:12 | |
| clarkb | zl03 has roghly 2x the nodes as 01 and 02 combined | 18:12 |
| jeblair | clarkb: yes, it's the usual imperfect method of gearman load distribution (if the server is busy, it's slower, right?) | 18:12 |
| mordred | clarkb, jlk: I hope to get the bluebox cloud re-bootstrapped tomorrow or thursday | 18:12 |
| pabelanger | rcarrillocruz: crinkle: clarkb: ansible is now connected to baremetal00.vanilla.ic.openstack.org. puppet currently failing with exit code 6 | 18:13 |
| jeblair | clarkb: maybe we should do "sleep(number_of_nodes)" before grabbing a job :) | 18:13 |
| pabelanger | mordred: excellent | 18:13 |
| pabelanger | clarkb: I've noticed that too | 18:13 |
| jeblair | clarkb, pabelanger: before we jump to that though, we should see whether it's a problem | 18:14 |
| jeblair | like, are we hitting resource limits | 18:14 |
| clarkb | ya I don't actualy know if it is a problem | 18:14 |
| *** eezhova has quit IRC | 18:14 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: ensure every publisher runs https://review.openstack.org/329612 | 18:14 |
| crinkle | pabelanger: WARNING: The following packages cannot be authenticated! libevent-2.0-5 libunbound2 unbound-anchor unbound | 18:14 |
| pabelanger | what about shutting down jenkins03 to see how zuul-launcher handle more nodes in the pool? | 18:15 |
| rcarrillocruz | pabelanger: sweet | 18:15 |
| pabelanger | crinkle: Hmm | 18:15 |
| *** Sukhdev has quit IRC | 18:15 | |
| rcarrillocruz | opening jiras for different issues i'm encointering in rack12 servers | 18:15 |
| jeblair | pabelanger: i think we should wait until we publish ansible logs so that people can diagnose POST_FAILURE conditions before we do that | 18:15 |
| jeblair | clarkb, Shrews, pabelanger: ignore_errors change: https://review.openstack.org/329612 | 18:15 |
| pabelanger | jeblair: agreed. | 18:16 |
| jeblair | oops, one sec on 612 | 18:16 |
| pabelanger | really looking forward to an ansible-playbook test fixture in zuul! Will make parsing dynamic playbooks way easier! | 18:17 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: ensure every publisher runs https://review.openstack.org/329612 | 18:17 |
| jeblair | pabelanger: i don't understand the last thing you just said | 18:17 |
| Rockyg | mordred, you around? What's the status of phabricator? I'm in nova-bugs meeting and enquiring minds want to know | 18:19 |
| *** sarob has joined #openstack-infra | 18:19 | |
| Rockyg | mordred, sorry, meeting-4 | 18:20 |
| *** sarob has quit IRC | 18:20 | |
| Shrews | jeblair: lgtm. i'm excited to see if it works! :) | 18:20 |
| mordred | Rockyg: https://review.openstack.org/#/c/314185/ | 18:20 |
| clarkb | Rockyg: I believe there was a recent ml update. Also fungi would like you to know that your lp account was possibly compromised and you should address that | 18:20 |
| clarkb | Rockyg: it was part of the wiki spam attack | 18:20 |
| clarkb | aha mordred has the link | 18:20 |
| *** PollyZ has joined #openstack-infra | 18:21 | |
| *** jamesmcarthur has joined #openstack-infra | 18:22 | |
| pabelanger | jeblair: nothing major, hard to parse our generated playbooks from python code. Was hoping one day we'd do something like JJB / grafyaml to compare the dynamic playbook to a know fixture. | 18:22 |
| *** eezhova has joined #openstack-infra | 18:23 | |
| *** mhickey has quit IRC | 18:23 | |
| *** mhickey has joined #openstack-infra | 18:24 | |
| openstackgerrit | Paul Carver proposed openstack-infra/irc-meetings: Correction to meeting ID for Common Flow Classifier meeting https://review.openstack.org/329616 | 18:24 |
| pabelanger | jeblair: +2 for stack | 18:26 |
| Rockyg | clarkb: dang! or maybe it was that I had found more spam pages and couldn't delete them so I moved them somewhere less obtrusive/irritating? | 18:27 |
| Rockyg | clarkb, and thanks. I guess I go and change my pw | 18:28 |
| Rockyg | Is tha all I need to do? | 18:28 |
| anteaya | Rockyg: did you move spam pages? | 18:28 |
| clarkb | Rockyg: fungi will have more info, I think he found spam pages iwth your name on them | 18:28 |
| anteaya | spam pages showed up in your account on June 8th | 18:28 |
| fungi | Rockyg: there were 5 pages in total which claimed to have been created by the Rocky account on the wiki | 18:28 |
| fungi | i'll link one in a sec | 18:28 |
| Rockyg | Yeah. They were in the middle of stuff some user groups would search so I moved them. Sounds like the ones I moved. I would have loved to have blown them away, but couldn | 18:29 |
| Rockyg | 't | 18:29 |
| anteaya | Rockyg: why couldn't you? | 18:29 |
| Rockyg | For the future, what is the best wway to address? Didn't have permission to delete. | 18:29 |
| *** baoli_ has quit IRC | 18:29 | |
| *** dmk0202 has quit IRC | 18:30 | |
| fungi | Rockyg: https://wiki.openstack.org/w/index.php?title=JJJJ@_1866-532-0850_Turbotax_Customer_care_Technical_Support_Help_Phone_Number&action=edit&redlink=1 was one | 18:30 |
| *** hasharOut has quit IRC | 18:30 | |
| *** harlowja has quit IRC | 18:30 | |
| Rockyg | Which is actually a good thing for most peopl.e | 18:30 |
| *** esikachev has quit IRC | 18:30 | |
| fungi | it does say something like "Rockyg moved page [spammy title] to Obsolete:[spammy title] (spam)" so that matches what you're saying | 18:31 |
| fungi | i think that means your account is not compromised ;) | 18:31 |
| fungi | i'll unblock it immediately | 18:31 |
| anteaya | which is a relief | 18:31 |
| Rockyg | Yup. I could move the page to obsolete, so I did the top page of the ones that turned up in the search. | 18:31 |
| fungi | Rockyg: i've unblocked it now. if you want to help patrol the wiki for spam and delete old material, i'm happy to add you to the wiki admins group there | 18:32 |
| *** harlowja has joined #openstack-infra | 18:32 | |
| Rockyg | Thanks you! And sorry for the scare. I hate those f*&% spammers for grafittiing all over our wiki | 18:32 |
| mtreinish | jeblair: hmm, it failed on the same thing running on jenkins: http://logs.openstack.org/67/328467/1/check/gate-puppet-openstack_health-puppet-beaker-rspec-ubuntu-trusty/6da3f67/console.html#_2016-06-14_17_23_00_360 | 18:33 |
| Rockyg | fungi, sure. Just need a delete page option, and a process so I don't scare the infra team when I do it. Like a tag, or whatever. | 18:34 |
| mordred | Rockyg: you will never scare us | 18:35 |
| *** e0ne has joined #openstack-infra | 18:35 | |
| fungi | Rockyg: deletion is reversible | 18:35 |
| mordred | Rockyg: well, I mean, except when you do | 18:35 |
| pabelanger | mtreinish: that is going to be a bug with EmilienM and copy_logs.sh. I think we need to add logic to chown / chmod the logs folder to jenkins:jenkins | 18:35 |
| Rockyg | Bwa ha ha ha ha! | 18:35 |
| anteaya | Rockyg: my take on it is that there are a whole lot of people with not enough money to live on, so this is what they can do for pay | 18:35 |
| mtreinish | pabelanger: ah, ok | 18:35 |
| anteaya | Rockyg: I don't fault the people, trying to earn a living, I fault a system that rewards them for doing this rather than useful work | 18:36 |
| *** xenogear has joined #openstack-infra | 18:36 | |
| Rockyg | anteaya, yup. Sad. Especially since they seem to have at least rudimentory (or if they are writing the scripts, good) computer skills. | 18:37 |
| pabelanger | crinkle: clarkb: rcarrillocruz: puppet looks to have run correctly this time on baremetal00.vanilla | 18:37 |
| rcarrillocruz | \o/ | 18:37 |
| fungi | Rockyg: your wiki account is unblocked, and you're now in the administrator and bureaucrat groups in the wiki like our other caretaker volunteers. thanks for helping! | 18:38 |
| anteaya | Rockyg: agreed, they could be doing so much more, if only they could feed themselves when doing it | 18:42 |
| *** hongbin has joined #openstack-infra | 18:45 | |
| *** salv-orlando has quit IRC | 18:45 | |
| Rockyg | Cool. Thanks! | 18:45 |
| openstackgerrit | greghaynes proposed openstack-infra/jeepyb: Allow setting a default org in create_cgitrepos https://review.openstack.org/329626 | 18:46 |
| *** sdague has quit IRC | 18:46 | |
| *** salv-orlando has joined #openstack-infra | 18:46 | |
| *** lifeless has quit IRC | 18:46 | |
| Rockyg | I will wield my new power conservatively in the pursit of community. As opposed to like a bureaucrat. | 18:46 |
| *** lifeless has joined #openstack-infra | 18:47 | |
| *** qb_ has quit IRC | 18:48 | |
| *** qb_ has joined #openstack-infra | 18:49 | |
| Shrews | jeblair: processed your stack | 18:49 |
| *** sarob has joined #openstack-infra | 18:50 | |
| *** e0ne has quit IRC | 18:53 | |
| *** mrmartin has joined #openstack-infra | 18:53 | |
| *** syjulian has quit IRC | 18:54 | |
| *** syjulian has joined #openstack-infra | 18:54 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Don't run post_playbook if ansible-playbook is aborted https://review.openstack.org/329631 | 18:55 |
| *** ddieterly is now known as ddieterly[away] | 18:55 | |
| pabelanger | jeblair: clarkb: I believe that will stop our post_playbook from running on abort.' | 18:55 |
| *** eezhova has quit IRC | 18:56 | |
| syjulian | hi guys, I'm trying to update a wiki page to grab some tasks, but for some reason I'm getting 500 errors when I log in with my Ubuntu One account. Anybody know what to do? | 18:56 |
| *** eezhova has joined #openstack-infra | 18:56 | |
| anteaya | I'm not a guy | 18:56 |
| pabelanger | syjulian: this is likely because you haven't logged into the wiki before | 18:56 |
| anteaya | and we had to disable new account creation on the wiki due to spamming | 18:57 |
| pabelanger | syjulian: if you give me a minute I can allow you to register for a 15mins window | 18:57 |
| *** amoralej is now known as amoralej|off | 18:57 | |
| syjulian | yes that would be great | 18:57 |
| pabelanger | syjulian: please try again | 18:57 |
| *** sarob has quit IRC | 18:58 | |
| syjulian | pabelanger: it works. thank you! | 18:59 |
| pabelanger | syjulian: np | 18:59 |
| *** eezhova has quit IRC | 18:59 | |
| pleia2 | meeting time | 19:00 |
| *** eezhova has joined #openstack-infra | 19:01 | |
| openstackgerrit | Merged openstack-infra/infra-specs: Mark 'Use Diskimage Builder in Nodepool' finished https://review.openstack.org/329080 | 19:01 |
| pabelanger | ^ \o/ | 19:01 |
| *** PollyZ_ has joined #openstack-infra | 19:02 | |
| *** PollyZ has quit IRC | 19:05 | |
| *** dmk0202 has joined #openstack-infra | 19:05 | |
| *** bpokorny has quit IRC | 19:05 | |
| *** sdake has quit IRC | 19:06 | |
| *** sdake has joined #openstack-infra | 19:07 | |
| *** PollyZ has joined #openstack-infra | 19:07 | |
| *** PollyZ_ has quit IRC | 19:07 | |
| *** eezhova has quit IRC | 19:08 | |
| *** eezhova has joined #openstack-infra | 19:08 | |
| *** bpokorny has joined #openstack-infra | 19:09 | |
| *** jcoufal_ has quit IRC | 19:10 | |
| *** PollyZ has quit IRC | 19:14 | |
| openstackgerrit | Jp Maxwell proposed openstack-infra/system-config: Adding ssh key for JP Maxwell to wiki.o.o https://review.openstack.org/329638 | 19:15 |
| *** xarses_ has quit IRC | 19:16 | |
| rcarrillocruz | clarkb: re: syslog on puppetmaster being interleaved, yup, i'll log to /var/log/ansible-puppet/<hostname> , being /var/log/ansible-puppet a role default, so it can tweaked | 19:16 |
| *** markus__ has quit IRC | 19:17 | |
| *** ddieterly[away] is now known as ddieterly | 19:17 | |
| nibalizer | pabelanger: awesome | 19:19 |
| nibalizer | re: puppet running correctly on baremetal00 | 19:19 |
| nibalizer | exciting to have a cloud again | 19:19 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: translations: Fix error abort https://review.openstack.org/329642 | 19:19 |
| *** PollyZ has joined #openstack-infra | 19:20 | |
| *** mrmartin has quit IRC | 19:20 | |
| *** kzaitsev_mb has quit IRC | 19:22 | |
| *** tonytan4ever has quit IRC | 19:25 | |
| *** dtardivel has quit IRC | 19:27 | |
| *** eezhova has quit IRC | 19:27 | |
| *** ociuhandu has joined #openstack-infra | 19:29 | |
| *** baoli has joined #openstack-infra | 19:31 | |
| *** mriedem has quit IRC | 19:31 | |
| *** sarob has joined #openstack-infra | 19:32 | |
| *** mriedem has joined #openstack-infra | 19:34 | |
| *** ayoung_ has joined #openstack-infra | 19:34 | |
| *** tkelsey has joined #openstack-infra | 19:35 | |
| *** gyee has quit IRC | 19:36 | |
| *** mrmartin has joined #openstack-infra | 19:37 | |
| *** eezhova has joined #openstack-infra | 19:39 | |
| *** tkelsey has quit IRC | 19:40 | |
| *** weshay is now known as weshay_mtg | 19:42 | |
| *** Sukhdev has joined #openstack-infra | 19:42 | |
| *** rfolco has quit IRC | 19:43 | |
| *** salv-orlando has quit IRC | 19:43 | |
| *** ajmiller_ has quit IRC | 19:44 | |
| *** salv-orlando has joined #openstack-infra | 19:44 | |
| *** eezhova has quit IRC | 19:44 | |
| *** dimtruck is now known as zz_dimtruck | 19:44 | |
| *** baoli has quit IRC | 19:44 | |
| *** roxanagh_ has joined #openstack-infra | 19:44 | |
| *** sarob has quit IRC | 19:45 | |
| *** baoli has joined #openstack-infra | 19:45 | |
| *** harlowja_ has joined #openstack-infra | 19:45 | |
| *** sarob has joined #openstack-infra | 19:45 | |
| *** derekh has quit IRC | 19:46 | |
| *** burgerk has quit IRC | 19:46 | |
| *** roxanagh_ has quit IRC | 19:49 | |
| *** harlowja has quit IRC | 19:49 | |
| *** jsavak has joined #openstack-infra | 19:50 | |
| *** cody-somerville has joined #openstack-infra | 19:50 | |
| *** cody-somerville has quit IRC | 19:50 | |
| *** cody-somerville has joined #openstack-infra | 19:50 | |
| *** rfolco has joined #openstack-infra | 19:50 | |
| jpmaxman | jeblair I put the patch here - copied what Marton did. https://review.openstack.org/329638 | 19:51 |
| jeblair | jpmaxman: thanks! | 19:52 |
| jpmaxman | it looks like Jenkins may not have liked it though in a couple of instances :( | 19:52 |
| *** ffrank has joined #openstack-infra | 19:52 | |
| ttx | jpmaxman: because in those tense situations we just get lucky :) | 19:52 |
| ttx | "Could not find puppet-4.5.2 in any of the sources" | 19:53 |
| jeblair | jpmaxman: yeah, a quick look at that suggests it's unrelated to your patch... we can probably track it down in a few minutes when the infra meeting wraps up | 19:53 |
| jeblair | ttx: yeah, that | 19:53 |
| pabelanger | o.0 | 19:54 |
| pabelanger | Right, we use the puppet modules beaker job, which doesn't use the OS version of puppet | 19:54 |
| pabelanger | it uninstalls it and them gem installs it | 19:54 |
| jeblair | 4.5.2 was released today, so i wonder if this is rubygems.org being inconsistent | 19:55 |
| jpmaxman | ok cool - it's weird because it seemed pretty vanilla. anyway let me know - happy to adjust | 19:55 |
| *** sdague has joined #openstack-infra | 19:56 | |
| *** pilgrimstack has joined #openstack-infra | 19:56 | |
| jeblair | jpmaxman: probably just bad timing; we can re-run the tests when we think rubygems.org is fixed | 19:57 |
| nibalizer | i didn't realize we'd moved the beaker jobs to 4.x | 19:57 |
| jeblair | Total downloads 6,423,373 | 19:57 |
| jeblair | For this version 24 | 19:57 |
| nibalizer | and yea... that is super strange | 19:57 |
| *** ffrank has quit IRC | 19:58 | |
| jeblair | that's for version 4.5.2 | 19:58 |
| jeblair | so we might have tried to pull it literally the moment it went up | 19:58 |
| *** e0ne has joined #openstack-infra | 19:58 | |
| *** rwsu_ has joined #openstack-infra | 19:58 | |
| jeblair | i'm just going to recheck now and see if it's already cleared | 19:58 |
| *** burgerk has joined #openstack-infra | 19:58 | |
| openstackgerrit | Jim Rollenhagen proposed openstack-infra/project-config: Fix grenade exclusion for ironic https://review.openstack.org/329658 | 19:58 |
| jroll | easy regex fix to save a few nodes ^ | 19:58 |
| nibalizer | jroll: ++ | 19:59 |
| jroll | :) | 19:59 |
| anteaya | jpmaxman: if you feel like attending infra weekly meetings, you may find them useful | 20:00 |
| anteaya | jpmaxman: this week's just ended | 20:00 |
| anteaya | jpmaxman: they take place in #openstack-meeting on Tuesdays at 19:00 utc | 20:00 |
| *** rwsu has quit IRC | 20:00 | |
| *** jsavak has quit IRC | 20:03 | |
| jpmaxman | cool I put it in my calendar | 20:03 |
| *** eezhova has joined #openstack-infra | 20:03 | |
| jpmaxman | no wonder it was so quiet around here | 20:03 |
| *** sarob has quit IRC | 20:04 | |
| anteaya | jpmaxman: great, it helps to get a feel for our flow | 20:04 |
| fungi | yeah, we were all focused in #openstack-meeting | 20:04 |
| *** rwsu_ is now known as rwsu | 20:04 | |
| anteaya | jpmaxman: you never know when you learn something useful in a meeting | 20:04 |
| fungi | we also tend to get pretty quiet for the hour after because we're either focused on the tc meeting or getting drunk. could be both | 20:04 |
| jpmaxman | fungi +1 | 20:04 |
| *** jsavak has joined #openstack-infra | 20:05 | |
| *** yamahata has quit IRC | 20:05 | |
| *** zz_dimtruck is now known as dimtruck | 20:06 | |
| *** ajmiller has joined #openstack-infra | 20:06 | |
| *** Goneri has quit IRC | 20:06 | |
| *** mrmartin has quit IRC | 20:09 | |
| *** flwang has quit IRC | 20:10 | |
| *** eezhova has quit IRC | 20:10 | |
| pabelanger | clarkb: going back to the boot-timeout for osic-cloud1, did we want to try upping it to 240 (from 120)? | 20:10 |
| clarkb | pabelanger: might be worthwhile to at least confirm it isn't something silly like slow boots occasionally happening | 20:11 |
| *** syjulian has left #openstack-infra | 20:12 | |
| *** pilgrimstack has quit IRC | 20:12 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Double our boot-timeout for osic-cloud1 https://review.openstack.org/329662 | 20:13 |
| pabelanger | clarkb: ^ | 20:13 |
| pleia2 | off to lunch for a bit | 20:13 |
| *** PollyZ has quit IRC | 20:14 | |
| *** kgiusti has quit IRC | 20:15 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: toci_gate_test: disable ipv6 on upgrade job for stable/liberty https://review.openstack.org/329663 | 20:16 |
| anteaya | pleia2: enjoy lunch | 20:16 |
| *** kzaitsev_mb has joined #openstack-infra | 20:17 | |
| *** xarses_ has joined #openstack-infra | 20:19 | |
| *** xarses_ has quit IRC | 20:24 | |
| *** xarses_ has joined #openstack-infra | 20:24 | |
| *** sarob has joined #openstack-infra | 20:24 | |
| *** flwang has joined #openstack-infra | 20:26 | |
| *** dmk0202 has quit IRC | 20:26 | |
| fungi | jeblair: 329638 lgtm, if you want to be the second +2 looks like it's all set for approval | 20:26 |
| *** dmk0202 has joined #openstack-infra | 20:26 | |
| *** xarses_ has quit IRC | 20:27 | |
| *** xarses_ has joined #openstack-infra | 20:27 | |
| *** mriedem1 has joined #openstack-infra | 20:27 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: toci_gate_test: disable ipv6 on upgrade job for stable/liberty https://review.openstack.org/329663 | 20:27 |
| pabelanger | fungi: clarkb: do you mind reviewing: 329536 Our wheel-builds are currently broken | 20:28 |
| *** sarob has quit IRC | 20:28 | |
| *** sarob has joined #openstack-infra | 20:28 | |
| *** e0ne has quit IRC | 20:29 | |
| *** PollyZ has joined #openstack-infra | 20:29 | |
| *** mriedem has quit IRC | 20:30 | |
| *** pradk has quit IRC | 20:31 | |
| *** rbrndt has quit IRC | 20:31 | |
| *** mriedem1 is now known as mriedem | 20:32 | |
| *** dmk0202 has quit IRC | 20:33 | |
| *** jsavak has quit IRC | 20:33 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: toci_gate_test: disable ipv6 on upgrade job for stable/liberty https://review.openstack.org/329663 | 20:33 |
| *** jsavak has joined #openstack-infra | 20:33 | |
| *** cody-somerville has quit IRC | 20:35 | |
| *** gouthamr has quit IRC | 20:36 | |
| *** ddieterly is now known as ddieterly[away] | 20:37 | |
| openstackgerrit | Merged openstack-infra/system-config: Adding ssh key for JP Maxwell to wiki.o.o https://review.openstack.org/329638 | 20:38 |
| prometheanfire | ianw: well, building now, thanks for the reminder, I didn't make a taks for that so it slipped my mind | 20:38 |
| *** PollyZ has quit IRC | 20:40 | |
| *** ccamacho has quit IRC | 20:40 | |
| openstackgerrit | Merged openstack-infra/project-config: Fix wheel-build macro issue https://review.openstack.org/329536 | 20:42 |
| *** rfolco has quit IRC | 20:42 | |
| *** mrmartin has joined #openstack-infra | 20:43 | |
| Shrews | pabelanger: left you question on https://review.openstack.org/329631 | 20:44 |
| anteaya | jpmaxman: so now we wait for a puppet run to ensure your ssh key is deployed to the wiki server | 20:44 |
| anteaya | what are we currently using as the puppet dashboard to track runs? | 20:44 |
| *** roxanagh_ has joined #openstack-infra | 20:45 | |
| fungi | anteaya: hopefully still puppetboard.openstack.org but it's severely limited with the switch to ansible | 20:46 |
| *** raildo is now known as raildo-afk | 20:47 | |
| *** PollyZ has joined #openstack-infra | 20:48 | |
| anteaya | any objections to getting rid of the puppet-dashboard.openstack.org entry in cacti? | 20:48 |
| pabelanger | Shrews: replied. we reason for leaving it | 20:48 |
| anteaya | fungi: and thank you | 20:48 |
| anteaya | jpmaxman: http://puppetboard.openstack.org/node/wiki.openstack.org | 20:48 |
| fungi | anteaya: none from me. if it's still in the config, propose removal | 20:48 |
| anteaya | I will, thanks | 20:49 |
| pabelanger | Shrews: basically, I've seen the abort request happen as we are uploading our publisher logs. Which is too late for the -9 to happen | 20:49 |
| *** roxanagh_ has quit IRC | 20:50 | |
| *** _nadya_ has joined #openstack-infra | 20:51 | |
| anteaya | fungi: it isn't in system-config, I wonder how it is still displaying in cacti | 20:51 |
| *** mrmartin has quit IRC | 20:52 | |
| fungi | anteaya: it may be that we don't have automated cleanup for things removed from cacti configs | 20:52 |
| *** edmondsw has quit IRC | 20:52 | |
| anteaya | ah | 20:52 |
| *** yolanda has quit IRC | 20:52 | |
| Shrews | pabelanger: ah, i see | 20:52 |
| anteaya | perhaps that is the thing | 20:53 |
| pabelanger | Shrews: I think we can revisit that logic and optimize it better. Sure jeblair has some ideas | 20:53 |
| *** shardy has quit IRC | 20:53 | |
| *** thorst has quit IRC | 20:54 | |
| *** thorst has joined #openstack-infra | 20:55 | |
| Shrews | pabelanger: these magic numbers concern me a bit. perhaps we should log the return code | 20:55 |
| Shrews | oh, we do | 20:55 |
| Shrews | duh | 20:55 |
| *** abregman is now known as abregman|ZzZzZz | 20:56 | |
| *** ayoung_ has quit IRC | 20:57 | |
| *** ayoung has quit IRC | 20:57 | |
| *** eezhova has joined #openstack-infra | 20:58 | |
| *** trown is now known as trown|outtypewww | 20:59 | |
| *** matt-borland has joined #openstack-infra | 20:59 | |
| *** thorst has quit IRC | 20:59 | |
| *** akshai has joined #openstack-infra | 20:59 | |
| *** _nadya_ has quit IRC | 20:59 | |
| jeblair | The conditional check 'task0.rc != 0 or task1.rc != 0 or task2.rc != 0 or task3.rc != 0 or task4.rc != 0' failed. The error was: error while evaluating conditional (task0.rc != 0 or task1.rc != 0 or task2.rc != 0 or task3.rc != 0 or task4.rc != 0): 'dict object' has no attribute 'rc' | 21:00 |
| jeblair | Shrews, pabelanger: ^ | 21:00 |
| timothyb89 | are there any gerrit wizards around who could help setup initial permissions for karma-subunit-reporter? | 21:00 |
| *** cody-somerville has joined #openstack-infra | 21:01 | |
| *** cody-somerville has joined #openstack-infra | 21:01 | |
| *** jd__ has quit IRC | 21:01 | |
| *** jd__ has joined #openstack-infra | 21:01 | |
| *** matt-borland_ has quit IRC | 21:01 | |
| *** eharney has quit IRC | 21:01 | |
| pabelanger | jeblair: any more to the log? | 21:01 |
| Shrews | jeblair: oh, hrm. rc might only be valid on shell commands | 21:01 |
| Shrews | which would stink | 21:01 |
| pabelanger | Oh, hmm | 21:02 |
| jeblair | pabelanger, Shrews: http://paste.openstack.org/show/516063/ | 21:02 |
| Shrews | jeblair: ugh. yeah: http://docs.ansible.com/ansible/shell_module.html#return-values | 21:03 |
| jeblair | so probably synchronize or copy are the culprits | 21:03 |
| Shrews | yeah | 21:03 |
| *** csomerville has joined #openstack-infra | 21:03 | |
| *** darrenc is now known as darrenc_afk | 21:03 | |
| *** _ari_ has quit IRC | 21:04 | |
| pabelanger | maybe checking stderr ? | 21:05 |
| pabelanger | but, that might only be shell again | 21:05 |
| fungi | timothyb89: what was the change that added that repo? | 21:05 |
| Shrews | pabelanger: yep, only shell | 21:05 |
| jeblair | pabelanger, Shrews: for checking stderr -- we could look for the 'ignoring' message from ansible | 21:05 |
| timothyb89 | fungi: https://review.openstack.org/#/c/327945/ | 21:05 |
| jeblair | (i do not like log scraping as a solution, but if that's the best we have, that might work) | 21:06 |
| *** e0ne has joined #openstack-infra | 21:06 | |
| jeblair | i wonder if we should head over to #ansible for this one | 21:06 |
| *** priteau has joined #openstack-infra | 21:06 | |
| *** gyee has joined #openstack-infra | 21:06 | |
| *** inc0 has quit IRC | 21:06 | |
| *** ddieterly[away] is now known as ddieterly | 21:06 | |
| Shrews | jeblair: probably | 21:06 |
| *** cody-somerville has quit IRC | 21:07 | |
| fungi | if you dangle "we have to parse your logs to solve this" in front of them, they might be compelled to point out far more palatable solutions | 21:07 |
| *** psilvad has quit IRC | 21:07 | |
| *** aysyd has quit IRC | 21:08 | |
| *** gouthamr has joined #openstack-infra | 21:08 | |
| fungi | timothyb89: done | 21:08 |
| timothyb89 | fungi: great, thanks! | 21:09 |
| *** abregman|ZzZzZz has quit IRC | 21:09 | |
| *** inc0 has joined #openstack-infra | 21:10 | |
| Shrews | jeblair: i'm stumped for ideas | 21:11 |
| Shrews | we should as bcoca | 21:11 |
| Shrews | ask* | 21:12 |
| *** yamahata has joined #openstack-infra | 21:12 | |
| *** julim has quit IRC | 21:12 | |
| *** javeriak has joined #openstack-infra | 21:13 | |
| *** javeriak has quit IRC | 21:13 | |
| *** piet has quit IRC | 21:14 | |
| pabelanger | jeblair: Shrews: same, unless we switch to using shell over synchronize. But not sure I like that | 21:15 |
| jeblair | i like log scraping better than that :( | 21:16 |
| *** javeriak_ has quit IRC | 21:16 | |
| jeblair | (and to reiterate, i do not like log scraping) | 21:16 |
| Shrews | ditto | 21:16 |
| jeblair | i asked in #ansible; we'll see if that merits a response :) | 21:17 |
| *** csomerville has quit IRC | 21:17 | |
| pabelanger | agreed | 21:17 |
| *** ldnunes has quit IRC | 21:18 | |
| *** csomerville has joined #openstack-infra | 21:18 | |
| *** woodster_ has quit IRC | 21:18 | |
| *** EricGonczer_ has quit IRC | 21:19 | |
| Shrews | jeblair: pabelanger: i have a feeling what we want is outside the norm and cannot be done w/o going all shell or scraping | 21:20 |
| jeblair | Shrews: perhaps so, but i don't feel it's *that* crazy | 21:20 |
| *** annegentle has quit IRC | 21:21 | |
| jeblair | (it follows fairly naturally from the idea of idempotency) | 21:21 |
| Shrews | jeblair: not *crazy*, just not often needed | 21:21 |
| *** kzaitsev_mb has quit IRC | 21:24 | |
| *** sarob has quit IRC | 21:25 | |
| *** burgerk has quit IRC | 21:26 | |
| *** rhallisey has quit IRC | 21:26 | |
| *** kzaitsev_mb has joined #openstack-infra | 21:27 | |
| *** amitgandhinz has quit IRC | 21:27 | |
| *** amitgandhinz has joined #openstack-infra | 21:27 | |
| *** matt-borland has quit IRC | 21:28 | |
| *** eezhova has quit IRC | 21:30 | |
| jeblair | Shrews, pabelanger: perhaps the best thing to scan for would be "fatal:" at the start of a log line | 21:32 |
| *** bpokorny has quit IRC | 21:32 | |
| openstackgerrit | Merged openstack-infra/project-config: Double our boot-timeout for osic-cloud1 https://review.openstack.org/329662 | 21:33 |
| *** eezhova has joined #openstack-infra | 21:34 | |
| *** inc0 has quit IRC | 21:34 | |
| *** elarson has joined #openstack-infra | 21:34 | |
| elarson | hey all, I've been getting disconnected from the list due to bounces and the folks managing my email could use some help figuring out why it is happening. is there a good contact I can provide them to help see what is happening? | 21:35 |
| pabelanger | jeblair: Shrews: quick test local: http://paste.openstack.org/show/516066/ I see failed = true for my register variable | 21:35 |
| pabelanger | jeblair: both for copy and synchronize | 21:35 |
| *** dizquierdo has joined #openstack-infra | 21:35 | |
| pabelanger | I'd have to test more to see if that world work | 21:36 |
| jeblair | pabelanger: i don't understand what you did | 21:36 |
| Shrews | pabelanger: with ignore_errors=true? | 21:36 |
| jeblair | pabelanger: can you post the playbook? | 21:37 |
| *** haaj has joined #openstack-infra | 21:38 | |
| *** pradk has joined #openstack-infra | 21:39 | |
| fungi | elarson: i can look into it. you're not at rackspace by any chance? was just looking into a bunch of their mass-unsubscribe events with ttx | 21:39 |
| *** bpokorny has joined #openstack-infra | 21:39 | |
| elarson | fungi: I am! | 21:39 |
| elarson | fungi: there are a bunch of folks that are seeing the same thing. let me know how I can help! | 21:40 |
| pabelanger | Shrews: jeblair: http://paste.openstack.org/show/516067/ was my example | 21:40 |
| fungi | elarson: <fireeye@raxempsscanner.corp>: conversation with 50.56.228.92[50.56.228.92] timed out while sending end of data -- message may be sent more than once | 21:40 |
| pabelanger | looks like .failed is set on failure | 21:40 |
| fungi | elarson: mailman is getting lots of bounces with that back from rackspace e-mail subscribers | 21:40 |
| jeblair | pabelanger, Shrews, clarkb: an alternative might be to use the regular behavior for most of the publishers inside of a block and then put the console log publisher (if present) at the end inside of an always | 21:40 |
| *** thiagop has quit IRC | 21:41 | |
| *** eezhova has quit IRC | 21:41 | |
| fungi | elarson: i interpret that as a fireeye scanner at rackspace getting overloaded and the previous hop mta bouncing the message back rather than deferring/queuing | 21:41 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul: Turn off Ansible host fact gathering https://review.openstack.org/329688 | 21:41 |
| dmsimard | pleia2: you can merge https://review.openstack.org/#/c/321229/ whenever you'd like | 21:41 |
| fungi | elarson: hopefully that's enough for your mailserver admins to go on? | 21:42 |
| *** e0ne has quit IRC | 21:42 | |
| elarson | fungi: I'll update my internal ticket and see! | 21:42 |
| fungi | elarson: ttx and i were just pondering who on that end to reach out to. much better if it comes from within | 21:42 |
| jeblair | fungi: oh, sorry, i somehow missed you and ttx working on this. anything i can do to help? | 21:43 |
| *** amitgandhinz has quit IRC | 21:43 | |
| Shrews | pabelanger: if that exists on shell commands (don't see why it wouldn't), then we could use that in place of 'rc' | 21:43 |
| *** bpokorny_ has joined #openstack-infra | 21:43 | |
| pabelanger | - debug: msg="{{ runner.failed|default(true) }}" could be the check | 21:44 |
| *** bpokorny_ has quit IRC | 21:44 | |
| pabelanger | if .failed is not setup, then default true | 21:44 |
| fungi | jeblair: came up yesterday, some rackspace subscribers (adrian otto for one) had reached out to him and he was wondering if there were options to troubleshoot. i suggested he turn on the setting that sends unsub notices to the listadmin since it will attach the bounce | 21:45 |
| *** eezhova has joined #openstack-infra | 21:45 | |
| pabelanger | err default(false( | 21:45 |
| *** bpokorny_ has joined #openstack-infra | 21:45 | |
| pabelanger | shell also setups .failed on failure | 21:45 |
| fungi | jeblair: i don't think it was in channel, but we can move the conversation here when he's back at the computer (perhaps tomorrow) | 21:46 |
| elarson | fungi: are there any logs of the bounces I can send along? | 21:46 |
| *** Sukhdev has quit IRC | 21:46 | |
| pabelanger | Shrews: jeblair: updated test http://paste.openstack.org/show/516068/ | 21:46 |
| *** bpokorny has quit IRC | 21:47 | |
| fungi | elarson: ttx forwarded one to me that i can forward to you. what's the address? | 21:48 |
| elarson | eric.larson@rackspace.com | 21:48 |
| elarson | thanks! | 21:48 |
| ianw | prometheanfire: i'm not 100% sure what the status is, but yolanda had changes out to use ansible instead of puppet for the infra bits | 21:49 |
| ianw | if there's puppet issues, it might be worth just focusing on that | 21:49 |
| *** stajkowski has joined #openstack-infra | 21:49 | |
| fungi | elarson: i just redirected a copy of what ttx sent me to you (Subject: Fwd: Bounce action notification) | 21:50 |
| stajkowski | looking for some reviews for a new project https://review.openstack.org/#/c/319368 | 21:50 |
| prometheanfire | ianw: possibly, I think I'm about as good in either | 21:50 |
| *** sileht has quit IRC | 21:51 | |
| ianw | prometheanfire: https://review.openstack.org/#/c/283788/ was the review i was thinking of | 21:51 |
| prometheanfire | just finished the cache-bindep portion I think on that image test | 21:51 |
| Shrews | pabelanger: jeblair: so i think 'failed' is only there on actual failure (https://github.com/ansible/ansible/blob/devel/lib/ansible/executor/task_executor.py#L115-L119) so what pabelanger proposed *should* work, i think | 21:51 |
| elarson | fungi: got it, thanks! | 21:52 |
| *** sileht has joined #openstack-infra | 21:52 | |
| jeblair | Shrews, pabelanger, clarkb: okay... should we try that, or should we consider the "exit on first failure except always run log upload" approach? | 21:52 |
| *** sarob has joined #openstack-infra | 21:53 | |
| jeblair | i think that will be sufficient for what we need -- it's just a matter of whether we think we also might benefit from partial uploads of other things (devstack logs... tarballs... etc) | 21:54 |
| *** darrenc_afk is now known as darrenc | 21:54 | |
| Shrews | jeblair: honestly, if the only task we want to always execute is the log upload, that seems simpler | 21:54 |
| clarkb | partial uploads of other things could be useful if the host has really ogtten into an unhappy place | 21:54 |
| clarkb | dstat logs are really important for example | 21:54 |
| Shrews | jeblair: but isn't ansible-playbook exit code what we're really after here? | 21:55 |
| jeblair | clarkb: what we get in the partial upload could be very random | 21:55 |
| pabelanger | agreed, our zuul_launcher logs are key to detecting failues, so uploading them always is a +1 for me. But, think we can also give .failed a go too | 21:55 |
| jeblair | Shrews: yes | 21:55 |
| jeblair | to try to rephrase the situation: | 21:55 |
| elarson | n | 21:56 |
| elarson | whoops! | 21:56 |
| *** csomerville has quit IRC | 21:56 | |
| jeblair | we *require* that we always upload the console+ansible logs, and we *require* that upon any publisher failure, we know about that. *optionally* we could try to continue past each publisher individually failing and try the next one. | 21:56 |
| *** csomerville has joined #openstack-infra | 21:57 | |
| jeblair | i think the last thing is the question -- whether we want to do that or not | 21:57 |
| clarkb | I like the idea of doing our best on error because when in an error situation you odn't know what will be useful info | 21:57 |
| *** dimtruck is now known as zz_dimtruck | 21:58 | |
| *** gouthamr_ has joined #openstack-infra | 21:58 | |
| *** ddieterly is now known as ddieterly[away] | 21:58 | |
| *** sarob has quit IRC | 21:58 | |
| jeblair | clarkb: ok. i just want to make sure that we understand that if, say, the devstack logs fail because they try to copy kern.log, that doesn't mean we will get dstat. it means we will copy whatever rsync happened to have copied so far (possibly nothing!) and still move on to the next publisher. | 21:58 |
| jeblair | so it's not "ignore individual file failures" it's "ignore publisher failures" | 21:59 |
| *** Sukhdev has joined #openstack-infra | 21:59 | |
| clarkb | gotcha | 21:59 |
| jeblair | still worth it? | 21:59 |
| clarkb | maybe we can ask rsync to be more greedy? | 21:59 |
| pabelanger | I have to step away, but agree that jeblair said on priority. I'll defer to which we do for now | 21:59 |
| jeblair | clarkb: a quick google says no | 21:59 |
| jeblair | clarkb: (solution is basically find+rsync) | 22:00 |
| clarkb | sadness | 22:00 |
| *** kzaitsev_mb has quit IRC | 22:00 | |
| *** ddieterly[away] is now known as ddieterly | 22:01 | |
| *** gouthamr has quit IRC | 22:01 | |
| anteaya | priteau: hello, are you available for a chat? | 22:02 |
| jeblair | clarkb: still worth it? | 22:02 |
| priteau | Yes for a quick chat | 22:02 |
| fungi | gonna go find some dinner, back in a while | 22:02 |
| anteaya | priteau: thank you, I understand you have done a lot of work on Blazar | 22:02 |
| priteau | anteaya: Sorry I meant to reply to your email today | 22:02 |
| clarkb | jeblair: probably not | 22:02 |
| anteaya | fungi: enjoy dinner | 22:02 |
| pleia2 | dmsimard: thanks, approved | 22:02 |
| anteaya | priteau: no worries, irc is better anyway, larger audience more help | 22:03 |
| anteaya | priteau: first of all thank you for using Blazar | 22:03 |
| anteaya | priteau: that is great news | 22:03 |
| anteaya | priteau: what can I do to help you upstream your patches? | 22:03 |
| jeblair | clarkb, Shrews: so does "block: [publishers]; always: [log upload]" sound like a close-enough approximation of what we want? | 22:03 |
| clarkb | jeblair: yes I think so | 22:04 |
| Shrews | jeblair: i think either is valid. but sounds like functionally, using the block seems to be more what you want | 22:04 |
| priteau | anteaya: I think the most helpful would be help to develop the tests, which are mostly missing at the moment | 22:05 |
| jeblair | it's at least simpler; i'll write that up and we can see what it looks like | 22:05 |
| anteaya | priteau: well I'm mostly interested in starting to get your work using gerrit | 22:05 |
| anteaya | priteau: was there a reason you didn't use our gerrit when you started added code to blazar in your fork? | 22:06 |
| anteaya | priteau: I should be clear, I'm a member of the infrastructure team, we help the developers use our workflow | 22:06 |
| anteaya | priteau: so having your work use gerrit would be helpful to having other openstack folks contribute to blazar, including getting tests working | 22:07 |
| anteaya | priteau: you wanted to upstream your work and I'd like to help you do that | 22:07 |
| *** njohnston is now known as njohnston|afk | 22:07 | |
| priteau | anteaya: when we started using Blazar in Chameleon, it was inactive with nothing happening in Gerrit, and we needed to get our project released so little time to iterate over patches | 22:08 |
| openstackgerrit | Merged openstack-infra/system-config: Add #ara to the list of channels https://review.openstack.org/321229 | 22:08 |
| anteaya | okay fair enough | 22:08 |
| anteaya | priteau: and yes there has been little activity in blazar for a while now | 22:09 |
| priteau | anteaya: I should be able to submit most of our patches in July | 22:09 |
| anteaya | okay great, may I ask why July? | 22:09 |
| priteau | anteaya: I am out of the office starting this Friday | 22:10 |
| priteau | for two weeks | 22:10 |
| anteaya | priteau: oh how lovely | 22:10 |
| priteau | sorry, bad timing | 22:10 |
| anteaya | I hope you have a marvelous time off | 22:10 |
| anteaya | not at all, glad to meet you | 22:10 |
| anteaya | long enough to wish you good holiday | 22:10 |
| priteau | (honeymoon ;-) ) | 22:10 |
| *** jamesmcarthur has quit IRC | 22:10 | |
| anteaya | oh even better | 22:10 |
| priteau | but I will come back refreshed and eager to push these patches | 22:10 |
| anteaya | may you and your spouse share much love and happiness together | 22:10 |
| Rockyg | congratulations, priteau ! | 22:11 |
| anteaya | ha ha ha | 22:11 |
| priteau | thanks | 22:11 |
| anteaya | you have a lovely time | 22:11 |
| *** mdrabe has quit IRC | 22:11 | |
| anteaya | and you know where to find us now | 22:11 |
| priteau | yep, thanks for getting in touch | 22:11 |
| anteaya | and I look forward to picking this up when you are back online | 22:11 |
| anteaya | sure thanks for the interest in Blazar | 22:11 |
| *** kzaitsev_mb has joined #openstack-infra | 22:11 | |
| *** mriedem is now known as mriedem_away | 22:12 | |
| *** Rockyg has quit IRC | 22:12 | |
| *** fungi_lounge has joined #openstack-infra | 22:13 | |
| anteaya | <-- dinner | 22:13 |
| priteau | have a good dinner anteaya! talk to you soon | 22:14 |
| *** Sukhdev has quit IRC | 22:14 | |
| *** psilvad has joined #openstack-infra | 22:15 | |
| *** Sukhdev has joined #openstack-infra | 22:15 | |
| anteaya | priteau: thank you | 22:15 |
| *** baoli has quit IRC | 22:15 | |
| *** Sukhdev has quit IRC | 22:16 | |
| *** dizquierdo has quit IRC | 22:16 | |
| *** akshai has quit IRC | 22:16 | |
| *** Sukhdev has joined #openstack-infra | 22:17 | |
| *** akshai has joined #openstack-infra | 22:17 | |
| *** priteau has quit IRC | 22:17 | |
| *** kzaitsev_mb has quit IRC | 22:18 | |
| *** eezhova has quit IRC | 22:19 | |
| *** mhickey has quit IRC | 22:20 | |
| *** netsin has quit IRC | 22:20 | |
| *** kzaitsev_mb has joined #openstack-infra | 22:21 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 22:23 | |
| *** openstackstatus has quit IRC | 22:25 | |
| *** openstack has joined #openstack-infra | 22:26 | |
| *** openstackstatus has joined #openstack-infra | 22:26 | |
| *** ChanServ sets mode: +v openstackstatus | 22:26 | |
| *** rcernin has joined #openstack-infra | 22:28 | |
| *** xarses has joined #openstack-infra | 22:30 | |
| *** xarses_ has quit IRC | 22:30 | |
| *** jsavak has quit IRC | 22:33 | |
| *** bpokorny_ has quit IRC | 22:35 | |
| *** ddieterly has quit IRC | 22:35 | |
| *** bpokorny has joined #openstack-infra | 22:35 | |
| *** bpokorny has quit IRC | 22:35 | |
| *** bpokorny has joined #openstack-infra | 22:36 | |
| *** psilvad has quit IRC | 22:37 | |
| *** furlongm_ has joined #openstack-infra | 22:37 | |
| *** rbrndt has joined #openstack-infra | 22:37 | |
| *** furlongm has quit IRC | 22:37 | |
| *** mtanino has quit IRC | 22:38 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: toci_gate_test: disable ipv6 on upgrade job for stable/liberty https://review.openstack.org/329663 | 22:39 |
| *** tkelsey has joined #openstack-infra | 22:39 | |
| *** rbrndt has quit IRC | 22:39 | |
| asselin | anyone know how to use ansible to find attached volumes and format partition them? | 22:40 |
| *** mtanino has joined #openstack-infra | 22:43 | |
| *** tkelsey has quit IRC | 22:43 | |
| *** hongbin has quit IRC | 22:43 | |
| jhesketh | Morning | 22:43 |
| *** amitgandhinz has joined #openstack-infra | 22:44 | |
| clarkb | asselin: at the very least you should be able to format and partition using shell directives. I do know not how to get the list of attached volumes in ansible though. | 22:44 |
| *** ajmiller has quit IRC | 22:44 | |
| asselin | yeah, I was hoping there was an 'ansible' way to do it. infra-ansbile has openstack.py which has the info, but I don't understand how it integrates... | 22:45 |
| *** bhavik1 has joined #openstack-infra | 22:46 | |
| jeblair | pabelanger: it looks like ansible logging is buffered, so when we upload the in-progress logs, we lose the last little bit | 22:46 |
| *** roxanagh_ has joined #openstack-infra | 22:47 | |
| jeblair | pabelanger: i mean, i was expecting not to have the results of actually uploading the log itself (for obvious reasons). but we lose the last few tasks before that too it seems | 22:47 |
| *** PollyZ_ has joined #openstack-infra | 22:47 | |
| clarkb | asselin: typically I thin kyo uwould assert the attachment in ansible then operate off of that | 22:47 |
| jeblair | Shrews, mordred: ^ | 22:48 |
| *** amitgandhinz has quit IRC | 22:48 | |
| clarkb | asselin: but http://docs.ansible.com/ansible/os_server_facts_module.html will likely include that info | 22:48 |
| *** bhavik has quit IRC | 22:48 | |
| *** bhavik1 is now known as bhavik | 22:48 | |
| jeblair | Shrews, mordred, pabelanger: doh! i'm wrong | 22:49 |
| mordred | jeblair: whee | 22:49 |
| jeblair | Shrews, mordred, pabelanger: i saw the log end before i expected because we *copy* it, so it stopped at the copy task, which is before the upload task. | 22:49 |
| *** PollyZ has quit IRC | 22:50 | |
| asselin | clarkb, looks interesting. will try it out | 22:50 |
| *** gongysh has joined #openstack-infra | 22:50 | |
| *** roxanagh_ has quit IRC | 22:51 | |
| *** ajmiller has joined #openstack-infra | 22:51 | |
| clarkb | mordred: btw I notice in http://docs.ansible.com/ansible/os_server_volume_module.html that you allow setting the device. It should be noted that that is not reliable and is going away soon aiui | 22:51 |
| *** Sukhdev has quit IRC | 22:51 | |
| *** PollyZ_ has quit IRC | 22:52 | |
| mordred | clarkb: that's fantastic | 22:52 |
| mordred | clarkb: I love it when OpenStack removes interfaces to things | 22:52 |
| *** vhosakot has quit IRC | 22:52 | |
| asselin | lol...glad i didn't use that option | 22:53 |
| csomerville | lol | 22:53 |
| *** gouthamr_ has quit IRC | 22:54 | |
| asselin | got an unexpected error: http://paste.openstack.org/show/516069/ "shade is required for this module"} | 22:54 |
| clarkb | install shade? | 22:55 |
| asselin | it is installed.... | 22:55 |
| *** gouthamr has joined #openstack-infra | 22:55 | |
| asselin | oh...maybe it needs to be installed on the target host? | 22:56 |
| clarkb | asselin: no virtualenvs are in use? anything like that | 22:56 |
| clarkb | asselin: it needs to be installed where ansible is running that lookup | 22:56 |
| clarkb | usually you would delegate to localhost I think since cloud | 22:57 |
| *** tpsilva has quit IRC | 22:57 | |
| *** ajmiller_ has joined #openstack-infra | 23:01 | |
| notmorgan | anteaya: (ignore this checking something) | 23:02 |
| notmorgan | anteaya: (your nick colour again:P) | 23:02 |
| notmorgan | anteaya: awww... it's not as cool a color now. boo. | 23:03 |
| *** ams__ has quit IRC | 23:03 | |
| *** netsin has joined #openstack-infra | 23:04 | |
| *** ajmiller has quit IRC | 23:05 | |
| anteaya | boo | 23:06 |
| anteaya | what colour am I now? | 23:06 |
| anteaya | my cat was torquoise and orange last night | 23:07 |
| anteaya | she looked so beautiful I couldn't stop petting her | 23:07 |
| anteaya | notmorgan: am I green still? green-blue? | 23:08 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Upload ansible playbook directory to logs server https://review.openstack.org/325602 | 23:10 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: ensure that console/ansible publishing is last https://review.openstack.org/329608 | 23:10 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: ensure log publishing always runs https://review.openstack.org/329612 | 23:10 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: set keep-hierarchy on log copying https://review.openstack.org/329712 | 23:10 |
| jeblair | clarkb, Shrews, pabelanger, mordred: ^ | 23:10 |
| pabelanger | back | 23:11 |
| pabelanger | checking | 23:11 |
| jeblair | i tested that, and it seems to work | 23:11 |
| nibalizer | asselin: clarkb yea when you have ansible in a venv it gets derpy | 23:12 |
| nibalizer | because the local connection loses the venv | 23:12 |
| asselin | nibalizer, go figure | 23:13 |
| nibalizer | sometimes ill set python_interpreter on the host | 23:13 |
| nibalizer | to get aroubd it | 23:13 |
| *** salv-orlando has quit IRC | 23:14 | |
| notmorgan | lifeless: let me know when you're about. :) | 23:14 |
| anteaya | notmorgan: well keep me posted if I become a better colour | 23:14 |
| *** salv-orlando has joined #openstack-infra | 23:14 | |
| * anteaya thinks pink | 23:14 | |
| notmorgan | lifeless: want to start talking testrepository and such :) since ... I'm looking at things to dive into and fixup to make it all better for all of us | 23:15 |
| notmorgan | anteaya: oh you don't want pink :P you want magenta ;) [i say this cause pink is kinda dull on this terminal] | 23:15 |
| anteaya | magenta | 23:15 |
| anteaya | I want magenta | 23:15 |
| anteaya | how right you are | 23:16 |
| pabelanger | jeblair: +2 for the stack. Much simpler | 23:17 |
| *** ayoung has joined #openstack-infra | 23:17 | |
| jeblair | btw, i did test that you can have a 'block' without an always (obvious to me); and an 'always' without 'block' (less obvious to me) | 23:17 |
| jeblair | (but still sensible i think) | 23:18 |
| notmorgan | wwwwwwwww doh, pabelanger and anteaya now have the same coloured nicks now. I.. dunno how to fix this. | 23:18 |
| mordred | jeblair: you can have an always without a block? | 23:18 |
| anteaya | notmorgan: we're both magenta? well we are both canadian, so that fits | 23:19 |
| notmorgan | no, trying to figure out if i can set a specific color for a specific nick | 23:19 |
| anteaya | plus our nicks are different lengths of characters | 23:19 |
| anteaya | notmorgan: what client? | 23:19 |
| notmorgan | anteaya: weechat | 23:19 |
| anteaya | yeah I just go with what it gives me | 23:20 |
| mordred | anteaya: in my terminal you are the same color of blue as jeblair - and notmorgan and pabelanger are the same magenta | 23:20 |
| anteaya | you're purple | 23:20 |
| *** Sukhdev has joined #openstack-infra | 23:20 | |
| anteaya | really? | 23:20 |
| notmorgan | mordred: you have... dark cyan i think | 23:20 |
| anteaya | mordred: you are the same blue as jeblair in my terminal | 23:20 |
| anteaya | jeblair just goes with everything | 23:20 |
| mordred | hahaha. like a nice credenza | 23:20 |
| *** sarob has joined #openstack-infra | 23:20 | |
| anteaya | ha ha ha | 23:21 |
| notmorgan | anteaya: ++ | 23:21 |
| jeblair | everyone is the same color of grey in my terminal. | 23:21 |
| * notmorgan is looking at the colorize nick code (yay python) | 23:21 | |
| anteaya | ha ha ha | 23:21 |
| notmorgan | jeblair: so your terminal is color blind? | 23:21 |
| mordred | jeblair: how do you know who is talking? | 23:21 |
| notmorgan | :) | 23:21 |
| notmorgan | mordred: i know right? | 23:21 |
| jeblair | it makes you all easier to ignore ;) | 23:21 |
| mordred | jeblair: great plan | 23:22 |
| anteaya | yeah, like how can I tell who to reply to unless I evaluate them with the credenza | 23:22 |
| * jeblair tries out black | 23:22 | |
| anteaya | ha ha ha | 23:22 |
| jeblair | SO MUCH BETTER | 23:22 |
| notmorgan | jeblair: i see what you did there... | 23:22 |
| anteaya | jeblair's productivity goes up, way up | 23:22 |
| *** akshai has quit IRC | 23:23 | |
| jeblair | notmorgan: i didn't! ;) | 23:23 |
| notmorgan | jeblair: ^5 | 23:23 |
| jeblair | notmorgan and i are here all week folks! | 23:24 |
| jeblair | tip your irc bot! | 23:24 |
| *** sarob has quit IRC | 23:24 | |
| *** sarob has joined #openstack-infra | 23:24 | |
| *** Guest16087 has quit IRC | 23:28 | |
| pabelanger | TIL: people want to see what I say | 23:29 |
| *** Jeffrey4l has joined #openstack-infra | 23:29 | |
| *** salv-orl_ has joined #openstack-infra | 23:29 | |
| clarkb | pabelanger: error launches in osic seem to have fallen off but that may also be because overall demand has fallen | 23:29 |
| *** yamada-h has joined #openstack-infra | 23:30 | |
| pabelanger | clarkb: could be. I was working with somebody at rackspace today. They are looking into the issue and expect to hear back from them tomorrow | 23:31 |
| *** salv-orlando has quit IRC | 23:32 | |
| *** sdake has quit IRC | 23:32 | |
| *** sdake has joined #openstack-infra | 23:33 | |
| anteaya | pabelanger: yes, we do | 23:34 |
| *** yamada-h has quit IRC | 23:34 | |
| *** xarses has quit IRC | 23:37 | |
| *** piet_ has joined #openstack-infra | 23:39 | |
| *** rcernin has quit IRC | 23:40 | |
| *** asselin_ has joined #openstack-infra | 23:40 | |
| lifeless | notmorgan: hai | 23:40 |
| *** amitgandhinz has joined #openstack-infra | 23:44 | |
| *** asselin_ has quit IRC | 23:45 | |
| *** Sukhdev has quit IRC | 23:45 | |
| notmorgan | lifeless: ohai | 23:48 |
| notmorgan | lifeless: so, in this channel or -dev better to talk testrepository and such? | 23:49 |
| *** amitgandhinz has quit IRC | 23:49 | |
| lifeless | notmorgan: #subunit probably, its where most of the contributors idle | 23:49 |
| notmorgan | okie. | 23:49 |
| openstackgerrit | Jeffrey Zhang proposed openstack-infra/project-config: Enable the Ubuntu binary build and deploy for all branch in kolla https://review.openstack.org/329721 | 23:52 |
| *** sdague has quit IRC | 23:53 | |
| *** johnny_ has quit IRC | 23:54 | |
| *** xyang1 has quit IRC | 23:56 | |
| openstackgerrit | Hisashi Osanai proposed openstack-infra/project-config: Add jobs for Monasca-Analytics https://review.openstack.org/329376 | 23:57 |
| openstackgerrit | Andrea Frittoli proposed openstack-infra/project-config: Start using pre-prov creds in dsvm-neutron-full https://review.openstack.org/329723 | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!