| anteaya | if you share error messages and stack traces with us on a paste service, things usually go faster | 00:00 |
|---|---|---|
| brucet | I'll take a break | 00:00 |
| *** ramishra has joined #openstack-infra | 00:00 | |
| fungi | jeblair: thanks! | 00:00 |
| brucet | Thanks greatly for all your help | 00:00 |
| brucet | I'll be back tomorrow | 00:00 |
| mtreinish | this is the best proposal bot patch I've seen in a while: https://review.openstack.org/#/c/304901/1 | 00:00 |
| brucet | See you then | 00:00 |
| fungi | brucet: you're welcome, i'm convinced we can get you the rest of the way through this when you're ready to continue | 00:00 |
| *** sputnik13 has joined #openstack-infra | 00:01 | |
| jeblair | mtreinish: um. apparently we've trained our bots to behave like us. what have we done? WHAT HAVE WE DONE? | 00:01 |
| fungi | mtreinish: that extra space offends me | 00:01 |
| anteaya | mtreinish: good thing that person got in there to +1 that patch | 00:01 |
| anteaya | whew | 00:01 |
| fungi | i'm glad i won't be the first with my back against the wall when the bots take over | 00:01 |
| *** baoli has quit IRC | 00:01 | |
| *** brucet has quit IRC | 00:02 | |
| mordred | mtreinish: excellent | 00:02 |
| *** aeng_ has joined #openstack-infra | 00:03 | |
| jeblair | wow, that helped me find a gertty bug | 00:03 |
| *** dchen has joined #openstack-infra | 00:03 | |
| jeblair | so thanks | 00:03 |
| fungi | what bug? | 00:04 |
| mtreinish | heh, yw | 00:04 |
| *** tphummel has quit IRC | 00:04 | |
| fungi | i was looking at it with gertty, didn't spot an issue | 00:04 |
| jeblair | i reviewed it and it did not post my review | 00:04 |
| *** aeng has quit IRC | 00:04 | |
| mordred | weird | 00:04 |
| fungi | ahh | 00:04 |
| jeblair | i don't have a cause, but i have a debug log so i'll be able to figure it out | 00:04 |
| *** camunoz has quit IRC | 00:05 | |
| mordred | jeblair: I see your vote | 00:05 |
| jeblair | oh that's weird | 00:05 |
| fungi | i should go back to running with debug enabled. i've discovered that something (git gc/prune maybe?) is causing gertty to blow up failing to find older patchsets for changes it has sync'd | 00:05 |
| jeblair | i loaded it up in the web interface afterwords and did not see it | 00:05 |
| jeblair | but i see it now | 00:05 |
| jeblair | oh | 00:05 |
| jeblair | it's 00:00 | 00:05 |
| fungi | hah | 00:06 |
| jeblair | this is not a gertty bug | 00:06 |
| fungi | backups strike again! | 00:06 |
| jeblair | nevermind :) | 00:06 |
| *** sdake has joined #openstack-infra | 00:06 | |
| *** sputnik13 has quit IRC | 00:07 | |
| EmilienM | hi, can I have a +A on https://review.openstack.org/#/c/303715/ please? | 00:08 |
| *** salv-orlando has joined #openstack-infra | 00:09 | |
| clarkb | fungi: jeblair just occurred to me that the backup server's ssh ley will need to be accepted on new review | 00:09 |
| jeblair | clarkb: ++ | 00:09 |
| ianw | mordred: i strongly disagree that the subject of the commit should finish in a period and would be out of style. if anything, it should contain some sort of message, like "this was proposed automatically to keep this file in sync" for anyone who is new to all this | 00:10 |
| fungi | EmilienM: approved. i assume the previous reviewers held off waiting for the dependency to merge | 00:10 |
| fungi | looks simple enough | 00:10 |
| *** yuanying has joined #openstack-infra | 00:10 | |
| fungi | clarkb: yeah, we should probably try to remember that in subsequent server replacements where we're doing bup | 00:10 |
| EmilienM | fungi: thanks | 00:10 |
| fungi | clarkb: at least until we get around to managing a global known_hosts file or sshfp/dnssec records | 00:10 |
| mordred | ianw: :) | 00:12 |
| mordred | ianw: (my vote was a joke, I also agree with what you have said) | 00:12 |
| *** thorst_ has joined #openstack-infra | 00:14 | |
| *** claudiub has quit IRC | 00:14 | |
| *** salv-orlando has quit IRC | 00:14 | |
| EmilienM | I have a stupid questions about periodic jobs. How to you monitor them usually? | 00:15 |
| *** kengo_sakai_mba has quit IRC | 00:15 | |
| clarkb | EmilienM: they log to http://logs.openstack.org/periodic and can send email | 00:16 |
| *** sflanigan has quit IRC | 00:16 | |
| ianw | EmilienM: you get https://review.openstack.org/#/c/301989/ merged, and write a shell script :) | 00:16 |
| EmilienM | clarkb: awesome | 00:16 |
| jeblair | EmilienM: and maybe can be tracked in the health dashboard | 00:16 |
| EmilienM | http://status.openstack.org/openstack-health/#/ is only for gate, isn't? | 00:17 |
| *** ddieterly has joined #openstack-infra | 00:17 | |
| clarkb | it was, but semi recently expanded to include everything | 00:17 |
| anteaya | oh was it | 00:18 |
| EmilienM | clarkb: do you have an example of existing job in your mind about sending an email? | 00:18 |
| *** kengo_sakai_mba has joined #openstack-infra | 00:18 | |
| anteaya | what is the definition of everything? | 00:18 |
| *** asettle has joined #openstack-infra | 00:18 | |
| clarkb | EmilienM: all of the tempest periodic jobs sent email I think | 00:18 |
| clarkb | anteaya: any subunit file the worker canfind | 00:18 |
| EmilienM | clarkb: cool thanks | 00:18 |
| clarkb | when jobs complete | 00:18 |
| krotscheck | nibalizer: Heya, anything that I can help with? | 00:18 |
| anteaya | does that include the check queue? | 00:18 |
| clarkb | anteaya: yes I think so | 00:19 |
| *** camunoz has joined #openstack-infra | 00:19 | |
| *** ramishra has quit IRC | 00:19 | |
| anteaya | wow | 00:19 |
| EmilienM | I like it | 00:19 |
| anteaya | mtreinish: I'm impressed | 00:19 |
| *** ramishra has joined #openstack-infra | 00:19 | |
| EmilienM | it really help us: http://status.openstack.org/openstack-health/#/?groupKey=project&resolutionKey=hour&end=2016-04-13T00:16:12.291Z&searchProject=puppet | 00:19 |
| fungi | EmilienM: http://lists.openstack.org/pipermail/openstack-stable-maint/2016-April/004216.html | 00:19 |
| fungi | is an example | 00:20 |
| EmilienM | fungi: excellent! thanks for searching it. | 00:20 |
| openstackgerrit | Merged openstack-infra/project-config: zuul/layout: run intergation jobs on puppet-zaqar https://review.openstack.org/303715 | 00:20 |
| fungi | EmilienM: the openstack-stable-maint and openstack-qa mailing lists are basically just a dumping ground for periodic job results | 00:20 |
| EmilienM | my idea is to have puppet-openstack-integration on puppet4 running in periodic | 00:20 |
| *** fawadkhaliq has quit IRC | 00:21 | |
| EmilienM | as puppet 3.x is still the version that everyone use, we already support puppet 4 and I want to continue tests on v4 | 00:21 |
| *** fawadkhaliq has joined #openstack-infra | 00:21 | |
| mtreinish | clarkb, EmilienM, anteaya: it's just gate and periodic | 00:22 |
| *** sdake_ has joined #openstack-infra | 00:22 | |
| anteaya | mtreinish: thanks | 00:22 |
| clarkb | mtreinish: oh I thought you were going to open it up more since the data could be expired | 00:22 |
| clarkb | and that was the whole reason for doing that? | 00:22 |
| mtreinish | clarkb: we opened it to all test jobs not just tempest | 00:22 |
| *** kengo_sakai_mba has quit IRC | 00:23 | |
| mtreinish | keeping 6 months of everything with subunit on gate and periodic is close to pushing the limits of what we can handle | 00:23 |
| *** Swami_ has quit IRC | 00:23 | |
| *** sdake has quit IRC | 00:23 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Start stamping the has_service debug messages https://review.openstack.org/303853 | 00:24 |
| fungi | mtreinish: good thing the flap on our big tent isn't getting any wider | 00:24 |
| mordred | mtreinish: I think mongodb solves that problem | 00:24 |
| mtreinish | clarkb: there is also a data seperation issue on the check queue. It makes doing some of the analysis more difficult because you have to add filter on queue to everything | 00:24 |
| *** asettle has quit IRC | 00:24 | |
| fungi | web scale | 00:24 |
| mtreinish | mordred: heh | 00:24 |
| *** julim has quit IRC | 00:24 | |
| clarkb | mtreinish: ya its less trusted info | 00:24 |
| mordred | https://devnull-as-a-service.com/ | 00:25 |
| mtreinish | mordred: SpamapS has been trying to convince me to use influxdb | 00:25 |
| mordred | mtreinish: of course he has :) | 00:25 |
| fungi | i guess that's what years of maintaining mysql in debian has driven him to | 00:25 |
| mtreinish | clarkb: my current thinking is keeping 2 DBs one for gate and periodic and one for everything else, and the everything else db has like only a months worth of data | 00:26 |
| mtreinish | it'll let us keep things seperate and make sure we don't overload the dbs with too much data | 00:26 |
| mtreinish | and it shouldn't be too much work on openstack-health side to handle that | 00:27 |
| *** shashank_hegde has quit IRC | 00:27 | |
| clarkb | the zuul job queue looks funny | 00:29 |
| anteaya | clarkb: I agree | 00:30 |
| mordred | clarkb: like haha funny? | 00:30 |
| anteaya | like a plateau | 00:30 |
| anteaya | or mesa | 00:30 |
| mordred | oh. yeah | 00:30 |
| mordred | mesa zuul | 00:30 |
| anteaya | straight up kind of | 00:30 |
| mordred | jeblair: ^^ might be worth gandering at | 00:30 |
| anteaya | I don't see anything strange in cacti | 00:31 |
| anteaya | memory usage looks nice and low | 00:31 |
| *** pfallenop has joined #openstack-infra | 00:32 | |
| *** sflanigan has joined #openstack-infra | 00:32 | |
| anteaya | are the mergers doing odd things I wonder | 00:32 |
| *** ramishra has quit IRC | 00:32 | |
| *** tqtran has joined #openstack-infra | 00:33 | |
| *** ramishra has joined #openstack-infra | 00:33 | |
| *** zz_dimtruck is now known as dimtruck | 00:33 | |
| anteaya | so far cacti on mergers 1-3 looks fine | 00:33 |
| *** pvaneck has quit IRC | 00:35 | |
| *** sputnik13 has joined #openstack-infra | 00:35 | |
| anteaya | mergers 4-8 also don't look unusual in cacti | 00:36 |
| *** tqtran has quit IRC | 00:37 | |
| *** IlyaG has quit IRC | 00:37 | |
| *** knikolla has quit IRC | 00:37 | |
| *** shu-mutou-AFK has quit IRC | 00:39 | |
| *** knikolla_ has joined #openstack-infra | 00:39 | |
| *** sputnik13 has quit IRC | 00:41 | |
| *** knikolla_ has quit IRC | 00:41 | |
| *** arif-ali has quit IRC | 00:43 | |
| *** ramishra has quit IRC | 00:44 | |
| jeblair | could be a bunch of jobs submitted by old-nodepool | 00:47 |
| *** tiswanso has joined #openstack-infra | 00:47 | |
| *** tiswanso has quit IRC | 00:48 | |
| *** vilobhmm11 has quit IRC | 00:48 | |
| *** tiswanso has joined #openstack-infra | 00:48 | |
| mordred | jeblair: you konw - maybe _this_ is the use of the "pause" feature in the cloud | 00:49 |
| jeblair | hrm | 00:49 |
| mordred | jeblair: when we want to keep a server around for a bit without deleting | 00:49 |
| jeblair | stop:jenkins04.openstack.org257800 | 00:49 |
| mordred | but don't _really_ want it running | 00:49 |
| jeblair | that's what it actually is | 00:50 |
| mordred | hrm | 00:50 |
| mordred | well, there you go with debugging | 00:50 |
| *** ramishra has joined #openstack-infra | 00:50 | |
| *** arif-ali has joined #openstack-infra | 00:50 | |
| jeblair | the only stop: function registered in jenkins01 | 00:50 |
| jeblair | stop:jenkins01.openstack.org001 | 00:50 |
| jeblair | the others are all offline | 00:50 |
| jeblair | and have hundreds or thousands of stop requests | 00:51 |
| jeblair | i wonder if something is wrong with the gearman plugin | 00:52 |
| jeblair | this could have a significant adverse affect on node usage too, if we aren't able to cancel jobs and return nodes to the pool | 00:53 |
| jeblair | (and it looks like this started a while ago, so not related to old-nodepool) | 00:53 |
| jeblair | it is past eod for me, but someone should look into this | 00:54 |
| jeblair | perhaps by gracefully restarting a jenkins master and seeing if its stop function registers. also, checking the jenkins log for anyone but jenkins01 and seeing if there's anything amiss | 00:55 |
| *** ramishra has quit IRC | 00:55 | |
| *** kengo_sakai has joined #openstack-infra | 00:55 | |
| *** baoli has joined #openstack-infra | 00:55 | |
| *** mtanino has joined #openstack-infra | 00:55 | |
| *** ramishra has joined #openstack-infra | 00:56 | |
| *** Swami has joined #openstack-infra | 00:57 | |
| *** sdake_ has quit IRC | 00:58 | |
| *** Qiming has quit IRC | 00:58 | |
| *** baoli has quit IRC | 00:59 | |
| *** brucet has joined #openstack-infra | 00:59 | |
| *** darrenc is now known as darrenc_afk | 01:00 | |
| clarkb | 03 should gracefully restart soon | 01:00 |
| clarkb | 01 is likely registered because it was just restarted earlier | 01:00 |
| *** kengo_sakai_mba has joined #openstack-infra | 01:01 | |
| *** rossella_s has quit IRC | 01:03 | |
| *** rossella_s has joined #openstack-infra | 01:03 | |
| *** Swami has quit IRC | 01:06 | |
| *** zhurong has joined #openstack-infra | 01:07 | |
| *** gongysh has joined #openstack-infra | 01:08 | |
| *** thorst_ has quit IRC | 01:09 | |
| *** eric_lopez has joined #openstack-infra | 01:09 | |
| *** elo has joined #openstack-infra | 01:09 | |
| *** eric_lopez has left #openstack-infra | 01:10 | |
| *** elo has quit IRC | 01:11 | |
| *** ybathia has joined #openstack-infra | 01:12 | |
| *** bpokorny has quit IRC | 01:12 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/shade: [WIP] Test with a python-keystoneclient change from git/gerrit https://review.openstack.org/304833 | 01:13 |
| *** rfolco_ has joined #openstack-infra | 01:13 | |
| *** ybathia_ has joined #openstack-infra | 01:13 | |
| *** elo has joined #openstack-infra | 01:14 | |
| *** r-mibu has joined #openstack-infra | 01:15 | |
| *** brucet has quit IRC | 01:16 | |
| *** ybathia has quit IRC | 01:17 | |
| *** ybathia_ is now known as ybathia | 01:17 | |
| *** baoli has joined #openstack-infra | 01:17 | |
| *** ddieterly has quit IRC | 01:17 | |
| *** ramishra has quit IRC | 01:21 | |
| *** ramishra has joined #openstack-infra | 01:21 | |
| *** baoli has quit IRC | 01:22 | |
| *** baoli has joined #openstack-infra | 01:22 | |
| *** baoli has quit IRC | 01:25 | |
| *** baoli has joined #openstack-infra | 01:25 | |
| *** _amrith_ is now known as amrith | 01:26 | |
| *** mylu has quit IRC | 01:27 | |
| *** mylu has joined #openstack-infra | 01:28 | |
| *** mylu has quit IRC | 01:30 | |
| *** mylu has joined #openstack-infra | 01:31 | |
| *** ramishra has quit IRC | 01:32 | |
| anteaya | jhesketh: you busy at all? | 01:33 |
| *** mylu has quit IRC | 01:33 | |
| jhesketh | anteaya: just catching up on email etc... what can I do for you? | 01:33 |
| anteaya | jhesketh: we seem to have a thing happening but not sure what exactly | 01:33 |
| anteaya | jhesketh: the zuul queue graph looks funny: http://status.openstack.org/zuul/ | 01:34 |
| anteaya | as clarkb noted at 00:29 | 01:34 |
| *** darrenc_afk is now known as darrenc | 01:34 | |
| anteaya | jeblair is wondering if something is happening with the gearman plugin but also feels it is past his end of day | 01:35 |
| *** mylu has joined #openstack-infra | 01:35 | |
| anteaya | jhesketh: was wondering if you felt like taking a peek | 01:35 |
| * jhesketh is looking | 01:36 | |
| anteaya | thanks | 01:36 |
| *** salv-orlando has joined #openstack-infra | 01:39 | |
| *** sflanigan has quit IRC | 01:39 | |
| *** aeng_ has quit IRC | 01:39 | |
| *** knikolla has joined #openstack-infra | 01:39 | |
| *** dchen has quit IRC | 01:40 | |
| *** salv-orlando has quit IRC | 01:41 | |
| *** fawadkhaliq has quit IRC | 01:43 | |
| *** sflanigan has joined #openstack-infra | 01:43 | |
| *** sflanigan has joined #openstack-infra | 01:43 | |
| jhesketh | looks like jenkins03 has its stop function registered... not sure if that's due to jenkins restarting as clarkb mentioned | 01:43 |
| clarkb | jhesketh: it just restarted a few minutes ao | 01:44 |
| clarkb | I started 06 on its graceful restart | 01:44 |
| jhesketh | yeah I can see that running | 01:44 |
| clarkb | jhesketh: you might be able to disable a job then enable it to trigger the gearman plugin reregistering | 01:44 |
| *** fawadkhaliq has joined #openstack-infra | 01:44 | |
| clarkb | jhesketh: my guess is that that worker thread may have actually died though | 01:44 |
| clarkb | and so disable/enable won't help | 01:44 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/shade: [WIP] Test with a python-keystoneclient change from git/gerrit https://review.openstack.org/304833 | 01:44 |
| jhesketh | clarkb: not sure which job you'd disable/enable sorry? | 01:45 |
| clarkb | jhesketh: the way the plugin works is any time you disable enable a job its supposed to register any that aren't registered | 01:45 |
| *** kengo_sakai_mba has quit IRC | 01:46 | |
| jhesketh | oh it'll register all jobs? | 01:46 |
| jhesketh | clarkb: shall I try it on 02? | 01:46 |
| clarkb | jhesketh: it should figure out the delta and register what isn't registered | 01:47 |
| *** mylu has quit IRC | 01:47 | |
| clarkb | but I don't know that it will work because I bet the thread is dead | 01:47 |
| jhesketh | clarkb: right, want me to try anyway? | 01:48 |
| clarkb | jhesketh: ya worth a shot since its easy to try | 01:48 |
| *** mylu has joined #openstack-infra | 01:49 | |
| *** salv-orlando has joined #openstack-infra | 01:50 | |
| jhesketh | clarkb: I assume you mean the Disable Build in the job configuration? | 01:50 |
| clarkb | jhesketh: ya in the jenkins ui disable the build then reenable it | 01:51 |
| clarkb | that should cause registration things to fire | 01:51 |
| *** aeng_ has joined #openstack-infra | 01:52 | |
| *** dchen has joined #openstack-infra | 01:52 | |
| *** ramishra has joined #openstack-infra | 01:53 | |
| *** thorst_ has joined #openstack-infra | 01:53 | |
| *** apoorvad has quit IRC | 01:53 | |
| *** moravec has joined #openstack-infra | 01:54 | |
| *** Rockyg has quit IRC | 01:56 | |
| *** salv-orlando has quit IRC | 01:57 | |
| *** thorst_ has quit IRC | 01:57 | |
| jhesketh | clarkb: that didn't seem to help | 01:58 |
| jhesketh | clarkb: so I guess the best thing is to just wait for the restarts to finish? | 01:58 |
| *** Qiming has joined #openstack-infra | 01:58 | |
| *** yamamot__ has joined #openstack-infra | 02:00 | |
| *** Sukhdev_ has quit IRC | 02:01 | |
| *** baoli has quit IRC | 02:01 | |
| *** yamamoto has quit IRC | 02:03 | |
| *** ramishra has quit IRC | 02:03 | |
| *** ramishra has joined #openstack-infra | 02:04 | |
| *** ramishra has quit IRC | 02:05 | |
| *** rfolco_ has quit IRC | 02:08 | |
| *** ramishra has joined #openstack-infra | 02:09 | |
| *** kus has quit IRC | 02:09 | |
| AJaeger | jhesketh, clarkb, any other project-config cores, could you review https://review.openstack.org/#/c/304494 , please? | 02:13 |
| openstackgerrit | Ian Wienand proposed openstack-infra/glean: Add selinux context manager for writing files https://review.openstack.org/304357 | 02:13 |
| jhesketh | AJaeger: lgtm | 02:14 |
| *** ramishra has quit IRC | 02:14 | |
| *** dimtruck is now known as zz_dimtruck | 02:14 | |
| AJaeger | thanks, jhesketh | 02:14 |
| *** mylu has quit IRC | 02:14 | |
| *** mylu has joined #openstack-infra | 02:16 | |
| *** ramishra has joined #openstack-infra | 02:20 | |
| anteaya | AJaeger: +A | 02:20 |
| *** yamahata has quit IRC | 02:20 | |
| anteaya | AJaeger: you are up early/late | 02:20 |
| *** mriedem has quit IRC | 02:21 | |
| AJaeger | thanks, anteaya | 02:21 |
| AJaeger | anteaya: kids ill ;( | 02:21 |
| anteaya | AJaeger: oh no :( | 02:21 |
| anteaya | do you have to go to the doctor? | 02:21 |
| anteaya | or just be up and hold them? | 02:22 |
| AJaeger | just be up and give them something to drink - let's see for the rest later today ... | 02:23 |
| anteaya | AJaeger: I hope the kids feel better soon | 02:25 |
| anteaya | AJaeger: wishing you all good sleep | 02:25 |
| AJaeger | thanks, anteaya ! | 02:26 |
| anteaya | :) | 02:26 |
| *** vilobhmm11 has joined #openstack-infra | 02:26 | |
| * AJaeger goes back to bed in a few minutes... | 02:27 | |
| anteaya | I think I'm going to go offline now too | 02:27 |
| *** Volundr has joined #openstack-infra | 02:27 | |
| anteaya | jhesketh: best of luck figuring out what is happening with zuul | 02:27 |
| anteaya | jhesketh: sorry to dump it on you and leave | 02:27 |
| anteaya | I hope you feel supported even when we're offline | 02:27 |
| anteaya | :) | 02:27 |
| anteaya | jhesketh: go you! | 02:27 |
| clarkb | jhesketh: we can probably queue up the graceful restart script for jenkins.o.o, 02, 04,05, and 07 to get them to reregister | 02:28 |
| openstackgerrit | Merged openstack-infra/project-config: Translations: Enabling syncing again https://review.openstack.org/304494 | 02:28 |
| *** whoops has quit IRC | 02:28 | |
| *** sdake has joined #openstack-infra | 02:32 | |
| *** sdake has quit IRC | 02:32 | |
| *** ramishra has quit IRC | 02:32 | |
| *** sdake has joined #openstack-infra | 02:32 | |
| *** ramishra has joined #openstack-infra | 02:33 | |
| r-mibu | hi, infra team, could you review https://review.openstack.org/#/c/303921/ ? it is just adding non-voting tests for ceilometer | 02:34 |
| *** thorst_ has joined #openstack-infra | 02:34 | |
| *** daemontool has quit IRC | 02:36 | |
| *** asettle has joined #openstack-infra | 02:37 | |
| *** erlon has quit IRC | 02:37 | |
| *** daemontool has joined #openstack-infra | 02:37 | |
| *** amotoki has quit IRC | 02:38 | |
| *** netsin has quit IRC | 02:39 | |
| *** thorst_ has quit IRC | 02:40 | |
| jhesketh | anteaya: no worries, thanks for bringing it to my attention. That's what I'm here for | 02:41 |
| *** kengo_sakai_mba has joined #openstack-infra | 02:41 | |
| jhesketh | clarkb: are you running the 06 restart in a screen or something? | 02:41 |
| jhesketh | I figure it'd be best to do them in serial right | 02:42 |
| clarkb | ya on puppetmaster owned by me | 02:42 |
| *** sdake_ has joined #openstack-infra | 02:42 | |
| *** gyee has quit IRC | 02:42 | |
| *** achanda has joined #openstack-infra | 02:44 | |
| *** sdake has quit IRC | 02:45 | |
| *** yamahata has joined #openstack-infra | 02:45 | |
| *** kengo_sakai_mba has quit IRC | 02:46 | |
| *** ramishra has quit IRC | 02:46 | |
| *** netsin has joined #openstack-infra | 02:47 | |
| clarkb | this horizon test is slow, I wonder ifwe can have them switch to testr for the integration tests and speed stuff up | 02:47 |
| clarkb | (not sure what the browser overhead is on that if any) | 02:47 |
| *** achanda has quit IRC | 02:49 | |
| *** baoli has joined #openstack-infra | 02:51 | |
| *** dchen has quit IRC | 02:51 | |
| *** ramishra has joined #openstack-infra | 02:51 | |
| jhesketh | clarkb: I've joined your screen session with -x | 02:52 |
| jhesketh | clarkb: want me to kick off 02? | 02:53 |
| clarkb | jhesketh: go for it I am eating dinner and with 06 done am about done for the day | 02:53 |
| jhesketh | clarkb: no worries, I can keep an eye on it | 02:53 |
| *** yuanying has quit IRC | 02:54 | |
| *** bpokorny has joined #openstack-infra | 02:55 | |
| jhesketh | clarkb: fyi 06 has it's stop method registered | 02:56 |
| jhesketh | so this strategy appears to be working | 02:56 |
| *** vilobhmm11 has quit IRC | 02:57 | |
| *** asettle has quit IRC | 02:57 | |
| *** asettle has joined #openstack-infra | 02:58 | |
| *** armax has quit IRC | 02:59 | |
| *** maishsk has quit IRC | 02:59 | |
| *** ramishra has quit IRC | 02:59 | |
| *** ramishra has joined #openstack-infra | 03:00 | |
| *** sdake_ is now known as sdkae | 03:05 | |
| *** daemontool has quit IRC | 03:11 | |
| *** ramishra has quit IRC | 03:11 | |
| *** ramishra has joined #openstack-infra | 03:16 | |
| *** kengo_sakai_mba has joined #openstack-infra | 03:17 | |
| *** yamamot__ has quit IRC | 03:20 | |
| openstackgerrit | Jamie Lennox proposed openstack/requirements: Allow maintaining extras from project https://review.openstack.org/303271 | 03:20 |
| *** yamamot__ has joined #openstack-infra | 03:21 | |
| *** fawadkhaliq has quit IRC | 03:22 | |
| *** esker has quit IRC | 03:23 | |
| *** fawadkhaliq has joined #openstack-infra | 03:23 | |
| *** mylu has quit IRC | 03:24 | |
| *** kengo_sakai_mba has quit IRC | 03:25 | |
| *** yamamot__ has quit IRC | 03:26 | |
| openstackgerrit | Merged openstack/requirements: Updated from generate-constraints https://review.openstack.org/304516 | 03:31 |
| *** ramishra_ has joined #openstack-infra | 03:31 | |
| *** aeng_ has quit IRC | 03:35 | |
| *** sflanigan has quit IRC | 03:35 | |
| *** IlyaG has joined #openstack-infra | 03:37 | |
| *** vilobhmm11 has joined #openstack-infra | 03:39 | |
| *** achanda has joined #openstack-infra | 03:41 | |
| *** tiswanso has quit IRC | 03:41 | |
| *** esker has joined #openstack-infra | 03:44 | |
| *** jamesmcarthur has joined #openstack-infra | 03:45 | |
| *** yuanying has joined #openstack-infra | 03:47 | |
| *** esker has quit IRC | 03:48 | |
| *** sflanigan has joined #openstack-infra | 03:51 | |
| *** aeng_ has joined #openstack-infra | 03:51 | |
| *** amotoki has joined #openstack-infra | 03:51 | |
| *** ramishra_ has quit IRC | 03:51 | |
| *** ramishra_ has joined #openstack-infra | 03:53 | |
| *** baoli has quit IRC | 03:53 | |
| *** nikhil_k has joined #openstack-infra | 03:56 | |
| *** binbincong has joined #openstack-infra | 03:58 | |
| *** jamielennox is now known as jamielennox|away | 03:58 | |
| *** Nisha has joined #openstack-infra | 03:59 | |
| *** nikhil has quit IRC | 04:00 | |
| *** ramishra_ has quit IRC | 04:01 | |
| *** ramishra_ has joined #openstack-infra | 04:02 | |
| *** shashank_hegde has joined #openstack-infra | 04:02 | |
| *** yamamot__ has joined #openstack-infra | 04:04 | |
| *** jamielennox|away is now known as jamielennox | 04:06 | |
| openstackgerrit | Spencer Krum proposed openstack-infra/system-config: Create unprivileged user to run npm mirroring https://review.openstack.org/304891 | 04:06 |
| *** andymaier has joined #openstack-infra | 04:07 | |
| *** yarkot_ has joined #openstack-infra | 04:10 | |
| *** roxanaghe has joined #openstack-infra | 04:11 | |
| *** mylu has joined #openstack-infra | 04:14 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config: Upgrade infracloud modules to mitaka https://review.openstack.org/304946 | 04:15 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-infracloud: (WIP) Upgrade to mitaka https://review.openstack.org/304888 | 04:16 |
| *** ramishra_ has quit IRC | 04:16 | |
| *** ramishra_ has joined #openstack-infra | 04:17 | |
| *** achanda has quit IRC | 04:17 | |
| openstackgerrit | Ryota MIBU proposed openstack-infra/project-config: [telemetry] enable ceilo/aodh tempest plugin tests https://review.openstack.org/303921 | 04:18 |
| *** hdd has joined #openstack-infra | 04:22 | |
| *** bpokorny has quit IRC | 04:22 | |
| *** roxanaghe has quit IRC | 04:22 | |
| *** shashank_hegde has quit IRC | 04:24 | |
| *** claudiub has joined #openstack-infra | 04:27 | |
| *** roxanaghe has joined #openstack-infra | 04:27 | |
| *** asettle has quit IRC | 04:37 | |
| *** mrmartin has joined #openstack-infra | 04:38 | |
| *** mrmartin has quit IRC | 04:39 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Rework slave-cleanup https://review.openstack.org/304949 | 04:40 |
| AJaeger | fungi, anteaya: Our slave cleanup did never run, see change above with lengthy commit message for details ^ | 04:40 |
| *** ramishra_ has quit IRC | 04:43 | |
| *** ramishra_ has joined #openstack-infra | 04:44 | |
| *** links has joined #openstack-infra | 04:46 | |
| *** salv-orlando has joined #openstack-infra | 04:46 | |
| *** sridhar_ram has quit IRC | 04:49 | |
| *** fedexo has joined #openstack-infra | 04:49 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Delete more empty pot files https://review.openstack.org/304952 | 04:50 |
| AJaeger | infra-root, could you check why 304494 is not active on the proposal slave? It should have been pushed by puppet there but looking at recent (8mins ago) updates, it's not there. It merged 90 mins ago. | 04:52 |
| AJaeger | http://puppetboard.openstack.org/node/proposal.slave.openstack.org shows that the proposal slave was last updated 7h ago | 04:53 |
| AJaeger | infra-root, looking at puppetboard, last puppet run was 6 hours ago | 04:54 |
| *** salv-orlando has quit IRC | 04:54 | |
| *** ramishra_ has quit IRC | 04:54 | |
| *** ramishra_ has joined #openstack-infra | 04:55 | |
| *** shashank_hegde has joined #openstack-infra | 04:58 | |
| *** roxanaghe has quit IRC | 04:58 | |
| *** amotoki_ has joined #openstack-infra | 05:02 | |
| *** abregman has joined #openstack-infra | 05:02 | |
| *** roxanaghe has joined #openstack-infra | 05:03 | |
| *** flwang1 has quit IRC | 05:04 | |
| *** amotoki has quit IRC | 05:06 | |
| *** Sukhdev has joined #openstack-infra | 05:06 | |
| *** woodster_ has quit IRC | 05:07 | |
| *** jogo has quit IRC | 05:08 | |
| *** jogo has joined #openstack-infra | 05:10 | |
| *** jamesmcarthur has quit IRC | 05:10 | |
| *** fawadkhaliq has quit IRC | 05:11 | |
| *** fawadkhaliq has joined #openstack-infra | 05:11 | |
| nibalizer | AJaeger: a lot of this in the run_all log | 05:14 |
| nibalizer | 2016-04-12 22:13:40,774 p=28718 u=root | fatal: [refstack.openstack.org]: FAILED! => {"failed": true, "msg": "Unexpected failure during module execution.", "stdout": ""} | 05:14 |
| nibalizer | 2016-04-12 22:13:40,814 p=28718 u=root | An exception occurred during task execution. To see the full traceback, use -vvv. The error was: OSError: [Errno 12] Cannot allocate memory | 05:14 |
| *** oanson has joined #openstack-infra | 05:14 | |
| nibalizer | lot of oom killing going on | 05:15 |
| nibalizer | still, its less than 4 * 6 ooms | 05:16 |
| *** cody-somerville has quit IRC | 05:17 | |
| *** ihrachys has joined #openstack-infra | 05:17 | |
| *** ihrachys has quit IRC | 05:17 | |
| *** mestery has quit IRC | 05:17 | |
| *** maishsk has joined #openstack-infra | 05:18 | |
| openstackgerrit | Tony Breeds proposed openstack-infra/irc-meetings: WiP Display skipped meetings on the eavesdrop homepage https://review.openstack.org/304956 | 05:18 |
| *** r-mibu has quit IRC | 05:20 | |
| *** fawadkhaliq has quit IRC | 05:20 | |
| *** fawadkhaliq has joined #openstack-infra | 05:20 | |
| *** mestery has joined #openstack-infra | 05:23 | |
| *** andymaier has quit IRC | 05:23 | |
| *** ramishra_ has quit IRC | 05:24 | |
| *** mylu has quit IRC | 05:24 | |
| *** ramishra_ has joined #openstack-infra | 05:24 | |
| *** roxanaghe has quit IRC | 05:26 | |
| *** yamamot__ has quit IRC | 05:28 | |
| *** yamamot__ has joined #openstack-infra | 05:29 | |
| *** r-mibu has joined #openstack-infra | 05:30 | |
| *** harlowja_at_home has joined #openstack-infra | 05:32 | |
| *** cody-somerville has joined #openstack-infra | 05:32 | |
| *** sflanigan has quit IRC | 05:34 | |
| *** yamahata has quit IRC | 05:34 | |
| *** yamamo___ has joined #openstack-infra | 05:34 | |
| *** aeng_ has quit IRC | 05:35 | |
| *** camunoz has quit IRC | 05:35 | |
| *** asettle has joined #openstack-infra | 05:35 | |
| *** yamamot__ has quit IRC | 05:36 | |
| *** maishsk has quit IRC | 05:43 | |
| *** ramishra_ has quit IRC | 05:44 | |
| *** ramishra_ has joined #openstack-infra | 05:44 | |
| *** infra-red has joined #openstack-infra | 05:44 | |
| nibalizer | AJaeger: looking at the log, nothing new has appeared in /var/log/puppet_run_all_cron.log for about half an hour | 05:45 |
| *** shashank_hegde has quit IRC | 05:45 | |
| *** sflanigan has joined #openstack-infra | 05:48 | |
| *** aeng_ has joined #openstack-infra | 05:48 | |
| *** Nisha has quit IRC | 05:56 | |
| *** rcernin has joined #openstack-infra | 05:56 | |
| *** kushal has quit IRC | 05:57 | |
| *** cody-somerville has quit IRC | 05:57 | |
| *** jogo has quit IRC | 06:00 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/304962 | 06:01 |
| *** denisra has quit IRC | 06:02 | |
| *** amrith is now known as _amrith_ | 06:02 | |
| *** shashank_hegde has joined #openstack-infra | 06:02 | |
| *** harlowja_at_home has quit IRC | 06:03 | |
| *** _amrith_ is now known as amrith | 06:04 | |
| *** ramishra_ has quit IRC | 06:05 | |
| *** ramishra_ has joined #openstack-infra | 06:07 | |
| *** jogo has joined #openstack-infra | 06:08 | |
| *** yamamo___ has quit IRC | 06:10 | |
| AJaeger | nibalizer: thanks for checking. Is puppet running now? | 06:10 |
| *** yamamot__ has joined #openstack-infra | 06:11 | |
| *** sdkae is now known as sdake | 06:12 | |
| openstackgerrit | Christian Schwede proposed openstack-infra/project-config: Enable gate-grenade-dsvm-multinode for Swift https://review.openstack.org/304505 | 06:13 |
| *** jaosorior has joined #openstack-infra | 06:14 | |
| *** kushal has joined #openstack-infra | 06:14 | |
| *** kengo_sakai_mba has joined #openstack-infra | 06:16 | |
| *** ybathia has quit IRC | 06:16 | |
| *** sflanigan has quit IRC | 06:19 | |
| *** maishsk has joined #openstack-infra | 06:19 | |
| openstackgerrit | Tony Breeds proposed openstack-infra/project-config: Run the additional-pkg-repos devstack-plugin as self-test https://review.openstack.org/304968 | 06:19 |
| *** nadya has joined #openstack-infra | 06:20 | |
| *** Hal has joined #openstack-infra | 06:21 | |
| *** Hal is now known as Guest91520 | 06:21 | |
| *** vikram_ has joined #openstack-infra | 06:21 | |
| vikram_ | AJaeger: Need one help .. | 06:22 |
| *** nadya has quit IRC | 06:22 | |
| vikram_ | AJaeger: What's the diff b/w branch-designator & job-suffix | 06:22 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/304969 | 06:23 |
| AJaeger | branch-designator allows you to check out a different branch and is passed to the script. | 06:23 |
| AJaeger | job-suffix is just a different suffix of the name and has no effect on the change | 06:23 |
| AJaeger | We use a job-suffix of "-nv" to mark jobs as non-voting | 06:23 |
| *** Sukhdev has quit IRC | 06:24 | |
| *** salv-orlando has joined #openstack-infra | 06:24 | |
| AJaeger | With job suffix, you can instantiate your template twice - noce with "" and once with "-nv" so that you have the job voting for some repository and non-voting for others... | 06:24 |
| *** Sukhdev has joined #openstack-infra | 06:24 | |
| yolanda | good morning | 06:24 |
| *** yamahata has joined #openstack-infra | 06:25 | |
| AJaeger | morning, yolanda | 06:27 |
| *** Sukhdev has quit IRC | 06:27 | |
| AJaeger | Puppet is not running ;( nibalizer did some initial check but haven't seen any update for 45 mins from him | 06:28 |
| yolanda | ok let me check. I am leaving today for a pair of hours, but i'm still around | 06:28 |
| yolanda | i actually saw same problem with puppet yesterday, I had to run it manually last morning | 06:29 |
| yolanda | it times out on remote_puppet_else | 06:30 |
| *** salv-orlando has quit IRC | 06:30 | |
| yolanda | i killed the puppet processes and ran again | 06:31 |
| vikram_ | AJaeger: thanks | 06:31 |
| *** ybathia has joined #openstack-infra | 06:31 | |
| yolanda | but looks as some process when running the remote puppet is causing it to hang | 06:31 |
| yolanda | tailing logs now | 06:31 |
| *** infra-re_ has joined #openstack-infra | 06:32 | |
| yolanda | An exception occurred during task execution. To see the full traceback, use -vvv. The error was: OSError: [Errno 12] Cannot allocate memory | 06:32 |
| yolanda | fatal: [openstackid-dev.openstack.org]: FAILED! => {"failed": true, "msg": "Unexpected failure during module execution.", "stdout": ""} | 06:32 |
| AJaeger | Thanks, yolanda for looking. Better leave a message for the rest of the team so that somebody else can investigate... | 06:32 |
| AJaeger | looks familiar to what nibalizer posted an hour ago | 06:32 |
| yolanda | AJaeger, i will be back in a pair of hours so I can resume later, before other infra-root connect | 06:33 |
| AJaeger | thanks, yolanda ! | 06:33 |
| *** ihrachys has joined #openstack-infra | 06:34 | |
| * AJaeger will not approve changes for infra projects since they won't be effective anyway | 06:34 | |
| *** infra-red has quit IRC | 06:35 | |
| yolanda | AJaeger, it should actually run now, becauase i killed the locked process | 06:35 |
| *** yamahata has quit IRC | 06:35 | |
| yolanda | so puppet will run remote_puppet_git and replicate changes on all servers | 06:35 |
| AJaeger | ok, let's see - I'm waiting for update on proposal slave ;) | 06:35 |
| *** ramishra_ has quit IRC | 06:35 | |
| yolanda | it's running puppet on slaves now | 06:36 |
| vikram_ | all the tests for my review link https://review.openstack.org/#/c/304597/ has passed, but still jerkis is shown as failed | 06:37 |
| vikram_ | How to figure out what's the issue | 06:37 |
| *** ihrachys has quit IRC | 06:37 | |
| AJaeger | vikram_: Press the "Toggle CI" button | 06:37 |
| *** ramishra_ has joined #openstack-infra | 06:37 | |
| AJaeger | And then check the Jenkins output. | 06:38 |
| AJaeger | What does it say? | 06:38 |
| vikram_ | AJaeger: Ok | 06:38 |
| yolanda | ok leaving now | 06:39 |
| yolanda | i see puppet ran on release.slave, AJaeger | 06:39 |
| AJaeger | proposal slave is the one I need - but then there's hope. | 06:40 |
| AJaeger | yolanda: gave a great morning! | 06:40 |
| *** vgridnev has joined #openstack-infra | 06:40 | |
| AJaeger | s/gave/have/ | 06:41 |
| *** yamahata has joined #openstack-infra | 06:43 | |
| vikram_ | AJaeger: Thanks I have found the issue | 06:44 |
| vikram_ | AJaeger: Your suggestion helped | 06:44 |
| *** infra-re_ has quit IRC | 06:44 | |
| *** ramishra_ has quit IRC | 06:45 | |
| *** tesseract has joined #openstack-infra | 06:46 | |
| *** tesseract is now known as Guest12699 | 06:46 | |
| AJaeger | vikram_: glad to hear! | 06:47 |
| *** mikelk has joined #openstack-infra | 06:49 | |
| *** ramishra_ has joined #openstack-infra | 06:49 | |
| *** salv-orlando has joined #openstack-infra | 06:51 | |
| AJaeger | yolanda: seems that was how far puppet came, no further progress on puppetboard | 06:51 |
| *** rlandy has quit IRC | 06:52 | |
| *** divyah_ has quit IRC | 06:53 | |
| *** sflanigan has joined #openstack-infra | 06:54 | |
| AJaeger | infra-root: Part of puppet not running means that openstack/fuel-plugin-tacker is not imported yet | 06:55 |
| *** salv-orlando has quit IRC | 06:55 | |
| *** IlyaG has quit IRC | 06:56 | |
| *** ihrachys has joined #openstack-infra | 06:56 | |
| *** ramishra_ has quit IRC | 06:56 | |
| *** ramishra_ has joined #openstack-infra | 06:58 | |
| *** scheuran has joined #openstack-infra | 07:00 | |
| *** Guest91520 has quit IRC | 07:00 | |
| *** MarkAtwood has quit IRC | 07:01 | |
| *** camunoz has joined #openstack-infra | 07:03 | |
| *** obondarev has quit IRC | 07:03 | |
| *** obondarev has joined #openstack-infra | 07:04 | |
| *** esker has joined #openstack-infra | 07:05 | |
| *** fedexo has quit IRC | 07:06 | |
| *** nmagnezi has joined #openstack-infra | 07:06 | |
| AJaeger | another broken babel release ;( | 07:06 |
| openstackgerrit | Andreas Jaeger proposed openstack/requirements: Blacklist broken Babel releases https://review.openstack.org/305000 | 07:09 |
| AJaeger | ttx, would be great to get the change above merged ^ I'm failing a bug report now... | 07:09 |
| *** esker has quit IRC | 07:10 | |
| *** fawadkhaliq has quit IRC | 07:10 | |
| *** asettle has quit IRC | 07:12 | |
| *** dtardivel has joined #openstack-infra | 07:13 | |
| *** vincentll has joined #openstack-infra | 07:14 | |
| *** lucas-brno is now known as lucasagomes | 07:14 | |
| AJaeger | s/failing/filing/ | 07:15 |
| *** ramishra_ has quit IRC | 07:16 | |
| openstackgerrit | Vasyl Saienko proposed openstack-infra/project-config: Add ironic cleaning job https://review.openstack.org/304473 | 07:19 |
| *** ramishra_ has joined #openstack-infra | 07:19 | |
| *** jcooley has joined #openstack-infra | 07:21 | |
| openstackgerrit | vikram.choudhary proposed openstack-infra/project-config: Adding neutron-dynamic-routing as neutron's stadium project https://review.openstack.org/268726 | 07:21 |
| *** ybathia has quit IRC | 07:21 | |
| *** infra-red has joined #openstack-infra | 07:22 | |
| *** asettle has joined #openstack-infra | 07:24 | |
| *** esker has joined #openstack-infra | 07:26 | |
| *** ramishra_ has quit IRC | 07:26 | |
| *** ramishra_ has joined #openstack-infra | 07:27 | |
| *** jlanoux has joined #openstack-infra | 07:28 | |
| *** jlanoux has quit IRC | 07:29 | |
| *** vilobhmm11 has quit IRC | 07:29 | |
| *** ihrachys has quit IRC | 07:29 | |
| *** jlanoux has joined #openstack-infra | 07:29 | |
| *** esker has quit IRC | 07:31 | |
| *** asettle has quit IRC | 07:32 | |
| *** vilobhmm11 has joined #openstack-infra | 07:32 | |
| *** ifarkas has joined #openstack-infra | 07:33 | |
| *** daemontool has joined #openstack-infra | 07:34 | |
| *** daemontool has quit IRC | 07:35 | |
| *** daemontool has joined #openstack-infra | 07:35 | |
| *** ramishra_ has quit IRC | 07:36 | |
| *** ramishra_ has joined #openstack-infra | 07:36 | |
| *** jcoufal has joined #openstack-infra | 07:37 | |
| *** matrohon has joined #openstack-infra | 07:38 | |
| *** jordanP has joined #openstack-infra | 07:38 | |
| *** arxcruz has joined #openstack-infra | 07:40 | |
| *** achanda has joined #openstack-infra | 07:41 | |
| *** yarkot_ has quit IRC | 07:42 | |
| openstackgerrit | Vasyl Saienko proposed openstack-infra/project-config: Add ironic cleaning job https://review.openstack.org/304473 | 07:42 |
| masayukig | Hi, I'd like to see test coverage data of projects. Does anyone know where I can get the data (especially post job's)? | 07:42 |
| masayukig | I found there are many '{name}-coverage*' job in project-config/zuul/layout.yaml post phase. But I couldn't find the deliverables.. | 07:43 |
| AJaeger | masayukig: logs.openstack.org/first-two-chars-of-sha/complete-sha - where sha is the sha of your commit | 07:44 |
| ttx | AJaeger: on it | 07:44 |
| AJaeger | thanks, ttx | 07:45 |
| ttx | AJaeger: shouldn't you update the upper-constraints file too, so that the change kicks in asap ? | 07:45 |
| *** sdake_ has joined #openstack-infra | 07:46 | |
| * ttx is not sure | 07:46 | |
| masayukig | AJaeger: Thanks! | 07:46 |
| AJaeger | yeah, probably... | 07:46 |
| AJaeger | ttx, I'll update... | 07:46 |
| ttx | that can't hurt | 07:46 |
| *** andymaier has joined #openstack-infra | 07:46 | |
| *** Jeffrey4l has joined #openstack-infra | 07:47 | |
| *** jaosorior has quit IRC | 07:48 | |
| openstackgerrit | Andreas Jaeger proposed openstack/requirements: Blacklist broken Babel releases https://review.openstack.org/305000 | 07:48 |
| AJaeger | ttx ^ | 07:48 |
| *** jaosorior has joined #openstack-infra | 07:48 | |
| *** sdake has quit IRC | 07:49 | |
| *** jordanP has quit IRC | 07:49 | |
| * masayukig is finding the sha... | 07:49 | |
| *** sdake_ is now known as sdake | 07:50 | |
| ttx | AJaeger: how broken are we currently ? Should I +2A or just +2 waiting for the tests and dims to +2A ? | 07:50 |
| ttx | I can fast-track if we can justify being blocked | 07:50 |
| ttx | Feels like we are only partially broken | 07:50 |
| *** zeih has joined #openstack-infra | 07:52 | |
| *** scheuran has quit IRC | 07:53 | |
| AJaeger | lucky for us I disabled syncing of translations yesterday - and due to puppet not running the enable patch is not active ;) | 07:54 |
| * AJaeger is figuring out whether there's an easy way to work around on infra side... | 07:55 | |
| openstackgerrit | Dong Ma proposed openstack-infra/subunit2sql: Add more db api tests part 3 https://review.openstack.org/305030 | 07:57 |
| *** ramishra_ has quit IRC | 07:59 | |
| *** shardy has joined #openstack-infra | 08:00 | |
| ttx | AJaeger: ok, we can wait a couple hours to get dims review on it then | 08:00 |
| *** placko_ has joined #openstack-infra | 08:00 | |
| *** chem has quit IRC | 08:03 | |
| *** ramishra_ has joined #openstack-infra | 08:04 | |
| *** achanda has quit IRC | 08:06 | |
| *** jistr has joined #openstack-infra | 08:07 | |
| *** esker has joined #openstack-infra | 08:07 | |
| *** chem has joined #openstack-infra | 08:08 | |
| *** shashank_hegde has quit IRC | 08:10 | |
| *** yamahata has quit IRC | 08:11 | |
| *** Nisha has joined #openstack-infra | 08:11 | |
| *** Hal has joined #openstack-infra | 08:12 | |
| *** esker has quit IRC | 08:12 | |
| *** salv-orlando has joined #openstack-infra | 08:12 | |
| *** Hal is now known as Guest30579 | 08:12 | |
| *** gongysh has quit IRC | 08:12 | |
| *** ramishra_ has quit IRC | 08:13 | |
| *** ramishra_ has joined #openstack-infra | 08:14 | |
| *** Douhet has quit IRC | 08:14 | |
| *** Douhet has joined #openstack-infra | 08:15 | |
| vilobhmm11 | AJaeger : ping | 08:16 |
| vilobhmm11 | created a new group chat for cross-project quotas | 08:16 |
| vilobhmm11 | http://docs.openstack.org/infra/system-config/irc.html#access | 08:16 |
| vilobhmm11 | but need to register the channel with ChanServ | 08:17 |
| *** salv-orlando has quit IRC | 08:17 | |
| AJaeger | vilobhmm11: I canot help you with that besides pointing to the documentation that you found already. If you have questions, better ask during US time the IRC experts... | 08:18 |
| vilobhmm11 | what is the ChanServ we are talking about here ? | 08:19 |
| vilobhmm11 | AJaeger : ok | 08:19 |
| *** ramishra_ has quit IRC | 08:21 | |
| *** ramishra_ has joined #openstack-infra | 08:22 | |
| AJaeger | vilobhmm11: ChanServ is the freenode chanserv | 08:23 |
| vilobhmm11 | ok | 08:23 |
| *** amotoki_ has quit IRC | 08:23 | |
| *** salv-orlando has joined #openstack-infra | 08:24 | |
| *** vikram_ has quit IRC | 08:26 | |
| *** mrmartin has joined #openstack-infra | 08:27 | |
| *** jordanP has joined #openstack-infra | 08:27 | |
| *** keedya has quit IRC | 08:28 | |
| *** salv-orlando has quit IRC | 08:28 | |
| *** esker has joined #openstack-infra | 08:29 | |
| *** vikram_ has joined #openstack-infra | 08:29 | |
| *** _degorenko|afk is now known as degorenko | 08:29 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Use pybabel instead of python setup.py https://review.openstack.org/305043 | 08:31 |
| *** esker has quit IRC | 08:33 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Tempest for tripleo CI https://review.openstack.org/295844 | 08:35 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Disable translations completely https://review.openstack.org/305050 | 08:36 |
| AJaeger | ianw, jhesketh, is any of you around to +2 ^, please? | 08:36 |
| *** asettle has joined #openstack-infra | 08:37 | |
| *** ihrachys has joined #openstack-infra | 08:38 | |
| *** vgridnev has quit IRC | 08:41 | |
| *** asettle has quit IRC | 08:41 | |
| *** ivc_ has joined #openstack-infra | 08:44 | |
| *** vgridnev has joined #openstack-infra | 08:47 | |
| AJaeger | infra team, I ninja-approved the change now to buy us time ^ | 08:47 |
| *** ramishra_ has quit IRC | 08:47 | |
| AJaeger | infra team, notice that most translations are already disabled - due to my change yesterday and puppet not running to push the enablement... | 08:48 |
| *** derekh has joined #openstack-infra | 08:48 | |
| AJaeger | But I want to be sure that once puppet works, the big hammer is in place... | 08:48 |
| *** ramishra_ has joined #openstack-infra | 08:48 | |
| *** ramishra_ has quit IRC | 08:50 | |
| *** esker has joined #openstack-infra | 08:50 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Nothing to see here https://review.openstack.org/111011 | 08:50 |
| *** dingyichen has quit IRC | 08:52 | |
| *** e0ne has joined #openstack-infra | 08:53 | |
| *** ramishra_ has joined #openstack-infra | 08:53 | |
| openstackgerrit | Merged openstack-infra/project-config: Disable translations completely https://review.openstack.org/305050 | 08:54 |
| *** esker has quit IRC | 08:55 | |
| openstackgerrit | 10173677 proposed openstack-infra/jenkins-job-builder: test file only https://review.openstack.org/305057 | 08:56 |
| *** ramishra_ has quit IRC | 08:58 | |
| *** ramishra_ has joined #openstack-infra | 08:59 | |
| *** cdent has joined #openstack-infra | 09:03 | |
| *** rossella_s has quit IRC | 09:03 | |
| *** rossella_s has joined #openstack-infra | 09:03 | |
| *** persia has quit IRC | 09:04 | |
| yolanda | ok i'm back now | 09:04 |
| AJaeger | yolanda: puppet did not update the proposal slave - but that was great! | 09:04 |
| *** ramishra_ has quit IRC | 09:05 | |
| yolanda | AJaeger, i see now that was stucked, asking for an ssh key confirmation | 09:05 |
| yolanda | The authenticity of host '2604:e100:1:0:f816:3eff:feb1:40bd (2604:e100:1:0:f816:3eff:feb1:40bd)' can't be established. | 09:05 |
| *** ramishra_ has joined #openstack-infra | 09:05 | |
| AJaeger | yolanda: see above: Babel 2.3.3 broke our translations for log-level files and therefore I ninja pushed a change in to disable translations *completely*. So, once puppet runs, we disable everything - and either wait for new Babel release or work around it... | 09:06 |
| *** persia has joined #openstack-infra | 09:06 | |
| yolanda | AJaeger, i can also force puppet apply on proposal directly | 09:06 |
| yolanda | i think it will be faster | 09:06 |
| AJaeger | yolanda: Why would that be confirmation be needed? | 09:06 |
| AJaeger | yolanda: yes, would be nice. | 09:06 |
| yolanda | AJaeger, i think it's a new host, so puppet doesn't rely on connecting to it for first time | 09:07 |
| *** ociuhandu has quit IRC | 09:07 | |
| *** ramishra_ has quit IRC | 09:10 | |
| *** ramishra_ has joined #openstack-infra | 09:11 | |
| *** esker has joined #openstack-infra | 09:11 | |
| AJaeger | I didn't knew that we updated the proposal slave... | 09:13 |
| AJaeger | but might have missed that | 09:13 |
| yolanda | puppet ran successfully on proposal slave | 09:14 |
| *** kengo_sakai_mba has quit IRC | 09:14 | |
| yolanda | AJaeger, the host i mentioned before was not proposal, it's a mirror | 09:14 |
| yolanda | maybe some mirror was added, but ssh key was not properly accepted | 09:14 |
| yolanda | we are doing that manually normally, just by ssh to the host in puppetmaster | 09:14 |
| yolanda | going to run a full puppet run again, to see if there are no problems now | 09:15 |
| AJaeger | Ah, an npm mirror was added | 09:15 |
| AJaeger | thanks, yolanda | 09:15 |
| *** esker has quit IRC | 09:15 | |
| *** kengo_sakai has quit IRC | 09:17 | |
| *** cdent has left #openstack-infra | 09:18 | |
| *** zhurong has quit IRC | 09:18 | |
| AJaeger | yolanda: translations are indeed disabled - thanks! | 09:18 |
| yolanda | woot! | 09:18 |
| yolanda | going to start with review queue... | 09:19 |
| *** ramishra_ has quit IRC | 09:25 | |
| *** ramishra_ has joined #openstack-infra | 09:26 | |
| *** kzaitsev_mb has joined #openstack-infra | 09:27 | |
| *** jkraj has joined #openstack-infra | 09:29 | |
| *** esker has joined #openstack-infra | 09:32 | |
| *** jkraj has quit IRC | 09:35 | |
| *** jkraj has joined #openstack-infra | 09:36 | |
| *** dtantsur|afk is now known as dtantsur | 09:36 | |
| *** esker has quit IRC | 09:37 | |
| *** nadya has joined #openstack-infra | 09:37 | |
| *** daemontool_ has joined #openstack-infra | 09:38 | |
| *** vikram_ has quit IRC | 09:39 | |
| *** andymccr_ has joined #openstack-infra | 09:40 | |
| *** flepied1 has joined #openstack-infra | 09:40 | |
| *** edwarnicke_ has joined #openstack-infra | 09:40 | |
| *** scheuran has joined #openstack-infra | 09:40 | |
| *** mestery_ has joined #openstack-infra | 09:40 | |
| *** roaet- has joined #openstack-infra | 09:41 | |
| *** ramishra_ has quit IRC | 09:41 | |
| yolanda | AJaeger, puppet ran completely now, i see some puppet errors on some of the servers, so going to check them | 09:41 |
| *** ramishra_ has joined #openstack-infra | 09:42 | |
| *** gsilvis_ has joined #openstack-infra | 09:42 | |
| *** tristanC_ has joined #openstack-infra | 09:43 | |
| *** aloga has left #openstack-infra | 09:44 | |
| openstackgerrit | Merged openstack-infra/project-config: Add tempest variations of Designate DSVM jobs https://review.openstack.org/304677 | 09:44 |
| *** cargonza_ has joined #openstack-infra | 09:44 | |
| *** calebb1 has joined #openstack-infra | 09:47 | |
| *** jmccrory_ has joined #openstack-infra | 09:47 | |
| *** jordanP has quit IRC | 09:47 | |
| *** daemontool has quit IRC | 09:47 | |
| *** mestery has quit IRC | 09:47 | |
| *** flepied has quit IRC | 09:47 | |
| *** sshnaidm has quit IRC | 09:47 | |
| *** sbelous has quit IRC | 09:47 | |
| *** Swanson has quit IRC | 09:47 | |
| *** tristanC has quit IRC | 09:47 | |
| *** chlong has quit IRC | 09:47 | |
| *** SpamapS has quit IRC | 09:47 | |
| *** calebb has quit IRC | 09:47 | |
| *** jmccrory has quit IRC | 09:47 | |
| *** clayton has quit IRC | 09:47 | |
| *** tchaypo has quit IRC | 09:47 | |
| *** dstufft has quit IRC | 09:47 | |
| *** jklare_ has quit IRC | 09:47 | |
| *** dougwig has quit IRC | 09:47 | |
| *** sballe_ has quit IRC | 09:47 | |
| *** mrunge has quit IRC | 09:47 | |
| *** andymccr has quit IRC | 09:47 | |
| *** skraynev has quit IRC | 09:47 | |
| *** electrical has quit IRC | 09:47 | |
| *** cargonza has quit IRC | 09:47 | |
| *** mancdaz has quit IRC | 09:47 | |
| *** edwarnicke has quit IRC | 09:47 | |
| *** fungi has quit IRC | 09:47 | |
| *** gsilvis has quit IRC | 09:47 | |
| *** russellb has quit IRC | 09:47 | |
| *** roaet_ has quit IRC | 09:47 | |
| *** mestery_ is now known as mestery | 09:47 | |
| openstackgerrit | Aparna proposed openstack/diskimage-builder: Install proliantutils in IPA's virtualenv https://review.openstack.org/299116 | 09:48 |
| *** mrunge has joined #openstack-infra | 09:48 | |
| *** dstufft has joined #openstack-infra | 09:48 | |
| *** jmccrory_ is now known as jmccrory | 09:48 | |
| *** cargonza_ is now known as cargonza | 09:48 | |
| *** jklare has joined #openstack-infra | 09:48 | |
| *** sbelous has joined #openstack-infra | 09:48 | |
| *** clayton has joined #openstack-infra | 09:49 | |
| *** salv-orlando has joined #openstack-infra | 09:49 | |
| *** russellb has joined #openstack-infra | 09:50 | |
| *** ramishra_ has quit IRC | 09:50 | |
| *** edwarnicke_ is now known as edwarnicke | 09:50 | |
| *** lucasagomes is now known as lucas-hungry | 09:51 | |
| *** ramishra_ has joined #openstack-infra | 09:51 | |
| openstackgerrit | Boris Bobrov proposed openstack-infra/shade: [WIP] Test with a python-keystoneclient change from git/gerrit https://review.openstack.org/304833 | 09:51 |
| *** sballe_ has joined #openstack-infra | 09:52 | |
| *** jistr has quit IRC | 09:52 | |
| *** electrical has joined #openstack-infra | 09:52 | |
| *** fungi has joined #openstack-infra | 09:52 | |
| *** esker has joined #openstack-infra | 09:53 | |
| *** daemontool_ is now known as daemontool | 09:53 | |
| openstackgerrit | Merged openstack-infra/project-config: Run identity v3 only job in swift https://review.openstack.org/304263 | 09:53 |
| *** salv-orlando has quit IRC | 09:54 | |
| *** skraynev has joined #openstack-infra | 09:54 | |
| *** Swanson has joined #openstack-infra | 09:54 | |
| *** chlong has joined #openstack-infra | 09:54 | |
| *** tchaypo has joined #openstack-infra | 09:55 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Update wanted_ci_job to reflect current status of things https://review.openstack.org/294071 | 09:55 |
| *** jordanP has joined #openstack-infra | 09:56 | |
| *** SpamapS has joined #openstack-infra | 09:56 | |
| *** sshnaidm has joined #openstack-infra | 09:56 | |
| openstackgerrit | Merged openstack-infra/project-config: Run identity v3 only job in cinder https://review.openstack.org/304253 | 09:57 |
| openstackgerrit | Merged openstack-infra/project-config: Run identity v3 only job in nova https://review.openstack.org/304243 | 09:57 |
| *** shashank_hegde has joined #openstack-infra | 09:57 | |
| *** esker has quit IRC | 09:58 | |
| *** kengo_sakai_mba has joined #openstack-infra | 10:01 | |
| *** jcooley has quit IRC | 10:02 | |
| *** shashank_hegde has quit IRC | 10:02 | |
| AJaeger | yolanda: https://github.com/openstack/fuel-plugin-tacker is still empty, the repo was not imported correctly. | 10:03 |
| yolanda | ah, github... ok, will fix it manually | 10:03 |
| AJaeger | could you check this, please? | 10:03 |
| AJaeger | Thanks! | 10:03 |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-dev/pbr: Make from_pip_string a little more robust https://review.openstack.org/305080 | 10:03 |
| *** sdague has joined #openstack-infra | 10:04 | |
| *** ramishra_ has quit IRC | 10:05 | |
| *** ramishra_ has joined #openstack-infra | 10:06 | |
| *** tosky has joined #openstack-infra | 10:07 | |
| *** tosky has quit IRC | 10:08 | |
| *** ramishra_ has quit IRC | 10:08 | |
| *** tosky has joined #openstack-infra | 10:09 | |
| yolanda | AJaeger, fixed, it should have contents on next run | 10:11 |
| *** ramishra_ has joined #openstack-infra | 10:12 | |
| *** esker has joined #openstack-infra | 10:14 | |
| *** kengo_sakai_mba has quit IRC | 10:15 | |
| *** mancdaz has joined #openstack-infra | 10:16 | |
| *** asettle has joined #openstack-infra | 10:18 | |
| *** esker has quit IRC | 10:19 | |
| *** sdake_ has joined #openstack-infra | 10:20 | |
| *** Qiming has quit IRC | 10:20 | |
| *** vilobhmm11 has quit IRC | 10:21 | |
| *** sdake has quit IRC | 10:22 | |
| *** amotoki has joined #openstack-infra | 10:22 | |
| *** asettle has quit IRC | 10:23 | |
| *** dizquierdo has joined #openstack-infra | 10:24 | |
| *** yamamot__ has quit IRC | 10:25 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-reviewday: Fix path for reviewday dependencies https://review.openstack.org/305102 | 10:27 |
| *** nmagnezi has quit IRC | 10:27 | |
| *** kengo_sakai_mba has joined #openstack-infra | 10:28 | |
| *** johnsom_ has joined #openstack-infra | 10:30 | |
| *** nikhil has joined #openstack-infra | 10:30 | |
| *** tojuvone_ has joined #openstack-infra | 10:31 | |
| *** nikhil_k has quit IRC | 10:32 | |
| *** openstackstatus has quit IRC | 10:32 | |
| *** vtapia_ has quit IRC | 10:32 | |
| *** jcoufal_ has joined #openstack-infra | 10:32 | |
| *** tosky_ has joined #openstack-infra | 10:32 | |
| *** vtapia_ has joined #openstack-infra | 10:32 | |
| yolanda | AJaeger, actually i need to test that commands for reviewday, i don't think it will install the dependencies from the right directory | 10:32 |
| *** hdd_ has joined #openstack-infra | 10:33 | |
| *** tosky has quit IRC | 10:33 | |
| *** dims has quit IRC | 10:33 | |
| *** johnsom has quit IRC | 10:33 | |
| *** openstack has joined #openstack-infra | 11:15 | |
| *** amotoki has quit IRC | 11:16 | |
| *** openstack has quit IRC | 11:16 | |
| *** openstack has joined #openstack-infra | 11:31 | |
| *** zeih_ has joined #openstack-infra | 11:33 | |
| *** salv-orl_ has joined #openstack-infra | 11:34 | |
| *** watersoul_ has joined #openstack-infra | 11:34 | |
| *** mylu_ has joined #openstack-infra | 11:35 | |
| *** bkeller` has quit IRC | 11:35 | |
| *** openstackgerrit has quit IRC | 11:35 | |
| *** dteselkin has quit IRC | 11:35 | |
| *** bradjones has quit IRC | 11:35 | |
| *** zeih has quit IRC | 11:35 | |
| *** alex_xu has quit IRC | 11:35 | |
| *** ptoohill has quit IRC | 11:35 | |
| *** amotoki has quit IRC | 11:36 | |
| *** mylu has quit IRC | 11:36 | |
| *** salv-orlando has quit IRC | 11:36 | |
| *** bapalm has quit IRC | 11:36 | |
| *** ekarlso has quit IRC | 11:36 | |
| *** bnemec has quit IRC | 11:36 | |
| *** rcarrillocruz has quit IRC | 11:36 | |
| *** Daviey has quit IRC | 11:36 | |
| *** sileht has quit IRC | 11:36 | |
| *** andrewbogott has quit IRC | 11:36 | |
| *** sabari has quit IRC | 11:36 | |
| *** dosaboy has quit IRC | 11:36 | |
| *** TheJulia has quit IRC | 11:36 | |
| *** EmilienM has quit IRC | 11:36 | |
| *** dtantsur is now known as dtantsur|mtg | 11:36 | |
| *** nmagnezi has quit IRC | 11:37 | |
| *** sdague has quit IRC | 11:37 | |
| *** SpamapS has quit IRC | 11:37 | |
| *** cargonza has quit IRC | 11:37 | |
| *** scheuran has quit IRC | 11:37 | |
| *** oanson has quit IRC | 11:37 | |
| *** links has quit IRC | 11:37 | |
| *** ericksonsantos has quit IRC | 11:37 | |
| *** bswartz has quit IRC | 11:37 | |
| *** toabctl has quit IRC | 11:37 | |
| *** watersoul has quit IRC | 11:37 | |
| *** otherwiseguy has quit IRC | 11:37 | |
| *** trown|outtypewww has quit IRC | 11:37 | |
| *** ianw has quit IRC | 11:37 | |
| *** trown has joined #openstack-infra | 11:37 | |
| *** sabari has joined #openstack-infra | 11:37 | |
| *** TheJulia has joined #openstack-infra | 11:37 | |
| *** Daviey has joined #openstack-infra | 11:37 | |
| *** dosaboy has joined #openstack-infra | 11:38 | |
| *** bnemec has joined #openstack-infra | 11:38 | |
| *** andrewbogott has joined #openstack-infra | 11:38 | |
| dims | AJaeger : sigh! approved the babel blacklist | 11:38 |
| *** cargonza has joined #openstack-infra | 11:39 | |
| *** Nisha has quit IRC | 11:40 | |
| *** nadya has joined #openstack-infra | 11:40 | |
| *** bradjones has joined #openstack-infra | 11:41 | |
| *** bradjones has quit IRC | 11:41 | |
| *** bradjones has joined #openstack-infra | 11:41 | |
| *** bapalm has joined #openstack-infra | 11:41 | |
| *** bkeller` has joined #openstack-infra | 11:41 | |
| *** toabctl has joined #openstack-infra | 11:41 | |
| *** tristanC_ is now known as tristanC | 11:42 | |
| *** ptoohill has joined #openstack-infra | 11:42 | |
| *** amotoki has joined #openstack-infra | 11:42 | |
| *** alex_xu has joined #openstack-infra | 11:42 | |
| *** Nisha has joined #openstack-infra | 11:42 | |
| *** dteselkin has joined #openstack-infra | 11:44 | |
| *** EmilienM has joined #openstack-infra | 11:44 | |
| *** EmilienM is now known as Guest38251 | 11:45 | |
| *** dprince has joined #openstack-infra | 11:45 | |
| AJaeger | dims, thanks. I've already disabled translation sync completely ... | 11:45 |
| *** scheuran has joined #openstack-infra | 11:45 | |
| *** otherwiseguy has joined #openstack-infra | 11:45 | |
| AJaeger | dims: have a look at https://review.openstack.org/#/c/304993/ - all log files are now 11573 files large and have the same content ;( | 11:45 |
| *** scheuran has quit IRC | 11:45 | |
| *** ianw has joined #openstack-infra | 11:46 | |
| AJaeger | s/files/lines/ | 11:46 |
| *** scheuran has joined #openstack-infra | 11:46 | |
| *** Qiming has joined #openstack-infra | 11:47 | |
| *** mwagner_lap has joined #openstack-infra | 11:48 | |
| dims | AJaeger : ack so the job itself did not fail, just bad output | 11:48 |
| *** ekarlso has joined #openstack-infra | 11:48 | |
| *** ericksonsantos has joined #openstack-infra | 11:48 | |
| *** nmagnezi has joined #openstack-infra | 11:49 | |
| *** rcarrillocruz has joined #openstack-infra | 11:49 | |
| *** sileht has joined #openstack-infra | 11:49 | |
| *** oanson has joined #openstack-infra | 11:49 | |
| *** links has joined #openstack-infra | 11:50 | |
| *** SpamapS has joined #openstack-infra | 11:50 | |
| AJaeger | dims: even worse: Our other scripts pushed bad content to our translation server ;( | 11:50 |
| *** openstackgerrit has joined #openstack-infra | 11:50 | |
| dims | oh dear! | 11:50 |
| AJaeger | zanata is AFAIK robust enough with it'S database so that we can easily recover it. | 11:51 |
| *** erlon has joined #openstack-infra | 11:51 | |
| AJaeger | But I didn't want to stress test that ;/ | 11:51 |
| dims | we should probably add a upper limit AJaeger | 11:51 |
| *** sdake is now known as sdake_busy_webin | 11:51 | |
| *** sdague has joined #openstack-infra | 11:51 | |
| *** Guest38251 has quit IRC | 11:51 | |
| dims | when we can verify new babel release, then we can raise the limit | 11:52 |
| AJaeger | yes, one option - or as done in: https://review.openstack.org/305043 - install a specific version... | 11:52 |
| AJaeger | I'm not sure what's the best way ;( | 11:52 |
| AJaeger | At least we have now time to discuss. | 11:53 |
| *** thorst_ has joined #openstack-infra | 11:53 | |
| *** nmagnezi has quit IRC | 11:54 | |
| *** weshay has joined #openstack-infra | 11:55 | |
| *** mtanino has quit IRC | 11:56 | |
| *** _amrith_ is now known as amrith | 11:56 | |
| *** pradk has quit IRC | 11:59 | |
| *** esker has joined #openstack-infra | 11:59 | |
| *** shashank_hegde has joined #openstack-infra | 11:59 | |
| *** zeih_ has quit IRC | 12:03 | |
| *** rfolco_ has joined #openstack-infra | 12:03 | |
| *** kzaitsev_mb has joined #openstack-infra | 12:04 | |
| *** esker has quit IRC | 12:04 | |
| *** shashank_hegde has quit IRC | 12:04 | |
| *** zeih has joined #openstack-infra | 12:07 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/shade: Add new test with betamax for create flavors https://review.openstack.org/302192 | 12:08 |
| *** EmilienM_ has joined #openstack-infra | 12:08 | |
| *** amotoki has quit IRC | 12:08 | |
| pabelanger | morning | 12:09 |
| *** nadya has quit IRC | 12:09 | |
| *** thorst_ has quit IRC | 12:09 | |
| krotscheck | Such scrollback. wow | 12:09 |
| pabelanger | krotscheck: how did you make out with npm mirrors yesterday? | 12:10 |
| *** EmilienM_ is now known as EmilienM | 12:10 | |
| *** EmilienM is now known as Guest50220 | 12:11 | |
| *** Guest50220 has quit IRC | 12:11 | |
| *** Guest50220 has joined #openstack-infra | 12:11 | |
| krotscheck | pabelanger: From what I understand, nibalizer has a screen session open where it's running the initial sync right now, but that there were some hiccups vis-a-vis AFS credentials | 12:11 |
| *** Guest50220 is now known as EmilienM | 12:11 | |
| krotscheck | pabelanger: Things only started moving once I got into daycare mode, so I wasn't able to be present for the discussion. | 12:11 |
| pabelanger | Ah, nice, so sounds like clone is in progress | 12:12 |
| *** zeih has quit IRC | 12:12 | |
| pabelanger | will check back in 3 days :D | 12:12 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Switch to a local dns server https://review.openstack.org/305147 | 12:12 |
| mordred | woot! | 12:12 |
| *** nadya has joined #openstack-infra | 12:12 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/shade: [WIP] Test with a python-keystoneclient change from git/gerrit https://review.openstack.org/304833 | 12:13 |
| *** nmagnezi has joined #openstack-infra | 12:13 | |
| *** yamamoto has joined #openstack-infra | 12:14 | |
| *** zhurong has joined #openstack-infra | 12:15 | |
| *** kengo_sakai_mba has quit IRC | 12:16 | |
| *** zhurong has quit IRC | 12:18 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Actually test stack-update to new tht https://review.openstack.org/304707 | 12:18 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Actually test stack-update to new tht https://review.openstack.org/304707 | 12:19 |
| *** Nisha has quit IRC | 12:19 | |
| *** esker has joined #openstack-infra | 12:21 | |
| *** zhurong has joined #openstack-infra | 12:22 | |
| *** Nisha has joined #openstack-infra | 12:23 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Honor default_network for interface_ip https://review.openstack.org/304569 | 12:25 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Start stamping the has_service debug messages https://review.openstack.org/303853 | 12:25 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Consume floating_ip_source config value https://review.openstack.org/304577 | 12:25 |
| *** esker has quit IRC | 12:25 | |
| *** eharney has joined #openstack-infra | 12:26 | |
| mordred | yolanda: ^^ fixed | 12:26 |
| yolanda | nice, going to review | 12:26 |
| *** aysyd has joined #openstack-infra | 12:27 | |
| *** xyang1 has joined #openstack-infra | 12:27 | |
| *** achanda has joined #openstack-infra | 12:30 | |
| *** Jeffrey4l has quit IRC | 12:31 | |
| *** yamamoto has quit IRC | 12:31 | |
| krotscheck | infra-core: Can anyone pop onto mirror_update and see if there's anything being written to /afs/.openstack.org/mirror/npm? | 12:32 |
| krotscheck | and/or if there's a process with registry-static in the string? | 12:32 |
| *** bswartz has joined #openstack-infra | 12:34 | |
| *** zeih has joined #openstack-infra | 12:35 | |
| *** achanda has quit IRC | 12:36 | |
| *** e0ne has joined #openstack-infra | 12:38 | |
| openstackgerrit | Graham Hayes proposed openstack-infra/project-config: Update Designate repos tags and ACLs https://review.openstack.org/305160 | 12:38 |
| openstackgerrit | Merged openstack/requirements: Blacklist broken Babel releases https://review.openstack.org/305000 | 12:39 |
| yolanda | krotscheck, give me 10 minutes and i can help, just finishing lunch | 12:39 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/diskimage-builder: Updated from global requirements https://review.openstack.org/303416 | 12:42 |
| *** amrith is now known as _amrith_ | 12:42 | |
| *** thorst_ has joined #openstack-infra | 12:42 | |
| *** thorst_ has quit IRC | 12:42 | |
| *** mylu_ has quit IRC | 12:43 | |
| *** mtanino has joined #openstack-infra | 12:43 | |
| *** Jeffrey4l has joined #openstack-infra | 12:44 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/shade: Add new test with betamax for create flavors https://review.openstack.org/302192 | 12:45 |
| *** rodrigods has quit IRC | 12:46 | |
| *** rodrigods has joined #openstack-infra | 12:46 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Add NPM mirror hosting to unified mirrors https://review.openstack.org/303650 | 12:46 |
| krotscheck | doh rebase. | 12:46 |
| yolanda | krotscheck, i am here now, which server do you need to check? | 12:46 |
| yolanda | npm mirror? | 12:46 |
| krotscheck | yolanda: mirror_update | 12:46 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-testr: Updated from global requirements https://review.openstack.org/305208 | 12:47 |
| krotscheck | yolanda: I'm looking for a screen process, something running registry-static, and/or content in /afs/.openstack.org/mirror/npm | 12:47 |
| yolanda | that directory is empty | 12:48 |
| yolanda | krotscheck , puppet apply shall be running there or is disabled? | 12:48 |
| *** Jeffrey4l has quit IRC | 12:49 | |
| *** bdemers has joined #openstack-infra | 12:49 | |
| *** gordc has joined #openstack-infra | 12:49 | |
| krotscheck | yolanda: That's irellevant. The code is there, we just need to do the initial sync. | 12:50 |
| *** achanda has joined #openstack-infra | 12:50 | |
| krotscheck | yolanda: I'll wait for nibs to wake up, he'll be able to help out. | 12:50 |
| yolanda | npm-mirror-update | 12:51 |
| yolanda | Obtaining npm tokens and running registry-static. | 12:51 |
| yolanda | Kerberos initialization for service/npm@OPENSTACK.ORG | 12:51 |
| yolanda | k5start: error getting credentials: Permission denied | 12:51 |
| *** Goneri has joined #openstack-infra | 12:52 | |
| openstackgerrit | Graham Hayes proposed openstack-infra/project-config: Update Designate repos tags and ACLs https://review.openstack.org/305160 | 12:52 |
| *** achanda has quit IRC | 12:52 | |
| *** nelsnelson has quit IRC | 12:52 | |
| *** Nisha has quit IRC | 12:53 | |
| *** jheroux has joined #openstack-infra | 12:53 | |
| *** baoli has joined #openstack-infra | 12:54 | |
| mordred | yolanda: are you looking at nibz's screen? | 12:54 |
| *** aspiers has quit IRC | 12:54 | |
| yolanda | no, i executed the npm update script on the mirror | 12:54 |
| mordred | hrm | 12:54 |
| *** mrmartin has joined #openstack-infra | 12:55 | |
| openstackgerrit | Merged openstack/requirements: Updated from generate-constraints https://review.openstack.org/304969 | 12:55 |
| mordred | yolanda: mind if I poke too? | 12:55 |
| yolanda | sure | 12:55 |
| yolanda | i actually logged out | 12:55 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Initial commit for RPM afs mirror https://review.openstack.org/289520 | 12:55 |
| *** bunting has joined #openstack-infra | 12:56 | |
| mordred | yolanda: it is working for me ... running in screen. let me stop and check something | 12:56 |
| *** baoli_ has joined #openstack-infra | 12:56 | |
| mordred | krotscheck: so - it's downloading Reston ALOT | 12:56 |
| krotscheck | mordred: IS it writing it? | 12:57 |
| *** matt-borland has joined #openstack-infra | 12:57 | |
| mordred | krotscheck: no | 12:57 |
| mordred | of course it isn't | 12:57 |
| krotscheck | mordred: jeblair mentioned that the keytab in use didn't have access to the mirror/npm directory, could you check that? | 12:57 |
| mordred | it has no acls | 12:57 |
| *** rlandy has joined #openstack-infra | 12:57 | |
| mordred | yeah | 12:57 |
| *** julim has joined #openstack-infra | 12:57 | |
| krotscheck | That would make things tricky | 12:57 |
| krotscheck | Oh man, whoever owns that reston library is going to be SUPER EXCITED by their download stats. | 12:58 |
| *** andymaier_ has joined #openstack-infra | 12:58 | |
| krotscheck | mordred: Thankfully it seems to be stuck on one package, and not advancing. | 12:59 |
| *** baoli has quit IRC | 12:59 | |
| *** andymaier_ has quit IRC | 12:59 | |
| *** zz_dimtruck is now known as dimtruck | 12:59 | |
| *** amotoki has joined #openstack-infra | 13:00 | |
| *** shashank_hegde has joined #openstack-infra | 13:00 | |
| *** andymaier has quit IRC | 13:00 | |
| mordred | ok. I've added permissions | 13:00 |
| *** andymaier has joined #openstack-infra | 13:01 | |
| *** kgiusti has joined #openstack-infra | 13:01 | |
| *** yamamoto has joined #openstack-infra | 13:02 | |
| mordred | ok | 13:02 |
| mordred | we are now writing to the AFS dir | 13:02 |
| krotscheck | mordred: Woot! | 13:02 |
| mordred | you can verify by looking at /afs/.openstack.org/mirror/npm | 13:02 |
| krotscheck | mordred: I'd need my own afs cell. | 13:03 |
| *** rossella_s has quit IRC | 13:03 | |
| *** zeih has quit IRC | 13:03 | |
| *** rossella_s has joined #openstack-infra | 13:03 | |
| *** zeih has joined #openstack-infra | 13:03 | |
| mordred | krotscheck: yah. they're awesome | 13:03 |
| *** gordc has quit IRC | 13:04 | |
| *** shashank_hegde has quit IRC | 13:05 | |
| krotscheck | mordred: This is me, trying to install AFS on OSX | 13:06 |
| mordred | krotscheck: isn't there a decently easy-to-install afs client on osc? | 13:07 |
| mordred | krotscheck: isn't there a decently easy-to-install afs client on osx? | 13:07 |
| openstackgerrit | yolanda.robla proposed openstack-infra/shade: Update delete flavor test to work with betamax https://review.openstack.org/303459 | 13:07 |
| *** jaypipes has quit IRC | 13:07 | |
| *** amotoki has quit IRC | 13:07 | |
| krotscheck | mordred I found one, not certain how easy it is. | 13:08 |
| *** jamesmcarthur has joined #openstack-infra | 13:08 | |
| *** woodster_ has joined #openstack-infra | 13:08 | |
| *** mwagner_lap has quit IRC | 13:09 | |
| krotscheck | mordred: Most recent release is for OSX-two-versions-ago | 13:09 |
| mordred | ah. well that sucks | 13:09 |
| mordred | krotscheck: https://www.auristor.com/openafs/client-installer/ | 13:10 |
| *** sdake has joined #openstack-infra | 13:10 | |
| krotscheck | Ohneat | 13:10 |
| krotscheck | Product registration is required. | 13:11 |
| *** tiswanso has joined #openstack-infra | 13:11 | |
| krotscheck | Thankfully, there's Henry Bovik | 13:11 |
| *** sdake_busy_webin has quit IRC | 13:12 | |
| *** sdake_ has joined #openstack-infra | 13:13 | |
| EmilienM | fungi, clarkb: hi, yesterday we talked about periodic pipelines and sending e-mails. It's only for stable branches. In Puppet, we want to run periodic jobs for puppet4, and send emails too. What would you suggest? | 13:14 |
| *** pfallenop has quit IRC | 13:15 | |
| *** sdake has quit IRC | 13:15 | |
| AJaeger | EmilienM: can you use the periodic-stable queue? Best talk first with stable team... | 13:15 |
| EmilienM | AJaeger: ok | 13:15 |
| EmilienM | AJaeger: asked on #openstack-stable | 13:17 |
| *** dimtruck is now known as zz_dimtruck | 13:18 | |
| krotscheck | mordred: I appear to have /afs/. How do I get /afs/.openstack.org/? | 13:18 |
| *** sdake_ is now known as sdake | 13:18 | |
| *** gordc has joined #openstack-infra | 13:18 | |
| *** sdake is now known as sdake_busy | 13:18 | |
| *** infra-red has quit IRC | 13:19 | |
| *** tiswanso has quit IRC | 13:19 | |
| *** tiswanso has joined #openstack-infra | 13:20 | |
| mordred | krotscheck: hrm. it should be advertised in dns ... have you tried just ls /afs/.openstack.org ? | 13:20 |
| openstackgerrit | Boris Bobrov proposed openstack-infra/shade: [WIP] Test with a python-keystoneclient change from git/gerrit https://review.openstack.org/304833 | 13:20 |
| krotscheck | mordred: And suddenly it appears! | 13:20 |
| mordred | woot! | 13:21 |
| AJaeger | woot! | 13:21 |
| AJaeger | ;) | 13:21 |
| mordred | krotscheck: just like back in college, right? | 13:21 |
| krotscheck | mordred: It's been a while. | 13:21 |
| krotscheck | Also, back then I was in the fine arts. | 13:21 |
| *** amitgandhinz has joined #openstack-infra | 13:21 | |
| krotscheck | AFS didn't support drafting tables at that point. | 13:22 |
| *** infra-red has joined #openstack-infra | 13:23 | |
| mordred | krotscheck: blashphemy | 13:23 |
| mordred | krotscheck: your drafting works much better in AFS | 13:23 |
| *** hrubi has quit IRC | 13:23 | |
| *** hdd__ has quit IRC | 13:23 | |
| *** infra-re_ has joined #openstack-infra | 13:24 | |
| krotscheck | mordred: You sure that AFS is PaperFS compliant? Does it support FilingCabinet1900? | 13:24 |
| krotscheck | Well, that folder structure seems to be correct. | 13:25 |
| mordred | krotscheck: dude. it's so old school it uses UDP because TCP hadn't been ratified yet when it was written. I'm sure it supports anything with 1900 in the name | 13:25 |
| * Clint chuckles. | 13:25 | |
| * krotscheck is totally going to try to run AFS on his TRS-80 driven plotter. | 13:26 | |
| openstackgerrit | Vasyl Saienko proposed openstack-infra/devstack-gate: Increase default OS_TEST_TIMEOUT to 1200 https://review.openstack.org/305270 | 13:26 |
| *** jbernard1 is now known as jbernard_ | 13:27 | |
| *** jsavak has joined #openstack-infra | 13:27 | |
| *** infra-red has quit IRC | 13:27 | |
| *** _ari_ has joined #openstack-infra | 13:29 | |
| *** denisra has joined #openstack-infra | 13:30 | |
| *** hrubi has joined #openstack-infra | 13:30 | |
| *** ayoung has quit IRC | 13:31 | |
| *** moravec has quit IRC | 13:32 | |
| krotscheck | mordred: Since you're around.... https://review.openstack.org/#/c/303650/ | 13:32 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/release-tools: Pick last SHA for PREVIOUS_VERSION https://review.openstack.org/305272 | 13:32 |
| krotscheck | That's the otherhalf | 13:32 |
| *** infra-re_ has quit IRC | 13:33 | |
| *** vgridnev_ has joined #openstack-infra | 13:34 | |
| *** denisra has quit IRC | 13:34 | |
| *** rfolco__ has joined #openstack-infra | 13:34 | |
| *** denisra has joined #openstack-infra | 13:34 | |
| *** sheeprine_ has joined #openstack-infra | 13:35 | |
| *** denisra has quit IRC | 13:35 | |
| *** [HeOS] has joined #openstack-infra | 13:35 | |
| *** denisra has joined #openstack-infra | 13:35 | |
| *** infra-red has joined #openstack-infra | 13:36 | |
| *** slagle_ has joined #openstack-infra | 13:36 | |
| *** dosaboy_ has joined #openstack-infra | 13:36 | |
| *** gordc_ has joined #openstack-infra | 13:36 | |
| *** lezbar__ has joined #openstack-infra | 13:36 | |
| *** jamesmcarthur has quit IRC | 13:36 | |
| yolanda | infra-root, can you take a look at https://review.openstack.org/305102 ? it's a fix for reviewday, puppet failing there | 13:37 |
| *** pleia2_ has joined #openstack-infra | 13:38 | |
| *** vgridnev_ has quit IRC | 13:38 | |
| coreyob | anyone know why I'm getting 500s from elasticsearch on logstash.openstack.org? | 13:38 |
| *** zhurong_ has joined #openstack-infra | 13:38 | |
| *** infra-red has quit IRC | 13:39 | |
| *** infra-red has joined #openstack-infra | 13:39 | |
| *** keedya has joined #openstack-infra | 13:39 | |
| *** tonytan4ever has joined #openstack-infra | 13:40 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: run puppet4 openstack integ jobs in periodic-stable https://review.openstack.org/305278 | 13:40 |
| *** therve` has joined #openstack-infra | 13:41 | |
| *** andymaier_ has joined #openstack-infra | 13:41 | |
| clarkb | coreyob: what url? | 13:41 |
| *** MarkAtwood has joined #openstack-infra | 13:42 | |
| coreyob | I just followed one the recheck links: http://logstash.openstack.org/#/dashboard/file/logstash.json?query=message:%5C%22%5BFail%5D%20Couldn't%20ping%20server%5C%22%20AND%20tags:%5C%22grenade.sh.txt%5C%22%20AND%20voting:1&from=864000s | 13:42 |
| *** markmcclain1 has joined #openstack-infra | 13:42 | |
| *** fbo_ has joined #openstack-infra | 13:43 | |
| *** dims_ has joined #openstack-infra | 13:43 | |
| *** jklare_ has joined #openstack-infra | 13:43 | |
| *** manfred has joined #openstack-infra | 13:43 | |
| *** gordc has quit IRC | 13:43 | |
| *** andymaier has quit IRC | 13:43 | |
| *** rlandy has quit IRC | 13:43 | |
| *** aysyd has quit IRC | 13:43 | |
| *** zhurong has quit IRC | 13:43 | |
| *** rfolco_ has quit IRC | 13:43 | |
| *** oanson has quit IRC | 13:43 | |
| *** dosaboy has quit IRC | 13:43 | |
| *** vgridnev has quit IRC | 13:43 | |
| *** dims has quit IRC | 13:43 | |
| *** jklare has quit IRC | 13:43 | |
| *** lezbar has quit IRC | 13:43 | |
| *** LinuxJedi has quit IRC | 13:43 | |
| *** AJaeger has quit IRC | 13:43 | |
| *** HeOS has quit IRC | 13:43 | |
| *** sheeprine has quit IRC | 13:43 | |
| *** pcrews has quit IRC | 13:43 | |
| *** wfoster has quit IRC | 13:43 | |
| *** slagle has quit IRC | 13:43 | |
| *** fbo has quit IRC | 13:43 | |
| *** gtmanfred has quit IRC | 13:43 | |
| *** lindsayk has quit IRC | 13:43 | |
| *** dmellado has quit IRC | 13:43 | |
| *** pleia2 has quit IRC | 13:43 | |
| *** rbradfor has quit IRC | 13:43 | |
| *** markmcclain has quit IRC | 13:43 | |
| *** gmann has quit IRC | 13:43 | |
| *** therve has quit IRC | 13:43 | |
| *** ianychoi has quit IRC | 13:43 | |
| *** manfred is now known as gtmanfred | 13:43 | |
| *** dmellado_ has joined #openstack-infra | 13:43 | |
| *** markmcclain1 is now known as markmcclain | 13:43 | |
| clarkb | if I had to guess we have run out of rams to accomodate all of the timestamps again, but I dont have a computer with keys that can confirm this early in the morning | 13:43 |
| *** [HeOS] is now known as HeOS | 13:43 | |
| *** rbradfor has joined #openstack-infra | 13:44 | |
| *** LinuxJedi has joined #openstack-infra | 13:44 | |
| *** vgridnev has joined #openstack-infra | 13:44 | |
| *** _amrith_ is now known as amrith | 13:44 | |
| openstackgerrit | Merged openstack-infra/system-config: Add NPM mirror hosting to unified mirrors https://review.openstack.org/303650 | 13:44 |
| *** AJaeger has joined #openstack-infra | 13:44 | |
| *** gordc_ is now known as gordc | 13:47 | |
| *** akshai has joined #openstack-infra | 13:48 | |
| *** thiagop has joined #openstack-infra | 13:48 | |
| *** |-paul-| has quit IRC | 13:49 | |
| *** hdd__ has joined #openstack-infra | 13:49 | |
| *** BigWillie has joined #openstack-infra | 13:49 | |
| *** rlandy has joined #openstack-infra | 13:50 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 13:51 | |
| *** ianychoi has joined #openstack-infra | 13:51 | |
| *** sheeprine_ is now known as sheeprine | 13:51 | |
| *** sheeprine has quit IRC | 13:51 | |
| *** sheeprine has joined #openstack-infra | 13:51 | |
| *** gmann has joined #openstack-infra | 13:52 | |
| *** wfoster has joined #openstack-infra | 13:52 | |
| *** pradk has joined #openstack-infra | 13:52 | |
| *** pcrews has joined #openstack-infra | 13:53 | |
| *** zeih has quit IRC | 13:55 | |
| *** thorst_ has joined #openstack-infra | 13:55 | |
| clarkb | infra-root you can check /var/log/elasticsearch/elasticsearch.log on elasticsearch02 | 13:56 |
| *** jsavak has quit IRC | 13:56 | |
| clarkb | or I can maybe check when actually awake. the workaround we have been using is to delete the oldest index | 13:56 |
| *** jsavak has joined #openstack-infra | 13:57 | |
| krotscheck | I.... suddenly can't access any of the mirrors anymore. | 13:58 |
| krotscheck | http://mirror.dfw.rax.openstack.org | 13:59 |
| krotscheck | mordred, yolanda ^^ | 13:59 |
| mordred | I agree with you | 13:59 |
| mordred | I will check | 13:59 |
| mordred | krotscheck: can you propose a revert commit real quick? | 14:00 |
| krotscheck | Bad vhost | 14:00 |
| krotscheck | Yep | 14:00 |
| mordred | AH00526: Syntax error on line 53 of /etc/apache2/sites-enabled/50-mirror.dfw.rax.openstack.org.conf: | 14:00 |
| mordred | Bad Substitute flag, only s///[infq] are supported | 14:00 |
| mordred | krotscheck: fwiw | 14:00 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Revert "Add NPM mirror hosting to unified mirrors" https://review.openstack.org/305292 | 14:01 |
| krotscheck | mordred https://review.openstack.org/305292 | 14:01 |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/project-config: Move gate-grenade-dsvm-neutron-dvr-multinode into check queue https://review.openstack.org/299844 | 14:01 |
| *** yamahata has joined #openstack-infra | 14:02 | |
| *** jsavak has quit IRC | 14:02 | |
| openstackgerrit | Merged openstack-infra/system-config: Revert "Add NPM mirror hosting to unified mirrors" https://review.openstack.org/305292 | 14:02 |
| openstackgerrit | yolanda.robla proposed openstack-infra/shade: Add new test with betamax for create flavors https://review.openstack.org/302192 | 14:02 |
| mordred | I have ninja-merged that - will now kick a puppet run on the mirrors | 14:02 |
| *** jsavak has joined #openstack-infra | 14:02 | |
| *** persia has quit IRC | 14:02 | |
| *** masayukig has quit IRC | 14:02 | |
| *** gmann has quit IRC | 14:02 | |
| *** itsuugo has quit IRC | 14:02 | |
| *** r-mibu has quit IRC | 14:04 | |
| *** markvan has quit IRC | 14:04 | |
| pabelanger | jeblair: so I was thinking this morning, is there anyway to leverage existing AFS mirrors for centos7? eg: /afs/cs.stanford.edu/ftp/mirrors/centos/7 vs us writing our own mirror scripts? | 14:04 |
| krotscheck | mordred https://review.openstack.org/305292 | 14:04 |
| *** ayoung has joined #openstack-infra | 14:04 | |
| *** aspiers has joined #openstack-infra | 14:05 | |
| mordred | krotscheck: mirrrs back | 14:05 |
| mordred | krotscheck: and yes, I force-merged it | 14:05 |
| krotscheck | mordred: Good. | 14:05 |
| *** r-mibu has joined #openstack-infra | 14:06 | |
| mordred | pabelanger: sadly not really | 14:06 |
| mordred | pabelanger: or, wait. I'm wrong | 14:06 |
| mordred | pabelanger: we _could_ leverage it - no idea what its uptime is though | 14:06 |
| krotscheck | pabelanger: Its all symlink based in AFS itself. | 14:07 |
| krotscheck | I mean in the vhost | 14:07 |
| *** gmann has joined #openstack-infra | 14:07 | |
| *** masayukig has joined #openstack-infra | 14:07 | |
| mordred | we could point a vhost at that path | 14:07 |
| pabelanger | mordred: Ya, not sure if we could somehow just clone it to afs01 directly, over using rsync then vos release on our local copy | 14:07 |
| mordred | my main concern would be that our friends at stanford may not be intending for us to hit it | 14:07 |
| *** itsuugo has joined #openstack-infra | 14:08 | |
| mordred | most of our performance from afs comes from our local caches on the mirror servers | 14:08 |
| *** markvan has joined #openstack-infra | 14:08 | |
| pabelanger | that is true | 14:08 |
| *** rbrndt has joined #openstack-infra | 14:08 | |
| mordred | instead of rsync though, we _could_ do a cp -a and then an rsync :) | 14:08 |
| mordred | or we could rsync from the afs path instead of over http | 14:08 |
| pabelanger | right, we'd still have to do some magic for repodata I guess | 14:09 |
| pabelanger | okay, let me play with it more | 14:10 |
| *** thorst_ has quit IRC | 14:10 | |
| *** zz_dimtruck is now known as dimtruck | 14:11 | |
| mordred | my main thing is that I want to make sure we're always releasing an internally consistent repo | 14:11 |
| mordred | and that we're not purging files from it aggresively | 14:11 |
| *** Guest30579 has quit IRC | 14:12 | |
| *** links has quit IRC | 14:13 | |
| *** nelsnelson has joined #openstack-infra | 14:14 | |
| AJaeger | mordred: https://review.openstack.org/304721 is a new test for shade jobs in project-config, could you review it, please? | 14:15 |
| AJaeger | mordred: and https://review.openstack.org/#/c/305122 as well, please | 14:16 |
| fungi | masayukig: i think rbradfor was working on coming up with a dashboard solution for coverage trending across and within projects too, not sure how far he got. i'm hoping it's something that'll integrate with the health dashboard in the future | 14:16 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Remove puppetlabs-concat pin https://review.openstack.org/305304 | 14:17 |
| mordred | AJaeger: let's go for it | 14:17 |
| AJaeger | mordred: thanks for reviewing | 14:18 |
| mordred | yolanda: on the magnum tests above ^^ can we add that as an experimental job first? given that w ehad timeout issues before, seems like a good idea to be able to test this one a few times before enabling | 14:18 |
| yolanda | mordred ok, i can do it | 14:19 |
| fungi | yolanda: still catching up on scrollback, did you figure out what 2604:e100:1:0:f816:3eff:feb1:40bd is? seems to be an address in vexxhost with no reverse dns... nova says it's "mirror.ca-ymq-1.vexxhost.openstack.org" but we only have an ipv4 address published for that in dns so presumably shade is pulling the address from nova instead | 14:20 |
| mordred | fungi: shade/ansible will always use the ip from openstack and does not reference dns at all | 14:20 |
| *** Nisha has joined #openstack-infra | 14:20 | |
| *** jaosorior has quit IRC | 14:21 | |
| *** aysyd has joined #openstack-infra | 14:21 | |
| *** Jeffrey4l has joined #openstack-infra | 14:22 | |
| *** asettle has joined #openstack-infra | 14:22 | |
| yolanda | fungi yes, i was able to manually ssh, and it was a mirror | 14:22 |
| mordred | we should probably add that address to forward dns ... | 14:23 |
| fungi | EmilienM: AJaeger: we've been adding zuul periodic pipelines if they need to e-mail a different mailing list with results (or recommending the plain "periodic" pipeline if they only need to be available from the logserver/health dashboard) | 14:24 |
| fungi | krotscheck: you used to be a draftsman too? we can share terrible stories over beer sometime | 14:24 |
| EmilienM | fungi: emails would be awesome. | 14:25 |
| * fungi still has a drafting table in his lab... "just in case" | 14:25 | |
| EmilienM | fungi: why periodic does not send emails? | 14:25 |
| krotscheck | fungi: Architecture degree. 5 years bent over a drafting table, but never went for the AIA creds. | 14:25 |
| *** Jeffrey4l_ has joined #openstack-infra | 14:25 | |
| *** kus has joined #openstack-infra | 14:25 | |
| *** kus has quit IRC | 14:25 | |
| *** Jeffrey4l has quit IRC | 14:26 | |
| *** Jeffrey4l__ has joined #openstack-infra | 14:26 | |
| anteaya | krotscheck: did you design a shopping mall? | 14:26 |
| anteaya | krotscheck: symphonic hall? | 14:26 |
| fungi | mordred: agreed on the dns addition. i'll see if we have enough open slots to add a record, or figure out one to clean up | 14:26 |
| anteaya | krotscheck: houses? | 14:27 |
| *** asettle has quit IRC | 14:27 | |
| krotscheck | anteaya: Student. So a little of everything. | 14:27 |
| anteaya | krotscheck: nice | 14:27 |
| fungi | EmilienM: the regular "periodic" pipeline is a generic one, no great place to send reports unless we created a special ml for that... don't want to spam the -dev ml | 14:27 |
| *** zhurong_ has quit IRC | 14:28 | |
| fungi | krotscheck: neat, i didn't get a degree in it but i worked as a draftsman (mostly cad operator) for architectural and engineering firms for years | 14:28 |
| *** Nikolay_St has joined #openstack-infra | 14:28 | |
| EmilienM | fungi: maybe we could create a separated ML | 14:28 |
| EmilienM | fungi: the problem is today there is nothing that sends emails except for periodic-stable | 14:28 |
| fungi | terrible memories of hours spent in a converted janitorial closet getting lightheaded from the amonia duplicator fumes running blueprints | 14:28 |
| EmilienM | fungi: the thing I propose does not really fit with stable ML | 14:29 |
| fungi | EmilienM: periodic-stable does sent e-mail. periodic does not | 14:29 |
| fungi | er, does send | 14:29 |
| EmilienM | fungi: right, and I propose periodic send too, to a new ML | 14:29 |
| EmilienM | fungi: in the meantime, I proposed https://review.openstack.org/#/c/305278/ but I don't like using periodic-stable for that | 14:30 |
| fungi | EmilienM: there are several timer-triggered pipelines currently in our zuul configuration: periodic, periodic-qa and periodic-stable | 14:30 |
| EmilienM | fungi: iiuc, only periodic-stable sends emails. | 14:30 |
| EmilienM | fungi: do you suggest me to create a new pipeline that sends emails to a new ML? | 14:30 |
| fungi | oh, right, periodic-qa used to send to the openstack-qa ml | 14:30 |
| *** Jeffrey4l_ has quit IRC | 14:30 | |
| fungi | i think maybe we collapsed that one into periodic recently | 14:30 |
| *** Swami has joined #openstack-infra | 14:31 | |
| *** pleia2_ is now known as pleia2 | 14:31 | |
| *** placko_ has quit IRC | 14:31 | |
| fungi | EmilienM: yeah, if you want an puppet-job-results ml or something which people can subscribe to and receive failure reports for those periodic jobs, that seems fine | 14:31 |
| AJaeger | fungi: yes, seems we collapsed them | 14:32 |
| EmilienM | fungi: I would have prefered a standard ML for all OpenStack projects that want a periodic job sending an email to a common ML and use email fitlers to find their jobs | 14:32 |
| fungi | because the qa team decided at some point the e-mail results were useless and getting ignored anyway | 14:32 |
| EmilienM | but I might be wrong | 14:32 |
| EmilienM | ok so I'll create a new ML and a new pipeline. | 14:33 |
| fungi | EmilienM: a periodic-job-results ml seems like it might be reasonable too, though it's possible other infra reviewers disagree | 14:33 |
| *** mylu has joined #openstack-infra | 14:34 | |
| AJaeger | as long as I don't need to subscribe to that list ;) | 14:34 |
| openstackgerrit | Merged openstack-infra/project-config: Enable designate on shade dsvm jobs https://review.openstack.org/304721 | 14:34 |
| fungi | i don't think having the standard periodic pipeline send to a generic ml and letting people subscribe and filter is bad | 14:34 |
| jeblair | does the health dashboard not work for this? | 14:34 |
| openstackgerrit | yolanda.robla proposed openstack-infra/shade: Add new test with betamax for create flavors https://review.openstack.org/302192 | 14:34 |
| EmilienM | fungi: me neither, I'll propose that | 14:34 |
| *** elo has quit IRC | 14:34 | |
| jeblair | fungi: i agree. my only objection is to more pipelines. | 14:34 |
| *** johnsom_ has quit IRC | 14:35 | |
| EmilienM | jeblair: the dashboard is fine. But it requires to go on it | 14:35 |
| EmilienM | jeblair: emails are notifs, that can help people who take care of our CI | 14:35 |
| *** claudiub|2 has quit IRC | 14:35 | |
| *** matt-borland_ has joined #openstack-infra | 14:35 | |
| *** infra-re_ has joined #openstack-infra | 14:35 | |
| *** elo has joined #openstack-infra | 14:35 | |
| fungi | EmilienM: though if it's just the failure event interrupt you're looking for, maybe the health dashboard could grow rss/atom integration or something? | 14:35 |
| krotscheck | fungi: Yeah, I had a come-to-jesus moment when I realized how many hours that career would be. | 14:35 |
| *** johnsom has joined #openstack-infra | 14:35 | |
| *** thorst_ has joined #openstack-infra | 14:37 | |
| EmilienM | fungi: maybe. But in the meantime, maybe we can use ML. How to proceed to create it on http://lists.openstack.org/cgi-bin/mailman/listinfo ? | 14:37 |
| fungi | krotscheck: still more useful than the math theory degree i never finished when i realized the only endgame there was teaching bratty/hungover college kids stuff they could care less about every day for the rest of my life | 14:37 |
| *** infra-red has quit IRC | 14:37 | |
| *** kgiusti has quit IRC | 14:37 | |
| *** thorst_ has quit IRC | 14:38 | |
| *** pcrews has quit IRC | 14:38 | |
| *** rbradfor has quit IRC | 14:38 | |
| *** keedya has quit IRC | 14:38 | |
| *** gordc has quit IRC | 14:38 | |
| fungi | EmilienM: there's a lists class in our openstack_project module in openstack-infra/system-configit sounds like jeblair was more in favor of adding a generic ml | 14:38 |
| *** trown is now known as trown|brb | 14:38 | |
| fungi | add a stanza to that | 14:38 |
| fungi | it relies on the mailman module, iirc | 14:38 |
| *** dims_ has quit IRC | 14:38 | |
| *** matt-borland has quit IRC | 14:38 | |
| *** dprince has quit IRC | 14:38 | |
| *** rossella_s has quit IRC | 14:38 | |
| *** keedya has joined #openstack-infra | 14:39 | |
| EmilienM | fungi: ok thx | 14:39 |
| fungi | EmilienM: sounds like consensus is building around adding something like a periodic-job-results@lists.openstack.org ml and then configuring the generic "periodic" pipeline to send failure reports there | 14:39 |
| *** gsilvis_ has quit IRC | 14:39 | |
| fungi | or maybe periodic-ci-reports@ or something. i will stop bikeshedding in irc | 14:39 |
| *** gordc has joined #openstack-infra | 14:40 | |
| *** gsilvis has joined #openstack-infra | 14:40 | |
| *** dprince has joined #openstack-infra | 14:40 | |
| * fungi writes it in a text editor 100 times | 14:40 | |
| *** ddieterly has joined #openstack-infra | 14:40 | |
| AJaeger | fungi, that's cheating ;) | 14:40 |
| fungi | yy100p | 14:40 |
| openstackgerrit | Merged openstack-dev/hacking: Run all flake8 checks https://review.openstack.org/276500 | 14:40 |
| openstackgerrit | Merged openstack-dev/hacking: Updated from global requirements https://review.openstack.org/303122 | 14:41 |
| *** dims has joined #openstack-infra | 14:41 | |
| anteaya | ha ha ha | 14:41 |
| AJaeger | fungi, do you remember the slave cleanup discussion we had? The approach we merged is a nop, see https://review.openstack.org/304949 for details. Alternative would be to remove the script completely... | 14:41 |
| fungi | er, ii will stop bikeshedding in irc^]yy99p | 14:41 |
| *** jamesmcarthur has joined #openstack-infra | 14:41 | |
| fungi | AJaeger: oh, yes i saw you proposed that in scrollback and i had started looking at it | 14:41 |
| anteaya | AJaeger: I have that open in a tab | 14:41 |
| fungi | that's actually a lot closer to my original suggestion anyway | 14:41 |
| AJaeger | thanks, fungi and anteaya. | 14:41 |
| AJaeger | fungi: yeah - but now I know why ;) | 14:42 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Nothing to see here https://review.openstack.org/111011 | 14:42 |
| fungi | AJaeger: way simpler ;) | 14:42 |
| AJaeger | unfortunately ;) | 14:42 |
| fungi | it appeases my innate sense of laziness | 14:42 |
| *** rosmaita has joined #openstack-infra | 14:43 | |
| anteaya | I +2'd it, waiting for fungi to review | 14:43 |
| *** ayoung has quit IRC | 14:43 | |
| EmilienM | fungi: is it fine if you're admin of the list? :-) | 14:43 |
| *** rbradfor has joined #openstack-infra | 14:43 | |
| anteaya | it is fine with me if fungi is an admin of the list :) | 14:44 |
| AJaeger | thanks, anteaya and fungi. | 14:44 |
| anteaya | AJaeger: welcome, thanks for following that up | 14:44 |
| *** mtanino_ has joined #openstack-infra | 14:44 | |
| anteaya | AJaeger: I'm curious to see if this helps | 14:44 |
| AJaeger | Won't make much of a difference at the moment since I disabled all translation syncs for now due to broken Babel. | 14:44 |
| *** yarkot_ has joined #openstack-infra | 14:45 | |
| AJaeger | Once pleia2 and clarkb are awake and I'll have some more time, I'd like to discuss how to move forward there. | 14:45 |
| AJaeger | team, for background see also http://lists.openstack.org/pipermail/openstack-i18n/2016-April/002116.html | 14:45 |
| fungi | EmilienM: that's fine by me, sure | 14:46 |
| *** mtanino has quit IRC | 14:46 | |
| pleia2 | I'm awake, but it's bug day today so I've got my hands ful | 14:46 |
| anteaya | AJaeger: well today is bug squash day in #openstack-sprint | 14:46 |
| anteaya | yup, that | 14:46 |
| AJaeger | guess I need to file a bug for it then ;) | 14:47 |
| pleia2 | lol | 14:47 |
| *** erlon has quit IRC | 14:47 | |
| clarkb | is babel broken in a way we can workaround or should we just pin and move on for now? | 14:47 |
| AJaeger | pleia2: I'm running out of time now, so will be back online later | 14:47 |
| anteaya | ha ha ha | 14:47 |
| anteaya | clarkb: I think AJaeger blacklisted it on requirements | 14:47 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/system-config: create periodic-ci-reports mailing-list https://review.openstack.org/305326 | 14:47 |
| openstackgerrit | Graham Hayes proposed openstack-infra/project-config: Update Designate repos tags and ACLs https://review.openstack.org/305160 | 14:48 |
| krotscheck | nibalizer: What is the appropriate ruby string method for replace all instances of string x with string Y in this string? | 14:48 |
| AJaeger | clarkb: see https://review.openstack.org/305043 for one way to workaround it | 14:48 |
| * krotscheck does not know how to ruby | 14:48 | |
| *** jsavak has quit IRC | 14:48 | |
| anteaya | https://review.openstack.org/#/c/305000/ | 14:48 |
| AJaeger | clarkb: and please read my email with details | 14:48 |
| krotscheck | Apparently, replace() just replaces the whole string | 14:48 |
| *** jsavak has joined #openstack-infra | 14:48 | |
| *** trown|brb is now known as trown | 14:48 | |
| mordred | krotscheck: didn't you say you expected the npm mirror to be something liek 200G? | 14:49 |
| *** akshai has quit IRC | 14:49 | |
| *** Swami_ has joined #openstack-infra | 14:49 | |
| *** mtanino has joined #openstack-infra | 14:49 | |
| AJaeger | clarkb: we use python setup.py and pybabel in our translation scripts - I wonder whether we can use only pybabel directly - and then use a pin of the babel version. That'S what 305043 does | 14:49 |
| krotscheck | mordred: The docs say 236G | 14:49 |
| krotscheck | mordred: Is it bigger? | 14:50 |
| *** pcrews has joined #openstack-infra | 14:50 | |
| mordred | smaller | 14:50 |
| krotscheck | mordred: Urm. | 14:50 |
| mordred | we currently have 2G on disk best I can tell | 14:50 |
| krotscheck | mordred: That... doesn't seem right | 14:50 |
| jeblair | fs lq -human | 14:50 |
| jeblair | Volume Name Quota Used %Used Partition | 14:50 |
| jeblair | mirror.npm 95.4G 2.0G 2% 43% | 14:50 |
| jeblair | oh look, there's a -human flag :) | 14:50 |
| jeblair | that helps pre-$coffee | 14:50 |
| *** akshai has joined #openstack-infra | 14:50 | |
| *** rossella_s has joined #openstack-infra | 14:50 | |
| mordred | I'm going to run just registry-static again | 14:51 |
| *** mtanino_ has quit IRC | 14:51 | |
| mordred | it is now downloading more things | 14:51 |
| krotscheck | mordred: The command from the cron file times out after 30 minutes | 14:51 |
| mordred | oh - you know what? the timeout command may have stopped it before | 14:51 |
| mordred | yah | 14:51 |
| *** burgerk has joined #openstack-infra | 14:51 | |
| mordred | lemme run it without that | 14:51 |
| mordred | :) | 14:51 |
| krotscheck | Okies | 14:51 |
| jeblair | whatever you do, don't run the script | 14:51 |
| *** kgiusti has joined #openstack-infra | 14:51 | |
| mordred | it is now running in screen | 14:52 |
| openstackgerrit | Merged openstack-infra/project-config: Rework slave-cleanup https://review.openstack.org/304949 | 14:52 |
| jeblair | because we don't want to vos release yet | 14:52 |
| *** Swami has quit IRC | 14:52 | |
| mordred | without the script or the timeout | 14:52 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: run puppet4 openstack integ jobs in periodic https://review.openstack.org/305278 | 14:52 |
| mordred | I expect it to take 5 hours to run | 14:52 |
| mordred | because it took 30 minutes to get 2G | 14:52 |
| mordred | oh. wait | 14:52 |
| mordred | 50 hours | 14:52 |
| mordred | I expect it to take 2 days | 14:52 |
| jeblair | mordred: having said that -- we might be able to get a jump on the vos release thing by going ahead and running it periodically while we're still doing the initial sync -- so... | 14:53 |
| jeblair | mordred: if you want to do that, i recommend starting a screen on afs01.dfw | 14:53 |
| mordred | jeblair: we could just run the script over and over again - run for 30 minutes, release. run for 30 minutes, release | 14:53 |
| jeblair | mordred: and running 'vos release mirror.npm -localauth' there | 14:53 |
| *** fedexo has joined #openstack-infra | 14:53 | |
| jeblair | mordred: that *might* work :) | 14:53 |
| jeblair | mordred: but if vos release times out, it's a mess to clean up | 14:54 |
| mordred | yah | 14:54 |
| jeblair | so i'd rather not chance it :) | 14:54 |
| mordred | ++ | 14:54 |
| openstackgerrit | Carlos Camacho proposed openstack-infra/tripleo-ci: Removing previously created resources in pingtest and --skip-pingtest-cleanup option https://review.openstack.org/304560 | 14:55 |
| *** andymaier_ has quit IRC | 14:55 | |
| jeblair | mordred: but anyway, as long as we don't mind doing partial mirror releases (which i don't think we do mind since we're not in prod yet), that may speed up the final vos release, which we estimated at about 2.5 days | 14:55 |
| jeblair | or was it 1.5? | 14:55 |
| *** tiswanso has quit IRC | 14:55 | |
| jeblair | i can't remember. days. | 14:55 |
| *** mriedem has joined #openstack-infra | 14:56 | |
| *** esker has joined #openstack-infra | 14:56 | |
| *** esker has quit IRC | 14:56 | |
| *** esker has joined #openstack-infra | 14:56 | |
| mordred | I have a screen going on afs01.dfw doing vos release | 14:57 |
| mordred | and a screen going on mirror-update doing the registry-static | 14:57 |
| *** esker has quit IRC | 14:57 | |
| *** BigWillie has quit IRC | 14:57 | |
| jeblair | yayz | 14:57 |
| fungi | somewhere between o(1) and o(10). trying to guess between orders of magnitude is too much effort, better to just let it go and plan to do other stuff for $WHILE | 14:58 |
| AJaeger | EmilienM: once https://review.openstack.org/#/c/305278 is in, could you send an email to openstack-dev announcing the new mailing list, please? | 14:58 |
| openstackgerrit | Brant Knudson proposed openstack/requirements: Allow maintaining extras from project https://review.openstack.org/303271 | 14:58 |
| EmilienM | AJaeger: I was writting it | 14:58 |
| AJaeger | great, EmilienM ! | 14:58 |
| EmilienM | AJaeger: and wondering which tag to use | 14:58 |
| EmilienM | AJaeger: [all] probably? | 14:58 |
| AJaeger | "[all]" | 14:58 |
| AJaeger | EmilienM: yes ;) | 14:58 |
| *** esker has joined #openstack-infra | 14:58 | |
| EmilienM | AJaeger: sent | 14:58 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Switch to a local dns server https://review.openstack.org/305147 | 14:59 |
| *** therve` is now known as therve | 14:59 | |
| *** jsavak has left #openstack-infra | 14:59 | |
| *** amrith is now known as _amrith_ | 14:59 | |
| AJaeger | bbl | 15:00 |
| *** netsin has quit IRC | 15:00 | |
| *** roxanaghe has joined #openstack-infra | 15:00 | |
| *** BigWillie has joined #openstack-infra | 15:00 | |
| pleia2 | bug day time over in #openstack-sprint :) | 15:01 |
| *** cody-somerville has joined #openstack-infra | 15:02 | |
| *** amitgandhinz has quit IRC | 15:05 | |
| *** roxanaghe has quit IRC | 15:05 | |
| *** maishsk has quit IRC | 15:06 | |
| *** jaypipes has joined #openstack-infra | 15:06 | |
| EmilienM | mhh, OVH cloud is having issues | 15:06 |
| EmilienM | Failed to fetch http://mirror.gra1.ovh.openstack.org/ubuntu/dists/trusty/Release.gpg | 15:06 |
| EmilienM | http://logs.openstack.org/14/304614/5/gate/gate-puppet-openstack-integration-puppet-lint/71c8182/console.html#_2016-04-13_13_54_54_381 | 15:06 |
| openstackgerrit | yolanda.robla proposed openstack-infra/shade: Update delete flavor test to work with betamax https://review.openstack.org/303459 | 15:06 |
| clarkb | EmilienM: that wasnt ovh. the mirros were turned off earlier today | 15:07 |
| *** alex_xu has quit IRC | 15:07 | |
| clarkb | should be fairly universal | 15:07 |
| EmilienM | well, the job just ran this morning | 15:07 |
| clarkb | yes | 15:07 |
| EmilienM | bluebox too | 15:07 |
| EmilienM | ok | 15:07 |
| EmilienM | should I just recheck? | 15:07 |
| clarkb | ya | 15:07 |
| EmilienM | k | 15:07 |
| *** ayoung has joined #openstack-infra | 15:08 | |
| *** sdake_busy has quit IRC | 15:08 | |
| krotscheck | EmilienM: We had a brief hiccup this morning with our mirrors. | 15:08 |
| EmilienM | ok np, just reporting just in case | 15:09 |
| *** doug-fish has joined #openstack-infra | 15:10 | |
| *** amitgandhinz has joined #openstack-infra | 15:10 | |
| *** alex_xu has joined #openstack-infra | 15:11 | |
| *** Jeffrey4l__ has quit IRC | 15:15 | |
| mordred | jeblair: for nodepool-shade-restart ... how about we install shade from source on the nodepool server so that if there is a problem we can fix it without a release churn cycle? | 15:15 |
| *** nmagnezi has quit IRC | 15:15 | |
| *** furlongm has quit IRC | 15:15 | |
| *** harlowja_at_home has joined #openstack-infra | 15:15 | |
| *** furlongm has joined #openstack-infra | 15:16 | |
| *** fedexo has quit IRC | 15:16 | |
| *** pfallenop has joined #openstack-infra | 15:17 | |
| *** e0ne has quit IRC | 15:17 | |
| *** armax has joined #openstack-infra | 15:17 | |
| *** sdake has joined #openstack-infra | 15:17 | |
| *** nikhil has quit IRC | 15:18 | |
| *** nmagnezi has joined #openstack-infra | 15:18 | |
| *** ifarkas has quit IRC | 15:19 | |
| *** harlowja_at_home has quit IRC | 15:20 | |
| *** ddieterly is now known as ddieterly[away] | 15:21 | |
| *** armax has quit IRC | 15:22 | |
| *** scheuran has quit IRC | 15:24 | |
| *** akshai has quit IRC | 15:26 | |
| *** ddieterly[away] is now known as ddieterly | 15:26 | |
| *** esikachev has quit IRC | 15:26 | |
| *** Jeffrey4l__ has joined #openstack-infra | 15:27 | |
| nibalizer | good morning | 15:27 |
| *** Guest12699 has quit IRC | 15:28 | |
| *** Qiming has quit IRC | 15:28 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Only use mirrored cache for git repos that exist https://review.openstack.org/305354 | 15:28 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Update the list of mirrored git repositories https://review.openstack.org/305355 | 15:28 |
| *** Guest14243 is now known as BobBall | 15:28 | |
| *** nikhil has joined #openstack-infra | 15:28 | |
| krotscheck | Hey there, nibalizer. | 15:29 |
| *** achanda has joined #openstack-infra | 15:29 | |
| krotscheck | nibalizer: Mordred got the npm mirror thing sorted. | 15:29 |
| nibalizer | wooo | 15:29 |
| nibalizer | gj mordred | 15:29 |
| mordred | it is currently running | 15:29 |
| mordred | in a screen on mirror-update | 15:29 |
| nibalizer | awesome | 15:29 |
| nibalizer | as root or as user? | 15:30 |
| mordred | root | 15:30 |
| *** armax has joined #openstack-infra | 15:30 | |
| nibalizer | cool | 15:30 |
| nibalizer | also im pretty sure we really do need this https://review.openstack.org/#/c/304318/ | 15:30 |
| mordred | yah | 15:30 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Add NPM mirror hosting to unified mirrors https://review.openstack.org/305356 | 15:30 |
| krotscheck | mordred: Ok, I think ^^ won't kill our mirrors. | 15:31 |
| nibalizer | are we bugdaying in the sprint channel or here or what? | 15:31 |
| *** kmartin has joined #openstack-infra | 15:31 | |
| krotscheck | nibalizer #openstack-sprint | 15:31 |
| fungi | clarkb: what was the package mirror issue? i must have missed it in scrollback | 15:31 |
| clarkb | fungi: bad vhost turned off apache | 15:32 |
| jeblair | mordred: wfm | 15:32 |
| *** jlanoux has quit IRC | 15:32 | |
| fungi | wow, i totally missed that | 15:32 |
| * fungi checks system-config history | 15:32 | |
| *** esker has quit IRC | 15:32 | |
| mordred | fungi: we landed it, it broke, I force-merged the revert and then ran puppet on the mirrors | 15:33 |
| *** dizquierdo has quit IRC | 15:33 | |
| *** esker has joined #openstack-infra | 15:33 | |
| *** Jeffrey4l__ has quit IRC | 15:34 | |
| *** ifarkas has joined #openstack-infra | 15:36 | |
| *** aspiers has quit IRC | 15:37 | |
| fungi | hopefully adding new mirrors is something we do infrequently and with closer scrutiny | 15:37 |
| jeblair | krotscheck: can you summarize why localhost is in the json output from the mirror such that we need to filter/replace it in apache? | 15:37 |
| *** infra-red has joined #openstack-infra | 15:37 | |
| *** minot has joined #openstack-infra | 15:37 | |
| jeblair | fungi: i wonder if we can add an apache configtest to the puppet tests... | 15:37 |
| *** johnsom_ has joined #openstack-infra | 15:37 | |
| *** baoli has joined #openstack-infra | 15:37 | |
| mordred | jeblair: I was having the same thought | 15:37 |
| *** aspiers has joined #openstack-infra | 15:37 | |
| krotscheck | jeblair: Yep- so, the npm client does not understand relative links. It pulls the tarball from the absolute path it receives. | 15:37 |
| *** aysyd_ has joined #openstack-infra | 15:37 | |
| krotscheck | jeblair: Which is fine, if you're mirroring to one host. But we're actually mirroring to lots of hosts with different names. | 15:38 |
| krotscheck | So I was looking for a good 'default' string to use. | 15:38 |
| krotscheck | localhost seemed as good as any. | 15:38 |
| *** nikhil has quit IRC | 15:38 | |
| krotscheck | Though, now that I think about it, I bet there's a package whose name is localhost. | 15:38 |
| *** fitoduarte has joined #openstack-infra | 15:38 | |
| jeblair | krotscheck: oh interesting. | 15:38 |
| jeblair | krotscheck: we won't be using it! | 15:38 |
| krotscheck | jeblair: Well, I'm pretty certain I can change the susbstitution to include the protocol prefix. | 15:38 |
| krotscheck | That'll make things a bit easier | 15:38 |
| *** yarkot_ has quit IRC | 15:38 | |
| *** Nisha has quit IRC | 15:38 | |
| *** nikhil has joined #openstack-infra | 15:38 | |
| *** openstack has joined #openstack-infra | 15:51 | |
| *** aspiers has joined #openstack-infra | 15:51 | |
| *** Rockyg has joined #openstack-infra | 15:51 | |
| *** tosky has quit IRC | 15:51 | |
| *** electrical has joined #openstack-infra | 15:51 | |
| *** pcrews has joined #openstack-infra | 15:51 | |
| krotscheck | jeblair: I _think_ ^^ will do it. | 15:52 |
| *** kozhukalov has joined #openstack-infra | 15:52 | |
| anteaya | jeblair: thank you | 15:52 |
| jeblair | how many meeting channels do we have now? | 15:52 |
| *** mikelk has quit IRC | 15:52 | |
| *** sballe_ has joined #openstack-infra | 15:53 | |
| *** openstack has joined #openstack-infra | 15:54 | |
| *** larainema has joined #openstack-infra | 15:54 | |
| *** bpokorny has joined #openstack-infra | 15:54 | |
| jeblair | anteaya: thanks, i forgot about cp | 15:55 |
| *** links has joined #openstack-infra | 15:55 | |
| *** roxanaghe has joined #openstack-infra | 15:56 | |
| * anteaya nods | 15:56 | |
| *** vaidy has joined #openstack-infra | 15:56 | |
| *** Jeffrey4l__ has joined #openstack-infra | 15:57 | |
| *** Rocky_g has joined #openstack-infra | 15:57 | |
| *** Rockyg has quit IRC | 15:57 | |
| *** Sukhdev has joined #openstack-infra | 15:57 | |
| *** Rocky_g has quit IRC | 15:58 | |
| *** Rocky_g has joined #openstack-infra | 15:58 | |
| *** apoorvad has joined #openstack-infra | 16:00 | |
| fungi | i often forget about -cp until i get pinged for the cross-project-specs meeting | 16:00 |
| *** erlon has joined #openstack-infra | 16:00 | |
| *** openstack has joined #openstack-infra | 16:08 | |
| *** moravec has joined #openstack-infra | 16:08 | |
| *** nmagnezi has quit IRC | 16:08 | |
| *** amotoki has joined #openstack-infra | 16:08 | |
| krtaylor | pleia2, sure, I'll make sure it's up to date | 16:09 |
| *** lakshmiS has joined #openstack-infra | 16:09 | |
| fungi | but twisted blows up if we point it at an rr that also has aaaa records in it | 16:09 |
| pleia2 | krtaylor: thank you! | 16:09 |
| *** sdake_ has joined #openstack-infra | 16:09 | |
| openstackgerrit | Niall Bunting proposed openstack-infra/project-config: Glance_store swift functional tests https://review.openstack.org/305378 | 16:09 |
| rcarrillocruz | wootz, shade designate is coming https://review.openstack.org/#/c/302770/ | 16:09 |
| rcarrillocruz | mordred: Shrews , passed tests after gate jobs having designate enabled | 16:09 |
| mordred | rcarrillocruz: woot! | 16:10 |
| jeblair | fungi: i'll ask em about that | 16:10 |
| fungi | thanks! | 16:10 |
| openstackgerrit | Brant Knudson proposed openstack-dev/pbr: Correct server test https://review.openstack.org/305377 | 16:10 |
| *** sdake has quit IRC | 16:11 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-hound: Add foundation for functional tests https://review.openstack.org/305376 | 16:12 |
| *** esikachev has quit IRC | 16:12 | |
| *** ams_ has joined #openstack-infra | 16:13 | |
| anteaya | rbradfor: we are doing an infra bug squash in the #openstack-sprint channel, are you able to update this story at all? https://storyboard.openstack.org/#!/story/2000343 | 16:13 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-stackalytics: Add foundation for functional tests https://review.openstack.org/305382 | 16:13 |
| anteaya | rbradfor: also as a note, links to our test logs only work for 30 days then the logs are deleted, perhaps putting the log output in a paste might be a better idea for things longer than 30 days | 16:13 |
| jeblair | anteaya: i think it's 4 months | 16:14 |
| *** akshai has joined #openstack-infra | 16:14 | |
| *** Jeffrey4l__ has joined #openstack-infra | 16:14 | |
| jeblair | (but yeah) | 16:14 |
| anteaya | jeblair: ah yes, sorry 4 months for us, 30 days for third party ci | 16:14 |
| fungi | jeblair: if you're looking for a concise description of the twisted ssl+ipv6 issue, the end of section 4.1.1 at http://supybook.fealdia.org/devel/ mentions it | 16:14 |
| anteaya | trying to keep things separate in my head is getting harder and harder | 16:14 |
| *** sdake has joined #openstack-infra | 16:15 | |
| fungi | "Warning: Due to a bug in python-twisted-names, after you change the network driver to Twisted, even non-SSL servers having AAAA records stop working. Freenode is a good example of this serious problem." | 16:15 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-translation_checksite: Add foundation for functional tests https://review.openstack.org/305384 | 16:15 |
| *** aspiers_ has quit IRC | 16:15 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-stackalytics: Add foundation for functional tests https://review.openstack.org/305382 | 16:16 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: update tag ACLs for all official projects https://review.openstack.org/298866 | 16:16 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-hound: Add foundation for functional tests https://review.openstack.org/305376 | 16:16 |
| *** sdake_ has quit IRC | 16:17 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-ciwatch: Add foundation for functional tests https://review.openstack.org/305373 | 16:17 |
| *** tosky has joined #openstack-infra | 16:19 | |
| *** nadya has quit IRC | 16:20 | |
| openstackgerrit | Spencer Krum proposed openstack-infra/system-config: Create unprivileged user to run npm mirroring https://review.openstack.org/304891 | 16:22 |
| *** newjersey has quit IRC | 16:23 | |
| jeblair | mordred: i think 299679 has a problem. i left a comment. how should nat_destination be plumbed into shade? in the create server call, or via the occ cloud kwargs? | 16:23 |
| *** ybathia has joined #openstack-infra | 16:24 | |
| anteaya | jesusaur: have you any update on this story? https://storyboard.openstack.org/#!/story/2000271 | 16:24 |
| *** amphiprions has quit IRC | 16:24 | |
| mordred | jeblair: cloud kwargs | 16:24 |
| *** Nisha has joined #openstack-infra | 16:25 | |
| mordred | but let me go read your comment | 16:25 |
| *** esker has quit IRC | 16:25 | |
| mordred | jeblair: oh! I agree with your review | 16:25 |
| *** esker has joined #openstack-infra | 16:27 | |
| *** brad_behle has joined #openstack-infra | 16:27 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/shade: Update delete flavor test to work with betamax https://review.openstack.org/303459 | 16:27 |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/release-tools: Add tool that annotates bug numbers with details from LP https://review.openstack.org/296470 | 16:27 |
| jeblair | nibalizer: inline comment in 891 | 16:28 |
| *** matrohon has quit IRC | 16:28 | |
| nibalizer | jeblair: fixing, lol | 16:29 |
| openstackgerrit | Spencer Krum proposed openstack-infra/system-config: Create unprivileged user to run npm mirroring https://review.openstack.org/304891 | 16:29 |
| jesusaur | anteaya: I don't recall the details of that issue, but I suspect it is no longer the case | 16:29 |
| *** _amrith_ is now known as amrith | 16:30 | |
| jesusaur | anteaya: though it seems easy enough to test, I could probably find some time this afternoon to deploy a kibana host and report back in the bug | 16:30 |
| anteaya | jesusaur: are you willing to add an update to that story, perhaps close it if you feel it is appropriate? | 16:30 |
| anteaya | jesusaur: oh, would you? that would be lovely, thank you | 16:30 |
| jesusaur | sure | 16:31 |
| anteaya | :) | 16:32 |
| *** jistr has quit IRC | 16:32 | |
| brad_behle | sc68cal_: Thanks for reviewing https://review.openstack.org/#/c/304871/ Do you have a minute? I'm interested in your thoughts on how a plugin should indicate to devstack to create the initial network (even if neutron l3 agent isn't enabled) | 16:33 |
| brad_behle | sc68cal_: I see you're working on a patch to move the L3 related code out of neutron-legacy, which seems to be a good start. | 16:34 |
| *** mrmartin has quit IRC | 16:35 | |
| *** esikachev has joined #openstack-infra | 16:35 | |
| *** thorst_ has joined #openstack-infra | 16:35 | |
| *** lezbar__ has quit IRC | 16:36 | |
| *** tosky has quit IRC | 16:36 | |
| *** lezbar has joined #openstack-infra | 16:36 | |
| *** mkwiek has joined #openstack-infra | 16:37 | |
| *** lezbar has quit IRC | 16:38 | |
| *** ociuhandu has quit IRC | 16:38 | |
| *** lezbar has joined #openstack-infra | 16:38 | |
| *** shashank_hegde has joined #openstack-infra | 16:39 | |
| *** rcernin has quit IRC | 16:39 | |
| *** ybathia has quit IRC | 16:39 | |
| *** sputnik13 has joined #openstack-infra | 16:41 | |
| *** ddieterly has quit IRC | 16:42 | |
| *** hdd__ has quit IRC | 16:42 | |
| sc68cal_ | brad_behle: yeah. My thinking is, make the code where we make the actual Neutron API calls for networking modular | 16:42 |
| pabelanger | clarkb: Did you mention bumping our dib builds for nodepool earlier? | 16:42 |
| jeblair | fungi: mquin suggested putting the bot behind znc or something else that can 'talk competently to the rest of the internet for it' | 16:42 |
| sc68cal_ | brad_behle: right now it's all tied up in the l3 agent creation, then some static route injection on the node, yadda yadda. It's messy | 16:42 |
| jeblair | fungi: i hadn't thought of that. but it sounds like a good idea under the circumstances. | 16:43 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool: Pass extended network information in to occ/shade https://review.openstack.org/299679 | 16:43 |
| brad_behle | sc68cal_: Yeah, I second the "messy" part, it took me a while just to figure out what it was trying to do | 16:43 |
| fungi | jeblair: proxying an irc bot through a bouncer? novel approach. seems like it should work | 16:43 |
| fungi | just have it listen on localhost | 16:43 |
| jeblair | fungi: yeah. i raised one eyebrow. | 16:44 |
| fungi | run one for statusbot as well i guess, as they both have this issue | 16:44 |
| jeblair | i don't think statusbot has this issue | 16:44 |
| brad_behle | sc68cal_: I talked to Sam-I-Am over in networking-ovn, and he seemed okay with putting something like this in for now, and then reworking it once your neutron-legacy L3 refactoring was done | 16:44 |
| fungi | oh, i thought it was also based on supybot | 16:44 |
| *** thorst_ has quit IRC | 16:44 | |
| *** vikram_ has quit IRC | 16:45 | |
| jeblair | no, only meetbot. all the others are homegrown python-irc | 16:45 |
| *** mrmartin has joined #openstack-infra | 16:45 | |
| *** esker has quit IRC | 16:45 | |
| fungi | indeed! i just checked its requirements.txt | 16:45 |
| brad_behle | sc68cal_: I'm trying to get that initial network/subnet/router created by devstack for a gate check job for OVN that doesn't use the neutron L3 agent | 16:45 |
| mordred | I'm surprised to say I think that's a good idea | 16:45 |
| fungi | so agreed, only meetbot needs it, statusbot is simply a victim of running on the same server with the same /etc/hosts hack | 16:45 |
| *** thingee has quit IRC | 16:46 | |
| jeblair | is znc multi-user? :) | 16:47 |
| rcarrillocruz | mordred: i would have sworn we supported url to upload glance images? | 16:47 |
| * rcarrillocruz recalls some change related to it | 16:47 | |
| *** imcsk8 is now known as imcsk8|lunch | 16:47 | |
| *** thingee has joined #openstack-infra | 16:48 | |
| fungi | jeblair: i have never used an irc bouncer, so no idea. having my irc client on a shell server has always been a lot more convenient | 16:48 |
| jeblair | mordred: ^? | 16:48 |
| *** sdake has quit IRC | 16:49 | |
| rcarrillocruz | ah | 16:49 |
| rcarrillocruz | heat | 16:49 |
| rcarrillocruz | create a heat template from a url | 16:49 |
| mordred | jeblair: the bouncer | 16:49 |
| mordred | rcarrillocruz: yah - we do not yet support "import from url" for glance | 16:50 |
| mordred | rcarrillocruz: it's on my todo list to add support for | 16:50 |
| fungi | jeblair: it's also possible that an irc bouncer knows how to reconnect without tanking the client connection, so this could also solve a lot of our stale topic problems in meeting channels | 16:50 |
| sc68cal_ | brad_behle: I think the short term fix is: "CREATE_INITIAL_NETWORKS=False" then build it up in your plugin | 16:50 |
| sc68cal_ | it sucks but it's the quickest way to unblock you | 16:51 |
| rcarrillocruz | k, i''ve assigned the story to myself, in case you haven't tackled and i need to find things to work on | 16:51 |
| *** rosmaita has left #openstack-infra | 16:51 | |
| sc68cal_ | jeblair: yeah it is | 16:51 |
| jeblair | fungi: yeah, the meetbot may never think it's offline. we may have the opposite problem -- we might have meetings that keep going forever if the bouncer gets disconnected -- except! we added the timeout so we should be good. | 16:51 |
| fungi | rcarrillocruz: mordred: what's the context? support for it in shade, or support for it in glance itself isn't implemented? | 16:51 |
| *** sc68cal_ is now known as sc68cal | 16:51 | |
| *** trown is now known as trown|lunch | 16:51 | |
| mordred | fungi: shade | 16:51 |
| rcarrillocruz | fungi: shade | 16:51 |
| jeblair | sc68cal_: cool, we might be able to use one bouncer for multiple bots then | 16:52 |
| mordred | fungi, rcarrillocruz: as it turns out there is an undocument glance task in v2 that does import from url | 16:52 |
| mordred | and v1 has it directly | 16:52 |
| rcarrillocruz | triaging shade open bugs now, a story about 'glance allows it, shade should' | 16:52 |
| mordred | so it's 'supportable' in shade | 16:52 |
| sc68cal | jeblair: cool. I ran a znc bouncer for a couple people so just ping me with q's | 16:52 |
| brad_behle | sc68cal: I get your point, but that's a lot of code duplication for the network/subnet/router create. | 16:52 |
| *** mylu has quit IRC | 16:52 | |
| fungi | mordred: isn't "undocumented glance task" a redundant phrase? | 16:52 |
| jeblair | sc68cal: thanks! | 16:52 |
| mordred | fungi: yup | 16:52 |
| jeblair | it's double undocumented | 16:52 |
| fungi | double secret undocumented | 16:53 |
| mordred | it works well on all of our clouds except for rackspace | 16:53 |
| *** Swami_ has quit IRC | 16:53 | |
| mordred | it's extra cool - because it'll do qcow2->raw conversion for you | 16:53 |
| fungi | oh, i thought that was a different task | 16:53 |
| mordred | so we could build qcow2, upload to tarballs, and then have clouds import-from that url | 16:53 |
| *** infra-red has joined #openstack-infra | 16:54 | |
| mordred | and only have to upload a qcow2 | 16:54 |
| fungi | but yes, that's super helpful for uploading smaller qcow2 images (except rackspace i guess) | 16:54 |
| mordred | (obviously rax is different for all of this) | 16:54 |
| *** salv-orl_ has quit IRC | 16:54 | |
| *** amotoki has quit IRC | 16:54 | |
| *** salv-orlando has joined #openstack-infra | 16:55 | |
| *** ddieterly has joined #openstack-infra | 16:55 | |
| openstackgerrit | Merged openstack-infra/puppet-translation_checksite: Puppet module to provide a translation checksite https://review.openstack.org/274701 | 16:55 |
| *** jogo has quit IRC | 16:56 | |
| *** rp_ has joined #openstack-infra | 16:57 | |
| *** achanda has joined #openstack-infra | 16:57 | |
| *** esikachev has quit IRC | 16:58 | |
| *** Sukhdev has quit IRC | 16:58 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-ciwatch: Add foundation for functional tests https://review.openstack.org/305373 | 16:58 |
| rp_ | I have a problem in submitting my code for review | 16:58 |
| sc68cal | brad_behle: I know, but the current code is too tightly bound to the neutron l3 agent at this point | 16:58 |
| rp_ | I get "fatal: ICLA contributor agreement requires current contact information." | 16:58 |
| timrc | fungi: An irclog now exists for this glance task, it's officially documented. | 16:59 |
| sc68cal | brad_behle: and I don't want to keep adding stuff that keeps it around longer | 16:59 |
| *** BobBall is now known as BobBall-AWOL | 16:59 | |
| *** salv-orlando has quit IRC | 16:59 | |
| *** ihrachys has quit IRC | 16:59 | |
| fungi | timrc: neat | 16:59 |
| fungi | so that's how it works | 17:00 |
| timrc | fungi: Yep! | 17:00 |
| brad_behle | sc68cal: Okay, I'll see if I can get the network/subnet/router create code to run in the OVN plugin for now. Thanks. | 17:00 |
| *** e0ne has quit IRC | 17:01 | |
| sc68cal | brad_behle: thanks. Sorry to make more work for you | 17:01 |
| *** jogo has joined #openstack-infra | 17:01 | |
| fungi | rp_: you need to put something in at https://review.openstack.org/#/settings/contact | 17:01 |
| brad_behle | sc68cal: No worries, I understand. | 17:01 |
| *** jamesmcarthur has quit IRC | 17:01 | |
| fungi | rp_: it doesn't have to be accurate/complete, though the preferred e-mail address there does need to match your foundation member account's primary address at www.openstack.org | 17:02 |
| fungi | rp_: see http://docs.openstack.org/infra/manual/developers.html#account-setup for details | 17:02 |
| *** rossella_s has quit IRC | 17:03 | |
| *** rossella_s has joined #openstack-infra | 17:03 | |
| rp_ | I tried updating those details, it fails with an error | 17:04 |
| *** infra-red has quit IRC | 17:04 | |
| fungi | rp_: this is not surprising. did you do the other things mentioned in the account setup process first? | 17:04 |
| rp_ | fungi: it had submitted earlier with couple of patches | 17:04 |
| *** salv-orlando has joined #openstack-infra | 17:05 | |
| *** mrmartin has quit IRC | 17:05 | |
| rp_ | but today my sandbox crashed and had to bring up a new one | 17:05 |
| fungi | rp_: it's possible you never completely set up other accounts and were contributing to a project which doesn't have cla enforcement turned on | 17:05 |
| fungi | rp_: or it's possible you've accidentally (and incompletely) set up a new account | 17:05 |
| rp_ | fungi: I have only one account with 2 emails ids | 17:06 |
| *** aspiers_ has joined #openstack-infra | 17:06 | |
| fungi | rp_: and you joined the openstack foundation as an individual member per the setup instructions? | 17:06 |
| *** esikachev has joined #openstack-infra | 17:06 | |
| fungi | rp_: and checked that your gerrit preferred e-mail address matches your foundation membership primary e-mail address? | 17:06 |
| rp_ | fungi: yes, they match | 17:07 |
| fungi | rp_: and you're a foundation member, not a community member? | 17:07 |
| fungi | rp_: there's a screenshot at https://ask.openstack.org/en/question/56720/cannot-store-contact-information-when-updating-info-in-openstack-gerrit/ which shows how to tell what your foundation member system account type is | 17:08 |
| fungi | er, https://ask.openstack.org/question/56720 is a shorter url to teh same | 17:08 |
| *** ramishra_ has quit IRC | 17:09 | |
| *** Swami has joined #openstack-infra | 17:09 | |
| rp_ | fungi: let me check | 17:10 |
| *** mrmartin has joined #openstack-infra | 17:10 | |
| *** mrmartin has quit IRC | 17:10 | |
| *** imcsk8|lunch is now known as imcsk8 | 17:10 | |
| crinkle | can i query openstack-health to see what non-voting jobs are failing? | 17:13 |
| *** abregman has quit IRC | 17:14 | |
| jeblair | mordred: 299679 looks good -- should we go ahead and manually install shade and that on nodepool and restart? | 17:14 |
| *** aspiers_ has quit IRC | 17:14 | |
| *** jsavak has quit IRC | 17:16 | |
| mordred | jeblair: yes! EXCEPT ... | 17:17 |
| mordred | duhnduhnduhn ... | 17:17 |
| *** andreww is now known as xarses | 17:17 | |
| mordred | I'm going to the dentist in 30 minutes | 17:17 |
| *** tphummel has joined #openstack-infra | 17:18 | |
| *** mylu has joined #openstack-infra | 17:18 | |
| *** ddieterly is now known as ddieterly[away] | 17:19 | |
| jeblair | mordred: ruh roh: http://logs.openstack.org/79/299679/3/check/gate-nodepool-python27-db/50d050c/console.html#_2016-04-13_16_56_55_720 | 17:20 |
| *** shashank_hegde has quit IRC | 17:21 | |
| mordred | jeblair: FASCINATING | 17:21 |
| *** ddieterly[away] is now known as ddieterly | 17:21 | |
| *** jsavak has joined #openstack-infra | 17:21 | |
| *** mrmartin has joined #openstack-infra | 17:22 | |
| jeblair | mordred: that don't rightly make no sense. | 17:22 |
| mordred | jeblair: I agree | 17:22 |
| *** fitoduarte has quit IRC | 17:22 | |
| *** mrmartin has quit IRC | 17:22 | |
| mordred | jeblair: I will investigate whence I return from yonder dentist | 17:22 |
| *** achanda_ has joined #openstack-infra | 17:22 | |
| jeblair | okiedokie | 17:23 |
| *** mriedem1 has joined #openstack-infra | 17:23 | |
| mordred | jeblair: I mean, unless you investigate in the mean time | 17:23 |
| *** mriedem has quit IRC | 17:23 | |
| *** achanda_ has quit IRC | 17:23 | |
| mordred | jeblair: I don't want to steal all the investigation fun | 17:23 |
| openstackgerrit | Zara proposed openstack-infra/python-storyboardclient: Get storyboard resources by properties other than ID https://review.openstack.org/305411 | 17:24 |
| anteaya | mordred: I hope your dentist is in a good mood | 17:25 |
| mordred | anteaya: me too | 17:26 |
| fungi | and not jack nicholson in little shop of horrors | 17:26 |
| *** kgiusti has left #openstack-infra | 17:26 | |
| anteaya | and not that | 17:26 |
| anteaya | happy dentist | 17:26 |
| Clint | or steve martin in any of the movies he was a dentist in | 17:26 |
| *** achanda has quit IRC | 17:27 | |
| anteaya | with an aura of healing blue light | 17:27 |
| anteaya | and using gentle words | 17:27 |
| fungi | Clint is right. it was steve martin | 17:27 |
| fungi | nicholson was the dental patient | 17:27 |
| *** mriedem1 is now known as mriedem | 17:27 | |
| *** sridhar_ram has joined #openstack-infra | 17:27 | |
| fungi | er, i'm conflating the original and remake now | 17:28 |
| Clint | but steve martin was also a non-littleshop dentist | 17:28 |
| *** Rockyg has joined #openstack-infra | 17:28 | |
| *** tosky has joined #openstack-infra | 17:28 | |
| Zara | well now I've got the littleshop dentist song stuck in my head | 17:28 |
| * anteaya pictures mordred in a glow of healing blue light | 17:28 | |
| fungi | john shaner was apparently cast as dr. farb in the original | 17:29 |
| *** Rockyg has quit IRC | 17:30 | |
| fungi | bill murray played nicholson's role in the remake | 17:30 |
| *** aspiers_ has joined #openstack-infra | 17:30 | |
| mordred | there were two? | 17:31 |
| *** Rockyg has joined #openstack-infra | 17:31 | |
| *** Rocky_g has quit IRC | 17:31 | |
| mordred | ahhhh | 17:31 |
| fungi | i always preferred the 1960 version | 17:31 |
| fungi | one of nicholson's best performances | 17:32 |
| *** doug-fis_ has quit IRC | 17:32 | |
| fungi | also, of the two, the original had teh better (darker) ending | 17:32 |
| *** hdd__ has joined #openstack-infra | 17:32 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Run tempest tests with periodic jobs https://review.openstack.org/297038 | 17:33 |
| mordred | yah. the musical also has the dark ending | 17:33 |
| *** mrmartin has joined #openstack-infra | 17:35 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/project-config: Make Infra's puppet beaker jobs voting https://review.openstack.org/305416 | 17:39 |
| *** doug-fish has joined #openstack-infra | 17:40 | |
| *** Rocky_g has joined #openstack-infra | 17:41 | |
| openstackgerrit | Steve Martinelli proposed openstack/requirements: bump keystoneauth upper constraints https://review.openstack.org/305418 | 17:42 |
| *** rp_ has quit IRC | 17:42 | |
| *** mylu has quit IRC | 17:43 | |
| *** e0ne has joined #openstack-infra | 17:43 | |
| *** davideagnello has joined #openstack-infra | 17:43 | |
| *** shashank_hegde has joined #openstack-infra | 17:44 | |
| *** woodster_ has quit IRC | 17:44 | |
| *** Rockyg has quit IRC | 17:45 | |
| *** e0ne has quit IRC | 17:45 | |
| *** jsavak has quit IRC | 17:45 | |
| *** e0ne has joined #openstack-infra | 17:45 | |
| *** jamesmcarthur has joined #openstack-infra | 17:45 | |
| *** mylu has joined #openstack-infra | 17:46 | |
| *** jordanP has quit IRC | 17:46 | |
| *** hdd_ has joined #openstack-infra | 17:47 | |
| *** degorenko is now known as _degorenko|afk | 17:49 | |
| *** hdd__ has quit IRC | 17:50 | |
| *** mrmartin has quit IRC | 17:52 | |
| *** ddieterly is now known as ddieterly[away] | 17:53 | |
| *** sambetts is now known as sambetts|afk | 17:53 | |
| *** Rocky_g has quit IRC | 17:55 | |
| *** Jeffrey4l__ has quit IRC | 17:55 | |
| *** fhubik has quit IRC | 17:55 | |
| *** pvaneck has joined #openstack-infra | 17:55 | |
| *** sdake has joined #openstack-infra | 17:55 | |
| *** Rockyg has joined #openstack-infra | 17:56 | |
| *** flwang1 has joined #openstack-infra | 17:56 | |
| *** tiswanso has joined #openstack-infra | 17:56 | |
| *** trown|lunch is now known as trown | 17:57 | |
| *** nadya has joined #openstack-infra | 17:59 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/infra-specs: Mark Translation spec as finished https://review.openstack.org/305426 | 18:00 |
| *** tqtran has joined #openstack-infra | 18:02 | |
| *** rbrndt has quit IRC | 18:02 | |
| *** lindsayk_ has quit IRC | 18:03 | |
| *** rcrit has joined #openstack-infra | 18:03 | |
| *** jsavak has joined #openstack-infra | 18:03 | |
| *** kzaitsev_mb has quit IRC | 18:03 | |
| *** armax has quit IRC | 18:04 | |
| rcrit | have a question about experimental tasks. When do they run? | 18:05 |
| rcrit | I added a test to openstack-dev/devstack in experimental but I don't see it executed | 18:05 |
| rcrit | (or any of the others in experimental, so it isn't just my test) | 18:06 |
| *** Jeffrey4l__ has joined #openstack-infra | 18:07 | |
| rcarrillocruz | docaedo: heya, around | 18:07 |
| jeblair | rcrit: leave a comment on the change with the text 'check experimental' | 18:07 |
| rcarrillocruz | ? | 18:07 |
| *** persia has joined #openstack-infra | 18:07 | |
| rcrit | ah, of course | 18:07 |
| rcrit | jeblair, is it considered rude to add that to some random change? | 18:07 |
| rcrit | e.g. I just want to see hwo well it works, can I update someone's existing, pending patch with that? | 18:08 |
| JayF | rcrit: if you did that, I'd be explicit in the comment that you're just testing your experimental job and the results of the test should be disregarded | 18:08 |
| openstackgerrit | Merged openstack-infra/infra-manual: Fix "contributing on behalf of a company" formatting https://review.openstack.org/302372 | 18:08 |
| rcrit | is there a better way to do it? | 18:08 |
| * JayF wouldn't mind you doing that on one of my changes, but I don't have anything up to devstack and don't know the general etiquette | 18:08 | |
| JayF | I mean, the alternative is throwing up a minor docs change (I always try to find something to actually improve so it can land eventually) | 18:09 |
| rcrit | I can make some trivial change in devstack, mark it as DON'T COMMIT/REVIEW, and add it there I guess. Just wondering what best practice is. | 18:09 |
| JayF | then you have your own patch to test on | 18:09 |
| fungi | rcrit: certainly doesn't seem rude to me. it doesn't hold up normal testing/reviewing/approving, doesn't add votes to a change, and the results are separated out by pipeline | 18:09 |
| rcrit | oh, heh, now that I think about it, I actually have a pending change, I can add it to my own patch | 18:09 |
| rcrit | ok cool | 18:10 |
| fungi | also if i've added an experimental job, i often pick some changes to try it on. especially ones i think it might fail | 18:10 |
| *** Nikolay_St has joined #openstack-infra | 18:10 | |
| openstackgerrit | Merged openstack-infra/puppet-reviewday: Fix path for reviewday dependencies https://review.openstack.org/305102 | 18:11 |
| rcarrillocruz | folks, i'm triaging https://storyboard.openstack.org/#!/story/2000541 , i see the jobs have passed latest runs on at least jenkins01 and jenkins02, recentish | 18:11 |
| rcarrillocruz | i'm leaning to mark it as fixed, but would like to know what the fix was about to reference it | 18:11 |
| rcarrillocruz | story topic is "NPM gate tests failing with "npm ERR! registry error parsing json"" | 18:11 |
| *** maishsk has joined #openstack-infra | 18:11 | |
| rcarrillocruz | rings any bell? | 18:12 |
| openstackgerrit | Merged openstack-infra/puppet-reviewday: Order of the class parameters is refactored https://review.openstack.org/295357 | 18:13 |
| anteaya | EmilienM: are you able to update this story at all? https://storyboard.openstack.org/#!/story/2000247 | 18:14 |
| openstackgerrit | Cedric Brandily proposed openstack/requirements: Bump python-openstacksdk to 0.8.5 https://review.openstack.org/305430 | 18:15 |
| EmilienM | anteaya: if you look at the patch, it's depending of some work that I'm not working on, krotscheck might give an update | 18:15 |
| anteaya | EmilienM: I just saw that the patch was abandoned, and wondered if that fact was enough to update the story at all | 18:16 |
| fungi | rcarrillocruz: that may have been inadvertently opened against the wrong project | 18:16 |
| EmilienM | anteaya: abandonned by who? | 18:16 |
| EmilienM | the patch is here https://review.openstack.org/#/c/253616/ | 18:17 |
| fungi | rcarrillocruz: general "these jobs are failing" bugs get reported and tracked against the openstack-gate project in launchpad, not on storyboard | 18:17 |
| EmilienM | and it depends on https://review.openstack.org/#/c/242161/7 | 18:17 |
| anteaya | EmilienM: the patch linked in the story is https://review.openstack.org/#/c/178026/ | 18:17 |
| *** dizquierdo has joined #openstack-infra | 18:17 | |
| anteaya | which is abandoned | 18:17 |
| EmilienM | anteaya: ok let me finish my current task and I'll take care of this one, a bit later | 18:18 |
| anteaya | EmilienM: thank you | 18:18 |
| fungi | rcarrillocruz: while some job failures are due to actual bugs in an infra project, the vast majority are more incidents than bugs and get filed against openstack-gate so that elastic-recheck can track and link them | 18:18 |
| *** cody-somerville has quit IRC | 18:19 | |
| anteaya | anyone know Pavel Bondar's irc nick? hoping to get him to update this story: https://storyboard.openstack.org/#!/story/2000470 | 18:20 |
| *** brucet has joined #openstack-infra | 18:21 | |
| *** vilobhmm11 has joined #openstack-infra | 18:22 | |
| *** vilobhmm11 has quit IRC | 18:23 | |
| *** vilobhmm11 has joined #openstack-infra | 18:23 | |
| *** nadya has quit IRC | 18:23 | |
| *** esker has joined #openstack-infra | 18:23 | |
| anteaya | pavel_bondar seems to be his nick according to launchpad but he doesn't seem to be on irc | 18:23 |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config: Add experimental job for shade and magnum https://review.openstack.org/305434 | 18:25 |
| *** Rockyg has quit IRC | 18:26 | |
| *** gordc_ is now known as gordc | 18:26 | |
| *** jamesmcarthur has quit IRC | 18:27 | |
| *** Rockyg has joined #openstack-infra | 18:27 | |
| *** vilobhmm11 has quit IRC | 18:27 | |
| openstackgerrit | Brant Knudson proposed openstack-dev/pbr: Correct server test https://review.openstack.org/305377 | 18:27 |
| *** baoli_ has joined #openstack-infra | 18:27 | |
| *** vilobhmm11 has joined #openstack-infra | 18:28 | |
| *** Nikolay_St has quit IRC | 18:28 | |
| openstackgerrit | Merged openstack-infra/statusbot: Post message to twitter on 'ok' status https://review.openstack.org/263099 | 18:28 |
| *** baoli__ has joined #openstack-infra | 18:29 | |
| *** eharney has quit IRC | 18:29 | |
| *** kzaitsev_mb has joined #openstack-infra | 18:29 | |
| *** baoli has quit IRC | 18:29 | |
| *** ericksonsantos has quit IRC | 18:32 | |
| *** rbrndt has joined #openstack-infra | 18:32 | |
| *** ericksonsantos has joined #openstack-infra | 18:32 | |
| *** baoli_ has quit IRC | 18:32 | |
| *** mylu has quit IRC | 18:33 | |
| krotscheck | indeed | 18:33 |
| krotscheck | mordred: Mirrors, now with more npm | 18:33 |
| krotscheck | And less vhost fail | 18:33 |
| fungi | vhost fail is rarely enjoyable | 18:34 |
| *** mylu has joined #openstack-infra | 18:34 | |
| *** johnsom_ has joined #openstack-infra | 18:36 | |
| *** tonytan_brb has joined #openstack-infra | 18:36 | |
| *** jordanP has joined #openstack-infra | 18:36 | |
| *** hdd__ has joined #openstack-infra | 18:36 | |
| *** dgonzalez has quit IRC | 18:37 | |
| *** mat128 has quit IRC | 18:37 | |
| *** katyafervent_awa has quit IRC | 18:37 | |
| *** hogepodge has quit IRC | 18:38 | |
| *** mnencia has quit IRC | 18:38 | |
| *** bexelbie has quit IRC | 18:38 | |
| *** daemontool_ has joined #openstack-infra | 18:38 | |
| *** hdd_ has quit IRC | 18:38 | |
| *** wfoster has quit IRC | 18:38 | |
| *** flaper87 has quit IRC | 18:38 | |
| *** lucas-brno has quit IRC | 18:38 | |
| *** charz has quit IRC | 18:38 | |
| *** HeOS_ has quit IRC | 18:38 | |
| *** Jeffrey4l__ has quit IRC | 18:39 | |
| *** johnsom has quit IRC | 18:39 | |
| *** tonytan4ever has quit IRC | 18:39 | |
| *** bapalm has quit IRC | 18:39 | |
| *** afazekas has quit IRC | 18:39 | |
| *** cschwede has quit IRC | 18:39 | |
| *** jpeeler has quit IRC | 18:39 | |
| *** hamzy has quit IRC | 18:39 | |
| *** jzerebecki has quit IRC | 18:39 | |
| *** hockeynut has quit IRC | 18:39 | |
| *** Vivek has quit IRC | 18:39 | |
| *** sbadia has quit IRC | 18:39 | |
| *** csaba has quit IRC | 18:39 | |
| *** bunting has quit IRC | 18:39 | |
| *** yamamoto has quit IRC | 18:39 | |
| *** tphummel has quit IRC | 18:39 | |
| *** rossella_s has quit IRC | 18:39 | |
| *** thingee has quit IRC | 18:39 | |
| *** weshay has quit IRC | 18:39 | |
| *** openstackgerrit has quit IRC | 18:39 | |
| *** daemontool has quit IRC | 18:39 | |
| *** coreyob has quit IRC | 18:39 | |
| *** jtomasek has quit IRC | 18:39 | |
| *** tsufiev has quit IRC | 18:39 | |
| *** amrith has quit IRC | 18:39 | |
| krotscheck | fungi It really is | 18:40 |
| krotscheck | Hrm, those rewrite rules seem to not be working. | 18:40 |
| *** HeOS_ has joined #openstack-infra | 18:40 | |
| fungi | krotscheck: which ones, where? | 18:40 |
| krotscheck | fungi: The ones for the new npm mirror. | 18:41 |
| *** jzerebecki has joined #openstack-infra | 18:41 | |
| krotscheck | fungi: I'm checking | 18:41 |
| krotscheck | fungi: Example-> http://mirror.dfw.rax.openstack.org/npm/k works, http://mirror.dfw.rax.openstack.org/npm/kademlia doesn't | 18:41 |
| fungi | you're testing them out locally? | 18:41 |
| fungi | oh | 18:41 |
| *** dalgaaf has quit IRC | 18:41 | |
| *** DuncanT has quit IRC | 18:41 | |
| *** erlon has quit IRC | 18:41 | |
| *** lindsayk has joined #openstack-infra | 18:41 | |
| fungi | constructed similarly to the one we have for pypi? | 18:41 |
| *** doug-fish has quit IRC | 18:42 | |
| *** doug-fish has joined #openstack-infra | 18:43 | |
| *** amrith has joined #openstack-infra | 18:43 | |
| fungi | something like RewriteRule ^/npm/([^/])([^/]*)(/.*)?$ /npm/$1/$1$2$3 [L] | 18:43 |
| *** salv-orlando has quit IRC | 18:43 | |
| *** doug-fis_ has joined #openstack-infra | 18:44 | |
| *** Rockyg has quit IRC | 18:44 | |
| *** bapalm has joined #openstack-infra | 18:44 | |
| *** tphummel has joined #openstack-infra | 18:44 | |
| *** dgonzalez has joined #openstack-infra | 18:44 | |
| *** Rockyg has joined #openstack-infra | 18:44 | |
| *** mnencia has joined #openstack-infra | 18:44 | |
| *** afazekas has joined #openstack-infra | 18:44 | |
| *** mat128 has joined #openstack-infra | 18:44 | |
| *** hockeynut has joined #openstack-infra | 18:44 | |
| fungi | krotscheck: i have to assume you're testing locally as http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/templates/mirror.vhost.erb doesn't reflect there's any npm rewriting in place right now | 18:45 |
| *** cschwede has joined #openstack-infra | 18:45 | |
| *** bexelbie has joined #openstack-infra | 18:45 | |
| *** coreyob has joined #openstack-infra | 18:45 | |
| *** charz has joined #openstack-infra | 18:45 | |
| *** flaper87 has joined #openstack-infra | 18:45 | |
| *** vilobhmm11 has quit IRC | 18:45 | |
| jeblair | krotscheck, mordred: i kicked off another intermediate vos release | 18:45 |
| krotscheck | fungi: Oh! Derp. I totally thought that https://review.openstack.org/#/c/305356/ landed. | 18:45 |
| *** sbadia has joined #openstack-infra | 18:46 | |
| jeblair | erm | 18:46 |
| *** tonytan_brb has quit IRC | 18:46 | |
| jeblair | it sure looks like the cronjob is running on mirror-update | 18:46 |
| *** erlon has joined #openstack-infra | 18:47 | |
| jeblair | nibalizer: did you release the lock? | 18:47 |
| krotscheck | ....uh-oh | 18:47 |
| nibalizer | jeblair: ... yes i did | 18:47 |
| jeblair | nibalizer: ok, i'll take care of it | 18:47 |
| nibalizer | it seemed like mordred an yall had it in hand | 18:47 |
| nibalizer | sorry! | 18:47 |
| *** katyafervent_awa has joined #openstack-infra | 18:47 | |
| *** doug-fish has quit IRC | 18:47 | |
| *** lucasagomes has joined #openstack-infra | 18:48 | |
| *** vilobhmm11 has joined #openstack-infra | 18:48 | |
| *** Vivek has joined #openstack-infra | 18:48 | |
| *** tsufiev has joined #openstack-infra | 18:48 | |
| *** tonytan4ever has joined #openstack-infra | 18:48 | |
| *** Vivek is now known as Guest59521 | 18:48 | |
| *** gabriel-bezerra has joined #openstack-infra | 18:49 | |
| *** dalgaaf has joined #openstack-infra | 18:49 | |
| *** armax has joined #openstack-infra | 18:50 | |
| *** nadya has joined #openstack-infra | 18:50 | |
| *** Jeffrey4l__ has joined #openstack-infra | 18:51 | |
| *** jtomasek has joined #openstack-infra | 18:51 | |
| *** esikachev has quit IRC | 18:51 | |
| *** rossella_s has joined #openstack-infra | 18:51 | |
| *** hamzy has joined #openstack-infra | 18:52 | |
| jeblair | mordred, krotscheck, nibalizer: okay, the lock is being held in a root screen in one window, and the mirror script is running in another window of the same screen | 18:52 |
| krotscheck | Cool. | 18:52 |
| *** weshay has joined #openstack-infra | 18:52 | |
| jeblair | mordred, krotscheck, nibalizer: meanwhile, i also have a vos release running in in a screen on afs01.dfw | 18:52 |
| *** jamesmcarthur has joined #openstack-infra | 18:52 | |
| nibalizer | jeblair: awesome | 18:52 |
| *** jpeeler has joined #openstack-infra | 18:52 | |
| nibalizer | sorry i released it without saying something in channel | 18:52 |
| jeblair | np, it all worked out | 18:53 |
| *** csaba has joined #openstack-infra | 18:53 | |
| *** ddieterly[away] has quit IRC | 18:53 | |
| nibalizer | i was a bit unclear on exactly how much of it was being handled by you and mordred | 18:54 |
| nibalizer | so i stopped working last night before dealing with afs groups | 18:54 |
| *** amotoki has joined #openstack-infra | 18:55 | |
| *** openstackgerrit has joined #openstack-infra | 18:55 | |
| jeblair | nibalizer, fungi, clarkb: want to take a look at https://review.openstack.org/305356 ? | 18:55 |
| *** thorst_ has joined #openstack-infra | 18:56 | |
| *** pvaneck_ has joined #openstack-infra | 18:57 | |
| *** pvaneck has quit IRC | 18:58 | |
| *** woodster_ has joined #openstack-infra | 18:58 | |
| *** fitoduarte has joined #openstack-infra | 18:58 | |
| fungi | krotscheck: jeblair: does 305356 need some manual config testing to avoid the issues from earlier? | 18:58 |
| nibalizer | i see /afs/.openstack.org/mirror/npm/ as having write for the npm-mirror group, but i dont see where service.npm is in the npm-mirror group | 18:58 |
| nibalizer | when i pts examine npm-mirror all i see is mordred | 18:58 |
| fungi | i can hand-patch it on, say, the vexxhost mirror and run apachectl configtest there real quick or something | 18:58 |
| krotscheck | fungi: The issue was that mod_substitute is weird about regex separator characters, and didn't correctly interpret \/ as an escaped / | 18:59 |
| fungi | as long as apache doesn't get reloaded, should be safe | 18:59 |
| krotscheck | fungi: Thankfully it also allows other kinds of separators, i.e. | | 18:59 |
| *** amotoki has quit IRC | 19:00 | |
| fungi | actually, testing on mirror.ord.rax.o.o should be entirely safe :/ | 19:00 |
| fungi | i need to check back and see where they got with those traceroutes i provided | 19:00 |
| jeblair | nibalizer: http://paste.openstack.org/show/493985/ | 19:00 |
| *** wfoster has joined #openstack-infra | 19:00 | |
| *** eharney has joined #openstack-infra | 19:01 | |
| *** thorst_ has quit IRC | 19:01 | |
| *** sdake_ has joined #openstack-infra | 19:01 | |
| nibalizer | jeblair: thanks | 19:02 |
| nibalizer | one day afs, one day I will grok you | 19:02 |
| *** lindsayk_ has joined #openstack-infra | 19:02 | |
| fungi | Syntax OK | 19:03 |
| *** vilobhmm11 has quit IRC | 19:03 | |
| *** MarkAtwood has quit IRC | 19:03 | |
| *** sdake has quit IRC | 19:03 | |
| *** vilobhmm11 has joined #openstack-infra | 19:04 | |
| krotscheck | woot | 19:04 |
| EmilienM | anteaya: https://review.openstack.org/#/c/242161/ is not abandonned | 19:04 |
| EmilienM | anteaya: so I'll work on https://review.openstack.org/#/c/253616/ once 242161 is landed. | 19:04 |
| krotscheck | fungi: Did I understand it correctly that you applied the new vhost to mirror.ord? | 19:05 |
| *** DuncanT has joined #openstack-infra | 19:05 | |
| fungi | krotscheck: i did, reloaded just now but... | 19:05 |
| fungi | 2001:470:e0d6:0:8200:bff:fe67:1b0f - - [13/Apr/2016:19:04:01 +0000] "GET /npm/kademlia HTTP/1.1" 404 426 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:44.0) Gecko/20100101 Firefox/44.0 Iceweasel/44.0" | 19:05 |
| krotscheck | Hrm. 404 | 19:05 |
| *** kzaitsev_mb has quit IRC | 19:05 | |
| fungi | should we add a rewrite.log | 19:06 |
| fungi | ? | 19:06 |
| krotscheck | fungi: Let me see if I can figure out what is going on a local v,. | 19:06 |
| krotscheck | vm | 19:06 |
| anteaya | EmilienM: I'm looking at this story: https://storyboard.openstack.org/#!/story/2000247 | 19:07 |
| *** e0ne has quit IRC | 19:07 | |
| anteaya | EmilienM: the story links to https://review.openstack.org/#/c/178026/ | 19:07 |
| * krotscheck could have sworn that worked. | 19:07 | |
| EmilienM | anteaya: me too | 19:07 |
| anteaya | EmilienM: which is abandoned | 19:07 |
| EmilienM | anteaya: yes please read last comment: in favor of https://review.openstack.org/253616 | 19:07 |
| EmilienM | and https://review.openstack.org/253616 is WIP | 19:07 |
| EmilienM | until 242161 is landed | 19:08 |
| anteaya | the last comment I see was added 2015-04-28 at 02:11:07 | 19:08 |
| anteaya | on https://storyboard.openstack.org/#!/story/2000247 | 19:09 |
| anteaya | EmilienM: where are you wanting me to look at the last comment? | 19:09 |
| *** esikachev has joined #openstack-infra | 19:09 | |
| EmilienM | anteaya: last comment on https://review.openstack.org/#/c/178026/ | 19:09 |
| *** jsavak has quit IRC | 19:10 | |
| anteaya | EmilienM: can you add a comment to the story? | 19:10 |
| *** Sukhdev has joined #openstack-infra | 19:10 | |
| EmilienM | a comment for what? | 19:10 |
| anteaya | pointing at the reviews you would like the story to know about | 19:10 |
| EmilienM | it's WIP? it's mentionned in Gerrit | 19:10 |
| *** jsavak has joined #openstack-infra | 19:10 | |
| fungi | krotscheck: trace3 level from mod_rewrite http://paste.openstack.org/show/493986/ | 19:10 |
| *** ddieterly has joined #openstack-infra | 19:10 | |
| anteaya | can you add a comment to the story pointing to the reviews that are not abandoned that you want the story viewers to know about? | 19:11 |
| fungi | krotscheck: looks like it's entering the perdir condition at least | 19:11 |
| krotscheck | fungi: huhn- it's not getting picked up at all. | 19:11 |
| krotscheck | Oh, wasn't there a mod_rewrite bug about different layers of nested rewrites? | 19:11 |
| fungi | krotscheck: oh... maybe so | 19:12 |
| fungi | krotscheck: was the directory container on this just for convenience? | 19:12 |
| fungi | ahh, for the non-rewrite bits | 19:12 |
| krotscheck | fungi: It was. Also, because it seemed like the rewrite rules could stand to be isolated. | 19:12 |
| *** jordanP has quit IRC | 19:13 | |
| fungi | krotscheck: wait! i copied poorly | 19:13 |
| *** ivc_ has quit IRC | 19:13 | |
| fungi | i got the rewriteconds but skipped the rewriterile | 19:13 |
| fungi | rewriterule | 19:13 |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/puppet-kibana: Remove support for Kibana2 https://review.openstack.org/305448 | 19:14 |
| fungi | krotscheck: now http://mirror.ord.rax.openstack.org/npm/k/kademlia/ is returning content | 19:14 |
| fungi | so lgtm | 19:14 |
| fungi | sorry about that | 19:14 |
| krotscheck | fungi Rmoeve the /k | 19:14 |
| fungi | krotscheck: it redirected | 19:14 |
| krotscheck | fungi: Ah, there we go | 19:14 |
| krotscheck | sweet | 19:14 |
| krotscheck | Let's see if I can install stuff with the npm client :) | 19:14 |
| fungi | yeah, i copied the post-redirect url, so obviously the url i pasted here is a bad example ;_ | 19:15 |
| krotscheck | Righto | 19:15 |
| fungi | krotscheck: approved, should be on its way to the servers briefly | 19:16 |
| openstackgerrit | Brant Knudson proposed openstack-dev/pbr: Correct server test https://review.openstack.org/305377 | 19:16 |
| krotscheck | Well, it succeeded at installing kademlia, plus a couple of other related packages before it died on underscore.string, which hasn't synced yet. | 19:16 |
| krotscheck | So, that's encouraging | 19:17 |
| openstackgerrit | Brant Knudson proposed openstack-dev/pbr: Correct server test https://review.openstack.org/305377 | 19:17 |
| anteaya | jeblair: would you care to update this story you created? https://storyboard.openstack.org/#!/story/2000068 | 19:18 |
| fungi | krotscheck: sounds as thorough a test as we can confirm for now | 19:18 |
| fungi | krotscheck: and more importantly, doesn't seem to break the bits we're relying on for our other mirrors | 19:18 |
| krotscheck | fungi: Yep. I'll do a smoke test on the various npm-using projects once everything has synced. | 19:18 |
| krotscheck | fungi: Hence the <directory> | 19:18 |
| odyssey4me | anteaya did I miss today's meeting, or is it still to come - I thought it was at 8pm UTC? | 19:19 |
| * fungi wonders if we should add a mirror-dev.openstack.org where we can play around with possibly breaking refactors of the puppetry/apache config and whatnot | 19:19 | |
| anteaya | odyssey4me: yesterday | 19:20 |
| odyssey4me | oh fun, according to https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting it's on Tue at 19:00 UTC so clearly I had the wrong day :/ | 19:20 |
| anteaya | odyssey4me: the meeting is Tuesday at 19:00 | 19:20 |
| odyssey4me | I'll join next week then :/ | 19:20 |
| anteaya | odyssey4me: we talked about it | 19:20 |
| anteaya | and we had no volunteers to do a rename on such short notice | 19:20 |
| anteaya | so we will discuss it again after the summit | 19:21 |
| odyssey4me | ok, that's ok - thanks for discussing anyway | 19:21 |
| odyssey4me | it's completely understandable | 19:21 |
| anteaya | sure, thanks for helping me understand your timetable | 19:21 |
| *** thorst_ has joined #openstack-infra | 19:21 | |
| *** thorst_ has quit IRC | 19:21 | |
| fungi | odyssey4me: yeah, sorry about that. a repo rename currently means a multi-hour outage in gerrit to do offline regeneration of its lucene query indices | 19:21 |
| *** thorst_ has joined #openstack-infra | 19:22 | |
| openstackgerrit | Merged openstack-infra/system-config: Add NPM mirror hosting to unified mirrors https://review.openstack.org/305356 | 19:22 |
| fungi | it's not pretty | 19:22 |
| *** e0ne has joined #openstack-infra | 19:24 | |
| krotscheck | hoookay | 19:25 |
| fungi | hookah? | 19:25 |
| *** ajmiller has quit IRC | 19:25 | |
| *** maishsk has quit IRC | 19:26 | |
| *** Nisha has quit IRC | 19:26 | |
| odyssey4me | fungi yeah, completely understandable - we have a reasonable workaround in place to manage the side-effects so we can manage | 19:26 |
| *** salv-orlando has joined #openstack-infra | 19:26 | |
| odyssey4me | and everyone needs enough of a break to get the creative juices flowing for the summit, of course :) | 19:27 |
| odyssey4me | I'm out for the night. Have a great day y'all! | 19:27 |
| *** yamahata has quit IRC | 19:27 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-subunit2sql: Let puppet start the subunit worker daemons https://review.openstack.org/305457 | 19:28 |
| anteaya | scottda: hello are you available for a chat about https://storyboard.openstack.org/#!/story/2000266 | 19:28 |
| anteaya | odyssey4me: thanks you too | 19:28 |
| *** flwang1 has quit IRC | 19:28 | |
| mtreinish | jeblair: ^^^ I'm not sure that's actually a good idea | 19:28 |
| mtreinish | because if it kills things in the middle of the write_to_db function that would be bad | 19:29 |
| *** netsin has joined #openstack-infra | 19:30 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: run puppet4 openstack integ jobs in periodic and experimental https://review.openstack.org/305278 | 19:34 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: zuul/periodic: send failures on periodic-ci-reports ML https://review.openstack.org/305458 | 19:34 |
| EmilienM | mtreinish: ^ comments addressed | 19:34 |
| EmilienM | mtreinish: what we would need in the dashboard is a notification system, maybe RSS? | 19:34 |
| *** akaszuba has joined #openstack-infra | 19:34 | |
| EmilienM | pabelanger: I splited the patches, like you suggested ^ | 19:34 |
| *** apoorvad has quit IRC | 19:35 | |
| *** matrohon has joined #openstack-infra | 19:35 | |
| mtreinish | EmilienM: I thinking adding rss support to the dashboard is a good idea. I'm not sure the best way to go about doing it (I've never had to do it before) | 19:35 |
| mtreinish | but we can figure that out :) | 19:35 |
| openstackgerrit | Christian Schwede proposed openstack-infra/devstack-gate: Use subnode IPs as Swift storage IPs in multinode setup https://review.openstack.org/304465 | 19:35 |
| EmilienM | mtreinish: cool, let me look a bit | 19:35 |
| *** jsavak has quit IRC | 19:36 | |
| mtreinish | EmilienM: I was thinking maybe add support for using something like: https://pypi.python.org/pypi/feedgen/ to the api server | 19:36 |
| mtreinish | and then have status.o.o just redirect to that | 19:37 |
| mtreinish | but there is probably a better way | 19:37 |
| EmilienM | oh nice | 19:37 |
| mordred | jeblair: I cannot reproduce that nodepool test failure. nor does it make any sense | 19:38 |
| *** esikachev has quit IRC | 19:38 | |
| *** e0ne has quit IRC | 19:38 | |
| *** niska has quit IRC | 19:38 | |
| EmilienM | mtreinish: it looks really easy http://lkiesow.github.io/python-feedgen/#add-feed-entries | 19:39 |
| *** jsavak has joined #openstack-infra | 19:39 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: run puppet4 openstack integ jobs in periodic and experimental https://review.openstack.org/305278 | 19:39 |
| *** yamamoto has joined #openstack-infra | 19:40 | |
| mtreinish | EmilienM: yeah, so we just have to add something to poll the db at a reasonable interval and call that when there are new results | 19:40 |
| EmilienM | mtreinish: every hour would be fine imho | 19:40 |
| *** niska has joined #openstack-infra | 19:42 | |
| mtreinish | EmilienM: yeah, that would probably be sufficient | 19:42 |
| *** imcsk8_ has joined #openstack-infra | 19:42 | |
| *** mrmartin has joined #openstack-infra | 19:42 | |
| mtreinish | the other thing to figure out is at what level to generate feeds? Per project, per job, per queue, etc | 19:42 |
| *** sdague_ is now known as sdague | 19:42 | |
| *** thingee has joined #openstack-infra | 19:42 | |
| EmilienM | mtreinish: we can probably generate multiple feeds? one per project, one per job one per queue? is it too much? | 19:43 |
| mtreinish | EmilienM: nah, I was thinking it would be good to do it that way. I'm just thinking about how best to implement it | 19:44 |
| mtreinish | that gives people the option to subscribe at what level they want | 19:44 |
| *** hogepodge has joined #openstack-infra | 19:46 | |
| mtreinish | but it makes me think we might want to do the feed generation dynamically, because there are a lot of jobs, projects and combinations there of | 19:46 |
| mtreinish | like only initially create a feed if it's been requested by someone | 19:46 |
| *** mrmartin has quit IRC | 19:46 | |
| *** HenryG_ has joined #openstack-infra | 19:46 | |
| openstackgerrit | Christian Schwede proposed openstack-infra/devstack-gate: Use subnode IPs as Swift storage IPs in multinode setup https://review.openstack.org/304465 | 19:46 |
| *** yamamoto has quit IRC | 19:47 | |
| EmilienM | mtreinish: I use a tool that converts RSS to emails :-P so I'll just filter what I actually care... | 19:47 |
| *** sdake_ has quit IRC | 19:48 | |
| *** nadya has quit IRC | 19:48 | |
| *** Guest59521 has quit IRC | 19:48 | |
| *** lucasagomes has quit IRC | 19:48 | |
| *** dgonzalez has quit IRC | 19:48 | |
| *** doug-fis_ has quit IRC | 19:48 | |
| *** HeOS_ has quit IRC | 19:48 | |
| *** hdd__ has quit IRC | 19:48 | |
| *** mylu has quit IRC | 19:48 | |
| *** brucet has quit IRC | 19:48 | |
| *** tqtran has quit IRC | 19:48 | |
| *** jogo has quit IRC | 19:48 | |
| *** furlongm has quit IRC | 19:48 | |
| *** elo has quit IRC | 19:48 | |
| *** bradjones has quit IRC | 19:48 | |
| *** ldnunes has quit IRC | 19:48 | |
| *** vtapia_ has quit IRC | 19:48 | |
| *** markvoelker has quit IRC | 19:48 | |
| *** zul has quit IRC | 19:48 | |
| *** clsacramento has quit IRC | 19:48 | |
| *** penguinolog has quit IRC | 19:48 | |
| *** pblaho has quit IRC | 19:48 | |
| *** david-lyle has quit IRC | 19:48 | |
| *** imcsk8 has quit IRC | 19:48 | |
| *** lyz has quit IRC | 19:48 | |
| *** mkoderer__ has quit IRC | 19:48 | |
| *** swat30 has quit IRC | 19:48 | |
| *** wznoinsk has quit IRC | 19:48 | |
| *** redrobot has quit IRC | 19:48 | |
| *** dmsimard has quit IRC | 19:48 | |
| *** pc_m has quit IRC | 19:48 | |
| *** mordred has quit IRC | 19:48 | |
| *** dansmith has quit IRC | 19:48 | |
| *** andreaf has quit IRC | 19:48 | |
| *** jokke_ has quit IRC | 19:48 | |
| *** hughhalf has quit IRC | 19:48 | |
| *** filler has quit IRC | 19:48 | |
| *** HenryG has quit IRC | 19:48 | |
| *** HenryG_ is now known as HenryG | 19:49 | |
| mtreinish | EmilienM: so I just checked the db, we have 327 projects in the db and 822 jobs | 19:50 |
| *** hughhalf_ has joined #openstack-infra | 19:50 | |
| *** dizquierdo has quit IRC | 19:51 | |
| openstackgerrit | Sean Dague proposed openstack-infra/project-config: make api-guide and api-ref targets voting https://review.openstack.org/305464 | 19:52 |
| jeblair | mtreinish: we usually don't have ensure->running set (mostly cause i get grumpy when i stop a service and puppet helpfully starts it for me, causing corruption); but we should be able to restart on updates | 19:52 |
| *** furlongm_ has joined #openstack-infra | 19:52 | |
| jeblair | mtreinish: and if we make sure the db update is in a transaction, that should be safe | 19:52 |
| jeblair | anteaya: what kind of updates are you looking for? | 19:53 |
| mtreinish | jeblair: ok cool, the restarting on updates is what I really wanted. that way I don't have to bug you guys about it every time | 19:53 |
| *** pblaho_ has joined #openstack-infra | 19:53 | |
| anteaya | jeblair: has there been any progress on that story? | 19:53 |
| mtreinish | jeblair: I think it might be, lemme double check the code | 19:53 |
| *** dansmith has joined #openstack-infra | 19:53 | |
| *** Vivek has joined #openstack-infra | 19:53 | |
| *** bradjones_ has joined #openstack-infra | 19:53 | |
| *** bradjones_ has quit IRC | 19:53 | |
| *** bradjones_ has joined #openstack-infra | 19:53 | |
| jeblair | anteaya: not that i'm aware of | 19:53 |
| *** Vivek is now known as Guest84824 | 19:54 | |
| *** Goneri has quit IRC | 19:54 | |
| *** HeOS__ has joined #openstack-infra | 19:54 | |
| anteaya | jeblair: thank you | 19:54 |
| mtreinish | jeblair: http://git.openstack.org/cgit/openstack-infra/subunit2sql/tree/subunit2sql/shell.py#n132 | 19:54 |
| jeblair | anteaya: we can add the low hanging fruit tag to it i guess? | 19:54 |
| jeblair | anteaya: (it has LHF in the title; this predates the tag) | 19:54 |
| *** swat30_ has joined #openstack-infra | 19:55 | |
| jeblair | mtreinish: i don't understand oslo.db, sorry. | 19:55 |
| anteaya | jeblair: ah thank you, I will try to add that tag | 19:55 |
| *** dmsimard1 has joined #openstack-infra | 19:55 | |
| scottda | anteaya: Hi | 19:55 |
| *** mkoderer___ has joined #openstack-infra | 19:55 | |
| anteaya | scottda: hello | 19:56 |
| mtreinish | jeblair: there is nothing oslo.db specific there, it gets a session from the subunit2sql api call to do that (which uses oslo.db under the covers, but it's a normal sqlalchemy session object) | 19:56 |
| scottda | anteaya: That is an old storyboard... | 19:56 |
| *** amotoki has joined #openstack-infra | 19:56 | |
| anteaya | scottda: yes it is | 19:56 |
| anteaya | scottda: is is still a concern? | 19:56 |
| scottda | anteaya: No. I just logged it originally to see if there was something that should be investigated.... | 19:57 |
| *** clsacramento_ has joined #openstack-infra | 19:57 | |
| scottda | anteaya: I reckon it can be closed, or whatever the process is. | 19:57 |
| *** pcm_ has joined #openstack-infra | 19:57 | |
| anteaya | scottda: ah thank you, has the cinderclient bugs since become tracked? | 19:57 |
| scottda | anteaya: Is there something I should do for that? | 19:57 |
| anteaya | have the | 19:57 |
| mtreinish | jeblair: but looking at the db api it looks like it's a transaction per resource instead of 1 big one: http://git.openstack.org/cgit/openstack-infra/subunit2sql/tree/subunit2sql/db/api.py#n198 | 19:58 |
| anteaya | scottda: you can log in and mark it invalid | 19:58 |
| anteaya | scottda: and thank you | 19:58 |
| *** dtardivel has quit IRC | 19:58 | |
| scottda | anteaya: Yes, the bug is merged and released. I'll mark the story as invalid. Thanks. | 19:58 |
| anteaya | scottda: thanks for filing the story :) | 19:58 |
| *** roxanaghe has quit IRC | 19:59 | |
| *** roxanaghe has joined #openstack-infra | 20:00 | |
| *** dmsimard1 has quit IRC | 20:01 | |
| krotscheck | durm dee durm dee durm. | 20:01 |
| *** amotoki has quit IRC | 20:01 | |
| *** dgonzalez_ has joined #openstack-infra | 20:02 | |
| *** dmsimard1 has joined #openstack-infra | 20:03 | |
| *** sdake_ has joined #openstack-infra | 20:03 | |
| *** Guest59521 has joined #openstack-infra | 20:03 | |
| *** lucasagomes has joined #openstack-infra | 20:03 | |
| *** dgonzalez has joined #openstack-infra | 20:03 | |
| *** doug-fis_ has joined #openstack-infra | 20:03 | |
| *** HeOS_ has joined #openstack-infra | 20:03 | |
| *** hdd__ has joined #openstack-infra | 20:03 | |
| *** mylu has joined #openstack-infra | 20:03 | |
| *** brucet has joined #openstack-infra | 20:03 | |
| *** tqtran has joined #openstack-infra | 20:03 | |
| *** jogo has joined #openstack-infra | 20:03 | |
| *** furlongm has joined #openstack-infra | 20:03 | |
| *** elo has joined #openstack-infra | 20:03 | |
| *** bradjones has joined #openstack-infra | 20:03 | |
| *** ldnunes has joined #openstack-infra | 20:03 | |
| *** vtapia_ has joined #openstack-infra | 20:03 | |
| *** markvoelker has joined #openstack-infra | 20:03 | |
| *** zul has joined #openstack-infra | 20:03 | |
| *** clsacramento has joined #openstack-infra | 20:03 | |
| *** penguinolog has joined #openstack-infra | 20:03 | |
| *** pblaho has joined #openstack-infra | 20:03 | |
| *** david-lyle has joined #openstack-infra | 20:03 | |
| *** lyz has joined #openstack-infra | 20:03 | |
| *** mkoderer__ has joined #openstack-infra | 20:03 | |
| *** swat30 has joined #openstack-infra | 20:03 | |
| *** wznoinsk has joined #openstack-infra | 20:03 | |
| *** redrobot has joined #openstack-infra | 20:03 | |
| *** dmsimard has joined #openstack-infra | 20:03 | |
| *** pc_m has joined #openstack-infra | 20:03 | |
| *** mordred has joined #openstack-infra | 20:03 | |
| *** andreaf has joined #openstack-infra | 20:03 | |
| *** jokke_ has joined #openstack-infra | 20:03 | |
| *** hughhalf has joined #openstack-infra | 20:03 | |
| *** filler has joined #openstack-infra | 20:03 | |
| *** lyz_ has joined #openstack-infra | 20:03 | |
| openstackgerrit | Merged openstack-infra/project-config: run puppet4 openstack integ jobs in periodic and experimental https://review.openstack.org/305278 | 20:03 |
| *** sdake_ has quit IRC | 20:04 | |
| *** Guest59521 has quit IRC | 20:04 | |
| *** lucasagomes has quit IRC | 20:04 | |
| *** dgonzalez has quit IRC | 20:04 | |
| *** doug-fis_ has quit IRC | 20:04 | |
| *** HeOS_ has quit IRC | 20:04 | |
| *** hdd__ has quit IRC | 20:04 | |
| *** mylu has quit IRC | 20:04 | |
| *** brucet has quit IRC | 20:04 | |
| *** tqtran has quit IRC | 20:04 | |
| *** jogo has quit IRC | 20:04 | |
| *** furlongm has quit IRC | 20:04 | |
| *** elo has quit IRC | 20:04 | |
| *** bradjones has quit IRC | 20:04 | |
| *** ldnunes has quit IRC | 20:04 | |
| *** vtapia_ has quit IRC | 20:04 | |
| *** markvoelker has quit IRC | 20:04 | |
| *** zul has quit IRC | 20:04 | |
| *** clsacramento has quit IRC | 20:04 | |
| *** penguinolog has quit IRC | 20:04 | |
| *** pblaho has quit IRC | 20:04 | |
| *** david-lyle has quit IRC | 20:04 | |
| *** lyz has quit IRC | 20:04 | |
| *** mkoderer__ has quit IRC | 20:04 | |
| *** swat30 has quit IRC | 20:04 | |
| *** wznoinsk has quit IRC | 20:04 | |
| *** redrobot has quit IRC | 20:04 | |
| *** dmsimard has quit IRC | 20:04 | |
| *** pc_m has quit IRC | 20:04 | |
| *** mordred has quit IRC | 20:04 | |
| *** andreaf has quit IRC | 20:04 | |
| *** jokke_ has quit IRC | 20:04 | |
| *** hughhalf has quit IRC | 20:04 | |
| *** filler has quit IRC | 20:04 | |
| *** bradjones_ is now known as bradjones | 20:04 | |
| *** swat30_ is now known as swat30 | 20:04 | |
| *** dgonzalez_ is now known as dgonzalez | 20:04 | |
| jeblair | clarkb, fungi, mordred: i have what i need from old-nodepool; anyone else need anything or shall i delete it? | 20:04 |
| clarkb | jeblair: I don't need anything, the new server seems to be working well | 20:05 |
| *** dmsimard1 is now known as dmsimard | 20:05 | |
| clarkb | jeblair: its possible the dib people may want to keep the old volume around to do further debugging on why our images are smaller now | 20:05 |
| jeblair | clarkb: i wasn't planning on deleting the image or volume, just server. | 20:05 |
| clarkb | ah gotcha | 20:05 |
| *** david-lyle has joined #openstack-infra | 20:05 | |
| *** jsavak has quit IRC | 20:06 | |
| *** maishsk has joined #openstack-infra | 20:06 | |
| *** trown is now known as dmsimard1 | 20:06 | |
| *** dmsimard1 is now known as trown | 20:06 | |
| *** jsavak has joined #openstack-infra | 20:06 | |
| *** doug-fish has joined #openstack-infra | 20:07 | |
| jeblair | i'm deleting it | 20:08 |
| *** doug-fish has quit IRC | 20:08 | |
| *** doug-fish has joined #openstack-infra | 20:08 | |
| *** f1ller has joined #openstack-infra | 20:08 | |
| *** yamahata has joined #openstack-infra | 20:09 | |
| *** amrith is now known as _amrith_ | 20:09 | |
| clarkb | looks like todays image builds were successful and we have 35 uploads in the queue | 20:09 |
| clarkb | so that is working well | 20:09 |
| clarkb | did anyone make sure that all the jenkinses have stop jobs registered with gearman now? | 20:10 |
| * clarkb checks | 20:10 | |
| clarkb | jeblair: all of hte numbered jenkins masters have stop jobs registerd, I assume jenkins.o.o should as well? | 20:10 |
| jeblair | clarkb: i did not check, though i looked at the graph and saw it step down to 0 in a manner consistent with what i would expect from the jenkins restarts | 20:11 |
| jeblair | clarkb: yes, it should | 20:11 |
| clarkb | ok, I will run the safe restart script against jenkins.o.o | 20:11 |
| *** ybathia has joined #openstack-infra | 20:11 | |
| *** sdake_ has joined #openstack-infra | 20:11 | |
| *** lucasagomes has joined #openstack-infra | 20:11 | |
| *** doug-fis_ has joined #openstack-infra | 20:11 | |
| *** hdd__ has joined #openstack-infra | 20:11 | |
| *** brucet has joined #openstack-infra | 20:11 | |
| *** tqtran has joined #openstack-infra | 20:11 | |
| *** jogo has joined #openstack-infra | 20:11 | |
| *** elo has joined #openstack-infra | 20:11 | |
| *** ldnunes has joined #openstack-infra | 20:11 | |
| *** vtapia_ has joined #openstack-infra | 20:11 | |
| *** markvoelker has joined #openstack-infra | 20:11 | |
| *** zul has joined #openstack-infra | 20:11 | |
| *** penguinolog has joined #openstack-infra | 20:11 | |
| *** wznoinsk has joined #openstack-infra | 20:11 | |
| *** redrobot has joined #openstack-infra | 20:11 | |
| *** mordred has joined #openstack-infra | 20:11 | |
| *** andreaf has joined #openstack-infra | 20:11 | |
| *** jokke_ has joined #openstack-infra | 20:11 | |
| *** doug-fis_ has quit IRC | 20:11 | |
| jeblair | will the permanent nodes mess up the script? | 20:12 |
| jeblair | (or is the script just looking for 'no jobs running'?) | 20:12 |
| clarkb | only if they are offline, I will double check none of them are offline | 20:12 |
| *** mordred has quit IRC | 20:12 | |
| clarkb | they are all online (except for master which is special cased) so it should be fine | 20:12 |
| *** eric_lopez has joined #openstack-infra | 20:12 | |
| *** sdake has joined #openstack-infra | 20:12 | |
| *** ldnunes has quit IRC | 20:13 | |
| *** tqtran_ has joined #openstack-infra | 20:14 | |
| *** markvoelker_ has joined #openstack-infra | 20:14 | |
| *** lucasagomes has quit IRC | 20:15 | |
| *** hdd__ has quit IRC | 20:15 | |
| *** tqtran has quit IRC | 20:15 | |
| *** vtapia_ has quit IRC | 20:15 | |
| *** markvoelker has quit IRC | 20:15 | |
| *** zul has quit IRC | 20:15 | |
| *** penguinolog has quit IRC | 20:15 | |
| *** wznoinsk has quit IRC | 20:15 | |
| *** redrobot has quit IRC | 20:15 | |
| *** andreaf has quit IRC | 20:15 | |
| *** jokke_ has quit IRC | 20:15 | |
| *** hdd_ has joined #openstack-infra | 20:15 | |
| *** ybathia has quit IRC | 20:16 | |
| *** sdake_ has quit IRC | 20:16 | |
| *** brucet has quit IRC | 20:16 | |
| *** jogo has quit IRC | 20:16 | |
| *** elo has quit IRC | 20:16 | |
| krotscheck | jeblair: FYI, the yahoo guy just told me there's a chance there may be more than 64K packages that start with the letter 's'. | 20:16 |
| pabelanger | kool | 20:16 |
| pabelanger | I still in shock there is 200GB of npm libraries | 20:17 |
| *** Rockyg has quit IRC | 20:17 | |
| *** jokke_ has joined #openstack-infra | 20:17 | |
| *** Rockyg has joined #openstack-infra | 20:17 | |
| clarkb | hrm I Think the auth might be different on jenkins.o.o. I may just quickly do it by hand since it is idle | 20:17 |
| *** apoorvad has joined #openstack-infra | 20:18 | |
| clarkb | actually there it goes, nevermind its fine | 20:18 |
| *** mylu has joined #openstack-infra | 20:18 | |
| *** roxanaghe has quit IRC | 20:18 | |
| krotscheck | pabelanger: 200GB of text-based tarballs no less. | 20:19 |
| krotscheck | Those shrink by lots. | 20:19 |
| *** armax has quit IRC | 20:19 | |
| jeblair | krotscheck: neat. if so, we may need to add an extra directory level. and get really clever with regexes. | 20:19 |
| krotscheck | jeblair: yeaaaahh we'll burn that bridge when we get to it :) | 20:20 |
| jeblair | krotscheck: also, the limit isn't actually 64k -- it's 64k if every filename in the directory are less than 16 characters... | 20:20 |
| jeblair | krotscheck: the limit shrinks if some are longer | 20:20 |
| krotscheck | .........uuuuuwhaaaaaaaa? | 20:20 |
| clarkb | "Please wait while your jenkins is getting ready to work" so slow | 20:21 |
| jeblair | krotscheck: (really there are 64k slots -- each file <16chars takes 1 slot; 16<len<32 is 2 slots, etc) | 20:21 |
| *** lucasagomes has joined #openstack-infra | 20:21 | |
| krotscheck | Gotcha. | 20:21 |
| krotscheck | Well, we're splitting /S/ and /s/, so that'll divide it down | 20:21 |
| jeblair | krotscheck, pabelanger: http://www.haneycodes.net/npm-left-pad-have-we-forgotten-how-to-program/ this may be relevant :) | 20:22 |
| *** ybathia has joined #openstack-infra | 20:22 | |
| jeblair | fortunately, microsaas's like http://left-pad.io/ will save us; though i assume we'll need a library to interface with it | 20:23 |
| clarkb | jeblair: so the filename length affects the number of files you can have? | 20:23 |
| jeblair | clarkb: yep | 20:23 |
| *** redrobot has joined #openstack-infra | 20:23 | |
| krotscheck | jeblair: IsArray. | 20:24 |
| *** roxanaghe has joined #openstack-infra | 20:24 | |
| krotscheck | I love that one. | 20:24 |
| *** redrobot is now known as Guest78786 | 20:24 | |
| *** jamesmcarthur has quit IRC | 20:24 | |
| jeblair | krotscheck: right? | 20:24 |
| *** mordred has joined #openstack-infra | 20:24 | |
| jeblair | someone has 'Maintainer of isArray' on their resume | 20:24 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Run tempest tests with periodic jobs https://review.openstack.org/297038 | 20:24 |
| *** andreaf has joined #openstack-infra | 20:25 | |
| krotscheck | jeblair: I want "maintainer of isNull" on mine. | 20:25 |
| fungi | jeblair: all clear to delete old nodepool as far as i'm concerned | 20:25 |
| mordred | that's awesome | 20:25 |
| mordred | jeblair: I was on the other side of a netsplit - did I miss anything good? | 20:26 |
| openstackgerrit | Hongbin Lu proposed openstack-infra/project-config: Magnum: switch neutron-lbaas devstack plugin https://review.openstack.org/305475 | 20:26 |
| *** ldnunes has joined #openstack-infra | 20:26 | |
| *** avarner has quit IRC | 20:26 | |
| jeblair | mordred: if you got the left-pad bit, that was it :) | 20:26 |
| jeblair | mordred: clarkb says image builds are happy and we have uploads in the queue | 20:26 |
| mordred | jeblair: nope. I got "20:24:43 jeblair | someone has 'Maintainer of isArray' on their resume | 20:26 |
| *** imcsk8_ is now known as imcsk8 | 20:26 | |
| mordred | jeblair: woot! | 20:27 |
| mordred | jeblair: what's up with left pad? | 20:27 |
| jeblair | mordred: oh, then you want to read http://www.haneycodes.net/npm-left-pad-have-we-forgotten-how-to-program/ | 20:27 |
| jeblair | mordred: it explains, among other things, why we are downloading so many packages :) | 20:27 |
| jeblair | mordred, clarkb: we may want to consider the image uploads when we restart nodepool | 20:27 |
| *** zul has joined #openstack-infra | 20:28 | |
| *** penguinolog has joined #openstack-infra | 20:29 | |
| *** vtapia_ has joined #openstack-infra | 20:29 | |
| pabelanger | jeblair: I really like this one from the article you linked: https://www.npmjs.com/package/isarray | 20:30 |
| pabelanger | 21,145,967 downloads in the last month | 20:30 |
| *** avarner has joined #openstack-infra | 20:30 | |
| jeblair | makes me want to override string coercion to return '[object Array]' on anything i write :) | 20:31 |
| *** ybathia has quit IRC | 20:31 | |
| mordred | jeblair: that is an amazing blog post | 20:32 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Install afs on all servers https://review.openstack.org/305477 | 20:32 |
| mordred | +3 | 20:32 |
| *** arxcruz has quit IRC | 20:33 | |
| openstackgerrit | Merged openstack-infra/puppet-stackalytics: Add foundation for functional tests https://review.openstack.org/305382 | 20:33 |
| *** mylu has quit IRC | 20:34 | |
| *** hashar has joined #openstack-infra | 20:35 | |
| *** weshay has quit IRC | 20:35 | |
| *** krotscheck is now known as krotscheck_dcm | 20:36 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/project-config: Make Infra's puppet beaker jobs voting https://review.openstack.org/305416 | 20:36 |
| *** denisra_ has joined #openstack-infra | 20:37 | |
| *** mylu has joined #openstack-infra | 20:37 | |
| *** bpokorny_ has joined #openstack-infra | 20:37 | |
| *** obondarev_ has joined #openstack-infra | 20:38 | |
| *** odyssey4me_ has joined #openstack-infra | 20:38 | |
| *** _amrith_ is now known as amrith | 20:38 | |
| mordred | jeblair: SOOOO ... nodepool path | 20:38 |
| mordred | patch | 20:39 |
| *** cbader has quit IRC | 20:39 | |
| *** mnencia_ has joined #openstack-infra | 20:39 | |
| *** akshai_ has joined #openstack-infra | 20:39 | |
| mordred | jeblair: I cannot logically reason about how that patch causes that error. nor, mind you, how _any_ patch causes that error | 20:39 |
| mordred | jeblair: nor can I reproduce it | 20:39 |
| *** nijaba_ has joined #openstack-infra | 20:39 | |
| *** tsufiev has quit IRC | 20:40 | |
| *** tsufiev_ has joined #openstack-infra | 20:40 | |
| *** tsufiev_ is now known as tsufiev__ | 20:40 | |
| *** tosky__ has joined #openstack-infra | 20:40 | |
| *** johnsom has joined #openstack-infra | 20:40 | |
| jeblair | mordred: i have rechecked it | 20:40 |
| *** AJaeger_ has joined #openstack-infra | 20:40 | |
| *** timburke_ has joined #openstack-infra | 20:40 | |
| *** wolsen_ has joined #openstack-infra | 20:40 | |
| *** xwizard___ has joined #openstack-infra | 20:41 | |
| *** tonyb_ has joined #openstack-infra | 20:41 | |
| mordred | jeblair: cool | 20:41 |
| *** Vivek___ has joined #openstack-infra | 20:41 | |
| mordred | jeblair: I mean, I'd LOVE to know what happened there | 20:41 |
| mordred | because it's a VERY weird error | 20:41 |
| mordred | and makes the think there is a real bug _somewhere_ | 20:42 |
| *** promethe1nfire has joined #openstack-infra | 20:42 | |
| *** markvoelker has joined #openstack-infra | 20:42 | |
| *** tqtran has joined #openstack-infra | 20:42 | |
| *** dprince_ has joined #openstack-infra | 20:42 | |
| *** r-mibu_ has joined #openstack-infra | 20:42 | |
| *** tobiash_ has joined #openstack-infra | 20:42 | |
| *** rkukura_ has joined #openstack-infra | 20:42 | |
| clarkb | mordred: was https://storyboard.openstack.org/#!/story/2000520 fixed? have a specific commit I can point to as being the fix for that? | 20:42 |
| clarkb | (I want to say it was fixed) | 20:43 |
| *** dteselkin_ has joined #openstack-infra | 20:43 | |
| clarkb | oh wait the review is in there /me looks to see if it merged | 20:43 |
| *** raginbaj- has joined #openstack-infra | 20:43 | |
| *** Guest78786 is now known as redrobot | 20:43 | |
| mordred | clarkb: yes. it's fixed | 20:43 |
| mordred | for the record, the description in the story was not the actual problem | 20:44 |
| mordred | but the actual problem was solved | 20:44 |
| clarkb | mordred: feel free to update it :) | 20:44 |
| *** lbragstad_ has joined #openstack-infra | 20:44 | |
| clarkb | nibalizer: https://storyboard.openstack.org/#!/story/2000172 is done I think right? | 20:44 |
| *** masayukig_ has joined #openstack-infra | 20:44 | |
| *** andrewbogott has quit IRC | 20:44 | |
| *** andrewbogott has joined #openstack-infra | 20:44 | |
| *** erikwilson has joined #openstack-infra | 20:45 | |
| *** ajo_ has joined #openstack-infra | 20:45 | |
| nibalizer | well my hope had been to delete one of the files completely | 20:46 |
| *** hdd_ has quit IRC | 20:46 | |
| nibalizer | but its really really really close | 20:46 |
| mordred | nibalizer: more flat | 20:46 |
| *** jokke_ has quit IRC | 20:47 | |
| *** markvoelker_ has quit IRC | 20:47 | |
| *** tqtran_ has quit IRC | 20:47 | |
| *** Guest84824 has quit IRC | 20:47 | |
| *** netsin has quit IRC | 20:47 | |
| *** csaba has quit IRC | 20:47 | |
| *** jpeeler has quit IRC | 20:47 | |
| *** katyafervent_awa has quit IRC | 20:47 | |
| *** mnencia has quit IRC | 20:47 | |
| *** johnsom_ has quit IRC | 20:47 | |
| *** tosky has quit IRC | 20:47 | |
| *** akshai has quit IRC | 20:47 | |
| *** moravec has quit IRC | 20:47 | |
| *** bpokorny has quit IRC | 20:47 | |
| *** amitgandhinz has quit IRC | 20:47 | |
| *** dprince has quit IRC | 20:47 | |
| *** masayukig has quit IRC | 20:47 | |
| *** r-mibu has quit IRC | 20:47 | |
| *** ianychoi has quit IRC | 20:47 | |
| *** AJaeger has quit IRC | 20:47 | |
| *** LinuxJedi has quit IRC | 20:47 | |
| *** denisra has quit IRC | 20:47 | |
| *** sileht has quit IRC | 20:47 | |
| *** rcarrillocruz has quit IRC | 20:47 | |
| *** dteselkin has quit IRC | 20:47 | |
| *** SergK has quit IRC | 20:47 | |
| *** xwizard has quit IRC | 20:47 | |
| *** russellb has quit IRC | 20:47 | |
| *** obondarev has quit IRC | 20:47 | |
| *** sflanigan has quit IRC | 20:47 | |
| *** rkukura has quit IRC | 20:47 | |
| *** dtantsur|afk has quit IRC | 20:47 | |
| *** wolsen has quit IRC | 20:47 | |
| *** harlowja has quit IRC | 20:47 | |
| *** nijaba has quit IRC | 20:47 | |
| *** prometheanfire has quit IRC | 20:47 | |
| *** tobiash has quit IRC | 20:47 | |
| *** ajo has quit IRC | 20:47 | |
| *** raginbajin has quit IRC | 20:47 | |
| *** cgross has quit IRC | 20:47 | |
| *** odyssey4me has quit IRC | 20:47 | |
| *** qb_ has quit IRC | 20:47 | |
| *** erikmwilson has quit IRC | 20:47 | |
| *** lbragstad has quit IRC | 20:47 | |
| *** pabelanger has quit IRC | 20:47 | |
| *** timburke has quit IRC | 20:47 | |
| *** trey has quit IRC | 20:47 | |
| *** tonyb has quit IRC | 20:47 | |
| *** mnencia_ is now known as mnencia | 20:47 | |
| *** rkukura_ is now known as rkukura | 20:47 | |
| *** amitgandhinz has joined #openstack-infra | 20:47 | |
| *** SergK_ has joined #openstack-infra | 20:47 | |
| *** raginbaj- is now known as raginbajin | 20:47 | |
| *** LinuxJedi_ has joined #openstack-infra | 20:47 | |
| *** mylu has quit IRC | 20:47 | |
| *** LinuxJedi_ is now known as LinuxJedi | 20:47 | |
| nibalizer | clarkb: oh and as for pulling o_p::server out thats not done | 20:47 |
| *** maishsk_ has joined #openstack-infra | 20:47 | |
| *** FallenPegasus has joined #openstack-infra | 20:48 | |
| *** ybathia has joined #openstack-infra | 20:48 | |
| *** hashar has quit IRC | 20:48 | |
| *** russellb has joined #openstack-infra | 20:48 | |
| openstackgerrit | Christian Schwede proposed openstack-infra/devstack-gate: Use subnode IPs as Swift storage IPs in multinode setup https://review.openstack.org/304465 | 20:48 |
| *** promethe1nfire is now known as prometheanfire | 20:49 | |
| *** pabelanger has joined #openstack-infra | 20:49 | |
| *** dtantsur has joined #openstack-infra | 20:49 | |
| EmilienM | I'm trying to understand why we don't have puppet-aodh tarball published on tarballs.o.o | 20:49 |
| *** jokke_ has joined #openstack-infra | 20:49 | |
| clarkb | zaro: can you update https://storyboard.openstack.org/#!/story/1397481 with the progress on the upstream change to have gerrit include the votes on every event? | 20:49 |
| openstackgerrit | Zara proposed openstack-infra/storyboard-webclient: Make it clear when user reaches end of events timeline https://review.openstack.org/305483 | 20:49 |
| clarkb | zaro I can dig it up my self if you are not able, but I think you have all the details already | 20:49 |
| *** maishsk has quit IRC | 20:50 | |
| *** maishsk_ is now known as maishsk | 20:50 | |
| *** trey has joined #openstack-infra | 20:50 | |
| *** katyafervent_awa has joined #openstack-infra | 20:50 | |
| EmilienM | and I don't find link for publish logs | 20:51 |
| openstackgerrit | Merged openstack-infra/project-config: Make gate-trove-python34-db voting; add gate job https://review.openstack.org/304030 | 20:51 |
| openstackgerrit | Merged openstack-infra/puppet-hound: Add foundation for functional tests https://review.openstack.org/305376 | 20:51 |
| *** mylu has joined #openstack-infra | 20:51 | |
| *** cgross has joined #openstack-infra | 20:52 | |
| *** edmondsw has joined #openstack-infra | 20:52 | |
| *** sflanigan has joined #openstack-infra | 20:52 | |
| *** sflanigan has joined #openstack-infra | 20:52 | |
| *** ianychoi has joined #openstack-infra | 20:52 | |
| openstackgerrit | Merged openstack-infra/puppet-ciwatch: Add foundation for functional tests https://review.openstack.org/305373 | 20:53 |
| *** claudiub|2 has joined #openstack-infra | 20:53 | |
| *** rcarrillocruz has joined #openstack-infra | 20:53 | |
| *** csaba has joined #openstack-infra | 20:54 | |
| pabelanger | I don't think my last comments made it to freenode, so apologies if this is duplicated | 20:54 |
| pabelanger | looks like ansible-playbook is hung again, 2.5 hours | 20:54 |
| *** jpeeler has joined #openstack-infra | 20:54 | |
| pabelanger | should be kill the process or wait and see if ansible eventually times out | 20:54 |
| *** jheroux has quit IRC | 20:54 | |
| *** BigWillie has quit IRC | 20:55 | |
| *** esker has quit IRC | 20:55 | |
| openstackgerrit | Anne Gentle proposed openstack-infra/project-config: Adds post job for nova-api-ref https://review.openstack.org/305485 | 20:56 |
| melwitt | does anyone know what made the http://status.openstack.org/zuul/ page fast again when filtering on a change number? really happy to see that | 20:56 |
| *** bpokorny has joined #openstack-infra | 20:57 | |
| *** pabelanger has quit IRC | 20:58 | |
| *** pabelanger has joined #openstack-infra | 20:58 | |
| *** sileht has joined #openstack-infra | 20:58 | |
| clarkb | melwitt: less data probably | 20:58 |
| clarkb | right now there aren't many changes in it | 20:58 |
| *** bpokorny has quit IRC | 20:58 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Actually test stack-update to new tht https://review.openstack.org/304707 | 20:58 |
| melwitt | clarkb: hm. last week I thought I remembered the progress bar graphics looked different/updated and things were slower | 20:59 |
| *** akaszuba_ has joined #openstack-infra | 20:59 | |
| *** bpokorny has joined #openstack-infra | 20:59 | |
| *** akaszuba_ has quit IRC | 20:59 | |
| clarkb | progress bars haven't changed | 20:59 |
| melwitt | wish I had screen capped what I saw. oh well :P | 21:00 |
| *** esker has joined #openstack-infra | 21:00 | |
| timrc | melwitt: Were you thinking of http://zuul.openstack.org/? | 21:00 |
| *** bpokorny_ has quit IRC | 21:00 | |
| melwitt | timrc: uh oh, you're right | 21:01 |
| EmilienM | clarkb: I'm trying to investigate why http://tarballs.openstack.org/puppet-aodh does not exist, where do you check post logs for a specific project? | 21:01 |
| *** bpokorny has quit IRC | 21:01 | |
| melwitt | clarkb, timrc: my bad, I didn't know they were different until now | 21:01 |
| pabelanger | EmilienM: when did you run the job? | 21:01 |
| *** akaszuba has quit IRC | 21:01 | |
| *** cbader has joined #openstack-infra | 21:01 | |
| timrc | melwitt: Confused me when I first stumbled upon zuul.openstack.org ;-) | 21:01 |
| *** bpokorny has joined #openstack-infra | 21:01 | |
| clarkb | EmilienM: http://logs.openstack.org/firsttwoofsha1/sha1 | 21:01 |
| EmilienM | pabelanger: they run in post, after each pupept-aodh merge | 21:02 |
| melwitt | timrc: I've recently onboarded to a new laptop and typed that in thinking I was going to status.openstack. oops :) | 21:02 |
| *** rossella_s has quit IRC | 21:03 | |
| pabelanger | EmilienM: this is for the changes you made to run-tarball.sh right? | 21:03 |
| pabelanger | EmilienM: if so, image uploads are still in progress so chances are the post pipeline is not using the latest version of the script | 21:04 |
| *** angdraug has joined #openstack-infra | 21:05 | |
| *** amrith is now known as _amrith_ | 21:05 | |
| *** achanda has joined #openstack-infra | 21:05 | |
| timrc | pabelanger: re: http://grafana.openstack.org/dashboard/db/nodepool-rackspace - Time to ready is when the node is in Jenkins ready to do work, right? | 21:05 |
| *** rossella_s has joined #openstack-infra | 21:06 | |
| *** achanda has quit IRC | 21:06 | |
| *** fitoduarte has quit IRC | 21:06 | |
| *** achanda has joined #openstack-infra | 21:06 | |
| *** sridhar_ram1 has joined #openstack-infra | 21:07 | |
| clarkb | timrc: yes the nodepool state change doesn't happen until after the post boot script runs and the instance is added to jenkins | 21:08 |
| EmilienM | pabelanger: yes | 21:08 |
| *** sdake has quit IRC | 21:08 | |
| timrc | clarkb: So that's a little scary that rackspace nodes are booting in less than a second but not ready to do work for 4-ish minutes? | 21:08 |
| EmilienM | pabelanger: ah, it makes sense ! | 21:08 |
| pabelanger | timrc: was looking at code, it looks like it. | 21:08 |
| clarkb | timrc: I think the "boot" there is actually how much time it takes nova to say "ya Iam booting things now" | 21:09 |
| *** sridhar_ram has quit IRC | 21:09 | |
| clarkb | then it takes 4 minutes to boot, run the ready script, and add to jenkins | 21:09 |
| *** dprince_ has quit IRC | 21:09 | |
| clarkb | not very scary | 21:09 |
| *** gordc has quit IRC | 21:09 | |
| docaedo | hellllo! since meetbot was asleep at the switch during the start of the defcore meeting, I was wondering how I can best supply the missing lines for todays meeting log? | 21:09 |
| timrc | clarkb: Ah so it's not "vm is active"? | 21:10 |
| clarkb | timrc: correct | 21:10 |
| pabelanger | EmilienM: still 28 images uploading | 21:10 |
| *** baoli__ has quit IRC | 21:10 | |
| timrc | clarkb: Not that I want to compare providers but time-to-ready for blue box is sub 30 seconds. | 21:11 |
| timrc | I guess it's not necessarily apples-to-apples but that is a stark contrast none-the-less. | 21:11 |
| *** achanda has quit IRC | 21:11 | |
| pabelanger | clarkb: did you mention moving nodepool image builds a few hours earlier in the day? | 21:11 |
| clarkb | timrc: the big difference htere is 17GB vhd vs 5GB qcow2 | 21:12 |
| mordred | clarkb: yay for vhd! | 21:12 |
| clarkb | timrc: and also xen vs kvm | 21:12 |
| clarkb | pabelanger: no I didn't move anything, just checked that they ran | 21:12 |
| mordred | I will say that the rax time to ready is REMARKABLY consistent | 21:12 |
| *** trown is now known as trown|outtypewww | 21:13 | |
| *** cody-somerville has joined #openstack-infra | 21:13 | |
| *** aspiers_ has quit IRC | 21:13 | |
| clarkb | mordred: vhd is literally the worst format | 21:14 |
| *** flwang1 has joined #openstack-infra | 21:14 | |
| clarkb | mordred: did you know that the raw images use less disk than vhd? | 21:14 |
| clarkb | mordred: because they can be trimed | 21:14 |
| clarkb | mordred: so their actual size is 17GB or whatever bu the on disk hit is closer to 11GB | 21:14 |
| mordred | clarkb: but $reasons!!!! | 21:15 |
| *** ldnunes has quit IRC | 21:15 | |
| *** esker has quit IRC | 21:15 | |
| *** esker has joined #openstack-infra | 21:16 | |
| *** |-paul-| has joined #openstack-infra | 21:16 | |
| timrc | clarkb: Thanks for the clarifications. I really love that infra has this "real world" data. | 21:16 |
| mordred | timrc: data. so much data. | 21:16 |
| mordred | also, quite a bit of da da | 21:16 |
| timrc | mordred: And not filtered through a marketing department. | 21:17 |
| *** asettle has joined #openstack-infra | 21:17 | |
| clarkb | it turns out that qcow2s as a transport format are pretty decent | 21:17 |
| timrc | So I really need to get my jenkins change tested and approved so that we start getting finer grained timing data from zuul. | 21:17 |
| timrc | Because this is really interesting to me for no other reason than that. | 21:18 |
| clarkb | they compress decently well (as compared to xzing a raw image) and are supported | 21:18 |
| mordred | clarkb: yup | 21:18 |
| timrc | time-to-ready hides a bunch of stuff | 21:18 |
| mordred | this is the reason I keep arguing that ALL clouds should accept qcow2 as input | 21:18 |
| mordred | even if that's not their native format | 21:18 |
| clarkb | the only real issue we have had with them is the qcow3 change that they didn't call out | 21:18 |
| timrc | beyond just hypervisors and image formats | 21:18 |
| *** cody-somerville has quit IRC | 21:18 | |
| mordred | clarkb: yup. but at this point it's all qcow3 | 21:18 |
| clarkb | mordred: are you sure? we still build with the compat flag | 21:19 |
| clarkb | and its impossible to debug when it isn't working | 21:19 |
| mordred | clarkb: I believe that's just because of history | 21:19 |
| *** ddieterly is now known as ddieterly[away] | 21:19 | |
| clarkb | that said I have tested builds with qcow2 vs qcow3 and I couldn't find a difference | 21:19 |
| clarkb | so not sure its a big deal | 21:19 |
| openstackgerrit | Ian Wienand proposed openstack-infra/glean: Add selinux context manager for writing files https://review.openstack.org/304357 | 21:19 |
| mordred | clarkb: HP was the only place I know of that needed the compat | 21:19 |
| clarkb | under the hood I am sure there are differences but difference to the user is nil | 21:19 |
| mordred | clarkb: but I think none of us have cared enough to test that it works right everywhere | 21:20 |
| *** lbragstad_ is now known as lbragstad | 21:21 | |
| mordred | jeblair: WOAH | 21:22 |
| mordred | jeblair: it's a thing | 21:22 |
| *** doug-fish has quit IRC | 21:22 | |
| jeblair | wth | 21:22 |
| mordred | jeblair: but it seems to be maybe racey? | 21:22 |
| jeblair | mordred: yeah, different job this time | 21:22 |
| *** doug-fish has joined #openstack-infra | 21:23 | |
| mordred | jeblair: so - gut hunch ... | 21:23 |
| mordred | jeblair: we're reusing an object when we think we're copying it | 21:23 |
| mordred | and something is stepping on internals | 21:23 |
| *** matrohon has quit IRC | 21:23 | |
| *** cody-somerville has joined #openstack-infra | 21:24 | |
| *** cody-somerville has quit IRC | 21:24 | |
| *** cody-somerville has joined #openstack-infra | 21:24 | |
| *** julim has quit IRC | 21:24 | |
| timrc | Sounds shady to me. | 21:24 |
| timrc | I've encountered that exact issue before with shade. | 21:24 |
| *** doug-fish has quit IRC | 21:26 | |
| *** doug-fish has joined #openstack-infra | 21:26 | |
| mordred | yah. I thought we squashed all of those a while back - there were some times where we were returning un-copied dicts | 21:27 |
| *** fitoduarte has joined #openstack-infra | 21:27 | |
| *** gyee has joined #openstack-infra | 21:28 | |
| clarkb | greghaynes: jroll adam_g https://storyboard.openstack.org/#!/story/1371319 is the ironic + dib integration testing the way we want it today? | 21:30 |
| *** esker has quit IRC | 21:30 | |
| *** thorst_ has quit IRC | 21:30 | |
| *** esker has joined #openstack-infra | 21:31 | |
| *** armax has joined #openstack-infra | 21:31 | |
| *** doug-fish has quit IRC | 21:31 | |
| *** sridhar_ram1 has quit IRC | 21:31 | |
| *** bpokorny has quit IRC | 21:32 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 21:32 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 21:32 | |
| *** doug-fish has joined #openstack-infra | 21:32 | |
| *** kzaitsev_mb has joined #openstack-infra | 21:33 | |
| *** _ari_ has quit IRC | 21:33 | |
| *** dgonzalez has quit IRC | 21:35 | |
| *** doug-fis_ has joined #openstack-infra | 21:35 | |
| mordred | jeblair: I think something else is weird | 21:35 |
| mordred | http://logs.openstack.org/79/299679/3/check/nodepool-coverage-db/1b90bb2/console.html#_2016-04-13_20_43_17_603 | 21:36 |
| *** thorst has joined #openstack-infra | 21:36 | |
| mordred | jeblair: "[os_client_config.config] Deferring keystone exception: __init__() got an unexpected keyword argument 'username'" | 21:36 |
| mordred | morgan: ^^ | 21:36 |
| * morgan reads backscroll | 21:36 | |
| *** sridhar_ram has joined #openstack-infra | 21:36 | |
| jeblair | that did not happen on successful runs | 21:37 |
| *** FallenPegasus has quit IRC | 21:37 | |
| *** matt-borland_ has quit IRC | 21:37 | |
| morgan | wait what? unknown kwargs "username"? | 21:37 |
| *** doug-fish has quit IRC | 21:37 | |
| morgan | thats.. weird. | 21:38 |
| mordred | yah | 21:38 |
| mordred | that makes me think that the error we're seeing is a symptom of something else | 21:38 |
| mordred | like, a weird failure in the occ constructor which is causing a half-created object or something | 21:38 |
| morgan | yeah | 21:39 |
| mordred | (image_format, fwiw, should ALWAYS be in the config dict - it's one of the things set in defaults.json | 21:39 |
| *** dalgaaf has quit IRC | 21:39 | |
| clarkb | dstufft: is https://storyboard.openstack.org/#!/story/1355050 fixed in latest pip? I want to say it is | 21:39 |
| *** doug-fis_ has quit IRC | 21:40 | |
| *** Volundr has quit IRC | 21:40 | |
| *** thorst has quit IRC | 21:40 | |
| *** aysyd has quit IRC | 21:41 | |
| *** imcsk8 has quit IRC | 21:41 | |
| *** cody-somerville has quit IRC | 21:41 | |
| *** claudiub|2 has quit IRC | 21:41 | |
| *** cgross has quit IRC | 21:41 | |
| *** amitgandhinz has quit IRC | 21:41 | |
| *** tobiash_ has quit IRC | 21:41 | |
| *** Vivek___ has quit IRC | 21:41 | |
| *** penguinolog has quit IRC | 21:41 | |
| *** clsacramento_ has quit IRC | 21:41 | |
| *** Jeffrey4l__ has quit IRC | 21:41 | |
| *** claudiub has joined #openstack-infra | 21:41 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add api to get recent failed runs by run_metadata https://review.openstack.org/305495 | 21:41 |
| *** tobiash has joined #openstack-infra | 21:41 | |
| morgan | mordred: did this error just start happening? | 21:42 |
| mordred | morgan: it's happening _sometimes_ on test run patches now | 21:42 |
| *** imcsk8 has joined #openstack-infra | 21:42 | |
| morgan | mordred: *blink8 | 21:42 |
| morgan | ok. that is weird. | 21:43 |
| mordred | morgan: it all passed yesterday | 21:43 |
| SpamapS | mordred: did I hear you say the other day that diskimage-builder works on xenial? | 21:43 |
| *** ddieterly[away] is now known as ddieterly | 21:43 | |
| *** clsacramento has joined #openstack-infra | 21:43 | |
| mordred | we've run it twice today and it has failed one of the two unittest related jobs | 21:43 |
| mordred | SpamapS: yes | 21:43 |
| SpamapS | mordred: k.. doesn't work for me... | 21:43 |
| mordred | SpamapS: I should be clear ... | 21:43 |
| mordred | SpamapS: dib + ubuntu-minimal work on xenial | 21:44 |
| SpamapS | yeah, I'm just doing ubuntu-minimal + simple-init | 21:44 |
| mordred | pabelanger: ^^ you were the one who worked on that right? | 21:44 |
| *** whoops has joined #openstack-infra | 21:44 | |
| *** doug-fish has joined #openstack-infra | 21:44 | |
| mordred | SpamapS: what issue are you running in to? | 21:44 |
| *** dgonzalez has joined #openstack-infra | 21:45 | |
| *** bpokorny has joined #openstack-infra | 21:45 | |
| SpamapS | + sudo find /tmp/image.RqWnNvOs/mnt/etc/ssh -name 'ssh_host*' -type f -delete | 21:45 |
| SpamapS | find: `/tmp/image.RqWnNvOs/mnt/etc/ssh': No such file or directory | 21:45 |
| mordred | hrm | 21:45 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool: Expose nodepool-builder workers settings https://review.openstack.org/294339 | 21:46 |
| SpamapS | just ubuntu-minimal, simple-init, and vm. | 21:46 |
| *** cgross has joined #openstack-infra | 21:46 | |
| mordred | SpamapS: http://nodepool.openstack.org/dib.ubuntu-xenial.log.2016-04-13_00 | 21:46 |
| pabelanger | mordred: yes. they are uploaded to nodepool too | 21:46 |
| mordred | SpamapS: fwiw | 21:46 |
| *** tsufiev__ is now known as tsufiev | 21:46 | |
| morgan | mordred: i have to go get food. late lunch can start looking again when done | 21:47 |
| mordred | morgan: k. thanks! | 21:47 |
| mordred | SpamapS: we have a few more elements in ours | 21:47 |
| *** ybathia has quit IRC | 21:47 | |
| SpamapS | yeah probably sshd isn't being pulled in | 21:47 |
| mordred | SpamapS: it's possible one of our elements installed something that fix that issue? | 21:47 |
| pabelanger | right, minimal doesn't have SSH | 21:48 |
| *** ybathia has joined #openstack-infra | 21:48 | |
| mordred | that seems like a bug in the simple-init element then | 21:48 |
| *** timburke_ is now known as timburke | 21:48 | |
| *** ybathia has quit IRC | 21:48 | |
| SpamapS | yep | 21:49 |
| SpamapS | assumes sshd | 21:49 |
| mordred | simple-init really shouldn't be deleting ssh hostkeys in a way that assumes sshd | 21:49 |
| *** doug-fish has quit IRC | 21:49 | |
| SpamapS | should just [ -d /etc/ssh ] && .. | 21:49 |
| mordred | ++ | 21:50 |
| mordred | SpamapS: I would +1 that patch | 21:50 |
| pabelanger | must be related to the recent changes that nuked nodepool.o.o the other day. When I tried ubuntu-minimal vm simple-init initially, it worked as expected | 21:50 |
| *** Vivek has joined #openstack-infra | 21:50 | |
| SpamapS | the ssh key cleanup is recent | 21:50 |
| SpamapS | I believe after somebody discovered all the machines had the same SSH key ;) | 21:50 |
| *** Vivek is now known as Guest18995 | 21:50 | |
| *** tonytan4ever has quit IRC | 21:51 | |
| pabelanger | Ooo, nice | 21:51 |
| *** tonytan4ever has joined #openstack-infra | 21:51 | |
| pabelanger | clarkb: when you have spare cycles, https://review.openstack.org/#/c/294339/ exposes worker settings via CLI | 21:53 |
| clarkb | SpamapS: mordred pretty sure it does check if the ssh dir is there before cleaing it up | 21:53 |
| *** Jeffrey4l__ has joined #openstack-infra | 21:54 | |
| *** keedya has quit IRC | 21:54 | |
| clarkb | SpamapS: mordred can you explain to me what the current issue is so that we don't explode nodepool again | 21:54 |
| *** penguinolog has joined #openstack-infra | 21:54 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack/diskimage-builder: simple-init: Fix path for /etc/ssh test https://review.openstack.org/305498 | 21:54 |
| SpamapS | ^^ | 21:54 |
| clarkb | and if you make any changes ot simple init can you either A) test it on all the images or B) let me review it first? | 21:54 |
| SpamapS | that's why | 21:54 |
| SpamapS | same bug | 21:54 |
| SpamapS | different line | 21:54 |
| *** thorst has joined #openstack-infra | 21:54 | |
| clarkb | damnit | 21:54 |
| * clarkb throws wet cats at dib | 21:54 | |
| * SpamapS watches dib take a shower in wet cats | 21:54 | |
| mordred | I'd wondered where those had gotten off to | 21:54 |
| clarkb | for the record | 21:54 |
| clarkb | I extensively tested the change | 21:55 |
| clarkb | and it worked on debian, ubuntu, fedora, and centos | 21:55 |
| *** bpokorny has quit IRC | 21:55 | |
| SpamapS | it works fine | 21:55 |
| SpamapS | as long as you have something that pulls in sshd | 21:55 |
| clarkb | then it get painted a different color and broke noepool | 21:55 |
| clarkb | SpamapS: no no | 21:55 |
| *** bpokorny has joined #openstack-infra | 21:55 | |
| clarkb | SpamapS: there was a first patchset taht was fine | 21:55 |
| clarkb | SpamapS: then a new patchset got made without testing that broke my world | 21:55 |
| SpamapS | :( | 21:55 |
| SpamapS | I know that feeling | 21:55 |
| clarkb | anyways I will get over it eventually | 21:55 |
| *** bpokorny has quit IRC | 21:56 | |
| mordred | oh! | 21:56 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-openstackci: Use our custom pip provider for keyring package https://review.openstack.org/305499 | 21:56 |
| SpamapS | Like, that's not what was in the box I shipped to you. ;) | 21:56 |
| clarkb | that change seems fine I will +1 | 21:56 |
| mordred | I has +2 on dib!!!!! | 21:56 |
| mordred | does infra share core on that now? | 21:56 |
| *** tonytan4ever has quit IRC | 21:56 | |
| clarkb | mordred: no | 21:56 |
| mordred | oh. bother | 21:56 |
| mordred | I'm still bootstrapper | 21:56 |
| fungi | the original change was funcitonal, but fell victim to "it would be better if" bikesheddingh | 21:56 |
| * mordred fixes | 21:56 | |
| SpamapS | hm, we need to make diskimage-builder log which package installs file each package was mentioned in. | 21:56 |
| * fungi is badtyping | 21:56 | |
| *** bpokorny has joined #openstack-infra | 21:56 | |
| SpamapS | It's not actually possible to see what element adds a particular package now. | 21:57 |
| *** doug-fish has joined #openstack-infra | 21:57 | |
| *** Swami has quit IRC | 21:57 | |
| *** tonytan4ever has joined #openstack-infra | 21:57 | |
| *** cody-somerville has joined #openstack-infra | 21:58 | |
| mordred | it's important to remove oneself from project bootstrappers immediately after one force-pushes a change | 21:58 |
| *** thorst has quit IRC | 21:58 | |
| mordred | otherwise one might accidentally vote +2 on a project they don't have core in | 21:59 |
| clarkb | mordred: yes | 21:59 |
| *** xarses_ has joined #openstack-infra | 21:59 | |
| mordred | s/force-push/force-merge/ | 21:59 |
| *** jkraj has quit IRC | 21:59 | |
| *** FallenPegasus has joined #openstack-infra | 21:59 | |
| *** Hal has joined #openstack-infra | 22:00 | |
| *** Hal is now known as Guest14592 | 22:00 | |
| *** shashank_hegde has quit IRC | 22:01 | |
| *** doug-fish has quit IRC | 22:01 | |
| *** ybathia has joined #openstack-infra | 22:02 | |
| *** xarses has quit IRC | 22:02 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool: Pass extended network information in to occ/shade https://review.openstack.org/299679 | 22:02 |
| * mordred adds debugging | 22:02 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 22:02 | |
| *** sdake has joined #openstack-infra | 22:02 | |
| *** amitgandhinz has joined #openstack-infra | 22:04 | |
| jesusaur | jeblair: did you ever look into https://storyboard.openstack.org/#!/story/2000141 any more? | 22:04 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/infra-manual: update project creator's guide section on tag ACLs https://review.openstack.org/305502 | 22:04 |
| *** thiagop has quit IRC | 22:05 | |
| *** ssancheztrujillo has joined #openstack-infra | 22:05 | |
| *** amitgandhinz has quit IRC | 22:05 | |
| *** pots has quit IRC | 22:06 | |
| *** mriedem has quit IRC | 22:06 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources: Fix on data update https://review.openstack.org/305503 | 22:06 |
| *** tonytan4ever has quit IRC | 22:07 | |
| *** mylu_ has joined #openstack-infra | 22:07 | |
| *** harlowja has joined #openstack-infra | 22:07 | |
| *** mylu has quit IRC | 22:07 | |
| openstackgerrit | Merged openstack-infra/openstackid-resources: Fix on data update https://review.openstack.org/305503 | 22:07 |
| *** achanda has joined #openstack-infra | 22:08 | |
| jeblair | jesusaur: i have no memory of that. | 22:08 |
| openstackgerrit | James E. Blair proposed openstack-infra/infra-specs: Zuul v3: use Zookeeper for Nodepool-Zuul protocol https://review.openstack.org/305506 | 22:08 |
| jeblair | greghaynes, mordred, jhesketh: ^ | 22:08 |
| jeblair | clarkb: ^ | 22:08 |
| *** dteselkin_ has quit IRC | 22:09 | |
| *** toabctl has quit IRC | 22:09 | |
| mordred | jeblair: woot | 22:09 |
| *** rakhmerov has quit IRC | 22:09 | |
| mordred | jeblair: also, don't look too closely at the debugging patch I just pushed up | 22:09 |
| jeblair | it has hooks? | 22:10 |
| *** katyafervent_awa has quit IRC | 22:10 | |
| *** ssancheztrujillo has quit IRC | 22:10 | |
| openstackgerrit | Mikhail S Medvedev proposed openstack-infra/puppet-ciwatch: Add initial implementation to deploy ciwatch https://review.openstack.org/238606 | 22:10 |
| *** jzerebecki has quit IRC | 22:11 | |
| *** _fortis has quit IRC | 22:11 | |
| *** hdd has joined #openstack-infra | 22:11 | |
| *** burgerk has quit IRC | 22:11 | |
| *** katyafervent_awa has joined #openstack-infra | 22:11 | |
| *** mriedem1 has joined #openstack-infra | 22:12 | |
| *** ybathia has quit IRC | 22:13 | |
| *** rakhmerov has joined #openstack-infra | 22:13 | |
| *** vilobhmm11 has quit IRC | 22:14 | |
| *** vilobhmm11 has joined #openstack-infra | 22:14 | |
| *** achanda has quit IRC | 22:14 | |
| *** toabctl has joined #openstack-infra | 22:15 | |
| *** shashank_hegde has joined #openstack-infra | 22:15 | |
| *** jsavak has quit IRC | 22:15 | |
| *** Guest14592 has quit IRC | 22:16 | |
| openstackgerrit | Fei Long Wang proposed openstack-infra/project-config: Rename the rally job of Zaqar https://review.openstack.org/305511 | 22:16 |
| mordred | jeblair: it has one hook | 22:16 |
| *** jzerebecki has joined #openstack-infra | 22:17 | |
| *** hdd has quit IRC | 22:17 | |
| *** _fortis has joined #openstack-infra | 22:17 | |
| *** mriedem1 has quit IRC | 22:18 | |
| *** baoli has joined #openstack-infra | 22:18 | |
| *** tonytan4ever has joined #openstack-infra | 22:18 | |
| *** dteselkin has joined #openstack-infra | 22:21 | |
| *** dalgaaf has joined #openstack-infra | 22:22 | |
| dstufft | clarkb: should be | 22:22 |
| clarkb | dstufft: thanks | 22:23 |
| sc` | i didn't see any apparent way to do so, but is it possible for a bot to call out periodic build jobs' status when they pass or fail? | 22:23 |
| clarkb | sc`: EmilienM has started a ml thread about this. basically they can send email or you can consume results via the health dashboard | 22:24 |
| sc` | just checked my ml label. will read further | 22:24 |
| *** roxanaghe has quit IRC | 22:25 | |
| *** tonytan4ever has quit IRC | 22:25 | |
| *** pradk has quit IRC | 22:25 | |
| *** flwang has joined #openstack-infra | 22:26 | |
| flwang | hi team, is there any way to check which time slot is free for proposing a new IRC meeting time? thanks | 22:27 |
| clarkb | mordred: jeblair as either current or former TC members want to make sure my comment about AGPLv3 and global reqs in https://storyboard.openstack.org/#!/story/1352348 is correct or set the record straight if they are wrong? | 22:28 |
| clarkb | flwang: eavesdrop.openstack.org will give you an ics file you can load into a calendar then look for gaps | 22:28 |
| pleia2 | flwang: http://eavesdrop.openstack.org/ has a link to an ical feed that has everything you can load up in a calendar and see | 22:28 |
| pleia2 | can also search through that page for timeslots you want | 22:28 |
| fungi | sc`: if you hop over to #openstack-qa, EmilienM and mtreinish are implementing solutions for it already | 22:29 |
| fungi | you could probably whip up a bot which watches rss feeds and spams irc channels | 22:29 |
| flwang | clarkb: pleia2: thank you, guys | 22:30 |
| dhellmann | fungi : I believe we're ready to merge the tagging ACL change for newton: https://review.openstack.org/298866 | 22:30 |
| *** tonyb_ is now known as tonyb | 22:30 | |
| mtreinish | sc`: http://status.openstack.org/openstack-health/#/g/build_queue/periodic?groupKey=build_queue&resolutionKey=hour&end=2016-04-13T22:31:19.256Z | 22:31 |
| *** ddieterly is now known as ddieterly[away] | 22:32 | |
| openstackgerrit | Fei Long Wang proposed openstack-infra/project-config: Rename the rally job of Zaqar https://review.openstack.org/305511 | 22:35 |
| fungi | dhellmann: what's with the one acl that has stable branch management stuff added? | 22:36 |
| fungi | openstack/fuel-ui | 22:36 |
| flwang | clarkb: hi do you know who is maintaining the page http://eavesdrop.openstack.org/ ? | 22:37 |
| dhellmann | fungi : let me look, that might be a rebase issue | 22:37 |
| clarkb | flwang: the infrastructure team | 22:37 |
| flwang | though i can see Zaqar's time at the bottom, but i can't find a quick link for zaqar in Meeting details section | 22:37 |
| clarkb | flwang: the data is user generated though, let me get you a link to that file | 22:38 |
| jeblair | flwang: check out this repository: http://git.openstack.org/cgit/openstack-infra/irc-meetings/ | 22:38 |
| jeblair | http://git.openstack.org/cgit/openstack-infra/irc-meetings/tree/meetings/zaqar-team-meeting.yaml | 22:38 |
| *** ybathia has joined #openstack-infra | 22:38 | |
| clarkb | jeblair++ ty | 22:38 |
| flwang | so i should be able to edit the page with a patch, right? | 22:38 |
| jeblair | if there's a bug in the generation, it'll be in this project: http://git.openstack.org/cgit/openstack-infra/yaml2ical/ | 22:39 |
| *** brad_behle has quit IRC | 22:39 | |
| dhellmann | fungi : yeah, it looks like I caught part of cddf19002d635304a46eafc0ae03d90bc914455a in my rebase | 22:39 |
| flwang | jeblair: i think it's a bug | 22:39 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: update tag ACLs for all official projects https://review.openstack.org/298866 | 22:40 |
| jlvillal | flwang: Agreed on bug. tonyb and I are probably to blame... :( | 22:40 |
| dhellmann | fungi, AJaeger_ : ^^ | 22:41 |
| flwang | jlvillal: :) no worries, i think i should debug here http://git.openstack.org/cgit/openstack-infra/irc-meetings/tree/meetingindex.jinja#n37 right? | 22:41 |
| * tonyb pays attention ... | 22:41 | |
| jlvillal | tonyb: Look at very bottom of eavesdrop page. | 22:41 |
| jlvillal | See Zagar entry | 22:41 |
| *** akshai_ has quit IRC | 22:41 | |
| jlvillal | Then look up top and now Zaqar entry :( | 22:41 |
| jlvillal | s/now/no/ | 22:41 |
| jlvillal | flwang: I think that is the area of the error. | 22:42 |
| flwang | jlvillal: i will leave it to you guys :) | 22:42 |
| *** vilobhmm111 has joined #openstack-infra | 22:42 | |
| tonyb | jlvillal: Huh, strange. That's my bug I can see what's happened | 22:42 |
| flwang | if i can't see it's fixed after zaqar's new time merged, i will come back to bug you guys :D | 22:43 |
| tonyb | jlvillal: no idea how to fix it just yet :/ | 22:43 |
| jlvillal | tonyb: Me either, but I haven't looked at it. Starting now | 22:43 |
| tonyb | jlvillal: I knew my "batch processing" was too simple | 22:43 |
| tonyb | jlvillal: It'll be a yaml2ical fix | 22:44 |
| * jlvillal pulls tip | 22:44 | |
| *** vilobhmm11 has quit IRC | 22:44 | |
| *** ybathia has quit IRC | 22:45 | |
| * jlvillal gasps in horror when he realizes there are no unit tests.... ;) | 22:45 | |
| *** ybathia has joined #openstack-infra | 22:46 | |
| *** ybathia has quit IRC | 22:46 | |
| fungi | wow, linux wheels of numpy on pypi now! | 22:47 |
| openstackgerrit | Fei Long Wang proposed openstack-infra/project-config: Rename the rally job of Zaqar https://review.openstack.org/305511 | 22:47 |
| *** resker has joined #openstack-infra | 22:48 | |
| SpamapS | fungi: o_O | 22:48 |
| *** avarner has quit IRC | 22:48 | |
| *** sdake has quit IRC | 22:48 | |
| fungi | SpamapS: need pip >=8.1 but you can just pip install numpy without even needing a compiler | 22:49 |
| SpamapS | errrmahgerd | 22:50 |
| *** claudiub has quit IRC | 22:50 | |
| *** FallenPegasus has quit IRC | 22:50 | |
| fungi | i just did it | 22:51 |
| fungi | took 5 seconds | 22:51 |
| *** roxanaghe has joined #openstack-infra | 22:51 | |
| *** esker has quit IRC | 22:51 | |
| *** akshai has joined #openstack-infra | 22:51 | |
| *** akshai_ has joined #openstack-infra | 22:52 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool: Pass extended network information in to occ/shade https://review.openstack.org/299679 | 22:54 |
| *** |-paul-| has quit IRC | 22:54 | |
| *** ddieterly[away] is now known as ddieterly | 22:55 | |
| jeblair | krotscheck_dcm, mordred, nibalizer: if we expect the npm archive to be 200G, we may want to increase the quota of the mirror.npm volume | 22:55 |
| jeblair | i'll seet it to 400G | 22:55 |
| *** akshai has quit IRC | 22:56 | |
| mordred | fungi: watching builds ... we could probably put language-pack-en into base images | 22:56 |
| *** kzaitsev_mb has quit IRC | 22:56 | |
| jeblair | vos release finished after 213m, i'm running it again | 22:57 |
| mordred | fungi: generating the locales takes 12 seconds and is completely not valuable to do for each test run | 22:57 |
| mordred | jeblair: awesome | 22:57 |
| *** dimtruck is now known as zz_dimtruck | 22:57 | |
| *** ddieterly is now known as ddieterly[away] | 22:57 | |
| *** ddieterly[away] is now known as ddieterly | 22:58 | |
| *** ddieterly has quit IRC | 22:58 | |
| mordred | jeblair: sigh. this time it passed | 22:59 |
| *** sdake has joined #openstack-infra | 23:00 | |
| nibalizer | jeblair: groovy | 23:02 |
| *** andymccr_ is now known as andymccr | 23:02 | |
| openstackgerrit | Fei Long Wang proposed openstack-infra/project-config: Rename the rally job of Zaqar https://review.openstack.org/305511 | 23:03 |
| *** akshai_ has quit IRC | 23:03 | |
| *** rbrndt has quit IRC | 23:05 | |
| *** dingyichen has joined #openstack-infra | 23:05 | |
| craige | o/ | 23:06 |
| ianw | clarkb: -rw-r--r--. 1 root root 102M Apr 13 22:58 locale-archive <- that's the size of all locales fyi. i think i was underestimating :/ | 23:06 |
| openstackgerrit | James E. Blair proposed openstack-infra/puppet-kerberos: Add an @ sign because puppet says to https://review.openstack.org/305518 | 23:08 |
| openstackgerrit | James E. Blair proposed openstack-infra/puppet-jeepyb: Protect gcc package from duplicate declaration https://review.openstack.org/305519 | 23:10 |
| *** doug-fish has joined #openstack-infra | 23:10 | |
| *** otherwiseguy has quit IRC | 23:10 | |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Install afs on all servers https://review.openstack.org/305477 | 23:10 |
| *** achanda has joined #openstack-infra | 23:11 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid: Upgrade to LV 5.2 https://review.openstack.org/305521 | 23:12 |
| openstackgerrit | James E. Blair proposed openstack-infra/puppet-subunit2sql: Ensure all jobs send a completion packet https://review.openstack.org/304303 | 23:12 |
| *** angdraug has quit IRC | 23:12 | |
| *** flepied1 has quit IRC | 23:12 | |
| *** xyang1 has quit IRC | 23:13 | |
| *** ams_ has quit IRC | 23:13 | |
| *** baoli has quit IRC | 23:14 | |
| *** doug-fish has quit IRC | 23:14 | |
| *** baoli has joined #openstack-infra | 23:15 | |
| *** achanda has quit IRC | 23:16 | |
| mordred | jeblair: do you have any thoughts about how ssh + kinit + afs might work for AFS home dirs for us? | 23:17 |
| mordred | let me rephrase | 23:17 |
| mordred | jeblair: as I'm sure you've thought about that, what are your thoughts on how that works? | 23:17 |
| clarkb | mordred: jeblair: two things, what changed to make afs homedirs desireable? and can we make sure that we further automate our afs stuff before relying on it for more than mirrors? | 23:18 |
| clarkb | (also is the afs wire protocol in the clear?) | 23:18 |
| mordred | clarkb: afs homedirs are awesome | 23:18 |
| *** Sukhdev has quit IRC | 23:18 | |
| mordred | clarkb: they let you have the same homedir everywhere | 23:18 |
| clarkb | mordred: ya I et that, but I have never needed that | 23:19 |
| clarkb | because my primary work device is not our cloud servers | 23:19 |
| *** rvba has quit IRC | 23:19 | |
| *** otherwiseguy has joined #openstack-infra | 23:19 | |
| jeblair | mordred: i have not thought that much about it -- i will be happy to achieve the interim step of just having access to a user dir even if it's not my homedir and maybe actually make it a real homedir later | 23:19 |
| mordred | jeblair: ++ | 23:19 |
| mordred | k | 23:19 |
| clarkb | so I am curious if something made that change? | 23:19 |
| jeblair | clarkb: it's in production now, and i thought it would be useful. re-populating my homedir on nodepool made me think of it. | 23:20 |
| * mordred has wanted it the whole time - just not enough to do work around it | 23:21 | |
| jeblair | clarkb: (i wasn't going to advocate using it on many/all of our servers before we used it for anything real production like, but we are now) | 23:21 |
| clarkb | ok, having tried to poke at the ceph mirror and getting distracted before I can figure afs out I really think we need better automation around afs management before we use it for more than mirrors | 23:21 |
| clarkb | and as far as homedirs go if the wire protocl is in the clear we must be careful about using clouds.yaml and friends | 23:22 |
| clarkb | but I can't find info with a quick google if it is in the clear or not | 23:22 |
| *** rvba has joined #openstack-infra | 23:23 | |
| jeblair | clarkb: the protocol is in the clear unless you 'fs setencrypt' in which case it is, erm, modified des. while not broken due to obscurity, it is not particularly private. i would not put sensitive information on it. it is, however, secure (that's via kerberos). | 23:23 |
| clarkb | (I did find an old security advisory for leaked kernel memory over the wire :)) | 23:23 |
| *** rvba has quit IRC | 23:23 | |
| *** rvba has joined #openstack-infra | 23:23 | |
| clarkb | ok, so we would need to make sure that rooters understand that sensitive data like clouds.yaml shold stay out of homedirs | 23:25 |
| *** angdraug has joined #openstack-infra | 23:25 | |
| *** sdague has quit IRC | 23:25 | |
| nibalizer | so it can't be tampered with or it can? | 23:26 |
| *** sshnaidm has quit IRC | 23:26 | |
| jeblair | clarkb: we are probably sufficiently paranoid for that to, yes. | 23:26 |
| nibalizer | because someone putting something nasty in my bashrc would be pretty bad | 23:26 |
| clarkb | nibalizer: you control that with the afs acls | 23:27 |
| jeblair | nibalizer: not tampered, possibly observed if you want to crack some weird old encryption. | 23:27 |
| *** angdraug has quit IRC | 23:27 | |
| nibalizer | this just doesn't seem that valuable | 23:27 |
| jeblair | what doesn't? | 23:27 |
| nibalizer | maybe it is though, who knows | 23:27 |
| nibalizer | afs homedirs | 23:27 |
| jeblair | oh. well | 23:27 |
| mordred | that's fine. I mean, I don't think jeblair or I are saying that everyone should be required to have afs homedirs | 23:28 |
| jeblair | i'm not actually proposing that. | 23:28 |
| nibalizer | oh ok | 23:28 |
| flwang | hi anybody can give me an idea on this one https://review.openstack.org/#/c/305511 to avoid /me playing with infra many times :) thanks | 23:28 |
| jeblair | i just want it on the servers so i can copy things around. | 23:28 |
| mordred | ++ | 23:28 |
| nibalizer | i would worry about ansible and other utilities scratching .cache files | 23:28 |
| jeblair | i'll probably symlink ~corvus/afs to /afs/openstack.org/users/corvus | 23:28 |
| nibalizer | we had a lot of trouble with that when we ran nfs homedirs | 23:28 |
| *** ddieterly has joined #openstack-infra | 23:28 | |
| nibalizer | thats good | 23:28 |
| nibalizer | having a document store everywhere is much less likely to break than having actual ~ | 23:29 |
| jeblair | someday, i may suggest real afs homedirs, but only after i'm prepared to advocate for it. because, yeah, there are lots of things to consider. :) | 23:29 |
| clarkb | flwang: http://logs.openstack.org/11/305511/4/check/gate-project-config-layout/2938159/console.html#_2016-04-13_23_04_16_016 is the error | 23:29 |
| jeblair | but for now, i'm going for best of both worlds -- added convenience without added problems | 23:29 |
| clarkb | flwang: if you search in project-config/jenkins/jobs does that job template exist? | 23:30 |
| jeblair | clarkb: re automation -- lots of stuff could be automated in afs, but doing so may have diminishing returns. the problem is similar to the ansible-launch-node problem (think attaching cinder volumes to servers, etc). having said that, we are establishing some patterns, and we could probably write an ansible playbook to, say, 'create a new kind of mirror'. | 23:32 |
| flwang | clarkb: yep, i think i know why it's complaining. since there is no job defined in zaqar job template. but i'm still don't know how to fix the original issue. now in zaqar, the rally job is using name gate-rally-dsvm-zaqar-zaqar | 23:32 |
| *** watersoul_ has quit IRC | 23:33 | |
| jeblair | clarkb: but also, i'm always happy to help. i think global filesystems are cool. :) | 23:33 |
| flwang | i would like to see gate-rally-dsvm-zaqar， i'm a neat freak :) | 23:33 |
| *** shashank_hegde has quit IRC | 23:34 | |
| *** watersoul has joined #openstack-infra | 23:34 | |
| clarkb | flwang: edit the existing template to remove the extra name | 23:34 |
| clarkb | jeblair: ya, its mostly that with everything else going on I haven't had time to spend the day it would require to get it done | 23:34 |
| clarkb | jeblair: and it would be nice if it was a 15 minute thing instead of a day of clarkb braindumping | 23:34 |
| openstackgerrit | Fei Long Wang proposed openstack-infra/project-config: Rename the rally job of Zaqar https://review.openstack.org/305511 | 23:36 |
| *** Qiming has joined #openstack-infra | 23:37 | |
| *** shashank_hegde has joined #openstack-infra | 23:37 | |
| jeblair | clarkb: yeah. once you know it, it won't take a day (i'd allocate 20 minutes to the currently-manual afs steps to create a completely new mirror, then allocate a few days to watch it sync :). but there's a tradeoff with how much of the fundamentals you want to learn. | 23:37 |
| *** zhurong has joined #openstack-infra | 23:38 | |
| *** lezbar__ has joined #openstack-infra | 23:38 | |
| *** andreww has joined #openstack-infra | 23:39 | |
| mordred | jeblair: so, what are your feelings towards this nodepool patch? | 23:40 |
| *** lezbar has quit IRC | 23:40 | |
| jeblair | mordred: i can't imagine why the problem would appear with that patch. i'm worried it's something about occ. | 23:41 |
| mordred | jeblair: yah. I've been walking through the OCC code with a comb looking for stuff | 23:41 |
| jeblair | i don't object to landing it per-se, but i'm worried that we're about to have lots of trouble landing nodepool changes. | 23:41 |
| jeblair | mordred: (i have looked too and nothing has jumped out at me) | 23:41 |
| mordred | jeblair: but we've also been using occ in nodepool for a while and haven't had any issues | 23:42 |
| clarkb | what does the crowd think about closing https://storyboard.openstack.org/#!/story/1311066 we have the thing we can fingerprint and track on | 23:42 |
| jeblair | mordred: didn't we just update? | 23:42 |
| *** salv-orlando has quit IRC | 23:42 | |
| clarkb | but otherwise we would never close that bug bceause yes indeed sometimes things are slow | 23:42 |
| *** xarses_ has quit IRC | 23:42 | |
| mordred | well, occ did release a few days | 23:42 |
| jeblair | clarkb: yeah, i think that was probably a tracking bug which is now duplicated in openstack-gate | 23:43 |
| mordred | ago | 23:43 |
| jeblair | mordred: yeah, that's what i was thinking | 23:43 |
| jeblair | mordred: it may be useful to push up a noop test to nodepool and compare | 23:43 |
| mordred | let me look at the difference | 23:43 |
| *** rfolco__ has quit IRC | 23:44 | |
| mordred | jeblair: since 1.15.0 the only difference in substance in the main files is the additoin of the network list processing | 23:45 |
| *** watersoul has joined #openstack-infra | 23:45 | |
| clarkb | jeblair: kk marked merged due to tracking being put in place | 23:45 |
| *** ybathia has joined #openstack-infra | 23:46 | |
| clarkb | jeblair: zuul can handle or does handle https://storyboard.openstack.org/#!/story/1211717 now right? any chance you can update it | 23:46 |
| mordred | jeblair: os_client_config/config.py and os_client_config/cloud_config.py being the two, in case you also want to look | 23:46 |
| mordred | jeblair: it seems that occ is properly pulling the network list from the config dict and putting it back in a way that should not affect the overall dict | 23:47 |
| mordred | jeblair: do you think there is any chance that in a multi-threaded context, the pattern of config = something(config) could be breaking anything? | 23:48 |
| jeblair | clarkb: i don't think we do anything about that yet? | 23:48 |
| mordred | jeblair: (where config is a dict) | 23:49 |
| clarkb | jeblair: did the feature to zuul merge? we could address it by using the lock stuff around the tarball jobs? | 23:49 |
| jeblair | clarkb: (we've talked about adding a new kind of manager to only run at most one post job set for a project at a time, which would handle it) | 23:49 |
| jeblair | clarkb: we could mutex it; it wouldn't be pretty though -- we'd need a mutex per-project. | 23:50 |
| jeblair | i'd rather solve it the other way. | 23:50 |
| clarkb | gotcha | 23:50 |
| *** ajmiller has joined #openstack-infra | 23:50 | |
| *** zhurong has quit IRC | 23:51 | |
| pleia2 | alright, I'm heading out to the faraway land of Pleasanton to go see thingee chat about cinder things | 23:51 |
| pleia2 | I'll check in on bug day when I return | 23:51 |
| clarkb | have fun | 23:51 |
| clarkb | I am about to pack up laptop and change physical locations myself | 23:52 |
| *** ddieterly is now known as ddieterly[away] | 23:52 | |
| clarkb | system-config is getting chewed through. Good mix of "hey we fixed this" and "this belongs in JJB" and "this is not an infra issue" | 23:52 |
| pleia2 | \o/ | 23:52 |
| pleia2 | thanks for diving into that mess | 23:52 |
| *** ajmiller_ has joined #openstack-infra | 23:52 | |
| openstackgerrit | John L. Villalovos proposed openstack-infra/yaml2ical: Fix off by 1 error in batch_messages & add unit tests https://review.openstack.org/305528 | 23:54 |
| jlvillal | tonyb: ^^^^ | 23:54 |
| tonyb | jlvillal: danke! | 23:54 |
| jlvillal | tonyb: My pleasure. Also if you have a good idea for the docstring, feel free to comment. | 23:55 |
| *** ajmiller has quit IRC | 23:55 | |
| *** FallenPegasus has joined #openstack-infra | 23:56 | |
| fungi | mordred: i'm hearing rumblings from downstream that pbr 1.9 now barfs if you have non-semver tags in your history | 23:57 |
| mordred | lifeless: ^^ | 23:58 |
| mordred | fungi: I do not believe that is supposed to be the case | 23:58 |
| *** mylu_ has quit IRC | 23:58 | |
| mordred | fungi: but lifeless would know better than I | 23:58 |
| mordred | or Nakato ^^ | 23:58 |
| *** mylu has joined #openstack-infra | 23:58 | |
| *** amotoki has joined #openstack-infra | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!