| *** zhurong has quit IRC | 00:00 | |
| *** jistr has quit IRC | 00:00 | |
| *** jistr has joined #openstack-infra | 00:01 | |
| *** spzala has quit IRC | 00:02 | |
| *** timello has quit IRC | 00:02 | |
| pabelanger | fungi: Hmm, I am not sure then | 00:02 |
|---|---|---|
| fungi | pabelanger: looking at the graph, we saw zuul catch back up around this time yesterday, so maybe it's about to start burning down today's backlog? it's gotten pretty far down on check already | 00:02 |
| *** spzala has joined #openstack-infra | 00:02 | |
| *** spzala has joined #openstack-infra | 00:03 | |
| pabelanger | fungi: right, but I'm not sure why we'd have 72 ubuntu-trusty ready nodes, when we have min-ready of 10 | 00:03 |
| *** spzala has quit IRC | 00:03 | |
| fungi | pabelanger: i agree it seems to be miscalculating the number of trusty nodes it needs to prepare for the current job demand, but i don't know that "just restart zuul" is necessarily the solution this time | 00:04 |
| * pabelanger nods | 00:04 | |
| *** timello has joined #openstack-infra | 00:04 | |
| fungi | as the explanation we had for it last time this happened doesn't seem to be the case this time | 00:04 |
| pabelanger | Deficit: ubuntu-trusty: 0 (start: 239 min-ready: 10 ready: 250 capacity: 22) | 00:05 |
| fungi | but i don't have a good theory on the cause at the moment, and i need to disappear. hopefully someone further west has some input before they reach end of day | 00:05 |
| pabelanger | I guess I should better understand how the start value is calculated | 00:05 |
| *** nwkarsten has joined #openstack-infra | 00:06 | |
| fungi | i have a feeling it's something we've done with recent nodepool patches | 00:06 |
| fungi | since this time, the basic demand numbers look like what i would expect, but nodepool is not acting on the demand it's calculating in the way i would expect | 00:07 |
| fungi | the xenial demand is reported as being an order of magnitude greater than the trusty demand | 00:07 |
| *** kzaitsev_mb has quit IRC | 00:08 | |
| *** gouthamr has joined #openstack-infra | 00:08 | |
| fungi | the base deficits also look reasonable for the present state of zuul | 00:08 |
| fungi | 2016-08-10 00:07:41,758 DEBUG nodepool.NodePool: Deficit: ubuntu-xenial: 2036 (start: 2291 min-ready: 10 ready: 255 capacity: 32) | 00:08 |
| fungi | i can believe it has ~2k pending jobs for xenial nodes, based on some spot checking of changes sitting there in various pipelines | 00:09 |
| pabelanger | ya | 00:09 |
| fungi | the question is why it seems to keep booting more trusty nodes anyway, and mostly ignoring the xenial deficit it's )seemingly accurately) reporting | 00:09 |
| *** zz_dimtruck is now known as dimtruck | 00:10 | |
| fungi | but i need to go disappear | 00:10 |
| pabelanger | ack | 00:10 |
| fungi | i'm not entirely opposed to a zuul restart, i just think it would be a shame to lose the current post pipeline backlog and then discover that a restart didn't actually help this time | 00:10 |
| *** sputnik13 has quit IRC | 00:10 | |
| pabelanger | agreed | 00:11 |
| pabelanger | however being down 90 nodes I don't think we'll clear it out today | 00:12 |
| pabelanger | I'll keep working on bluebox and try to get everything in working order for tomorrow | 00:12 |
| fungi | well, things have turned around. now that it's got ready trusty nodes, at least it's currently building several times as many new xenial nodes than trusty nodes now | 00:13 |
| fungi | ~8x as many in fact | 00:13 |
| *** Apoorva has quit IRC | 00:13 | |
| *** timello has quit IRC | 00:13 | |
| anteaya | pabelanger: and thanks for keeping an eye on things :) | 00:14 |
| openstackgerrit | Henry Gessau proposed openstack-dev/cookiecutter: Enable Python DeprecationWarning in test environments https://review.openstack.org/353197 | 00:14 |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update cfp builder to use convert xml https://review.openstack.org/353198 | 00:15 |
| *** sin_ has quit IRC | 00:16 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add n1.large flavor for bluebox https://review.openstack.org/353199 | 00:17 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Update admin-bluebox with new settings https://review.openstack.org/353055 | 00:17 |
| *** timello has joined #openstack-infra | 00:18 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Update admin-bluebox with new settings https://review.openstack.org/353055 | 00:21 |
| *** tonytan4ever has quit IRC | 00:22 | |
| *** mtanino has quit IRC | 00:23 | |
| *** mrhillsman is now known as mrhillsman_ | 00:24 | |
| *** mrhillsman_ is now known as mrhillsman_Away | 00:24 | |
| *** _ari_ has quit IRC | 00:25 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Use n1.large for bluebox-sjc1 https://review.openstack.org/353201 | 00:26 |
| *** kzaitsev_mb has joined #openstack-infra | 00:27 | |
| *** timello has quit IRC | 00:27 | |
| *** timello has joined #openstack-infra | 00:27 | |
| *** yamamot__ has joined #openstack-infra | 00:27 | |
| *** tphummel has quit IRC | 00:30 | |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update findbugs_settings to use convert xml https://review.openstack.org/353204 | 00:30 |
| *** Julien-zte has joined #openstack-infra | 00:31 | |
| *** spzala has joined #openstack-infra | 00:33 | |
| *** sarob has joined #openstack-infra | 00:33 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Update admin-bluebox with new settings https://review.openstack.org/353055 | 00:33 |
| *** yamamot__ has quit IRC | 00:34 | |
| *** timello has quit IRC | 00:36 | |
| nibalizer | oh hey | 00:38 |
| *** sarob has quit IRC | 00:38 | |
| nibalizer | i now have administrative access to 'OpenStack Foundation Upstream CICD-SJC (formerly montytaylor-sjc)' | 00:38 |
| nibalizer | thanks bluebox, jlk and etc | 00:38 |
| jlk | nice | 00:38 |
| pabelanger | is that another cloud? | 00:39 |
| pabelanger | I'm working on montytaylor-sjc ATM | 00:39 |
| nibalizer | hrm | 00:39 |
| nibalizer | I have a dashboard that says 'Your Blue Box Clouds' | 00:39 |
| pabelanger | https://review.openstack.org/#/q/topic:bluebox-nodepool is the topic for my open patches right now | 00:39 |
| nibalizer | and lists 'OpenStack Foundation Upstream CICD-SJC (formerly montytaylor-sjc)' as one of my clouds (my only cloud) | 00:39 |
| pabelanger | nibalizer: any projects created? | 00:40 |
| pabelanger | eg: openstackzuul | 00:40 |
| nibalizer | pabelanger: this is admin at the bluebox layer I think | 00:40 |
| nibalizer | not the openstack layer | 00:40 |
| pabelanger | Ah, I see | 00:40 |
| pabelanger | neat then | 00:41 |
| nibalizer | yea, so I can hopefully help us get this back into utility | 00:41 |
| * nibalizer really needs to start helping out in here again | 00:41 | |
| anteaya | nibalizer: we miss you | 00:41 |
| pabelanger | for sure, were using rcarrillocruz ansible-role-cloud-launcher to bring up everything now | 00:41 |
| nibalizer | totes | 00:41 |
| *** timello has joined #openstack-infra | 00:41 | |
| pabelanger | it is working great | 00:41 |
| nibalizer | im going to a thing, but when I return or maybe thursday we can bang on this | 00:41 |
| nibalizer | you have credentials on the openstack right now right? I don't have to do anything yet? | 00:42 |
| pabelanger | yes, should be lauching AFS servers tonight or first thing in the morning | 00:42 |
| nibalizer | qeqs | 00:42 |
| nibalizer | wewt | 00:42 |
| nibalizer | ttyl | 00:42 |
| *** gouthamr_ has joined #openstack-infra | 00:43 | |
| jeblair | pabelanger: so that i understand -- nodepool sees ubuntu-trusty demand as 169 right now, but you think there is none? | 00:43 |
| pabelanger | jeblair: I think there is 86 jobs running ubuntu-trusty | 00:45 |
| pabelanger | jeblair: because we have 83 ubuntu-trusty nodes in ready state | 00:46 |
| *** vvalderrv has joined #openstack-infra | 00:46 | |
| *** gouthamr has quit IRC | 00:47 | |
| *** tonytan4ever has joined #openstack-infra | 00:48 | |
| jeblair | pabelanger, fungi: we have many jobs registered with both ubuntu-trusty and ubuntu-xenial | 00:50 |
| jeblair | most of them releasenotes | 00:50 |
| jeblair | did we change the label for that job recently? | 00:50 |
| *** timello has quit IRC | 00:50 | |
| *** timello has joined #openstack-infra | 00:51 | |
| pabelanger | let me check | 00:51 |
| fungi | jeblair: oh, i had searched for jobs ending in -trusty registered for ubuntu-xenial nodes or vice versa | 00:51 |
| fungi | for which we seem to have none | 00:51 |
| *** valderrv has joined #openstack-infra | 00:52 | |
| fungi | though, yes, there are likely some jobs which aren't suffixed and got switched from trusty to xenial recently. those could certainly be throwing off the demand counts some | 00:53 |
| pabelanger | jeblair: I believe we merged that yesterday, 349181. which changed our nodes from ubuntu-trusty to ubuntu-xenial | 00:54 |
| fungi | but before we caught up on outstanding trusty jobs, even though nodepool was reporting way more demand for xenial nodes it was continuing to boot many more trusty nodes than xenial | 00:54 |
| jeblair | the current demand is coming from the releasenotes job (i ran the allocator manually to confirm) | 00:55 |
| jeblair | there is a single node registered with gearman that can run *-releasenotes:ubuntu-trusty | 00:55 |
| jeblair | all the others are xenial | 00:55 |
| pabelanger | ah | 00:55 |
| jeblair | so we'll need to delete that one before restarting zuul | 00:56 |
| jeblair | (otherwise it will re-register and we'll be right back here) | 00:56 |
| jeblair | ubuntu-trusty-ovh-bhs1-3139002 | 00:56 |
| fungi | that makes sense and accounts for the current extra trusty ready nodes | 00:56 |
| *** csomerville has quit IRC | 00:56 | |
| jeblair | it doesn't show up in nodepool's list, so is probably a ghost connection | 00:57 |
| fungi | though i'd be surprised if that alone accounts for the significant imbalance we were seeing between demand ratios and building ratios | 00:57 |
| jeblair | fungi: the imbalence is likely due to the intentional skewing of the demand algorithm to favor lesser-used nodes | 00:57 |
| jeblair | but that shouldn't extend to creating nodes that aren't needed, like here | 00:58 |
| *** sdake has joined #openstack-infra | 00:59 | |
| jeblair | so anyway, i think at this point a zuul restart should take care of it. otoh, this situation isn't too crazy. demand is already decreasing significantly as it works through the backlog. we could probably periodically delete the extra ready nodes and run it down. | 00:59 |
| fungi | does the ratio between min-ready values factor into it too? are we maybe overdue to rebalance min-ready for xenial vs trusty? | 00:59 |
| *** timello has quit IRC | 00:59 | |
| jeblair | fungi: i'm not sure anymore :) | 01:00 |
| fungi | factor into how it compares demand vs start ratios that is | 01:00 |
| *** timello has joined #openstack-infra | 01:00 | |
| openstackgerrit | Merged openstack-infra/project-config: Add Fedora 24 work-around kernel https://review.openstack.org/352211 | 01:00 |
| *** dimtruck is now known as zz_dimtruck | 01:01 | |
| fungi | hrm, min-ready for ubuntu-trusty and ubuntu-xenial are both 10 right now, so probably not a factor | 01:02 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add n1.large flavor for bluebox https://review.openstack.org/353199 | 01:02 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Update admin-bluebox with new settings https://review.openstack.org/353055 | 01:02 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add bluebox to cloud_layouts https://review.openstack.org/353013 | 01:02 |
| *** amitgandhinz has joined #openstack-infra | 01:03 | |
| *** raunak has quit IRC | 01:03 | |
| *** pvaneck has quit IRC | 01:04 | |
| anteaya | yeah we split the min ready between trusty and xenial last week as part of the move the things to xenial direction | 01:04 |
| anteaya | it was 20 on trusty and about 3 on xenial before that | 01:05 |
| *** Julien-zte has quit IRC | 01:05 | |
| fungi | at this point we're probably tipped over to mostly running jobs on xenial | 01:05 |
| anteaya | https://review.openstack.org/#/c/347573/ | 01:05 |
| fungi | or not, i haven't been following closely | 01:05 |
| anteaya | sorry that was 2 weeks ago | 01:06 |
| *** zz_dimtruck is now known as dimtruck | 01:06 | |
| anteaya | nor have I | 01:06 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add gate-python-tripleoclient-buildimage jobs https://review.openstack.org/351278 | 01:06 |
| *** Sukhdev has quit IRC | 01:06 | |
| *** kzaitsev_mb has quit IRC | 01:07 | |
| *** zhurong has joined #openstack-infra | 01:07 | |
| pabelanger | fungi: we are close I think | 01:07 |
| *** hongbin_ has joined #openstack-infra | 01:08 | |
| *** amitgandhinz has quit IRC | 01:08 | |
| pabelanger | we still have tox-db to migrate | 01:08 |
| *** Julien-zte has joined #openstack-infra | 01:09 | |
| *** timello has quit IRC | 01:09 | |
| *** timello has joined #openstack-infra | 01:09 | |
| pabelanger | ianw: fungi: jeblair: jhesketh: If you don't mind help land https://review.openstack.org/#/c/353055/ this evening? Will help with some pre tasks bringing our bluebox cloud online for nodepool | 01:10 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add DB API function to get list of runs from ids https://review.openstack.org/352558 | 01:12 |
| jhesketh | pabelanger: sure, looking | 01:13 |
| jeblair | pabelanger: i have a nit on https://review.openstack.org/353199 | 01:14 |
| jhesketh | pabelanger: +W | 01:14 |
| jeblair | pabelanger: not too particular on the name, but would prefer something meaningful if we can :) | 01:14 |
| ianw | jhesketh: hey, i was looking at that ;) | 01:14 |
| ianw | i'll take it though, nice position to be in when reviews are happening *too* fast :) | 01:15 |
| jhesketh | heh :-) | 01:16 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/devstack-gate: Add sanity check on devstack generated tempest.conf https://review.openstack.org/353060 | 01:16 |
| jhesketh | need some kind of review lock | 01:16 |
| *** xnox has quit IRC | 01:17 | |
| openstackgerrit | Merged openstack-infra/subunit2sql: Add option to ensure get_test_runs_by_test_id() is ordered by date https://review.openstack.org/352506 | 01:18 |
| *** Apoorva has joined #openstack-infra | 01:19 | |
| *** timello has quit IRC | 01:20 | |
| *** timello has joined #openstack-infra | 01:20 | |
| *** gongysh has quit IRC | 01:21 | |
| pabelanger | jeblair: replied. But same, I went with n1 since we have m1 today | 01:21 |
| pabelanger | will switch to nodepool | 01:22 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add nodepool flavor for bluebox https://review.openstack.org/353199 | 01:22 |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update helpers.artifactory_optional_props https://review.openstack.org/353208 | 01:24 |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update helpers.artifactory https://review.openstack.org/353208 | 01:25 |
| pabelanger | jeblair: fungi: Did we decided to restart zuul or just leave it alone for now? | 01:26 |
| *** tphummel has joined #openstack-infra | 01:28 | |
| *** timello has quit IRC | 01:29 | |
| *** timello has joined #openstack-infra | 01:31 | |
| *** karthik__ has quit IRC | 01:34 | |
| ianw | fyi i'm about to kick off an Fedora 24 build to hopefully get a node we can run tempest on | 01:36 |
| *** yanyanhu has joined #openstack-infra | 01:38 | |
| *** _ari_ has joined #openstack-infra | 01:39 | |
| *** _ari_ has quit IRC | 01:40 | |
| *** amotoki has joined #openstack-infra | 01:40 | |
| *** timello has quit IRC | 01:40 | |
| *** spzala has quit IRC | 01:42 | |
| *** timello has joined #openstack-infra | 01:43 | |
| *** gongysh has joined #openstack-infra | 01:43 | |
| *** xnox has joined #openstack-infra | 01:43 | |
| *** xnox has quit IRC | 01:46 | |
| *** xnox has joined #openstack-infra | 01:48 | |
| *** kushal has joined #openstack-infra | 01:48 | |
| *** amotoki has quit IRC | 01:48 | |
| *** mrhillsman_Away is now known as mrhillsman | 01:50 | |
| *** timello has quit IRC | 01:52 | |
| *** timello has joined #openstack-infra | 01:53 | |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Fix issue that cause JCloud instance to be set https://review.openstack.org/353211 | 01:56 |
| *** amotoki has joined #openstack-infra | 01:58 | |
| *** raunak has joined #openstack-infra | 01:59 | |
| *** amotoki has quit IRC | 01:59 | |
| *** amotoki has joined #openstack-infra | 02:00 | |
| *** valderrv has quit IRC | 02:03 | |
| *** vvalderrv has quit IRC | 02:04 | |
| *** amitgandhinz has joined #openstack-infra | 02:05 | |
| *** timello has quit IRC | 02:05 | |
| *** timello has joined #openstack-infra | 02:06 | |
| *** tphummel has quit IRC | 02:06 | |
| *** flepied has joined #openstack-infra | 02:09 | |
| *** amitgandhinz has quit IRC | 02:09 | |
| *** ayoung has quit IRC | 02:11 | |
| *** claudiub has quit IRC | 02:14 | |
| *** raunak_ has joined #openstack-infra | 02:14 | |
| *** timello has quit IRC | 02:14 | |
| fungi | pabelanger: probably would help clear out the release notes job double-registration, but might want to see if jhesketh can tackle that once the post pipeline burns down (disclaimer, i'm not on a computer where i can easily check the zuul status, so not sure how far it's got to go for that) | 02:15 |
| jhesketh | fungi: post pipeline is 293 long | 02:16 |
| jhesketh | I haven't read the scrollback, what's the double-registration issue? | 02:16 |
| *** raunak has quit IRC | 02:17 | |
| *** raunak_ is now known as raunak | 02:17 | |
| *** timello has joined #openstack-infra | 02:17 | |
| openstackgerrit | Isaku Yamahata proposed openstack-infra/project-config: networking-odl: cover more combinations of version https://review.openstack.org/347045 | 02:20 |
| fungi | jhesketh: probably mild, but there's a stale ubuntu-trusty registration for release-notes jobs which were recently switched to ubuntu-xenial | 02:20 |
| fungi | so nodepool is allocating trusty nodes which won't be used for a while | 02:21 |
| fungi | anyway, i'm hitting the hay, night all | 02:21 |
| jhesketh | how come a zuul reload doesn't solve that? | 02:21 |
| jhesketh | fungi: did you want me to restart zuul overnight? | 02:22 |
| *** tphummel has joined #openstack-infra | 02:26 | |
| *** timello has quit IRC | 02:26 | |
| *** timello has joined #openstack-infra | 02:27 | |
| *** senk_ has joined #openstack-infra | 02:28 | |
| *** Apoorva has quit IRC | 02:29 | |
| *** aviau has quit IRC | 02:34 | |
| *** aviau has joined #openstack-infra | 02:34 | |
| *** sarob has joined #openstack-infra | 02:34 | |
| *** PollyZ has joined #openstack-infra | 02:35 | |
| *** timello has quit IRC | 02:36 | |
| *** sarob has quit IRC | 02:38 | |
| clarkb | because that doesnt affect gearman | 02:40 |
| clarkb | need to restart that service iirc | 02:41 |
| anteaya | 299 in post with a 20 hour wait time for the top patches | 02:41 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/project-config: Remove the ironic-parallel-nv from tempest experimental https://review.openstack.org/353225 | 02:41 |
| *** zhurong has quit IRC | 02:42 | |
| *** timello has joined #openstack-infra | 02:42 | |
| *** zhurong has joined #openstack-infra | 02:44 | |
| *** zhurong has quit IRC | 02:48 | |
| *** coolsvap has joined #openstack-infra | 02:49 | |
| *** zhurong has joined #openstack-infra | 02:49 | |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update helpers.artifactory https://review.openstack.org/353208 | 02:52 |
| *** gyee has quit IRC | 02:57 | |
| *** sarob has joined #openstack-infra | 02:57 | |
| *** zhurong has quit IRC | 02:58 | |
| *** timello has quit IRC | 02:58 | |
| openstackgerrit | Isaku Yamahata proposed openstack-infra/project-config: Add networking-odl for grafana dashboard https://review.openstack.org/353226 | 02:58 |
| *** zhurong has joined #openstack-infra | 02:58 | |
| openstackgerrit | Merged openstack-infra/system-config: Update admin-bluebox with new settings https://review.openstack.org/353055 | 02:59 |
| *** dimtruck is now known as zz_dimtruck | 03:00 | |
| *** timello has joined #openstack-infra | 03:01 | |
| *** psilvad has quit IRC | 03:01 | |
| *** sarob has quit IRC | 03:02 | |
| *** amitgandhinz has joined #openstack-infra | 03:06 | |
| *** tphummel has quit IRC | 03:08 | |
| *** woodster_ has quit IRC | 03:09 | |
| *** timello has quit IRC | 03:10 | |
| *** timello has joined #openstack-infra | 03:10 | |
| *** amitgandhinz has quit IRC | 03:10 | |
| *** esberglu has joined #openstack-infra | 03:11 | |
| *** esberglu has quit IRC | 03:11 | |
| *** tonytan4ever has quit IRC | 03:14 | |
| *** jerryz has quit IRC | 03:16 | |
| *** sdake has quit IRC | 03:17 | |
| *** timello has quit IRC | 03:19 | |
| *** timello has joined #openstack-infra | 03:19 | |
| *** yamamot__ has joined #openstack-infra | 03:20 | |
| *** yamamot__ has quit IRC | 03:20 | |
| *** yamamot__ has joined #openstack-infra | 03:22 | |
| *** hongbin_ has quit IRC | 03:22 | |
| openstackgerrit | John L. Villalovos proposed openstack-infra/devstack-gate: Ironic needs fewer services on the subnode https://review.openstack.org/353191 | 03:27 |
| *** timello has quit IRC | 03:29 | |
| *** julim has quit IRC | 03:30 | |
| *** timello has joined #openstack-infra | 03:30 | |
| *** xarses has quit IRC | 03:32 | |
| *** xarses has joined #openstack-infra | 03:32 | |
| *** csomerville has joined #openstack-infra | 03:37 | |
| *** timello has quit IRC | 03:39 | |
| *** spzala has joined #openstack-infra | 03:43 | |
| *** yamamot__ has quit IRC | 03:44 | |
| *** timello has joined #openstack-infra | 03:44 | |
| *** armax has quit IRC | 03:48 | |
| *** spzala has quit IRC | 03:48 | |
| *** timello has quit IRC | 03:53 | |
| *** timello has joined #openstack-infra | 03:55 | |
| *** links has joined #openstack-infra | 04:01 | |
| *** cshastri has joined #openstack-infra | 04:02 | |
| *** timello has quit IRC | 04:04 | |
| *** jlk has quit IRC | 04:04 | |
| *** timello has joined #openstack-infra | 04:04 | |
| *** jlk has joined #openstack-infra | 04:04 | |
| *** jlk has quit IRC | 04:04 | |
| *** jlk has joined #openstack-infra | 04:04 | |
| *** tangchen has joined #openstack-infra | 04:05 | |
| *** amitgandhinz has joined #openstack-infra | 04:06 | |
| *** TheodoreBe has joined #openstack-infra | 04:09 | |
| *** TheodoreBe has quit IRC | 04:10 | |
| *** TheodoreBe has joined #openstack-infra | 04:11 | |
| *** amitgandhinz has quit IRC | 04:11 | |
| *** yamamot__ has joined #openstack-infra | 04:11 | |
| *** TheodoreBe has quit IRC | 04:13 | |
| *** tonytan4ever has joined #openstack-infra | 04:15 | |
| *** gouthamr_ has quit IRC | 04:15 | |
| *** tsufiev has quit IRC | 04:18 | |
| *** tsufiev has joined #openstack-infra | 04:18 | |
| *** timello has quit IRC | 04:18 | |
| *** timello has joined #openstack-infra | 04:18 | |
| *** tonytan4ever has quit IRC | 04:20 | |
| *** Julien-zte has quit IRC | 04:24 | |
| *** Julien-zte has joined #openstack-infra | 04:25 | |
| *** permalac has quit IRC | 04:25 | |
| *** permalac has joined #openstack-infra | 04:26 | |
| openstackgerrit | Merged openstack-infra/project-config: Add gate-python-tripleoclient-buildimage jobs https://review.openstack.org/351278 | 04:27 |
| openstackgerrit | Merged openstack-infra/project-config: try harder to update constraint when a library is released https://review.openstack.org/352929 | 04:27 |
| *** roxanaghe has joined #openstack-infra | 04:27 | |
| *** timello has quit IRC | 04:28 | |
| *** timello has joined #openstack-infra | 04:29 | |
| *** gongysh has quit IRC | 04:30 | |
| AJaeger | fungi, pabelanger : We did change some labels - I remember changing releasenotes from trusty to xenial. | 04:31 |
| *** senk_ has quit IRC | 04:33 | |
| *** senk_ has joined #openstack-infra | 04:33 | |
| *** karthikp_ has joined #openstack-infra | 04:34 | |
| AJaeger | ah, you noticed as well - now read full backlog | 04:36 |
| *** yamamot__ has quit IRC | 04:36 | |
| AJaeger | we're down to 16 entries in post queue. | 04:36 |
| AJaeger | jhesketh: my understanding is that fungi suggested a restart of zuul and gearman | 04:37 |
| *** timello has quit IRC | 04:38 | |
| *** timello has joined #openstack-infra | 04:38 | |
| jhesketh | AJaeger: yeah, it'll affect the other pipelines too so I'm going to wait until it's very small | 04:39 |
| *** psachin has joined #openstack-infra | 04:40 | |
| AJaeger | let's do it in the next 60 mins - before the periodic jobs starts ;) | 04:41 |
| *** timello has quit IRC | 04:47 | |
| *** coolsvap is now known as _coolsvap_ | 04:49 | |
| *** timello has joined #openstack-infra | 04:49 | |
| *** PollyZ has quit IRC | 04:50 | |
| *** roxanaghe has quit IRC | 04:51 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove the ironic-parallel-nv from tempest experimental https://review.openstack.org/353225 | 04:51 |
| *** nwkarsten has quit IRC | 04:52 | |
| *** nwkarsten has joined #openstack-infra | 04:53 | |
| *** tphummel has joined #openstack-infra | 04:53 | |
| *** mrmartin has joined #openstack-infra | 04:54 | |
| *** jaosorior has joined #openstack-infra | 04:55 | |
| *** nwkarsten has quit IRC | 04:57 | |
| *** timello has quit IRC | 04:58 | |
| *** adrian_otto has joined #openstack-infra | 04:59 | |
| *** javeriak has joined #openstack-infra | 05:00 | |
| *** pcaruana has quit IRC | 05:01 | |
| *** timello has joined #openstack-infra | 05:02 | |
| *** amitgandhinz has joined #openstack-infra | 05:07 | |
| AJaeger | jhesketh: down to 10 post jobs - I suggest to wait for the tempest change in the gate to merge and then restart. Won't get much lower in the next few hours and the periodic jobs start in 53 mins - and Europe is waking up as well... | 05:08 |
| jhesketh | agreed | 05:08 |
| AJaeger | and if that tempest job does not finish in the next 20 mins - or you don't have time, I would not wait for it ;) | 05:09 |
| *** senk_ has quit IRC | 05:09 | |
| AJaeger | will be back later | 05:09 |
| *** timello has quit IRC | 05:11 | |
| *** amitgandhinz has quit IRC | 05:11 | |
| jhesketh | yep | 05:12 |
| *** timello has joined #openstack-infra | 05:12 | |
| tangchen | Ajaeger: Hi, Andreas, are you still around ? | 05:13 |
| tangchen | Ajaeger: I posted this patch yesterday. https://review.openstack.org/#/c/352772/ | 05:13 |
| tangchen | Ajaeger: It was merged. But I think I made a mistake and it was incorrect. | 05:13 |
| tangchen | Ajaeger: What would you prefer me to do? Post another patch to fix it or will you revert it ? | 05:14 |
| *** karthikp_ has quit IRC | 05:14 | |
| zaro | tangchen: i would post another patch for him to review. i assume he can provide suggestions from there. | 05:15 |
| *** _coolsvap_ is now known as coolsvap | 05:16 | |
| tangchen | Ajaeger: "openstack/ironic-python-agent" should be added for ironic functional test, but I added it to ironicclient. | 05:16 |
| *** yamamot__ has joined #openstack-infra | 05:16 | |
| tangchen | zaro: Hi, zaro, thanks. :) But I think I have already made a patch for it. Would you like me to post ? | 05:16 |
| *** roxanaghe has joined #openstack-infra | 05:17 | |
| zaro | tangchen: it doesn't hurt anything, you can add AJaeger as a reviewer. | 05:17 |
| tangchen | zaro: Er...I mean the incorrect patch I posted yesterday has been merged. | 05:17 |
| tangchen | zaro: So, I just post another patch to fix it, right ? | 05:18 |
| tangchen | zaro: And you are right, it doesn't hurt anything. | 05:18 |
| *** rbuzatu has joined #openstack-infra | 05:18 | |
| zaro | tangchen: yes, post one to fix it and have AJaeger review it. then he will let you know whether it's better to revert or merge the fix | 05:20 |
| tangchen | zaro: OK, thank you very much. :) | 05:21 |
| openstackgerrit | Tang Chen proposed openstack-infra/project-config: Fix incorrect Ironic functional job setting https://review.openstack.org/353254 | 05:21 |
| *** timello has quit IRC | 05:21 | |
| jhesketh | #status notice zuul is being restarted to reload configuration. Jobs should be re-enqueued but if you're missing anything (and it's not on http://status.openstack.org/zuul/) please issue a recheck in 30min. | 05:23 |
| openstackstatus | jhesketh: sending notice | 05:23 |
| -openstackstatus- NOTICE: zuul is being restarted to reload configuration. Jobs should be re-enqueued but if you're missing anything (and it's not on http://status.openstack.org/zuul/) please issue a recheck in 30min. | 05:24 | |
| *** timello has joined #openstack-infra | 05:25 | |
| *** tphummel has quit IRC | 05:26 | |
| openstackstatus | jhesketh: finished sending notice | 05:26 |
| *** Sukhdev has joined #openstack-infra | 05:31 | |
| yamamot__ | stats_counts.zuul.pipeline.periodic.job.periodic-tempest-dsvm-networking-midonet-v2-mitaka.FAILURE doesn't exist while the corresponding SUCCESS does. does it mean it has never failed yet? | 05:33 |
| yamamot__ | or something wrong? | 05:33 |
| *** hurgleburgler has quit IRC | 05:34 | |
| *** timello has quit IRC | 05:34 | |
| jhesketh | yamamot__: I think that could be the case | 05:34 |
| *** roxanaghe has quit IRC | 05:35 | |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/project-config: grafana: Fix a typo (missing hyphen) in midonet UT metrics https://review.openstack.org/353256 | 05:36 |
| yamamot__ | jhesketh: thank you | 05:36 |
| AJaeger | zaro, tangchen : Please do not add me. I go through the reviews on my own pace... | 05:38 |
| AJaeger | if it's urgent, just ask here and somebody will pick it up | 05:38 |
| AJaeger | tangchen: regarding revert: If you can fix it quickly, no need to revert... | 05:39 |
| *** timello has joined #openstack-infra | 05:40 | |
| *** mrmartin has quit IRC | 05:40 | |
| AJaeger | jhesketh: thanks for restart! | 05:41 |
| jhesketh | all good :-) | 05:42 |
| *** jpeeler has quit IRC | 05:45 | |
| *** mikelk has joined #openstack-infra | 05:45 | |
| *** timello has quit IRC | 05:49 | |
| *** jpeeler has joined #openstack-infra | 05:49 | |
| *** timello has joined #openstack-infra | 05:52 | |
| tangchen | Ajaeger: Hi Andreas, Thanks for the quick +2. :) | 05:58 |
| AJaeger | tangchen: you're welcome | 05:59 |
| *** timello has quit IRC | 06:01 | |
| *** timello has joined #openstack-infra | 06:02 | |
| AJaeger | jhesketh: could you review https://review.openstack.org/#/c/352788/ , please? | 06:02 |
| *** _nadya_ has joined #openstack-infra | 06:03 | |
| jhesketh | AJaeger: looking | 06:03 |
| *** rcernin has joined #openstack-infra | 06:04 | |
| jhesketh | AJaeger: lgtm | 06:04 |
| AJaeger | thanks! | 06:05 |
| *** amitgandhinz has joined #openstack-infra | 06:08 | |
| *** timello has quit IRC | 06:11 | |
| *** timello has joined #openstack-infra | 06:11 | |
| *** savihou has joined #openstack-infra | 06:11 | |
| *** amitgandhinz has quit IRC | 06:12 | |
| *** claudiub has joined #openstack-infra | 06:13 | |
| *** _nadya_ has quit IRC | 06:13 | |
| *** mrmartin has joined #openstack-infra | 06:17 | |
| *** kien-ha has quit IRC | 06:19 | |
| *** raunak has quit IRC | 06:19 | |
| *** mrmartin has quit IRC | 06:19 | |
| *** timello has quit IRC | 06:20 | |
| *** nherciu has joined #openstack-infra | 06:22 | |
| *** oanson has joined #openstack-infra | 06:22 | |
| *** ihrachys has joined #openstack-infra | 06:22 | |
| *** mrmartin has joined #openstack-infra | 06:24 | |
| *** maishsk has joined #openstack-infra | 06:24 | |
| *** timello has joined #openstack-infra | 06:24 | |
| *** mrmartin has quit IRC | 06:28 | |
| *** senk_ has joined #openstack-infra | 06:28 | |
| *** ihrachys has quit IRC | 06:29 | |
| *** andymaier has joined #openstack-infra | 06:31 | |
| *** rbuzatu has quit IRC | 06:31 | |
| *** andymaier has quit IRC | 06:32 | |
| *** maishsk has quit IRC | 06:32 | |
| *** timello has quit IRC | 06:33 | |
| *** maishsk has joined #openstack-infra | 06:34 | |
| *** timello has joined #openstack-infra | 06:35 | |
| openstackgerrit | Craige McWhirter proposed openstack-infra/puppet-phabricator: Patches Required to Deliver Pholio https://review.openstack.org/342481 | 06:36 |
| *** inc0 has joined #openstack-infra | 06:39 | |
| *** spzala has joined #openstack-infra | 06:43 | |
| *** timello has quit IRC | 06:44 | |
| *** tesseract- has joined #openstack-infra | 06:44 | |
| *** javeriak has quit IRC | 06:44 | |
| *** witek has quit IRC | 06:44 | |
| *** javeriak has joined #openstack-infra | 06:45 | |
| *** inc0 has quit IRC | 06:45 | |
| jhesketh | craige: some comments/feedback on ^ | 06:45 |
| craige | oh, very snappy. | 06:45 |
| * craige goes off to read | 06:45 | |
| *** timello has joined #openstack-infra | 06:45 | |
| *** e0ne has joined #openstack-infra | 06:47 | |
| *** spzala has quit IRC | 06:47 | |
| craige | In your first comment jhesketh, were you referring to the File or the Exec in require? | 06:48 |
| *** tosky has joined #openstack-infra | 06:49 | |
| *** _nadya_ has joined #openstack-infra | 06:50 | |
| *** jlanoux has joined #openstack-infra | 06:50 | |
| yolanda | good morning | 06:51 |
| *** Sukhdev has quit IRC | 06:52 | |
| *** Ravikiran_K has joined #openstack-infra | 06:52 | |
| *** e0ne has quit IRC | 06:54 | |
| *** adrian_otto has quit IRC | 06:54 | |
| *** timello has quit IRC | 06:54 | |
| *** timello has joined #openstack-infra | 06:56 | |
| *** sshnaidm has quit IRC | 06:56 | |
| *** adrian_otto has joined #openstack-infra | 06:57 | |
| craige | morning yolanda! | 06:57 |
| *** e0ne has joined #openstack-infra | 06:58 | |
| *** e0ne has quit IRC | 07:00 | |
| *** ccamacho has quit IRC | 07:01 | |
| *** ccamacho has joined #openstack-infra | 07:04 | |
| *** timello has quit IRC | 07:05 | |
| *** e0ne has joined #openstack-infra | 07:05 | |
| openstackgerrit | Craige McWhirter proposed openstack-infra/puppet-phabricator: Patches Required to Deliver Pholio https://review.openstack.org/342481 | 07:05 |
| *** e0ne has quit IRC | 07:07 | |
| *** timello has joined #openstack-infra | 07:08 | |
| jhesketh | craige: Exec | 07:08 |
| craige | cool. will punt the patch up for that, jhesketh | 07:10 |
| *** adrian_otto has quit IRC | 07:11 | |
| *** e0ne has joined #openstack-infra | 07:11 | |
| openstackgerrit | Craige McWhirter proposed openstack-infra/puppet-phabricator: Patches Required to Deliver Pholio https://review.openstack.org/342481 | 07:13 |
| *** mrmartin has joined #openstack-infra | 07:15 | |
| *** hongbin has quit IRC | 07:15 | |
| jhesketh | craige: lgtm | 07:16 |
| craige | \o/ | 07:16 |
| *** timello has quit IRC | 07:17 | |
| craige | I haven't tested that latest patch yet though, jhesketh because http://imgur.com/a/QTU8d | 07:17 |
| *** jistr has quit IRC | 07:18 | |
| openstackgerrit | Merged openstack-infra/project-config: grafana: Fix a typo (missing hyphen) in midonet UT metrics https://review.openstack.org/353256 | 07:18 |
| openstackgerrit | Merged openstack-infra/project-config: Adding project for Gluon fuel plugin. https://review.openstack.org/339488 | 07:19 |
| *** Hunner has quit IRC | 07:19 | |
| openstackgerrit | Alexey Stepanov proposed openstack-infra/project-config: fuel-qa: ubuntu-xenial gates excluded for 6.1 and older https://review.openstack.org/353276 | 07:19 |
| *** crinkle has quit IRC | 07:19 | |
| openstackgerrit | Merged openstack-infra/project-config: translations: Update UPPER_CONSTRAINTS usage https://review.openstack.org/352788 | 07:20 |
| *** greghaynes has quit IRC | 07:20 | |
| openstackgerrit | Merged openstack-infra/project-config: Add {murano-app}-unittest job https://review.openstack.org/351757 | 07:20 |
| *** bmjen has quit IRC | 07:20 | |
| *** greghaynes has joined #openstack-infra | 07:20 | |
| *** crinkle has joined #openstack-infra | 07:21 | |
| *** Hunner has joined #openstack-infra | 07:22 | |
| *** Hunner has quit IRC | 07:22 | |
| *** Hunner has joined #openstack-infra | 07:22 | |
| *** pcaruana has joined #openstack-infra | 07:24 | |
| *** timello has joined #openstack-infra | 07:24 | |
| *** ifarkas_afk is now known as ifarkas | 07:25 | |
| *** maishsk has quit IRC | 07:25 | |
| *** jistr has joined #openstack-infra | 07:25 | |
| *** bmjen has joined #openstack-infra | 07:26 | |
| openstackgerrit | Luka Peschke proposed openstack-infra/project-config: Added dsvm CloudKitty installation gate jobs https://review.openstack.org/352997 | 07:29 |
| openstackgerrit | Merged openstack-infra/project-config: [trove] Promote scenario tests to voting and gating https://review.openstack.org/353129 | 07:29 |
| *** sshnaidm has joined #openstack-infra | 07:29 | |
| openstackgerrit | Merged openstack-infra/project-config: New project - Almanach https://review.openstack.org/346152 | 07:30 |
| penguinolog | Hello! Review. please: https://review.openstack.org/#/c/353276/ - we have blocked old branch, which is required to modify fo upgrades tests | 07:30 |
| *** vincentll has joined #openstack-infra | 07:31 | |
| *** florianf has joined #openstack-infra | 07:31 | |
| *** e0ne has quit IRC | 07:31 | |
| *** javeriak has quit IRC | 07:31 | |
| openstackgerrit | Merged openstack-infra/project-config: Add non-voting unit test coverage job for murano-dashboard https://review.openstack.org/352412 | 07:32 |
| openstackgerrit | Fatih Degirmenci proposed openstack-infra/jenkins-job-builder: Add support for Parameterized Scheduler Plugin https://review.openstack.org/353165 | 07:32 |
| *** timello has quit IRC | 07:33 | |
| *** ccamacho is now known as ccamacho|afk | 07:33 | |
| *** matrohon has joined #openstack-infra | 07:33 | |
| *** timello has joined #openstack-infra | 07:33 | |
| *** kushal has quit IRC | 07:35 | |
| *** shardy has joined #openstack-infra | 07:35 | |
| *** ihrachys has joined #openstack-infra | 07:36 | |
| openstackgerrit | Xiang Chen proposed openstack/diskimage-builder: Explain difference between 'DIB_OFFLINE' and 'DIB_DEBIAN_USE_DEBOOTSTRAP_CACHE' https://review.openstack.org/345935 | 07:36 |
| *** maishsk has joined #openstack-infra | 07:37 | |
| openstackgerrit | Merged openstack-infra/project-config: Enable voting for designate grenade job https://review.openstack.org/352904 | 07:37 |
| openstackgerrit | Merged openstack-infra/project-config: Add one new Fuel's plugin https://review.openstack.org/352446 | 07:37 |
| openstackgerrit | Merged openstack-infra/subunit2sql: Add DB API function to get list of runs from ids https://review.openstack.org/352558 | 07:38 |
| openstackgerrit | zhangyanxian proposed openstack-infra/project-config: Fix typo in the query-zanata-project-version.py https://review.openstack.org/353288 | 07:38 |
| *** amitgandhinz has joined #openstack-infra | 07:39 | |
| openstackgerrit | zhangyanxian proposed openstack-infra/project-config: Fix typo in the query-zanata-project-version.py https://review.openstack.org/353288 | 07:40 |
| *** ramishra has quit IRC | 07:41 | |
| *** matrohon has quit IRC | 07:41 | |
| *** timello has quit IRC | 07:42 | |
| *** jpich has joined #openstack-infra | 07:43 | |
| *** amitgandhinz has quit IRC | 07:43 | |
| *** timello has joined #openstack-infra | 07:44 | |
| *** ramishra has joined #openstack-infra | 07:45 | |
| *** dtantsur|afk is now known as dtantsur | 07:45 | |
| *** matthewbodkin has joined #openstack-infra | 07:48 | |
| *** maishsk has quit IRC | 07:50 | |
| *** kong has quit IRC | 07:50 | |
| *** maishsk has joined #openstack-infra | 07:51 | |
| *** timello has quit IRC | 07:53 | |
| *** timello has joined #openstack-infra | 07:54 | |
| *** javeriak has joined #openstack-infra | 07:54 | |
| *** nwkarsten has joined #openstack-infra | 07:54 | |
| *** ccamacho|afk is now known as ccamacho | 07:56 | |
| *** matrohon has joined #openstack-infra | 07:57 | |
| *** yanyanhu has quit IRC | 07:58 | |
| *** mhickey has joined #openstack-infra | 07:58 | |
| *** nwkarsten has quit IRC | 07:59 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** zzzeek has joined #openstack-infra | 08:00 | |
| *** yanyanhu has joined #openstack-infra | 08:00 | |
| *** yanyanhu has quit IRC | 08:01 | |
| *** timello has quit IRC | 08:03 | |
| *** mrunge_ is now known as mrunge | 08:03 | |
| *** yanyanhu has joined #openstack-infra | 08:03 | |
| openstackgerrit | Alexey Stepanov proposed openstack-infra/project-config: fuel-qa: ubuntu-xenial gates excluded for 6.1 and older https://review.openstack.org/353276 | 08:03 |
| *** mrmartin has quit IRC | 08:04 | |
| *** lucas-dinner is now known as lucasagomes | 08:06 | |
| *** timello has joined #openstack-infra | 08:08 | |
| *** rbuzatu has joined #openstack-infra | 08:08 | |
| *** simonmcc has quit IRC | 08:08 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Change gateway on baremetal machine https://review.openstack.org/353299 | 08:09 |
| *** simonmcc has joined #openstack-infra | 08:09 | |
| *** chem has joined #openstack-infra | 08:10 | |
| *** rbuzatu has quit IRC | 08:12 | |
| *** matrohon has quit IRC | 08:15 | |
| *** javeriak has quit IRC | 08:15 | |
| *** furlongm has joined #openstack-infra | 08:18 | |
| *** openstackgerrit has quit IRC | 08:18 | |
| *** openstackgerrit has joined #openstack-infra | 08:18 | |
| *** acoles_ is now known as acoles | 08:19 | |
| openstackgerrit | Yuval Brik proposed openstack-infra/project-config: Rename Smaug to Karbor https://review.openstack.org/353304 | 08:19 |
| *** rbuzatu has joined #openstack-infra | 08:19 | |
| *** yaume has joined #openstack-infra | 08:20 | |
| *** yaume has quit IRC | 08:21 | |
| *** timello has quit IRC | 08:24 | |
| openstackgerrit | Yuval Brik proposed openstack-infra/irc-meetings: Rename Smaug to Karbor https://review.openstack.org/353306 | 08:25 |
| *** amoralej|off is now known as amoralej | 08:26 | |
| *** timello has joined #openstack-infra | 08:28 | |
| openstackgerrit | Yuval Brik proposed openstack-infra/system-config: Rename Smaug to Karbor https://review.openstack.org/353309 | 08:29 |
| *** matrohon has joined #openstack-infra | 08:31 | |
| *** larainema has quit IRC | 08:31 | |
| *** bethwhite has joined #openstack-infra | 08:33 | |
| *** larainema has joined #openstack-infra | 08:33 | |
| *** electrofelix has joined #openstack-infra | 08:36 | |
| *** timello has quit IRC | 08:37 | |
| *** timello has joined #openstack-infra | 08:38 | |
| *** dfflanders has quit IRC | 08:38 | |
| *** amitgandhinz has joined #openstack-infra | 08:39 | |
| *** permalac has quit IRC | 08:40 | |
| zigo | AJaeger: Any idea why there's no post job in there: http://logs.openstack.org/d8/d8e495f9ce2983fd9fb8ffc4de65d09791d845ea/post/ ? | 08:42 |
| zigo | Normally, there's more than a single one... | 08:43 |
| *** _degorenko|afk is now known as degorenko | 08:43 | |
| zigo | It also didn't publish python-imagesize | 08:43 |
| *** amitgandhinz has quit IRC | 08:44 | |
| *** shardy has quit IRC | 08:44 | |
| *** shardy has joined #openstack-infra | 08:45 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Remove baremetal00.hpuseast from site.pp https://review.openstack.org/353317 | 08:45 |
| *** timello has quit IRC | 08:47 | |
| openstackgerrit | Timur Sufiev proposed openstack-infra/project-config: A follow-up on 'Splitting current/deprecated integ. test' https://review.openstack.org/353321 | 08:48 |
| *** pgadiya has joined #openstack-infra | 08:51 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Change vlan to 2551 https://review.openstack.org/353324 | 08:51 |
| *** kzaitsev_mb has joined #openstack-infra | 08:53 | |
| *** timello has joined #openstack-infra | 08:55 | |
| *** Na3iL has joined #openstack-infra | 08:55 | |
| *** dkehn has quit IRC | 08:56 | |
| *** nwkarsten has joined #openstack-infra | 08:57 | |
| *** shardy is now known as shardy_mtg | 09:00 | |
| *** yaume has joined #openstack-infra | 09:00 | |
| *** dkehn has joined #openstack-infra | 09:01 | |
| *** nwkarsten has quit IRC | 09:01 | |
| openstackgerrit | Yuval Brik proposed openstack-infra/project-config: Rename Smaug to Karbor https://review.openstack.org/353304 | 09:02 |
| *** jed56 has joined #openstack-infra | 09:03 | |
| *** timello has quit IRC | 09:04 | |
| *** nijaba has quit IRC | 09:04 | |
| *** timello has joined #openstack-infra | 09:04 | |
| openstackgerrit | Yuval Brik proposed openstack-infra/system-config: Rename Smaug to Karbor https://review.openstack.org/353309 | 09:05 |
| *** yanyanhu has quit IRC | 09:06 | |
| *** nijaba has joined #openstack-infra | 09:07 | |
| *** yanyanhu has joined #openstack-infra | 09:08 | |
| openstackgerrit | Andrea Scarpino proposed openstack-infra/jenkins-job-builder: Add ansible builder which wraps Ansible Plugin https://review.openstack.org/353332 | 09:08 |
| *** mrmartin has joined #openstack-infra | 09:14 | |
| *** timello has quit IRC | 09:15 | |
| *** kaisers has joined #openstack-infra | 09:15 | |
| *** timello has joined #openstack-infra | 09:16 | |
| openstackgerrit | Merged openstack-infra/system-config: Add bluebox to cloud_layouts https://review.openstack.org/353013 | 09:16 |
| *** vicky has joined #openstack-infra | 09:17 | |
| *** vicky is now known as Guest42782 | 09:17 | |
| *** matrohon has quit IRC | 09:22 | |
| openstackgerrit | Merged openstack-infra/system-config: Workaround for its-storyboard change-restored event https://review.openstack.org/353046 | 09:24 |
| *** timello has quit IRC | 09:24 | |
| *** timello has joined #openstack-infra | 09:25 | |
| openstackgerrit | Ardar proposed openstack/python-jenkins: a bug when process jenkins url with prefix, like: http://yourIP:port/prefix/ https://review.openstack.org/353339 | 09:27 |
| *** e0ne has joined #openstack-infra | 09:29 | |
| *** lezbar has quit IRC | 09:31 | |
| *** kaisers has quit IRC | 09:31 | |
| *** kaisers has joined #openstack-infra | 09:32 | |
| *** timello has quit IRC | 09:34 | |
| openstackgerrit | Andrea Scarpino proposed openstack-infra/jenkins-job-builder: Add ansible builder which wraps Ansible Plugin https://review.openstack.org/353332 | 09:34 |
| *** timello has joined #openstack-infra | 09:37 | |
| openstackgerrit | Luka Peschke proposed openstack-infra/project-config: Added dsvm CloudKitty installation gate jobs https://review.openstack.org/352997 | 09:37 |
| *** dkehn has quit IRC | 09:37 | |
| *** dkehn_ has quit IRC | 09:38 | |
| openstackgerrit | Merged openstack-infra/system-config: Add nodepool flavor for bluebox https://review.openstack.org/353199 | 09:38 |
| *** matrohon has joined #openstack-infra | 09:38 | |
| *** amitgandhinz has joined #openstack-infra | 09:40 | |
| *** yamamot__ has quit IRC | 09:44 | |
| openstackgerrit | Merged openstack-infra/system-config: Change gateway on baremetal machine https://review.openstack.org/353299 | 09:44 |
| *** dkehn has joined #openstack-infra | 09:44 | |
| *** amitgandhinz has quit IRC | 09:44 | |
| *** electrical has joined #openstack-infra | 09:45 | |
| *** timello has quit IRC | 09:46 | |
| *** sdake has joined #openstack-infra | 09:47 | |
| *** timello has joined #openstack-infra | 09:48 | |
| openstackgerrit | Luka Peschke proposed openstack-infra/project-config: Added dsvm CloudKitty installation gate jobs https://review.openstack.org/352997 | 09:49 |
| *** Guest42782 is now known as vikrant | 09:49 | |
| *** mrmartin has quit IRC | 09:49 | |
| *** tangchen has quit IRC | 09:50 | |
| *** dkehn_ has joined #openstack-infra | 09:51 | |
| *** andreykurilin has joined #openstack-infra | 09:51 | |
| *** Na3iL has quit IRC | 09:52 | |
| *** kdas__ has joined #openstack-infra | 09:54 | |
| *** timello has quit IRC | 09:57 | |
| *** sambetts|afk is now known as sambetts | 09:59 | |
| *** kdas__ is now known as kushal | 10:00 | |
| *** kushal has quit IRC | 10:00 | |
| *** kushal has joined #openstack-infra | 10:00 | |
| *** timello has joined #openstack-infra | 10:00 | |
| *** kzaitsev_mb has quit IRC | 10:00 | |
| zigo | AJaeger: Is it possible that the job needs rescheduling after zuul got restarted? Or it's just infra being super overcommitted? (the POST jobs are still in the queue, as per http://status.openstack.org/zuul/) | 10:00 |
| openstackgerrit | Andrea Scarpino proposed openstack-infra/jenkins-job-builder: Add ansible builder which wraps Ansible Plugin https://review.openstack.org/353332 | 10:01 |
| *** zhurong has quit IRC | 10:02 | |
| *** matrohon has quit IRC | 10:04 | |
| openstackgerrit | Merged openstack-infra/system-config: Remove baremetal00.hpuseast from site.pp https://review.openstack.org/353317 | 10:06 |
| *** mrmartin has joined #openstack-infra | 10:08 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Change vlan to 2551 https://review.openstack.org/353324 | 10:09 |
| *** timello has quit IRC | 10:09 | |
| *** timello has joined #openstack-infra | 10:11 | |
| *** jlanoux has quit IRC | 10:12 | |
| *** yanyanhu has quit IRC | 10:13 | |
| *** Julien-zte has quit IRC | 10:19 | |
| *** tangchen has joined #openstack-infra | 10:19 | |
| *** timello has quit IRC | 10:20 | |
| *** timello has joined #openstack-infra | 10:21 | |
| *** matrohon has joined #openstack-infra | 10:21 | |
| openstackgerrit | Merged openstack-infra/python-storyboardclient: Add system info https://review.openstack.org/347405 | 10:22 |
| *** shardy_mtg is now known as shardy | 10:25 | |
| *** matrohon has quit IRC | 10:29 | |
| *** Na3iL has joined #openstack-infra | 10:29 | |
| *** timello has quit IRC | 10:29 | |
| openstackgerrit | Merged openstack-infra/python-storyboardclient: Add support for private stories https://review.openstack.org/347448 | 10:29 |
| *** timello has joined #openstack-infra | 10:32 | |
| *** mrmartin has quit IRC | 10:33 | |
| *** jlanoux has joined #openstack-infra | 10:38 | |
| *** sdague has joined #openstack-infra | 10:38 | |
| *** sdague has quit IRC | 10:38 | |
| *** sdague has joined #openstack-infra | 10:39 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Comment some settings in worklist resource https://review.openstack.org/342290 | 10:39 |
| *** amitgandhinz has joined #openstack-infra | 10:41 | |
| *** javeriak has joined #openstack-infra | 10:41 | |
| *** timello has quit IRC | 10:41 | |
| *** mkoderer_ has joined #openstack-infra | 10:42 | |
| *** timello has joined #openstack-infra | 10:42 | |
| *** mkoderer_ has quit IRC | 10:42 | |
| *** spzala has joined #openstack-infra | 10:44 | |
| *** jlanoux has quit IRC | 10:44 | |
| *** jlanoux has joined #openstack-infra | 10:44 | |
| *** rajinir has quit IRC | 10:45 | |
| *** matrohon has joined #openstack-infra | 10:45 | |
| *** amitgandhinz has quit IRC | 10:45 | |
| *** jlanoux has quit IRC | 10:46 | |
| *** apetrich has joined #openstack-infra | 10:46 | |
| *** Julien-zte has joined #openstack-infra | 10:46 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/353378 | 10:47 |
| *** spzala has quit IRC | 10:48 | |
| *** rbuzatu has quit IRC | 10:49 | |
| *** timello has quit IRC | 10:51 | |
| *** timello has joined #openstack-infra | 10:52 | |
| *** permalac has joined #openstack-infra | 10:53 | |
| *** matrohon has quit IRC | 10:53 | |
| AJaeger | zigo: if it's a post job, we do no restart after a zuul restart. The post queue had only 10 entries when we restarted, so it finally was small... | 10:55 |
| zigo | AJaeger: It looks like to me that d8e495f is completely stuck. | 10:56 |
| zigo | AJaeger: It's been 4 hours it's in queue, with only one of the 3 consecutive POST jobs... | 10:56 |
| openstackgerrit | Andrea Scarpino proposed openstack-infra/jenkins-job-builder: Add ansible builder which wraps Ansible Plugin https://review.openstack.org/353332 | 10:56 |
| AJaeger | zigo, busy CI ;( | 10:57 |
| zigo | Seriously? | 10:57 |
| zigo | :( | 10:57 |
| AJaeger | zigo, post queue has lower priority | 10:57 |
| AJaeger | and looking at the graphs, we run at full capacity ;( | 10:57 |
| AJaeger | but we're slow indeed - periodic jobs are still in the queue ;( | 10:58 |
| *** mrmartin has joined #openstack-infra | 10:58 | |
| zigo | AJaeger: Is it possible to provision just a little bit more Jessie VMs, maybe? Graphana shows that they get provisionned, then killed, then provisionned again ... | 10:59 |
| zigo | Looks like a waste of resources to me. | 10:59 |
| *** dprince has joined #openstack-infra | 10:59 | |
| *** mrunge has quit IRC | 11:00 | |
| *** timello has quit IRC | 11:01 | |
| *** jlanoux has joined #openstack-infra | 11:01 | |
| *** timello has joined #openstack-infra | 11:02 | |
| *** Ravikiran_K has quit IRC | 11:06 | |
| AJaeger | zigo, we hand out root access to VMs, so really have to restart... | 11:06 |
| *** kzaitsev_mb has joined #openstack-infra | 11:07 | |
| AJaeger | ... otherwise you won't have a clean environment to start from | 11:07 |
| *** mrmartin has quit IRC | 11:08 | |
| AJaeger | zigo: let's see whether one of our experts has an idea. My best bet: Let's help pabelanger to get bluemix ready again so that we have more VMs for testing... | 11:08 |
| zigo | Sounds about the right thing to do indeed. | 11:09 |
| *** matrohon has joined #openstack-infra | 11:09 | |
| *** Anticime1 is now known as Anticimex | 11:10 | |
| *** timello has quit IRC | 11:11 | |
| *** mrunge has joined #openstack-infra | 11:11 | |
| openstackgerrit | Merged openstack-infra/project-config: Make networking-generic-switch-dsvm-tempest voting https://review.openstack.org/352553 | 11:11 |
| *** timello has joined #openstack-infra | 11:11 | |
| *** vikrant is now known as vikrant|T | 11:11 | |
| *** matrohon has quit IRC | 11:12 | |
| *** kong has joined #openstack-infra | 11:16 | |
| *** jkilpatr has joined #openstack-infra | 11:18 | |
| *** mrmartin has joined #openstack-infra | 11:19 | |
| *** r-mibu has quit IRC | 11:19 | |
| *** r-mibu has joined #openstack-infra | 11:19 | |
| *** e0ne has quit IRC | 11:20 | |
| *** timello has quit IRC | 11:20 | |
| *** timello has joined #openstack-infra | 11:21 | |
| *** tpsilva has joined #openstack-infra | 11:21 | |
| *** dkehn has quit IRC | 11:21 | |
| *** lucasagomes is now known as lucas-hungry | 11:22 | |
| *** rbuzatu has joined #openstack-infra | 11:22 | |
| *** dkehn_ has quit IRC | 11:22 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/puppet-infracloud: Enable inventory_dhcp_static_ip on dnsmasq https://review.openstack.org/353390 | 11:23 |
| *** dkehn has joined #openstack-infra | 11:23 | |
| *** javeriak has quit IRC | 11:26 | |
| *** hrubi has quit IRC | 11:27 | |
| *** matthewbodkin has quit IRC | 11:28 | |
| *** jaosorior has quit IRC | 11:28 | |
| *** matrohon has joined #openstack-infra | 11:28 | |
| *** jaosorior has joined #openstack-infra | 11:29 | |
| openstackgerrit | Sam Betts proposed openstack-infra/project-config: Fix TinyIPA POST job make clean call https://review.openstack.org/353393 | 11:30 |
| *** timello has quit IRC | 11:30 | |
| *** timello has joined #openstack-infra | 11:31 | |
| dtantsur | folks, please review ^^ it blocks our image building job | 11:31 |
| *** jaosorior has quit IRC | 11:33 | |
| *** jaosorior has joined #openstack-infra | 11:33 | |
| *** dkehn_ has joined #openstack-infra | 11:34 | |
| *** kdas__ has joined #openstack-infra | 11:36 | |
| *** matrohon has quit IRC | 11:36 | |
| *** martinkopec has joined #openstack-infra | 11:37 | |
| *** kushal has quit IRC | 11:38 | |
| *** kdas__ is now known as kushal | 11:39 | |
| *** kushal has quit IRC | 11:39 | |
| *** kushal has joined #openstack-infra | 11:39 | |
| *** hrubi has joined #openstack-infra | 11:39 | |
| *** timello has quit IRC | 11:40 | |
| mordred | rcarrillocruz: dammit | 11:40 |
| *** timello has joined #openstack-infra | 11:40 | |
| mordred | rcarrillocruz: the baymodel patch landed in master and I want to cut a release | 11:40 |
| mordred | which normally would be awesome | 11:40 |
| mordred | except that magnum are now saying they're changing the word bay | 11:41 |
| *** vikrant|T is now known as vikrant | 11:41 | |
| *** amitgandhinz has joined #openstack-infra | 11:41 | |
| *** ccamacho is now known as ccamacho|lunch | 11:42 | |
| mordred | strigazi: if you're still around ... can you give me an update on the magnum bay rename? or at least point me to something I can read? | 11:43 |
| *** amitgandhinz has quit IRC | 11:46 | |
| openstackgerrit | Merged openstack-infra/system-config: Change vlan to 2551 https://review.openstack.org/353324 | 11:46 |
| mordred | strigazi: more specifically, if bay -> cluster, is baymodel going to be called cluster_model? | 11:46 |
| *** mrmartin has quit IRC | 11:46 | |
| mordred | strigazi: ah! https://etherpad.openstack.org/p/magnum-newton-midcycle-heterogenous-cluster ... Cluster and ClusterTemplate | 11:47 |
| mordred | rcarrillocruz: hrm. maybe we should just have some method aliases - use 'cluster' in the 'real' methods and the docs, but also support the 'bay' and 'baymodel' word since that's what magnum users are used to right now | 11:48 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Reinstall python-requests in bootstrap script https://review.openstack.org/350521 | 11:48 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Fix resetting /etc/hosts https://review.openstack.org/351917 | 11:48 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Raise ping test count after multinode setup https://review.openstack.org/351916 | 11:48 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Make $TRIPLEO_ROOT overrideable https://review.openstack.org/350539 | 11:48 |
| *** timello has quit IRC | 11:50 | |
| *** timello has joined #openstack-infra | 11:50 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/project-config: Add experimental tripleo jobs https://review.openstack.org/352880 | 11:51 |
| *** amotoki has quit IRC | 11:51 | |
| *** sdake has quit IRC | 11:51 | |
| *** sdake has joined #openstack-infra | 11:51 | |
| sshnaidm | AJaeger, hi, can you please look at this in your time? https://review.openstack.org/352880 I'm not sure I do it right, need a little assistance, thanks | 11:51 |
| *** matrohon has joined #openstack-infra | 11:52 | |
| *** jcoufal has joined #openstack-infra | 11:54 | |
| *** amotoki has joined #openstack-infra | 11:56 | |
| *** sdake_ has joined #openstack-infra | 11:56 | |
| rcarrillocruz | mordred: that sounds sensible to me | 11:56 |
| *** rfolco has joined #openstack-infra | 11:56 | |
| mordred | rcarrillocruz: patch coming | 11:56 |
| *** weshay has joined #openstack-infra | 11:56 | |
| rcarrillocruz | ++ | 11:56 |
| strigazi | mordred: ping | 11:57 |
| mordred | strigazi: heya! I think I answered my own question :) | 11:58 |
| *** sdake has quit IRC | 11:59 | |
| *** nwkarsten has joined #openstack-infra | 11:59 | |
| mordred | strigazi: but the cluster/clustertemplate rename is decided on for sure? | 11:59 |
| *** maishsk has quit IRC | 11:59 | |
| *** timello has quit IRC | 11:59 | |
| strigazi | it's decided, but I don't think we can make it in this cycle | 12:00 |
| rcarrillocruz | to be honest, cluster and cluster template makes more sense in my brain than bay and baymodel | 12:00 |
| *** timello has joined #openstack-infra | 12:00 | |
| rcarrillocruz | oh, then i guess we can wait mordred ? | 12:00 |
| *** amotoki has quit IRC | 12:00 | |
| strigazi | That was the reasoning | 12:00 |
| mordred | nah - I'd rather to have shade's 'official' api be forward looking and match what reality intends to be when we can | 12:00 |
| strigazi | new people to magnum get a little confused with bay | 12:00 |
| mordred | yah. I think it's a good move | 12:01 |
| *** links has quit IRC | 12:01 | |
| strigazi | Whenever I was trying to explain to someone magnum I was telling at start bay == cluster | 12:01 |
| mordred | bay came from somewhere, right? is that the docker swarm term? | 12:02 |
| strigazi | I think it was trying to be close to swarm | 12:02 |
| mordred | nod. makes sense | 12:03 |
| strigazi | whales bays | 12:03 |
| mordred | so many whales | 12:03 |
| strigazi | not we'll have a shark as a logo which might be invalid | 12:03 |
| strigazi | :) | 12:04 |
| mordred | of course, 'whale' makes me think of the twitter fail-whale instead of docker ... which got me in trouble with the marketing folks a couple of companies ago :) | 12:04 |
| mordred | hahahaha | 12:04 |
| *** nwkarsten has quit IRC | 12:04 | |
| *** maishsk has joined #openstack-infra | 12:04 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Properly set up UPPER_CONSTRAINTS for translations https://review.openstack.org/353418 | 12:04 |
| *** rhallisey has joined #openstack-infra | 12:04 | |
| AJaeger | sshnaidm, dtantsur : Looking... | 12:05 |
| *** mrmartin has joined #openstack-infra | 12:06 | |
| *** pfallenop has quit IRC | 12:07 | |
| *** psilvad has joined #openstack-infra | 12:08 | |
| *** timello has quit IRC | 12:09 | |
| *** dprince has quit IRC | 12:10 | |
| *** dprince has joined #openstack-infra | 12:11 | |
| *** matthewbodkin has joined #openstack-infra | 12:11 | |
| *** timello has joined #openstack-infra | 12:11 | |
| *** pfallenop has joined #openstack-infra | 12:14 | |
| *** amotoki has joined #openstack-infra | 12:15 | |
| *** icey has quit IRC | 12:16 | |
| *** thiagop has joined #openstack-infra | 12:16 | |
| *** e0ne has joined #openstack-infra | 12:17 | |
| *** psilvad has quit IRC | 12:18 | |
| *** icey has joined #openstack-infra | 12:18 | |
| *** jcoufal has quit IRC | 12:18 | |
| *** burgerk has joined #openstack-infra | 12:19 | |
| openstackgerrit | vikas choudhary proposed openstack-infra/project-config: Fix dsvm installation job https://review.openstack.org/346717 | 12:19 |
| *** baoli has joined #openstack-infra | 12:19 | |
| *** maishsk has quit IRC | 12:19 | |
| *** baoli_ has joined #openstack-infra | 12:20 | |
| *** timello has quit IRC | 12:20 | |
| *** maishsk has joined #openstack-infra | 12:21 | |
| *** timello has joined #openstack-infra | 12:21 | |
| *** maishsk has quit IRC | 12:21 | |
| *** matrohon has quit IRC | 12:21 | |
| sdague | so... could we get zuul to rerun jobs that are POST_FAILURE? | 12:22 |
| sdague | because I've lost I think 6 patches to that this week | 12:23 |
| sdague | which requires a full recheck | 12:23 |
| *** cdent has joined #openstack-infra | 12:23 | |
| sdague | which further chews up our nodes | 12:23 |
| *** rlandy has joined #openstack-infra | 12:23 | |
| sdague | also... any idea why we're getting so many POST_FAILURES? | 12:23 |
| *** baoli has quit IRC | 12:24 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Set new private and public IPs for controller00.vanilla https://review.openstack.org/353442 | 12:25 |
| *** zhurong has joined #openstack-infra | 12:25 | |
| *** amitgandhinz has joined #openstack-infra | 12:26 | |
| *** ldnunes has joined #openstack-infra | 12:26 | |
| AJaeger | sdague: POST_FAILURE in gate queue? best to show up here with links to the job so that an infra-root can check the log files... | 12:27 |
| openstackgerrit | Merged openstack-infra/storyboard: Add example commands for the Branches api https://review.openstack.org/338396 | 12:27 |
| sdague | AJaeger: yes, right now | 12:27 |
| sdague | http://logs.openstack.org/93/346893/1/gate/gate-devstack-docs-ubuntu-xenial/a9ff518/ | 12:27 |
| AJaeger | sdague: That's new AFAIK | 12:27 |
| *** amotoki has quit IRC | 12:28 | |
| AJaeger | sdague: http://logs.openstack.org/93/346893/1/gate/gate-devstack-docs-ubuntu-xenial/a9ff518/_zuul_ansible/ansible_log.txt shows that the rsync failed | 12:28 |
| sdague | AJaeger: ok, I've had a bunch of my patches die like this recently | 12:28 |
| *** lucas-hungry is now known as lucasagomes | 12:29 | |
| AJaeger | sdague: but looking at http://logs.openstack.org/93/346893/1/gate/gate-devstack-docs-ubuntu-xenial/a9ff518/console.html#_2016-08-10_11_59_14_916896, there was no content generated. This is a git read timeout. | 12:29 |
| AJaeger | So, networking problem. | 12:29 |
| *** mtanino has joined #openstack-infra | 12:30 | |
| sdague | ok, so, that's kind of why it would be nice if zuul retried those | 12:30 |
| AJaeger | "ReadTimeoutError: HTTPConnectionPool(host='mirror.ord.rax.openstack.org', port=80): Read timed out." | 12:30 |
| *** timello has quit IRC | 12:30 | |
| sdague | because right now, they have to be retried | 12:30 |
| sdague | except by humans | 12:30 |
| sdague | consuming much more resource | 12:30 |
| AJaeger | sdague: best discuss with our zuul expert like jeblair later | 12:30 |
| sdague | yeh | 12:30 |
| AJaeger | This looks like normal FAILURE, not a POST_FAILURE. Please record a few of these for digging into them to see what's up... | 12:31 |
| openstackgerrit | Merged openstack-infra/release-tools: Initial version for ACL manager https://review.openstack.org/351637 | 12:32 |
| sdague | yeh, I guess I just assumed there would be no logs on the POST_FAILURE, so I didn't even look further | 12:33 |
| AJaeger | sdague: for POST_FAILURE there can be logs - depending on the kind of failure. | 12:33 |
| *** ccamacho|lunch is now known as ccamacho | 12:33 | |
| AJaeger | sdague: and there might be something in the server logs as well... | 12:33 |
| sdague | any way to be able to query all the POST_FAILURE jobs? | 12:34 |
| AJaeger | none that I know | 12:34 |
| *** jcoufal has joined #openstack-infra | 12:35 | |
| *** woodster_ has joined #openstack-infra | 12:36 | |
| *** timello has joined #openstack-infra | 12:37 | |
| *** M-docaedo_vector has quit IRC | 12:38 | |
| *** matrohon has joined #openstack-infra | 12:38 | |
| openstackgerrit | Csaba Henk proposed openstack-infra/project-config: Add jobs for Manila glusterfs, glusterfs-native drivers with Heketi layout https://review.openstack.org/287632 | 12:38 |
| openstackgerrit | Csaba Henk proposed openstack-infra/project-config: templatize Manila GlusterFS jobs https://review.openstack.org/350120 | 12:38 |
| sdague | it's in logstash, so I can get some of it there | 12:38 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Properly set up UPPER_CONSTRAINTS_FILE for translations https://review.openstack.org/353418 | 12:39 |
| sdague | http://logstash.openstack.org/#dashboard/file/logstash.json?query=build_status%3APOST_FAILURE%20AND%20message%3A%5C%22Building%20remotely%20on%5C%22 is pretty reasonable | 12:39 |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/353378 | 12:40 |
| sdague | here is another one - http://logs.openstack.org/53/336953/1/gate/gate-devstack-docs-ubuntu-xenial/d8beda0/_zuul_ansible/ansible_log.txt | 12:40 |
| *** gordc has joined #openstack-infra | 12:41 | |
| *** zz_dimtruck is now known as dimtruck | 12:41 | |
| sdague | http://logs.openstack.org/50/336950/1/gate/gate-devstack-docs-ubuntu-xenial/23655be/_zuul_ansible/ansible_log.txt | 12:41 |
| AJaeger | sdague: and that one shows SUCCESS in console log file | 12:41 |
| sdague | http://logs.openstack.org/37/349637/2/gate/gate-devstack-docs-ubuntu-xenial/3dd1b79/_zuul_ansible/ansible_log.txt | 12:42 |
| sdague | AJaeger: yes, in the console | 12:42 |
| sdague | but it was still a POST_FAILURE reported to zuul | 12:42 |
| *** e0ne has quit IRC | 12:43 | |
| sdague | http://logs.openstack.org/50/336950/1/gate/gate-devstack-docs-ubuntu-xenial/1044329/_zuul_ansible/ansible_log.txt | 12:43 |
| sdague | http://logs.openstack.org/77/351477/4/check/gate-rally-dsvm-neutron-neutron/d3c4960/_zuul_ansible/ansible_log.txt | 12:43 |
| *** vikrant has quit IRC | 12:43 | |
| *** e0ne has joined #openstack-infra | 12:43 | |
| sdague | so, 6 gate-devstack-docs-ubuntu-xenial fails in the last 2 days | 12:44 |
| sdague | 1 is a mirror pull failure | 12:44 |
| sdague | the other 5 are rsync failures in ansible syncing logs at the end | 12:44 |
| sdague | for otherwise successful runs | 12:44 |
| AJaeger | Let's see whether jeblair or other have any ideas about this ^ | 12:46 |
| AJaeger | sdague: on another topic: You can use constraints anywhere now, https://review.openstack.org/352900 is ready ;) | 12:46 |
| *** baoli_ has quit IRC | 12:48 | |
| *** pradk has quit IRC | 12:48 | |
| *** kushal has quit IRC | 12:48 | |
| *** matrohon has quit IRC | 12:49 | |
| *** amoralej is now known as amoralej|lunch | 12:51 | |
| *** mtanino has quit IRC | 12:56 | |
| *** Na3iL has quit IRC | 12:59 | |
| *** nwkarsten has joined #openstack-infra | 12:59 | |
| *** kzaitsev_mb has quit IRC | 13:00 | |
| *** edmondsw has joined #openstack-infra | 13:00 | |
| *** xyang1 has joined #openstack-infra | 13:01 | |
| *** priteau has joined #openstack-infra | 13:02 | |
| *** amitgandhinz has quit IRC | 13:02 | |
| *** _nadya_ has quit IRC | 13:02 | |
| *** jkilpatr has quit IRC | 13:02 | |
| *** amitgandhinz has joined #openstack-infra | 13:02 | |
| *** gouthamr has joined #openstack-infra | 13:06 | |
| *** gouthamr has quit IRC | 13:06 | |
| *** matrohon has joined #openstack-infra | 13:06 | |
| *** kushal has joined #openstack-infra | 13:06 | |
| *** matt-borland has joined #openstack-infra | 13:06 | |
| *** gouthamr has joined #openstack-infra | 13:07 | |
| *** amitgandhinz has quit IRC | 13:08 | |
| *** weshay is now known as weshay_brb | 13:08 | |
| *** amitgandhinz has joined #openstack-infra | 13:08 | |
| anteaya | <-- offline most of today | 13:08 |
| *** spzala_ has joined #openstack-infra | 13:08 | |
| *** mdrabe has joined #openstack-infra | 13:09 | |
| *** berendt has joined #openstack-infra | 13:09 | |
| *** M-docaedo_vector has joined #openstack-infra | 13:11 | |
| *** weshay_brb is now known as weshay | 13:13 | |
| *** jkilpatr has joined #openstack-infra | 13:15 | |
| *** cshastri has quit IRC | 13:15 | |
| *** amotoki has joined #openstack-infra | 13:15 | |
| *** PollyZ has joined #openstack-infra | 13:17 | |
| *** matrohon has quit IRC | 13:17 | |
| *** sshnaidm has quit IRC | 13:17 | |
| *** sdake_ is now known as sdake | 13:18 | |
| *** eharney has joined #openstack-infra | 13:18 | |
| *** yaume has quit IRC | 13:20 | |
| *** mriedem has joined #openstack-infra | 13:21 | |
| *** tonytan4ever has joined #openstack-infra | 13:21 | |
| *** pgadiya has quit IRC | 13:22 | |
| *** apetrich has quit IRC | 13:22 | |
| *** _ari_ has joined #openstack-infra | 13:23 | |
| *** harlowja has quit IRC | 13:23 | |
| *** julim has joined #openstack-infra | 13:24 | |
| *** _nadya_ has joined #openstack-infra | 13:33 | |
| *** matrohon has joined #openstack-infra | 13:34 | |
| *** amitgandhinz has quit IRC | 13:35 | |
| *** amitgandhinz has joined #openstack-infra | 13:36 | |
| *** mrmartin has quit IRC | 13:37 | |
| *** yaume has joined #openstack-infra | 13:38 | |
| *** burgerk has quit IRC | 13:40 | |
| *** kushal has quit IRC | 13:41 | |
| *** yamamoto has joined #openstack-infra | 13:41 | |
| *** matrohon has quit IRC | 13:41 | |
| *** kushal has joined #openstack-infra | 13:42 | |
| *** esberglu has joined #openstack-infra | 13:42 | |
| *** sshnaidm has joined #openstack-infra | 13:43 | |
| *** kzaitsev_mb has joined #openstack-infra | 13:43 | |
| *** amoralej|lunch is now known as amoralej | 13:44 | |
| openstackgerrit | Merged openstack-infra/puppet-infracloud: Enable inventory_dhcp_static_ip on dnsmasq https://review.openstack.org/353390 | 13:45 |
| openstackgerrit | Thomas Goirand proposed openstack-infra/project-config: Use --sbuild-chroot-name jessie-newton-backports https://review.openstack.org/353489 | 13:49 |
| *** m1st3rwr0ng has joined #openstack-infra | 13:53 | |
| *** thorst_ has joined #openstack-infra | 13:53 | |
| *** ayoung has joined #openstack-infra | 13:53 | |
| *** cdnchris has joined #openstack-infra | 13:53 | |
| openstackgerrit | Alexey Stepanov proposed openstack-infra/project-config: fuel-qa: ubuntu-xenial gates excluded for stable/mitaka and older https://review.openstack.org/353276 | 13:54 |
| *** rodrigods has quit IRC | 13:56 | |
| *** rodrigods has joined #openstack-infra | 13:56 | |
| *** PollyZ has quit IRC | 13:57 | |
| *** matrohon has joined #openstack-infra | 13:58 | |
| *** yuval has joined #openstack-infra | 13:58 | |
| *** yamamoto has quit IRC | 13:59 | |
| yuval | Hey, does anyone knows what's going on with project-config tests? Some are queued for 2 hours | 13:59 |
| *** tongli has joined #openstack-infra | 13:59 | |
| *** hongbin has joined #openstack-infra | 13:59 | |
| *** shamail has joined #openstack-infra | 14:00 | |
| AJaeger | yuval: check status.openstack.org/zuul - and the graph at the bottom | 14:00 |
| AJaeger | yuval: our CI system is busy and has some backlog. | 14:01 |
| openstackgerrit | Merged openstack-infra/project-config: Fix TinyIPA POST job make clean call https://review.openstack.org/353393 | 14:01 |
| *** hurgleburgler has joined #openstack-infra | 14:02 | |
| *** PollyZ has joined #openstack-infra | 14:03 | |
| yuval | AJaeger: thanks, I know zuul status :) | 14:03 |
| yuval | AJaeger: will wait patiently ^^ | 14:03 |
| *** dimtruck is now known as zz_dimtruck | 14:04 | |
| *** asettle has joined #openstack-infra | 14:06 | |
| openstackgerrit | Beth Elwell proposed openstack-infra/project-config: Added openstack-server-publish-jobs to eslint-config-openstack https://review.openstack.org/353512 | 14:06 |
| *** pradk has joined #openstack-infra | 14:06 | |
| *** andymaier has joined #openstack-infra | 14:07 | |
| *** zhurong has quit IRC | 14:09 | |
| openstackgerrit | Matthew Bodkin proposed openstack-infra/storyboard: Making docs easier to understand https://review.openstack.org/353517 | 14:09 |
| timrc | It is strange to me that http://health.openstack.org/ would return JSON... | 14:09 |
| *** mikelk has quit IRC | 14:09 | |
| AJaeger | timrc: are you looking for http://status.openstack.org/openstack-health/#/ ? | 14:10 |
| *** kong has quit IRC | 14:10 | |
| timrc | AJaeger: I am there, but I was just curious if health.openstack.org would resolve to something and it did.. ;-) | 14:11 |
| *** edtubill has joined #openstack-infra | 14:12 | |
| *** nherciu has quit IRC | 14:12 | |
| *** zhurong has joined #openstack-infra | 14:12 | |
| *** fguillot has joined #openstack-infra | 14:13 | |
| *** matrohon has quit IRC | 14:13 | |
| *** hieulq_ has joined #openstack-infra | 14:14 | |
| *** rbrndt has joined #openstack-infra | 14:15 | |
| *** asettle has quit IRC | 14:19 | |
| timrc | rcarrillocruz: So https://github.com/openstack/ansible-role-cloud-launcher -- we provision all of our test clouds (for now) with Heat and I hate it and cloud-launcher solves a couple of those reasons but one of the things that I really like about Heat is its ability to clean up everything a stack owns when you delete it which makes it decent for provisioning and tearing down cloud envs Off the top | 14:19 |
| timrc | of my head I don't know if something similar would be achievable with Ansible but I really, really like how simplistic the format is for specifying the env you want to provision. | 14:19 |
| *** elmiko has joined #openstack-infra | 14:19 | |
| elmiko | fungi: ping | 14:19 |
| *** asettle has joined #openstack-infra | 14:20 | |
| elmiko | anyone know if fungi is around, it appears like someone is using his account to make changes on the openstack wiki | 14:20 |
| AJaeger | elmiko: what kind of changes? | 14:20 |
| elmiko | like "14:18, 10 August 2016 Fungi (Talk | contribs) moved page API Working Group weekly email template to Module talk:HP Printer Support Phone Number 1 (888) 990-8801 without leaving a redirect (revert)" | 14:20 |
| AJaeger | elmiko: fungi is regularly cleaning up the wiki. | 14:21 |
| AJaeger | elmiko: but that comment surprises me, let's see what he will tell us... | 14:21 |
| elmiko | there is also " 14:11, 10 August 2016 Fungi (Talk | contribs) deleted page API Working Group weekly email template (spam) " | 14:22 |
| elmiko | that page was definitely not spam | 14:22 |
| *** burgerk has joined #openstack-infra | 14:22 | |
| elmiko | also see this, " 13:51, 10 August 2016 Davidthompsonnew (Talk | contribs) moved page API Working Group weekly email template to Module talk:HP Printer Support Phone Number 1 (888) 990-8801 (tji) " | 14:22 |
| elmiko | just looks highly suspicious | 14:22 |
| elmiko | and the redirect page is most certainly spam | 14:22 |
| AJaeger | elmiko: we really have a spam problem. It's getting better thanks to fungi's daily fights | 14:23 |
| elmiko | AJaeger: oh yeah, i have tremendous respect for fungi's work. which is why this seemed so odd | 14:23 |
| *** yamamoto has joined #openstack-infra | 14:24 | |
| elmiko | AJaeger: now i'm getting updates that fungi(or whoever is using those credentials) is modifying the redirect page (the hp support stuff) | 14:25 |
| elmiko | this is nuts | 14:25 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add openstackci-bluebox to ansible to manage AFS server https://review.openstack.org/353526 | 14:26 |
| *** liusheng has quit IRC | 14:27 | |
| openstackgerrit | Yuval Brik proposed openstack-infra/project-config: Rename Smaug to Karbor https://review.openstack.org/353304 | 14:28 |
| rcarrillocruz | timrc: deleting a server is easy, there's even a delete_fip flag on os_server now | 14:29 |
| rcarrillocruz | deleting stuff like os_network is trickier, cos you have to check for ports associated to the network/router, prior to do the deletion | 14:29 |
| *** matrohon has joined #openstack-infra | 14:30 | |
| openstackgerrit | Sam Betts proposed openstack-infra/project-config: Get IPA buildimage POST job into healthcheck https://review.openstack.org/351806 | 14:30 |
| rcarrillocruz | i.e. dependencies must be handled prior to hitting delete | 14:30 |
| rcarrillocruz | but the launcher allows to set the state attribute | 14:30 |
| *** mtanino has joined #openstack-infra | 14:30 | |
| rcarrillocruz | so if you put | 14:30 |
| rcarrillocruz | state: absent , the role will try to delete that resource | 14:30 |
| rcarrillocruz | as servers are at the end of the dependency chain of resources, that's not a problem | 14:31 |
| *** liusheng has joined #openstack-infra | 14:31 | |
| *** mdrabe has quit IRC | 14:31 | |
| *** karthikp_ has joined #openstack-infra | 14:31 | |
| rcarrillocruz | the good thing about the launcher is that it is allows bare minimum bootstrapping, no heat dependency | 14:32 |
| *** csomerville has quit IRC | 14:33 | |
| *** ivyyyy-brb is now known as ivyyy | 14:33 | |
| robcresswell | Just in case this is relevant elsewhere, requests 2.11.0 (released two days ago) has caused all the Django OpenStack Auth UTs to fail. Reverting makes them pass. Not sure if our fault or requests bug yet. | 14:34 |
| *** martinkopec has quit IRC | 14:34 | |
| *** martinkopec has joined #openstack-infra | 14:37 | |
| *** martinkopec has quit IRC | 14:37 | |
| *** adrian_otto has joined #openstack-infra | 14:38 | |
| *** martinkopec has joined #openstack-infra | 14:39 | |
| *** zxiiro is now known as zxiiro|away | 14:43 | |
| *** dkehn_ has quit IRC | 14:43 | |
| *** dkehn has quit IRC | 14:43 | |
| *** zz_dimtruck is now known as dimtruck | 14:43 | |
| *** matrohon has quit IRC | 14:44 | |
| *** karthikp_ has quit IRC | 14:45 | |
| *** Swami_ has joined #openstack-infra | 14:47 | |
| *** mdrabe has joined #openstack-infra | 14:47 | |
| AJaeger | robcresswell: 2.11.0 is currently shown in upper-constraints. Do you have a patch up for reversal? | 14:48 |
| robcresswell | AJaeger: investigating at the moment. It might be incorrect usage by us | 14:48 |
| *** yamamoto has quit IRC | 14:49 | |
| *** zhurong has quit IRC | 14:49 | |
| *** yamamoto has joined #openstack-infra | 14:50 | |
| *** yamamoto has quit IRC | 14:50 | |
| *** yamamoto has joined #openstack-infra | 14:50 | |
| *** dkehn has joined #openstack-infra | 14:50 | |
| *** yamamoto has quit IRC | 14:50 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/system-config: Switch to ironic vanilla inventory https://review.openstack.org/353545 | 14:50 |
| *** yamamoto has joined #openstack-infra | 14:51 | |
| yuval | AJaeger: regarding smaug rename patches - any preference on whether the channels change should be first or after the rest of the changes? ( https://review.openstack.org/#/c/353304/ ) | 14:51 |
| *** d34dh0r531234123 is now known as d34dh0r53 | 14:51 | |
| *** senk_ has quit IRC | 14:51 | |
| rcarrillocruz | pabelanger, yolanda , care reviewing ^ ? | 14:51 |
| *** martinkopec has quit IRC | 14:51 | |
| rcarrillocruz | baremetal00 is not resolving the inventory correctly, since the key on hiera is now on the 'vanilla' namespace | 14:51 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Rename baymodel to cluster_template https://review.openstack.org/353546 | 14:52 |
| *** martinkopec has joined #openstack-infra | 14:52 | |
| mordred | rcarrillocruz: I think that gets it | 14:52 |
| AJaeger | yuval: it will take weeks until we have the next maintenance window and can do the rename. Renaming repos means several hours of gerrit downtime - for everybody. | 14:52 |
| mordred | rcarrillocruz: we should maybe think about something similar to delete_fip for deleting networks and stuff | 14:52 |
| AJaeger | yuval: So, if you want to use the channels already today, you can split the change up: Now change rename, later the rest. | 14:53 |
| AJaeger | yuval: just an option for you to consider. | 14:53 |
| rcarrillocruz | mordred: ++ , i expressed that thing to Shrews in the past, having a cascade flag or something | 14:53 |
| yuval | AJaeger: I see. BTW, what about github mirror? Is it done in the same time of the maintenance window? Is anything required on smaug's part? | 14:53 |
| AJaeger | yuval: also, let's first rename any repositories in project-config - and then everywhere else. I've given some -1 where you had dependencies on the governance change. That would be wrong. | 14:54 |
| yuval | AJaeger: yes, that's my mistake, will fix asap | 14:54 |
| AJaeger | yuval: github rename is part of infra'S change... | 14:54 |
| AJaeger | yuval: you could change your project name in governance now - and later change the repositories. | 14:54 |
| *** pblaho has joined #openstack-infra | 14:54 | |
| AJaeger | but governance should reflect existing repos as is. And not what it might be in a few weeks... | 14:55 |
| *** dkehn_ has joined #openstack-infra | 14:55 | |
| rcarrillocruz | mordred: i believe python is pretty loose on having python filename vs main class name , not like in Java. But I think you want to change 'class TestBaymodel' to 'class ClusterTemplate' | 14:56 |
| *** nwkarsten has quit IRC | 14:56 | |
| *** yamamoto has quit IRC | 14:56 | |
| *** Na3iL has joined #openstack-infra | 14:57 | |
| *** nwkarsten has joined #openstack-infra | 14:57 | |
| *** adrian_otto has quit IRC | 14:57 | |
| openstackgerrit | Sam Yaple proposed openstack-infra/shade: Add update_endpoint() https://review.openstack.org/289052 | 14:58 |
| *** matrohon has joined #openstack-infra | 14:59 | |
| mordred | rcarrillocruz: ++ | 14:59 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: retry publisher sync tasks https://review.openstack.org/353553 | 14:59 |
| jeblair | AJaeger, sdague: ^ | 15:00 |
| *** adrian_otto has joined #openstack-infra | 15:00 | |
| *** nwkarsten has quit IRC | 15:00 | |
| sdague | jeblair: cool | 15:00 |
| jeblair | mordred, Shrews, rcarrillocruz, pabelanger: can you take a look at that ^ | 15:00 |
| mordred | jeblair: yup | 15:00 |
| rcarrillocruz | yeah, i guess that's for sdague question from earlier | 15:00 |
| rcarrillocruz | checking | 15:00 |
| *** cody-somerville has joined #openstack-infra | 15:00 | |
| *** hieulq_ has quit IRC | 15:01 | |
| *** ayoung has quit IRC | 15:01 | |
| *** d0ugal has quit IRC | 15:01 | |
| mordred | jeblair: that seems reasonable | 15:01 |
| *** martinkopec has quit IRC | 15:01 | |
| jeblair | (i pasted all the links from sdague as a review message for more context) | 15:01 |
| *** hieulq_ has joined #openstack-infra | 15:02 | |
| rcarrillocruz | lgtm | 15:02 |
| *** shamail has quit IRC | 15:02 | |
| *** Qiming_ has joined #openstack-infra | 15:03 | |
| mordred | jeblair: Imma land that, k? | 15:03 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade: Rename baymodel to cluster_template https://review.openstack.org/353546 | 15:03 |
| openstackgerrit | Yuval Brik proposed openstack-infra/project-config: Rename Smaug to Karbor https://review.openstack.org/353304 | 15:03 |
| jeblair | oh, also, one of those commands is an ftp command, but it's also safe to retry | 15:04 |
| *** vhosakot has joined #openstack-infra | 15:04 | |
| jeblair | mordred: sure thing, thanks. i tested locally. | 15:04 |
| *** sdake_ has joined #openstack-infra | 15:04 | |
| mordred | +A! | 15:04 |
| Shrews | jeblair: so, result.rc is only valid on shell commands | 15:04 |
| *** yuval has quit IRC | 15:04 | |
| fungi | elmiko: i was cleaning up spam pages, yes. one tactic the spammers like to employ is to move a valid content page to a new title, then edit it, maybe a couple times | 15:04 |
| *** Qiming has quit IRC | 15:04 | |
| jeblair | Shrews: ah! | 15:05 |
| * mordred removes +A real quick | 15:05 | |
| Shrews | lol | 15:05 |
| jeblair | Shrews: i thought i tested it with synchronize | 15:05 |
| elmiko | fungi: ack, just wanted to make sure someone hadn't compromised your creds | 15:05 |
| fungi | elmiko: this time around i got tripped up by trying to revert the page move before rolling back the page edits, which confuses mediawiki's revert capability | 15:05 |
| jeblair | let me see if i actually did | 15:05 |
| Shrews | jeblair: you'd have to check synchronize docs to see if it returns something similar | 15:05 |
| elmiko | fungi: yeah, once all the edits were through i started to see what happened | 15:05 |
| elmiko | fungi: thanks for keeping the wall safe ;) | 15:06 |
| *** dimtruck is now known as zz_dimtruck | 15:06 | |
| fungi | elmiko: apparently you have to be very careful and undo the spam edits _first_, then revert the page move | 15:06 |
| *** zz_dimtruck is now known as dimtruck | 15:06 | |
| fungi | lesson learned :/ | 15:06 |
| elmiko | interesting... | 15:06 |
| elmiko | regardless, thanks again =) | 15:06 |
| fungi | you're welcome, of course. thanks for keeping an eye out | 15:07 |
| *** sdake has quit IRC | 15:07 | |
| elmiko | glad it wasn't what i suspected | 15:07 |
| jeblair | Shrews: experimentally it works. i don't see anything about return values in the docs though. | 15:07 |
| *** cdnchris has quit IRC | 15:08 | |
| matt-borland | krotscheck...since you're familiar with the predecessor, can you take a look at https://review.openstack.org/#/c/353321/ ? It's super simple, just parameterizing another shell script. | 15:09 |
| rcarrillocruz | i believe that synchronize may be just a wrapper around shell tasks, thus the rc is implied | 15:09 |
| *** savihou has quit IRC | 15:09 | |
| Shrews | jeblair: seems that module DOES set it: https://github.com/ansible/ansible-modules-core/blob/5d7b46e0ddb7f55f1dbc6f69b973f1bcd385102d/files/synchronize.py#L454 | 15:09 |
| * rcarrillocruz is pushing hard to get infracloud rolling to get more CI capacity, but changes to move it along are being slow to merge due to CI capacity | 15:09 | |
| rcarrillocruz | YAY | 15:09 |
| Shrews | jeblair: can't say the same about all others though, so be careful with that check :) | 15:10 |
| pabelanger | rcarrillocruz: \o/ | 15:10 |
| *** amitgandhinz has quit IRC | 15:10 | |
| Shrews | rcarrillocruz: the module has to explicitly return it from exit_json(), which that module does | 15:10 |
| rcarrillocruz | pabelanger: can you look at https://review.openstack.org/#/c/353545/ https://review.openstack.org/#/c/353442/ | 15:11 |
| jeblair | Shrews: yeah, i found that 'copy' does not set it, but fortunately a local copy is less likely to fail in the way we are seeing :) | 15:11 |
| jeblair | Shrews: all the rest are either synchronize or shell | 15:11 |
| *** vhosakot_ has joined #openstack-infra | 15:11 | |
| mordred | maybe until ('rc' in result and result.rc == 0) or 'rc' not in result | 15:11 |
| rcarrillocruz | Shrews: yay for consistency | 15:11 |
| mordred | jeblair: good point | 15:11 |
| *** amitgandhinz has joined #openstack-infra | 15:11 | |
| jeblair | (the ftp is a shell as well, cause for some reason, ansible hasn't grown an ftp module yet ???) | 15:11 |
| mordred | jeblair: who uses FTP? EVERYONE uses object storage didn't you know? | 15:12 |
| *** david-lyle has quit IRC | 15:12 | |
| *** vhosakot has quit IRC | 15:12 | |
| mordred | jeblair: also, we could write one - I think we know some people who might merge it | 15:12 |
| mordred | or, we could also stop using ftp | 15:12 |
| jeblair | mordred: how about we stop using our ftp server instead ;) | 15:12 |
| mordred | ++ | 15:12 |
| *** yamamoto has joined #openstack-infra | 15:12 | |
| *** yamamoto has quit IRC | 15:12 | |
| *** ayoung has joined #openstack-infra | 15:12 | |
| pabelanger | rcarrillocruz: +2 | 15:12 |
| *** david-lyle has joined #openstack-infra | 15:12 | |
| rcarrillocruz | thx | 15:13 |
| *** armax has joined #openstack-infra | 15:13 | |
| jeblair | rcarrillocruz: i'd be okay enqueing infracloud changes directly to gate | 15:13 |
| * rcarrillocruz takes a note | 15:13 | |
| rcarrillocruz | cos i was +2 ing myself stuff due to urgency | 15:13 |
| rcarrillocruz | thx | 15:13 |
| *** nwkarsten has joined #openstack-infra | 15:14 | |
| rcarrillocruz | but that works for me too :-) | 15:14 |
| jeblair | it's an operational issue (and actually, we traditionally are okay with patches going straight to gate -- the clean check thing is more for openstack itself) | 15:14 |
| jeblair | rcarrillocruz: you know how to do that? | 15:14 |
| *** senk_ has joined #openstack-infra | 15:14 | |
| *** nwkarste_ has joined #openstack-infra | 15:14 | |
| rcarrillocruz | hmm, good point, i used to with the previous gozer gerrit (gerrit 2.4) | 15:15 |
| jeblair | rcarrillocruz: example from my history on zuul.o.o: zuul enqueue --trigger gerrit --pipeline gate --project openstack/requirements --change 352454,2 | 15:15 |
| rcarrillocruz | ah good, thru zuul | 15:15 |
| rcarrillocruz | thanks | 15:15 |
| *** dimtruck is now known as zz_dimtruck | 15:15 | |
| jeblair | rcarrillocruz: it'll still need a W+1, but won't need the +1 from jenkins | 15:16 |
| *** zz_dimtruck is now known as dimtruck | 15:16 | |
| *** amitgandhinz has quit IRC | 15:16 | |
| *** amitgandhinz has joined #openstack-infra | 15:16 | |
| *** wewe0901 has joined #openstack-infra | 15:18 | |
| prometheanfire | was anyone able to look at https://review.openstack.org/#/c/310865/ | 15:18 |
| *** nwkarsten has quit IRC | 15:18 | |
| zigo | pabelanger: The jessie-newton-backports is now working great, packages gets slowly built, one after the other. Thanks so much for all your help. | 15:18 |
| prometheanfire | gentoo support for the elements in project-config | 15:18 |
| *** aimeeu has joined #openstack-infra | 15:18 | |
| *** senk_ has quit IRC | 15:19 | |
| zigo | pabelanger: I've wrote a "named schroot" patch, so we can get rid of the --ignore=wrongdistribution, it's currently broken, but the fix is on its way. I'll let you know when it's really working, so we can merge https://review.openstack.org/353489 | 15:19 |
| pabelanger | zigo: great | 15:20 |
| *** oanson has quit IRC | 15:20 | |
| pabelanger | zigo: ya, ping me once you land your change, we can land the revert | 15:20 |
| *** stewie925 has joined #openstack-infra | 15:20 | |
| *** pedroalvarez has joined #openstack-infra | 15:21 | |
| stewie925 | hello infra gurus | 15:21 |
| YorikSar | Hello, Infra wizards. Is everything OK with Gerrit login? I get page asking for OpenID (instead of redirecting to LP) with error "Provider is not supported, or was incorrectly entered." | 15:21 |
| stewie925 | YorikSar: I have the same question too! :) | 15:21 |
| zigo | pabelanger: It's best if you abandon your patch, so that the new --sbuild-chroot-name gets added at the same time (see #353489) | 15:21 |
| aimeeu | Launchpad login is impacted as well - I received a 503 Gateway error when I tried to log in | 15:21 |
| pabelanger | zigo: sure | 15:21 |
| YorikSar | stewie925: But yours is with "gurus" ;) | 15:22 |
| stewie925 | lol | 15:22 |
| stewie925 | thanks for doing the typing for me, YorikSar | 15:22 |
| YorikSar | When I enter what supposed to be my Launchpad OpenID URL (https://launchpad.net/~yorik-sar) I get "OpenID provider not permitted by site policy." | 15:23 |
| *** sambetts is now known as sambetts|afk | 15:24 | |
| openstackgerrit | Zhenguo Niu proposed openstack-infra/project-config: Create new project nimble https://review.openstack.org/353575 | 15:25 |
| stewie925 | strange that I was able to get into Launchpad.net - it was able to auto-log me in | 15:25 |
| *** vincentll has quit IRC | 15:26 | |
| armax | YorikSar: I get the same error | 15:26 |
| YorikSar | Ok, now Launchpad says "OpenID Provider Is Unavailable at This Time" when I try to sign in. Seems to be Ubuntu SSO error. | 15:28 |
| openstackgerrit | Zhenguo Niu proposed openstack-infra/project-config: Create new project Nimble https://review.openstack.org/353575 | 15:28 |
| rcarrillocruz | pabelanger: a couple of changes for bluebox you pushed merged during my morning | 15:28 |
| *** andymaier has quit IRC | 15:28 | |
| fungi | stewie925: armax: YorikSar: you probably want to ask in #launchpad since it looks like their sso system has died again | 15:28 |
| armax | fungi: ack | 15:28 |
| armax | thanks | 15:28 |
| pabelanger | rcarrillocruz: great, thanks | 15:29 |
| fungi | hopefully this is one of those short-term outages, and not like that time it was out of commission for half a day | 15:29 |
| YorikSar | fungi: BTW, how come we still depend on their OpenID when we have our openstackid.org? | 15:29 |
| fungi | YorikSar: partly due to the complexity of planning the migration. there's no direct mapping between launchpad users and foundation accounts used on openstackid.org | 15:30 |
| YorikSar | Is it possible to add two OpenID URLs to a user in Gerrit? | 15:30 |
| fungi | there are some things we can try to use to map them up, for example e-mail addresses, but they're not guaranteed to match, lp lets you make them private, some lp users on gerrit et cetera may not have foundation accounts at all, et cetera | 15:30 |
| aimeeu | fungi: thanks for the info and suggestion | 15:30 |
| *** adrian_otto has quit IRC | 15:31 | |
| zigo | pabelanger: My fix worked, we now just need #353552 to land. | 15:31 |
| pabelanger | zigo: ack | 15:31 |
| fungi | YorikSar: yes, we can do it by editing the database, so any we're able to programmatically map up we could make temporarily usable from either openid provider i think. the challenge is that anyone logging in via openstackid.org without a predetermined mapping will end up with a new gerrit account disconnected from their old one | 15:31 |
| zigo | (this includes the POST job being fully processed, which these days can take hours...) | 15:31 |
| *** itisha has joined #openstack-infra | 15:32 | |
| *** weshay is now known as weshay_mtg | 15:32 | |
| pabelanger | rcarrillocruz: mind helping https://review.openstack.org/#/c/353489/ land | 15:32 |
| rcarrillocruz | sure | 15:32 |
| YorikSar | fungi: Like for backup when Launchpad is down to be able to login with openstackid.org. The same could be used for migration (like stop supporting Launchpad URLs on some date). | 15:32 |
| *** senk_ has joined #openstack-infra | 15:32 | |
| fungi | YorikSar: the gerrit webui/api don't give users the ability to add and remove openids themselves, and gerrit autocreates accounts any time it's presented with an unrecognized openid | 15:32 |
| fungi | so that's the challenge | 15:33 |
| rcarrillocruz | +A | 15:33 |
| *** pt_15 has joined #openstack-infra | 15:33 | |
| fungi | YorikSar: it's possible we can solve it with a gerrit plugin, but need someone who doesn't mind spaghetti java to have free time to write it | 15:33 |
| eglute | Hello Infra, I am getting "OpenID provider not permitted by site policy." when trying to login to gerrit. | 15:33 |
| fungi | eglute: yes, launchpad's sso seems to have broken recently | 15:34 |
| fungi | someone in #launchpad might have details | 15:34 |
| hogepodge | I was seeing similar problems to eglute, but it came back up in the last few minutes. | 15:34 |
| YorikSar | fungi: Ah... Then this burden should be on openstackid.org side. Like let Gerrit login via openstackid.org and that could allow to either add launchpad-id or create new openstack-id from launchpad-id... | 15:34 |
| * hogepodge shrugs | 15:34 | |
| *** pedroalvarez has left #openstack-infra | 15:34 | |
| *** dimtruck is now known as zz_dimtruck | 15:34 | |
| fungi | hogepodge: maybe they're load-balancing to multiple frontend servers and one is broken? | 15:35 |
| stewie925 | openstack gerrit login is working | 15:35 |
| fungi | or maybe they just fixed it | 15:35 |
| *** rfolco has quit IRC | 15:35 | |
| stewie925 | its back in business | 15:35 |
| YorikSar | fungi: They say they've just fixed it. | 15:35 |
| pleia2 | good morning | 15:35 |
| rcarrillocruz | hey pleia2 | 15:36 |
| rcarrillocruz | you returned safely? | 15:36 |
| *** ifarkas is now known as ifarkas_afk | 15:36 | |
| mordred | pleia2: zomg. you're in a timezone that makes sense | 15:36 |
| rcarrillocruz | i hope you weren't hit by the Delta outage | 15:36 |
| *** rwsu has quit IRC | 15:36 | |
| fungi | YorikSar: i'm trying to figure out how that would work if we implemented it on the foundation id provider side of things. precreate "legacy" urls for openstackid.org that can be matched to lp sso openid urls we already have in gerrit, and then make users "claim" them somehow with a foundation account? | 15:36 |
| mordred | honestly, I think just getting to the point where we can migrate will be the biggest win | 15:37 |
| mordred | then we don't have to worry about lp urls at all | 15:38 |
| eglute | thank you fungi | 15:38 |
| *** valderrv has joined #openstack-infra | 15:39 | |
| *** senk_ has quit IRC | 15:39 | |
| *** jlanoux has quit IRC | 15:40 | |
| fungi | mordred: agreed, the challenge, as i outlined, is how to match teh accounts already in our systems (especially gerrit) up to new openids | 15:40 |
| fungi | gerrit's implementation makes that especially tricky | 15:40 |
| *** gyee has joined #openstack-infra | 15:41 | |
| mordred | indeed | 15:41 |
| YorikSar | fungi: When user registers LP ID in openstackid.org, it should replace appropriate LP ID with OS ID in Gerrit DB. Then users will have to just once login via openstackid.org and click "add LP ID". | 15:41 |
| *** karthikp_ has joined #openstack-infra | 15:41 | |
| openstackgerrit | Julien Danjou proposed openstack-infra/project-config: panko: fix py34 gating https://review.openstack.org/353588 | 15:42 |
| pleia2 | rcarrillocruz: yes, thanks :) I was flying british airways, they left my luggage in heathrow (still waiting for it, I got back Monday night...) but at least it's on this end | 15:42 |
| rcarrillocruz | ++ | 15:42 |
| mordred | fungi, YorikSar: oh ... so ... we could maybe have openstackid, if the referrer is review.openstack.org, check to see if it has a launchpad id for a user, and if it doesn't, force them to add a launchpad id before it will go on | 15:42 |
| fungi | YorikSar: "it" is where this is breaking down. what is this "it" that has access to edit the gerrit database? | 15:42 |
| YorikSar | fungi: Yeah... We shouldn't trust "small Python script" that much, right? | 15:43 |
| mordred | fungi: then how we get that collected id from openstackid back in to gerrit for the mapping becomes tricky - but there is a point where we can trap and make sure we've collected it that's under our control | 15:43 |
| fungi | the other outstanding concern i have before we migrate, is that openstackid currently has a malleable id: it's based on your "first" and "last" name in your foundation profile. because of course (1) everybody has those? and (2) they never change them? | 15:43 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Refactor layout for clouds_layouts.yaml https://review.openstack.org/353594 | 15:44 |
| mordred | fungi: ++ | 15:44 |
| openstackgerrit | Ivan Udovichenko proposed openstack-infra/project-config: Add new projects https://review.openstack.org/347047 | 15:44 |
| pabelanger | rcarrillocruz: At a problem with ansible-role-cloud-launcher layout, I've refactored to v2.0 ^ | 15:44 |
| pabelanger | rcarrillocruz: basically, everything was setup in the admin-bluebox project because we only use that cloud in shade | 15:44 |
| YorikSar | fungi: Wait... It can change if I change my name?? | 15:44 |
| mordred | pabelanger: ++ | 15:44 |
| rcarrillocruz | ah k, i had clicked to 347047 :D | 15:45 |
| pabelanger | rcarrillocruz: now, we are using 3 different clouds names for bluebox | 15:45 |
| fungi | YorikSar: as i understand it, yes | 15:45 |
| *** zz_dimtruck is now known as dimtruck | 15:45 | |
| *** sdake has joined #openstack-infra | 15:45 | |
| rcarrillocruz | interesting, so one cloud just for flavors? | 15:46 |
| fungi | YorikSar: you're still hand-waving with the "small python script" idea. the foundation member system doesn't have an event stream for it to trigger from (or if it does, i don't know where that's documented) | 15:46 |
| pabelanger | rcarrillocruz: no, 2 clouds using the same flavor | 15:46 |
| pabelanger | rcarrillocruz: well, 2 projects | 15:46 |
| rcarrillocruz | yeah | 15:47 |
| YorikSar | fungi: Isn't that system under Infra control as well? | 15:47 |
| fungi | YorikSar: we don't maintain the foundation member system, so would also be hesitant to give it credentials to directly modify our gerrit database | 15:47 |
| fungi | YorikSar: no, it's not | 15:47 |
| *** sdake_ has quit IRC | 15:48 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Use openstackzuul-bluebox not that jenkins is gone https://review.openstack.org/353602 | 15:48 |
| fungi | YorikSar: and it's written in php. i have very limited confidence in my ability to do much of a security review on whatever feature it grew to modify gerrit's database, if that were to come to pass | 15:48 |
| openstackgerrit | Michael Ionkin proposed openstack-infra/project-config: Disable Heat integration tests https://review.openstack.org/353603 | 15:49 |
| YorikSar | fungi: Oh, ok... So we need it to collect LP IDs from users and then fetch it from there and put it into Gerrit in big batches. | 15:49 |
| *** matrohon has quit IRC | 15:50 | |
| fungi | YorikSar: probably, yes, or as i said precreate legacy ids and add them as sencdary entries in gerrit, then map them up where we can and have some feature in the foundation member system where people can "claim" the rest (maybe by validating one of their e-mail address from gerrit or something) | 15:50 |
| *** mhickey has quit IRC | 15:51 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Use openstackzuul-bluebox not that jenkins is gone https://review.openstack.org/353602 | 15:51 |
| openstackgerrit | Michael Ionkin proposed openstack-infra/project-config: Disable Heat integration tests https://review.openstack.org/353603 | 15:51 |
| rcarrillocruz | pabelanger: did you create the flavor fine with the launcher last night? | 15:52 |
| *** m1st3rwr0ng has quit IRC | 15:52 | |
| rcarrillocruz | or you just pushed the change and you still have to run it, now that it has merged | 15:52 |
| pabelanger | rcarrillocruz: actually, I think we can create the flavor under admin-bluebox, since it will be used my all projects | 15:52 |
| pabelanger | rcarrillocruz: yes, flavor created properly | 15:52 |
| YorikSar | fungi: I'm just afraid those secondare entries will bite us after you said that if (or rather when) for anyone smth goes wrong and OS ID will be missing from Gerrit DB, it'll create a new entry. | 15:52 |
| rcarrillocruz | cool | 15:52 |
| fungi | YorikSar: we're going to need to do a similar exercise with other systems too (wiki, storyboard, et cetera) which will have some overlapping users but also new ones that perhaps didn't have accounts in the systems we've migrated prior to that | 15:53 |
| pabelanger | http://mirror.regionone.bluebox-sjc1.openstack.org is backonline too, dns updated. I used launch_node.py to create that | 15:53 |
| pabelanger | confirmed nodepool.o.o is setup properly too | 15:54 |
| fungi | YorikSar: yeah, the alternative is to replace the lp openids with the mapping we've precreated in a maintenance window and switch openid providers at the same time | 15:54 |
| pabelanger | so, I think we can land https://review.openstack.org/#/c/352973/ and restart nodepool-builder so we can start uploading images again to bluebox | 15:54 |
| fungi | YorikSar: that just makes it a bit harder to switch back and forth if we discover an issue | 15:55 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Use openstackzuul-bluebox now that jenkins is gone https://review.openstack.org/353602 | 15:55 |
| *** piet has joined #openstack-infra | 15:56 | |
| *** Sukhdev has joined #openstack-infra | 15:56 | |
| pabelanger | fungi: mordred: did you want review bluebox-sjc1 before we started uploading images with nodepool? max-servers is still -1 | 15:56 |
| fungi | pabelanger: as much as i would like to, my plate is overflowing so i'll pass | 15:57 |
| pabelanger | fungi: ack | 15:57 |
| mordred | pabelanger: lemme take a quick peak | 15:59 |
| *** matthewbodkin has quit IRC | 16:00 | |
| *** inc0 has joined #openstack-infra | 16:00 | |
| openstackgerrit | Merged openstack-infra/system-config: Set new private and public IPs for controller00.vanilla https://review.openstack.org/353442 | 16:00 |
| *** seldenr has joined #openstack-infra | 16:01 | |
| *** rcernin has quit IRC | 16:01 | |
| zaro | morning | 16:01 |
| mnaser | forgive me but does anyone else' browser blows up on zuul.openstack.org | 16:01 |
| *** vhosakot_ has quit IRC | 16:01 | |
| mnaser | and is there a better way of looking at where things are | 16:01 |
| mnaser | i usually go there to check logs for an on-going job for a review but it takes ages to get any info out | 16:02 |
| *** vhosakot has joined #openstack-infra | 16:03 | |
| *** raunak has joined #openstack-infra | 16:04 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Fix resetting /etc/hosts https://review.openstack.org/351917 | 16:05 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Raise ping test count after multinode setup https://review.openstack.org/351916 | 16:05 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Make $TRIPLEO_ROOT overrideable https://review.openstack.org/350539 | 16:05 |
| openstackgerrit | Merged openstack-infra/system-config: Switch to ironic vanilla inventory https://review.openstack.org/353545 | 16:05 |
| openstackgerrit | Henry Gessau proposed openstack-infra/elastic-recheck: Remove query for bug 1526675 https://review.openstack.org/353619 | 16:06 |
| openstack | bug 1526675 in Glance liberty "test_models_sync fails with 'Models and migration scripts aren't in sync'" [Critical,Fix committed] https://launchpad.net/bugs/1526675 - Assigned to Flavio Percoco (flaper87) | 16:06 |
| mtreinish | yolanda, rcarrillocruz, fungi: if you get a sec can you take a look at: https://review.openstack.org/#/c/353060 it should be quick | 16:06 |
| *** Na3iL has quit IRC | 16:07 | |
| jeblair | mordred: do you want to restore your +A on https://review.openstack.org/353553 | 16:08 |
| mordred | jeblair: I do! | 16:08 |
| *** roxanaghe has joined #openstack-infra | 16:08 | |
| *** shamail has joined #openstack-infra | 16:08 | |
| krotscheck | matt-borland Looking | 16:08 |
| Krenair | pleia2, re https://review.openstack.org/352057 - I thought you weren't using Xenial? | 16:08 |
| pabelanger | mordred: thanks | 16:08 |
| matt-borland | thanks krotscheck! | 16:08 |
| mordred | pabelanger: cloud looks good - although amusingly the admin user does not have a role in the openstackci or openstackzuul projects :) | 16:09 |
| mordred | pabelanger: did you make the AFS mirror already? | 16:09 |
| pleia2 | Krenair: we aren't, just a note for the future | 16:09 |
| Krenair | ok.. | 16:09 |
| pabelanger | mordred: Ya, I didn't make any changes to existing user permissions | 16:10 |
| pabelanger | mordred: yes! Worked first time using launch_node | 16:10 |
| mordred | pabelanger: and it's not necessary to | 16:10 |
| mordred | pabelanger: woot! | 16:10 |
| pleia2 | Krenair: I approved it :) | 16:10 |
| Krenair | pleia2, will the system-config stuff work on xenial? if so should I attempt to prepare puppet-mediawiki for that, after trusty is done? | 16:10 |
| krotscheck | matt-borland LTM | 16:10 |
| krotscheck | LGTM | 16:10 |
| pabelanger | mordred: if all is good, https://review.openstack.org/#/c/352973/ will start uploading DIBs to bluebox | 16:10 |
| krotscheck | AJaeger: I'm confused. One of your comments says "this requires a bindep release" another says it doesn't? | 16:10 |
| pleia2 | Krenair: I haven't had a close look at it, for most of these I'd like to get jpmaxman to chime in since he's the one who has been working to get the demo server going on trusty | 16:11 |
| *** nherciu has joined #openstack-infra | 16:11 | |
| *** weshay_mtg is now known as weshay | 16:11 | |
| *** amitgandhinz has quit IRC | 16:12 | |
| *** amitgandhinz has joined #openstack-infra | 16:12 | |
| Krenair | pleia2, okay. these puppet-mediawiki patches are based on pretty much my own local demo server | 16:13 |
| *** yamamoto has joined #openstack-infra | 16:13 | |
| pleia2 | Krenair: btw, we're using a gerrit topic for these, so if you could use "git review -t wiki-upgrade" when submitting patches, it will help us with reviewing them | 16:13 |
| Krenair | yep | 16:13 |
| pleia2 | I updated the ones that are pending for now so I could go through them all | 16:13 |
| pleia2 | thanks | 16:14 |
| Krenair | thanks | 16:14 |
| *** Sukhdev has quit IRC | 16:15 | |
| *** rfolco has joined #openstack-infra | 16:18 | |
| *** kushal has quit IRC | 16:18 | |
| *** hieulq_ has quit IRC | 16:19 | |
| mtreinish | fungi: so tls hiera magic? :) (I also noticed puppet hasn't run on firehose in a while: http://puppetdb.openstack.org/node/firehose01.openstack.org ) | 16:19 |
| *** yamamoto has quit IRC | 16:19 | |
| *** rfolco has quit IRC | 16:20 | |
| rcarrillocruz | o-k | 16:20 |
| rcarrillocruz | now baremetal00 has the baremetal.json populated | 16:20 |
| pabelanger | nice | 16:20 |
| rcarrillocruz | afk a couple hours, babysitting my son now | 16:20 |
| rcarrillocruz | pabelanger: i'll ping you later, we could monitor a provision job | 16:21 |
| pabelanger | rcarrillocruz: yes please | 16:21 |
| *** rfolco has joined #openstack-infra | 16:21 | |
| *** tonytan_brb has joined #openstack-infra | 16:21 | |
| *** Apoorva has joined #openstack-infra | 16:22 | |
| jpmaxman | Krenair: sorry what's the question? | 16:24 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Fix the integrated gate filtering on uncategorized https://review.openstack.org/351924 | 16:24 |
| *** tonytan4ever has quit IRC | 16:24 | |
| Krenair | jpmaxman, I have a ton of puppet-mediawiki patches | 16:24 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/elastic-recheck: Add grenade to the integrated gate projects list https://review.openstack.org/353628 | 16:24 |
| Krenair | open for review | 16:24 |
| Krenair | Me and pleia2 would like you to review them, since you've done effectively the same thing but manually | 16:25 |
| *** raildo has joined #openstack-infra | 16:25 | |
| *** Julien-zte has quit IRC | 16:25 | |
| jpmaxman | Krenair: ok. Is this to setup a new server? | 16:26 |
| Krenair | guess so | 16:26 |
| *** vhosakot has quit IRC | 16:26 | |
| jpmaxman | I have effectively done very little other than to migrate the db to 1.27 and get the no captcha working | 16:26 |
| *** vhosakot has joined #openstack-infra | 16:26 | |
| jpmaxman | Where are the review? | 16:27 |
| pleia2 | anteaya: can you have a look at https://review.openstack.org/#/c/352056/ when you have a chance? (I think you looked into ffmpeg previously for the wiki?) | 16:27 |
| mordred | pabelanger: btw - I delayed on releasing shade becuase I learned about a magnum rename and we had a new magnum api call in shade master that I wanted to change to match before we released it | 16:27 |
| pleia2 | jpmaxman: https://review.openstack.org/#/q/topic:wiki-upgrade+is:open | 16:27 |
| pleia2 | we're using the topic we defined in the spec | 16:27 |
| jpmaxman | The only thing under puppet was the settings file | 16:27 |
| openstackgerrit | Merged openstack-infra/project-config: Revert "Remove bluebox" https://review.openstack.org/352973 | 16:28 |
| openstackgerrit | Ryan Selden proposed openstack-infra/irc-meetings: Add a UTC clock for ease of use https://review.openstack.org/353008 | 16:28 |
| *** pcaruana has quit IRC | 16:29 | |
| pleia2 | jpmaxman: did you use the puppet module to set up the test instance? | 16:29 |
| *** jaosorior has quit IRC | 16:29 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add gpg signing job for reprepro https://review.openstack.org/343336 | 16:31 |
| pabelanger | mordred: ack, why the name change? | 16:31 |
| wznoinsk | ls | 16:31 |
| jpmaxman | Ok I think I see what you are trying to do Krenair | 16:31 |
| pabelanger | any project-config-core like to review 343336, add signed packages to debian-openstack jobs | 16:31 |
| jpmaxman | pleia2: no it is a clone of the current wiki instance | 16:31 |
| openstackgerrit | Alfredo Moralejo proposed openstack-infra/system-config: Add packstack installation logs to logstash https://review.openstack.org/321604 | 16:31 |
| fungi | pleia2: the test instance was created by takign a snapshot of wiki.o.o and then booting a new instance from that | 16:32 |
| mordred | pabelanger: the current name "bay" is associated with docker swarm - but k8s and mesos both use the word "cluster" | 16:32 |
| mordred | pabelanger: the magnum team (rightly I think) thinks that adopting cluster is a more meaninful name to people than bay | 16:32 |
| pleia2 | jpmaxman, fungi, ah, gotcha, so it's still precise? | 16:32 |
| fungi | pleia2: it was upgraded from precise to trusty in-place between mw 1.25 and 1.26 | 16:32 |
| pabelanger | mordred: ya, make sense | 16:32 |
| pleia2 | Krenair's patches are very good, and help get us to trusty | 16:32 |
| pleia2 | fungi: ok, thanks | 16:33 |
| fungi | pleia2: since you have to do progressive upgrades of mw to migrate the database version to bersion, and 1.26 required trusty | 16:33 |
| *** Joostvdg has joined #openstack-infra | 16:33 | |
| fungi | s/bersion/version/ | 16:33 |
| * pleia2 nods | 16:33 | |
| openstackgerrit | Merged openstack-infra/project-config: Increase rh1 max-servers to 75 https://review.openstack.org/353111 | 16:33 |
| Joostvdg | Hi, anyone who can help me with openstack review account problems? | 16:34 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-elastic_recheck: Rework uncategorized script to handle multiple html files https://review.openstack.org/353630 | 16:34 |
| mtreinish | pleia2, rcarrillocruz, fungi, jesusaur, asselin: ^^^ we need to land that since: the corresponding e-r change already merged (it was faster than I expected) | 16:35 |
| *** amoralej is now known as amoralej|off | 16:35 | |
| pleia2 | mtreinish: lgtm | 16:35 |
| fungi | Joostvdg: can you elaborate? hard to help you if we don't know what review account problems you have | 16:35 |
| mtreinish | pleia2: thanks | 16:36 |
| *** tonytan_brb is now known as tonytan4ever | 16:36 | |
| *** Qiming_ has quit IRC | 16:36 | |
| jpmaxman | fungi pleia2 Krenair I think the general plan was to move the wiki to the new instance running 1.27 w/ the db on trove. Then we can launch and test a puppetized instance in parallel as the DB will be on trove both could point to the same or a clone of the production db. Once it is certified as fully puppeted and working as desired could do a final switch | 16:36 |
| jpmaxman | to that server. | 16:36 |
| *** psachin has quit IRC | 16:37 | |
| pabelanger | fungi: kolla has asked to move their IRC logs from #kolla to #openstack-kolla, is that something we want to do or just leave them for historical purposes: http://lists.openstack.org/pipermail/openstack-infra/2016-August/004617.html | 16:38 |
| pleia2 | jpmaxman: great, I think Krenair's patches will help with the puppetized instance | 16:38 |
| fungi | jpmaxman: i agree that's the fastest way to get 1.27 running in production, which hopefully frees up time currently wasted on spam fighting to devote to configuration management efforts on it | 16:38 |
| *** yaume has quit IRC | 16:38 | |
| Joostvdg | Before I start a review - following the sandbox "tutorial" - I need to sign an agreement. This agreement is only valid if it contains contact detail. | 16:38 |
| *** mrmartin has joined #openstack-infra | 16:39 | |
| Joostvdg | When I try to save contact details, it fails with "Could not save" | 16:39 |
| fungi | Joostvdg: it needs your gerrit preferred e-mail address to match the primary address on your foundation member profile | 16:39 |
| jpmaxman | Krenair: I'll review your patches. fungi What is the base image you all start with? if I have time I'd love to try to go from zero to wiki. One thing | 16:39 |
| *** ihrachys has quit IRC | 16:39 | |
| fungi | Joostvdg: did you follow these instructions when setting up your accounts? http://docs.openstack.org/infra/manual/developers.html#account-setup | 16:39 |
| *** degorenko is now known as _degorenko|afk | 16:39 | |
| jpmaxman | Krenair: one thing I was looking at was I created the extensions & skins directory in /srv/mediawiki then symbolic linked to the extension in the mediawikis extensions and skins folder - this matches how the image assets were being handled | 16:40 |
| fungi | jpmaxman: 4b3e2127-4623-43df-a938-e60268c236c2 | Ubuntu 14.04 LTS (Trusty Tahr) (PVHVM) | 16:40 |
| jpmaxman | potentially this would mean you could keep the mediawiki connected to the mediawiki git | 16:41 |
| Krenair | thanks jpmaxman | 16:41 |
| pabelanger | fungi: pleia2: mordred: jeblair: any objections to restarting nodepool-builder to pick up the config changes for bluebox-sjc1? This is needed to start uploading our DIBs | 16:41 |
| jpmaxman | but I'm not sure that's the best practice you want to follow | 16:41 |
| *** Qiming has joined #openstack-infra | 16:41 | |
| jpmaxman | so it's kind of up to you all how the mediawiki code is managed | 16:41 |
| *** stew925 has joined #openstack-infra | 16:41 | |
| Krenair | jpmaxman, so wait... | 16:41 |
| Joostvdg | fungi: I did, and email account is the same | 16:41 |
| Krenair | You cloned the extensions and skins in /srv/mediawiki | 16:42 |
| jpmaxman | either way what was clear is we need to stay on top of the updates to mediawiki much better | 16:42 |
| Krenair | Then added symlinks to them in the appropriate subdirectory of /srv/mediawiki/w ? | 16:42 |
| mordred | pabelanger: I do not have any objections | 16:42 |
| jpmaxman | Krenair: not exactly - I left the extensions and skins in place that are included with mediawiki | 16:42 |
| fungi | Joostvdg: and you did a foundation member account there like the instructions specify, not a community member account? take a look at https://ask.openstack.org/question/56720 and see if those troubleshooting suggestions help | 16:43 |
| jpmaxman | and then I symlinked to ones that were added in addition to the included ones | 16:43 |
| *** sdague has quit IRC | 16:43 | |
| *** florianf has quit IRC | 16:43 | |
| *** cdent has quit IRC | 16:43 | |
| jpmaxman | the /srv/mediawiki/w is just a symlink to the latest version | 16:43 |
| *** asettle has quit IRC | 16:43 | |
| *** sshnaidm is now known as sshnaidm|afk | 16:43 | |
| *** asettle has joined #openstack-infra | 16:43 | |
| jpmaxman | on the existing machine there was /srv/mediawiki/slot0 /srv/mediawiki/slot1 | 16:44 |
| jpmaxman | slot0 was the previous version. slot1 was 1.25 pre release | 16:44 |
| jpmaxman | I made slot2 for 1.25 release latest, slot3 for 1.26 release latest & slot4 for 1.27 release latest | 16:44 |
| pleia2 | do we still want that red/green anymore? I assumed we'd just have a wiki-dev moving forward for that | 16:44 |
| jpmaxman | in a new server with an upgraded db you don't really need this - could just start with slot4 | 16:44 |
| jpmaxman | but I think that's how they were managing the forward release upgrades | 16:45 |
| pleia2 | (like other services we run) | 16:45 |
| *** Swami_ has quit IRC | 16:45 | |
| jpmaxman | then just repointed the /srv/mediawiki/w symlink to the latest slot or the previous slot if needed to roll back | 16:45 |
| fungi | yeah, i think long-term we don't need "slots" for this, though it made for a convenient way to step through the upgrades | 16:45 |
| Krenair | I don't think you need to keep that setup on the new system | 16:45 |
| fungi | the way Ryan_Lane had been doing upgrades previously was to switch back and forth between slot0 and slot1 | 16:45 |
| jpmaxman | I don't really have an opinion on how to do it - just was trying to follow established convention as best as possible | 16:45 |
| jpmaxman | ahhh so you only had two I see | 16:46 |
| *** sdague has joined #openstack-infra | 16:46 | |
| pabelanger | mordred: thanks | 16:46 |
| pleia2 | jpmaxman: in this case, I think we can migrate away from the old practice :) | 16:46 |
| Joostvdg | fungi: thanks, it seems I have read over that | 16:46 |
| fungi | but ultimately i expect we'll just replace the code in place through configuration management and have it fire database migrations whenever that happens | 16:46 |
| pabelanger | I'll restart nodepool-builder in 5mins, if others don't have objections | 16:46 |
| pleia2 | I'm treating Krenair like our new Ryan_Lane | 16:46 |
| AJaeger | krotscheck: I first thought it needs one - convinced myself it does not need one. We just need to update the project-config script, it uses -f to read the file and thus does not use the defaults at all | 16:47 |
| pleia2 | Krenair: in case you lack background, Ryan_Lane used to work for wikimedia and maintained the old server for us | 16:47 |
| jpmaxman | so Krenair you should probably get the settings file from the new machine | 16:47 |
| jpmaxman | other than the database the main thing that changed were settings file conventions | 16:47 |
| jpmaxman | between the version we're on now and 1.27 | 16:48 |
| *** PollyZ has quit IRC | 16:48 | |
| pleia2 | jpmaxman: think you could compare the one on the new machine and submit a change in geerit? not everyone has access to these servers | 16:48 |
| pleia2 | gerrit too | 16:48 |
| fungi | the main benefit we got from the old slot-based upgrades was the ability to test an update in production and still be able to roll back for the most part to the previous increment if something was wrong, but instead we'd rather have a separate wiki-dev where we try out the latest versions and then when we are comfortable it's working we bump a version pin (git commit, tag, tarball url, whatever) for | 16:48 |
| fungi | the production server | 16:48 |
| AJaeger | jeblair: thanks for the docs retry patch for zuul-cloner | 16:49 |
| jpmaxman | pleia2: yup I'll try to do that | 16:49 |
| pleia2 | jpmaxman: awesome, thanks :) | 16:50 |
| jpmaxman | brb | 16:50 |
| *** tosky has quit IRC | 16:50 | |
| *** Joostvdg has quit IRC | 16:50 | |
| pabelanger | #status log nodepool-builder restarted on nodepool.o.o to pickup nodepool.yaml changes for bluebox-sjc1 | 16:51 |
| openstackstatus | pabelanger: finished logging | 16:51 |
| jeblair | AJaeger: zuul-launcher. you're welcome. :) | 16:52 |
| *** lucasagomes is now known as lucas-dinner | 16:52 | |
| *** kdas__ has joined #openstack-infra | 16:52 | |
| *** jpich has quit IRC | 16:53 | |
| *** oanson has joined #openstack-infra | 16:53 | |
| *** tphummel has joined #openstack-infra | 16:53 | |
| *** _nadya_ has quit IRC | 16:53 | |
| pabelanger | re: bluebox-sjc1, how many nodes do we want to start with? Our previous max-servers was 39, so do we want to start there or slowly ramp up? | 16:54 |
| *** rlandy is now known as rlandy|afk | 16:54 | |
| *** xgerman has quit IRC | 16:56 | |
| AJaeger | project-config cores, can we merge https://review.openstack.org/352871, please? - I first thought we need a new bindep release but that's wrong, we use -f and thus can merge this any time so that it's available in images for use. | 16:56 |
| AJaeger | fungi, do you agree with my reasoning? ^ | 16:56 |
| *** xgerman has joined #openstack-infra | 16:56 | |
| *** kdas__ is now known as kushal | 16:57 | |
| *** jerryz has joined #openstack-infra | 16:57 | |
| *** kushal has quit IRC | 16:57 | |
| *** kushal has joined #openstack-infra | 16:57 | |
| *** karthikp_ has quit IRC | 16:57 | |
| openstackgerrit | devdatta-kulkarni proposed openstack-infra/project-config: Move to Ubuntu-trust for Libvirt/LXC test https://review.openstack.org/353640 | 16:57 |
| *** thorst_ has quit IRC | 16:58 | |
| fungi | AJaeger: yep, that can go in now. we still need to dig into our cache-bindep element, which likely will rely on a bindep release before we begin caching packages listed in bindep.txt files | 16:58 |
| *** nwkarste_ has quit IRC | 16:59 | |
| AJaeger | oh, good point... | 16:59 |
| Krenair | pleia2, I knew Ryan Lane a little bit, last saw him (in person) in January. I think this isn't the first wiki he was maintaining that I'm now picking up :) | 16:59 |
| *** nwkarsten has joined #openstack-infra | 16:59 | |
| fungi | pabelanger: which won't actually prevent it from working, but could result in additional downloads over the network in jobs for projects switching to bindep.txt | 16:59 |
| fungi | er, AJaeger: ^ | 17:00 |
| fungi | sorry pabelanger | 17:00 |
| *** asettle has quit IRC | 17:00 | |
| *** Swami_ has joined #openstack-infra | 17:00 | |
| AJaeger | fungi, working on it ... | 17:00 |
| *** shamail has quit IRC | 17:00 | |
| Krenair | your setup has some similarities with our wikitech-static site | 17:00 |
| pleia2 | Krenair: haha, nice :) | 17:00 |
| *** cody-somerville has quit IRC | 17:01 | |
| *** xarses has quit IRC | 17:01 | |
| fungi | Ryal_Lane is an awesome guy. i hope he's having fun, wherever he's off to now | 17:01 |
| pleia2 | Krenair: feel free to lmk if you ever come to the mediawiki HQ in SF, I live next door and would be happy to buy you a coffee | 17:01 |
| *** oanson has quit IRC | 17:01 | |
| fungi | er, Ryan_Lane | 17:01 |
| pabelanger | fungi: do you mind adding https://review.openstack.org/#/c/343336/ to your review list? signing jobs for reprepro, pretty sure it is ready to try out | 17:02 |
| Krenair | Next time I go over there I'll let you know pleia2 :) | 17:03 |
| *** nwkarsten has quit IRC | 17:04 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: cache-bindep: Handle bindep.txt https://review.openstack.org/353647 | 17:04 |
| AJaeger | fungi ^ | 17:04 |
| mordred | pabelanger: so, wehn we get around to it - we should totally move https://launchpad.net/~openstack-ci-core/+archive/ubuntu/vhd-util to our apt repo farm | 17:05 |
| openstackgerrit | devdatta-kulkarni proposed openstack-infra/project-config: Move to Ubuntu-trusty for Libvirt/LXC test https://review.openstack.org/353640 | 17:05 |
| Krenair | fungi, it's on his LinkedIn profile :) | 17:05 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Correct configuration file usage. https://review.openstack.org/347984 | 17:06 |
| *** ayoung has quit IRC | 17:06 | |
| pabelanger | mordred: ++ | 17:06 |
| *** xenogear_ has joined #openstack-infra | 17:06 | |
| *** Na3iL has joined #openstack-infra | 17:07 | |
| *** tbarron is now known as tbarron|lunch | 17:09 | |
| openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: Pass a volume_type for the pingtest https://review.openstack.org/353654 | 17:10 |
| *** berendt has quit IRC | 17:11 | |
| AJaeger | team, translations and constraints are not working together for me ;( I used the wrong variable name and expected that it was set by magic. Could you review https://review.openstack.org/#/c/353418/ , please? | 17:11 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Bump baremetal vm memory to 5.5 GB https://review.openstack.org/353655 | 17:13 |
| openstackgerrit | Merged openstack-infra/project-config: Handle new bindep file https://review.openstack.org/352871 | 17:14 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Bump baremetal vm memory to 5.5 GB https://review.openstack.org/353655 | 17:14 |
| openstackgerrit | Steve Martinelli proposed openstack-infra/project-config: test heat-translator gate fail https://review.openstack.org/353656 | 17:15 |
| *** stewie925 has quit IRC | 17:15 | |
| *** dtantsur is now known as dtantsur|afk | 17:19 | |
| *** tongli has quit IRC | 17:19 | |
| *** inc0 has quit IRC | 17:19 | |
| *** tphummel has quit IRC | 17:20 | |
| *** ayoung has joined #openstack-infra | 17:20 | |
| *** tphummel has joined #openstack-infra | 17:21 | |
| *** e0ne has quit IRC | 17:21 | |
| *** mrhillsman is now known as mrhillsman_away | 17:22 | |
| openstackgerrit | Isaku Yamahata proposed openstack-infra/project-config: networking-odl: cover more combinations of version https://review.openstack.org/347045 | 17:24 |
| *** sputnik13 has joined #openstack-infra | 17:24 | |
| *** harlowja has joined #openstack-infra | 17:27 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Bring bluebox-sjc1 online to 1/2 capacity https://review.openstack.org/353659 | 17:29 |
| mgagne | is there a way to cancel a graceful shutdown of Zuul? | 17:30 |
| pabelanger | infra-root: project-config-core: ^ brings bluebox-sjc1 back online with 20 nodes. That is 1/2 the capacity before it was shutdown for upgrade | 17:30 |
| jeblair | mgagne: i don't believe so | 17:30 |
| pabelanger | mgagne: I don't believe so | 17:30 |
| mgagne | I guess people don't believe so | 17:30 |
| * fungi wants to believe | 17:30 | |
| *** kzaitsev_mb has quit IRC | 17:30 | |
| pabelanger | I lol'd | 17:31 |
| * AJaeger hopes | 17:31 | |
| fungi | ghostbusters: "we're ready to believe you" | 17:31 |
| mgagne | can state be saved so I can forcefully shut it down and restore previous state then? | 17:31 |
| AJaeger | pabelanger: So, we have images already for bluebox-sjc1? | 17:32 |
| jeblair | mgagne: there is a script in tools/zuul-changes.py that grabs changes currently in the queue and outputs commands to re-enqueue them after a restart | 17:32 |
| pabelanger | AJaeger: yes, they are slowly coming on line now | 17:32 |
| *** thorst_ has joined #openstack-infra | 17:32 | |
| mgagne | jeblair: will it start the job from scratch? | 17:32 |
| jeblair | mgagne: yes | 17:32 |
| mgagne | ok =( | 17:33 |
| fungi | mgagne: and you have to run it once for each pipeline you want to export | 17:33 |
| fungi | assuming you have more than one pipeline defined | 17:33 |
| AJaeger | pabelanger: then let's promote your change to the gate asap - after somebody else +2Aed it | 17:33 |
| mgagne | yea, I do | 17:33 |
| pabelanger | AJaeger: sure, we can do that | 17:33 |
| jeblair | and probably only works for pre-merge pipelines (ie, check/gate) | 17:33 |
| mgagne | alright, lesson learned | 17:33 |
| fungi | mgagne: also i don't know if we've tested it with non-change pipelines (e.g., for those with ref-updated triggers) | 17:33 |
| fungi | er, what jeblair said | 17:33 |
| AJaeger | pabelanger: thanks a lot for getting it online! | 17:34 |
| pabelanger | AJaeger: np! | 17:35 |
| openstackgerrit | Zara proposed openstack-infra/storyboard-webclient: Search for boards and worklists https://review.openstack.org/341562 | 17:35 |
| *** tonytan4ever has quit IRC | 17:35 | |
| pabelanger | ansible-role-cloud-launcher is starting to make it easy | 17:35 |
| openstackgerrit | Pradeep Kilambi proposed openstack-infra/project-config: Create puppet-panko repository https://review.openstack.org/353662 | 17:35 |
| AJaeger | pabelanger: could you review my change 353418, please? | 17:36 |
| *** thorst_ has quit IRC | 17:37 | |
| pabelanger | AJaeger: +3 | 17:37 |
| *** ganesan has joined #openstack-infra | 17:38 | |
| AJaeger | thanks - now waiting for it to get a node ... | 17:38 |
| *** claudiub has quit IRC | 17:39 | |
| *** woodster_ has quit IRC | 17:39 | |
| *** annegentle has joined #openstack-infra | 17:39 | |
| *** thorst_ has joined #openstack-infra | 17:40 | |
| *** annegentle is now known as Guest27834 | 17:40 | |
| *** xenogear_ has quit IRC | 17:40 | |
| *** xenogear_ has joined #openstack-infra | 17:40 | |
| *** priteau has quit IRC | 17:41 | |
| *** mrmartin has quit IRC | 17:41 | |
| *** nwkarsten has joined #openstack-infra | 17:43 | |
| *** inc0 has joined #openstack-infra | 17:43 | |
| *** Apoorva has quit IRC | 17:44 | |
| *** senk_ has joined #openstack-infra | 17:44 | |
| *** Apoorva has joined #openstack-infra | 17:45 | |
| jlk | Where is the what-broke script? | 17:45 |
| *** senk_ has quit IRC | 17:45 | |
| *** rbuzatu has quit IRC | 17:45 | |
| *** priteau has joined #openstack-infra | 17:45 | |
| openstackgerrit | Alexander Monk proposed openstack-infra/puppet-mediawiki: Also require php5-memcached https://review.openstack.org/352057 | 17:45 |
| openstackgerrit | Alexander Monk proposed openstack-infra/puppet-mediawiki: image_scaler: Partially separate packages for Trusty https://review.openstack.org/352056 | 17:45 |
| openstackgerrit | Alexander Monk proposed openstack-infra/puppet-mediawiki: Ensure cache directory is owned by the right user/group https://review.openstack.org/352058 | 17:45 |
| openstackgerrit | Alexander Monk proposed openstack-infra/puppet-mediawiki: Stick to REL1_27 on Trusty, not master https://review.openstack.org/352036 | 17:45 |
| openstackgerrit | Alexander Monk proposed openstack-infra/puppet-mediawiki: Puppetise extension repositories https://review.openstack.org/352055 | 17:45 |
| openstackgerrit | Alexander Monk proposed openstack-infra/puppet-mediawiki: Puppetise LocalSettings.php -> ../Settings.php symlink https://review.openstack.org/352034 | 17:45 |
| openstackgerrit | Alexander Monk proposed openstack-infra/puppet-mediawiki: Also clone the vendor repository for newer MW versions https://review.openstack.org/352035 | 17:45 |
| AJaeger | jlk, I think in requirements repo - but if not, use codesearch.openstack.org | 17:46 |
| jlk | bingo, thanks | 17:46 |
| *** electrofelix has quit IRC | 17:49 | |
| AJaeger | Is any project-config cores around to approve pabelanger's change to give us 20 more VMs, please? https://review.openstack.org/353659 | 17:49 |
| mordred | AJaeger: done | 17:50 |
| mordred | pabelanger is a machine | 17:50 |
| AJaeger | mordred, pabelanger, do you want to promote to gate? Otherwise we wait 5+ hours... | 17:51 |
| AJaeger | thanks, mordred | 17:51 |
| *** Qiming has quit IRC | 17:51 | |
| openstackgerrit | Ryan Selden proposed openstack-infra/irc-meetings: Add a UTC clock and integrate local meeting time https://review.openstack.org/353008 | 17:51 |
| pabelanger | AJaeger: mordred: I can promote now | 17:52 |
| *** asettle has joined #openstack-infra | 17:52 | |
| AJaeger | great! | 17:53 |
| mgagne | so I thought Zuul would queue and store events to be later process when gracefully shutting down, that's the impression I got from the queue only mode warning. | 17:54 |
| *** matt-borland has quit IRC | 17:55 | |
| fungi | mtreinish: the firehose.o.o key/cert/chain are in hiera now using the keys specified in our global site manifest, so will likely get applied on the next puppet pulse | 17:55 |
| *** karthikp_ has joined #openstack-infra | 17:55 | |
| mtreinish | fungi: cool, thanks | 17:55 |
| *** Qiming has joined #openstack-infra | 17:55 | |
| openstackgerrit | Merged openstack-infra/shade: Make shared an optional keyword param to create_network https://review.openstack.org/352881 | 17:56 |
| mtreinish | fungi: although it looks like it's been some time since puppet last ran on firehose: http://puppetdb.openstack.org/node/firehose01.openstack.org | 17:56 |
| openstackgerrit | Merged openstack-infra/shade: Allow creating a floating ip on an arbitrary port https://review.openstack.org/352114 | 17:56 |
| pabelanger | mgagne: did you re-enqueue the events from zuul-changes.py? | 17:57 |
| fungi | mtreinish: i haven't looked for the telltale errors, but since someone approved the system-config change without actually checking that the corresponding keys were in hiera, my guess is that merging corresponds with when puppet stopped running? | 17:57 |
| *** berendt has joined #openstack-infra | 17:57 | |
| pabelanger | mgagne: http://docs.openstack.org/infra/system-config/zuul.html#restarts is the process we do to restart zuul | 17:57 |
| mgagne | pabelanger: no because I thought Zuul would have done it for me (based on the maintenance message) | 17:57 |
| mtreinish | fungi: ah, ok. That makes sense | 17:58 |
| jeblair | mgagne: yes, that's the way it's supposed to work. though because it takes us so long to shutdown gracefully (something like 24 hours) we haven't used that facility in years. | 17:58 |
| *** Na3iL has quit IRC | 17:58 | |
| pabelanger | Ah, right | 17:58 |
| pabelanger | I forgot about that | 17:58 |
| mgagne | so this means I lost all events | 17:58 |
| *** valderrv has quit IRC | 17:58 | |
| jeblair | mgagne: tell me more about what happened | 17:59 |
| *** ajmiller has joined #openstack-infra | 18:00 | |
| *** valderrv has joined #openstack-infra | 18:00 | |
| mgagne | jeblair: graceful shutdown. Read in Zuul UI that it's in queue only mode (X events). Zuul restarts and somehow, no event is reprocessed. that's my impression, maybe very wrong and some events weren't really triggering any jobs but were just replication events and such. | 18:00 |
| *** Apoorva_ has joined #openstack-infra | 18:00 | |
| AJaeger | team, I've written a short text to announce the constraints changes, reviews and edits are welcome: https://etherpad.openstack.org/p/ldhFWa3PWk | 18:01 |
| mgagne | but I found a change that wasn't processed as expected so I think some legitimate events were lost. | 18:01 |
| *** Apoorva_ has quit IRC | 18:02 | |
| timrc | rcarrillocruz: The problem is harder than that... you want to delete everything associatd with the env you build. | 18:02 |
| *** Apoorva_ has joined #openstack-infra | 18:02 | |
| jeblair | mgagne: are you running with debug logging? if so, is "Saving queue" and "Loading queue" in the log? | 18:02 |
| *** senk_ has joined #openstack-infra | 18:03 | |
| mtreinish | fungi: hopefully the puppet bits we added for adding the tls ports actually work :) | 18:03 |
| timrc | rcarrillocruz: So if you created keys, for example, when you delete the env you'd also want to delete those keys. Extrapolate that to any resource created with the env. | 18:03 |
| mgagne | jeblair: found both messages | 18:03 |
| *** ajmiller_ has quit IRC | 18:03 | |
| fungi | mtreinish: if they don't seem to, i can find time to dig deeper | 18:03 |
| jeblair | mgagne: how about "Queue length is ..."? | 18:03 |
| mgagne | and those ones too: | 18:03 |
| mgagne | DEBUG zuul.Scheduler: Queue length is 19 | 18:03 |
| mgagne | DEBUG zuul.Scheduler: Deleting saved queue | 18:03 |
| *** xnox has quit IRC | 18:03 | |
| mgagne | and Resuming queue processing | 18:03 |
| jeblair | hrm, so it should have put those 19 events into the queue for processing | 18:04 |
| fguillot | I need advices concerning gerrit groups, I submitted a new project in "project-config" and everything have been merged successfully, however I don't seems to have the permissions to add people and myself in $project-core and $project-release, is there any procedure for that? | 18:04 |
| *** Apoorva has quit IRC | 18:04 | |
| openstackgerrit | Merged openstack-infra/project-config: Bring bluebox-sjc1 online to 1/2 capacity https://review.openstack.org/353659 | 18:05 |
| jeblair | mgagne: do you see it processing events after that? | 18:05 |
| mgagne | jeblair: ok, maybe there was some weird Gerrit dependencies which made it so Zuul didn't run jobs as expected | 18:05 |
| *** sneti_ has joined #openstack-infra | 18:05 | |
| mgagne | "Processing trigger event" | 18:05 |
| *** elo has joined #openstack-infra | 18:05 | |
| mgagne | I guess that's it | 18:05 |
| *** kcobb has joined #openstack-infra | 18:05 | |
| *** berendt has quit IRC | 18:05 | |
| jeblair | mgagne: yeah, you should see 19 of those | 18:05 |
| *** Guest27834 has quit IRC | 18:05 | |
| jeblair | (possibly separated by lots of logspam) | 18:06 |
| jeblair | mgagne: maybe you can find the logs relating to processing the event for the change you expected to see | 18:06 |
| mgagne | ok, found all of them | 18:06 |
| pabelanger | mordred: oops, we didn't increase our quota for openstackci | 18:06 |
| AJaeger | fguillot: yes, asking here - see http://docs.openstack.org/infra/manual/creators.html#update-the-gerrit-group-members . Best to link to the review that merged your repo, then an infra-root can set you up. | 18:06 |
| pabelanger | mordred: doing that now | 18:07 |
| AJaeger | fguillot: it's a manual step... | 18:07 |
| mgagne | so now it's just a matter of checking if my expectations were a bit too high regarding the number of jobs that should have run | 18:07 |
| *** sarob has joined #openstack-infra | 18:07 | |
| *** amitgandhinz has quit IRC | 18:07 | |
| mtreinish | fungi: there it is: http://puppetdb.openstack.org/report/firehose01.openstack.org/a7b9793d657d423507a3694ffa3a8d08b9242bc1 | 18:07 |
| mgagne | alright, looks all ok I guess. thanks for the assistance jeblair! | 18:08 |
| *** sarob has quit IRC | 18:08 | |
| jeblair | mgagne: whew! np :) | 18:08 |
| *** amitgandhinz has joined #openstack-infra | 18:08 | |
| *** sarob has joined #openstack-infra | 18:09 | |
| AJaeger | pabelanger: woot, building 4 nodes for bluebox! | 18:09 |
| mtreinish | fungi: although I'm not sure how to test it because I need the ca cert to encrypt use it | 18:09 |
| mtreinish | fungi: also I think we need to restart mosquitto | 18:10 |
| *** PollyZ has joined #openstack-infra | 18:10 | |
| pabelanger | AJaeger: ya, fixing our quota issues | 18:10 |
| openstackgerrit | Merged openstack-infra/devstack-gate: Add sanity check on devstack generated tempest.conf https://review.openstack.org/353060 | 18:10 |
| fguillot | AJaeger: the review is here https://review.openstack.org/#/c/346152/ and the groups are almanach-core and almanach-release, to whom should I ask? | 18:11 |
| fungi | mtreinish: the ca cert should be obtainable from commodo but is also in the chain cert which i put in the ca-file. you just need a copy of that? | 18:11 |
| AJaeger | infra-root, could somebody add fguillot ,please? ^ | 18:11 |
| AJaeger | fguillot: just asking for the team to help normally works - but depends on what people are working on right now. Might take some time until somebody does it... | 18:12 |
| pabelanger | rcarrillocruz: guess we can't do quotas with ansible-role-cloud-launcher yet? | 18:12 |
| fungi | mtreinish: i tried to restart mosquitto just now and it failed. checking logs to see why | 18:12 |
| *** thorst_ has quit IRC | 18:12 | |
| mtreinish | fungi: I think so. I think that's the bare min of what I need to give the client to test the encrypted ports | 18:12 |
| fguillot | AJaeger: ok thanks | 18:12 |
| *** berendt has joined #openstack-infra | 18:12 | |
| *** e0ne has joined #openstack-infra | 18:12 | |
| *** berendt has quit IRC | 18:13 | |
| *** tesseract- has quit IRC | 18:13 | |
| mordred | rcarrillocruz: consider this a feature request for arcl for quota management :) | 18:13 |
| fungi | mtreinish: it doesn't use client certificates or anything like that, right? | 18:13 |
| rcarrillocruz | pabelanger, mordred: iirc GheRivero had some shade changes to set quotas | 18:13 |
| mtreinish | fungi: it can, but that's mostly used for authentication and identification | 18:13 |
| fungi | i was assuming you were just asking for an x.509 cert signed by a third-party ca | 18:13 |
| mgagne | jeblair: and Grafana shows me events being queued and consumed =) | 18:14 |
| mtreinish | I'm pretty sure I didn't set the option to require the client certs on inbound requests on the tls listeners | 18:14 |
| *** senk_ has quit IRC | 18:14 | |
| rcarrillocruz | So we need to check that, and then write corresponding ansible modules | 18:14 |
| pabelanger | mordred: for the moment, I've manually set: core=9999 instances=39 ram=99999999 If you know what our limits actually are, I can update them | 18:14 |
| *** thorst_ has joined #openstack-infra | 18:14 | |
| *** elo has quit IRC | 18:14 | |
| *** Qiming has quit IRC | 18:14 | |
| *** pmalik has joined #openstack-infra | 18:14 | |
| fungi | joy... as before, mosquitto doesn't start and doesn't log anything explaining why (or anything at all) | 18:15 |
| mtreinish | fungi: oh, fun... | 18:15 |
| *** hurgleburgler has quit IRC | 18:15 | |
| mtreinish | fungi: strace away? :) | 18:15 |
| *** hurgleburgler has joined #openstack-infra | 18:15 | |
| fungi | checking that the files we expect have the content we expect first | 18:16 |
| mtreinish | fungi: although the config file diff should be fairly isolated this time, it's just those 2 new listener blocks | 18:16 |
| mordred | pabelanger: I think that sounds fine | 18:18 |
| *** Ravikiran_K has joined #openstack-infra | 18:18 | |
| fungi | mtreinish: running the daemon directly gives me useful errors that systemd doesn't seem to journal and which mosquitto fails to write to its log | 18:20 |
| *** Qiming has joined #openstack-infra | 18:20 | |
| mtreinish | hmm, well at least we got an error this time :) | 18:20 |
| *** elo has joined #openstack-infra | 18:22 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources: Upgrade Laravel Version and ORM Framework https://review.openstack.org/322307 | 18:23 |
| pabelanger | there we go | 18:23 |
| AJaeger | yeah! 2 nodes in use! | 18:23 |
| *** Qiming has quit IRC | 18:23 | |
| pabelanger | had an issue with security groups | 18:24 |
| pabelanger | fixed it | 18:24 |
| AJaeger | anything to update in our config, pabelanger ? | 18:24 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/puppet-mosquitto: Fix typo for websocket_tls_port https://review.openstack.org/353674 | 18:24 |
| fungi | mtreinish: ^ | 18:24 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Refactor layout for clouds_layouts.yaml https://review.openstack.org/353594 | 18:24 |
| mtreinish | fungi: haha, good catch | 18:25 |
| mtreinish | although I kinda like webocket :) | 18:25 |
| pabelanger | AJaeger: https://review.openstack.org/#/c/353602/ hasn't landed yet | 18:25 |
| fungi | mtreinish: apparently referencing an uninitialized/undefined class variable from an erb template doesn't error, and just gives you an empty string? | 18:25 |
| pabelanger | AJaeger: I applied it manually, reran ansible-role-cloud-launcher, fixed the problem | 18:25 |
| fungi | mtreinish: disappointingly shell-like | 18:25 |
| mtreinish | hmm, I didn't realize that either | 18:25 |
| AJaeger | pabelanger: let's ask another infra-root to +2a 353602 and then promote again... | 18:25 |
| AJaeger | pabelanger: is it still montytaylor-sjc.openstack.blueboxgrid.com? | 18:26 |
| pabelanger | yes | 18:26 |
| fungi | AJaeger: pabelanger: approved | 18:26 |
| AJaeger | thanks, fungi | 18:27 |
| *** Swami_ has quit IRC | 18:27 | |
| *** claudiub has joined #openstack-infra | 18:28 | |
| ganesan | I am seeing ssh error on nodepool "http://paste.openstack.org/show/553628/" | 18:28 |
| ganesan | also seeing node is deleted due to building state for long time | 18:29 |
| ganesan | Deleting node id: 201 which has been in building state for 0.0117117494345 hours | 18:29 |
| *** ajmiller_ has joined #openstack-infra | 18:29 | |
| *** Qiming has joined #openstack-infra | 18:29 | |
| pabelanger | ganesan: is your private key valid? Seems to be the SSHException | 18:30 |
| *** valderrv has quit IRC | 18:30 | |
| mtreinish | fungi: is germqtt still going? I'm not seeing any events | 18:31 |
| *** ayoung has quit IRC | 18:31 | |
| *** valderrv has joined #openstack-infra | 18:32 | |
| pabelanger | AJaeger: nc 169.45.71.89 19885 | 18:33 |
| pabelanger | AJaeger: bluebox node | 18:33 |
| ganesan | pabelanger , yes it is valid, I have another working setup, so I copied it from there | 18:33 |
| *** Apoorva_ has quit IRC | 18:33 | |
| *** ajmiller has quit IRC | 18:33 | |
| mordred | pabelanger: woot! | 18:34 |
| *** Apoorva has joined #openstack-infra | 18:34 | |
| mordred | jlk: ^^ you see we're back up and running on the bluebox cloud | 18:34 |
| fungi | mtreinish: it was not, but is now that i've restarted it. perhaps it choked when i killed mosquitto earlier? | 18:34 |
| pabelanger | ganesan: trying manually SSHing into a node using the private key. Nodepool thinks it is bad for some reason | 18:34 |
| mtreinish | fungi: ah there we go, I'm getting evets over both the tls and regular mqtt ports now | 18:34 |
| mordred | pabelanger: so - I have not yet released the shade change which includes the ultimate fix for fip leaking | 18:34 |
| jlk | mordred: I see, is it working? | 18:34 |
| mordred | pabelanger: but this should give us some data on how the new bb cloud is doing with that and then we can see what effect the change has | 18:35 |
| mordred | jlk: yup! | 18:35 |
| pabelanger | mordred: Ya, I haven't checked FIPs yet to see if we are leaking | 18:35 |
| mtreinish | fungi: mosquitto_sub --topic '#' -h firehose.openstack.org --cafile Downloads/\[new\]instantssl_26may.ca-bundle -p 8883 (with that file from https://support.comodo.com/index.php?/Default/Knowledgebase/Article/View/921/75/new-instantssl-ca-bundle ) | 18:35 |
| jlk | *whew* | 18:35 |
| *** shardy is now known as shardy_afk | 18:35 | |
| mordred | jlk: also, pabelanger used rcarrillocruz's ansible-role-cloud-launcher to set up the projects and flavors and stuff | 18:35 |
| fungi | mtreinish: awesome | 18:35 |
| *** tonytan4ever has joined #openstack-infra | 18:35 | |
| mtreinish | I still need to tcpdump it to make sure it's actually encrypted though | 18:36 |
| ganesan | hey, I see node is in building state from the output of "nodepool list" | 18:36 |
| *** valderrv has quit IRC | 18:36 | |
| mordred | jlk: http://git.openstack.org/cgit/openstack-infra/system-config/tree/cloud_launcher/clouds_layouts.yml if you haven't seen the inputs for it | 18:36 |
| *** Qiming has quit IRC | 18:37 | |
| AJaeger | fguillot: if nobody helps, best to write an email to openstack-infra mailing list with the details... Sorry, I cannot help you myself | 18:38 |
| *** valderrv has joined #openstack-infra | 18:38 | |
| *** pvaneck has joined #openstack-infra | 18:39 | |
| *** valderrv has quit IRC | 18:39 | |
| AJaeger | woot, pabelanger ! Success! | 18:39 |
| pabelanger | mordred: so some quick math, the amount of vcpus will be our limit for the number of nodes we can run. Using 8vcpus in our nodes, puts us back at 39 servers. Moving to 4vcpu give us 78 server we could run. With left over RAM and HDD | 18:40 |
| pabelanger | AJaeger: yay | 18:40 |
| *** valderrv has joined #openstack-infra | 18:40 | |
| fguillot | AJaeger: no problem, I understand that people are busy.. | 18:40 |
| *** tonytan4ever has quit IRC | 18:40 | |
| *** Sukhdev has joined #openstack-infra | 18:41 | |
| *** Qiming has joined #openstack-infra | 18:42 | |
| *** sarob has quit IRC | 18:42 | |
| openstackgerrit | Ryan Selden proposed openstack-infra/irc-meetings: Add a UTC clock and integrate local meeting time https://review.openstack.org/353008 | 18:42 |
| *** e0ne has quit IRC | 18:43 | |
| mtreinish | fungi: I think it looks all good. It at least appears to be encyprted when comparing it to the regular port. Also wireshark identifies the traffic on the regular port as mqtt traffic, while it can't identify anything on the tls port | 18:44 |
| *** e0ne has joined #openstack-infra | 18:44 | |
| *** zul has quit IRC | 18:44 | |
| fungi | mtreinish: sounds good | 18:44 |
| fungi | fguillot: taking care of it now | 18:45 |
| pabelanger | AJaeger: lets wait for a few more to pass then we can bump it up another 19 nodes | 18:45 |
| mtreinish | fungi: the only issue I ahve is if you don't specify the ca cert mosquitto_sub just doesn't print anything. There aren't an errors or anything (which I guess is pretty typical in our experience) | 18:46 |
| AJaeger | EmilienM: could you review https://review.openstack.org/#/c/353662/ , please? Will that repo be part of your team? | 18:46 |
| fungi | fguillot: done | 18:46 |
| *** rlandy|afk is now known as rlandy | 18:47 | |
| fungi | mtreinish: yeah, seems error handling was a bit of an afterthought in that community | 18:47 |
| EmilienM | nice ! | 18:47 |
| EmilienM | AJaeger: thanks ! | 18:47 |
| *** xarses has joined #openstack-infra | 18:48 | |
| pabelanger | mordred: jlk: do we have IPv6 for bluebox-sjc1? | 18:48 |
| jlk | You... I think you can have ipv6 internally, but not plumbled all the way in | 18:48 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Increase bluebox-sjc1 to 39 nodes https://review.openstack.org/353686 | 18:48 |
| pabelanger | jlk: ack | 18:48 |
| jlk | softlayer problem | 18:48 |
| *** e0ne_ has joined #openstack-infra | 18:49 | |
| EmilienM | AJaeger: done | 18:49 |
| *** e0ne has quit IRC | 18:49 | |
| fguillot | fungi: thanks! | 18:49 |
| mtreinish | nibalizer, crinkle: any hints as to what: http://logs.openstack.org/44/351344/3/check/gate-puppet-logstash-puppet-lint/93b80e3/console.html#_2016-08-10_00_32_39_517222 means on: https://review.openstack.org/#/c/351344/ | 18:49 |
| *** spzala_ has quit IRC | 18:49 | |
| *** zul has joined #openstack-infra | 18:49 | |
| *** spzala has joined #openstack-infra | 18:50 | |
| mtreinish | oh, is it just catching my typo in a roundabout way? | 18:50 |
| crinkle | mtreinish: yes | 18:50 |
| mtreinish | crinkle: ok, thanks | 18:50 |
| *** xnox has joined #openstack-infra | 18:50 | |
| *** inc0 has quit IRC | 18:51 | |
| *** tbarron|lunch is now known as tbarron | 18:52 | |
| *** Swanson has joined #openstack-infra | 18:55 | |
| *** ganesan has quit IRC | 18:55 | |
| *** oanson has joined #openstack-infra | 18:55 | |
| *** cryptarium has joined #openstack-infra | 18:56 | |
| *** cryptarium has quit IRC | 18:56 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-logstash: Add mqtt output plugin to logstash indexers https://review.openstack.org/351344 | 18:56 |
| mtreinish | fungi, clarkb, rcarrillocruz, jesusaur: ^^^ I think that should be right | 18:56 |
| *** cryptarium has joined #openstack-infra | 18:57 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Set up HTTPS for releases.openstack.org site https://review.openstack.org/353692 | 18:59 |
| fungi | dhellmann: ^ | 18:59 |
| dhellmann | fungi : nice | 18:59 |
| fungi | dhellmann: i plan to be working on the launchpadlib stuff for the signing node later today | 19:00 |
| *** sshnaidm|afk has quit IRC | 19:00 | |
| *** e0ne_ has quit IRC | 19:00 | |
| dhellmann | fungi : good news, thanks | 19:00 |
| openstackgerrit | Pradeep Kilambi proposed openstack-infra/project-config: Create puppet-panko repository https://review.openstack.org/353662 | 19:00 |
| *** cryptarium has quit IRC | 19:01 | |
| dhellmann | fungi : I'm still reading this patch, but do we want to redirect from http to https once the cert is in place? | 19:01 |
| *** xnox has quit IRC | 19:02 | |
| dhellmann | indeed, I see that in here now | 19:02 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Set up HTTPS for releases.openstack.org site https://review.openstack.org/353692 | 19:02 |
| *** fifieldt has quit IRC | 19:02 | |
| fungi | dhellmann: that was going to be my question for you actually. do you want http and https working, or http permanently redirecting to https? i did the latter but can revise easily enough | 19:03 |
| *** annegentle_ has joined #openstack-infra | 19:03 | |
| dhellmann | fungi : I think we want the redirect. We might want to wait until we can consult with ttx, though. | 19:03 |
| fungi | given we will be serving some security-relevant details (checksums, key fingerprints) from there, it seemed wise to not allow people to trust an http version of teh content | 19:04 |
| zigo | AJaeger: dhellmann: git-os-job just reached Debian! :) | 19:04 |
| zigo | (FTP master approved) | 19:04 |
| dhellmann | zigo : nice | 19:04 |
| zigo | dhellmann: Next time you write something nice like that, let me know, I discovered it 2 days ago... :P | 19:05 |
| *** tonytan4ever has joined #openstack-infra | 19:05 | |
| annegentle_ | hehehe we all hero-worship dhellman :) | 19:05 |
| zigo | It could have been included in Xenial... | 19:05 |
| dhellmann | fungi : right, I agree. maybe we can go ahead with it and if ttx presents a reason to not redirect we can remove that | 19:05 |
| *** xarses has quit IRC | 19:05 | |
| jeblair | zigo: have you seen git-restack? http://docs.openstack.org/infra/git-restack/ | 19:05 |
| dhellmann | zigo : I never know what anyone is going to find useful. People still use a bunch of hacky shell scripts I wrote a zillion years ago to manage their python virtualenvs... | 19:06 |
| openstackgerrit | Merged openstack-infra/project-config: Properly set up UPPER_CONSTRAINTS_FILE for translations https://review.openstack.org/353418 | 19:06 |
| annegentle_ | AJaeger: or anyone familiar with project-config, where can I find the folder name/ service name for api-ref-jobs target? | 19:06 |
| zigo | jeblair: Do you want it in Debian / Ubuntu as well? | 19:06 |
| pabelanger | zigo: looks like you have an issue with your build script: http://logs.openstack.org/5e/5ebe5fd9380356b0bcb1bf6318bbe8bbfea4ea06/post/reprepro-import-debian-openstack-jessie-backports/781b1d6//console.html#_2016-08-10_19_05_39_526233 | 19:06 |
| *** zul has quit IRC | 19:06 | |
| annegentle_ | I can't figure it out for manila, sahara, searchlight specifically, do they have publish jobs for api-ref and if so where is the target? | 19:06 |
| jeblair | zigo: it has a similar potential audience as git-review. i think it's generally useful enough. i use it many times per day. | 19:06 |
| jeblair | zigo: (it's also forked from git-review, so already has the same infrastructure build-out as far as docs/packaging/etc) | 19:07 |
| pabelanger | zigo: looks like you are increasing your build number | 19:07 |
| zigo | Ok, I'll try to find time to do it too. | 19:07 |
| jeblair | zigo: thanks :) | 19:08 |
| *** raunak has quit IRC | 19:08 | |
| *** oanson has quit IRC | 19:10 | |
| *** valderrv has quit IRC | 19:12 | |
| *** asettle has quit IRC | 19:12 | |
| *** rbuzatu has joined #openstack-infra | 19:12 | |
| *** valderrv has joined #openstack-infra | 19:13 | |
| *** woodster_ has joined #openstack-infra | 19:13 | |
| *** baoli_ has joined #openstack-infra | 19:14 | |
| AJaeger | annegentle_: in jenkins/jobs/projects.yaml | 19:14 |
| annegentle_ | AJaeger: ok I'm in there... what am I looking for, service: | 19:14 |
| AJaeger | annegentle_: service: shared-file-systems | 19:14 |
| AJaeger | annegentle_: that's manila ^ | 19:15 |
| AJaeger | annegentle_: just search for api-ref-jobs in jenkins/jobs/projects.yaml | 19:15 |
| AJaeger | sahara has data-processing and searchlight uses search | 19:16 |
| *** rbuzatu has quit IRC | 19:17 | |
| *** acoles is now known as acoles_ | 19:18 | |
| *** fifieldt has joined #openstack-infra | 19:18 | |
| AJaeger | pabelanger: do you want to ask for another +2a for the bluebox increase change and promote again? https://review.openstack.org/353686 - or wait a bit longer? | 19:21 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Reduce name for python-tripleoclient buildimages https://review.openstack.org/353700 | 19:21 |
| pabelanger | AJaeger: lets check logstash | 19:21 |
| openstackgerrit | Pradeep Kilambi proposed openstack-infra/project-config: Create puppet-panko repository https://review.openstack.org/353662 | 19:21 |
| pabelanger | see if there are any failures | 19:21 |
| pabelanger | AJaeger: A quick sampling show we appear to be okay | 19:24 |
| AJaeger | great! | 19:24 |
| *** elo has quit IRC | 19:24 | |
| annegentle_ | AJaeger: thanks! that's it! | 19:24 |
| *** rbuzatu has joined #openstack-infra | 19:25 | |
| AJaeger | project-config cores, could you review pabelanger's change https://review.openstack.org/353686 to add 19 more VMs, please? | 19:25 |
| openstackgerrit | Sean Dague proposed openstack-infra/project-config: test python-novaclient master changes against a stable/mitaka https://review.openstack.org/353703 | 19:27 |
| *** tphummel has quit IRC | 19:27 | |
| sdague | AJaeger: ... does that look remotely correct ^^ | 19:27 |
| AJaeger | looking... | 19:28 |
| *** roxanaghe has quit IRC | 19:29 | |
| *** senk_ has joined #openstack-infra | 19:29 | |
| openstackgerrit | devdatta-kulkarni proposed openstack-infra/project-config: Move to Ubuntu-trusty for Libvirt/LXC test https://review.openstack.org/353640 | 19:30 |
| AJaeger | sdague: not really sure, worth a try... | 19:30 |
| *** zul has joined #openstack-infra | 19:30 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-logstash: Add mqtt output plugin to logstash indexers https://review.openstack.org/351344 | 19:33 |
| AJaeger | team, I've written a short text to announce the constraints changes, reviews and edits are welcome: https://etherpad.openstack.org/p/ldhFWa3PWk | 19:33 |
| * AJaeger waves good night, will send tomorrow ^ | 19:33 | |
| *** roxanaghe has joined #openstack-infra | 19:34 | |
| *** mtanino has quit IRC | 19:34 | |
| *** piet has quit IRC | 19:35 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config: Add mqtt support to logstash workers https://review.openstack.org/353706 | 19:39 |
| *** hurgleburgler1 has joined #openstack-infra | 19:42 | |
| *** hurgleburgler has quit IRC | 19:42 | |
| sdague | AJaeger: damn, I was hoping for a +2 first :) | 19:42 |
| *** mrhillsman_away is now known as mrhillsman | 19:43 | |
| *** ayoung has joined #openstack-infra | 19:43 | |
| openstackgerrit | Sean Dague proposed openstack-infra/project-config: test python-novaclient master changes against a stable/mitaka https://review.openstack.org/353703 | 19:45 |
| *** xnox has joined #openstack-infra | 19:46 | |
| *** amitgandhinz has quit IRC | 19:47 | |
| openstackgerrit | Merged openstack-infra/project-config: Increase bluebox-sjc1 to 39 nodes https://review.openstack.org/353686 | 19:47 |
| *** amitgandhinz has joined #openstack-infra | 19:48 | |
| *** roxanaghe has quit IRC | 19:49 | |
| *** Na3iL has joined #openstack-infra | 19:50 | |
| *** xnox has quit IRC | 19:52 | |
| *** claudiub has quit IRC | 19:53 | |
| *** andymaier has joined #openstack-infra | 19:56 | |
| *** roxanaghe has joined #openstack-infra | 19:56 | |
| *** amitgandhinz has quit IRC | 19:59 | |
| *** xarses has joined #openstack-infra | 19:59 | |
| *** amitgandhinz has joined #openstack-infra | 19:59 | |
| *** d0ugal has joined #openstack-infra | 20:01 | |
| *** thorst_ has quit IRC | 20:02 | |
| *** senk_ has quit IRC | 20:02 | |
| *** sshnaidm|afk has joined #openstack-infra | 20:02 | |
| *** roxanaghe has quit IRC | 20:04 | |
| mtreinish | umm, I just pushed up a new tag for subunit2sql and the only job I see triggered is the release notes one | 20:06 |
| *** roxanaghe has joined #openstack-infra | 20:07 | |
| *** raunak has joined #openstack-infra | 20:07 | |
| *** thorst_ has joined #openstack-infra | 20:08 | |
| *** thorst_ has quit IRC | 20:08 | |
| *** thorst_ has joined #openstack-infra | 20:08 | |
| mtreinish | oh, nm I was looking at the tag pipeline, I missed the release one because there is so much going on right now | 20:09 |
| pabelanger | jlk: mordred: looking at the Hypervisor Summary for montytaylor-sjc we only have 63vCPUs. Is there a reason for the change? It looks like before we had 64vCPUs | 20:09 |
| *** asettle has joined #openstack-infra | 20:09 | |
| jlk | Now I have to page all that info back into my head. | 20:10 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Update category mapping in tests https://review.openstack.org/352636 | 20:11 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Fix addFailTest https://review.openstack.org/352637 | 20:11 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Re-enable test_connection tests https://review.openstack.org/351949 | 20:11 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Stop sharing Gerrit event queues in tests https://review.openstack.org/352635 | 20:11 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Rename test_connection to test_connections https://review.openstack.org/353712 | 20:11 |
| *** _ari_ has quit IRC | 20:11 | |
| jlk | pabelanger: the HV itself has 64 CPUs, and I believe we reserve 1 for the nova services on the compute node | 20:11 |
| pabelanger | jlk: ya, that is what I was thinking too | 20:12 |
| jlk | I think that's a change from your previous release to the current release. | 20:12 |
| pabelanger | Seems to be | 20:13 |
| jlk | it gets expressed as a `vcpu_pin_set` | 20:13 |
| jesusaur | mtreinish: is there a way to tell if the logstash plugin install has happened? currently that change will run the install on every puppet run | 20:13 |
| jlk | I _thought_ that didn't impact the HV summary, and it just wound up double stacking one of the CPUs, but... | 20:13 |
| jlk | it's nova, so I'm not 100% certain on that | 20:13 |
| pabelanger | okay, I'll have to update our max-servers to 35 since we can only do 7 instances | 20:14 |
| jlk | but that's matching other clusters of that release. the CPU count - 1 | 20:14 |
| jlk | We _can_ do CPU overcommit if you want | 20:14 |
| jlk | the default is 1.0 ratio | 20:15 |
| mtreinish | jesusaur: this maybe: https://www.elastic.co/guide/en/logstash/current/working-with-plugins.html#listing-plugins | 20:15 |
| jlk | We can do more. | 20:15 |
| pabelanger | mordred: jeblair: ^ thoughts? we only have 63vcpus in bluebox today. Which is a drop to 35 from 39 max-servers | 20:15 |
| pabelanger | jlk: lets see what others say | 20:15 |
| *** rbrndt has quit IRC | 20:15 | |
| *** rwsu has joined #openstack-infra | 20:15 | |
| *** rbrndt has joined #openstack-infra | 20:16 | |
| *** dprince has quit IRC | 20:16 | |
| *** elo has joined #openstack-infra | 20:16 | |
| jesusaur | mtreinish: yeah, if it has sane exit codes, we can put a plugin list command in an "unless =>" in the puppet to only run if the module isn't listed | 20:18 |
| openstackgerrit | Timothy R. Chavez proposed openstack-infra/jenkins-job-builder: Add support for the random string parameter https://review.openstack.org/351384 | 20:19 |
| mtreinish | jesusaur: I have no way to test the exit code. I don't have a logstash anywhere | 20:19 |
| *** kzaitsev_mb has joined #openstack-infra | 20:20 | |
| *** tphummel has joined #openstack-infra | 20:20 | |
| pabelanger | jlk: I think if we overcommit just enough for 64, I'd say lets do it | 20:20 |
| *** rwsu has quit IRC | 20:20 | |
| mtreinish | jesusaur: is your concern that we say enable_mqtt => true and the gem install fails but we configure everything anyway? | 20:21 |
| *** mhickey has joined #openstack-infra | 20:21 | |
| jesusaur | mtreinish: no, my concern is that installing an already installed plugin would give an error | 20:22 |
| mtreinish | ah, ok | 20:22 |
| *** inc0 has joined #openstack-infra | 20:24 | |
| mtreinish | jesusaur: if you have a logstash somewhere, we check the list format and then just grep for the plugin in there | 20:24 |
| pabelanger | jlk: quick math says a ratio of 1.0159 | 20:26 |
| jesusaur | mtreinish: yeah, I'm doing that now. It looks like it does exit with an error code if the module is not found, but it's also not finding 'logstash-output-mqtt' as an available plugin | 20:26 |
| *** claudiub has joined #openstack-infra | 20:27 | |
| mtreinish | jesusaur: that might be related to: https://github.com/elastic/logstash/issues/4487 | 20:27 |
| jeblair | pabelanger, jlk: we might be able to handle a little bit of (actual) overcommit, but based on our experience with other providers, probably not much. | 20:27 |
| sdague | proposal bot pushes a new change even if there is no content - https://review.openstack.org/#/c/351991/ ? | 20:27 |
| jeblair | (it's unfortunate that pinning that single vcpu causes us to loose 7 more) | 20:28 |
| jeblair | er, lose. not loose. in fact, not being able to loose them is the problem. ;) | 20:28 |
| pabelanger | ya | 20:28 |
| sdague | the global requirements job seems to be pushing rebased proposal bot changes on every g-r merge even if there are no changes to the project patch, which I'm sure is not helping our 538 jobs in the check queue | 20:29 |
| jesusaur | mtreinish: ah, yeah, that would do it | 20:29 |
| *** julim has quit IRC | 20:30 | |
| *** shardy_afk has quit IRC | 20:31 | |
| *** sarob has joined #openstack-infra | 20:31 | |
| *** coolsvap has quit IRC | 20:31 | |
| *** karthikp_ has quit IRC | 20:31 | |
| *** karthikp_ has joined #openstack-infra | 20:32 | |
| *** cody-somerville has joined #openstack-infra | 20:32 | |
| *** elo has quit IRC | 20:33 | |
| *** roxanaghe has quit IRC | 20:33 | |
| *** Jeffrey4l has quit IRC | 20:34 | |
| openstackgerrit | Sean Dague proposed openstack-infra/project-config: test python-novaclient master changes against a stable/mitaka https://review.openstack.org/353703 | 20:35 |
| *** _nadya_ has joined #openstack-infra | 20:35 | |
| *** Jeffrey4l has joined #openstack-infra | 20:35 | |
| mtreinish | AJaeger: we should be good to go on: https://review.openstack.org/#/c/333461/1 the new subunit2sql release is up now: https://pypi.python.org/pypi/subunit2sql/1.7.0 | 20:36 |
| *** elo has joined #openstack-infra | 20:36 | |
| *** rwsu has joined #openstack-infra | 20:37 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Use updated overcloud-full image name https://review.openstack.org/353729 | 20:37 |
| *** roxanaghe has joined #openstack-infra | 20:38 | |
| *** elo has quit IRC | 20:40 | |
| openstackgerrit | James Slagle proposed openstack-infra/project-config: Make Tripleo nonha-multinode and undercloud jobs voting https://review.openstack.org/353019 | 20:41 |
| sdague | hmmm... we just got a 3rd proposal bot requirements patch with no content changes this hour on the nova repo | 20:41 |
| sdague | is proposal bot drunk? | 20:41 |
| jeblair | or not drunk enough? | 20:42 |
| jeblair | we should have the bot link to the run which created a proposal in a review comment | 20:42 |
| jesusaur | mtreinish: it also looks like 'plugin list' lists all available plugins by default, 'plugin list --installed $pluginname' does what we want | 20:42 |
| mtreinish | jesusaur: ah, ok. That's not what the docs say, but I could see that | 20:43 |
| jesusaur | yeah, the --help output isn't quite in line with the docs, though maybe that's a logstash version issue | 20:44 |
| zaro | fungi: it looks like i'm going to need more time to test online reindex. can we still go ahead with the gerrit upgrade this friday? we don't have to use online reindex even if we upgrade. | 20:44 |
| *** elo has joined #openstack-infra | 20:44 | |
| sdague | jeblair: seems like they are running through post right now | 20:45 |
| jesusaur | mtreinish: fyi im testing with 2.0.0 | 20:45 |
| sdague | I'm just really confused why it's pushing new content to nova | 20:45 |
| mtreinish | jesusaur: is that the same as what we have deployed? (I honestly don't know) | 20:46 |
| sdague | because every time it does, it's going to restart 20 some odd jobs | 20:46 |
| jeblair | sdague: i'm looking at http://logs.openstack.org/b3/b32777a1553d008f1f3940d831cdaf5d9be564ee/post/propose-requirements-updates/debc1bb/console.html | 20:46 |
| *** roxanaghe has quit IRC | 20:46 | |
| sdague | jeblair: ok, that's a pretty old one | 20:46 |
| sdague | we're up to rev 12 on that | 20:47 |
| jesusaur | mtreinish: I'm pretty sure it is, but not 100% | 20:47 |
| sdague | and that's rev 7 | 20:47 |
| jeblair | sdague: yeah, it looks like that was at 7 pushing 8? | 20:47 |
| sdague | yeh | 20:47 |
| *** roxanaghe has joined #openstack-infra | 20:47 | |
| sdague | although 7 -> 8 is the same issue | 20:47 |
| *** shardy_afk has joined #openstack-infra | 20:48 | |
| sdague | as it's just a rebase | 20:48 |
| jeblair | yep | 20:48 |
| *** rwsu has quit IRC | 20:48 | |
| *** mhickey has quit IRC | 20:51 | |
| *** sarob has quit IRC | 20:52 | |
| *** sarob has joined #openstack-infra | 20:52 | |
| jeblair | sdague: i don't see anything to bail out if there are no changes. there is a check_already_approved function which causes us to avoid pushing a new version if the change has been approved, but that's only used by translation updates, not requirements updates. | 20:55 |
| *** PollyZ has quit IRC | 20:55 | |
| *** valderrv has quit IRC | 20:55 | |
| *** elo has quit IRC | 20:55 | |
| openstackgerrit | Merged openstack-infra/project-config: Use --sbuild-chroot-name jessie-newton-backports https://review.openstack.org/353489 | 20:55 |
| sdague | yeh, maybe it's always been like this, but the requirements group was slow enough, that we didn't see as much churn here | 20:55 |
| sdague | now that the team is more active, it's causing more load | 20:55 |
| *** roxanaghe has quit IRC | 20:56 | |
| jeblair | yep. it's probably worth making that improvement. | 20:56 |
| *** baoli_ has quit IRC | 20:56 | |
| *** nwkarsten has quit IRC | 20:56 | |
| *** nwkarsten has joined #openstack-infra | 20:57 | |
| *** shardy_afk has quit IRC | 20:58 | |
| jeblair | yolanda: i had to remove your email address from the root list in hiera because your mail server is rejecting mail which is causing backscatter for the rest of us on the list. | 20:59 |
| jeblair | infra-root: cc ^ | 21:00 |
| *** nwkarste_ has joined #openstack-infra | 21:00 | |
| sdague | jeblair: oh wait, didn't requirements updates only come in nightly before? | 21:00 |
| pabelanger | jeblair: I'm not sure I am on that list, how do I check? | 21:00 |
| jeblair | pabelanger: common.yaml in secret puppet hiera. | 21:00 |
| *** rbuzatu has quit IRC | 21:00 | |
| pabelanger | jeblair: thank you | 21:00 |
| jeblair | pabelanger: root email addresses are a "secret" because otherwise, people run our puppet and we get their cronspam. :( | 21:01 |
| *** acoles_ is now known as acoles | 21:01 | |
| * pabelanger nods | 21:01 | |
| jeblair | it's "funny". | 21:01 |
| jeblair | sdague: hrm, my memory is fuzzy, but i want to say we've been doing both nightly and upon-merge for a while.. | 21:01 |
| *** nwkarsten has quit IRC | 21:01 | |
| *** sarob has quit IRC | 21:01 | |
| sdague | yeh, that's what project-config history seems to say | 21:01 |
| jeblair | (on to catch external changes, the other for internal ones) | 21:01 |
| *** sarob has joined #openstack-infra | 21:01 | |
| pabelanger | jeblair: added. thanks | 21:02 |
| jeblair | pabelanger: if your address is handled by google you might have the same problem | 21:02 |
| *** jkilpatr has quit IRC | 21:03 | |
| *** roxanaghe has joined #openstack-infra | 21:03 | |
| *** Apoorva_ has joined #openstack-infra | 21:04 | |
| pabelanger | jeblair: used my corp address, if an issue, I can drop it | 21:04 |
| *** PollyZ has joined #openstack-infra | 21:04 | |
| *** elo has joined #openstack-infra | 21:05 | |
| *** Apoorva has quit IRC | 21:07 | |
| *** _nadya_ has quit IRC | 21:10 | |
| jlk | jeblair: pabelanger: what we recommend is no more than an overcommit of 2x | 21:11 |
| jlk | unlike some public cloud vendors, who run like 16x | 21:11 |
| *** elo has quit IRC | 21:13 | |
| pabelanger | jlk: Ya, what ever gets us to 64vcpus is enough | 21:14 |
| *** ldnunes has quit IRC | 21:14 | |
| *** sarob has quit IRC | 21:14 | |
| *** HeOS has quit IRC | 21:16 | |
| karthikp_ | AJaeger: Hi | 21:16 |
| *** sarob has joined #openstack-infra | 21:18 | |
| *** sarob has quit IRC | 21:18 | |
| *** tpsilva has quit IRC | 21:18 | |
| jlk | I believe nibalizer has boxpanel access to create a ticket requesting 2x over commit. | 21:20 |
| *** amitgandhinz has quit IRC | 21:22 | |
| *** amitgandhinz has joined #openstack-infra | 21:22 | |
| *** claudiub has quit IRC | 21:28 | |
| *** amitgandhinz has quit IRC | 21:28 | |
| *** inc0 has quit IRC | 21:30 | |
| *** nwkarste_ has quit IRC | 21:30 | |
| *** nwkarsten has joined #openstack-infra | 21:31 | |
| *** nwkarsten has quit IRC | 21:35 | |
| rcarrillocruz | pabelanger: i'm seeing issues on ironic installation on the baremetal | 21:36 |
| rcarrillocruz | http://paste.openstack.org/show/553673/ | 21:36 |
| rcarrillocruz | not sure what is, but we have to fix this , we can't provision things now | 21:37 |
| *** Ravikiran_K has quit IRC | 21:38 | |
| *** aeng has joined #openstack-infra | 21:40 | |
| *** shardy has joined #openstack-infra | 21:40 | |
| fungi | zaro: yeah, i can send an announcement now for a gerrit restart onto the updated version. how's 20:00 utc friday sound? | 21:40 |
| *** _nadya_ has joined #openstack-infra | 21:40 | |
| fungi | zaro: also, does this include the its-storyboard plugin? Zara was asking in the storyboard meeting earlier today and i was unsure | 21:40 |
| odyssey4me | any infra roots available to complete the process to get this finalised? https://review.openstack.org/307831 | 21:42 |
| *** inc0 has joined #openstack-infra | 21:42 | |
| *** gildub has joined #openstack-infra | 21:43 | |
| *** burgerk has quit IRC | 21:44 | |
| *** nwkarsten has joined #openstack-infra | 21:44 | |
| *** jheroux has quit IRC | 21:45 | |
| *** _nadya_ has quit IRC | 21:45 | |
| *** esberglu has quit IRC | 21:48 | |
| *** nwkarsten has quit IRC | 21:49 | |
| *** jcoufal has quit IRC | 21:49 | |
| *** edtubill has quit IRC | 21:49 | |
| ianw | odyssey4me / pabelanger : i have zero context on this, but could make some time today | 21:50 |
| nibalizer | jlk: yes i have 'panel' access | 21:50 |
| nibalizer | really busy right now, but let me know what we need and ill press the right buttons | 21:50 |
| jlk | nibalizer: a ticket opened requesting 2x CPU overcommit. | 21:51 |
| *** annegentle_ has quit IRC | 21:52 | |
| nibalizer | ok | 21:52 |
| mordred | nibalizer: ossum. you have the boxpanel for the account now? that's happy making | 21:53 |
| nibalizer | i do | 21:53 |
| nibalizer | proudman ftw | 21:54 |
| nibalizer | iirc we are supposed to press the upgrade button or something | 21:54 |
| *** priteau has quit IRC | 21:54 | |
| nibalizer | but, tomorrow, today is alll *waves hands wildiy* | 21:54 |
| odyssey4me | ianw thanks - it's just the addition of another mirror, so I believe that other than approving the patch, there's some admin work on AFS to do - pabelanger should be able to fill you in | 21:55 |
| *** inc0 has quit IRC | 21:56 | |
| zaro | fungi: 20:00 utc sounds fine. | 21:56 |
| zaro | fungi: we'll need to merge this for its-storyboard https://review.openstack.org/#/c/347486/ | 21:57 |
| zaro | fungi: note the commit message | 21:57 |
| fungi | thanks, reviewing now | 21:57 |
| pabelanger | rcarrillocruz: ack | 21:57 |
| *** tonytan4ever has quit IRC | 21:58 | |
| fungi | zaro: oh, that's pretty simple. we can debate whether to do per-project or all once we've restarted | 21:58 |
| openstackgerrit | Merged openstack-infra/zuul: Ansible launcher: retry publisher sync tasks https://review.openstack.org/353553 | 21:58 |
| zaro | fungi: i believe zara has already tested on review-dev. | 21:58 |
| *** inc0 has joined #openstack-infra | 21:59 | |
| zaro | fungi: yeah, i guess up to you. but i've enabled for all-projects on review-dev | 21:59 |
| karthikp_ | Hi All, I have a question. what exactly is the role of primary and sub node in case of devstack-gate project? | 21:59 |
| *** vhosakot has quit IRC | 21:59 | |
| pabelanger | ianw: I won't be around tonight, but http://docs.openstack.org/infra/system-config/afs.html is the steps needed. You'll need to create yourself a User Principal first | 21:59 |
| ianw | pabelanger / odssey4me : ok, i'll take a look in about a hour or so, just gotta wrap some things up | 22:00 |
| *** gordc has quit IRC | 22:00 | |
| *** gouthamr has quit IRC | 22:00 | |
| fungi | karthikp_: for single-node jobs, not much. for multi-node jobs, the "primary" node is the one where devstack-gate is run and it needs ssh access into all defined subnodes to set up devstack on them | 22:01 |
| *** vhosakot has joined #openstack-infra | 22:01 | |
| zaro | fungi: might want to take a look to see if you want to add this plugin as well? https://review.openstack.org/#/c/347033/ | 22:02 |
| *** gouthamr has joined #openstack-infra | 22:02 | |
| fungi | zaro: that probably needs wider discussion. we don't really have "deprecated" repos, but we do have "retired" repos, and sometimes they come back out of retirement | 22:04 |
| *** kong has joined #openstack-infra | 22:04 | |
| zaro | fungi: ok, np. the change to put it on review-dev already merged. so it's there if you want to play with it | 22:04 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add specs dir https://review.openstack.org/336109 | 22:05 |
| zaro | fungi: wanted to see if you have suggestions on how to test online reindex. | 22:05 |
| *** mriedem has quit IRC | 22:05 | |
| fungi | zaro: you tried importing nova, i guess. how did that go? | 22:05 |
| *** kzaitsev_mb has quit IRC | 22:06 | |
| zaro | i attempted to import the nova repo from review.o.o using the gerrit importer plugin but it doesn't work all that well. looks like there's some bugs in it that keep causing the import to stop | 22:06 |
| *** xyang1 has quit IRC | 22:06 | |
| zaro | changes are tied to user accounts and when a user account doesn't exist for a change that is being imported then the plugin attempts to create an account for each user it encounters. | 22:07 |
| *** shardy has quit IRC | 22:07 | |
| zaro | the auto account creation piece seems to have lots of issues. | 22:07 |
| zaro | i event attempt to manually add users but import still fails. | 22:08 |
| *** nherciu has quit IRC | 22:09 | |
| zaro | i'm not sure how else to import a single project into review-dev with the plugin. | 22:09 |
| *** elo has joined #openstack-infra | 22:09 | |
| karthikp_ | fungi: what exactly are the sub nodes required for? | 22:09 |
| zaro | so i guess we have 2 options. we can do all or just do all on a seperate gerrit test instance | 22:09 |
| fungi | karthikp_: multi-node testing | 22:09 |
| zaro | fungi: or i guess the 3rd option is to fix stuff with import plugin | 22:10 |
| fungi | zaro: perhaps you could script up some api calls in a loop to submit a bunch of changes into the gtest repo on git-review | 22:10 |
| *** kzaitsev_mb has joined #openstack-infra | 22:10 | |
| fungi | just commit garbage/noise so there's a few thousand or so changes to make sure reindexing will take some minutes | 22:11 |
| zaro | fungi: that's an option but my concern with that is it's not prod like data | 22:11 |
| *** yamamoto has joined #openstack-infra | 22:11 | |
| *** elmiko is now known as _elmiko | 22:12 | |
| fungi | zaro: i don't know how prod-like it needs to be, as long as we've got a scenario that will slow down reindexing enough to give us time to exercise review-dev while it's running, that's probably good enough? | 22:12 |
| karthikp_ | fungi: Does it do multi-node testing for upgrade? Does it install base, target in primary , sub-node respectively? sorry I am pretty new | 22:13 |
| karthikp_ | fungi: base,primary from grenade testing environment | 22:14 |
| *** mdrabe has quit IRC | 22:14 | |
| fungi | karthikp_: i don't know what the scenario is for multi-node grenade jobs (if there are any). i know we have jobs that do things like test nova live migration between multiple hosts, or neutron distributed virtual routers with >1x distribution | 22:15 |
| fungi | karthikp_: questions about the sorts of testing we do are more likely to get good answers in #openstack-qa | 22:16 |
| zaro | fungi: ok. as long as your comfortable with that, then it sounds fine to me. | 22:16 |
| *** kzaitsev_mb has quit IRC | 22:17 | |
| openstackgerrit | Merged openstack-infra/system-config: Add support for logstash-filter integration tests https://review.openstack.org/320728 | 22:17 |
| fungi | zaro: yeah, i don't want to make this unnecessarily complicated, but it would be good to at least have some cursory testing before we try to run it in production | 22:18 |
| *** karthikp_ has quit IRC | 22:18 | |
| * craige awoke to find that last night's tests failed with the last committ yesterday jhesketh :-) | 22:20 | |
| *** yamamoto has quit IRC | 22:20 | |
| * craige will sort it when he gets to Melbourne this afternoon (PyConAu \o/ ) | 22:20 | |
| *** eric_lopez has joined #openstack-infra | 22:20 | |
| *** elo has quit IRC | 22:21 | |
| *** kzaitsev_mb has joined #openstack-infra | 22:22 | |
| *** eharney has quit IRC | 22:23 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Remove query for bug 1526675 https://review.openstack.org/353619 | 22:23 |
| *** vhosakot has quit IRC | 22:23 | |
| openstack | bug 1526675 in Glance liberty "test_models_sync fails with 'Models and migration scripts aren't in sync'" [Critical,Fix committed] https://launchpad.net/bugs/1526675 - Assigned to Flavio Percoco (flaper87) | 22:23 |
| *** karthikp_ has joined #openstack-infra | 22:25 | |
| *** inc0 has quit IRC | 22:25 | |
| *** seldenr has quit IRC | 22:27 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Tests: update references from worker to launch_server https://review.openstack.org/353768 | 22:27 |
| *** acoles is now known as acoles_ | 22:28 | |
| *** amitgandhinz has joined #openstack-infra | 22:29 | |
| *** kzaitsev_mb has quit IRC | 22:29 | |
| *** IlyaG has joined #openstack-infra | 22:30 | |
| *** yamamoto has joined #openstack-infra | 22:31 | |
| *** dimtruck is now known as zz_dimtruck | 22:31 | |
| *** thorst_ has quit IRC | 22:32 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Tests: add an extra debug line when stalled https://review.openstack.org/353769 | 22:33 |
| *** yamamoto has quit IRC | 22:33 | |
| *** amitgandhinz has quit IRC | 22:33 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add grenade to the integrated gate projects list https://review.openstack.org/353628 | 22:34 |
| jhesketh | Morning | 22:35 |
| *** edmondsw has quit IRC | 22:36 | |
| *** Julien-zte has joined #openstack-infra | 22:37 | |
| *** yamahata has joined #openstack-infra | 22:37 | |
| karthikp_ | fungi: thanks. I just want to know how primary, sub are different from base & target of the grenade in a regular setup | 22:38 |
| *** kdas__ has joined #openstack-infra | 22:38 | |
| *** sdake has quit IRC | 22:38 | |
| *** kushal has quit IRC | 22:39 | |
| *** kzaitsev_mb has joined #openstack-infra | 22:39 | |
| fungi | karthikp_: base and target in grenade are basically what version of openstack you're upgrading from and what version you're upgrading to | 22:39 |
| fungi | karthikp_: primary node and subnodes in devstack-gate are the node which runs devstack-gate vs the additional devstack nodes (if any) that are controlled by it | 22:40 |
| *** thiagop has quit IRC | 22:41 | |
| *** inc0 has joined #openstack-infra | 22:41 | |
| *** yamamoto has joined #openstack-infra | 22:47 | |
| *** pradk has quit IRC | 22:47 | |
| *** aeng has quit IRC | 22:47 | |
| *** asettle has quit IRC | 22:47 | |
| *** rbrndt has quit IRC | 22:49 | |
| *** inc0 has quit IRC | 22:49 | |
| *** thorst_ has joined #openstack-infra | 22:49 | |
| *** andymaier has quit IRC | 22:50 | |
| *** kzaitsev_mb has quit IRC | 22:50 | |
| karthikp_ | @fungi; is there connection between them by default? | 22:51 |
| fungi | karthikp_: none whatsoever. they're completely unrelated constructs | 22:51 |
| fungi | karthikp_: grenade performs an in-place upgrade of openstack services, so is typically a single-node affair. primary/sub nodes have to do with multi-node devstack deployments | 22:52 |
| *** yamamoto has quit IRC | 22:54 | |
| *** thorst_ has quit IRC | 22:54 | |
| *** PollyZ has quit IRC | 22:56 | |
| openstackgerrit | Merged openstack-infra/system-config: Use openstackzuul-bluebox now that jenkins is gone https://review.openstack.org/353602 | 22:56 |
| *** inc0 has joined #openstack-infra | 22:57 | |
| *** inc0 has quit IRC | 22:57 | |
| *** kzaitsev_mb has joined #openstack-infra | 22:59 | |
| *** tonytan4ever has joined #openstack-infra | 22:59 | |
| karthikp_ | fungi: perfect makes sense.. | 22:59 |
| openstackgerrit | Zhenguo Niu proposed openstack-infra/project-config: Create new project nimble https://review.openstack.org/353575 | 23:00 |
| openstackgerrit | Zhenguo Niu proposed openstack-infra/project-config: Create new project Nimble https://review.openstack.org/353575 | 23:01 |
| *** inc0 has joined #openstack-infra | 23:01 | |
| *** mriedem has joined #openstack-infra | 23:01 | |
| *** Sukhdev has quit IRC | 23:01 | |
| *** bryan_att has quit IRC | 23:03 | |
| *** tonytan4ever has quit IRC | 23:03 | |
| *** aeng has joined #openstack-infra | 23:03 | |
| *** rwsu has joined #openstack-infra | 23:04 | |
| openstackgerrit | Merged openstack-infra/puppet-elastic_recheck: Rework uncategorized script to handle multiple html files https://review.openstack.org/353630 | 23:04 |
| *** tonytan4ever has joined #openstack-infra | 23:07 | |
| *** eric_lopez has quit IRC | 23:10 | |
| fungi | okay, maintenance announcements sent for gerrit and wiki upgrades friday | 23:12 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Stub out aborting jobs in ansible launcher https://review.openstack.org/353778 | 23:13 |
| fungi | i sent the wiki one to -dev, -operators and the general ml for a broader audience, since it's also a bit of a call for testing | 23:13 |
| mtreinish | fungi: all the announcements remind me I think we put in the firehose spec sending a ML announcement about it | 23:16 |
| mtreinish | we should do that at some point | 23:16 |
| fungi | mtreinish: yep, i think we're about at that point | 23:16 |
| fungi | docs merged, we can probably put the infra spec for it to bed now too? | 23:17 |
| mtreinish | yeah, I think we finished all the pieces in it | 23:17 |
| *** inc0 has quit IRC | 23:17 | |
| fungi | that's just accomplished with a patch to the infra-specs repo moving the entry for it in the index.rst from the approved list to the implemented list | 23:18 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/infra-specs: Move firehose spec to implemented https://review.openstack.org/353779 | 23:18 |
| mtreinish | fungi: ^^^ | 23:18 |
| mtreinish | I would have been faster but I kept mistyping my gpg passphrase | 23:19 |
| fungi | hah | 23:19 |
| fungi | proof that signed commits are teh debbil's work | 23:19 |
| ianw | you would think "boot this kernel please" would be something rather simple | 23:19 |
| fungi | ianw: i don't think i've ever actually thought that. wished it many times though | 23:20 |
| *** pvaneck has quit IRC | 23:20 | |
| *** elo has joined #openstack-infra | 23:20 | |
| fungi | then again, i'm usually surprised when any of this stuff works at all | 23:20 |
| *** kien-ha has joined #openstack-infra | 23:22 | |
| *** Julien-zte has quit IRC | 23:24 | |
| *** eric_lopez has joined #openstack-infra | 23:25 | |
| *** elo has quit IRC | 23:25 | |
| anteaya | pleia2: actually it was pabelanger who told me that our wiki doesn't use ffmpeg | 23:25 |
| anteaya | pabelanger: have you time to review https://review.openstack.org/#/c/352056/4 regarding our wiki's usage or not of ffmpeg? | 23:26 |
| *** sdake has joined #openstack-infra | 23:26 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Use updated overcloud-full image name https://review.openstack.org/353729 | 23:28 |
| *** amitgandhinz has joined #openstack-infra | 23:30 | |
| *** ayoung has quit IRC | 23:30 | |
| *** chlong|mtg has quit IRC | 23:31 | |
| *** hongbin has quit IRC | 23:31 | |
| *** Sukhdev has joined #openstack-infra | 23:32 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config: Force install of F24 workaround kernel https://review.openstack.org/353783 | 23:32 |
| pleia2 | anteaya: thanks | 23:32 |
| *** claudiub has joined #openstack-infra | 23:32 | |
| *** cody-somerville has quit IRC | 23:33 | |
| kien-ha | zaro: Hey Khai, you around? | 23:33 |
| *** amitgandhinz has quit IRC | 23:34 | |
| *** sdague has quit IRC | 23:35 | |
| *** karthik__ has joined #openstack-infra | 23:37 | |
| *** zhurong has joined #openstack-infra | 23:37 | |
| anteaya | pleia2: :) | 23:38 |
| *** karthikp_ has quit IRC | 23:39 | |
| *** zz_dimtruck is now known as dimtruck | 23:44 | |
| ianw | hodyssey4me: what about clarkb's comments in https://review.openstack.org/#/c/307831 ? | 23:44 |
| anteaya | the tale of a person and their journey with their coal hod | 23:45 |
| ianw | ahh, hang on, might have been looking at cs6 | 23:47 |
| ianw | ok, yeah. let's see here then ... | 23:47 |
| *** woodster_ has quit IRC | 23:49 | |
| *** nwkarsten has joined #openstack-infra | 23:50 | |
| *** raunak has quit IRC | 23:53 | |
| *** rfolco has quit IRC | 23:53 | |
| *** nwkarsten has quit IRC | 23:55 | |
| openstackgerrit | Steve Martinelli proposed openstack-infra/project-config: test heat-translator gate fail https://review.openstack.org/353656 | 23:56 |
| *** kzaitsev_mb has quit IRC | 23:57 | |
| *** dingyichen has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!