| *** asselin has joined #openstack-infra | 00:03 | |
| *** dimtruck is now known as zz_dimtruck | 00:04 | |
| *** rbrndt has quit IRC | 00:05 | |
| *** zz_dimtruck is now known as dimtruck | 00:05 | |
| ianw | greghaynes: so, yeah, looks like all the DIB failures are due to diskimage-builder in an odd way ... it has cached all the old devstack branches so it's trying to download images that don't exist. I think we need a "--prune" in there to make sure old branches are not in the cache | 00:06 |
|---|---|---|
| *** pradk has quit IRC | 00:06 | |
| ianw | of course, this is only a problem where there are *really* old caches, so it doesn't replicate in a recent build because your base cache just doesn't have the branches | 00:06 |
| greghaynes | ianw: wah | 00:07 |
| greghaynes | ianw: why is dib trying to download images | 00:07 |
| greghaynes | ianw: oh, you mean the base fedora cloud images? Why would devstack have anything to do with that | 00:07 |
| ianw | greghaynes: well, it's not ... the devstack cache does. but that is going through the branches of the devstack git tree and building up a list of images to build | 00:08 |
| zaro | bardia: not sure, but ignoring replication events might be normal. what you'll want to watch for is patchset-created events which is the event that should kick off a build | 00:08 |
| greghaynes | ianw: hrm, that doesn't reconcile with the bits I know about our dib builds, is there a log or something? | 00:08 |
| *** amrith is now known as _amrith_ | 00:08 | |
| *** markmcclain has quit IRC | 00:09 | |
| bardia | zaro: would that be in the zuul debug.log? | 00:09 |
| ianw | greghaynes: see -> http://paste.openstack.org/show/483506/ | 00:09 |
| asselin | asselin | 00:09 |
| ianw | icehouse and juno branches are gone, but they're still in the cached version on disk | 00:10 |
| *** markmcclain has joined #openstack-infra | 00:10 | |
| oomichi | lifeless: dhellmann: hi, can you take a look at g-r patch: https://review.openstack.org/#/c/263106/ ? | 00:10 |
| greghaynes | ianw: ahhh | 00:10 |
| zaro | bardia: yes, i believe so. here are the gerrit events, https://gerrit-documentation.storage.googleapis.com/Documentation/2.11/cmd-stream-events.html#events | 00:11 |
| ianw | greghaynes: i *think* https://git.openstack.org/cgit/openstack/diskimage-builder/tree/elements/source-repositories/extra-data.d/98-source-repositories#n128 needs a "--prune"? | 00:13 |
| ianw | fungi: ^ seems like you added this bit... | 00:13 |
| bardia | zaro: yeah I can see the patchset-created events in the logs but still no jobs are started for it | 00:13 |
| bardia | zaro: I have checked the gearman server and it has builders ready | 00:14 |
| zaro | bardia: is the gearman queue empty? | 00:15 |
| bardia | zaro: yes | 00:15 |
| bardia | build:dsvm-tempest-coho-cinder-driver:d-p-c001 | 00:15 |
| bardia | thats the response from the status call to gearman server | 00:15 |
| bardia | empty queue with one builder ready | 00:16 |
| greghaynes | ianw: ok, so that is just the bit to compile images which devstack will install in to glance | 00:16 |
| greghaynes | ianw: ya, just needs a prune. That is a pretty wierd bug | 00:16 |
| zaro | hmm, not sure why zuul isn't sending any jobs. probably best one to ask is jeblair | 00:17 |
| zaro | bardia: have you maybe tried to restart zuul? | 00:17 |
| ianw | greghaynes: yep, so it gets an old fedora image, which must have disappeared into the archives at the end of the year, and so now all builds fail with a 404 trying to grab that | 00:17 |
| jeblair | bardia: can you paste your layout.yaml file to paste.openstack.org? | 00:18 |
| *** rkukura has joined #openstack-infra | 00:19 | |
| bardia | zaro: yes many times | 00:19 |
| bardia | jeblair: will do | 00:19 |
| *** mudassirlatif has quit IRC | 00:19 | |
| bardia | jeblair: http://paste.openstack.org/show/483510/ | 00:20 |
| *** markvoelker has quit IRC | 00:20 | |
| *** otter768 has quit IRC | 00:20 | |
| *** annegentle has quit IRC | 00:24 | |
| *** Guest5635 has quit IRC | 00:24 | |
| Thelo | asselin: let's move here, it is more appropriate ( I am running the CI again to see the content of pip freeze, I will let you know when its done) | 00:25 |
| *** ajmiller has quit IRC | 00:25 | |
| asselin | Thelo, up to you, your questions are related to cinder, so cinder channel is fine too | 00:26 |
| Thelo | asselin: Besides, Does the CI automatically update the code related to the patchset? | 00:26 |
| asselin | Thelo, normally yes, that's what zuul does, it prepares all the references for all the repos. | 00:27 |
| bardia | jeblair: any insight? | 00:27 |
| *** lvdongbing has joined #openstack-infra | 00:27 | |
| Thelo | asselin: Regarding my other questions | 00:28 |
| Thelo | I am submitting a new driver and wander how should os-brick be tested in the CI for the first commit( since the driver is divided in three patchs os-brick, nova and cinder. Besides Nova's and cinder's patchsets depends of os-brick) | 00:28 |
| Thelo | do you know about this? | 00:28 |
| asselin | Thelo, well you can setup your ci to run on os-brick patches too that pull in your unmerged driver....similarly for nova. But if you can get them unit tested and the cores of those projects are ok with your changes, perhaps you don't neeed to go so far. | 00:29 |
| asselin | Thelo, I suggest talking to hemna in cinder channel | 00:29 |
| Thelo | is he in your time zone? | 00:30 |
| Thelo | asselin: Thanks | 00:30 |
| asselin | yes | 00:30 |
| *** Sukhdev has quit IRC | 00:31 | |
| *** piet has joined #openstack-infra | 00:31 | |
| *** rkukura has quit IRC | 00:32 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Prune old branches when updating cache https://review.openstack.org/266086 | 00:32 |
| ianw | greghaynes: ^ *think* this will work ... hard to test fully | 00:33 |
| ianw | i guess i could add the old branch to my cached checkout ... hmm, will try | 00:33 |
| openstackgerrit | Christian Hofstädtler proposed openstack-infra/jenkins-job-builder: gitlab trigger: Support new "addCiMessage" flag https://review.openstack.org/266087 | 00:34 |
| *** dizquierdo has quit IRC | 00:34 | |
| *** dimtruck is now known as zz_dimtruck | 00:36 | |
| *** dims_ has joined #openstack-infra | 00:39 | |
| *** markmcclain has quit IRC | 00:39 | |
| *** dims has quit IRC | 00:40 | |
| fungi | ianw: greghaynes: agreed, pruning makes sense. my brains were turned toward making sure the things we use get included, but not considering making sure things we need to _not_ be present are removed | 00:40 |
| *** Sukhdev has joined #openstack-infra | 00:40 | |
| *** rfolco has quit IRC | 00:41 | |
| fungi | i'm in favor of the idea behind 266086 as long as someone tests it | 00:41 |
| *** sputnik13 has quit IRC | 00:41 | |
| *** sputnik13 has joined #openstack-infra | 00:42 | |
| *** markmcclain has joined #openstack-infra | 00:42 | |
| *** sputnik13 has quit IRC | 00:42 | |
| *** sputnik13 has joined #openstack-infra | 00:42 | |
| *** rkukura has joined #openstack-infra | 00:43 | |
| *** yolanda has quit IRC | 00:44 | |
| jeblair | bardia: can you paste the relevant lines from the zuul debug log? | 00:45 |
| *** bpokorny_ has joined #openstack-infra | 00:45 | |
| *** yolanda has joined #openstack-infra | 00:45 | |
| bardia | jeblair: http://paste.openstack.org/show/483514/ | 00:48 |
| bardia | jeblair: do you know about this change in zuul? https://github.com/openstack-infra/zuul/commit/514db293af344e15d7a1f1585c2ac84c3694eb0f | 00:48 |
| bardia | it was posted the day before our zuul stopped working | 00:49 |
| jeblair | bardia: there should be a section of the debug log right when zuul starts where it logs its configuration, can you paste that? | 00:49 |
| *** bpokorny has quit IRC | 00:49 | |
| bardia | jeblair: thats the thing. its not logging that | 00:49 |
| bardia | jeblair: now that you mentioned it i noticed it | 00:49 |
| jeblair | bardia: oh, that log you just pasted is from the beginning isn't it? | 00:49 |
| bardia | jeblair: yeah thats right from the restart | 00:50 |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Allow puppet to configure the Gerrit's sshd.batchThreads option https://review.openstack.org/262053 | 00:50 |
| bardia | jeblair: but its not logging the config that its loading like it used to | 00:50 |
| *** moravec has quit IRC | 00:50 | |
| jeblair | bardia: okay, that explains why it isn't doing anything -- it doesn't actually have any pipelines, etc, defined... | 00:50 |
| bardia | jeblair: hmmmm so its not loading the config | 00:51 |
| bardia | :q | 00:51 |
| bardia | oops | 00:51 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add DB API method to get a list of recent runs by metadata https://review.openstack.org/264303 | 00:51 |
| *** angdraug has quit IRC | 00:52 | |
| bardia | jeblair: any ideas as to why zuul is not loading its config? | 00:53 |
| jeblair | bardia: not yet | 00:53 |
| bardia | jeblair: thanks | 00:53 |
| bardia | jeblair: I will do some debugging on my sys to see if i can figure it out | 00:54 |
| bardia | -r-------- 1 zuul root 790 Jan 11 15:40 zuul.conf | 00:54 |
| bardia | jeblair: does the perm look right? | 00:54 |
| clarkb | fungi: greghaynes ianw ya we do that same prune step on gerrit git prep | 00:54 |
| jeblair | bardia: yes, that's how we run | 00:54 |
| clarkb | we can just run that in the repo that is copied in from the cache | 00:54 |
| jeblair | bardia: (would be good to double check zuul is running as the zuul user :) | 00:54 |
| clarkb | zaro: so I am confused about that because I don't think we ever published it anywhere and certainly wouldn't delete it | 00:55 |
| bardia | jeblair: it is. one thing I have noticed is that the service.d stop script is not working for zuul | 00:55 |
| *** gyee has quit IRC | 00:55 | |
| openstackgerrit | Merged openstack/diskimage-builder: deploy-ironic: Fix syntax error when checking for root device hints https://review.openstack.org/264736 | 00:55 |
| bardia | jeblair: on start its not creating the zuul.pid file so it can stop it | 00:55 |
| bardia | jeblair: to restart zuul i have to manually kill the zuul-server processes | 00:56 |
| *** woodster_ has quit IRC | 00:56 | |
| jeblair | bardia: hrm, i thought that was fixed -- at least i haven't had that problem in a long time | 00:56 |
| *** piet has quit IRC | 00:57 | |
| *** piet has joined #openstack-infra | 00:57 | |
| zaro | clarkb: the jenkins-plugin job publishes it to jenkins repo on a release. | 00:57 |
| *** rkukura has quit IRC | 00:58 | |
| *** aviau_ is now known as aviau | 00:58 | |
| bardia | jeblair: i actually never had that problem until I recently reran the puppet scripts | 00:58 |
| *** Sukhdev has quit IRC | 00:58 | |
| *** amuller_afk is now known as amuller | 00:58 | |
| clarkb | zaro: hvae we ever made a release for it? | 00:58 |
| zaro | i think the latest version then gets automatatically put on the jenkins update site. | 00:58 |
| *** mmedvede has quit IRC | 00:58 | |
| zaro | i just assumed so because it got published to jenkins repo. | 00:59 |
| zaro | but maybe not. | 00:59 |
| *** Swami has quit IRC | 00:59 | |
| clarkb | I mean its 404ing so I can't make any statements about it | 00:59 |
| zaro | clarkb: i see a few tags, https://git.openstack.org/cgit/openstack-infra/zmq-event-publisher/ | 00:59 |
| clarkb | and seems like the fix for that would be to have jenkins pull from the maven repo rather than us making a new release | 01:00 |
| *** eil397 has quit IRC | 01:00 | |
| jeblair | bardia: does "zuul-server -c /etc/zuul/zuul.conf -l /etc/zuul/layout/layout.yaml -t" work? | 01:00 |
| zaro | clarkb: yeah something's wrong with things going from jenkins repo to jenkins update site. | 01:00 |
| jeblair | bardia: (and double check that /etc/zuul/zuul.conf points to /etc/zuul/layout/layout.yaml) | 01:01 |
| bardia | :q | 01:01 |
| zaro | fwiw, gearman plugin is on update site | 01:01 |
| zaro | clarkb: i was just hoping new release might fix it on jenkins end. not sure if it will though. | 01:01 |
| bardia | jeblair: no "zuul-server -c /etc/zuul/zuul.conf -l /etc/zuul/layout/layout.yaml -t" did not work | 01:02 |
| greghaynes | ianw: Hey can I get a quick review on https://review.openstack.org/#/c/259734/ ? | 01:02 |
| openstackgerrit | Merged openstack-infra/subunit2sql: Add DB API method to get a list of recent runs by metadata https://review.openstack.org/264303 | 01:02 |
| bardia | jeblair: traceback http://paste.openstack.org/show/483515/ | 01:02 |
| greghaynes | ianw: The dependent patch merged and I'd like to cut a release with that in it | 01:02 |
| clarkb | zaro: ah gotcha | 01:03 |
| clarkb | zaro: I mean I don't mind making a release if we need one, but might be better to figure out if thats the best step first | 01:03 |
| clarkb | greghaynes: fungi ianw do we hvae a fix for the old branches yet? | 01:03 |
| jeblair | bardia: did you run that as zuul? is /etc/zuul/zuul.conf the correct path to your zuul config? does it have a [zuul] section? | 01:04 |
| clarkb | I can review that if so and write it if not | 01:04 |
| greghaynes | clarkb: https://review.openstack.org/#/c/266086/1 | 01:04 |
| *** oomichi has quit IRC | 01:04 | |
| *** ddieterly has joined #openstack-infra | 01:04 | |
| clarkb | hrm I am not sure we use that in our dib builds | 01:05 |
| clarkb | for hysterical raisins | 01:05 |
| *** eil397 has joined #openstack-infra | 01:05 | |
| bardia | jeblair: ok I didn't run it as zuul. but now as zuul its failing with this traceback: http://paste.openstack.org/show/483516/ | 01:05 |
| greghaynes | clarkb: right - I think there is a copy of that script somewhere in the infra elmeents | 01:05 |
| greghaynes | clarkb: which needs to be fixed | 01:05 |
| clarkb | greghaynes: ya ... | 01:05 |
| clarkb | greghaynes: maybe we can just rm it and make it pull from dib | 01:05 |
| clarkb | mordred: ^ why was it forked? | 01:05 |
| *** ybathia has quit IRC | 01:05 | |
| greghaynes | clarkb: It is a copy because it needs to run on a dynamically generated list of repos | 01:06 |
| greghaynes | clarkb: IIRC | 01:06 |
| clarkb | it actually looks like it may no longer be a fork | 01:06 |
| clarkb | http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/elements/openstack-repos | 01:06 |
| jeblair | bardia: ah, you need to upgrade apscheduler | 01:06 |
| *** kzaitsev_mb has quit IRC | 01:07 | |
| clarkb | greghaynes: so we may actually be good once dib updates | 01:07 |
| bardia | jeblair: how? | 01:07 |
| *** Qiming has joined #openstack-infra | 01:07 | |
| greghaynes | clarkb: ok, so there are 3 things I need to release with dib now :) | 01:07 |
| clarkb | greghaynes: gg | 01:07 |
| jeblair | bardia: are you using puppet-openstackci? | 01:07 |
| bardia | yes | 01:07 |
| jeblair | bardia: hrm, then it should have upgraded it automatically when zuul's dependency on it updated | 01:08 |
| jeblair | bardia: did puppet upgrade your zuul or did you do that manually? | 01:08 |
| bardia | jeblair: puppet did. I have ran this multiple times today: sudo puppet apply --verbose /etc/puppet/manifests/site.pp | 01:09 |
| *** aeng has quit IRC | 01:09 | |
| jeblair | bardia: to fix, you should be able to run "sudo pip install -U /opt/zuul" and it should reinstall zuul and upgrade any needed deps. | 01:09 |
| jeblair | bardia: if something is wrong with that, it might print out useful info | 01:09 |
| *** rkukura has joined #openstack-infra | 01:09 | |
| bardia | jeblair: ah now this "zuul-server -c /etc/zuul/zuul.conf -l /etc/zuul/layout/layout.yaml -t" worked | 01:10 |
| bardia | jeblair: let me try restarting the zuul service | 01:10 |
| jeblair | bardia: cool | 01:10 |
| *** pvaneck has quit IRC | 01:11 | |
| bardia | jeblair: yup that did it. it loaded the config now | 01:12 |
| bardia | jeblair: started the queue process too. | 01:12 |
| bardia | jeblair: thanks a bunch!!!!!!!!!!!!! | 01:12 |
| *** lvdongbing has quit IRC | 01:14 | |
| *** markmcclain has quit IRC | 01:17 | |
| *** lvdongbing has joined #openstack-infra | 01:18 | |
| jeblair | bardia: np! glad it works | 01:20 |
| *** markvoelker has joined #openstack-infra | 01:21 | |
| *** markmcclain has joined #openstack-infra | 01:21 | |
| *** markvoelker has quit IRC | 01:25 | |
| *** markvoelker has joined #openstack-infra | 01:25 | |
| *** sputnik13 has quit IRC | 01:26 | |
| *** unicell1 has quit IRC | 01:27 | |
| ianw | greghaynes: is adding that pipefail intentional? | 01:27 |
| *** unicell has joined #openstack-infra | 01:27 | |
| bardia | jeblair: should I open a bug against puppet-openstackci as to why it wasn't updating zuul properly? | 01:28 |
| nibalizer | bardia: i think so | 01:28 |
| greghaynes | ianw: Yep - I need the | tee to actually fail if the run-parts fails | 01:28 |
| bardia | nibalizer: will do | 01:28 |
| *** ianychoi has quit IRC | 01:29 | |
| *** aeng has joined #openstack-infra | 01:29 | |
| bardia | nibalizer: which project would be? puppet-openstack-integration? | 01:30 |
| *** asselin has quit IRC | 01:34 | |
| ianw | greghaynes: just added a couple of comments there] | 01:35 |
| *** thorst_ has quit IRC | 01:39 | |
| *** thorst has joined #openstack-infra | 01:40 | |
| greghaynes | ianw: ah, ok | 01:40 |
| *** mmedvede has joined #openstack-infra | 01:41 | |
| *** ianychoi has joined #openstack-infra | 01:42 | |
| *** Sukhdev has joined #openstack-infra | 01:43 | |
| *** zz_dimtruck is now known as dimtruck | 01:47 | |
| *** dims_ has quit IRC | 01:48 | |
| *** thorst has quit IRC | 01:48 | |
| *** dims has joined #openstack-infra | 01:48 | |
| *** markmcclain has quit IRC | 01:51 | |
| *** mtanino has quit IRC | 01:52 | |
| *** dims has quit IRC | 01:54 | |
| *** markmcclain has joined #openstack-infra | 01:55 | |
| *** _amrith_ is now known as amrith | 01:56 | |
| *** yolanda has quit IRC | 01:57 | |
| *** asselin has joined #openstack-infra | 01:57 | |
| *** [1]Thelo has joined #openstack-infra | 02:01 | |
| *** Sukhdev has quit IRC | 02:01 | |
| *** yolanda has joined #openstack-infra | 02:02 | |
| cody-somerville | Has anyone made any progress on that zuul memory leak? | 02:03 |
| *** Thelo has quit IRC | 02:03 | |
| *** [1]Thelo is now known as Thelo | 02:03 | |
| *** markmcclain has quit IRC | 02:03 | |
| *** piet has quit IRC | 02:04 | |
| asselin | bardia, the location for bugs is documented here: http://git.openstack.org/cgit/openstack-infra/puppet-openstackci/tree/CONTRIBUTING.rst | 02:04 |
| *** jhesketh has quit IRC | 02:05 | |
| bardia | asselin: thanks | 02:05 |
| *** markmcclain has joined #openstack-infra | 02:05 | |
| asselin | bardia, that said, it's probably an issue with puppet-zuul | 02:05 |
| *** jhesketh has joined #openstack-infra | 02:05 | |
| asselin | bardia, here's where it would get installed: http://git.openstack.org/cgit/openstack-infra/puppet-zuul/tree/manifests/init.pp#n129 | 02:06 |
| *** dims has joined #openstack-infra | 02:06 | |
| asselin | bardia, I don't know why apscheduler didn't install. if you can, try uninstalling that package. | 02:07 |
| *** eil397 has quit IRC | 02:07 | |
| bardia | asselin: ah ok. is puppet-zuul part of storyboard.openstack too? | 02:07 |
| asselin | bardia, then go to /opt/zuul and git checkout HEAD^^ | 02:07 |
| asselin | bardia, and run puppet again | 02:08 |
| *** aeng has quit IRC | 02:08 | |
| asselin | bardia, probably yes | 02:08 |
| *** zhurong has joined #openstack-infra | 02:08 | |
| bardia | asselin: I had to manually reinstall zuul with "sudo pip install -U /opt/zuul" | 02:09 |
| asselin | bardia, yes I read that in the scrollback...but I don't know why | 02:10 |
| *** kaisers1 has quit IRC | 02:10 | |
| bardia | asselin: hmmm ok I will try uninstalling apscheduler then running puppet | 02:10 |
| asselin | bardia, do this before running puppet: go to /opt/zuul and git checkout HEAD^^ | 02:11 |
| *** kaisers has joined #openstack-infra | 02:11 | |
| bardia | asselin: ok | 02:11 |
| *** bpokorny_ has quit IRC | 02:13 | |
| *** bpokorny has joined #openstack-infra | 02:14 | |
| *** markmcclain has quit IRC | 02:15 | |
| bardia | asselin: yup it didn't install apscheduler | 02:15 |
| asselin | bardia, can you paste the puppet output? | 02:16 |
| asselin | bardia, and also git log -n 1 in /opt/zuul | 02:16 |
| bardia | asselin: it looks like that package is missing from here: http://git.openstack.org/cgit/openstack-infra/puppet-zuul/tree/manifests/init.pp#n129 | 02:16 |
| bardia | asselin: http://paste.openstack.org/show/483521/ | 02:16 |
| *** yamamoto has joined #openstack-infra | 02:17 | |
| asselin | bardia, those aren't pip packages | 02:17 |
| *** rguillebert has quit IRC | 02:17 | |
| bardia | asselin: oh i see | 02:17 |
| openstackgerrit | hardik proposed openstack-infra/project-config: Fixed mistral experimental job https://review.openstack.org/265030 | 02:17 |
| asselin | bardia, and if you run this now? sudo pip install -U /opt/zuul | 02:18 |
| *** markmcclain has joined #openstack-infra | 02:18 | |
| bardia | asselin: now apscheduler is installed | 02:19 |
| asselin | bardia, /etc/puppet/modules/zuul | 02:19 |
| asselin | git log -n 1 from there | 02:20 |
| bardia | asselin: Merge "Add ensure parameter to zuul services" | 02:20 |
| asselin | bardia, which pip | 02:21 |
| *** otter768 has joined #openstack-infra | 02:21 | |
| bardia | asselin: pip 7.1.2 from /usr/local/lib/python2.7/dist-packages (python 2.7) | 02:22 |
| *** aeng has joined #openstack-infra | 02:22 | |
| asselin | bardia, I wonder if there's an issue with pip referenced on line 131: http://git.openstack.org/cgit/openstack-infra/puppet-zuul/tree/manifests/init.pp?id=6e79104957c387e248dbf92e34481977ea7729a0 | 02:22 |
| asselin | bardia, seems to be the only difference... | 02:23 |
| asselin | bardia, can you check the versions of pip in those paths specified? | 02:23 |
| *** dimtruck is now known as zz_dimtruck | 02:24 | |
| bardia | asselin: how do I do that? | 02:25 |
| bardia | asselin: its pip 7.1.2 from cmd line | 02:26 |
| *** rkukura has quit IRC | 02:26 | |
| asselin | bardia, specify the full path | 02:26 |
| asselin | bardia, /usr/local/bin/pip -version | 02:26 |
| *** gongysh has joined #openstack-infra | 02:26 | |
| *** otter768 has quit IRC | 02:26 | |
| asselin | there's 3 paths listed: '/usr/local/bin:/usr/bin:/bin/', | 02:26 |
| bardia | /usr/local/bin/pip2.7 --version | 02:27 |
| bardia | pip 7.1.2 from /usr/local/lib/python2.7/dist-packages (python 2.7) | 02:27 |
| bardia | /usr/local/bin/pip --version | 02:27 |
| bardia | pip 7.1.2 from /usr/local/lib/python2.7/dist-packages (python 2.7) | 02:27 |
| bardia | pip: /usr/local/bin/pip /usr/local/bin/pip2.7 | 02:28 |
| *** keedya has joined #openstack-infra | 02:29 | |
| asselin | bardia, no idea why it doesn't behave the same way when run by puppet... | 02:30 |
| clarkb | I am rebuilding bare-trusty image in rax-dfw to get a head start with new images and the mysql grant fix | 02:30 |
| greghaynes | clarkb: w00t | 02:30 |
| asselin | bardia, perhaps you can try again with more puppet verbose or debug logging | 02:30 |
| *** bpokorny_ has joined #openstack-infra | 02:30 | |
| bardia | asselin: ok I will try later. Gotta head home in a bit :) | 02:31 |
| asselin | bardia, and add this to the /etc/puppet/modules/zuul/manifest/init.pp: https://docs.puppetlabs.com/references/latest/type.html#exec-attribute-logoutput | 02:31 |
| asselin | logoutput => true | 02:31 |
| asselin | bardia, sure | 02:31 |
| *** yamamoto has quit IRC | 02:31 | |
| *** kzaitsev_mb has joined #openstack-infra | 02:32 | |
| *** bpokorny has quit IRC | 02:34 | |
| *** bpokorny_ has quit IRC | 02:35 | |
| *** bardia has quit IRC | 02:36 | |
| *** yamamoto_ has joined #openstack-infra | 02:38 | |
| *** bharathm has quit IRC | 02:39 | |
| *** yamamoto_ has quit IRC | 02:40 | |
| clarkb | anteaya: worker20 looks happy, I think a rough plan on my end would be to try deploying that across the other 19 sometime after wednesday since that is when I am doing the nodepool image builders | 02:41 |
| *** fawadkhaliq has joined #openstack-infra | 02:44 | |
| clarkb | if this bare-trusty run works out I am going to rebuild the rest of bare-trusty | 02:46 |
| clarkb | also, we should remember that at the end of the month only rax will be providing bare-* | 02:46 |
| *** salv-orlando has quit IRC | 02:47 | |
| *** thorst has joined #openstack-infra | 02:48 | |
| *** thorst has quit IRC | 02:53 | |
| *** amotoki has joined #openstack-infra | 02:54 | |
| *** ramineni has joined #openstack-infra | 02:57 | |
| *** EricGonczer_ has quit IRC | 02:58 | |
| *** salv-orlando has joined #openstack-infra | 02:58 | |
| *** ramineni1 has joined #openstack-infra | 03:00 | |
| *** kevinbenton_ has joined #openstack-infra | 03:00 | |
| *** ramineni has quit IRC | 03:01 | |
| *** ramineni has joined #openstack-infra | 03:01 | |
| *** apoorvad has quit IRC | 03:02 | |
| *** oomichi has joined #openstack-infra | 03:03 | |
| *** kevinbenton_ has quit IRC | 03:03 | |
| *** ramineni2 has joined #openstack-infra | 03:03 | |
| *** amuller has quit IRC | 03:04 | |
| *** markmcclain has quit IRC | 03:04 | |
| *** bharathm has joined #openstack-infra | 03:04 | |
| *** ramineni1 has quit IRC | 03:04 | |
| *** ramineni1 has joined #openstack-infra | 03:04 | |
| *** MarkAtwood has quit IRC | 03:05 | |
| *** ramineni3 has joined #openstack-infra | 03:05 | |
| *** zz_dimtruck is now known as dimtruck | 03:06 | |
| *** ramineni has quit IRC | 03:06 | |
| *** spredzy has quit IRC | 03:07 | |
| *** ramineni2 has quit IRC | 03:07 | |
| *** markmcclain has joined #openstack-infra | 03:07 | |
| *** ramineni has joined #openstack-infra | 03:08 | |
| *** ramineni1 has quit IRC | 03:09 | |
| *** piet has joined #openstack-infra | 03:10 | |
| *** spredzy has joined #openstack-infra | 03:10 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Remove Nova object store service https://review.openstack.org/266119 | 03:10 |
| *** ramineni3 has quit IRC | 03:10 | |
| *** amotoki has quit IRC | 03:12 | |
| *** ramineni has quit IRC | 03:12 | |
| *** markmcclain has quit IRC | 03:12 | |
| *** amotoki has joined #openstack-infra | 03:14 | |
| *** kzaitsev_mb has quit IRC | 03:14 | |
| *** markmcclain has joined #openstack-infra | 03:18 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 03:22 | |
| *** dims has quit IRC | 03:24 | |
| clarkb | dimtruck: I am amazed that shows up in so many places | 03:25 |
| *** markmcclain has quit IRC | 03:25 | |
| *** doug-fish has joined #openstack-infra | 03:30 | |
| *** yamamoto_ has joined #openstack-infra | 03:33 | |
| *** amotoki has quit IRC | 03:36 | |
| clarkb | greghaynes: we can probably recheck those nodepool changes in about 15 minutes | 03:36 |
| clarkb | we are in the make a snapshot portion of the image builds | 03:37 |
| *** ramineni1 has joined #openstack-infra | 03:37 | |
| greghaynes | clarkb: awesome | 03:37 |
| *** ramineni has joined #openstack-infra | 03:38 | |
| *** ramineni has quit IRC | 03:38 | |
| *** ramineni_ has joined #openstack-infra | 03:38 | |
| *** piet has quit IRC | 03:38 | |
| ramineni_ | AJaeger: Could you please review https://review.openstack.org/#/c/258799/ | 03:38 |
| *** ramineni1 has quit IRC | 03:41 | |
| clarkb | Exception in thread hpcloud-b4 (most likely raised during interpreter shutdown): | 03:42 |
| clarkb | thats a new one | 03:42 |
| *** ddieterly has quit IRC | 03:42 | |
| clarkb | but the new image is there so apparently nothing fatal | 03:43 |
| clarkb | greghaynes: there is one outstanding build at athis point out of 8, so you can roll the dice now if you want | 03:48 |
| greghaynes | Roll d-8 | 03:48 |
| * craige waves. | 03:48 | |
| clarkb | craige: how goes phabrication? | 03:49 |
| craige | Anyone, particularly Clint if he's still, feel like trying the latest iteration of the phab migration? | 03:49 |
| craige | http://os01.mcwhirter.io/ | 03:49 |
| clarkb | oh hey I have bugs assigned to me | 03:49 |
| craige | heh | 03:50 |
| clarkb | so that worked :) | 03:50 |
| craige | It's an import form storyboard from about 6 weeks ago. | 03:50 |
| craige | Latest update is that people with Launchpad accounts should have sane usernames. | 03:50 |
| clarkb | still not seeing the bug comments I owuld expect on 1010621 | 03:51 |
| craige | and people with duplicates like clarkb are sanely merged. | 03:51 |
| craige | *hopfeully* | 03:51 |
| craige | oh, not forgotten clarkb. Hadn;t even had a chance to move on that one! | 03:51 |
| clarkb | kk | 03:51 |
| craige | Thanks for reminding me. | 03:51 |
| *** thorst has joined #openstack-infra | 03:51 | |
| craige | (There was a Phab schema update last week that broke all the things :-) | 03:51 |
| clarkb | craige: is that the sort of thing you can migrate to an older schema with then just use their migrations to uplift with? | 03:52 |
| craige | Probably. | 03:52 |
| craige | In this case I just updated our migration schema. | 03:53 |
| craige | Upated the field matching. | 03:53 |
| *** vivekd has joined #openstack-infra | 03:53 | |
| *** ddieterly has joined #openstack-infra | 03:53 | |
| clarkb | greghaynes: did you roll the d8? | 03:54 |
| greghaynes | clarkb: nope | 03:54 |
| clarkb | I can if you havne't | 03:54 |
| greghaynes | Ya, go for it | 03:54 |
| *** doug-fish has quit IRC | 03:58 | |
| *** tlian has quit IRC | 03:58 | |
| *** thorst has quit IRC | 03:58 | |
| *** EricGonczer_ has joined #openstack-infra | 04:01 | |
| clarkb | greghaynes: jobs pass on the first one | 04:02 |
| *** akshai has joined #openstack-infra | 04:03 | |
| *** amotoki has joined #openstack-infra | 04:04 | |
| *** akshai_ has joined #openstack-infra | 04:04 | |
| *** kevinbenton_ has joined #openstack-infra | 04:07 | |
| *** akshai has quit IRC | 04:08 | |
| *** markmcclain has joined #openstack-infra | 04:09 | |
| *** salv-orl_ has joined #openstack-infra | 04:10 | |
| *** yamahata has quit IRC | 04:10 | |
| *** kzaitsev_mb has joined #openstack-infra | 04:11 | |
| *** salv-orlando has quit IRC | 04:13 | |
| greghaynes | clarkb: w00t | 04:15 |
| *** markmcclain has quit IRC | 04:17 | |
| *** rhallisey has quit IRC | 04:19 | |
| clarkb | greghaynes: looks like some may be failing though? Not sure I will have energy to wade through it tonight | 04:20 |
| clarkb | greghaynes: tests are timing out, was that how the db problem manifested? | 04:20 |
| greghaynes | clarkb: Ya, it is possible, I use CI to test it because ETOOMANYPATCHES | 04:20 |
| greghaynes | clarkb: uhhh, I dont think so, I can debug tomorrow though | 04:21 |
| clarkb | ok | 04:21 |
| greghaynes | clarkb: might be a rebase derp I did | 04:21 |
| clarkb | I will probablyt have time to debug tomorrow as well | 04:21 |
| *** otter768 has joined #openstack-infra | 04:22 | |
| clarkb | nibalizer: for omfra cloud is virt passthrough turned on? might be good to test it with and without that on just as another data point | 04:25 |
| clarkb | nibalizer: based on ovh numbers it doesn't affect runtime much but we think we had some unreliableness from cirros boots with it on | 04:25 |
| *** harlowja_at_home has joined #openstack-infra | 04:26 | |
| *** links has joined #openstack-infra | 04:26 | |
| *** otter768 has quit IRC | 04:26 | |
| clarkb | nibalizer: but it would still probably be good if we can see where else we are spending time on those jobs, it may not all be cpu bound | 04:29 |
| *** coolsvap|away is now known as coolsvap | 04:29 | |
| *** EricGonczer_ has quit IRC | 04:29 | |
| *** dimtruck is now known as zz_dimtruck | 04:29 | |
| *** aeng_ has joined #openstack-infra | 04:30 | |
| *** aeng has quit IRC | 04:31 | |
| *** kzaitsev_mb has quit IRC | 04:32 | |
| *** akshai_ has quit IRC | 04:35 | |
| *** piet has joined #openstack-infra | 04:38 | |
| *** treaki__ has joined #openstack-infra | 04:39 | |
| clarkb | TheJulia: FYI from roughly this point on the ironic unittests with database access should start working again | 04:41 |
| clarkb | TheJulia: we should probably work on removing the special db for those and switch to having tests create the DBs they need | 04:41 |
| *** treaki_ has quit IRC | 04:43 | |
| *** bardia has joined #openstack-infra | 04:47 | |
| *** fawadkhaliq has quit IRC | 04:52 | |
| *** MarkAtwood has joined #openstack-infra | 04:53 | |
| *** zz_dimtruck is now known as dimtruck | 04:55 | |
| *** ddieterly has quit IRC | 04:55 | |
| *** thorst has joined #openstack-infra | 04:57 | |
| *** MarkAtwood has quit IRC | 04:58 | |
| openstackgerrit | Masayuki Igawa proposed openstack/os-testr: Add unit test for colorizer https://review.openstack.org/253409 | 04:58 |
| *** markvoelker has quit IRC | 04:58 | |
| openstackgerrit | Merged openstack/requirements: Bump minimum tempest-lib version https://review.openstack.org/263106 | 04:59 |
| *** baoli has quit IRC | 04:59 | |
| *** piet has quit IRC | 05:00 | |
| *** baoli has joined #openstack-infra | 05:00 | |
| *** thorst has quit IRC | 05:03 | |
| *** mudassirlatif has joined #openstack-infra | 05:08 | |
| *** dimtruck is now known as zz_dimtruck | 05:09 | |
| *** sridhar_ram has joined #openstack-infra | 05:10 | |
| *** rlandy has quit IRC | 05:12 | |
| *** fawadkhaliq has joined #openstack-infra | 05:15 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 05:18 | |
| *** markmcclain has joined #openstack-infra | 05:19 | |
| *** kzaitsev_mb has joined #openstack-infra | 05:28 | |
| *** aeng_ has quit IRC | 05:30 | |
| *** zhurong has quit IRC | 05:32 | |
| *** markmcclain has quit IRC | 05:34 | |
| *** vgridnev has joined #openstack-infra | 05:35 | |
| *** abregman has joined #openstack-infra | 05:36 | |
| *** lvdongbing has left #openstack-infra | 05:36 | |
| *** _nadya_ has joined #openstack-infra | 05:42 | |
| *** mrmartin has joined #openstack-infra | 05:42 | |
| *** _nadya_ has quit IRC | 05:44 | |
| *** jaosorior has joined #openstack-infra | 05:47 | |
| *** _nadya_ has joined #openstack-infra | 05:49 | |
| *** harlowja_at_home has quit IRC | 05:50 | |
| *** aeng_ has joined #openstack-infra | 05:50 | |
| *** yamahata has joined #openstack-infra | 05:52 | |
| openstackgerrit | Angus Lees proposed openstack/requirements: Add oslo.privsep to global-requirements.txt https://review.openstack.org/266189 | 05:55 |
| *** ddieterly has joined #openstack-infra | 05:56 | |
| *** grue_pm_ has joined #openstack-infra | 05:57 | |
| *** kzaitsev_mb has quit IRC | 05:57 | |
| *** ildikov has quit IRC | 05:57 | |
| *** grue_pm_ has quit IRC | 05:58 | |
| *** markvoelker has joined #openstack-infra | 05:59 | |
| openstackgerrit | Angus Lees proposed openstack/requirements: Add oslo.privsep to global-requirements.txt https://review.openstack.org/266189 | 06:00 |
| *** dfflanders has joined #openstack-infra | 06:01 | |
| *** ddieterly has quit IRC | 06:01 | |
| *** HeOS has joined #openstack-infra | 06:01 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/266195 | 06:01 |
| *** thorst has joined #openstack-infra | 06:01 | |
| *** edmondsw has quit IRC | 06:02 | |
| *** kevinbenton_ has quit IRC | 06:04 | |
| *** markvoelker has quit IRC | 06:06 | |
| *** _nadya_ has quit IRC | 06:07 | |
| *** thorst has quit IRC | 06:08 | |
| openstackgerrit | Angus Lees proposed openstack/requirements: Add oslo.privsep to global-requirements.txt https://review.openstack.org/266189 | 06:09 |
| *** mrmartin has quit IRC | 06:14 | |
| *** binbincong has quit IRC | 06:14 | |
| *** mudassirlatif has quit IRC | 06:15 | |
| *** ParsectiX has joined #openstack-infra | 06:17 | |
| *** dfflanders has quit IRC | 06:20 | |
| *** rossella_s has quit IRC | 06:22 | |
| *** rossella_s has joined #openstack-infra | 06:22 | |
| openstackgerrit | Merged openstack-infra/irc-meetings: Drop bi-weekly schedule for the neutron team meeting https://review.openstack.org/266019 | 06:22 |
| openstackgerrit | Alex Xu proposed openstack-infra/irc-meetings: Add agenda url for nova api meeting https://review.openstack.org/266217 | 06:24 |
| openstackgerrit | Merged openstack-infra/irc-meetings: Add agenda url for nova api meeting https://review.openstack.org/266217 | 06:28 |
| *** zhurong has joined #openstack-infra | 06:33 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/requirements: Updated from generate-constraints https://review.openstack.org/265697 | 06:34 |
| *** aeng_ has quit IRC | 06:34 | |
| *** yolanda has quit IRC | 06:36 | |
| *** hdd has quit IRC | 06:40 | |
| *** hdd_ has joined #openstack-infra | 06:40 | |
| openstackgerrit | Craige McWhirter proposed openstack-infra/system-config: Added RemoteUser Provider & OpenID configs https://review.openstack.org/254066 | 06:42 |
| *** MarkAtwood has joined #openstack-infra | 06:42 | |
| *** MarkAtwood has quit IRC | 06:47 | |
| *** jaosorior has quit IRC | 06:48 | |
| *** esker has joined #openstack-infra | 06:49 | |
| *** fawadkhaliq has quit IRC | 06:49 | |
| *** salv-orl_ has quit IRC | 06:50 | |
| *** salv-orlando has joined #openstack-infra | 06:50 | |
| *** fawadkhaliq has joined #openstack-infra | 06:51 | |
| *** aeng_ has joined #openstack-infra | 06:52 | |
| *** sridhar_ram has quit IRC | 06:52 | |
| *** kzaitsev_mb has joined #openstack-infra | 06:53 | |
| *** bardia has quit IRC | 06:53 | |
| *** esker has quit IRC | 06:53 | |
| *** yolanda has joined #openstack-infra | 06:56 | |
| *** kzaitsev_mb has quit IRC | 06:58 | |
| *** hdd_ has quit IRC | 07:03 | |
| *** vgridnev has quit IRC | 07:05 | |
| *** thorst has joined #openstack-infra | 07:06 | |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/266195 | 07:07 |
| *** HeOS has quit IRC | 07:07 | |
| *** jaosorior has joined #openstack-infra | 07:09 | |
| *** thorst has quit IRC | 07:13 | |
| *** Zara_ has quit IRC | 07:17 | |
| *** aopincar_ has joined #openstack-infra | 07:19 | |
| *** Zara_ has joined #openstack-infra | 07:19 | |
| AJaeger | ramineni_: please talk with sdague first | 07:20 |
| ramineni_ | AJaeger: yes, i have replied to his comment, but couldn't catch him on IRC | 07:23 |
| ramineni_ | AJaeger: Any idea about his timezone? | 07:24 |
| *** odlD2 has quit IRC | 07:25 | |
| *** gongysh has quit IRC | 07:30 | |
| AJaeger | ramineni_: US | 07:31 |
| *** gongysh has joined #openstack-infra | 07:32 | |
| *** vgridnev has joined #openstack-infra | 07:35 | |
| *** esikachev has joined #openstack-infra | 07:41 | |
| *** e0ne has joined #openstack-infra | 07:42 | |
| *** ddieterly has joined #openstack-infra | 07:42 | |
| *** egonzalez90 has joined #openstack-infra | 07:44 | |
| *** ddieterly has quit IRC | 07:47 | |
| *** odlD2 has joined #openstack-infra | 07:48 | |
| *** vivekd_ has joined #openstack-infra | 07:49 | |
| *** mpavone has joined #openstack-infra | 07:49 | |
| *** mpavone has quit IRC | 07:49 | |
| openstackgerrit | hardik proposed openstack-infra/project-config: Fixed mistral experimental job https://review.openstack.org/265030 | 07:50 |
| yolanda | morning | 07:50 |
| *** Hal has joined #openstack-infra | 07:51 | |
| *** vivekd has quit IRC | 07:52 | |
| *** vivekd_ is now known as vivekd | 07:52 | |
| *** Hal is now known as Guest84591 | 07:52 | |
| *** Qiming has quit IRC | 07:52 | |
| *** isaacb has joined #openstack-infra | 07:53 | |
| *** kzaitsev_mb has joined #openstack-infra | 07:54 | |
| *** jtomasek has joined #openstack-infra | 07:57 | |
| AJaeger | morning, yolanda ! | 07:57 |
| *** egonzalez90 has quit IRC | 07:58 | |
| *** yolanda has quit IRC | 07:59 | |
| *** kzaitsev_mb has quit IRC | 07:59 | |
| *** yolanda has joined #openstack-infra | 07:59 | |
| *** vgridnev has quit IRC | 08:00 | |
| *** otter768 has joined #openstack-infra | 08:00 | |
| *** salv-orl_ has joined #openstack-infra | 08:01 | |
| *** nmagnezi has joined #openstack-infra | 08:02 | |
| *** otter768_ has joined #openstack-infra | 08:02 | |
| *** otter768 has quit IRC | 08:02 | |
| *** otter768 has joined #openstack-infra | 08:04 | |
| *** salv-orlando has quit IRC | 08:05 | |
| *** salv-orl_ has quit IRC | 08:06 | |
| *** otter76__ has joined #openstack-infra | 08:06 | |
| *** jistr has joined #openstack-infra | 08:06 | |
| *** otter768_ has quit IRC | 08:06 | |
| *** jistr is now known as jistr|doc | 08:07 | |
| *** gchamoul has left #openstack-infra | 08:07 | |
| *** otter768_ has joined #openstack-infra | 08:08 | |
| *** otter76__ has quit IRC | 08:08 | |
| *** jed56 has joined #openstack-infra | 08:08 | |
| *** HeOS has joined #openstack-infra | 08:09 | |
| *** otter768 has quit IRC | 08:09 | |
| *** otter768_ has quit IRC | 08:09 | |
| *** otter768 has joined #openstack-infra | 08:10 | |
| *** pahuang_ has quit IRC | 08:11 | |
| *** thorst has joined #openstack-infra | 08:11 | |
| *** egonzalez90 has joined #openstack-infra | 08:11 | |
| *** otter768_ has joined #openstack-infra | 08:12 | |
| *** esker has joined #openstack-infra | 08:12 | |
| *** matrohon has joined #openstack-infra | 08:12 | |
| *** otter768_ has quit IRC | 08:14 | |
| *** otter76__ has joined #openstack-infra | 08:14 | |
| *** otter768 has quit IRC | 08:14 | |
| *** ildikov has joined #openstack-infra | 08:15 | |
| openstackgerrit | Einst Crazy proposed openstack-infra/zuul: Use assertTrue/False instead of assertEqual(T/F) https://review.openstack.org/263575 | 08:15 |
| *** otter768 has joined #openstack-infra | 08:16 | |
| *** e0ne has quit IRC | 08:16 | |
| *** esker has quit IRC | 08:17 | |
| *** otter768 has quit IRC | 08:17 | |
| hparekh | AJaegar, Can you please review https://review.openstack.org/#/c/265030/ ? | 08:17 |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder: Add dib element to generate logical volumes https://review.openstack.org/252041 | 08:17 |
| *** otter768 has joined #openstack-infra | 08:17 | |
| *** otter76__ has quit IRC | 08:18 | |
| *** eliqiao has quit IRC | 08:18 | |
| *** thorst has quit IRC | 08:18 | |
| *** otter768 has quit IRC | 08:19 | |
| *** otter768 has joined #openstack-infra | 08:19 | |
| *** otter768_ has joined #openstack-infra | 08:21 | |
| *** fawadkhaliq has quit IRC | 08:22 | |
| *** fawadkhaliq has joined #openstack-infra | 08:22 | |
| *** otter76__ has joined #openstack-infra | 08:23 | |
| *** otter768 has quit IRC | 08:23 | |
| *** mrmartin has joined #openstack-infra | 08:24 | |
| *** abregman has quit IRC | 08:24 | |
| *** abregman has joined #openstack-infra | 08:25 | |
| *** otter76__ has quit IRC | 08:25 | |
| *** otter768 has joined #openstack-infra | 08:25 | |
| *** otter768_ has quit IRC | 08:25 | |
| *** otter768 has quit IRC | 08:26 | |
| *** Daisy has joined #openstack-infra | 08:27 | |
| *** otter768 has joined #openstack-infra | 08:27 | |
| *** samuelBartel has joined #openstack-infra | 08:27 | |
| *** shardy has joined #openstack-infra | 08:28 | |
| *** otter768_ has joined #openstack-infra | 08:29 | |
| *** otter768 has quit IRC | 08:29 | |
| *** bryan_att has quit IRC | 08:29 | |
| *** _nadya_ has joined #openstack-infra | 08:30 | |
| *** otter768 has joined #openstack-infra | 08:31 | |
| *** otter768_ has quit IRC | 08:31 | |
| *** otter768 has quit IRC | 08:32 | |
| *** otter768 has joined #openstack-infra | 08:33 | |
| *** kzaitsev_mb has joined #openstack-infra | 08:34 | |
| *** otter768_ has joined #openstack-infra | 08:35 | |
| *** markus_z has joined #openstack-infra | 08:35 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack-infra/tripleo-ci: Keep py3.X compatibility for urllib https://review.openstack.org/261182 | 08:36 |
| *** otter768_ has quit IRC | 08:36 | |
| *** otter768_ has joined #openstack-infra | 08:37 | |
| *** hichihara has quit IRC | 08:37 | |
| *** otter768 has quit IRC | 08:37 | |
| openstackgerrit | Merged openstack-infra/project-config: Update for fuel-plugin-xenserver group's name on LP https://review.openstack.org/265879 | 08:38 |
| *** egonzalez90 has quit IRC | 08:38 | |
| *** otter768_ has quit IRC | 08:38 | |
| *** otter768 has joined #openstack-infra | 08:38 | |
| *** otter768 has quit IRC | 08:40 | |
| *** otter768 has joined #openstack-infra | 08:40 | |
| *** markmcclain has joined #openstack-infra | 08:41 | |
| *** otter768_ has joined #openstack-infra | 08:42 | |
| *** ddieterly has joined #openstack-infra | 08:43 | |
| *** otter768_ has quit IRC | 08:44 | |
| *** yaume has joined #openstack-infra | 08:44 | |
| *** otter768 has quit IRC | 08:45 | |
| AJaeger | yolanda: regarding https://review.openstack.org/#/c/265560 - mkdir -p return sucess if the directory already exists, no need to if -d around it. | 08:45 |
| *** markmcclain has quit IRC | 08:45 | |
| openstackgerrit | Merged openstack-infra/project-config: Add releasenotes jobs for os-client-config https://review.openstack.org/262600 | 08:45 |
| yolanda | ++ | 08:46 |
| *** otter768 has joined #openstack-infra | 08:46 | |
| *** zeih has joined #openstack-infra | 08:46 | |
| *** jlanoux has joined #openstack-infra | 08:47 | |
| *** jlanoux has quit IRC | 08:48 | |
| *** otter768 has quit IRC | 08:48 | |
| *** jlanoux has joined #openstack-infra | 08:48 | |
| *** ddieterly has quit IRC | 08:48 | |
| *** otter768 has joined #openstack-infra | 08:48 | |
| *** vgridnev has joined #openstack-infra | 08:50 | |
| *** otter768_ has joined #openstack-infra | 08:50 | |
| openstackgerrit | Merged openstack-infra/project-config: Add gerritbot entry for openstack-ux https://review.openstack.org/265965 | 08:50 |
| openstackgerrit | Merged openstack-infra/project-config: Switch pika related jobs to master oslo.messaging branch https://review.openstack.org/265934 | 08:51 |
| *** dizquierdo has joined #openstack-infra | 08:51 | |
| openstackgerrit | Merged openstack-infra/project-config: Add check-requirements for ekko https://review.openstack.org/265553 | 08:51 |
| *** yamahata has quit IRC | 08:51 | |
| *** otter768_ has quit IRC | 08:51 | |
| *** otter768_ has joined #openstack-infra | 08:52 | |
| *** yamahata has joined #openstack-infra | 08:52 | |
| *** otter768 has quit IRC | 08:53 | |
| *** arxcruz has joined #openstack-infra | 08:53 | |
| *** esker has joined #openstack-infra | 08:53 | |
| *** otter768 has joined #openstack-infra | 08:56 | |
| *** otter768_ has quit IRC | 08:57 | |
| *** ifarkas has joined #openstack-infra | 08:57 | |
| *** dtantsur|afk is now known as dtantsur | 08:57 | |
| *** otter768_ has joined #openstack-infra | 08:58 | |
| *** otter768 has quit IRC | 08:58 | |
| *** esker has quit IRC | 08:58 | |
| *** jistr|doc has quit IRC | 08:59 | |
| *** otter768 has joined #openstack-infra | 08:59 | |
| isaacb | yolanda: I saw your reply in the mailing list. I totally agree that it is better to have the jenkins.default as a template but I'm not sure that it is preferable to load the puppet class with a lot of parameters. Do you have any parameter in mind? | 08:59 |
| *** otter768_ has quit IRC | 09:00 | |
| yolanda | isaacb , we are doing that in other modules such as gerrit. So i'd prefer to parameterize what we can, and set sane defaults | 09:00 |
| *** otter768 has quit IRC | 09:01 | |
| *** otter768_ has joined #openstack-infra | 09:02 | |
| yolanda | i'd parameterize java_args, maxopenfiles, jenkins_args... | 09:02 |
| isaacb | yolanda: ok, I understand the spirit. thanks. | 09:02 |
| *** otter768_ has quit IRC | 09:03 | |
| *** otter768_ has joined #openstack-infra | 09:04 | |
| *** otter768_ has quit IRC | 09:05 | |
| *** zhurong has quit IRC | 09:05 | |
| *** amotoki has quit IRC | 09:05 | |
| *** otter76__ has joined #openstack-infra | 09:05 | |
| *** fhubik has joined #openstack-infra | 09:06 | |
| *** dtardivel has joined #openstack-infra | 09:06 | |
| *** amotoki has joined #openstack-infra | 09:07 | |
| *** otter7___ has joined #openstack-infra | 09:07 | |
| *** otter76__ has quit IRC | 09:07 | |
| *** amotoki has quit IRC | 09:07 | |
| *** jordanP has joined #openstack-infra | 09:08 | |
| *** otter____ has joined #openstack-infra | 09:09 | |
| *** daemontool has joined #openstack-infra | 09:11 | |
| *** otter____ has quit IRC | 09:11 | |
| *** otte_____ has joined #openstack-infra | 09:11 | |
| *** otter7___ has quit IRC | 09:11 | |
| *** otte_____ has quit IRC | 09:13 | |
| *** otter____ has joined #openstack-infra | 09:13 | |
| *** otter768 has joined #openstack-infra | 09:15 | |
| *** otter____ has quit IRC | 09:15 | |
| *** thorst has joined #openstack-infra | 09:16 | |
| *** otter768_ has joined #openstack-infra | 09:17 | |
| *** otter768 has quit IRC | 09:17 | |
| *** openstackgerrit has quit IRC | 09:17 | |
| *** openstackgerrit has joined #openstack-infra | 09:17 | |
| *** nmagnezi has quit IRC | 09:18 | |
| *** otter768 has joined #openstack-infra | 09:19 | |
| *** nmagnezi has joined #openstack-infra | 09:19 | |
| *** salv-orlando has joined #openstack-infra | 09:19 | |
| openstackgerrit | Boris Pavlovic proposed openstack-infra/project-config: Move gate-rally-install-devstack-centos to check & gate https://review.openstack.org/266253 | 09:19 |
| *** jistr has joined #openstack-infra | 09:20 | |
| *** otter76__ has joined #openstack-infra | 09:21 | |
| *** kzaitsev_mb has quit IRC | 09:21 | |
| *** otter768_ has quit IRC | 09:22 | |
| *** otter76__ has quit IRC | 09:22 | |
| *** jordanP has quit IRC | 09:22 | |
| *** otter768_ has joined #openstack-infra | 09:23 | |
| *** kzaitsev_mb has joined #openstack-infra | 09:23 | |
| *** otter768 has quit IRC | 09:23 | |
| *** thorst has quit IRC | 09:23 | |
| *** sshnaidm has joined #openstack-infra | 09:24 | |
| *** otter768 has joined #openstack-infra | 09:24 | |
| *** otter768_ has quit IRC | 09:24 | |
| *** otter768_ has joined #openstack-infra | 09:26 | |
| *** otter768 has quit IRC | 09:26 | |
| *** yamahata has quit IRC | 09:27 | |
| *** fhubik is now known as fhubik_brb | 09:27 | |
| *** kzaitsev_mb has quit IRC | 09:28 | |
| *** otter768 has joined #openstack-infra | 09:28 | |
| *** otter768_ has quit IRC | 09:28 | |
| *** otter768 has quit IRC | 09:30 | |
| *** Daisy has quit IRC | 09:30 | |
| *** otter768 has joined #openstack-infra | 09:30 | |
| *** otter768 has quit IRC | 09:31 | |
| *** sputnik13 has joined #openstack-infra | 09:32 | |
| *** aeng_ has quit IRC | 09:32 | |
| *** ggillies has quit IRC | 09:32 | |
| *** camunoz has quit IRC | 09:32 | |
| *** otter768 has joined #openstack-infra | 09:32 | |
| *** sflanigan has quit IRC | 09:32 | |
| *** pahuang has quit IRC | 09:32 | |
| *** jordanP has joined #openstack-infra | 09:33 | |
| *** otter768 has quit IRC | 09:34 | |
| *** otter768 has joined #openstack-infra | 09:34 | |
| *** esker has joined #openstack-infra | 09:35 | |
| *** derekh has joined #openstack-infra | 09:35 | |
| *** otter768 has quit IRC | 09:35 | |
| *** otter768 has joined #openstack-infra | 09:36 | |
| *** larainema has quit IRC | 09:38 | |
| *** otter768_ has joined #openstack-infra | 09:38 | |
| *** otter768_ has quit IRC | 09:39 | |
| *** otter768_ has joined #openstack-infra | 09:40 | |
| *** esker has quit IRC | 09:40 | |
| *** otter768 has quit IRC | 09:40 | |
| *** larainema has joined #openstack-infra | 09:41 | |
| *** otter768_ has quit IRC | 09:41 | |
| *** otter768 has joined #openstack-infra | 09:41 | |
| *** otter768 has quit IRC | 09:43 | |
| *** otter768 has joined #openstack-infra | 09:44 | |
| openstackgerrit | Boris Pavlovic proposed openstack/requirements: Change OSprofiler constraints to >=0.4.0 https://review.openstack.org/266264 | 09:44 |
| *** ggillies has joined #openstack-infra | 09:44 | |
| *** ddieterly has joined #openstack-infra | 09:44 | |
| *** sflanigan has joined #openstack-infra | 09:44 | |
| *** sflanigan has joined #openstack-infra | 09:44 | |
| *** yolanda has quit IRC | 09:44 | |
| *** pahuang has joined #openstack-infra | 09:44 | |
| *** aeng_ has joined #openstack-infra | 09:45 | |
| *** camunoz has joined #openstack-infra | 09:45 | |
| *** otter768 has quit IRC | 09:45 | |
| *** otter768 has joined #openstack-infra | 09:45 | |
| *** otter768 has quit IRC | 09:47 | |
| *** rguillebert has joined #openstack-infra | 09:47 | |
| *** otter768 has joined #openstack-infra | 09:47 | |
| *** e0ne has joined #openstack-infra | 09:48 | |
| *** ddieterly has quit IRC | 09:48 | |
| *** tzn has joined #openstack-infra | 09:49 | |
| *** otter768 has quit IRC | 09:49 | |
| *** yolanda has joined #openstack-infra | 09:49 | |
| *** sfinucan has joined #openstack-infra | 09:49 | |
| *** otter768 has joined #openstack-infra | 09:49 | |
| *** fhubik_brb is now known as fhubik | 09:50 | |
| *** otter768 has quit IRC | 09:51 | |
| *** clayton has quit IRC | 09:51 | |
| *** salv-orlando has quit IRC | 09:51 | |
| *** fawadkhaliq has quit IRC | 09:51 | |
| *** otter768 has joined #openstack-infra | 09:51 | |
| *** mrmartin has quit IRC | 09:51 | |
| *** clayton has joined #openstack-infra | 09:51 | |
| *** fawadkhaliq has joined #openstack-infra | 09:52 | |
| *** keedya has quit IRC | 09:52 | |
| *** darvon has quit IRC | 09:52 | |
| *** slagle has quit IRC | 09:52 | |
| *** sileht has quit IRC | 09:52 | |
| *** rossella_s has quit IRC | 09:53 | |
| *** davideag_ has joined #openstack-infra | 09:53 | |
| *** rcarrillocruz has quit IRC | 09:53 | |
| *** keedya has joined #openstack-infra | 09:53 | |
| *** grassass has quit IRC | 09:53 | |
| *** otter768_ has joined #openstack-infra | 09:53 | |
| *** acabot has quit IRC | 09:53 | |
| *** ParsectiX has quit IRC | 09:53 | |
| *** derekh has quit IRC | 09:54 | |
| *** baoli_ has joined #openstack-infra | 09:54 | |
| *** darvon has joined #openstack-infra | 09:54 | |
| *** slagle has joined #openstack-infra | 09:54 | |
| *** yolanda has quit IRC | 09:54 | |
| *** otter768_ has quit IRC | 09:54 | |
| *** gongysh has quit IRC | 09:55 | |
| *** cbader_ has joined #openstack-infra | 09:55 | |
| *** derekh has joined #openstack-infra | 09:55 | |
| *** otter768_ has joined #openstack-infra | 09:55 | |
| *** davideagnello has quit IRC | 09:56 | |
| *** mdenny has quit IRC | 09:56 | |
| *** cbader has quit IRC | 09:56 | |
| *** mdenny has joined #openstack-infra | 09:56 | |
| *** otter768 has quit IRC | 09:56 | |
| *** baoli has quit IRC | 09:56 | |
| *** otter768 has joined #openstack-infra | 09:57 | |
| *** salv-orlando has joined #openstack-infra | 09:57 | |
| *** rcarrillocruz has joined #openstack-infra | 09:58 | |
| *** otter768 has quit IRC | 09:59 | |
| *** amotoki has joined #openstack-infra | 09:59 | |
| *** otter768_ has quit IRC | 09:59 | |
| *** yolanda has joined #openstack-infra | 10:00 | |
| *** grassass has joined #openstack-infra | 10:00 | |
| *** otter768 has joined #openstack-infra | 10:01 | |
| *** fawadkhaliq has quit IRC | 10:01 | |
| *** fawadkhaliq has joined #openstack-infra | 10:01 | |
| *** bharathm has quit IRC | 10:02 | |
| *** acabot has joined #openstack-infra | 10:02 | |
| *** rossella_s has joined #openstack-infra | 10:02 | |
| *** sileht has joined #openstack-infra | 10:02 | |
| *** markvoelker has joined #openstack-infra | 10:02 | |
| *** otter768 has quit IRC | 10:04 | |
| *** otter768_ has joined #openstack-infra | 10:04 | |
| *** jordanP has quit IRC | 10:04 | |
| *** ParsectiX has joined #openstack-infra | 10:05 | |
| *** otter768_ has quit IRC | 10:06 | |
| *** otter768 has joined #openstack-infra | 10:07 | |
| *** markvoelker has quit IRC | 10:07 | |
| *** otter768 has quit IRC | 10:08 | |
| *** eliqiao has joined #openstack-infra | 10:08 | |
| *** otter768 has joined #openstack-infra | 10:08 | |
| *** eliqiao has quit IRC | 10:09 | |
| *** salv-orlando has quit IRC | 10:10 | |
| *** otter768_ has joined #openstack-infra | 10:10 | |
| *** otter768 has quit IRC | 10:10 | |
| *** eliqiao has joined #openstack-infra | 10:10 | |
| Odd_Bloke | clarkb: So I've got something working for my configuration thing; '- project:\n ...\n configuration:\n !include: configuration.yaml.inc' and then {configuration[config_value]} in shell templates. | 10:11 |
| Odd_Bloke | clarkb: So thanks again for the pointer to putting it on the project, this has made my life much easier. :) | 10:12 |
| *** otter768_ has quit IRC | 10:12 | |
| *** otter768 has joined #openstack-infra | 10:12 | |
| *** otter768 has quit IRC | 10:14 | |
| Zara | anteaya: we don't have that yet, which is really annoying. It's work in progress (story here: https://storyboard.openstack.org/#!/story/2000415 ) at the moment I'm linking things in the comments as a workaround. | 10:15 |
| *** devvesa has joined #openstack-infra | 10:15 | |
| *** rossella_s has quit IRC | 10:15 | |
| *** sputnik13 has quit IRC | 10:16 | |
| *** denisra_ has joined #openstack-infra | 10:16 | |
| *** derekh has quit IRC | 10:16 | |
| *** acabot has quit IRC | 10:16 | |
| *** esker has joined #openstack-infra | 10:16 | |
| *** otter768_ has joined #openstack-infra | 10:16 | |
| *** sputnik13 has joined #openstack-infra | 10:17 | |
| *** grassass has quit IRC | 10:17 | |
| *** sputnik13 has quit IRC | 10:17 | |
| Zara | anteaya: I can bump it up my todo list; I generally wait until at least one other person mentions something that annoys me. on those lines, I'd also like stories to have some kind of +1 feature so people can register that its affected them... so many things! | 10:17 |
| *** otter768_ has quit IRC | 10:18 | |
| *** otter768 has joined #openstack-infra | 10:18 | |
| *** derekh has joined #openstack-infra | 10:19 | |
| *** Verilium_ has joined #openstack-infra | 10:19 | |
| *** denisra has quit IRC | 10:19 | |
| *** Verilium has quit IRC | 10:19 | |
| *** sputnik13 has joined #openstack-infra | 10:19 | |
| *** ParsectiX has quit IRC | 10:20 | |
| *** otter768 has quit IRC | 10:20 | |
| *** otter768_ has joined #openstack-infra | 10:20 | |
| *** thorst has joined #openstack-infra | 10:21 | |
| *** esker has quit IRC | 10:21 | |
| *** otter768_ has quit IRC | 10:21 | |
| *** otter768 has joined #openstack-infra | 10:22 | |
| *** acabot has joined #openstack-infra | 10:22 | |
| *** rossella_s has joined #openstack-infra | 10:22 | |
| *** grassass has joined #openstack-infra | 10:22 | |
| *** otter768_ has joined #openstack-infra | 10:24 | |
| *** otter76__ has joined #openstack-infra | 10:25 | |
| *** otter768_ has quit IRC | 10:26 | |
| *** otter768 has quit IRC | 10:26 | |
| *** kzaitsev_mb has joined #openstack-infra | 10:27 | |
| *** otter76__ has quit IRC | 10:27 | |
| *** yolanda has quit IRC | 10:27 | |
| *** otter768 has joined #openstack-infra | 10:28 | |
| *** thorst has quit IRC | 10:28 | |
| *** salv-orlando has joined #openstack-infra | 10:28 | |
| *** salv-orlando has quit IRC | 10:28 | |
| *** otter768 has quit IRC | 10:29 | |
| *** mrmartin has joined #openstack-infra | 10:29 | |
| *** yolanda has joined #openstack-infra | 10:29 | |
| *** otter768 has joined #openstack-infra | 10:29 | |
| *** ParsectiX has joined #openstack-infra | 10:32 | |
| *** kzaitsev_mb has quit IRC | 10:32 | |
| *** yolanda has quit IRC | 10:32 | |
| *** weshay_xchat has joined #openstack-infra | 10:34 | |
| *** yolanda has joined #openstack-infra | 10:34 | |
| *** otter768 has quit IRC | 10:34 | |
| openstackgerrit | Boris Pavlovic proposed openstack/requirements: Change OSprofiler constraints to >=0.4.0 https://review.openstack.org/266264 | 10:34 |
| *** jordanP has joined #openstack-infra | 10:35 | |
| *** yolanda has quit IRC | 10:35 | |
| *** electrofelix has joined #openstack-infra | 10:36 | |
| *** yolanda has joined #openstack-infra | 10:36 | |
| openstackgerrit | Merged openstack/diskimage-builder: Add pip-and-virtualenv element https://review.openstack.org/239561 | 10:37 |
| *** kzaitsev_mb has joined #openstack-infra | 10:42 | |
| *** ildikov has quit IRC | 10:43 | |
| *** dizquierdo has quit IRC | 10:44 | |
| *** ildikov has joined #openstack-infra | 10:44 | |
| *** ddieterly has joined #openstack-infra | 10:45 | |
| *** ldnunes has joined #openstack-infra | 10:48 | |
| *** ddieterly has quit IRC | 10:50 | |
| *** baoli has joined #openstack-infra | 10:52 | |
| *** fawadkhaliq has quit IRC | 10:52 | |
| *** baoli_ has quit IRC | 10:55 | |
| *** jaosorior has quit IRC | 10:55 | |
| *** jaosorior has joined #openstack-infra | 10:55 | |
| *** weshay_xchat has quit IRC | 10:56 | |
| *** coolsvap is now known as coolsvap|away | 10:56 | |
| *** esker has joined #openstack-infra | 10:58 | |
| *** esker has quit IRC | 11:02 | |
| *** _nadya_ has quit IRC | 11:03 | |
| *** dims has joined #openstack-infra | 11:05 | |
| *** yolanda has quit IRC | 11:05 | |
| *** yolanda has joined #openstack-infra | 11:06 | |
| openstackgerrit | Merged openstack-infra/project-config: Add puppet-midonet to OpenStack infrastructure https://review.openstack.org/260093 | 11:07 |
| *** sputnik13 has quit IRC | 11:09 | |
| openstackgerrit | Ivan Kolodyazhny proposed openstack-infra/project-config: Add functional test job for python-brick-cinderclient-ext https://review.openstack.org/265925 | 11:09 |
| *** dizquierdo has joined #openstack-infra | 11:14 | |
| *** weshay_xchat has joined #openstack-infra | 11:14 | |
| *** sputnik13 has joined #openstack-infra | 11:15 | |
| *** MarkAtwood has joined #openstack-infra | 11:15 | |
| *** vivekd has quit IRC | 11:17 | |
| *** openstackgerrit has quit IRC | 11:17 | |
| *** openstackgerrit has joined #openstack-infra | 11:17 | |
| *** fhubik is now known as fhubik_brb | 11:19 | |
| *** ociuhandu has quit IRC | 11:19 | |
| *** MarkAtwood has quit IRC | 11:19 | |
| *** zeih has quit IRC | 11:21 | |
| *** aysyd has joined #openstack-infra | 11:25 | |
| *** thorst has joined #openstack-infra | 11:26 | |
| *** jcoufal has joined #openstack-infra | 11:26 | |
| *** lucas-dinner is now known as lucasagomes | 11:27 | |
| *** jlanoux has quit IRC | 11:29 | |
| *** jlanoux has joined #openstack-infra | 11:30 | |
| *** thorst has quit IRC | 11:33 | |
| *** ramineni_ has quit IRC | 11:35 | |
| *** zeih has joined #openstack-infra | 11:38 | |
| *** ildikov has quit IRC | 11:40 | |
| *** vivekd has joined #openstack-infra | 11:40 | |
| *** kzaitsev_mb has quit IRC | 11:41 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Remove Nova object store service https://review.openstack.org/266119 | 11:41 |
| *** vivekd has quit IRC | 11:41 | |
| *** vivekd has joined #openstack-infra | 11:42 | |
| *** gongysh has joined #openstack-infra | 11:44 | |
| *** amotoki has quit IRC | 11:46 | |
| *** jlanoux_ has joined #openstack-infra | 11:47 | |
| *** jlanoux has quit IRC | 11:47 | |
| *** ddieterly has joined #openstack-infra | 11:47 | |
| *** sputnik13 has quit IRC | 11:48 | |
| *** salv-orl_ has joined #openstack-infra | 11:49 | |
| *** jaosorior has quit IRC | 11:49 | |
| *** jaosorior has joined #openstack-infra | 11:50 | |
| *** amotoki has joined #openstack-infra | 11:50 | |
| *** ddieterly has quit IRC | 11:51 | |
| *** fhubik_brb is now known as fhubik | 11:52 | |
| *** vivekd has quit IRC | 11:54 | |
| *** ihrachys has joined #openstack-infra | 11:54 | |
| *** _nadya_ has joined #openstack-infra | 11:54 | |
| *** ociuhandu has joined #openstack-infra | 11:59 | |
| *** [1]Thelo has joined #openstack-infra | 12:01 | |
| *** sfinucan_ has joined #openstack-infra | 12:01 | |
| *** pblaho has joined #openstack-infra | 12:01 | |
| openstackgerrit | Merged openstack-infra/project-config: congress: disable tempest plugin for stable branches https://review.openstack.org/258799 | 12:01 |
| *** amotoki has quit IRC | 12:02 | |
| *** markvoelker has joined #openstack-infra | 12:03 | |
| *** Thelo has quit IRC | 12:04 | |
| *** [1]Thelo is now known as Thelo | 12:04 | |
| *** fhubik is now known as fhubik_brb | 12:04 | |
| *** sfinucan has quit IRC | 12:04 | |
| *** mrmartin has quit IRC | 12:06 | |
| *** jcoufal has quit IRC | 12:07 | |
| *** kzaitsev_mb has joined #openstack-infra | 12:07 | |
| *** mrmartin has joined #openstack-infra | 12:08 | |
| *** markvoelker has quit IRC | 12:08 | |
| openstackgerrit | Miles Gould proposed openstack/requirements: Bump minimum python-ironicclient version https://review.openstack.org/266320 | 12:09 |
| *** vgridnev has quit IRC | 12:10 | |
| *** vgridnev has joined #openstack-infra | 12:11 | |
| *** mrmartin has quit IRC | 12:11 | |
| *** notnownikki has quit IRC | 12:13 | |
| openstackgerrit | Zara proposed openstack-infra/storyboard: Add rudimentary templates for tags https://review.openstack.org/266327 | 12:19 |
| openstackgerrit | Slava Tabolin proposed openstack-infra/project-config: Add project vmware-dvs https://review.openstack.org/266331 | 12:24 |
| *** eliqiao has quit IRC | 12:24 | |
| *** eliqiao has joined #openstack-infra | 12:25 | |
| *** _nadya_ has quit IRC | 12:25 | |
| *** EricGonczer_ has joined #openstack-infra | 12:25 | |
| openstackgerrit | Zara proposed openstack-infra/storyboard: Add rudimentary templates for tags https://review.openstack.org/266327 | 12:27 |
| *** _nadya_ has joined #openstack-infra | 12:28 | |
| *** jcoufal has joined #openstack-infra | 12:29 | |
| *** mgould has joined #openstack-infra | 12:29 | |
| mgould | morning everyone! Can anyone review this requirements patch? https://review.openstack.org/#/c/266320/ | 12:30 |
| *** vgridnev has quit IRC | 12:30 | |
| *** otter768 has joined #openstack-infra | 12:30 | |
| *** ildikov has joined #openstack-infra | 12:32 | |
| *** fhubik_brb is now known as fhubik | 12:33 | |
| *** aopincar_ has quit IRC | 12:34 | |
| *** otter768 has quit IRC | 12:35 | |
| TheJulia | clarkb: well, funtionatl test creating the databases it needs for any execution, although if there is a particular way you recommend database creation, I'd be more than happy to change things. | 12:38 |
| *** thorst has joined #openstack-infra | 12:39 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Remove Nova object store service https://review.openstack.org/266119 | 12:40 |
| *** dizquierdo has quit IRC | 12:41 | |
| dims | AJaeger : thanks for the reviews. hoping this ^^ would do the trick | 12:41 |
| *** vgridnev has joined #openstack-infra | 12:41 | |
| *** d0ugal has quit IRC | 12:42 | |
| *** d0ugal has joined #openstack-infra | 12:43 | |
| *** d0ugal is now known as Guest58385 | 12:43 | |
| *** aopincar_ has joined #openstack-infra | 12:43 | |
| *** slava__ has joined #openstack-infra | 12:43 | |
| *** salv-orl_ has quit IRC | 12:43 | |
| *** Guest58385 is now known as d0ugal | 12:45 | |
| *** d0ugal has quit IRC | 12:45 | |
| *** d0ugal has joined #openstack-infra | 12:45 | |
| *** esikachev has quit IRC | 12:46 | |
| *** ddieterly has joined #openstack-infra | 12:47 | |
| *** esikachev has joined #openstack-infra | 12:50 | |
| clarkb | TheJulia every one of oyr test images makes a DB just for ironic testing. But since we have started allowing tests to create their own DBs the special ironoc DB is not required | 12:52 |
| *** ddieterly has quit IRC | 12:52 | |
| clarkb | it is possible ironic fixed it on their end and never cleaned up the extra database that is created for it | 12:52 |
| clarkb | in that case we can just stop making the extra db | 12:53 |
| TheJulia | clarkb: can you point me to a reference implementation of this? | 12:53 |
| clarkb | https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/slave_db.pp#n46 is where we create the db | 12:55 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Remove Nova object store service https://review.openstack.org/266119 | 12:55 |
| clarkb | not sure what reference implementation there would be, ironic is the only thing that should have ever used it | 12:56 |
| clarkb | there is one under postgres too | 12:57 |
| *** dprince has joined #openstack-infra | 12:58 | |
| TheJulia | clarkb: I was wondering what the intent of that was.... I'm not sure I've ever seen that database used anywhere. Hmm | 12:59 |
| *** kgiusti has joined #openstack-infra | 13:00 | |
| *** kgiusti has left #openstack-infra | 13:01 | |
| clarkb | I think it was for migration testing. devananda insisted ironic couldnt use the existing test db we were already creating because nova/ironic have separate schemas that had to be upgraded side by side or something | 13:01 |
| clarkb | (I'm just trying to cleanup our db stuff so that when hpcloud goes away the amlunt of effort to run unittests on more than just rax is minimal) | 13:02 |
| *** kgiusti has joined #openstack-infra | 13:02 | |
| *** amuller has joined #openstack-infra | 13:03 | |
| *** MarkAtwood has joined #openstack-infra | 13:03 | |
| clarkb | speaking of, I think we can dib bare-* images prior to using bindep if we add a small db setup step to the unittest jobs | 13:04 |
| *** markvoelker has joined #openstack-infra | 13:04 | |
| clarkb | which will be a good interim step to remain multicloud on those jpbs | 13:04 |
| *** Keedya_ has joined #openstack-infra | 13:04 | |
| *** fhubik is now known as fhubik_brb | 13:05 | |
| *** rfolco has joined #openstack-infra | 13:06 | |
| *** jcoufal has quit IRC | 13:07 | |
| *** Keedya_ has quit IRC | 13:08 | |
| *** baoli has quit IRC | 13:08 | |
| *** MarkAtwood has quit IRC | 13:08 | |
| *** fhubik_brb is now known as fhubik | 13:09 | |
| *** baoli has joined #openstack-infra | 13:09 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/elastic-recheck: Query for Neutron Gate failures in TestGettingAddress https://review.openstack.org/266356 | 13:11 |
| *** derekh has quit IRC | 13:11 | |
| *** grassass has quit IRC | 13:11 | |
| dims | sdague : mriedem_away : just spotted this in gate ^^^ | 13:11 |
| *** hashar has joined #openstack-infra | 13:12 | |
| *** ddieterly has joined #openstack-infra | 13:12 | |
| *** MIDENN_ has joined #openstack-infra | 13:12 | |
| *** markvoelker has quit IRC | 13:15 | |
| *** filler has quit IRC | 13:15 | |
| *** lindsayk- has quit IRC | 13:15 | |
| *** mriedem_away has quit IRC | 13:15 | |
| *** mdenny has quit IRC | 13:15 | |
| *** f1ller has joined #openstack-infra | 13:15 | |
| *** lindsayk has joined #openstack-infra | 13:15 | |
| *** derekh has joined #openstack-infra | 13:16 | |
| *** grassass has joined #openstack-infra | 13:17 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/ansible-puppet: Update to use blocks from ansible 2.0 https://review.openstack.org/266361 | 13:17 |
| sdague | dims: you don't need to add the voting:1 | 13:18 |
| sdague | we do that automatically for all the queries | 13:19 |
| dims | sdague : ah ok | 13:19 |
| *** fhubik has quit IRC | 13:19 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/elastic-recheck: Query for Neutron Gate failures in TestGettingAddress https://review.openstack.org/266356 | 13:22 |
| dims | sdague : fixed ^^ | 13:22 |
| *** oomichi is now known as oomichi_away | 13:23 | |
| *** yolanda has quit IRC | 13:24 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config: Add script to make ansible groups from patterns https://review.openstack.org/265647 | 13:25 |
| *** yolanda has joined #openstack-infra | 13:26 | |
| sdague | dims: the problem with queries like that is they are 100% symptom and no root cause | 13:26 |
| *** yolanda has quit IRC | 13:26 | |
| *** rhallisey has joined #openstack-infra | 13:27 | |
| *** fawadkhaliq has joined #openstack-infra | 13:28 | |
| *** yolanda has joined #openstack-infra | 13:28 | |
| *** trown|outttypeww is now known as trown | 13:28 | |
| dims | sdague : ack abandoning if it is not useful | 13:29 |
| openstackgerrit | Zara proposed openstack-infra/storyboard: Rudimentary Templates for Tasks https://review.openstack.org/264255 | 13:29 |
| dims | hopefully the bug in neutron is enough | 13:30 |
| *** erlon has joined #openstack-infra | 13:30 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 13:31 | |
| *** edmondsw has joined #openstack-infra | 13:32 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/ansible-puppet: Clean up codestyle in conditionals https://review.openstack.org/266369 | 13:32 |
| openstackgerrit | Slava Tabolin proposed openstack-infra/project-config: Add project vmware-dvs https://review.openstack.org/266331 | 13:32 |
| mordred | SamYaple: ^^ I addressed your comments in https://review.openstack.org/#/c/251580 in a follow-on patch ^^ | 13:32 |
| *** EricGonc_ has joined #openstack-infra | 13:34 | |
| *** fawadkhaliq has quit IRC | 13:34 | |
| sdague | this looks suspicious - http://logs.openstack.org/95/262395/7/check/gate-tempest-dsvm-neutron-full/4293c5f/logs/screen-q-svc.txt.gz?level=INFO#_2016-01-12_07_29_54_417 - the subnet that we're supposed to be using isn't there when the delete happens | 13:35 |
| *** markvoelker has joined #openstack-infra | 13:35 | |
| mordred | clarkb: ++ | 13:38 |
| *** EricGonczer_ has quit IRC | 13:38 | |
| *** abregman has quit IRC | 13:38 | |
| *** eliqiao_ has joined #openstack-infra | 13:39 | |
| *** regXboi has joined #openstack-infra | 13:39 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 13:40 | |
| *** eliqiao has quit IRC | 13:41 | |
| *** dslev has joined #openstack-infra | 13:42 | |
| *** eliqiao_ is now known as eliqiao | 13:42 | |
| mordred | clarkb: feel like https://review.openstack.org/#/c/251580/ ? | 13:43 |
| *** aopincar_ has quit IRC | 13:45 | |
| stevemar | htruta: looks like it's failing the legacy tests | 13:46 |
| *** larainema has quit IRC | 13:48 | |
| openstackgerrit | Slava Tabolin proposed openstack-infra/project-config: Add project vmware-dvs https://review.openstack.org/266331 | 13:48 |
| *** hichihara has joined #openstack-infra | 13:49 | |
| SamYaple | mordred: sounds good | 13:50 |
| *** cdent has joined #openstack-infra | 13:51 | |
| *** dkranz has joined #openstack-infra | 13:53 | |
| *** tlian has joined #openstack-infra | 13:55 | |
| *** miqui has joined #openstack-infra | 13:56 | |
| *** aysyd has quit IRC | 14:00 | |
| *** aysyd has joined #openstack-infra | 14:01 | |
| *** akshai has joined #openstack-infra | 14:02 | |
| *** burgerk has joined #openstack-infra | 14:02 | |
| *** dslev has quit IRC | 14:02 | |
| igorbelikov | Folks, can you please review https://review.openstack.org/#/c/261540/, it didn't receive any attention since the year started:) AJaeger: looks like your comments were implemented, can you take a look once again? | 14:03 |
| *** rlandy has joined #openstack-infra | 14:03 | |
| *** gordc has joined #openstack-infra | 14:04 | |
| *** dkranz has quit IRC | 14:05 | |
| *** mtanino has joined #openstack-infra | 14:07 | |
| *** danpb has joined #openstack-infra | 14:08 | |
| danpb | before xmas infra created the os-vif project for me. can someone here now add me to the os-vif-core team so I can do reviews and add other core team members myself :-) https://review.openstack.org/#/admin/groups/1175,members | 14:09 |
| *** adreznec_ is now known as adreznec | 14:09 | |
| *** jcoufal has joined #openstack-infra | 14:10 | |
| anteaya | Zara: it affects me, do bump, my thanks | 14:10 |
| mordred | WOOT! | 14:11 |
| mordred | https://pypi.python.org/pypi/ansible | 14:11 |
| *** salv-orlando has joined #openstack-infra | 14:11 | |
| mordred | ansible 2.0 is out | 14:11 |
| *** dkranz has joined #openstack-infra | 14:11 | |
| *** yamamoto_ has quit IRC | 14:11 | |
| anteaya | clarkb: ah okay so after wednesday then, thanks | 14:11 |
| *** piet has joined #openstack-infra | 14:11 | |
| *** xyang1 has joined #openstack-infra | 14:11 | |
| anteaya | mordred: yay | 14:12 |
| mordred | infra can now start writing playbooks that use the shade-based openstack modules in ansible 2.0 | 14:12 |
| mordred | Shrews: ^^ woot | 14:12 |
| *** krtaylor has quit IRC | 14:13 | |
| mordred | Shrews: also, ZOMG NOW ALL THE PEOPLE ARE GOING TO START SENDING BUGS!!!! | 14:13 |
| Shrews | mordred: umm... yay? | 14:14 |
| *** julim has joined #openstack-infra | 14:14 | |
| odyssey4me | :) | 14:14 |
| *** larainema has joined #openstack-infra | 14:15 | |
| *** jsavak has joined #openstack-infra | 14:16 | |
| *** ddieterly has quit IRC | 14:17 | |
| *** tongli has joined #openstack-infra | 14:18 | |
| *** ZZelle has joined #openstack-infra | 14:18 | |
| *** amotoki has joined #openstack-infra | 14:20 | |
| *** eharney has joined #openstack-infra | 14:21 | |
| *** yolanda has quit IRC | 14:21 | |
| *** amotoki has quit IRC | 14:21 | |
| *** jsavak has quit IRC | 14:21 | |
| *** rossella_s has quit IRC | 14:22 | |
| *** yolanda has joined #openstack-infra | 14:22 | |
| *** ajmiller has joined #openstack-infra | 14:22 | |
| *** rossella_s has joined #openstack-infra | 14:22 | |
| *** jsavak has joined #openstack-infra | 14:23 | |
| *** zz_dimtruck is now known as dimtruck | 14:23 | |
| *** krtaylor has joined #openstack-infra | 14:24 | |
| *** mriedem has joined #openstack-infra | 14:25 | |
| *** mtanino has quit IRC | 14:26 | |
| *** shardy is now known as shardy_mtg | 14:27 | |
| *** _nadya_ has quit IRC | 14:28 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Allow inventory filering by cloud name https://review.openstack.org/263513 | 14:29 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade: Allow inventory filtering by cloud name https://review.openstack.org/263513 | 14:29 |
| *** esker has joined #openstack-infra | 14:29 | |
| *** peristeri has joined #openstack-infra | 14:29 | |
| Shrews | nibalizer: ^^^^ see the changes to the unit tests i made for you | 14:29 |
| *** otter768 has joined #openstack-infra | 14:31 | |
| *** shardy_mtg has quit IRC | 14:32 | |
| *** esker has quit IRC | 14:32 | |
| *** esker has joined #openstack-infra | 14:33 | |
| *** shardy_mtg has joined #openstack-infra | 14:34 | |
| *** dimtruck is now known as zz_dimtruck | 14:34 | |
| *** ajmiller has quit IRC | 14:34 | |
| *** otter768 has quit IRC | 14:36 | |
| openstackgerrit | Slava Tabolin proposed openstack-infra/project-config: Add project vmware-dvs https://review.openstack.org/266331 | 14:37 |
| *** annegentle has joined #openstack-infra | 14:37 | |
| *** esker has quit IRC | 14:38 | |
| *** dslev has joined #openstack-infra | 14:39 | |
| openstackgerrit | Mikhail S Medvedev proposed openstack-infra/puppet-ciwatch: Add initial implementation to deploy ciwatch https://review.openstack.org/238606 | 14:39 |
| *** amotoki has joined #openstack-infra | 14:40 | |
| *** jaosorior has quit IRC | 14:40 | |
| *** salv-orlando has quit IRC | 14:42 | |
| *** hichihara has quit IRC | 14:42 | |
| *** hichihara has joined #openstack-infra | 14:42 | |
| *** burgerk has quit IRC | 14:43 | |
| *** ddieterly has joined #openstack-infra | 14:44 | |
| *** ddieterly has quit IRC | 14:45 | |
| *** ddieterly has joined #openstack-infra | 14:45 | |
| anteaya | so my single sign in logging didn't work for me trying to log into gerrit this morning, it redirected me to the launchpad page, I input my email and password, I was redirected to the Personal Data Request page and I clicked Yes, log me in | 14:45 |
| anteaya | the gerbils worked hard to no avail and then gave up | 14:46 |
| *** woodster_ has joined #openstack-infra | 14:46 | |
| anteaya | no error, nothing | 14:46 |
| anteaya | the Personal Data Request page has returned to me with the Yes, log me in button greyed out, refreshing the page produces no different action | 14:46 |
| *** bgmccollum has left #openstack-infra | 14:47 | |
| *** amotoki has quit IRC | 14:47 | |
| *** hichihara has quit IRC | 14:47 | |
| anteaya | trying again with review.openstack.org again, I am not signed in, taken straight to the Personal Data Request page and signed in now | 14:48 |
| anteaya | odd | 14:48 |
| *** zeih_ has joined #openstack-infra | 14:48 | |
| *** rossella_s has quit IRC | 14:48 | |
| *** zeih has quit IRC | 14:48 | |
| *** dslev has quit IRC | 14:49 | |
| *** links has quit IRC | 14:50 | |
| *** hichihara has joined #openstack-infra | 14:51 | |
| *** amrith is now known as _amrith_ | 14:52 | |
| *** rossella_s has joined #openstack-infra | 14:53 | |
| *** salv-orlando has joined #openstack-infra | 14:54 | |
| *** salv-orlando has quit IRC | 14:56 | |
| *** jsavak has quit IRC | 14:56 | |
| *** salv-orlando has joined #openstack-infra | 14:56 | |
| *** annegentle has quit IRC | 14:56 | |
| *** annegentle has joined #openstack-infra | 14:57 | |
| *** hichihara has quit IRC | 14:58 | |
| *** dslev has joined #openstack-infra | 14:58 | |
| *** hichihara has joined #openstack-infra | 14:58 | |
| *** akshai has quit IRC | 14:58 | |
| mordred | anteaya: I blame jaypipes - but I have no reason to do so | 14:58 |
| anteaya | mordred: okay | 14:59 |
| anteaya | jaypipes: I do not blame you, for what that's worth | 14:59 |
| *** akshai has joined #openstack-infra | 14:59 | |
| Zara | does openstack have a preferred place for openstack-related ansible playbooks to go up? I've asked around a couple of times but I think I've picked bad moments. | 15:00 |
| *** jlanoux_ has quit IRC | 15:01 | |
| *** hichihara has quit IRC | 15:01 | |
| *** jlanoux has joined #openstack-infra | 15:01 | |
| anteaya | mordred: hey might you have a response to the where should storyboard ansible things go? | 15:02 |
| *** hichihara has joined #openstack-infra | 15:02 | |
| *** markmcclain has joined #openstack-infra | 15:02 | |
| *** doug-fish has joined #openstack-infra | 15:02 | |
| odyssey4me | Zara it all depends on what the playbooks are trying to achieve. There is the OpenStack-Ansible project, which specifically deploys OpenStack with Ansible. It has a series of role repositories focused on deploying different OpenStack and related services. There are also -infra related playbooks and roles around. | 15:02 |
| odyssey4me | Zara if I understand you correctly you have a set of Ansible playbooks which deploys Storyboard? | 15:03 |
| *** annegentle has quit IRC | 15:03 | |
| mordred | uhgm | 15:03 |
| *** markmcclain has quit IRC | 15:03 | |
| *** markmcclain1 has joined #openstack-infra | 15:03 | |
| Zara | oddyssey4me: ah, thanks. yes, I think so ( https://galaxy.ansible.com/detail#/role/6187 ) | 15:04 |
| fungi | this sounds probably more along the lines of pabelanger's question about where to put his ansible-nodepool role | 15:04 |
| *** annegentle has joined #openstack-infra | 15:04 | |
| Zara | * odyssey4me^ | 15:04 |
| *** mtanino has joined #openstack-infra | 15:04 | |
| *** markmcclain1 is now known as markmcclain | 15:04 | |
| anteaya | trying to connect to the wiki is taking me rather a long time | 15:04 |
| mordred | fungi, Zara ++ | 15:04 |
| odyssey4me | yeah, so considering that storyboard is more around deploying openstack-infra, it would seem to me that this should reside in the openstack-infra namespace, IMO | 15:05 |
| *** salv-orl_ has joined #openstack-infra | 15:05 | |
| mordred | I thnik that's where ansible-nodepool wound up | 15:05 |
| *** salv-orlando has quit IRC | 15:05 | |
| *** esker has joined #openstack-infra | 15:05 | |
| mordred | right? | 15:05 |
| fungi | e.g. we decided that openstack-infra is not a great umbrella for tools which aren't used by the infra team and with no clear path to migration (we deploy most everything with puppet right now and would want an all-or-nothing plan to migrate), so the options are either a new project team or an unofficial project in the openstack git namespace | 15:05 |
| mordred | or, maybe it wound up nowhere | 15:05 |
| *** skolekonov has quit IRC | 15:06 | |
| *** esker has quit IRC | 15:06 | |
| odyssey4me | Zara You could recruit reviewers from openstack-ansible - there may be people interested in assisting with that. | 15:06 |
| *** lucasagomes is now known as lucas-hungry | 15:06 | |
| odyssey4me | mordred yeah, I think it's in the -infra namespace... | 15:06 |
| fungi | i think unofficial project was what happened to ansible-nodepool, but have to check that assertion | 15:06 |
| mordred | nope. openstack | 15:07 |
| mordred | yah | 15:07 |
| *** esker has joined #openstack-infra | 15:07 | |
| mordred | openstack/ansible-role-openstack | 15:07 |
| odyssey4me | ah ok | 15:07 |
| fungi | ansible-role-nodepool yeah | 15:07 |
| fungi | unofficial repo | 15:07 |
| *** burgerk has joined #openstack-infra | 15:07 | |
| mordred | btw - ansible- and ansible-role- as prefixes are things that ansible galaxy knows how to strip | 15:07 |
| fungi | which was seen as sufficient to get collaboration happening on it in gerrit without having to make governance decisions immediately | 15:07 |
| *** HeOS has quit IRC | 15:08 | |
| mordred | so in a future world where we have a mechanism that's usable to sync things with galaxy, openstack/ansible-role-nodepool should become "openstack.nodepool" - which is pretty cool | 15:08 |
| Zara | ah, okay, so openstack/ansible-role-storyboard or openstack/ansible-storyboard would probably be the way to name it? | 15:09 |
| mordred | yah | 15:10 |
| *** salv-orl_ has quit IRC | 15:11 | |
| *** ari has joined #openstack-infra | 15:11 | |
| Zara | cool, thank you. :) btw, is there any preferred course between making a new project team or making an unofficial project? or does it just depend on the context? | 15:11 |
| *** HeOS has joined #openstack-infra | 15:11 | |
| Zara | odyssey4me: thanks for the tip re: asking around openstack-ansible :) | 15:12 |
| mordred | odyssey4me: uhoh | 15:12 |
| mordred | odyssey4me: ansible 2.0 broke ansible-lint | 15:13 |
| *** yamamoto has joined #openstack-infra | 15:13 | |
| mordred | odyssey4me: http://logs.openstack.org/61/266361/1/check/gate-ansible-puppet-ansible-lint/d503db1/console.html | 15:14 |
| mordred | odyssey4me: will likely affect you more than me at the moment - but we probably might need to dive in and make a patch and get that guy to cut a release | 15:15 |
| odyssey4me | mordred hmm, ansible-lint - afaik - should be run against the playbook that calls the role... not against each role file | 15:15 |
| pabelanger | fungi: mordred: odyssey4me: I actually have an email to post to ML about this. There was some positive discussions about moving the stuff I am working on under the openstack ansible team, but was very high level a few weeks ago | 15:15 |
| *** wznoinsk1 has joined #openstack-infra | 15:16 | |
| anteaya | if anyone wants to join the stable meeting in -3 that is in progress | 15:16 |
| pabelanger | Zara: mordred: I prefer ansible-role-foo | 15:16 |
| pabelanger | over ansible-foo | 15:16 |
| Zara | pabelanger: useful to know! :) why is that, ooi? | 15:16 |
| pabelanger | Zara: IIRC, seems to be what people are defaulting there role names in the wild. Like mordred said, galaxy strips is, so it must be somewhat official | 15:17 |
| *** wznoinsk1 has quit IRC | 15:17 | |
| mordred | odyssey4me: that's what it's doing in that test - I have a test.yml playbook that calls the role | 15:17 |
| danpb | is there anyone around who can update initial gerrit group membership for os-vif per the documented process here http://docs.openstack.org/infra/manual/creators.html | 15:17 |
| pabelanger | also.... \o/ for ansible 2.0. Guess what I'm doing today :) | 15:17 |
| mordred | odyssey4me: (I use your ansible jobs on our infra ansible role) | 15:17 |
| odyssey4me | pabelanger fixing ansible-lint? :) | 15:18 |
| pabelanger | odyssey4me: :) | 15:18 |
| openstackgerrit | Merged openstack-infra/project-config: fix stable/liberty telemetry integration job https://review.openstack.org/264769 | 15:18 |
| pabelanger | looking now | 15:18 |
| mordred | pabelanger: yes. please. please to be fixing ansible-lint kthxbai | 15:18 |
| *** katyafervent is now known as katyafervent_awa | 15:18 | |
| openstackgerrit | Jim Rollenhagen proposed openstack-infra/project-config: Allow ironic jobs to run with tempest plugin https://review.openstack.org/265311 | 15:18 |
| mordred | pabelanger: http://logs.openstack.org/61/266361/1/check/gate-ansible-puppet-ansible-lint/d503db1/console.html is the first break that's surfaced - I believe the internal python api in ansible has changed | 15:18 |
| mordred | s/I believe// | 15:19 |
| pabelanger | mordred: Ya, checking now | 15:19 |
| Zara | pabelanger: okay, that's good to know, thanks. (I don't know very much about ansible personally but other people on storyboard like it a lot!) | 15:19 |
| *** shardy_mtg is now known as shardy | 15:19 | |
| *** isaacb has quit IRC | 15:19 | |
| pabelanger | Zara: Ya, I've started moving some infra tools under ansible. So, ansible-role-(nodepool|zuul|jenkins-job-builder|diskimage-builder) all exist today | 15:20 |
| pabelanger | I have more locally, just need to clean them up and push up | 15:20 |
| openstackgerrit | Jim Rollenhagen proposed openstack-infra/project-config: Allow ironic jobs to run with tempest plugin https://review.openstack.org/265311 | 15:21 |
| fungi | we definitely like ansible, but we have a lot of puppet inertia we'd need to overcome and don't want to be deploying half our systems with one tool and the other half with another | 15:21 |
| openstackgerrit | Vincent Françoise proposed openstack-infra/project-config: Update Watcher Tempest tests package name https://review.openstack.org/266430 | 15:21 |
| *** akshai has quit IRC | 15:21 | |
| fungi | at least not as a long-term situation (for a short-term migration it would be expected of course) | 15:22 |
| odyssey4me | fungi sure, but pabelanger's tooling is useful for external CI's and other people wanting to setup similar tooling for their own CI | 15:22 |
| openstackgerrit | Jim Rollenhagen proposed openstack-infra/project-config: Allow ironic jobs to run with tempest plugin https://review.openstack.org/265311 | 15:22 |
| fungi | absolutely | 15:22 |
| Zara | fungi: yeah, makes sense, I'd imagine it would get very confusing | 15:22 |
| *** akshai has joined #openstack-infra | 15:23 | |
| fungi | we simply reached a sort of team consensus (as much as we ever can i guess) that we shouldn't put tools under infra governance unless we're at least using them or plan to be using them very soon | 15:23 |
| *** yamamoto has quit IRC | 15:24 | |
| *** jsavak has joined #openstack-infra | 15:25 | |
| fungi | because otherwise it's a disservice to those projects--we'd have less incentive as a group to keep them reviewed and supported | 15:25 |
| openstackgerrit | Slava Tabolin proposed openstack-infra/project-config: Add project vmware-dvs https://review.openstack.org/266331 | 15:26 |
| *** ajmiller has joined #openstack-infra | 15:28 | |
| openstackgerrit | Jim Rollenhagen proposed openstack-infra/project-config: Allow ironic jobs to run with tempest plugin https://review.openstack.org/265311 | 15:28 |
| *** nmagnezi has quit IRC | 15:29 | |
| *** nmagnezi has joined #openstack-infra | 15:30 | |
| *** Thelo has quit IRC | 15:31 | |
| *** katyafervent_awa is now known as katyafervent | 15:31 | |
| *** Thelo has joined #openstack-infra | 15:32 | |
| *** levi_b has joined #openstack-infra | 15:32 | |
| *** zz_dimtruck is now known as dimtruck | 15:33 | |
| *** Keedya_ has joined #openstack-infra | 15:34 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/ansible-puppet: Collapse redundant hieradata tasks https://review.openstack.org/265652 | 15:34 |
| openstackgerrit | Monty Taylor proposed openstack-infra/ansible-puppet: Perform filepath checking in python https://review.openstack.org/266439 | 15:34 |
| *** isaacb has joined #openstack-infra | 15:34 | |
| *** yaume_ has joined #openstack-infra | 15:34 | |
| openstackgerrit | gordon chung proposed openstack/requirements: bump ceilometerclient to 2.2.0 https://review.openstack.org/266440 | 15:35 |
| *** abregman has joined #openstack-infra | 15:38 | |
| *** yaume has quit IRC | 15:38 | |
| *** _amrith_ is now known as amrith | 15:39 | |
| *** dizquierdo has joined #openstack-infra | 15:44 | |
| anteaya | fungi: ovh has a thin shim of error, I think it started when we surpassed the prior threashold of servers previously created there | 15:44 |
| *** Verilium_ is now known as Verilium | 15:46 | |
| fungi | anteaya: yeah, looking at the last week's graph, the lack of errors over the past few days is more of an anomaly | 15:47 |
| anteaya | I do like the lack of errors | 15:47 |
| anteaya | ah utopia if only as an anomaly | 15:47 |
| *** yamamoto has joined #openstack-infra | 15:48 | |
| clarkb | internap seems to be doing well at 24 nodes fwiw | 15:49 |
| anteaya | yay | 15:49 |
| clarkb | at least on the job result side | 15:49 |
| anteaya | the side that counts | 15:49 |
| anteaya | mgagne: ^^ yay | 15:49 |
| anteaya | clarkb: you were up early this morning | 15:50 |
| clarkb | anteaya: ya I was up at 4am for some reason then managed to fall back asleep | 15:50 |
| anteaya | glad you could fall back to sleep | 15:50 |
| anteaya | did you stick a bottle in someone's mouth just because you were up? | 15:51 |
| * anteaya wonders if clarkb has a cat | 15:51 | |
| clarkb | no cat | 15:52 |
| dims | anteaya : first review request for this year :) https://review.openstack.org/#/c/266119/ - please take a look when you get a chance | 15:53 |
| anteaya | from you | 15:54 |
| dims | yes, from me :) | 15:54 |
| * anteaya hangs red bow on 266119 | 15:54 | |
| clarkb | looks like neutron tempest runs are still on the slow side. I wonder if we have made any progress improving the ipv6 tests | 15:56 |
| anteaya | armax: would you happen to know if there has been any progress on the ipv6 tests for tempest? | 15:57 |
| *** rbrndt has joined #openstack-infra | 15:57 | |
| dims | armax : i saw some gate failures this morning - https://bugs.launchpad.net/neutron/+bug/1533194 | 15:57 |
| openstack | Launchpad bug 1533194 in tempest "Gate failures for neutron in TestGettingAddress" [Undecided,New] | 15:57 |
| *** notnownikki has joined #openstack-infra | 15:58 | |
| armax | anteaya: not right now | 15:58 |
| *** fawadkhaliq has joined #openstack-infra | 15:58 | |
| *** amrith is now known as _amrith_ | 15:59 | |
| armax | anteaya: I’ll have a look if no-one beats me to them, thanks for the reminder | 15:59 |
| *** _amrith_ is now known as amrith | 16:00 | |
| *** isaacb has quit IRC | 16:00 | |
| *** hichihara has quit IRC | 16:01 | |
| *** krtaylor has quit IRC | 16:03 | |
| *** thomasem-away is now known as thomasem | 16:03 | |
| anteaya | armax: thank you | 16:03 |
| *** sridhar_ram has joined #openstack-infra | 16:03 | |
| *** tzn has quit IRC | 16:04 | |
| anteaya | dims: okay so here is a problem, which we just were trying to understand the breadth of in the stable meeting, not all those projects have follow the stable branch policy | 16:04 |
| anteaya | ttx: ^^ | 16:04 |
| anteaya | dims: so for example, rally has a 0.1 branch | 16:04 |
| gtmanfred | got the keystoneauth plugin for rackspace to work | 16:04 |
| gtmanfred | https://github.com/rackerlabs/rackspace-keystoneauth-plugin/pull/3 | 16:05 |
| anteaya | dims: so I have no idea what will happen to rally's testing if we impliment your testing code | 16:05 |
| *** lucas-hungry is now known as lucasagomes | 16:05 | |
| *** gokrokve has joined #openstack-infra | 16:05 | |
| anteaya | dims ttx making it difficult for nova to deprecate their testing for n-obj | 16:05 |
| anteaya | dims: I'd like for your patch as written to work | 16:06 |
| dims | anteaya : that pattern is already there in some of the jovs | 16:06 |
| dims | jobs | 16:06 |
| *** julim has quit IRC | 16:06 | |
| dims | anteaya : nova already ripped out n-obj service | 16:06 |
| dims | in master | 16:06 |
| fungi | granted, they're pretty much on their own with that... the rally deliverable already doesn't claim release:has-stable-branches and they have to do funky workarounds already since devstack-gate tests unrecognized branches against master | 16:07 |
| *** tzn has joined #openstack-infra | 16:07 | |
| *** julim has joined #openstack-infra | 16:07 | |
| anteaya | dims: yeah I saw that patch | 16:07 |
| anteaya | fungi: can you review 266119 with that in mind? | 16:08 |
| anteaya | please | 16:08 |
| fungi | gtmanfred: exciting, though it's not part of openstack (neither that plugin nor the non-keystone authentication it's communicating with) | 16:08 |
| anteaya | I just don't want to do something that breaks assumptions without clearly stating I'm understanding I'm breaking assumed agreements that don't actually exist | 16:09 |
| anteaya | dims: and I confirm I found http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/rally.yaml#n130 | 16:09 |
| clarkb | I think n-obj if it needs specific testing on older branches can have special jobs for that, but the generic run foo test doesn't need to have n-obj running | 16:10 |
| gtmanfred | fungi: i know, was just making a comment from talking here yesterday | 16:10 |
| *** vgridnev has quit IRC | 16:11 | |
| openstackgerrit | Slava Tabolin proposed openstack-infra/project-config: Add project vmware-dvs https://review.openstack.org/266331 | 16:11 |
| fungi | clarkb: well, 266119 is removing it from a rally-specific job definition too, not just our generic ones | 16:11 |
| fungi | also congress | 16:11 |
| clarkb | right but rally and congress et al aren't using n-obj | 16:11 |
| clarkb | (I was actually really surprised we had n-obj running anywhere) | 16:11 |
| fungi | watcher as well | 16:12 |
| dims | right. me too | 16:12 |
| anteaya | so the goal is to get rid of n-obj in testing as it is removed from nova master, dims I'm on board with that | 16:12 |
| dims | thanks anteaya | 16:13 |
| *** kzaitsev_mb has quit IRC | 16:13 | |
| anteaya | I'm waiting to see if we feel there is a better approach | 16:13 |
| anteaya | dims: the code appears to be what AJaeger asked for | 16:13 |
| anteaya | dims: I'd like to hear what fungi and clarkb suggest | 16:13 |
| fungi | so, yeah i agree that if rally is going to use n-obj on their 1.0 branch then they need special handling for it anyway | 16:13 |
| fungi | and if not, then this shouldn't break for them | 16:14 |
| *** krtaylor has joined #openstack-infra | 16:14 | |
| *** fawadkhaliq has quit IRC | 16:14 | |
| fungi | so either way the change is correct, but boris-42 probably needs a heads up. also maybe thinrichs and whoever's running the unofficial watcher repo dev work (that's not a job for an official project) | 16:15 |
| dims | boris-42 : please see https://review.openstack.org/#/c/266119/ | 16:16 |
| anteaya | dims: thanks | 16:16 |
| *** ayoung has quit IRC | 16:16 | |
| boris-42 | dims: fungi just woke up | 16:16 |
| fungi | but since n-obj is gone from nova in master then any dependence they have is already breaking, so 266119 is just making sure we don't run obviously broken configurations | 16:16 |
| boris-42 | dims: fungi one minute | 16:16 |
| *** matrohon has quit IRC | 16:16 | |
| boris-42 | dims: thank you for spec! | 16:16 |
| dmellado | is gerrit down? | 16:17 |
| clarkb | dmellado: not for me | 16:18 |
| anteaya | dmellado: what are you seeing? | 16:18 |
| dmellado | i.e. https://review.openstack.org/259931 service unavailable | 16:18 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 16:18 | |
| dmellado | oh, now it loads | 16:18 |
| dmellado | nevermind ;) | 16:18 |
| anteaya | dmellado: I see it | 16:18 |
| dmellado | thanks anteaya and clarkb ;) | 16:18 |
| *** yamamoto has quit IRC | 16:18 | |
| anteaya | dmellado: okay do share error messages if you see them, thanks | 16:19 |
| dmellado | anteaya: it wasn't loading before | 16:19 |
| dmellado | it was showing service unavailable, just FYI | 16:19 |
| anteaya | dmellado: I believe you, new gerrit is not operating as smoothly as old gerrit, we are working on solutions | 16:19 |
| anteaya | dmellado: if you do see errors, please report them | 16:19 |
| dmellado | anteaya: will do, thanks anyway! | 16:19 |
| anteaya | thank you | 16:20 |
| boris-42 | dims: so I am not sure that Rally is deeply connected with nova object service | 16:20 |
| dims | boris-42 : quite sure about that. heads up just in case you see something break in stable jobs for rally | 16:21 |
| anteaya | boris-42: would it make sense for you to offer a patch to rally jobs removing it entirely and dims can rebase his patch on yours | 16:21 |
| anteaya | then we don't have a conditional we don't need | 16:21 |
| boris-42 | anteaya: okay I will work on it today | 16:22 |
| anteaya | dims: is that acceptable for you? | 16:22 |
| *** egonzalez has joined #openstack-infra | 16:22 | |
| dims | anteaya : or we could let this in and he can remove conditional :) | 16:22 |
| anteaya | boris-42: is that okay for you? | 16:22 |
| anteaya | dims: can you find someone associated with watcher, or at least make an attempt? | 16:23 |
| dims | anteaya : for watcher, this is defensive code anyway since they don't have stable/* branches at all | 16:24 |
| dims | anteaya : and their master would have broken when nova yanked n-obj out | 16:25 |
| anteaya | yeah they only have master branch | 16:25 |
| *** BobBall has quit IRC | 16:25 | |
| *** sparkyco_ has joined #openstack-infra | 16:26 | |
| anteaya | boris-42: I'm going to approve 266119 but it would still be great to see your clean up patch | 16:26 |
| *** MarkAtwood has joined #openstack-infra | 16:26 | |
| *** gongysh has quit IRC | 16:26 | |
| *** esikachev has quit IRC | 16:26 | |
| boris-42 | anteaya: it's ok for me both cases | 16:27 |
| boris-42 | anteaya: if something will go wrong we can always revert do the quick fix | 16:27 |
| boris-42 | or do* | 16:27 |
| dims | thanks anteaya | 16:27 |
| anteaya | boris-42: okay great thanks, at least you are aware | 16:27 |
| boris-42 | anteaya: yep thank you for pinging me | 16:27 |
| anteaya | dims: thank you for cleaning it up | 16:27 |
| anteaya | boris-42: yup, thanks | 16:28 |
| *** isaacb has joined #openstack-infra | 16:29 | |
| *** eharney has quit IRC | 16:29 | |
| mgagne | clarkb I think we have yet to "really" test overcommit due to Zuul not using ALL nodes despite having a greater max-servers. | 16:30 |
| clarkb | mgagne: ah | 16:30 |
| clarkb | just due to overhead for cycling nodes in and out we never really use all capacity at one time | 16:31 |
| mgagne | clarkb yea, I think infra is only loaded to ~33% so we have 8 nodes used at max | 16:31 |
| mgagne | clarkb right | 16:31 |
| clarkb | mgagne: should we keep ramping up slowly? | 16:31 |
| *** slava__ has quit IRC | 16:31 | |
| mgagne | clarkb I think we should, let me check the next step | 16:31 |
| mgagne | clarkb there are 2 limits so far for capacity: hardware capacity and IPs | 16:32 |
| *** cdent has quit IRC | 16:32 | |
| mgagne | all can be fixed | 16:32 |
| mgagne | but might need a bit of time | 16:32 |
| *** otter768 has joined #openstack-infra | 16:32 | |
| anteaya | mgagne: do you want to up the number of max_servers? | 16:33 |
| mgagne | checking for next limit | 16:33 |
| mgagne | 36 is the theoretical next limit | 16:34 |
| anteaya | mgagne: I'll offer a patch and we can discuss when to merge | 16:35 |
| *** Keedya_ has quit IRC | 16:35 | |
| mgagne | thanks | 16:36 |
| *** otter768 has quit IRC | 16:37 | |
| *** jamielennox is now known as jamielennox|away | 16:37 | |
| *** Keedya_ has joined #openstack-infra | 16:38 | |
| openstackgerrit | Anita Kuno proposed openstack-infra/project-config: Increase internap max servers from 24 to 36 https://review.openstack.org/266468 | 16:38 |
| clarkb | fungi: I know you are busy with the atc collcetion, but on the bare-* image build front does it make sense to separate the db setup from bindep so that we can get those images transitioned to dib sooner? | 16:39 |
| clarkb | fungi: trying to make sure that we can run those jobs on >1 cloud | 16:39 |
| *** rkukura has joined #openstack-infra | 16:40 | |
| fungi | clarkb: sure, they're basically already separate. i have job macros tested and working, just need to add them to the jobs or job-templates which need them | 16:40 |
| *** piet has quit IRC | 16:41 | |
| fungi | in theory the bindep part is working too, except that we're only installing bindep via a dib element right now | 16:41 |
| fungi | so was holding off pushing that transition forward until we have glance working in rackspace | 16:41 |
| clarkb | hrm | 16:42 |
| *** pradk has joined #openstack-infra | 16:42 | |
| *** rkukura has quit IRC | 16:42 | |
| clarkb | maybe it is simpler to get bindep on the existing rax images | 16:42 |
| clarkb | mostly just don't want to be running all unittests on one cloud next month so trying to figure out what a good short term path is | 16:42 |
| clarkb | and maybe just doing the thing is best | 16:43 |
| fungi | the only part of the database setup depending on package installation during job runtime is that we need to ensure the platform-specific packages for database services get installed | 16:43 |
| anteaya | mgagne: can you comment on when you would like 266468 to merge? | 16:43 |
| mgagne | anteaya sure | 16:43 |
| anteaya | mgagne: thanks, that helps us | 16:43 |
| *** jordanP has quit IRC | 16:43 | |
| *** jordanP has joined #openstack-infra | 16:43 | |
| mordred | clarkb: just to be clear - s/next month/in two weeks/ | 16:43 |
| clarkb | mordred: right | 16:44 |
| sfinucan_ | Hi folks - can anyone explain to me what might cause a persistent 'This change depends on a change that failed to merge.' error in Jenkins/ZUUL? | 16:44 |
| clarkb | let me check on how reliable rax image uploads are now that shade should be creating a new swift connection for each upload | 16:44 |
| anteaya | sfinucan_: can you share the link of the patch you see that on and I will explain with patch numbers? | 16:44 |
| *** jsavak has quit IRC | 16:44 | |
| sfinucan_ | anteaya: Sure. https://review.openstack.org/#/c/258984/ | 16:44 |
| *** egonzalez has quit IRC | 16:45 | |
| *** dtantsur is now known as dtantsur|brb | 16:45 | |
| clarkb | rax ubuntu-trusty is 6 days old | 16:45 |
| sfinucan_ | Though all the predecessors of that patch will are affected | 16:45 |
| sfinucan_ | *will be/are | 16:45 |
| anteaya | sfinucan_: in the top right hand corner do you see there are 9 related changes? | 16:45 |
| *** hockeynut_afk is now known as hockeynut | 16:45 | |
| sfinucan_ | anteaya: Yeah? | 16:45 |
| *** derekh has quit IRC | 16:46 | |
| anteaya | it seems you are rebasing them all | 16:46 |
| anteaya | that should address the error | 16:46 |
| openstackgerrit | Ivan Kolodyazhny proposed openstack-infra/project-config: Add functional test job for python-brick-cinderclient-ext https://review.openstack.org/265925 | 16:46 |
| clarkb | also the git pruning that dib's git caching will do should trim down the size of some of our images | 16:47 |
| clarkb | I was also thinking that maybe we should be gc'ing and repacking the git cache | 16:47 |
| greghaynes | clarkb: SGTM | 16:47 |
| anteaya | basically that error is due to a dependency chain either within the same repo via git parents and children or via the use of Depends-On in the commit message to create a cross repo dependency | 16:47 |
| fungi | yeah, seems reasonable | 16:47 |
| anteaya | sfinucan_: if a patch that others are dependant on can't be merged that error shows up on the dependencies | 16:48 |
| *** levi_b has left #openstack-infra | 16:48 | |
| sfinucan_ | anteaya: But that's the problem - I don't think I have any dependencies | 16:48 |
| anteaya | sfinucan_: you have a series of 9 dependant patches | 16:48 |
| sfinucan_ | I have no 'Depends-On' in the commits messages | 16:49 |
| sfinucan_ | sorry | 16:49 |
| *** fawadkhaliq has joined #openstack-infra | 16:49 | |
| sfinucan_ | I don't have any dependencies that don't apply | 16:49 |
| *** yolanda has quit IRC | 16:49 | |
| *** notnownikki has quit IRC | 16:49 | |
| *** yolanda has joined #openstack-infra | 16:50 | |
| sfinucan_ | I've downloaded the topmost patch of that "series" (i.e. 258984) and rebased that onto master. That should rebase its dependencies also, right/ | 16:50 |
| sfinucan_ | at least that's my understanding of rebase... | 16:50 |
| anteaya | sfinucan_: that seems to be what is happening if you go through all the patches in that series | 16:51 |
| anteaya | and all those 9 patches are dependant on the parent before | 16:51 |
| anteaya | so taht is a dependency | 16:51 |
| *** harlowja_at_home has joined #openstack-infra | 16:52 | |
| *** sparkyco_ has quit IRC | 16:52 | |
| *** yolanda has quit IRC | 16:52 | |
| clarkb | OpenStackCloudException: Could not determine capabilities (Inner Exception: No tenant specified) thats still happening /me checks shade version | 16:52 |
| sfinucan_ | anteaya: True, but that dependency is merged (commit id: fb3a062cecfad6ff74510bcb411db37de51ae059) so it shouldn't introduce any issues | 16:52 |
| *** _nadya_ has joined #openstack-infra | 16:52 | |
| greghaynes | clarkb: In nodepool? :/ | 16:53 |
| anteaya | sfinucan_: how about we revisit this after jenkins returns with results of your rebase | 16:53 |
| greghaynes | clarkb: That was an os-c-c thing I thought | 16:53 |
| *** Keedya_ has quit IRC | 16:53 | |
| fungi | sfinucan_: it looks like you rebased https://review.openstack.org/257956 but not the changes depending on that one | 16:53 |
| clarkb | greghaynes: its the shade doesn't make a new swift connection for each upload problem, token times out and auth breaks | 16:53 |
| *** yolanda has joined #openstack-infra | 16:53 | |
| greghaynes | clarkb: oh, that one | 16:53 |
| clarkb | looks like we need shade 1.4.0 but have 1.3.0 | 16:53 |
| *** tonytan4ever has joined #openstack-infra | 16:54 | |
| greghaynes | That error should be made a lot better | 16:54 |
| greghaynes | error message | 16:54 |
| fungi | sfinucan_: oh, you've rebased at least a few of them | 16:54 |
| sfinucan_ | anteaya: Sure :) Though it's worth noting this has done this...four times, I think? | 16:54 |
| *** jsavak has joined #openstack-infra | 16:54 | |
| sfinucan_ | fungi: Yup, and I'm doing it again now | 16:54 |
| rhedlind | anteaya: the X-IO CI is back up and running the latest zuul. I would like to request for the account to be enabled again. | 16:54 |
| clarkb | 1.4.0 is old enough that we should have had it updated by puppet | 16:54 |
| openstackgerrit | Sean Dague proposed openstack-infra/project-config: remove gate-tempest-dsvm-ceilometer-mysql-neutron-full from tempest gate queue https://review.openstack.org/266478 | 16:55 |
| *** sputnik13 has joined #openstack-infra | 16:55 | |
| *** amitgandhinz has joined #openstack-infra | 16:55 | |
| *** yolanda has quit IRC | 16:55 | |
| *** bradjones has quit IRC | 16:55 | |
| anteaya | rhedlind: can you post to the -announce mailing list thread that I started for this incident with details of why this incident occured and what you have done to address it? | 16:55 |
| fungi | sfinucan_: you're rebasing the entire series together at the same time using something like rebase -i right? | 16:55 |
| anteaya | rhedlind: so we have the paper trail in order? | 16:56 |
| anteaya | any objection from anyone regarding turning the X-IO CI back on? | 16:56 |
| anteaya | Roamer`: ^^ | 16:56 |
| *** yolanda has joined #openstack-infra | 16:56 | |
| odyssey4me | mordred FYI https://github.com/willthames/ansible-lint/issues/75 | 16:56 |
| sfinucan_ | fungi: anteaya: Sure am. My exact commands (verbose, I know): 'git checkout master; git pull --ff-only; git review -d 258984; git rebase -i master; #edit first patch; git commit -a --amend; git rebase --continue; git review -fy' | 16:56 |
| sfinucan_ | which is exactly how I work normally | 16:57 |
| *** sputnik13 has quit IRC | 16:57 | |
| rhedlind | anteaya: Ok, I will post to the mailing list. | 16:58 |
| *** bradjones has joined #openstack-infra | 16:58 | |
| *** bradjones has quit IRC | 16:58 | |
| *** bradjones has joined #openstack-infra | 16:58 | |
| *** nmagnezi has quit IRC | 16:58 | |
| clarkb | puppet is puppeting on nodepool (I am going to figure out why latest shade isn't used | 16:58 |
| *** sputnik13 has joined #openstack-infra | 16:58 | |
| anteaya | rhedlind: thank you | 16:59 |
| *** rkukura has joined #openstack-infra | 16:59 | |
| *** piet has joined #openstack-infra | 16:59 | |
| *** yolanda has quit IRC | 16:59 | |
| clarkb | oh because we only update when nodepool code merges and we haven't done that for a while for raisins | 16:59 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config: Use file permissions instead of ownership to grant access to files https://review.openstack.org/266482 | 16:59 |
| *** yolanda has joined #openstack-infra | 17:00 | |
| *** yolanda has quit IRC | 17:01 | |
| *** dslev has quit IRC | 17:01 | |
| fungi | sfinucan_: definitely odd. once testing comes back on those it would be good to see where in the series it starts complaining about merge conflicts | 17:01 |
| *** dizquierdo has quit IRC | 17:01 | |
| clarkb | I can manually upgrade shade and restart nodepool (after today's image builds finish) or we can wait for the image builder changes to merge and let it happen then | 17:01 |
| *** BobBall has joined #openstack-infra | 17:01 | |
| *** gyee has joined #openstack-infra | 17:01 | |
| sfinucan_ | fungi: sounds good. I have a rework of the first patch that will cause tests to fail, but I'll hold off pushing | 17:02 |
| *** yolanda has joined #openstack-infra | 17:02 | |
| mordred | odyssey4me: good - there is a patch already | 17:02 |
| *** yamamoto has joined #openstack-infra | 17:02 | |
| anteaya | sfinucan_: yes, giving us a chance to see what you are reporting is helpful to us getting an understanding of your situation | 17:02 |
| anteaya | since as soon as you rebase the error you are reporting disappears | 17:03 |
| *** bharathm has joined #openstack-infra | 17:03 | |
| *** salv-orlando has joined #openstack-infra | 17:03 | |
| *** claudiub has joined #openstack-infra | 17:04 | |
| *** samuelBartel has quit IRC | 17:04 | |
| *** jlanoux has quit IRC | 17:04 | |
| openstackgerrit | Mikhail S Medvedev proposed openstack-infra/system-config: Remove logstash/elasticsearch classes, use puppet-openstackci https://review.openstack.org/240011 | 17:05 |
| mtreinish | fungi: do you have a sec to help me manually test the subunit2sql db pruning? | 17:05 |
| *** Keedya_ has joined #openstack-infra | 17:05 | |
| mgagne | now most nodes are used @internap ^^' | 17:05 |
| *** yolanda has quit IRC | 17:05 | |
| *** yolanda has joined #openstack-infra | 17:06 | |
| mgagne | if there could be a way to correlate graphs with logs, that would be great. Some nodes are taking up to 10m to be ready and I would like to understand if it's a problem on our side or not. | 17:06 |
| fungi | mtreinish: sort of scrambling under a deadline. what do you need? | 17:06 |
| *** vgridnev has joined #openstack-infra | 17:06 | |
| *** kzaitsev_mb has joined #openstack-infra | 17:06 | |
| clarkb | mgagne: I can get you uuids from our nodepool logs | 17:07 |
| *** tongli has quit IRC | 17:07 | |
| mgagne | clarkb I have a bunch of UUID too but don't know which one reported 10m ready time | 17:07 |
| clarkb | mgagne: unfortunately we can't publish them publicly until we get better non disclosure of passwords (which is in progress aiui so someday soon hopefully) | 17:07 |
| *** fawadkhaliq has quit IRC | 17:07 | |
| *** yolanda has quit IRC | 17:07 | |
| *** devvesa has quit IRC | 17:07 | |
| clarkb | mgagne: ya I think I can determine which one took longer | 17:07 |
| mtreinish | fungi: just for someone to run L66 from: https://review.openstack.org/#/c/176564/3/manifests/server.pp with $expire_age=365 on logstash.o.o and see if it works (and how long it takes) | 17:07 |
| mtreinish | but it's not critical so no rush | 17:08 |
| mgagne | clarkb but I guess it could be related to 9 nodes being created at the same time. Nova has a max_io_ops thing which limits the number of ops per node IIRC | 17:08 |
| *** yolanda has joined #openstack-infra | 17:09 | |
| anteaya | mmedvede: I'm unclear what you are trying to do with 240011, I don't see that logstash and elastic search occupies the same space as puppet-openstackci at all | 17:09 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for NameError os-win bug 1533290 https://review.openstack.org/266487 | 17:09 |
| openstack | bug 1533290 in os-win "nova unit tests failing with os-win 0.1.0" [Undecided,New] https://launchpad.net/bugs/1533290 | 17:09 |
| mgagne | hmm we don't have this filter | 17:09 |
| zaro | morning | 17:10 |
| *** Swami has joined #openstack-infra | 17:10 | |
| anteaya | morning zaro | 17:10 |
| *** Keedya_ has quit IRC | 17:10 | |
| *** bpokorny has joined #openstack-infra | 17:10 | |
| mmedvede | anteaya: this was part of initial spec to move logstash to openstackci. I see your point though | 17:10 |
| *** yolanda has quit IRC | 17:10 | |
| *** kgiusti has quit IRC | 17:10 | |
| *** isaacb has quit IRC | 17:10 | |
| anteaya | mmedvede: ah okay perhaps a clearer title on that commit and maybe a commit message body with a link to the spec? | 17:11 |
| openstackgerrit | Matt Riedemann proposed openstack/requirements: Block os-win 0.1.0 https://review.openstack.org/266490 | 17:11 |
| mriedem | sdague: ^ | 17:11 |
| asselin | mmedvede, anteaya yes, we can consider removing it from the original spec and take it on the follow-up spec | 17:12 |
| *** yolanda has joined #openstack-infra | 17:12 | |
| anteaya | mmedvede: right now my mind thinks you are intending to replace logstash with puppet-openstackci | 17:12 |
| anteaya | great, so some details in that commit message would help me a lot | 17:12 |
| fungi | mtreinish: i have this running... sudo sh -c 'subunit2sql-db-manage --config-file /etc/subunit2sql.conf expire --expire-age 365 >> /var/log/subunit2sql_migration.log 2>&1' | 17:12 |
| clarkb | mgagne: that would probably explain it if you had it :) | 17:13 |
| *** apoorvad has joined #openstack-infra | 17:13 | |
| clarkb | mgagne: our logs aren't quite as verbose on how long it takes these things as I expected | 17:13 |
| mmedvede | anteaya: the spec called to move logstash setup into puppet-openstackci, so that third-party CIs would have easier time setting it up | 17:13 |
| mgagne | yea, I'm not sure if Nova really throttles ops despite not having this filter | 17:13 |
| anteaya | mmedvede: that is fine, my point is that commit message I'm seeing doesn't link to the spec | 17:13 |
| mgagne | clarkb because a filter would reject your request IIRC | 17:13 |
| nibalizer | Shrews: cool thanks! | 17:13 |
| clarkb | mgagne: oh it would error rather than block? | 17:13 |
| anteaya | mmedvede: having a link to the spec or specs would be helpful | 17:14 |
| mgagne | clarkb yea, instance would go in error state or in fact, host will be removed from available hosts | 17:14 |
| *** yolanda has quit IRC | 17:14 | |
| mtreinish | fungi: ok cool, thanks | 17:14 |
| mtreinish | that'll prune the db to 1 years worth of records | 17:14 |
| mgagne | clarkb or could it be a multi-nodes thing? | 17:15 |
| mgagne | clarkb where node is ready only if all nodes are ready? | 17:15 |
| mgagne | sub-nodes* | 17:15 |
| clarkb | mgagne: yes primary node only goes ready once all subnodes are ready too | 17:15 |
| mgagne | that could be it then | 17:15 |
| fungi | mtreinish: any guesses as to expected runtime? i'll check in on it periodically to see when it seems to have completed | 17:15 |
| clarkb | mgagne: but it boots them in parallel so it shouldn't increase the ready time drastically | 17:15 |
| mgagne | we have 2 multi-nodes node | 17:16 |
| mgagne | hopefully | 17:16 |
| *** austin81 has joined #openstack-infra | 17:16 | |
| *** sfinucan_ has quit IRC | 17:16 | |
| openstackgerrit | Sean Dague proposed openstack-infra/project-config: switch nova over to constraints jobs https://review.openstack.org/266492 | 17:16 |
| *** yolanda has joined #openstack-infra | 17:16 | |
| sdague | fungi / clarkb if we could approve ^^^^ | 17:17 |
| mtreinish | fungi: no clue, I haven't ever used it before on a real db. Just some synthetic test cases I set up | 17:17 |
| sdague | there is an upstream requirements break, a g-r block coming, but honestly, just flipping to the constraints job is probably about time | 17:17 |
| mgagne | but I'm not sure what ready time really is. is it time between ACTIVE and SSH access? | 17:17 |
| mgagne | if so, can only comes from delay in OS boot process or bad arp cache | 17:18 |
| mtreinish | fungi: I imagine the test_run and test_run metadata deletes will take some time, those tables are quite big | 17:18 |
| *** akshai has quit IRC | 17:18 | |
| *** yaume_ has quit IRC | 17:20 | |
| openstackgerrit | Sean Dague proposed openstack-infra/project-config: switch nova over to constraints jobs https://review.openstack.org/266492 | 17:20 |
| clarkb | mgagne: I think it is total time from sending nova boot api request to ssh access | 17:20 |
| clarkb | mgagne: but I would have to check where that is reported to graphite to confirm | 17:20 |
| mgagne | ok, I think I found the nodes | 17:20 |
| mgagne | created vs OS-SRV-USG:launched_at I would say | 17:20 |
| mgagne | + SSH ready time | 17:20 |
| openstackgerrit | Khai Do proposed openstack-infra/puppet-gerrit: Allow puppet to configure the Gerrit's sshd.batchThreads option https://review.openstack.org/262053 | 17:21 |
| zaro | clarkb: i'm noticing that pushes to gerrit are slightly slow today, or is it just me? | 17:21 |
| clarkb | zaro: I haven't pushed anything to gerrit today so don't have any data to compare with | 17:22 |
| fungi | sdague: what's the review urgency there? is something broken for nova with not constraints right now? also, it looks like you're making constraints voting on stable/kilo... is that desired? | 17:22 |
| zaro | clarkb: nobody else complaining is good sign i guess. | 17:22 |
| *** daemontool has quit IRC | 17:22 | |
| sdague | fungi: yes, it's broken now | 17:22 |
| *** gokrokve has quit IRC | 17:22 | |
| anteaya | zaro: no they are tired of complaining I think | 17:22 |
| sdague | there is a global override for no constraints on kilo | 17:22 |
| mgagne | will have to ask our team to get more timing logs and graphs, I'm really conscious about performance =) | 17:23 |
| fungi | sdague: oh, i see, there's a match up at line 2122 to only run constraints jobs on not-kilo | 17:23 |
| zaro | anteaya: haha, even better! | 17:23 |
| *** hashar has quit IRC | 17:23 | |
| anteaya | zaro: well not so much from my point of view | 17:23 |
| sdague | https://github.com/openstack-infra/project-config/blob/298940babd42e4dfbfc0ddbd42396905b35cab8e/zuul/layout.yaml#L2123 | 17:23 |
| anteaya | zaro: it would be great if we could figure out why gerrit is having difficulty being as performant as prior gerrit | 17:23 |
| anteaya | mgagne: thank you | 17:24 |
| mmedvede | anteaya: the spec http://specs.openstack.org/openstack-infra/infra-specs/specs/openstackci.html | 17:24 |
| anteaya | mmedvede: I am asking you to fix the commit message | 17:24 |
| clarkb | anteaya: its the memory, which older gerrit had trouble with too | 17:25 |
| anteaya | mmedvede: please fix the commit message | 17:25 |
| zaro | anteaya: where are you seeing the perf degrade? | 17:25 |
| *** sdake has joined #openstack-infra | 17:25 | |
| mmedvede | anteaya: would do | 17:25 |
| clarkb | but this version may have more issues with that | 17:25 |
| anteaya | mmedvede: thank you | 17:25 |
| anteaya | frequent user issues, service not available, 502 and 503 issues | 17:25 |
| clarkb | anteaya: yes thats the memory problem | 17:25 |
| anteaya | I had an issue logging in this morning | 17:26 |
| clarkb | we know why it happens | 17:26 |
| anteaya | okay can we have some sort of written thing, like maybe an etherpad I can read and perhaps show people when they have issues? | 17:26 |
| *** yamamoto has quit IRC | 17:27 | |
| *** MarkAtwood has quit IRC | 17:27 | |
| anteaya | with maybe a list of patches up to propose solutions so folks who are frustrated can review? | 17:27 |
| *** dtantsur|brb is now known as dtantsur|afk | 17:27 | |
| zaro | anteaya: i thought giving gerrit more memory resolved all those? | 17:27 |
| anteaya | zaro: can you perhaps add something to today's agenda? | 17:28 |
| anteaya | zaro: because if you think the problem is fixed and clarkb things this is a known issue I think it deserves some discussion time | 17:28 |
| openstackgerrit | Merged openstack-infra/subunit2sql: Add tests to check migrations and models match https://review.openstack.org/253301 | 17:28 |
| *** yamamoto has joined #openstack-infra | 17:28 | |
| openstackgerrit | Merged openstack-infra/subunit2sql: Add command deleting *.pyc before executing ostestr https://review.openstack.org/265125 | 17:28 |
| openstackgerrit | Merged openstack-infra/subunit2sql: Add min ver to extra libs for graph https://review.openstack.org/265153 | 17:28 |
| *** fawadkhaliq has joined #openstack-infra | 17:29 | |
| *** mudassirlatif has joined #openstack-infra | 17:30 | |
| *** yamamoto has quit IRC | 17:30 | |
| *** yamamoto has joined #openstack-infra | 17:30 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for NameError os-win bug 1533290 https://review.openstack.org/266487 | 17:31 |
| openstack | bug 1533290 in OpenStack Compute (nova) "nova unit tests failing with os-win 0.1.0" [High,In progress] https://launchpad.net/bugs/1533290 - Assigned to Claudiu Belu (cbelu) | 17:31 |
| *** rkukura has quit IRC | 17:31 | |
| *** yamamoto has quit IRC | 17:31 | |
| *** rkukura has joined #openstack-infra | 17:32 | |
| *** yamamoto has joined #openstack-infra | 17:32 | |
| *** yamamoto has quit IRC | 17:33 | |
| AJaeger | clarkb: can we merge the Fedora 21 nodepool patch? https://review.openstack.org/#/c/264379/ - I would approve it now but like to hear that it's safe... | 17:33 |
| clarkb | AJaeger: I want to delete al lthe images for it then merge it, if you can +2 that owuld be great thoiugh | 17:34 |
| pabelanger | jeblair: so, if not nodepool for templating, the git servers? | 17:34 |
| *** yamamoto has joined #openstack-infra | 17:34 | |
| *** mrmartin has joined #openstack-infra | 17:34 | |
| *** yamamoto has quit IRC | 17:35 | |
| *** yamamoto has joined #openstack-infra | 17:36 | |
| annegentle | did gerritbot move? I'm reading http://docs.openstack.org/infra/manual/drivers.html#gerrit-irc-notifications and not finding it in my cloned, updated copy of openstack-infra/config. Any pointers? | 17:36 |
| *** yamamoto has quit IRC | 17:36 | |
| anteaya | annegentle: openstack-infra/config split | 17:37 |
| *** yamamoto has joined #openstack-infra | 17:37 | |
| anteaya | annegentle: it became openstack-infra/project-config and openstack-infra/system-config | 17:37 |
| annegentle | anteaya: ok, thanks! Where can I update that doc also? | 17:38 |
| anteaya | annegentle: http://git.openstack.org/cgit/openstack-infra/project-config/tree/gerritbot/channels.yaml | 17:38 |
| *** yamamoto has quit IRC | 17:38 | |
| annegentle | anteaya: probably doc is infra-manual? | 17:38 |
| annegentle | anteaya: perfect, thanks | 17:38 |
| anteaya | annegentle: let me read the doc you linked for a second | 17:38 |
| *** esker has quit IRC | 17:39 | |
| *** yamamoto has joined #openstack-infra | 17:39 | |
| anteaya | annegentle: so the infra-manual needs an update | 17:39 |
| anteaya | annegentle: do you want to offer the patch? | 17:39 |
| anteaya | annegentle: or shall I? | 17:39 |
| *** markus_z has quit IRC | 17:39 | |
| annegentle | anteaya: I can do it | 17:40 |
| anteaya | annegentle: wonderful, I'll review | 17:40 |
| anteaya | thanks | 17:40 |
| annegentle | anteaya: though, you'll need to review :) | 17:40 |
| annegentle | anteaya: heh faster typing :) | 17:40 |
| *** hdd has joined #openstack-infra | 17:41 | |
| anteaya | annegentle: I believe you, one thing that will need to be added is that people need to give channel permissions to infra via accessbot before gerritbot can be added to a channel | 17:42 |
| *** e0ne has quit IRC | 17:42 | |
| anteaya | annegentle: I'm trying to find the accessbot file, I can never find that | 17:42 |
| openstackgerrit | Guido Günther proposed openstack/python-jenkins: Add support for handling promotions https://review.openstack.org/258032 | 17:42 |
| *** zeih_ has quit IRC | 17:42 | |
| *** matrohon has joined #openstack-infra | 17:43 | |
| *** yamamoto_ has joined #openstack-infra | 17:43 | |
| *** yamamoto has quit IRC | 17:44 | |
| *** unicell has quit IRC | 17:44 | |
| AJaeger | clarkb: +2 done, you can +W yourself... | 17:45 |
| *** yamamoto_ has quit IRC | 17:45 | |
| *** _nadya_ has quit IRC | 17:45 | |
| anteaya | annegentle: it moved! now easier to find: http://git.openstack.org/cgit/openstack-infra/project-config/tree/accessbot/channels.yaml | 17:46 |
| *** eil397 has joined #openstack-infra | 17:46 | |
| *** yamamoto has joined #openstack-infra | 17:47 | |
| harlowja_at_home | hi there infra folks, can we get https://review.openstack.org/#/c/263678/ merged so that etcd can start to be tested | 17:48 |
| harlowja_at_home | (in an automated manner) | 17:48 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/infra-specs: Move docs.openstack.org/releases to releases.openstack.org https://review.openstack.org/266506 | 17:48 |
| *** jordanP has quit IRC | 17:48 | |
| openstackgerrit | Khai Do proposed openstack-infra/system-config: Allow the review history panel to resize https://review.openstack.org/259560 | 17:49 |
| *** tonytan4ever has quit IRC | 17:51 | |
| *** rkukura has quit IRC | 17:51 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Add release notes for pending 1.2.0 release https://review.openstack.org/266507 | 17:51 |
| jeblair | pabelanger: honestly, the thing i like about dashboards is that they show a lot of information at once and make it easy to correlate... i just haven't had the problem where i want to see less information on a dashboard yet. | 17:51 |
| openstackgerrit | Anne Gentle proposed openstack-infra/infra-manual: Updates Gerrit IRC Notifications section https://review.openstack.org/266508 | 17:52 |
| *** yamamoto has quit IRC | 17:52 | |
| jeblair | clarkb, mordred, Shrews: i found and fixed an ansible bug while attempting to use it in zuul: https://github.com/ansible/ansible/issues/13822 | 17:53 |
| pabelanger | jeblair: right. But, we can default the templating to select all, so it looks like today. It just give the ability to toggle if you need to dive deeper I find | 17:53 |
| mtreinish | fungi: is it still running? | 17:53 |
| *** akshai has joined #openstack-infra | 17:53 | |
| *** mrmartin has quit IRC | 17:53 | |
| mtreinish | if so I guess I need to background it, because we'll trip timeouts | 17:53 |
| *** rkukura has joined #openstack-infra | 17:53 | |
| jeblair | pabelanger: oh, okay, that coulde be useful then | 17:54 |
| Shrews | jeblair: that's fun | 17:54 |
| *** harlowja_at_home has quit IRC | 17:54 | |
| krotscheck | jeblair: I rewrote that rewrite of yours here -> https://review.openstack.org/#/c/253521/ | 17:54 |
| krotscheck | mod_rewrite | 17:54 |
| pabelanger | jeblair: I'll switch to git servers then | 17:54 |
| jeblair | pabelanger: if we can have all regions selected by default, i'm okay with trying on the nodepool graph | 17:55 |
| jeblair | er dash | 17:55 |
| pabelanger | jeblair: Ya, that is how it works now | 17:55 |
| pabelanger | all regions are selected | 17:55 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/system-config: add new vhost for releases.openstack.org https://review.openstack.org/266510 | 17:56 |
| mordred | jeblair: you failed CI! :) | 17:56 |
| *** bardia has joined #openstack-infra | 17:56 | |
| mordred | jeblair: https://travis-ci.org/ansible/ansible/builds/101900288 | 17:56 |
| jeblair | mordred: then clearly the existing code must be correct and my code in error! | 17:56 |
| pabelanger | jeblair: includeAll is the field to select all by default | 17:56 |
| krotscheck | greghaynes: One question on https://review.openstack.org/#/c/165240 - since you were part of the original discussions. Was there ever any thought about having a single wheel-mirror slave build - and then publish - to all the mirrors? | 17:56 |
| krotscheck | Right now it looks like there's > 1. | 17:57 |
| krotscheck | And I'm not certain that's necessary. | 17:57 |
| clarkb | unsupported parameter _ansible_verbosity | 17:57 |
| *** kashyap has joined #openstack-infra | 17:57 | |
| jeblair | clarkb, mordred: wth? | 17:58 |
| clarkb | jeblair: ya Idunno | 17:58 |
| jeblair | mordred: istr they had a "build: failing" badge on their github ready thingy | 17:58 |
| mordred | jeblair: that might just be yeah a known failing travis | 17:58 |
| jeblair | mordred: maybe, on account of they don't have a gate, they broke their stuff generally? | 17:58 |
| kashyap | Hi folks, what does the 'updown' means in this Gate job? - gate-devstack-dsvm-updown | 17:58 |
| jeblair | krotscheck: i think wheel builds need to happen per platform, so there would be one per supported distro/os thingy | 17:59 |
| openstackgerrit | Sean Dague proposed openstack-infra/project-config: switch nova over to constraints jobs https://review.openstack.org/266492 | 17:59 |
| krotscheck | jeblair: Ah. right. Obvious. Thank you. | 17:59 |
| *** zeih has joined #openstack-infra | 18:00 | |
| greghaynes | krotscheck: ohai, yes, what jeblair said | 18:01 |
| fungi | mtreinish: http://paste.openstack.org/show/483636/ | 18:01 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Add wheel mirror to mirrors https://review.openstack.org/253521 | 18:01 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/system-config: Add wheel_mirror_slaves that use pypi-mirror https://review.openstack.org/165240 | 18:01 |
| fungi | mtreinish: no idea if that log content is from this run or not. i didn't look to see whether it had prior content | 18:02 |
| krotscheck | greghaynes: Ack. In that case, ^^ should be ready to review | 18:02 |
| krotscheck | That'll set up the directory on the mirrors, and the wheel slaves. | 18:02 |
| * krotscheck is updating the jobs patch | 18:02 | |
| *** trown is now known as trown|lunch | 18:02 | |
| *** jistr has quit IRC | 18:03 | |
| mtreinish | fungi: the migration stuff would be from a previous run | 18:03 |
| mtreinish | and the time stuff is | 18:04 |
| mtreinish | I'm not sure about 'Killed' though | 18:04 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: publish from openstack/releases to releases.openstack.org https://review.openstack.org/266515 | 18:04 |
| *** mrmartin has joined #openstack-infra | 18:04 | |
| *** zeih has quit IRC | 18:04 | |
| mtreinish | fungi: from that paste it means it exited without an error code? | 18:04 |
| *** sridhar_ram has quit IRC | 18:05 | |
| fungi | mtreinish: i didn't think to check the exit code | 18:05 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/infra-specs: Move docs.openstack.org/releases to releases.openstack.org https://review.openstack.org/266506 | 18:05 |
| fungi | mtreinish: however i can rerun it now (should be much faster the second time?) | 18:05 |
| mtreinish | hmm, looking at the db it cleaned some stuff but not others :( | 18:05 |
| mtreinish | fungi: yeah lets give it another go | 18:06 |
| mtreinish | it should be faster especially because it doesn't have to do the run deletes anymore just test_runs | 18:06 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/infra-specs: Move docs.openstack.org/releases to releases.openstack.org https://review.openstack.org/266506 | 18:06 |
| fungi | running again. i'll check the exit code once it finishes | 18:06 |
| *** MarkAtwood has joined #openstack-infra | 18:07 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/infra-specs: Move docs.openstack.org/releases to releases.openstack.org https://review.openstack.org/266506 | 18:07 |
| openstackgerrit | Mikhail S Medvedev proposed openstack-infra/system-config: Remove logstash/elasticsearch classes, moved to puppet-openstackci https://review.openstack.org/240011 | 18:07 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: publish from openstack/releases to releases.openstack.org https://review.openstack.org/266515 | 18:07 |
| anteaya | <- gerrit 503 | 18:08 |
| dstanek | anteaya: me too | 18:08 |
| dstanek | anteaya: off and on it seems | 18:09 |
| lifeless | fungi: https://review.openstack.org/#/c/263297/ may have implications for infra; could you, or someone else in -core, eyeball and add some thoughts? | 18:09 |
| *** rkukura has quit IRC | 18:09 | |
| mordred | look! it's a lifeless | 18:09 |
| jeblair | clarkb, mordred: yeah, pretty sure it's an unrelated failure; not sure it's known though, the pr that caused it merged 1hr ago; i brought it up in #ansible-devel | 18:09 |
| lifeless | mordred: tis not | 18:09 |
| anteaya | dstanek: yes, we were discussing earlier the status of gerrit currently | 18:09 |
| anteaya | dstanek: thanks for sharing your experience as a datapoint | 18:09 |
| mordred | lifeless: so - I was going to ask you about a pbr design thing, but then I just took a stance and updated my patch | 18:09 |
| mtreinish | fungi: I did just realize we're probably doing the deletes backwards it should remove test_runs first then runs, but that shouldn't break anything | 18:09 |
| *** jsavak has quit IRC | 18:10 | |
| lifeless | mordred: of which patch doth you speak? | 18:10 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Switch to templating for nodepool rackspace https://review.openstack.org/264994 | 18:10 |
| mordred | lifeless: https://review.openstack.org/#/c/263748/ | 18:10 |
| openstackgerrit | Anne Gentle proposed openstack-infra/infra-manual: Updates Gerrit IRC Notifications section https://review.openstack.org/266508 | 18:10 |
| openstackgerrit | Randall Burt proposed openstack-infra/project-config: Fix syntax error in gate-heat-templates-dsvm https://review.openstack.org/266518 | 18:10 |
| mordred | lifeless: turns out, if you have tags in your git history that are not pbr versions, they show up in the changelog history anyway | 18:10 |
| mordred | lifeless: which I discovered adding tests for tags containing parens | 18:11 |
| fungi | mtreinish: it exited 137 and there's another 'Killed' appended to the end of the log | 18:11 |
| openstackgerrit | Randall Burt proposed openstack-infra/project-config: Fix syntax error in gate-heat-templates-dsvm https://review.openstack.org/266519 | 18:11 |
| fungi | mtreinish: ran much faster this time though | 18:11 |
| mtreinish | hmm, that's no good. I wonder what 137 means | 18:11 |
| *** mgould has quit IRC | 18:12 | |
| mtreinish | and I wonder what's killing it | 18:12 |
| pabelanger | jeblair: ^ I didn't touch the test nodes section this time. Going to wait until a follow up patch. | 18:12 |
| lifeless | mordred: I agree, if we don't honour it, its not changelog appropriate | 18:12 |
| nibalizer | clarkb: looks like all the f21 nodes are gone, so I'll approve 264379 unless you see a reason not to | 18:12 |
| clarkb | nibalizer: lets make sure the images are gone too | 18:13 |
| jeblair | pabelanger: ok | 18:13 |
| pabelanger | jeblair: Also wonder if we should have grafana-dev to allow us to generate dummy dashboards, so we can share them easily | 18:13 |
| pabelanger | since it is hard to see what a yaml file generates sometime | 18:13 |
| nibalizer | looks like we still have a 'devstack-f21' | 18:13 |
| AJaeger | nibalizer: clarkb wanted to approve himself | 18:14 |
| lifeless | mordred: now I'm wondering about ,'s in tags :) | 18:14 |
| clarkb | ya I wanted opportunity to check and remove any images | 18:14 |
| mtreinish | fungi: I'm gonna go get lunch, and I'll think about the next step. Because I'm kinda stumped as to what could be causing this | 18:14 |
| AJaeger | nibalizer: where do have a devstack-f21? | 18:14 |
| clarkb | if someone else does that and approves that works too | 18:14 |
| *** matrohon has quit IRC | 18:14 | |
| *** Paratosh has joined #openstack-infra | 18:14 | |
| pabelanger | timrc: up for reviewing 250893? | 18:14 |
| nibalizer | we have two devstack-f21 images per cloud provider | 18:15 |
| *** yamahata has joined #openstack-infra | 18:15 | |
| nibalizer | er per region | 18:15 |
| anteaya | X-IO CI has requested reactivation | 18:15 |
| lifeless | curse you gerrit, for breaking middle-mouse-paste-buffers | 18:16 |
| nibalizer | well hpcloud-b{1,2,3,4,5} and rax-{ord,iad,dfw} | 18:16 |
| anteaya | gerrit id 14969 | 18:16 |
| mtreinish | fungi: oh, wait isn't 137 the exit code after something gets SIGKILL? | 18:16 |
| *** angdraug has joined #openstack-infra | 18:16 | |
| anteaya | could an infra-root reactivate gerrit id 14969 please? | 18:16 |
| *** nmagnezi has joined #openstack-infra | 18:17 | |
| *** ddieterly has quit IRC | 18:17 | |
| *** unicell has joined #openstack-infra | 18:17 | |
| clarkb | nibalizer: so I think we can delete those and any associated dib images | 18:18 |
| clarkb | nibalizer: then merge the change | 18:18 |
| lifeless | mordred: reviewed | 18:18 |
| fungi | mtreinish: typically, yes, good point! checking dmesg reveals it's running out of memory | 18:18 |
| fungi | mtreinish: [Tue Jan 12 18:08:47 2016] Killed process 17353 (subunit2sql-db-) total-vm:3367220kB, anon-rss:1635812kB, file-rss:0kB | 18:19 |
| mordred | lifeless: I cannot see the difference in what you wrote in your first in-line comment and what's in the code | 18:19 |
| nibalizer | clarkb: okay i will do these things | 18:19 |
| mordred | lifeless: it's possible I just have old-man eyes | 18:19 |
| mtreinish | fungi: hmm, well thats no good | 18:19 |
| anteaya | mordred: where'd you get those? | 18:19 |
| clarkb | My font size keeps going up | 18:20 |
| fungi | mtreinish: physical memory on the system is 2gb and swap is another 2gb. ignoring buffers/cache you seem to have about 3gb open for this process to consume before it hits oom | 18:20 |
| mtreinish | fungi: I wasn't expecting it to do much locally, and just issue the SQL to do the deletes | 18:20 |
| *** tonytan4ever has joined #openstack-infra | 18:20 | |
| mtreinish | it is deleting a lot of data, so if it's doing some local caching of stuff that would explain it | 18:20 |
| lifeless | mordred: the second digit separator is a comma | 18:21 |
| mtreinish | I'll have to check the sqlalchemy docs, but I'll likely respin it to be a raw sql.execute() and push a release | 18:21 |
| lifeless | mordred: so 1.2,3 will be processed as 1.2 and thus a valid version | 18:21 |
| lifeless | mordred: when it is not | 18:21 |
| pleia2 | anteaya: didn't seen an ack, so doing that now | 18:21 |
| mordred | lifeless: yes, I grok that- I do not understand your comment: "--format=%h%x00%s%x00%d | 18:22 |
| anteaya | pleia2: sorry I lost context | 18:22 |
| lifeless | mordred: here's a loely test string for you: (HEAD, tag: f,b, tag: (12), master) | 18:22 |
| lifeless | mordred: oh, that was me working around the inability to copy-paste from the gerrit web ui | 18:22 |
| mordred | ok | 18:22 |
| anteaya | pleia2: did I ask something earlier that I clearly have forgotten? | 18:22 |
| mordred | good. then I understand your review | 18:22 |
| lifeless | mordred: I used the browser copy-paste to put it in a comment, and then selected and pasted from the un-broken edit buffer | 18:22 |
| mordred | lifeless: what about a tag named "tag:" ? | 18:22 |
| *** rkukura has joined #openstack-infra | 18:22 | |
| lifeless | mordred: ooh ooh I love | 18:23 |
| mordred | lifeless: :) | 18:23 |
| lifeless | mordred: but I think as it has to be a prefix it will probably pass tests right now | 18:23 |
| mordred | yah - if someone does that, they deserve whatever pain they find in life | 18:23 |
| pleia2 | anteaya: you asked that an account me made active again, it's done now | 18:24 |
| lifeless | mordred: what about a tag "1, tag: 2" | 18:24 |
| lifeless | mordred: aka little bobby tables | 18:24 |
| anteaya | pleia2: oh thank you | 18:24 |
| fungi | gah, so in gerrit 2.8 the successfully merged comment gets attributed to the account which made the submit api call rather than being an authorless system message. no wonder my data was all missing matches prior to the gerrit upgrade date | 18:24 |
| anteaya | rhedlind: pleia2 reactivated your account | 18:24 |
| anteaya | rhedlind: I'll post to the list | 18:25 |
| lifeless | oh nice, git protects us from that one | 18:25 |
| anteaya | fungi: that's fun | 18:25 |
| openstackgerrit | Merged openstack-infra/ciwatch: Separate votes from Jenkins queues https://review.openstack.org/258125 | 18:25 |
| lifeless | mordred: space and : are not allowed in tags by the git version I have here | 18:26 |
| anteaya | fungi: did I hear the sound of a deadline exploding? | 18:26 |
| jeblair | fungi: wasn't it always like that? (i thought jenkins always made our merge commits?) | 18:26 |
| mordred | lifeless: woot! | 18:26 |
| *** yamamoto has joined #openstack-infra | 18:26 | |
| clarkb | fungi: in this case the jenkins user | 18:26 |
| lifeless | mordred: so yeah, just comma and brackets to be defensive over | 18:26 |
| *** rkukura has quit IRC | 18:27 | |
| *** jaosorior has joined #openstack-infra | 18:27 | |
| clarkb | lifeless: note I think the version in precise was known to be a little too relaxed on how it handled git tags | 18:27 |
| *** zeih has joined #openstack-infra | 18:27 | |
| clarkb | thankfully precise is mostly gone so not a huge deal | 18:27 |
| *** yamamoto has quit IRC | 18:27 | |
| fungi | jeblair: the "Change has been successfully merged into the git repository." comments claim to come from jenkins and say "Change has been successfully merged into the git repository." | 18:27 |
| fungi | jeblair: er, redundancy mine | 18:28 |
| *** yamamoto has joined #openstack-infra | 18:28 | |
| fungi | jeblair: that's for things merged in 2.8 timeframe | 18:28 |
| openstackgerrit | Monty Taylor proposed openstack-dev/pbr: Split changelog on nulls instead of ( https://review.openstack.org/263748 | 18:28 |
| mordred | lifeless: ^^ there ya go | 18:28 |
| openstackgerrit | Merged openstack-infra/grafyaml: Add support for template query https://review.openstack.org/250893 | 18:28 |
| openstackgerrit | Merged openstack-infra/grafyaml: Add support for interval template type https://review.openstack.org/265487 | 18:28 |
| pabelanger | yolanda: jeblair: clarkb: any other objections for pyzmq uncap on zuul? https://review.openstack.org/#/c/262019/ | 18:28 |
| fungi | jeblair: for changes merged in 2.11 timeframe the comments are authorless but displayed as coming from the "Gerrit Code Review" account and say something like "Change has been successfully merged into the git repository by Jenkins" | 18:28 |
| lifeless | clarkb: yeah, I'm more worried about folk adding new torturous stuff to trees | 18:29 |
| cbader_ | I am trying to setup an external log file sever to replace one that is going away, I have run the common-ci steps for setting up the log server but when I try to do the test scp it doesn't accept the jenkins user is there something I am missing? | 18:29 |
| *** arxcruz has quit IRC | 18:29 | |
| jeblair | fungi: oh, comments, not commits | 18:29 |
| *** yamamoto has quit IRC | 18:29 | |
| *** yamamoto has joined #openstack-infra | 18:29 | |
| fungi | jeblair: yeah. so my ugly workaround for https://code.google.com/p/gerrit/issues/detail?id=3579 becomes more hideous still | 18:30 |
| anteaya | cbader_: have you a stacktrace of the error message that you can paste somewhere and share? | 18:30 |
| cbader_ | I get an error with the sudo puppet apply --verbose /etc/puppet/manifests/site.pp Error: Could not find data item domain in any Hiera data file and no default supplied at /etc/puppet/manifests/site.pp:24 on node ip-172-31-34-189.us-west-2.compute.internal | 18:30 |
| lifeless | mordred: so, I still don't understand what the split on ", " is needed for | 18:30 |
| *** rkukura has joined #openstack-infra | 18:30 | |
| lifeless | mordred: isn't it a no-op here ? | 18:30 |
| yolanda | pabelanger, i wanted to know the reason of the cap, and if that was tested properly. If that is, i'm fine | 18:30 |
| mordred | lifeless: it's not | 18:30 |
| pabelanger | yolanda: I left a note in the review, it was the initial commit the cap was placed. | 18:30 |
| pabelanger | yolanda: I believe the newer versions didn't exist at the time | 18:31 |
| yolanda | yep, saw it some days ago but didn't review, sorry | 18:31 |
| mordred | lifeless: becuase (HEAD, tag: 1.2.3, master) split on "tag:" will give you a value of "1.2.3, master" | 18:31 |
| yolanda | +2 now | 18:31 |
| *** yamamoto has quit IRC | 18:31 | |
| cbader_ | anteaya, I just get Permission denied (publickey). I have the jenkins key in the authorized_key file. | 18:31 |
| anteaya | cbader_: where did you get your code you are using? | 18:31 |
| cbader_ | anteaya, I can login into the server with the key so I know it works for that. | 18:31 |
| clarkb | cbader_: anteaya I think you may have to configure the jenkins scp plugin with a user to use and the key to use | 18:32 |
| *** zeih has quit IRC | 18:32 | |
| anteaya | ah thanks clarkb | 18:32 |
| cbader_ | clarkb, I haven't had to config a scp plugin for the jenkins user but I will look that up thanks | 18:32 |
| *** piet has quit IRC | 18:32 | |
| *** HeOS has quit IRC | 18:32 | |
| *** otter768 has joined #openstack-infra | 18:33 | |
| lifeless | mordred: hmm,not for me | 18:33 |
| *** tzn has quit IRC | 18:34 | |
| *** aopincar_ has joined #openstack-infra | 18:34 | |
| *** yamamoto has joined #openstack-infra | 18:34 | |
| cbader_ | clarkb, with the common-ci instructions I don't see where it creates the jenkins user since this is a remote log file server and jenkins is not present should I create a jenkins user? | 18:34 |
| *** eharney has joined #openstack-infra | 18:35 | |
| *** yamamoto has quit IRC | 18:35 | |
| *** krtaylor has quit IRC | 18:35 | |
| *** e0ne has joined #openstack-infra | 18:35 | |
| clarkb | cbader_: I am assuming this setup uses the jenkins scp plugin, the way that works is the jenkins master copies files from the slave then to the remote log server. It needs a user to log into the remote server with | 18:35 |
| mordred | lifeless: >>> "(HEAD, tag: 1.2.3, master)".split('tag:') | 18:36 |
| mordred | ['(HEAD, ', ' 1.2.3, master)'] | 18:36 |
| *** yamamoto has joined #openstack-infra | 18:36 | |
| lifeless | mordred: yeah, it was double tags I was pondering | 18:37 |
| *** rockyg has joined #openstack-infra | 18:37 | |
| mordred | ah | 18:37 |
| *** yamamoto has quit IRC | 18:37 | |
| *** otter768 has quit IRC | 18:37 | |
| *** yamamoto has joined #openstack-infra | 18:38 | |
| *** yamamoto has quit IRC | 18:38 | |
| lifeless | mordred: eviewed again | 18:39 |
| *** mriedem has quit IRC | 18:40 | |
| *** fawadkhaliq has quit IRC | 18:42 | |
| *** bardia has quit IRC | 18:42 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Create jobs for a wheel mirror https://review.openstack.org/164927 | 18:42 |
| krotscheck | Ok, I think ^^ makes the wheel mirrors happen | 18:42 |
| *** asselin_ has joined #openstack-infra | 18:42 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/release-tools: stop sending folks to launchpad milestones for release details https://review.openstack.org/266064 | 18:43 |
| *** mriedem has joined #openstack-infra | 18:43 | |
| *** aopincar_ has quit IRC | 18:43 | |
| *** yamamoto has joined #openstack-infra | 18:45 | |
| nibalizer | clarkb: i've approved 264379 so once it lands we're free of f21 | 18:46 |
| clarkb | nibalizer: and you lceaned up the images? | 18:46 |
| clarkb | or should I be doing that really quickly? | 18:46 |
| fungi | jeblair: oh, and this also means that we'll end up missing changes like https://review.openstack.org/245457 where one of us directly hit submit (that's probably safe, it's unlikely to have been anyone's only patch in a given window, so i'll just emit a warning) | 18:46 |
| nibalizer | clarkb: i cleaned the images and dib-images | 18:47 |
| *** yamamoto has quit IRC | 18:47 | |
| *** yamamoto_ has joined #openstack-infra | 18:47 | |
| clarkb | nibalizer: woot | 18:47 |
| *** jsavak has joined #openstack-infra | 18:48 | |
| *** yamamoto_ has quit IRC | 18:48 | |
| *** yamamoto has joined #openstack-infra | 18:49 | |
| *** ifarkas has quit IRC | 18:49 | |
| *** yamamoto has quit IRC | 18:50 | |
| *** yamamoto has joined #openstack-infra | 18:50 | |
| *** Guest84591 has quit IRC | 18:51 | |
| *** rossella_s has quit IRC | 18:52 | |
| *** yamamoto has quit IRC | 18:52 | |
| *** yamamoto has joined #openstack-infra | 18:52 | |
| *** rossella_s has joined #openstack-infra | 18:52 | |
| *** ayoung has joined #openstack-infra | 18:52 | |
| *** rkukura has quit IRC | 18:53 | |
| *** yamamoto has quit IRC | 18:53 | |
| *** piet has joined #openstack-infra | 18:53 | |
| *** yamamoto has joined #openstack-infra | 18:54 | |
| clarkb | I may have to rebase my hpcloud changes once that merges so will be sure to hceck on that | 18:55 |
| *** yamamoto has quit IRC | 18:55 | |
| *** yamamoto has joined #openstack-infra | 18:56 | |
| *** yamamoto has quit IRC | 18:56 | |
| *** mrmartin has quit IRC | 18:56 | |
| *** danpb has quit IRC | 18:57 | |
| *** yamamoto has joined #openstack-infra | 18:57 | |
| *** yamamoto has quit IRC | 18:57 | |
| *** sabeen has joined #openstack-infra | 18:58 | |
| AJaeger | any project-config core for an obvious typo review? https://review.openstack.org/266519 | 18:58 |
| openstackgerrit | Spencer Krum proposed openstack-infra/ansible-puppet: Conditionally manage puppet config files https://review.openstack.org/243399 | 18:58 |
| openstackgerrit | Merged openstack-infra/project-config: Remove Fedora 21 from nodepool https://review.openstack.org/264379 | 18:58 |
| jhesketh | Morning | 18:58 |
| *** yamamoto has joined #openstack-infra | 18:58 | |
| anteaya | morning jhesketh | 18:59 |
| clarkb | AJaeger: done | 18:59 |
| AJaeger | thanks, clarkb | 18:59 |
| *** tzn has joined #openstack-infra | 18:59 | |
| *** EricGonczer_ has joined #openstack-infra | 18:59 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config: Add Infra Cloud controller node https://review.openstack.org/209698 | 19:00 |
| *** jaosorior has quit IRC | 19:00 | |
| pleia2 | meeting time | 19:00 |
| *** yamamoto has quit IRC | 19:00 | |
| *** daemontool has joined #openstack-infra | 19:00 | |
| *** jaosorior has joined #openstack-infra | 19:00 | |
| *** bpokorny_ has joined #openstack-infra | 19:00 | |
| anteaya | crinkle: I was unable to get a gerrit url associated with a storyboard task, Zara says that functionality is not there yet but she also wants it to be there | 19:01 |
| anteaya | crinkle: so I couldn't do the thing I said I would do yesterday, sorry about that | 19:01 |
| *** rockyg has quit IRC | 19:02 | |
| *** EricGonc_ has quit IRC | 19:02 | |
| crinkle | anteaya: no problem, thank you for trying | 19:02 |
| *** arxcruz has joined #openstack-infra | 19:02 | |
| anteaya | sure, if you care I can let you know when I have any progress to report | 19:03 |
| openstackgerrit | Daniel Wallace proposed openstack-infra/shade: allow for saving of the initial admin password https://review.openstack.org/266532 | 19:03 |
| *** rkukura has joined #openstack-infra | 19:03 | |
| *** yamamoto has joined #openstack-infra | 19:03 | |
| Shrews | O.o | 19:03 |
| clarkb | Shrews: all your password are belong to shade | 19:04 |
| *** electrofelix has quit IRC | 19:04 | |
| *** sabeen3 has joined #openstack-infra | 19:04 | |
| *** EricGonc_ has joined #openstack-infra | 19:04 | |
| *** bpokorny has quit IRC | 19:04 | |
| clarkb | I think that may be a requirement for windows? | 19:04 |
| *** sabeen has quit IRC | 19:05 | |
| *** yamamoto has quit IRC | 19:05 | |
| openstackgerrit | Merged openstack-infra/project-config: Fix syntax error in gate-heat-templates-dsvm https://review.openstack.org/266519 | 19:05 |
| anteaya | that's passwords plus account numbers | 19:05 |
| *** yamamoto has joined #openstack-infra | 19:05 | |
| Shrews | this does not seem proper | 19:05 |
| *** amitgandhinz has quit IRC | 19:05 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config: Split puppet installation into its own element https://review.openstack.org/198417 | 19:05 |
| *** amitgandhinz has joined #openstack-infra | 19:06 | |
| Shrews | shade is not a in-memory data store... but maybe it should be | 19:06 |
| *** yamamoto has quit IRC | 19:06 | |
| *** julim has quit IRC | 19:06 | |
| *** trown|lunch is now known as trown | 19:06 | |
| * Shrews repurposes mordred as in-memory organism | 19:06 | |
| *** yamamoto has joined #openstack-infra | 19:07 | |
| clarkb | Shrews: the issue aiui is nova will only give you that passwd on the first boot repsonse | 19:07 |
| clarkb | if you show the host later it won't have that info | 19:07 |
| clarkb | since shade is booting the machines it has to get the passwd and expose it | 19:07 |
| clarkb | the alternative is to use glean/cloud-init/windows-cloud-init | 19:07 |
| *** julim has joined #openstack-infra | 19:08 | |
| *** rockyg has joined #openstack-infra | 19:08 | |
| *** EricGonczer_ has quit IRC | 19:08 | |
| *** yamamoto has quit IRC | 19:08 | |
| *** ayoung has quit IRC | 19:08 | |
| *** yamamoto has joined #openstack-infra | 19:08 | |
| Shrews | clarkb: since server data can be cached, we now get into security issues, which is my concern | 19:09 |
| *** yamamoto has quit IRC | 19:09 | |
| *** bardia has joined #openstack-infra | 19:09 | |
| clarkb | yup it definitely exposes a security issue | 19:09 |
| *** ashaw has joined #openstack-infra | 19:09 | |
| clarkb | and the alternative is quite honestly much better | 19:09 |
| openstackgerrit | Ben Swartzlander proposed openstack/requirements: Raise manilaclient upper to 1.6.0 https://review.openstack.org/266534 | 19:09 |
| *** ashaw has left #openstack-infra | 19:10 | |
| *** yamamoto has joined #openstack-infra | 19:10 | |
| *** yamamoto has quit IRC | 19:11 | |
| *** yamamoto has joined #openstack-infra | 19:12 | |
| *** yamamoto has quit IRC | 19:12 | |
| *** gyee has quit IRC | 19:13 | |
| *** ihrachys has quit IRC | 19:14 | |
| *** bpokorny_ has quit IRC | 19:15 | |
| *** yamamoto has joined #openstack-infra | 19:15 | |
| openstackgerrit | amrith proposed openstack/requirements: Update global-requirements.txt https://review.openstack.org/266535 | 19:15 |
| *** bpokorny has joined #openstack-infra | 19:15 | |
| *** yamamoto has quit IRC | 19:16 | |
| openstackgerrit | amrith proposed openstack/requirements: Update global-requirements.txt https://review.openstack.org/266535 | 19:16 |
| *** yamamoto has joined #openstack-infra | 19:17 | |
| *** salv-orlando has quit IRC | 19:17 | |
| *** salv-orlando has joined #openstack-infra | 19:18 | |
| *** yamamoto has quit IRC | 19:18 | |
| openstackgerrit | Daniel Wallace proposed openstack-infra/shade: allow for saving of the initial admin password https://review.openstack.org/266532 | 19:18 |
| *** mrmartin has joined #openstack-infra | 19:19 | |
| *** yamamoto has joined #openstack-infra | 19:20 | |
| *** rockyg has quit IRC | 19:20 | |
| openstackgerrit | James E. Blair proposed openstack-infra/infra-specs: Add some assignees to zuulv3 https://review.openstack.org/266538 | 19:20 |
| *** yamamoto has quit IRC | 19:21 | |
| *** yamamoto has joined #openstack-infra | 19:22 | |
| *** yamamoto has quit IRC | 19:23 | |
| *** yamamoto has joined #openstack-infra | 19:23 | |
| *** yamamoto has quit IRC | 19:25 | |
| *** yamamoto has joined #openstack-infra | 19:25 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/infra-specs: Make translation set up consistent https://review.openstack.org/262545 | 19:25 |
| *** rkukura has quit IRC | 19:25 | |
| *** yamamoto has quit IRC | 19:26 | |
| *** rkukura has joined #openstack-infra | 19:26 | |
| *** yamamoto has joined #openstack-infra | 19:27 | |
| *** yamamoto has quit IRC | 19:27 | |
| *** nmagnezi has quit IRC | 19:28 | |
| *** rkukura has quit IRC | 19:28 | |
| *** yamamoto has joined #openstack-infra | 19:28 | |
| AJaeger | EmilienM: could you review https://review.openstack.org/#/c/266482/ ,please? | 19:28 |
| *** yamamoto has quit IRC | 19:29 | |
| *** MarkAtwo_ has joined #openstack-infra | 19:30 | |
| *** MarkAtwood has quit IRC | 19:30 | |
| *** yamamoto has joined #openstack-infra | 19:30 | |
| *** ybathia has joined #openstack-infra | 19:30 | |
| *** baoli has quit IRC | 19:31 | |
| *** yamamoto_ has joined #openstack-infra | 19:31 | |
| *** jaosorior has quit IRC | 19:31 | |
| *** yamamoto has quit IRC | 19:31 | |
| *** jaosorior has joined #openstack-infra | 19:32 | |
| *** mrmartin has quit IRC | 19:32 | |
| *** yamamoto_ has quit IRC | 19:32 | |
| *** austin81 has left #openstack-infra | 19:34 | |
| *** rkukura has joined #openstack-infra | 19:35 | |
| *** yamamoto has joined #openstack-infra | 19:35 | |
| *** yamamoto has quit IRC | 19:36 | |
| *** yamamoto has joined #openstack-infra | 19:37 | |
| *** _nadya_ has joined #openstack-infra | 19:37 | |
| *** mrunge has quit IRC | 19:37 | |
| *** ddieterly has joined #openstack-infra | 19:37 | |
| *** yamamoto has quit IRC | 19:38 | |
| *** yamamoto has joined #openstack-infra | 19:38 | |
| *** yamamoto has quit IRC | 19:39 | |
| *** yamamoto has joined #openstack-infra | 19:40 | |
| *** julim has quit IRC | 19:40 | |
| *** armax has quit IRC | 19:40 | |
| *** mrmartin has joined #openstack-infra | 19:40 | |
| *** ayoung has joined #openstack-infra | 19:41 | |
| openstackgerrit | Daniel Wallace proposed openstack-infra/shade: Save the adminPass if returned on server create https://review.openstack.org/266532 | 19:41 |
| *** yamamoto has quit IRC | 19:41 | |
| *** julim has joined #openstack-infra | 19:41 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config: Use file permissions instead of ownership to grant access to files https://review.openstack.org/266482 | 19:42 |
| *** yamamoto has joined #openstack-infra | 19:43 | |
| *** Sukhdev has joined #openstack-infra | 19:43 | |
| *** yamamoto has quit IRC | 19:45 | |
| *** yamamoto has joined #openstack-infra | 19:45 | |
| *** yamamoto has quit IRC | 19:46 | |
| *** armax has joined #openstack-infra | 19:46 | |
| *** yamamoto has joined #openstack-infra | 19:47 | |
| *** yamamoto has quit IRC | 19:47 | |
| *** yamamoto has joined #openstack-infra | 19:48 | |
| *** HeOS has joined #openstack-infra | 19:49 | |
| *** yamamoto has quit IRC | 19:49 | |
| *** yamamoto has joined #openstack-infra | 19:50 | |
| *** armax has quit IRC | 19:51 | |
| *** yamamoto has quit IRC | 19:51 | |
| *** yamamoto has joined #openstack-infra | 19:51 | |
| openstackgerrit | Chris Jones proposed openstack-infra/project-config: Add project configuration for Vahana. https://review.openstack.org/266549 | 19:52 |
| *** yamamoto has quit IRC | 19:52 | |
| *** bharathm has quit IRC | 19:52 | |
| *** yamamoto has joined #openstack-infra | 19:53 | |
| EmilienM | AJaeger: ack | 19:54 |
| *** yamamoto_ has joined #openstack-infra | 19:55 | |
| EmilienM | AJaeger: +1 I think permissions are good | 19:55 |
| AJaeger | thanks, EmilienM. I know how often you had to update that part so wanted you to be comfortable with it ;) | 19:56 |
| *** yamamoto_ has quit IRC | 19:56 | |
| *** yamamoto_ has joined #openstack-infra | 19:56 | |
| EmilienM | AJaeger: yeah, thanks a lot | 19:57 |
| *** ddieterly has quit IRC | 19:57 | |
| *** ddieterly has joined #openstack-infra | 19:58 | |
| *** yamamoto_ has quit IRC | 19:58 | |
| *** mrmartin has quit IRC | 19:58 | |
| *** yamamoto_ has joined #openstack-infra | 19:58 | |
| *** yamamoto has quit IRC | 19:58 | |
| *** yamamoto_ has quit IRC | 19:59 | |
| pleia2 | thanks fungi! | 20:00 |
| *** ddieterly has quit IRC | 20:00 | |
| *** yamamoto has joined #openstack-infra | 20:00 | |
| ianw | clarkb / greghaynes : where have we ended up with the current dib image build breakage? it would be really nice to get things going again | 20:00 |
| clarkb | ianw: we need a dib release with the fixes aiui | 20:00 |
| clarkb | and for rackspace uploads we need to update shade and restart nodepoold | 20:00 |
| *** yamamoto has quit IRC | 20:00 | |
| ianw | clarkb: right, https://review.openstack.org/#/c/266086/ is still outstanding (/ping SpamapS ?) | 20:01 |
| *** rockyg has joined #openstack-infra | 20:02 | |
| *** yamamoto has joined #openstack-infra | 20:02 | |
| *** yamamoto has quit IRC | 20:02 | |
| krotscheck | dstufft: If I have a wheel mirror at one URL, and a regular mirror at a different URL, should I do a pip --index-url=<wheel> --extra-index-url=<simple>, the other way around, or should I be using --find-links? | 20:02 |
| SpamapS | ianw: +A | 20:02 |
| *** rfolco has quit IRC | 20:02 | |
| greghaynes | krotscheck: other way around, --index-url=simple --extra-index-url=wheel | 20:03 |
| craige | Ran out of time in the infra meeting so I'll drop a Phabricator migration update here ( mordred ttx ). There's two outstanding issues I'm aware of: 1. A regression causing comments to no longer migrate 2. Usage of fake email addresses blocking matching of usernames to Launchpad usernames. Currently randomnly generating those that can't be matched. | 20:03 |
| krotscheck | Additional constraint: The directory structure might not fully match the <foldername>/<package.wheel> pattern. | 20:03 |
| greghaynes | krotscheck: I think technically it doesn't matter, but that is how I would do it | 20:03 |
| *** yamamoto has joined #openstack-infra | 20:03 | |
| krotscheck | greghaynes: Well, I want to make sure that it grabs wheels first. Extra implies secondary, so I wanted to make sure. | 20:03 |
| fungi | ianw: on the bindep stuff, right now we have a dib element which caches things mentioned in the fallback list and any repo-specific bindep lists. it was never implemented as a nodepool prep script because the assumption was that we'd have dib-built images uploadable everywhere before we were ready to push forward with bindep in voting jobs. next step is probably to temporarily duplicate | 20:03 |
| fungi | http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/elements/cache-bindep so that it works for snapshot-based image builds too | 20:03 |
| craige | It would be good to have some discussion time around Phabricator next week fungi. I have some direction questions for the team. | 20:04 |
| *** yamamoto has quit IRC | 20:04 | |
| fungi | craige: i'll add it to the agenda, but feel free to ask in here before then | 20:04 |
| craige | Will do :-) | 20:04 |
| dstufft | krotscheck: the order of --index-url and --extra-index-url doesn't really matter (techinally there is a "implementation defined thing where it matters, but only if both indexes provide a file of the same precedence) | 20:05 |
| *** yamamoto has joined #openstack-infra | 20:05 | |
| krotscheck | dstufft: Cool. | 20:05 |
| dstufft | however the format for --find-links and (--extra)--index-url are different and you have to use the correct one for that | 20:05 |
| anteaya | dhellmann: I have to depart soon for an errand, I would like to be aware of any discussion regarding where release tools end up so I uphold my end of making what ever decision work | 20:05 |
| *** yamamoto has quit IRC | 20:05 | |
| dstufft | index urls are assumed to follow the format as laid down in PEP 503 | 20:06 |
| fungi | anteaya: we agreed to continue discussion sometime tomorrow | 20:06 |
| dstufft | find-links are assumed to follow the format of "the url you give us links to every single file" | 20:06 |
| krotscheck | dstufft: Ok, so the current thought is that the real folder structure is /s/sp/split/split.whl style, to support AFS practical folder limits, but to make it look like /split/split.whl via mod_rewrite. Do you see any issues with this? | 20:06 |
| anteaya | fungi: okay thank you, I expect to be around tomorrow | 20:06 |
| krotscheck | dstufft: Wow, every file. | 20:06 |
| anteaya | fungi: I'm fine with whatever is decided I just want to ensure I know what is expected of me to make it work | 20:07 |
| * krotscheck reads PEP503 | 20:07 | |
| ianw | fungi: ah yes, i remember that, with the yum download only stuff | 20:07 |
| dstufft | PEP 503 boils down to $ROOT/$NORMALIZED_NAME/ | 20:07 |
| dstufft | must link to every file for $NORMALIZED_NAME | 20:07 |
| krotscheck | dstufft: Ok, so I'll have to figure out a way to make that directory structure show up correctly in apache, while also being sane for AFS. | 20:08 |
| *** yamamoto has joined #openstack-infra | 20:08 | |
| *** Keedya_ has joined #openstack-infra | 20:08 | |
| *** yamamoto has quit IRC | 20:09 | |
| krotscheck | dstufft: I assume that pip reads $ROOT/ to see what packages are available? | 20:10 |
| *** yamamoto has joined #openstack-infra | 20:10 | |
| dstufft | krotscheck: maybe. | 20:10 |
| dstufft | it depends on what version of pip we're talking about | 20:10 |
| *** yamamoto has quit IRC | 20:10 | |
| dstufft | all versions of pip will do $ROOT/$NAME/, in older versions of pip $NAME was "exactly what the user typed", in newer versions $NAME is "a normalized version of the name so pip and other tools can know if it should be Django or django" | 20:11 |
| krotscheck | So, newer versions would read the $ROOT directory listing, normalize all those names, and index off of that? | 20:12 |
| dstufft | In older versions of pip, if $ROOT/$NAME/ returned a 404, it would fall back to fetching $ROOT/ and normalized the names there to fiscover the link | 20:12 |
| dstufft | however in newer versions that has been deprecated... and I think pip 8 might be the one that completely removes it | 20:12 |
| dstufft | mostly because a 3MB download penality because you fat fingered a name is crappy | 20:13 |
| krotscheck | dstufft: point | 20:14 |
| *** lucasagomes is now known as lucas-dinner | 20:14 | |
| * krotscheck is thecking trusty pip version | 20:15 | |
| ianw | anyone know why trying to sign up for an OVH account would result in a red bar with "error_country_values"? | 20:16 |
| *** ldnunes has quit IRC | 20:16 | |
| dstufft | krotscheck: 1.5.something | 20:16 |
| dstufft | 1.5.6? | 20:16 |
| krotscheck | dstufft: 1.5.4 | 20:16 |
| dstufft | + random patches because that's how distros roll | 20:16 |
| anteaya | ianw: I do not know, this is via the ovh website? | 20:16 |
| *** yamamoto has joined #openstack-infra | 20:17 | |
| ianw | anteaya: yes. seems they don't like australians | 20:17 |
| *** yamamoto has quit IRC | 20:17 | |
| krotscheck | dstufft: Debian sid is 1.5.6 | 20:17 |
| krotscheck | So, one or two versions behind | 20:17 |
| * krotscheck is understating for comedic effect. | 20:17 | |
| anteaya | ianw: interesting, well I'm sure they like your money same as anyone else's | 20:17 |
| *** yamamoto has joined #openstack-infra | 20:18 | |
| anteaya | ianw: I don't know as we have an ovh contact in channel | 20:18 |
| dstufft | krotscheck: to be fair, we switched version numbering in between 1.5 and 8 | 20:18 |
| dstufft | 1.5 would have been pip 5 in the new version numbering | 20:18 |
| anteaya | ianw: I think we mostly communicate with them via email | 20:18 |
| clarkb | I want to say as a french company they have extra rules to abide by | 20:18 |
| *** Sukhdev has quit IRC | 20:18 | |
| clarkb | which may be related? | 20:18 |
| anteaya | they have a data center in canada | 20:19 |
| anteaya | perhaps their canadian data centre can take australian money | 20:19 |
| clarkb | anteaya: typically if you have a company whose headquarters are in one country they are still bound to those rules even when doing business elsewhere | 20:19 |
| anteaya | true | 20:19 |
| clarkb | as an American I can't legally bribe people in countries where such things are legal | 20:19 |
| *** yamamoto has quit IRC | 20:19 | |
| anteaya | they have more customers than amazon according to a graph I saw somewhere | 20:20 |
| clarkb | (granted the just take my money for real services is another category of what) | 20:20 |
| anteaya | they must have figured it out so as to do business with other countries | 20:20 |
| krotscheck | dstufft: So, one or two versions :) | 20:20 |
| SamYaple | clarkb: not with taht attitude | 20:20 |
| *** Swami has quit IRC | 20:20 | |
| ianw | we'll see if they believe i live in 90210 | 20:20 |
| anteaya | ha ha ha | 20:20 |
| krotscheck | dstufft: Ok, so it sounds like I'm going to have to generate a sane index. | 20:20 |
| clarkb | SamYaple: I just need to pay someone else to lobby for me >_> | 20:20 |
| anteaya | bribery is legal some places? | 20:21 |
| SamYaple | oh political burn | 20:21 |
| clarkb | anteaya: yes, in much of the world it is not only legal but expected | 20:21 |
| *** yamamoto has joined #openstack-infra | 20:21 | |
| *** dslev has joined #openstack-infra | 20:22 | |
| * clarkb recounts the annual corportate "don't bribe people" training | 20:22 | |
| anteaya | wow | 20:22 |
| *** bharathm has joined #openstack-infra | 20:22 | |
| *** arxcruz has quit IRC | 20:22 | |
| anteaya | oh yeah I saw the vid but I didn't think that it was legal | 20:22 |
| anteaya | expected is based on who you do business with | 20:22 |
| *** piet has quit IRC | 20:22 | |
| mordred | anteaya: in some places, the laws written on paper are not the actual laws | 20:22 |
| openstackgerrit | yolanda.robla proposed openstack-infra/ansible-puppet: Conditionally manage puppet config files https://review.openstack.org/243399 | 20:23 |
| anteaya | someone expecting me to sell them drugs can expect all they want, they will be disapointed | 20:23 |
| *** yamamoto has quit IRC | 20:23 | |
| anteaya | mordred: ah okay so yes, the paper laws don't make bribery legal | 20:23 |
| clarkb | anteaya: oh because I am remembering it, someone I went to high school is now in your parliament | 20:23 |
| mordred | anteaya: sure - but it's possible then that you might avoid that by avoiding going certain places | 20:23 |
| anteaya | just the culture of business for that country | 20:23 |
| *** dprince has quit IRC | 20:23 | |
| *** yamamoto has joined #openstack-infra | 20:23 | |
| mordred | yes | 20:23 |
| anteaya | clarkb: do tell | 20:23 |
| *** yamamoto has quit IRC | 20:24 | |
| anteaya | I lived in some very seedy places | 20:24 |
| mordred | anteaya: might not just be business - in some places you MUST bribe a government official to do things | 20:24 |
| clarkb | anteaya: https://en.wikipedia.org/wiki/Alupa_Clarke | 20:24 |
| clarkb | he doesn't represent you | 20:24 |
| mordred | anteaya: which I'd argue makes bribery de-facto legal | 20:24 |
| anteaya | don't look like you are in the marked for drugs or prostitues and they treat you like you don't exist | 20:24 |
| anteaya | everyone is happy | 20:24 |
| *** yamamoto has joined #openstack-infra | 20:25 | |
| anteaya | clarkb: dude represents part of Quebec City | 20:25 |
| anteaya | mordred: okay fair argument | 20:25 |
| jeblair | clarkb, fungi: any thougts on https://etherpad.openstack.org/p/austin-upstream-dev-track-ideas ? | 20:25 |
| *** dslev has quit IRC | 20:25 | |
| clarkb | jeblair: I was thinking "get a new job in the gate" might make a good topic | 20:25 |
| mordred | anteaya: also, I agree about not looking like you are in the market | 20:25 |
| clarkb | jeblair: also wading through all of the data we collect to discover and fix bugs | 20:26 |
| jeblair | clarkb: oh both good | 20:26 |
| mordred | anteaya: with the exception being the places where being in the place is the qualifying criteria for what a person needs to do to look like they are in the market | 20:26 |
| mordred | anteaya: the bar area of Tijuana from 10 years ago comes to mind | 20:26 |
| anteaya | mordred: yes, I walked right past crack dealers alone at 5am, no problem since I was clearly communicating I was not a consumer | 20:26 |
| mordred | anteaya: yup | 20:26 |
| anteaya | mordred: fair enough | 20:26 |
| anteaya | my area was crack houses and film studios | 20:27 |
| clarkb | jeblair: where that might a story from "graphite/health dashboard say test foo fails a lot", use wget to collect the logs for a recent run, find error cause, write logstash query, drill in to see if it is that one problem or many, eyc | 20:27 |
| mordred | anteaya: yah - behavior is variable there | 20:27 |
| anteaya | I communicated the film studio leaning | 20:27 |
| *** yamamoto has quit IRC | 20:27 | |
| *** yamamoto_ has joined #openstack-infra | 20:27 | |
| anteaya | jeblair: is anyone signing up to lead these tracks? | 20:28 |
| *** yamamoto_ has quit IRC | 20:28 | |
| fungi | jeblair: thanks--it's about all i can do to keep up with discussions in this channel and -meeting simultaneously ;) | 20:28 |
| *** jsavak has quit IRC | 20:28 | |
| *** tonytan4ever has quit IRC | 20:28 | |
| *** yamamoto has joined #openstack-infra | 20:29 | |
| jeblair | anteaya: i think track chairs were collected last tc meeting? | 20:29 |
| *** yamamoto has quit IRC | 20:29 | |
| fungi | jeblair: as someone who rarely has time to attend non-design-summit tracks i have a hard time guessing at what the audience will appreciate, but i did agree to try and drum up some interested infra team members to give talks (if there are any--we seem to have some people who like to talk) | 20:29 |
| *** jsavak has joined #openstack-infra | 20:29 | |
| anteaya | jeblair: sorry I mean is anyone agreeing to lead any of these suggested sessions | 20:30 |
| *** yamamoto has joined #openstack-infra | 20:30 | |
| jeblair | anteaya: this is a main conference presentation track | 20:30 |
| anteaya | really? | 20:31 |
| jeblair | anteaya: there will be proposals as normal, but the tc is trying to decide on a direction they'd like to steer things | 20:31 |
| anteaya | I thought thsi was just for developers | 20:31 |
| *** yamamoto has quit IRC | 20:31 | |
| anteaya | jeblair: ah okay | 20:31 |
| anteaya | I didn't think this would be in the main conference | 20:31 |
| jeblair | anteaya: it's for developers on a non-summit day | 20:31 |
| *** _tzn has joined #openstack-infra | 20:31 | |
| *** MarkAtwo_ has quit IRC | 20:31 | |
| anteaya | yes that is what I had thought | 20:32 |
| krotscheck | jeblair: Do you have any suggestions on how to generate an HTML5 index page of that AFS directory structure? I'm reading through the mod_autoindex documentation that fungi suggested some teim ago and I don't think it's going to do what I need it to (I may be blind). | 20:33 |
| *** yamamoto has joined #openstack-infra | 20:34 | |
| *** otter768 has joined #openstack-infra | 20:34 | |
| *** yamamoto has quit IRC | 20:34 | |
| clarkb | krotscheck: mod_autoindex should work since afs presents itself like a file system | 20:34 |
| *** tzn has quit IRC | 20:34 | |
| *** topolznc has joined #openstack-infra | 20:35 | |
| *** yamamoto has joined #openstack-infra | 20:35 | |
| krotscheck | clarkb: PEP503, and the old version of pip available on debian*, requires that the various indexes be ROOT/packagename/*, and that an index.html file lists them all. The AFS folder structure we're using is ROOT/p/pa/packagename/* | 20:36 |
| krotscheck | I'm currently trying to figure out if I can just run a bash script and have apache call that. | 20:36 |
| krotscheck | (Seems to work) | 20:36 |
| *** yamamoto has quit IRC | 20:37 | |
| krotscheck | Wait, I'm wrong. It's ROOT/Distro-DistroVersion/p/pa/packagename | 20:37 |
| *** yamamoto has joined #openstack-infra | 20:37 | |
| *** rguillebert has quit IRC | 20:37 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Disable session synchronization on deletes https://review.openstack.org/266562 | 20:37 |
| mtreinish | fungi: ^^^ I think that should fix the OOM killer issues | 20:37 |
| *** yamamoto has quit IRC | 20:38 | |
| mtreinish | I've got a release all queue up to go so I'll include that patch in it | 20:38 |
| *** yamamoto has joined #openstack-infra | 20:38 | |
| *** otter768 has quit IRC | 20:39 | |
| anteaya | annegentle: jimmy macarthur is the one who answers my bugs to the openstack.org website about seo issues | 20:39 |
| anteaya | annegentle: like not being able to search videos and find what I am looking for | 20:39 |
| *** yamamoto has quit IRC | 20:39 | |
| *** tonytan4ever has joined #openstack-infra | 20:39 | |
| *** salv-orlando has quit IRC | 20:39 | |
| clarkb | krotscheck: I see, so you need a pep503 compliant index onto the other format | 20:40 |
| anteaya | annegentle: and not being able to find links to git.openstack.org when searching for openstack project source code | 20:40 |
| krotscheck | clarkb: Yes | 20:40 |
| krotscheck | The rewrite is already there. | 20:40 |
| krotscheck | So to HTTP requests it'll look correct. | 20:40 |
| *** yamamoto has joined #openstack-infra | 20:40 | |
| clarkb | but for humans navigating you need an index | 20:40 |
| *** salv-orlando has joined #openstack-infra | 20:40 | |
| *** yamamoto has quit IRC | 20:41 | |
| annegentle | anteaya: cool thanks | 20:41 |
| krotscheck | clarkb: So from ^^ discussion, older versions of pip (pre 8) initially do a request for /ROOT/THING/*, and if that got fat fingered, will request /ROOT/ to figure out if there's a normalized version (Babel vs. babel) | 20:41 |
| *** bharathm has quit IRC | 20:41 | |
| anteaya | 5 deer on my lawn | 20:42 |
| krotscheck | So, yes, humans can use it, but it's really for Pip. | 20:42 |
| anteaya | usually just get 2 at a time | 20:42 |
| anteaya | annegentle: welcome, I hope you get something useful | 20:42 |
| *** yamamoto has joined #openstack-infra | 20:43 | |
| *** jaosorior has quit IRC | 20:44 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: WIP: Add new django extraction https://review.openstack.org/266565 | 20:44 |
| *** bharathm has joined #openstack-infra | 20:44 | |
| *** _nadya_ has quit IRC | 20:45 | |
| AJaeger | pleia2, clarkb : here's an implementation of the django extraction for the spec ^. I plan to update the change over the next days to see that the can be implemented ;) | 20:45 |
| fungi | mtreinish: the synchronize_session parameter takes a string but uses boolean False to disable rather than empty string or None? | 20:45 |
| *** yamamoto has quit IRC | 20:45 | |
| fungi | mtreinish: and defaults to something not disabled? | 20:45 |
| *** yamamoto has joined #openstack-infra | 20:45 | |
| *** yamamoto has quit IRC | 20:46 | |
| mtreinish | fungi: that's how I read the docs: http://docs.sqlalchemy.org/en/latest/orm/query.html#sqlalchemy.orm.query.Query.delete | 20:46 |
| *** rockyg has quit IRC | 20:46 | |
| mtreinish | it did seem really weird | 20:46 |
| *** yamamoto has joined #openstack-infra | 20:47 | |
| fungi | zzzeek probably has a good reason for it being that way | 20:47 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: WIP: Add new django extraction https://review.openstack.org/266565 | 20:47 |
| mtreinish | heh, yeah I wasn't questioning it. Just using it incorrectly :) | 20:47 |
| *** yamamoto has quit IRC | 20:47 | |
| *** amuller is now known as amuller_afk | 20:47 | |
| *** cburgess_ has quit IRC | 20:48 | |
| *** mudassirlatif has quit IRC | 20:48 | |
| *** matrohon has joined #openstack-infra | 20:48 | |
| *** cburgess has joined #openstack-infra | 20:48 | |
| fungi | yep, lgtm | 20:49 |
| zzzeek | fungi: which part, the default setting, or the mixture of False / strings | 20:50 |
| *** yamamoto has joined #openstack-infra | 20:50 | |
| fungi | zzzeek: mixture of False and strings. just seemed weird | 20:50 |
| *** rockyg has joined #openstack-infra | 20:50 | |
| fungi | but as long as it works, no complaints from me | 20:51 |
| zzzeek | fungi: theres things like that in many places, sometimes because it started as True/False and then we needed extra states | 20:51 |
| *** yamamoto has quit IRC | 20:51 | |
| zzzeek | fungi: main example is relationship() -> lazy=True|False | 20:51 |
| fungi | simply triggered my "are you sure?" filter | 20:51 |
| fungi | makes sense | 20:51 |
| *** topolznc is now known as topol | 20:52 | |
| zzzeek | fungi: for synchronize_session, it probably shoudl be the value 'never' | 20:52 |
| *** topol is now known as Guest45335 | 20:52 | |
| *** daemontool has quit IRC | 20:52 | |
| *** sjmc7 has joined #openstack-infra | 20:53 | |
| zzzeek | fungi: there's at least five parameters where a True|False|<string> thing is happening, most for legacy reasons. ideas welcome | 20:54 |
| zzzeek | in most cases we deprecate the True|False aspect | 20:54 |
| zzzeek | like this one http://docs.sqlalchemy.org/en/rel_1_0/core/pooling.html?highlight=reset_on_return#sqlalchemy.pool.Pool.params.reset_on_return | 20:55 |
| fungi | zzzeek: yeah, deprecation was my only real suggestion | 20:55 |
| *** krtaylor has joined #openstack-infra | 20:55 | |
| *** shardy has quit IRC | 20:55 | |
| zzzeek | http://docs.sqlalchemy.org/en/rel_1_0/core/metadata.html?highlight=autoincrement#sqlalchemy.schema.Column.params.autoincrement | 20:55 |
| fungi | but again, not a big deal, just caused me to want to double-check the docs | 20:56 |
| zzzeek | http://docs.sqlalchemy.org/en/rel_1_0/orm/relationship_api.html?highlight=passive_deletes#sqlalchemy.orm.relationship.params.passive_deletes | 20:56 |
| * zzzeek is just finding them all | 20:56 | |
| clarkb | something looks broken with nodepool/zuul | 20:56 |
| *** hongbin has joined #openstack-infra | 20:56 | |
| *** yamamoto has joined #openstack-infra | 20:57 | |
| clarkb | KeyError: 'nodepool.label.devstack-fedora21-dib.nodes.delete' | 20:57 |
| clarkb | nibalizer: ^ | 20:57 |
| clarkb | not sure if that is related yes | 20:57 |
| *** Guest45335 is now known as topol_ | 20:57 | |
| *** bcourt has quit IRC | 20:57 | |
| *** topol_ is now known as Guest12581 | 20:58 | |
| clarkb | ok ya fun | 20:58 |
| *** Guest12581 is now known as topol2 | 20:58 | |
| *** yamamoto has quit IRC | 20:58 | |
| *** yamamoto_ has joined #openstack-infra | 20:58 | |
| zzzeek | i think that's it | 20:59 |
| clarkb | I believe the issue is that when we try to process the state for new nodes it already has a key for the fedora21 stuffwhen trying to update graphite | 20:59 |
| clarkb | thats a new one | 20:59 |
| dims | zuul's busted i guess? | 20:59 |
| clarkb | dims: not zuul, nodepool | 20:59 |
| *** yamamoto_ has quit IRC | 20:59 | |
| *** jsavak has quit IRC | 20:59 | |
| dims | clarkb : ah ok | 20:59 |
| clarkb | we havne't run into this bceause while we have removed things before we hanve't completely removed a flavor | 20:59 |
| *** erikwilson has joined #openstack-infra | 20:59 | |
| *** ddieterly has joined #openstack-infra | 20:59 | |
| dims | clarkb : ack thanks! | 21:00 |
| *** yamamoto has joined #openstack-infra | 21:00 | |
| clarkb | I *think* this will be fixed if we remove 6697292 and restart nodepool | 21:00 |
| *** amrith is now known as _amrith_ | 21:00 | |
| *** jsavak has joined #openstack-infra | 21:00 | |
| * clarkb attempts the delete first | 21:00 | |
| hongbin | Hi all. A question. Magnum gate currently ran into an error due to lack of disk space. In particular, the nodes "devstack-trusty-bluebox-*" have 57G disk which is far less than other nodes | 21:01 |
| *** armax has joined #openstack-infra | 21:01 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: WIP: Add new django extraction https://review.openstack.org/266565 | 21:01 |
| clarkb | mordred: ^ were yoiu going to modify the flavor? | 21:01 |
| *** yamamoto has quit IRC | 21:01 | |
| clarkb | hongbin: its a known issue, mordred mentioned we hvae control of the flavor but not sure how we actually get it changed | 21:02 |
| *** yamamoto has joined #openstack-infra | 21:02 | |
| hongbin | clarkb: ack | 21:02 |
| *** yamamoto has quit IRC | 21:02 | |
| clarkb | ok that fedora21-dib node was deleted, nodepool seems like it may be happier just wit hthat | 21:02 |
| *** dslevin has joined #openstack-infra | 21:03 | |
| *** rlandy has quit IRC | 21:03 | |
| *** yamamoto has joined #openstack-infra | 21:03 | |
| *** dslevin has quit IRC | 21:04 | |
| clarkb | timrc: jlk maybe you know hwat is involved in changing the flavor size on that bluebox cloud? | 21:04 |
| clarkb | timrc: jlk I think we would be wanting 80GB root disk. or smaller root and an ephemeral disk | 21:04 |
| *** bharathm has quit IRC | 21:04 | |
| *** yamamoto has quit IRC | 21:05 | |
| *** yamamoto has joined #openstack-infra | 21:05 | |
| *** yamamoto has quit IRC | 21:06 | |
| clarkb | it appears to be successfully deleting our backlog of delete state nodes | 21:06 |
| clarkb | after which we should trnasition to building and running jobs again | 21:06 |
| *** topol2 is now known as topol_ | 21:06 | |
| *** yamamoto has joined #openstack-infra | 21:07 | |
| *** topol_ is now known as topol | 21:07 | |
| *** bharathm has joined #openstack-infra | 21:08 | |
| *** sjmc7 has left #openstack-infra | 21:08 | |
| *** yamamoto has quit IRC | 21:08 | |
| *** yamamoto_ has joined #openstack-infra | 21:08 | |
| *** rguillebert has joined #openstack-infra | 21:08 | |
| *** krtaylor has quit IRC | 21:09 | |
| *** yamamoto_ has quit IRC | 21:10 | |
| *** yamamoto has joined #openstack-infra | 21:10 | |
| *** jsavak has quit IRC | 21:10 | |
| *** jsavak has joined #openstack-infra | 21:10 | |
| *** aysyd has quit IRC | 21:11 | |
| *** yamamoto has quit IRC | 21:12 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 21:12 | |
| *** yamamoto has joined #openstack-infra | 21:12 | |
| *** jsavak has quit IRC | 21:12 | |
| *** Swami has joined #openstack-infra | 21:12 | |
| *** yamamoto has quit IRC | 21:13 | |
| *** sigmavirus24_awa is now known as sigmavirus24 | 21:13 | |
| *** armax has quit IRC | 21:13 | |
| *** yamamoto has joined #openstack-infra | 21:14 | |
| *** yamamoto has quit IRC | 21:14 | |
| *** ayoung has quit IRC | 21:14 | |
| *** yamamoto has joined #openstack-infra | 21:15 | |
| *** yamamoto has quit IRC | 21:16 | |
| mgagne | our no launch node error record is now over =( | 21:16 |
| clarkb | mgagne: well we just launched all the nodes :) | 21:16 |
| mgagne | clarkb yea, was doing well so far | 21:16 |
| *** yamamoto has joined #openstack-infra | 21:17 | |
| clarkb | we stopped being able to delete as nodes were being used so we piled up a delete/build backlog | 21:17 |
| *** ari has quit IRC | 21:17 | |
| clarkb | which I got fixed and set lose | 21:17 |
| clarkb | so we probably deleted all 24 in a short time frame then booted all 24 | 21:17 |
| krotscheck | Ok, were I to use "something" to autogenerate that index, is there a preference on how that's built? I can do cgi-perl, php, but I'm not certain if I can have apache execute a bash script and/or a simple python script. | 21:18 |
| *** rkukura has quit IRC | 21:18 | |
| clarkb | krotscheck: you can use wsgi for python | 21:18 |
| clarkb | and probably cgi too but wsgi is the more accepted method | 21:18 |
| *** yamamoto has quit IRC | 21:18 | |
| krotscheck | clarkb: That...seems like overkill? | 21:18 |
| *** jtomasek has quit IRC | 21:18 | |
| *** rhallisey has quit IRC | 21:18 | |
| *** yamamoto_ has joined #openstack-infra | 21:18 | |
| clarkb | yes probably | 21:18 |
| *** itsuugo has quit IRC | 21:18 | |
| mgagne | that was at ~9pm, errors were at ~7h15pm | 21:18 |
| *** rkukura has joined #openstack-infra | 21:19 | |
| *** yamamoto_ has quit IRC | 21:19 | |
| clarkb | krotscheck: and hte reaosn we don't use the pep prescribed layout is because afs does poorly stating many entries in dirents right? | 21:20 |
| clarkb | just making sure I understand the problem we are trying to solve | 21:20 |
| *** yamamoto has joined #openstack-infra | 21:20 | |
| *** jsavak has joined #openstack-infra | 21:20 | |
| *** itsuugo has joined #openstack-infra | 21:20 | |
| clarkb | krotscheck: we might be able to get away with writing an index to local disk that symlinks onto afs | 21:20 |
| *** yamamoto has quit IRC | 21:21 | |
| krotscheck | clarkb: Yes. According to jeblair there's a "limit" (fuzzy, complicated answer) of 64K records | 21:21 |
| clarkb | hrm thats the same limit as ext4 | 21:21 |
| *** yamamoto has joined #openstack-infra | 21:21 | |
| *** rlandy has joined #openstack-infra | 21:22 | |
| krotscheck | clarkb: Static index would be awesome. Bash via cron? | 21:22 |
| clarkb | krotscheck: ya, except if 64k is the issue I don't know that it solves anythin since ext4 hsa the same limit (thought we don't have problems iwth current ext4 filesytems) | 21:23 |
| *** yamamoto has quit IRC | 21:23 | |
| fungi | bash (or python) called at the end of any modification to the tree preferably? don't want that index out of sync with the files it's indexing for longer than we can help | 21:23 |
| clarkb | you would just do it at a directory level | 21:24 |
| clarkb | so as long as pypi doesn't completely delete projects we should be fine | 21:24 |
| clarkb | but having said that you know it will appen now :) | 21:24 |
| fungi | oh, i see, so we're just worried about indexing directory names, not file names | 21:24 |
| krotscheck | fungi: Yep. | 21:25 |
| *** yamamoto has joined #openstack-infra | 21:25 | |
| fungi | and leave the file listings in directories to the usual suspects | 21:25 |
| clarkb | ya since pip always does a listing first iirc | 21:25 |
| krotscheck | So there'd be a bit of a lag if a new library is added to wheel. | 21:25 |
| krotscheck | But otherwise it'd be fine. | 21:25 |
| * krotscheck wonders if we can do that in puppet. | 21:26 | |
| clarkb | pip install foo==1.1.1 will say list foo at path, find 1.1.1 in that list, download that | 21:26 |
| clarkb | krotscheck: I would detach it form puppet so that puppet doesn't need to be running to update it | 21:26 |
| *** yamamoto has quit IRC | 21:26 | |
| krotscheck | Right, and if foo is actually in Foo, then pip takes the 404 from /foo/, checks /, sees /Foo/, and reads that instead. | 21:26 |
| krotscheck | So, bash & cron? | 21:27 |
| clarkb | ya bash + cron or have pypi-mirror write it out orsomething | 21:28 |
| krotscheck | Optimally it could be part of the job, but there's no guarantee that wheel-slave's wheelhouse represents the same files as on the mirror. | 21:28 |
| AJaeger | clarkb, pleia2: If you find time this week to review my untested translation patch https://review.openstack.org/266565 and tell me whether that goes in the right direction, I would appreciate it... | 21:28 |
| jasondotstar | 21:28 | |
| *** yamamoto has joined #openstack-infra | 21:29 | |
| *** kzaitsev_mb has quit IRC | 21:29 | |
| mgagne | clarkb can't find an explanation for the failed launch attempt :-/ no logs on our side | 21:29 |
| jasondotstar | does the -infra team have an environment for api testing that accessible publicly? | 21:29 |
| jasondotstar | s/that/that's | 21:29 |
| clarkb | jasondotstar: what do you mean by api testing? | 21:29 |
| jasondotstar | writing my own client code (i.e. for launching nova instances, etc) | 21:30 |
| *** yamamoto has quit IRC | 21:30 | |
| *** dslevin has joined #openstack-infra | 21:30 | |
| jasondotstar | specifically this is for the (very early stage) openstack-vahana project | 21:30 |
| clarkb | jasondotstar: there is no permanent install that we manage for that no, instead we spin up clouds on demand using devstack and tear them down when done | 21:30 |
| *** esker has joined #openstack-infra | 21:30 | |
| clarkb | shade does its testing this way as do some of the other client libs | 21:31 |
| clarkb | there is also trystack | 21:31 |
| *** esker has quit IRC | 21:31 | |
| jasondotstar | ah | 21:31 |
| jasondotstar | trystack | 21:31 |
| jasondotstar | perhaps that is enough for now | 21:31 |
| *** esker has joined #openstack-infra | 21:31 | |
| *** amitgandhinz has quit IRC | 21:31 | |
| *** hdd has quit IRC | 21:32 | |
| *** yamamoto has joined #openstack-infra | 21:32 | |
| mordred | jasondotstar: I got myself accounts on 18 different public openstack clouds | 21:32 |
| *** ZZelle_ has joined #openstack-infra | 21:32 | |
| jasondotstar | mordred: wow.... all free? | 21:33 |
| *** yamamoto has quit IRC | 21:33 | |
| mordred | jasondotstar: nope | 21:33 |
| jasondotstar | ah | 21:33 |
| jasondotstar | ok | 21:33 |
| *** rkukura has quit IRC | 21:33 | |
| clarkb | it actually works pretty well to hit devstack | 21:33 |
| *** yamamoto has joined #openstack-infra | 21:34 | |
| clarkb | you don't get to tickle cloud specific odditites but for asnwering the question of whether or not you mostly function against the api its good | 21:34 |
| *** yamamoto has quit IRC | 21:34 | |
| *** amitgandhinz has joined #openstack-infra | 21:34 | |
| jasondotstar | yep, that's basically what I'm looking for | 21:34 |
| jasondotstar | something that's a step above running locally | 21:35 |
| *** jsavak has quit IRC | 21:35 | |
| *** yamamoto has joined #openstack-infra | 21:35 | |
| jasondotstar | right now, we need just something reliable to test against | 21:35 |
| *** hdd has joined #openstack-infra | 21:35 | |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: Initial add of gentoo support for diskimage-builder https://review.openstack.org/263349 | 21:35 |
| *** abregman is now known as abregman|afk | 21:36 | |
| *** yamahata has quit IRC | 21:36 | |
| *** yamamoto has quit IRC | 21:36 | |
| clarkb | I don't see vahana in gerrit so we wouldn't be able to run that for you just yet | 21:36 |
| clarkb | but ya, make job that spins up devstack, executes test suite against live cloud, win | 21:36 |
| AJaeger | clarkb: there's a proposal to add vahana | 21:37 |
| clarkb | ah ok | 21:37 |
| jasondotstar | we just added one | 21:37 |
| jasondotstar | AJaeger: +1 | 21:37 |
| *** yamamoto has joined #openstack-infra | 21:37 | |
| AJaeger | clarkb: https://review.openstack.org/#/c/266549/ | 21:37 |
| jasondotstar | a prop that is | 21:37 |
| jasondotstar | https://review.openstack.org/#/c/266549 | 21:37 |
| clarkb | oh, iOS | 21:37 |
| *** ildikov has quit IRC | 21:37 | |
| jasondotstar | clarkb: +1 | 21:37 |
| clarkb | we may have trouble executing that code | 21:37 |
| AJaeger | jasondotstar: please update it | 21:37 |
| jasondotstar | AJaeger: with? | 21:38 |
| *** bharathm has quit IRC | 21:38 | |
| AJaeger | jasondotstar: see my comments... | 21:38 |
| clarkb | as iOS dev env requires OS X right? | 21:38 |
| clarkb | and isn't freely available etc | 21:38 |
| *** yamamoto has quit IRC | 21:38 | |
| *** yamamoto has joined #openstack-infra | 21:38 | |
| jasondotstar | AJaeger: ack | 21:39 |
| jasondotstar | native iOS (swift, and objc) does require osx | 21:39 |
| *** ociuhandu has quit IRC | 21:39 | |
| *** yamamoto has quit IRC | 21:40 | |
| *** yamamoto has joined #openstack-infra | 21:40 | |
| *** ildikov has joined #openstack-infra | 21:40 | |
| *** bharathm has joined #openstack-infra | 21:41 | |
| jeblair | clarkb: ext4 doesn't have an (inbuilt) limit on directory entries | 21:41 |
| clarkb | jeblair: I thought it did, we ran into it with ext3 | 21:41 |
| jeblair | clarkb: ext3 does, but not ext4 | 21:41 |
| clarkb | and ext4 doubled it so we remounted ext4 and continued on | 21:41 |
| * AJaeger waves good night | 21:41 | |
| clarkb | ah ok | 21:42 |
| clarkb | the difference was mounting ext3 as ext4 instead of converting | 21:42 |
| *** abregman|afk has quit IRC | 21:42 | |
| jeblair | i maintain it's never a good idea to have lots of things in a directory anyway, regardless of what your fs actually supports | 21:42 |
| jeblair | but that's just my opinion as a sysadmin; i really like "ls" to return. :) | 21:43 |
| jlk | clarkb: timrc: Did you get sorted out on flavor creation ? | 21:43 |
| clarkb | jlk: not that I have heard | 21:43 |
| *** salv-orlando has quit IRC | 21:43 | |
| *** yamamoto_ has joined #openstack-infra | 21:43 | |
| jlk | clarkb: so, you should be able to create new flavors at will | 21:44 |
| *** salv-orlando has joined #openstack-infra | 21:44 | |
| clarkb | oh you mean we can create them ourselves? | 21:44 |
| jlk | and sort them out however you want | 21:44 |
| jlk | through the nova API | 21:44 |
| jlk | yeah | 21:44 |
| *** gyee has joined #openstack-infra | 21:44 | |
| *** yamamoto_ has quit IRC | 21:44 | |
| clarkb | huh, also I am not familar enough with that cloud to know if we can safely bump the root disk size without changing our quota | 21:44 |
| clarkb | mordred: ^ | 21:44 |
| jeblair | clarkb: yes, i created the flavor originally | 21:44 |
| jlk | http://developer.openstack.org/api-ref-compute-v2.1.html#createFlavor | 21:45 |
| *** yamamoto_ has joined #openstack-infra | 21:45 | |
| *** yamamoto has quit IRC | 21:45 | |
| mordred | clarkb: yes. you can use admin creds to use nova api - or you can use the horizon | 21:45 |
| jeblair | give me a sec before you do anything; i believe i chose that value intentionally | 21:46 |
| clarkb | jeblair: ya that was my concern | 21:46 |
| *** yamamoto_ has quit IRC | 21:46 | |
| mordred | clarkb: also, now that ansible 2.0 is out, I was thinking we should make some ansible playbooks that manage some of our cloud things like flavor size | 21:46 |
| clarkb | that we did maths and thats what the number was | 21:46 |
| mordred | jeblair: ^^ (to playbooks for managing cloud things like flavors) | 21:46 |
| *** erlon has quit IRC | 21:46 | |
| openstackgerrit | Steve Martinelli proposed openstack-infra/project-config: set keystone's legacy_drivers job to voting https://review.openstack.org/266582 | 21:47 |
| *** yamamoto has joined #openstack-infra | 21:47 | |
| *** baoli has joined #openstack-infra | 21:47 | |
| *** yamamoto has quit IRC | 21:47 | |
| *** yamamoto has joined #openstack-infra | 21:48 | |
| *** julim has quit IRC | 21:49 | |
| jeblair | clarkb: the math is as follows: each hypervisor has 915G of space and 64 vcpus; the mirror is 300G. (915-300)/8=76. the current flavor is 60, which i think i chose because i wasn't sure how much we would have from ovh and didn't want to overshoot. | 21:49 |
| *** yamamoto has quit IRC | 21:50 | |
| *** yamamoto has joined #openstack-infra | 21:50 | |
| jeblair | note, that we've since clarified we need 500G for the mirror, which leaves less than that. | 21:50 |
| jeblair | we can increase the volume to 80G but we will lose a few instances. | 21:51 |
| clarkb | gotcha | 21:51 |
| *** yamamoto has quit IRC | 21:51 | |
| jeblair | i think we have since standardized on saying we'll have 80G, so i guess we should do that | 21:52 |
| *** yamamoto has joined #openstack-infra | 21:52 | |
| openstackgerrit | Ivan Kolodyazhny proposed openstack-infra/project-config: Add functional test job for python-brick-cinderclient-ext https://review.openstack.org/265925 | 21:52 |
| clarkb | ya and as far as I know all of our current jobs are happy with that number | 21:52 |
| jeblair | clarkb: i have the screen up, shall i go ahead and punch in the number? | 21:52 |
| clarkb | jeblair: sure | 21:52 |
| openstackgerrit | Ramy Asselin proposed openstack-infra/puppet-openstackci: Make initial puppet steps more explcit https://review.openstack.org/266588 | 21:52 |
| clarkb | worst case nova scheudler will 500 if it can't place a node | 21:52 |
| jeblair | clarkb: okay, it's now 80G root 0 ephemeral | 21:53 |
| jeblair | clarkb: should we change to using ephemeral? | 21:53 |
| *** Keedya_ has quit IRC | 21:53 | |
| clarkb | jeblair: I am not sure if it makes a difference if using local storage | 21:53 |
| *** yamamoto has quit IRC | 21:53 | |
| clarkb | good question for someone that knows nova probably | 21:53 |
| jeblair | clarkb: i guess it will trigger the swapfile code path in devstack-gate? | 21:53 |
| *** trown is now known as trown|outttypeww | 21:53 | |
| *** yamamoto has joined #openstack-infra | 21:53 | |
| clarkb | ephemeral would yes | 21:53 |
| clarkb | er no ephemeral would | 21:53 |
| *** mudassirlatif has joined #openstack-infra | 21:54 | |
| clarkb | using ephemeral would trigger the repartition with swap fs | 21:54 |
| jeblair | i'm not really sure i have an opinion :) | 21:54 |
| *** rockyg has quit IRC | 21:54 | |
| jlk | what's the question? | 21:54 |
| jeblair | jlk: relative performance merits of just having a large root partition vs ephemeral drive | 21:55 |
| jeblair | jlk: i think clarkb suspects with the local storage in blue box, there's probably not a difference | 21:55 |
| jlk | hrm. | 21:55 |
| jlk | honestly I'm not sure what the perf differences are between local instance storage vs a cinder volume | 21:56 |
| *** yamamoto has quit IRC | 21:56 | |
| jlk | one involve more network than the other. | 21:56 |
| clarkb | well cinder is served via iscsi, so depends on networking | 21:56 |
| fungi | i think if we're trying to squeeze the most out of our available backend disk capacity, then we want an 80gb rootfs, no ephemeral (so that we end up only using sparse swapfile in devstack-gate as needed) | 21:56 |
| jeblair | jlk: the ephemeral disk that we get by configuring that in a flavor is still the hypervisor-local disk in blue box, right? | 21:57 |
| clarkb | and what the storage serving thta is based on | 21:57 |
| *** MarkAtwood has joined #openstack-infra | 21:58 | |
| *** baoli has quit IRC | 21:58 | |
| cody-somerville | with mirror you might need a lot of concurrent access which storage cluster via cinder might be able to provide better than smaller number of local disks, assuming the storage cluster has hundreds of disks vs. handful of local disks per compute node. | 21:59 |
| *** baoli has joined #openstack-infra | 21:59 | |
| *** jsavak has joined #openstack-infra | 22:00 | |
| fungi | i think the question was just about how to provision the filesystem(s) for worker instances | 22:00 |
| *** ayoung has joined #openstack-infra | 22:00 | |
| openstackgerrit | Merged openstack-infra/subunit2sql: Add release notes for pending 1.2.0 release https://review.openstack.org/266507 | 22:00 |
| *** eil397 has quit IRC | 22:00 | |
| fungi | for mirrors we already provision a separate cinder volume when possible | 22:01 |
| cody-somerville | ah | 22:01 |
| *** [1]Thelo has joined #openstack-infra | 22:02 | |
| cody-somerville | jeblair: I was curious, did you ever figure out the zuul memory leak? | 22:02 |
| fungi | SamYaple: sdake: thanks for the clarification on the missing python-kollaclient repo. in the future, you probably also want to make the governance addition depends-on the project-config addition so that the tc doesn't approve another nonexistent repo into the list | 22:03 |
| openstackgerrit | Merged openstack-infra/shade: Allow inventory filtering by cloud name https://review.openstack.org/263513 | 22:03 |
| SamYaple | fungi: agreed | 22:03 |
| jeblair | cody-somerville: no | 22:03 |
| sdake | fungi ack | 22:03 |
| cody-somerville | jeblair: Did anyone ever send in any clues? | 22:04 |
| jeblair | cody-somerville: no | 22:04 |
| *** Thelo has quit IRC | 22:04 | |
| *** [1]Thelo is now known as Thelo | 22:04 | |
| *** krtaylor has joined #openstack-infra | 22:05 | |
| cody-somerville | jeblair: Is your gut feeling that it is a change in zuul itself or a change in a dependency or maybe even configuration? | 22:05 |
| openstackgerrit | Merged openstack-infra/subunit2sql: Disable session synchronization on deletes https://review.openstack.org/266562 | 22:05 |
| clarkb | jeblair: I was trying to remember, did the connections stuff merge around then? | 22:05 |
| *** dkranz has quit IRC | 22:06 | |
| *** EricGonczer_ has joined #openstack-infra | 22:07 | |
| *** bharathm has quit IRC | 22:08 | |
| *** davidlenwell has quit IRC | 22:09 | |
| *** salv-orl_ has joined #openstack-infra | 22:09 | |
| jeblair | clarkb: yes, it's high on my list of suspects. however, the support for undefined projects is a candidate too. | 22:09 |
| jeblair | cody-somerville: ^ zuul | 22:09 |
| *** thorst has quit IRC | 22:10 | |
| jeblair | note, i'm not actually investigating it. i hope someone else will. | 22:10 |
| *** baoli has quit IRC | 22:10 | |
| *** thorst has joined #openstack-infra | 22:10 | |
| *** EricGonc_ has quit IRC | 22:10 | |
| jeblair | i believe jhesketh was going to start looking at it when he got back | 22:10 |
| jeblair | i was just hoping to collect more data for him | 22:11 |
| *** baoli has joined #openstack-infra | 22:11 | |
| *** davidlenwell has joined #openstack-infra | 22:11 | |
| *** hongbin has left #openstack-infra | 22:12 | |
| *** vgridnev has quit IRC | 22:12 | |
| *** eharney has quit IRC | 22:12 | |
| *** salv-orlando has quit IRC | 22:12 | |
| *** jcoufal has quit IRC | 22:13 | |
| *** bharathm has joined #openstack-infra | 22:14 | |
| *** matrohon has quit IRC | 22:14 | |
| *** yamahata has joined #openstack-infra | 22:14 | |
| *** dims has quit IRC | 22:15 | |
| *** thorst has quit IRC | 22:15 | |
| *** eil397 has joined #openstack-infra | 22:16 | |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: Initial add of gentoo support for diskimage-builder https://review.openstack.org/263349 | 22:16 |
| *** annegentle has quit IRC | 22:16 | |
| *** jamielennox|away is now known as jamielennox | 22:17 | |
| *** ggillies has quit IRC | 22:17 | |
| clarkb | ok bluebox nodes are starting to be deleted should see them using new flavor soon | 22:19 |
| *** EricGonc_ has joined #openstack-infra | 22:19 | |
| *** EricGonczer_ has quit IRC | 22:20 | |
| *** ggillies has joined #openstack-infra | 22:20 | |
| *** peristeri has quit IRC | 22:20 | |
| *** baoli has quit IRC | 22:22 | |
| *** baoli has joined #openstack-infra | 22:22 | |
| *** ayoung has quit IRC | 22:23 | |
| *** EricGonc_ has quit IRC | 22:24 | |
| *** zz_ja is now known as zz_zz_ja | 22:24 | |
| *** _amrith_ is now known as amrith | 22:25 | |
| *** bharathm has quit IRC | 22:25 | |
| *** amitgandhinz has quit IRC | 22:25 | |
| SamYaple | clarkb: jeblair re: disk flavor. My vote would be for having the ephemeral drive. We repartition to use it for swap and for Docker things. Using a loopback file for docker is drastically slower in previous tests (20m to 90m+) | 22:26 |
| openstackgerrit | Fei Long Wang proposed openstack-infra/project-config: Make zaqarclient-dsvm-functional job as voting https://review.openstack.org/266596 | 22:26 |
| clarkb | SamYaple: there is already swap | 22:26 |
| *** amitgandhinz has joined #openstack-infra | 22:26 | |
| SamYaple | unless its change bluebox did not have a swap disk | 22:26 |
| *** bharathm has joined #openstack-infra | 22:26 | |
| clarkb | SamYaple: and we don't have this control on every cloud | 22:26 |
| clarkb | SamYaple: we make a swap file | 22:27 |
| dmsimard | o/ any chance for an easy +A on https://review.openstack.org/#/c/266482/ ? :) | 22:27 |
| *** Hal has joined #openstack-infra | 22:27 | |
| clarkb | SamYaple: basically you don't want to depend on an epehemeral disk, we may still want to do it in bluebox but its not as easy in other clouds | 22:27 |
| SamYaple | right we do not at the Kolla side, but swap wasnt the real issue it was docker | 22:27 |
| SamYaple | yea im working on getting a decent performing gate without ephermal | 22:27 |
| *** Hal is now known as Guest17316 | 22:27 | |
| clarkb | SamYaple: well anyone can then, also I highly recommend using the setup stuff we have in place for you | 22:27 |
| clarkb | but whatever | 22:27 |
| clarkb | it will make sure you have the correct git repos, a swapfile, etc | 22:28 |
| *** jpeeler has quit IRC | 22:28 | |
| SamYaple | yea we have different requirements than most gates is all so we don't run those scripts | 22:28 |
| SamYaple | but i think weve talked about this before | 22:28 |
| clarkb | SamYaple: I don't recall but it should be fine to run them... | 22:28 |
| clarkb | especially if yo umiss things like swap :) thats what they exist for | 22:29 |
| SamYaple | swap is not an issue, its the docker needing a different backend | 22:29 |
| *** ddieterly has quit IRC | 22:29 | |
| SamYaple | if it had swap (like RAX) we would swapoff and repartition it for docker then do a swapfile in the past | 22:29 |
| *** ddieterly has joined #openstack-infra | 22:30 | |
| clarkb | you can also repartition the root device | 22:30 |
| asselin_ | hi, can someone create feature/unit-tests branch based off of master for this project? https://review.openstack.org/#/admin/projects/openstack-infra/ciwatch,branches | 22:30 |
| clarkb | but that is trickier | 22:30 |
| *** piet has joined #openstack-infra | 22:30 | |
| *** erikmwilson_ has joined #openstack-infra | 22:31 | |
| clarkb | like maybe we don't extend to the end of the disk on boot | 22:31 |
| *** erikmwilson_ has quit IRC | 22:31 | |
| *** davideag_ has quit IRC | 22:31 | |
| *** _tzn has quit IRC | 22:31 | |
| SamYaple | i suggested taht to mordred before, but as a configurable | 22:31 |
| SamYaple | he pointed out these are built ahead of time, so this change would affect everyone | 22:31 |
| SamYaple | i actually like that idea personally | 22:31 |
| clarkb | yes it would affect everyone, but we could by default leave an 8GB tail that we just extend over on most jobs | 22:31 |
| clarkb | (not sure if it is a good alternative but would be doable) | 22:32 |
| *** davideagnello has joined #openstack-infra | 22:32 | |
| *** gordc has quit IRC | 22:32 | |
| SamYaple | ill get back to you on that. ideally a filebackend would be the _best_ solution here. let me attempt again to get it performance. havent tried in about 6 months | 22:32 |
| SamYaple | as i told him this issue goes away with 16.04 gates because the newer kernel means i can use OverlayFS | 22:33 |
| SamYaple | then all my problems are solved, so just trying to stop gap here | 22:33 |
| *** Guest17316 has quit IRC | 22:33 | |
| *** baoli has quit IRC | 22:34 | |
| *** erikwilson has quit IRC | 22:34 | |
| *** baoli has joined #openstack-infra | 22:34 | |
| *** annegentle has joined #openstack-infra | 22:34 | |
| *** rkukura has joined #openstack-infra | 22:35 | |
| *** otter768 has joined #openstack-infra | 22:35 | |
| *** davideagnello has quit IRC | 22:35 | |
| asselin_ | patrickeast, does this really work for you? I'm getting exceptions: https://review.openstack.org/#/c/266041/2 | 22:35 |
| *** amitgandhinz has quit IRC | 22:35 | |
| *** davideagnello has joined #openstack-infra | 22:36 | |
| mtreinish | fungi: ok, the new version of subunit2sql is up: https://pypi.python.org/pypi/subunit2sql | 22:36 |
| *** ociuhandu has joined #openstack-infra | 22:36 | |
| openstackgerrit | Chris Jones proposed openstack-infra/project-config: Add project configuration for Vahana. https://review.openstack.org/266549 | 22:36 |
| mtreinish | once puppet on logstash.o.o picks it up we can give the expire command another shot and it should hopefully work this time :) | 22:36 |
| *** otter768 has quit IRC | 22:39 | |
| *** dingyichen has joined #openstack-infra | 22:41 | |
| *** armax has joined #openstack-infra | 22:42 | |
| rm_work | is there a devstack specific channel? | 22:43 |
| sc68cal | rm_work: #openstack-qa | 22:43 |
| rm_work | ok, devstack is included in qa, got it :P thanks | 22:43 |
| rm_work | (I always ask here because it seems #infra is pretty much all-knowing) | 22:43 |
| *** _nadya_ has joined #openstack-infra | 22:45 | |
| *** Swami has quit IRC | 22:46 | |
| *** baoli has quit IRC | 22:46 | |
| *** baoli has joined #openstack-infra | 22:47 | |
| *** bharathm has quit IRC | 22:47 | |
| *** bharathm has joined #openstack-infra | 22:48 | |
| *** Hal has joined #openstack-infra | 22:48 | |
| *** Hal is now known as Guest91463 | 22:49 | |
| *** _nadya_ has quit IRC | 22:50 | |
| *** thorst has joined #openstack-infra | 22:51 | |
| *** piet has quit IRC | 22:51 | |
| *** rossella_s has quit IRC | 22:52 | |
| *** rossella_s has joined #openstack-infra | 22:52 | |
| *** thorst_ has joined #openstack-infra | 22:53 | |
| *** eliqiao has quit IRC | 22:54 | |
| *** davideagnello has quit IRC | 22:54 | |
| jhesketh | jeblair: sorry, missed your highlight earlier. Yes, I'm poking at the memory leak.. Haven't been able to obvoiusly reproduce it yet so I haven't gotten very far | 22:54 |
| *** mriedem has quit IRC | 22:54 | |
| *** bharathm has quit IRC | 22:54 | |
| jhesketh | jeblair: the "Project None not found" errors should be fixed by https://review.openstack.org/#/c/135332/3 | 22:54 |
| jhesketh | those extra events have been there from before the memory leak so I don't think it's that (although it may help) | 22:55 |
| *** eliqiao has joined #openstack-infra | 22:55 | |
| jhesketh | it also doesn't look to be the cache from grepping over the debug logs | 22:55 |
| jhesketh | mind you it appears the cache is only cleared on a reconfigure. We do those often enough but we should also find another time that is good for clearing it | 22:56 |
| *** thorst has quit IRC | 22:56 | |
| *** davideagnello has joined #openstack-infra | 22:56 | |
| *** Paratosh has quit IRC | 22:57 | |
| *** bharathm has joined #openstack-infra | 22:57 | |
| jhesketh | I don't think we reconfigure enough for any of the old triggers/sources/reporters etc to be hanging around taking up memory, so I'm leaning away from that | 22:57 |
| jhesketh | given the steady growth in memory consumption I'd lean towards some kind of change or event management, but that's as far as I've gotten | 22:58 |
| jhesketh | and it's not very certain | 22:58 |
| *** baoli has quit IRC | 22:58 | |
| *** baoli has joined #openstack-infra | 22:58 | |
| mtreinish | fungi: looks like logstash.o.o has the latest version now: http://puppetboard.openstack.org/report/logstash.openstack.org/f360ea4f3dab380333a4a3d253cc1af3dc520861 | 23:00 |
| mtreinish | so whenever you get a sec we should be good to try it again | 23:00 |
| *** ccrouch has joined #openstack-infra | 23:01 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 23:01 | |
| jeblair | jhesketh: oh, the project not found thing i was referring to was the support for undefined projects that was added for third-party ci systems which don't have every project defined in layout.yaml | 23:02 |
| *** burgerk has quit IRC | 23:03 | |
| jhesketh | jeblair: ah right. You had mentioned the "None project" last week when the leak was found | 23:03 |
| jhesketh | but yes, that could be related | 23:03 |
| *** Kiall has quit IRC | 23:03 | |
| *** tonytan4ever has quit IRC | 23:04 | |
| *** Kiall has joined #openstack-infra | 23:04 | |
| *** armax has quit IRC | 23:05 | |
| *** armax has joined #openstack-infra | 23:06 | |
| asselin_ | patrickeast, nevermind...user error | 23:06 |
| *** amrith is now known as _amrith_ | 23:07 | |
| *** markmcclain has quit IRC | 23:07 | |
| *** pradk has quit IRC | 23:08 | |
| *** markmcclain has joined #openstack-infra | 23:09 | |
| *** armax has quit IRC | 23:10 | |
| clarkb | greghaynes: did you get anywhere on those nodepool test fails? | 23:10 |
| *** baoli has quit IRC | 23:10 | |
| *** baoli has joined #openstack-infra | 23:10 | |
| *** armax has joined #openstack-infra | 23:11 | |
| clarkb | hrm gerrit is 503ing | 23:11 |
| *** armax has quit IRC | 23:11 | |
| *** regXboi has quit IRC | 23:12 | |
| greghaynes | clarkb: started looking at it - it is wierd | 23:12 |
| clarkb | memory use has climbed quickly but GC doesn't seem to be a problem | 23:12 |
| clarkb | greghaynes: it starts with https://review.openstack.org/#/c/231592/18 so looking there | 23:13 |
| *** rkukura has quit IRC | 23:13 | |
| greghaynes | clarkb: yep - basically the provider is not getting started after a reconfigure | 23:14 |
| *** davideagnello has quit IRC | 23:14 | |
| greghaynes | clarkb: in the log youll see a fake-provider is no longer running | 23:14 |
| clarkb | ya | 23:15 |
| clarkb | my browser really dislikes that log file | 23:15 |
| *** ybathia has quit IRC | 23:15 | |
| *** ybathia has joined #openstack-infra | 23:16 | |
| *** davideagnello has joined #openstack-infra | 23:16 | |
| *** rkukura has joined #openstack-infra | 23:16 | |
| *** davidlenwell has quit IRC | 23:17 | |
| *** tzn has joined #openstack-infra | 23:17 | |
| *** davidlenwell has joined #openstack-infra | 23:18 | |
| *** armax has joined #openstack-infra | 23:18 | |
| clarkb | greghaynes: one odd thing is I thought we had jbelair updates tests for the new network stuff | 23:19 |
| clarkb | greghaynes: but if you diff ps17 and ps18 there are no test differences | 23:19 |
| asselin_ | bardia, btw I reproduced your issue: ImportError: No module named schedulers.background | 23:19 |
| asselin_ | on my test system | 23:19 |
| clarkb | greghaynes: I am going to diff your patch against master | 23:19 |
| clarkb | hrm I don't see the delta I expected so I must be imaging int | 23:21 |
| *** Sukhdev has joined #openstack-infra | 23:22 | |
| *** Guest91463 has quit IRC | 23:22 | |
| *** baoli has quit IRC | 23:22 | |
| *** baoli has joined #openstack-infra | 23:23 | |
| *** MarkAtwood has quit IRC | 23:24 | |
| clarkb | greghaynes: I think I may have figured it out | 23:24 |
| *** markmcclain has quit IRC | 23:25 | |
| clarkb | greghaynes: the definitions of ConfigValue are different and the merge resolution won't be able to move that to the new file | 23:25 |
| *** markmcclain has joined #openstack-infra | 23:26 | |
| *** e0ne has quit IRC | 23:26 | |
| jhesketh | jeblair: you know how some patches have failed to get into the gate... I saw a case yesterday where a change was behind an older patchset in a merged change. Zuul would do the check pipeline fine (presumably building on the old patchset as per gerrit), but it wouldn't enter the gate pipeline | 23:26 |
| jhesketh | jeblair: for some reason it wouldn't grab the newer patchset to test against in the gate pipeline | 23:26 |
| jhesketh | have you seen that before? | 23:26 |
| clarkb | greghaynes: anyways I am almost positive that is the issue, I can push a patch for it too if you want | 23:26 |
| *** xyang1 has quit IRC | 23:27 | |
| *** dimtruck is now known as zz_dimtruck | 23:28 | |
| greghaynes | clarkb: fun. yesplz | 23:29 |
| greghaynes | clarkb: busy with other stuff ATM | 23:29 |
| *** rkukura has quit IRC | 23:29 | |
| clarkb | I figured | 23:29 |
| *** annegentle has quit IRC | 23:31 | |
| *** eliqiao has quit IRC | 23:31 | |
| *** eliqiao has joined #openstack-infra | 23:32 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool: Builders can be run on their own https://review.openstack.org/232251 | 23:33 |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool: Decouple builders from nodepool instance https://review.openstack.org/232135 | 23:33 |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool: Builders distinguish between failure and exception https://review.openstack.org/242131 | 23:33 |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool: Make secure.conf optional https://review.openstack.org/232800 | 23:33 |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool: Move nodepool config loading into module https://review.openstack.org/231592 | 23:33 |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool: Decouple nodepool db and config from builders https://review.openstack.org/222398 | 23:33 |
| clarkb | lets see how those do | 23:33 |
| *** armax has quit IRC | 23:34 | |
| asselin_ | bardia, log output for reference: http://paste.openstack.org/show/483674/ | 23:34 |
| *** baoli has quit IRC | 23:34 | |
| *** baoli has joined #openstack-infra | 23:35 | |
| *** armax has joined #openstack-infra | 23:35 | |
| *** _amrith_ is now known as amrith | 23:36 | |
| *** weshay_xchat has quit IRC | 23:37 | |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: Initial add of gentoo support for diskimage-builder https://review.openstack.org/263349 | 23:38 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 23:38 | |
| *** sabeen3 has quit IRC | 23:40 | |
| *** armax has quit IRC | 23:40 | |
| *** armax has joined #openstack-infra | 23:41 | |
| *** ddieterly has quit IRC | 23:41 | |
| greghaynes | clarkb: AH | 23:41 |
| greghaynes | clarkb: I see | 23:41 |
| openstackgerrit | hardik proposed openstack-infra/project-config: Fixed mistral experimental job https://review.openstack.org/265030 | 23:41 |
| *** ddieterly has joined #openstack-infra | 23:42 | |
| bardia | asselin_: its looks like the puppet installed apscheduler correctly in the logs. How did you repro the issue? | 23:42 |
| clarkb | greghaynes: ya git won't realiez that it has a merge conflict on that | 23:42 |
| clarkb | he dangers of moving code | 23:42 |
| Thelo | asselin: regarding yesterday issue (LIBS FROM GIT does not pick up), I put the logs here (http://paste.openstack.org/show/483675/) | 23:42 |
| greghaynes | clarkb: yep. fortunately we have tests | 23:42 |
| asselin_ | bardia, I had an old test setup. I renewed it and zuul was starting but failing to create the pid. I ran it as non-daemon and found the same? stack trace as you | 23:43 |
| asselin_ | bardia, but in my case poking puppet a bit, it did install the needed dependency correctly | 23:43 |
| *** Hal has joined #openstack-infra | 23:43 | |
| bardia | asselin_: yeah thats pretty much what happened in my case | 23:44 |
| asselin_ | Thelo, do you have the full devstacklog? | 23:44 |
| *** Hal is now known as Guest18860 | 23:44 | |
| asselin_ | bardia, no, in your case, even poking puppet caused it to not update that dependency...at least that's what I understood | 23:44 |
| *** armax has quit IRC | 23:44 | |
| *** akshai has quit IRC | 23:44 | |
| bardia | asselin_: yeah I meant the issue minus the puppet poking :) | 23:44 |
| *** sdake has quit IRC | 23:45 | |
| bardia | asselin_: what was the poking you did to puppet | 23:45 |
| *** rkukura has joined #openstack-infra | 23:45 | |
| *** armax has joined #openstack-infra | 23:45 | |
| jlk | jeblair: ephemeral disk is local to the hypervisor. It's a RAID 10 array, 6 device, SAS 10K rpm | 23:45 |
| jlk | clarkb: ^^ | 23:45 |
| asselin_ | bardia, basically the same this I had you do yesterday, uninstall the dependency, change the version of /opt/zuul, and run puppet to have it reinstall it | 23:46 |
| bardia | asselin_: ah okay | 23:46 |
| *** baoli has quit IRC | 23:46 | |
| *** baoli has joined #openstack-infra | 23:47 | |
| *** armax has quit IRC | 23:47 | |
| clarkb | jlk: and that is separate from what the root disks are served from? or is it shared? | 23:47 |
| *** sigmavirus24 is now known as sigmavirus24_awa | 23:47 | |
| *** armax has joined #openstack-infra | 23:47 | |
| fungi | mtreinish: i'm back and have it running again now | 23:47 |
| mtreinish | fungi: ok cool, thanks | 23:48 |
| * mtreinish crosses his fingers | 23:48 | |
| jlk | clarkb: separate from the root disks | 23:48 |
| clarkb | hrm | 23:48 |
| asselin_ | clarkb, can you create a feature branch for us? https://review.openstack.org/#/admin/projects/openstack-infra/ciwatch,branches | 23:48 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 23:48 | |
| clarkb | jlk: what is the useable disk on that? | 23:48 |
| jlk | clarkb: the root OS is on a raid 1 of 2 disks | 23:48 |
| clarkb | jlk: er total useable disk | 23:48 |
| *** angdraug has quit IRC | 23:48 | |
| jlk | clarkb: the ephermal, it depends on if these are "enterprise" class nodes, or standard nodes. | 23:48 |
| clarkb | uh | 23:48 |
| asselin_ | clarkb, feature/unit-tests based off of master | 23:49 |
| jeblair | jlk, clarkb: i think we are enterprise? | 23:49 |
| *** oomichi_away is now known as oomichi | 23:49 | |
| jlk | it's either 1.2TB of storage per compute, or 2.4TB of storage per compute | 23:49 |
| clarkb | in either case thats plenty for an 80GB ephemeral disk right? | 23:49 |
| * clarkb remembers napkin math and believes that is correct | 23:49 | |
| jlk | oh yes | 23:49 |
| jeblair | jlk: should that show up in horizon? i only recall seeing a single pie chart for disk, which sounds like it might be a simplified view. | 23:49 |
| clarkb | well its 80GB x 8 | 23:50 |
| clarkb | I think | 23:50 |
| jlk | yeah, horizon shows the aggregate in the overview | 23:50 |
| jlk | there might be a way to view specific hypervisor data in the admin tab | 23:50 |
| jeblair | clarkb: yeah, we have vcpus for 8 instances / hypervisor | 23:50 |
| *** weshay_xchat has joined #openstack-infra | 23:50 | |
| clarkb | so maybe we want to do 30GB / and 80GB epehemeral | 23:51 |
| clarkb | could do bigger epehemeral too | 23:51 |
| *** armax has quit IRC | 23:51 | |
| clarkb | but this keeps it from being crazy | 23:51 |
| jeblair | jlk: so if horizon says we have 915G per, and both the root and ehpmeral are included in that, sounds like we're actually 'standard'? | 23:51 |
| jlk | Horizon is going ot report the disk space available to Nova | 23:51 |
| jlk | which would not include the root partition | 23:51 |
| jlk | if it says 915G per, then yeah I think that's "standard" | 23:52 |
| jlk | not enterprise | 23:52 |
| jlk | how many cores does it say? | 23:52 |
| jlk | VCPUs | 23:52 |
| jeblair | 64 vcpus | 23:52 |
| jlk | https://www.blueboxcloud.com/products/pricing is our price sheet for the Dedicated product. | 23:52 |
| jlk | oh hrm. | 23:52 |
| jlk | 64 CPUs sounds like .. weird. | 23:53 |
| jlk | let me actually look at your account. | 23:53 |
| *** rkukura has quit IRC | 23:54 | |
| *** annegentle has joined #openstack-infra | 23:54 | |
| jeblair | clarkb: btw, we need to restart nodepool to pick up the flavor changes | 23:55 |
| *** bpokorny_ has joined #openstack-infra | 23:55 | |
| clarkb | jeblair: oh, I didn't realize | 23:55 |
| jeblair | clarkb: (the id changed out from under us when i edited the existing flavor) | 23:55 |
| jeblair | clarkb: and nodepool caches that permanently | 23:55 |
| *** amuller_afk is now known as amuller | 23:55 | |
| jlk | yeah, your account says these are 64 cores, and 2TB of storage, that might be raw storage before raid? | 23:55 |
| clarkb | gotcha | 23:55 |
| *** Guest18860 has quit IRC | 23:55 | |
| jeblair | clarkb: but i'll wait until we finish this conversation in case we want to change the flavor again | 23:56 |
| clarkb | ya soundling like we may want to use ephemeral to free up root disk for the mirror | 23:56 |
| jeblair | jlk: i don't see any more detailed info in horizon; maybe nova commands would split them out? | 23:57 |
| jlk | I was going to log directly into the box and see what it says there | 23:57 |
| jeblair | hard to argue with that | 23:57 |
| *** piet has joined #openstack-infra | 23:57 | |
| Thelo | asselin: here https://drive.google.com/file/d/0B08i3ee8d70RVFI4dUhoUUpuQWc/view?usp=sharing | 23:58 |
| *** baoli has quit IRC | 23:58 | |
| Thelo | asselin: ( sorry my log server ip is not reachable from outside my company yet) | 23:59 |
| *** bpokorny has quit IRC | 23:59 | |
| *** bharathm has quit IRC | 23:59 | |
| *** baoli has joined #openstack-infra | 23:59 | |
| *** rkukura has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!