| *** thingee is now known as thingee_zzz | 00:00 | |
| *** xchu has joined #openstack-infra | 00:00 | |
| *** senk has joined #openstack-infra | 00:05 | |
| *** hogepodge has joined #openstack-infra | 00:06 | |
| *** senk has quit IRC | 00:09 | |
| *** seagulls is now known as beagles | 00:10 | |
| *** flashgordon is now known as jog0 | 00:11 | |
| *** fifieldt has quit IRC | 00:15 | |
| *** jerryz has quit IRC | 00:16 | |
| *** vipul-away is now known as vipul | 00:19 | |
| *** sdake has quit IRC | 00:26 | |
| *** sdake_ has quit IRC | 00:29 | |
| *** reed has quit IRC | 00:29 | |
| *** reed has joined #openstack-infra | 00:29 | |
| openstackgerrit | Chris Hoge proposed a change to openstack-infra/config: Adding puppet-vswitch project to Stackforge https://review.openstack.org/49020 | 00:30 |
|---|---|---|
| hogepodge | fungi: thanks for your feedback. | 00:31 |
| hogepodge | fungi: hope you have a good weekend. | 00:31 |
| hogepodge | et al | 00:31 |
| fungi | you're welcome, and thanks! | 00:32 |
| *** wenlock has quit IRC | 00:34 | |
| openstackgerrit | A change was merged to openstack-infra/config: Adding puppet-vswitch project to Stackforge https://review.openstack.org/49020 | 00:38 |
| *** reed has quit IRC | 00:44 | |
| *** senk has joined #openstack-infra | 00:47 | |
| *** briancline has joined #openstack-infra | 00:50 | |
| *** openstackgerrit has quit IRC | 00:54 | |
| *** openstackgerrit has joined #openstack-infra | 00:55 | |
| jeblair | fungi: i think the thing that would make it do that is the min-ready -- it thought it needed to spin up 20 nodes immediately to meet the minimum | 00:59 |
| jeblair | fungi: maybe | 01:00 |
| jeblair | fungi: yeah, i'll think about that. things like this would make good unit tests for nodepool too. | 01:01 |
| jeblair | fungi: thanks for restarting jenkins02 | 01:01 |
| fungi | my pleasure | 01:03 |
| *** melwitt has quit IRC | 01:05 | |
| *** nati_uen_ has quit IRC | 01:23 | |
| clarkb | ++ thank you fungi. I have been completely distracted | 01:29 |
| *** nati_ueno has joined #openstack-infra | 01:48 | |
| *** beagles has quit IRC | 01:49 | |
| *** sarob has quit IRC | 01:53 | |
| *** sarob has joined #openstack-infra | 01:54 | |
| *** yaguang has joined #openstack-infra | 01:56 | |
| *** luhrs1 has quit IRC | 01:57 | |
| *** che-arne has quit IRC | 01:57 | |
| *** lcheng has quit IRC | 01:58 | |
| *** sarob has quit IRC | 01:58 | |
| *** nati_ueno has quit IRC | 02:01 | |
| *** wenlock has joined #openstack-infra | 02:01 | |
| *** nati_ueno has joined #openstack-infra | 02:01 | |
| *** nati_ueno has quit IRC | 02:01 | |
| *** nati_ueno has joined #openstack-infra | 02:02 | |
| *** thingee_zzz has quit IRC | 02:09 | |
| *** senk has quit IRC | 02:09 | |
| *** nati_ueno has quit IRC | 02:24 | |
| *** changbl has joined #openstack-infra | 02:24 | |
| *** nati_ueno has joined #openstack-infra | 02:24 | |
| *** DennyZhang has joined #openstack-infra | 02:27 | |
| *** nati_ueno has quit IRC | 02:28 | |
| *** Ryan_Lane has quit IRC | 02:52 | |
| *** fbo is now known as fbo_away | 02:59 | |
| *** esker has joined #openstack-infra | 03:03 | |
| *** guohliu has joined #openstack-infra | 03:03 | |
| *** dims has quit IRC | 03:04 | |
| *** sarob has joined #openstack-infra | 03:04 | |
| *** wenlock-mobile has joined #openstack-infra | 03:09 | |
| *** sarob has quit IRC | 03:10 | |
| *** wenlock-mobile has quit IRC | 03:17 | |
| fungi | be distracted. it's a friday night, after all | 03:24 |
| *** mriedem has quit IRC | 03:33 | |
| *** DennyZhang has quit IRC | 03:41 | |
| *** DennyZhang has joined #openstack-infra | 03:41 | |
| *** markmcclain has quit IRC | 03:42 | |
| *** esker has quit IRC | 03:55 | |
| *** esker has joined #openstack-infra | 03:56 | |
| *** esker has quit IRC | 04:00 | |
| *** colinmcnamara has joined #openstack-infra | 04:03 | |
| *** esker has joined #openstack-infra | 04:06 | |
| *** wenlock has quit IRC | 04:08 | |
| *** Steely_Spam is now known as dansmith | 04:12 | |
| *** nati_ueno has joined #openstack-infra | 04:13 | |
| *** CaptTofu has joined #openstack-infra | 04:13 | |
| *** CaptTofu has quit IRC | 04:14 | |
| *** CaptTofu has joined #openstack-infra | 04:15 | |
| *** CaptTofu has quit IRC | 04:19 | |
| *** HenryG has quit IRC | 04:34 | |
| *** nati_ueno has quit IRC | 04:34 | |
| *** nati_ueno has joined #openstack-infra | 04:35 | |
| *** DennyZhang has quit IRC | 04:37 | |
| *** nati_ueno has quit IRC | 04:40 | |
| *** vipul is now known as vipul-away | 04:41 | |
| *** nati_ueno has joined #openstack-infra | 04:46 | |
| *** Ryan_Lane has joined #openstack-infra | 04:47 | |
| *** colinmcnamara has quit IRC | 04:51 | |
| *** dguitarbite has joined #openstack-infra | 04:53 | |
| *** esker has quit IRC | 05:06 | |
| *** davidhadas_ has joined #openstack-infra | 05:48 | |
| *** davidhadas has quit IRC | 05:49 | |
| openstackgerrit | Zhongyue Luo proposed a change to openstack-dev/hacking: Add assertNotEquals check to E602 https://review.openstack.org/51355 | 05:53 |
| openstackgerrit | Zhongyue Luo proposed a change to openstack-dev/hacking: Add assertNotEquals check to E602 https://review.openstack.org/51355 | 06:00 |
| *** nati_ueno has quit IRC | 06:23 | |
| *** nati_ueno has joined #openstack-infra | 06:24 | |
| *** SergeyLukjanov has joined #openstack-infra | 06:24 | |
| *** nati_ueno has quit IRC | 06:28 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 06:41 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 06:44 | |
| *** dguitarbite has quit IRC | 07:03 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 07:03 | |
| *** dguitarbite has joined #openstack-infra | 07:04 | |
| *** dguitarbite has quit IRC | 07:04 | |
| *** dguitarbite has joined #openstack-infra | 07:04 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 07:10 | |
| *** yolanda has joined #openstack-infra | 07:20 | |
| *** cody-somerville has quit IRC | 07:21 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 07:21 | |
| *** _SergeyLukjanov is now known as SergeyLukjanov | 07:38 | |
| *** garyk has quit IRC | 07:51 | |
| *** SergeyLukjanov has quit IRC | 08:12 | |
| *** dafter has joined #openstack-infra | 08:14 | |
| *** dafter has quit IRC | 08:14 | |
| *** dafter has joined #openstack-infra | 08:14 | |
| *** boris-42 has joined #openstack-infra | 08:25 | |
| *** davidhadas_ has quit IRC | 08:36 | |
| *** dguitarbite has quit IRC | 08:42 | |
| *** SergeyLukjanov has joined #openstack-infra | 08:45 | |
| *** SergeyLukjanov has quit IRC | 08:49 | |
| *** cody-somerville has joined #openstack-infra | 09:30 | |
| *** xchu has quit IRC | 10:01 | |
| sdague | hmmmm so the first couple patches of os-loganalyze are failing because of checkout problems | 10:10 |
| *** yaguang has quit IRC | 10:24 | |
| *** ruhe has joined #openstack-infra | 10:35 | |
| *** ruhe has quit IRC | 10:48 | |
| *** rickerc has quit IRC | 11:01 | |
| *** anteaya has joined #openstack-infra | 11:12 | |
| anteaya | clarkb: so far I have some urls for blog posts about elastic search: http://www.found.no/foundation/ http://www.found.no/foundation/elasticsearch-in-production/#how-much-memory-is-enough http://www.found.no/foundation/elasticsearch-from-the-bottom-up/ | 11:13 |
| anteaya | which mention some suggestions for optimization | 11:13 |
| anteaya | first I am assuming we index on the day, which helps with deletion since we just delete the index for the day - deletion is expensive | 11:14 |
| anteaya | next were questions about our memory capacity, we looked at the cacti for elasticsearch but we didnt' see stats on how much heap we have | 11:15 |
| anteaya | we need a lot of heap, so an initial suggestion was optimizing the JVM so that we have enough heap | 11:15 |
| anteaya | a maximum of 32G heap is suggested, since "Java doesn't have a pointer, so to actually get more space than 32G we need to go up to 48G" | 11:16 |
| anteaya | but then we run into garbage collection issues, so a maximum of 32G heap is suggested | 11:16 |
| anteaya | this is per node | 11:16 |
| anteaya | this is what I have so far from Alex who works at found | 11:17 |
| anteaya | I have a meeting with Honsa from elastic search in a few hours | 11:17 |
| anteaya | hopefully I will learn more then | 11:17 |
| *** Bada has joined #openstack-infra | 11:18 | |
| anteaya | and I mis-understood the comment from last night about many small clusters, he was referring to his business has many customers who each have small clusters | 11:19 |
| anteaya | so that was a red herring - sorry | 11:19 |
| sdague | anteaya: I think one of the issues is the large nodes we can get are 32GB total | 11:21 |
| anteaya | yes | 11:21 |
| anteaya | sdague: do you know if we have any access to tweak how the memory is allocated? | 11:22 |
| *** rickerc has joined #openstack-infra | 11:22 | |
| sdague | I don't know what the heap settings currently are on those nodes, and what else needs to run. Presumably if we start swapping we're sort of dead | 11:22 |
| anteaya | yes | 11:23 |
| anteaya | swapping is bad, I did get that message | 11:23 |
| sdague | fungi (should you be signing on today): https://review.openstack.org/#/c/51382/ really weird that I'm having whack-a-mole with os-loganalyze in being able to get unit tests to run because the tree won't check out | 11:24 |
| anteaya | I have to use my laptop for the tv presentation so I can't talk any more right now :( I will see about different arrangements next time | 11:24 |
| anteaya | so quiet for a bit | 11:24 |
| *** Bada has quit IRC | 11:25 | |
| *** davidhadas has joined #openstack-infra | 11:25 | |
| *** guohliu has quit IRC | 11:29 | |
| *** SergeyLukjanov has joined #openstack-infra | 11:39 | |
| *** cody-somerville has quit IRC | 11:41 | |
| *** Bada has joined #openstack-infra | 11:45 | |
| *** SergeyLukjanov has quit IRC | 12:20 | |
| *** CaptTofu has joined #openstack-infra | 12:23 | |
| *** SergeyLukjanov has joined #openstack-infra | 12:26 | |
| *** Bada has quit IRC | 12:27 | |
| *** SergeyLukjanov has quit IRC | 12:31 | |
| fungi | sdague: might be that race between repo creation on gerrit vs the git servers again. i've trigger a forced replication just now... see if that helps | 12:39 |
| *** openstackstatus has quit IRC | 12:51 | |
| *** SergeyLukjanov has joined #openstack-infra | 12:52 | |
| *** cody-somerville has joined #openstack-infra | 12:58 | |
| *** SergeyLukjanov has quit IRC | 13:02 | |
| *** CaptTofu has quit IRC | 13:02 | |
| *** CaptTofu has joined #openstack-infra | 13:03 | |
| anteaya | ES_HEAP needs to be 50% of RAM | 13:05 |
| *** CaptTofu has quit IRC | 13:07 | |
| *** SergeyLukjanov has joined #openstack-infra | 13:10 | |
| *** SergeyLukjanov has quit IRC | 13:10 | |
| *** thomasbiege has joined #openstack-infra | 13:15 | |
| *** garyk has joined #openstack-infra | 13:17 | |
| *** pcm_ has joined #openstack-infra | 13:18 | |
| *** thomasbiege has quit IRC | 13:19 | |
| anteaya | I'll but the rest in an etherpad | 13:20 |
| anteaya | s/but/put | 13:20 |
| *** pcm_ has quit IRC | 13:21 | |
| *** pcm_ has joined #openstack-infra | 13:22 | |
| *** thomasbiege has joined #openstack-infra | 13:22 | |
| *** SergeyLukjanov has joined #openstack-infra | 13:31 | |
| *** thomasbiege has quit IRC | 13:33 | |
| *** thomasbiege has joined #openstack-infra | 13:39 | |
| *** dizquierdo has joined #openstack-infra | 13:39 | |
| *** dhouck_ has joined #openstack-infra | 13:42 | |
| *** thomasbiege has quit IRC | 13:49 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 13:52 | |
| anteaya | etherpad url: https://etherpad.openstack.org/elasticsearch-optimization-suggestions | 13:55 |
| fungi | jeblair: more food for thought when you're pondering the job distribution imbalance, right now there's 2 devstack jobs going on jenkins01 and 16 going on jenkins02, so a similar proportion a good 12 hours after the restart | 14:11 |
| *** senk has joined #openstack-infra | 14:24 | |
| fungi | sdague: so apparently something was also wacky with puppet on git02 which prevented it from noticing your new project at all and creating a repo. ran puppet manually and it seemed to work but is still throwing some errors i'm looking into now. the change in question merged though and your repo looks good on all the git servers now | 14:36 |
| *** jaypipes has joined #openstack-infra | 14:38 | |
| fungi | ahh, yep... that'd do it! http://paste.openstack.org/show/48337/ | 14:41 |
| fungi | fix: sudo yum reinstall python-setuptools | 14:43 |
| *** cody-somerville has quit IRC | 14:44 | |
| fungi | oh, followed by: sudo pip install -U setuptools | 14:45 |
| mordred | morning | 14:45 |
| fungi | morning | 14:46 |
| mordred | fungi: setuptools broke somewhere new? | 14:46 |
| *** pcm_ has quit IRC | 14:46 | |
| fungi | mordred: the gitXX farm | 14:46 |
| fungi | fixing it on all 4 of them now | 14:47 |
| mordred | guh | 14:47 |
| mordred | fungi: you want me to dive in? | 14:47 |
| fungi | i suspect we're in a bad way with distro package automated upgrades when new versions of, e.g., setuptools get installed | 14:48 |
| fungi | mordred: i can fix it up manually, which i'm doing now | 14:48 |
| mordred | ok. I really want to figure out a better strategy for this | 14:48 |
| mordred | it's going to keep coming up | 14:48 |
| *** CaptTofu has joined #openstack-infra | 14:49 | |
| *** senk has quit IRC | 14:49 | |
| jeblair | fungi: ah, i think the nodepool allocator is unfairly distributing | 14:50 |
| jeblair | i think that perhaps it should consider nodes already assigned to targets as well as nodes it is about to assign to targets; then it should even out. | 14:51 |
| *** davidhadas has quit IRC | 14:51 | |
| *** CaptTofu has quit IRC | 14:52 | |
| *** CaptTofu has joined #openstack-infra | 14:53 | |
| *** anteaya has quit IRC | 14:53 | |
| fungi | that sounds reasonable | 14:54 |
| dstufft | mordred: make a fedora package of setuptools | 14:54 |
| dstufft | or don't use the global install | 14:55 |
| *** CaptTofu has quit IRC | 14:57 | |
| mordred | dstufft: those are both a non-trivial amount of work | 14:58 |
| mordred | dstufft: we consume python from source in a continuous delivery manner for 10-12 different applications over _several_ machines - on both centos and ubuntu | 14:59 |
| dstufft | mordred: fpm can probably make the first one reasonably easy | 14:59 |
| mordred | so whatever we do needs to be systemic and automatable and stuff | 14:59 |
| mordred | dstufft: if we make debs and rpms, we then need to make a place to put those. and we need to figure out when/how we make new ones | 15:00 |
| *** senk has joined #openstack-infra | 15:00 | |
| fungi | or preferably, automate creation of new ones each time a new version is released upstream, because security | 15:00 |
| mordred | do we make a packaging repo for each thing that we're making packages for and have that drive the uploading of packages into repos that we consume? | 15:00 |
| mordred | yah. | 15:00 |
| dstufft | surely puppet has a place to put them? I've never used puppet but when I needed to make an override package in chef I just chucked it in chef :V | 15:01 |
| mordred | dstufft: at heart, I agree with both of your suggestions - sorting the actual mechanics is tteh suck | 15:01 |
| mordred | dstufft: nope | 15:01 |
| mordred | at least, not in the way we use puppet | 15:01 |
| dstufft | welp | 15:01 |
| mordred | thing is - we don't wnat to be special | 15:01 |
| fungi | well, puppet could stash them onto the system and then use rpm to upgrade | 15:01 |
| mordred | we don't want fungi or I to upload some artifact by hand into a special location | 15:01 |
| jeblair | checking rpms into our puppet repo doesn't sound ideal | 15:02 |
| mordred | we want every aspect of our system to be potentially changable/patchable by anyone | 15:02 |
| fungi | but that would mean puppet reimplementing things like version parsing and other stuff that apt/yum already know how to do with their own repos, so suboptimal | 15:02 |
| mordred | no - I think the analog here | 15:02 |
| mordred | is that you can do things like "knife upload blah to my chef server" | 15:02 |
| mordred | and then you can reference that file in your recipes | 15:02 |
| dstufft | no I just commit it as part of the cookbook | 15:02 |
| mordred | puppet can do that too | 15:02 |
| mordred | what? | 15:02 |
| dstufft | cookbooks can have files associated with them | 15:03 |
| mordred | and you commit binary packages into them? | 15:03 |
| fungi | committing binary blob packages into your git repositories? | 15:03 |
| fungi | jinx ;) | 15:03 |
| dstufft | https://github.com/python/psf-chef/tree/master/cookbooks/stud/files/ubuntu-12.04 | 15:03 |
| dstufft | sure | 15:03 |
| mordred | wow. | 15:03 |
| dstufft | better then alternatives | 15:03 |
| dstufft | pragmatism, a thing ;P | 15:03 |
| dstufft | alternatives being setup an apt repo somewhere which isn't managed by git and requirs it's own infrastructure, or build from source on production machines | 15:04 |
| jeblair | i think if we're going to deal with packages, then we should have proper repos, and build packages in them from control files in git | 15:04 |
| mordred | or set up an apt repo somewhere which is managed by git and driven by our infrastructure | 15:04 |
| mordred | jeblair: I agree with that | 15:05 |
| jeblair | mordred: it's mutual :) | 15:05 |
| fungi | completely agree | 15:05 |
| mordred | jeblair: the question in my mind is - is it more work to do that | 15:05 |
| mordred | jeblair: or to rework our puppet installs of python things to do them into virtualenvs | 15:05 |
| dstufft | mordred: fundamentally the problem is you're trying to upgrade a system controlled package with pip and those two systems are going to fight each other | 15:05 |
| dstufft | so you need to stop doing that :V one way or another | 15:05 |
| fungi | yeah, which is why the virtualenv option is tempting | 15:05 |
| mordred | dstufft: totally grok the fundamental issue. it makes me unhappy and all, but totally grok | 15:05 |
| jeblair | mordred: i thought the last time we discussed this, we decided packages were desirable | 15:06 |
| mordred | jeblair: we did. | 15:06 |
| *** Lingxian has quit IRC | 15:06 | |
| mordred | jeblair: and setting up the repo is a piece of cake | 15:06 |
| *** Lingxian has joined #openstack-infra | 15:06 | |
| mordred | jeblair: I have not yet found a dev workflow for us that doesn't suck ass though | 15:06 |
| fungi | if we package-all-the-things it's preferable. and if we automate packaging one thing, then that's 99% of the work anyway | 15:06 |
| fungi | if we don't package-all-the-things then virtualenv is the only sane alternative i see | 15:07 |
| mordred | the main issue that wants to be solved is that what we _want_ on some of these things is to be running CD from trunk | 15:07 |
| jeblair | we have several friends already packaging much of our work | 15:07 |
| jeblair | mordred: so we let jenkins build packages on each commit and upload? | 15:08 |
| mordred | so making sure that the combination of the packaging code and the python code doesn't produce extra work for the "run every commit" workflow | 15:08 |
| mordred | right | 15:08 |
| fungi | well, i meant if we automate package generation and uploading for one thing, that's 99% of the work | 15:08 |
| mordred | I'm sorry, I'm being unclear | 15:08 |
| mordred | I'm talking about working out the specifics of how that works | 15:08 |
| jeblair | mordred: oh, details are important? | 15:08 |
| mordred | yes | 15:08 |
| jeblair | :) | 15:08 |
| jeblair | mordred: i think changes to package info is ~= changes to puppet now... | 15:09 |
| fungi | i suspect for anything using pbr we can just about cookie-cutter the actual deb/rpm package configurations | 15:10 |
| fungi | yeah | 15:10 |
| mordred | yup | 15:10 |
| mordred | so - we will have a basic cookiecutter packaging repo and a source code repo | 15:10 |
| fungi | at least, for sufficiently consistent/recent versions of pbr | 15:10 |
| mordred | we want to make a package on changes to either of those | 15:10 |
| jeblair | mordred: in that, if, say, we change how something is installed, we might have to change the code and change something in puppet. so from a high level, having to change something in code + a packaging repo shouldn't be a huge burden. | 15:10 |
| jeblair | mordred: sounds good so far | 15:11 |
| *** Lingxian has quit IRC | 15:11 | |
| dstufft | personally I think shoving everything in vitualenvs is the saner option but that's just me :] Sooner or later when you go down this path you end up with OpenStack linux instead of Fedora/Ubuntu | 15:11 |
| mordred | the version management in the debian/changelog needs to be automatically drivable from the package creation scripts (because thank you debian for making that a part of the system) | 15:11 |
| mordred | dstufft: oh - we know, that's why we don't make openstack packages | 15:11 |
| *** Lingxian has joined #openstack-infra | 15:11 | |
| mordred | dstufft: this is for deploying the infra pieces | 15:11 |
| mordred | dstufft: like zuul or nodepool | 15:12 |
| mordred | or the few other things that we have no other choice but to deploy from python directly | 15:12 |
| dstufft | mordred: yea, I've been messing around with stuff like that, I think my favorite method ATM is virtualenvs inside of system packages | 15:12 |
| mordred | my debian background would never let me do that | 15:12 |
| mordred | however... | 15:13 |
| dstufft | installing stuff globally has always ended up resulting in pain, sooner or later. For me at least | 15:13 |
| mordred | jeblair: dstufft has a point | 15:13 |
| * fungi is similarly allergic to that idea | 15:13 | |
| mordred | which is that the problem is not packaging zuul | 15:13 |
| fungi | but could be convinced it's sfe | 15:13 |
| fungi | safe | 15:13 |
| dstufft | but the debian package system has some nice stuff | 15:13 |
| mordred | it's packaging the things that zuul depends on | 15:13 |
| mordred | that we are installing via pip | 15:13 |
| dstufft | so combining the two gives you most of the benefits of packages, with all the messyness of python packaging constrained to build time | 15:14 |
| dstufft | debian packages* | 15:14 |
| jeblair | mordred: hashar has already done most of this work; we can work together and it won't be huge. | 15:14 |
| jeblair | i think it's crazy to need to install a virtualenv on a dedicated server | 15:14 |
| dstufft | as soon as you need a different version of something that the system provides (especally if it requires that system) it's prepare for pain time | 15:15 |
| mordred | jeblair: we do it for quoins though. I mean, to a certain extent, it seems that python basically just punted on the idea of being installable | 15:15 |
| fungi | yes, true, we stand to benefit from the fact that wmf insists on debs for everything they run, and they run our stuff, which means they made the debs for all the dependencies already | 15:15 |
| dstufft | it's Ok as long as your stuff and the system stuff doesn't converge | 15:15 |
| mordred | fungi: are those debs in a consumable repo? | 15:15 |
| jeblair | mordred: we _want_ to isolate quoins -- that's a shared server. | 15:15 |
| fungi | mordred: they could almost certainly be made so | 15:15 |
| mordred | jeblair: nod | 15:16 |
| dstufft | jeblair: I'm unsure why you'd ever not want to isolate something, :D | 15:16 |
| fungi | though i don't want to speak for the wmf peeps of course | 15:16 |
| jeblair | mordred: but yes, i agree that virtualenv has gone from being about isolation to being a crutch for installation. | 15:16 |
| jeblair | dstufft: isolate it from what? | 15:16 |
| jeblair | dstufft: the server literally does nothing else | 15:16 |
| dstufft | jeblair: the system | 15:16 |
| dstufft | the system uses python | 15:16 |
| dstufft | so no matter what you already have 1 "application" on the server | 15:16 |
| dstufft | which is the system itself | 15:16 |
| mordred | I think this is why docker is so popular right now | 15:17 |
| dstufft | somewhat yea I think | 15:17 |
| dstufft | isolation++ | 15:17 |
| mordred | well, isolation | 15:17 |
| jeblair | dstufft: perhaps fedora; not so much debian. there is really i promise absolutely nothing else on the system that uses python. but i do believe i understand your point. | 15:17 |
| mordred | and actually, the distros design is that pretty much everythign you install just becomes another module of the system | 15:18 |
| mordred | rather than having an OS/app model | 15:18 |
| mordred | where you have a system and you can run applications on that OS | 15:18 |
| mordred | I used to LOVE the conjoined nature of that design | 15:18 |
| mordred | but as time goes on, I believe that it's less and less advantageous | 15:18 |
| dstufft | Personally I want the OS to be it's own thing and leave my apps the hell alone | 15:19 |
| jeblair | mordred: i believe it's highly advantageous, but it requires the discipline to not use the shiny new thing right away. | 15:19 |
| fungi | well, stepping back to the stable server concept, even continuous deployment of a particular application shouldn't necessarily mean continuous deployment of all the dependencies too | 15:19 |
| *** guohliu has joined #openstack-infra | 15:19 | |
| mordred | jeblair: except for when the disto doesn't package the shiny new thing until you've started using it, because they're waiting for people to adopt it before they provide it | 15:19 |
| fungi | so being able to rely on a stable server distribution to provide you carefully backported security fixes for the same versions of those dependencies over a span of years can be a good thing, security and stability wise | 15:20 |
| jeblair | mordred: perhaps we could punt on this and say it is appropriate for some circumstances. | 15:20 |
| mordred | jeblair: yes! | 15:20 |
| mordred | jeblair: I totally agree with that statement | 15:20 |
| mordred | I actually think that's the key | 15:20 |
| mordred | depending on what you are doing, there are a couple of different sensible designs | 15:20 |
| jeblair | i'm really glad we don't have to compile and install apache | 15:20 |
| mordred | yup | 15:20 |
| mordred | and, you know, we install _some_ of our things from pypi instead of from master | 15:21 |
| mordred | because they're stable-er | 15:21 |
| mordred | and it makes more sense | 15:21 |
| dstufft | funny that, half the time I compile nginx :V | 15:21 |
| dstufft | instead of use system packages | 15:21 |
| mordred | dstufft: see, now, you're just showing your crazypants :) | 15:21 |
| fungi | dstufft: and recompile it every time there's a security release? | 15:21 |
| dstufft | fungi: yea | 15:21 |
| * mordred is with jeblair on this one | 15:22 | |
| fungi | fun. i used to do that with bind. it was a not-insignificant percentage of my work time | 15:22 |
| dstufft | mordred: well convince nginx to let me load modules dynamically and then we'll talk :d | 15:22 |
| mordred | ah | 15:22 |
| mordred | as an apache user, I have not be incited by that issue | 15:22 |
| mordred | s/be/been/ | 15:22 |
| dstufft | I track security releases for work anyways | 15:22 |
| dstufft | so it's nbd for me to add another one | 15:22 |
| mordred | tbf - we do install node.js from source | 15:22 |
| mordred | that is, however, because there is no packages | 15:23 |
| fungi | with bind, at least the excuse was that i got advance notice of vulnerabilities and source patches before the distros had updated packages, and as a service provider with very large dns deployments we couldn't afford to be vulnerable to any more 0-day issues than we could help | 15:23 |
| mordred | anyway - I thnk the key point is that packages are a bunch of extra work for tip-of-trunk things that move faster than important-release-per-6-months | 15:23 |
| mordred | BUT | 15:23 |
| mordred | still awesome | 15:23 |
| mordred | jeblair: I think I may start work on specific suggestions around packaging stuff for us | 15:24 |
| mordred | since I need to do it for internal anyway | 15:24 |
| dstufft | all my installs from source are in chef so typically unless building it has changed it's just bumping the URL I install from :] | 15:24 |
| dstufft | mordred: yea I can agree w/ that statement | 15:24 |
| jeblair | mordred: cool. i agree it's a lot of work, but if a little of your internal work can be combined with hashars work, it may not be terrible. | 15:24 |
| mordred | yah | 15:24 |
| jeblair | and pabelanger's work... | 15:24 |
| mordred | the main sticking point is going to be packaging repo lifecycle question | 15:25 |
| *** yolanda has quit IRC | 15:25 | |
| mordred | because I don't want to have different processes for package building for each thing we need packaged | 15:26 |
| mordred | but we now potentially have 4 different 'upstream' homes for the packaging work | 15:26 |
| mordred | us, debian/ubuntu, pabelanger, hashar | 15:26 |
| dstufft | mordred: FWIW this only solves the debian side of things but I started using http://labs.spotify.com/2013/10/10/packaging-in-your-packaging-dh-virtualenv/ the other day, it's got some rough edges but overall seems to work well… if you decide to go down that path | 15:26 |
| mordred | dstufft: thanks | 15:27 |
| mordred | jeblair: ok. actually, I might want you to look at the above | 15:28 |
| dstufft | I had to make a change to make it work on ubunt 12.04 (https://github.com/spotify/dh-virtualenv/blob/master/bin/dh_virtualenv#L66 should just be DebHelper(options) I think) | 15:30 |
| dstufft | note: I'm not a debian (or any other OS package) whiz, so it could be horrible and I just don't know it | 15:31 |
| *** dhouck_ has quit IRC | 15:32 | |
| *** ruhe has joined #openstack-infra | 15:33 | |
| *** w_ is now known as olaph | 15:41 | |
| *** dims has joined #openstack-infra | 15:50 | |
| *** markmcclain has joined #openstack-infra | 15:52 | |
| clarkb | you guys have been busy | 15:56 |
| clarkb | anteaya: sdague: the biggest node we can get is 30GB of that 16GB is hard allocated to jvm heap | 15:56 |
| *** cody-somerville has joined #openstack-infra | 15:57 | |
| *** davidhadas has joined #openstack-infra | 15:57 | |
| *** jean__ has joined #openstack-infra | 16:05 | |
| * pabelanger reads scrollback | 16:06 | |
| pabelanger | FWIW: I'm about to rebuild my zuul / jjb / gerrit / etc stuff so I plan to bump everything to the latest and greatest. So, I plan to package nodepool, get zuul going again, etc for debian. If that helps anybody | 16:07 |
| sdague | fungi: thanks for looking into the git thing, looks good now | 16:13 |
| fungi | sdague: no worries, and sorry about the broken | 16:15 |
| *** oubiwann_ has joined #openstack-infra | 16:15 | |
| *** senk has quit IRC | 16:16 | |
| *** dizquierdo has left #openstack-infra | 16:18 | |
| *** paul-- has quit IRC | 16:25 | |
| *** dhouck_ has joined #openstack-infra | 16:25 | |
| mordred | pabelanger: where do you keep your packaging repos and how are you dealing with dependent libs? | 16:32 |
| fungi | i get the impression he keeps (at least some of) them in the debian new queue, sid, jessie... | 16:34 |
| *** paul-- has joined #openstack-infra | 16:34 | |
| *** mrodden has joined #openstack-infra | 16:36 | |
| *** dhouck_ has quit IRC | 16:37 | |
| fungi | http://qa.debian.org/developer.php?login=paul.belanger@polybeacon.com | 16:37 |
| *** markmcclain has quit IRC | 16:38 | |
| mordred | fungi: https://github.com/pabelanger/jenkins-job-builder-deb | 16:38 |
| mordred | https://github.com/pabelanger/zuul-deb | 16:38 |
| fungi | oh, you mean the git repos with his packaging files. i thought you meant his package repos | 16:38 |
| *** markmcclain has joined #openstack-infra | 16:39 | |
| pabelanger | Yup, github | 16:39 |
| pabelanger | if a lib is missing, I add it to the list to package | 16:39 |
| *** markmcclain has quit IRC | 16:39 | |
| pabelanger | hasar has been doing some good work too | 16:39 |
| pabelanger | hashar* | 16:40 |
| fungi | yeah, i've seen itp bugs from him come across debian-devel recently for some of our ci bits | 16:40 |
| *** dcramer_ has quit IRC | 16:49 | |
| *** jean__ has quit IRC | 16:52 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Check devstack-gate changes on stable branch jobs https://review.openstack.org/49805 | 16:55 |
| mordred | fungi: if I want to test a puppet change on review-dev - puppet agent --test --environment=blah | 16:58 |
| mordred | is the way, yeah? | 16:58 |
| *** oubiwann_ has quit IRC | 16:58 | |
| fungi | stop puppet (if running, or comment out the cron job if on a slave) and then from a root shell (not via sudo) puppet agent --test --environment fungi (or whatever your environment is named) | 16:59 |
| mordred | yah | 17:00 |
| fungi | mordred: ^ | 17:00 |
| mordred | cool | 17:00 |
| mordred | that's working | 17:00 |
| fungi | i've also found, when testing on slaves which have cron-based updating, i need to patch my puppet environment to not turn that back on when it gets applied | 17:00 |
| fungi | so i just keep a simple commit for that around which i can cherry-pick onto whatever i'm testing | 17:01 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Parameterize server OpenStack-isms https://review.openstack.org/33926 | 17:01 |
| mordred | fungi: notice: /Stage[main]/Openstack_project::Gerrit/File[/home/gerrit2/review_site/etc/GerritSiteHeader.html]/owner: owner changed 'linuxjedi' to 'root' | 17:03 |
| mordred | I see this on each test | 17:03 |
| mordred | but when I look at the file, it's owned by root already | 17:03 |
| fungi | i think it's translating uids from the puppet master to the system where you're testing | 17:03 |
| *** mihgen_ has joined #openstack-infra | 17:03 | |
| mordred | ah | 17:03 |
| mordred | fascinating | 17:03 |
| fungi | is that file owned by you on the master? | 17:04 |
| fungi | if you don't set explicit ownership on file objects in a puppet manifest, i've found it makes some odd choices based on the original ownership on the master environment | 17:04 |
| mordred | fungi: yup | 17:05 |
| fungi | and your uid on the master is == to LinuxJedi's on the server where you're applying it? | 17:05 |
| mordred | ok. I've tested the above change and am going to apply it now | 17:05 |
| fungi | fun times | 17:05 |
| mordred | it does not change things in our confit | 17:05 |
| mordred | config | 17:05 |
| fungi | excellent. teh emailz must flow | 17:06 |
| fungi | also, confit sounds much tastier | 17:06 |
| openstackgerrit | A change was merged to openstack-infra/config: Parameterize server OpenStack-isms https://review.openstack.org/33926 | 17:07 |
| mordred | neat. I believe that all of the internal HP changes to infra/config are now merged in | 17:09 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Un-Revert "Make gerrit email server configurable" https://review.openstack.org/50501 | 17:10 |
| mordred | nope. that's not true. I need to check that one | 17:10 |
| *** SergeyLukjanov has joined #openstack-infra | 17:10 | |
| mordred | cool. that version of the patch works too | 17:11 |
| openstackgerrit | A change was merged to openstack-infra/config: Un-Revert "Make gerrit email server configurable" https://review.openstack.org/50501 | 17:15 |
| *** mihgen_ has quit IRC | 17:17 | |
| mordred | ok. both applied. watched the diffs go by. we should be good on both | 17:20 |
| *** CaptTofu has joined #openstack-infra | 17:26 | |
| *** SergeyLukjanov has quit IRC | 17:29 | |
| *** dcramer_ has joined #openstack-infra | 17:32 | |
| *** SergeyLukjanov has joined #openstack-infra | 17:33 | |
| *** yolanda has joined #openstack-infra | 17:38 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Remove ::1 mysql root user. https://review.openstack.org/45347 | 17:41 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Upgrade puppetlabs-mysql to 0.6.1. https://review.openstack.org/45348 | 17:41 |
| *** senk has joined #openstack-infra | 17:47 | |
| *** senk has quit IRC | 17:51 | |
| mordred | fungi: I couldn't get a review from you on https://review.openstack.org/#/c/35535/ could I? | 17:52 |
| mordred | fungi: I've been running it internally for the last week, fwiw | 17:52 |
| *** CaptTofu has quit IRC | 17:52 | |
| *** CaptTofu has joined #openstack-infra | 17:53 | |
| fungi | sure, i'll take a peek | 17:54 |
| mordred | thanks! | 17:54 |
| * mordred trying to remove local divergence so I can get the local thing to just pull directly from upstream and thus incite more direct upstreaming | 17:54 | |
| fungi | always good to be inciteful ;) | 17:56 |
| mordred | hrm. why do we directly depend on pygithub and gerritlib in jeepyb? | 17:56 |
| mordred | (in the puppet module) | 17:56 |
| mordred | shouldn't those just get installed via jeepyb? | 17:57 |
| *** CaptTofu has quit IRC | 17:57 | |
| fungi | i think in the past they did not, or we went through gyrations not to pull any additional transitive dependencies in from pypi so as not to replace older distro-provided versions where sufficient? | 17:58 |
| mordred | yah. I do believe that is why they are there currently | 17:58 |
| mordred | I believe that given how we are installing now, it's a bug-ish | 17:58 |
| fungi | possibly | 17:59 |
| fungi | it may be a pre-pbr-ism for jeepyb | 17:59 |
| mordred | yup | 17:59 |
| mordred | same with gerritlib in the gerrit module | 17:59 |
| mordred | it's there for things that used it before we put them into jeepyb | 17:59 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Remove explicit depends on transitive pip deps https://review.openstack.org/51402 | 18:00 |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: Do repo processing in an existing local dir https://review.openstack.org/35535 | 18:02 |
| mordred | fungi: woot! thank you! | 18:04 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Remove direct depend on virtualenv https://review.openstack.org/51403 | 18:05 |
| fungi | i'm taking puppet agent offline on review-dev for a bit to test 46339 if there are no objections | 18:05 |
| mordred | fungi: do it | 18:05 |
| * fungi clearly doesn't realize it's saturday | 18:06 | |
| mordred | fungi: well, it was a busy week | 18:07 |
| mordred | and some of these things, while not NEARLY as important as keeping the gate running, nag at is | 18:07 |
| mordred | us | 18:07 |
| fungi | yeah, i feel very behind | 18:08 |
| fungi | and i'll be mostly afk on thursday and semi-afk on friday (travelling to asheville to do a q&a for some comp-sci seniors) | 18:09 |
| *** mihgen_ has joined #openstack-infra | 18:14 | |
| mordred | oy | 18:15 |
| lifeless | vey | 18:15 |
| mordred | oh look. it's a lifeless | 18:17 |
| * lifeless looks around | 18:17 | |
| fungi | review-dev seems to be working as advertised after replacing its java dependencies with symlink to distro-provided equivalents and restarting | 18:24 |
| mordred | fungi: neat | 18:24 |
| fungi | i'll leave it in this state with puppet disabled for a bit in case anyone wants to beat on it | 18:24 |
| fungi | because, you know, EWEEKEND | 18:24 |
| lifeless | whats that | 18:24 |
| *** jean__ has joined #openstack-infra | 18:26 | |
| *** jean__ has quit IRC | 18:29 | |
| *** dcramer_ has quit IRC | 18:30 | |
| *** ruhe has quit IRC | 18:33 | |
| openstackgerrit | Roman Podolyaka proposed a change to openstack-infra/config: Move sa-migrate experimental jobs to check/gate https://review.openstack.org/51405 | 18:41 |
| *** mihgen_ has quit IRC | 18:47 | |
| *** Bada has joined #openstack-infra | 18:50 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Add branch tarballs for tuskar projects https://review.openstack.org/51406 | 18:53 |
| *** Bada has quit IRC | 19:03 | |
| *** senk has joined #openstack-infra | 19:18 | |
| *** Prince_Africanus has joined #openstack-infra | 19:21 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Update official list of ATC-qualifying projects https://review.openstack.org/50468 | 19:22 |
| *** senk has quit IRC | 19:22 | |
| *** rickerc has quit IRC | 19:26 | |
| *** rickerc has joined #openstack-infra | 19:36 | |
| *** SergeyLukjanov has quit IRC | 19:39 | |
| *** SergeyLukjanov has joined #openstack-infra | 19:40 | |
| *** dafter has quit IRC | 19:40 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Adding a Gerrit third-party testing account https://review.openstack.org/50448 | 19:41 |
| *** Prince_Africanus has quit IRC | 19:44 | |
| *** pcm_ has joined #openstack-infra | 19:47 | |
| *** davidhadas has quit IRC | 19:48 | |
| *** pcm_ has quit IRC | 19:51 | |
| *** pcm_ has joined #openstack-infra | 19:51 | |
| *** pcm_ has quit IRC | 19:51 | |
| *** pcm_ has joined #openstack-infra | 19:52 | |
| *** yolanda has quit IRC | 19:54 | |
| *** senk has joined #openstack-infra | 19:55 | |
| *** senk has quit IRC | 19:59 | |
| *** sarob has joined #openstack-infra | 20:06 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Puppet the Gerrit bcprov and mysql-connector deps https://review.openstack.org/46339 | 20:06 |
| fungi | thanks for the rebase, mordred | 20:08 |
| fungi | (though i tested and the previous patchset seemed to merge on the tip of master cleanly?) | 20:08 |
| mordred | fungi: I'm testing my remote launchpad_sync module, so I was testing it on top of your change ... | 20:08 |
| fungi | k | 20:08 |
| mordred | but then I couldn't see what difference my patch would cause to be installed | 20:08 |
| mordred | (pure selfishness :) ) | 20:08 |
| fungi | heh. cherry-pick to update your development environment instead of checkout ;) | 20:09 |
| mordred | yup | 20:09 |
| fungi | at least, that's how i usually get around it | 20:09 |
| mordred | ah - I totally shoud have done that | 20:09 |
| mordred | duh | 20:09 |
| mordred | I cherry picked _my_ change in - didn't think about cherry picking your change too | 20:10 |
| fungi | puppet development environment, i mean | 20:10 |
| * mordred si stoopid | 20:10 | |
| dstufft | mordred: y'all might like this https://gist.github.com/dstufft/6954117 | 20:10 |
| fungi | no worries | 20:10 |
| fungi | dstufft: neat! that's a potentially useful set of demographics to consider when working on packaging tools | 20:11 |
| mordred | nice | 20:12 |
| mordred | dstufft: I should make pbr set something so that I show up in your stats... | 20:12 |
| dstufft | PBR just uses pip right? | 20:12 |
| dstufft | so it'd show up under pip | 20:13 |
| mordred | ya. I was also kidding | 20:13 |
| dstufft | I'm thinking about making the user agent for pip configurable | 20:13 |
| *** dafter has joined #openstack-infra | 20:14 | |
| mordred | that might be interesting ... I mean, pip/1.5.dev1.micktwomey2 is showing up | 20:16 |
| mordred | I could make it be pip/%(actual_pip_version)s.pbr.%(pbr_version)s - but honestly, that's vanity - I'm much more interested to see the massively higher amount of work in 2.6 and 2.7 rather than 3.3 | 20:17 |
| lifeless | oh look infra people | 20:18 |
| mordred | it's also really nice to see that pip has such a high hit count over setuptools/distribute | 20:18 |
| * mordred looks | 20:18 | |
| lifeless | fungi: so, we now have the new overcloud existing | 20:18 |
| fungi | lifeless: exciting! | 20:18 |
| lifeless | fungi: I'd like to create some initial creds for infra for that | 20:18 |
| mordred | lifeless: I thought your overcloud went away every 50 minutes | 20:19 |
| lifeless | fungi: but rather than stashing them in a spreadsheet somewhere, could we stash them in hiera, ready for when the cloud does seamless upgrades? | 20:19 |
| lifeless | mordred: it does | 20:19 |
| mordred | ah. I follow your conversational thread now | 20:19 |
| dstufft | mordred: sooner or later these kind of stats will be recorded automatically and be part of PyPI | 20:19 |
| mordred | neat | 20:19 |
| *** dafter has quit IRC | 20:19 | |
| *** SergeyLukjanov has quit IRC | 20:20 | |
| fungi | lifeless: which hiera? is the puppetry for the overcloud going into openstack-infra/config or elsewhere? using the openstack.org puppet master server? et cetera? | 20:20 |
| mordred | fungi: overcloud no have puppet | 20:20 |
| mordred | fungi: I believe he's saying: | 20:20 |
| mordred | fungi: "I'd like to give infra an account on my cloud so that they can spin up nodes on it in nodepool" | 20:21 |
| lifeless | fungi: once it's continuously available (vs continously deployed) we'll move the tripleo nodes (managed by nodepool) from our grizzly cloud to thise one | 20:21 |
| mordred | fungi: but it's not doing seamless upgrades yet, so it's ready for that to be turned on | 20:21 |
| fungi | lifeless: i see, so you mean for integration with our ci jobs | 20:22 |
| lifeless | fungi: so -infra has one set of creds for a tripleo cloud today - the poc.tripleo.org cloud | 20:22 |
| openstackgerrit | A change was merged to openstack-infra/config: Remove launchpad_sync module https://review.openstack.org/47643 | 20:22 |
| lifeless | fungi: I want to give -infra creds for cd-overcloud.tripleo.org (which isn't yet suitable for -infra to use, but it's one less thing to remember later) | 20:22 |
| fungi | so stuff we would reference in our puppet configuration to tell our other systems how to authenticate to the overcloud | 20:22 |
| lifeless | yes | 20:22 |
| fungi | sounds great. if you want to encrypt them to me i'll be happy to put them in there. key is the same one published at http://www.openstack.org/projects/openstack-security/ | 20:24 |
| lifeless | we need an email address and account name | 20:24 |
| lifeless | openstack-infra is what I'd propose for the account name | 20:24 |
| mordred | fungi: ok. I've put review-dev back to being on your env | 20:25 |
| mordred | lifeless: this is a tenant for non-precious build hosts, yeah? | 20:25 |
| fungi | mordred: i had already switched review-dev back to production anyway since doing so doesn't undo the library linking | 20:25 |
| mordred | fungi: oh! ok. I'll just do that | 20:25 |
| fungi | previously not puppet-managed files, so it just leaves them there | 20:26 |
| mordred | ++ | 20:26 |
| *** dafter has joined #openstack-infra | 20:26 | |
| *** dafter has quit IRC | 20:26 | |
| *** dafter has joined #openstack-infra | 20:26 | |
| fungi | (that's the reason for my change, in fact... making sure they get managed) | 20:26 |
| *** jecarey has joined #openstack-infra | 20:26 | |
| lifeless | mordred: account and tenant, https://review.openstack.org/#/c/51354/5/scripts/assert-users | 20:26 |
| lifeless | mordred: or do you mean 'what sort of vm's would -infra run in this account' ? | 20:26 |
| mordred | right | 20:27 |
| *** yolanda has joined #openstack-infra | 20:27 | |
| lifeless | mordred: nodepool managed tripleo test nodes initially. | 20:27 |
| lifeless | mordred: for the baremetal test story pleia2 is building up | 20:27 |
| fungi | mordred: do we have a particular infra e-mail address we want to associate with that overcloud admin account, or create a new one? | 20:27 |
| lifeless | mordred: so perhaps openstack-infra isn't the right account name? | 20:27 |
| lifeless | fungi: you won't have admin acls on the cloud | 20:28 |
| lifeless | fungi: thats separate (and doable if you want, but we look for significant involvement with tripleo...) | 20:28 |
| mordred | lifeless: so - what we do now | 20:28 |
| lifeless | fungi: https://wiki.openstack.org/wiki/TripleO/TripleOCloud | 20:28 |
| fungi | lifeless: sure, so the tenant account, however that works out | 20:29 |
| fungi | i guess we don't need to be an admin for our tenant to spin machines up and down | 20:29 |
| mordred | lifeless: is on each cloud, we tend to have an account in which we put nodes that have absolutely no precious data (nodepool nodes) and a completely separate account for nodes that might want to stick around for longer than a second | 20:29 |
| mordred | lifeless: mainly, so that if there is a bug in one of our auto-spin-up-nodes scripts, we don't delete nodes that might take extra effort to recreate | 20:30 |
| lifeless | fungi: so the simple automation (see above) for asserting accounts doesn't currently do multi-user tenants | 20:30 |
| lifeless | fungi: if you need that I'll add it in | 20:30 |
| fungi | got it | 20:30 |
| mordred | lifeless: openstackci is what we call the long lived and openstackjenkins the short-lived - HOWEVER, neither of those names are actually great | 20:30 |
| lifeless | fungi: but you don't need the admin role to do vms and networks etc | 20:30 |
| lifeless | fungi: the admin role gets you well, admin | 20:30 |
| fungi | makes sense | 20:31 |
| mordred | I'd think perhaps "openstack-infra" and "openstack-nodepool" | 20:31 |
| lifeless | ok | 20:31 |
| mordred | since those express the intent better | 20:31 |
| lifeless | so lets make two accounts | 20:31 |
| fungi | i like those names | 20:31 |
| fungi | i suppose we can respin them later if jeblair and clarkb don't | 20:31 |
| lifeless | ok | 20:32 |
| lifeless | I need two email addresses then | 20:32 |
| mordred | infra-root@openstack.org for openstack-infra | 20:32 |
| mordred | jenkins@openstack.org for openstack-nodepool | 20:32 |
| mordred | unless fungi disagrees | 20:33 |
| fungi | mordred: sounds fine. we should, maybe, change the e-mail addresses on our other provider accounts at some point to match? | 20:33 |
| fungi | (for example, right now openstackci in rackspace goes to your inbox alone) | 20:33 |
| mordred | fungi: probably. we might want to make a nodepool@openstack.org to replace the jenkins@openstack.org too | 20:34 |
| mordred | fungi: GROSS | 20:34 |
| lifeless | https://review.openstack.org/51410 | 20:34 |
| mordred | lifeless: ossum | 20:34 |
| mordred | +1 | 20:34 |
| lifeless | I've monkeypatched that into prod, the next cycle will create the creds which I can then send to fungi | 20:35 |
| fungi | perfect, thanks! | 20:35 |
| openstackgerrit | A change was merged to openstack-dev/cookiecutter: doc: disable intersphinx mapping https://review.openstack.org/50009 | 20:35 |
| openstackgerrit | A change was merged to openstack-dev/cookiecutter: Ignore editor files. https://review.openstack.org/49318 | 20:35 |
| openstackgerrit | A change was merged to openstack-dev/cookiecutter: Actually git add the __init__.py file https://review.openstack.org/51238 | 20:36 |
| mordred | neat | 20:36 |
| mordred | I'm getting so close to having all of my outstanding config patches landed: https://review.openstack.org/#/q/status:open+project:openstack-infra/config+owner:mordred%2540inaugust.com,n,z | 20:39 |
| mordred | it probably would have helped it I hadn't uploaded three more today | 20:40 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Gate current clients on stable branches https://review.openstack.org/41931 | 20:41 |
| fungi | two patches forward, three patches back | 20:42 |
| mordred | yup | 20:43 |
| mordred | ok. I'm going to try to burn through my review load for openstack-infra/config | 20:44 |
| mordred | because it's MASSIVE | 20:44 |
| *** sarob has quit IRC | 20:45 | |
| mordred | fungi: I'm starting to think - not because any of them are urgent for me, but just because we do have a giant review load at all times | 20:47 |
| openstackgerrit | A change was merged to openstack-infra/config: Adding a Gerrit third-party testing account https://review.openstack.org/50448 | 20:47 |
| lifeless | maybe you need more reviewers | 20:47 |
| mordred | fungi: perhaps on 'trivial' things like small doc fixes or addition of straight-forward jobs that was done by another core person (such as your tuskar tarball patch) - that 2 core reviews might be overkill | 20:48 |
| mordred | lifeless: we DEFINITELY need more reviewers | 20:48 |
| mordred | lifeless: problem is, "but we look for significant involvement with infra" for that. | 20:49 |
| * mordred cries | 20:49 | |
| lifeless | I am reviewing infra dailyish | 20:49 |
| mordred | yah, which is appreciated, and I think your context for patches is definitely growing | 20:51 |
| mordred | GAH. I starred two of my own patches for review | 20:52 |
| openstackgerrit | A change was merged to openstack-infra/config: Remove ::1 mysql root user. https://review.openstack.org/45347 | 20:56 |
| openstackgerrit | A change was merged to openstack-infra/config: Upgrade puppetlabs-mysql to 0.6.1. https://review.openstack.org/45348 | 20:57 |
| clarkb | o_O | 20:58 |
| *** dmakogon_ has joined #openstack-infra | 20:58 | |
| clarkb | you will probably want to keep an eye on that one :) | 20:58 |
| mordred | clarkb: watching | 21:00 |
| mordred | clarkb: today is the day of watching puppet run on places | 21:00 |
| *** UtahDave has joined #openstack-infra | 21:01 | |
| mordred | clarkb: worked like a charm | 21:02 |
| mordred | clarkb: if you wanted to review https://review.openstack.org/#/c/51004/ and https://review.openstack.org/#/c/51402/ and https://review.openstack.org/#/c/51403/ while watching the football, I would not throw rotten vegetables at you | 21:04 |
| clarkb | :) maybe when I get home. on my way back to seattle just passing through centralia now | 21:05 |
| mordred | clarkb: ah. you probably don't want to review things just yet then | 21:07 |
| *** dcramer_ has joined #openstack-infra | 21:09 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add IRC bot on #openstack-rally for Gerrit changes https://review.openstack.org/51207 | 21:11 |
| *** nicedice has joined #openstack-infra | 21:11 | |
| *** _david_ has joined #openstack-infra | 21:14 | |
| *** SergeyLukjanov has joined #openstack-infra | 21:14 | |
| openstackgerrit | A change was merged to openstack-infra/config: Make novaclient-python33 gating https://review.openstack.org/51058 | 21:16 |
| openstackgerrit | A change was merged to openstack-infra/config: Correct the default cacti tree name https://review.openstack.org/50765 | 21:16 |
| openstackgerrit | A change was merged to openstack-infra/config: Add RTFD hook to trigger auto-build of docs https://review.openstack.org/48437 | 21:17 |
| openstackgerrit | A change was merged to openstack-infra/config: auto restart elastic recheck on update https://review.openstack.org/50688 | 21:18 |
| *** dafter has quit IRC | 21:18 | |
| lifeless | fungi: pasted to you | 21:18 |
| openstackgerrit | A change was merged to openstack-infra/config: Add missed savanna-docs job to post pipeline https://review.openstack.org/50616 | 21:18 |
| openstackgerrit | A change was merged to openstack-infra/config: Add Jenkins jobs for tuskar-ui https://review.openstack.org/50317 | 21:19 |
| SergeyLukjanov | is it possible to merge https://review.openstack.org/#/c/49167/ ? I hope that we'll be able to release our image-elements repo Oct 17... | 21:20 |
| mordred | SergeyLukjanov: "savanna-image-elements hasn't been registered on pypi" ... I think perhaps I shoudl do that real quick | 21:26 |
| SergeyLukjanov | is it real to do it before the Oct 17? | 21:26 |
| *** jaypipes has quit IRC | 21:27 | |
| SergeyLukjanov | if not - I'll remove pypi-upload from this CR to have at least tarballs published :) | 21:27 |
| mordred | SergeyLukjanov: easy. I'm going to grab your repo real quick and register it | 21:27 |
| SergeyLukjanov | mordred, oh, it'll be great, thank you! | 21:28 |
| mordred | this is reminding me - we need to set up that git grok thing from kernel.org to help with syncing local repos... | 21:29 |
| mordred | SergeyLukjanov: savanna-extra-0.3.rc2 | 21:30 |
| fungi | lifeless: received, thanks | 21:30 |
| mordred | SergeyLukjanov: https://review.openstack.org/51413 | 21:31 |
| SergeyLukjanov | merged, thank you | 21:32 |
| openstackgerrit | A change was merged to openstack-infra/config: Add tempest-devstack-vm-neutron-large-ops job https://review.openstack.org/50449 | 21:35 |
| *** jaypipes has joined #openstack-infra | 21:40 | |
| *** mihgen_ has joined #openstack-infra | 21:46 | |
| mordred | dstufft: is there a 'good' way to use pip to check to see if something needs to be upgraded? | 21:51 |
| dstufft | mordred: pip list -o ? | 22:01 |
| mordred | dstufft: that shows me all the packages I've got | 22:01 |
| dstufft | mordred: I don't think there's much else besides using package finder itself | 22:02 |
| mordred | k | 22:02 |
| dstufft | I normally just pip list -o | grep foo | 22:02 |
| mordred | I'm trying to make puppet be less stupid about what it's doing | 22:02 |
| mordred | and defer to pip, since pip knows how to read pip.conf files | 22:03 |
| dstufft | there's a open ticket from some chef people about adding a dry run mode | 22:05 |
| SergeyLukjanov | should gerrit work ok now? | 22:05 |
| *** dkranz has joined #openstack-infra | 22:06 | |
| mordred | SergeyLukjanov: it should. are you having issues? | 22:07 |
| SergeyLukjanov | mordred, yeah, can't push tag | 22:09 |
| SergeyLukjanov | ! [remote rejected] 0.3.rc3 -> 0.3.rc3 (can not create new references) | 22:09 |
| SergeyLukjanov | error: failed to push some refs to 'ssh://slukjanov@review.openstack.org:29418/openstack/python-savannaclient.git' | 22:09 |
| mordred | looking | 22:09 |
| mordred | SergeyLukjanov: you have no teams configured to be able to push tags | 22:09 |
| openstackgerrit | Endre Karlson proposed a change to openstack-infra/config: Add / Change python-libraclient jobs https://review.openstack.org/51069 | 22:10 |
| mordred | SergeyLukjanov: you need a refs/tags/* section with savanna-ptl listed in it (see the python-novaclient.config acl file) | 22:11 |
| ekarlso | approve that now mordred ? ;) | 22:11 |
| mordred | SergeyLukjanov: I'd do it for you, but my girlfriend has just told me that I need to go get some boxes from storage :) | 22:11 |
| SergeyLukjanov | mordred, I've create CR for updating ACL for savanna | 22:11 |
| *** dkranz has quit IRC | 22:11 | |
| SergeyLukjanov | mordred, thanks for info | 22:12 |
| mordred | SergeyLukjanov: sure thing! | 22:12 |
| mordred | SergeyLukjanov: also, I created s-i-e on pypi, and approved that change | 22:13 |
| *** _david_ has quit IRC | 22:13 | |
| SergeyLukjanov | great! | 22:14 |
| openstackgerrit | A change was merged to openstack-infra/config: Add jobs for extra and dib repos https://review.openstack.org/49167 | 22:14 |
| *** garyk has quit IRC | 22:14 | |
| *** garyk has joined #openstack-infra | 22:14 | |
| *** pcm_ has quit IRC | 22:14 | |
| *** dims has quit IRC | 22:21 | |
| SergeyLukjanov | mordred, hmm, I can't understand why I have permissions to push tag when https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/gerrit/acls/openstack/savanna.config#L4 | 22:24 |
| *** boris-42 has quit IRC | 22:27 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Limit savanna tagging to savanna-ptl group https://review.openstack.org/51417 | 22:29 |
| SergeyLukjanov | mordred, I think that ACL should looks like https://review.openstack.org/51417 | 22:29 |
| SergeyLukjanov | for Savanna | 22:29 |
| SergeyLukjanov | does it mean that savanna-ptl will be able to push tags to all branches | 22:30 |
| SergeyLukjanov | ? | 22:30 |
| *** yolanda has quit IRC | 22:30 | |
| fungi | SergeyLukjanov: yes, because tags go in a separate refs hierarchy from branches (heads) | 22:31 |
| fungi | tags aren't a branch-specific concept | 22:31 |
| SergeyLukjanov | fungi, now it looks much clearer for me:) | 22:31 |
| *** mihgen_ has quit IRC | 22:33 | |
| fungi | SergeyLukjanov: who are the ptl delegates for savanna with knowledge of how to correctly tag releases? | 22:33 |
| fungi | or really i can just add one person to the ptl group and they can add their delegates as they see fit | 22:34 |
| SergeyLukjanov | fungi, please, add me and I'll found someone else to avoid scary bus factor | 22:34 |
| openstackgerrit | A change was merged to openstack-infra/config: Limit savanna tagging to savanna-ptl group https://review.openstack.org/51417 | 22:34 |
| fungi | SergeyLukjanov: will do | 22:35 |
| SergeyLukjanov | fungi, thank you | 22:35 |
| fungi | it'll be a few minutes before that's reflected in gerrit, and then i'll add you to the group it creates | 22:35 |
| *** dmakogon_ has quit IRC | 22:43 | |
| SergeyLukjanov | fungi, I've pushed tag, so, it looks like it's already lanede | 22:46 |
| SergeyLukjanov | landed* | 22:46 |
| fungi | SergeyLukjanov: did it show up? it was still using the old acl (the new one isn't loaded yet)... maybe you mistyped your push command before? | 22:56 |
| SergeyLukjanov | hmmm | 22:56 |
| SergeyLukjanov | maybe I've used incorrect email | 22:57 |
| SergeyLukjanov | that is not added to gerrit | 22:57 |
| fungi | yeah, at the moment it's still using the old acl... https://review.openstack.org/#/admin/projects/openstack/python-savannaclient,access | 22:57 |
| fungi | but that acl change was a good idea anyway since it'll align you closer to how the official clients are working their release controls | 22:58 |
| SergeyLukjanov | fungi, yeah | 22:58 |
| *** wenlock has joined #openstack-infra | 23:00 | |
| SergeyLukjanov | thank you guys! | 23:09 |
| *** SergeyLukjanov has quit IRC | 23:09 | |
| *** mihgen has joined #openstack-infra | 23:13 | |
| *** openstackgerrit has quit IRC | 23:26 | |
| *** openstackgerrit has joined #openstack-infra | 23:26 | |
| *** senk has joined #openstack-infra | 23:43 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!