19:00:54 #startmeeting infra 19:00:55 Meeting started Tue Sep 30 19:00:54 2014 UTC and is due to finish in 60 minutes. The chair is fungi. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:56 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:00:58 The meeting name has been set to 'infra' 19:01:12 o/ 19:01:13 #link https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting 19:01:15 Hello everyone 19:01:16 o/ 19:01:24 This is Vijay Tripathi from Microsoft 19:01:31 hi there 19:01:39 hey there 19:01:44 we have a long agenda so i'm going to try to burn though it quickly in hopes we can fit it all into the hour 19:01:47 hi all 19:01:55 hey VijayTripathi 19:01:57 hi ociuhandu 19:02:11 #topic Actions from last meeting 19:02:24 #link http://eavesdrop.openstack.org/meetings/infra/2014/infra.2014-09-23-19.02.html 19:02:46 nibalizer file a story with a task for each project for http://specs.openstack.org/openstack-infra/infra-specs/specs/puppet-modules.html 19:02:55 looks like you did? 19:02:58 #link https://storyboard.openstack.org/#!/story/302 19:02:58 o/ 19:03:32 o/ 19:03:50 i will take his silence as assent, as well as a sign that he'll volunteer for any unclaimed action items henceforth ;) 19:04:02 jeblair send project-config announcement to -dev list 19:04:08 he's not here today, but he did 19:04:10 o/ 19:04:16 #link http://lists.openstack.org/pipermail/openstack-dev/2014-September/047207.html 19:04:30 mordred to write up autoincrement plan for infra bug import 19:04:34 looks like krotscheck did that 19:04:42 #link https://review.openstack.org/123821 19:04:47 Ayup. 19:04:51 krotscheck make list of infra projects in launchpad for krotscheck to use in testing 19:04:52 Landed 19:04:56 Also done. 19:05:02 #link https://etherpad.openstack.org/p/infra-storyboard-migration 19:05:05 awesome 19:05:06 Furthermore, testing complete, and bugs encountered have been landed. 19:05:15 krotscheck: you landed more bugs :P 19:05:22 I did! 19:05:36 clarkb: A ladybug and a grashopper. 19:05:43 and now we come to the last action item, which breaks our "accomplish all the things" mantra 19:05:45 fungi get devstack.org served by openstack rackspace dns account 19:06:00 minor snag with cruft in the provider's dns, blocked on service provider ticket 140930-ord-0001529 19:06:07 woooo 19:06:19 fungi: is there a tl;dr for that? 19:06:39 (I don't think the majority of folks here can see that ticket, I assume its a request for authority tarnsfer?) 19:06:45 yeah, the dns interface complains when i try to add devstack.org that there is a subdomain of that belonging to another tenant 19:06:49 well that knocks my item lower in the agenda off the list for this week 19:06:58 fungi: terrific 19:07:01 likely cruft in their database 19:07:32 anteaya: no problem. we can discuss that when we get to it, or skip and move it to next week, or address out of band in #openstack-infra later 19:07:40 yup 19:07:44 your call, chari 19:07:44 #topic Kilo summit topic brainstorming pad 19:07:47 chair 19:07:52 chair's perogative 19:08:05 #link https://etherpad.openstack.org/p/kilo-infrastructure-summit-topics 19:08:10 seems we didn't have one yet 19:08:21 let's all take some time to throw spaghetti at that and see what sticks 19:08:25 but not in this meeting 19:08:38 we'll need to chat with translators at some point, but I'm not sure whether that's our track or theirs (historically theirs I think?) 19:08:48 put it down 19:08:52 right 19:08:55 o/ yes i made a the story 19:08:55 k 19:08:56 translators don't have a track AFAIR 19:08:57 and then we can find the best home after 19:09:02 #topic Priority Efforts (fungi) 19:09:10 Puppet 3 Migration 19:09:28 I think the old master is still alive 19:09:32 is there pending cleanup out there for this? i think we can nail the coffin shut now right? 19:09:35 but other than that we are basically done 19:09:42 cool 19:10:03 i think it can be killed yea 19:10:05 #action fungi delete old ci-puppetmaster.openstack.org 19:10:15 Swift logs 19:10:23 #link https://etherpad.openstack.org/p/swift_logs_next_steps 19:10:41 jhesketh is around or still asleep? 19:10:47 fungi: can close bug 1263164 when that's done (puppetmaster) 19:10:48 Launchpad bug 1263164 in openstack-ci "Need to rebuild Puppet Master server" [Medium,Triaged] https://launchpad.net/bugs/1263164 19:11:52 pleia2: oh, thanks 19:11:55 fungi: I haven't seen him, must be sleeping. We are now running swift log uploads for more infra jobs though 19:12:02 fungi: which should give us better performance data 19:12:04 yep, saw that 19:12:08 yay 19:12:09 we should probably wait and collect stats 19:12:18 he was around during my morning, hacking long into the night... 19:12:22 yup I think next step is to come back here with a bit more data 19:13:02 #agreed continue watching infra project logs in swift for signs of bugs and performance issues, revisit next meeting 19:13:24 Config repo split 19:13:43 thanks to the hard work of a lot of people that seems to have gone really well 19:13:44 that is still pending the rename to system-config 19:13:53 yes 19:13:57 but yes, the hard part is behind us 19:14:01 thank you everyone 19:14:07 yes thank you, and it did go well 19:14:11 are there any other outstanding cleanup changes? 19:14:14 yes good job everyone! anteaya and corvus especially! 19:14:18 I know we are still sorting out jobs so that they work properly 19:14:21 nibalizer: thanks 19:14:21 sorry i was way out of this because of conference 19:14:29 nibalizer: np 19:14:37 i went looking for further cleanup an hour ago and found one more to approve 19:14:44 the jjb compare-xml merged I think 19:14:47 fungi: oh? 19:14:53 anteaya: yes, it merged 19:14:58 AJaeger_: yay 19:15:03 but in general now we're just treating anything else outstanding as bugs and attacking as they crop up 19:15:13 anteaya: your deletion change for layout-dev.yaml 19:15:18 yes 19:15:25 did that merge yet then? 19:15:32 i think so 19:15:34 have a link or number? 19:15:36 k, thanks 19:15:40 oh if it merged then nevermind 19:15:52 in any case point me at changes like that and I will happily prioritize them 19:15:59 same here 19:16:00 #link https://review.openstack.org/#/c/124898/ 19:16:09 missed removing a file 19:16:29 but there were no updates to it recently, so I didn't have to backport any content with it 19:16:34 so anyway, next phase for this spec is the rename, right? 19:16:38 yes 19:16:46 shall we schedule that 19:16:58 ya it will require gerrit downtime 19:16:59 is anyone already working on those bits or do we want to defer discussion to a later time? 19:17:04 we can piggyback the stackforge renames 19:17:08 I am not working on it 19:17:15 I am not working on it 19:17:20 presumably we can take care of the outstanding stackforge project renames in te same window, assuming their changes are ready 19:17:28 er, what anteaya said 19:17:37 the changes are in project-config and look ready to me 19:17:41 for stackforge 19:18:05 did someoen want to volunteer to start prepping changes for the system-cofnig rename? 19:18:10 so next we need rename changes for openstack-infra/config or at least some digging to see how much will break when we do that 19:18:17 there are probably quite a few locations that need to be renamed 19:18:20 I don't know what to do to prepare 19:18:26 I don't have a test env locally 19:18:30 i can take a stab at it 19:18:35 anteaya: (and others) we will need a change to do a normal project rename 19:18:44 that I can do 19:18:50 but if someone else wants to volunteer for it, all the better 19:18:52 along with changes that update url paths everywhere that we clone openstack.org/openstack-infra/config 19:18:57 but not the testing stuff, though I can review the testing stuff 19:19:06 thanks anteaya 19:19:06 since we clone that path for image builds and puppet installs and so on 19:19:14 anteaya: sounds good and thank you 19:19:29 kk 19:19:38 #action anteaya take a stab at writing the openstack-infra/config rename changes 19:19:58 Nodepool DIB 19:20:03 what's the current state? 19:20:07 we have nodes now, right? 19:20:14 yay nodes! 19:20:24 we do 19:20:27 and i think some tests got run on them and passed? 19:20:33 and we have jobs under d-g experimental and they pass 19:20:41 what's the current situation in rax? 19:21:02 ianw has written a nodepool patch that needs reviewing. It allows us to set image metadata when we upload images 19:21:12 that patch allows us to set the things we need for rax 19:21:40 so I think what I would like to do is switch hpcloud-* devstack-trusty to run off of the dib image, then put the devstack-trusty-dib image in rax 19:22:13 but we are in rc time so not sure how comfortable we are with that 19:22:25 * clarkb gets link to nodepool change 19:22:27 have a link 19:22:28 ? 19:22:43 ianw_pto seems to be on vacation 19:22:53 #link https://review.openstack.org/#/c/123639/ 19:23:04 ya he is out but I asked for the change to go up so that we can keep iterating 19:23:23 which i assume means i can also scratch his agenda items for today unless others are taking them on 19:23:41 probably 19:24:19 is there opposition to moving hpcloud devstack-trusty to the dib image because it is RC time? 19:24:20 I don't see any problem switching hpcloud over 19:24:27 if the tests are passing 19:24:37 mordred: ya the mysql and postgres normal jobs are passing 19:24:40 like, I don't expect these nodes to all of a sudden be flaky 19:24:52 I expect them to diaf if we didn't get something installed right :) 19:24:58 good point 19:25:11 thats the golden image manifesto 19:25:14 I will also be proposing devstack-precise-dib images asap 19:25:23 okay, awesome 19:25:34 any specific action items here or just revisit next week? 19:25:52 not really, I will keep at it as able and beg peopel for reviews :) 19:26:00 but please review that nodepool change 19:26:39 Docs publishing 19:26:46 anything new on this?> 19:26:54 I like publishing docs 19:26:54 * annegent_ sits on the edge of her seat 19:27:11 I think that is mostly needing humans to be thrown at it 19:27:21 clarkb: they bounce you know 19:27:45 or raisinettes. those actually stick 19:27:48 I want to work on this after devstack and system-config 19:27:48 there is a spec though so volunteers can start throwing code up for that 19:27:56 because I think I can do something here 19:28:10 but someone else is welcome to beat me to it since it is down my list 19:28:20 #link http://specs.openstack.org/openstack-infra/infra-specs/specs/doc-publishing.html 19:28:39 I will share at Rackspace in case anyone is interested 19:28:56 okay, sounds like nothing new on this front yet. sorry to get your hopes up annegent_ 19:29:01 thanks, annegent_ and anteaya ! 19:29:06 fungi: no worries :) 19:29:07 annegent_: and thanks for the advertising ;) 19:29:22 Jobs on trusty 19:29:57 #link https://etherpad.openstack.org/p/py34-transition 19:30:00 still holding on new py34 ya? 19:30:06 yeah 19:30:20 and there are a few projects down at the bottom of that etherpad which still need investigating 19:30:45 anyone with some spare cycles who wants to debug random projects, try cloning those and running their py34 tox envs 19:30:54 on a generic ubuntu trusty box 19:31:24 but we can't cut over until at least the backport in 1367907 is addressed 19:31:29 #link https://launchpad.net/bugs/1367907 19:31:31 Launchpad bug 1367907 in python3.4 "Segfault in gc with cyclic trash" [High,In progress] 19:32:03 okay, that covers the priority efforts 19:32:12 #topic Call for help: lists.o.o and wiki.o.o (reed) 19:32:48 just a quick heads up that our community volunteer admins who were taking care of projects related to lists.openstack.org and wiki.openstack.org haven't been around much lately 19:33:15 we have a few outstanding bugs for them which could use some attention if there are mediawiki or mailman experienced people with free time to devote 19:33:25 #link https://launchpad.net/bugs/1254911 19:33:26 Launchpad bug 1254911 in openstack-ci "UniversalLanguageSelector breaks wiki skin" [Medium,Triaged] 19:33:40 I wonder if hashar has interest in the mediawiki things? 19:33:48 maybe we can draft hashar into doing that :P 19:33:51 #link https://launchpad.net/bugs/1339284 19:33:53 Launchpad bug 1339284 in openstack-ci "Import mailman archive for General list" [Medium,Confirmed] 19:33:59 #link https://launchpad.net/bugs/1182690 19:34:00 Launchpad bug 1182690 in openstack-ci "Add mailman templates in other languages" [Medium,Triaged] 19:34:24 and yeah, maybe we can shanghai him with our press gang when we find him in paris ;) 19:34:45 #topic Fedora/Centos testing updates (ianw) 19:35:11 looks like the change for "Initial centos7 support for build-image.sh" conflicted with the project-config split and got abandoned 19:35:18 not sure if it's been reproposed to project-config yet 19:35:31 I haven't seen it 19:35:33 with ianw_pto on vacation, we can probably defer discussion to next week 19:35:45 #topic Log Download (ianw) 19:36:08 looks like he had a review for "Add ability to download log bundles" he also wanted to discuss 19:36:15 #link https://review.openstack.org/122615 19:36:29 so we can defer that discussion to next time he's around as well 19:36:41 #topic Publish devstack.org content under infra (anteaya) 19:36:45 I'm wrong 19:36:48 #link https://review.openstack.org/#/c/124256/ 19:36:49 fungi, it was reproposed 19:36:52 anteaya: defer while we work on my failure to execute? 19:37:01 yes 19:37:07 AJaeger_: anteaya: thanks 19:37:16 #topic StoryBoard Migration (krotscheck) 19:37:21 Wooo! 19:37:22 take it away, krotscheck 19:37:29 clarkb: unlikely :/ 19:37:32 So, issues raised in the meeting last week have been addressed. 19:37:47 The list of commands that need to be run are here: https://etherpad.openstack.org/p/infra-storyboard-migration 19:37:57 The outstanding issue is that we don’t have full infra features ready yet. 19:38:01 clarkb: but I will be happy to relay your MediaWiki related issues to our community. 19:38:09 hashar: thanks, that works 19:38:29 So that’s my update, and I’ll be actively pestering all of you over the next few weeks as we get those features polished. 19:39:15 A discussion on whether y’all want to do a test migration of one of the projects before throwing the big switch might be good. 19:39:20 krotscheck: full infra features? 19:39:41 #link https://etherpad.openstack.org/p/infra-storyboard-migration 19:39:44 clarkb: The version 1.1 features agreed to in Atlanta. Outstanding is Project Groups, Emails, Subscriptions. 19:39:57 gotcha 19:40:07 clarkb: Oh, and tags. 19:40:21 The active list is in our meeting notes, here: https://wiki.openstack.org/wiki/StoryBoard 19:40:30 o/ 19:40:51 * krotscheck is done 19:40:54 #link https://wiki.openstack.org/wiki/StoryBoard 19:40:55 Unless anyone has questions. 19:41:20 no questions from me. did we need to discuss migration timeline now or wait a week? 19:41:54 I mean, I'm keen to just do it even without all of those things 19:42:02 but people should maybe ignore me 19:42:04 mordred: ya part of me thinks that may help snowball the process 19:42:11 mordred: but those features sound usefull too :) 19:42:11 we could just migrate everything while jeblair's away, to keep him on his toes 19:42:18 ha ha ha 19:42:23 that makes him happy 19:42:40 hehe 19:42:48 Hey, I’m just building the tools. Using them for good or ill is up to you crazy people :D 19:42:50 he has to be expecting SOMETHING big 19:42:54 * krotscheck suggests a vote :D 19:43:45 #vote Migrate any infra project bugs to Storyboard this week? none, some, zomgeverything 19:44:00 er, i always forget the syntax 19:44:08 #vote some 19:44:19 you want startvote 19:44:23 #startvote Migrate any infra project bugs to Storyboard this week? none, some, zomgeverything 19:44:24 Begin voting on: Migrate any infra project bugs to Storyboard this week? Valid vote options are none, some, zomgeverything. 19:44:26 Vote using '#vote OPTION'. Only your last vote counts. 19:44:37 #vote some 19:44:37 * AJaeger_ abstains 19:44:41 yeah, just found our documentation ;) 19:45:06 #vote none 19:45:08 * krotscheck none 19:45:11 * nibalizer abstains 19:45:12 #vote none 19:45:18 i'd like to see how much further the featureset comes first 19:45:21 The world won’t end if we wait a week. 19:45:27 #vote none 19:45:29 #vote none 19:45:30 you guys have convinced me 19:45:39 * anteaya is out on the cliff dancing alone 19:46:02 anteaya: very brave spot :) 19:46:13 and a fool sometimes too 19:46:24 but the view is awesome 19:46:33 I know jeblair wants them mvoed soon, another bug day with lp might kill him 19:46:58 we could vote on which thing jeblair would hate more 19:47:04 last call 19:47:08 #vote zomgeverything 19:47:13 just for completeness 19:47:16 #vote vote 19:47:17 awesome 19:47:17 lifeless: vote is not a valid option. Valid options are none, some, zomgeverything. 19:47:30 #vote none 19:47:33 Oh, forget it. 19:47:36 * anteaya sees mordred far ahead in the distance, dancing on clouds 19:47:38 #vote zomgeverything 19:47:47 Because, really, what could possibly go wrong. 19:47:51 BOOM! 19:47:52 (tm) 19:47:57 #endvote 19:47:57 Voted on "Migrate any infra project bugs to Storyboard this week?" Results are 19:47:58 zomgeverything (2): mordred, krotscheck 19:47:59 none (4): SergeyLukjanov, pleia2, fungi, clarkb 19:48:00 some (1): anteaya 19:48:18 SergeyLukjanov is here? 19:48:23 there you are 19:48:32 okay, so let's revisit next week and see if we think we have cycles to give it a shot 19:48:49 anteaya, yup, trying to be here 19:48:52 cool 19:48:53 #topic Havana EOL (fungi) 19:48:57 yay 19:49:01 nice work fungi \o/ 19:49:07 indeed 19:49:14 ++ 19:49:18 pubic service announcement: havana is dead 19:49:23 #link http://lists.openstack.org/pipermail/openstack-announce/2014-September/000286.html 19:49:53 yay!!! 19:50:06 keep an eye out for any unanticipated breakage, but i plan to delete the stable/havana branches of official projects listed in that announcement later today 19:50:18 cool 19:50:52 i'm thinking we might want to defer cleanup of devstack/grenade/devstack-gate until we're ready to add juno to them. otherwise we lose some (particularly grenade-specific) logic in the scripts 19:51:03 agreed 19:51:08 but the jobs are no longer running and have been removed from our configuration 19:51:26 #topic How should 3rd party ci leverage new project-config? fork? branch? don't directly, just as reference? (asselin, 2nd half of meeting) 19:51:41 asselin: you here? 19:51:44 Hi, that's my question 19:51:45 asselin: you don't have much time left, but you're welcome to 8 minutes anyway ;) 19:51:47 yay 19:51:56 go asselin ! 19:52:20 so I'm looking at project-config, and not sure how best to leverage for third party ci 19:52:46 asselin: which sorts of things do you think make sense for you in 3rd party land? jjb templates? 19:52:50 fork it? create my own branch to store my 'private' jobs, or ?? 19:53:21 like, are you wanting all of the infra jobs plus some jobs of your own? 19:53:33 * mordred wants to understand which pieces are compelling to leverage 19:53:46 honestly, I was hoping to see if I can use project-config as a template to hook into the rest of the infra-config. i.e. to help setup zuul, nodepool, etc. 19:54:13 sot he piece is really the layout, and maybe some of the scripts. 19:54:29 gotcha. 19:54:40 asselin: are you also running the openstack jobs, or totally your own tests? 19:54:56 asselin. i had a similar impression 19:54:59 basically my own (copied and modified from openstack) 19:55:00 I wonder if having a third-party-project-config repo which contains stuff that third parties would want makes sense 19:55:10 i think maintaining a (very loose) fork is probably going to work best. the codiest bits are the nodepool prep scripts, dib elements, and jenkins slave scripts. maybe also some job definitions/templates/macros 19:55:13 mordred: it does to me 19:55:26 mordred, i think it would 19:55:35 I spoke with hogepodge last week and he said he was interested in making things more consumable for third party ci folks 19:55:36 can we fork it into a third party config repo 19:55:46 I can follow up with him to help out 19:56:13 it would help give a single point of focus 19:56:19 i need to convince hogepodge to lurk in all the channels, now that he's at the foundation 19:56:20 anteaya: yeah 19:56:24 folks are trying hard to do the right thing 19:56:26 we discussed that a few weeks ago ++ from me 19:56:28 fungi: yeah, he's in -infra but not here :) 19:56:38 they just get conflicting advice on what that is 19:56:51 anteaya, one point i have is that sometimes we need a hybrid of both upstream and our own scripts 19:56:52 so having one repo would really help everyone 19:57:05 so we would need both repositories, not only a fork 19:57:07 we need a place to hook in then 19:57:16 on forj redstone, general use scripts would certainly be allot better to maintain , +1 for that 19:57:18 so you can use your own scripts 19:57:24 "your scripts go here" 19:57:45 so my plan for doing that in our 3rd party, is to build a tool that consumes the upstream layout.yaml, our private layout.yaml and generates the combination of both 19:57:58 ooh. tools! 19:57:58 anteaya, that could work. or some way to also "override" one of the existing scripts 19:58:00 but i haven't done any of that yet so its just a plan 19:58:05 adalbas: right 19:58:11 * mordred throws fluffy bunny at nibalizer 19:58:13 to minimize the need to fork 19:58:23 yes, override is a good word... 19:58:30 and just use the thing, because the thing is designed for you to use 19:58:39 the way you override things is by passing ina different project-cofnig repo 19:59:04 clarkb, right...so what's the best way to create your own project-config repo? 19:59:18 almost at time 19:59:26 yeah 19:59:41 but we want to stay as close to upstream as possible, a new repo becomes a problem 19:59:42 so maybe go around again 19:59:42 ok, seems there's no answer, so I'll play around with it a bit. 19:59:42 we can continue this in #openstack-infra, as usual 19:59:51 since I sense agreement in principle 20:00:00 the project-config split was done to facilitate all of this. all you should need to do is put your additional/different scripts in a repo that looks like project-config 20:00:00 thanks everybody! 20:00:03 #endmeeting