19:01:48 <clarkb> #startmeeting infra
19:01:49 <openstack> Meeting started Tue Jan 15 19:01:48 2019 UTC and is due to finish in 60 minutes.  The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:01:51 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:01:53 <openstack> The meeting name has been set to 'infra'
19:02:01 <gary_perkins> o/ Hi!
19:02:22 <diablo_rojo> o/
19:02:43 <fungi> i'm around, just multi-tasking
19:03:23 <clarkb> #link https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting
19:03:26 <clarkb> I've undistracted myselc
19:03:43 <clarkb> #topic Announcements
19:03:59 <ianw> o/
19:04:05 <clarkb> First announcement is that fungi and I are in meetings next week and won't be able to pay attention to this meeting
19:04:15 <fungi> and i'm on vacation the week after
19:04:27 <clarkb> We'll need a volunteer to chair the meeting. I should be able to chair the week after though
19:04:46 <clarkb> The other announcement (which I added after the email went out) is that openstack board election is happening now
19:05:04 <clarkb> don't forget to vote and if you didn't get a ballot but expected one please reach out and we'll see what is going on
19:05:16 <fungi> or do like the major networks and play reruns from 2012 ci program meetings
19:06:15 <clarkb> #topic Actions from last meeting
19:06:26 <clarkb> #link http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-01-08-19.01.txt minutes from last meeting
19:06:40 <clarkb> One action was recorded and that was corvus to update the dns server documentation
19:06:46 <clarkb> fungi: ^ I think you were involved in that too
19:06:56 <corvus> i did not do it
19:06:57 <clarkb> corvus: fungi: is there a change that has merged or needs review yet?
19:07:05 <corvus> i have not forgotten, but omg so many things
19:07:11 <fungi> oh, yeah i just wasn't entirely sure what needed to go in there
19:07:44 <fungi> i guess it's stripping out the puppetish bits
19:08:05 <clarkb> #topic Specs approval
19:08:27 <clarkb> I actually pulled the anomoly detection change off the list hee due to the omg so many things problem
19:08:52 <clarkb> are there any urgent spec related items we should consider before moving on though?
19:09:59 <clarkb> sounds like no
19:10:01 <clarkb> #topic Priority Efforts
19:10:10 <clarkb> #topic Update Configuration Management
19:10:19 <clarkb> This is where many of the omg things have happened
19:10:35 <clarkb> in the last week we've started to build out a real docker image publishing and promotion pipeline
19:10:45 <clarkb> corvus: mordred ^ what should we all know about that?
19:10:52 <fungi> it's amazing. i'm reviewing more of it while i type this
19:11:02 <corvus> i'm going to move it to zuul-jobs today
19:11:28 <mordred> yah - it's the best.
19:11:41 <clarkb> up to and including the bazel memory problems :)
19:11:52 <corvus> and then write up a ml post about it and send it to openstack-discuss (so that openstack folks know they can use it), openstack-infra (so that we can use it for the container stuff), and zuul-discuss (so that the world knows about the promote pattern and the new roles)
19:12:05 <corvus> so i guess, probably 3 ml posts :)
19:12:47 <fungi> but maybe you can copy between them some
19:13:03 <clarkb> On the puppet 4 side of things cmurphy has a change up to flip one server over then another to flip all the servers over to puppet 4. I have unfortunately not had time to look at that as gerrit updates have distracted me. However I think this is also exciting and we should get around to review those changes topic:puppet-4
19:13:04 <corvus> i sure hope so
19:13:05 <mordred> I expect three custom written posts
19:13:15 <mordred> clarkb: ++
19:13:31 <clarkb> cmurphy: also in writing that last statement I realize that maybe we want to do an ubuntu server to puppet-4 then a centos7 server then the rest (since those are slightly different)
19:14:42 <clarkb> cmurphy: I can leave a note in review after the meeting too
19:15:08 <mordred> clarkb: good point
19:15:44 <clarkb> are there any other changes or topics within this space we should be aware of or reviewing?
19:17:09 <clarkb> sounds like no. We'll focus on rolling out the mat for puppet-4 and docker then
19:17:10 <fungi> the storyboard container stuff mordred started, maybe?
19:17:18 <clarkb> did that get picked up again?
19:17:34 <mordred> I'd like to change approaches there
19:17:41 <mordred> I guess it's worth talking about ...
19:17:45 <clarkb> go for it
19:17:55 <mordred> with the work we've done on gitea - and now on gerrit and the other things ...
19:18:01 <mordred> I believe the pbrx approach is the wrong approach
19:18:20 <mordred> I want to make a bindep builder image that we publish that can be used in other multi-stage dockerfiles
19:18:32 <mordred> and then make dockerfiles for storyboard and zuul and friends
19:18:55 <clarkb> the benefit there is being able to more completely control what goes into the final image?
19:18:58 <mordred> there's too much surrounding toolchain that understand working with dockerfiles - I don't thnik we get enough benefit from pbrx
19:19:23 <fungi> is this because of the non-python bits?
19:19:30 <mordred> clarkb: that and just not being super different/weird - it lets people more easily rebuild images from source in places that assume source repos have dockerfiles
19:19:37 <fungi> (javascript, images, css...)
19:19:38 <clarkb> gotcha
19:19:46 <mordred> not really ... those are actually fairly decently handled
19:19:53 <clarkb> I do like the idea of a consistent process for gerrit and storyboard and grafana and zuul
19:19:56 <mordred> it's just that it's divergent tooling for not enough benefit
19:19:59 <mordred> clarkb: ++
19:20:05 <mordred> that's the real key
19:20:46 <mordred> anywho - I'll try to get a bindep image cranked up RSN
19:20:55 <corvus> we'll end with a nice collection of dockerfiles, and docker-compose files
19:20:58 <mordred> yah
19:21:15 <corvus> might make local testing easier, but i've learned never to lead with that promise :)
19:21:49 <clarkb> #topic Storyboard
19:21:56 <clarkb> seems like that was a good lead into this topic
19:22:02 <clarkb> fungi: still fighting openid?
19:22:32 <fungi> if fighting is defined as caught up in start-of-year paperwork then sure ;)
19:23:20 <fungi> at least since last week, the server is up as storyboard-dev01.opendev.org and it's serving a storyboard-dev.openstack.org vhost successfully, but logging in leads to exceptions in the oauth module
19:23:32 <fungi> something about an invalid redirect (on returning from the provider)
19:24:17 <fungi> ruled out dns not being canonical, and ssl snakeoil cert not having a matching cn for the site name
19:24:19 <clarkb> fungi: maybe you can share the ip addr? I may have time to take a look at that when waiting for puppet runs later today
19:24:30 <fungi> it's in dns already as storyboard-dev01.opendev.org
19:24:36 <clarkb> set it up in /etc/hosts and see what firefox browser debuger says
19:24:42 <clarkb> oh perfect
19:24:50 <fungi> so just copy one of those addresses into your /etc/hosts
19:24:54 <fungi> and yeah, that's how i was testing
19:25:31 <fungi> i think next steps are to 1. compare pip freeze between the old and new servers, 2. try to sidexside diff the http parameters in the urls for a working and failing redirect
19:25:49 <clarkb> ya I was thinking I'd work on 2.
19:26:20 <fungi> one other note, they're both sharing the same trove instance at the moment, not sure if that could be causing an issue but i wouldn't be surprised if writes from one were wonky/colliding in the other at times
19:26:49 <fungi> but that seems unlikely to be what's surfacing this particular oauth/openid exception
19:27:30 <clarkb> good to know though
19:27:39 <clarkb> any other storyboard topics before we talk opendev?
19:27:57 <fungi> the stestr switch landed
19:28:14 <fungi> other than that, nothing i'm aware of
19:28:25 <clarkb> #topic OpenDev
19:28:33 <clarkb> some of the omg things happened in opendev land too
19:28:37 * diablo_rojo missed my window to mention https://review.openstack.org/#/c/555397/
19:29:12 <clarkb> diablo_rojo: looks like it needs a recehck?
19:29:17 <clarkb> (or maybe debugging)
19:29:28 <clarkb> #link https://opendev.org/walkthrough.html k8s walkthrough
19:29:42 <clarkb> if you missed it corvus recordred the k8s walkthrough we did last wee and you can find it at ^
19:29:45 <diablo_rojo> Yeah was roing to do a little of both.
19:30:11 <clarkb> Additionally I cleaned out all the old dns servers and we are now running on opendev.org name servers \o/
19:30:51 <clarkb> There are discussions around hosting zuul.opendev.org to enable some zuul multitenancy on our instance starting up. Let us know if you are interested in this (in particular zuul itself would like a tenant as would kata)
19:32:37 <clarkb> Other than that I think the big next item is making a decision on cert management for the new domain. ianw if you get a chance can you get that spec rebased?
19:32:48 <clarkb> https://review.openstack.org/#/c/587283/ specifically
19:33:43 <clarkb> Any other opendev topics to bring up before we move on?
19:34:46 <fungi> signs point to no
19:34:54 <clarkb> #topic General Topics
19:34:56 <ianw> clarkb: yep, i think a POC would be great there and will try to get something going ...
19:35:01 <clarkb> ianw: thanks!
19:35:02 * Shrews shakes fungi for more wisdom
19:35:23 <fungi> Shrews: outlook cloudy?
19:35:24 <clarkb> Ok I sent email about the PTG. Gave a rough outline of what I thought I'd ask for and didn't hear any dissent
19:35:41 <clarkb> I've got to get that in by Sunday so I'll probably just go ahead and do it hwne I'm waiting for puppet to run one of these times
19:36:09 <clarkb> to tl;dr Infra would like a room for ~2 days and ~10 people. I know this is all handwavy at this point and I'm sure our planners understand (right diablo_rojo ?)
19:36:29 <clarkb> I expect we'll be able to use that time to work on config managemnt updates and opendev work
19:37:01 <diablo_rojo> Yep :) We get it. get as close as you can and we can refine later.
19:37:28 <clarkb> Once we get a bit closer to the event I'll spin up a planning document (probably etherpad) where we can dig into details
19:38:14 <clarkb> any other thoughts about the PTG?
19:38:42 <fungi> looking forward to seeing folks in a place i can get to with a direct flight for a change ;)
19:39:08 <clarkb> fungi: I don't think I have a direct flight (but thats an airline thing not total lack of availability)
19:39:25 <corvus> it's hub-hub for me
19:39:35 <clarkb> Denver is beginning tofeel like the unofficial train depot for openstack though
19:39:44 <fungi> living where i do, i limit my flight choices to airlines which are willing to land planes here
19:39:48 <corvus> they have a nice train depot
19:40:13 <clarkb> #topic Open Discussion
19:40:20 <clarkb> may as well open it open to other items. We have time
19:40:21 * mordred is driving to denver again
19:40:33 <fungi> mordred: a cattle drive?
19:40:43 <mordred> fungi: yes. except with no cattle - or gasoline!
19:40:44 <clarkb> mordred: I was thinking about that too and trying to make a family trip out of it. That makes sense with 3 day conference but not 8 day marathon though
19:40:58 <clarkb> so we'll just do road trip nearer to home in the summer instead
19:41:02 <mordred> clarkb: ++
19:41:12 <mordred> clarkb: your family has more moving parts than mine does
19:41:13 <clarkb> https://review.openstack.org/#/c/631052/1 should fix the gerrit war publishing more and I've cleaned up the extra file copies from the last publish
19:41:22 <clarkb> if anyone has a quick moment to review that
19:41:49 <mordred> https://review.openstack.org/#/c/631047/ <-- I left out a gerrit plugin earlier
19:42:20 <mordred> oh - also ..
19:42:35 <mordred> drumroll please ...
19:42:37 <mordred> https://review.openstack.org/#/c/629585/
19:42:52 <mordred> puts all the new artifact/log copying into the base job!
19:42:55 <fungi> i was just looking at that one
19:42:57 <mordred> and is ready to land
19:43:05 <clarkb> I'll let fungi review it then :)
19:43:09 <fungi> (mainly trying to make sure this matches what has been done in base-test)
19:43:13 <corvus> woo!
19:43:24 <clarkb> mordred: my super secret plan was trying to somehow make transporting the smoker work out then we could do an infra bbq :)
19:43:28 <mordred> there's still a couple of patches to get artifact publication finished
19:43:35 <clarkb> mordred: reality is that smoker probably wasn't going to fit in with luggage anyway
19:43:41 <mordred> clarkb: I am in favor of that plan
19:43:43 <corvus> i could probably check my flowerpot smoker
19:43:52 <fungi> clarkb: i expect denver will have no shortage of bbq
19:44:11 <clarkb> fungi: ya and late april weather may not work out with being in a park
19:44:19 <clarkb> I just like sitting around outside with fire and beer :)
19:44:38 <anteaya> and mosquitos
19:44:43 <mordred> it's not a smoker ... but there's this: https://www.gozney.com/products/home-ovens/roccbox/
19:44:52 <corvus> fire smoke repells mosquitos
19:45:10 <mordred> corvus: ++
19:45:15 <anteaya> indeed, the magic of the combination
19:45:35 <mordred> I'll be driving through new mexico - could bring a bunch of piñon wood
19:45:51 <mordred> (for the mosquitos - not for the pizza oven)
19:46:12 <anteaya> I hope someday to smell what that wood smells like
19:47:17 <clarkb> considering discussion is mostly around wood smoke I'm going to call it a meeting and give you ~13 minutes for lunch/dinner/braekfast
19:47:21 <clarkb> thanks everyone!
19:47:21 <mordred> anteaya: imagine "pine" ... but more new mexican
19:47:24 <clarkb> #endmeeting