19:01:04 <clarkb> #startmeeting infra
19:01:05 <openstack> Meeting started Tue Jul 24 19:01:04 2018 UTC and is due to finish in 60 minutes.  The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:01:06 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:01:08 <openstack> The meeting name has been set to 'infra'
19:01:16 <clarkb> #link https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting
19:01:43 <clarkb> #topic Announcements
19:02:00 <clarkb> OpenStack PTL nominations open at the end of day today UTC
19:02:22 <clarkb> Also it is feature freeze week so everyone is running around crazy busy
19:02:54 <clarkb> #link https://etherpad.openstack.org/p/infra-ptg-denver-2018 Also the PTG is coming up, brainstorming ideas there much appreciated (also more detail on config management changes may be good)
19:03:33 <clarkb> #topic Actions from last meeting
19:03:51 <clarkb> #link http://eavesdrop.openstack.org/meetings/infra/2018/infra.2018-07-17-19.01.txt Minutes from last meeting
19:03:53 <mordred> o/
19:04:15 <clarkb> ianw and fungi were going to POC letsencrypt but I think fungi is still trying his hardest to do that vacation thing. ianw have you managed to look at that yet?
19:04:44 <clarkb> and mordred was going to start looking at porting our base puppet module to ansible (but I may have tricked him into reviewing some of the other config spec related puppet-4 changes instead)
19:04:54 <mordred> yeah - I reviewed a bunch of puppet
19:05:05 <mordred> just now have system-config open looking at the base ansible
19:05:32 <ianw> i have not done anything useful with that yet
19:05:45 <clarkb> fwiw the puppet-4 work from cmurphy seems to be moving steadily along. We've got two servers running with the puppet 4 parser now (review-dev and groups-dev) and I plan to add a third after this meeting (groups.o.o)
19:05:49 <mordred> also - so much puppet
19:06:19 <clarkb> when I say "I plan to" I really mean cmurphy did most of the work and I just get to approve the change and watch that the service remains happy
19:07:07 <clarkb> ianw: no worries. I know I'm deifnitely in this feature freeze making things unhappy firefighting state
19:07:23 <mordred> clarkb: that's what I usually mean when I say I'm going to do something
19:07:31 <mordred> clarkb: although sometimes s/cmurphy/clarkb/ :)
19:07:54 <clarkb> #topic Specs approval
19:08:07 <clarkb> #link https://review.openstack.org/585514 Follow up on config management update spec to make it a priority effort
19:08:08 <patchbot> patch 585514 - openstack-infra/infra-specs - Make update-config-management priority effort
19:08:11 <clarkb> I didn't catch that until today
19:08:31 <corvus> that seems more than reasonable :)
19:08:48 <clarkb> corvus: I think so :) appreciate it if we can get general consensus of +1s o nthat though
19:09:44 <clarkb> I'll admit I've largely been focused on this spec the last little while, but I've caught some movement on other specs from ianw and tristanC
19:10:06 <clarkb> ianw: ^ is the third party ci direction spec somethin we should be looking at nowish?
19:10:50 <clarkb> #link https://review.openstack.org/#/c/563849/ Third party CI direction spec
19:10:51 <patchbot> patch 563849 - openstack-infra/infra-specs - Direction setting for 3rd Party CI
19:11:00 <ianw> there was some chatter on it last week.  i will update from comments, then maybe we should loop back on it
19:11:11 <clarkb> ianw: ok, sounds like a plan
19:11:52 <clarkb> #link https://review.openstack.org/581214 Anomoly detection in CI/CD jobs spec
19:11:53 <patchbot> patch 581214 - openstack-infra/infra-specs - Add anomaly detection in CI/CD jobs specification ...
19:12:07 <clarkb> is the other spec, looks like tristanc may still be calling it a draft but I'm sure input if you have time would be appreciated
19:12:44 <clarkb> #topic Priority Efforts
19:13:28 <clarkb> I'm not sure how much of an update for storyboard we'll have. Seems like the biggest things happening there are feedback from new airship and starlingx users
19:14:15 <mordred> I thought cfriessen had a nice clear email on that topic to the list that felt like good feedback from a fresh set of eyes
19:14:25 <mordred> but I didn't have anything use to respond
19:14:46 <clarkb> yup the feedback was good, but I similarly don't really have enough of storyboard paged in to write a useful response
19:15:06 <clarkb> Storyboard's outreachy intern is also pushing up patches and getting local testing to work (setting up the database is always fun)
19:15:14 <clarkb> #link https://review.openstack.org/#/c/556648/
19:15:15 <patchbot> patch 556648 - openstack-infra/storyboard - Validating task status in the API
19:15:23 <clarkb> #link https://review.openstack.org/#/c/583085/
19:15:24 <patchbot> patch 583085 - openstack-infra/storyboard-webclient - Exposing task creator name in  search results
19:16:07 <clarkb> On the update config management spec (lets just assume it is a priority spec forthis meeting :) ) cmurphy continues to push on the puppet life support changes. topic:puppet-4 if you are interested in reviewing those
19:16:26 <clarkb> and the big chang ethere is we are now using the puppet 4 parser in puppet 3 on a couple nodes
19:16:58 <clarkb> I think the rough plan is to migrate things over to that newer parser as a pre step as it is just a line of config should we need to revert
19:17:07 <clarkb> then once those look good we can upgrade to puppet 4 proper
19:17:17 <clarkb> and that will get us on puppet that has life support
19:17:43 <mordred> ++
19:18:01 <clarkb> pbrx jobs are also showing up on some projects so I expect we'll start publishing images in the near future and possible work on converting some services (zuul nodepool?) directly to container based deployments?
19:18:33 <clarkb> pbrx is a tool monty wrote to build docker images for python projects given they use pbr and have bindep files
19:18:44 <clarkb> standardizes that process in a self contained way
19:19:23 <clarkb> mordred: any other config management update items for today?
19:20:08 <mordred> not really ... although at some point we might want to talk about whether pbrx being a random project is the right choice long term
19:20:36 <mordred> atm, infra-core (cause infra) and oslo-core (cause pbr) have +2 on it
19:20:44 <mordred> I think it's fine for now - just a note for the future
19:21:12 <mordred> oh, yeah - on the images
19:21:14 <corvus> i find it an un-objectionable project.  :)
19:21:19 <mordred> the images are buiding fine, whic his great
19:21:27 <mordred> next step is publishing them
19:21:37 <pabelanger> o/
19:21:58 <mordred> then yeah, they'll be available for us to start thinking about using in our zuul/nodepool deployments
19:22:13 <mordred> we should be able to add the job to any of our python/pbr-based projects
19:22:37 <clarkb> mordred: is there a goal of pushing that to openstack as well?
19:22:42 <mordred> for non-zuul things, I grabbed the openstackinfra org on dockerhub ... that sound like a good place to put things to folks?
19:22:51 <clarkb> mordred: wfm
19:22:57 <mordred> clarkb: I dunno - maybe? I mean, I don't want to drive that
19:23:03 <clarkb> mordred: fair enough
19:23:12 <mordred> it's certainly possible to use it on any openstack project
19:24:06 <mordred> in fact, I did chat with notmyname a bit about it - and it's _possible_ he might want to look at it for swift
19:24:20 <clarkb> mordred: oh right swift was why the alpine work in bindep started
19:25:13 <mordred> although there is a patch that needs to be submitted up to pyxattr first to make it able to build in alpine
19:25:41 <clarkb> that is independent of bindep and pbrx though right? that is a separate part of swift's dep tree?
19:25:45 <mordred> yup
19:26:11 <mordred> pyxattr is missing a #include <sys/types.h>  - in xattr/lib_build.py on in the final #else around line 40
19:26:20 <mordred> in case anybody feels like figuring out sending them a patch
19:26:42 <clarkb> heh
19:26:49 <clarkb> #topic General Topics
19:27:12 <clarkb> aiui jbryce does still intend on putting a followup email together on the winterscale naming effort
19:27:26 <clarkb> but not much other news on that front today
19:27:32 <clarkb> corvus: swift logs still something you would like to talk about
19:27:34 <clarkb> ?
19:28:25 <corvus> clarkb: left over from last week, but if folks have questions i'm happy to answer.  i expect to push up a reviewable version of the upload role ... today?
19:28:54 <clarkb> I look forward to reviewing that
19:29:03 <ssbarnea> one topic: git-review tool maintenance and reviews.
19:30:01 <clarkb> ssbarnea: we should have time at the tail end of items at https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting
19:30:10 <clarkb> pabelanger: base job update is next in the list
19:30:20 <clarkb> #link http://lists.openstack.org/pipermail/openstack-infra/2018-July/006030.html
19:30:43 <pabelanger> yay, this was mostly to see if people that it was a good idea or not for me to push up some patches
19:31:25 <scas> o/
19:31:27 <pabelanger> I think it would be something good to land to help minimize the base / base-test steps we need to do for untrusted changes, like configure-mirror
19:31:37 <clarkb> the summary is basically to reduce the size of our base job to the things that need privileged access to secrets and put the remainder of tasks in a separate base job that didn't need to go in a trusted repo
19:31:47 <pabelanger> clarkb: exactly
19:32:02 <clarkb> I'm in favor of it, my one concern is that this will affect all jobs and we are in the middle of feature freeze for openstack
19:32:15 <clarkb> waiting for next week to land changes is probably a good idea (though we could start proposing things now)
19:32:24 <mordred> ++
19:33:00 <pabelanger> sure, I can propose now, and we can review and decided when to land. We managed to do it in rdoproject without breaking anything :)
19:33:08 <clarkb> It will make it easier to test changes to the base too
19:33:10 <clarkb> which is ++ good
19:33:24 <clarkb> (because more changes will be self testing)
19:33:46 <pabelanger> ++
19:33:49 <corvus> yeah, i see no reason not to start on it, even if we avoid switching 'base' itself for a week or so
19:34:05 <pabelanger> thanks, all I had
19:34:24 <clarkb> pabelanger: you also wanted to talk about the long standingtodo to swich up how we grab logs on jobs
19:34:32 <clarkb> #link http://lists.openstack.org/pipermail/openstack-infra/2017-October/005606.html
19:34:45 <clarkb> I want to say mordred and ajaeger had been working on this
19:34:57 <clarkb> and it is largely a matter of moving things through in a backward compat manner?
19:35:01 <pabelanger> oh, yes. This is mostly saying I want take over driving this from mordred, I've started testing again, and seem like something we could also land after the base change above
19:35:15 <mordred> yes. pabelanger has picked up that ball. I also removed a -2 from frickler on the first job in the stack, since it was a procedural one and he's out righ tnow
19:35:38 <pabelanger> yah, I think it would be too late in the cycle to do it now, feature free, but something once stien starts
19:35:43 <clarkb> pabelanger: the base job split won't make this easier though as this requires secrets right?
19:36:10 <clarkb> ya making large scale changes to job function (not just organization) might be difficult
19:36:19 <pabelanger> clarkb: the first part of copying logs back to the executor won't need secrets, so that can be tested
19:36:23 <clarkb> but I think we can work on preparing for it while we wait
19:36:48 <pabelanger> yah, think it just needs eyes on the jobs, make sure we all understand, and schedule cut over
19:36:57 <pabelanger> and would like to do that after base changes
19:37:05 <clarkb> sounds reasonable to me
19:37:31 <clarkb> anything else on that?
19:37:32 <pabelanger> the goal is to also use this in rdoproject, log publisher
19:37:36 <pabelanger> none, thanks
19:37:42 <corvus> oh good
19:37:50 <corvus> that's the document i was asking about the other day
19:38:25 <corvus> i'll look that over again and talk with pabelanger if i see any potential conflicts with the swift logs work
19:38:32 <clarkb> corvus: ++
19:38:38 <pabelanger> wfm
19:38:56 <clarkb> ssbarnea: git-review tool maintenance and reviews you have the floor
19:39:52 <ssbarnea> i want to get git-review CRs reviewed and merged (or rejected).
19:40:26 <ssbarnea> not sure who to ping on these, https://review.openstack.org/#/q/project:openstack-infra/git-review+status:open
19:41:42 <clarkb> ssbarnea: fungi volunteered a long time ago to be the "maintainer" for git-review, I think many of us tend to defer to fungi on those sorts of decisions. Maybe a good step forward is for you to followup to the mailing list thread with a categorization of changes "bug fixes we need to merge, features that would be good to have under fungis criteria, features probably not acceptable" (or whatver the
19:41:44 <clarkb> criteria are) then we can get fungi and other infra reviewers to take a look at them?
19:42:10 <clarkb> ideally that would make it easy to get important bug fixes in asap then we can sort through the feature work
19:42:53 <ssbarnea> without mentioning specific reviews it looks like lack of  enough reviews for making them pass. we can remove those with negative votes and the list would still be long.
19:43:00 <ssbarnea> i will try the mailing list
19:43:31 <corvus> triage will help here
19:43:37 <clarkb> even if you just wanted to start by identifying some important bug fixes I think that would get the ball moving
19:45:50 <clarkb> and hopefully as the list gets smaller it will be easier to keep up
19:46:34 <ssbarnea> clarkb: that's the idea, to triage (including abandonin old reviews that are obsolete or not wanted).
19:47:10 <clarkb> ssbarnea: yup, I think if you are able to provide your triage to fungi (and the rest of us via the mailing list) that will help us further push things along
19:47:11 <ssbarnea> we did the same on jjb, it was a mess one year ago. time to "sanitize" the CR queue for git-review.
19:47:26 <mordred> ++
19:47:48 <ssbarnea> ok, i will do so. soon elextrofelix will be back from vacation and he will help too as he expressed interest in that.
19:48:30 <ssbarnea> i can +/- on that but is not helping too much, better make a list and send email to mailing list and cc fungi.
19:49:03 <clarkb> sounds good, thanks
19:49:10 <clarkb> #topic Open Discussion
19:49:23 <clarkb> With 10 minutes left anything else we want ot bring up?
19:50:15 <scas> i have an open review for unretiring the openstack-chef project i mentioned on #openstack-infra earlier. i understand that it requires gerrit downtime
19:50:57 <clarkb> scas: looks like you added it to the rename list on the meeting agenda
19:51:09 <clarkb> scas: our current penciled in date for doing those renames in August 3 around 1600UTC
19:51:24 <scas> that works for me
19:52:14 <clarkb> we'll probably follow up with you next week on making sure everything is ready to go on that
19:52:49 <scas> of course.
19:54:31 <clarkb> in general news we stopped using our mirror of pypi as we ran out of disk and are caching that with a proxy instead
19:54:56 <ianw> i've spent a bit of time getting readthedocs new api working; which now requires authentication.  see  https://review.openstack.org/#/c/583449/ .  however i think it will all work better with per-project info which i've proposed in zuul with https://review.openstack.org/584230 .  so reviews welcome on that
19:54:57 <clarkb> I've also been working with john_studarus to improve the reliability of packethost which seems to be pretty close to happy now
19:57:15 <clarkb> ianw: neat, feature would allow you to define project level variables that jobs consumed?
19:57:21 <clarkb> I think we actually used a lot of this in jjb once upon a time
19:58:54 <ianw> clarkb: yep, that's the intent of the zuul change @ https://review.openstack.org/#/c/583449
20:00:05 <clarkb> ianw: I like that
20:00:08 <clarkb> and we are out of time
20:00:12 <clarkb> thanks everyone
20:00:14 <clarkb> #endmeeting