19:02:42 <jeblair> #startmeeting infra
19:02:43 <openstack> Meeting started Tue Aug 19 19:02:42 2014 UTC and is due to finish in 60 minutes.  The chair is jeblair. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:02:44 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:02:46 <openstack> The meeting name has been set to 'infra'
19:02:57 <jeblair> #link agenda https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting
19:03:00 <jeblair> #link actions from last meeting: http://eavesdrop.openstack.org/meetings/infra/2014/infra.2014-08-12-19.02.html
19:03:12 <jeblair> #topic  Actions from last meeting
19:03:16 <pleia2> o/
19:03:18 <nibalizer> o/
19:03:19 <grantbow> o/
19:03:19 <zaro> o/
19:03:19 <wenlock> o/
19:03:26 <dougwig> o/
19:03:28 <krtaylor> o/
19:03:38 <fungi> hey-o
19:03:46 <jeblair> jeblair Publish python-jenkins 0.3.3 (last release without pbr) tarball to pypi
19:03:49 <anteaya> o/
19:03:50 <jeblair> i did not do that
19:03:52 <clarkb> o/
19:03:59 <jeblair> #action jeblair Publish python-jenkins 0.3.3 (last release without pbr) tarball to pypi
19:04:09 <jeblair> #topic  Priority Specs (jeblair)
19:04:38 <jeblair> hey, so what with the fact that we're constantly swamped with reviews....
19:04:44 <fungi> we should review those
19:04:54 <jeblair> some projects have taken to highlighting changes that are a priority
19:05:04 <jeblair> and i thought maybe a good first step for that would be specs
19:05:24 <fungi> wholeheartedly agree
19:05:38 <clarkb> ++
19:05:40 <jeblair> so these are some specs that are really strategic for what we want to do this cycle (and probably next too :)
19:06:11 <jeblair> #link priority spec: https://review.openstack.org/#/c/100363/
19:06:11 <jeblair> #link priority spec: https://review.openstack.org/#/c/99990/
19:06:11 <jeblair> #link priority spec: https://review.openstack.org/#/c/110730/
19:06:11 <jeblair> #link priority spec: https://review.openstack.org/#/c/110793/
19:06:24 <jeblair> so if you have to pick some, those would be a good place to start
19:06:56 <jeblair> and i think maybe once we have some more specs approved, we might be able to prioritize the reviews that implement those
19:07:18 <fungi> sounds like a fantastic plan
19:07:26 <jeblair> i don't know how formal we want to be about this, but i'm guessing that at least nominating these and communicating will probably help us a lot
19:07:39 <anteaya> a good start
19:07:41 <clarkb> yup
19:08:03 <fungi> part of the problem with review backlog is lack of explicit prioritization. i hope this helps as much as i think it will
19:08:56 <jeblair> so anyway, if i've missed something feel free to contact me and i'll consider it.  but i also want to keep the count fairly small and manageable
19:09:03 * krtaylor imagines a weighted review assignment to produce a prioritized list
19:09:50 <jeblair> krtaylor: that would be great; we've imagined that for a few years but need a feature in gerrit to store the weight/priority/whatever
19:10:14 <jeblair> krtaylor: maybe if the ci report plugin works out, that could be the next step
19:10:28 <krtaylor> ++
19:10:43 <jeblair> #topic  email for gerrit third party ci accounts status (anteaya)
19:10:50 <anteaya> hello
19:10:51 <fungi> but add to that wishlist feature, transitive weighting (association with a spec grants similar priority value)
19:11:17 <anteaya> so we used to require new gerrit accounts include the email for the gerrit account in the request
19:11:29 <anteaya> we have now changed that requirement
19:11:33 <anteaya> #link http://ci.openstack.org/third_party.html#requirements
19:11:51 <anteaya> and now require third party gerrit ci accounts to have a wikipage with contact info on it
19:11:55 <pleia2> \o/
19:11:59 <anteaya> so the question is, how do we transition?
19:12:30 <jeblair> anteaya: maybe you mean http://ci.openstack.org/third_party.html#requesting-a-service-account  ?
19:12:37 <fungi> i think the higher-order question which has been raised is first, do we transition (or keep contact info in both places)
19:12:40 <anteaya> likely I do
19:12:56 <jeblair> so, i was ambivalent about this, until on friday when i disabled 4 accounts
19:13:09 <anteaya> how do you feel now, jeblair?
19:13:18 <jeblair> i wanted to send them mail, so i used the info in gerrit to get their addresses to cc them
19:13:24 <jeblair> i would not have clicked through to the wiki to do that
19:13:34 <krtaylor> and, only about 30 have created wiki pages so far
19:13:41 <jeblair> krtaylor: wow that's a lot :)
19:13:47 <fungi> (heh, "only about 30")
19:13:57 <clarkb> ya I am partial to having the info in gerrit too
19:14:07 <krtaylor> hehheh, well, I guess its a matter of how you look at it  :)
19:14:13 <jeblair> so i definitely still want them in the wiki, because i think all that info should be easily available for people wanting to learn about a system
19:14:19 <anteaya> having the email in gerrit means that mulitple accounts need separate email addresses
19:14:29 <fungi> while i like having that info in gerrit, there are two complications which we've been living with so far but are pretty inconvenient
19:14:34 <fungi> right, that's one
19:14:41 <fungi> it's hard to explain to people
19:14:57 <anteaya> the need for separate emails is hard to explain?
19:15:03 <fungi> so they get it wrong and we iterate over new account requests several times
19:15:07 <jeblair> but i think that if we expect that when we perform administrative actions on those accounts that we cc them, then we should keep them in gerrit
19:15:24 <jeblair> if we don't want to expect that, then i'm okay dropping them :)
19:15:25 <clarkb> fungi: anteaya: what was the other?
19:15:33 <clarkb> I am familiar with the email dances that have happened
19:15:33 <fungi> surprisingly, yes. also because they often don't follow instructions and create an account or two through lp openid first with the same address
19:15:43 <anteaya> we need to be able to email them efficiently
19:15:46 <fungi> second annoyance is that they can't change it. they have to ask us to change it
19:16:04 <clarkb> fungi: gotcha
19:16:17 <anteaya> fungi: they don't follow instructions - describes newcomers to this group very well
19:16:38 <jeblair> so the issue about having multiple accounts with the same address is mostly related to 'duplicate' lp accounts?
19:16:41 <fungi> so i think if we do keep it in as a requirement, we need some better way of explaining it such that they're more likely to get it right the first time, and we need to also drive home that they should do everything in their power to not change what that address is
19:16:46 <jeblair> rather than multiple ci systems operated by the same people?
19:16:50 <anteaya> and them not following instructions and what to do about it deserves its own agenda item, in my mind
19:17:03 <anteaya> since it compounds so many other actions
19:17:20 <fungi> well, depends on what you count as "duplicate" because also sometimes they try to provide their personal e-mail address as contact info even though they already are a reviewer in gerrit with an account associated with that address
19:18:13 <fungi> and basically these don't come to light until we try to add the account and gerrit barfs
19:18:44 <fungi> maybe explicit instructions explaining how to check that the address is not already in gerrit first before requesting the account?
19:19:02 <anteaya> do you think they would follow those instructions?
19:19:30 <anteaya> creating better instructions tends to result in just different questions in channel or them falling down in different places
19:19:48 <anteaya> the folks that would follow the new instructions also already follow the old ones
19:20:01 <jeblair> what if we had a third-party-ci-announce list, and asked all 3p operators to subscribe to it, and sent administrative action notifications there
19:20:07 <fungi> i try not to assume anything about what anyone will actually do, just hope we can provide them with the greatest chance of getting it right the first time, understanding that english is often not their first language so reading english instructions may be hit-and-miss
19:20:13 <anteaya> we can try
19:20:28 <anteaya> I know that others want this back and forth out of their mind space
19:20:36 <anteaya> and I can't tell you how frustrated it makes me
19:20:40 <clarkb> jeblair: thats a good idea
19:20:45 <anteaya> I want to cry sometimes I get so tired
19:20:48 <clarkb> jeblair: or just -infra with a subject?
19:20:50 <krtaylor> well, we could direct them to the third-party meetings to get started too
19:21:16 <anteaya> krtaylor: that redirect usually happens in response to an email
19:21:20 <jeblair> my personal pov is that i'm not particularly concerned if a 3p operator gets the message that their system has been disabled, but i want to give them a reasonable chance of getting it :)
19:21:34 <jeblair> which is why i did direct cc's on friday
19:21:39 <fungi> i definitely appreciate how the third-party ci operator community is starting to come together and provide examples/mentoring to one another. any way we can facilitate increases and improvements in that behavior means less work for infra in the long term
19:21:52 <anteaya> fungi: yes
19:21:56 <anteaya> me too
19:22:15 <jeblair> but a dedicated announce list that is easy to follow should get a fairly similar result, without needing email addresses in gerrit (and it makes it easier for us too)
19:22:29 <fungi> i think that's a great idea
19:22:31 <krtaylor> we are brainstorming how to best police ourselves
19:22:31 <anteaya> let's try
19:22:41 <anteaya> at least it will be different
19:23:04 <jeblair> clarkb: and to your point about -infra, that's kind of anticipating another thing i put on the agenda
19:23:13 <anteaya> krtaylor: yes we are getting to a better place
19:23:14 <krtaylor> I like the maillist idea too, or a way to email blast everyone on the gerrit service group
19:23:17 * wenlock wonders why requesting account isn't a commit request like everything else
19:23:29 <jeblair> " separate mailing list for third-party ci account requests (jeblair) "
19:23:34 <anteaya> wenlock: oh these folks couldn't get that at all
19:23:41 <grantbow> any way to filter requests up front with a lookup for some of these known cases?
19:23:41 <jeblair> wenlock: because we'd have to give a bot admin access :(
19:23:47 <anteaya> wenlock: have you read some of the interactions?
19:23:48 <wenlock> make a form that submits the commit request?
19:24:00 <dougwig> what about a simple web form, which could do things like the duplicate check before accepting the submission?
19:24:01 <dougwig> jinx
19:24:31 <anteaya> well actually if I had more help responding to the existing emails I would feel way more supported
19:24:35 <jeblair> well, that specific thing is not an issue if we're willing to drop the email-in-gerrit requirement
19:24:36 <anteaya> I feel very alone now
19:24:49 <anteaya> I would appreciate help way more than work on a web form
19:24:49 <krtaylor> anteaya, my bad
19:25:02 <anteaya> krtaylor: not at all, you are very supportive
19:25:15 <krtaylor> I am plagued by poor mail reader/filters
19:25:31 <anteaya> krtaylor: let's discuss how to fix that later?
19:26:18 <krtaylor> the mail list would be for infra announces to all third-party service accounts only?
19:26:44 <clarkb> krtaylor: I think jeblair is indicating it would be used for the account requests too
19:27:16 <jeblair> so which way are people leaning?  a) anounce list and drop the gerrit email requirement?  b) drop the email requirement with no mitigating measures?  c) keep the email requirement?  possibly with automation to help make them more correct?
19:27:21 <jeblair> clarkb: i think i have 2 lists in mind now
19:27:22 <fungi> so third-party-ci-announce@lists.openstack.org (or whatever) as one possible action item, maybe a different (or combined) list for related discussion and requests (can wait for that topic in the agenda)?
19:27:31 <clarkb> jeblair: gotcha
19:27:45 <anteaya> jeblair: I'd like to add before we remove anything
19:27:49 <clarkb> jeblair: that works for me too and means 3p operators only need to subscribe to one?
19:28:01 <anteaya> jeblair: to assess the effectivness of whatever we decide to add
19:28:33 <clarkb> I really do like the list idea
19:28:44 <anteaya> I'm fine trying it
19:28:47 <fungi> i like the mailing list with no other contact info collection option if 1. it is well received in the weekly third-party meeting and 2. we continue to be quick with the disable-first-worry-later approach
19:29:06 <jeblair> fungi: i intend to do 2 regardless :)
19:29:20 <anteaya> yay jeblair :D
19:29:28 <anteaya> quick on the trigger
19:29:51 <fungi> yeah. i agree that it's incumbent on the operators to subscribe to an ml and pay attention in case we let them know that whatever account was disabled why and it happens to be the one they maintain
19:30:04 <jeblair> it's best for everyone in the long run if people are not annoyed by 3p ci systems :)
19:30:11 <anteaya> very much so
19:30:11 <fungi> assuming we want to do that (and give them a heads up that this is a requirement in some official documentation)
19:30:18 <krtaylor> ++
19:30:27 <jeblair> so let's segue into that for a bit
19:30:29 <jeblair> #topic  separate mailing list for third-party ci account requests (jeblair)
19:30:33 <anteaya> well I'm usually annoyed so I guess that is everybody else
19:30:56 * anteaya has to stop ranting
19:30:59 <jeblair> so i originally proposed this because i think there are some folks on the -infra list who probably don't care about our third-party account request workflow
19:31:07 <jeblair> like folks from other organizations using zuul, etc
19:31:10 <anteaya> I'd say most
19:31:40 <pleia2> they are like 80% of the non-subscriber emails I let through
19:31:50 <fungi> i worry that moving account requests to a separate ml, because the frequency of them has started to drown out conversation in the existing list, is indicative of other issues and we're going to see substantial scaling challenges if they continue
19:31:54 <pleia2> I don't mind, I can listadmin new lists too, just a thing :)
19:32:05 <anteaya> if they are a non-subscriber, do they get a nudge to encourage them to subscribe?
19:32:20 <pleia2> a nudge can be given, I don't do so right now
19:32:25 <jeblair> so i don't want the list to become a place where people can't actually talk about openstack infrostructure projects :)
19:32:25 <anteaya> fungi: I'd go with yes
19:32:36 <anteaya> pleia2: I'd like to request a nudge please
19:32:44 <anteaya> jeblair: yes, I get that
19:32:54 <anteaya> I'm amazed at how long some threads go on
19:32:56 <pleia2> anteaya: to infra ml, or to this new 3rd party one(s)?
19:33:03 <anteaya> pleia2: both?
19:33:08 <jeblair> fungi: what kind of scaling challenges?
19:33:10 <pleia2> wfm
19:33:14 <anteaya> pleia2: thanks
19:33:25 <fungi> human scaling challenges, working through the incoming requests
19:33:33 <anteaya> thanks fungi
19:33:37 <fungi> unless we make it self-service or something longer term
19:33:38 <clarkb> right there is quite a bit of handholding that seems to be done
19:33:43 <anteaya> yes
19:33:54 <anteaya> this group doesn't seem capable of self-serve
19:33:58 <clarkb> fungi: even then I am not sure that it would remove the scaling issue. It would just help with the first part of it
19:34:06 <anteaya> the ones that do get it are already openstack contributors
19:34:18 <krtaylor> anteaya, that is the key
19:34:24 <fungi> anyway, it's tangential to the need for it to stop impacting the usability of the existing list, so i'm in favor of the split-out. could be the same list as the third-party-ci announcements if we want
19:34:25 <anteaya> yup
19:34:40 <anteaya> doesn't matter to me either way
19:34:52 <anteaya> two new lists, one new list *shrug*
19:35:15 <krtaylor> the idea for patchset for creating an account isnt a bad one, it helps fix the contributor problem
19:35:20 <anteaya> any change will have problems in the transition, the end goal just has to be clear for me
19:35:43 <jeblair> so i think the proposal is:
19:35:48 <anteaya> krtaylor: ah, except the panic for getting an email right becomes a panic for submitting a patch
19:36:02 <anteaya> they don't have to set up accounts or sign a cla to send an email
19:36:04 <fungi> "steps to request a new account: #1 subscribe to the ml. #2 check these assumptions are valid (list of assumptions). #3 post a request in the following format..."
19:36:36 <pleia2> fungi: +1
19:36:46 <jeblair> a low-volume announce list that we require all third-party operators to subscribe to; only infra has unmoderated posting access.  AND...
19:37:12 <jeblair> a third-party ci admins list where new account and account maintenance requests are sent to
19:37:15 <clarkb> what if (and this is just me brainstorming things) we force people to use lp/openid for these accounts
19:37:17 <pleia2> which list do their own outage alerts go to?
19:37:21 <clarkb> then the request is simply for group membership?
19:37:48 <jeblair> the third-party community would be welcome to participate in the second list to help new systems work through the process
19:37:55 <fungi> clarkb: well, right now we don't allow web interface access for those accounts. it would be a paradigm shift
19:38:05 <clarkb> fungi: right, but one that would mostly allow self service
19:38:06 * anteaya notes that at krtaylor's request she is searching for a new word that means admins but isn't admins, she hasn't found one yet
19:38:15 <krtaylor> hehheh
19:38:28 <jeblair> pleia2: why do they send out outage alerts?
19:38:30 <fungi> i think announcements list posts should also have the reply-to set to the discussion list of we go dual-list
19:39:22 <fungi> anteaya: i've called them operators. no idea if krtaylor sees a stigma with that term
19:39:33 <anteaya> krtaylor: ?
19:39:35 <pleia2> jeblair: some send them to -dev and get in trouble
19:39:37 <jeblair> fungi: i think infra is the discussion list
19:39:41 <fungi> i also call myself a systems administrator, so i'm not entirely sure i understand teh concern
19:39:50 <krtaylor> not a big deal really
19:39:56 <pleia2> so it seems some do want to announce outage alerts, but maybe they shouldn't?
19:39:57 <fungi> jeblair: fair enough. we could set follow up to -infra in the headers then
19:39:59 <anteaya> I'm fine with operators
19:40:32 <jeblair> pleia2: if dev doesn't want announcements of third-party ci sytem outages, then perhaps dev doesn't want their third-party ci system at all and they can just close up shop and save everyone some work.  :)
19:40:42 <pleia2> fair enough
19:40:45 <anteaya> pleia2: we addressed the need to announce outage alerts with the wikipage
19:41:07 <jeblair> but i _definitely_ don't want to see them; i think the only potential audience is in -dev.
19:41:26 <jeblair> (i only care about systems that are posting, not systems that aren't)
19:41:38 <fungi> i guess it's reasonable for the third-party operators to update the wiki systems list with status information when they have/resolve an outage
19:41:47 <pleia2> anteaya: ok, guess I missed that, sorry
19:42:00 <fungi> as long as all the projects relying on input from these know where to look when they notice the absence of an expected vote on a change
19:42:03 <anteaya> pleia2: np
19:42:07 <anteaya> #link https://wiki.openstack.org/wiki/ThirdPartySystems
19:42:18 <jeblair> so let me draft some 'agreed' statements and you can tell me if we're really agreed
19:42:19 <anteaya> we having pointed to it yet on the ml
19:42:34 <anteaya> fungi: yes
19:42:41 <grantbow> anteaya: thanks for the link
19:42:57 <anteaya> also they need to announce status so their overall status in the project is uptodate
19:43:03 <anteaya> as in they are communicating
19:43:15 <anteaya> so they won't have their driver pulled from master
19:43:43 <anteaya> so I get the need, we just need an agreed upon, used, non-spammy way to do it
19:43:56 <anteaya> grantbow: welcome
19:44:00 <krtaylor> anteaya, we should put status on the individual template too
19:44:20 <anteaya> krtaylor: I leave that magic to you, I am not opposed
19:44:34 <jeblair> agreed create third-party announce ml which is low-volume, infra -> third-party broadcast only, expect all third-party admins to subscribe
19:44:34 <jeblair> agreed create third-party request ml which is open subscription where third-party systems sent account requests
19:44:34 <jeblair> agreed third-party accounts no longer need to provide gerrit email addresses
19:44:34 <jeblair> agreed third-party system status goes in wiki page
19:44:38 <fungi> agreed subscription to third party announcements list required of all operators, agreed separate third party account request mailing list where other operators are encouraged to help newcomers through the process
19:44:42 <jeblair> do those all look right? ^
19:44:46 * krtaylor wishes he was a better magician
19:44:47 <krtaylor> yes
19:44:58 <clarkb> jeblair: yup
19:45:07 <anteaya> jeblair: yes
19:45:13 <fungi> jeblair: in favor, aye to all four
19:45:19 <anteaya> what does open subscription mean?
19:45:29 <clarkb> anteaya: it means anyone can join. like the -dev list
19:45:36 <anteaya> clarkb: k, thanks
19:45:39 <jeblair> #agreed create third-party announce ml which is low-volume, infra -> third-party broadcast only, expect all third-party admins to subscribe
19:45:40 <jeblair> #agreed create third-party request ml which is open subscription where third-party systems sent account requests
19:45:40 <jeblair> #agreed third-party accounts no longer need to provide gerrit email addresses
19:45:40 <jeblair> #agreed third-party system status goes in wiki page
19:45:41 <fungi> anteaya: anyone who joins can post to the list
19:45:53 <anteaya> fungi: thanks
19:46:05 <jeblair> clarkb: to your point about lp accounts and group membership.  it's tempting, but i'm worried that they will be used the wrong way if they can be... but i think it merits further thought.
19:46:07 <anteaya> who is the ml creator?
19:46:12 <anteaya> pleia2: is that you?
19:46:22 <clarkb> jeblair: ya I think that is the biggest concern. people may use them to push code and other things
19:46:34 <clarkb> jeblair: so we may have to figure out gerrit side acls that make thigns safish
19:46:34 <pleia2> anteaya: there is a file in config you update
19:46:39 <anteaya> if there is an unimaginable wrong way to do something, this group will find it
19:46:43 <jeblair> clarkb: maybe... maybe with sufficient acls we could get that to work.
19:46:47 <anteaya> they should be quality control
19:46:49 <fungi> jeblair: clarkb: worth more thought. maybe we can restrict the access currently granted to normal users if they're in the systems group or whatever
19:46:54 <fungi> er, that
19:47:05 <anteaya> who wants to create the patch?
19:47:21 <anteaya> I'm still conferencing so I am a bad choice
19:47:26 <anteaya> I would otherwise
19:47:28 <anteaya> :(
19:47:34 <pleia2> it's an easy enough patch, I can do it
19:47:36 <fungi> i am happy to submit the change to create the mailing lists
19:47:41 <fungi> or pleia2 can to it!
19:47:50 * fungi quickly backs away from the keyboard
19:47:51 <jeblair> fungi, clarkb: i think we should proceed with what we've got (the only thing that may change is we may drop the request list), and plan out the other idea further
19:47:55 <grantbow> lol
19:48:03 <jeblair> #action pleia2 create new mailing lists
19:48:06 <anteaya> pleia2: thanks, link me for the review
19:48:06 <clarkb> jeblair: sounds good
19:48:16 <clarkb> jeblair: ya I think my brainstorming alternative will need more time and thought
19:48:24 <clarkb> whereas the list stuff is cheap and quick
19:48:26 <clarkb> good for now
19:48:32 <jeblair> oh my, this meeting is almost all third-party this week
19:48:38 <jeblair> #topic  comment syntax for third-party CI recheck (dougwig, from third-party CI meeting)
19:48:47 <anteaya> we are a rowdy bunch
19:48:56 <anteaya> do dougwig had to jet at half past
19:49:06 <fungi> it's almost like the third-party operators need their own weekly meeting. oh, wait, they have one!
19:49:12 <jeblair> whoopsie; if people mention that in the agenda, i can accomodate
19:49:17 <anteaya> the upshot is that third party doesn't want to break things like vmware did last week
19:49:37 <anteaya> so how should we recheck for now, until we can agree to how we should recheck long term?
19:49:38 <fungi> this rolls into the next topic, so maybe we just cover both as one discussion
19:49:51 <anteaya> I have told them manual recheck is acceptable, not generated scripts
19:50:07 <krtaylor> pleia2, can you tag the topic of the maillist patch third-party?
19:50:10 <anteaya> what else should they know to safely recheck?
19:50:17 <jeblair> oh yeah, it's never okay to mass-recheck by leaving comments
19:50:24 <pleia2> krtaylor: sure
19:50:24 <fungi> oh, how should they do mass rechecks? they should trigger them in whatever way their systems allow without leaving comments in gerrit to do so
19:50:25 <jeblair> neither jenkins nor zuul requires that
19:50:50 <anteaya> jeblair: yes, we covered not mass-rechecking in yesterday's meeting
19:50:55 <dougwig> here's my topic intro: "hi there.  A topic came up in the third-party CI meeting, which was to refrain from using recheck.* as a prefix for triggering re-checks in 3rd party CI's, as it causes standard jenkins to also re-run.  What syntax/prefix should we use/standardize for retriggering a single third-party CI?"
19:50:59 <dougwig> (i'm not really here)
19:51:00 <anteaya> which motivated the current agenda item
19:51:05 <anteaya> dougwig: yay
19:51:12 <jeblair> #link https://review.openstack.org/#/c/109565/
19:51:21 <jeblair> so there's a change that proposes a syntax.
19:51:26 <jeblair> it has nothing but red votes
19:51:30 <anteaya> yes
19:51:36 <anteaya> so there is a conversation
19:51:48 <anteaya> they just want to know what is safe which the conversation takes place
19:52:02 <anteaya> so they aren't blocked until resolution of the conversation
19:52:07 <jeblair> i think some people can't agree on the syntax, and i think some people (myself and fungi included) don't agree that third-party ci specific recheck comments should be used at all
19:52:12 <anteaya> or is there a safe way
19:52:23 <fungi> there are also a bug and an infra ml thread on this topic
19:52:27 <jeblair> anteaya: what is the safe way to do a third-party specific recheck comment?
19:52:39 <anteaya> yes, that is the question
19:52:43 <anteaya> I do believe
19:52:49 <fungi> #link https://launchpad.net/bugs/1355480
19:52:50 <uvirtbot> Launchpad bug 1355480 in openstack-ci "recheck/reverify comment_filter is too loose" [Undecided,Opinion]
19:52:54 <jeblair> anteaya: there is no safe way, it's not supported or recommended
19:53:00 <anteaya> hmmmm
19:53:00 <clarkb> I agree with jeblair and fungi
19:53:04 <clarkb> its the wild west we don't enforce it
19:53:05 <fungi> #link http://lists.openstack.org/pipermail/openstack-infra/2014-August/001681.html
19:53:07 <clarkb> there is no "safe"
19:53:25 <anteaya> okay well this places those operators wanting to play by the rules in a difficult spot
19:53:34 <anteaya> I don't know what to take back to them
19:53:51 <anteaya> I've worked so hard to get them to ask
19:54:07 <anteaya> I would be really disapointed if I went back with "do whatever you want"
19:54:14 <fungi> the rules say they should rerun their checks when someone leave a "recheck bug [0-9]+" or "recheck no bug" comment
19:54:25 <jeblair> anteaya: i don't think i'd say that.  i think i'd say what fungi said.
19:54:26 <fungi> at least in present documentation
19:54:45 <dougwig> i mean, something like "thirdparty-recheck-a10", or anything that will never match your jenkins, is all we want.
19:55:04 <fungi> dougwig: i'm unconvinced there's value in rerunning only one ci system
19:55:04 <jeblair> i believe we attempted to have a stronger indication that nothing else should be supported, but that did not gain unanimous approval.  thus the 3p ci operates in an undefined area if they do anything more.
19:55:06 <dougwig> we can make it up; wasn't sure if there should be a standard.
19:55:14 <anteaya> is dougwig's suggestion safe?
19:55:19 <dougwig> fungi: retrying false negatives. we are not all as perfect as infra.  :)
19:55:24 <fungi> it's not like we allow developers to only rerun the jobs which fail and leave the passing results for other jobs. it should eb all-or-nothing
19:55:32 <jeblair> yeah, everyone wants to make some super complicated language for controlling ci systems in comments
19:55:50 <jeblair> that's an INSANE idea, and the only reason we do it at all is because our tooling is not good enough to support something else
19:55:59 <jeblair> i mean, it's a horrible user interface for developers
19:56:00 <dougwig> we ask people to start commenting before they vote, to shake out bugs.  not every failure is a fault in the submitted code (many are not for 3p)
19:56:04 <jeblair> and we should minimize it's use totally
19:56:13 <anteaya> I think they are just trying their best to not step on infra's toes and allow functionality they percieve as necessary
19:56:37 <jeblair> i don't think we'd be upset if someone says "recheck".  i mean right now we re-run jobs all the time for no particular reason :)
19:56:53 <asselin_> the issue with recheck bug # is that all ci systems will start over again. As there are more and more, the probability of all of them passing gets smaller, especially when there are intermittent bugs.
19:57:05 <anteaya> we just don't want to create a scenario like vmware did that brought down zuul last week
19:57:16 <jeblair> anteaya: these are two separate issues
19:57:21 <anteaya> jeblair: yes
19:57:22 <clarkb> anteaya: a different syntax doesn't solve that problem
19:57:27 <anteaya> clarkb: oh
19:57:43 <anteaya> we are almost at time and I feel we are at a stalemate for now
19:57:45 <clarkb> anteaya: at least as long as clean check si a thing any comment can trigger CI
19:57:53 <fungi> anteaya: the main problem i saw is that a bot went and left hundreds of useless comments on changes
19:57:57 <jeblair> anteaya: we will disable any account for spamming regardless of what syntax they use
19:58:00 <anteaya> so no bots
19:58:09 <anteaya> jeblair: true
19:58:22 <fungi> not no bots. just bots should only leave comments which contain useful information for reviewers
19:58:23 <dougwig> what i'm hearing is that you think the mechanism is a bad idea in general.  i think that's overly optimistic in terms of the quality of all of these disparate CI systems.
19:58:25 <anteaya> I think we covered not bots for comments, but I can cover that again
19:58:38 <anteaya> fungi: okay
19:58:49 <dougwig> in an ideal CI world, it would not be needed.  the world is not currently ideal.  or even close.
19:58:53 <fungi> (useful and nonredundant)
19:59:12 <anteaya> fungi: k
19:59:24 <jeblair> dougwig: there's an opportunity in https://review.openstack.org/#/c/109565/
19:59:38 <fungi> i'm also worried that we risk sending a signal that it's okay for things to work poorly, and we should give up bothering to improve that
19:59:47 <jeblair> dougwig: right now, no one likes that change at all.  perhaps if even a handful of people could agree on what would be a good idea, it will go somewhere.
19:59:56 <anteaya> I'm all for improving our signalling
20:00:18 <anteaya> and we have no more time
20:00:24 <jeblair> or maybe someone should add authentication to zuul's webui so recheck requests don't have to go in comments :)
20:00:31 <jeblair> that would be a far better use of all of our time, i think.
20:00:34 <jeblair> thanks everyone!
20:00:35 <dougwig> not specifying a syntax will result in every 3p system making up its own recheck trigger.  i think the idealist case won't fly in reality.
20:00:37 <jeblair> #endmeeting