| *** GonZo2000 has joined #zuul | 00:00 | |
| *** GonZo2000 has quit IRC | 00:00 | |
| *** GonZo2000 has joined #zuul | 00:00 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: Fix deprecation warning of ansible-2.5 https://review.openstack.org/571996 | 00:18 |
|---|---|---|
| *** TheAceOfHearts has quit IRC | 00:40 | |
| *** GonZo2000 has quit IRC | 00:52 | |
| *** GonZo2000 has joined #zuul | 00:57 | |
| *** D3VIATION has quit IRC | 01:07 | |
| *** xinliang has quit IRC | 01:14 | |
| tristanC | mordred: would you also happen to have a wip for the zuul_stream refactor (e.g.: using ssh port forward) on your laptop ? :) | 01:21 |
| tristanC | this would be useful to have for running parallel job on a single host. Alternatively i can rebase these log_stream executor settings: https://review.openstack.org/535538 | 01:24 |
| *** pwhalen_ has quit IRC | 01:27 | |
| *** xinliang has joined #zuul | 01:30 | |
| *** xinliang has quit IRC | 01:30 | |
| *** xinliang has joined #zuul | 01:30 | |
| *** weshay has quit IRC | 01:30 | |
| *** weshay has joined #zuul | 01:36 | |
| *** TheAceOfHearts has joined #zuul | 01:50 | |
| *** TheAceO__ has joined #zuul | 01:58 | |
| *** TheAceOfHearts has quit IRC | 02:00 | |
| *** TheAceOfHearts has joined #zuul | 02:31 | |
| *** TheAceO__ has quit IRC | 02:34 | |
| *** TheAceOf_ has joined #zuul | 02:48 | |
| *** TheAceOfHearts has quit IRC | 02:51 | |
| *** TheAceOf_ has quit IRC | 02:57 | |
| *** TheAceOfHearts has joined #zuul | 02:58 | |
| *** TheAceOfHearts has quit IRC | 03:04 | |
| *** EmilienM has quit IRC | 03:22 | |
| *** EmilienM has joined #zuul | 03:23 | |
| *** D3VIATION has joined #zuul | 04:07 | |
| *** D3VIATION has quit IRC | 04:11 | |
| *** gtema has joined #zuul | 06:32 | |
| *** ssbarnea has quit IRC | 06:43 | |
| *** ssbarnea has joined #zuul | 06:44 | |
| *** AJaeger has joined #zuul | 07:00 | |
| *** sshnaidm has joined #zuul | 07:10 | |
| *** Rohaan has joined #zuul | 07:10 | |
| *** pcaruana has joined #zuul | 07:12 | |
| *** pcaruana is now known as pcaruana|worksho | 07:14 | |
| *** Rohaan___ has joined #zuul | 07:17 | |
| *** Rohaan has quit IRC | 07:17 | |
| *** yolanda_ has joined #zuul | 07:20 | |
| openstackgerrit | Artem Goncharov proposed openstack-infra/nodepool master: Add support for specifying security_group in nodepool https://review.openstack.org/570857 | 07:23 |
| *** yolanda has quit IRC | 07:23 | |
| *** Wei_Liu has joined #zuul | 07:27 | |
| *** sshnaidm is now known as sshnaidm|brq | 07:38 | |
| *** jpena|off is now known as jpena | 07:53 | |
| tristanC | zuul.openstack.org didn't restarted to pick the cherrypy change? | 07:54 |
| *** hashar has joined #zuul | 07:56 | |
| *** Rohaan___ has quit IRC | 07:58 | |
| gtema | faced couple of times situation, that DIB is failing to build an image and then do a proper cleanup (latest example switched SELinux on host in, but in the images semanage was not present). DIB fails to cleanup (target busy). Nodepool-builder retries permanently to build new image, unless disk is full. What about adding basic enough storage check to the nodepool-builder? | 08:13 |
| gtema | DIB should be also fixed - it's clear. But I think nodepool-builder should not continuosly retry flooding the disk | 08:14 |
| *** Rohaan___ has joined #zuul | 08:26 | |
| *** Rohaan___ has quit IRC | 08:30 | |
| *** GonZo2000 has quit IRC | 08:39 | |
| *** GonZo2000 has joined #zuul | 08:40 | |
| *** GonZo2000 has quit IRC | 09:23 | |
| *** GonZo2000 has joined #zuul | 09:25 | |
| *** GonZo2000 has quit IRC | 09:35 | |
| *** GonZo2000 has joined #zuul | 09:48 | |
| *** GonZo2000 has quit IRC | 09:48 | |
| *** GonZo2000 has joined #zuul | 09:48 | |
| *** pleia2_ has joined #zuul | 10:15 | |
| *** GonZo2000 has quit IRC | 10:16 | |
| *** gouthamr_ has joined #zuul | 10:20 | |
| *** gouthamr has quit IRC | 10:20 | |
| *** pleia2 has quit IRC | 10:20 | |
| *** Rohaan has joined #zuul | 10:27 | |
| *** jpena is now known as jpena|lunch | 11:09 | |
| *** pwhalen has joined #zuul | 11:33 | |
| *** pwhalen has joined #zuul | 11:33 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Remove getRepoPermission from FakeGithubConnection https://review.openstack.org/570428 | 12:22 |
| *** openstackgerrit has quit IRC | 12:34 | |
| *** rlandy has joined #zuul | 12:34 | |
| *** rlandy is now known as rlandy|rover | 12:35 | |
| *** openstackgerrit has joined #zuul | 12:44 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Status branch protection checking for github https://review.openstack.org/535680 | 12:44 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Remove getRepoPermission from FakeGithubConnection https://review.openstack.org/570428 | 12:44 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Status branch protection checking for github https://review.openstack.org/535680 | 13:13 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Remove getRepoPermission from FakeGithubConnection https://review.openstack.org/570428 | 13:13 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Status branch protection checking for github https://review.openstack.org/535680 | 13:27 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Remove getRepoPermission from FakeGithubConnection https://review.openstack.org/570428 | 13:27 |
| *** gtema has quit IRC | 13:32 | |
| *** jpena|lunch is now known as jpena | 13:42 | |
| *** jimi|ansible has joined #zuul | 13:43 | |
| *** jimi|ansible has joined #zuul | 13:43 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Status branch protection checking for github https://review.openstack.org/535680 | 13:47 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Remove getRepoPermission from FakeGithubConnection https://review.openstack.org/570428 | 13:47 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add supercedent pipeline manager https://review.openstack.org/571932 | 13:55 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Fix link in mqtt reno https://review.openstack.org/572110 | 13:55 |
| *** Rohaan___ has joined #zuul | 14:00 | |
| *** Rohaan has quit IRC | 14:01 | |
| * Shrews getting a late start after a very long weekend, but will attempt to catch up on all-the-things | 14:03 | |
| corvus | Shrews, tobiash: i have lost all context for this -- are we at the point where we should just decide to do the 2.5 upgrade on, say, wednesday or thursday this week and stand by to handle fallout by quickly merging changes to zuul-jobs to fix anything that's broken? | 14:06 |
| tobiash | corvus: I think so | 14:06 |
| tobiash | we have it in production since before the summit without problems | 14:06 |
| Shrews | corvus: i believe that was the decision we had come to, previously | 14:07 |
| tobiash | the only problem we had was the breakage of shell tasks which has been fixed | 14:07 |
| corvus | anyone have a preference for wednesday or thursday? | 14:08 |
| jhesketh | perhaps a silly question (and certainly one I should know the answer to) but does zuul ever push into gerrit? It only does a 'submit' review, right, and gerrit re-performs the merge action? | 14:08 |
| tobiash | jhesketh: yes, it presses the merge-button via api | 14:08 |
| tobiash | gerrit re-performs the merge action | 14:08 |
| corvus | jhesketh: correct; we may very well push to gerrit later (that would be required for bidirectional cross-repo-dependency, and also make unidirectional more reliable) | 14:08 |
| jhesketh | right, that was my understanding :-) | 14:09 |
| Shrews | corvus: either of those days wfm | 14:09 |
| jhesketh | so there's a risk that the merge-commit zuul makes is slightly different to gerrits | 14:09 |
| tobiash | (and is currently not possible as a github app) | 14:09 |
| corvus | jhesketh: correct | 14:09 |
| tobiash | corvus: I'm ok with both | 14:09 |
| jhesketh | I've never looked at ths SHA's, but do they compute the same? | 14:09 |
| tobiash | probably not | 14:09 |
| corvus | right, timestamps are different | 14:10 |
| tristanC | corvus: that would requires a gerrit acl change, atm zuul ci doesn't need push permission | 14:10 |
| corvus | tristanC: yep. but it's possible with gerrit, and gerrit will do the right thing with the change (close it) | 14:10 |
| jhesketh | ah, of course | 14:10 |
| jhesketh | thanks :-) | 14:10 |
| corvus | github will also dtrt with the pr, if there were a way to give it perms :/ | 14:11 |
| jhesketh | oh neat | 14:12 |
| *** TheJulia is now known as needsleep | 14:18 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add supercedent pipeline manager https://review.openstack.org/571932 | 14:22 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Fix link in mqtt reno https://review.openstack.org/572110 | 14:23 |
| tobiash | this is the blocking part in github: https://platform.github.community/t/repositories-which-have-protected-branches-with-push-restrictions-have-no-ability-to-grant-push-rights-to-integrations/1376/45 | 14:25 |
| *** hashar is now known as hasharAway | 14:26 | |
| tobiash | but there is also a comment there from github stuff that they're working on that | 14:26 |
| *** Guest57137 is now known as sc68cal | 14:36 | |
| *** sc68cal is now known as Guest86524 | 14:36 | |
| *** D3VIATION has joined #zuul | 14:39 | |
| corvus | pabelanger: i'll abandon your aiohttp version fixes since cherrypy landed | 14:45 |
| Shrews | ooh, that landed? neat | 14:52 |
| corvus | Shrews: yeah, i figured out the test race at the end of last week. we haven't restarted openstack with it yet... i should probably do that today :) | 14:53 |
| *** needsleep is now known as TheJulia | 14:56 | |
| *** Guest86524 has quit IRC | 14:56 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Add driver interface documentation https://review.openstack.org/571538 | 15:01 |
| *** sshnaidm|brq has quit IRC | 15:10 | |
| *** Rohaan___ has quit IRC | 15:12 | |
| clarkb | corvus: I have to take kids to the dentist wednesday so thursday is easier for me from a handling fallout perspective | 15:28 |
| clarkb | corvus: ++ to restarting zuul with cherrypy updates along with the other bug fixes and chagnes that merged late last week | 15:29 |
| hughsaunders | Hey, does nodepool support sending cloud-init data when creating instances? I see it has a boolean for enabling config drive: https://github.com/openstack-infra/nodepool/blob/master/nodepool/driver/openstack/handler.py#L108-L121 but where would the config drive data come from if its enabled? | 15:31 |
| clarkb | hughsaunders: it currently only supports image metadata (which maybe is exposed via config drive/meta data server) and ssh keys metadata. I think there is a need for adding general metadata | 15:32 |
| hughsaunders | clarkb: To confirm, the current state is that nodepool can't supply cloud init data when launching an openstack instance? | 15:34 |
| clarkb | hughsaunders: no, it can supply a subset of the data | 15:34 |
| *** pcaruana|worksho has quit IRC | 15:35 | |
| clarkb | double checking I don't see expected image metadata in the instance metadata contents on rax so that probably doesn't make it through (at least not in all cases). That means the ssh keys metadata is the only metadata passed through I think | 15:38 |
| *** pleia2_ is now known as pleia2 | 15:39 | |
| hughsaunders | clarkb: Thanks, another question... so is there any way I can get a nodepool-launcher to run a script on the node once its up? | 15:39 |
| hughsaunders | I was thinking cloud-init may be a way to achieve that. | 15:39 |
| hughsaunders | I know DIB us the usual solution, but I want to use OnMetal nodes that I can't image. | 15:40 |
| clarkb | hughsaunders: we dropped that feature from nodepool with the idea being your workload generator (like zuul) should do that instead since that is what they are good at | 15:40 |
| clarkb | hughsaunders: rax OnMetal accepts qcow2 images I thought | 15:40 |
| hughsaunders | oooh, I better check that then. | 15:41 |
| tobiash | Shrews, tristanC: we observed a race within the static node driver which resulted in static nodes used by several jobs at the same time | 15:51 |
| tobiash | we think _checkConcurrency is the source of this (http://git.openstack.org/cgit/openstack-infra/nodepool/tree/nodepool/driver/static/handler.py#n27) | 15:52 |
| tobiash | Shrews: I think this may be fixed once your refactor and pre-registration have landed | 15:54 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Directly link providers and request handlers https://review.openstack.org/571584 | 15:55 |
| Shrews | tobiash: i'm finding that the driver needs to be totally rewritten to support pre-reg | 15:56 |
| Shrews | frustratingly | 15:56 |
| tobiash | hrm | 15:56 |
| tobiash | Shrews: let me know if I can help with that | 15:57 |
| Shrews | tobiash: thx | 15:58 |
| Shrews | i had to put it out of my head for a few days. will get back into it after some of these driver api changes | 15:59 |
| tobiash | I think without pre-reg we would need a zk-lock for the provider to make that race free | 16:00 |
| pabelanger | can https://zuul-ci.org/docs/zuul/user/config.html#attr-job.roles.zuul accept multiple connection name to a project? eg: zuul: git.openstack.org/openstack-infra/openstack-zuul-jobs and zuul: git.zuul.org/zuul ? | 16:02 |
| pabelanger | err | 16:03 |
| pabelanger | zuul: git.zuul.org/zuul-jobs | 16:03 |
| clarkb | pabelanger: I think if they have different connections and thus have different canonical names it would work | 16:04 |
| pabelanger | clarkb: okay great, I'll update based on testing | 16:05 |
| tobiash | Shrews, corvus: I posted a question on 571584 | 16:09 |
| Shrews | tobiash: yes, that was just for QuotaInfo. If you want to move that in a follow up, feel free | 16:10 |
| tobiash | k, maybe tomorrow | 16:11 |
| clarkb | tobiash: re your comment on https://review.openstack.org/#/c/567227/1 do you think we should do both things? fix it as best as possible in the merge call so that most of the time we don't have problems then address any specific issues where they show up too ? | 16:11 |
| tobiash | clarkb: I think merges in most cases don't harm git fetch (.gitmodules is an exception) | 16:12 |
| clarkb | tobiash: ah | 16:13 |
| tobiash | clarkb: and as long as we need to cope with this anyway I tend to handle this only when some operation fails | 16:13 |
| clarkb | ya that makes sense since it is a remote fetch source config file | 16:13 |
| dmsimard | jlk: so I guess we need to get Zuul into Microsoft now :p | 16:15 |
| jlk | I'm afraid of the words mordred hasn't spoken | 16:15 |
| corvus | well, actually, we chatted with the kata folks at the summit, and they have some azure they'd like to use with zuul | 16:16 |
| *** sshnaidm|brq has joined #zuul | 16:16 | |
| tobiash | windows gated by zuul sounds somewhat cool ;) | 16:17 |
| corvus | so shortly, we plan on borrowing their account to help finish and test tristanC's azure driver for nodepool :) | 16:17 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul master: zuul-migrate: run / post-run playbooks need .yaml extensions https://review.openstack.org/572160 | 16:17 |
| pabelanger | anybody see reportportal.io before? Just came across my inbox this morning, looks interesting. Tagged as AI-powered Test Automation Dashboard | 16:19 |
| jlk | corvus: NEAT | 16:21 |
| jlk | I... may have reasons to want a nodepool to Azure like tool | 16:21 |
| Shrews | jlk: i'm afraid of the reasons you haven't spoken | 16:26 |
| fungi | i'm getting questions from events folks at osf about ansiblefest in austin this october... anyone planning to submit talks/trainings to the cfp? | 16:26 |
| jlk | Shrews: hehe, I'm as surprised by the announcement as the rest of you are. I have no insider information. | 16:26 |
| jlk | but I really don't think I have the ... capability to push for how MSFT does CI for their software. | 16:27 |
| *** harlowja has joined #zuul | 16:28 | |
| *** ianychoi has quit IRC | 16:32 | |
| Shrews | tobiash: thx for the nodepool reviews. any reason you're not adding the +W? | 16:36 |
| Shrews | something i need to change? | 16:36 |
| tobiash | Shrews: just wasn't sure if you want to land it at once | 16:36 |
| Shrews | tobiash: i see no reason to wait | 16:37 |
| tobiash | Shrews: ok, will add +w then :) | 16:37 |
| Shrews | w00t. thx | 16:37 |
| openstackgerrit | Merged openstack-infra/zuul master: Gracefully handle broken .gitmodule files https://review.openstack.org/567227 | 16:39 |
| clarkb | tobiash: ^ | 16:39 |
| tobiash | \o/ | 16:39 |
| openstackgerrit | Merged openstack-infra/nodepool master: Fix ConfigValue comparisons https://review.openstack.org/567927 | 16:46 |
| openstackgerrit | Merged openstack-infra/nodepool master: Make imagesAvailable() part of the driver API https://review.openstack.org/568702 | 16:46 |
| openstackgerrit | Merged openstack-infra/nodepool master: Use ProviderConfig iface to validate labels https://review.openstack.org/568703 | 16:48 |
| *** ianychoi has joined #zuul | 16:48 | |
| dmsimard | fungi: I plan on submitting something to AnsibleFest. Not 100% sure what about yet. Likely a mix of ARA and Zuul. | 16:49 |
| *** pcaruana|worksho has joined #zuul | 16:51 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Simplify driver API https://review.openstack.org/568704 | 16:54 |
| *** gouthamr_ is now known as gouthamr | 16:56 | |
| fungi | dmsimard: thanks. i think they're trying to make booth et cetera decisions based on whether there'll be zuul content (and how much) | 16:57 |
| tobiash | Shrews: do we need to make a zuul change together with the multi label change in nodepool? | 16:58 |
| Shrews | tobiash: excellent question. i don't *think* so, but let me inspect the code to make sure | 17:01 |
| *** GonZo2000 has joined #zuul | 17:02 | |
| Shrews | corvus: did you happen to consider that when reviewing the multi-label change? ^^ | 17:04 |
| Shrews | (in case i miss something when reviewing zuul code) | 17:05 |
| corvus | Shrews: i *thought* not, but i'm not sure now. we should double check. | 17:05 |
| corvus | it doesn't look like zuul reads node.type from zk | 17:08 |
| corvus | i believe it assumes that it got what it asked for :) | 17:08 |
| corvus | so i think we're ok | 17:08 |
| tobiash | Shrews, corvus so we can +w it? | 17:09 |
| Shrews | corvus: actually, it seems to use it for some statsd stuff | 17:09 |
| corvus | Shrews: where's that? | 17:09 |
| Shrews | zuul/nodepool.py | 17:10 |
| Shrews | emitStats | 17:10 |
| *** harlowja has quit IRC | 17:10 | |
| Shrews | and i'm not sure how to fix that since it seems to assume Node.label == requested.label | 17:10 |
| Shrews | the zuul model has Node.label, np has Node.type | 17:11 |
| corvus | right, we never finished the conversion of type -> label on the nodepool side | 17:12 |
| corvus | but zuul never reads the 'type' field from zookeeper | 17:12 |
| Shrews | corvus: oh. so the zuul model is actually not what's read from ZK? | 17:13 |
| Shrews | perhaps just used internally | 17:13 |
| corvus | it only writes the labels that it requests to the noderequest znode, then assumes a 1:1 mapping of nodes when the request is fulfilled | 17:13 |
| Shrews | oh. so seems like we're ok then | 17:14 |
| corvus | the zuul node model object does read from zk. | 17:14 |
| corvus | but nothing on the zuul side uses type | 17:14 |
| corvus | label is just a copy of the requested label that was sent | 17:15 |
| Shrews | yep. and seems to store any other values in storeNode() as-is. | 17:15 |
| corvus | so, after we merge this change, we need to be careful if we finish the type -> label conversion on the nodepool side. if we did that, we could end up overwriting zuul's label with the new list-of-labels, *then* we'd be in trouble | 17:16 |
| Shrews | yep | 17:16 |
| *** acozine1 has joined #zuul | 17:16 | |
| Shrews | tobiash: ok, i think that means we are ok on the zuul side | 17:16 |
| corvus | (not insurmountable, just another step) | 17:16 |
| Shrews | corvus: thx, sorry for diverting your attention | 17:17 |
| *** pcaruana|worksho is now known as pcaruana | 17:18 | |
| *** jpena is now known as jpena|off | 17:19 | |
| corvus | np | 17:20 |
| tobiash | Shrews: ok +w'ed that change | 17:25 |
| Shrews | tobiash: thx! | 17:25 |
| tobiash | np | 17:25 |
| openstackgerrit | Merged openstack-infra/zuul master: Status branch protection checking for github https://review.openstack.org/535680 | 17:31 |
| openstackgerrit | Merged openstack-infra/nodepool master: Add multilabel support to ZooKeeper Node model https://review.openstack.org/570227 | 17:31 |
| openstackgerrit | Merged openstack-infra/nodepool master: Add driver interface documentation https://review.openstack.org/571538 | 17:31 |
| openstackgerrit | Merged openstack-infra/nodepool master: Directly link providers and request handlers https://review.openstack.org/571584 | 17:33 |
| corvus | i just restarted zuul-web in openstack-infra and it seems to be working okay (except that our tooling was incorrectly writing the config file with the empty string as the listen address, which is no longer supported) | 17:34 |
| pabelanger | secrets are tenant scoped right? | 17:41 |
| pabelanger | I haven't tested yet, but can shortly | 17:42 |
| clarkb | pabelanger: they are project scoped (so I think that means yes) | 17:44 |
| pabelanger | great, that's what I assumed | 17:44 |
| *** harlowja has joined #zuul | 17:49 | |
| mnaser | the email for the confirmation to join zuul mailing lists went to my spam folder | 17:51 |
| mnaser | :( | 17:51 |
| corvus | mnaser: yay it was delivered! | 18:29 |
| fungi | mnaser: do you have any insight into the spam classification algorithm in use? | 18:29 |
| mnaser | fungi: no :( gsuite/google apps | 18:30 |
| fungi | ahh... i have spamassassin set to inject a list of matching rule ids into a header | 18:30 |
| fungi | hooray for free software? | 18:30 |
| fungi | i never have to ask "why did this end up in my spam box?" | 18:31 |
| fungi | just hit the h key in mutt to examine message headers and it tells me | 18:31 |
| mnaser | fungi: i agree but operating a mail server is quite the hassle :( | 18:39 |
| fungi | fair. my soapbox requires it, i suppose | 18:40 |
| *** myoung is now known as myoung|lunch | 18:43 | |
| mordred | corvus: in the time that I have spent not finishing the angular patches - angular v6 has been released | 18:52 |
| mordred | corvus: in good news though - angular v6 is an LTS release, so we should be good on it for $time | 18:53 |
| corvus | mordred: i really hope ansible 2.6 doesn't release before thursday :) | 18:54 |
| clarkb | ha | 18:54 |
| mordred | corvus: :) | 18:55 |
| mordred | 2018-06-28 Final Release <-- 2.6 expected release date | 18:55 |
| mordred | so we should be good for thursday | 18:55 |
| *** D3VIATION has quit IRC | 18:57 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Stop scheduler more quickly https://review.openstack.org/572186 | 18:57 |
| corvus | oh thank goodness | 18:57 |
| SpamapS | wtf | 18:58 |
| SpamapS | are they like, in "break everything every 2 months" mode? | 18:58 |
| corvus | speaking of break everything -- we suspect a problem with openstack-infra's connection to github with current zuul master | 19:04 |
| corvus | caveat deployer :) | 19:04 |
| *** GonZo2000 has quit IRC | 19:21 | |
| SpamapS | it senses a disturbance in the github force | 19:22 |
| clarkb | jlk: related to ^ we are not seeing events from Monty on https://github.com/ansible/ansible/pull/37170 but do see the recent shipit comment event on https://github.com/ansible/ansible/pull/39877#pullrequestreview-125706973 in the github side events log for our zuul app | 19:23 |
| clarkb | mordred double checked the permissions on the app and seems happy with them also we are getting comments like that shipit comment so it should be fine perms wise | 19:23 |
| clarkb | is it possible github is losing those events are is it pebkac | 19:24 |
| *** harlowja has quit IRC | 19:29 | |
| *** myoung|lunch is now known as myoung | 19:57 | |
| *** harlowja has joined #zuul | 19:59 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add cherrypy to built-in logging config https://review.openstack.org/572200 | 19:59 |
| jlk | hrm. | 20:17 |
| jlk | We don't often lose events | 20:17 |
| jlk | You should be able to see the event in the GitHub application interface, there is a space to see all the events sent out from an app | 20:17 |
| jlk | and you can click to re-send them too | 20:17 |
| clarkb | jlk: ya the events don't show up there | 20:18 |
| clarkb | https://github.com/ansible/ansible/pull/37170#issuecomment-394462095 specifically the event for that comment | 20:18 |
| jlk | the recheck? | 20:19 |
| jlk | (the link doesn't highlight which of the events on that stream :( ) | 20:19 |
| clarkb | ya tge rechecks | 20:21 |
| jlk | that's really bizarre. | 20:31 |
| jlk | I just added my own, do you see the event in the list? | 20:31 |
| clarkb | not yet at least. Most recent event is from 6 minutes ago | 20:32 |
| clarkb | and now there are more recent events | 20:33 |
| clarkb | but not yours | 20:33 |
| jlk | that's super strange. I don't see any internal incidents ongoing either | 20:34 |
| clarkb | public status looks happy too | 20:34 |
| jlk | We started getting some Zuul reporting on github3.py today. | 20:46 |
| clarkb | jlk: have an example I can look at (want to compare timestamps to see if any ran after we updated zuul) | 20:50 |
| jlk | https://github.com/sigmavirus24/github3.py/pull/852#issuecomment-394363768 | 20:51 |
| jlk | which is http://logs.openstack.org/52/852/2d2146647fdb7096c9ad815dc2e52c585c6fd421/third-party-check/tox-py35-on-zuul/810aa6b/ | 20:51 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Fix zuul-web sql connections https://review.openstack.org/572210 | 20:56 |
| *** acozine1 has quit IRC | 20:56 | |
| *** pcaruana has quit IRC | 20:59 | |
| *** hasharAway has quit IRC | 21:02 | |
| *** harlowja has quit IRC | 21:04 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Upgrade from angularjs (v1) to angular (v5) https://review.openstack.org/551989 | 21:14 |
| *** myoung is now known as myoung|off | 21:31 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Fix zuul-web sql connections https://review.openstack.org/572210 | 21:32 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Upgrade from angularjs (v1) to angular (v6) https://review.openstack.org/551989 | 21:35 |
| mordred | tristanC: ^^ that works for the multi-tenant case in local testing | 21:35 |
| *** harlowja has joined #zuul | 21:36 | |
| clarkb | dmsimard: thinking about the ansible 2.5 update (sounds like we are doing that thursady) are there any major gotchas we should be looking at in eg zuul-jobs before doing so | 21:43 |
| clarkb | I think tobiash says that they've got it working at bmw so maybe it will mostly just work | 21:43 |
| dmsimard | Upstream has some docs on that, sec | 21:44 |
| dmsimard | https://docs.ansible.com/ansible/2.5/porting_guides/porting_guide_2.5.html | 21:45 |
| dmsimard | https://docs.ansible.com/ansible/2.5/porting_guides/porting_guide_2.4.html | 21:46 |
| clarkb | thanks | 21:47 |
| dmsimard | There's a lot of new deprecations (which shouldn't break anything but might bring a lot of warning spam) | 21:47 |
| clarkb | ya include will do that | 21:47 |
| dmsimard | I wonder if we could get some elastic-recheck queries ready ahead of time to identify issues quickly | 21:48 |
| clarkb | skimming those two pages I don't see anything that should be too bad other than the deprecations | 21:49 |
| dmsimard | I also never got around to graphing the metrics I added to executor phase results. Would probably be useful. | 21:49 |
| dmsimard | I'll get something going to add to grafana so we have some amount of visibility. | 21:51 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Translate zuulStartStream into typescript https://review.openstack.org/558618 | 21:53 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Shift log streaming code into StreamComponent https://review.openstack.org/558619 | 21:53 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Stop falling back to job name for missing url https://review.openstack.org/554056 | 21:58 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Remove dashboard workaround for missing log_url https://review.openstack.org/554066 | 21:59 |
| openstackgerrit | Merged openstack-infra/zuul master: Add cherrypy to built-in logging config https://review.openstack.org/572200 | 22:16 |
| openstackgerrit | Merged openstack-infra/zuul master: Stop scheduler more quickly https://review.openstack.org/572186 | 22:16 |
| *** D3VIATION has joined #zuul | 22:19 | |
| corvus | jlk, clarkb, mordred: any thoughts on what to do next about the missing github events? | 22:19 |
| clarkb | corvus: we should probably have a fresh set of eyeballs double check the config on our end to rule out pebkac | 22:19 |
| corvus | ok i'll look | 22:20 |
| mordred | corvus: I think we might need to loop in gundalow when he gets back so that we can make sure the settings on the installation are ok | 22:20 |
| corvus | it's not just on ansible/ansible, right? | 22:20 |
| clarkb | I don't know if anyone has tested outside of ansible/ansible | 22:20 |
| mordred | I do not know the answer to that question | 22:20 |
| corvus | aha! then that's the next step :) | 22:20 |
| corvus | let's test something on gtest-org/ansible | 22:20 |
| mordred | ++ | 22:21 |
| gundalow | #ansible-devel should be able to help in my absence | 22:21 |
| mordred | gundalow: I think we've found another vector to explore where we don't have to bother anybody for now | 22:24 |
| mordred | gundalow: but cool. also, you're not vacationing properly | 22:24 |
| corvus | according to the zuul docs, we should have the perms "issues: read/write" but we have "issues: no access". and events should have "issue comment, issues" checked, but they don't exist (presumably because the perm isn't set). also we should have "release". | 22:26 |
| corvus | that seems very likely relevant to me | 22:27 |
| corvus | also the "commit comment" event | 22:27 |
| gundalow | mordred: cool. Thanks. | 22:28 |
| corvus | here's my plan: leave a recheck comment on gtest-org/ansible to make sure it's broken there. then update the perms to match our docs. then accept the updated perms in gtest-org. assuming they work, ask ansible to accept new perms. | 22:29 |
| corvus | if that doesn't work, then i'll panic | 22:29 |
| clarkb | corvus: ok | 22:30 |
| corvus | oh, only zuul-test is installed there | 22:31 |
| corvus | i'll add openstack-zuul to gtest-org first | 22:31 |
| mordred | corvus: I like your proposed plan | 22:31 |
| mordred | corvus: although I think panicing first would be more amusing | 22:31 |
| clarkb | ok the global config is missing issues too | 22:31 |
| corvus | clarkb: yeah, that's what i was looking at | 22:32 |
| clarkb | wouldn't surprise me if this is carrying over due to the issues are PRs | 22:32 |
| clarkb | I have no idea why this would've chagned or if it did in our config at all (maybe issues are PRs is affecting this as of recently) | 22:32 |
| corvus | i was wrong -- openstack-zuul is installed on gtest-org, so we should be all set. i did not see an event for my comment. | 22:32 |
| corvus | as expected | 22:32 |
| corvus | i closed/reopened a pr and got an event from that | 22:33 |
| corvus | so i'm ready to proceed to updating the perms | 22:33 |
| clarkb | ok | 22:33 |
| clarkb | when we are done with that we should turn on ssl verification too | 22:33 |
| clarkb | (since we broke the integration anyways) | 22:34 |
| corvus | X-GitHub-Event: issue_comment | 22:36 |
| corvus | that's looking good | 22:36 |
| corvus | event b3893300-6847-11e8-9faa-d8616ab45b6c | 22:37 |
| corvus | so i think we're set now, we just need to ask #ansible-devel to accept the updated perms and let jlk know we figured it out, sorry | 22:38 |
| clarkb | any idea if that was set before? | 22:39 |
| *** rbergeron has joined #zuul | 22:43 | |
| corvus | clarkb: the event doesn't show up in our debug logs before today, so very likely it was never set | 22:43 |
| rbergeron | corvus: did a nice humsn request updated permissions to the zuul app in ansible? | 22:44 |
| rbergeron | (or if anyone else knows) | 22:45 |
| corvus | rbergeron: well, i did... i don't feel qualified to speak on whether i'm nice or human :) | 22:45 |
| rbergeron | mostly, i clicked yes :) all done. | 22:45 |
| corvus | rbergeron: \o/ thanks! | 22:45 |
| rbergeron | you are nice and human :) | 22:45 |
| corvus | rbergeron: well, i'll see what i can do about that ;) | 22:46 |
| rbergeron | :D | 22:51 |
| *** sshnaidm|brq has quit IRC | 22:55 | |
| *** TheAceOfHearts has joined #zuul | 22:56 | |
| *** TheAceOfHearts has quit IRC | 23:13 | |
| *** AJaeger has quit IRC | 23:49 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: Handle multiple stats in assertReportedStats https://review.openstack.org/572230 | 23:55 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!