Wednesday, 2017-05-10

openstackgerritJamie Lennox proposed openstack-infra/zuul feature/zuulv3: Add tenant to url formatting.  https://review.openstack.org/46362800:31
jamielennoxso i haven't been able to run zuul tests for ages, but04:11
jamielennoxpymysql.err.OperationalError: (2003, "Can't connect to MySQL server on 'localhost' ([Errno 0] Error)")04:11
jamielennox*** Error in `python': double free or corruption (out): 0x00007fad10006610 ***04:11
jamielennoxis new04:11
*** isaacb has joined #zuul06:31
*** Cibo_ has joined #zuul07:22
*** hashar has joined #zuul07:37
*** Cibo has joined #zuul09:11
*** Cibo_ has quit IRC09:13
*** Cibo has quit IRC09:36
*** bhavik1 has joined #zuul10:04
mordredjamielennox: that doesn't seem good10:15
*** bhavik1 has quit IRC10:42
Shrewswow, we made python crash? neat11:04
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul feature/zuulv3: WIP: Add web-based console log streaming  https://review.openstack.org/46335311:11
jeblairjamielennox: there are several places where we make the assumption that we only run one instance of a job for a change.  aside from that, there are two reasons we allow implied job variants in the project pipeline config: first, so that you can add extra selectors to a job for only a single project (ie, only run devstack on changes to neutron if these files are changed).  having that at the global job level rather than at the project level is ...11:15
jeblair... one of the worst things about zuulv2 (it has made openstack's config incomprehensible).  second, implied project variants are the only way to modify a job defined in another project (because it's not safe to globally modify a job definition from a different project).11:15
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul feature/zuulv3: WIP: Add web-based console log streaming  https://review.openstack.org/46335311:28
SpamapSShrews: are we moving forward on that w/o asyncio?12:25
* SpamapS hasn't looked at the actual patch just curious12:25
ShrewsSpamapS: it still makes heavy use of asyncio12:26
Shrewsi'm still figuring it out. i *just* got it to successfully stream random text, so that's fun12:26
SpamapSah, so we'll just python3 that bit12:29
Shrewsyup12:29
SpamapSI keep getting distracted off of it, but I'd like to see us start running tox py3 tests on whatever tests do pass now.12:32
*** jkilpatr has joined #zuul13:01
*** jkilpatr has quit IRC13:48
pabelangermorning13:49
pabelangerSpamapS: if you are around this morning, would love to chat about flatpak, if you are aware of it13:49
*** jkilpatr has joined #zuul13:55
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Use six.moves.urllib for python3 compat  https://review.openstack.org/46359513:57
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Use raise Exception for python3  https://review.openstack.org/46359413:57
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Remove FakeFile from fakeprovider.py  https://review.openstack.org/46358713:57
*** isaacb has quit IRC13:58
jeblairi found a cozy spot near room 108 if anyone wants to join me14:02
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul feature/zuulv3: WIP: Add web-based console log streaming  https://review.openstack.org/46335314:03
*** isaacb has joined #zuul14:22
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: WIP: Fix imports for python3  https://review.openstack.org/46376914:23
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul feature/zuulv3: WIP: Add web-based console log streaming  https://review.openstack.org/46335314:24
pabelangerShrews: ^maybe you have a few minutes today to help explain imports for nodepool, because imports for py27 are not working local for me and I do not know why14:26
Shrewspabelanger: relative imports changed14:30
Shrewspabelanger: so "import zk" needs to be "from . import zk", for example14:30
Shrewsthe 2nd form should work for both version14:31
Shrewspabelanger: what's not working for you?14:32
pabelangerShrews: I am having this issue: http://logs.openstack.org/69/463769/1/check/gate-nodepool-python27-ubuntu-xenial/fef81bd/console.html#_2017-05-10_14_27_43_61605214:34
pabelangerwhich the gate has also found, yay14:34
pabelangeragree, from . import exceptions should work14:35
pabelangerlet me make a change14:35
* SpamapS doesn't like .14:35
SpamapSI prefer the full module path.14:35
pabelangeragree, I thought: from nodepool import exceptions was that, full module path14:37
pabelangerbut, I must be wrong14:37
Shrewsi do not know why the full path is not working for you14:38
pabelangerI am assuming tox might be doing something here, but haven't figure it out yet14:38
*** jkilpatr has quit IRC14:40
Shrewspabelanger: is that the only file where you get that error?14:40
Shrewsif so, it might have something to do with the name of the file also being 'nodepool', so maybe it's confusing the .py file and the entire library? not sure14:41
pabelangerShrews: Oh, that could be14:42
pabelangerShrews: Yes, i think that is correct14:42
pabelangerShrews: thank you14:43
Shrewsthat's caught me before in other scenarios14:44
pabelangeragree, I think for now we can easy mode with from . import foo14:44
pabelangerbut maybe start to break up nodepool.py into other bits14:44
pabelangerlaster14:44
pabelangerlater*14:44
*** jkilpatr has joined #zuul14:47
*** hashar has quit IRC14:50
pabelangerShrews: jeblair: clarkb: SpamapS: So, for python3 imports, maybe it is better just to rename nodepool.py now? instead of from . import foo.  So, if we want to bikeshed on new filename does nodepool.py->pool.py work?14:50
Shrewsi think launcher.py makes sense since i've had that in the back of my head since the daemon rename14:51
pabelangerYes, that does make sense actually14:51
SpamapS+114:52
jeblairwfm15:05
*** jkilpatr has quit IRC15:07
*** isaacb has quit IRC15:08
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul feature/zuulv3: WIP: Add web-based console log streaming  https://review.openstack.org/46335315:19
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Rename nodepool.py to launcher.py  https://review.openstack.org/46380715:23
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Fix imports for python3  https://review.openstack.org/46380815:23
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul feature/zuulv3: WIP: Add web-based console log streaming  https://review.openstack.org/46335315:24
*** jkilpatr has joined #zuul15:32
*** jkilpatr has quit IRC15:41
*** DangerousDaren has joined #zuul15:47
*** hashar has joined #zuul15:53
*** Cibo has joined #zuul16:52
*** hashar has quit IRC17:05
*** hashar has joined #zuul17:06
*** hashar has quit IRC17:07
*** jkilpatr has joined #zuul17:10
*** smyers has quit IRC17:15
*** smyers has joined #zuul17:16
*** DangerousDaren has quit IRC17:20
*** tobiash_ has joined #zuul17:36
*** jkilpatr has quit IRC17:42
tobiash_hi, is there already a way in zuulv3 to add deployment defined variables to the jobs?17:42
tobiash_e.g I have an automated deployment together with an automatic creation of a cache server which is not known in advance17:43
tobiash_so I would like to have some way to add a (ansible generated) variable to the zuul config (tenant config?) which is then accessible by any job17:45
tobiash_I'm thinking of something like http://paste.openstack.org/show/609295/17:54
pabelangertobiash_: we actually talked a little about this today in person.  But I think we have the need for a playbook to generare some variable, which is dynamic, so the next playbook run is able to consume it18:09
tobiash_pabelanger: ah, ok18:11
tobiash_so you mean a trusted playbook could generate some variables (depending on stuff on the executor as it's not running sandboxed) which can be used on later (untrusted) playbooks?18:13
pabelangertobiash_: I believe jeblair and mordred will be able to provide more information too18:13
tobiash_sounds pretty generic18:13
SpamapStobiash_: indeed, your timing is excellent. I think we have a good plan and I expect us to express it in doc/spec form soon.18:13
tobiash_:)18:13
pabelangertobiash_: yes, or even untrusted to untrusted in different repos18:13
SpamapSI believe the idea was for every playbook to generate variables for zuul or the next job to inspect.18:13
tobiash_I think that would be a good fit for my use case18:14
tobiash_I hope you enjoy the summit. Unfortunately I wasn't able to attend. Would have been cool to meet you guys in person.18:16
jeblairtobiash_: indeed!  we'll cross paths eventually :)18:17
jeblairtobiash_: to add to what SpamapS and pabelanger said: we'll also probably have something like this change: https://review.openstack.org/447734  to add zuul site variables too.18:18
tobiash_Judging from your timezones I assume no one will attend the german openstack days end of june?18:18
tobiash_jeblair: cool, looks like I overlooked this change18:19
jeblairtobiash_: mordred sometimes travels to those, we should check with him.  it's not on my calendar at the moment, though that could change18:19
tobiash_this year it's just in the neighbor building of my employer...18:21
SpamapSjeblair: so I just pushed a change to gear that does appendData. But I'm wondering if, instead, we should make a TextList which is just a list that converts appends/extends on write, so we don't API-break.18:22
*** clarkb has joined #zuul18:22
SpamapS(but I know we said it should be ok to API-break because of the language port)18:22
SpamapSwe could also just tag this 0.8 and call it macaroni18:23
jeblairtobiash_: oh nice!  looks like that's the weekend after ansiblefest london (which mordred and i are attending)18:23
jeblairSpamapS: TextList wfm18:24
tobiash_interesting18:25
*** Cibo has quit IRC18:30
*** Cibo has joined #zuul18:39
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Rename nodepool.py to launcher.py  https://review.openstack.org/46380718:40
openstackgerritPaul Belanger proposed openstack-infra/nodepool feature/zuulv3: Fix imports for python3  https://review.openstack.org/46380818:40
*** jkilpatr has joined #zuul18:41
jeblairjlk: question on 44451118:43
*** tobiash_ has quit IRC18:54
SpamapSjeblair: unfortunately this wasn't the problem causing my internal geard to disconnect. :-/18:55
SpamapSactually hm18:56
jeblairjlk: and let me know if you want to chat about 44529218:56
SpamapSthis is interesting...18:56
jeblairSpamapS: i'll start setting up a test env18:57
SpamapShttp://paste.openstack.org/show/609302/18:57
SpamapSthe test passes18:57
SpamapSbut the teardown gets stuck18:57
SpamapSwhich I think is actually better than before18:57
SpamapSjeblair: I'm over by 108 again, will be heading to airport in about 45-60 minu18:58
jeblairSpamapS: yeah, that doesn't, at first glance, look gear related18:58
jeblairSpamapS: is there an open seat near you?18:59
SpamapShttp://paste.openstack.org/show/609303/18:59
SpamapSjeblair: there is one yes18:59
jeblairSpamapS: be right there18:59
*** jkilpatr has quit IRC19:12
*** jkilpatr has joined #zuul19:16
jlkjeblair: I'm in now, happy to chat. Looking at the reviews now19:39
SpamapSwoohoo, got a test to pass in py35!19:39
jlkA+19:39
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: WIP: working on python3 compat  https://review.openstack.org/46146819:40
SpamapSnow time to go fly home :-P19:41
*** jkilpatr has quit IRC19:42
SpamapSThat commit at least gets the test suite to run without locking up...19:42
* SpamapS waiting for how many tests actually pass19:42
SpamapSLength too long: 1587008719:42
SpamapS"too many"19:43
jlkjeblair: Reading coment on 445292, I see what you mean. we can definitely set that value when an event comes in and we do the API call to get the pull request object. What I wasn't sure of is the action path should this method return a False, stating that the change cannot be merged. Since that's going to be a fairly common result.19:45
*** jkilpatr has joined #zuul20:29
jeblairjlk: in a dependent pipeline, it will not enqueue the change if it's false at enqueue time.  if it changes while already in the queue, it should get kicked out20:32
jlkalright, so for GitHub, the simple data we can get from the event will say mergeable basically if the patch(es) can merge cleanly. It doesn't look at vote status at all. So yeah, we'd get some false positives on "this could merge" when in reality it may not have all the required things in place.20:35
jeblairjlk: oh, well we can perform that api call then, right?20:37
jlkNo, because the API call also doesn't give you that information20:37
jlkat this time, github does not expose a True/False value for whether or not all the merge requirements are in place20:37
jeblairjlk: i'm confused :(20:38
jlkThe best we could do is query the branch object through the API to discover what "protections" are in place for the branch, and then investigate the PullRequest to see if the PR has met all those requirements.20:38
jlkGithub doesn't have a vote requirement system like gerrit20:38
jlkwhat it has is "branch protection"20:38
jeblairjlk: that's pretty close, in spirit, to what we do for gerrit20:38
jlkyou can configure each branch to be "protected" from improper merges20:39
jlkprotections can be "needs a human with write access to review it", can be "needs these passing statuses set on the tip of the PR", and could even be "only allow this sub-set of people to commit to the branch"20:40
jeblairjlk: then that's probably what we should be doing here.  i don't think we need to worry about file-level merge conflicts in relation to this20:40
jlkand the even MORE insane one of "must be a fast forward merge" essentially.20:40
jlkWould you be willing to accept us just returning True here in this PR, and having a follow up one to do the investigation for more accurate data?20:41
jlkWe don't even support depends-on type changes anyway yet20:41
jeblairjlk: absolutely20:41
jeblairjlk: i mostly didn't want to land a todo which looked wrong :)20:42
jlkgotcha. I'll fix that up and continue the rebase-a-thon20:43
SpamapSoh that's fun.21:33
SpamapSin py3k, if you define __eq__ , __hash__ gets undefined21:33
jlker,... what?21:34
SpamapS(to coerce you into making things that are equal likely to have the same hash)21:34
jlkwhy would it do that?21:34
jlkeww21:34
jlkI mean, there's probably a good reason, but that seems heavy handed21:34
SpamapSyou can just do __hash__ = object.__hash__ if you want python 2's behavior21:34
SpamapSwhich is to return the id21:34
SpamapSjlk: I guess they figure you're going to have to revisit every object anyway21:35
jlkINCOMING!21:40
* SpamapS ducks21:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Allow github trigger to match on branches/refs  https://review.openstack.org/44562521:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Add cachecontrol to requests to github  https://review.openstack.org/46158721:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Add support for requiring github pr head status  https://review.openstack.org/44939021:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Create github client each time it's called.  https://review.openstack.org/46342121:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Use integration_id with github  https://review.openstack.org/46342021:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Refactor integration key loading to always load  https://review.openstack.org/46342321:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Decode JSON body once for requests  https://review.openstack.org/46342221:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Include exc_info in reporter failure  https://review.openstack.org/46076521:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Add support for github enterprise  https://review.openstack.org/44925821:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Log GitHub API rate limit  https://review.openstack.org/44615021:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Ensure PRs arent rejected for stale negative reviews  https://review.openstack.org/46070021:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Comment on PRs if a remote call to merge a change failed  https://review.openstack.org/46076221:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: support github pull request labels  https://review.openstack.org/44451121:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Set filter according to PR/Change in URL  https://review.openstack.org/44678221:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Support for dependent pipelines with github  https://review.openstack.org/44529221:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Adds github triggering from status updates  https://review.openstack.org/45384421:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Implement trigger require/reject-approvals  https://review.openstack.org/46338721:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Implement pipeline requirement on github reviews  https://review.openstack.org/45384521:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Implement github trigger requirement status  https://review.openstack.org/46338621:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Actually filter status triggers  https://review.openstack.org/46338521:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Represent github change ID in status page by PR number  https://review.openstack.org/46071621:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Better merge message for GitHub pull reqeusts  https://review.openstack.org/44564421:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Test gerrit and github drivers in same tenant  https://review.openstack.org/44825721:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Encapsulate determining the event purpose  https://review.openstack.org/44524221:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Save installation ids to a cache and fetch them per project  https://review.openstack.org/46342421:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Catch integration key file not found in zuul  https://review.openstack.org/46342521:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: GitHub file matching support  https://review.openstack.org/44611321:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Add trigger capability on github pr review  https://review.openstack.org/44936521:41
openstackgerritJesse Keating proposed openstack-infra/zuul feature/zuulv3: Include tenant name in github context  https://review.openstack.org/46387121:41
SpamapSwow this is a special type of error21:50
SpamapSTypeError: initializer for ctype 'unsigned char *' must be a bytes or list or tuple, not str21:50
clarkbbecause str is list of ints?21:52
SpamapSprobably21:52
SpamapSbytes is a list of ints too21:52
SpamapSactually no, in this case, it's because str's aren't bytes period21:58
SpamapSso there's no way to feed that into a C lib21:58
SpamapSwhich is really the whole point of the bytes/str change21:58
*** jkilpatr has quit IRC22:11
jeblairSpamapS: where's that from?22:24
jamielennoxoh, in case anyone is interested: https://github.com/BonnyCI/requests-mock/pull/1 is zuul v3 leaving status on a github PR via integration23:46

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!