Friday, 2018-01-19

openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Make subunit processing more resilient  https://review.openstack.org/53443100:03
tristanCShrews: pabelanger: a nodepool.d would be great to have indeed, SF uses a nodepool-conf-merger script to produce a single nodepool.yaml file out of the project configs repository00:19
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: license: remove dos line break  https://review.openstack.org/53553100:27
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: Add noop job description  https://review.openstack.org/53553400:29
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: pep8: ignore E124 rule  https://review.openstack.org/53553600:30
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: license: remove dos line break  https://review.openstack.org/53553100:30
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: zk: use kazoo retry facilities  https://review.openstack.org/53553700:30
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: executor: add log_stream_port and log_stream_file settings  https://review.openstack.org/53553800:30
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: scheduler: better handle format status error  https://review.openstack.org/53553900:31
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: requirements: remove paramiko <2.0 cap  https://review.openstack.org/53554000:31
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add OpenAPI documentation  https://review.openstack.org/53554100:31
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: ansible: honor command no_log module attribute  https://review.openstack.org/53554200:32
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: mqtt: add basic reporter  https://review.openstack.org/53554300:32
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: web: add /{tenant}/jobs/{job_name} route  https://review.openstack.org/53554500:33
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: Add restricted-node-labels global limits  https://review.openstack.org/53554600:35
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: Ensure build.start_time is defined  https://review.openstack.org/53554900:39
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: handler: fix support for handler without launch_manager  https://review.openstack.org/53555000:44
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Add a plugin interface for drivers  https://review.openstack.org/53555100:46
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: builder: do not cleanup image for driver not managing image  https://review.openstack.org/53555200:46
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement a static driver for Nodepool  https://review.openstack.org/53555300:47
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Refactor run_handler to be generic  https://review.openstack.org/53555400:47
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Refactor NodeLauncher to be generic  https://review.openstack.org/53555500:48
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement an OpenContainer driver  https://review.openstack.org/53555600:49
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement a Kubernetes driver  https://review.openstack.org/53555700:49
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Implement an Amazon EC2 driver  https://review.openstack.org/53555800:50
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: zk: check for client in properties  https://review.openstack.org/53555900:51
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: config: add statsd-server config parameter  https://review.openstack.org/53556000:51
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Add /node-list to the webapp  https://review.openstack.org/53556200:52
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool master: Add /label-list to the webapp  https://review.openstack.org/53556300:53
pabelangerclarkb: I can do that in the morning00:54
pabelangeralso, \o/00:55
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Make subunit processing more resilient  https://review.openstack.org/53443100:56
clarkbcorvus: fyi https://storyboard.openstack.org/#!/story/2001488 filed from the zuulv3 issues etherpad. I think that could be a potential dos vector. I've not marked it with the 3.0 tag in case I am just missing something about it but I think we may want to consider bumping thei mportant of that up01:13
clarkbalso I've cleared out the zuulv3-issues etherpad finally01:14
*** weshay|dentist is now known as weshay|ruck02:00
*** rlandy has quit IRC02:26
*** harlowja has quit IRC02:31
openstackgerritTristan Cacqueray proposed openstack-infra/zuul master: executor: block stat get_mime on localhost  https://review.openstack.org/53557602:34
*** threestrands_ has joined #zuul02:49
*** threestrands_ has quit IRC02:49
*** threestrands_ has joined #zuul02:49
*** threestrands has quit IRC02:51
*** jappleii__ has joined #zuul03:44
*** jappleii__ has quit IRC03:45
*** jappleii__ has joined #zuul03:45
*** jappleii__ has quit IRC03:46
*** threestrands_ has quit IRC03:46
*** jappleii__ has joined #zuul03:47
*** jappleii__ has quit IRC03:48
*** jappleii__ has joined #zuul03:48
*** harlowja has joined #zuul05:21
openstackgerritAndreas Jaeger proposed openstack-infra/zuul-jobs master: Make subunit processing more resilient  https://review.openstack.org/53443105:22
*** bstinson has quit IRC05:28
*** bstinson has joined #zuul05:46
*** bhavik1 has joined #zuul05:49
*** jappleii__ has quit IRC05:52
*** bhavik1 has quit IRC05:58
*** logan- has quit IRC06:16
*** harlowja has quit IRC06:29
openstackgerritTristan Cacqueray proposed openstack-infra/zuul-jobs master: unittests: make bindep role optional  https://review.openstack.org/52225506:34
*** logan- has joined #zuul06:40
*** pleia2 has quit IRC07:02
*** pleia2 has joined #zuul07:28
*** harlowja has joined #zuul08:17
*** persia has quit IRC08:28
*** persia has joined #zuul08:28
*** jamielennox has quit IRC08:33
*** harlowja has quit IRC08:37
*** jpena|off is now known as jpena08:54
openstackgerritTobias Henkel proposed openstack-infra/zuul master: WIP: Status branch protection checking for github  https://review.openstack.org/53568009:10
openstackgerritTobias Henkel proposed openstack-infra/nodepool master: Don't gather host keys for non ssh connections  https://review.openstack.org/53568509:17
openstackgerritTobias Henkel proposed openstack-infra/nodepool master: Add connection-port to provider diskimage  https://review.openstack.org/53568609:17
*** xinliang has quit IRC09:44
*** jamielennox has joined #zuul09:46
*** xinliang has joined #zuul09:56
*** xinliang has quit IRC09:56
*** xinliang has joined #zuul09:56
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Move github webhook from webapp to zuul-web  https://review.openstack.org/53571110:20
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Add regex support to project stanzas  https://review.openstack.org/53571310:25
openstackgerritTobias Henkel proposed openstack-infra/nodepool master: Add node list to webapp  https://review.openstack.org/53571410:26
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Set remote url on every getRepo in merger  https://review.openstack.org/53571610:28
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Add winrm certificate handling  https://review.openstack.org/53571710:29
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Also prefix the indexes if needed  https://review.openstack.org/53571810:29
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Fix sql reporting with postgres  https://review.openstack.org/53572010:30
openstackgerritTobias Henkel proposed openstack-infra/nodepool master: Don't gather host keys for non ssh connections  https://review.openstack.org/53568510:41
openstackgerritTobias Henkel proposed openstack-infra/nodepool master: Add connection-port to provider diskimage  https://review.openstack.org/53568610:41
*** ankkumar has joined #zuul10:43
ankkumarHi All10:45
ankkumarhow to see running devstack logs in zuul web for debugging purposes?10:46
openstackgerritFabien Boucher proposed openstack-infra/zuul master: Add tenant project group definition example and definition in the doc  https://review.openstack.org/53573011:24
*** saop has joined #zuul12:20
saophello all12:21
saopI am trying to see the zuul ansible running logs in stream, websocket, how would it work?12:21
saopsaop, Or in simple way, how would i post my console logs to zuul websocket ?12:22
*** hogepodge has quit IRC12:26
*** sc68cal has quit IRC12:27
*** gothicmindfood has quit IRC12:27
*** nhicher has quit IRC12:28
*** hogepodge has joined #zuul12:28
*** gothicmindfood has joined #zuul12:28
*** nhicher has joined #zuul12:29
*** sc68cal has joined #zuul12:29
saoptristanC, Any idea how to make this stream.html in zuul to work?12:30
saoptristanC, Its showing End of Stream only.12:30
*** Shrews has quit IRC12:33
tristanCsaop: the executor needs to connect to tcp port 19885 on the slave, is this enabled in your nodepool tenant security group?12:45
saoptristanC, I don't think so12:46
tristanCthen that may be the reason why the streaming doesn't work12:48
*** saop has quit IRC12:50
*** saop has joined #zuul12:54
*** jpena is now known as jpena|lunch13:07
*** Shrews has joined #zuul13:21
*** rlandy has joined #zuul13:32
*** xinliang has quit IRC13:50
*** xinliang has joined #zuul13:52
*** weshay|ruck is now known as weshay|rover14:08
*** jpena|lunch is now known as jpena14:13
saoptristanC, i enabled 19885 in securitygroup but still not showing in stream anything14:13
saoptristanC, also though job is buidling, i could see in status.html its not showing anything in builds.html14:14
saoptristanC, there should be any listing kind of thing i suppose?14:14
tristanCperhaps the zuul-web endpoint are not accessible, how are you running the service?14:14
saoptristanC, sudo zuul-web -d14:15
saoptristanC, and scheduler and executor the same14:16
tristanCsaop: no need to sudo anything in zuul...14:16
saoptristanC, is that a problem for zuul-web14:17
tristanCsaop: by default it listen to port 9000, if you don't use a proxy, you need to change default port in zuul.conf [web] port14:17
saoptristanC, we are able to open zuul dashboard14:17
saoptristanC, even i can do <>/stream.html it showing empty websocket14:18
tristanCpabelanger: leifmadsen_: is there quick start documentation for zuul-web somewhere?14:20
leifmadsen_tristanC: I haven't gotten that far yet14:21
leifmadsen_so nothing written by me at least14:21
leifmadsen_I figured the approach would be to add that on as a documentation component later14:21
*** ankkumar has quit IRC14:24
*** saop has quit IRC14:25
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul master: Allow Ansible 2.4  https://review.openstack.org/53578114:32
*** dkranz has joined #zuul14:40
*** hashar has joined #zuul14:49
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Make subunit processing more resilient  https://review.openstack.org/53443114:54
pabelangertristanC: thanks, is that something you'd want to push up for nodepool.d? I don't mind seeing how we did it in zuul for zuul.d too15:06
tristanCpabelanger: it's pretty low in my todo list atm...15:07
pabelangertristanC: okay, I can look into it15:12
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Make subunit processing more resilient  https://review.openstack.org/53443115:29
corvusclarkb: i pushed the same change mentioned in story 2001488 and zuul -1d it.  i think that was fixed a long time ago.  https://review.openstack.org/53580015:34
openstackgerritMatthieu Huin proposed openstack-infra/zuul-jobs master: [WIP] Debugging playbook: inject public keys in case of failure  https://review.openstack.org/53580315:35
clarkbcorvus: see if I was smart I would've done the same test :) thank you for checking15:56
corvusnp :)15:56
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Add provider info command  https://review.openstack.org/53581716:19
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: Add provider info command  https://review.openstack.org/53581716:23
tobiashShrews, mordred: is https://review.openstack.org/#/c/505430/2 and parent also necessary for ansible 2.4?16:36
Shrewstobiash: i'm not entirely sure, but our tests pass without them16:39
Shrewshttps://review.openstack.org/53578116:39
tobiashShrews: at least 505419 looks important to me from security point of view16:41
tobiashjudging from the commit message16:41
clarkbdoes putting network.py as the only contents of a python file do anything but make python parser unhappy?16:44
clarkbcurious if there is some mechanism there that is magical that isn't clear16:44
openstackgerritMonty Taylor proposed openstack-infra/zuul-jobs master: Only install bindep when we've found a file  https://review.openstack.org/53583616:47
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Disable action and lookup plugins from 2.4  https://review.openstack.org/53583916:54
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Port in changes from ansible 2.4 command module  https://review.openstack.org/53584016:54
tobiashShrews, mordred: reproposed that on top of the ansible 2.4 change16:55
tobiashlet's see if that still works16:55
mordredclarkb: I thnk that's gerrit showing us that it's a symlink17:01
mordredclarkb: s/I thnk//17:01
clarkbsymlinks break setuptools fyi17:01
clarkbI am not sure we should be using them17:01
mordredclarkb: they seem tohave been working fine for us so far (we already use them for all of the existing network modules)17:02
clarkbmordred: have we checked that the symlinks get installed properly?17:03
clarkbI want to say they worked in the past on the packaging side but not post install?17:03
clarkbneutron iirc specifically ran into problems with them that got all the way to integration testing using some import path17:03
mordredclarkb: the installation on zuul.o.o has copies of network.py installed for each of the symlinks17:05
clarkbthey must've fixed it then cool17:05
mordredclarkb: so it installs each file as if they were copies and does not install them as symlinks17:05
mordredyah. good call on checking17:05
clarkb Distribute #183: Symlinked files are now extracted from source distributions.17:06
clarkbI don't remember the issue being that old but apparently it is/was :)17:06
mordredhahaha17:09
openstackgerritJames E. Blair proposed openstack-infra/zuul master: Don't rely on canonical hostnames for CRD  https://review.openstack.org/53585817:12
corvustobiash, mordred: ^ can we merge that quickly?  i'd like to get that in place for the upcoming openstack zuul restart17:12
tobiashlooking17:12
corvus(i just tried to use the new url syntax, and i think that's why it failed)17:12
tobiashlgtm17:14
corvusclarkb: what's the network.py thing you were looking at?17:14
clarkbcorvus: in 505419 gerrit shows you symlinks in a confusing manner17:14
clarkbyou have to read the file mode very carefully otherwise there is no indication this isn't a regular file17:14
corvusclarkb: gotcha17:15
mordredcorvus: +A17:23
mordredgundalow: heads up - we've approved the patch to run shade's ansible functional tests on openstack related patches to ansible/ansible: https://review.openstack.org/#/c/531916/17:31
mordredgundalow: once it's landed and live, i'll update one of the existing PRs and let you know how it goes17:31
openstackgerritTobias Henkel proposed openstack-infra/zuul master: Correctly document default git dirs  https://review.openstack.org/53586417:43
openstackgerritMatthieu Huin proposed openstack-infra/zuul-jobs master: [WIP] Debugging playbook: inject public keys in case of failure  https://review.openstack.org/53580317:56
openstackgerritFabien Boucher proposed openstack-infra/zuul master: Tenant config dynamic loading of project from external script  https://review.openstack.org/53587818:14
* rbergeron rubs her hands eagerly to see the magic from ansible to shade testing18:16
openstackgerritMerged openstack-infra/zuul master: Don't rely on canonical hostnames for CRD  https://review.openstack.org/53585818:25
*** weshay|rover is now known as weshay|bbiab18:31
openstackgerritDavid Shrewsbury proposed openstack-infra/nodepool master: WIP: Convert from legacy to native devstack job  https://review.openstack.org/53589918:43
Shrewsyay! with the branch merge, ^^^ is doing things now18:48
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Port per-change status to zuul-web  https://review.openstack.org/53590318:59
*** myoung is now known as myoung|food18:59
mordredShrews: woot!19:02
*** electrofelix has quit IRC19:09
*** harlowja has joined #zuul19:31
*** jpena is now known as jpena|afk19:33
*** hashar has quit IRC19:38
*** myoung|food is now known as myoung19:41
pabelangerwhy would http://paste.openstack.org/show/647305/ have different values?19:45
pabelangerlooking at quota for rax-ord19:46
openstackgerritMonty Taylor proposed openstack-infra/zuul master: Port per-change status to zuul-web  https://review.openstack.org/53590319:47
clarkbpabelanger: one is the total quota you have in the cloud the otheris total quota less usage iirc19:48
clarkbor it might be total quota in cloud and the other is quota after applying local restrictions like max-servers19:48
pabelangeryah, I wonder if we might have leaked some nodes or something19:48
pabelangerthat's what I am looking at now19:49
pabelangerclarkb: we can remove max-servers from nodepool.yaml now right?19:51
pabelangerI mean, we want to just use what our providers give us I think19:51
clarkbin the case of max-servers 0 we can't19:51
clarkbbut otherwise we maybe can. I dunno19:51
pabelangerkk, something to test next week I guess19:52
pabelangeryah, looks like we leaks some nodes19:53
pabelangerI'll clean them up now19:53
dmsimardI use Firefox on Fedora.. does the zuul status page kill anyone else's browser ?19:59
clarkbdmsimard: is that what has been making firefox so slow?19:59
dmsimardLike, every time it refreshes I feel firefox lag -- even if it's loading in another background tab19:59
mordreddmsimard: it's currently suboptimal, yes19:59
clarkbI just restart it about once a week and that keeps it happy19:59
dmsimardclarkb: yeah, as soon as I close the zuul status page it's fine so I try to only keep it opened when I need it20:00
mordreddmsimard: it's on the todo-list to clean up a bit right after I finish up the javascript stack20:00
dmsimardmordred: it's fine so long as we're aware of it, was just asking to make sure I wasn't crazy20:00
mordredclarkb, dmsimard: that said -it shouldn't be doing anything when it's not your active tab20:00
mordreddmsimard: but yes - are aware20:00
dmsimardmordred: it keeps reloading like every 5 seconds or something.. I suspect that it downloads the status.json etc20:00
dmsimardthe zuul status page for RDO isn't as busy and it doesn't keep locking up my firefox if I keep it opened20:01
pabelangeryah, my CPU under chrome will pin20:01
dmsimardso it's probably something around parsing the json and drawing/rendering20:01
fungiyeah, it has a similar impact for me (firefox-esr on debian/unstable)20:01
dmsimardEverything is rendered client side I guess ?20:02
fungii've taken to retrieving and parsing the json with python20:02
mordredyup. but it's actually just more about how we're doign the poll loop20:02
dmsimardfungi: ouch20:02
mordredwhich can and will be improved20:02
dmsimardfungi: there is a zuultty somewhere20:02
fungibecause the "lag" for my browser to redraw the statsu page when it's relatively active is around 10-15 seconds20:02
fungimost of the time it takes longer for ff to render the status page than the delay before the next status update20:03
dmsimardfungi: https://github.com/harlowja/gerrit_view/blob/master/README.rst#czuul20:03
fungiand also does seem to cause my browser to lock up every few seconds for 10+ seconds even if in a background tab20:03
dmsimardmordred: maybe in the meantime we can tone down the refresh rate or something ? It's probably not lightweight to poke Zuul directly every time either.. That's why I was asking the other day if we were serving status.json from disk20:05
dmsimardfungi: yeah, same behavior for me.20:05
dmsimardfungi: Chrome doesn't lock up, however. Only Firefox -- I thought their electrolysis project was supposed to solve that. Similar thing happens when I load a large logs.o.o page, my other tabs tend to lock down20:06
fungidmsimard: we serve it from a cache last i remember20:06
mordredwe do serve it from a cache20:06
dmsimardHeavy frontends like grafana and kibana tend to have the same locking effect as well20:06
dmsimardfungi, mordred: cached by apache ?20:07
fungidmsimard: part of the performance issue for me is likely due to using firefox-esr instead of a post-xul release since i still rely heavily on one xul-based plugin20:07
mordreddmsimard: no, in ram in zuul20:07
dmsimardfungi: I don't know, it's the same for me in the latest version of firefox20:07
fungioh, good to know20:07
fungii'm on like ff52 i think20:08
dmsimardmordred: so you mean it's not recalculated every time it's queried, basically20:08
mordredyah20:08
clarkb57 was faster for sure20:08
dmsimardokay.20:08
clarkb58 maybe a regression20:08
fungicnfirmed, ff52.5.3 esr20:08
dmsimardyeah 57 made me switch from Chrome but this locking is a bit annoying20:08
mordreddmsimard: I should have the javascript stack finished up early next week, so we should be able to dig in to improvements shortly after20:08
dmsimardmordred: you rock, I don't know how you do all the things that you do, keep rocking20:08
mordreddmsimard: booze. lots of booze20:09
*** jpena|afk is now known as jpena|off20:24
SpamapSclarkb: 58 has spectre mitigations no?20:27
SpamapSbut.. czuul.. yeah.. I keep forgetting about that amazingness20:28
clarkbSpamapS: and 57 too20:30
clarkbSpamapS: its just timer precision reduction20:30
SpamapSclarkb: ah. Well I've found that FF is kicking chrome's butt lately since the quantum beta.20:34
*** weshay|bbiab is now known as weshay20:38
*** weshay is now known as weshay|ruck20:38
*** weshay|ruck is now known as weshay|rover20:38
corvusthe status page is supposed to stop refreshing when not visible, but that breaks a lot20:44
mrhillsmanwhat does a cat job consist of?20:52
mrhillsman2018-01-19 03:28:55,212 DEBUG zuul.ExecutorServer: Got cat job: f17193f179684fd2b7d9a61b7d124dd320:52
clarkbmrhillsman: it cats the file contents for the zuul config out of the git repos20:52
clarkbmrhillsman: it is how zuul reads its config from all the repositories20:52
mrhillsmangot it20:53
mrhillsmanthought my executor picked up a job from gearman...or did it?20:53
clarkbif you don't run separate zuul mergers I believe the executors do the cat jobs20:54
mrhillsmanback to looking at executor behind vpn and saw that in my logs20:54
mrhillsmangot it20:54
clarkbexecutors have mergers built in as they need the code to push into the test envs20:55
SpamapSclarkb: do we run zuul-merger in infra?20:55
SpamapSI figure we do since executors are so loaded20:55
clarkbyes we do20:56
gundalowmordred: ace. Good work to all involved21:10
*** rlandy has quit IRC21:43
*** dkranz has quit IRC23:18
*** corvus is now known as mr_book23:45
*** mr_book is now known as corvus23:45

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!