Friday, 2018-01-05

*** JasonCL has quit IRC00:25
*** JasonCL has joined #zuul00:26
*** EmilienM has quit IRC00:30
*** JasonCL has quit IRC00:31
*** JasonCL has joined #zuul00:35
*** JasonCL has quit IRC00:40
*** JasonCL has joined #zuul00:47
*** JasonCL has quit IRC00:48
*** JasonCL has joined #zuul00:48
*** JasonCL has quit IRC00:54
*** JasonCL has joined #zuul00:54
*** rlandy has quit IRC01:22
*** JasonCL has quit IRC01:23
*** JasonCL has joined #zuul01:24
*** JasonCL has quit IRC01:30
*** JasonCL has joined #zuul01:40
*** JasonCL has quit IRC01:45
*** JasonCL has joined #zuul01:46
*** JasonCL has quit IRC01:48
*** JasonCL has joined #zuul01:49
*** JasonCL has quit IRC01:50
*** JasonCL has joined #zuul01:51
*** JasonCL has quit IRC02:01
*** JasonCL has joined #zuul02:02
*** JasonCL has quit IRC02:12
*** threestrands_ has joined #zuul02:14
*** threestrands_ has quit IRC02:14
*** threestrands_ has joined #zuul02:14
*** threestrands has quit IRC02:16
*** harlowja has quit IRC02:36
*** xinliang has quit IRC02:52
*** xinliang has joined #zuul03:04
*** xinliang has quit IRC03:04
*** xinliang has joined #zuul03:04
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Do pep8 housekeeping according to zuul rules  https://review.openstack.org/52294504:10
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Use same flake8 config as in zuul  https://review.openstack.org/50971504:10
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: handler: fix support for handler without launch_manager  https://review.openstack.org/52477304:10
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Add a plugin interface for drivers  https://review.openstack.org/52462004:10
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: builder: do not cleanup image for driver not managing image  https://review.openstack.org/51692004:10
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement a static driver for Nodepool  https://review.openstack.org/46862404:13
*** jkilpatr has quit IRC04:27
SpamapSFYI, the Netflix zuul maintainer was nice enough to close our request for them to disambiguate without explanation, so I submitted this: https://github.com/Netflix/zuul/pull/39104:46
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Do pep8 housekeeping according to zuul rules  https://review.openstack.org/52294504:50
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Use same flake8 config as in zuul  https://review.openstack.org/50971504:50
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/zuul feature/zuulv3: Disambiguate with Netflix and Javascript zuul  https://review.openstack.org/53129204:58
SpamapShttps://github.com/defunctzombie/zuul/pull/32405:01
*** dmsimard has quit IRC05:09
*** dmsimard has joined #zuul05:26
*** dmsimard has joined #zuul05:26
*** dmsimard has quit IRC05:32
*** dmsimard has joined #zuul05:34
*** threestrands_ has quit IRC06:13
*** bhavik1 has joined #zuul06:29
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: handler: fix support for handler without launch_manager  https://review.openstack.org/52477307:52
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Add a plugin interface for drivers  https://review.openstack.org/52462007:52
*** nguyentrihai has joined #zuul08:06
openstackgerritTristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Add a plugin interface for drivers  https://review.openstack.org/52462009:00
*** bhavik1 has quit IRC09:44
*** JasonCL has joined #zuul09:46
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: [WIP] Refactor status functions, add more status endpoints  https://review.openstack.org/53135410:39
*** electrofelix has joined #zuul10:55
openstackgerritMatthieu Huin proposed openstack-infra/nodepool master: [WIP] Refactor status functions, add more status endpoints  https://review.openstack.org/53135411:01
*** JasonCL has quit IRC12:44
*** JasonCL has joined #zuul12:52
*** yolanda__ has joined #zuul12:56
*** yolanda has quit IRC12:59
*** jkilpatr has joined #zuul13:09
openstackgerritMatthieu Huin proposed openstack-infra/nodepool feature/zuulv3: Add /node-list to the webapp  https://review.openstack.org/49996913:26
openstackgerritMatthieu Huin proposed openstack-infra/nodepool feature/zuulv3: Add /label-list to the webapp  https://review.openstack.org/50025413:26
openstackgerritMerged openstack-infra/zuul-jobs master: Add tox-nodejs-npm base job for python and javascript  https://review.openstack.org/52966313:40
*** rlandy has joined #zuul13:41
*** JasonCL has quit IRC14:21
*** JasonCL has joined #zuul14:26
*** JasonCL has quit IRC14:30
*** JasonCL has joined #zuul14:32
*** JasonCL has quit IRC14:36
*** maxamillion has quit IRC14:52
*** JasonCL has joined #zuul14:54
*** JasonCL has quit IRC14:59
*** mhu has joined #zuul15:04
*** JasonCL has joined #zuul15:09
*** JasonCL has quit IRC15:17
*** maxamillion has joined #zuul15:18
*** JasonCL has joined #zuul15:21
openstackgerritMatthieu Huin proposed openstack-infra/nodepool feature/zuulv3: [WIP] Refactor status functions, add web endpoints, allow params  https://review.openstack.org/53139015:21
corvustobiash, mordred, jhesketh, jlk: i left a really long comment in 52831215:27
*** weshay is now known as weshay_interview15:28
openstackgerritMatthieu Huin proposed openstack-infra/nodepool feature/zuulv3: [WIP] Refactor status functions, add web endpoints, allow params  https://review.openstack.org/53139015:39
Shrewscorvus: how do we plan to expose the build UUID for using the fingergw?15:54
* Shrews has typed jebl^D^D^Dcorvus multiple times so far in 201815:56
corvusShrews: i was thinking we should put it in the status page, so next to the web streaming link, there's a second link (probably just an icon or something) so you can copy-paste the finger url15:57
mordredShrews: it's in the status.json - but the exact mechanism we'd use on the status page isn't decided yet15:57
mordredor what corvus said :)15:57
* mordred was also thinking we could do a preference check box like the expand-all one or something "I prefer finger links"15:58
mordredcorvus, tobiash, jlk: re: 528312 ... gh pr urls in depends-on footers have urls lke https://github.com/ansible/ansible/pull/33098 which have the canonical name of the src repo in it - could we use that as part of your suggested solution to determine if it's a repo we know about  (and thus probably can see via authenticated installation lookup) or foreign, and thus must necessarily use unauthenticated?15:59
corvusmordred: that works too, though i think it's likely that some people may choose to use the web sometimes and finger others, so having both there would be good.  but you could use that box to switch the main url...16:00
mordredcorvus: yah. agree16:00
corvusmordred: this is for the reverse dependency (ie, needed by) and the urls in question could point anywhere.  example, we're asking github to give us all the PRs that have "Depends-On: https://review.openstack.org/1234"16:01
corvus(because we're enqueing gerrit change 1234, and want to enqueue things that depend on it behind it...)16:02
corvuswhich actually made me realize that we can narrow the scope slightly -- we only need to find these for projects that share a change queue -- so we could iterate over all the installations for all the projects in the shared change queue, which could be much smaller than all the installations in the system.16:03
corvusmight just be 1.16:03
mordredcorvus: ahhhhhh16:04
mordredgotcha16:04
openstackgerritFabien Boucher proposed openstack-infra/zuul-jobs master: Add roles for static web content build and publication  https://review.openstack.org/53140616:05
jlkI will try to carve out time to read the issue and the comments today or this weekend16:06
openstackgerritDavid Shrewsbury proposed openstack-infra/zuul feature/zuulv3: Don't treat finger client disconnect as exception  https://review.openstack.org/53140816:12
openstackgerritMatthieu Huin proposed openstack-infra/nodepool feature/zuulv3: [WIP] Refactor status functions, add web endpoints, allow params  https://review.openstack.org/53139016:18
pabelangercorvus: mordred: helping leifmadsen with some docs, what do we want to do about secure.conf for nodepool? right now it isn't used, but we need to create an empty file to properly start nodepool.  https://review.openstack.org/480310/ was created but abandoned, should we just remove secure.conf or remove the default value16:25
rcarrillocruzheh, yeah16:29
Shrewsi'm of the opinion now that it should never be required. but if it's present, use whatever data is in it (though that 2nd part isn't supported yet)16:29
rcarrillocruzhttps://github.com/rcarrillocruz/zuul-ci-container/blob/master/roles/ansible-role-nodepool-launcher-container/tasks/main.yml#L8716:29
*** weshay_interview is now known as weshay16:30
corvuswe need a place to put zookeeper authentication information.  Shrews's suggestion would let folks put it in nodepool.conf or secure.conf.  i like that.16:30
mordredcorvus, Shrews, pabelanger: ++ agree16:31
leifmadsenship it!16:33
leifmadsen:D16:33
*** JasonCL has quit IRC16:35
*** JasonCL has joined #zuul16:37
Shrewspabelanger: corvus: i can jump on that grenade to shepherd jamie's patch through and add secure.conf support for zk. seems like a good thing to have for a v3.0 release16:39
pabelangerwfm16:40
pabelangerdanke!16:40
corvusShrews: thanks!16:40
Shrewsjamielennox: fyi and thx for the patch ^^^16:41
rcarrillocruzdoes https://review.openstack.org/#/c/530265/6 look good to people ?16:42
*** JasonCL has quit IRC16:44
openstackgerritMatthieu Huin proposed openstack-infra/nodepool feature/zuulv3: [WIP] Refactor status functions, add web endpoints, allow params  https://review.openstack.org/53139016:45
dmsimardrcarrillocruz: network devices have no implementation of setup to gather facts and such ?16:46
rcarrillocruznot yet16:46
dmsimardrcarrillocruz: I thought setup was one of the basic things in a connection plugin16:46
rcarrillocruzthey are modules16:46
rcarrillocruzios_facts16:46
rcarrillocruziosxr_facts16:46
dmsimardoh, so it still uses ssh ?16:46
gundalowdmsimard: depends on the connection plugin16:46
*** JasonCL has joined #zuul16:46
rcarrillocruzthere's an item in the works to have machinery to appropiately call the module for underlying OS16:46
rcarrillocruzso setup does the right thing16:47
gundalowmay use SSH/HTTP or other stuff16:47
dmsimardah yeah I guess it depends on whatever API16:47
gundalownetconf, etc, etc,16:47
* dmsimard nods16:47
rcarrillocruznetwork_cli is special, in the sense it creates a persistent connection for CLI based (ssh) commands16:47
rcarrillocruzcos we use paramiko16:48
rcarrillocruzwe don't have controlpersist16:48
rcarrillocruzthus network_cli gives the persistence16:48
rcarrillocruzso we can reuse a connection made to pass thru multiple commands16:48
dmsimardrcarrillocruz: so the purpose of that setup task is a bit to validate that the host works properly16:48
dmsimardif we're skipping that validation for network devices we're missing out on that validation16:49
rcarrillocruzwell, we could have different validation mechanisms per connection types16:49
rcarrillocruzbut what we agreed for now is to have that blacklist mechanism16:49
rcarrillocruzsince in the end, we'll have gather_facts "just work" for network devices in 2.616:50
rcarrillocruzand then revert that patch16:50
dmsimardrcarrillocruz: https://github.com/openstack-infra/zuul/commit/f641f4fd34cc93c6deb837a5099048492130c0eb16:50
dmsimardrcarrillocruz: oh, I didn't know anything had been agreed on16:51
rcarrillocruzyeah, i know what it does, and it makes sense16:51
dmsimardcarry on, I'm a zuul noob anyway :p16:51
rcarrillocruzit's just it assumes all nodes are going to have python in the other end, thus the '-m setup' was chosen16:51
dmsimardrcarrillocruz: yeah that's fair, I was thinking perhaps it was worth considering different ways of validating the network depending on the kind of device16:52
rcarrillocruzwe discussed to maybe do a wait_for16:52
rcarrillocruzbut16:52
dmsimardvs "skip validation" depending on the kind of device16:52
rcarrillocruzcorvus said it was important to have end to end validation16:52
rcarrillocruznot just 'hey is the socket opened?'16:52
gundalowYou can do a basic "Is this running?": ansible -m $platform_command -a "commands=?"16:52
gundalow(assuming connection type & credentials are specified in inventory16:52
mordredShrews: oh - speaking of zk - I lost track of the discussion about moving to a 3-node cluster ... where did we ever get with that?16:53
dmsimardyeah, that's a bit what I had in mind16:53
dmsimardparameterize the "-m setup" and do $setup instead, $setup being something relevant to the connection type or device type16:53
mordredrcarrillocruz, gundalow: -m setup is used to prime the fact cache16:53
Shrewsmordred: there was a discussion?16:53
gundalowansible -m vyos_command -a 'commands=?' -u vyos -i vyos-1-1-8, -e 'ansible_host=ec2-52-3-245-147.compute-1.amazonaws.com ansible_port=22 ansible_connection=local ansible_ssh_private_key_file=test/cache/id_rsa' vyos-1-1-816:54
gundalow^ What we do in Shippable to ensure the network VM is running before we start tests16:54
mordredrcarrillocruz, gundalow: since we currently set gathering to 'smart' - will ansible network connection plugins do the right thing and not gather facts?16:55
dmsimardI guess all those vars would be provided by the inventory already in the context of zuul16:55
gundalowrcarrillocruz: will gather_facts *just work* for *all* platforms in 2.6, or will there be some per-platform changes needed?16:55
rcarrillocruzthat's influx, initial work started16:56
rcarrillocruzlet me link16:56
mordredrcarrillocruz: also - if gather_facts won't work with gathering=smart ... should we write empty json dicts to the fact cache dir for those hosts that are blacklisted?16:56
mordredrcarrillocruz: we already write out fake facts for localhost ... https://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/executor/server.py?h=refs/changes/65/530265/6#n33916:57
openstackgerritDavid Moreau Simard proposed openstack-infra/zuul feature/zuulv3: Delete IncludeRole object from result object for include_role tasks  https://review.openstack.org/50423816:57
rcarrillocruzmordred: not sure, in my tests zuul just work on network devices with that patch, why you think we should add an empty json fact dict for them16:57
rcarrillocruzfor consistency then?16:57
mordredrcarrillocruz: oh - don't know- if they work, then we don't need to16:57
mordredrcarrillocruz: was just bringing it up in case fact gathering was also sad16:57
rcarrillocruzhttps://github.com/ansible/ansible/pull/3178316:58
rcarrillocruzthat's some starter bcoca pushed16:58
rcarrillocruzto 'unhardcode' gather_facts to be just a unix game16:58
rcarrillocruzgundalow, dmsimard , mordred ^16:59
rcarrillocruzbut that needs more work for us... i expect we'll need to have some 'introspection' or hint16:59
rcarrillocruzfor networking at least16:59
rcarrillocruzlike16:59
rcarrillocruz'ok, let's check ansible_network_os, if it's set and it's IOS, let's use ios_facts for gather_facts'16:59
mordred++17:00
openstackgerritMatthieu Huin proposed openstack-infra/nodepool feature/zuulv3: [WIP] Refactor status functions, add web endpoints, allow params  https://review.openstack.org/53139017:02
SpamapSwow. such backlog. very chat.17:02
pabelangerHmm, for some reason getting nodepool provider is no longer running in local testing: nodepool.task_manager.ManagerStoppedException: Manager myprovider is no longer running17:03
pabelangerbut, nothing in logs to stay we stopped it17:04
*** JasonCL has quit IRC17:05
rcarrillocruzmordred: i don't think gathering protects network_cli. I mean, gathering is a caching mechanism that prevents doing subsequent facts gathering to the cache, but in this case it's the initial facts gathering, so network_cli or not ansible will try to gather it17:07
*** JasonCL has joined #zuul17:08
mordredrcarrillocruz: awesome.17:09
mordredrcarrillocruz: https://review.openstack.org/#/c/530265 lgtm17:10
rcarrillocruzcoolio, thx17:10
*** JasonCL has quit IRC17:12
rcarrillocruzotoh tho, even if you ask me, doing other mchanisms for testing just net connectivity would be a bit moot17:19
rcarrillocruznodepool does ssh-keyscan17:19
rcarrillocruzso yeah, i see what mordred says that the main reason for that is getting facts (along with another net connectivity test before zuul job run)17:19
*** JasonCL has joined #zuul17:21
mordredrcarrillocruz: yah - it's actually also that we want to gather and cache facts but we don't want to let non-privileged playbooks have write access to the cache directory17:37
mordredrcarrillocruz: so we run the pre-gathering out of band in the executor so that we can do it in an appropriate security context for all the nodes in question safely17:38
dmsimardhmm, https://review.openstack.org/#/c/504238/ no longer triggers the JSON finger URL issue after just rebasing17:42
rcarrillocruzAck17:45
pabelangerokay, think I figured provider no longer running issue. Looks like we didn't properly stop nodepool-launcher17:52
fungilooking at adding playbook readmes to the base jobs... we currently have none at all for the playbooks in the zuul-jobs repo (only the roles), and none in project-config playbooks for that matter with the sole exception of base-minimal18:09
*** harlowja has joined #zuul18:09
fungiare readmes for playbooks something we actually want?18:09
dmsimardwe have readme for roles yeah, playbooks should be self documenting for the most part I would guess18:18
dmsimardin the sense that ansible is verbose to begin with18:19
clarkbalso playbooks aren't shared outside of current repo so aren't a reconsumable unit (not that having docs for internal use is a bad idea)18:21
leifmadsenALL THE DOCS!18:23
mordreddmsimard, fungi, clarkb: I made a crazy POC patch a few months ago ... lemme find it18:27
dmsimardthere's something like this that exists apparently: https://github.com/shirou/sphinxcontrib-ansibleautodoc18:27
mordreddmsimard, fungi, clarkb: https://review.openstack.org/#/c/500612/18:28
dmsimardmeta roles that only include other roles eh18:29
mordreddmsimard, fungi, clarkb: the idea in my brain was that since roles are reconsumable and we have a documentation story for them, that for 'important' units of re-use we could create some roll-up roles18:30
mordredyah. not sure it's a great idea, just tossing it out there in the context of the other thing18:30
*** electrofelix has quit IRC18:30
mordredalso - bcoca has been noodling on the idea of 'bundles' for ansible that include playbooks and roles and stuff ...18:31
mordredhttps://github.com/bcoca/playbook_bundle <-- is an example of one18:31
mordredthe idea being that there would be a tool similar to ansible-playbook that could deal with them, and that the playbooks in them could/would correspond to known or named 'phases'18:32
dmsimardI remember that18:32
mordredif that manages to come about, I imagine for us it could be very convenient, since then we could have a sphinx bundle that has a pre-run a run and a post-run playbook in it ... for instance18:33
*** rlandy has quit IRC18:56
*** JasonCL has quit IRC19:03
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Fake more of the github3 api  https://review.openstack.org/53147819:10
*** openstack has joined #zuul21:17
*** ChanServ sets mode: +o openstack21:17
*** JasonCL has quit IRC21:26
*** JasonCL has joined #zuul21:34
openstackgerritTobias Henkel proposed openstack-infra/zuul feature/zuulv3: Add regex support to project stanzas  https://review.openstack.org/51336821:35
dmsimardclarkb: looking (re: job banner)21:37
*** hashar has joined #zuul21:41
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Share a fake pull request database across connections  https://review.openstack.org/53150621:47
*** qwc has quit IRC22:11
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: WIP: Support cross-source dependencies  https://review.openstack.org/53080622:16
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Delete stale jobdirs at startup  https://review.openstack.org/53151022:23
*** qwc has joined #zuul22:34
*** hashar has quit IRC22:41
openstackgerritJames E. Blair proposed openstack-infra/zuul feature/zuulv3: Delete stale jobdirs at startup  https://review.openstack.org/53151023:12
*** jpena|off has quit IRC23:14
SpamapSwow23:23
SpamapShttps://github.com/Netflix/zuul/pull/39123:23
SpamapS@artgon artgon locked as resolved and limited conversation to collaborators 4 hours ago23:23
corvusthat seems rude... i mean, if he hadn't slammed the door in your face you probably could have pointed him at the article published in the trade press that suffered from that confusion...23:52
dmsimardthere was a link somewhere23:53
dmsimardhttps://thenewstack.io/ibm-openstack-engineer-urges-cncf-consider-augmenting-jenkins-zuul/ from the referenced issue23:54
corvusdmsimard: well, yes, but the netflix folks are never going to see that23:54
corvusoh he did see it23:54
SpamapSI've been extremely patient and polite. I'm done. Internet /ignore.23:55
corvusSpamapS: yep23:55
SpamapSI was hoping we'd have it all wrapped up by 3.0 release time.23:56
mordredI love it when people are nice23:56
SpamapSBut at this point, all we can do is hope nobody hears about Zuul from that, and winds up in that repo.23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!