| pabelanger | SpamapS: for the purpose of the installfest, we basically did the following: https://etherpad.openstack.org/p/SYD-windmill-101 The idea was just to have 'software' configure everything, which it could (in future) but was a quick way to setup dependencies and services in place. | 00:10 |
|---|---|---|
| *** ricky_summit has quit IRC | 00:18 | |
| SpamapS | pabelanger: cool, yeah windmill may be simpler than hoist. :) | 00:47 |
| *** qwc has quit IRC | 01:01 | |
| *** qwc has joined #zuul | 01:09 | |
| *** qwc has quit IRC | 01:36 | |
| *** qwc has joined #zuul | 01:42 | |
| *** qwc has quit IRC | 01:52 | |
| *** qwc has joined #zuul | 02:15 | |
| *** jkilpatr has quit IRC | 02:21 | |
| *** qwc has quit IRC | 02:23 | |
| *** qwc has joined #zuul | 02:28 | |
| *** ricky_summit has joined #zuul | 02:34 | |
| *** qwc has quit IRC | 02:35 | |
| *** qwc has joined #zuul | 02:42 | |
| *** ricky_summit has quit IRC | 03:03 | |
| *** bhavik1 has joined #zuul | 05:49 | |
| *** bhavik1 has quit IRC | 06:15 | |
| *** xinliang has quit IRC | 07:26 | |
| *** xinliang has joined #zuul | 07:43 | |
| *** hashar has joined #zuul | 07:45 | |
| *** xinliang has quit IRC | 08:17 | |
| *** xinliang has joined #zuul | 08:30 | |
| *** xinliang has quit IRC | 08:30 | |
| *** xinliang has joined #zuul | 08:30 | |
| *** hogepodge has quit IRC | 09:43 | |
| *** timrc has quit IRC | 09:43 | |
| *** jamielennox has quit IRC | 09:43 | |
| *** mordred has quit IRC | 09:43 | |
| *** rbergeron has quit IRC | 09:43 | |
| *** mrhillsman has quit IRC | 09:43 | |
| *** electrofelix has joined #zuul | 09:48 | |
| *** hogepodge has joined #zuul | 09:49 | |
| *** timrc has joined #zuul | 09:49 | |
| *** jamielennox has joined #zuul | 09:49 | |
| *** mordred has joined #zuul | 09:49 | |
| *** rbergeron has joined #zuul | 09:49 | |
| *** mrhillsman has joined #zuul | 09:49 | |
| *** lennyb has quit IRC | 10:48 | |
| *** hashar is now known as hasharAway | 10:50 | |
| *** lennyb has joined #zuul | 11:45 | |
| *** jkilpatr has joined #zuul | 12:32 | |
| *** xinliang has quit IRC | 12:39 | |
| *** xinliang has joined #zuul | 12:53 | |
| *** bhavik1 has joined #zuul | 12:57 | |
| *** xinliang has quit IRC | 13:19 | |
| *** xinliang has joined #zuul | 13:23 | |
| *** bhavik1 has quit IRC | 13:27 | |
| *** dkranz has joined #zuul | 14:15 | |
| *** Guest5156 has joined #zuul | 14:48 | |
| *** Guest5156 has quit IRC | 14:52 | |
| *** dkranz has left #zuul | 14:55 | |
| odyssey4me | I'm doing some work to implement nodepool v3 in our org for testing. We're still using jenkins for now and don't want to switch from jenkins to zuul just yet... but we definitely want to use nodepool. | 15:58 |
| odyssey4me | I have a working implementation up and am trying to understand how I can have jenkins consume nodes from nodepool given nodepool v3. | 15:58 |
| odyssey4me | In nodepool v2/v2.5 it appears that gearman and zeromq were involved... I need a little help connecting the dots if someone is able to help with that. In a previous chat with mordred there was mention made of there being a wish for a new jenkins plugin for nodepool v3... so I'm happy to help there, but would like a little help gaining enough understanding of what the integration needs to be to make that happen. | 16:01 |
| dmsimard | odyssey4me: in v2, zuul jenkins and nodepool communicate with each other through geard, yes | 16:15 |
| dmsimard | odyssey4me: review.rdoproject.org still uses a setup with zuul v2 with jenkins and nodepool | 16:15 |
| dmsimard | nodepool (v2) adds the nodes to jenkins as slaves (i.e, https://github.com/openstack-infra/nodepool/blob/master/nodepool/jenkins_manager.py ) | 16:16 |
| dmsimard | that code is gone from nodepool v3 | 16:17 |
| dmsimard | and then zuul allocates a job to run on a slave | 16:17 |
| dmsimard | if you don't run zuul, you could probably have logic to run jobs based on labels https://github.com/openstack-infra/nodepool/blob/master/nodepool/jenkins_manager.py#L54 | 16:18 |
| dmsimard | So basically nodepool would create a slave in jenkins with label, say, 'centos-7' (even though the slave is going to be name centos-7-34832987292) and then in your jenkins jobs, you restrict the jobs to run only on slaves with that particular label | 16:19 |
| dmsimard | I'm not sure what triggers the eventual deletion of the slave without a zuul involved, though | 16:20 |
| odyssey4me | well, if I understand it correctly all nodes are now in zookeeper and all nodepool's understanding of the node's state comes from there | 16:34 |
| odyssey4me | so I'm guessing that whatever zuul does to set node states in that inventory is the same that needs to be done by jenkins | 16:35 |
| *** jkilpatr has quit IRC | 16:41 | |
| dmsimard | odyssey4me: zookeeper came with nodepoolv3, if you use v3 there's no native integration for jenkins (yet) | 16:45 |
| dmsimard | nodepool v2, with jenkins support, does not use zookeper | 16:45 |
| dmsimard | zookeeper* | 16:45 |
| *** openstackgerrit has quit IRC | 16:48 | |
| *** jkilpatr has joined #zuul | 16:54 | |
| odyssey4me | dmsimard yes, that's the point - I want to build the integration so I'm trying to understand the flow of work and where jenkins would need to interface | 17:01 |
| dmsimard | ahhh :) | 17:02 |
| dmsimard | afraid I can't quite help you there | 17:02 |
| dmsimard | Shrews would be able to point you in the right direction | 17:03 |
| *** jkilpatr has quit IRC | 17:05 | |
| *** jkilpatr has joined #zuul | 17:18 | |
| odyssey4me | dmsimard it would appear that you helped me think in the right direction - it would seem that https://github.com/openstack-infra/zuul/blob/feature/zuulv3/zuul/nodepool.py is a good place to start | 17:32 |
| dmsimard | I was not useless /o/ | 17:36 |
| leifmadsen | huzzuh! | 17:50 |
| Shrews | odyssey4me: as far as <something> communicating with nodepool to request nodes, it only understands the simple request protocol somewhat outlined in the zuulv3 spec: http://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html | 17:52 |
| odyssey4me | Shrews yeah, I have that referenced too | 17:52 |
| Shrews | odyssey4me: that's pretty simple to implement. the jenkins side of it would have to be handled by a backend driver of sorts that you'd have to create | 17:53 |
| Shrews | see nodepool/drivers/* | 17:53 |
| Shrews | odyssey4me: but i've never actually done much with jenkins, so i've no idea what that would look like | 17:54 |
| odyssey4me | Shrews hmm, I hadn't thought of the driver interface | 17:54 |
| odyssey4me | I'm guessing it may make sense to do something similar to what there was before - have a jenkins plugin which publishes and subscribes events, then a driver to consume those | 17:55 |
| Shrews | odyssey4me: nodepool didn't really have a concept of "drivers" until just recently when tristanC (iirc) broke that code out. so it's still somewhat in development, i think | 17:55 |
| odyssey4me | the alternative would be to totally decouple it and work through zookeeper exclusively - jenkins <-> zookeeper <-> nodepool | 17:56 |
| Shrews | odyssey4me: i could see a jenkins plugin that speaks the zookeeper protocol, making the request/response handling | 17:56 |
| Shrews | that actually makes more sense in my mind, and doesn't require nodepool changes | 17:57 |
| odyssey4me | yeah, the only thing it requires is for the data model in zookeeper to be consistent | 17:57 |
| odyssey4me | that's easy enough to manage though - each plugin version is tested against a specific nodepool version | 17:59 |
| odyssey4me | and that's that | 17:59 |
| Shrews | odyssey4me: We don't do any sort of versioning of that. We've tried not to change any existing data values, but only add new ones as we see the need. Code just sort of looks for the data fields it needs, ignoring any irrelevant data. I'd imagine the plugin could do the same. | 18:00 |
| Shrews | jeblair's handling of that in the zuul data model is actually very flexible, and a good model to follow. | 18:00 |
| odyssey4me | yeah, good plan | 18:00 |
| Shrews | zuul/zk.py, fwiw | 18:01 |
| Shrews | oh, no. zuul/model.py, i think | 18:01 |
| *** openstackgerrit has joined #zuul | 18:02 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Add nodejs-npm jobs https://review.openstack.org/517808 | 18:02 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Collect npm-shrinkwrap for javascript https://review.openstack.org/517774 | 18:02 |
| Shrews | see the Node class for a good example | 18:02 |
| *** jkilpatr has quit IRC | 18:05 | |
| odyssey4me | Shrews dmsimard thank you for helping me coalesce the right bits - with a fresh head I'll revisit this tomorrow and see if I can knock up some sort of PoC/hack to validate it | 18:12 |
| *** jkilpatr has joined #zuul | 18:18 | |
| *** jkilpatr has quit IRC | 19:00 | |
| *** electrofelix has quit IRC | 19:11 | |
| *** jkilpatr has joined #zuul | 19:12 | |
| *** jkilpatr_ has joined #zuul | 20:32 | |
| *** jkilpatr has quit IRC | 20:35 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/nodepool feature/zuulv3: Document security group https://review.openstack.org/518620 | 21:19 |
| mordred | openstackgerrit: morning! | 21:22 |
| mordred | odyssey4me, Shrews: https://etherpad.openstack.org/p/zuulv3-jenkins-integration is the etherpad I wrote up with thoughts on possibilities for jenkins integratoin | 21:22 |
| dmsimard | mordred: I see a use case for using jenkins with nodepool (without zuul), but using nodepool+zuul+jenkins ?? | 21:24 |
| dmsimard | (especially with v3) | 21:25 |
| mordred | dmsimard: yes - I think both things are valid use cases ... | 21:29 |
| mordred | dmsimard: for one, an org could keep one nodepool that is shared between a zuul and a jenkins - or between several jenkinses | 21:29 |
| dmsimard | mordred: so s/zuul-executor/jenkins/ ? | 21:29 |
| dmsimard | oh, in that sense | 21:29 |
| mordred | dmsimard: but also it's possible there is a jenkins pipeline something job that someone really needs to keep - so having a zuul job that can trigger it seems perfectly valid too | 21:30 |
| mordred | also - stepwise migratoin :) | 21:30 |
| *** jkilpatr_ has quit IRC | 21:32 | |
| *** jkilpatr has joined #zuul | 21:57 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!