| *** jkilpatr has quit IRC | 00:22 | |
| *** jasondotstar has quit IRC | 02:02 | |
| *** jasondotstar has joined #zuul | 02:03 | |
| *** jasondotstar has quit IRC | 02:19 | |
| *** jasondotstar has joined #zuul | 02:20 | |
| *** jasondotstar has quit IRC | 02:22 | |
| *** jasondotstar has joined #zuul | 02:22 | |
| *** jasondotstar has quit IRC | 02:27 | |
| *** jasondotstar has joined #zuul | 02:27 | |
| *** jasondotstar has quit IRC | 02:28 | |
| *** jasondotstar has joined #zuul | 02:28 | |
| *** jasondotstar has quit IRC | 02:30 | |
| *** jasondotstar has joined #zuul | 02:30 | |
| *** jasondotstar has quit IRC | 02:31 | |
| *** jasondotstar has joined #zuul | 02:31 | |
| *** jasondotstar has quit IRC | 02:39 | |
| *** jasondotstar has joined #zuul | 02:39 | |
| *** jasondotstar has quit IRC | 02:45 | |
| *** threestrands_ has joined #zuul | 02:49 | |
| *** threestrands_ has quit IRC | 02:49 | |
| *** threestrands_ has joined #zuul | 02:49 | |
| *** threestrands_ has quit IRC | 02:50 | |
| *** threestrands has quit IRC | 02:50 | |
| *** threestrands_ has joined #zuul | 02:50 | |
| *** threestrands_ has quit IRC | 02:50 | |
| *** threestrands_ has joined #zuul | 02:50 | |
| *** jasondotstar has joined #zuul | 03:24 | |
| *** jasondotstar has quit IRC | 03:44 | |
| *** jasondotstar has joined #zuul | 03:45 | |
| *** jasondotstar has quit IRC | 03:45 | |
| *** jasondotstar has joined #zuul | 03:45 | |
| *** bhavik1 has joined #zuul | 04:35 | |
| *** jaianshu has joined #zuul | 04:57 | |
| *** jasondotstar has quit IRC | 05:11 | |
| *** jasondotstar has joined #zuul | 05:12 | |
| *** jasondotstar has quit IRC | 05:16 | |
| *** jasondotstar has joined #zuul | 05:35 | |
| *** jasondotstar has quit IRC | 05:39 | |
| *** jaianshu has quit IRC | 06:11 | |
| *** jaianshu has joined #zuul | 06:11 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Reorg non detailed instance listing columns https://review.openstack.org/522103 | 06:27 |
|---|---|---|
| tobiash | clarkb: fixed your change ^^ | 06:28 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Improve test case node_assignment_at_quota https://review.openstack.org/506134 | 06:41 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Add cloud quota handling https://review.openstack.org/503838 | 06:41 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Don't fail on quota exceeded https://review.openstack.org/503051 | 06:41 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Make max-servers optional https://review.openstack.org/504282 | 06:42 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Support cores limit per pool https://review.openstack.org/504283 | 06:42 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Support ram limit per pool https://review.openstack.org/504284 | 06:42 |
| jaianshu | Hello everyone.\ | 06:58 |
| jaianshu | when zuul-schedulor tries to communicate to nodepool...i | 06:59 |
| jaianshu | i'm getting Received error(xid=1) NoNodeError() .. | 06:59 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Reorg non detailed instance listing columns https://review.openstack.org/522103 | 06:59 |
| *** saop has joined #zuul | 07:00 | |
| saop | hello everyone | 07:01 |
| saop | Just installed zuulv3 setup. My scheduler is not seem to communicating with zuul executor, its not able to clone the git repo | 07:03 |
| saop | I have zuul.d in my repo | 07:04 |
| saop | and it have all basic config files | 07:04 |
| saop | there is no devstack thing though I am creating setup for personal CI project | 07:04 |
| saop | I have zookeeper and Nodepool also | 07:05 |
| saop | though nodepool image list is empty | 07:05 |
| saop | did i am missing anything? | 07:05 |
| SpamapS | saop: hi | 07:12 |
| SpamapS | saop: first, thanks for being brave enough to try out zuulv3. :) | 07:13 |
| SpamapS | saop: when you start up zuul-executor and zuul-scheduler, you should see zuul-scheduler fetching all the branches for all of the projects listed in your tenant configuration. | 07:14 |
| saop | SpamapS, yes i can see | 07:14 |
| SpamapS | saop: or more accurately, you should see zuul-scheduler submitting jobs that the executor executes by fetching all the branches. | 07:14 |
| SpamapS | saop: so are those merge jobs failing the clones? | 07:15 |
| saop | SpamapS, there is no clones | 07:15 |
| saop | SpamapS, i submit the patch for test | 07:15 |
| saop | SpamapS, its showing in scheduler all ref number also but on the same time executor is idle | 07:16 |
| SpamapS | saop: Before you submit any patches, the executor will have to clone all the repos to read configuration. | 07:16 |
| saop | SpamapS, that it do | 07:16 |
| SpamapS | saop: Ok, so are you certain it loaded all of the pipelines from your config-project repo(s)? | 07:16 |
| saop | SpamapS, it says Updating repository /var/lib/zuul/git/<>/repo | 07:17 |
| SpamapS | saop: after all of those merge jobs complete, the scheduler should log a summary of the configuration. | 07:17 |
| saop | SpamapS, i got only two events which are, patchset-created and ref-updated | 07:19 |
| saop | SpamapS, there is no check pipeline event | 07:19 |
| saop | SpamapS, i didnt see any error also | 07:20 |
| SpamapS | saop: do you have something like this in your scheduler log: | 07:21 |
| SpamapS | http://paste.openstack.org/show/627408/ | 07:21 |
| SpamapS | (mine is github based, but yours may be gerrit based, they should be relatively similar) | 07:21 |
| saop | SpamapS, no | 07:21 |
| SpamapS | Ok, then your executor isn't communicating with your source. | 07:22 |
| saop | SpamapS, zuul.pipeline is not there | 07:22 |
| saop | SpamapS, okay | 07:22 |
| saop | SpamapS, so how to correct it | 07:22 |
| SpamapS | saop: can you pastebin the executor log? | 07:22 |
| SpamapS | should be lots of things like this: | 07:23 |
| SpamapS | http://paste.openstack.org/show/627409/ | 07:23 |
| *** saop has quit IRC | 07:24 | |
| SpamapS | doh | 07:25 |
| *** saop has joined #zuul | 07:29 | |
| saop | SpamapS, Executor logs: http://paste.openstack.org/show/627410/ | 07:37 |
| SpamapS | saop: ok, that's good. | 07:37 |
| SpamapS | saop: do you have base setup as a config-project ? | 07:37 |
| saop | SpamapS, but it dont change | 07:38 |
| saop | SpamapS, it will be same no matter how many patch you raise | 07:38 |
| saop | SpamapS, no activity i found in executor side | 07:38 |
| SpamapS | saop: Unless you have pipelines in the 'base' repo, (which must be in your tenant config as a 'config-project'), your Zuul won't know what to do with events. | 07:40 |
| SpamapS | saop: can you paste your tenant configuration? | 07:40 |
| saop | SpamapS, you mean main.yaml file? | 07:40 |
| saop | SpamapS, tenant config: http://paste.openstack.org/show/627411/ | 07:42 |
| SpamapS | saop: it can be named anything, there's no default, 'tenant_config' from zuul.conf | 07:42 |
| SpamapS | saop: ok, so you SHOULD get a tenant configuration block on zuul-scheduler startup. | 07:42 |
| SpamapS | saop: if you restart zuul-scheduler, it's logged at INFO level | 07:42 |
| SpamapS | I have to go afk for a bit. If you are getting those merge jobs completing, then you should get a pipeline config. If that's happening, then you need to make sure your user can see all the events. That's where my knowledge gets fuzzy, because I am more versed in Github than Gerrit. :-P | 07:43 |
| saop | SpamapS, thanks | 07:44 |
| saop | Do we need nodepool to communicate with zuul executor? | 07:58 |
| SpamapS | saop: nodepool only communicates with zookeeper and openstack. | 07:59 |
| SpamapS | saop: the zuul scheduler communicates with the same zookeeper to request nodes and release them. | 07:59 |
| SpamapS | saop: the executor is given nodes by the scheduler | 07:59 |
| saop | SpamapS, so if scheduler don't give any node then | 08:00 |
| saop | SpamapS, is executor dont clone repo? | 08:00 |
| SpamapS | if your jobs use nodesets, then no jobs will run | 08:00 |
| saop | saop, i dont use nodesets | 08:00 |
| SpamapS | if you have nodeless jobs, they'll run fine without nodepool. | 08:01 |
| saop | SpamapS, okay thanks, | 08:01 |
| saop | SpamapS, can you give me any points, now how would i debug whats wrong with my zuul scheduler? | 08:01 |
| saop | SpamapS, why its not talking to executor? | 08:02 |
| saop | SpamapS, on the same time if i uninstalled zuulv3 and installed zuul v2 it works like charm | 08:02 |
| saop | SpamapS, whats the hack in zuulv3? | 08:02 |
| SpamapS | saop: can you paste your scheduler log? like, restart it, and then paste exactly what comes out from startup -> quiet ? | 08:03 |
| saop | SpamapS, okay | 08:03 |
| SpamapS | because your executor actually seems to be talking to gerrit just fine | 08:03 |
| SpamapS | saop: also maybe paste contents of zuul.d if you can. | 08:03 |
| *** hashar has joined #zuul | 08:04 | |
| saop | SpamapS, scheduler logs: http://paste.openstack.org/show/627418/ | 08:05 |
| SpamapS | um, you don't need to run it as root. | 08:06 |
| saop | SpamapS, we tried without it also | 08:08 |
| saop | SpamapS, same thing | 08:08 |
| SpamapS | saop: running as root won't break it, but yeah, don't do that. ;) | 08:08 |
| SpamapS | anyway, | 08:08 |
| saop | SpamapS, zuul.d logs: http://paste.openstack.org/show/627420/ | 08:08 |
| SpamapS | your config isn't loading | 08:08 |
| saop | SpamapS, okay | 08:08 |
| saop | SpamapS, how to solve? | 08:08 |
| SpamapS | saop: Not 100% sure just yet... looking at your zuul.d | 08:10 |
| SpamapS | hm | 08:11 |
| SpamapS | 2017-11-27 03:08:40,704 DEBUG zuul.TenantParser: Cat job <gear.Job 0x7fcfb8121a58 handle: b'H:127.0.0.1:1' name: merger:cat unique: f6af241c5b8e4a35ad4 | 08:11 |
| SpamapS | 93161209de1ba> got files dict_keys(['zuul.yaml', '.zuul.yaml']) | 08:11 |
| SpamapS | saop: perhaps try catting zuul.d/* into zuul.yaml in the base repo. | 08:12 |
| SpamapS | saop: I don't see why it would only load zuul.yaml, but.. maybe config projects are different | 08:12 |
| saop | SpamapS, yeah i saw that and i tried also using zuul.yaml only but still it was the same | 08:13 |
| SpamapS | saop: hm I'm stumped then. I can see where the cat job comes back, so it should have all the files needed. | 08:14 |
| SpamapS | 09de1ba> with data {'dirs': ['zuul.d', '.zuul.d'], 'branch': 'master', 'files': ['zuul.yaml', '.zuul.yaml'], 'project': 'base', 'connection': 'gerri | 08:15 |
| SpamapS | that should load | 08:15 |
| SpamapS | saop: and your config looks like it should parse. | 08:16 |
| saop | SpamapS, okay | 08:16 |
| SpamapS | it even says "Got files [zuul.yaml, .zuul.yaml]" | 08:17 |
| saop | SpamapS, but i dont have those files | 08:18 |
| saop | SpamapS, i once tried with zuul.d and then again with zuul.yaml | 08:18 |
| SpamapS | weird because the next line should be "Loading configuration from zuul.yaml" | 08:19 |
| SpamapS | saop: yeah if you have both I believe zuul explodes because it doesn't want to choose between conflicting configs. | 08:19 |
| SpamapS | ah no it doesn't explode it just warns that there are multiple | 08:20 |
| SpamapS | well anyway | 08:22 |
| SpamapS | it's time for me to sleep | 08:22 |
| saop | SpamapS, thanks by the way | 08:23 |
| SpamapS | saop: Sorry I couldn't help. There is likely something small going wrong. You look to have the right idea. | 08:23 |
| saop | SpamapS, i will try :) | 08:23 |
| SpamapS | good luck | 08:23 |
| * SpamapS sleeps | 08:23 | |
| jaianshu | Hello All | 08:24 |
| jaianshu | i face similar config issue where zuul scheduler is not able to load the zuul.yaml and "zuul.Pipeline.{tenant-name}.check" event is not generated for me. | 08:27 |
| *** jaianshu_ has joined #zuul | 08:34 | |
| *** flepied__ has joined #zuul | 08:35 | |
| *** jaianshu has quit IRC | 08:37 | |
| *** flepied_ has quit IRC | 08:38 | |
| *** bhavik1 has quit IRC | 09:04 | |
| tobiash | saop: you need a base job: http://paste.openstack.org/show/627426/ | 09:51 |
| tobiash | jaianshu_: do you have logs? | 09:51 |
| saop | tobiash, hello base job means? | 09:52 |
| saop | tobiash, in my jobs.yaml i have job name merge | 09:52 |
| tobiash | zuul3 has job inheritance | 09:53 |
| saop | tobiash, okay | 09:53 |
| tobiash | and this must be rooted at a job | 09:53 |
| tobiash | we call this job base job | 09:53 |
| rcarrillocruz | saop: https://github.com/openstack-infra/project-config/blob/master/zuul.d/jobs.yaml#L27 | 09:53 |
| tobiash | it sets parent explicitly to null | 09:54 |
| rcarrillocruz | something along those lines | 09:54 |
| tobiash | and can only be defined in a config project | 09:54 |
| tobiash | a base job typically does things like pushing the repos to the node and log collection | 09:54 |
| saop | tobiash, please look at my zuul.d http://paste.openstack.org/show/627420/ | 09:54 |
| saop | tobiash, i am little confused now | 09:54 |
| saop | tobiash, i have project name 'base' job name 'merge' | 09:55 |
| saop | tobiash, do you mean i need to make project name and job name same | 09:55 |
| tobiash | saop: http://paste.openstack.org/show/627429/ | 09:55 |
| tobiash | saop: if a job doesn't define a parent it implicitly inherits from a job called 'base' | 09:56 |
| tobiash | you didn't define this job | 09:56 |
| tobiash | it has nothing to do with your project 'base' | 09:56 |
| tobiash | you probably want to call it differently ;) | 09:56 |
| saop | tobiash, thanks a lot | 09:57 |
| saop | tobiash, i will run a test | 09:57 |
| saop | tobiash, let see it works :) | 09:57 |
| tobiash | saop: you also should read this: https://docs.openstack.org/infra/zuul/user/config.html#job | 09:57 |
| tobiash | there the job inheritance etc is described in detail | 09:58 |
| saop | tobiash, okay | 09:58 |
| saop | tobiash, one more question, can i put my zuul.d in /etc/zuul? | 10:00 |
| tobiash | saop: no, this is only read from the projects referred to in the tenant config file | 10:01 |
| rcarrillocruz | saop: if it helps you, this is a superminimal repo containing a config project | 10:08 |
| rcarrillocruz | https://github.com/rcarrillocruz-org/common-config | 10:08 |
| rcarrillocruz | openstack project-config can be daunting if you're new to zuul | 10:09 |
| saop | tobiash, rcarrillocruz I tried this now but still the same | 10:14 |
| saop | tobiash, no luck | 10:14 |
| tobiash | saop: can you collect the logs of a complete startup of both, zuul-scheduler and zuul-executor? | 10:15 |
| saop | tobiash, yeah | 10:15 |
| tobiash | saop: you run an executor don't you? | 10:15 |
| saop | tobiash, yes | 10:15 |
| tobiash | great | 10:15 |
| saop | tobiash, wait i will paste | 10:15 |
| saop | tobiash, scheduler logs: http://paste.openstack.org/show/627436/ http://paste.openstack.org/show/627438/ | 10:31 |
| saop | tobiash, executor logs: http://paste.openstack.org/show/627439/ | 10:32 |
| tobiash | saop: how does your tenant config look like? | 10:41 |
| tobiash | to me it looks like it just didn't find project configs | 10:42 |
| saop | tobiash, http://paste.openstack.org/show/627454/ | 10:43 |
| tobiash | saop: and what's the directory structure in the ironman repo (current master branch on gerrit)? | 10:44 |
| saop | tobiash, master branch | 10:45 |
| saop | tobiash, its just a test repo | 10:45 |
| tobiash | are there multiple matches of [zuul.yaml, .zuul.yaml, zuul.d, .zuul.d] ? | 10:45 |
| tobiash | you should only have one of these in there | 10:45 |
| saop | tobiash, zuul.d we created | 10:45 |
| saop | tobiash, it have all three jobs.yaml pipelines.yaml and projects.yaml | 10:45 |
| saop | tobiash, it only have zuul.d | 10:46 |
| tobiash | saop: and you're sure you have this upstream on the master branch? | 10:48 |
| tobiash | that looks really strange | 10:48 |
| tobiash | a config project is not speculative so it must be already merged into master and not be still in code review | 10:49 |
| jaianshu_ | tobiash, this ironman repo is created on a gerrit setup created y us only | 10:49 |
| tobiash | with upstream I mean is it merged on your gerrit? ;) | 10:50 |
| jaianshu_ | tobiash, zuul.d is not merged | 10:50 |
| tobiash | it needs to be merged | 10:50 |
| tobiash | zuul reads the master branch of a config repo | 10:50 |
| jaianshu_ | tobiash, ok. | 10:51 |
| tobiash | if zuul.d is not part of the current master branch on your gerrit it will have no config | 10:51 |
| *** jasondotstar has joined #zuul | 10:54 | |
| *** electrofelix has joined #zuul | 10:55 | |
| *** jasondotstar has quit IRC | 10:56 | |
| *** jasondotstar has joined #zuul | 10:56 | |
| *** ianw has quit IRC | 11:03 | |
| *** ianw_ has joined #zuul | 11:03 | |
| *** ianw_ is now known as ianw | 11:04 | |
| jaianshu_ | tobiash, thank you...it worked after that... i'm able to see the cloning working | 11:32 |
| tobiash | :) | 11:33 |
| jaianshu_ | tobiash, i'm not getting one issue while running ansible job - FileNotFoundError: [Errno 2] No such file or directory: 'bwrap' | 11:33 |
| tobiash | jaianshu_: you need to install bubblewrap | 11:34 |
| tobiash | this is used to sandbox the ansible runs | 11:34 |
| jaianshu_ | ah. | 11:35 |
| rcarrillocruz | jaianshu_: if you're on xenial, https://launchpad.net/~openstack-ci-core/+archive/ubuntu/bubblewrap | 11:36 |
| *** threestrands_ has quit IRC | 11:42 | |
| *** jasondotstar has quit IRC | 11:44 | |
| *** jasondotstar has joined #zuul | 11:44 | |
| *** jasondotstar has quit IRC | 11:49 | |
| *** jasondotstar has joined #zuul | 11:49 | |
| *** mattclay has quit IRC | 11:53 | |
| *** patrickeast has quit IRC | 11:53 | |
| *** kmalloc has quit IRC | 11:54 | |
| *** kklimonda has quit IRC | 11:54 | |
| *** zaro has quit IRC | 11:54 | |
| *** mattclay has joined #zuul | 11:54 | |
| *** zaro has joined #zuul | 11:55 | |
| *** patrickeast has joined #zuul | 11:55 | |
| *** kklimonda has joined #zuul | 11:56 | |
| *** kmalloc has joined #zuul | 11:56 | |
| *** jkilpatr has joined #zuul | 12:13 | |
| *** jkilpatr has quit IRC | 12:14 | |
| *** jkilpatr has joined #zuul | 12:14 | |
| jaianshu_ | rcarrillocruz, thanks for link..passed that error. Now I see that thouch i have defined hosts in ansible still its not able to find any... " Ansible output: b' [WARNING]: No hosts matched, nothing to do' | 12:15 |
| rcarrillocruz | if you don't have a nodepool hooked up to zuul | 12:17 |
| rcarrillocruz | there will be no nodes created for jobs | 12:17 |
| rcarrillocruz | meaning | 12:17 |
| rcarrillocruz | you'll just have 'localhost' | 12:17 |
| rcarrillocruz | which is the zuul executor | 12:17 |
| rcarrillocruz | if you want to test your zuul is functional | 12:17 |
| rcarrillocruz | you can put as job 'noop' | 12:17 |
| rcarrillocruz | and should do 'something' | 12:17 |
| rcarrillocruz | without requiring a nodepool | 12:17 |
| *** saop has quit IRC | 12:32 | |
| *** draculax has joined #zuul | 12:32 | |
| draculax | whois openstack | 12:33 |
| draculax | whois draculax | 12:33 |
| *** draculax has quit IRC | 12:34 | |
| *** lennyb_ has joined #zuul | 12:49 | |
| *** jkilpatr has quit IRC | 13:11 | |
| *** jkilpatr has joined #zuul | 13:29 | |
| *** jaianshu_ has quit IRC | 13:49 | |
| Shrews | ".ical is not updated for Neutron Drivers meeting" <-- is this srsly a zuul issue? | 13:49 |
| Shrews | also, what is zuul? | 13:50 |
| Shrews | also... hi everybody! | 13:50 |
| tobiash | hi Shrews :) | 13:55 |
| tobiash | where is this listed? | 13:55 |
| Shrews | tobiash: zuul-issues etherpad | 14:10 |
| tobiash | ah | 14:11 |
| *** maxamillion has quit IRC | 14:42 | |
| *** dkranz has joined #zuul | 14:51 | |
| *** maxamillion has joined #zuul | 14:57 | |
| clarkb | its a zuul ve job issue yes | 15:38 |
| clarkb | since a job produces and publishes that data | 15:38 |
| *** xinliang has quit IRC | 16:14 | |
| *** xinliang has joined #zuul | 16:28 | |
| *** xinliang has quit IRC | 16:28 | |
| *** xinliang has joined #zuul | 16:28 | |
| *** xinliang has quit IRC | 16:34 | |
| *** xinliang has joined #zuul | 16:39 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: WIP: Fix broken use of pre-existing cloud images https://review.openstack.org/498050 | 16:56 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Use py3 in docs generation https://review.openstack.org/523169 | 17:05 |
| *** sshnaidm has joined #zuul | 17:34 | |
| sshnaidm | hi, is there RFE for support ansible tags in zuul? | 17:35 |
| SpamapS | sshnaidm: No, it has been discussed. For now, you'll want to boil things down into smaller and smaller includes and use variables. | 17:35 |
| SpamapS | includes/roles I should say | 17:36 |
| sshnaidm | SpamapS, ack | 17:36 |
| jeblair | fyi i'm cleaning up the v3 board in preparation for moving the roadmap into storyboard | 17:37 |
| SpamapS | nice. I haven't looked at storyboard in a while. I am sure there are things that we did that can be closed. | 17:38 |
| jeblair | yep | 17:38 |
| sshnaidm | SpamapS, external vars and external vars files are not gonna be supported too? | 17:39 |
| SpamapS | sshnaidm: you can pass vars in via the job def | 17:40 |
| SpamapS | sshnaidm: and if they're sensitive, via the secrets mechanism. | 17:40 |
| jeblair | sshnaidm: iiuc your use case is a large collection of static vars, like "when running in this configuration, use these 100 values" | 17:40 |
| sshnaidm | a few hundreds lines currently | 17:41 |
| sshnaidm | and if we make a lot of small includes, I need to use vars_files in every include too (in case we use vars_files in playbook) | 17:42 |
| pabelanger | you could maybe try moving your variables file into hosts_vars / group_vars direct next to playbooks. Ansible will then autoload them | 17:44 |
| pabelanger | don't help with --tags | 17:44 |
| sshnaidm | pabelanger, we load sometimes different files depending on job | 17:45 |
| sshnaidm | pabelanger, so it seems like I need to configure what to load in job.vars and then to add vars_files: {{ config }} in every include of playbook | 17:46 |
| sshnaidm | either somebody has better ideas | 17:46 |
| *** harlowja has joined #zuul | 17:47 | |
| pabelanger | I'm sure there is another way to do it, but would need to look at big picture of playbooks. for things I've done, I've opted to use different hosts but same ansible_host=<ip> to node, that allowed some better control on which variables would be loaded when ansible-playbooks ran | 17:47 |
| pabelanger | I opted that approach over using different switches to ansible-playbook on CLI | 17:48 |
| jeblair | sshnaidm: yeah, i think that's the closest we can come for now. i agree this is a good use case for either additional vars files on command line or tags. i originally read your request as being to 'load vars from files into zuul' but when i read it as 'add more -e' command line options i agree it fits in better with the intent for zuul. | 17:50 |
| jeblair | sshnaidm: we'll need to be extra careful about that though because of variable precedence in ansible. there may be security issues, and if so, we may not be able to implement it. | 17:50 |
| jeblair | sshnaidm: tags still seems pretty straightforward though. | 17:51 |
| sshnaidm | jeblair, yeah, I think if you set all secrets as last extra-vars, it will override everything before | 17:52 |
| sshnaidm | so nobody could override them | 17:52 |
| pabelanger | thought you cannot override extra-vars | 17:53 |
| pabelanger | so, secrets would need to be first I think | 17:53 |
| jeblair | sshnaidm, pabelanger: https://docs.openstack.org/infra/zuul/feature/zuulv3/user/jobs.html#variables is a delicate balance we need not to upset :) | 17:53 |
| sshnaidm | pabelanger, you can override extra-vars with next extra-vars :) | 17:53 |
| jeblair | if we can fit it into there, it may work | 17:53 |
| pabelanger | jeblair: agree | 17:54 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: WIP: Fix broken use of pre-existing cloud images https://review.openstack.org/498050 | 18:00 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: WIP: Fix broken use of pre-existing cloud images https://review.openstack.org/498050 | 18:09 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Fix broken use of pre-existing cloud images https://review.openstack.org/498050 | 18:29 |
| Shrews | jeblair: mordred: rebased/fixed/added test ^^^, and should hopefully pass tests now | 18:32 |
| Shrews | leifmadsen: although i agree with your comments on 498050, they're outside the scope of the change. Feel free to submit a separate change for those if you like. | 18:33 |
| Shrews | our docs definitely could use some cleanup | 18:33 |
| leifmadsen | huh, was commenting on areas that were being changed | 18:33 |
| leifmadsen | at least according to the diff | 18:33 |
| leifmadsen | do as you wish though | 18:33 |
| Shrews | leifmadsen: not sure what you're seeing. nothing about auto-floating-ip is being changed | 18:34 |
| leifmadsen | .shrug | 18:34 |
| Shrews | leifmadsen: maybe you're seeing changes due to the rebase? *shrug x2* | 18:34 |
| leifmadsen | no idea | 18:34 |
| leifmadsen | :) | 18:34 |
| jeblair | let's see if that's the case | 18:35 |
| jeblair | i'd hate to miss something or have leif feel ignored :) | 18:35 |
| leifmadsen | don't worry about me :) | 18:35 |
| leifmadsen | just working on other things before kubecon, so don't really have time to do much else | 18:35 |
| Shrews | pretty sure it's the rebase. seeing that diff when comparing PS 4 & 10 | 18:36 |
| jeblair | and 4..7 which is where leif left the comments | 18:36 |
| leifmadsen | yea, I was comparing 6 to 7 | 18:37 |
| jeblair | leifmadsen: ah yep. so patchset 7 includes a rebase and gerrit shows that as a diff | 18:37 |
| jeblair | er, shows that in the diff | 18:37 |
| jeblair | but if you switch to showing "base" to 7 | 18:37 |
| jeblair | then you'll see that it's not actually in the change itself | 18:38 |
| jeblair | which explains the confusion | 18:38 |
| jeblair | after spending 6 years with this, we've basically determined that there's no perfect way to deal with this situation (git makes interdiff hard) :/ | 18:38 |
| *** jasondotstar has quit IRC | 18:39 | |
| tobiash | yes, best thing is to avoid rebases if possible and use the same base for subsequent change versions | 18:40 |
| tobiash | of course this is not possible if there are merge conflicts | 18:41 |
| Shrews | yep. i wanted to avoid the rebase, but that review was pretty old | 18:43 |
| leifmadsen | also, it didn't work for me on patch 6, had to go back to patch 1 | 18:44 |
| leifmadsen | based on the tested was doing with jeblair in latest doc session | 18:44 |
| jeblair | next best thing i've found is to do the rebase as it's own patchset, and then leave a comment about that. but you have to think ahead for that, and it's extra work. | 18:44 |
| dmsimard | Oh, btw, I found the issue with the ara sqlite middleware | 18:54 |
| dmsimard | It's a mess of rewrite rules between htmlify and ara | 18:54 |
| dmsimard | If I disable the htmlify rewrite rules, everything works just fine .. so need to figure that out. | 18:54 |
| *** jasondotstar has joined #zuul | 18:56 | |
| *** electrofelix has quit IRC | 19:04 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: WIP: Add command socket support to zuul-merger https://review.openstack.org/523197 | 19:11 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Use py3 in docs generation https://review.openstack.org/523169 | 19:21 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Add command socket support to zuul-merger https://review.openstack.org/523197 | 20:07 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Move send_command() into ZuulApp https://review.openstack.org/523211 | 20:07 |
| pabelanger | jeblair: are you okay with command_socket as merger config setting in zuul.conf? If naming is fine, I'll refactor others processes to use the same idea | 20:08 |
| *** jasondotstar has quit IRC | 20:11 | |
| Shrews | jeblair: ooooh, i think i might have a theory on the finger daemon going away bit... | 20:26 |
| Shrews | jeblair: "Out of memory: Kill process 20077 (zuul-executor) score 10 or sacrifice child" | 20:27 |
| Shrews | jeblair: those last two words of that message | 20:27 |
| Shrews | jeblair: when i look at the current executors, the main zuul-executor proc is the one using most of the memory, not the child | 20:27 |
| Shrews | jeblair: i'm guessing the child (finger daemon) is being sacrificed for the "least amount of damage" to the system here | 20:28 |
| *** jasondotstar has joined #zuul | 20:29 | |
| jeblair | Shrews: ++! | 20:29 |
| jeblair | Shrews: do we need some more ram headroom? | 20:30 |
| Shrews | i guess one option is to split out the finger daemon to a solo process, but then i fear that the main zuul-executor is going to be the one killed | 20:30 |
| Shrews | jeblair: or do your thing | 20:30 |
| Shrews | what that headroom amount would be, i do not know | 20:31 |
| *** threestrands_ has joined #zuul | 20:32 | |
| *** threestrands_ has quit IRC | 20:32 | |
| *** threestrands_ has joined #zuul | 20:32 | |
| *** threestrands_ has quit IRC | 20:33 | |
| jeblair | Shrews: we have fine-grained load average graphs in graphite for executors, but not ram. cacti has coarse-grained ram. it might be possible to correlate cacti data with the oom kills. if not, we may want to add ram to the executor statsd code so we get fine-grained graphs for that. | 20:33 |
| *** threestrands_ has joined #zuul | 20:34 | |
| jeblair | http://grafana.openstack.org/dashboard/db/zuul-status | 20:34 |
| *** threestrands has joined #zuul | 20:34 | |
| *** threestrands has quit IRC | 20:34 | |
| *** threestrands has joined #zuul | 20:34 | |
| jeblair | http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=64003&rra_id=all | 20:34 |
| Shrews | http://cacti.openstack.org/cacti/graph.php?action=zoom&local_graph_id=64003&rra_id=2&view_type=&graph_start=1511395936&graph_end=1511397284&graph_height=120&graph_width=500&title_font_size=10 | 20:41 |
| Shrews | gah... nm. wrong machine | 20:41 |
| Shrews | http://cacti.openstack.org/cacti/graph.php?action=zoom&local_graph_id=64203&rra_id=2&view_type=&graph_start=1511396325&graph_end=1511397503&graph_height=120&graph_width=500&title_font_size=10 | 20:44 |
| Shrews | Nov 23 00:26:22 ze04 kernel: [8494719.679925] Out of memory: Kill process 20077 (zuul-executor) score 10 or sacrifice child | 20:44 |
| Shrews | that kill message corresponds to that graph. looks like we were pretty low on free mem | 20:44 |
| *** threestrands_ has quit IRC | 20:44 | |
| Shrews | we'd still be guessing at how much additional memory zuul would eat up if we gave it more | 20:48 |
| Shrews | nom nom nom nom | 20:48 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: Move to dictionary list of projects zuul._projects (take 2) https://review.openstack.org/518815 | 20:53 |
| *** flepied_ has joined #zuul | 21:23 | |
| *** flepied__ has quit IRC | 21:26 | |
| *** dkranz has quit IRC | 21:31 | |
| jhesketh | o/ | 21:56 |
| jeblair | it's zuul meeting time in #openstack-meeting-alt | 22:00 |
| pabelanger | o/ | 22:01 |
| dmsimard | btw pro tip if you're not familiar with storyboard, if you click on the star at the top of a story it'll send you email notifications when things happen in them.. and if you click the start at the top of a project, it'll send you email notifications for that entire project | 22:30 |
| *** smyers has quit IRC | 22:36 | |
| *** smyers has joined #zuul | 22:38 | |
| *** jasondotstar has quit IRC | 22:41 | |
| *** jasondotstar has joined #zuul | 23:00 | |
| *** hashar has quit IRC | 23:07 | |
| *** flepied_ has quit IRC | 23:09 | |
| *** flepied_ has joined #zuul | 23:11 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!