| dmsimard | grep -I --color='auto' -P "[\x80-\xFF]" * -R | 00:00 |
|---|---|---|
| dmsimard | I meant to add it as a local global pre-commit hook but never got around to it | 00:00 |
| jamielennox | has anyone seen errors like: | 00:11 |
| jamielennox | 2017-09-04 22:26:02,098 DEBUG zuul.AnsibleJob: [build: 99c01a347def47b6988518d9761ce66b] Ansible output: b' [WARNING]: provided hosts list is empty, only localhost is available | 00:11 |
| jamielennox | ' | 00:11 |
| jamielennox | 2017-09-04 22:26:02,400 DEBUG zuul.AnsibleJob: [build: 99c01a347def47b6988518d9761ce66b] Ansible output: b' [WARNING]: Failure using method (v2_playbook_on_play_start) in callb | 00:11 |
| jamielennox | ack plugin' | 00:11 |
| jamielennox | 2017-09-04 22:26:02,401 DEBUG zuul.AnsibleJob: [build: 99c01a347def47b6988518d9761ce66b] Ansible output: b'(<ansible.plugins.callback.zuul_stream.CallbackModule object at' | 00:12 |
| jamielennox | 2017-09-04 22:26:02,401 DEBUG zuul.AnsibleJob: [build: 99c01a347def47b6988518d9761ce66b] Ansible output: b"0x7f5aee225f60>): 'NoneType' object has no attribute 'values'" | 00:12 |
| jamielennox | best guess from that is that http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/ansible/callback/zuul_stream.py?h=feature/zuulv3#n173 | 00:13 |
| jamielennox | _variable_manager._hostvars is None. I do have a play that targets localhost, but afaik that should be ok | 00:14 |
| tristanC | dmsimard: to avoid non-breaking space, you can use this xmodmap -e "keycode 65 = space space space space underscore underscore space space" | 00:15 |
| dmsimard | tristanC: what's that do ? | 00:16 |
| tristanC | dmsimard: by default, xorg bind those weird space character at alt-space or altgr-space iirc | 00:16 |
| dmsimard | yeah that's the one | 00:17 |
| tristanC | dmsimard: this modmap make sure that when you press space it doesn't send non-breakable space | 00:17 |
| dmsimard | I tend to produce that whitespace around pipes which I press alt+gr + tilde for (below esc key) | 00:17 |
| dmsimard | and since there is a space before and after the pipe, sometimes I don't depress the alt key fast enough | 00:17 |
| dmsimard | tristanC: is that persisted or do I need to save it/load it somewhere ? | 00:19 |
| tristanC | dmsimard: you'll have to run it at starttime, e.g. in your .xinitrc | 00:19 |
| jamielennox | oh - wait, somehow zuul is generating an inventory with an empty hosts: {} | 00:20 |
| dmsimard | tristanC: thanks, this will save me a lot of headaches | 00:21 |
| dmsimard | jamielennox: where is that happening ? | 00:21 |
| jamielennox | in zuul-executor | 00:21 |
| jamielennox | hostvars is None because there are no hosts in the inventory | 00:22 |
| tristanC | dmsimard: heh you're welcome :-) | 00:22 |
| dmsimard | jamielennox: right, in zuul-executor but what patch ? what job ? | 00:23 |
| jamielennox | dmsimard: one of Bonny's | 00:23 |
| dmsimard | I guess it's technically possible for there to be no hosts for a job -- but there should at least be localhost (the executor?) | 00:24 |
| jamielennox | i think i figured it out, i have been pushing towards trying to reuse upstream roles and i had nodes: defined on a role higher than base | 00:28 |
| jamielennox | so in the swap i've got no nodes: attached to the job | 00:29 |
| openstackgerrit | Jamie Lennox proposed openstack-infra/zuul-jobs master: Pass correct variables to install bindep https://review.openstack.org/500648 | 00:40 |
| tristanC | jamielennox: shouldn't nodes default to localhost? | 01:16 |
| jamielennox | tristanC: it doesn't look like it does | 01:26 |
| jamielennox | but localhost typically is not displayed in an inventory | 01:27 |
| dmsimard | jeblair: looked at the generate failure -- the error is the following: Unexpected status '404 NOT FOUND' on URL /reports/ajax/results/ce9cbcd1-eeb9-472c-9096-d60561419124.txt | 02:24 |
| dmsimard | There's an inconsistency -- that uuid is a playbook: ce9cbcd1-eeb9-472c-9096-d60561419124 ( /var/lib/zuul/builds/7740680fc95042649e17e2ee655ad39b/work/src/git.openstack.org/openstack-infra/devstack-gate/playbooks/devstack.yaml ) | 02:27 |
| dmsimard | However, that playbook did not finish running, it was apparently interrupted | 02:27 |
| dmsimard | Or something that made it so the callback didn't make it to "v2_playbook_on_stats", maybe an exception raised somewhere before ? | 02:28 |
| dmsimard | jeblair: yeah, I can't pull more information than that using just the database. The problem is that something prevented that playbook run from finishing. It either crashed or was interrupted -- so there are basically no results for that particular playbook. | 02:32 |
| dmsimard | Looking at the console output, we would probably see some exception crashing the playbook run or something like that. | 02:33 |
| dmsimard | FWIW 404's during static generation are no longer fatal in 1.0 due to the nature of how the API works and 404's are somewhat expected | 02:35 |
| dmsimard | https://github.com/openstack/ara/commit/3b5db8b9311ce2efa7dc98c66f7ef6b2783f881e | 02:35 |
| openstackgerrit | Jamie Lennox proposed openstack-infra/zuul-jobs master: Only do the bindep_command if bindep_file is defined https://review.openstack.org/500683 | 06:23 |
| tobiash | mordred: I was wrong yesterday, something did break | 07:12 |
| tobiash | mordred: zuul tests don't work anymore within pycharm (but work within tox) | 07:13 |
| tobiash | mordred: bisected the problem to change 498127 | 07:13 |
| tobiash | mordred: looking now into this problem | 07:13 |
| *** hashar has joined #zuul | 08:08 | |
| *** electrofelix has joined #zuul | 08:40 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Fix missing logconfig when running tests in pycharm https://review.openstack.org/500748 | 09:28 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Close logging config file after write https://review.openstack.org/500754 | 09:43 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Close command socket after sending stop https://review.openstack.org/500755 | 09:44 |
| tobiash | jamielennox: do you mind if I rebase and update these patches which I think would be useful for windows support? https://review.openstack.org/#/q/topic:username+status:open | 09:57 |
| *** Shrews has quit IRC | 10:16 | |
| *** Shrews has joined #zuul | 10:43 | |
| *** jkilpatr has joined #zuul | 10:59 | |
| *** jkilpatr_ has joined #zuul | 11:01 | |
| *** jkilpatr has quit IRC | 11:05 | |
| jamielennox | tobiash: go for it - there was some dissent about where the username should be stored and communicated, but i think that was the preferred way | 11:09 |
| tobiash | jamielennox: thanks | 11:09 |
| tobiash | jamielennox: I'll add also add password and connection type to it so zuul/ansible can use that for connecting to windows nodes | 11:10 |
| tobiash | (on top of the stack) | 11:10 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Use username from node information if available https://review.openstack.org/453983 | 11:22 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Add username to build and upload information https://review.openstack.org/453968 | 11:45 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Add username to build and upload information https://review.openstack.org/453968 | 12:16 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Rename ssh_port to connection_port https://review.openstack.org/500799 | 12:37 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Rename ssh_port to connection_port https://review.openstack.org/500800 | 12:37 |
| *** dmsimard is now known as dmsimard|afk | 12:47 | |
| leifmadsen_ | jeblair: mordred: not sure if you'll have any time before the PTG for quickstart, but let me know if you might :) | 12:49 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Support username also for unmanaged cloud images https://review.openstack.org/500808 | 12:52 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Rename ssh_port to connection_port https://review.openstack.org/500800 | 12:57 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool feature/zuulv3: Support username also for unmanaged cloud images https://review.openstack.org/500808 | 12:57 |
| *** persia has joined #zuul | 12:58 | |
| *** dkranz has quit IRC | 13:05 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Use password supplied from nodepool https://review.openstack.org/500823 | 13:29 |
| Shrews | tobiash: not sure you want to store a password in ZK. that's totally unencrypted | 13:33 |
| tobiash | Shrews: that's based from a discussion from 9th of august: http://paste.openstack.org/show/620410/ | 13:37 |
| mordred | rcarrillocruz: https://docs.openstack.org/infra/system-config/github.html#openstack-zuul-app are docs on our app | 14:00 |
| mordred | rcarrillocruz: to create one, you go to the settings page of an organization, so for infra we went to https://github.com/organizations/openstack-infra/settings/profile | 14:01 |
| rcarrillocruz | Oh! | 14:01 |
| mordred | rcarrillocruz: then on the left-hand side you'll see "GitHub Apps" | 14:01 |
| rcarrillocruz | That looks better, thx! | 14:01 |
| mordred | and "Register a new GitHub App" | 14:01 |
| *** dkranz has joined #zuul | 14:22 | |
| jeblair | Shrews: we need to add support for encryption and auth to zk anyway | 14:26 |
| jeblair | tobiash: ^ | 14:26 |
| jeblair | dmsimard|afk: yes, that invocation of ansible-playbook was killed because the job timed out. that is a thing that will happen a lot and we need to handle it gracefully. i'm surprised though since even the normal post playbook which generates ara does so before it is completed. i guess the difference here is that there is an interrupted playbook run followed by more playbook runs? | 14:29 |
| tobiash | jeblair: yes | 14:34 |
| tobiash | jeblair: hm, just found this: https://github.com/python-zk/kazoo/issues/382 | 14:34 |
| tobiash | jeblair: looks like kazoo doesn't support ssl yet | 14:34 |
| jeblair | i guess we'll have to add that | 14:36 |
| fungi | mordred: any guess why 499843 is still resulting in that "extra keys not allowed" validation error from zuulv3 even after its parent merged? | 14:44 |
| jeblair | fungi: hrm, the periodic pipeline definition is wrong | 14:48 |
| jeblair | there was no vote from zuul on the change that added it | 14:48 |
| fungi | oh, also, not sure whether anyone else has noticed yet but http://zuulv3.openstack.org/status.json seems to include some cruft changes which aren't represented in the html rendering | 14:49 |
| jeblair | fungi: aha! | 14:49 |
| jeblair | fungi: mordred made a change that (mostly) hides changes with no jobs from the status page. however, it doesn't hide the queues. so we could see there was something stuck there, but didn't know what. | 14:50 |
| jeblair | fungi: it turns out the change that is stuck there is the change to add the periodic pipeline, which has an error and did not report. | 14:50 |
| fungi | gah! | 14:50 |
| jeblair | fungi: failing_reasons: "it has an invalid configuration" | 14:50 |
| jeblair | it's a non-live item with nothing behind it, so it should have been removed. | 14:51 |
| jeblair | oh wait, it does have an item behind it | 14:52 |
| jeblair | okay, so i think i understand the bug now | 14:53 |
| jeblair | i think it's because there's a change behind a change with a config error | 14:54 |
| jeblair | remote: https://review.openstack.org/500864 Zuul v3: Revert "Add periodic pipeline" | 14:57 |
| jeblair | fungi, mordred: let's merge that for now; i'll work on reproducing the zuul bug | 14:57 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Revert "Only add changes to status page with jobs" https://review.openstack.org/500865 | 15:00 |
| fungi | thanks jeblair! | 15:04 |
| mordred | jeblair: ++ | 15:17 |
| mordred | jeblair: ah! so that's why the shade patches were failing with merge errors too | 15:17 |
| fungi | mordred: not urgent, but are these topic:zuulv3 nodepool changes from last year still needed and just on the back burner, or a dead end to be abandoned? 411512 411473 | 15:26 |
| mordred | fungi: 411512 is bong, abandoned | 15:28 |
| fungi | thanks. still working my way through some of the older unapproved zuulv3 reviews trying to make sure i know what's actually going in | 15:29 |
| mordred | fungi: 411473 I _think_ is superseeded by other logic in v3 - Shrews could you double-check me on that? | 15:30 |
| mordred | Shrews: I think our use of nodepool_build_id makes the check for is_public not needed in v3 | 15:30 |
| Shrews | mordred: looking | 15:37 |
| *** hashar is now known as hasharAway | 15:38 | |
| Shrews | mordred: i believe you are correct | 15:38 |
| fungi | thanks for confirming, Shrews! | 15:38 |
| mordred | thanks fungi Shrews | 15:50 |
| Shrews | was it intentional to overwrite the zuul 2.5 docs with v3? https://docs.openstack.org/infra/zuul/ | 15:56 |
| Shrews | guess it really doesn't matter for this last week | 15:57 |
| clarkb | Shrews: also noticing that the current template doesn't show the version in it | 15:59 |
| jeblair | Shrews: no it wasn't intentional; that probably means there's an error in the doc publishing jobs somewhere | 16:03 |
| jeblair | pabelanger: ^ | 16:03 |
| Shrews | jeblair: qq... for the cloner shim, i can safely ignore the --branch and --project-branch arguments, yeah? | 16:08 |
| jeblair | Shrews: yep | 16:08 |
| Shrews | coolio | 16:08 |
| fungi | i'm in the process of trying to rebase jhesketh's stack of 456090 and 456162 for zuul, but a fair amount of the underlying logic seems to have changed/moved. can anyone skim and tell me whether that's a futile effort before i get much deeper into those? | 16:10 |
| jeblair | fungi: there is a very distinct possibility that all 456090 needs is the TODO comment removed. so i'd say focus on verifying that the underlying problem has been solved in the interim. | 16:13 |
| jeblair | fungi: (i'm pretty sure i had to make all that work a while back for something else i was doing) | 16:13 |
| jeblair | fungi: 456162 should still be applicable | 16:15 |
| fungi | jeblair: ahh, okay cool. the merge conflicts on 456090 were mostly around a parameter name change | 16:15 |
| fungi | 456162 merge-conflicted on moving some logic into a new method which seems to have already been created in the interim | 16:16 |
| fungi | also, those changes _seem_ unrelated to me... is there actually any reason to keep them in series with each other? | 16:17 |
| fungi | though i guess if the first one turns into just a comment removal then there's no way they truly depend there any longer, i suppose | 16:19 |
| jeblair | fungi: yeah, they can be split | 16:21 |
| fungi | thanks | 16:32 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul feature/zuulv3: WIP: Add zuul-cloner shim https://review.openstack.org/500922 | 16:39 |
| fungi | jeblair: digging into history there, is it 460769 you're saying corrected the issue related to that ssh connection context, making that todo obsolete? | 16:48 |
| jeblair | fungi: yes, i think that's the one | 16:50 |
| fungi | thanks, just making sure i provide adequate references | 16:50 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul feature/zuulv3: Clean up obsolete TODO for _get_git_ssh() https://review.openstack.org/500925 | 16:55 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul feature/zuulv3: Clean up obsolete TODO for _setGitSsh() https://review.openstack.org/500925 | 16:57 |
| fungi | 456162 unfortunately moved _and_ changed logic, so most of the work will be untangling what it was actually changing from what was simply moved around | 16:59 |
| *** harlowja has joined #zuul | 17:08 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul feature/zuulv3: Only grab the gerrit change if necessary https://review.openstack.org/456162 | 17:17 |
| fungi | okay, i *think* that should be it ^ | 17:17 |
| fungi | not too hard to straighten out | 17:17 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Dequeue non-live items with errors https://review.openstack.org/500931 | 17:20 |
| jeblair | fungi, mordred: ^ that should fix both the error with the change getting stuck, as well as the non-reporting of the invalid pipeline config. | 17:20 |
| fungi | jeblair: the whitespace gods demand the sacrifice of your tab key | 17:29 |
| fungi | they are quick to anger | 17:29 |
| fungi | my rebase of 456162 is failing unit testing, however i made the mistake of attempting to load the zuul unit test console log in a graphical browser | 17:31 |
| fungi | failed on test_untrusted_yaml_error | 17:32 |
| fungi | which seems like an odd test for that change to break | 17:33 |
| fungi | oh, nevermind i misread. it failed 131 tests | 17:33 |
| fungi | AttributeError: 'GerritEventConnector' object has no attribute 'abide' | 17:36 |
| fungi | i guess it did at some point | 17:37 |
| *** dmsimard|afk is now known as dmsimard | 17:38 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul feature/zuulv3: Only grab the gerrit change if necessary https://review.openstack.org/456162 | 17:45 |
| dmsimard | jeblair: re: generate failure -- I could likely backport that patch I sent you back to master and do a dot release. | 17:58 |
| dmsimard | The only thing it does is to make 404's non-fatal on static generation | 17:59 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Dequeue non-live items with errors https://review.openstack.org/500931 | 18:07 |
| jeblair | dmsimard: what does 404 mean in this context? | 18:07 |
| fungi | i think i've gotten in over my head on the unit test failures for 456162. i'm not sure if switching from GerritEventConnector.abide to GerritEventConnector.connection.sched.abide was the correct way to address the exceptions raised in patchset 3, or if doing that is what leads to the exceptions for ambiguous project naming from tenant.getProject(event.project_name) in patchset 4 | 18:32 |
| mordred | jeblair: 500931 looks great to me - let's get it landed and restarted, then I can propose a revert revert of the periodic and see what zuul wanted to tell me | 18:38 |
| mordred | pabelanger, jlk, SpamapS, clarkb, Shrews, fungi, tobiash: anybody feel like +3ing https://review.openstack.org/#/c/500931 ? | 18:39 |
| fungi | yeah, i had just gotten back to reviewing that one | 18:41 |
| fungi | lgtm | 18:49 |
| pabelanger | jeblair: Shrews: Yes, that is a mistake. We need to land https://review.openstack.org/500229/ and https://review.openstack.org/500591/ In the mean time we can revert publish-openstack-python-docs on zuulv3 post if needed | 18:49 |
| pabelanger | jeblair: mordred: is there anything I should be focusing on or looking at? Our afs publishing jobs are in good shape, just pending reviews. | 18:52 |
| clarkb | pabelanger: shouldn't only master changes publish to the document root though? (I guess on the other openstack docs jobs its to latest/ now) | 18:57 |
| clarkb | pabelanger: just reading thsoe commit messages I'm a little confused why that would change zuulv3 branch from publishing to master destination | 18:57 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Dequeue non-live items with errors https://review.openstack.org/500931 | 18:59 |
| jeblair | reminder: we'll have a zuul ptg status checkin at the infra meeting starting nowish in #openstack-meeting | 19:00 |
| pabelanger | clarkb: yes, master will be doc root. That works atm, but for feature/zuulv3 branch, we need to land 500229, since it was still using old zuul variables | 19:01 |
| clarkb | pabelanger: right but why did zuulv3 publish to master/root/latest? | 19:01 |
| pabelanger | clarkb: it is using run_docs.sh right now | 19:02 |
| pabelanger | which, still has old zuul branch vars | 19:02 |
| pabelanger | so, currently a bug can we can stop publishing for a moment | 19:02 |
| Shrews | jeblair: ooh, thx for the reminder | 19:05 |
| mordred | pabelanger: -1 on https://review.openstack.org/#/c/500591 - but I'd like to get https://review.openstack.org/500229/ landed asap | 19:11 |
| pabelanger | mordred: looking | 19:13 |
| pabelanger | mordred: ah, forgot | 19:13 |
| pabelanger | fixing | 19:13 |
| dmsimard | jeblair: sorry, caught in between meetings. The 404 is because the UI does an "ajax" call to an endpoint to fetch the results for a playbook. There were no results for that particular playbook so the endpoint returned a 404 and this is (currently) fatal for static generation. | 19:59 |
| clarkb | pabelanger: can you see comments on 229? | 20:00 |
| clarkb | pabelanger: also do you know where we end up publishing to /latest for master? I'm not seeing that in v2 or v3 jobs | 20:01 |
| pabelanger | looking | 20:01 |
| mordred | clarkb: I don't believe we do | 20:01 |
| jeblair | dmsimard: ah cool. yeah, that would be helpful to have backported if possible. i think as long as we set ignore_errors it's not critical, we just won't have ara in those cases otherwise, and it would be nice to. | 20:02 |
| clarkb | mordred: https://docs.openstack.org/nova/latest/ that works somehow though? | 20:02 |
| mordred | clarkb: oh - hrm, I'm totally wrong about that | 20:02 |
| jeblair | clarkb ,mordred: http://files.openstack.org/docs/nova/ | 20:03 |
| dmsimard | jeblair: yeah, since the playbook run would be incomplete, there would be that grey icon that we typically have for the post playbook for log collection for example. | 20:03 |
| dmsimard | jeblair: I'll go ahead and backport it. | 20:03 |
| jeblair | dmsimard: cool, thanks | 20:04 |
| clarkb | jeblair: ya so something is creating that, but I don't see it in 229 or in existing project-config jobs | 20:04 |
| jeblair | there's no .root-marker in that directory; the next one up is Project: openstack/openstack-manuals Ref: master Build: b2d85b0da8174434ad2532d3f057b3e0 Revision: cad68a203fde595a20f06ce19408ac69da907235 | 20:05 |
| pabelanger | clarkb: replied, but for master we just accept the default docs folder structure, so no moving things | 20:06 |
| clarkb | pabelanger: right but that is a regresion isn't it? | 20:06 |
| pabelanger | clarkb: no, that is how we do it today in run_docs.sh | 20:06 |
| pabelanger | if master, we skip | 20:06 |
| clarkb | ah ok, then that leaves me more confused for latest/ | 20:07 |
| pabelanger | http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/scripts/run-docs.sh#n33 | 20:07 |
| pabelanger | clarkb: ya, I haven't actually looked at latest stuff yet | 20:07 |
| clarkb | since latest is basicaly a mv of / (master) | 20:07 |
| pabelanger | clarkb: which job is using that? | 20:07 |
| pabelanger | Oh, I wonder if it is on the publisher side | 20:08 |
| clarkb | pabelanger: thats the problem I don't see it in jobs but its definitely in the output | 20:08 |
| clarkb | oh maybe | 20:08 |
| pabelanger | Ya, I think at PTG, we can simplify some of this. but might be too much to do before cut over | 20:08 |
| pabelanger | I know some jobs have different publisher directories | 20:08 |
| pabelanger | but all call run-docs.sh | 20:09 |
| clarkb | pabelanger: then the other concern is do we need to skip things more aggressively if doing tags only? | 20:09 |
| mordred | yah - tags-only is handled | 20:09 |
| fungi | i thought the docs jobs refactor a few weeks back basically stopped publishing on tag events | 20:10 |
| pabelanger | ya, I haven't look much into tag stuff, was leaning on mordred changes for that | 20:10 |
| mordred | there is branching logic - depending on whether or not a tags-only parameter is passed - there are 3 repos that pass that parameter | 20:10 |
| clarkb | fungi: http://files.openstack.org/docs/nova/ does lack a 16.0.0 tag for pike so that wouldn't surprise me | 20:10 |
| clarkb | mordred: can you see my comment on 229 to see what I mean about handling tags only? | 20:11 |
| mordred | https://review.openstack.org/#/c/500229/7/roles/prepare-docs-for-afs/tasks/main.yaml <-- the left hand side has the existing code | 20:11 |
| clarkb | mordred: theer are a couple branches that would run with tags only that I think should not run | 20:11 |
| pabelanger | Oh, interesting: http://files.openstack.org/docs/nova/ | 20:11 |
| pabelanger | now I see what you mean about latest | 20:12 |
| pabelanger | so ya, we'd need to make sure shade published into a different directory | 20:12 |
| clarkb | oh you know what I bet we handle that in zuulls layout tody | 20:12 |
| clarkb | where we only run the job in the release pipeline | 20:12 |
| mordred | clarkb: so - the 3 logic branches there are, I thnk the same as the 4 on the left hand side | 20:13 |
| mordred | clarkb: there isn't an explicit no-op for branch == 'master' - because branch == 'master' doesn't match the other three anyway | 20:13 |
| jeblair | okay, looking at the most recent run of openstack-manuals-tox-doc-publishdocs, i can confirm that job isn't doing anything with the project directories. it is *trying* to delete them because they don't have .root-marker files, but it's failing because their children *do* have .root-marker files. that's probably okay, if slightly weird (theoretically, there should be some job somewhere that sticks a .root-marker in nova/). i don't think that ... | 20:13 |
| jeblair | ... was accounted for with the new jobs. | 20:13 |
| jeblair | nova's latest .root-marker is Project: openstack/nova Ref: master Build: 9d0798a6a0ef4fa79b8df1db95cf2e36 Revision: 019f7ecb88e031d7c56472c378473e48b26ea59d | 20:15 |
| jeblair | http://logs.openstack.org/01/019f7ecb88e031d7c56472c378473e48b26ea59d/post/ | 20:15 |
| jeblair | http://logs.openstack.org/01/019f7ecb88e031d7c56472c378473e48b26ea59d/post/nova-docs-unified-ubuntu-xenial/9d0798a/console.html#_2017-09-05_19_18_07_564008 | 20:15 |
| clarkb | mordred: ya its basically removing the first case by explicitly blocking out master in the subsequent 3 | 20:15 |
| mordred | so the first non-master branch is for there being a tag (and not on master) the second is are we on a branch with stable in it and not master or tag, the third is we're on a branch without a tag but that isn't a stable branch or master | 20:15 |
| mordred | clarkb: yah | 20:15 |
| mordred | clarkb: becaue there's no elif in ansible -so each when: has to be self-sufficient | 20:16 |
| clarkb | mordred: ya, if only ansible had proper branching syntax :) | 20:17 |
| jeblair | http://logs.openstack.org/01/019f7ecb88e031d7c56472c378473e48b26ea59d/post/nova-docs-unified-ubuntu-xenial/9d0798a/_zuul_ansible/scripts/07-ad0138d0065a4c82ac0f58a3371d07f2.sh | 20:17 |
| jeblair | that's where latest comes from | 20:17 |
| clarkb | mordred: note the oldcode seems to use dirname not basename where you -1'd? | 20:18 |
| clarkb | jeblair: thanks for finding that, so I think that would be a good candidate for cleanup once past the initial transition | 20:18 |
| mordred | clarkb, pabelanger: yah - I see that ... I think we should actually not do either | 20:19 |
| clarkb | mordred: and just publish to feature/zuulv3 ? | 20:19 |
| mordred | since we're working with zuul.tag which is feature/zuulv3 - and we want to publish the docs for non-stable branches to feature/zuulv3 not to feature or zuulv3 | 20:19 |
| jeblair | http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/openstack-publish-jobs.yaml#n22 | 20:19 |
| jeblair | that's where it's defined | 20:19 |
| mordred | https://docs.openstack.org/infra/zuul/feature/zuulv3/ | 20:19 |
| mordred | oh good | 20:20 |
| clarkb | mordred: ya | 20:20 |
| mordred | did we translate the wrong thing? | 20:20 |
| jeblair | that looks like an docs publishing job for a non-infra official openstack project | 20:20 |
| mordred | yah. so I think what we have currently is a docs job for a non-official project - which really should only apply to infra projects at this point | 20:21 |
| mordred | I also don't see any issue with infra using that same doc publication job | 20:22 |
| jeblair | mordred: well, it's official, but infra :) | 20:22 |
| mordred | yah - I mean - having https://docs.openstack.org/infra/zuul/latest doesn't seem like it would be bad or a thing we should keep an infra-specific publish job for | 20:22 |
| *** hasharAway has quit IRC | 20:23 | |
| jeblair | mordred: yep -- as long as it can handle publisihng one level down (infra/) | 20:23 |
| mordred | other than the location of the base dir | 20:23 |
| mordred | ++ | 20:23 |
| mordred | so - I mean, we need an infra-specific job - but we don't need an infra-specific role | 20:23 |
| jeblair | that sounds reasonable | 20:23 |
| pabelanger | okay, so we'll need a little update then | 20:23 |
| jeblair | mordred: i think we need a redirect though? | 20:23 |
| jeblair | or is that in the server config? | 20:24 |
| pabelanger | let me see how latest stuff works | 20:24 |
| jeblair | something does nova/ -> nova/latest/ | 20:24 |
| mordred | jeblair: yah - I'm not sure where theredirect is coming from | 20:24 |
| jeblair | .htaccess in openstack-manuals | 20:25 |
| mordred | awesome | 20:25 |
| *** jkilpatr_ has quit IRC | 20:25 | |
| mordred | jeblair: should we just add a redirect there for infra? or to root-level config? | 20:25 |
| pabelanger | looking at the link jeblair posted: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/openstack-publish-jobs.yaml#n22 it doesn't support feature branches, how would that look with feature/zuulv3? | 20:26 |
| clarkb | pabelanger: thats what mordred was saying dropping the dirname/basename entitely | 20:27 |
| jeblair | mordred: it's templated; we'd need to update the script to add /latest/ redirects for the infra projects once they exist | 20:27 |
| jeblair | pabelanger, clarkb: it does seem that not publishing feature branches is a bit of a problem for us at the moment though. | 20:27 |
| jeblair | so i think we're going to need the infra publish jobs to continue to work the way they do now, and we should talk about switching to the unified jobs after cutover (maybe after v3 release) | 20:28 |
| mordred | ++ | 20:28 |
| mordred | agree | 20:28 |
| pabelanger | okay | 20:28 |
| mordred | so we fix up 500229 and rename it to be openstack-publish-infra-docs | 20:29 |
| pabelanger | publish-openstack-python-docs-infra and publish-openstack-python-docs jobs ? | 20:29 |
| pabelanger | ah | 20:29 |
| mordred | then make a new one that is openstack-publish-docs that uses the content from docs-unified | 20:29 |
| pabelanger | k | 20:29 |
| mordred | yah | 20:29 |
| mordred | pabelanger: your names are clearly better :) | 20:30 |
| pabelanger | k, I'm going to need some food, so I'll start on that after I eat | 20:30 |
| mordred | pabelanger: and we can have zuul use the new publish-openstack-python-docs-infra and shade use publish-openstack-python-docs when we have it | 20:30 |
| mordred | since shade does unified openstack publishing already | 20:30 |
| pabelanger | mordred: ++ | 20:30 |
| mordred | that way we can verify both work | 20:30 |
| mordred | pabelanger: cool - youre on this one? | 20:30 |
| pabelanger | ya | 20:30 |
| mordred | sweet | 20:30 |
| mordred | pabelanger, jeblair, clarkb, fungi: https://etherpad.openstack.org/p/zuulv3-constraints-jobs | 20:31 |
| pabelanger | yup, I'll start hammer out the jobs here in 45mins | 20:31 |
| jeblair | mordred: i will read your constraints change now | 20:31 |
| jeblair | mordred: and your treatise :) | 20:31 |
| mordred | I made an etherpad with the background, what's trying to be accomplished and what the current options are | 20:31 |
| mordred | jeblair: you might want to just start with the etherpad and ignore the changes themselves for now | 20:31 |
| mordred | jeblair, fungi: btw - we've been doing this long enough that I was able to know which of you was jeblair and which was fungi just from wording and writing style :) | 20:41 |
| *** jkilpatr has joined #zuul | 20:41 | |
| jeblair | also fungi is always chocolate | 20:41 |
| fungi | i shall endeavor to be more opaque in the future ;) | 20:41 |
| mordred | you'd think that would be how I knew ... | 20:41 |
| mordred | and people say tone doesn't come across when chatting via text ... | 20:42 |
| jeblair | mordred: i'd like to explore the option that we have openstack-tox-py35 with parent: tox-py35 and the addition of openstack/requirements and any job var needed | 20:42 |
| mordred | fungi, jeblair, pabelanger: so I think we can probably switch back to here having enough context for stuff to talk here | 20:43 |
| jlk | mordred: text has its own tone | 20:43 |
| jeblair | mordred: i feel like that's actually a really good use of zuul-jobs -- there's no duplicate job definition, we're just enhancing the zuul-jobs tox job with the openstack-specific add-ons | 20:43 |
| mordred | jeblair: what about openstack-tox-py35 do you like more than project-template: openstack-python-jobs ? | 20:43 |
| mordred | (I mean, we already have project-template openstack-python-jobs, so it's more a question of what you like about openstack-tox-py35 more than just setting the repo and var in the template def) | 20:44 |
| pabelanger | interesting question, since both should do the same thing | 20:45 |
| jeblair | mordred: it means you have to use the project-template in order to not have a mess; if you need a different project template for some reason, that would need to duplicate the settings. and if you just wanted to add a one-off (tox-py36?) again, duplicated settings. | 20:45 |
| mordred | jeblair: k. I can understand that - from my side I prefer the tempate because the thing that reports to the user is "tox-py35" whether the job uses a constraints file or not, so people experiencing the system here and then using it elsewhere can learn that "tox-py35" is the thing they want to run on their zuul back home | 20:46 |
| jeblair | mordred: i was actually about to type into the etherpad that i like that it has a different name. fundamentally, it behaves differently. it's not *just* a tox-py35 job, it's a tox-py35 job with the *openstack constraints* | 20:47 |
| mordred | at the end of the day, since it'll be in a template for a large portion of the users anyway I can just eat the openstack-tox-py35 and get over it | 20:48 |
| fungi | what's the downside to the project-template alternative? | 20:48 |
| mordred | jeblair: but ... so - the thing is | 20:48 |
| jeblair | i consider that less important than the usability, but still desirable | 20:48 |
| pabelanger | right, I thought that is why we named it tox-py35-constraints to better indicate the constraints in comments | 20:48 |
| mordred | jeblair: thejob will run with upper-constraints whether we have a job that does those things or not | 20:48 |
| fungi | oh, nevermind, i see jeblair answered that further up | 20:49 |
| * fungi is a slow reader | 20:49 | |
| mordred | since the upper-constraints processing is actually logic that's in the tox.ini | 20:49 |
| jeblair | (like, if you just gave me that choice with no context about cost/benifit, i'd still say "two job names because they behave differently") | 20:49 |
| mordred | so they don't actualy behave differently from a content execution perspective - the difference would be providing depends-on with openstack/requirements and not having the job wget the file but instead have it pushed to the node | 20:50 |
| mordred | now - that may still be enough to warrant "it works differently" | 20:50 |
| mordred | but to me those are deployer-specific optimizations that should be hidden from the user | 20:50 |
| jeblair | oh wait, i thought tox-py35 isn't going to go wgetting files | 20:50 |
| mordred | the tox.ini file in shade will TOTALLY wget files if something doesn't put them there | 20:50 |
| mordred | or, rather, pip will | 20:51 |
| jeblair | ah. but zuul tox-py35 won't. | 20:51 |
| mordred | the default value of the variable is "https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt" | 20:51 |
| mordred | right | 20:51 |
| mordred | so zuul using the thing I pushed up will not do anything since its tox.ini doesn't know or care about constraints files | 20:52 |
| fungi | part of my concern over how much of this is generalizable outside openstack is the upper-constraints idea vs constraints as a whole. we have upper-constraints.txt as a rendering of the highest usable versions of our transitive deps for a synchronized set of global requirements. that's very openstack-specific. having a mechanism to generally apply a constraints set would be nice and non-openstack-specific | 20:52 |
| fungi | assuming we don't need a lot of arbitrary shell callout from the tox.ini to handle it (e.g. if we get direct support for constraints in tox) | 20:52 |
| mordred | even with the excessive version of adding o/r to all required-projects- zuul's tox-py35 will operate unchanged | 20:52 |
| mordred | fungi: yes - I agree with that - I think that medium-term goal should be adding feature to tox to support arbitrary constraints set | 20:53 |
| mordred | fungi: at that point, theimpl details of the tox role passing UPPER_CONSTRAINTS_FILE as an env var could be replaced with a different mechanism | 20:53 |
| pabelanger | once concern about adding o/r to base required-projects, is puppet-jobs won't use it. So, unneeded repo on disk now | 20:54 |
| mordred | pabelanger: yah - that's the bad part about that and I think I'm fairly convinced doing that is a bad idea | 20:55 |
| jeblair | mordred: another option (which i don't like, but has a potentially better shape in the future) added to etherpad | 20:55 |
| mordred | jeblair: yah - that's what I was starting to ask about before ... | 20:57 |
| mordred | jeblair: can we just shadow the tox base job instead of tox-py35? | 20:57 |
| jeblair | mordred: yeah, that might be a bit better | 20:57 |
| fungi | having zuul-jobs:tox-py35 inherit from project-config:tox would be a layering violation wouldn't it? | 20:58 |
| jeblair | mordred: hrm, 2 problems with that: 1) tox has 3 playbooks, so we'd need to copy those. 2) we'd still be pushing the requirements repo to projects that don't need it. | 20:59 |
| mordred | yah | 20:59 |
| mordred | ok - how about this as a resolution for now | 20:59 |
| mordred | we do the openstack-tox-py35 job for now | 20:59 |
| fungi | or do you mean shadow both zuul-jobs:tox-py35 _and_ zuul-jobs:tox in project-config? | 20:59 |
| jeblair | fungi: if we shadow tox in project-config, that's the job that zuul-jobs will inherit from | 20:59 |
| mordred | and we put it into a openstack-python-jobs-constraints template | 20:59 |
| mordred | but - we put an asterisk by it with a goal that we'd like to get openstack-python-jobs-constraints to have just tox-py* in it and not openstack-tox-py* | 21:00 |
| fungi | and there's nothing wrong with having baser-level zuul-jobs definitions inherit modified replacement jobs from our openstack-specific project-config set? | 21:01 |
| jeblair | mordred: i'm really tempted to agree with that, except wouldn't that still mean that we'd end up with all tox jobs getting the requiremnts repo regardless of whether they use it? | 21:01 |
| mordred | fungi: zuul-jobs already inherit from jobs in project-config :) | 21:01 |
| mordred | jeblair: nope | 21:01 |
| mordred | oh - the repo. hrm | 21:02 |
| mordred | jeblair: well - I think I was thinking about your statement about post-ptg work about inheriting and shadowing | 21:02 |
| fungi | mordred: which ones? mostly curious at this point if we've somehow added jobs to zuul-jobs which aren't usable outside openstack because they depend on things in project-config | 21:02 |
| mordred | fungi: base | 21:02 |
| mordred | fungi: all zuul installs must define a base job | 21:03 |
| jeblair | mordred: yeah, i'm starting to talk myself out of that, because i think we may have been dancing around the central question: | 21:03 |
| jeblair | should tox-py35 in openstack's zuul automatically include the openstack/requirements repo? | 21:03 |
| fungi | oh, got it. so the base job definition is something that we expect the deployer to provide from somewhere, possibly customized? | 21:03 |
| mordred | jeblair: I think, for me, if we could have a tox base job in project-config that didn't have to copy playbooks and essentially only added a repo and a variable | 21:04 |
| mordred | fungi: yes. because secrets and log publication and whatnot | 21:04 |
| pabelanger | If voting, I would say no to automatically include openstack/requirements | 21:04 |
| mordred | jeblair: then o/r always being added to tox jobs in openstack's zuul would not bother me personally | 21:04 |
| jeblair | fungi: we're really close to being able to provide a 'base' job that's reusable, but back-burnered that till after ptg. | 21:04 |
| fungi | makes sense | 21:05 |
| mordred | jeblair: (I actually did some hackingon that too this weekend) | 21:05 |
| mordred | jeblair: but yes - back-burnered so I haven't pointed it out :) | 21:05 |
| jeblair | mordred: i think i lean toward 'no' on that question -- i'd prefer not to have that in every tox job. | 21:06 |
| jeblair | as a zuul developer, i *would* like to have the tools available to *do* that :) | 21:06 |
| mordred | yah | 21:06 |
| mordred | jeblair: also - once we have those tools, perhaps it's also a place where tenant differences can be used | 21:07 |
| jeblair | but as an openstack zuul operator, i'm currently inclined to say that it's not an assumption we sholud make in openstack | 21:07 |
| jeblair | mordred: agreed; if we ended up with an official openstack tenant, that would be an easier sell for me | 21:07 |
| mordred | jeblair: like, have only 'openstack' projects in the openstack zuul tenant, do the tox shadow + o/r repo there | 21:07 |
| mordred | jeblair: because then there really are no cases when there's a repo that has a good reason to opt-out of that | 21:07 |
| pabelanger | Oh, that would be fun | 21:07 |
| mordred | jeblair: but I think that's a level of complexity we should avoid for this week | 21:08 |
| mordred | k. I'll update the constraints patch ... do we prefer tox-py35-constraints? or openstack-tox-py35? or openstack-tox-py35-with-constraints? | 21:09 |
| pabelanger | I don't mind tox-py35-constraints | 21:09 |
| jeblair | wfm | 21:09 |
| fungi | i think openstack-tox-py35 is accurate and sufficient | 21:09 |
| jeblair | also wfm :) | 21:10 |
| pabelanger | dealers choice | 21:10 |
| fungi | particularly if we find there's anything else "special" that openstack-specific tox-based jobs will also need to do | 21:10 |
| jeblair | mordred: heads or tails? | 21:10 |
| jeblair | fungi makes a good point :) | 21:10 |
| fungi | whereas teh odds that things outside the openstack set will want to apply openstack's upper-constraints.txt file are fairly slim | 21:11 |
| * jeblair changes vote to openstack-tox-py35 | 21:11 | |
| pabelanger | ha, I think we originally called tox-py35-constraints that :D | 21:11 |
| pabelanger | ++ | 21:11 |
| fungi | gives us a convenient place to contain the openstackisms | 21:11 |
| jeblair | i also updated etherpad with example | 21:12 |
| *** harlowja has quit IRC | 21:13 | |
| mordred | ok. cool | 21:13 |
| mordred | btw - https://review.openstack.org/#/c/500320 itself is good to go - the part we're discussing was related to a project-config change | 21:13 |
| mordred | jeblair, fungi, pabelanger: WHILE I HAVE YOUR ATTENTION ... | 21:14 |
| jeblair | you certainly have it now! | 21:14 |
| mordred | https://review.openstack.org/#/c/489719 is also readyfor review ... and is the thing that magically adds python projects on the system from required-projects into tox | 21:14 |
| mordred | and I have a set of shade and keystoneauth patches that show it doing its job, both in not making changes when there are no changes to make, making them when there are, and responding to an opt-out flag | 21:15 |
| mordred | the shade patches end here: https://review.openstack.org/#/c/500388 | 21:15 |
| mordred | looking at http://logs.openstack.org/87/500387/1/check/tox-py35/98fc1f1/testr_results.html.gz is how you can see that it failed becaue it applied the keystoneauth depends-on patch | 21:16 |
| mordred | since the exceptoin in the unittest is "This is an intentionally failing change" | 21:16 |
| fungi | if anyone with deeper understanding of gerritconnection module has a moment, i think i've gotten in over my head on the unit test failures for 456162. i'm not sure if switching from GerritEventConnector.abide to GerritEventConnector.connection.sched.abide was the correct way to address the exceptions raised in patchset 3, or if doing that is what leads to the exceptions for ambiguous project naming from | 21:19 |
| fungi | tenant.getProject(event.project_name) in patchset 4 | 21:19 |
| fungi | i would have assumed it should be able to figure out the host by context, but i guess that gets dereferenced by washing it through the scheduler object? | 21:22 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Add UPPER_CONSTRAINTS_FILE file if it exists https://review.openstack.org/500320 | 21:23 |
| mordred | jeblair: ^^ fixed from your comments | 21:23 |
| jeblair | mordred: thx | 21:23 |
| mordred | jeblair: also, it depends on https://review.openstack.org/#/c/500324 | 21:24 |
| jeblair | fungi: lookin | 21:24 |
| fungi | just starting to attempt to delve into how to get from the event connector to a tenants list with connection context, or whether i need to assemble a host-specific project name to pass into tenant.getProject() now | 21:25 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Override tox requirments with zuul git repos https://review.openstack.org/489719 | 21:25 |
| mordred | jeblair: have we restarted zuul scheduler since your patch to fix the reporting? | 21:27 |
| jeblair | mordred: not i | 21:27 |
| fungi | yeah, i assume we still need a restart to clear out the cobwebs in the check pipeline | 21:28 |
| mordred | jeblair: k. I'd like to do that | 21:28 |
| jeblair | mordred: ++ you have a 'go' from me. | 21:29 |
| * mordred restarting zuulv3 scheduler | 21:29 | |
| fungi | if it were working now the check queue count wouldn't be off-by-one | 21:29 |
| fungi | mordred: graceful restart i guess? | 21:29 |
| pabelanger | mordred: do you mind getting executors too? Want to make sure we are running pastebin friendly debug logs | 21:30 |
| fungi | we have several changes in the zuulv3 gate | 21:30 |
| mordred | pabelanger: sure thing | 21:30 |
| pabelanger | danke | 21:30 |
| mordred | fungi: oh - no - sorry - will need to recheck those | 21:30 |
| mordred | my bad - I'll dump/restore next time | 21:31 |
| clarkb | pabelanger: pastebin friendyl logs? like text? | 21:31 |
| fungi | 500865,1 500476,2 and 500324,1 need reapproving i guess? | 21:31 |
| fungi | 500320,9 489719,29 and 500943,3 will need rechecking | 21:31 |
| mordred | clarkb: pabelanger found a place where secrets could leak into log | 21:31 |
| clarkb | oh fun | 21:32 |
| clarkb | thats the service logs though not the job logs. I see the change now | 21:32 |
| mordred | clarkb: yah | 21:33 |
| jeblair | fungi: left some text on 456162 for ya | 21:33 |
| mordred | pabelanger: restarted | 21:33 |
| fungi | i'll get those 6 changes reenqueued | 21:33 |
| fungi | thanks jeblair! | 21:33 |
| mordred | v3 should be running with all latest changes | 21:33 |
| jeblair | mordred: thanks! | 21:33 |
| clarkb | mordred: pabelanger should we make that more obvious in the code too? reading the patch I don't think that woudl be clear to me as a reviewer | 21:34 |
| fungi | what tenant name do i need to pass to zuul enqueue now on zuulv3.o.o? | 21:36 |
| fungi | is that "openstack"? | 21:36 |
| jeblair | fungi: yes | 21:36 |
| fungi | thanks | 21:36 |
| jeblair | clarkb, pabelanger: well, maybe in the commit message; i'm not sure the code needs extra annotation. | 21:37 |
| clarkb | jeblair: I guess its never "safe" to print playbook objects? | 21:38 |
| jeblair | clarkb: correct (not at the moment, now that we moved secrets there) | 21:38 |
| fungi | reenqueued 3 gate jobs and 3 check jobs following the scheduler restart | 21:39 |
| clarkb | jeblair: maybe make stringification of playbook objects filter out secrets? | 21:41 |
| jeblair | clarkb: they're just dicts there; we could make them special | 21:43 |
| jeblair | clarkb: i'm not really worried about this as a systemic problem because we've knowingly logged excessively while in development; i'd consider this merely initial cleanup rather than a developer oops. | 21:43 |
| clarkb | ok | 21:44 |
| pabelanger | ya | 21:44 |
| jeblair | (for instance, we still log the entirely of the cat job data we get back on dynamic reconfiguration; that's not going to last long into production scale :) | 21:44 |
| clarkb | (I just bring it up because similar problem has existed in openstack since well forever and it has hampered efforts to make certain logs public) | 21:44 |
| pabelanger | well, I'm sure we might be sharing debug logs this weekend with non infra-root people too. That was my main concern about the restarts :) | 21:45 |
| jeblair | yeah. the intent is not to log any secret data. all we need to do is not log any secret data. :) | 21:45 |
| jeblair | clarkb: and, yeah if we *do* find that we want to log the playbook object, we should make a safe formatter | 21:46 |
| jeblair | i'm going to take a short break then try to dig into the wget issue | 21:46 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul feature/zuulv3: Only grab the gerrit change if necessary https://review.openstack.org/456162 | 21:47 |
| fungi | going to go try and perform pre-travel yardwork while that ^ is tested | 21:47 |
| fungi | jeblair: thanks for the reference to connection.canonical_hostname there... i was worried assembling the host-specific project name there was going to end up being the answer (at least until that new todo is addressed_ | 21:48 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Revert "Only add changes to status page with jobs" https://review.openstack.org/500865 | 21:48 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Update tox/test-requirements https://review.openstack.org/500324 | 21:49 |
| mordred | remote: https://review.openstack.org/501001 Add OpenStack constraints versions of tox jobs | 21:49 |
| mordred | jeblair, fungi, pabelanger, clarkb: ^^ | 21:49 |
| mordred | pabelanger: I think you're going to need to make a copy of the job with the new name, then update zuul's use of it, then delete the old one | 21:52 |
| pabelanger | mordred: Yup, that is what I'm working on now | 21:52 |
| mordred | pabelanger: cool | 21:54 |
| mordred | pabelanger, fungi, jeblair, clarkb: for https://review.openstack.org/#/c/500229 - since we've now learned that it's actually just for publishing infra-docs - how about we drop support for the tags-only logic? | 21:56 |
| mordred | pabelanger, fungi, jeblair, clarkb: it's used in exactly three places: bindep-infra-docs-tags-only git-restack-infra-docs-tags-only git-review-infra-docs-tags-only | 21:56 |
| clarkb | mordred: and just publish on every change for those tools? I think that is likely ok for most. bindep may cause problems | 21:57 |
| pabelanger | will do what group wants to | 21:57 |
| clarkb | mordred: since bindep is largely consumed on releases? | 21:57 |
| fungi | same for git-review and git-restack | 21:57 |
| mordred | well - whether we publish is a which-pipieline-does-the-job-go-in thing | 21:57 |
| clarkb | git review is really static now and comes with a man page at least. Not sure about restack | 21:57 |
| *** dkranz has quit IRC | 21:58 | |
| mordred | the only difference with the tags-only flag is how we put files into a subdir | 21:58 |
| fungi | i wasn't too thrilled with the switch to continuously publishing per-branch docs in openstack as a whole, but i expect them to come back around when enough users complain | 21:58 |
| fungi | and get at least some back to a on-releases-only publication method | 21:58 |
| mordred | I honestly to god can't see the logic difference with the tag too | 21:58 |
| clarkb | mordred: I think its only supposed to publish to / when tags happen | 21:59 |
| clarkb | mordred: the idea being if you go to docs/foo youget the last tagged docs not the current dev docs | 21:59 |
| mordred | ahh - so - with tags-only flag is only moves the docs into a tag subdir if the tag is also the latest tag in the repo | 22:00 |
| fungi | i think it was mostly just that we only put the job in the release pipeline rather than post for some projects | 22:00 |
| mordred | clarkb: http://paste.openstack.org/show/620448/ | 22:00 |
| fungi | and, yeah, that was to keep stable point releases from overriding the most recent | 22:01 |
| mordred | GOTCHA | 22:01 |
| mordred | but that doesn't really apply to git-review, git-restack or bindep | 22:02 |
| mordred | the comment above it helps | 22:02 |
| mordred | publish to / and /$TAG if this is the latest version for projects and we are only publishing from the release pipeline, or just /$TAG otherwise | 22:03 |
| clarkb | ya that should be fine | 22:04 |
| clarkb | mordred: So ya I think just drop logic entirely would be a good simplification | 22:09 |
| pabelanger | mordred: clarkb: do you mind commenting on 500229 and I'll push those changes up | 22:10 |
| jeblair | enabling keep on ze01 | 22:16 |
| jamielennox | pabelanger: can you have a look at https://review.openstack.org/#/c/500683/ and https://review.openstack.org/#/c/500648/ | 22:16 |
| jamielennox | they're simple and at least the first one i can't work around locally without forking zuul-jobs | 22:16 |
| mordred | pabelanger: done | 22:18 |
| pabelanger | jamielennox: left comment on 500683 which likey will turn into bikeshed | 22:23 |
| mordred | pabelanger: sorry - +A'd it before seeing your comment | 22:24 |
| mordred | pabelanger: it's completely reasonable to not have bindep_file - the role should be "run bindep on the repo if it's useful to do so" | 22:25 |
| mordred | but we can't know if it's useful to do so without looking to see if there is a bindep_file in the repo | 22:25 |
| mordred | pabelanger: BUT - we should almost certainly move the installation of bindep to after we determine if we even have a bindep file | 22:26 |
| pabelanger | mordred: ya, that's fine. I think we can discuss at PTG, I'm not a fan of adding roles that just noop. Because, if you don't have a bindep.txt file, no reason to use bindep roll | 22:26 |
| pabelanger | role* | 22:26 |
| pabelanger | but, then we are getting into conditional role includes | 22:26 |
| pabelanger | which, is also tricky | 22:27 |
| mordred | pabelanger: I disagree | 22:27 |
| mordred | I think we always want to run the bindep role | 22:27 |
| mordred | as part of the unittest base job | 22:27 |
| pabelanger | yes | 22:27 |
| mordred | because it's a base job thats intent is to do its best to set things up for people | 22:27 |
| mordred | so we cannot know whether a given repo uses bindep or not without running the role | 22:27 |
| pabelanger | so, I would do include_role: bindep if bindep_file is defined, in playbook | 22:28 |
| pabelanger | but, I know people don't like include_role | 22:28 |
| pabelanger | over having bindep role install bindep, then skip calling it | 22:29 |
| mordred | oh - there's a problem though | 22:29 |
| mordred | the playbook can't do that | 22:29 |
| mordred | without running the role | 22:29 |
| mordred | because in 99% of the times, bindep_file is not defined by the job or user | 22:29 |
| mordred | bindep_file is a fact set by find.yaml | 22:29 |
| mordred | it's only an argument to allow people to do really weird things | 22:29 |
| pabelanger | Ah, I thought we dropped the find stuff | 22:30 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Pass correct variables to install bindep https://review.openstack.org/500648 | 22:31 |
| clarkb | run tox has always been a super set of what people generally need in order to do the right thing depending on what they do need | 22:31 |
| clarkb | subunit vs nose for example | 22:31 |
| clarkb | I think its ok to contineu doing that as it reduces the number of things people have to think about getting just right in order to have their jobs do what is expected of them | 22:32 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Don't install bindep if there's no bindep file https://review.openstack.org/501018 | 22:33 |
| mordred | pabelanger, clarkb, jamielennox: ^^ | 22:33 |
| mordred | I think that should further optimize for the case where there is no bindep file (no need to install anything if there's no file) | 22:33 |
| jamielennox | pabelanger: i'd agree with not installing bindep at all if not necessary | 22:36 |
| jamielennox | but like -infra i ended up embedding it in the image anyway | 22:36 |
| mordred | jamielennox, jlk: over the weekend I made these: https://review.openstack.org/#/q/topic:zuulv3-base-rollup | 22:36 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Only do the bindep_command if bindep_file is defined https://review.openstack.org/500683 | 22:37 |
| mordred | jamielennox, jlk: I don't expect us to have much time to think about them til post-ptg - but after chatting with jamielennox about consuming things directly last week, the idea popped into my head that we could make tracking that stuff easier -anyway - just wanted to toss them across your bow | 22:37 |
| jeblair | mordred: erm, can we WIP those till later? | 22:37 |
| mordred | jeblair: yup | 22:37 |
| mordred | jeblair: I just changed their topic so they wouldn't show up in people's review lists - but I'll go wip them explicitly | 22:38 |
| jlk | I saw those, and ignored them for the most part. LOTS of line changes, and failing tests. | 22:38 |
| jeblair | mordred: i think there's an alternative with more direct use of the base job but i don't have time to explore it now. | 22:38 |
| mordred | jeblair: wipd | 22:38 |
| mordred | jeblair: ++ - all marked WIP | 22:39 |
| clarkb | mordred: one potential problem with the bindep change is bindep_command no longer accepts a valid command but only a path to an executable | 22:43 |
| clarkb | mordred: how painful would it be to rename it to bindep_executable to make that clearer? | 22:44 |
| pabelanger | clarkb: mordred: fungi: jeblair: should we delete zuulv3-dev.o.o too to free up quota? | 22:44 |
| pabelanger | sorry, should have been in #openstack-infra | 22:44 |
| jlk | clarkb: that change shouldn't be too difficult to enact, as long as we agree on that color :D | 22:45 |
| mordred | clarkb, jlk: yah - we just need to add bindep_executable: to project-config zuul/site-variables first | 22:54 |
| mordred | then update the bindep role to consume executable instead of command - then remove command from zuul/site-variables | 22:54 |
| clarkb | mordred: can you see comment on https://review.openstack.org/#/c/500320/9 about requiring environment and environment_defaults? | 22:58 |
| mordred | clarkb: yah - can fix that - but I need to AFK for now | 23:02 |
| clarkb | mordred: oh doesn't look like environment defaults is used at all | 23:05 |
| clarkb | mordred: so maybe we collapse it down into a single required item? | 23:05 |
| clarkb | mordred: other than that stack for constraints lgtm. I think just want to make it clear what params are used such | 23:09 |
| jeblair | progress on the wget error -- i think the problem is in zuul_command, where we are getting some unexpected encoding from wget: | 23:12 |
| jeblair | 2017-09-05 23:10:08.979822 | ubuntu-xenial | 'Saving to: \xe2\x80\x98/opt/stack/devstack-corvus/files/etcd-v3.1.7-linux-amd64.tar.gz\xe2\x80\x99\n' | 23:12 |
| jeblair | (that's with zuul_command changed to use repr on the strings it gets) | 23:12 |
| jeblair | so we can see there are some unicode characters there | 23:12 |
| jlk | eww? | 23:12 |
| clarkb | those are quotes | 23:13 |
| jeblair | the devstack connection may be that devstack sets some local veriables to override system defaults | 23:13 |
| jeblair | clarkb: yep | 23:13 |
| clarkb | left and right single quote marks in utf8 | 23:13 |
| clarkb | mordred: pabelanger 229 lgtm now | 23:14 |
| jeblair | oh, actually, i think the devstack locale thing may be a red herring | 23:14 |
| jeblair | and in fact, my test wget job *did* fail | 23:14 |
| jeblair | i just didn't notice it | 23:15 |
| jeblair | because it just silently stopped logging but then continued | 23:15 |
| clarkb | jeblair: a quick local test of wget shows that saving to string uses utf8 quotes | 23:18 |
| clarkb | jeblair: so thats "normal" for wget in a utf8 locale looks like | 23:19 |
| jeblair | clarkb: yeah, that jives with what i'm seeing | 23:19 |
| jeblair | theoretically zuulv2 should be subject to this.. | 23:19 |
| clarkb | we don't stream that bit though | 23:20 |
| clarkb | in v2 I think we just redirect that to a file (the devstack log that is) | 23:20 |
| jeblair | clarkb: oooohhh... that's just redirected to a file? | 23:20 |
| clarkb | ya | 23:20 |
| jeblair | okay. neat. | 23:20 |
| jeblair | so we're a bit ahead of ourselves here :) | 23:20 |
| clarkb | pretty sure because in main console log we say "go look at the main log for this thing if it fails" | 23:20 |
| clarkb | and that file is devstacklog.txt | 23:20 |
| jeblair | i think the problem here is that we're attempting to encode the string at all -- it's a bytestring that we read from a pipe -- it should already be encoded | 23:23 |
| clarkb | I thought the websocket default was to utf8 or rather websockets require utf8? | 23:37 |
| clarkb | we may be failing at the read side though no the write I suppose | 23:37 |
| jeblair | clarkb: this is at the lowest level -- the ansible module that runs a command and pipes the output over a tcp socket | 23:38 |
| jeblair | (to be clear -- that's *zuul's* ansible module doing that) | 23:38 |
| jeblair | not an ansible problem per-se, it's where we hook into ansible to get command streaming | 23:39 |
| clarkb | jeblair: in ansible/callback/zuul_stream.py ? | 23:40 |
| jeblair | clarkb: ansible/library/zuul_console.py | 23:41 |
| jeblair | i have a fix; i'm just trying some more permutations of things that might go wrong now | 23:41 |
| jeblair | clarkb: Console.addLine | 23:41 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Be explicit about byte and encoding in command module https://review.openstack.org/501040 | 23:59 |
| jeblair | clarkb, mordred: ^ | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!