| SpamapS | ~/.local/share/systemd/user/{name of template goes here} means we can install one without root, assuming the executor gets write access to its HOME | 00:00 |
|---|---|---|
| SpamapS | or ~/.config/systemd/user | 00:01 |
| SpamapS | Yeah I think that might be the simple path to "pretty darn good" namespaced, cgrouped ansible-playbook | 00:04 |
| SpamapS | and if somebody wants to tack on selinux, that's their prerogative. | 00:04 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: Support swift logserver without Send-Temp-Url-Key https://review.openstack.org/270338 | 02:05 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: Add javascript license information https://review.openstack.org/335660 | 02:10 |
| openstackgerrit | Jamie Lennox proposed openstack-infra/nodepool feature/zuulv3: Remove remaining apscheduler variables https://review.openstack.org/445719 | 02:13 |
| mordred | jamielennox: ^^ there is a comment you remove in https://review.openstack.org/#/c/445719/1/nodepool/tests/__init__.py which indicates that that whole if condition might be removable ... | 03:03 |
| jamielennox | mordred: i tried removing that, however there are hundreds of those threads created over the life of the test | 03:04 |
| jamielennox | i have no idea what they are doing | 03:04 |
| mordred | oh fun! | 03:04 |
| jamielennox | so i removed the comment but not the conditional | 03:04 |
| mordred | jamielennox: wfm | 03:05 |
| mordred | jamielennox: for some reason when I get jetlag in europe it puts me on an australian schedule. I'm not sure what's up with that | 03:12 |
| jamielennox | mordred: adjusting to where you'd rather be? | 03:18 |
| mordred | jamielennox: that must be it | 03:23 |
| *** hashar has joined #zuul | 03:52 | |
| *** hashar has quit IRC | 05:29 | |
| *** isaacb has joined #zuul | 06:26 | |
| *** isaacb has quit IRC | 07:18 | |
| *** isaacb has joined #zuul | 07:25 | |
| rbergeron | /win 10 | 07:36 |
| *** Cibo_ has joined #zuul | 07:40 | |
| *** hashar has joined #zuul | 08:00 | |
| *** Cibo_ has quit IRC | 08:07 | |
| *** bhavik1 has joined #zuul | 08:12 | |
| *** bhavik1 has quit IRC | 08:25 | |
| *** dmellado has quit IRC | 09:54 | |
| *** dmellado has joined #zuul | 10:02 | |
| *** isaacb has quit IRC | 10:10 | |
| *** openstackgerrit has quit IRC | 10:18 | |
| *** Cibo_ has joined #zuul | 10:56 | |
| *** yolanda has quit IRC | 11:29 | |
| *** isaacb has joined #zuul | 11:29 | |
| *** yolanda has joined #zuul | 11:34 | |
| *** yolanda has quit IRC | 11:41 | |
| *** Cibo_ has quit IRC | 11:46 | |
| *** yolanda has joined #zuul | 11:48 | |
| *** yolanda has quit IRC | 12:05 | |
| *** yolanda has joined #zuul | 12:14 | |
| *** yolanda has quit IRC | 12:14 | |
| *** yolanda has joined #zuul | 12:14 | |
| *** hashar is now known as hasharLunch | 12:29 | |
| *** hasharLunch is now known as hashar | 12:39 | |
| Shrews | mordred: jamielennox: "Thread-" is the default name for threads (set by the threading lib) | 13:28 |
| mordred | Shrews: awesome | 13:28 |
| *** openstackgerrit has joined #zuul | 13:31 | |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Remove remaining apscheduler variables https://review.openstack.org/445719 | 13:31 |
| pabelanger | jhesketh: thanks for the reviews, will fix this morning | 13:45 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Remove noisy log line https://review.openstack.org/445964 | 13:49 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Populate requestor for min-ready requests https://review.openstack.org/445970 | 13:56 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Create run-cover role https://review.openstack.org/441332 | 14:22 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Create zuul_workspace_root job variable https://review.openstack.org/441441 | 14:22 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Organize playbooks folder https://review.openstack.org/441547 | 14:22 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Rename prepare-workspace role to bootstrap https://review.openstack.org/441440 | 14:22 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Add run-docs role and tox-docs job https://review.openstack.org/441345 | 14:22 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Add net-info to bootstrap role https://review.openstack.org/441617 | 14:22 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Add revoke-sudo role and update tox jobs https://review.openstack.org/441467 | 14:22 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Create tox-tarball job https://review.openstack.org/441609 | 14:22 |
| mordred | jeblair: sorry I missed the zuul meeting monday - this europe trip has decided to be one of the ones where my jetlag is terrible. should we sync on nodepool-shim? (I think that was the main outstandingquestion for me from reading the meeting minutes) | 14:26 |
| jeblair | mordred: lets! | 14:26 |
| mordred | jeblair: so - I have two patches up to get the ball rolling. of course, they break all the tests | 14:27 |
| mordred | jeblair: so first questoin is - what do we want to do about that? I think we didn't want to spend a ton of time making this thing rock solid ... but maybe that's a mistake and we _should_ go ahead and fix tests some? | 14:28 |
| jeblair | mordred: hrm. here's what i just wrote: yeah; i guess that's expected, unless we write a fake nodepool. that doesn't seem worth it (at the moment at least). | 14:28 |
| jeblair | mordred: however | 14:28 |
| mordred | setting up an integration test that spins up a v3 and a shim shouldn't be super hard - the unittests, on the other hand... | 14:28 |
| jeblair | mordred: i wrote a fake nodepool for zuul | 14:28 |
| mordred | hrm. good point | 14:29 |
| jeblair | mordred: so maybe getting tests to work is a matter of a couple hours? | 14:29 |
| mordred | maybe it is? maybe let's try to make them work for a couple of hours and if it's longer we just giv eup and do dorky empriical testing | 14:29 |
| jeblair | mordred: that sounds like a plan | 14:29 |
| mordred | jeblair: the patches so far are here: https://review.openstack.org/#/q/status:open+project:openstack-infra/nodepool+branch:feature/gearman-zk-shim - there's one that just tells the shim to ignore anything about min-ready )since the v3 should be handling that) | 14:31 |
| mordred | and then one that rips out all of the openstack stuff and just leaves the todo/stubs for single-node | 14:31 |
| jeblair | mordred: so aside from that, are the two patches that are up ready? if so, maybe the thing is to either rebase those 2 on (test fix | test removal). then i think there's at least one more thing that needs doing for multinode, right? | 14:31 |
| mordred | remaining non-test related steps are to put in zk calls in the todo place, and then to fix multi-node | 14:31 |
| jeblair | ok, so 2 things left to do | 14:32 |
| mordred | yah | 14:32 |
| mordred | and no, I need to fix tests for both patches - forcing min-ready to 0 turns out ot breaka lot of tests :) | 14:32 |
| mordred | I mean, they're enough what they need to be for a human to look at the patches and make sure the approach isn't stupid - but they are not landable | 14:33 |
| jeblair | mordred: yeah; so it probably makes sense to disable most of the tests anyway since they're not testing things of interest to the shim. | 14:33 |
| mordred | yah | 14:33 |
| mordred | hrm. actually - I'm not sure if the functional tests will test anything given that the functional tests _only_ test image upload and min-ready | 14:34 |
| jeblair | mordred: in fact, iirc, most of the v0 tests just rely on min-ready. | 14:34 |
| jeblair | mordred: :) | 14:34 |
| mordred | yah. this may actually just be a "remove tests and do empirical testing" - combined with adding some new tests to make sure that shim is making the right v3 zk calls | 14:35 |
| jeblair | mordred: yeah, i think that may be the case. i don't see any tests exercising the gearman demand pipepline. | 14:36 |
| mordred | me either | 14:36 |
| mordred | and since we're not touching that, we should be fairly confident that it's still working | 14:37 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Remove noisy log line https://review.openstack.org/445964 | 14:39 |
| *** mattclay has quit IRC | 14:40 | |
| *** mattclay has joined #zuul | 14:41 | |
| jeblair | mordred, Shrews: 445567 could use a review | 14:41 |
| Shrews | oh, wonder how i missed that | 14:44 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Update wait_for_threads https://review.openstack.org/445994 | 14:51 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Remove ready-script support https://review.openstack.org/445567 | 14:52 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Stop writing nodepool bash variable on nodes https://review.openstack.org/445572 | 14:52 |
| *** isaacb has quit IRC | 15:01 | |
| *** isaacb has joined #zuul | 15:15 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/gearman-zk-shim: Stub out methods needed for zk client https://review.openstack.org/435964 | 15:23 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/gearman-zk-shim: Hardcode min-ready to zero https://review.openstack.org/435963 | 15:23 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool feature/gearman-zk-shim: Pass zk object in to provider manager https://review.openstack.org/446012 | 15:23 |
| *** persia has quit IRC | 15:42 | |
| pabelanger | rcarrillocruz: jeblair: mordred: Shrews: With zuulv3 support for nodepool wrapping up, I'm curious when you think adding 141016 (nodepool rest api) to our weekly zuul meetings would be appropriate. It might be something I would be interested in helping develop. | 15:42 |
| *** persia has joined #zuul | 15:45 | |
| jeblair | pabelanger: i'd really rather we stay focused on the zuulv3 operational blockers. there's *so* much left to do there. | 15:47 |
| jeblair | pabelanger: also, there's still some substantial changes to nodepool (config syntax and docs) we need to make. | 15:47 |
| pabelanger | understood, I don't want to divert of that effort. No issues punting it for now | 15:49 |
| *** gundalow has quit IRC | 15:55 | |
| *** gundalow has joined #zuul | 15:55 | |
| *** hashar is now known as hasharMeeting | 16:01 | |
| *** hasharMeeting has quit IRC | 16:02 | |
| jeblair | pabelanger: i have a suggestion in 445594 | 16:05 |
| pabelanger | jeblair: looking | 16:06 |
| jeblair | mordred, pabelanger: 442114 442124 are ready for a +3 when you get a sec | 16:07 |
| *** isaacb has quit IRC | 16:08 | |
| SpamapS | looks like systemd-nspawn is about as simple as bubblewrap | 16:12 |
| SpamapS | but that name.. :-P | 16:12 |
| jeblair | SpamapS: i propose we rot13 encode the name in our code so we don't have to see it | 16:15 |
| jeblair | hereafter known as flfgrzq-afcnja | 16:15 |
| pabelanger | jeblair: 442124 appears to be in merge conflict | 16:18 |
| pabelanger | as is 442114 | 16:18 |
| jeblair | all the more reason to not let it sit another week | 16:18 |
| SpamapS | jeblair: genius | 16:20 |
| SpamapS | I'm finding systemd-nspawn harder to lock down | 16:20 |
| SpamapS | bubblewrap, by default, never wants to let root run in that namespace | 16:20 |
| SpamapS | systemd-nspawn seems to want to run a whole OS | 16:20 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Rename zuul-launcher to zuul-executor https://review.openstack.org/445594 | 16:21 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add per-repo public and private keys https://review.openstack.org/406382 | 16:23 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Port SQLAlchemy reporter to v3 driver structure https://review.openstack.org/442114 | 16:25 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Move alembic into sql driver https://review.openstack.org/442124 | 16:25 |
| pabelanger | jeblair: left comment on 442114 and +2 | 16:29 |
| mordred | SpamapS: that to me is a big vote in favor of bubblewrap :) | 16:31 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Remove more swift configurations https://review.openstack.org/446056 | 16:31 |
| jeblair | pabelanger: thanks ^ | 16:31 |
| SpamapS | mordred: yeah I may just be missing it because my systemd hate is hard to tamp down | 16:32 |
| mordred | SpamapS: embrace that hate | 16:32 |
| SpamapS | USE YOUR ANGER | 16:32 |
| pabelanger | jeblair: LGTM | 16:32 |
| SpamapS | https://review.openstack.org/444495 <-- updated just now to include all the facts we've learned, and moves "Run on nodes" to Alternatives with a message about why it is rejected. | 16:33 |
| mordred | SpamapS: I know it may be silly, but I would find it exceptionaly demoralizing if our new awesome thing had an explicit direct dependency on systemd | 16:33 |
| mordred | so, while my morale is not 100% of our concern, hopefully it's also not 0% of it | 16:33 |
| jeblair | mordred: this may be of little solace, but i think i read that systemd-nspawn isn't especially tied to pid 1 systemd. like, in the world of rainbows and unicorns, we'd drop systemd as an init system and keep systemd-nspawn as what it is because it's actually good at its job. | 16:36 |
| jeblair | but still. | 16:37 |
| mordred | jeblair: yah - and that is one of the reaosns I don't immediately hate the fact that rkt uses it | 16:37 |
| mordred | but yes. still. from a human/social perspective literally everything about the systemd suite of things represents everything I hate about where the tech industry is headed. and although I clearly cannot stop our mad dash into insanity, I don't have to actually actively help | 16:38 |
| SpamapS | jeblair is correct, systemd-nspawn does not depend on pid1 == systemd | 16:38 |
| mordred | jeblair: 442114 has test sad | 16:39 |
| SpamapS | bubblewrap though, seems like just what we want, truly | 16:39 |
| mordred | SpamapS: \o/ | 16:39 |
| mordred | SpamapS: thank you for doing rigorous work on this | 16:39 |
| SpamapS | I haven't deep dove on rkt yet | 16:40 |
| SpamapS | I might not bother | 16:40 |
| SpamapS | it's got so much stuff in it | 16:40 |
| SpamapS | AFAICT we don't really want any of that extra stuff | 16:40 |
| SpamapS | (less extra than Docker, but still.. extra) | 16:41 |
| mordred | yah. small/lean is win for us | 16:41 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Port SQLAlchemy reporter to v3 driver structure https://review.openstack.org/442114 | 16:41 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Move alembic into sql driver https://review.openstack.org/442124 | 16:41 |
| SpamapS | from what I see, bubblewrap is the new lxc 1.0 | 16:41 |
| jeblair | mordred: thanks. good thing we have a zuul. or two. | 16:41 |
| pabelanger | 445594 is also ready to be painted! Moving zuul-launcher to zuul-executor. I'll start working on puppet changes too | 16:41 |
| mordred | SpamapS: espeically since you can do things like bind-mount /usr as a readonly filesystem into the bubblewrap execution context, saving the need for creating an entire actual chroot | 16:42 |
| jeblair | yeah, let's merges 445594 asap because it's a major conflict magnet | 16:42 |
| mordred | +A | 16:44 |
| SpamapS | mordred: yeeeahh... I'm not sure I like that for this. | 16:44 |
| mordred | also - yes. I really like the new work | 16:44 |
| SpamapS | mordred: though it would accelerate adoption. | 16:44 |
| mordred | SpamapS: that's cool if not - just saying it's an option | 16:44 |
| SpamapS | mordred: but I really want to lock them down so at least all they have is python. ;) | 16:44 |
| SpamapS | but you know.. with python you can probably do whatever you need to get more. | 16:44 |
| SpamapS | so maybe there's an argument to just do it. | 16:44 |
| mordred | and rsync. and probably a few other small things | 16:45 |
| SpamapS | right there's a bunch of stuff needed... but python is the most dangerous | 16:45 |
| mordred | SpamapS: I believe you can give bubblewrap a list of directories and potentially even files to map in | 16:45 |
| * SpamapS alt-tabs back to ansibust-playbook | 16:45 | |
| SpamapS | mordred: yeah you can bind mount in whatever you wnat | 16:46 |
| mordred | SpamapS: so I _think_ we might be able to have the whitelist of exactly what we need, and then have bubblewrap do that for us | 16:46 |
| jeblair | SpamapS: updated spec lg | 16:46 |
| SpamapS | jeblair: \o/ | 16:46 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add per-repo public and private keys https://review.openstack.org/406382 | 16:47 |
| jeblair | die pep8 die | 16:47 |
| jeblair | rbergeron: are you going to make one of those snazzy update emails? | 16:51 |
| SpamapS | is there something like gofmt/rustfmt for Python where one can just put it in a pre-commit hook and not have to think? | 16:51 |
| jeblair | SpamapS: yes, but it's even more wrong than pep8 | 16:51 |
| mordred | yah. they're al terrible | 16:52 |
| mordred | I keep trying them | 16:52 |
| mordred | to see if I can make them not be terrible | 16:52 |
| mordred | but I can't | 16:52 |
| SpamapS | that sucks | 16:52 |
| mordred | yah | 16:52 |
| clarkb | jeblair: mordred correct you can run systemd nspawn independently | 16:52 |
| mordred | and I even have more extreme/simplistic views on code formatting | 16:52 |
| SpamapS | I very much like that whenever I save my stuff in rust it just gets formatted the right way. | 16:52 |
| SpamapS | (or tells me it can't figure out how to) | 16:53 |
| SpamapS | hey here's a funny thought... | 16:53 |
| mordred | and with those views, I _still_ can't get the python fmt tools to behave in accordance with them | 16:53 |
| * mordred waits for today's suggestoin from SpamapS that we rewrite everythign in rust ... :) | 16:53 | |
| SpamapS | nevermind | 16:53 |
| * mordred kidding - go ahead! | 16:54 | |
| * SpamapS has been up since 0330 .. probably need a nap | 16:54 | |
| mordred | oy | 16:54 |
| mordred | that's horrible | 16:54 |
| SpamapS | babies | 16:54 |
| SpamapS | I was going to suggest we turn ansible-playbook into something like a pantsbuild PEX thing so it won't give users /usr/bin/python | 16:54 |
| SpamapS | but that's not actually what pants or PEX do | 16:55 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Rename zuul-launcher to zuul-executor https://review.openstack.org/445594 | 16:55 |
| mordred | SpamapS: yah - I see where you're going with that ... but also yeah, I think there be dragons | 16:55 |
| clarkb | SpamapS: I like this in nspawn man page "Like all other systemd-nspawn features, this is not a security feature and provides protection against accidental destructive operations only" | 16:57 |
| clarkb | so it may be the wrong option given their intentions | 16:57 |
| mordred | ++ | 16:57 |
| clarkb | (also that goes nicely with what you said earlier about it being harder to lock down) | 16:57 |
| clarkb | that said, I think thats generally the state of linux containering in general | 16:58 |
| mordred | clarkb: yah - that's the reason I think it's important to still do our ansible-level lockdown too | 16:59 |
| SpamapS | clarkb: yeah, vs bubblewrap's claims "The maintainers of this tool believe that it does not, even when used in combination with typical software installed on that distribution, allow privilege escalation. It may increase the ability of a logged in user to perform denial of service attacks, however." | 17:01 |
| mordred | SpamapS: intent of maintainers matters, it turns out | 17:01 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Update wait_for_threads https://review.openstack.org/445994 | 17:03 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Clean up additional references to launcher https://review.openstack.org/446073 | 17:04 |
| pabelanger | mordred: jeblair: ^ missed 2 references of launcher | 17:05 |
| jlk | So bubble wrap seems the way forward? | 17:07 |
| SpamapS | jlk: it certainly seems like the biggest bang for buck. | 17:08 |
| mordred | jlk: it's looking very promising | 17:08 |
| SpamapS | I mean, at the end of the day, it's still a kernel vulnerability away from break out. | 17:08 |
| jlk | I had a thought last night, that depending on something like rkt or other containers, that it would make it awkward to run executerd itself in a container, should a site wish to go with container based deployments. | 17:08 |
| SpamapS | So layering SELinux on top of it seems like a good next step. | 17:08 |
| jlk | I need to read up on bubblewrap | 17:08 |
| jlk | probably telling that it's what Tower uses too | 17:08 |
| SpamapS | jlk: yes! bubblewrap is specifically designed to work in that environment. | 17:08 |
| pabelanger | SpamapS: do you mind sharing the syntax you used to launch bubblewrap? | 17:09 |
| SpamapS | (though not with default Xenial kernel) | 17:09 |
| jlk | SpamapS: oh good, so you can bubblewrap _inside_ a container | 17:09 |
| jlk | oh geez, has kernel requirements? | 17:09 |
| jlk | Can we just switch to Fedora based deployments? ;) | 17:09 |
| SpamapS | pabelanger: after untarring my chroot, I use this in the chroot root "bwrap --bind $PWD / --ro-bind /etc/resolv.conf /etc/resolv.conf --dev /dev --unshare-all --share-net --proc /proc bash" | 17:09 |
| SpamapS | jlk: something like 4.6 and later have extra USER_NS stuff that let bwrap work inside namespaces yeah. | 17:10 |
| pabelanger | SpamapS: danke, I'll give it a run later tonight | 17:10 |
| jeblair | good, we should keep the door open for container deployments of zuul. structurally speaking, it's a good fit. | 17:10 |
| mordred | ++ | 17:10 |
| jlk | ++ | 17:10 |
| jlk | I spent a couple hours last night playing with Ansible Container | 17:10 |
| SpamapS | jlk: it would be less awkward to switch to Yakkety (Ubuntu 16.10) which has the right kernel, than Fedora. ;) | 17:11 |
| jlk | I'm ... more pleased with it than my first run through. | 17:11 |
| clarkb | SpamapS: except that yakkety's support term is even worse than fedoras | 17:11 |
| clarkb | which is pretty hard to comprehend | 17:11 |
| jlk | SpamapS: awkward for whom? ;) I'm sure I could knock out a giant patch to Hoist in a few hours. | 17:11 |
| SpamapS | jeblair: I have it on my TODO to try putting Zuul in kubernetes for BonnyCI .. you know.. when I have time. | 17:11 |
| jeblair | SpamapS: ++ | 17:11 |
| jeblair | i would find rhel an easy sell :) | 17:11 |
| SpamapS | jlk: All of us who use Ubuntu every day. :-P | 17:12 |
| pabelanger | I've been trying to put zuul in openshift, but I have yet to find openshift credentials | 17:12 |
| jlk | SpamapS: I am going ot take another stab at doing zuul in ansible container, so that we can build and run locally, and also push to K8S | 17:12 |
| * mordred supports all of the people doing all of the container things | 17:12 | |
| SpamapS | clarkb: hah, yeah, they definitely made those irrelevant by shortening. :-P | 17:12 |
| jlk | I still really like hte idea of building images during CI, testing them, pushing them to registry, and using those exact images in dev/test/prod | 17:12 |
| clarkb | SpamapS: worse than irrelevant, I keep seeing people using them as if the old term was still in place so now its a massive security headache | 17:12 |
| pabelanger | is there like public, free, k8s things you can register for? | 17:12 |
| jlk | SpamapS: hand waves.... | 17:12 |
| mordred | jlk: yah - although at least in infra land we have a few tentpoles to deal with before we can do that | 17:13 |
| jlk | pabelanger: there's minikube | 17:13 |
| pabelanger | I'd like to avoid standing on up if possible | 17:13 |
| mordred | pabelanger: google compute | 17:13 |
| jlk | https://kubernetes.io/docs/getting-started-guides/minikube/ | 17:13 |
| jlk | oh yeah GCP | 17:13 |
| pabelanger | mordred: yes! I have an account for that | 17:13 |
| SpamapS | jlk: FTR, I'm totally down for Fedora'ing or even Tumbleweeding ;) | 17:13 |
| pabelanger | cool | 17:13 |
| clarkb | SpamapS: I'd go tumbleweed >_> | 17:13 |
| mordred | SpamapS, jlk: I'm totally in favor of y'all fedoring or tumbleweeding and think we should definitely support running on those | 17:14 |
| jlk | We all have our personal biases | 17:14 |
| clarkb | I do wish these distros would all stop making iptables impossible to use though | 17:14 |
| eventingmonkey | Arch? | 17:14 |
| eventingmonkey | <_< | 17:14 |
| clarkb | debuntu seem to get that right which is nice | 17:14 |
| jlk | bro, do you even firewalld ? | 17:14 |
| eventingmonkey | >_> | 17:14 |
| SpamapS | pretty sure iptables made iptables impossible to use | 17:14 |
| clarkb | jlk: no because it unworks :) | 17:14 |
| * SpamapS still longs for OpenBSD pf in Linux | 17:14 | |
| jlk | I mean, you get systemd, SELinux, firewalld, what more do you want‽ | 17:15 |
| pabelanger | sigh, pep8 | 17:15 |
| mordred | clarkb: you're forgetting, the trend in computers is to make them easy for people to use who don't know how to use computers at the expense of the people who do | 17:15 |
| clarkb | jlk: I don't want firewalld because it has a habit of actually not firwalling things | 17:15 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Clean up additional references to launcher https://review.openstack.org/446073 | 17:15 |
| pabelanger | mordred: jeblair: sorry, pep8 got me | 17:15 |
| clarkb | jlk: in addition to making services like libvirt really really slow | 17:15 |
| pabelanger | ^ | 17:15 |
| jlk | ugh yes | 17:15 |
| jlk | I'm pretty OKAY with getting out of the business of running libvirtd .... | 17:15 |
| SpamapS | yesplease | 17:19 |
| clarkb | but where would I run dib? | 17:21 |
| jlk | on Somebody Else's Computer | 17:22 |
| SpamapS | Now you're just an instance that I used to know. | 17:22 |
| * clarkb likes running things locally and also likes the added extra bit of security VMs provide. Also not worrying about kernels | 17:25 | |
| pabelanger | clarkb: I managed to put DIB in a docker over christmas (present to myself?), which I then build a ubuntu-xenial tarball. But so many turtles | 17:25 |
| pabelanger | rather just stick with kvm for that | 17:26 |
| openstackgerrit | Merged openstack-infra/nodepool master: Default config-drive to true https://review.openstack.org/349697 | 17:36 |
| jeblair | fungi: in the zuulv3 spec we said "PKCS1 with RSAES-OAEP (implemented by the Python cryptography library)" https://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html#secrets | 17:38 |
| jeblair | fungi: but i'm not seeing a lot of support for pkcs1/rsaes-oaep. maybe i'm not looking in the right place? | 17:38 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Clean up additional references to launcher https://review.openstack.org/446073 | 17:39 |
| jeblair | (pycryptodone does seem to have some easy-to-use rsaes-oaep methods) | 17:39 |
| jeblair | pycrpytodome even | 17:39 |
| fungi | jeblair: i could swear i pulled up the api docs for it to confirm. just a sec and i'll revisit | 17:39 |
| jeblair | yeah, that's how i remembered it too :/ | 17:40 |
| fungi | jeblair: https://cryptography.io/en/latest/hazmat/primitives/asymmetric/rsa/#encryption shows an example | 17:42 |
| jeblair | fungi: thank you, my search-fu was bad. :( | 17:43 |
| fungi | np, it's not the easiest documentation (or even field of technology) to follow | 17:43 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Reset lost requests https://review.openstack.org/446095 | 18:06 |
| Shrews | pabelanger: jeblair: I'm *hoping* that change ^^^ will allow us to restart nl01 and allow it to cleanup after itself. | 18:06 |
| pabelanger | looking | 18:07 |
| Shrews | The state has changed on nl01 since I last looked, so not quite sure what's going to happen on a restart :/ | 18:07 |
| Shrews | Hrm, we might need some code to reset Node.allocated_to if the request is gone, too | 18:08 |
| Shrews | yeah, i think we should add that, too, before a restart | 18:09 |
| *** Cibo_ has joined #zuul | 18:12 | |
| jlk | Oh for the love of Jibbers. | 18:13 |
| jlk | Just wasted more time than I'd like to admit because my zuul.yaml had a job that didn't indent keys far enough | 18:13 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Create run-cover role https://review.openstack.org/441332 | 18:16 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Create zuul_workspace_root job variable https://review.openstack.org/441441 | 18:16 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Organize playbooks folder https://review.openstack.org/441547 | 18:16 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Rename prepare-workspace role to bootstrap https://review.openstack.org/441440 | 18:16 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Add run-docs role and tox-docs job https://review.openstack.org/441345 | 18:16 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Add net-info to bootstrap role https://review.openstack.org/441617 | 18:16 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Add revoke-sudo role and update tox jobs https://review.openstack.org/441467 | 18:16 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Create tox-tarball job https://review.openstack.org/441609 | 18:16 |
| jeblair | jlk: current feature/zuulv3 tip should be pretty good about outputting error messages in config files; is there something we should improve? | 18:18 |
| jeblair | (cause that's not something we want to put users through) | 18:18 |
| jlk | I'll see if I can re-create it | 18:18 |
| jlk | it was an error about reading the config, but I was wrongly thinking it was a schema issue rather than a formatting issue | 18:19 |
| jeblair | ah, then we might be in the territory of heuristic suggestions, which we don't do (yet) | 18:19 |
| clarkb | ansible suffers the same issue. I think pyyaml may make detecting that hard | 18:19 |
| clarkb | jeblair: ya | 18:20 |
| clarkb | beacuse its not strictly invalid? but is invalid when you look for specific keys as specific levels? | 18:20 |
| jlk | this is what I see http://paste.openstack.org/show/602912/ | 18:21 |
| jlk | This is the diff that caused it http://paste.openstack.org/show/602914/ | 18:22 |
| jeblair | ah, we can make that error better; we should be able to point the user at a file location. | 18:24 |
| jeblair | instead of, you know, printing out the entire configuration as a string. :) | 18:25 |
| jlk | heheh. Yeah. Thankfully I knew only one thing had changed in the file since the last time tests passed | 18:25 |
| jlk | I knew _where_ the error was, just couldn't see the indentation for the forest. | 18:25 |
| jlk | http://i3.kym-cdn.com/photos/images/original/001/211/814/a1c.jpg WHAT INDENTATION | 18:27 |
| jeblair | i think this case is constrained enough we can apply a heuristic to that and say "i see a job definition but it looks like the following keys are not indented enough" | 18:28 |
| *** hashar has joined #zuul | 18:29 | |
| jeblair | alternatively, we can probably apply a basic top-level config object validator, which i don't think we do now.... | 18:29 |
| jeblair | i'll poke at those after i make progress on the encrypted stuff | 18:29 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Deallocate ready nodes with no requests https://review.openstack.org/446110 | 18:31 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: GitHub file matching support https://review.openstack.org/446113 | 18:34 |
| pabelanger | mordred: able to help land 442114, a merge conflict waiting to happen | 18:46 |
| SpamapS | I thought Ansible started using more deep yaml errors so they didn't just throw pyyaml's drivel at you. | 19:02 |
| rbergeron | jeblair: indeed i am, this afternoon or tonight is when ill get there (metal tubing yesterday) | 19:07 |
| clarkb | SpamapS: they may have but I think current release gives you the obtuse error still | 19:11 |
| SpamapS | clarkb: http://paste.openstack.org/show/602920/ <-- flamel just does pyyaml ... so ansible is definitely cleaning it up | 19:15 |
| SpamapS | that's with 2.2.0 | 19:15 |
| clarkb | oh neat | 19:16 |
| *** Cibo_ has quit IRC | 19:17 | |
| jeblair | structurally that's just the pyyaml error without the start marker highlighted | 19:18 |
| jeblair | (obviously, it's much more helpful text and includes file locations; we've already done that for zuul. so zuul errors look like a combination of the two) | 19:18 |
| SpamapS | Yeah, beyond that, as you said, you need heuristics | 19:19 |
| SpamapS | because if it parses as yaml, but the mapping is actually a list.. ruhroh. | 19:19 |
| jeblair | SpamapS: indeed -- that's where voluptuous comes in for us | 19:20 |
| jeblair | jlk found a hole between pyyaml and voluptuous | 19:20 |
| SpamapS | orly? | 19:20 |
| SpamapS | impressive. | 19:20 |
| jeblair | yep. we can plug it though. | 19:21 |
| SpamapS | Call that the Vezzini error. "You keep using tha format.. I do no think it means wha you think it means" | 19:21 |
| jeblair | we should apply that heuristic to the word 'gate' :) | 19:22 |
| jeblair | (so we can gate all our gates on the gate gate) | 19:22 |
| jeblair | i think i'll go eat some brisket now | 19:23 |
| jlk | I guess I'm good at finding holes. | 19:31 |
| *** Cibo_ has joined #zuul | 19:37 | |
| pabelanger | <3 voluptuous | 19:38 |
| *** Cibo_ has quit IRC | 19:42 | |
| *** Cibo_ has joined #zuul | 19:42 | |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Populate requestor for min-ready requests https://review.openstack.org/445970 | 19:44 |
| *** herlo has quit IRC | 19:49 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Log GitHub API rate limit https://review.openstack.org/446150 | 19:52 |
| openstackgerrit | Merged openstack-infra/nodepool feature/gearman-zk-shim: Hardcode min-ready to zero https://review.openstack.org/435963 | 19:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add secret top-level config object https://review.openstack.org/446156 | 20:06 |
| jeblair | i admit, that reads kind of weird. it's totally not a secret, you can tell people about it. | 20:06 |
| clarkb | I read it as "add top secret level config object" the first tiem | 20:10 |
| openstackgerrit | Jamie Lennox proposed openstack-infra/nodepool feature/zuulv3: Split webapp into its own nodepool application https://review.openstack.org/445675 | 20:18 |
| pabelanger | jeblair: A question on 446156 when you are free | 20:19 |
| jeblair | pabelanger: yep, that's an error | 20:20 |
| jeblair | (we would have caught it when i get around to using the value in a test) | 20:20 |
| pabelanger | okay, cool | 20:20 |
| pabelanger | I wasn't sure if it was optional | 20:20 |
| openstackgerrit | Jamie Lennox proposed openstack-infra/nodepool feature/zuulv3: Allow loading logging config from yaml https://review.openstack.org/445656 | 20:29 |
| jhesketh | Morning | 20:30 |
| pabelanger | jhesketh: ohai! Thanks for the reviews yesterday, I've pushed up new patches | 20:33 |
| jhesketh | pabelanger: no worries | 20:33 |
| jhesketh | cool, will take a look :-) | 20:33 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add secret top-level config object https://review.openstack.org/446156 | 20:40 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Add reconfigure_handler for executor logging https://review.openstack.org/446172 | 20:40 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add secret top-level config object https://review.openstack.org/446156 | 20:40 |
| jhesketh | pabelanger: did you see my comment on https://review.openstack.org/#/c/441440/ | 20:41 |
| pabelanger | jeblair: jamielennox: ^446172 might be something we want to zuul-executor too, the ability to reload logging with SIGHUP | 20:41 |
| pabelanger | jhesketh: I actually see that role expanding as we start adding more jobs, so bootstrap seem like a decent default for now. Eventually, I see us having more roles in the bare-pre.yaml playbook and possible that boostrap would become another name. | 20:44 |
| pabelanger | jhesketh: base role works too | 20:45 |
| jhesketh | pabelanger: you wouldn't have them as separate playbooks? | 20:45 |
| jhesketh | since they can be stacked | 20:45 |
| pabelanger | maybe? | 20:46 |
| pabelanger | I think right now, I'm happy to let it grow organically and see where the bootstrap role goes. Then, we can start to refactor things as more jobs need different base things bootstrapped | 20:47 |
| clarkb | jeblair: given the grump over using sha1 for secure things can/should we use something other than sha1 in the example encryption code? | 20:50 |
| jhesketh | pabelanger: sure, I'm not too fussed either. Happy to merge it, but organically I would have thought prepare-worksapce was a good starting point ;-) | 20:51 |
| clarkb | I'm assuming other hashes can be used and cryptography will sort it out, but its eaasy to copy pasta existing code and if it uses sha1 that could cause cargo culting that is undesirable? | 20:51 |
| SpamapS | clarkb: indeed, anything new should consider sha1 dead, even if the new thing is just example/doc code. | 20:52 |
| jlk | oh I see its going to be one of those kinds of days. zuul is failing my tests in review.openstack.org, but they pass locally | 20:53 |
| SpamapS | jlk: you forgot to hook up the doll? | 20:53 |
| jlk | on review. they are generating 67 megs worth of console output | 20:53 |
| * SpamapS hopes some day, somebody has also seen Weird Science, and gets that. | 20:54 | |
| jlk | I've seen it, a long time ago | 20:54 |
| jamielennox | pabelanger: i think you need to do a bit more to reconfigure logging rather than just add new ones to the existing, but can test that out later | 20:54 |
| SpamapS | It was practically on a loop in my house on account of my brother being in love with Kelly Labrocke. | 20:55 |
| pabelanger | jhesketh: I mostly renamed it because of some naming conventions I follow for role varibles. EG: <role name>_<variable_name>, And I found prepare_workspace_workspace_root a little odd, renaming to bootstrap, gave me bootstrap_workspace_root. That was the real motivation for the rename | 20:56 |
| pabelanger | jamielennox: should work, that logic is the same in cmd/zuul.py | 20:56 |
| jhesketh | pabelanger: fair enough | 20:56 |
| jeblair | pabelanger: is "change logging configuration" is something we expect to happen often enough to warrant implementing a partial reconfiguration handler? | 21:01 |
| jeblair | we usually go months or years between changing our logging config | 21:01 |
| jeblair | clarkb: indeed, the latest version of the cryptography docs have updated their example to sha256 | 21:03 |
| jeblair | clarkb: er, wait, i'm wrong it still uses sha1 there | 21:04 |
| pabelanger | jeblair: agree, but it does mean a difference between how scheduler and executor do things for logging | 21:05 |
| jeblair | pabelanger: true, though i think that may be a complete reconfiguration for the scheduler. | 21:07 |
| pabelanger | jeblair: right, it also does that, along with logging | 21:08 |
| jeblair | (it actually does reload the entire configuration file and update connections, etc) | 21:08 |
| clarkb | jeblair: ok left proper comment on the change | 21:08 |
| pabelanger | jeblair: it was just something I noticed updating ansible-role-zuul for zuul-executor were we reload the process if logging changes. Got me looking to see if that actually worked. | 21:09 |
| pabelanger | so, I can go either way | 21:09 |
| jeblair | clarkb: i'm not following your 2nd comment -- you're asking why the validator allows str|encrypted? | 21:11 |
| clarkb | jeblair: ya, and if str is allowed why bother with encrypted at all (just use built in str) | 21:11 |
| clarkb | using built in str solves the other comment when checking equality too I think | 21:11 |
| jeblair | clarkb: well, i guess we could have the pyyaml constructor create a str instead of an encrypted str object. but then later when we use it, we'll have to do substring comparisons everywhere to decide if we want to decrypt it. | 21:12 |
| clarkb | jeblair: oh, gotcha so intent there is to allow non encrypted strings too? | 21:13 |
| clarkb | for some reason I assumed all secrets would be encrypted | 21:13 |
| jeblair | clarkb: ah, yes. username might be plain text, password might be encrypted | 21:13 |
| jeblair | clarkb: we allow that so you can bundle them up together, rather than having to use two separate facilities. | 21:14 |
| clarkb | for the other thing I think because deepcopy is used you won't just get new refs to same object so foo == bar will always be false | 21:15 |
| clarkb | (except with primitives like str?) | 21:15 |
| clarkb | pretty sure my naive test confirmed that is a problem locally | 21:16 |
| jeblair | clarkb: i believe deepcopy will create new objects, however, i likely need to implement an eq method | 21:16 |
| clarkb | ya that | 21:16 |
| jlk | I apologize for the incoming flood | 21:43 |
| jlk | I had to rebase all my patches for the executor name change. | 21:44 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: support github pull reqeust labels https://review.openstack.org/444511 | 21:44 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Allow github trigger to match on branches/refs https://review.openstack.org/445625 | 21:44 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Support for dependent pipelines with github https://review.openstack.org/445292 | 21:44 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Add basic Github Zuul Reporter. https://review.openstack.org/443323 | 21:44 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Configurable SSH access to GitHub https://review.openstack.org/444034 | 21:44 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Add 'push' and 'tag' github webhook events. https://review.openstack.org/443947 | 21:44 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Add 'pr-comment' github webhook event https://review.openstack.org/443959 | 21:44 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Support for github commit status https://review.openstack.org/444060 | 21:44 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Better merge message for GitHub pull reqeusts https://review.openstack.org/445644 | 21:44 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Encapsulate determining the event purpose https://review.openstack.org/445242 | 21:44 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Allow using webapp from connections https://review.openstack.org/439831 | 21:44 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Merge pull requests from github reporter https://review.openstack.org/444463 | 21:44 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Support GitHub PR webhooks https://review.openstack.org/439834 | 21:44 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: GitHub file matching support https://review.openstack.org/446113 | 21:44 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Log GitHub API rate limit https://review.openstack.org/446150 | 21:44 |
| jeblair | clarkb, fungi: https://security.stackexchange.com/questions/112029/should-sha-1-be-used-with-rsa-oaep | 21:53 |
| *** jamielennox has quit IRC | 21:55 | |
| fungi | yeah, catching up on scrollback, i don't consider this a slippery slope use for sha-1, it's more that if the underlying library we want to use hasn't implemented an alternative yet, then we're using what's available (and it is a safe enough choice for a while to come, most likely) | 21:56 |
| jeblair | answered by someone with a mushroom on his homepage and a phd thesis in cryptography, if that helps you evaluate trust. | 21:56 |
| SpamapS | jeblair: good find .. that answer clarified some things for me. :) | 21:57 |
| SpamapS | the mushroom definitely sells it for me. | 21:57 |
| fungi | the mushroom definitely | 21:57 |
| jeblair | http://www.bolet.org/~pornin/cv-en.html | 21:57 |
| fungi | i mean, i have one on my homepage too | 21:57 |
| fungi | the phd less so, but meh | 21:57 |
| jeblair | fungi: you can put a phd on your homepage! | 21:57 |
| fungi | this is true | 21:57 |
| fungi | oh, hey, he's the same age as me too | 21:58 |
| jeblair | fungi: you found your french twin | 21:58 |
| fungi | scary, scary stuff | 21:58 |
| SpamapS | oh and he's french canadian, so the fact that the answer is a little cranky means he's definitely annoyed he has to tell people why SHA1 would be fine but we still use SHA256 because of perception. | 21:58 |
| SpamapS | Typing that answer probably ruined his Poutine. | 21:59 |
| jlk | lolol | 22:03 |
| jlk | now I want poutine | 22:03 |
| jeblair | fungi, clarkb, SpamapS: i wonder if the size limit is going to be a problem for us. it seems with 4096 bit keys and sha1 we can encrypt 470 bytes (3060 bits). | 22:03 |
| fungi | though it raises an interesting point... by using rsa-oaep we're limiting the length... | 22:03 |
| jeblair | er 3760 bits | 22:03 |
| fungi | reading my mind again | 22:03 |
| SpamapS | jeblair: I feel like we may be asked to encrypt giant API keys. | 22:04 |
| SpamapS | Or tempurls | 22:04 |
| fungi | may want to switch to a block cipher instead | 22:04 |
| jeblair | SpamapS: yes | 22:04 |
| SpamapS | 470 is still kinda ok for those.. | 22:04 |
| SpamapS | I'm pretty crypto-naive, but I would have chosen PGP encapsulation. I'm sure there are good reasons not to though. | 22:05 |
| clarkb | re sha1 being fine the problem is you have computing envs that kill it and md5 entirely | 22:05 |
| clarkb | then even when use of such things is fine you have problems. Swift has this problem with their use of md5 | 22:05 |
| SpamapS | clarkb: yeah, I think the answer kind of mentioned that. | 22:06 |
| clarkb | so its just easier overall to avoid the problem (if possible) | 22:06 |
| fungi | the original model was patterned on an existing implementation elsewhere (hiera?) | 22:06 |
| jeblair | fungi: yeah; i think it used pkcs7? | 22:06 |
| fungi | right, we originally said pkcs7 until we discovered none of the good libs available had a pkcs7 implementation | 22:06 |
| *** hashar has quit IRC | 22:09 | |
| fungi | and pkcs1 was selected as the next closest (albeit less featureful) alternative | 22:12 |
| fungi | we can probably just pick a padded block cipher and wrap it in pem encoding or something if length is a concern (takes up a lot more space though) | 22:14 |
| *** jamielennox has joined #zuul | 22:15 | |
| SpamapS | so, humor me. Why not just wrap it in pgp so we have a header and introspection tools? | 22:15 |
| fungi | no objection here | 22:16 |
| SpamapS | trying not to bikeshed on it.. You both know more than I do. :) | 22:16 |
| fungi | again, the original design was based on prior art. i agree that if constraints indicate we need to diverge heavily from that then something like openpgp is not out of the question | 22:16 |
| fungi | we need something asymmetric and suitable for embedding into yaml | 22:17 |
| fungi | beyond that, having an existing and vetted/popular implementation we can rely on in library form without needing to come up with it ourselves is a huge plus | 22:18 |
| jlk | yaml and secrets you say... | 22:20 |
| jlk | what does Ansible vault use? | 22:20 |
| fungi | asymmetric crypto algorithms pretty much all revolve around encrypting small amounts of data. if you want to encrypt a large(-ish) amount of data then you either need a proxy to symmetric crypto (e.g. asymmetrically encrypt a symmetric key) or some form of block chaining (which is tricky to get right, so don't implement it from scratch yourself) | 22:20 |
| jeblair | also, we did make this forward compatible with new systems by using the yaml tag; so we can do both. but if we decide we want to just do gpg out of the gate, then maybe we shouldn't bother with pkcs1 | 22:24 |
| jeblair | jlk: like http://docs.ansible.com/ansible/playbooks_vault.html#single-encrypted-variable ? | 22:27 |
| jlk | jeblair: yeah, ansible-vault is a way of encrypting yaml keys or whole files (that could be yaml). I have no idea what it's using under the hood, and I'm way out of my league here with crypto, just thought I'd mention it. | 22:28 |
| jlk | I honestly don't even know the problem space, so I'm likely just making incoherent noises. | 22:28 |
| fungi | the possibly unique twist in zuul's case is that we want everyone to be able to submit already-encrypted data into public git repos which only our zuul server can decrypt and use | 22:29 |
| fungi | so asymmetric crypto is an obvious solution, in the most general sense | 22:30 |
| jeblair | it looks like vault uses aes, which is symmetric | 22:31 |
| fungi | their use case is presumably much different (shared keys for writer and reader) | 22:31 |
| jlk | yup, fair enough | 22:31 |
| jeblair | ya | 22:31 |
| fungi | the prior art we were looking to for inspiration was presumably https://github.com/voxpupuli/hiera-eyaml | 22:33 |
| fungi | so to SpamapS's point, there's also https://github.com/sihil/hiera-eyaml-gpg | 22:33 |
| jeblair | it looks like the main beef with that was that it encrypted the whole file. i don't see why one still couldn't use gpg to encrypt individual values. | 22:35 |
| fungi | nor do i. encrypting arbitrary strings with gnupg or similar openpgp implementations is trivial | 22:36 |
| jeblair | i guess the downside is that they all mostly shell out to gpg, yeah? so we'd be managing a keyring file, and operations would be a little slower? | 22:36 |
| jeblair | (we can hide the keyring implementation detail, i bet) | 22:37 |
| fungi | probably. for that matter, we could shell out to openssl's pkcs#7 implementation | 22:37 |
| jeblair | huh. didn't think of that. :) | 22:38 |
| fungi | for some perspective, here's a three-byte string encrypted to my personal 4k rsa key: http://paste.openstack.org/show/602939/ | 22:39 |
| SpamapS | Does libgcrypt not implement all the bits of PGP? | 22:39 |
| fungi | considering https://pypi.python.org/pypi/pygcrypt for that i guess? | 22:42 |
| jeblair | the source is not as accessible as i would hope: https://framagit.org/okhin.pygcrypt/ | 22:43 |
| SpamapS | C extensions aren't that hard to write | 22:43 |
| pabelanger | wow scrollback | 22:45 |
| fungi | yeah, at least libgcrypt source is hosted with gnupg itself https://git.gnupg.org/cgi-bin/gitweb.cgi?p=libgcrypt.git | 22:45 |
| fungi | interesting that libgcrypt seems to be lgpl v2.1+ while pygcrypt is gpl v3 (if i'm reading them correctly) | 22:48 |
| jeblair | using libgcrypt looks a little more involved. perhaps easier to shoot feet. | 22:53 |
| *** Cibo_ has quit IRC | 23:17 | |
| *** Cibo_ has joined #zuul | 23:30 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
