| mattoliverau | go tdasilva! | 00:02 |
|---|---|---|
| NM | A lot of compliments from brazilian openstack community. | 00:04 |
| NM | notmyname: Can be a screenshot? It's on hangouts. :) | 00:11 |
| notmyname | nah, it's fine. I thought it was at a conference or meetup or something in person | 00:12 |
| *** ManojK has joined #openstack-swift | 00:16 | |
| *** NM has quit IRC | 00:19 | |
| *** Suyash has quit IRC | 00:19 | |
| notmyname | I had no idea there was a swift-related meetup in virginia today https://twitter.com/OpenStackNova/status/748293865111584768 | 00:24 |
| notmyname | buch of recent tweets on https://twitter.com/OpenStackNova | 00:25 |
| mattoliverau | Talking openstack swift at the openstack nova meetup... ok. | 00:28 |
| mattoliverau | cool of course, but still.. ok | 00:28 |
| mattoliverau | ?? | 00:28 |
| *** Jeffrey4l has joined #openstack-swift | 00:36 | |
| *** siva_krish has quit IRC | 00:44 | |
| *** lyrrad has quit IRC | 00:46 | |
| *** gyee has quit IRC | 00:50 | |
| *** links has joined #openstack-swift | 00:57 | |
| *** tqtran has quit IRC | 00:59 | |
| *** Suyash has joined #openstack-swift | 00:59 | |
| *** nadeem has quit IRC | 01:03 | |
| torgomatic | mattoliverau: openstack NoVa, as in Northern Virginia, I think | 01:04 |
| torgomatic | alternately, "openstack doesn't go" in Spanish :p | 01:04 |
| *** klamath has quit IRC | 01:05 | |
| *** klamath has joined #openstack-swift | 01:05 | |
| *** takashi has joined #openstack-swift | 01:10 | |
| *** klamath has quit IRC | 01:14 | |
| *** klamath has joined #openstack-swift | 01:15 | |
| *** greghaynes has quit IRC | 01:16 | |
| *** klrmn has quit IRC | 01:18 | |
| notmyname | torgomatic: ah ok. I had the same comfusion | 01:19 |
| *** asettle has joined #openstack-swift | 01:21 | |
| mattoliverau | torgomatic: ahhh, well that makes a whole lot more sense! | 01:22 |
| *** asettle has quit IRC | 01:25 | |
| *** lyrrad has joined #openstack-swift | 01:30 | |
| *** greghaynes has joined #openstack-swift | 01:31 | |
| *** lyrrad has quit IRC | 01:55 | |
| *** dmorita has quit IRC | 01:59 | |
| *** dmorita has joined #openstack-swift | 02:01 | |
| *** lyrrad has joined #openstack-swift | 02:01 | |
| *** siva_krish has joined #openstack-swift | 02:02 | |
| *** Suyash has quit IRC | 02:04 | |
| *** siva_krish has quit IRC | 02:05 | |
| *** dmorita has quit IRC | 02:05 | |
| *** dmorita has joined #openstack-swift | 02:08 | |
| *** dmorita has quit IRC | 02:10 | |
| *** dmorita has joined #openstack-swift | 02:10 | |
| *** klrmn has joined #openstack-swift | 02:11 | |
| *** lyrrad has quit IRC | 02:17 | |
| *** niknakpaddywak has quit IRC | 02:23 | |
| *** takashi has quit IRC | 02:29 | |
| *** baojg has joined #openstack-swift | 02:32 | |
| *** vinsh_ has quit IRC | 02:32 | |
| *** jraju has joined #openstack-swift | 02:34 | |
| *** links has quit IRC | 02:36 | |
| *** vinsh has joined #openstack-swift | 02:36 | |
| *** dmorita has quit IRC | 02:37 | |
| *** dmorita has joined #openstack-swift | 02:39 | |
| *** ManojK has quit IRC | 02:41 | |
| *** dmorita has quit IRC | 02:44 | |
| *** klamath has quit IRC | 02:45 | |
| *** baojg has quit IRC | 02:54 | |
| *** jraju has quit IRC | 03:03 | |
| *** vinsh has quit IRC | 03:07 | |
| *** ManojK has joined #openstack-swift | 03:16 | |
| *** baojg has joined #openstack-swift | 03:21 | |
| *** _JZ_ has quit IRC | 03:22 | |
| *** _JZ_ has joined #openstack-swift | 03:24 | |
| *** sheel has joined #openstack-swift | 03:28 | |
| *** baojg has quit IRC | 03:40 | |
| *** tqtran has joined #openstack-swift | 03:56 | |
| *** tqtran has quit IRC | 04:00 | |
| *** vinsh has joined #openstack-swift | 04:08 | |
| *** vinsh has quit IRC | 04:13 | |
| *** klrmn has quit IRC | 04:14 | |
| *** adu has joined #openstack-swift | 04:22 | |
| *** psachin has joined #openstack-swift | 04:25 | |
| *** vinsh has joined #openstack-swift | 04:39 | |
| *** vinsh has quit IRC | 04:43 | |
| *** ManojK has quit IRC | 04:52 | |
| *** SkyRocknRoll has joined #openstack-swift | 05:22 | |
| *** rcernin has joined #openstack-swift | 05:26 | |
| *** rcernin has quit IRC | 05:34 | |
| *** adu has quit IRC | 05:38 | |
| *** hseipp has joined #openstack-swift | 05:39 | |
| *** ChubYann has quit IRC | 05:40 | |
| *** ppai has joined #openstack-swift | 05:40 | |
| *** hseipp has quit IRC | 05:43 | |
| *** ppai has quit IRC | 05:46 | |
| *** tqtran has joined #openstack-swift | 05:46 | |
| *** tqtran has quit IRC | 05:50 | |
| *** ppai has joined #openstack-swift | 06:00 | |
| *** baojg has joined #openstack-swift | 06:04 | |
| *** _JZ_ has quit IRC | 06:07 | |
| *** rcernin has joined #openstack-swift | 06:09 | |
| *** vinsh has joined #openstack-swift | 06:09 | |
| *** _JZ_ has joined #openstack-swift | 06:10 | |
| *** vinsh has quit IRC | 06:15 | |
| *** manous has joined #openstack-swift | 06:17 | |
| *** arcimboldo has joined #openstack-swift | 06:33 | |
| *** baojg_ has joined #openstack-swift | 06:34 | |
| *** baojg has quit IRC | 06:35 | |
| *** pcaruana has joined #openstack-swift | 06:37 | |
| *** baojg has joined #openstack-swift | 06:38 | |
| *** baojg_ has quit IRC | 06:38 | |
| *** baojg has quit IRC | 06:42 | |
| *** hseipp has joined #openstack-swift | 06:51 | |
| *** asettle has joined #openstack-swift | 07:04 | |
| *** kei_yama has quit IRC | 07:07 | |
| *** kei_yama has joined #openstack-swift | 07:07 | |
| *** cschwede has joined #openstack-swift | 07:13 | |
| *** tesseract- has joined #openstack-swift | 07:15 | |
| *** kei_yama_ has joined #openstack-swift | 07:19 | |
| *** rledisez has joined #openstack-swift | 07:19 | |
| kota_ | I'm still thinking of patch 335641 yet. | 07:19 |
| patchbot | kota_: https://review.openstack.org/#/c/335641/ - swift (feature/crypto-review) - Use a single wsgi filter for the encrypter and dec... | 07:19 |
| *** kei_yama has quit IRC | 07:19 | |
| mattoliverau | kota_: I think it's a good idea. I.e anything that makes it easier for someone in the pipeline is great. I kinda like clayg's idea about putting it all under a crytpo namespace in the common/middleware to be more self contained. | 07:26 |
| kota_ | yes but... | 07:26 |
| kota_ | mattoliverau: i might be stupid though, I'm realizing like "why we don't merge keymaster into encryption module?" | 07:27 |
| kota_ | if it could be, we need just one module inserted in pipeline as a new stuff. | 07:28 |
| kota_ | mattoliverau: I don't say we should merge the classes into a class. | 07:28 |
| kota_ | mattoliverau: similary with Decrypter(Encrypter()) magic, we could do Keymaster(Decrypter(Encrypter())) in the pipeline filter right? | 07:29 |
| *** baojg has joined #openstack-swift | 07:29 | |
| mattoliverau | kota_: to keep in simpler? the keymaster is the part that would be replaced by others. Maybe there could be more then one. if it gets swapped out a lot then people need to hack the paste config function or add another level of "plugins" with isn't that clean anymore | 07:30 |
| kota_ | The reason such an idea comes is because crypto_callback key fetch is callable in left middleware for now. | 07:30 |
| timburke | kota_: my understanding is that keymasters may be pluggable. there could be a separate, out-of-tree keymaster for talking to barbican, or whatever other secret-keeping system an operator is using | 07:30 |
| kota_ | imattoliverau: if we could make it as a cupsle, probably we can drop req.environ["fetch key call back"] for outside of modules. | 07:31 |
| mattoliverau | kota_: yeah, but decryption deals with it from the response so the hook should already be there. | 07:31 |
| kota_ | timburke: yes, it's pluggable *BUT* in my idea, it's still pluggable because | 07:31 |
| kota_ | with " Keymaster(Decrypter(Encrypter()))" syntax, Keymaster is just a pluggable class, I think. | 07:32 |
| kota_ | still yet. | 07:32 |
| *** asettle has quit IRC | 07:32 | |
| mattoliverau | kota_: it's "pluggable" by changing the filter code, but not by a simple paste entry point in the config/pipeline. | 07:33 |
| mattoliverau | kota_: and what if the new 3rd party keymaster takes different config params (which it probably will). | 07:34 |
| *** baojg has quit IRC | 07:35 | |
| cschwede | ^^ this! i think it’s better to separate these two middlewares, more flexible for operators and maybe even easier to understand? | 07:35 |
| kota_ | so right now, the keymaster is a "not secure" reference model so it's ok it stands on independently but if it start to talk with other modules (e.g. barbican), we should prevent to fetch keys in outside of encrypter/decrypter, right? | 07:35 |
| *** ppai has quit IRC | 07:35 | |
| cschwede | i see the keymaster similar to tempauth - it’s working, and a reference implementation, but you probably don’t want to use it in production on large scale | 07:36 |
| timburke | mattoliverau: kota_: well...unless you referenced the paste entrypoint in the [filter:encryption] section. but that starts to feel a little silly; why re-invent both the app-wrapping *and* loading from paste? | 07:36 |
| kota_ | mattoliverau: oh, yeah, probably it makes complicated though. | 07:36 |
| *** baojg has joined #openstack-swift | 07:36 | |
| mattoliverau | reinvent all the things!!! | 07:37 |
| timburke | cschwede: i'm not entirely sure that's true. it may not satisfy all requirements for all use cases, but i think it's actually pretty solid | 07:37 |
| timburke | as far as preventing the fetching of keys outside of encrypter/decrypter, i'd maintain that we can't reasonably protect from rogue middlewares. especially given how encryption is expected to be to the far right of the pipeline | 07:38 |
| cschwede | timburke: i’m not saying that it’s a bad implementation; fully aggree that it’s pretty solid. but i already heard comments like „master key in config file? no way…“ - so ops might want to use other keymaster implementations, where one uses barbican, hardware modules or something other | 07:39 |
| kota_ | timburke: good point, alwyas my headake for that. | 07:39 |
| *** baojg has quit IRC | 07:39 | |
| kota_ | timburke: so my question starts from why keymaster can reuse the key in self._keys. | 07:40 |
| kota_ | in Keymaster.get_keys() | 07:40 |
| kota_ | I was playing to add "raise" if resusing it, a bunch of tests failed because there are some case both encrypter and decrypter calling for fetching. | 07:41 |
| kota_ | so I failed :'( | 07:41 |
| *** manous has quit IRC | 07:41 | |
| kota_ | so restricting call times seems a bad way. | 07:42 |
| kota_ | however if we could encupsulerate Keymaster(Decyrpter(Encrypter())), it enables us to maintain the fetch callback easily. | 07:43 |
| kota_ | (it probably needs more wrapping though) | 07:43 |
| timburke | cschwede: don't we already have rather valuable pieces of info in our configs? admin_key, credentials for authtoken, swift_hash_path_* | 07:44 |
| mattoliverau | k my parents have turned up (vistiing for dinner, dinner I'm suppose to be makeing) so I'd better do be social. I'll check in later once they've left. | 07:47 |
| mattoliverau | s/do/go/ | 07:48 |
| kota_ | right now, I don't have strong oppsite opinion at the point of merging encrypter/decrypter but not sure if jsut encypter/decypter is better way or not | 07:48 |
| kota_ | mattoliverau: have a fun | 07:48 |
| *** ppai has joined #openstack-swift | 07:48 | |
| kota_ | s/a// | 07:48 |
| *** tqtran has joined #openstack-swift | 07:48 | |
| timburke | kota_: i'm still not convinced that having multiple calls to fetch_crypto_keys is a bad thing, or that it should be restricted | 07:49 |
| *** geaaru has joined #openstack-swift | 07:50 | |
| timburke | the main thing i'm hoping for by combining encrypter/decrypter is to simplify what operators have to think about. in that light, 3->2 is a win; 2->1 could be another win, but i'm not convinced that we can do it yet | 07:52 |
| *** acoles_ is now known as acoles | 07:52 | |
| *** tqtran has quit IRC | 07:53 | |
| acoles | good morning | 07:53 |
| kota_ | timburke: yup | 07:53 |
| kota_ | acoles: morning \o/ | 07:53 |
| timburke | ok, that's enough crypto for now. sleepy time. i'll think more about the callback interface while i sleep | 07:53 |
| timburke | crap! i stayed up until acoles woke up :-( | 07:53 |
| kota_ | timburke: oh you can think while you are asleep, awesome!? | 07:54 |
| timburke | kota_: the key question is whether i can retain the ideas :-) | 07:54 |
| kota_ | timburke: kidding, but THANKS, please take a rest enough :-) | 07:54 |
| kota_ | lol | 07:55 |
| *** baojg has joined #openstack-swift | 07:55 | |
| timburke | good night | 07:56 |
| cschwede | timburke: true, but i think the masterkey is something special and you want to keep it from leaking even more than other credentials | 07:56 |
| cschwede | timburke: rest well! | 07:56 |
| * kota_ is heading for grabbing a cup of pop corn to consider more deeply. | 07:56 | |
| timburke | cschwede: one more reason i'd like to see key rotation soonish | 07:57 |
| * cschwede nods in timburke’s direction | 07:57 | |
| acoles | timburke: good morning! | 07:57 |
| acoles | about merging decrypter (D) and encrypter (E) - the thing is, there is no reason for them to be separate because (imo) we don't need the keymaster (KM) to be in the middle. | 08:00 |
| acoles | So although the doc says the pipeline must be D KM E, it can in fact be KM D E (or even KM E D) - try it, it works. | 08:00 |
| acoles | Given that, it just seems simpler to have them be a single filter. | 08:01 |
| acoles | Whereas, there are arguments for keeping the KM separate (and also arguments for not) but it is not such a no-brainer as combining D+E | 08:02 |
| kota_ | acoles: yes, that's true with current implementation. | 08:08 |
| kota_ | the reason I got the question was that I thought the *originial* idea for (D KM E) order was shutting the key leak to other middleware to right/left of encryption staffs. | 08:09 |
| kota_ | probably, I was wrong though. | 08:09 |
| *** _JZ__ has joined #openstack-swift | 08:09 | |
| *** _JZ_ has quit IRC | 08:10 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack/swift: [WIP] Testing latest u-c https://review.openstack.org/318441 | 08:10 |
| kota_ | (i.e. D, E can drop the fetch key functionality when passing a request to left/right middlewre. | 08:10 |
| *** vinsh has joined #openstack-swift | 08:12 | |
| kota_ | (I found/know, my expectations is wrong with current implementation. | 08:12 |
| kota_ | so now we can fetch_key in eny middleware in any places and that's why we can merge the left D into the right E. | 08:13 |
| acoles | kota_: that wasn't my understanding. We are not attempting to protect against an attack on proxy code. If I have access to proxy and malicious middleware M then I could insert it here D KM M E. I could also just read the plaintext data. | 08:13 |
| kota_ | but not sure it's by design or not. | 08:13 |
| kota_ | acoles: yes, yes, exactly malicious middleware can read plaintext data anyway. | 08:15 |
| acoles | kota_: we never have erased the callback function from the request env. But now you mention it we could do that. | 08:15 |
| kota_ | i might think too nervous :/ | 08:15 |
| *** vinsh has quit IRC | 08:16 | |
| *** jamielennox is now known as jamielennox|away | 08:16 | |
| acoles | kota_: With KM D+E, KM installs callback in req env, passes env to D+E. For a GET then once D+E have resp, callback is called to fetch keys, then callback is erased before resp is passed back to KM and on to other middleware. | 08:17 |
| acoles | kota_: Similar for PUT, the callback could be erased so no other middleware can access it. But again, that assumes correct pipeline config, not KM other D+E. | 08:18 |
| kota_ | acoles: yes, exactly | 08:18 |
| acoles | kota_: However, I think a security expert would tell us that it still does not protect us against a malicious attacker who has access to python runtime. | 08:19 |
| kota_ | acoes: agreed | 08:22 |
| kota_ | acoles: so summrized those, the merging D/E seems totally fine to me and we could still have a time to think of pipeline healthiness (probably after landed) anyway | 08:23 |
| acoles | kota_: ok. thanks for thinking about it, it's great review! | 08:24 |
| kota_ | acoles: :-) | 08:25 |
| kota_ | ah, can i finish up my review in 12 hours? | 08:33 |
| kota_ | the irc meeting starts since 6:00 am and now 5:30 p.m. | 08:33 |
| kota_ | it seems I did! | 08:34 |
| *** baojg has quit IRC | 08:34 | |
| *** asettle has joined #openstack-swift | 08:36 | |
| admin6 | Hi there, one dummy question about async pendings. When I use swift-recon -a and got "[async_pending] - No hosts returned valid data." answer. Does that means "all is ok", no hosts have someting to notice, or "There is a problem" because I receive no valid data from hosts ? | 08:36 |
| *** baojg has joined #openstack-swift | 08:37 | |
| ahale | are your hosts running the swift-recon-cron cron? that populates the recon cache with async info | 08:39 |
| *** baojg has quit IRC | 08:41 | |
| *** baojg has joined #openstack-swift | 08:43 | |
| *** sheel has quit IRC | 08:45 | |
| *** bapalm has quit IRC | 08:57 | |
| *** bapalm has joined #openstack-swift | 09:00 | |
| *** balajir_ has joined #openstack-swift | 09:01 | |
| *** treyd_ has joined #openstack-swift | 09:02 | |
| *** baojg has quit IRC | 09:04 | |
| *** diogogmt has quit IRC | 09:08 | |
| *** j_king has quit IRC | 09:08 | |
| *** balajir has quit IRC | 09:08 | |
| *** ndk_ has quit IRC | 09:08 | |
| *** mathiasb has quit IRC | 09:08 | |
| *** sgundur1 has quit IRC | 09:08 | |
| *** treyd has quit IRC | 09:08 | |
| *** mathiasb has joined #openstack-swift | 09:08 | |
| *** j_king has joined #openstack-swift | 09:09 | |
| *** ndk_ has joined #openstack-swift | 09:09 | |
| *** arcimboldo has quit IRC | 09:11 | |
| *** _JZ__ has quit IRC | 09:11 | |
| *** _JZ_ has joined #openstack-swift | 09:12 | |
| *** mathiasb has quit IRC | 09:13 | |
| *** mmcardle has joined #openstack-swift | 09:13 | |
| *** mathiasb has joined #openstack-swift | 09:14 | |
| *** sgundur1 has joined #openstack-swift | 09:15 | |
| *** baojg has joined #openstack-swift | 09:16 | |
| *** mmcardle has quit IRC | 09:27 | |
| *** ppai has quit IRC | 09:27 | |
| *** ppai has joined #openstack-swift | 09:29 | |
| *** baojg has quit IRC | 09:33 | |
| *** baojg has joined #openstack-swift | 09:38 | |
| *** tmoreira has joined #openstack-swift | 09:39 | |
| *** mmcardle has joined #openstack-swift | 09:40 | |
| *** arcimboldo has joined #openstack-swift | 10:03 | |
| *** baojg has quit IRC | 10:04 | |
| *** baojg has joined #openstack-swift | 10:06 | |
| *** vinsh has joined #openstack-swift | 10:10 | |
| *** rledisez has quit IRC | 10:15 | |
| *** baojg has quit IRC | 10:18 | |
| *** arcimboldo has quit IRC | 10:18 | |
| *** arcimboldo has joined #openstack-swift | 10:19 | |
| *** mmcardle has quit IRC | 10:33 | |
| *** vinsh has quit IRC | 10:39 | |
| *** _fortis has quit IRC | 10:43 | |
| *** mmcardle has joined #openstack-swift | 10:47 | |
| *** klamath has joined #openstack-swift | 10:56 | |
| *** _fortis has joined #openstack-swift | 10:58 | |
| *** kei_yama_ has quit IRC | 11:02 | |
| *** psachin has quit IRC | 11:04 | |
| *** psachin has joined #openstack-swift | 11:06 | |
| *** mvk has quit IRC | 11:29 | |
| *** klamath has quit IRC | 11:30 | |
| *** hseipp has quit IRC | 11:31 | |
| *** hseipp has joined #openstack-swift | 11:31 | |
| *** psachin has quit IRC | 11:35 | |
| *** ppai has quit IRC | 11:35 | |
| *** dmorita has joined #openstack-swift | 11:39 | |
| *** vinsh has joined #openstack-swift | 11:40 | |
| *** dmorita has quit IRC | 11:44 | |
| *** tqtran has joined #openstack-swift | 11:45 | |
| *** vinsh has quit IRC | 11:45 | |
| *** mmcardle has quit IRC | 11:46 | |
| *** psachin has joined #openstack-swift | 11:48 | |
| *** tqtran has quit IRC | 11:48 | |
| *** links has joined #openstack-swift | 11:48 | |
| *** links has quit IRC | 11:48 | |
| *** ppai has joined #openstack-swift | 11:49 | |
| *** asettle has quit IRC | 11:51 | |
| *** asettle has joined #openstack-swift | 11:57 | |
| *** mvk has joined #openstack-swift | 11:57 | |
| *** NM has joined #openstack-swift | 12:09 | |
| admin6 | ahale: you’re right I was missing the swift-recon-cron. thanks (and sorry for thte delay to answer) | 12:10 |
| *** NM has quit IRC | 12:10 | |
| *** NM1 has joined #openstack-swift | 12:10 | |
| ahale | :) | 12:11 |
| *** raildo-afk is now known as raildo | 12:13 | |
| *** daemontool has joined #openstack-swift | 12:14 | |
| *** vinsh has joined #openstack-swift | 12:20 | |
| *** jamie_h has joined #openstack-swift | 12:27 | |
| *** mmcardle has joined #openstack-swift | 12:46 | |
| *** psachin has quit IRC | 12:57 | |
| *** ManojK has joined #openstack-swift | 12:59 | |
| *** NM1 has quit IRC | 13:02 | |
| *** NM has joined #openstack-swift | 13:04 | |
| *** ametts has joined #openstack-swift | 13:21 | |
| *** mmcardle has quit IRC | 13:25 | |
| *** klamath has joined #openstack-swift | 13:27 | |
| *** klamath has quit IRC | 13:27 | |
| *** klamath has joined #openstack-swift | 13:27 | |
| *** zaitcev has joined #openstack-swift | 13:31 | |
| *** ChanServ sets mode: +v zaitcev | 13:31 | |
| *** psachin has joined #openstack-swift | 13:36 | |
| *** ManojK has quit IRC | 13:37 | |
| *** manous has joined #openstack-swift | 13:43 | |
| *** sheel has joined #openstack-swift | 13:44 | |
| *** manous has quit IRC | 13:51 | |
| *** diogogmt has joined #openstack-swift | 13:54 | |
| *** mmcardle has joined #openstack-swift | 13:57 | |
| *** ManojK has joined #openstack-swift | 13:58 | |
| *** manous has joined #openstack-swift | 14:04 | |
| *** adu has joined #openstack-swift | 14:05 | |
| *** ManojK has quit IRC | 14:13 | |
| *** ppai has quit IRC | 14:13 | |
| *** diogogmt has quit IRC | 14:14 | |
| *** rcernin has quit IRC | 14:14 | |
| *** _JZ_ has quit IRC | 14:18 | |
| *** vint_bra has joined #openstack-swift | 14:23 | |
| *** rcernin has joined #openstack-swift | 14:29 | |
| *** jistr is now known as jistr|mtg | 14:31 | |
| *** adu has quit IRC | 14:33 | |
| *** ManojK has joined #openstack-swift | 14:33 | |
| *** Suyash has joined #openstack-swift | 14:40 | |
| *** balajir_ has quit IRC | 14:43 | |
| *** cdelatte has joined #openstack-swift | 14:43 | |
| *** charz has quit IRC | 14:43 | |
| *** charz has joined #openstack-swift | 14:46 | |
| *** rcernin has quit IRC | 14:46 | |
| *** balajir has joined #openstack-swift | 14:46 | |
| *** pcaruana has quit IRC | 14:46 | |
| acoles | mahatic_: if you're still here, where is the log message made that you mention here https://review.openstack.org/#/c/328208/12/swift/common/middleware/encrypter.py@128 ? | 14:47 |
| patchbot | acoles: patch 328208 - swift (feature/crypto-review) - Enable object body and metadata encryption | 14:47 |
| *** npf has joined #openstack-swift | 14:54 | |
| *** rcernin has joined #openstack-swift | 14:59 | |
| *** ManojK has quit IRC | 15:01 | |
| *** ManojK has joined #openstack-swift | 15:03 | |
| *** manous has quit IRC | 15:05 | |
| asettle | notmyname you know, I don't even remember signing up for a swiftstack newsletter :P | 15:08 |
| *** vint_bra has quit IRC | 15:11 | |
| *** diogogmt has joined #openstack-swift | 15:12 | |
| *** psachin has quit IRC | 15:17 | |
| *** manous has joined #openstack-swift | 15:17 | |
| *** rcernin has quit IRC | 15:19 | |
| *** jistr|mtg is now known as jistr | 15:23 | |
| *** tesseract- has quit IRC | 15:27 | |
| *** dmorita has joined #openstack-swift | 15:33 | |
| *** thumpba has joined #openstack-swift | 15:33 | |
| *** dmorita has quit IRC | 15:39 | |
| *** jmccarthy has quit IRC | 15:45 | |
| *** jmccarthy has joined #openstack-swift | 15:46 | |
| admin6 | Hi there. Is the swift-dispersion report compatible with erasure coded rings ? and if not, is there an equivalent soution for EC? | 15:47 |
| *** jordanP has joined #openstack-swift | 15:50 | |
| *** silor has joined #openstack-swift | 15:51 | |
| notmyname | good morning | 15:52 |
| notmyname | asettle: doh! sorry about that. probably some conference hall badge scan sort of thing. I'll make sure you're off | 15:52 |
| asettle | notmyname: haha nah it's not a problem at all :) did wonder how I started getting them though... | 15:53 |
| *** silor1 has joined #openstack-swift | 15:56 | |
| *** silor has quit IRC | 15:58 | |
| *** silor1 is now known as silor | 15:58 | |
| *** nadeem has joined #openstack-swift | 15:59 | |
| *** jordanP has quit IRC | 16:00 | |
| notmyname | acoles: looks like some good review comments today | 16:02 |
| acoles | notmyname: yes, has kept me busy! | 16:03 |
| notmyname | acoles: where should I start this morning? | 16:04 |
| acoles | notmyname: re https://review.openstack.org/335641 I like clayg's idea to move to common/middleware/encryption and put the filter factory in __init__, and I think others were positive, but will that lose us git change history? | 16:05 |
| notmyname | no, not if patch 335641 is done with `git mv` | 16:06 |
| patchbot | notmyname: https://review.openstack.org/#/c/335641/ - swift (feature/crypto-review) - Use a single wsgi filter for the encrypter and dec... | 16:06 |
| *** Jeffrey4l has quit IRC | 16:08 | |
| *** hseipp has quit IRC | 16:12 | |
| acoles | notmyname: I'm about to push new versions so you can start by reviewing the changes there. | 16:13 |
| notmyname | ok | 16:13 |
| *** arcimboldo has quit IRC | 16:14 | |
| *** nadeem has quit IRC | 16:14 | |
| *** mwheckmann has joined #openstack-swift | 16:15 | |
| notmyname | I'm fascinated that every time Huge Corp (tm) makes an announcement of some new thing they're doing, especially when it's New! Innovative! thing, the announcement always starts with something like "we got a small group of smart people in a room and ..." | 16:15 |
| *** cdelatte has quit IRC | 16:16 | |
| acoles | notmyname: oic. I had intended to squash the changes into the crypto and docs patches, so we'd still have the original chain of 6. does that make sense to you? so we never merge a patch that had D KM E | 16:16 |
| notmyname | now, when I'm reading the ML thread about the proposed architecture working group, I'm doubly fascinated that OpenStack, which itself started as this New! Innovative! thing, has people who are trying to get a small group of people together to go do some innovative thing | 16:16 |
| notmyname | acoles: yeah, I'm fine with that. but in that case I'm missing where we'd lose git history | 16:17 |
| ahale | Openstack is always open to small groups of people doing innovative things! | 16:17 |
| notmyname | the patch chain is the history | 16:17 |
| notmyname | acoles: as long as it's the same innovative things that everyone else is doing in the same way everyone else is doing it ;-) | 16:18 |
| *** lyrrad has joined #openstack-swift | 16:18 | |
| ahale | absolutely, anything else might be too disruptive after all | 16:19 |
| notmyname | tab complete nick fail. oops | 16:20 |
| acoles | notmyname: duh, of course, we haven't merged anything yet so there is no git history. | 16:20 |
| * acoles is tired | 16:21 | |
| notmyname | acoles: we're inventing the history! | 16:21 |
| acoles | notmyname: yeah, let's get a small group of people in a room and invent history ;) | 16:22 |
| acoles | notmyname: so I am concerned that its is really hard to review a file rename in gerrit, so I propose that we take it as an isolated update to the patchset i.e. the only change is the filter factory stuff and file relocation | 16:23 |
| timburke | good morning | 16:24 |
| notmyname | ie a patch on top of the current chain (similar to how it's proposed now)? | 16:24 |
| acoles | notmyname: no. sorry. I mean, I push a new version of the last 2 patches (crypto, doc) where the *only* change is file relocation and stuff related to the filter-factory change. Then reviewers mainly need to check the file contents are identical. No other changes mixed in. | 16:27 |
| timburke | acoles: notmyname: i like that idea. and i think it can land on master | 16:27 |
| timburke | oh, i misread that, too | 16:28 |
| acoles | a 7th patch is *easier* but leaves us with weird stuff in the history, like we had 3 filters but then we didn't. IDK, I am trying to figure it out but would like to get it right | 16:29 |
| notmyname | acoles: I'd much prefer it to be a simpler history to look at in 6 months than whatever gerrit makes easiest for this week | 16:30 |
| notmyname | but I'm not sure that it's too weird int he history to have the 7th patch set | 16:31 |
| acoles | notmyname: that's hedging ;) let's go for 6 patches and clean history | 16:32 |
| acoles | wouldn't it be nice to have a button in the comment reply dialog in gerrit labelled 'Later' that created a link to the comment in trello or somewhere? | 16:33 |
| timburke | acoles: for all of my comments last night, you should absolutely feel free to leave a "Later" comment and rely on me writing the patch | 16:35 |
| acoles | timburke: too late! | 16:35 |
| *** mmcardle1 has joined #openstack-swift | 16:36 | |
| *** silor has quit IRC | 16:36 | |
| *** gyee has joined #openstack-swift | 16:36 | |
| timburke | haha i'd tried to make it clear in the over-all comment... | 16:36 |
| *** mmcardle has quit IRC | 16:38 | |
| acoles | timburke: the perfectionist in me can't resist (although I have left a few) | 16:38 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Enable object body and metadata encryption https://review.openstack.org/328208 | 16:38 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Add encryption overview doc https://review.openstack.org/328209 | 16:38 |
| timburke | whooo! | 16:38 |
| acoles | ^^ ok that is all changes I have APART from the filter factory change which I am now going to squash into those two reviews | 16:39 |
| notmyname | ok | 16:39 |
| acoles | notmyname: shall we start to merge the first few patches onto crypto-review to get a head start on zuul? | 16:39 |
| acoles | timburke: your dash with 'Has draft' has been very useful | 16:41 |
| *** cdelatte has joined #openstack-swift | 16:41 | |
| notmyname | acoles: I don't think it matters too much. as long as the merge commit gets landed by my monday morning, I think we're ok. ie we've got all weekend to let zuul chew on it | 16:41 |
| acoles | k | 16:42 |
| notmyname | acoles: but that being said, if you drop your -2 on the first one, then based on comments today (while you're sleeping), I can start landing things if they look good | 16:43 |
| notmyname | acoles: or we can start that when you wake up tomorrow | 16:43 |
| acoles | notmyname: ok sounds like a good plan (drop -2) - you could add a -2 just in case | 16:44 |
| notmyname | ok | 16:44 |
| notmyname | acoles: done | 16:44 |
| *** dmorita has joined #openstack-swift | 16:45 | |
| *** mmcardle1 has quit IRC | 16:48 | |
| *** dmorita has quit IRC | 16:50 | |
| *** nadeem has joined #openstack-swift | 16:50 | |
| *** vint_bra has joined #openstack-swift | 16:50 | |
| acoles | timburke: the change to not base64encode the path in keymaster was important, can i ask you to cast an eye of this related change though https://review.openstack.org/#/c/328208/13/swift/common/middleware/crypto_utils.py@238 ? I found I need this to fix some func tests that broke. | 16:50 |
| patchbot | acoles: patch 328208 - swift (feature/crypto-review) - Enable object body and metadata encryption | 16:50 |
| timburke | acoles: we'll need to update the docstring, and when we eventually move to py3, we may need to revisit | 16:54 |
| *** mmcardle has joined #openstack-swift | 16:54 | |
| timburke | alternatively, you could do something like `val.encode('utf8') if six.PY2 else val` (while still updating docstring a little | 16:54 |
| timburke | ) | 16:54 |
| acoles | ugh, the docstring :/ can we live with that for now? | 16:58 |
| notmyname | timburke: really? really?! we might have to change something when we move to py3? *that's* the sort of review you'll give on the crypto branch today?! ;-) | 16:59 |
| timburke | sure. adding it to my TODOs... | 16:59 |
| notmyname | lol | 16:59 |
| *** asettle has quit IRC | 16:59 | |
| timburke | i'm shocked (*shocked*!) to find changes necessary to move to py3! | 17:00 |
| timburke | acoles: fundamentally, the encoding seems sane | 17:00 |
| *** daemontool_ has joined #openstack-swift | 17:01 | |
| acoles | timburke: ok, thanks | 17:01 |
| *** ManojK has quit IRC | 17:01 | |
| timburke | acoles: i thought the point of the 32 was to give us some buffer for random headers we may need to add (like EC sysmeta, or now crypto sysmeta) | 17:02 |
| timburke | i don't see much harm in bumping to 36, but i was kinda hoping to not have any changes there at all | 17:03 |
| *** ManojK has joined #openstack-swift | 17:03 | |
| *** daemontool has quit IRC | 17:04 | |
| *** NM has quit IRC | 17:06 | |
| *** Suyash has quit IRC | 17:06 | |
| *** mvk has quit IRC | 17:06 | |
| *** siva_krish has joined #openstack-swift | 17:07 | |
| *** mmcardle has quit IRC | 17:08 | |
| clayg | timburke: i was sorta hoping we'd leave it at 132 "just cause" | 17:10 |
| notmyname | clayg: bigger numbers are better, right? | 17:10 |
| clayg | notmyname $%YUing gets it | 17:10 |
| timburke | clayg: notmyname: i guess you're right. like 600s bulk delete times! who wants to go *down* to 20s?!? | 17:12 |
| * notmyname doesn't care as long as the default config isn't broken | 17:12 | |
| acoles | timburke: IIRC someone, cschwede perhaps, did a count of the headers we might expect to reach 32. Since we're adding some it seemed logical to increase that number. It doesn't actually result in any resource being committed in httplib. | 17:12 |
| notmyname | 36 is good. let's not bnikeshed about it | 17:13 |
| timburke | yeah, i;m fine with leaving it | 17:14 |
| acoles | reminds me though, the limit doesn't come in til py2.7.9 (I think?) and I only have 2.7.6 on my saio - can anyone confirm func tests do work on 2.7.9 and that we don't need to allow *more* headers? | 17:15 |
| acoles | jrichli: you saw the max header problem on your machine right? ^^ | 17:15 |
| timburke | on the content-type thing, i think we *would* want to extract before decrypting, since we're using it to check for a multipart response. but we can talk about that more in san antonio (or whenever we get around to encrypting content-type) | 17:16 |
| notmyname | acoles: yeah, I've got 2.7.11+ IIRC, so I'll run functests | 17:16 |
| *** dmorita has joined #openstack-swift | 17:16 | |
| notmyname | patch 335641 is going away, right? | 17:17 |
| patchbot | notmyname: https://review.openstack.org/#/c/335641/ - swift (feature/crypto-review) - Use a single wsgi filter for the encrypter and dec... | 17:17 |
| *** peluse has quit IRC | 17:17 | |
| acoles | notmyname: thanks, and yes | 17:20 |
| *** Suyash has joined #openstack-swift | 17:21 | |
| *** chsc has joined #openstack-swift | 17:21 | |
| *** chsc has joined #openstack-swift | 17:21 | |
| notmyname | acoles: functests pass on my SAIO for patch 328209 with python 2.7.11+ | 17:23 |
| patchbot | notmyname: https://review.openstack.org/#/c/328209/ - swift (feature/crypto-review) - Add encryption overview doc | 17:23 |
| acoles | great. | 17:23 |
| acoles | notmyname: with EC? | 17:23 |
| notmyname | acoles: no. running that now | 17:25 |
| *** cdelatte has quit IRC | 17:27 | |
| clayg | timburke: think of it as *deletes per second* - bigger is always better, esspecially when its faster | 17:29 |
| notmyname | acoles: all functests passed with an EC policy as default | 17:30 |
| acoles | notmyname: excellent, thank you | 17:30 |
| acoles | notmyname: ...fast-post, fallocate? ;) | 17:30 |
| acoles | fast-post in checked in gate | 17:31 |
| *** manous has quit IRC | 17:32 | |
| *** npf has quit IRC | 17:32 | |
| notmyname | using fallocate | 17:32 |
| clayg | is this is then? are we going to merge this thing!? | 17:33 |
| jrichli | acoles: got back from lunch. reading now | 17:33 |
| notmyname | clayg: the unification patch is going to be submitted, I think (acoles?) | 17:34 |
| acoles | notmyname: clayg the unification patch is going to be squashed into review 323208 and patch 323209 | 17:35 |
| patchbot | acoles: https://review.openstack.org/#/c/323209/ - cookbook-openstack-image - initial commit for the newton development cycle (MERGED) | 17:35 |
| acoles | argh! patch 328208 and 328209 | 17:35 |
| patchbot | acoles: https://review.openstack.org/#/c/328208/ - swift (feature/crypto-review) - Enable object body and metadata encryption | 17:35 |
| acoles | patch 328209 | 17:36 |
| patchbot | acoles: https://review.openstack.org/#/c/328209/ - swift (feature/crypto-review) - Add encryption overview doc | 17:36 |
| *** klrmn has joined #openstack-swift | 17:36 | |
| *** asettle has joined #openstack-swift | 17:43 | |
| *** NM has joined #openstack-swift | 17:44 | |
| clayg | notmyname: are we sure? isn't 3 middlewares obviously better than only 2 middlewares? | 17:45 |
| *** ChubYann has joined #openstack-swift | 17:45 | |
| *** manous has joined #openstack-swift | 17:46 | |
| clayg | notmyname: maybe config options is the exception to the rool? Bigger version; Less options - PEFECT | 17:46 |
| notmyname | wait, I'm not sure which way you're advocating now | 17:47 |
| notmyname | clayg: which one do you want, and why? | 17:47 |
| *** asettle has quit IRC | 17:47 | |
| *** gyee has quit IRC | 17:48 | |
| clayg | notmyname: i'm j/k | 17:48 |
| notmyname | ABOUT WHICH ONE?! | 17:48 |
| notmyname | ;-) | 17:48 |
| acoles | clayg: I had this idea overnight for making it four | 17:49 |
| clayg | The views, opinions and positions expressed by clayg the IRC troll are his alone, and do not necessarily reflect the views, opinions or positions of clayg reviewer | 17:50 |
| clayg | acoles: i like where you going... | 17:51 |
| acoles | clayg: me too, I'm going for dinner... | 17:51 |
| jrichli | acoles: FWIW, functests pass on my python 2.7.6 as is, and I made sure my configs did not override default max | 17:57 |
| notmyname | good news, everyone. no merge conflicts between patch 328209 and master | 18:00 |
| patchbot | notmyname: https://review.openstack.org/#/c/328209/ - swift (feature/crypto-review) - Add encryption overview doc | 18:00 |
| *** manous has quit IRC | 18:01 | |
| notmyname | here's the plan | 18:02 |
| *** ManojK has quit IRC | 18:02 | |
| notmyname | acoles (after dinner) will push the 2 middleware version into the current patch chain, and we'll continue reviewing | 18:02 |
| notmyname | later today, I'll start landing stuff on the feature/crypto-review branch | 18:02 |
| notmyname | and wehn we've got the positive reviews to the end of the chain, I'll propose the merge commit to master | 18:02 |
| notmyname | i'm hoping to do that tonight | 18:02 |
| clayg | zomgbbq! | 18:03 |
| jrichli | so, i guess this is really happening ... | 18:03 |
| jrichli | ;-) | 18:03 |
| jrichli | so, where's the party Friday night? | 18:04 |
| notmyname | if all this goes according to plan, I'll bring beer to the swiftstack office on friday ;-) | 18:05 |
| jrichli | too bad you can't 'beam' us all there | 18:06 |
| clayg | which gate/devstack test/setup exactly cares if the default root crypto secret is set? | 18:06 |
| *** SkyRocknRoll has quit IRC | 18:06 | |
| jrichli | clayg: I did some initial looking into how we can setup a gate test when crypto is not in default pipeline | 18:07 |
| *** arcimboldo has joined #openstack-swift | 18:07 | |
| jrichli | but I didnt see a solution immediately | 18:07 |
| notmyname | I think we'll need to build one. like with the fast-post or ec test | 18:08 |
| *** mvk has joined #openstack-swift | 18:09 | |
| jrichli | and it looked like there will need to be something really new to be able to support internal client config changes | 18:12 |
| jrichli | acoles had mentioned maybe making use of SWIFT_EXTRAS_MIDDLEWARE. | 18:13 |
| *** manous has joined #openstack-swift | 18:14 | |
| jrichli | I am new to looking at the devstack setup. from what I see, you'd have to have some conditional configured with devstack to be able to selectively apply that, such as the "is_service_enabled" flag. | 18:14 |
| *** thumpba has quit IRC | 18:18 | |
| *** _JZ_ has joined #openstack-swift | 18:21 | |
| clayg | jrichli: yeah makes sense about the internal-client.conf :\ | 18:21 |
| clayg | toobad | 18:21 |
| clayg | bbiab | 18:21 |
| jrichli | clayg: but actually, the only probetests we have dont use devstack, right? its a swiftstack cluster? so maybe you know a good way to modify the internal client conf there? | 18:23 |
| timburke | now there's a good feeling. the first three sections of my crypto review dash are all empty :-) | 18:24 |
| jrichli | timburke: nice work! | 18:24 |
| notmyname | nice | 18:24 |
| jrichli | I meant to say, the only gate tests running probetests don't use devstack | 18:25 |
| *** ManojK has joined #openstack-swift | 18:26 | |
| *** manous has quit IRC | 18:27 | |
| acoles | notmyname: if i stop work now and drive to heathrow I can arrive in SF in time for beers | 18:29 |
| notmyname | :-) | 18:29 |
| *** thumpba has joined #openstack-swift | 18:29 | |
| timburke | acoles: oh no you don't! there's still a patch that needs squashing ;-) | 18:30 |
| tdasilva | acoles: they do have wifi on the train to heathrow and it works pretty well | 18:31 |
| acoles | oh bother i messed up the translations | 18:33 |
| acoles | timburke: well IDK if clayg is going to love me or not if I squash that change in | 18:33 |
| timburke | acoles: again, i'm happy to (1) be the one that fixes it and (2) do it on master | 18:33 |
| *** thumpba has quit IRC | 18:34 | |
| *** Fin1te has joined #openstack-swift | 18:34 | |
| *** thumpba has joined #openstack-swift | 18:35 | |
| *** thumpba has quit IRC | 18:35 | |
| *** thumpba has joined #openstack-swift | 18:35 | |
| *** geaaru has quit IRC | 18:41 | |
| *** vinsh has quit IRC | 18:48 | |
| *** manous has joined #openstack-swift | 18:51 | |
| *** manous has quit IRC | 19:05 | |
| torgomatic | so this is not something we should actually do, but it's sort of interesting that we can do this http://paste.openstack.org/show/DNgZXDrT71crJhiL24va/ to patch 328208 | 19:17 |
| patchbot | torgomatic: https://review.openstack.org/#/c/328208/ - swift (feature/crypto-review) - Enable object body and metadata encryption | 19:17 |
| timburke | torgomatic: yay CTR mode! | 19:19 |
| *** manous has joined #openstack-swift | 19:20 | |
| jrichli | torgomatic: true :-) | 19:21 |
| timburke | (that's not generally a property of block ciphers. OFB mode also has the property that D == E, but CBC and CFB (and of course ECB) modes don't) | 19:24 |
| *** manous has quit IRC | 19:28 | |
| *** vinsh has joined #openstack-swift | 19:29 | |
| jrichli | yes, i thought it was strange to see 2 diff methods in the library. but i figured we should go with using them both. | 19:32 |
| * jrichli is gonna be away for a bit | 19:34 | |
| *** sheel has quit IRC | 19:35 | |
| clayg | torgomatic: wtf is that? offset = 0 !? | 19:36 |
| torgomatic | clayg: just playing around. offset is only for ranged GETs and create_encryption_ctx() is only for PUTs, so offset=0 means start at the beginning | 19:37 |
| *** vint_bra has quit IRC | 19:39 | |
| *** tqtran has joined #openstack-swift | 19:39 | |
| clayg | torgomatic: ah, i see now - had to look at in context - that's goofy and I don't understand what timburke said :D | 19:40 |
| *** sgundur_ has joined #openstack-swift | 19:40 | |
| *** sgundur_ has left #openstack-swift | 19:40 | |
| *** vint_bra has joined #openstack-swift | 19:40 | |
| *** hoonetorg has quit IRC | 19:40 | |
| timburke | clayg: in general, the encryption and decryption functions are different. for certain modes of operation (https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation), however, they'll be the same | 19:42 |
| clayg | timburke: lol @ the location bar in my browser! https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation | 19:42 |
| acoles | clayg: btw I like the idea of path to file for secret, can it be a follow up? | 19:46 |
| *** daemontool_ has quit IRC | 19:46 | |
| clayg | acoles: mattoliverau seemed to think having it be a string in a config was useful too, he suggested that we support both with strong preference to a new option to specify path | 19:47 |
| clayg | givin that suggestion i'm ok fixing it as soon as it's on master | 19:47 |
| acoles | clayg: yeah +1 for that | 19:47 |
| clayg | I wish I could talk to someone who has some operational experience obtaining a encryption keys from a "cryptographically secure random number generator" so they could tell us what these things output? | 19:49 |
| clayg | I imagine something that looks more like a well-formed ssh id_rsa private key looking thing? | 19:49 |
| clayg | then we could just start adding support for these well known formats that such tools might be able to output too | 19:50 |
| clayg | cooking up our little base64 scheme seems like it's only useful for text in config files - a file on disk could just be binary for all we care - or maybe something more complicated | 19:50 |
| clayg | ... dunno | 19:50 |
| acoles | notmyname: timburke clayg jrichli ok I have squashed patch 335641 into the chain and moved everything to middleware/crypto. just running tests. you ready for me to push those new versions soon? | 19:51 |
| patchbot | acoles: https://review.openstack.org/#/c/335641/ - swift (feature/crypto-review) - Use a single wsgi filter for the encrypter and dec... | 19:51 |
| notmyname | acoles: yes! | 19:51 |
| *** hoonetorg has joined #openstack-swift | 19:51 | |
| acoles | clayg: this will give you something suitable 'openssl rand -base64 32' | 19:52 |
| clayg | acoles: i have to admit i'm also thinking about swiftstack product - while in one mode we may say "we'll handle the keys; trust us" another option would be something like "when you're adding a new node place your key into /etc/swift/root_secret" if it's not there we'll tell you and demand you fix it because we don't have that key | 19:52 |
| clayg | acoles: did i miss that in the docs? | 19:52 |
| acoles | clayg: you did ;) | 19:52 |
| clayg | acoles: indeed :) | 19:53 |
| timburke | clayg: id_rsa *is* base64-encoded, no? | 19:53 |
| acoles | clayg: its the same format of key that you use with google/aws byok I believe | 19:54 |
| clayg | timburke: yeah, sorry I shouldn't have trying to make it sound like I'm against base64 - the fact that openssl can output to that format gives pleanty of weight to it being a reasonable data exchange in this role | 19:54 |
| clayg | timburke: i'm looking at what other formats it supports | 19:55 |
| acoles | clayg: it also simplifies the config of proxy and container sync internal client, if both can point to a file | 19:56 |
| timburke | clayg: ah. was gonna say, `sed '1,/^$/d; /^--/d' <~/.ssh/id_rsa | base64 -D | hd` certainly *seems* to work... | 19:56 |
| clayg | acoles: yeah that's how I stumbled to it | 19:56 |
| acoles | possibly makes it easier to ship some sample config that points to a well known secret file that needs to be populated | 19:56 |
| clayg | acoles: like that too | 19:57 |
| acoles | clayg: tbh I hadn't anticipated container sync working with encryption without further work, so it was late on before I got into that internal client config thing | 19:58 |
| *** asettle has joined #openstack-swift | 19:59 | |
| *** asettle has quit IRC | 19:59 | |
| clayg | acoles: yeah no worries - glad it works! | 19:59 |
| acoles | notmyname: timburke I'm not sure what/where to link from middleware.rst now we have this composing filter factory, so I'm just putting something very minimal for now | 20:00 |
| timburke | sure | 20:00 |
| clayg | acoles: so is there a rebase coming or what | 20:04 |
| acoles | ok tests good, here goes | 20:05 |
| acoles | hope i didn't screw this up | 20:06 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Enable object body and metadata encryption https://review.openstack.org/328208 | 20:06 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Add encryption overview doc https://review.openstack.org/328209 | 20:06 |
| acoles | ^^ don't forget to change pipelines in proxy-server.conf and container sync client conf | 20:08 |
| timburke | and reinstall, either with pip or setup.py directly | 20:09 |
| *** tqtran has quit IRC | 20:09 | |
| *** tqtran has joined #openstack-swift | 20:10 | |
| *** mwheckmann has quit IRC | 20:11 | |
| *** tqtran has quit IRC | 20:15 | |
| *** tqtran has joined #openstack-swift | 20:15 | |
| notmyname | is the setup.cfg right? | 20:18 |
| notmyname | it has keymaster = swift.common.middleware.keymaster:filter_factory but shouldn't it be "keymaster = swift.common.middleware.crypto.keymaster:filter_factory" | 20:19 |
| notmyname | made that change locally, reinstalled, running tests now | 20:21 |
| *** ManojK has quit IRC | 20:24 | |
| clayg | FIXEET! | 20:25 |
| *** ManojK has joined #openstack-swift | 20:26 | |
| *** gyee has joined #openstack-swift | 20:26 | |
| clayg | wait, so there *is* or is not a crypto_helpers? | 20:27 |
| timburke | clayg: there is a crypto_helpers in the unit tests. comparable to test/unit/common/middleware/helpers.py | 20:29 |
| clayg | interesting.... | 20:29 |
| timburke | there is *not* one under swift/ -- thought here's a crypto_utils.py | 20:29 |
| acoles | so how did it install for me if setp.cfg is wrong? | 20:29 |
| timburke | acoles: stale pyc files | 20:30 |
| notmyname | because you might have an old one | 20:30 |
| timburke | git clean -fX | 20:30 |
| *** cdelatte has joined #openstack-swift | 20:30 | |
| notmyname | all tests passed locally for me | 20:30 |
| acoles | of course. it is wrong. | 20:30 |
| notmyname | all = functests | 20:30 |
| *** klrmn1 has joined #openstack-swift | 20:31 | |
| *** klrmn has quit IRC | 20:32 | |
| clayg | acoles: just keep pushing till it works - you have an army of testers waiting in the wings | 20:34 |
| acoles | you want me to push the fix or wait for any other bloopers? | 20:34 |
| *** cdelatte has quit IRC | 20:34 | |
| clayg | #vote push it | 20:35 |
| clayg | *before* we find any other bloopers - let's go ahead and make it installable ;) | 20:35 |
| *** Fin1te has quit IRC | 20:37 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Enable object body and metadata encryption https://review.openstack.org/328208 | 20:37 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Add encryption overview doc https://review.openstack.org/328209 | 20:38 |
| *** cdelatte has joined #openstack-swift | 20:39 | |
| *** thumpba has quit IRC | 20:40 | |
| notmyname | looks good so far :-) | 20:40 |
| *** rcernin has joined #openstack-swift | 20:41 | |
| acoles | timburke: what was I doing in the factory test? :/ | 20:42 |
| timburke | acoles: passing {} as the app | 20:42 |
| acoles | right! :/ | 20:43 |
| timburke | it'll do, but doesn't really reflect what we expect to happen. i'd left a suggestion on how we could improve it, but that can be later (if ever) | 20:43 |
| acoles | gerrit seems to have stopped sending me email so I'm having to poll gerrit | 20:45 |
| notmyname | ok, I've got a +2 on all of the patches now (except the cork -2 on the first one). I'm ready to go | 20:45 |
| timburke | i'm on board, too | 20:46 |
| notmyname | jrichli: I'd love to have your votes before you take off today | 20:47 |
| notmyname | and mattoliverau and kota_ should be around in a few hours | 20:47 |
| notmyname | clayg: torgomatic: your votes are appreciated too :-) | 20:47 |
| jrichli | notmyname: ack. will be a few hours from now | 20:48 |
| jrichli | is that ok? | 20:48 |
| notmyname | jrichli: just tell the manager types that meetings can wait--you gotta land crypto ;-) | 20:48 |
| notmyname | jrichli: yes, of course that's fine :-) | 20:48 |
| jrichli | notmyname: :-) | 20:48 |
| notmyname | it's a little before 2pm here, so I'd prefer to start landing the chain to crypto-review in a couple of hours, and then tonight I can do the merge commit to master | 20:49 |
| jrichli | i am reviewing now, btw | 20:49 |
| notmyname | assuming everyone is ok with it, of course | 20:49 |
| jrichli | ah, ok | 20:49 |
| *** raildo is now known as raildo-afk | 21:02 | |
| *** siva_krish has quit IRC | 21:05 | |
| *** cdelatte has quit IRC | 21:09 | |
| *** ametts has quit IRC | 21:13 | |
| notmyname | interesting. I open hacker news and see a post on the front page that's simply a link to intel's ISA-L stuff https://news.ycombinator.com/item?id=12010267 | 21:22 |
| clayg | jrichli: acoles: you know in the end i'm pretty happy with x-object-transient-sysmeta | 21:27 |
| *** notmyname has quit IRC | 21:28 | |
| *** notmyname_ has joined #openstack-swift | 21:28 | |
| *** ChanServ sets mode: +v notmyname_ | 21:28 | |
| *** notmyname_ is now known as notmyname | 21:29 | |
| *** NM has quit IRC | 21:34 | |
| *** diogogmt has quit IRC | 21:36 | |
| *** HenryG_ has joined #openstack-swift | 21:41 | |
| *** jroll|dupe has joined #openstack-swift | 21:44 | |
| *** jroll|dupe has quit IRC | 21:44 | |
| *** jroll|dupe has joined #openstack-swift | 21:44 | |
| *** kevinc has joined #openstack-swift | 21:44 | |
| *** hoonetorg has quit IRC | 21:45 | |
| *** HenryG has quit IRC | 21:45 | |
| *** MooingLemur has quit IRC | 21:45 | |
| *** jroll has quit IRC | 21:45 | |
| *** hogepodge has quit IRC | 21:45 | |
| *** jroll|dupe is now known as jroll | 21:45 | |
| jrichli | clayg: happy to hear it :-) | 21:46 |
| *** jamie_h has quit IRC | 21:47 | |
| kevinc | I am updating my container ring for the first time since upgrading to liberty and i get the following warning: RingValidationWarning: The partition xxx has been assigned to duplicate devices . How to i fix the issue? Should I ignore the warning? | 21:47 |
| notmyname | kevinc: that would normally happen if you've got less drives than replicas. eg 2 drives and 3x replication | 21:48 |
| kevinc | i have 4 replica with 20 drives | 21:49 |
| kevinc | 4 zones | 21:49 |
| notmyname | kevinc: and, FWIW, it's the warning that's new, not the underlying issue | 21:49 |
| notmyname | oh, that's interesting | 21:49 |
| kevinc | 1048576 partitions, 4.000000 replicas, 1 regions, 4 zones, 20 devices, 0.00 balance, 0.67 dispersion | 21:49 |
| kevinc | we have been using/updating the same ring since before grizzly, we did make a change from 3 replicas to 4 replicas about 2 years ago | 21:50 |
| notmyname | ok | 21:50 |
| clayg | kevinc: you're not on the latest code then - the latest code makes that an ERROR | 21:51 |
| clayg | kevinc: the reason we couldn't make it an error on the old code was the old code sucked and torgomatic wouldn't let me borrow his time machine | 21:51 |
| *** cdelatte has joined #openstack-swift | 21:51 | |
| clayg | so we had to fix it in the latest code | 21:51 |
| clayg | you should get on the latest code | 21:51 |
| clayg | but... honestly - I *thought* the latest official release had all the good ring-y bits | 21:51 |
| clayg | cirtainly at least 2.7 | 21:52 |
| kevinc | clayg: correct, we are on liberty (2.5), so 2.7 will fix the problem or just cause an error? | 21:52 |
| notmyname | liberty is 2.5.0 | 21:52 |
| clayg | FIXXEET! | 21:52 |
| *** HenryG_ is now known as HenryG | 21:52 | |
| clayg | you could generate rings with 2.7 if you need to and publish to a 2.5 cluster | 21:53 |
| clayg | not recommened but it would work | 21:53 |
| clayg | well... i'm not even sure it's not recommend - we do it | 21:53 |
| kevinc | ok, we can put off updating the ring for now until we upgrade to 2.7, thank you! | 21:53 |
| notmyname | if you can swing upgrading to 2.8, I'd recommend that | 21:53 |
| clayg | kevinc: not recommended - it's trying to tell you about a terrible terrible state of your ring from 2.5 - you *should* fix it | 21:53 |
| notmyname | kevinc: ...and to build on what clayg is saying, the new code will fix it with a rebalance | 21:54 |
| clayg | notmyname: YEAH IT WILL - LIKE A BOSS! | 21:54 |
| acoles | clayg: :D -transient- | 21:55 |
| clayg | did i mispell it? | 21:55 |
| notmyname | kevinc: your current situation is that you have multiple replicas on the same drive, which therefore aren't different replicas at all. so you *think* you're good, but one drive failure can take out a lot more than you expect | 21:55 |
| acoles | clayg: no, I'm just providing context for the smile | 21:55 |
| acoles | lol | 21:55 |
| clayg | ok, w/e - point is nice work acoles & jrichli for not backing down on a good idea despite my inability to understand it originally | 21:56 |
| acoles | clayg: but now you mention it, "mass" is a lot easier to spell :P | 21:56 |
| clayg | nope, that's a stupid name | 21:56 |
| clayg | ;) | 21:57 |
| notmyname | lol | 21:57 |
| acoles | lol | 21:57 |
| acoles | notmyname: I'm thinking of calling it a day - need anything from me? | 21:57 |
| notmyname | acoles: I'm good, and I think there should be some landed patches by the time you wake up | 21:58 |
| jrichli | have a restful night, acoles! | 21:58 |
| notmyname | mostly I'm waiting on people to add review votes to the patch chain | 21:58 |
| jrichli | speaking of which ... i am about to +1 :-) | 21:59 |
| timburke | whoooo! | 21:59 |
| acoles | jrichli: I'm about to imbibe some of clayg's favourite tipple to help with that | 21:59 |
| * clayg not sure if that's beer scotch or whisky | 22:00 | |
| acoles | notmyname: ok, thanks, I'll look forward to that! | 22:00 |
| kevinc | notmyname & clayg: thank you! I'll rebalance on my test cluster that is running mitaka (2.7) | 22:00 |
| jrichli | acoles: you are always increasing my vocab! :-) | 22:01 |
| acoles | clayg: one or more of those | 22:01 |
| kevinc | if i am reading the warning correctly, we have each partition on at least 3 different devices | 22:02 |
| *** catintheroof has joined #openstack-swift | 22:03 | |
| clayg | well... there's that i suppose :\ | 22:03 |
| acoles | jrichli: I'll run out of words soon ;) | 22:03 |
| acoles | good night | 22:03 |
| *** acoles is now known as acoles_ | 22:04 | |
| clayg | so openssl rand seems to be down for hex or base64, with -out <file> the output is still basically unadorned | 22:08 |
| *** cdelatte has quit IRC | 22:09 | |
| *** nadeem has quit IRC | 22:19 | |
| openstackgerrit | Tim Burke proposed openstack/swift: Change elifs to ifs https://review.openstack.org/336303 | 22:20 |
| *** ManojK has quit IRC | 22:20 | |
| timburke | notmyname: just for you^^ | 22:21 |
| *** cdelatte has joined #openstack-swift | 22:27 | |
| openstackgerrit | Tim Burke proposed openstack/swift: Stop digging for publicly_accessible ourselves https://review.openstack.org/336308 | 22:30 |
| clayg | timburke: that *is* acctually better - it's more obviously a series of guard returns instead of some kind of werid switch that might jump you out on the middle | 22:32 |
| mattoliverau | Morning | 22:33 |
| timburke | clayg: i'm not *opposed* to it. i just find both perfectly readable. but the important thing is, one more thing to scratch off my follow-up list | 22:33 |
| timburke | i've already got 4 more little guys like that just waiting for a merge commit to use as a parent | 22:35 |
| *** siva_krish has joined #openstack-swift | 22:36 | |
| *** jamielennox|away is now known as jamielennox | 22:37 | |
| *** siva_krish has quit IRC | 22:44 | |
| *** cdelatte has quit IRC | 22:52 | |
| *** tqtran has quit IRC | 22:59 | |
| notmyname | mattoliverau: good morning! | 23:01 |
| notmyname | mattoliverau: guess what? | 23:01 |
| *** tqtran has joined #openstack-swift | 23:03 | |
| *** vint_bra has quit IRC | 23:03 | |
| *** asettle has joined #openstack-swift | 23:04 | |
| *** NM has joined #openstack-swift | 23:05 | |
| *** tqtran_ has joined #openstack-swift | 23:07 | |
| *** tqtran has quit IRC | 23:07 | |
| torgomatic | mattoliverau: hint: the answer is "chicken butt". he's been like this all day; I think the stress is getting to him. :p | 23:09 |
| notmyname | heh | 23:09 |
| notmyname | I'm going to start landing some of the initial patches to crypto-review now | 23:10 |
| mattoliverau | notmyname: chicken butt?? | 23:10 |
| *** asettle has quit IRC | 23:10 | |
| *** kong has joined #openstack-swift | 23:10 | |
| notmyname | I'll leave https://review.openstack.org/#/c/328208/ and https://review.openstack.org/#/c/328209/ so that mattoliverau and kota_ have a chance to look at the new revisions | 23:10 |
| patchbot | notmyname: patch 328208 - swift (feature/crypto-review) - Enable object body and metadata encryption | 23:10 |
| patchbot | notmyname: patch 328209 - swift (feature/crypto-review) - Add encryption overview doc | 23:10 |
| kota_ | morning | 23:11 |
| mattoliverau | kota_: morning | 23:11 |
| notmyname | kota_: good morning! | 23:11 |
| notmyname | kota_: guess what? | 23:11 |
| kota_ | notmyname: done encryption work? | 23:11 |
| kota_ | oh, what | 23:11 |
| kota_ | not yet reading the back log | 23:11 |
| notmyname | kota_: pretty much. I'd appreciate your comments on the patches. there was a new revision to the last 2 in the chain today | 23:12 |
| kota_ | notmyname: ok, gotcha | 23:12 |
| * kota_ will look at | 23:13 | |
| notmyname | ok, let's see what that does to the gate | 23:13 |
| notmyname | buttons have been clicked | 23:14 |
| kota_ | notmyname: do you have a time limit for them? | 23:14 |
| kota_ | today is enough? or a few hours? | 23:14 |
| kota_ | (or an hour :/) | 23:14 |
| notmyname | kota_: I'd prefer it to be in the next 5 hours | 23:14 |
| * mattoliverau will look as soon as he gets out of his next meeting :) | 23:14 | |
| kota_ | notmyname: ok, trying to do that | 23:15 |
| notmyname | kota_: mattoliverau: thanks :-) | 23:15 |
| *** kei_yama has joined #openstack-swift | 23:17 | |
| torgomatic | question unrelated to crypto: of Swift developers, does anyone's /tmp filesystem *not* support extended attributes? | 23:17 |
| torgomatic | (or $TMPDIR; the spot where tempdir.mkdtemp() puts things) | 23:17 |
| zaitcev | mine for one | 23:18 |
| *** catintheroof has quit IRC | 23:18 | |
| zaitcev | I have a patch outstanding for it, but it ended with adding a gazillion /var/tmp, so I flag it as abandoned for now | 23:18 |
| clayg | torgomatic: atm, mine does not - why do we care about the *system* tmpdir tho? | 23:19 |
| *** tmoreira has quit IRC | 23:20 | |
| zaitcev | torgomatic: http://www.zaitcev.us/things/swift/swift-go-tmpf-1b.diff | 23:20 |
| clayg | torgomatic: is that part of the reason we monkeypatch xattr in test/__init__.py? | 23:20 |
| zaitcev | OOPS | 23:20 |
| clayg | zaitcev: that patch looks good too tho | 23:20 |
| zaitcev | clayg: sorry, please reload http://www.zaitcev.us/things/swift/swift-go-tmpf-1b.diff | 23:21 |
| zaitcev | I overwrote it with scp by accident | 23:21 |
| *** tqtran has joined #openstack-swift | 23:22 | |
| clayg | zaitcev: torgomatic: are you two even talking about the same thing? | 23:22 |
| *** tqtran_ has quit IRC | 23:24 | |
| clayg | zaitcev: why is /var/tmp better than /tmp in that patch? it's not using the cwd is it? | 23:24 |
| zaitcev | [zaitcev@lembas ~]$ setfattr -n user.swift.metadata /tmp/xxx | 23:24 |
| zaitcev | setfattr: /tmp/xxx: Operation not supported | 23:24 |
| zaitcev | [zaitcev@lembas ~]$ touch /var/tmp/xxx | 23:25 |
| zaitcev | [zaitcev@lembas ~]$ setfattr -n user.swift.metadata /var/tmp/xxx | 23:25 |
| zaitcev | [zaitcev@lembas ~]$ | 23:25 |
| zaitcev | What if your home directory is on NFS? | 23:25 |
| notmyname | don't we use a tmpfile on the same filesystem partition as the data itself? ie an xfs volume on the drive, not the system /tmp? | 23:25 |
| zaitcev | I do it all the time in a test box that has a bunch of test VMs | 23:26 |
| zaitcev | notmyname: we do, but this is "go test ./...", which I want to run on my laptop... | 23:26 |
| *** tqtran_ has joined #openstack-swift | 23:27 | |
| zaitcev | notmyname: so, I switched my $HOME to XFS just for that, but there's no Swift layout in /srv/node, not to mention you probably don't want to put test files into your actual Swift anyway. | 23:27 |
| clayg | torgomatic: my bad, /var/tmp and /tmp both have xattrs on my system - i forgot cp needs the -a :\ | 23:28 |
| zaitcev | Maybe it's easier to figure out why tmpfs does not support attributes. I remember vaguely that it was a problem before, and I did something to address it in kernel... There was a patch or other... And it was like 7 years ago. Yet it's 2016 and still nothing works. | 23:28 |
| *** tqtran has quit IRC | 23:29 | |
| *** tmoreira has joined #openstack-swift | 23:29 | |
| clayg | zaitcev: so do a interface trick where you can implement a fake xattr for tests like swift unittest do? | 23:29 |
| zaitcev | clayg: I don't know how to do it in Go. There's no monkey-patching. The xattr is a system library. I suppose we could do it to our own layer... | 23:31 |
| zaitcev | clayg: What's grep .tmp /proc/mounts on your system? Empty? | 23:34 |
| zaitcev | Er. grep /tmp /proc/mounts | 23:35 |
| zaitcev | here: | 23:35 |
| zaitcev | tmpfs /tmp tmpfs rw,seclabel 0 0 | 23:35 |
| clayg | zaitcev: interesting... vagrant@saio:~$ df /tmp/1467329166.42935.data | 23:37 |
| clayg | Filesystem 1K-blocks Used Available Use% Mounted on | 23:37 |
| clayg | /dev/sda1 41251136 2007508 37507716 6% / | 23:37 |
| clayg | as I said, I was wrong before - on this system /tmp has xattrs - because it's the root file system (i, know, i'm surprised) | 23:37 |
| zaitcev | that means not mounted, which is completely okay, just I thought every modern distro mounted /tmp | 23:37 |
| clayg | god only knows wtf the stupid macbook is doing | 23:37 |
| zaitcev | Maybe you're using Ubuntu LTS | 23:38 |
| zaitcev | Do you even have Systemd on it? | 23:38 |
| torgomatic | clayg: yeah, that monkeypatching thing... I hacked up some xattr checksumming to see if it would work, and so far it's been about 5% checksumming the xattrs and 95% dealing with stale data left in that monkeypatched thing | 23:38 |
| *** chsc has quit IRC | 23:38 | |
| clayg | yeah, dunno, could just be this particular box too - vagrant keeps telling me it's outdated - and xenial is out yada yada | 23:38 |
| clayg | torgomatic: so you and zaitcev *are* working on the same thing? | 23:39 |
| torgomatic | clayg: don't think so? unless zaitcev is also adding xattr checksumming in the python code | 23:39 |
| torgomatic | (it'd be the first I've heard of it) | 23:40 |
| zaitcev | So, not the same problem, but same root cause. | 23:40 |
| zaitcev | My problem is to run go test ./... in Hummingbird. | 23:41 |
| *** cdelatte has joined #openstack-swift | 23:44 | |
| clayg | torgomatic: zaitcev: sounds like you guys have *opposite* problems | 23:45 |
| torgomatic | well, I'm gonna just rip out the xattr monkeypatching and submit this thing; it'll get ignored for a while anyway due to crypto, and hopefully anyone with troubles will chime in | 23:50 |
| *** hogepodge has joined #openstack-swift | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!