Thursday, 2016-06-30

mattoliveraugo tdasilva!00:02
NMA lot of compliments from brazilian openstack community.00:04
NMnotmyname:   Can be a screenshot? It's on hangouts. :)00:11
notmynamenah, it's fine. I thought it was at a conference or meetup or something in person00:12
*** ManojK has joined #openstack-swift00:16
*** NM has quit IRC00:19
*** Suyash has quit IRC00:19
notmynameI had no idea there was a swift-related meetup in virginia today
notmynamebuch of recent tweets on
mattoliverauTalking openstack swift at the openstack nova meetup... ok.00:28
mattoliveraucool of course, but still.. ok00:28
*** Jeffrey4l has joined #openstack-swift00:36
*** siva_krish has quit IRC00:44
*** lyrrad has quit IRC00:46
*** gyee has quit IRC00:50
*** links has joined #openstack-swift00:57
*** tqtran has quit IRC00:59
*** Suyash has joined #openstack-swift00:59
*** nadeem has quit IRC01:03
torgomaticmattoliverau: openstack NoVa, as in Northern Virginia, I think01:04
torgomaticalternately, "openstack doesn't go" in Spanish :p01:04
*** klamath has quit IRC01:05
*** klamath has joined #openstack-swift01:05
*** takashi has joined #openstack-swift01:10
*** klamath has quit IRC01:14
*** klamath has joined #openstack-swift01:15
*** greghaynes has quit IRC01:16
*** klrmn has quit IRC01:18
notmynametorgomatic: ah ok. I had the same comfusion01:19
*** asettle has joined #openstack-swift01:21
mattoliverautorgomatic: ahhh, well that makes a whole lot more sense!01:22
*** asettle has quit IRC01:25
*** lyrrad has joined #openstack-swift01:30
*** greghaynes has joined #openstack-swift01:31
*** lyrrad has quit IRC01:55
*** dmorita has quit IRC01:59
*** dmorita has joined #openstack-swift02:01
*** lyrrad has joined #openstack-swift02:01
*** siva_krish has joined #openstack-swift02:02
*** Suyash has quit IRC02:04
*** siva_krish has quit IRC02:05
*** dmorita has quit IRC02:05
*** dmorita has joined #openstack-swift02:08
*** dmorita has quit IRC02:10
*** dmorita has joined #openstack-swift02:10
*** klrmn has joined #openstack-swift02:11
*** lyrrad has quit IRC02:17
*** niknakpaddywak has quit IRC02:23
*** takashi has quit IRC02:29
*** baojg has joined #openstack-swift02:32
*** vinsh_ has quit IRC02:32
*** jraju has joined #openstack-swift02:34
*** links has quit IRC02:36
*** vinsh has joined #openstack-swift02:36
*** dmorita has quit IRC02:37
*** dmorita has joined #openstack-swift02:39
*** ManojK has quit IRC02:41
*** dmorita has quit IRC02:44
*** klamath has quit IRC02:45
*** baojg has quit IRC02:54
*** jraju has quit IRC03:03
*** vinsh has quit IRC03:07
*** ManojK has joined #openstack-swift03:16
*** baojg has joined #openstack-swift03:21
*** _JZ_ has quit IRC03:22
*** _JZ_ has joined #openstack-swift03:24
*** sheel has joined #openstack-swift03:28
*** baojg has quit IRC03:40
*** tqtran has joined #openstack-swift03:56
*** tqtran has quit IRC04:00
*** vinsh has joined #openstack-swift04:08
*** vinsh has quit IRC04:13
*** klrmn has quit IRC04:14
*** adu has joined #openstack-swift04:22
*** psachin has joined #openstack-swift04:25
*** vinsh has joined #openstack-swift04:39
*** vinsh has quit IRC04:43
*** ManojK has quit IRC04:52
*** SkyRocknRoll has joined #openstack-swift05:22
*** rcernin has joined #openstack-swift05:26
*** rcernin has quit IRC05:34
*** adu has quit IRC05:38
*** hseipp has joined #openstack-swift05:39
*** ChubYann has quit IRC05:40
*** ppai has joined #openstack-swift05:40
*** hseipp has quit IRC05:43
*** ppai has quit IRC05:46
*** tqtran has joined #openstack-swift05:46
*** tqtran has quit IRC05:50
*** ppai has joined #openstack-swift06:00
*** baojg has joined #openstack-swift06:04
*** _JZ_ has quit IRC06:07
*** rcernin has joined #openstack-swift06:09
*** vinsh has joined #openstack-swift06:09
*** _JZ_ has joined #openstack-swift06:10
*** vinsh has quit IRC06:15
*** manous has joined #openstack-swift06:17
*** arcimboldo has joined #openstack-swift06:33
*** baojg_ has joined #openstack-swift06:34
*** baojg has quit IRC06:35
*** pcaruana has joined #openstack-swift06:37
*** baojg has joined #openstack-swift06:38
*** baojg_ has quit IRC06:38
*** baojg has quit IRC06:42
*** hseipp has joined #openstack-swift06:51
*** asettle has joined #openstack-swift07:04
*** kei_yama has quit IRC07:07
*** kei_yama has joined #openstack-swift07:07
*** cschwede has joined #openstack-swift07:13
*** tesseract- has joined #openstack-swift07:15
*** kei_yama_ has joined #openstack-swift07:19
*** rledisez has joined #openstack-swift07:19
kota_I'm still thinking of patch 335641 yet.07:19
patchbotkota_: - swift (feature/crypto-review) - Use a single wsgi filter for the encrypter and dec...07:19
*** kei_yama has quit IRC07:19
mattoliveraukota_: I think it's a good idea. I.e anything that makes it easier for someone in the pipeline is great. I kinda like clayg's idea about putting it all under a crytpo namespace in the common/middleware to be more self contained.07:26
kota_yes but...07:26
kota_mattoliverau: i might be stupid though, I'm realizing like "why we don't merge keymaster into encryption module?"07:27
kota_if it could be, we need just one module inserted in pipeline as a new stuff.07:28
kota_mattoliverau: I don't say we should merge the classes into a class.07:28
kota_mattoliverau: similary with Decrypter(Encrypter()) magic, we could do Keymaster(Decrypter(Encrypter())) in the pipeline filter right?07:29
*** baojg has joined #openstack-swift07:29
mattoliveraukota_: to keep in simpler? the keymaster is the part that would be replaced by others. Maybe there could be more then one. if it gets swapped out a lot then people need to hack the paste config function or add another level of "plugins" with isn't that clean anymore07:30
kota_The reason such an idea comes is because crypto_callback key fetch is callable in left middleware for now.07:30
timburkekota_: my understanding is that keymasters may be pluggable. there could be a separate, out-of-tree keymaster for talking to barbican, or whatever other secret-keeping system an operator is using07:30
kota_imattoliverau: if we could make it as a cupsle, probably we can drop req.environ["fetch key call back"] for outside of modules.07:31
mattoliveraukota_: yeah, but decryption deals with it from the response so the hook should already be there.07:31
kota_timburke: yes, it's pluggable *BUT* in my idea, it's still pluggable because07:31
kota_with " Keymaster(Decrypter(Encrypter()))" syntax, Keymaster is just a pluggable class, I think.07:32
kota_still yet.07:32
*** asettle has quit IRC07:32
mattoliveraukota_: it's "pluggable" by changing the filter code, but not by a simple paste entry point in the config/pipeline.07:33
mattoliveraukota_: and what if the new 3rd party keymaster takes different config params (which it probably will).07:34
*** baojg has quit IRC07:35
cschwede^^ this! i think it’s better to separate these two middlewares, more flexible for operators and maybe even easier to understand?07:35
kota_so right now, the keymaster is a "not secure" reference model so it's ok it stands on independently but if it start to talk with other modules (e.g. barbican), we should prevent to fetch keys in outside of encrypter/decrypter, right?07:35
*** ppai has quit IRC07:35
cschwedei see the keymaster similar to tempauth - it’s working, and a reference implementation, but you probably don’t want to use it in production on large scale07:36
timburkemattoliverau: kota_: well...unless you referenced the paste entrypoint in the [filter:encryption] section. but that starts to feel a little silly; why re-invent both the app-wrapping *and* loading from paste?07:36
kota_mattoliverau: oh, yeah, probably it makes complicated though.07:36
*** baojg has joined #openstack-swift07:36
mattoliveraureinvent all the things!!!07:37
timburkecschwede: i'm not entirely sure that's true. it may not satisfy all requirements for all use cases, but i think it's actually pretty solid07:37
timburkeas far as preventing the fetching of keys outside of encrypter/decrypter, i'd maintain that we can't reasonably protect from rogue middlewares. especially given how encryption is expected to be to the far right of the pipeline07:38
cschwedetimburke: i’m not saying that it’s a bad implementation; fully aggree that it’s pretty solid. but i already heard comments like „master key in config file? no way…“ - so ops might want to use other keymaster implementations, where one uses barbican, hardware modules or something other07:39
kota_timburke: good point, alwyas my headake for that.07:39
*** baojg has quit IRC07:39
kota_timburke: so my question starts from why keymaster can reuse the key in self._keys.07:40
kota_in Keymaster.get_keys()07:40
kota_I was playing to add "raise" if resusing it, a bunch of tests failed because there are some case both encrypter and decrypter calling for fetching.07:41
kota_so I failed :'(07:41
*** manous has quit IRC07:41
kota_so restricting call times seems a bad way.07:42
kota_however if we could encupsulerate Keymaster(Decyrpter(Encrypter())), it enables us to maintain the fetch callback easily.07:43
kota_(it probably needs more wrapping though)07:43
timburkecschwede: don't we already have rather valuable pieces of info in our configs? admin_key, credentials for authtoken, swift_hash_path_*07:44
mattoliverauk my parents have turned up (vistiing for dinner, dinner I'm suppose to be makeing) so I'd better do be social. I'll check in later once they've left.07:47
kota_right now, I don't have strong oppsite opinion at the point of merging encrypter/decrypter but not sure if jsut encypter/decypter is better way or not07:48
kota_mattoliverau: have a fun07:48
*** ppai has joined #openstack-swift07:48
*** tqtran has joined #openstack-swift07:48
timburkekota_: i'm still not convinced that having multiple calls to fetch_crypto_keys is a bad thing, or that it should be restricted07:49
*** geaaru has joined #openstack-swift07:50
timburkethe main thing i'm hoping for by combining encrypter/decrypter is to simplify what operators have to think about. in that light, 3->2 is a win; 2->1 could be another win, but i'm not convinced that we can do it yet07:52
*** acoles_ is now known as acoles07:52
*** tqtran has quit IRC07:53
acolesgood morning07:53
kota_timburke: yup07:53
kota_acoles: morning \o/07:53
timburkeok, that's enough crypto for now. sleepy time. i'll think more about the callback interface while i sleep07:53
timburkecrap! i stayed up until acoles woke up :-(07:53
kota_timburke: oh you can think while you are asleep, awesome!?07:54
timburkekota_: the key question is whether i can retain the ideas :-)07:54
kota_timburke: kidding, but THANKS, please take a rest enough :-)07:54
*** baojg has joined #openstack-swift07:55
timburkegood night07:56
cschwedetimburke: true, but i think the masterkey is something special and you want to keep it from leaking even more than other credentials07:56
cschwedetimburke: rest well!07:56
* kota_ is heading for grabbing a cup of pop corn to consider more deeply.07:56
timburkecschwede: one more reason i'd like to see key rotation soonish07:57
* cschwede nods in timburke’s direction07:57
acolestimburke: good morning!07:57
acolesabout merging decrypter (D) and encrypter (E) - the thing is, there is no reason for them to be separate because (imo) we don't need the keymaster (KM) to be in the middle.08:00
acolesSo although the doc says the pipeline must be D KM E, it can in fact be KM D E (or even KM E D) - try it, it works.08:00
acolesGiven that, it just seems simpler to have them be a single filter.08:01
acolesWhereas, there are arguments for keeping the KM separate (and also arguments for not) but it is not such a no-brainer as combining D+E08:02
kota_acoles: yes, that's true with current implementation.08:08
kota_the reason I got the question was that I thought the *originial* idea for (D KM E) order was shutting the key leak to other middleware to right/left of encryption staffs.08:09
kota_probably, I was wrong though.08:09
*** _JZ__ has joined #openstack-swift08:09
*** _JZ_ has quit IRC08:10
openstackgerritDavanum Srinivas (dims) proposed openstack/swift: [WIP] Testing latest u-c
kota_(i.e. D, E can drop the fetch key functionality when passing a request to left/right middlewre.08:10
*** vinsh has joined #openstack-swift08:12
kota_(I found/know, my expectations is wrong with current implementation.08:12
kota_so now we can fetch_key in eny middleware in any places and that's why we can merge the left D into the right E.08:13
acoleskota_: that wasn't my understanding. We are not attempting to protect against an attack on proxy code. If I have access to proxy and malicious middleware M then I could insert it here D KM M E. I could also just read the plaintext data.08:13
kota_but not sure it's by design or not.08:13
kota_acoles: yes, yes, exactly malicious middleware can read plaintext data anyway.08:15
acoleskota_: we never have erased the callback function from the request env. But now you mention it we could do that.08:15
kota_i might think too nervous :/08:15
*** vinsh has quit IRC08:16
*** jamielennox is now known as jamielennox|away08:16
acoleskota_: With KM D+E, KM installs callback in req env, passes env to D+E. For a GET then once D+E have resp, callback is called to fetch keys, then callback is erased before resp is passed back to KM and on to other middleware.08:17
acoleskota_: Similar for PUT, the callback could be erased so no other middleware can access it. But again, that assumes correct pipeline config, not KM other D+E.08:18
kota_acoles: yes, exactly08:18
acoleskota_: However, I think a security expert would tell us that it still does not protect us against a malicious attacker who has access to python runtime.08:19
kota_acoes: agreed08:22
kota_acoles: so summrized those, the merging D/E seems totally fine to me and we could still have a time to think of pipeline healthiness (probably after landed) anyway08:23
acoleskota_: ok. thanks for thinking about it, it's great review!08:24
kota_acoles: :-)08:25
kota_ah, can i finish up my review in 12 hours?08:33
kota_the irc meeting starts since 6:00 am and now 5:30 p.m.08:33
kota_it seems I did!08:34
*** baojg has quit IRC08:34
*** asettle has joined #openstack-swift08:36
admin6Hi there, one dummy question about async pendings. When I use swift-recon -a  and got "[async_pending] - No hosts returned valid data." answer. Does that means "all is ok", no hosts have someting to notice, or "There is a problem" because I receive no valid data from hosts ?08:36
*** baojg has joined #openstack-swift08:37
ahaleare your hosts running the swift-recon-cron cron? that populates the recon cache with async info08:39
*** baojg has quit IRC08:41
*** baojg has joined #openstack-swift08:43
*** sheel has quit IRC08:45
*** bapalm has quit IRC08:57
*** bapalm has joined #openstack-swift09:00
*** balajir_ has joined #openstack-swift09:01
*** treyd_ has joined #openstack-swift09:02
*** baojg has quit IRC09:04
*** diogogmt has quit IRC09:08
*** j_king has quit IRC09:08
*** balajir has quit IRC09:08
*** ndk_ has quit IRC09:08
*** mathiasb has quit IRC09:08
*** sgundur1 has quit IRC09:08
*** treyd has quit IRC09:08
*** mathiasb has joined #openstack-swift09:08
*** j_king has joined #openstack-swift09:09
*** ndk_ has joined #openstack-swift09:09
*** arcimboldo has quit IRC09:11
*** _JZ__ has quit IRC09:11
*** _JZ_ has joined #openstack-swift09:12
*** mathiasb has quit IRC09:13
*** mmcardle has joined #openstack-swift09:13
*** mathiasb has joined #openstack-swift09:14
*** sgundur1 has joined #openstack-swift09:15
*** baojg has joined #openstack-swift09:16
*** mmcardle has quit IRC09:27
*** ppai has quit IRC09:27
*** ppai has joined #openstack-swift09:29
*** baojg has quit IRC09:33
*** baojg has joined #openstack-swift09:38
*** tmoreira has joined #openstack-swift09:39
*** mmcardle has joined #openstack-swift09:40
*** arcimboldo has joined #openstack-swift10:03
*** baojg has quit IRC10:04
*** baojg has joined #openstack-swift10:06
*** vinsh has joined #openstack-swift10:10
*** rledisez has quit IRC10:15
*** baojg has quit IRC10:18
*** arcimboldo has quit IRC10:18
*** arcimboldo has joined #openstack-swift10:19
*** mmcardle has quit IRC10:33
*** vinsh has quit IRC10:39
*** _fortis has quit IRC10:43
*** mmcardle has joined #openstack-swift10:47
*** klamath has joined #openstack-swift10:56
*** _fortis has joined #openstack-swift10:58
*** kei_yama_ has quit IRC11:02
*** psachin has quit IRC11:04
*** psachin has joined #openstack-swift11:06
*** mvk has quit IRC11:29
*** klamath has quit IRC11:30
*** hseipp has quit IRC11:31
*** hseipp has joined #openstack-swift11:31
*** psachin has quit IRC11:35
*** ppai has quit IRC11:35
*** dmorita has joined #openstack-swift11:39
*** vinsh has joined #openstack-swift11:40
*** dmorita has quit IRC11:44
*** tqtran has joined #openstack-swift11:45
*** vinsh has quit IRC11:45
*** mmcardle has quit IRC11:46
*** psachin has joined #openstack-swift11:48
*** tqtran has quit IRC11:48
*** links has joined #openstack-swift11:48
*** links has quit IRC11:48
*** ppai has joined #openstack-swift11:49
*** asettle has quit IRC11:51
*** asettle has joined #openstack-swift11:57
*** mvk has joined #openstack-swift11:57
*** NM has joined #openstack-swift12:09
admin6ahale: you’re right I was missing the swift-recon-cron. thanks (and sorry for thte delay to answer)12:10
*** NM has quit IRC12:10
*** NM1 has joined #openstack-swift12:10
*** raildo-afk is now known as raildo12:13
*** daemontool has joined #openstack-swift12:14
*** vinsh has joined #openstack-swift12:20
*** jamie_h has joined #openstack-swift12:27
*** mmcardle has joined #openstack-swift12:46
*** psachin has quit IRC12:57
*** ManojK has joined #openstack-swift12:59
*** NM1 has quit IRC13:02
*** NM has joined #openstack-swift13:04
*** ametts has joined #openstack-swift13:21
*** mmcardle has quit IRC13:25
*** klamath has joined #openstack-swift13:27
*** klamath has quit IRC13:27
*** klamath has joined #openstack-swift13:27
*** zaitcev has joined #openstack-swift13:31
*** ChanServ sets mode: +v zaitcev13:31
*** psachin has joined #openstack-swift13:36
*** ManojK has quit IRC13:37
*** manous has joined #openstack-swift13:43
*** sheel has joined #openstack-swift13:44
*** manous has quit IRC13:51
*** diogogmt has joined #openstack-swift13:54
*** mmcardle has joined #openstack-swift13:57
*** ManojK has joined #openstack-swift13:58
*** manous has joined #openstack-swift14:04
*** adu has joined #openstack-swift14:05
*** ManojK has quit IRC14:13
*** ppai has quit IRC14:13
*** diogogmt has quit IRC14:14
*** rcernin has quit IRC14:14
*** _JZ_ has quit IRC14:18
*** vint_bra has joined #openstack-swift14:23
*** rcernin has joined #openstack-swift14:29
*** jistr is now known as jistr|mtg14:31
*** adu has quit IRC14:33
*** ManojK has joined #openstack-swift14:33
*** Suyash has joined #openstack-swift14:40
*** balajir_ has quit IRC14:43
*** cdelatte has joined #openstack-swift14:43
*** charz has quit IRC14:43
*** charz has joined #openstack-swift14:46
*** rcernin has quit IRC14:46
*** balajir has joined #openstack-swift14:46
*** pcaruana has quit IRC14:46
acolesmahatic_: if you're still here, where is the log message made that you mention here ?14:47
patchbotacoles: patch 328208 - swift (feature/crypto-review) - Enable object body and metadata encryption14:47
*** npf has joined #openstack-swift14:54
*** rcernin has joined #openstack-swift14:59
*** ManojK has quit IRC15:01
*** ManojK has joined #openstack-swift15:03
*** manous has quit IRC15:05
asettlenotmyname you know, I don't even remember signing up for a swiftstack newsletter :P15:08
*** vint_bra has quit IRC15:11
*** diogogmt has joined #openstack-swift15:12
*** psachin has quit IRC15:17
*** manous has joined #openstack-swift15:17
*** rcernin has quit IRC15:19
*** jistr|mtg is now known as jistr15:23
*** tesseract- has quit IRC15:27
*** dmorita has joined #openstack-swift15:33
*** thumpba has joined #openstack-swift15:33
*** dmorita has quit IRC15:39
*** jmccarthy has quit IRC15:45
*** jmccarthy has joined #openstack-swift15:46
admin6Hi there. Is the swift-dispersion report compatible with erasure coded rings ? and if not, is there an equivalent soution for EC?15:47
*** jordanP has joined #openstack-swift15:50
*** silor has joined #openstack-swift15:51
notmynamegood morning15:52
notmynameasettle: doh! sorry about that. probably some conference hall badge scan sort of thing. I'll make sure you're off15:52
asettlenotmyname: haha nah it's not a problem at all :) did wonder how I started getting them though...15:53
*** silor1 has joined #openstack-swift15:56
*** silor has quit IRC15:58
*** silor1 is now known as silor15:58
*** nadeem has joined #openstack-swift15:59
*** jordanP has quit IRC16:00
notmynameacoles: looks like some good review comments today16:02
acolesnotmyname: yes, has kept me busy!16:03
notmynameacoles: where should I start this morning?16:04
acolesnotmyname: re I like clayg's idea to move to common/middleware/encryption and put the filter factory in __init__, and I think others were positive, but will that lose us git change history?16:05
notmynameno, not if patch 335641 is done with `git mv`16:06
patchbotnotmyname: - swift (feature/crypto-review) - Use a single wsgi filter for the encrypter and dec...16:06
*** Jeffrey4l has quit IRC16:08
*** hseipp has quit IRC16:12
acolesnotmyname: I'm about to push new versions so you can start by reviewing the changes there.16:13
*** arcimboldo has quit IRC16:14
*** nadeem has quit IRC16:14
*** mwheckmann has joined #openstack-swift16:15
notmynameI'm fascinated that every time Huge Corp (tm) makes an announcement of some new thing they're doing, especially when it's New! Innovative! thing, the announcement always starts with something like "we got a small group of smart people in a room and ..."16:15
*** cdelatte has quit IRC16:16
acolesnotmyname: oic. I had intended to squash the changes into the crypto and docs patches, so we'd still have the original chain of 6. does that make sense to you? so we never merge a patch that had D KM E16:16
notmynamenow, when I'm reading the ML thread about the proposed architecture working group, I'm doubly fascinated that OpenStack, which itself started as this New! Innovative! thing, has people who are trying to get a small group of people together to go do some innovative thing16:16
notmynameacoles: yeah, I'm fine with that. but in that case I'm missing where we'd lose git history16:17
ahaleOpenstack is always open to small groups of people doing innovative things!16:17
notmynamethe patch chain is the history16:17
notmynameacoles: as long as it's the same innovative things that everyone else is doing in the same way everyone else is doing it ;-)16:18
*** lyrrad has joined #openstack-swift16:18
ahaleabsolutely, anything else might be too disruptive after all16:19
notmynametab complete nick fail. oops16:20
acolesnotmyname: duh, of course, we haven't merged anything yet so there is no git history.16:20
* acoles is tired 16:21
notmynameacoles: we're inventing the history!16:21
acolesnotmyname: yeah, let's get a small group of people in a room and invent history ;)16:22
acolesnotmyname: so I am concerned that its is really hard to review a file rename in gerrit, so I propose that we take it as an isolated update to the patchset i.e. the only change is the filter factory stuff and file relocation16:23
timburkegood morning16:24
notmynameie a patch on top of the current chain (similar to how it's proposed now)?16:24
acolesnotmyname: no. sorry. I mean, I push a new version of the last 2 patches (crypto, doc) where the *only* change is file relocation and stuff related to the filter-factory change. Then reviewers mainly need to check the file contents are identical. No other changes mixed in.16:27
timburkeacoles: notmyname: i like that idea. and i think it can land on master16:27
timburkeoh, i misread that, too16:28
acolesa 7th patch is *easier* but leaves us with weird stuff in the history, like we had 3 filters but then we didn't. IDK, I am trying to figure it out but would like to get it right16:29
notmynameacoles: I'd much prefer it to be a simpler history to look at in 6 months than whatever gerrit makes easiest for this week16:30
notmynamebut I'm not sure that it's too weird int he history to have the 7th patch set16:31
acolesnotmyname: that's hedging ;) let's go for 6 patches and clean history16:32
acoleswouldn't it be nice to have a button in the comment reply dialog in gerrit labelled 'Later' that created a link to the comment in trello or somewhere?16:33
timburkeacoles: for all of my comments last night, you should absolutely feel free to leave a "Later" comment and rely on me writing the patch16:35
acolestimburke: too late!16:35
*** mmcardle1 has joined #openstack-swift16:36
*** silor has quit IRC16:36
*** gyee has joined #openstack-swift16:36
timburkehaha i'd tried to make it clear in the over-all comment...16:36
*** mmcardle has quit IRC16:38
acolestimburke: the perfectionist in me can't resist (although I have left a few)16:38
openstackgerritAlistair Coles proposed openstack/swift: Enable object body and metadata encryption
openstackgerritAlistair Coles proposed openstack/swift: Add encryption overview doc
acoles^^ ok that is all changes I have APART from the filter factory change which I am now going to squash into those two reviews16:39
acolesnotmyname: shall we start to merge the first few patches onto crypto-review to get a head start on zuul?16:39
acolestimburke: your dash with 'Has draft' has been very useful16:41
*** cdelatte has joined #openstack-swift16:41
notmynameacoles: I don't think it matters too much. as long as the merge commit gets landed by my monday morning, I think we're ok. ie we've got all weekend to let zuul chew on it16:41
notmynameacoles: but that being said, if you drop your -2 on the first one, then based on comments today (while you're sleeping), I can start landing things if they look good16:43
notmynameacoles: or we can start that when you wake up tomorrow16:43
acolesnotmyname: ok sounds like a good plan (drop -2) - you could add a -2 just in case16:44
notmynameacoles: done16:44
*** dmorita has joined #openstack-swift16:45
*** mmcardle1 has quit IRC16:48
*** dmorita has quit IRC16:50
*** nadeem has joined #openstack-swift16:50
*** vint_bra has joined #openstack-swift16:50
acolestimburke: the change to not base64encode the path in keymaster was important, can i ask you to cast an eye of this related change though ? I found I need this to fix some func tests that broke.16:50
patchbotacoles: patch 328208 - swift (feature/crypto-review) - Enable object body and metadata encryption16:50
timburkeacoles: we'll need to update the docstring, and when we eventually move to py3, we may need to revisit16:54
*** mmcardle has joined #openstack-swift16:54
timburkealternatively, you could do something like `val.encode('utf8') if six.PY2 else val` (while still updating docstring a little16:54
acolesugh, the docstring :/ can we live with that for now?16:58
notmynametimburke: really? really?! we might have to change something when we move to py3? *that's* the sort of review you'll give on the crypto branch today?! ;-)16:59
timburkesure. adding it to my TODOs...16:59
*** asettle has quit IRC16:59
timburkei'm shocked (*shocked*!) to find changes necessary to move to py3!17:00
timburkeacoles: fundamentally, the encoding seems sane17:00
*** daemontool_ has joined #openstack-swift17:01
acolestimburke: ok, thanks17:01
*** ManojK has quit IRC17:01
timburkeacoles: i thought the point of the 32 was to give us some buffer for random headers we may need to add (like EC sysmeta, or now crypto sysmeta)17:02
timburkei don't see much harm in bumping to 36, but i was kinda hoping to not have any changes there at all17:03
*** ManojK has joined #openstack-swift17:03
*** daemontool has quit IRC17:04
*** NM has quit IRC17:06
*** Suyash has quit IRC17:06
*** mvk has quit IRC17:06
*** siva_krish has joined #openstack-swift17:07
*** mmcardle has quit IRC17:08
claygtimburke: i was sorta hoping we'd leave it at 132 "just cause"17:10
notmynameclayg: bigger numbers are better, right?17:10
claygnotmyname $%YUing gets it17:10
timburkeclayg: notmyname: i guess you're right. like 600s bulk delete times! who wants to go *down* to 20s?!?17:12
* notmyname doesn't care as long as the default config isn't broken17:12
acolestimburke: IIRC someone, cschwede perhaps, did a count of the headers we might expect to reach 32. Since we're adding some it seemed logical to increase that number. It doesn't actually result in any resource being committed in httplib.17:12
notmyname36 is good. let's not bnikeshed about it17:13
timburkeyeah, i;m fine with leaving it17:14
acolesreminds me though, the limit doesn't come in til py2.7.9 (I think?) and I only have 2.7.6 on my saio - can anyone confirm func tests do work on 2.7.9 and that we don't need to allow *more* headers?17:15
acolesjrichli: you saw the max header problem on your machine right? ^^17:15
timburkeon the content-type thing, i think we *would* want to extract before decrypting, since we're using it to check for a multipart response. but we can talk about that more in san antonio (or whenever we get around to encrypting content-type)17:16
notmynameacoles: yeah, I've got 2.7.11+ IIRC, so I'll run functests17:16
*** dmorita has joined #openstack-swift17:16
notmynamepatch 335641 is going away, right?17:17
patchbotnotmyname: - swift (feature/crypto-review) - Use a single wsgi filter for the encrypter and dec...17:17
*** peluse has quit IRC17:17
acolesnotmyname: thanks, and yes17:20
*** Suyash has joined #openstack-swift17:21
*** chsc has joined #openstack-swift17:21
*** chsc has joined #openstack-swift17:21
notmynameacoles: functests pass on my SAIO for patch 328209 with python 2.7.11+17:23
patchbotnotmyname: - swift (feature/crypto-review) - Add encryption overview doc17:23
acolesnotmyname: with EC?17:23
notmynameacoles: no. running that now17:25
*** cdelatte has quit IRC17:27
claygtimburke: think of it as *deletes per second* - bigger is always better, esspecially when its faster17:29
notmynameacoles: all functests passed with an EC policy as default17:30
acolesnotmyname: excellent, thank you17:30
acolesnotmyname:, fallocate? ;)17:30
acolesfast-post in checked in gate17:31
*** manous has quit IRC17:32
*** npf has quit IRC17:32
notmynameusing fallocate17:32
claygis this is then?  are we going to merge this thing!?17:33
jrichliacoles: got back from lunch.  reading now17:33
notmynameclayg: the unification patch is going to be submitted, I think (acoles?)17:34
acolesnotmyname: clayg the unification patch is going to be squashed into review 323208 and patch 32320917:35
patchbotacoles: - cookbook-openstack-image - initial commit for the newton development cycle (MERGED)17:35
acolesargh! patch 328208 and 32820917:35
patchbotacoles: - swift (feature/crypto-review) - Enable object body and metadata encryption17:35
acolespatch 32820917:36
patchbotacoles: - swift (feature/crypto-review) - Add encryption overview doc17:36
*** klrmn has joined #openstack-swift17:36
*** asettle has joined #openstack-swift17:43
*** NM has joined #openstack-swift17:44
claygnotmyname: are we sure?  isn't 3 middlewares obviously better than only 2 middlewares?17:45
*** ChubYann has joined #openstack-swift17:45
*** manous has joined #openstack-swift17:46
claygnotmyname: maybe config options is the exception to the rool?  Bigger version; Less options - PEFECT17:46
notmynamewait, I'm not sure which way you're advocating now17:47
notmynameclayg: which one do you want, and why?17:47
*** asettle has quit IRC17:47
*** gyee has quit IRC17:48
claygnotmyname: i'm j/k17:48
notmynameABOUT WHICH ONE?!17:48
acolesclayg: I had this idea overnight for making it four17:49
claygThe views, opinions and positions expressed by clayg the IRC troll are his alone, and do not necessarily reflect the views, opinions or positions of clayg reviewer17:50
claygacoles: i like where you going...17:51
acolesclayg: me too, I'm going for dinner...17:51
jrichliacoles: FWIW, functests pass on my python 2.7.6 as is, and I made sure my configs did not override default max17:57
notmynamegood news, everyone. no merge conflicts between patch 328209 and master18:00
patchbotnotmyname: - swift (feature/crypto-review) - Add encryption overview doc18:00
*** manous has quit IRC18:01
notmynamehere's the plan18:02
*** ManojK has quit IRC18:02
notmynameacoles (after dinner) will push the 2 middleware version into the current patch chain, and we'll continue reviewing18:02
notmynamelater today, I'll start landing stuff on the feature/crypto-review branch18:02
notmynameand wehn we've got the positive reviews to the end of the chain, I'll propose the merge commit to master18:02
notmynamei'm hoping to do that tonight18:02
jrichliso, i guess this is really happening ...18:03
jrichliso, where's the party Friday night?18:04
notmynameif all this goes according to plan, I'll bring beer to the swiftstack office on friday ;-)18:05
jrichlitoo bad you can't 'beam' us all there18:06
claygwhich gate/devstack test/setup exactly cares if the default root crypto secret is set?18:06
*** SkyRocknRoll has quit IRC18:06
jrichliclayg:  I did some initial looking into how we can setup a gate test when crypto is not in default pipeline18:07
*** arcimboldo has joined #openstack-swift18:07
jrichlibut I didnt see a solution immediately18:07
notmynameI think we'll need to build one. like with the fast-post or ec test18:08
*** mvk has joined #openstack-swift18:09
jrichliand it looked like there will need to be something really new to be able to support internal client config changes18:12
jrichliacoles had mentioned maybe making use of SWIFT_EXTRAS_MIDDLEWARE.18:13
*** manous has joined #openstack-swift18:14
jrichliI am new to looking at the devstack setup.  from what I see, you'd have to have some conditional configured with devstack to be able to selectively apply that, such as the "is_service_enabled" flag.18:14
*** thumpba has quit IRC18:18
*** _JZ_ has joined #openstack-swift18:21
claygjrichli: yeah makes sense about the internal-client.conf :\18:21
jrichliclayg: but actually, the only probetests we have dont use devstack, right?  its a swiftstack cluster?  so maybe you know a good way to modify the internal client conf there?18:23
timburkenow there's a good feeling. the first three sections of my crypto review dash are all empty :-)18:24
jrichlitimburke: nice work!18:24
jrichliI meant to say, the only gate tests running probetests don't use devstack18:25
*** ManojK has joined #openstack-swift18:26
*** manous has quit IRC18:27
acolesnotmyname: if i stop work now and drive to heathrow I can arrive in SF in time for beers18:29
*** thumpba has joined #openstack-swift18:29
timburkeacoles: oh no you don't! there's still a patch that needs squashing ;-)18:30
tdasilvaacoles: they do have wifi on the train to heathrow and it works pretty well18:31
acolesoh bother i messed up the translations18:33
acolestimburke: well IDK if clayg is going to love me or not if I squash that change in18:33
timburkeacoles: again, i'm happy to (1) be the one that fixes it and (2) do it on master18:33
*** thumpba has quit IRC18:34
*** Fin1te has joined #openstack-swift18:34
*** thumpba has joined #openstack-swift18:35
*** thumpba has quit IRC18:35
*** thumpba has joined #openstack-swift18:35
*** geaaru has quit IRC18:41
*** vinsh has quit IRC18:48
*** manous has joined #openstack-swift18:51
*** manous has quit IRC19:05
torgomaticso this is not something we should actually do, but it's sort of interesting that we can do this to patch 32820819:17
patchbottorgomatic: - swift (feature/crypto-review) - Enable object body and metadata encryption19:17
timburketorgomatic: yay CTR mode!19:19
*** manous has joined #openstack-swift19:20
jrichlitorgomatic: true :-)19:21
timburke(that's not generally a property of block ciphers. OFB mode also has the property that D == E, but CBC and CFB (and of course ECB) modes don't)19:24
*** manous has quit IRC19:28
*** vinsh has joined #openstack-swift19:29
jrichliyes, i thought it was strange to see 2 diff methods in the library.  but i figured we should go with using them both.19:32
* jrichli is gonna be away for a bit19:34
*** sheel has quit IRC19:35
claygtorgomatic: wtf is that?  offset = 0 !?19:36
torgomaticclayg: just playing around. offset is only for ranged GETs and create_encryption_ctx() is only for PUTs, so offset=0 means start at the beginning19:37
*** vint_bra has quit IRC19:39
*** tqtran has joined #openstack-swift19:39
claygtorgomatic: ah, i see now - had to look at in context - that's goofy and I don't understand what timburke said :D19:40
*** sgundur_ has joined #openstack-swift19:40
*** sgundur_ has left #openstack-swift19:40
*** vint_bra has joined #openstack-swift19:40
*** hoonetorg has quit IRC19:40
timburkeclayg: in general, the encryption and decryption functions are different. for certain modes of operation (, however, they'll be the same19:42
claygtimburke: lol @ the location bar in my browser!
acolesclayg: btw I like the idea of path to file for secret, can it be a follow up?19:46
*** daemontool_ has quit IRC19:46
claygacoles: mattoliverau seemed to think having it be a string in a config was useful too, he suggested that we support both with strong preference to a new option to specify path19:47
clayggivin that suggestion i'm ok fixing it as soon as it's on master19:47
acolesclayg: yeah +1 for that19:47
claygI wish I could talk to someone who has some operational experience obtaining a encryption keys from a "cryptographically secure random number generator" so they could tell us what these things output?19:49
claygI imagine something that looks more like a well-formed ssh id_rsa private key looking thing?19:49
claygthen we could just start adding support for these well known formats that such tools might be able to output too19:50
claygcooking up our little base64 scheme seems like it's only useful for text in config files - a file on disk could just be binary for all we care - or maybe something more complicated19:50
clayg... dunno19:50
acolesnotmyname: timburke clayg jrichli ok I have squashed patch 335641 into the chain and moved everything to middleware/crypto. just running tests. you ready for me to push those new versions soon?19:51
patchbotacoles: - swift (feature/crypto-review) - Use a single wsgi filter for the encrypter and dec...19:51
notmynameacoles: yes!19:51
*** hoonetorg has joined #openstack-swift19:51
acolesclayg: this will give you something suitable 'openssl rand -base64 32'19:52
claygacoles: i have to admit i'm also thinking about swiftstack product - while in one mode we may say "we'll handle the keys; trust us" another option would be something like "when you're adding a new node place your key into /etc/swift/root_secret" if it's not there we'll tell you and demand you fix it because we don't have that key19:52
claygacoles: did i miss that in the docs?19:52
acolesclayg: you did ;)19:52
claygacoles: indeed :)19:53
timburkeclayg: id_rsa *is* base64-encoded, no?19:53
acolesclayg: its the same format of key that you use with google/aws byok I believe19:54
claygtimburke: yeah, sorry I shouldn't have trying to make it sound like I'm against base64 - the fact that openssl can output to that format gives pleanty of weight to it being a reasonable data exchange in this role19:54
claygtimburke: i'm looking at what other formats it supports19:55
acolesclayg: it also simplifies the config of proxy and container sync internal client, if both can point to a file19:56
timburkeclayg: ah. was gonna say, `sed '1,/^$/d; /^--/d' <~/.ssh/id_rsa | base64 -D | hd` certainly *seems* to work...19:56
claygacoles: yeah that's how I stumbled to it19:56
acolespossibly makes it easier to ship some sample config that points to a well known secret file that needs to be populated19:56
claygacoles: like that too19:57
acolesclayg: tbh I hadn't anticipated container sync working with encryption without further work, so it was late on before I got into that internal client config thing19:58
*** asettle has joined #openstack-swift19:59
*** asettle has quit IRC19:59
claygacoles: yeah no worries - glad it works!19:59
acolesnotmyname: timburke I'm not sure what/where to link from middleware.rst now we have this composing filter factory, so I'm just putting something very minimal for now20:00
claygacoles: so is there a rebase coming or what20:04
acolesok tests good, here goes20:05
acoleshope i didn't screw this up20:06
openstackgerritAlistair Coles proposed openstack/swift: Enable object body and metadata encryption
openstackgerritAlistair Coles proposed openstack/swift: Add encryption overview doc
acoles^^ don't forget to change pipelines in proxy-server.conf and container sync client conf20:08
timburkeand reinstall, either with pip or directly20:09
*** tqtran has quit IRC20:09
*** tqtran has joined #openstack-swift20:10
*** mwheckmann has quit IRC20:11
*** tqtran has quit IRC20:15
*** tqtran has joined #openstack-swift20:15
notmynameis the setup.cfg right?20:18
notmynameit has keymaster = swift.common.middleware.keymaster:filter_factory but shouldn't it be "keymaster = swift.common.middleware.crypto.keymaster:filter_factory"20:19
notmynamemade that change locally, reinstalled, running tests now20:21
*** ManojK has quit IRC20:24
*** ManojK has joined #openstack-swift20:26
*** gyee has joined #openstack-swift20:26
claygwait, so there *is* or is not a crypto_helpers?20:27
timburkeclayg: there is a crypto_helpers in the unit tests. comparable to test/unit/common/middleware/helpers.py20:29
timburkethere is *not* one under swift/ -- thought here's a crypto_utils.py20:29
acolesso how did it install for me if setp.cfg is wrong?20:29
timburkeacoles: stale pyc files20:30
notmynamebecause you might have an old one20:30
timburkegit clean -fX20:30
*** cdelatte has joined #openstack-swift20:30
notmynameall tests passed locally for me20:30
acolesof course. it is wrong.20:30
notmynameall = functests20:30
*** klrmn1 has joined #openstack-swift20:31
*** klrmn has quit IRC20:32
claygacoles: just keep pushing till it works - you have an army of testers waiting in the wings20:34
acolesyou want me to push the fix or wait for any other bloopers?20:34
*** cdelatte has quit IRC20:34
clayg#vote push it20:35
clayg*before* we find any other bloopers - let's go ahead and make it installable ;)20:35
*** Fin1te has quit IRC20:37
openstackgerritAlistair Coles proposed openstack/swift: Enable object body and metadata encryption
openstackgerritAlistair Coles proposed openstack/swift: Add encryption overview doc
*** cdelatte has joined #openstack-swift20:39
*** thumpba has quit IRC20:40
notmynamelooks good so far :-)20:40
*** rcernin has joined #openstack-swift20:41
acolestimburke: what was I doing in the factory test? :/20:42
timburkeacoles: passing {} as the app20:42
acolesright! :/20:43
timburkeit'll do, but doesn't really reflect what we expect to happen. i'd left a suggestion on how we could improve it, but that can be later (if ever)20:43
acolesgerrit seems to have stopped sending me email so I'm having to poll gerrit20:45
notmynameok, I've got a +2 on all of the patches now (except the cork -2 on the first one). I'm ready to go20:45
timburkei'm on board, too20:46
notmynamejrichli: I'd love to have your votes before you take off today20:47
notmynameand mattoliverau and kota_ should be around in a few hours20:47
notmynameclayg: torgomatic: your votes are appreciated too :-)20:47
jrichlinotmyname: ack.  will be a few hours from now20:48
jrichliis that ok?20:48
notmynamejrichli: just tell the manager types that meetings can wait--you gotta land crypto ;-)20:48
notmynamejrichli: yes, of course that's fine :-)20:48
jrichlinotmyname: :-)20:48
notmynameit's a little before 2pm here, so I'd prefer to start landing the chain to crypto-review in a couple of hours, and then tonight I can do the merge commit to master20:49
jrichlii am reviewing now, btw20:49
notmynameassuming everyone is ok with it, of course20:49
jrichliah, ok20:49
*** raildo is now known as raildo-afk21:02
*** siva_krish has quit IRC21:05
*** cdelatte has quit IRC21:09
*** ametts has quit IRC21:13
notmynameinteresting. I open hacker news and see a post on the front page that's simply a link to intel's ISA-L stuff
claygjrichli: acoles: you know in the end i'm pretty happy with x-object-transient-sysmeta21:27
*** notmyname has quit IRC21:28
*** notmyname_ has joined #openstack-swift21:28
*** ChanServ sets mode: +v notmyname_21:28
*** notmyname_ is now known as notmyname21:29
*** NM has quit IRC21:34
*** diogogmt has quit IRC21:36
*** HenryG_ has joined #openstack-swift21:41
*** jroll|dupe has joined #openstack-swift21:44
*** jroll|dupe has quit IRC21:44
*** jroll|dupe has joined #openstack-swift21:44
*** kevinc has joined #openstack-swift21:44
*** hoonetorg has quit IRC21:45
*** HenryG has quit IRC21:45
*** MooingLemur has quit IRC21:45
*** jroll has quit IRC21:45
*** hogepodge has quit IRC21:45
*** jroll|dupe is now known as jroll21:45
jrichliclayg: happy to hear it :-)21:46
*** jamie_h has quit IRC21:47
kevincI am updating my container ring for the first time since upgrading to liberty and i get the following warning: RingValidationWarning: The partition xxx has been assigned to duplicate devices . How to i fix the issue? Should I ignore the warning?21:47
notmynamekevinc: that would normally happen if you've got less drives than replicas. eg 2 drives and 3x replication21:48
kevinci have 4 replica with 20 drives21:49
kevinc4 zones21:49
notmynamekevinc: and, FWIW, it's the warning that's new, not the underlying issue21:49
notmynameoh, that's interesting21:49
kevinc1048576 partitions, 4.000000 replicas, 1 regions, 4 zones, 20 devices, 0.00 balance, 0.67 dispersion21:49
kevincwe have been using/updating the same ring since before grizzly, we did make a change from 3 replicas to 4 replicas about 2 years ago21:50
claygkevinc: you're not on the latest code then - the latest code makes that an ERROR21:51
claygkevinc: the reason we couldn't make it an error on the old code was the old code sucked and torgomatic wouldn't let me borrow his time machine21:51
*** cdelatte has joined #openstack-swift21:51
claygso we had to fix it in the latest code21:51
claygyou should get on the latest code21:51
claygbut... honestly - I *thought* the latest official release had all the good ring-y bits21:51
claygcirtainly at least 2.721:52
kevincclayg: correct, we are on liberty (2.5), so 2.7 will fix the problem or just cause an error?21:52
notmynameliberty is 2.5.021:52
*** HenryG_ is now known as HenryG21:52
claygyou could generate rings with 2.7 if you need to and publish to a 2.5 cluster21:53
claygnot recommened but it would work21:53
claygwell... i'm not even sure it's not recommend - we do it21:53
kevincok, we can put off updating the ring for now until we upgrade to 2.7, thank you!21:53
notmynameif you can swing upgrading to 2.8, I'd recommend that21:53
claygkevinc: not recommended - it's trying to tell you about a terrible terrible state of your ring from 2.5 - you *should* fix it21:53
notmynamekevinc: ...and to build on what clayg is saying, the new code will fix it with a rebalance21:54
claygnotmyname: YEAH IT WILL - LIKE A BOSS!21:54
acolesclayg: :D -transient-21:55
claygdid i mispell it?21:55
notmynamekevinc: your current situation is that you have multiple replicas on the same drive, which therefore aren't different replicas at all. so you *think* you're good, but one drive failure can take out a lot more than you expect21:55
acolesclayg: no, I'm just providing context for the smile21:55
claygok, w/e - point is nice work acoles & jrichli for not backing down on a good idea despite my inability to understand it originally21:56
acolesclayg: but now you mention it, "mass" is a lot easier to spell :P21:56
claygnope, that's a stupid name21:56
acolesnotmyname: I'm thinking of calling it a day - need anything from me?21:57
notmynameacoles: I'm good, and I think there should be some landed patches by the time you wake up21:58
jrichlihave a restful night, acoles!21:58
notmynamemostly I'm waiting on people to add review votes to the patch chain21:58
jrichlispeaking of which ... i am about to +1 :-)21:59
acolesjrichli: I'm about to imbibe some of clayg's favourite tipple to help with that21:59
* clayg not sure if that's beer scotch or whisky22:00
acolesnotmyname: ok, thanks, I'll look forward to that!22:00
kevincnotmyname & clayg: thank you! I'll rebalance on my test cluster that is running mitaka (2.7)22:00
jrichliacoles: you are always increasing my vocab!  :-)22:01
acolesclayg: one or more of those22:01
kevincif i am reading the warning correctly, we have each partition on at least 3 different devices22:02
*** catintheroof has joined #openstack-swift22:03
claygwell... there's that i suppose :\22:03
acolesjrichli: I'll run out of words soon ;)22:03
acolesgood night22:03
*** acoles is now known as acoles_22:04
claygso openssl rand seems to be down for hex or base64, with -out <file> the output is still basically unadorned22:08
*** cdelatte has quit IRC22:09
*** nadeem has quit IRC22:19
openstackgerritTim Burke proposed openstack/swift: Change elifs to ifs
*** ManojK has quit IRC22:20
timburkenotmyname: just for you^^22:21
*** cdelatte has joined #openstack-swift22:27
openstackgerritTim Burke proposed openstack/swift: Stop digging for publicly_accessible ourselves
claygtimburke: that *is* acctually better - it's more obviously a series of guard returns instead of some kind of werid switch that might jump you out on the middle22:32
timburkeclayg: i'm not *opposed* to it. i just find both perfectly readable. but the important thing is, one more thing to scratch off my follow-up list22:33
timburkei've already got 4 more little guys like that just waiting for a merge commit to use as a parent22:35
*** siva_krish has joined #openstack-swift22:36
*** jamielennox|away is now known as jamielennox22:37
*** siva_krish has quit IRC22:44
*** cdelatte has quit IRC22:52
*** tqtran has quit IRC22:59
notmynamemattoliverau: good morning!23:01
notmynamemattoliverau: guess what?23:01
*** tqtran has joined #openstack-swift23:03
*** vint_bra has quit IRC23:03
*** asettle has joined #openstack-swift23:04
*** NM has joined #openstack-swift23:05
*** tqtran_ has joined #openstack-swift23:07
*** tqtran has quit IRC23:07
torgomaticmattoliverau: hint: the answer is "chicken butt". he's been like this all day; I think the stress is getting to him. :p23:09
notmynameI'm going to start landing some of the initial patches to crypto-review now23:10
mattoliveraunotmyname: chicken butt??23:10
*** asettle has quit IRC23:10
*** kong has joined #openstack-swift23:10
notmynameI'll leave and so that mattoliverau and kota_ have a chance to look at the new revisions23:10
patchbotnotmyname: patch 328208 - swift (feature/crypto-review) - Enable object body and metadata encryption23:10
patchbotnotmyname: patch 328209 - swift (feature/crypto-review) - Add encryption overview doc23:10
mattoliveraukota_: morning23:11
notmynamekota_: good morning!23:11
notmynamekota_: guess what?23:11
kota_notmyname: done encryption work?23:11
kota_oh, what23:11
kota_not yet reading the back log23:11
notmynamekota_: pretty much. I'd appreciate your comments on the patches. there was a new revision to the last 2 in the chain today23:12
kota_notmyname: ok, gotcha23:12
* kota_ will look at23:13
notmynameok, let's see what that does to the gate23:13
notmynamebuttons have been clicked23:14
kota_notmyname: do you have a time limit for them?23:14
kota_today is enough? or a few hours?23:14
kota_(or an hour :/)23:14
notmynamekota_: I'd prefer it to be in the next 5 hours23:14
* mattoliverau will look as soon as he gets out of his next meeting :)23:14
kota_notmyname: ok, trying to do that23:15
notmynamekota_: mattoliverau: thanks :-)23:15
*** kei_yama has joined #openstack-swift23:17
torgomaticquestion unrelated to crypto: of Swift developers, does anyone's /tmp filesystem *not* support extended attributes?23:17
torgomatic(or $TMPDIR; the spot where tempdir.mkdtemp() puts things)23:17
zaitcevmine for one23:18
*** catintheroof has quit IRC23:18
zaitcevI have a patch outstanding for it, but it ended with adding a gazillion /var/tmp, so I flag it as abandoned for now23:18
claygtorgomatic: atm, mine does not - why do we care about the *system* tmpdir tho?23:19
*** tmoreira has quit IRC23:20
claygtorgomatic: is that part of the reason we monkeypatch xattr in test/
claygzaitcev: that patch looks good too tho23:20
zaitcevclayg: sorry, please reload
zaitcevI overwrote it with scp by accident23:21
*** tqtran has joined #openstack-swift23:22
claygzaitcev: torgomatic: are you two even talking about the same thing?23:22
*** tqtran_ has quit IRC23:24
claygzaitcev: why is /var/tmp better than /tmp in that patch?  it's not using the cwd is it?23:24
zaitcev[zaitcev@lembas ~]$ setfattr -n user.swift.metadata /tmp/xxx23:24
zaitcevsetfattr: /tmp/xxx: Operation not supported23:24
zaitcev[zaitcev@lembas ~]$ touch /var/tmp/xxx23:25
zaitcev[zaitcev@lembas ~]$ setfattr -n user.swift.metadata /var/tmp/xxx23:25
zaitcev[zaitcev@lembas ~]$23:25
zaitcevWhat if your home directory is on NFS?23:25
notmynamedon't we use a tmpfile on the same filesystem partition as the data itself? ie an xfs volume on the drive, not the system /tmp?23:25
zaitcevI do it all the time in a test box that has a bunch of test VMs23:26
zaitcevnotmyname: we do, but this is "go test ./...", which I want to run on my laptop...23:26
*** tqtran_ has joined #openstack-swift23:27
zaitcevnotmyname: so, I switched my $HOME to XFS just for that, but there's no Swift layout in /srv/node, not to mention you probably don't want to put test files into your actual Swift anyway.23:27
claygtorgomatic: my bad, /var/tmp and /tmp both have xattrs on my system - i forgot cp needs the -a :\23:28
zaitcevMaybe it's easier to figure out why tmpfs does not support attributes. I remember vaguely that it was a problem before, and I did something to address it in kernel... There was a patch or other... And it was like 7 years ago. Yet it's 2016 and still nothing works.23:28
*** tqtran has quit IRC23:29
*** tmoreira has joined #openstack-swift23:29
claygzaitcev: so do a interface trick where you can implement a fake xattr for tests like swift unittest do?23:29
zaitcevclayg: I don't know how to do it in Go. There's no monkey-patching. The xattr is a system library. I suppose we could do it to our own layer...23:31
zaitcevclayg: What's  grep .tmp /proc/mounts  on your system? Empty?23:34
zaitcevEr.   grep /tmp /proc/mounts23:35
zaitcevtmpfs /tmp tmpfs rw,seclabel 0 023:35
claygzaitcev: interesting... vagrant@saio:~$ df /tmp/1467329166.42935.data23:37
claygFilesystem     1K-blocks    Used Available Use% Mounted on23:37
clayg/dev/sda1       41251136 2007508  37507716   6% /23:37
claygas I said, I was wrong before - on this system /tmp has xattrs - because it's the root file system (i, know, i'm surprised)23:37
zaitcevthat means not mounted, which is completely okay, just I thought every modern distro mounted /tmp23:37
clayggod only knows wtf the stupid macbook is doing23:37
zaitcevMaybe you're using Ubuntu LTS23:38
zaitcevDo you even have Systemd on it?23:38
torgomaticclayg: yeah, that monkeypatching thing... I hacked up some xattr checksumming to see if it would work, and so far it's been about 5% checksumming the xattrs and 95% dealing with stale data left in that monkeypatched thing23:38
*** chsc has quit IRC23:38
claygyeah, dunno, could just be this particular box too - vagrant keeps telling me it's outdated - and xenial is out yada yada23:38
claygtorgomatic: so you and zaitcev *are* working on the same thing?23:39
torgomaticclayg: don't think so? unless zaitcev is also adding xattr checksumming in the python code23:39
torgomatic(it'd be the first I've heard of it)23:40
zaitcevSo, not the same problem, but same root cause.23:40
zaitcevMy problem is to run  go test ./...  in Hummingbird.23:41
*** cdelatte has joined #openstack-swift23:44
claygtorgomatic: zaitcev: sounds like you guys have *opposite* problems23:45
torgomaticwell, I'm gonna just rip out the xattr monkeypatching and submit this thing; it'll get ignored for a while anyway due to crypto, and hopefully anyone with troubles will chime in23:50
*** hogepodge has joined #openstack-swift23:59

Generated by 2.14.0 by Marius Gedminas - find it at!