Tuesday, 2016-03-29

*** haomaiwa_ has quit IRC00:01
*** davideagnello has joined #openstack-trove00:01
*** haomaiwa_ has joined #openstack-trove00:01
*** imandhan has joined #openstack-trove00:03
*** davideag_ has joined #openstack-trove00:15
*** davideagnello has quit IRC00:15
*** imandhan has quit IRC00:17
*** _amrith_ is now known as amrith00:57
*** cheneydc has joined #openstack-trove00:59
*** haomaiwa_ has quit IRC01:01
*** haomaiwang has joined #openstack-trove01:01
*** haomai___ has joined #openstack-trove01:06
*** haomaiwang has quit IRC01:08
*** tqtran has quit IRC01:14
*** mingdang1 has joined #openstack-trove01:19
*** mingdang1 has quit IRC01:54
*** amrith is now known as _amrith_01:59
*** pmackinn has quit IRC02:00
*** haomai___ has quit IRC02:01
*** haomaiwang has joined #openstack-trove02:01
*** haomaiwang has quit IRC02:03
*** 7F1AAMJH0 has joined #openstack-trove02:08
*** tqtran has joined #openstack-trove02:12
*** 7F1AAMJH0 has quit IRC02:13
*** tqtran has quit IRC02:17
*** ediardo has quit IRC02:18
*** tellesnobrega is now known as tellesnobrega_af02:40
*** tellesnobrega_af is now known as tellesnobrega02:41
*** lifeless has quit IRC02:43
*** lifeless has joined #openstack-trove02:44
*** haomaiwang has joined #openstack-trove02:52
*** _amrith_ is now known as amrith02:59
*** haomaiwang has quit IRC03:01
*** haomaiwang has joined #openstack-trove03:01
*** tqtran has joined #openstack-trove03:13
*** links has joined #openstack-trove03:13
*** tqtran has quit IRC03:17
*** imandhan has joined #openstack-trove03:42
*** haomaiwang has quit IRC04:01
*** 7GHAAMTT3 has joined #openstack-trove04:01
*** sgotliv has quit IRC04:19
*** sgotliv has joined #openstack-trove04:20
*** ediardo has joined #openstack-trove04:42
*** 7GHAAMTT3 has quit IRC05:01
*** haomaiwang has joined #openstack-trove05:01
*** pcaruana has quit IRC05:09
*** ediardo has quit IRC05:10
*** haomaiwang has quit IRC06:01
*** haomaiwang has joined #openstack-trove06:01
*** sgotliv has quit IRC06:17
*** tesseract has joined #openstack-trove06:19
*** tesseract is now known as Guest5878206:19
*** Ravikiran_K has joined #openstack-trove06:22
*** haomaiwang has quit IRC07:01
*** haomaiwang has joined #openstack-trove07:01
*** tqtran has joined #openstack-trove07:14
*** tqtran has quit IRC07:18
*** rcernin has joined #openstack-trove07:36
*** mingdang1 has joined #openstack-trove07:37
*** haomaiwang has quit IRC07:39
*** mingdang1 has quit IRC07:39
*** mingdang1 has joined #openstack-trove07:40
*** haomaiwang has joined #openstack-trove07:43
*** haomaiwang has quit IRC07:48
*** pcaruana has joined #openstack-trove07:48
*** haomaiwa_ has joined #openstack-trove07:53
*** sgotliv has joined #openstack-trove07:53
*** haomaiwa_ has quit IRC08:01
*** haomaiwang has joined #openstack-trove08:01
*** haypo has joined #openstack-trove08:18
haypoamrith: hello08:18
*** mingdang1 has quit IRC08:21
*** haomaiwang has quit IRC09:01
*** haomaiwang has joined #openstack-trove09:01
*** tosky has joined #openstack-trove09:06
*** barclaac has quit IRC09:11
*** barclaac has joined #openstack-trove09:14
*** lifeless has quit IRC09:39
*** lifeless has joined #openstack-trove09:40
*** haomaiwang has quit IRC10:01
*** haomaiwang has joined #openstack-trove10:01
*** cheneydc has quit IRC10:02
*** tqtran has joined #openstack-trove10:15
*** tqtran has quit IRC10:20
*** flaper87 has quit IRC10:36
*** flaper87 has joined #openstack-trove10:36
*** haomaiwang has quit IRC11:01
*** haomaiwang has joined #openstack-trove11:01
*** tellesnobrega is now known as tellesnobrega_af11:05
*** Ravikiran_K has quit IRC11:17
*** flwang has joined #openstack-trove11:25
*** mingdang1 has joined #openstack-trove11:33
amrithhaypo, hello11:44
haypoamrith: hey. i would like to talk about https://review.openstack.org/#/c/297888/11:47
haypoamrith: i don't think that it's a good idea to have two versions of the code, one for py2, one for py311:52
haypoamrith: do you expect regressions from my change? maybe i should explain better the effect of the change on py2?11:53
haypo(already done in comments, but maybe i should enhance the commit message)11:53
amrithone second haypo, timeslicing a bit here.11:53
*** haomaiwang has quit IRC12:01
*** pmackinn has joined #openstack-trove12:01
amrithhi haypo, I'm back12:01
*** haomaiwa_ has joined #openstack-trove12:01
amrithI read your comments.12:01
amrithhere is my concern, and it may be the case that your change does not cause this problem, but I want to be sure.12:02
amrithassume that in one version of code, we have functions encrypt() and decrypt()12:02
amrithand encrypt() was used to store some data which is at rest.12:02
amrithsuppose you fix encrypt() and decrypt() in some way, after upgrading, you should still be able to decrypt() all the old data.12:03
amrithmy concern is that your change(s) are in that realm, and I'd like to see tests that can ensure that this doesn't introduce a regression; data becoming unecryptable.12:03
amrithsorry, un-decryptable.12:03
amrithhaypo ^^ does this question make sense or am I missing something very basic?12:05
haypoamrith: they are already unit tests, i don't think that it's worth to add new tests12:09
haypoamrith: IMHO my change adds a new feature on py2, unicode is now supported12:09
amrithbut the unit tests only test code in either the old version or the new version.12:09
haypoamrith: if you want new tests, they should be on unicode strings, so check that unicode is encoded to utf-8 as expected12:10
haypoamrith: i don't understand what do you mean by "old" and "new" version. my change only adds something like:12:11
haypoif isinstance(data, unicode):12:11
haypo  data = data.encode('utf-8')12:11
haypoif you pass str (bytes), it does nothing12:12
haypo(i'm talking about python 2 here)12:12
amrithhaypo, by old version I mean code before your fix and by new version I mean code with your fix.12:16
amriththese functions are used to encrypt data and then store them in files, or the database.12:16
amriththerefore data could be encrypted using the current code and later attempt to be decrypted using the new code.12:16
amrithif these functions don't function identically, you are missing an upgrade step here ...12:16
haypoamrith: my change doesn't break backward compatibility12:16
amrithplease tell me what you mean by backward compatibility.12:17
haypoamrith: if you pass bytes, it's a no-op12:17
haypoamrith: i mean that data encrypted by the old version is decryptable by the new version (for example)12:17
haypoamrith: data encrypted by the new version can be decrypted by the old version12:18
haypoetc.12:18
haypoamrith: the real question is if trove sometimes pass Unicode strings with non-ASCII characters12:18
haypoi don't think so, since you will likely get an unicode error in this case with the old version ;)12:18
amrithgiven where these routines may have been used, it is possible but I'm not sure.12:18
openstackgerritamrith proposed openstack/trove: Address change in Route v2.3  https://review.openstack.org/29841912:19
amrithhaypo, to be clear. I'm not able to give you a concrete example of "here, this is something that will break". hence the concern on my part.12:20
amrithI don't understand all the possible ways in which crypto_utils has been used, nor the complete import of your change. so let others weigh in as well?12:20
haypowith python 2:12:21
haypo>>> trove.common.crypto_utils.encrypt_data(u'data', u'key')12:21
haypo'oMfvEWglTIacQ4ql\x06\xba\xa1?\x0e\xec<:\xdc\xad\x9d\xf9\x05\xdc\xea\x82'12:21
haypowith the current code, you *can*  pass unicode string if they are encodable to ASCII12:21
haypo>>> trove.common.crypto_utils.encrypt_data(u'data', u'key\xff')12:21
haypoUnicodeEncodeError: 'ascii' codec can't encode character u'\xff' in position 3: ordinal not in range(128)12:21
haypobut you *cannot12:21
haypobut you *cannot* pass Unicode string with non-ASCII character12:21
haypoamrith: it confirms my guess, my code has *no* effect on python 2 :-) it only adds a new feature. you are now able to encode arbitrary Unicode on python 2, it's first encoded to UTF-812:22
haypoamrith: you can decode such string using the old code, as soon as you accepted UTF-8 encoded byte string12:22
haypoamrith: currently, Unicode is encoded to ASCII. But UTF-8 is a superset of ASCII. Data encoded to DATA is decodable from UTF-812:23
haypooops12:23
haypoData encoded to ASCII is decodable from UTF-812:23
amrithI take your word for it, let's see what other reviewers feel, shall we? I'll also spend some time working with your new code to understand better.12:23
haypoamrith: unicode is an hard problem. i know that very well. i fixed unicode issues in python 3 (the interpreter, C code, stdlib, etc.) during several years :-) (i'm a python core developer)12:24
haypoamrith: i even wrote http://unicodebook.readthedocs.org/ /o\12:24
amrithThat's very good, and reassuring to know. thanks.12:25
openstackgerritVictor Stinner proposed openstack/trove: Port crypto_utils to Python 3  https://review.openstack.org/29788812:30
haypoamrith: ^^ i completed the commit message to try to reasure you :)12:31
haypo"I'd like to see tests that can ensure that this doesn't introduce a regression; data becoming unecryptable." which kinds of test do you expect?12:32
haypoamrith: ^^12:32
haypoamrith: do you expect a test with encrypted data + the key and ensure that you can still decode data?12:33
haypoamrith: right now i only see tests like data == decrypt(encrypt(data))12:33
amrithhaypo, I'm looking for some way to mitigate the concern (either tests or code inspection) that existing encrypted data doesn't become undecryptable12:35
haypoamrith: i can add new tests if you want, but i would like to understand what you expect first :=)12:35
amrithhaypo, we're looking for you in #openstack-release :)12:37
amrithhaypo, please don't add anything at this stage. let's see what other reviewers feel. I will go with their counsel.12:39
*** Ravikiran_K has joined #openstack-trove12:49
*** pmackinn has quit IRC12:59
*** haomaiwa_ has quit IRC13:01
*** haomaiwang has joined #openstack-trove13:01
*** pmackinn has joined #openstack-trove13:02
*** links has quit IRC13:02
haypoamrith: i don't understand why Random.new() data is encoded by encode_data(), AES accepts arbitrary binary data13:19
haypoamrith: i fear that encode_data() reduces the quality of the IV (less random)13:19
*** haomaiwang has quit IRC13:22
*** amrith is now known as _amrith_13:35
haypohum, there is also a major performance issue in _find_mock_refs() of trove/tests/unittests/trove_testtools.py13:43
*** mingdang1 has quit IRC13:45
openstackgerritVictor Stinner proposed openstack/trove: Port crypto_utils to Python 3  https://review.openstack.org/29788813:46
*** haypo has left #openstack-trove13:50
*** sgotliv has quit IRC14:03
*** tqtran has joined #openstack-trove14:17
*** tellesnobrega_af is now known as tellesnobrega14:19
*** sgotliv has joined #openstack-trove14:20
*** tqtran has quit IRC14:21
*** _amrith_ is now known as amrith14:24
cp16netflaper87: yt?14:31
flaper87cp16net: yes14:31
*** dims has joined #openstack-trove14:32
cp16netmind seeing if you approve this for stable/mitaka? https://review.openstack.org/#/c/297564/14:32
cp16netflaper87: i'd like to get this merged so i can cut the RC2 for trove-dashboard today14:35
flaper87cp16net: looking into it!14:36
cp16netit looks like the final patch we need. thanks! :)14:36
*** boden has joined #openstack-trove14:36
*** haomaiwang has joined #openstack-trove14:43
flaper87cp16net: done14:52
flaper87:)14:52
*** tosky has quit IRC14:53
*** boden has left #openstack-trove14:57
*** tosky has joined #openstack-trove14:57
cp16netflaper87: thanks!15:00
flaper87anytime15:00
*** haomaiwang has quit IRC15:01
*** haomaiwang has joined #openstack-trove15:01
*** Ravikiran_K has quit IRC15:04
*** openstackgerrit has quit IRC15:06
*** openstackgerrit has joined #openstack-trove15:06
*** tellesnobrega is now known as tellesnobrega_af15:12
cp16netalright trove-dashboard rc2 is in the pipeline to be released15:22
cp16netzigo: ^^ your change is in there now15:22
*** rcernin has quit IRC15:27
*** links has joined #openstack-trove15:32
*** haomaiwang has quit IRC16:01
*** haomaiwang has joined #openstack-trove16:01
*** Guest58782 has quit IRC16:03
*** links has quit IRC16:04
openstackgerritamrith proposed openstack/trove: Trove's tox tests should respect upper-constraints.txt  https://review.openstack.org/29885016:09
*** tellesnobrega_af is now known as tellesnobrega16:16
*** tqtran has joined #openstack-trove16:30
*** haomaiwang has quit IRC16:32
*** haomaiwa_ has joined #openstack-trove16:34
*** haomaiwa_ has quit IRC16:37
*** haomaiwang has joined #openstack-trove16:38
*** Ravikiran_K has joined #openstack-trove16:41
*** haomaiwang has quit IRC16:42
*** pcaruana has quit IRC16:48
*** sgotliv has quit IRC16:59
*** zhiyan has quit IRC17:07
*** pmalik has quit IRC17:07
*** shakamunyi has quit IRC17:07
*** eglute has quit IRC17:07
*** vkmc has quit IRC17:07
*** tosky has quit IRC17:07
*** mfisch has quit IRC17:07
*** eglute has joined #openstack-trove17:07
*** barra204 has joined #openstack-trove17:07
*** pmalik_ has joined #openstack-trove17:07
*** tosky_ has joined #openstack-trove17:07
*** mfisch has joined #openstack-trove17:07
*** mfisch is now known as Guest2804917:08
*** vkmc has joined #openstack-trove17:09
openstackgerritamrith proposed openstack/trove: Trove's tox tests should respect upper-constraints.txt  https://review.openstack.org/29885017:12
*** zhiyan has joined #openstack-trove17:17
*** amrith is now known as _amrith_17:29
*** arist has quit IRC17:39
*** arist has joined #openstack-trove17:40
*** tosky_ has quit IRC17:53
*** pcaruana has joined #openstack-trove18:02
*** _amrith_ is now known as amrith18:19
*** david-lyle has quit IRC18:45
*** david-lyle has joined #openstack-trove18:46
openstackgerritPeter Stachowski proposed openstack/python-troveclient: Fixed output of cluster-create in CLI  https://review.openstack.org/29893218:47
*** Guest28049 is now known as mfisch19:01
*** mfisch is now known as Guest8245419:01
*** Guest82454 has quit IRC19:02
*** Guest82454 has joined #openstack-trove19:02
*** Guest82454 is now known as mfisch19:03
*** krotscheck is now known as krotscheck_dcm19:07
*** haypo has joined #openstack-trove19:19
haypopeterstac: hi. i saw your useful review on https://review.openstack.org/#/c/297888/ thanks19:21
haypopeterstac: i replied with new questions :)19:21
*** dalees` has quit IRC19:25
peterstachaypo, ok, I'll take a look :)19:33
haypopeterstac: it would be nice to start to have a working py3 gate, so focus on the first change19:34
haypopeterstac: do you have anything against https://review.openstack.org/#/c/279098/3 ? :)19:34
openstackgerritVictor Stinner proposed openstack/trove: Port more common unit tests to Python 3  https://review.openstack.org/29895219:35
peterstacYeah, I think I looked at that a while back (around the time we decided to push it into Newton)19:35
peterstacI think it's fine19:35
peterstac(I noticed the crypto_utils one because I worked on that, and I remember how difficult it was to get it right)19:36
haypopeterstac: i discussed recently with amrith who is ok to start for newton. my plan is my blueprint linked in my changes, https://blueprints.launchpad.net/trove/+spec/trove-python319:36
haypoah, it looks like you have a mitaka branch, so we can start to work on py3 ;)19:36
peterstacyep19:37
haypopeterstac: by the way, i think that encode_data() on the key is useless and reduces the entropy of the key19:37
haypoi should propose a separated change for that19:37
peterstacprobably correct - I can't remember why I added that at this point though19:37
peterstac(I hope I had a reason ;) )19:37
haypo(i don't understand neither why data is encoded *twice* to base64, but that's a different topic, it's not easy to change this part, and it doesn't affect security)19:37
*** dalees` has joined #openstack-trove19:38
peterstacbasically the scenario is that data is taken from a file (or passed into the python API) and needs to be stored in the db19:39
peterstacsince it has to be sent over the wire, it's b64 encoded to facilitate that19:39
peterstaconce it's encrypted it's encoded again to store it into the db19:40
haypopeterstac: yeah, i now that base64 solves many issues ;)19:40
haypopeterstac: but it's encoded twice to base64, i don't understand why :-p19:40
peterstac(mainly because Trove doesn't have code support for blobs, and I didn't have time to sort that out ;) - but it does support Text)19:40
haypoAES accepts arbitrary binary data as input19:40
peterstacsure, but then I'd have to decode it first (since it comes across the wire as b64), then encrypt it, then re-encode it for the db19:43
peterstacI just decided to skip the first decode19:43
peterstacor maybe you're thinking of something else - what part of the code are you referring to?19:43
*** dims has left #openstack-trove19:45
*** sgotliv has joined #openstack-trove19:48
haypopeterstac: according to unit tests, it looks like trove uses: data_for_db = base64(encrypt(base64(data_from_file), key))19:48
haypopeterstac: my question is, why not using directly: data_for_db = base64(encrypt(data_from_file, key))19:49
peterstacbecause the data_from_file is sent over the wire to the server19:50
peterstacand the client doesn't have the key to encrypt the data19:50
peterstac(first)19:50
peterstacsince it was already encoded, I chose to leave it that way19:51
haypo"data_from_file is sent over the wire to the server" hum ok19:51
peterstacor in other words, the client python API needs to convert it to a format that can be sent to the trove ReST server19:52
peterstacmaybe there's a better way, but that's what I came up with :)19:52
haypoin a previous job, i recall that we used something like wire_data = to_xml(base64(data)). worst serialization ever :) but the whole communication was based on XML-RPC, and we failed to find how to serialize BLOB using the <![CDATA[ thing19:53
haypopeterstac: when i read the code for the first time, it wasn't obvious that data already comes encoded19:53
peterstacYeah, I should probably have added a few more comments :)19:55
*** ediardo has joined #openstack-trove19:57
*** Ravikiran_K has quit IRC19:59
haypopeterstac: do you know if data coming from the client comes as text or binary on python3?19:59
peterstacfrom the CLI, it'll read the file as binary, and convert it to b64/utf-8 encoded20:00
peterstacbut from the python API, it could be a string also, so it also runs through the same convertion20:01
peterstacso the ReST server should always receive b64/utf-8 encoded data20:01
haypopeterstac: but then it goes through oslo messaging, i guess? i would like to know if you expect bytes or text (unicode) at the server side20:02
haypoIMHO we should accept bytes & unicode and simply encode unicode to utf-820:02
haypoyou may want to encode to ASCII to be pedantic, it's just that utf-8 is simpler (ex: there is the to_utf8() function in oslo.utils)20:03
peterstachow do you accept bytes from a ReST call?20:03
peterstacI just assumed it had to be encoded to fit into the data payload20:04
peterstacIf you're speaking of the client, that's essentially what it does20:07
peterstacour client does have py34 tests so I had to jump through a lot of hoops already to get that to work20:07
peterstac(which is why the Base64Codec looks the way it does)20:08
openstackgerritamrith proposed openstack/trove: Trove's tox tests should respect upper-constraints.txt  https://review.openstack.org/29885020:17
amrithcp16net, peterstac, vkmc, johnma, ... we need to merge https://review.openstack.org/298984 and https://review.openstack.org/#/c/298858/ for mitaka before thursday so we can cut an RC2.21:15
amriththe corresponding changes have worked on master21:15
vkmcyep, checking out21:16
amriththanks. the change to g-r won't be till after mitaka is released.21:16
amrithsee also the neutron solution ... https://review.openstack.org/#/c/298855/121:17
amrithlooks familiar?21:17
amrithI don't see the need to fix mappers; I've posted a comment on the neutron review.21:22
johnmawill look at amrith21:26
vkmcwhy there is a new change?21:29
vkmchttps://review.openstack.org/#/c/298984/1 == https://review.openstack.org/#/c/298419/1 right?21:29
vkmchttps://review.openstack.org/#/c/298984/ looks good21:32
*** cweid has joined #openstack-trove21:33
openstackgerritPeter Stachowski proposed openstack/trove-specs: Replication/cluster locality  https://review.openstack.org/29899421:39
*** cweid has quit IRC22:08
*** sgotliv has quit IRC22:35
*** haypo has left #openstack-trove22:43
*** mingdang1 has joined #openstack-trove23:26
*** mingdang1 has quit IRC23:40

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!