Monday, 2015-07-27

*** chlong has joined #openstack-swift00:00
*** hrou has joined #openstack-swift00:12
*** kota_ has joined #openstack-swift01:00
*** ChanServ sets mode: +v kota_01:00
kota_good morning01:01
hokota_: good morning!01:03
kota_ho: yeah, how's going?01:04
hokota_: i'm exhausted. too too too hot :-)01:05
kota_ho: ya, it's gettting hotter in Japan.01:08
*** wbhuber has joined #openstack-swift01:10
*** Kennan has quit IRC01:13
*** Kennan has joined #openstack-swift01:16
*** jasondotstar has joined #openstack-swift01:26
*** wbhuber_ has joined #openstack-swift01:31
*** wbhuber has quit IRC01:34
*** wbhuber_ has quit IRC01:48
jrichlimahatic: there has been a merge from master.  start now by picking up the latest patch on https://review.openstack.org/#/c/203454/02:08
jrichliyou won't have to rebase what is there now02:09
*** wbhuber has joined #openstack-swift02:28
*** jrichli has quit IRC02:48
notmynamehello, world02:50
*** wbhuber has quit IRC02:50
honotmyname: good evening!02:50
notmynamehow are things?02:55
hogreat! except the weather (too hot for me) :-)02:57
*** tsubic has quit IRC02:59
notmynameI went to seattle today (where I am now). similar temperature as home, but it's been raining lightly. very nice :-)03:02
notmynameho: what's the weather going to be in tokyo at the summit?03:06
*** kcaj has quit IRC03:07
*** kcaj has joined #openstack-swift03:14
kota_it could be good weather in October in Tokyo.03:17
kota_perhaps a bit colder than bay area.03:17
kota_but almost of days in Octover will be sunny.03:18
*** changbl has quit IRC03:19
*** jasondotstar has quit IRC03:20
notmynamenice!03:22
*** sanchitmalhotra has joined #openstack-swift03:24
*** sanchitmalhotra1 has joined #openstack-swift03:31
*** changbl has joined #openstack-swift03:32
*** sanchitmalhotra has quit IRC03:33
*** sanchitmalhotra has joined #openstack-swift03:38
*** sanchitmalhotra1 has quit IRC03:40
hoi was out for lunch. yeah, +1 for weather in october in tokyo03:50
*** zul has quit IRC03:52
*** zul has joined #openstack-swift03:53
*** nexusz99 has joined #openstack-swift04:00
*** haomaiwang has quit IRC04:02
*** haomaiwang has joined #openstack-swift04:04
*** proteusguy has quit IRC04:06
hoi didn't check all yet but meta data search has many entries. https://www.openstack.org/summit/tokyo-2015/vote-for-speakers/SearchForm04:07
*** hrou has quit IRC04:08
*** ppai has joined #openstack-swift04:13
*** proteusguy has joined #openstack-swift04:18
*** Kennan2 has joined #openstack-swift04:20
*** Kennan has quit IRC04:20
*** DericHorn-HP has joined #openstack-swift04:26
notmynameI'm going back through old local branches I have in my swift tree04:29
notmynameso far it seems like most are just stuff from reviews04:29
notmynamebut I did just fine one from Oct 2012 that I had never pushed up04:30
*** ho_ has joined #openstack-swift04:38
*** marzif has joined #openstack-swift04:41
*** ho_ has left #openstack-swift04:51
*** jome has joined #openstack-swift04:59
*** marzif has quit IRC05:11
*** sanchitmalhotra1 has joined #openstack-swift05:25
*** sanchitmalhotra has quit IRC05:27
*** nakagawamsa has joined #openstack-swift05:29
*** DericHorn-HP has quit IRC05:35
*** sanchitmalhotra has joined #openstack-swift05:55
*** sanchitmalhotra1 has quit IRC05:57
*** fifieldt has joined #openstack-swift06:00
*** sanchitmalhotra1 has joined #openstack-swift06:22
*** sanchitmalhotra has quit IRC06:24
*** ppai has quit IRC06:59
*** ppai has joined #openstack-swift07:09
hoacoles: around? are you back?07:22
*** chlong has quit IRC07:28
*** rledisez has joined #openstack-swift07:41
*** fifieldt has quit IRC07:42
*** geaaru has joined #openstack-swift07:43
*** jistr has joined #openstack-swift07:44
*** jome has quit IRC07:52
*** jome has joined #openstack-swift07:52
*** jome_ has joined #openstack-swift08:01
*** jome has quit IRC08:02
*** jome_ has quit IRC08:08
*** haypo has joined #openstack-swift08:16
*** jome has joined #openstack-swift08:19
*** jome_ has joined #openstack-swift08:22
*** silor has joined #openstack-swift08:24
*** jome has quit IRC08:24
hotoday i have a party so i will leave now. have a nice day!08:27
*** ho has quit IRC08:30
*** aix has joined #openstack-swift08:31
*** silor1 has joined #openstack-swift08:44
*** silor has quit IRC08:46
*** silor1 is now known as silor08:46
*** wbhuber has joined #openstack-swift08:46
*** wbhuber has quit IRC08:50
*** jome_ has quit IRC08:59
*** eandersson has joined #openstack-swift09:00
*** sanchitmalhotra has joined #openstack-swift09:21
openstackgerritVictor Stinner proposed openstack/swift: Fix warnings of hacking 0.10.2  https://review.openstack.org/20596409:22
*** sanchitmalhotra1 has quit IRC09:23
openstackgerritVictor Stinner proposed openstack/swift: Don't format using self.__dict__  https://review.openstack.org/20596609:26
*** jordanP has joined #openstack-swift09:32
*** jasondotstar has joined #openstack-swift09:32
*** sanchitmalhotra1 has joined #openstack-swift09:37
*** sanchitmalhotra has quit IRC09:39
*** jome has joined #openstack-swift09:40
openstackgerritVictor Stinner proposed openstack/swift: Fix warnings of hacking 0.10.2  https://review.openstack.org/20596409:42
openstackgerritVictor Stinner proposed openstack/swift: Update hacking to 0.10.0 or newer  https://review.openstack.org/20597709:42
*** jome has quit IRC09:46
*** jome has joined #openstack-swift09:49
*** jome has quit IRC09:54
*** jome has joined #openstack-swift09:56
*** jome has quit IRC10:00
*** jome has joined #openstack-swift10:02
*** jome has quit IRC10:02
*** kota_ has quit IRC10:04
*** jome has joined #openstack-swift10:09
*** ppai has quit IRC10:12
*** jome_ has joined #openstack-swift10:18
*** jome_ has quit IRC10:20
*** cppforlife__ has joined #openstack-swift10:21
*** jome has quit IRC10:21
*** cppforlife_ has quit IRC10:21
*** acoles has quit IRC10:21
*** blair has quit IRC10:21
*** tdasilva has quit IRC10:21
*** cppforlife__ is now known as cppforlife_10:22
*** jome has joined #openstack-swift10:23
*** ppai has joined #openstack-swift10:24
jith_hi all, how to upload data with more than 5GB10:33
jith_in swift10:33
nexusz99@jith_  you can use multipart upload (SLO, DLO)10:34
jith_nexusz99: Thanks... can u suggest some documentation for that10:35
nexusz99jith_,  see http://docs.openstack.org/developer/swift/overview_large_objects.html10:35
jith_nexusz99: thanks..  i was seeing that... i am not uploading via swift client but through API... means i have configured swift as backend for glance... so when i upload images in glance, swift throws an error like "Request entity too large". So i should change in some conf files??10:38
nexusz99jith_,  you mean api is the swift REST API?10:40
jith_yes i think so.. i am a beginner in this... if i upload images less than 5 GB its working fine.. but throwing error if it exceds 5gb...10:42
nexusz99jith_,  then  you have to choice which middleware will you use to multipart upload. swift support 2 types of middleware (SLO , DLO)10:44
*** kei_yama has quit IRC10:44
nexusz99jith_,  if you want to use dlo middleware, see this documents (http://docs.openstack.org/developer/swift/overview_large_objects.html#direct-api)10:45
nexusz99jith_,  fix your client code when upload object exceed 5GB splite the object into segment( min size 1MB) and upload segments.10:46
nexusz99jith_,  when upload segments finished, create manifest file with 'X-Object-Manifest'.10:50
jith_nexusz99: Thanks... but i dont have client code... i am using glance client with swift options10:52
jith_in glance api.conf i stored swift credentials and regarding the size limits i could see only two parameters in that..  they are  What size, in MB, should Glance start chunking image files10:53
jith_# and do a large object manifest in Swift? By default, this is10:53
jith_# the maximum object size in Swift, which is 5GB10:53
jith_swift_store_large_object_size = 512010:53
jith_# When doing a large object manifest, what size, in MB, should10:53
jith_# Glance write chunks to Swift? This amount of data is written10:53
jith_# to a temporary disk buffer during the process of chunking10:53
jith_# the image file, and the default is 200MB10:53
jith_swift_store_large_object_chunk_size = 20010:53
nexusz99jith_,  could you show your swift proxy-server's pipeline in config file?10:54
*** sanchitmalhotra has joined #openstack-swift10:55
*** sanchitmalhotra1 has quit IRC10:57
jith_sure11:05
jith_nexusz99: http://paste.openstack.org/show/405908/11:07
*** ppai has quit IRC11:08
jith_nexusz99: Sorry for late reply... thanks for guiding11:09
nexusz99jith_,  hmm..   i have to research about glance. i think glance does not upload object by multipart ..11:10
jith_nexusz99: Thanks no error is from swift side... if possible check the log.. http://paste.openstack.org/show/405909/11:13
*** acoles_ has joined #openstack-swift11:13
*** ChanServ sets mode: +v acoles_11:13
*** acoles has joined #openstack-swift11:17
*** ChanServ sets mode: +v acoles11:17
*** tdasilva has joined #openstack-swift11:18
*** acoles- has joined #openstack-swift11:19
*** acoles has quit IRC11:19
*** acoles- is now known as acoles11:19
*** ChanServ sets mode: +v acoles11:19
*** ppai has joined #openstack-swift11:20
*** acoles_ has quit IRC11:20
*** jome has quit IRC11:21
*** marzif has joined #openstack-swift11:31
*** blair has joined #openstack-swift11:33
*** sanchitmalhotra has quit IRC11:41
*** nexusz99 has quit IRC11:42
*** nexusz99 has joined #openstack-swift11:43
*** marcusvrn has joined #openstack-swift12:02
*** ppai has quit IRC12:05
openstackgerritVictor Stinner proposed openstack/swift: Replace deprecated assert_()  https://review.openstack.org/20603112:09
*** km has quit IRC12:10
*** jasondotstar has quit IRC12:10
*** thurloat_isgone is now known as thurloat12:18
*** jome has joined #openstack-swift12:18
*** ppai has joined #openstack-swift12:20
*** chlong has joined #openstack-swift12:30
*** jasondotstar has joined #openstack-swift12:32
*** vinsh_ has quit IRC12:38
*** hrou has joined #openstack-swift12:45
*** ppai has quit IRC12:47
openstackgerritVictor Stinner proposed openstack/swift: Update hacking to 0.10.0 or newer  https://review.openstack.org/20597712:54
openstackgerritVictor Stinner proposed openstack/swift: Fix warnings of hacking 0.10.2  https://review.openstack.org/20596412:54
openstackgerritVictor Stinner proposed openstack/swift: Don't format using self.__dict__  https://review.openstack.org/20596612:54
*** annegentle has joined #openstack-swift12:54
*** silor has quit IRC12:55
openstackgerritjanonymous proposed openstack/swift: Python3: do not use im_self/im_func/func_closure  https://review.openstack.org/20306612:59
*** ppai has joined #openstack-swift12:59
*** _hrou_ has joined #openstack-swift13:00
*** hrou has quit IRC13:01
*** tongli has joined #openstack-swift13:08
*** annegentle has quit IRC13:10
*** annegentle has joined #openstack-swift13:11
*** janonymous_ has joined #openstack-swift13:19
*** charlesw has joined #openstack-swift13:21
janonymous_Hi Guys, it has not been a long time since i have been a part of this community but i have always tried to do as much as i can, with this effort only i have submitted a small talk which might get approved with your votes: https://www.openstack.org/summit/tokyo-2015/vote-for-speakers/presentation/4570  ; Kindly vote if you feel if it's worth giving a chance  ;)13:24
*** ppai has quit IRC13:30
*** annegent_ has joined #openstack-swift13:38
*** annegentle has quit IRC13:41
*** chlong has quit IRC13:42
*** ppai has joined #openstack-swift13:45
*** jasondotstar has quit IRC13:48
*** jistr has quit IRC13:52
*** blmartin has joined #openstack-swift13:53
*** jistr has joined #openstack-swift13:54
*** jrichli has joined #openstack-swift13:54
*** janonymous_ has quit IRC13:57
*** nexusz99 has quit IRC14:07
*** jome_ has joined #openstack-swift14:10
*** jome has quit IRC14:10
*** jome_ has quit IRC14:10
*** jome has joined #openstack-swift14:13
*** wbhuber has joined #openstack-swift14:17
*** jlhinson has joined #openstack-swift14:18
jrichlimahatic: I will be offline this morning and the earlier part of tomorrow morning.14:19
*** jasondotstar has joined #openstack-swift14:28
*** changbl has quit IRC14:32
*** vince_ has joined #openstack-swift14:51
vince_hi guys, is there a way to post dynamic large objects from the python APIs of python-swiftclient?14:53
*** annegent_ has quit IRC14:54
vince_it's possible from the command line with the -S option, to split the object into segments and get the manifest automatically created, but I'd like a Python equivalent14:54
*** minwoob has joined #openstack-swift14:58
*** pgbridge has joined #openstack-swift14:58
openstackgerritNicolas Trangez proposed openstack/swift: Use entrypoints for storage policy implementation lookups  https://review.openstack.org/20610515:02
eikkenotmyname: some time ago we talked about this ^^ I went ahead and created a patch15:02
*** charlesw has quit IRC15:06
*** nadeem has joined #openstack-swift15:17
*** aix has quit IRC15:22
*** silor has joined #openstack-swift15:23
*** acoles is now known as acoles_away15:24
*** ppai has quit IRC15:24
*** acoles_away is now known as acoles15:25
*** early` has quit IRC15:29
acolesho: i'm back. catch you tomorrow maybe.15:29
peluseacoles, hey how are ya?15:30
jrichliacoles!  How was your vaca?15:32
*** early has joined #openstack-swift15:32
*** wer has quit IRC15:32
jrichliI didn't have to be offline as long as I thought I would15:32
acolespeluse: jrichli hi guys! vacation was great thanks15:32
jrichligood to hear it.  And good to have you back!  I guess you have seen that there is an etherpad with encryption issues?15:33
acolesjrichli: not seen that yet, got a link?15:33
acolesjrichli: still catching up...15:33
jrichliits in the topic, actually15:34
jrichlihttps://etherpad.openstack.org/p/swift_encryption_issues15:35
acolesoic thx15:35
jrichliacoles: mahatic has joined the effort.  She is working on the probetests that you had made trello cards for.15:37
acolesjrichli: nice, thanks mahatic15:37
pelusejrichli, ahh cool.  I'll read it too.  Need to get spun up on the crypto branch myself15:40
jrichlipeluse: sounds great.  would love your input15:41
pelusejrichli, so what's the current status of the feature branch wrt what should work/should fail in terms of unit/func/probe?15:41
*** thurloat is now known as thurloat_isgone15:41
* peluse will BRB15:42
jrichlipeluse: 2 unittests fail (POST as copy and containerSync), functests errors=9, failures=17, probetests errors=2, failures=115:45
notmynamegood morning15:45
notmynameacoles is back!15:45
acolesnotmyname: remind me what we do here? :P15:46
notmynameacoles: I think we mostly roll our face against the keyboard until clayg stops griping at us. ;-)15:47
jrichlilol15:49
notmynameeikke: interesting. that will be worth a look. thanks15:49
acolesnotmyname: lol!15:49
eikkenotmyname: I'll experiment with our diskfile impl to find out whether it's as useful as I expect it to be :-p15:49
notmynameeikke: I had originally been thinking about diskfile implementations instead of "Storage policies" as the pluggable piece.15:50
notmynameeikke: ah, that'd be great15:50
eikkenotmyname: well, I guess it makes sense to have a 'storage policy' define the underlying diskfile being used, otherwise there's some other strange indirection15:50
*** thurloat_isgone is now known as thurloat15:51
notmynameit depends :-)15:51
eikkenotmyname: in the end, diskfiles define which policy types they support, or something along those lines. not sure how the lines should be drawn here15:51
notmynamehmm15:51
eikkebut as noted in the commit message, having the split-in-3 (storage policy, proxy obj controller and diskfilemanager) makes not much sense15:52
eikkeit's unclear to me how tightly (or configurable) coupling should be15:52
eikkewhich combos make sense and which dont15:53
eikkein the current swift codebase it's fairly coupled: [repl SP, repl objcontroller, repl diskfile] or [ec SP, ec objcontroller, ec diskfile]15:54
eikkealthough I could imagine a PACO/in-process objcontroller to be something untied to SP or diskfile15:54
notmynameyeah. torgomatic and clayg and I have talked about it some.15:58
notmynameon the one hand, EC diskfile vs replication diskfile is important because they do different things. on the other, when it comes down to just talking to storage media, it shouldn't matter what cluster durability scheme is used15:58
pelusejrichli, thanks.  I see the same errors so guess we're ready to ship it eh?  :)15:59
jrichlipeluse: guess so!15:59
*** rledisez has quit IRC16:00
* notmyname is hearing that encryption is done. good job, team ;-)16:00
*** mgarza_ has joined #openstack-swift16:00
pelusewho is buying t-shirts?16:01
jrichliLOL.  only if you don't want to have a handful of features - you know, like COPY.  and you want some middleware with FAKE in the name16:01
*** wer has joined #openstack-swift16:01
jrichlipeluse: IBM.  You will get them at Hackathon :-)16:02
pelusedamn, so we have an entire two weeeks to get things done.  that's awesome :)  Feels like policies all over again!16:03
*** vinsh has joined #openstack-swift16:06
*** thurloat is now known as thurloat_isgone16:07
*** acoles is now known as acoles_away16:07
*** vince_ has quit IRC16:08
jrichlisince acoles already left, I guess I will go for an early lunch ;-)16:08
* peluse thinks maybe an early beer sounds good now for some reason. must be Monday...16:09
*** thurloat_isgone is now known as thurloat16:16
*** mragupat has joined #openstack-swift16:18
claygacoles_away: !16:30
*** acoles_away is now known as acoles16:30
claygnotmyname: last year I clicked through every openstack summit to vote - it was terrible "16:31
acolesclayg: !16:31
peluseclayg, yeah, I was wondering if it was me or is it really a PITA to go through all of them (even jsut storage)??16:31
notmynamewow, there's a lot https://swiftstack.com/blog/2015/07/27/cast-your-vote-openstack-tokyo-summit-2015-presentations-on-swift/16:32
claygnotmyname: yeah I think that's only the ones mario and his his team picked16:33
claygthere's probably tons more16:33
clayg*we're* doing a lot16:33
notmynamethere's more at the bottom16:33
claygwell - i guess we'll *see* how many we'll do16:33
claygnotmyname: yeah but that's not all "storage" - they've dont some filtering16:33
notmynameoh, yeah. I keep forgetting. storage != just swift ;-)16:33
pelusewhat??16:34
clayglol16:34
*** jasondotstar has quit IRC16:34
hayponotmyname: hi. for my change "Update hacking to 0.10.0 or newer", i now understand the consequence of my change on tox.ini: i disabled *all* Hxxx warnings, whereas before 6 warnings were enabled16:36
hayponotmyname: i can only blacklist Hxxx warnings which were disabled if you prefer16:36
hayponotmyname: the patches related to coding style are quite huge, so they are likely to conflict16:37
hayponotmyname: i'm not sure that i will want to spend too much time on them if you are not ready to review them16:37
notmynamehaypo: we settled on the current way as an explicit whitelist of stuff we wanted to test on. that way as stuff is added, it won't suddenly cause errors16:37
hayponotmyname: so what's the opinion on them?16:37
*** janonymous_ has joined #openstack-swift16:39
notmynameI want to keep testing the ones that are explicitly called out. if there are others that we should consider adding, let's discuss it16:39
notmynamedoes the hacking version update still allow that? having just a whitelist of stuff that is checked?16:39
*** thurloat is now known as thurloat_isgone16:40
*** jasondotstar has joined #openstack-swift16:40
hayponotmyname: i don't think that the behaviour of hacking (flake8) changed16:40
notmynameok (good :-)16:40
notmynameso update hacking, if needed, but leave the checks as-is16:40
hayponotmyname: the "ignore =" was already ignored before (but i'm not 100% sure)16:40
notmynameah16:41
hayponotmyname: my bet is more that pep8 & flake8 have more tests16:41
notmynameoh, I know they do. actually, it's hacking that has a lot more checks16:41
haypoanyway, swift cannot hack pep8 or flake8, we have to adopt the new behaviour anyway ;)16:41
notmynamepep8 should still all be checked16:41
notmynameit's the hacking checks we've got a whitelist on16:42
*** geaaru has quit IRC16:42
hayponotmyname: it's amazing how much stuff should be fixed first to enable python3 tests on Swift...16:44
hayponotmyname: dnspython3 (WIP), PyEClib (done!), hacking, what's next? :-)16:44
notmynamehaypo: which is why I'm impressed you and janonymous are sticking with it!16:45
haypohaha16:45
hayponotmyname: it's almost my job to port OpenStack to Python 316:45
hayponotmyname: would you be interested by patches fixing some flake8 warnings, like the one on assert_?16:46
notmynameonly if they can also be elevated to an error so that future me won't re-introduce it. ie it would have to break something obviously16:47
hayponotmyname: i see16:48
*** janonymous_ has quit IRC16:49
*** darrenc_ has joined #openstack-swift16:51
*** janonymous_ has joined #openstack-swift16:52
*** darrenc has quit IRC16:52
openstackgerritVictor Stinner proposed openstack/swift: Update hacking to 0.10.0 or newer  https://review.openstack.org/20597716:52
hayponotmyname: this new change enables _more_ tests than before, and keep existing Hxxx warnings: https://review.openstack.org/#/c/205977/3/tox.ini16:53
hayponotmyname: it's better than ignoring all hackings tests (Hxxx)16:53
*** nexusz99 has joined #openstack-swift16:53
*** jistr has quit IRC16:55
notmynamehaypo: why not stick with the selected list of things to check?16:56
hayponotmyname: it's not possible to use select _and_ ignore16:56
notmynameok, so why not just use select? won't that implicitly ignore everything else?16:56
hayponotmyname: it's a dummy limitation of pep8 (so also of flake8): https://github.com/PyCQA/pep8/blob/d48eef071003952ed0f98445dc3dc6bc29618f9c/pep8.py#L180816:56
hayponotmyname: i want to get the ability to ignore specific tests16:57
notmynamewhy?16:57
hayponotmyname: IHMO "F812: list comprehension redefines ..." warnings are really stupid16:57
notmynamesure. so don't put them in the select line :-)16:57
notmynameassuming that flake8 would actually allow both select and ignore, then the current version of our tox.ini says "ignore everything and then select these few". right?16:58
hayponotmyname: one example: https://github.com/openstack/swift/blob/master/swift/common/middleware/x_profile/html_viewer.py#L41516:58
hayponotmyname: "F812 list comprehension redefines '__' from line 410"16:58
hayponotmyname: the __ variable is replaced... well.. ok, so what? should I use the same "___" or write "del __" ?16:58
notmynameugh. yeah, that's annoying16:58
notmynameah, ok, I think I see16:59
hayponotmyname: the scope of the variable is just fine, it's ok to reuse variables, we don't do SSA in Python (Static single assignment form)16:59
notmynamewe had the select line F, E, W, and the Hxxx warnings17:00
hayponotmyname: with hacking 0.8, we had much less warnings, so it wasn't so much importable to ignore a single check17:00
notmynamewhich means that we can't have select = F and also ignore F81217:00
hayponotmyname: since you use the "select =" line, you cannot use "ignore =" anymore17:00
*** nexusz99 has quit IRC17:00
hayponotmyname: it's annoying17:00
notmynamewhat's the impact of the warning on F812? does it cause flake8 to fail?17:01
hayponotmyname: without ignore = F812, "tox -e pep8" fails17:02
notmynamewow17:02
*** jordanP has quit IRC17:02
notmynameand you're working on updating the hacking version because that's required if you update pbr which is required for py3?17:02
notmyname'cause so far I'm not seeing any advantage ot update hacking ;-)17:03
hayponotmyname: ahaha, that's insane17:03
hayponotmyname: but yes, you rebuilt my puzzle17:03
hayponotmyname: well, hacking helps you to have a better coding style17:03
hayponotmyname: at the end, the code should be more readable17:03
notmynamewhat's the upstream opinion on not being able to both select and ignore with flake8?17:05
notmynamebug or working as designed?17:05
hayponotmyname: i didn't ask17:05
hayponotmyname: for me, it's more a bug, but from flake8 point of view, it can be seen as a new feature :-D17:06
hayponotmyname: i'm not sure that pep8 is designed to use select, but more to use ignore17:06
openstackgerritVictor Stinner proposed openstack/swift: Fix hacking H232 warnings (octal)  https://review.openstack.org/20614917:07
openstackgerritVictor Stinner proposed openstack/swift: Fix hacking H403 warnings (docstrings)  https://review.openstack.org/20615017:07
notmynamethe problem I have with the ignore methodology is that the next time something is added that we don't pass, our gate will fail17:07
notmynamehmm..seems like it's time for our yearly "why are we actually using pbr because it just seems to get in the way" arguments17:09
notmyname;-)17:09
hayponotmyname: i'm checking nova. it also uses "hacking<0.11,>=0.10.0" in test-requirements.txt and "ignore = ..." in tox.ini17:09
notmynamehaypo: IIRC most project do only have a few they select and then ignore the rest17:09
redboyay17:09
notmynameredbo: you must have an alert on "pbr" ;-)17:09
hayponotmyname: nova dev hates other developers when they introduce a regression, so you can trust them that it's the well usage of flake8 :)17:09
hayponotmyname: "most project": ah? which ones?17:10
janonymous_I think https://review.openstack.org/#/c/204093/ will fix H23517:10
*** nexusz99 has joined #openstack-swift17:10
notmynameanecdotal based on my own hallway conversations a while (years) back17:10
haypocinder, glance: same than nova (ignore)17:10
hayponova: same than nova (ignore)17:11
hayponotmyname: i trust hacking developers to *not* break cinder, glance, neutron, nova by adding new random tests just to be pedantic17:11
notmynameyet here we are ;-)17:11
hayponotmyname: that's exactly why my patch uses "hacking>=0.10.0,<0.11"17:12
hayponotmyname: hacking must not add new tests in minor versions (0.10.x), only in major versions (0.11). well, that's what I understood17:12
hayponotmyname: FYI i wrote two patches removing checks from the ignore list, to test this approach (fixing a warning and ensure that it doesn't come back): https://review.openstack.org/206149 & https://review.openstack.org/20615017:13
notmynameI'm sure the hacking devs will do their best to do the right thing. I'm leery of planting a time bomb for ourselves. comes down to explicit vs implicit17:15
notmynameI need to step away for a fe minutes17:16
openstackgerritVictor Stinner proposed openstack/swift: Update hacking to 0.10.0  https://review.openstack.org/20597717:23
openstackgerritVictor Stinner proposed openstack/swift: Fix hacking H232 warnings (octal)  https://review.openstack.org/20614917:26
hayponotmyname: ok. i simplified my patch serie into one unique patch: https://review.openstack.org/#/c/205977/ (+ example: https://review.openstack.org/#/c/206149/)17:27
haypothe idea is then to fix issues reported by flake8 and each time enable one more check in tox.ini (flake8)17:28
hayponotmyname: i have to go, bye!17:28
openstackgerritAlistair Coles proposed openstack/swift: Refactor diskfile  https://review.openstack.org/19842917:28
notmynamehaypo: thanks17:28
* notmyname just got back17:28
*** acoles is now known as acoles_away17:30
notmynamehaypo: yes! that pattern of tacking it I like. get base functionality then add them each one at a time17:30
*** zaitcev has joined #openstack-swift17:31
*** ChanServ sets mode: +v zaitcev17:31
*** ChanServ sets mode: +v tdasilva17:31
*** mgarza_ has quit IRC17:44
*** nakagawamsa has quit IRC17:47
*** NM has joined #openstack-swift17:47
*** NM has quit IRC17:47
*** NM has joined #openstack-swift17:48
claygwell that was an interesting scrollback17:49
notmynameclayg: on hacking?17:51
*** jome_ has joined #openstack-swift17:57
*** annegentle has joined #openstack-swift17:58
*** jome has quit IRC18:01
*** blmartin has quit IRC18:09
*** changbl has joined #openstack-swift18:12
*** annegentle has quit IRC18:15
*** jome has joined #openstack-swift18:17
*** jome_ has quit IRC18:19
*** mahatic has joined #openstack-swift18:24
*** jasondot_ has joined #openstack-swift18:25
portanteAre folks aware of this PEP coming in Python 3.5? https://www.python.org/dev/peps/pep-0475/18:25
*** janonymous_ has quit IRC18:26
portanteretrying system calls in the python stdlib at the lowest levels18:26
notmynameportante: nope18:26
*** annegentle has joined #openstack-swift18:27
*** jasondotstar has quit IRC18:27
hurricanerixnotmyname: I remember seeing a blueprint for proposed /v2 stuff, but can't seem to find it now.  Do you know where it is?18:31
notmynamehurricanerix: auth or api?18:32
hurricanerixapi18:32
notmynamehurricanerix: like a v2 swift api?18:32
notmynameyeah18:32
hurricanerixyeah18:32
*** eandersson has quit IRC18:32
notmynameyeah, where was that?18:32
hurricanerixlol, I know right, I have been looking around for it, but haven't been able to find it.18:32
zaitcevHeh.18:32
zaitcevI suspect there may be dozens.18:33
hurricanerixzaitcev: yeah, I know I saw one, once.  But now when I look, all I find is auth stuff or people asking questions about when swift's version got bumped to 2.* and asking why the URL was still /v118:34
zaitcevI talked about it w/ Yehuda, the RGW leader. He has this idea to create tenantized buckets, which are containers in our case. But he insists that we use '/', e.g. "tenant/container". That offsets everything by a level in Swift.18:34
*** ohno13 has joined #openstack-swift18:34
zaitcevSo I said, "okay, worst come to worst, I'll propose a /v2, then we'll know there's an extra slash"18:35
notmynamethere's been lots of conversations about it. not sure what's been written down, if anything18:35
notmynamemost recent I idea I've heard is to have a middleware that translates stuff back to v1 for older clients18:36
hurricanerixnotmyname: I know I saw at least one thing written down once.  Maybe torgomatic or clayg gave it to me before, but I don't remember for sure who it was.18:36
notmynamehurricanerix: theres https://wiki.openstack.org/wiki/SwiftNextAPI and https://wiki.openstack.org/wiki/Swift/API which look like really old scratch pads18:37
*** proteusguy has quit IRC18:38
hurricanerixnotmyname: the next api one was the one I was looking for.  Thanks18:40
notmynamehurricanerix: should I read anything into your question? ;-)18:40
notmynamehurricanerix: FWIW, I found that linked at the bottom of https://wiki.openstack.org/wiki/Swift18:41
hurricanerixnotmyname: not really, I just had an idea that might fit a /v2 change, and wanted to re-read what existing things were proposed.18:41
hurricanerixAhh, I kept searching for "v2" which is probably why I didn't find it. =)18:41
*** annegentle has quit IRC18:43
hurricanerixnotmyname: I was thinking about how there is some middleware that gets auto enabled in the pipeline because it was migrated from swift to middleware.  And it seemed like maybe the /v2 API might be a good opportunity to remove that restriction so that if someone wanted to setup a vanilla swift cluster with no extra features, they are not forced into it.18:44
notmynamehurricanerix: depends on what you consider "extra" ;-)18:44
claygnotmyname: hurricanerix was wondering if we could make DLO's optional ;)18:45
notmynameyeah. I'm going the other way ;-)18:45
torgomaticso someone wants an extra URL path component of /tenant/ for reasons? boo18:45
torgomaticthat's nonsense when talking about non-Keystone auth systems18:45
notmynamemiddleware is a way to organize code and control flow. it doesn't mean something is optional18:45
hurricanerixTrue, then why require it be configured in the pipeline to begin with?18:46
hurricanerixwell i guess it isn't required18:46
claygnotmyname: i think when defining a v2 api we should have the *option* of not including some v1 features :P18:46
notmynamesure18:47
hurricanerixI was looking at it from the point that the only reason DLOs are required, is because they were written before we put everything in middleware, no?18:47
hurricanerix(or anything that is a feature in swift, and has been migrated to middleware)18:47
notmynameI just have  knee-jerk reaction to middleware==pluggable after trying to correct sales people a dozen times ;-)18:47
clayghurricanerix: i think they're required because of clients mostly :\18:47
hurricanerixyeah, so a client wouldn't have an expectation that they would be supported in /v218:48
hurricanerixand would have to check /info or somthing18:48
clayghurricanerix: +1 I think it's totally debatable what's included in the v2 api feature wise18:48
notmynameor rather that middleware==optional and every config option is middleware. (of course middleware is pluggable)18:48
clayghurricanerix: it's all about covering the use cases18:48
notmynamehurricanerix: yeah, I definitely thing any v2 api should be tied to what's in /info18:48
hurricanerixyeah, I was thinking about brining it up at the hackathon, but then couldn't find that link to read over.18:48
openstackgerritBen Martin proposed openstack/swift: Document method to avoid rsync filling root drive  https://review.openstack.org/19920318:49
clayghurricanerix: awesome!  good idea!18:49
hurricanerixclayg: thanks!18:49
notmynamethe https://etherpad.openstack.org/p/swift-midcycle-aug-2015 link in the topic is where we should put stuff like this :-)18:50
claygnotmyname: oh nice!  thanks!18:50
*** mragupat_ has joined #openstack-swift18:50
notmynameactually, hurricanerix created it and it was his idea!18:50
hurricanerixnotmyname: thanks! I had lost the link... =)18:50
hurricanerix(always back up your bookmarks when reinstalling your OS...)18:51
redboor quit jumping between linux and osx every 5 minutes18:51
notmynameI gotta run. I'll be out for a few hours, then back on tonight18:51
hurricanerixredbo: that has only happened two or three times...18:52
*** mragupat has quit IRC18:54
*** proteusguy has joined #openstack-swift18:55
redbohow about we take out middleware configuration entirely and put in feature flags that then set up the pipeline in the right order.  because it's getting pretty goofballs.18:56
*** mfalatic has joined #openstack-swift18:58
redbowith like 25 middlewares that have to be in the right order to not break things in weird ways or cause security problems18:59
hurricanerixredbo: Do you have an example of how that would work, because I am not sure I understand.  Are you proposing that it is just a "on/off" type thing?18:59
torgomaticredbo: that's probably not a bad idea19:00
redboYeah.  slo = true, then it puts that middleware in the right spot.  which might not be simple, but we're making people do it right now.19:00
hurricanerixredbo: so do they loose the ability to tweak them?  or they would just do it a different way?19:00
*** annegentle has joined #openstack-swift19:02
redboI don't know.  Maybe toss in a feature config section if it's really necessary.  Are we all really changing the defaults on lots of middlewares?19:02
hurricanerixredbo: min_segment_size helps a lot in testing envs19:02
hurricanerixAlso, how would third party middlewares work with that setup?19:04
redbooh, true19:05
redboStill.  goofballs.19:06
hurricanerixmaybe slo = true could just be a shorthand to add things to the pipeline. and modifying the pipeline yourself would become "use at your own risk".  =)19:06
*** _hrou_ is now known as hrou19:06
*** blmartin has joined #openstack-swift19:08
*** mahatic has quit IRC19:09
*** mahatic has joined #openstack-swift19:09
mahaticanyone know what this is supposed to mean? (although I do have that in entry_points.txt)19:10
mahatic"LookupError: Entry point 'decrypter' not found in egg 'swift' (dir: /home/mahati/swift; protocols: paste.filter_factory, paste.filter_app_factory; entry_points:"19:10
*** robefran has joined #openstack-swift19:11
*** annegentle has quit IRC19:11
mahaticI am getting that on "swift-init start main"19:12
*** annegentle has joined #openstack-swift19:12
claygslo = true is great - but how do I say *my* middleware needs to be after slo?19:13
claygmahatic: you need to checkout the crypto change (I think the tail review is on the priority review wiki) and then setup.py develop again (pip install -e .)19:13
clayghurricanerix: I think use at your own risk is sort of what we have now - only there's no other option19:15
clayghurricanerix: making it somewhat easier to do the stock thing (like copying what's there is "too hard") doesn't really solve a general way for middleware authors to not just *describe* but *maintain* a set of rules that will allow the computer to build the pipeline *correctly*19:16
*** robefran_ has joined #openstack-swift19:18
*** robefran has quit IRC19:21
*** mahatic has quit IRC19:21
*** mahatic has joined #openstack-swift19:21
openstackgerritBen Martin proposed openstack/swift: +Document method to avoid rsync filling root drive  https://review.openstack.org/19920319:25
*** ccavanna has joined #openstack-swift19:28
mahaticclayg: thanks. I did that. I was trying to run that on "feature/crypto" branch19:31
*** silor has quit IRC19:32
jrichlimahatic: are you still using the config file from the paste the other day?19:36
jrichlimahatic: although, I agree with clayg.  sounds like the code you have installed doesn't include the new stuff19:37
mahaticjrichli: hi, yes19:37
mahaticum, I took that code again19:38
mahaticjrichli: I did cross check the code on the tail review and my local19:40
jrichliits the develop and install step you might want to do again: and makes sure that is pointing to the right repo19:41
jrichliyour "egg" doesn't have the new middleware in it19:41
*** lpabon has joined #openstack-swift19:42
mahaticjrichli: I am not on "feature/crypto" branch (I took the code and created a new branch)19:43
jrichlican you see the decrypter.py file in your repo?19:45
mahaticyeah19:45
mahaticI get that error when I run the startmain script on "feature/crypto" or master19:45
mahatic(I did do the install and develop)19:46
jrichliyou shouldn't run against just feature/crypto.  it doesn't have that file19:46
jrichliyou have to start from the code on the latest review.  The new middleware has never been accepted onto the feature branch19:47
mahaticjrichli: thanks! that's what I wanted to confirm19:48
jrichlimake your new branch track the review branch19:48
mahaticyeah, that's there19:48
mahaticI'm still sweating out on pip issues!19:48
*** breitz has quit IRC20:19
*** breitz has joined #openstack-swift20:19
*** ohno13 has quit IRC20:23
*** jome has quit IRC20:25
peluseor maybe you mean "swearing" out on pip issues :)20:28
*** robefran_ has quit IRC20:30
*** dmsimard has quit IRC20:30
*** dmsimard has joined #openstack-swift20:34
*** lpabon has quit IRC20:35
*** mragupat_ has quit IRC20:36
*** mragupat has joined #openstack-swift20:37
*** mragupat has quit IRC20:37
*** annegentle has quit IRC20:37
*** mragupat has joined #openstack-swift20:37
*** mragupat has quit IRC20:38
*** mragupat has joined #openstack-swift20:39
clayg!@#$%ing pip20:49
openstackclayg: Error: "@#$%ing" is not a valid command.20:49
claygopenstack: !@#$% you20:49
*** blmartin_ has joined #openstack-swift20:56
*** blmartin has quit IRC20:56
*** blmartin_ has quit IRC20:57
*** blmartin__ has joined #openstack-swift20:57
*** tongli has quit IRC20:57
hayponotmyname: yeah, it took me some iterations to find the simple solution ;-) it's always like that20:57
jrichliclayg : LOL21:00
mahaticlol21:00
mahatic:D21:00
*** annegentle has joined #openstack-swift21:06
*** elmiko has joined #openstack-swift21:07
elmikohi folks, has there been any talk(or blueprints/reviews) about adding keystone Session support to the swiftclient?21:07
*** hrou has quit IRC21:16
*** CaioBrentano has joined #openstack-swift21:17
openstackgerritMerged openstack/swift: Use right most occurrence object dir in diskfile.extract_policy  https://review.openstack.org/20564121:26
*** ccavanna has quit IRC21:34
*** jrichli has quit IRC21:44
openstackgerritSamuel Merritt proposed openstack/swift: Rename WsgiStringIO -> WsgiBytesIO.  https://review.openstack.org/20377521:45
*** blmartin__ has quit IRC21:46
*** mragupat has quit IRC21:57
*** mahatic has quit IRC21:57
*** mahatic has joined #openstack-swift22:05
*** chlong has joined #openstack-swift22:09
*** jlhinson has quit IRC22:09
*** jasondot_ has quit IRC22:15
*** CaioBrentano has quit IRC22:18
*** tongli has joined #openstack-swift22:18
*** CaioBrentano has joined #openstack-swift22:18
*** CaioBrentano has quit IRC22:22
*** marcusvrn has quit IRC22:25
claygelmiko: timburke is working on something like that22:25
claygelmiko: well acctually wait - what is a keystone session22:26
*** NM has quit IRC22:27
*** haypo has left #openstack-swift22:42
*** ccavanna has joined #openstack-swift22:43
*** hrou has joined #openstack-swift22:44
*** annegentle has quit IRC22:50
*** DericHorn-HP has joined #openstack-swift22:51
*** nadeem has quit IRC22:53
peluseclayg, torgomatic - did you guys see my note on patch 201283?22:54
patchbotpeluse: https://review.openstack.org/#/c/201283/22:54
claygpeluse: yeah you like sent an email right - it was like "pls review, kthxbai"22:56
pelusepretty much :)22:57
peluseand now I'm doing the same thing here22:57
clayg:)22:57
peluseso basically the perf work we're trying to do with you guys on the 15 node cluster here in Phx is stalled on the EC side.  We're updating code today to close to master (Swiftstack 2.50) in an attempt to avoid having to debug a lot of misc shit22:58
peluselike FI's missing where they shouldn't be, etc22:58
*** DericHorn-HP has quit IRC22:58
claygpeluse: right on22:59
peluseso not only do I want to get our stuff ugpraded, want to get the other EC bug fixes landed too, as does everyone else, or we won't get good data out of this thing.  others don't seem to hit the bugs we're hitting but not sure anyone else has a full 15 node 12 disk per node all xeon cluster dedicated to beating on the Ec code23:00
claygpeluse: yes I agree these bugs are delaying a lot of testing23:01
peluseyeah, I'm guily of spending to much time on just that one, going to spend the rest of this week looking at the others as well (minus an all day Wed shin dig I'm stuck in)23:02
*** DericHorn-HP has joined #openstack-swift23:05
*** km has joined #openstack-swift23:06
*** DericHorn-HP has quit IRC23:06
*** jasondotstar has joined #openstack-swift23:08
*** jasondotstar has quit IRC23:13
*** kei_yama has joined #openstack-swift23:13
*** darrenc_ is now known as darrenc23:18
*** jasondot_ has joined #openstack-swift23:28
*** onovy has quit IRC23:29
*** onovy has joined #openstack-swift23:31
pelusehmmm, just went through the current list of Ec bugs.  Not too bad I don't think...23:36
*** jasondotstar has joined #openstack-swift23:44
*** DericHorn-HP has joined #openstack-swift23:44
*** jasondot_ has quit IRC23:47
*** tongli has quit IRC23:54
*** ho has joined #openstack-swift23:54
elmikoclayg: i was referring to http://docs.openstack.org/developer/python-keystoneclient/using-sessions.html#23:56
elmikokeystoneclient.session.Session and the related authentication plugins to authenticate/authorize clients, instead of parameterized auth23:56
elmikowell, i shouldn't say authorize just authenticate23:57
elmikobut the Session objects allow for things like connection pooling to keystone and a centralized abstraction for security updates and the like23:57
ctennis peluse: 2.50 release is hinging on clayg, so whisper sweet thoughts into his ear23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!