| *** david-ly_ has joined #openstack-swift | 00:06 | |
| *** david-lyle has quit IRC | 00:08 | |
| *** dmorita has joined #openstack-swift | 00:14 | |
| *** esker has quit IRC | 00:15 | |
| *** gyee_ has quit IRC | 00:17 | |
| *** esker has joined #openstack-swift | 00:25 | |
| *** kei_yama has quit IRC | 00:32 | |
| *** kei_yama has joined #openstack-swift | 00:36 | |
| *** nadeem has quit IRC | 00:44 | |
| *** nadeem has joined #openstack-swift | 00:47 | |
| *** nadeem has quit IRC | 00:52 | |
| *** jrichli has joined #openstack-swift | 00:53 | |
| *** wbhuber has joined #openstack-swift | 01:10 | |
| *** Kennan has left #openstack-swift | 01:11 | |
| *** DericHorn-HP has joined #openstack-swift | 01:16 | |
| asettle | notmyname, going through your instructions for my mac now. I"ll let you know how it goes ;) | 01:16 |
|---|---|---|
| notmyname | asettle: great. I'd appreciate any feedback | 01:23 |
| asettle | Will do :) | 01:24 |
| notmyname | asettle: if you could email them to me, that would be better than pasting them in here. I'll be traveling the next few days, and email will be easier to access | 01:24 |
| notmyname | asettle: me@not.mn | 01:24 |
| asettle | Will do :) so far, so good! :) | 01:24 |
| asettle | No complaints | 01:24 |
| asettle | Other than the provisioning is going to take another hour :p but that's hardly your fault. | 01:24 |
| notmyname | asettle: yeah, you just gotta get faster internets down under ;-) | 01:25 |
| asettle | notmyname, don't even start *facepalm* that's a never ending problem. | 01:26 |
| notmyname | asettle: I'm told that the best thing is to ask mikal about the NBN | 01:27 |
| asettle | Sure, if you'd like him to start running around screaming and hitting things. mattoliverau am I wrong?! :p | 01:27 |
| mattoliverau | That's right, that's why definitely mention it to him ;) | 01:30 |
| *** DericHorn-HP has quit IRC | 01:31 | |
| *** david-ly_ is now known as david-lyle | 01:36 | |
| asettle | Also notmyname Lana was interested to hear what you wanted regarding our email convo :) | 01:36 |
| mattoliverau | asettle: careful I get picked on for shortening works stralian like (re: convo) | 01:37 |
| mattoliverau | *words even | 01:37 |
| asettle | ... convo isn't common? | 01:38 |
| asettle | Damn. | 01:38 |
| asettle | I said 'arvo' and 'mate the other day and Andy McCrae laughed at me for 5 minutes. | 01:38 |
| asettle | :( | 01:39 |
| asettle | I never think I'm particularly Australian until I see people from overseas. | 01:39 |
| mattoliverau | lol, yup.. they used to make fun of me in the UK bout that stuff.. but it's my fault, I'd make fun of them too.. and I was in their country :P | 01:39 |
| mattoliverau | Like I do to Americans at swift mid-cycles :P | 01:39 |
| asettle | Heh. | 01:41 |
| occupant | one time I met an australian in a bar and he said "with all these SUVs driving about, it's like I'm in the I cheem". I paused for a momentm, wondering what the fuck he'd just said. Seeing my look of confusion, he said "you know, with Mr. T" | 01:41 |
| asettle | HAHA | 01:45 |
| asettle | Our pronunciation of the alphabet is pretty shocking | 01:46 |
| asettle | I asked one of the US guys to send me over a document so I could 'Take a squiz at it' | 01:46 |
| asettle | I got pulled up because apparently in urban dictionary the definition isn't as savoury as the Australian meaning. | 01:47 |
| *** darrenc has joined #openstack-swift | 01:55 | |
| *** dontalton has quit IRC | 01:56 | |
| *** asettle is now known as asettle-afk | 02:02 | |
| *** DericHorn-HP has joined #openstack-swift | 02:11 | |
| *** DericHorn-HP has quit IRC | 02:12 | |
| *** DericHorn-HP has joined #openstack-swift | 02:14 | |
| *** DericHorn-HP has quit IRC | 02:17 | |
| notmyname | mattoliverau: I just thought aussies weren't bothered by the "savory" meanings as much as americans are | 02:23 |
| mattoliverau | notmyname: we're just ignorant :P | 02:25 |
| *** bkopilov has quit IRC | 02:33 | |
| *** amit213 has quit IRC | 03:12 | |
| *** zul has quit IRC | 03:13 | |
| *** amit213 has joined #openstack-swift | 03:13 | |
| *** asettle-afk is now known as asettle | 03:15 | |
| *** blmartin_ has joined #openstack-swift | 03:16 | |
| *** zul has joined #openstack-swift | 03:16 | |
| *** doxavore has quit IRC | 03:20 | |
| asettle | notmyname, Completed the installation without a hitch! | 03:37 |
| *** kota_ has joined #openstack-swift | 03:42 | |
| *** ChanServ sets mode: +v kota_ | 03:42 | |
| *** nexusz99 has joined #openstack-swift | 03:50 | |
| *** bkopilov has joined #openstack-swift | 03:54 | |
| *** wbhuber has quit IRC | 04:07 | |
| *** jrichli has quit IRC | 04:17 | |
| *** esker has quit IRC | 04:18 | |
| *** esker has joined #openstack-swift | 04:18 | |
| *** esker has quit IRC | 04:23 | |
| *** nexusz99 has quit IRC | 04:23 | |
| *** nexusz99 has joined #openstack-swift | 04:23 | |
| blmartin_ | Good night all. Have a good day | 04:24 |
| *** blmartin_ has quit IRC | 04:24 | |
| *** nexusz99 has quit IRC | 04:28 | |
| *** DericHorn-HP has joined #openstack-swift | 04:28 | |
| *** madhuri_ has joined #openstack-swift | 04:33 | |
| *** sorrison_laptop has quit IRC | 04:45 | |
| *** amandap_ is now known as amandap | 04:48 | |
| *** blmartin has quit IRC | 05:00 | |
| *** SkyRocknRoll has joined #openstack-swift | 05:02 | |
| *** madhuri has joined #openstack-swift | 05:04 | |
| *** madhuri has quit IRC | 05:13 | |
| *** sorrison_laptop has joined #openstack-swift | 05:19 | |
| *** esker has joined #openstack-swift | 05:19 | |
| *** esker has quit IRC | 05:32 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Add one more test for ssync_receiver https://review.openstack.org/194518 | 05:35 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: WIP: Fix missing container update https://review.openstack.org/186735 | 05:43 |
| *** madhuri_ has left #openstack-swift | 05:44 | |
| clayg | kota_: i see the commit message is WIP - i'm guessing you either agree with torgomatic and my assment - or have a better plan? | 05:47 |
| *** nexusz99 has joined #openstack-swift | 05:48 | |
| kota_ | clayg: agree with torgomatic comments. | 05:48 |
| clayg | right oh | 05:48 |
| kota_ | clayg: I'm now planning to change my implementation to push the missing container_update instead of just 503 returning. | 05:49 |
| kota_ | push into footers | 05:49 |
| *** nexusz99 has quit IRC | 05:49 | |
| clayg | kota_: maybe i need to look at the tests closer - it wasn't obvious to me how the initial connections would ever *not* have the full set of outgoing headers (unless you run out of nodes) | 05:50 |
| kota_ | I guess, you are right, unless we run out of nodes, it should not happen. | 05:51 |
| *** asettle has quit IRC | 05:51 | |
| kota_ | but I guess the available nodes are depends of config... | 05:52 |
| kota_ | request_node_counts, maybe | 05:52 |
| *** zaitcev has quit IRC | 05:53 | |
| kota_ | s/request_node_counts/request_node_count/ | 05:53 |
| clayg | yeah - i knew what you ment | 05:53 |
| clayg | ok, gl | 05:54 |
| kota_ | clayg: thx! | 05:54 |
| clayg | i still like just sending down data quorum + container quorum updates and callin' it a day - i.e. itertools.islice(itertools.cycle(container_nodes)) in _backend_requests | 05:59 |
| *** esker has joined #openstack-swift | 05:59 | |
| *** DericHorn-HP has quit IRC | 06:04 | |
| *** tellesnobrega has quit IRC | 06:05 | |
| kota_ | clayg: ok, let me consider that. And I noticed now you commented before I pushed the rebased patch set. | 06:06 |
| clayg | no worries | 06:06 |
| cschwede | Good Morning! | 06:12 |
| kota_ | cschwede: Good morning! | 06:13 |
| *** esker has quit IRC | 06:15 | |
| clayg | cschwede: I think today i like x-object-fresh-sysmeta [and x-object-fresh-meta when we get around to implementing updatable object metadata and need some place to stick v1 style post behiavor metadata] | 06:28 |
| clayg | something about mass ~= en masse doesn't really convey the life cycle of the metadata they way I'd like | 06:28 |
| clayg | and I really would rather we not couple metadata lifecycle with use-case so tightly - it's already akward enough that x-object-sysmeta has a sperate life cycle than x-object-meta | 06:30 |
| clayg | idk, maybe it's more confusing if for the time being x-object-meta has the same lifecycle as x-object-fresh-sysmeta | 06:31 |
| clayg | ... and I don't even know if x-object-fresh-sysmeta replaces pre-eixsting x-object-sysmeta - or if it just adds to it? | 06:34 |
| *** nexusz99 has joined #openstack-swift | 06:39 | |
| *** ian_brown has quit IRC | 06:40 | |
| *** esker has joined #openstack-swift | 06:41 | |
| clayg | ok - so i don't know about the the massmeta usecase to understand something here | 06:42 |
| clayg | *if* we had updateable object metadata (a la container system metadata handling) - would the encryption metadata stuff that's replaced on PUT or POST want to change with every post? | 06:43 |
| *** proteusguy has quit IRC | 06:47 | |
| openstackgerrit | Matthew Oliver proposed openstack/swift-specs: Add Spec Lifecycle Rules to readme https://review.openstack.org/190066 | 06:53 |
| *** nexusz99 has quit IRC | 06:54 | |
| *** esker has quit IRC | 06:56 | |
| *** aix has joined #openstack-swift | 06:56 | |
| mattoliverau | cschwede: morning | 07:01 |
| *** rledisez has joined #openstack-swift | 07:16 | |
| *** welldannit has joined #openstack-swift | 07:22 | |
| *** welldannit has quit IRC | 07:22 | |
| *** esker has joined #openstack-swift | 07:23 | |
| clayg | mattoliverau: cschwede is ignoring me | 07:29 |
| mattoliverau | clayg: me too :( | 07:29 |
| cschwede | clayg: morning/evening! sorry, i’m not ignoring you, just need to wrap my head around it again | 07:30 |
| cschwede | kota_: mattoliverau: morning/evening too :) | 07:30 |
| mattoliverau | lol | 07:30 |
| mattoliverau | swift, where people are too friendly and get feelings hurt if you don't say hi :P | 07:31 |
| kota_ | mattoliverau: lol | 07:32 |
| kota_ | I'm getting back home, I might get cold this morning so finish up today's work. | 07:33 |
| kota_ | see you tommorow | 07:33 |
| cschwede | clayg: awesome writeup in patch 158401 ! | 07:34 |
| patchbot | cschwede: https://review.openstack.org/#/c/158401/ | 07:34 |
| cschwede | kota_: have a nice evening! | 07:34 |
| *** kota_ has quit IRC | 07:34 | |
| clayg | cschwede: you say that I feel like I have to go re-read what I wrote - I'm not seeing it - I rambled and bitched a little bit - bust mostly just admited there's no obvious answer and this is hard and we probably don't even yet have all the options on the table | 07:36 |
| clayg | which is a really crappy place to be trying to make a decision from | 07:36 |
| cschwede | clayg: you don’t have to re-read it, but it’s a really nice summary on what’s going with all the metadatas and that this needs more thought. i think you summarized the situation well! | 07:37 |
| clayg | ok :\ | 07:37 |
| *** esker has quit IRC | 07:38 | |
| clayg | i guess if you let middleware set x-object-sysmeta on post you loose two things 1) setting "some" x-object-sysmeta on post doesn't *guarantee* x-object-sysmeta that was set on PUT is cleared 2) if you set some x-object-sysmeta on PUT another piece of middleware could come along later and overwrite your sysmeta on POST with a new value in same key | 07:41 |
| *** sorrison_laptop has quit IRC | 07:41 | |
| clayg | #2 is a paper tiger - middlewares have a potential to conflict | 07:42 |
| *** jordanP has joined #openstack-swift | 07:42 | |
| clayg | #1 is something maybe - but can be solved by making sure you always add a x-object-sysmeta key/value to a POST for the keys that may have been set on a PUT you want wiped out | 07:44 |
| clayg | I don't *think* there's really a usecase for "set this one metadata value and wipe out everything else" - it's just a thing our existing crappy user-metadata implemenation *does* | 07:44 |
| clayg | but that existing crappy implementation already allowed *some* metadata with the data file to bleed up through a metadata update - that's why my quick "let post set sysmeta" works - if there's sysmeta down on the datafile it will "pass-through" to a metadata update | 07:46 |
| cschwede | oh yes, i think we could do better with this, but the challenge will be to don’t break existing implementations. | 07:46 |
| clayg | I think this is a totally reasonable behavior | 07:46 |
| cschwede | sometimes i think we’re juggling to many things at the same time | 07:46 |
| clayg | just need another layer of abstraction | 07:47 |
| clayg | shit notmyname is OOO tomorrow - maybe I find jirichi online :\ | 07:48 |
| clayg | if nothing else she might be able to point me down the patch chain at who is expecting massmeta to work and how | 07:49 |
| clayg | bah patch 165517 fake footers is the one that really needs some work | 07:52 |
| patchbot | clayg: https://review.openstack.org/#/c/165517/ | 07:52 |
| clayg | string the trailing metadata mime document conversation stuff back up into replication | 07:52 |
| *** geaaru has joined #openstack-swift | 07:52 | |
| clayg | cschwede: I think you might e right juggling to many things at the same time | 07:53 |
| *** sorrison_laptop has joined #openstack-swift | 07:54 | |
| mattoliverau | k, I'm going to cook some dinner.. but the wife is away tonight, so probably be back later :) | 07:56 |
| mattoliverau | clayg: I assume you'll finally be in bed by the time I get back, so night.. tho you are clayg and apparently you never need to sleep. | 07:56 |
| cschwede | ehh, i think swiftclient tests are broken on the gate :/ | 07:58 |
| cschwede | https://review.openstack.org/#/q/status:open+project:openstack/python-swiftclient,n,z | 07:59 |
| cschwede | failures are always the same | 07:59 |
| *** esker has joined #openstack-swift | 08:04 | |
| *** lyrrad has quit IRC | 08:10 | |
| *** lyrrad has joined #openstack-swift | 08:16 | |
| *** esker has quit IRC | 08:20 | |
| *** nexusz99 has joined #openstack-swift | 08:24 | |
| *** nexusz99 has quit IRC | 08:29 | |
| *** sorrison_laptop has quit IRC | 08:30 | |
| *** joeljwright has joined #openstack-swift | 08:33 | |
| *** ChanServ sets mode: +v joeljwright | 08:33 | |
| *** haypo has joined #openstack-swift | 08:39 | |
| openstackgerrit | Daisuke Morita proposed openstack/swift: Improvement of logging for invalid policies https://review.openstack.org/141238 | 08:41 |
| *** esker has joined #openstack-swift | 08:46 | |
| *** ppai has joined #openstack-swift | 08:46 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Fix ValueError in ssync_receiver https://review.openstack.org/195457 | 08:53 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Make ssync send a better HTTP client https://review.openstack.org/195458 | 08:53 |
| clayg | you have to *ease* back into these things | 08:54 |
| clayg | g'night | 08:54 |
| *** esker has quit IRC | 09:01 | |
| openstackgerrit | Christian Schwede proposed openstack/python-swiftclient: Rename `obj` to `name` in *_object wrappers https://review.openstack.org/189815 | 09:13 |
| openstackgerrit | Merged openstack/swift: Replace xrange() with six.moves.range() https://review.openstack.org/185460 | 09:13 |
| joeljwright | cschwede: I think we need to discuss https://review.openstack.org/189815 again | 09:14 |
| joeljwright | until last night I hadn't fully understood the impact this patch could have on people | 09:15 |
| cschwede | joeljwright: well, yes. though the error in the swiftclient functional tests are not related to that patch, the same happens for multiple swiftclient patches atm: https://review.openstack.org/#/q/status:open+project:openstack/python-swiftclient,n,z | 09:15 |
| cschwede | joeljwright: yes, i assumed an impact; i didn’t expect it on the gate though :) | 09:15 |
| joeljwright | should we save this patch for a larger API tidy up that breaks compatability? | 09:17 |
| joeljwright | for now I think it should probably be just a doc update | 09:18 |
| cschwede | agreed; if you have other api changes in mind we should bundle them, cut a release from that and raise a big red flag then | 09:21 |
| cschwede | but do you want to include in a doc update? the patch doesn’t update the docs? | 09:21 |
| joeljwright | well the 4 names for the same thing is confusing | 09:21 |
| joeljwright | the idea of the patch was to clear up confusion | 09:22 |
| joeljwright | instead of a code change, this could (for now) be extra docstrings | 09:22 |
| joeljwright | I don't have any good reason for breaking API compatibility at the moment, but that might change as we improve the swiftclient in the coming months | 09:24 |
| *** esker has joined #openstack-swift | 09:28 | |
| *** mfalatic has quit IRC | 09:29 | |
| cschwede | ah, ok - you mean docstrings. yes, that’s a good idea | 09:36 |
| *** nexusz99 has joined #openstack-swift | 09:40 | |
| *** dmgweb has joined #openstack-swift | 09:41 | |
| openstackgerrit | Merged openstack/swift: Add policy support to dispersion tools https://review.openstack.org/185459 | 09:42 |
| dmgweb | hello :) | 09:42 |
| dmgweb | anyone knows because swift can give the following error ? INFO:swiftclient:RESP STATUS: 503 Internal Server Error | 09:43 |
| dmgweb | I'm going crazy , searching in google but I can not solve . | 09:43 |
| *** esker has quit IRC | 09:44 | |
| *** nexusz99 has quit IRC | 09:47 | |
| openstackgerrit | Daisuke Morita proposed openstack/swift: object-updater runs for all async_pending directories https://review.openstack.org/141252 | 09:52 |
| *** dmorita has quit IRC | 10:06 | |
| *** dmgweb has quit IRC | 10:09 | |
| *** esker has joined #openstack-swift | 10:11 | |
| *** husanu3 has joined #openstack-swift | 10:19 | |
| *** husanu3 has quit IRC | 10:25 | |
| *** husanu1 has joined #openstack-swift | 10:26 | |
| *** husanu1 has quit IRC | 10:28 | |
| *** esker has quit IRC | 10:29 | |
| *** husanu2 has joined #openstack-swift | 10:30 | |
| *** husanu2 has quit IRC | 10:31 | |
| *** jasondot_ has joined #openstack-swift | 10:32 | |
| *** jasondot_ is now known as jasondotstar | 10:37 | |
| openstackgerrit | Michael Barton proposed openstack/swift: go: replicator only read files once https://review.openstack.org/195510 | 10:39 |
| *** SkyRocknRoll has quit IRC | 10:51 | |
| *** gar has joined #openstack-swift | 10:51 | |
| *** gar has joined #openstack-swift | 10:51 | |
| kragniz | joeljwright: cschwede: yup, I think at least put that patch on hold for a while | 10:54 |
| *** esker has joined #openstack-swift | 10:54 | |
| joeljwright | kragniz: thanks, I think that's for the best atm | 10:58 |
| *** breitz has quit IRC | 11:02 | |
| *** breitz has joined #openstack-swift | 11:02 | |
| *** esker has quit IRC | 11:11 | |
| *** SkyRocknRoll has joined #openstack-swift | 11:15 | |
| *** david-lyle has quit IRC | 11:17 | |
| *** david-lyle has joined #openstack-swift | 11:21 | |
| *** nexusz99 has joined #openstack-swift | 11:31 | |
| *** proteusguy has joined #openstack-swift | 11:32 | |
| *** nexusz99 has quit IRC | 11:36 | |
| *** esker has joined #openstack-swift | 11:37 | |
| *** EmilienM|off is now known as EmilienM | 11:38 | |
| *** SkyRocknRoll has quit IRC | 11:46 | |
| *** jasondotstar has quit IRC | 11:50 | |
| *** esker has quit IRC | 11:54 | |
| *** silor has joined #openstack-swift | 12:06 | |
| *** thurloat_isgone is now known as thurloat | 12:11 | |
| *** nexusz99 has joined #openstack-swift | 12:11 | |
| *** gar has quit IRC | 12:17 | |
| *** esker has joined #openstack-swift | 12:21 | |
| *** kei_yama has quit IRC | 12:26 | |
| *** bkopilov has quit IRC | 12:32 | |
| *** david-ly_ has joined #openstack-swift | 12:35 | |
| *** resker has joined #openstack-swift | 12:36 | |
| *** esker has quit IRC | 12:38 | |
| *** david-lyle has quit IRC | 12:39 | |
| *** gar has joined #openstack-swift | 12:45 | |
| *** gar has joined #openstack-swift | 12:45 | |
| *** jasondotstar has joined #openstack-swift | 12:46 | |
| *** husanu5 has joined #openstack-swift | 12:50 | |
| *** husanu5 has quit IRC | 12:52 | |
| *** tellesnobrega_ has joined #openstack-swift | 12:55 | |
| *** resker has quit IRC | 12:55 | |
| *** esker has joined #openstack-swift | 12:55 | |
| *** husanu5 has joined #openstack-swift | 12:56 | |
| *** husanu5 has quit IRC | 13:00 | |
| *** amoturi has joined #openstack-swift | 13:03 | |
| *** km has quit IRC | 13:04 | |
| *** petertr7_away is now known as petertr7 | 13:05 | |
| *** gar has quit IRC | 13:08 | |
| *** gar has joined #openstack-swift | 13:11 | |
| *** gar has joined #openstack-swift | 13:11 | |
| *** ppai has quit IRC | 13:11 | |
| *** husanux3 has joined #openstack-swift | 13:13 | |
| *** husanux3 has quit IRC | 13:13 | |
| *** barker has joined #openstack-swift | 13:14 | |
| *** husanux0 has joined #openstack-swift | 13:14 | |
| *** husanux0 has quit IRC | 13:18 | |
| *** husanux7 has joined #openstack-swift | 13:20 | |
| *** husanux7 has quit IRC | 13:21 | |
| *** husanux9 has joined #openstack-swift | 13:25 | |
| *** janonymous_ has joined #openstack-swift | 13:26 | |
| *** husanux9 has quit IRC | 13:29 | |
| *** barker has quit IRC | 13:33 | |
| *** barker has joined #openstack-swift | 13:45 | |
| *** tellesnobrega_ has quit IRC | 13:51 | |
| *** tellesnobrega has joined #openstack-swift | 14:01 | |
| *** bkopilov has joined #openstack-swift | 14:02 | |
| *** barker has quit IRC | 14:03 | |
| *** blmartin has joined #openstack-swift | 14:04 | |
| *** peluse has quit IRC | 14:04 | |
| *** tellesnobrega has quit IRC | 14:11 | |
| *** tellesnobrega has joined #openstack-swift | 14:14 | |
| *** jrichli has joined #openstack-swift | 14:15 | |
| *** wbhuber has joined #openstack-swift | 14:20 | |
| *** blmartin_ has joined #openstack-swift | 14:22 | |
| openstackgerrit | Victor Stinner proposed openstack/swift: Use six to fix imports on Python 3 https://review.openstack.org/185453 | 14:22 |
| *** pgbridge has joined #openstack-swift | 14:23 | |
| *** tellesnobrega has quit IRC | 14:24 | |
| *** esker has quit IRC | 14:29 | |
| *** esker has joined #openstack-swift | 14:30 | |
| *** esker has quit IRC | 14:35 | |
| *** jlhinson has joined #openstack-swift | 14:42 | |
| *** acoles_away is now known as acoles | 14:49 | |
| acoles | peluse: looks like mattoliverau and kota got to the dispersion tools review before me | 14:50 |
| acoles | mattoliverau: kota: thanks^^ | 14:50 |
| notmyname | hello | 14:54 |
| notmyname | (just checking in before I go pack--I'm traveling to visit my grandparents today. will be there through the weekend) | 14:54 |
| jrichli | notmyname: could you add the hotel block web link to the hackathon invite? | 15:00 |
| jrichli | http://home2suites.hilton.com/en/ht/groups/personalized/A/AUSTDHT-SWI-20150809/index.jhtml?WT.mc_id=POG | 15:00 |
| *** fifieldt has joined #openstack-swift | 15:00 | |
| notmyname | on it | 15:00 |
| jrichli | notmyname: thanks. Have fun with the grandparents! | 15:00 |
| *** david-ly_ is now known as david-lyle | 15:01 | |
| *** fifieldt has quit IRC | 15:01 | |
| *** fifieldt has joined #openstack-swift | 15:01 | |
| notmyname | jrichli: done | 15:02 |
| jrichli | great, thx! | 15:03 |
| *** lpabon has joined #openstack-swift | 15:05 | |
| *** jasondotstar has quit IRC | 15:10 | |
| jrichli | clayg: I am gathering some info about how massmeta is used. lemme know when u can chat | 15:14 |
| *** blmartin__ has joined #openstack-swift | 15:14 | |
| acoles | jrichli: clayg i am just looking at the massmeta review | 15:14 |
| jrichli | acoles: ok, thanks. My comment to clayg was re to some things he chatted about last night in channel | 15:16 |
| acoles | jrichli: ok. i'll reply to clayg review comments on gerritt | 15:17 |
| acoles | jrichli: lemme know if we need a skype or whatever to sort it all out | 15:17 |
| jrichli | will do | 15:17 |
| *** blmartin has quit IRC | 15:18 | |
| notmyname | https://bugs.launchpad.net/swift/+bug/1468782 | 15:24 |
| openstack | Launchpad bug 1468782 in OpenStack Object Storage (swift) "Swift proxy crashes on CORS request" [Undecided,New] | 15:24 |
| *** DericHorn-HP has joined #openstack-swift | 15:27 | |
| jrichli | notmyname: I will try Dmitry's fix. Although I am still learning, so somebody else might want to also be tackling this. | 15:31 |
| notmyname | jrichli: looks fairly straightforward. however, a test will be required. probably a good place to start | 15:31 |
| jrichli | got it | 15:32 |
| *** mjseger has joined #openstack-swift | 15:40 | |
| *** annegentle has joined #openstack-swift | 15:40 | |
| *** ppai has joined #openstack-swift | 15:43 | |
| *** jasondotstar has joined #openstack-swift | 15:45 | |
| *** annegentle has quit IRC | 15:52 | |
| *** annegentle has joined #openstack-swift | 15:52 | |
| acoles | jrichli: i was thinking about reversing the dependency of patch 163806 and patch 158401 so that 163806 could be merged | 15:59 |
| patchbot | acoles: https://review.openstack.org/#/c/163806/ | 15:59 |
| acoles | patch 158401 | 15:59 |
| patchbot | acoles: https://review.openstack.org/#/c/158401/ | 15:59 |
| acoles | jrichli: but that would mean your dependent chain would need to be rebased (although it will anyways once one of those changes) | 15:59 |
| acoles | jrichli: what do you think? | 16:00 |
| jrichli | acoles: sounds good | 16:00 |
| acoles | jrichli: ok | 16:00 |
| acoles | lets see if i can remember how to do this :D | 16:00 |
| *** nadeem has joined #openstack-swift | 16:01 | |
| *** nadeem has quit IRC | 16:01 | |
| *** SkyRocknRoll has joined #openstack-swift | 16:01 | |
| *** nadeem has joined #openstack-swift | 16:02 | |
| *** lcurtis has joined #openstack-swift | 16:05 | |
| *** ppai has quit IRC | 16:07 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Allow middleware to override metadata header checking https://review.openstack.org/163806 | 16:11 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Enable middleware to set metadata on object POST https://review.openstack.org/158401 | 16:11 |
| *** greghaynes has joined #openstack-swift | 16:12 | |
| *** jordanP has quit IRC | 16:16 | |
| *** annegentle has quit IRC | 16:17 | |
| *** DericHorn-HP has quit IRC | 16:20 | |
| *** peluse has joined #openstack-swift | 16:23 | |
| *** petertr7 is now known as petertr7_away | 16:24 | |
| *** esker has joined #openstack-swift | 16:25 | |
| *** esker has quit IRC | 16:25 | |
| *** esker has joined #openstack-swift | 16:25 | |
| *** doxavore has joined #openstack-swift | 16:35 | |
| *** rledisez has quit IRC | 16:42 | |
| *** jasondotstar has quit IRC | 16:55 | |
| *** DericHorn-HP has joined #openstack-swift | 17:05 | |
| *** esker has quit IRC | 17:05 | |
| *** jasondotstar has joined #openstack-swift | 17:08 | |
| clayg | i'm awake - probably going toward the office soon | 17:10 |
| clayg | jrichli: i really want to understand the life cycle of of the "system metadata updated on every post" in an post-updateable-object-metadata world | 17:11 |
| clayg | acoles: 1(a) isn't a symantic limitation of sysmeta - it's just a technical limitation - middleware can't change individual items because that's not possible - the question is if middleware *could* make use of what we have implemented and sysmeta seems to have missed the use-case where you want to overwrite a subset of keys via POST that may have been set on PUT (needed for encryption) | 17:16 |
| *** petertr7_away is now known as petertr7 | 17:16 | |
| *** nexusz99 has quit IRC | 17:17 | |
| clayg | 1 (b) is a paper tiger - middleware needs to coordinate on sysmeta namespacing - and since they're in the pipeline they can set whatever sysmeta they want on any PUT or POST | 17:17 |
| clayg | if a middleware doesn't want something lost it needs to a) set it on PUT and b) set it on every POST | 17:17 |
| clayg | ^ but that's the same as massmeta | 17:17 |
| clayg | sysmeta fixed the .data metadata getting lost on a post - if you squint - it's *already* updatable metadata - you just get two timestamps (data & meta) instead of N | 17:18 |
| openstackgerrit | Charles Hsu proposed openstack/swift: Fix reconstructor stats mssage. https://review.openstack.org/195275 | 17:32 |
| acoles | clayg: 1(a) agree (I think) its a current technical limitation that today you cannot update individual items of sysmeta on object POST. Let me think some more about how easy/hard it is to overcome that limitation. | 17:33 |
| *** jasondotstar has quit IRC | 17:34 | |
| acoles | clayg: 1(b) we can't let a middleware change *all* sysmeta on a POST because that would blat EC (and crypto) sysmeta (e.g. encrypted etag) that can't be regenerated easily on a POST. | 17:36 |
| clayg | ohai | 17:36 |
| acoles | 1(b) argues that any change to sysmeta on POST must be selective | 17:36 |
| clayg | acoles: isn't already if the keys don't collide? | 17:36 |
| clayg | acoles: I mean in my curl snippet with the oneline diff to allow sysmeta into .meta files | 17:37 |
| clayg | acoles: I set bar to baz on PUT and foo to bar on POST - afterwards I had a merge of the both | 17:37 |
| clayg | subsequent post wipes out foo, and I can overwrite bar with a POST | 17:37 |
| clayg | seems to be what crypto needs | 17:37 |
| clayg | all it has to do is not overwrite ec metadata that was set on PUT (I think) | 17:38 |
| *** occupant has quit IRC | 17:38 | |
| acoles | clayg: yes. the change you made was == "any change to sysmeta on POST must be selective", you didn't replace all sysmeta with the new set. | 17:38 |
| clayg | acoles: and that's what crypto needs right? | 17:38 |
| clayg | acoles: it's also sorta how you would expect to use updatable object metadata | 17:39 |
| *** jasondotstar has joined #openstack-swift | 17:39 | |
| acoles | clayg: yes but selective changes to sysmeta that was 1(a)and like i said i need think about the implications wrt consistency because you have challenged my status quo on that one :P | 17:40 |
| clayg | i'm such a bastard | 17:40 |
| acoles | clayg: like, maybe we can... | 17:41 |
| acoles | heh | 17:41 |
| clayg | acoles: I sorta want to just "try it out" | 17:41 |
| clayg | acoles: like take jrichli's patch change and rebase it on the oneline let sysmeta into posts and find out what breask? | 17:41 |
| jrichli | clayg, acoles: just got back from lunch. reading scrollback | 17:43 |
| *** occupant has joined #openstack-swift | 17:43 | |
| acoles | clayg: so it would be "any sysmeta that is ever changed on a POST must either be set again on subsequent POST or it gets lost" ? | 17:44 |
| clayg | jrichli: how far up the patch chain do I have to get to for something that is setting massmeta on PUT/POST | 17:44 |
| acoles | hmmm | 17:44 |
| clayg | acoles: yeah but that's true already! I mean - we just called it massmeta | 17:44 |
| acoles | clayg: is that what i did ? :P | 17:47 |
| clayg | anyway i'm not sure this life-cycle would exist if we had updatable object metadata - so i'm cautious of making a whole new namespace for it - esspecially since it seems like sysmeta is already really really close to updatable object metadata | 17:47 |
| clayg | you can set sysmeta on a data and it stays with the data unless you overwrite it in the meta - can easily extend to you can set sysmeta and it stays unless you overwrite it | 17:48 |
| clayg | acoles: YES! I'm totally just grabbing your cool thing like a thug and running with it! | 17:48 |
| clayg | acoles: anyways - it doesn't have to be answered this morning/evening | 17:49 |
| clayg | acoles: you think - and I'll try to get jrichli to get me a patch that I can rebase on expanded lifecycle sysmeta to see if it works | 17:49 |
| jrichli | clayg: https://review.openstack.org/#/c/194191/ uses metadata | 17:49 |
| clayg | jrichli: is that the *first* one? it's sorta deep :\ | 17:50 |
| jrichli | forgive me if I am saying something that is obvious or that you already know: one thing about the massmeta is that it is paired with user-meta items | 17:50 |
| jrichli | yes. the first 3 patches are changes alistair made in core swift - but they dont add any encryption | 17:51 |
| jrichli | the "life-cyle" of the massmeta (this is used for storing what we have deemed "crypto-meta") has to match that of the corresponding user-meta | 17:52 |
| jrichli | metadata items are storing off the cypher-alg and nonce - and whatever else is needed to decrypt | 17:53 |
| clayg | jrichli: i see 1) add massmeta 2) make fakefooters 3) keymaster 4) crypto 5) encrypter-uses-sys(mass?)-meta | 17:53 |
| jrichli | yes :-) | 17:54 |
| acoles | clayg: there is a difference - massmeta behaves exactly like user-meta, on a POST all existing massmeta is discarded. What you are suggesting is that on a POST only the keys in the posted sysmeta get discarded. SO you get a merge of data file and meta file items. Ain't saying that can't work, but thats the difference I want to ponder wrt eventual consistency. | 17:54 |
| jrichli | there was one more in there somewhere ... | 17:54 |
| *** breitz has quit IRC | 17:54 | |
| acoles | (had to say the "eventual consistency" words eventually :) | 17:55 |
| acoles | clayg: jrichli i just reordered the chain and +A'd the first - did it land? | 17:56 |
| clayg | acoles: i saw the +a | 17:56 |
| openstackgerrit | Greg Lange proposed openstack/swift: go: add ability to lock device/drive with file on file system https://review.openstack.org/195689 | 17:56 |
| *** tellesnobrega has joined #openstack-swift | 17:56 | |
| *** petertr7 is now known as petertr7_away | 17:57 | |
| *** petertr7_away is now known as petertr7 | 17:57 | |
| acoles | patch 163806 no its still in the queue | 17:57 |
| patchbot | acoles: https://review.openstack.org/#/c/163806/ | 17:57 |
| *** NM has joined #openstack-swift | 17:57 | |
| *** breitz has joined #openstack-swift | 17:57 | |
| clayg | acoles: so that's *1* down ;) | 17:58 |
| acoles | clayg: progresssss :) | 17:58 |
| clayg | jrichli: is sysmeta and/or massmeta encrypted? | 17:59 |
| acoles | clayg: ok i supposed to spend time with family this evening apparently. I will think while you run with it! | 17:59 |
| clayg | jrichli: you said massmeta lives "with the user-metadata" | 17:59 |
| jrichli | no. only user-meta, etag, and content-type | 17:59 |
| clayg | acoles: sounds like a great plan! | 17:59 |
| clayg | acoles: thank you so much | 17:59 |
| jrichli | acoles: thanks | 17:59 |
| acoles | clayg: jrichli : g'nite | 17:59 |
| acoles | clayg: no thank you for challenging the thinking | 18:00 |
| clayg | jrichli: so when you go to decrypt user-metadata - it's not akward hving like ec-sysmeta and x-timestamp and stuff in the bag of metadata - you just pick out the user-metadata and use the crypto-meta in the clear to do what you need to do? | 18:01 |
| clayg | maybe I can find that code... | 18:01 |
| *** SkyRocknRoll has quit IRC | 18:01 | |
| jrichli | clayg: yes. | 18:02 |
| jrichli | which code? | 18:02 |
| *** husanu5 has joined #openstack-swift | 18:02 | |
| *** dontalton has joined #openstack-swift | 18:02 | |
| jrichli | the code where crypto-meta is used? | 18:02 |
| clayg | the code that does the decrypting of user-metadata from the crypto-meta stored in massmeta | 18:02 |
| jrichli | one min | 18:03 |
| jrichli | ugh, wait: | 18:03 |
| jrichli | the patch you have for encrpter / decrypter now only uses massmeta for the crypto-meta of the object body | 18:04 |
| jrichli | i split that out from headers encryption to make an easier review | 18:04 |
| jrichli | so right now, you can look at the body_crypto_meta being used. Is that enough for now? Or do you want to see the header processing in the "prototype" review? | 18:05 |
| clayg | idk - i want to see if extending sysmeta the way i'm suggesting will cover the crypto use-case | 18:05 |
| *** jasondotstar has quit IRC | 18:05 | |
| clayg | it seems like the updated crypto metdata on the POST will be important to understanding that interplay | 18:06 |
| jrichli | prob best to look at the prototype with all the functionatlity: https://review.openstack.org/#/c/157907/ | 18:06 |
| clayg | jrichli: do you know what acoles and I are talking about when we say "updatable object metadata" | 18:06 |
| *** acoles is now known as acoles_away | 18:06 | |
| jrichli | clayg: one reason that we have a crypto-meta for each different user-meta item is that we need a different nonce for each one | 18:07 |
| clayg | oh wow - each user-meta item has it's own nonce! | 18:07 |
| clayg | that's good - that will work *better* with updatable object metadata - was that acoles idea? | 18:08 |
| jrichli | yes. and everytime a given user-meta item is set/updated, it will need to be re-encrypted with a new nonce. | 18:08 |
| *** esker has joined #openstack-swift | 18:08 | |
| jrichli | I know that acoles had said that *someday* there might be updatable object metadata. I didnt know he was suggesting it now. i haven't gotten to thoroughly read this scrollback. | 18:09 |
| clayg | no he's not suggesting it now | 18:09 |
| clayg | he's suggesting adding a new metadata namespace that we have to live with forever because we don't have updatable object metdata today | 18:09 |
| jrichli | for furture-proofing, I would suggest not doing something that locks our encrypted objects into never having updatable obj meta | 18:09 |
| clayg | i'm suggesting we don't need it because there's not really a usecase for what this new namespace is doing - it's just an enhanced version of sysmeta - that's acctually closer to where we want to go anyway | 18:10 |
| jrichli | well yes, that part I knew about :-) | 18:10 |
| jrichli | oh wait - read wrong, sorry | 18:10 |
| clayg | anyway - this patch with everything is perfect - i think my plan will work | 18:12 |
| clayg | it'll probably be annoying to describe the life-cycle of sysmeta++ but it'll be fine | 18:12 |
| jrichli | ok, cool. for me, I just want a way to store this info, have it persisted, and have it updatable when needed :-) | 18:12 |
| clayg | that's what I'm talking about - get shit done ;) | 18:13 |
| jrichli | and to explain a little more on the nonce : its a crypto thing that makes the meta-data require having a different nonce. if they use the same one, we can start leaking info | 18:13 |
| clayg | i got some ec bugs i'm working on too - but since i'm jazzed about this idea I'll try to get some cycles on it - maybe we present to acoles next week - like monday am skype or something? | 18:14 |
| jrichli | sounds good. | 18:14 |
| *** husanu5 has quit IRC | 18:14 | |
| * jrichli I guess I will need to install skype on my new mac | 18:14 | |
| petertr7 | Is it possible to remove/delete headers from objects such as X-delete-at? | 18:16 |
| *** jasondotstar has joined #openstack-swift | 18:19 | |
| jrichli | clayg: oh, and yes - I am pretty sure I know what you mean by updatable obj data. you mean b/c on a POST, if you update any of user-meta, then the whole set gets replaced with what is in that POST request | 18:21 |
| jrichli | so updatable would mean that you could update one piece - and not have to provide the whole set? | 18:21 |
| mjseger | clayg: are you aware of issues in which connection,py is not included with the requests library and as a result nagel isn't being disabled? I just discovered this in 14.04 and am seeing 1K PUTs more that 2X slower than 2K PUTs | 18:22 |
| hurricanerix | petertr7: you should be able to just supply a empty 'x-delete-at' header. http://pastebin.com/68phbTzP | 18:22 |
| mjseger | if I update requests, performance becomes what it should | 18:22 |
| petertr7 | hurricanerix, Thanks! I'm going to try that | 18:25 |
| *** geaaru has quit IRC | 18:26 | |
| petertr7 | hurricanerix, I actually get this error when I attempt to pass in an empty header "400 Bad Request Non-integer X-Delete-At" | 18:27 |
| petertr7 | I'm using python-swiftlicent post_object and passing in the empty string | 18:27 |
| tdasilva | anybody know of a flag to disable deletes on ssbench? I'd like to leave the data on the cluster, not just the initial data | 18:28 |
| hurricanerix | petertr7: ahh, let me take a look at the client. | 18:28 |
| tdasilva | swifterdarrell: ^^^ | 18:28 |
| *** annegentle has joined #openstack-swift | 18:28 | |
| swifterdarrell | tdasilva: ya there's a flag for that | 18:28 |
| swifterdarrell | one sec | 18:29 |
| swifterdarrell | tdasilva: -k, --keep-objects Keep all uploaded objects in cluster; | 18:29 |
| swifterdarrell | tdasilva: for "ssbench-master run-scenario ..." | 18:30 |
| swifterdarrell | tdasilva: (used that when benchmarking the server-per-port stuff, actually) :) | 18:30 |
| tdasilva | swifterdarrell: thanks! i'm actually looking to test object expiration. Saw the flag for x-delete-after, but then the data was actually getting deleted ;) | 18:31 |
| *** DericHorn-HP has quit IRC | 18:31 | |
| swifterdarrell | tdasilva: heh | 18:31 |
| hurricanerix | petertr7: Actually, I am pretty sure if you just do "swift post container object" it will remove all headers from the object. (since existing custom metadata does not persist, when posting new metadata) | 18:31 |
| hurricanerix | *all user-added metadata | 18:31 |
| swifterdarrell | tdasilva: I didn't add the delete-after bit and haven't used it, if that matters | 18:31 |
| tdasilva | swifterdarrell: it seems to be working correctly.... | 18:32 |
| tdasilva | running some tests now | 18:32 |
| swifterdarrell | tdasilva: nice! | 18:32 |
| petertr7 | Ohh that's so easy. Thanks hurricanerix! | 18:33 |
| hurricanerix | peluse: np | 18:33 |
| hurricanerix | petertr7: np | 18:34 |
| openstackgerrit | Merged openstack/swift: go: check for empty path parts in object server https://review.openstack.org/194789 | 18:34 |
| hurricanerix | petertr7: you can also find more info here: http://developer.openstack.org/api-ref-objectstorage-v1.html#updateObjectMeta | 18:34 |
| hurricanerix | (just click "detail" below the one that says "POST") | 18:35 |
| petertr7 | Thank you, this is such a conveniently compact page for the api! | 18:35 |
| openstackgerrit | Merged openstack/swift: go: make use of filepath.Join() in objectserver main https://review.openstack.org/195271 | 18:39 |
| *** dontalton has quit IRC | 18:44 | |
| mjseger | does anyone want to know why 1K puts are twice as slow of 2K puts? | 18:45 |
| *** annegentle has quit IRC | 18:45 | |
| *** jasondotstar has quit IRC | 18:47 | |
| *** rbrooker has joined #openstack-swift | 18:52 | |
| *** lastops has joined #openstack-swift | 18:54 | |
| *** lastops has quit IRC | 18:54 | |
| *** hemanthm has joined #openstack-swift | 19:01 | |
| *** jasondotstar has joined #openstack-swift | 19:07 | |
| torgomatic | mjseger: I thought you just said it was TCP_NODELAY not being set by an old version of requests library | 19:07 |
| torgomatic | is it something else? that'd be interesting | 19:07 |
| torgomatic | (not that requests's behavior is uninteresting, but it is known) | 19:08 |
| mjseger | torgomatic: it's complicated, I think. on a new 14.04 installation, requests is preinstalled, but only some of it, not the urllib3 substree | 19:08 |
| mjseger | when you setup.py install swiftclient, the urllib subtree IS populated BUT connection.py is left off and he's the guy who disables nagel | 19:09 |
| mjseger | if I manually install requests everyone is happy | 19:09 |
| mjseger | very odd | 19:09 |
| *** tellesnobrega has quit IRC | 19:10 | |
| *** blmartin has joined #openstack-swift | 19:11 | |
| *** blmartin_ has quit IRC | 19:13 | |
| *** blmartin__ has quit IRC | 19:13 | |
| mjseger | torgomatic: when you say requests's behavior is known, can you elaborate? | 19:15 |
| torgomatic | mjseger: I thought that old versions of requests did not set TCP_NODELAY, but at some point they added it in | 19:15 |
| mjseger | ahh, right. this is something very different. let me try again | 19:16 |
| mjseger | torgomatic: if you install a brand new 14.04 and clone swiftclient, the right version of requests is indeed there - 2.2.1 - BUT it's missing connection.py. if you reinstall, you get it, but reinstallation is NOT straightword | 19:17 |
| torgomatic | mjseger: so ubuntu has broken packages? :sadface: | 19:17 |
| * torgomatic is getting hauled into a meeting; sorry about that | 19:18 | |
| mjseger | torgomatic: I'm not sure. If I do pip install requests-2.2.1 it tells me it's already installed. if I try to uninstall it, it tells me it can't because owned by the o/s | 19:18 |
| mjseger | perhaps a conversation for another day? | 19:18 |
| *** blmartin has quit IRC | 19:21 | |
| *** tellesnobrega has joined #openstack-swift | 19:29 | |
| *** peluse has quit IRC | 19:33 | |
| *** tellesnobrega has quit IRC | 19:35 | |
| *** blmartin has joined #openstack-swift | 19:44 | |
| *** DericHorn-HP has joined #openstack-swift | 19:50 | |
| *** esker has quit IRC | 19:54 | |
| *** janonymous_ has quit IRC | 19:56 | |
| *** gar has quit IRC | 19:59 | |
| *** thurloat is now known as thurloat_isgone | 20:01 | |
| *** blmartin has quit IRC | 20:05 | |
| openstackgerrit | Victor Stinner proposed openstack/swift: Get StringIO and cStringIO from six.moves https://review.openstack.org/185457 | 20:06 |
| openstackgerrit | Victor Stinner proposed openstack/swift: Replace StringIO with BytesIO in ssync https://review.openstack.org/186073 | 20:06 |
| openstackgerrit | Victor Stinner proposed openstack/swift: Replace StringIO with BytesIO for file https://review.openstack.org/186072 | 20:06 |
| openstackgerrit | Victor Stinner proposed openstack/swift: Replace StringIO with BytesIO for WSGI input https://review.openstack.org/186071 | 20:06 |
| openstackgerrit | Minwoo Bae proposed openstack/swift: EC Reconstructor: Do not reconstruct existing fragments. https://review.openstack.org/193279 | 20:08 |
| minwoob | clayg: I've updated what I currently have, for the rebuilding fragments patch. It is now down to only 1 failing test, of the 5 that were originally failing. I've also added in the unit test for making sure that the reconstructor does not fail when it finds the fragment to be rebuilt. | 20:14 |
| *** amoturi has quit IRC | 20:14 | |
| minwoob | It now just needs a test for testing duplicates. | 20:14 |
| clayg | I didn't do it | 20:14 |
| minwoob | clayg: No worries. I'm working on it now. | 20:15 |
| clayg | minwoob: that's great! | 20:15 |
| clayg | minwoob: nice work! | 20:15 |
| clayg | minwoob: is there anything I can do to help you today? | 20:15 |
| clayg | should I look at what you have? debug the test failure? cheer from the sidelines? | 20:15 |
| minwoob | clayg: Possibly taking a look at the 1 remaining failing test, if you have the time. | 20:15 |
| clayg | right on! | 20:15 |
| minwoob | clayg: Cool! Thanks! | 20:15 |
| minwoob | clayg: I'll be uploading another patch set soon, for the duplicates. | 20:16 |
| openstackgerrit | Victor Stinner proposed openstack/swift: Fix Python 3 issues https://review.openstack.org/185450 | 20:16 |
| clayg | minwoob: so it's +6 lines of code and ~100 of test changes - and you're only part way done with tests | 20:19 |
| clayg | sounds bout right | 20:19 |
| clayg | minwoob: ah interesting - looks like test_reconstruct_fa_with_mixed_old_etag only failures 9/10 times! | 20:21 |
| clayg | for i in {1..10}; do nosetests swift/test/unit/obj/test_reconstructor.py:TestObjectReconstructor.test_reconstruct_fa_with_mixed_old_etag &> /dev/null; echo $?; done | 20:21 |
| minwoob | clayg: Do they normally fail all the time, when they fail? | 20:22 |
| clayg | ... to good ones do | 20:23 |
| clayg | tests that fail only sometimes are super annoying :'( | 20:23 |
| minwoob | Ah. | 20:23 |
| clayg | but 9/10 failures is better than 1/10 failures | 20:23 |
| minwoob | clayg: Right ... because then you might not catch it ... | 20:24 |
| minwoob | until it fails in production. | 20:25 |
| clayg | well a lot of times they're just shotty tests | 20:25 |
| clayg | but yeah sometimes there's a real bug there | 20:26 |
| clayg | more often the bugs lurk where there is NO test coverage because no one even thought to write a test for that failure! | 20:26 |
| *** peluse has joined #openstack-swift | 20:27 | |
| minwoob | By that, I assume you mean cases where no one could think a simple part could fail ... | 20:29 |
| clayg | something like that - or "how on earth did we end up with *no* data on primaries" or "why would a tombstone that's 6 months hold suddenly show up?" - when you do something a billion times that non-zero aspect of improbable things start to pop up | 20:31 |
| *** peluse has quit IRC | 20:32 | |
| clayg | one time the json guy - doug something - came to a company i was working for and talked about software engineering in broard strokes - one thing he said I loved "Saying 'that would rarely happen' is exactly the same as saying 'that happens'" | 20:32 |
| minwoob | clayg: Lol. | 20:33 |
| clayg | ok, i got it failing 10/10 now | 20:33 |
| *** jasondotstar has quit IRC | 20:39 | |
| *** silor has quit IRC | 20:44 | |
| *** peluse has joined #openstack-swift | 20:47 | |
| *** peluse_ has joined #openstack-swift | 20:48 | |
| *** peluse__ has joined #openstack-swift | 20:49 | |
| *** peluse has quit IRC | 20:52 | |
| clayg | minwoob: ok you have to be careful with those calls to next(ts) | 20:57 |
| clayg | it's an iterator - so moving where they are called in the test changes their value | 20:58 |
| clayg | when I get confused I tend to say "t0 = next(ts)" and "t1 = next(ts)" and then use t0 and t1 where I need them | 20:58 |
| clayg | but *sometimes* i just cheat and call next(ts) inline w/o naming it - which is when you have to be careful about moving it before/after another call or inside of a loop | 20:59 |
| *** ChanServ sets mode: +v peluse__ | 20:59 | |
| minwoob | clayg: That's interesting. I've also noticed that mixed_new_etag is very similar in this regard, but I have not seen it fail so far. | 21:02 |
| minwoob | clayg: mixed_new_etag the test* | 21:03 |
| clayg | yeah - i think that one is like an expected failure condition - a negative test if you will | 21:03 |
| *** annegentle has joined #openstack-swift | 21:03 | |
| clayg | minwoob: there was a typo in that gist - i updated it | 21:04 |
| *** peluse_ has quit IRC | 21:05 | |
| *** petertr7 is now known as petertr7_away | 21:07 | |
| *** DericHorn-HP has quit IRC | 21:08 | |
| *** DericHorn-HP has joined #openstack-swift | 21:11 | |
| *** amoturi has joined #openstack-swift | 21:13 | |
| clayg | minwoob: yeah mixed_new_etag assertRaises DiksFileError | 21:14 |
| clayg | in fact I think when you changed mixed_old_etag to make the bad_requests ts newer - it sort of *became* mixed_new_etag and raised diskfile error just like mixed_new_etag was expected to do :P | 21:14 |
| minwoob | Yeah, that's what it looks like. | 21:18 |
| openstackgerrit | Minwoo Bae proposed openstack/swift: EC Reconstructor: Do not reconstruct existing fragments. https://review.openstack.org/193279 | 21:20 |
| minwoob | I saw a failure in test.unit.obj.test_updater.TestObjectUpdater | 21:21 |
| minwoob | Looking into it now. | 21:21 |
| *** jrichli has quit IRC | 21:24 | |
| *** ian_brown has joined #openstack-swift | 21:41 | |
| *** ian_brown has quit IRC | 21:48 | |
| *** annegentle has quit IRC | 21:58 | |
| timburke | joeljwright: thanks for looking at some patches! it makes me sad that global requirements seems to have busted the gate, though | 21:58 |
| *** anteaya has joined #openstack-swift | 22:04 | |
| anteaya | anyone know what return 137 means in the swift-dsvm-funtional job? | 22:05 |
| anteaya | we can't find any logs | 22:05 |
| anteaya | and it is kind of blocking a patch that we would like to merge | 22:05 |
| anteaya | https://review.openstack.org/#/c/195213/ | 22:05 |
| anteaya | I am stuck on how to get more information about why/how this is failing | 22:05 |
| *** sc68cal has joined #openstack-swift | 22:06 | |
| *** NM has quit IRC | 22:07 | |
| *** pberis has joined #openstack-swift | 22:08 | |
| anteaya | notmyname mattoliverau ^^ | 22:09 |
| clayg | return 137? | 22:10 |
| clayg | maybe lits like 1337-speak? | 22:10 |
| clayg | anteaya: wow - that just looks like *all* the tests failed!? | 22:12 |
| clayg | swift must not be very happy :'( | 22:12 |
| clayg | wow - new to me! HTTPException: got more than 100 headers (txn: tx92bb01cf831142dd9e827-00558c3723) | 22:14 |
| clayg | in the proxy logs | 22:14 |
| clayg | i wonder what version of python 2.7 this is? | 22:16 |
| clayg | 2.7.6-8ubuntu0.2 - seems reasonable | 22:17 |
| mattoliverau | Yeah according to the logs, the > 100 headers error starts appearing on reposnses from the container server.. Then it looks like it gets error limited. Is it like a header dict that isn't cleaned up. | 22:22 |
| clayg | mattoliverau: i'm not sure why I folow that >100 headers is not reasonable? | 22:23 |
| joeljwright | timburke: shame about the gate :( | 22:23 |
| clayg | and I can't find any reference to MAXHEADER in my /usr/lib/python2.7/httplib.py | 22:23 |
| clayg | joeljwright: timburke: you get used to it | 22:23 |
| clayg | joeljwright: timburke: plus the first time you find the bug that *fixes* the gate you feel sorta like a hero | 22:24 |
| mattoliverau | I saw something about the rfc822.Message constructor.. So a limit there maybe | 22:24 |
| joeljwright | timburke: maybe you could have a look at this patch for me… https://review.openstack.org/#/c/171692/ | 22:24 |
| joeljwright | now, time for sleep | 22:24 |
| mattoliverau | clayg ^ | 22:24 |
| mattoliverau | A limit in that module I mean | 22:25 |
| clayg | but am i mis reading the traceback? should be in line 279 of httplib.py | 22:25 |
| mattoliverau | Oh yeah | 22:26 |
| timburke | joeljwright: i like that idea! i'll put it on my list. will probably get to it some time next week | 22:26 |
| *** asettle has joined #openstack-swift | 22:28 | |
| clayg | lol@ * SECURITY UPDATE: denial of service in multiple servers | 22:29 |
| clayg | - debian/patches/CVE-2013-1752-httplib-2.patch: limit amount of headers | 22:29 |
| clayg | in Lib/httplib.py, added test to Lib/test/test_httplib.py. | 22:29 |
| clayg | i'm sure we can monkey patch it to 1000 or something | 22:30 |
| *** jlhinson has quit IRC | 22:30 | |
| * clayg does all the updates | 22:30 | |
| mattoliverau | clayg: nice find | 22:31 |
| clayg | i can so imagine someone sitting there thinking "yeah I'm pretty sure you don't need more than 100 headers - I mean - that's two zeros for christ's sake" | 22:31 |
| *** dontalton has joined #openstack-swift | 22:32 | |
| mattoliverau | instead they dos us by making a container server in testing error limitted. That is teh suck. | 22:34 |
| clayg | oh wait - timburke - is this the thing that's breaking the gate!? | 22:34 |
| clayg | yay! there it is :) | 22:35 |
| clayg | _MAXHEADERS = 100 | 22:35 |
| timburke | clayg: what, header limits? no. it's something with global dependencies adding requirements that our gate can't parse | 22:35 |
| clayg | oh neat - so multiple brokens - that's what Im talking about | 22:36 |
| mattoliverau | Thats a great way to start a Friday morning :P | 22:36 |
| clayg | is it Friday already? | 22:36 |
| mattoliverau | yeah.. well here it is. | 22:37 |
| timburke | clayg: see http://logs.openstack.org/87/190887/5/check/check-swiftclient-dsvm-functional/1d3cad1/console.html#_2015-06-24_02_48_27_036 for an example | 22:37 |
| peluse__ | hell, I'm outta here then | 22:37 |
| mattoliverau | peluse__: lol, it is only 8:30am.. so a little too early to call it quits. :P | 22:37 |
| peluse__ | damn in | 22:38 |
| peluse__ | it | 22:38 |
| clayg | timburke: what the crap is that crap!? | 22:38 |
| *** peluse__ is now known as peluse | 22:38 | |
| *** thurloat_isgone is now known as thurloat | 22:39 | |
| timburke | clayg: i've been advised by our PTL to "head down to the pub for a pint until it all blows over" | 22:39 |
| clayg | timburke: PTL is good like that | 22:39 |
| joeljwright | timburke: sounds like adive you should listen to :) | 22:40 |
| joeljwright | advice even | 22:40 |
| clayg | timburke: OTOH, I ignore his sage advice more often than most | 22:40 |
| clayg | you could always just dive in and see what sort of rukus you can cause? | 22:40 |
| mattoliverau | but it's too early to start drinking!!!... but if the PTL says its ok :P | 22:40 |
| timburke | clayg: yeah, yeah; after the release | 22:40 |
| *** lcurtis has quit IRC | 22:41 | |
| notmyname | hello. I'm sitting in DFW for a few hours | 22:41 |
| notmyname | catching up on scrollback... | 22:41 |
| clayg | mattoliverau: lol! "PTL said i could" is my answer for all my bad decisions now | 22:41 |
| clayg | test_bad_metadata3 seems to be the test that maxheaders love to hate | 22:41 |
| *** joeljwright has quit IRC | 22:41 | |
| mattoliverau | lol, sounds like a great t-shirt | 22:42 |
| timburke | clayg: mattoliverau: in notmyname's defense, it was like 10PM local time when he gave me that advice :P | 22:45 |
| notmyname | seems like I logged in at just the right time ;-) | 22:45 |
| mattoliverau | notmyname: your PTL sense was tingling :P | 22:45 |
| *** DericHorn-HP has quit IRC | 22:45 | |
| *** dontalton has quit IRC | 22:45 | |
| *** pberis1 has joined #openstack-swift | 22:46 | |
| *** pberis has quit IRC | 22:46 | |
| notmyname | so everything is broken because a CVE added a limit to headers in httplib2? | 22:51 |
| notmyname | and the other broken thing about requirements parsing is still broken? ( timburke?) | 22:52 |
| mattoliverau | looks like it in both counts | 22:53 |
| timburke | notmyname: can't speak to the headers, but yes, requirements stuff is still broken | 22:54 |
| notmyname | oh, goody | 22:54 |
| notmyname | timburke: did you go to the pub for a pint? | 22:54 |
| notmyname | because clearly it hasn't blown over yet | 22:54 |
| *** thurloat is now known as thurloat_isgone | 22:54 | |
| timburke | notmyname: *that's* why it's still busted! i was never very good at following directions... | 22:55 |
| notmyname | :-) | 22:55 |
| clayg | that moment when you realize you have no idea how eventlet manages it's monkey patching of stdlib modules | 22:55 |
| openstackgerrit | Merged openstack/swift: Allow middleware to override metadata header checking https://review.openstack.org/163806 | 22:56 |
| *** wbhuber has quit IRC | 22:57 | |
| *** annegentle has joined #openstack-swift | 22:59 | |
| *** nadeem has quit IRC | 23:00 | |
| notmyname | that moment when you see a bot say a patch was merged, but you have no idea what the patch does ;-) | 23:02 |
| * notmyname looks at openstackgerrit | 23:02 | |
| notmyname | cool. it was a crypto patch. glad to see progress there :-) | 23:05 |
| *** annegentle has quit IRC | 23:05 | |
| clayg | notmyname: yeah acoles_away comes back for one day and shows us all up | 23:05 |
| mattoliverau | lol | 23:08 |
| mattoliverau | so acoles_away is why we ever get anything done.. I see that now | 23:09 |
| *** rbrooker has quit IRC | 23:22 | |
| *** kei_yama has joined #openstack-swift | 23:27 | |
| openstackgerrit | Merged openstack/swift: go: check for content length parse error on gets in object server https://review.openstack.org/195273 | 23:32 |
| *** doxavore has quit IRC | 23:44 | |
| *** esker has joined #openstack-swift | 23:46 | |
| *** pberis1 has quit IRC | 23:46 | |
| *** esker has quit IRC | 23:46 | |
| *** esker has joined #openstack-swift | 23:47 | |
| clayg | hurricanerix: is creith like still "around"? | 23:48 |
| clayg | I sort of imagine he may have been one of three people on the planet that acctully had some notion of how eventlet patching worked :'( | 23:48 |
| clayg | https://gist.github.com/clayg/3b8fb343786efe1b1e34 | 23:48 |
| clayg | ^ redbo | 23:48 |
| clayg | still waiting on the eventletdev ML post to show up in the archive so i can link that too | 23:51 |
| anteaya | clayg mattoliverau yes thanks for taking a look at that for me | 23:54 |
| anteaya | sorry I'm at the neutron mid-cycle | 23:54 |
| anteaya | about to leave for dinner will be looking at this again in about 16 hours | 23:55 |
| anteaya | any thoughts appreciated | 23:55 |
| *** esker has quit IRC | 23:57 | |
| clayg | here we go -> https://lists.secondlife.com/pipermail/eventletdev/2015-June/001213.html | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!