| *** dmsimard is now known as dmsimard_away | 00:11 | |
| *** abhirc has joined #openstack-swift | 00:15 | |
| *** jasondotstar has joined #openstack-swift | 00:16 | |
| notmyname | is it possible to encode json in a streaming fashion? eg do I have to read a 5GB chunk of data into memory in order to serialize it to json? | 00:23 |
|---|---|---|
| acorwin | notmyname: yes, it is possible to do that in a general sense | 00:23 |
| notmyname | even with arbitrary data? | 00:24 |
| acorwin | notmyname: sure. i suppose an especially pathological dataset could be pretty bad | 00:24 |
| acorwin | notmyname: i.e. if your data is `["a 5GB string"]` then you might not be very happy | 00:24 |
| *** yuanz has quit IRC | 00:25 | |
| notmyname | I want the result to be {'a': <contents of a.dat>, 'b': <contents of b.dat>} | 00:25 |
| acorwin | notmyname: i have actually encountered other people with this problem before! python does not seem to have a generally-agreed-upon streaming json encoder/decoder. | 00:25 |
| notmyname | oh good :-) | 00:25 |
| acorwin | notmyname: i wrote a streaming decoder but i'm afraid i dont have a streaming encoder solution for you | 00:26 |
| *** oomichi has joined #openstack-swift | 00:26 | |
| acorwin | notmyname: OTOH how are you going to stream in data and get python objects out without pulling in all the data beforehand? | 00:26 |
| notmyname | the API working group is proposing that the response to a GET request would be {'data': <data stuff>, 'metadata': {<metadata dict>}} | 00:26 |
| * acorwin cheats and goes over to notmyname's desk | 00:27 | |
| notmyname | :-) | 00:28 |
| torgomatic | well that's extensible; if I ever need metametadata, I'm not constrained by HTTP /s | 00:29 |
| notmyname | torgomatic: actually, thats a feature they mention | 00:31 |
| * torgomatic explodes | 00:31 | |
| notmyname | hmm..maybe that was just in the review comments | 00:32 |
| notmyname | but it was mentioned :-) | 00:32 |
| *** abhirc has quit IRC | 00:34 | |
| notmyname | torgomatic: if you want to play along from home... https://review.openstack.org/#/c/141229/ | 00:36 |
| *** dmorita has joined #openstack-swift | 00:38 | |
| notmyname | torgomatic: what are we going to do about https://review.openstack.org/#/c/152283/ (keystone v3)? | 00:38 |
| torgomatic | Poe's law meets OpenStack | 00:38 |
| notmyname | :-) | 00:39 |
| *** jasondotstar has quit IRC | 00:41 | |
| torgomatic | notmyname: I'm not planning to work too hard on it, honestly. If it lands, it lands. If not, then not. I haven't got any customers asking me for keystone-v3 support. | 00:43 |
| torgomatic | Folks were just saying that it was difficult, so I thought I'd see what could be done. | 00:44 |
| acorwin | torgomatic: but does the spec cover meatdata? | 00:44 |
| torgomatic | acorwin: it had better! that's just table steaks. | 00:45 |
| acorwin | torgomatic: wow, you're in rare form | 00:45 |
| torgomatic | acorwin: certainly not well done | 00:47 |
| torgomatic | notmyname: so you're saying that https://review.openstack.org/#/c/141229/ shows how the sausage is made? | 00:48 |
| acorwin | torgomatic: you win. i'm at a natural disadvantage when it comes to meat puns. | 00:49 |
| notmyname | torgomatic: it's watching an open-source project turn into "Enterprise Software" before your very eyes | 00:49 |
| *** abhirc has joined #openstack-swift | 00:50 | |
| *** david-lyle is now known as david-lyle_afk | 00:57 | |
| *** NM has left #openstack-swift | 00:58 | |
| *** abhirc has quit IRC | 01:08 | |
| *** abhirc has joined #openstack-swift | 01:10 | |
| *** nellysmitt has joined #openstack-swift | 01:11 | |
| *** jrichli has joined #openstack-swift | 01:13 | |
| openstackgerrit | Madhuri Kumari proposed openstack/swift: Check for existence of swift servers binaries. https://review.openstack.org/91753 | 01:15 |
| *** abhirc has quit IRC | 01:15 | |
| *** nellysmitt has quit IRC | 01:16 | |
| *** jasondotstar has joined #openstack-swift | 01:24 | |
| *** gyee has quit IRC | 01:27 | |
| *** jamielennox is now known as jamielennox|away | 01:33 | |
| *** joeljwright1 has quit IRC | 01:39 | |
| *** joeljwright has joined #openstack-swift | 01:40 | |
| *** abhirc has joined #openstack-swift | 01:45 | |
| *** jasondotstar has quit IRC | 01:50 | |
| *** haomaiwang has joined #openstack-swift | 02:20 | |
| *** stevage has joined #openstack-swift | 02:20 | |
| stevage | hello, I'm running into some "service unavailable" errors when uploading using python swift-client, and wondering if anyone knows a workaround that will at least keep trying each file, rather than simply skipping it | 02:21 |
| stevage | some context about the problem, including logs of the output: http://pastebin.com/G51WCXJK | 02:23 |
| notmyname | mattoliverau: ^^ you around? I gotta run upstairs for dinner | 02:32 |
| mattoliverau | notmyname: yup, what's up? | 02:33 |
| notmyname | mattoliverau: meet stevage. stevage meet mattoliverau. looks like you're both in Oz :-) | 02:33 |
| mattoliverau | Yay! Hay stevage | 02:34 |
| mattoliverau | Sorry been at lunch | 02:34 |
| stevage | hi matt! | 02:35 |
| stevage | yeah, I'm at Melbourne Uni, trying to use vicnode | 02:35 |
| *** abhirc has quit IRC | 02:36 | |
| stevage | I do sysadmin stuff for researchers, use NeCTAR a fair bit, but to be honest I'm really struggling with swift | 02:36 |
| stevage | what I'm trying to do atm is just upload a truckload of data, and have a really simple web interface to allow access to the files. after that I want to get fancier, but that's stage 1 | 02:38 |
| stevage | it should all be publicly accessible without authentication, which is maybe weird. | 02:38 |
| mattoliverau | Ahh nectar cool, I'm in Melbourne too! | 02:39 |
| mattoliverau | OK, let me read your paste first and we'll go from there :) | 02:40 |
| *** rmcall has joined #openstack-swift | 02:42 | |
| stevage | great | 02:44 |
| mattoliverau | So your having trouble with uploading then, are you scripting it to upload a file at a time using swiftclient? | 02:47 |
| stevage | at the moment I'm just using: swift upload CONTAINER * | 02:47 |
| stevage | so, using the CLI | 02:47 |
| mattoliverau | Intermittent failures from the Nectar swift server | 02:47 |
| stevage | yeah. I don't expect you to solve the server side issues :) | 02:48 |
| stevage | I've reported them, but no response yet | 02:48 |
| mattoliverau | Aww, why not :p | 02:48 |
| stevage | but swift client seems to just ignore failures, meaning some files don't get uploaded | 02:48 |
| mattoliverau | OK so you will probably have to script it to upload file by file then, on error, sleep, retry.. I can help if you want :) | 02:49 |
| mattoliverau | Though it could be more reliable when your not vpned in (knowing Australian internet :p ) | 02:50 |
| stevage | I'm not VPNed - just accessing swift directly from home | 02:51 |
| stevage | hmm. ok, to check whether I'm even going down the right path, what method would *you* use for uploading so much data? | 02:51 |
| stevage | and trying to get to this end result? | 02:52 |
| *** tellesnobrega_ has joined #openstack-swift | 03:03 | |
| mattoliverau | stevage: just looking some options on my local system | 03:05 |
| mattoliverau | stevage: I do more backend stuff so use curl, just looking at swift client :) | 03:05 |
| stevage | ah I see | 03:08 |
| stevage | well, look, if your suggestion is to just script it to handle errors, I guess I'll do that - but I have some other issues to pick your brains about too | 03:08 |
| stevage | for example - https://vault.melbourne.vicnode.org.au:8888/v1/AUTH_5efcf00a5431448586564b8341ba6a17/test_fromextdrive/# | 03:08 |
| stevage | it's asking for a username/password, and I don't know why (and I don't want it to) - and also I don't seem to be able to log in | 03:09 |
| *** nellysmitt has joined #openstack-swift | 03:12 | |
| *** nellysmitt has quit IRC | 03:16 | |
| mattoliverau | stevage: if your scripting 'swift upload CONTAINER <filename>', it will do the posix thing and return a 0 on success ($?) and a positive int on failure. so that's an easy check | 03:17 |
| mattoliverau | stevage: if its hitting swift, you;ll have to set the ACL on the container to be accessable without a username/password or use tempurls | 03:20 |
| mattoliverau | stevage: I'm looking at the swiftclient code, and there is a retry limit you can pass in, that might make it easier :) | 03:23 |
| mattoliverau | Look me see if I can get it to work. | 03:23 |
| mattoliverau | stevage: -R | 03:24 |
| mattoliverau | swift -R <num> upload CONTAINER * | 03:25 |
| mattoliverau | stevage: ^^ | 03:25 |
| stevage | oh, I didn't see that! | 03:27 |
| mattoliverau | stevage: me either ;) | 03:27 |
| stevage | what do you mean, set ACLs on containers, exactly. I have a bunch of commands that I've been using but I'm not certain what each one means | 03:27 |
| stevage | I've been using this to make files readable: swift post -r '.r:*' $CONTAINER | 03:28 |
| stevage | which I think sets each *file* publicly readable, but doesn't change anything about the container as a whole | 03:28 |
| stevage | I think I previously used temp urls but I don't quite see how they fit into the picture. I used a command like this: swift post $CONTAINER -m "Temp-URL-Key:" | 03:29 |
| mattoliverau | stevage: .r:* is access to files, and rlistings is for container listing | 03:38 |
| mattoliverau | stevage: for tempurl, you need to set a key and then once the key is added you can generate a tempurl | 03:39 |
| mattoliverau | stevage: sorry .rlistings | 03:39 |
| stevage | container listing means allowing people to view the contents of the container? | 03:39 |
| mattoliverau | stevage: yup, so: swift post -r '.r:*,.rlistings' | 03:42 |
| mattoliverau | will give everyone read access to listing the container and accessing the files. | 03:42 |
| mattoliverau | *reading the files | 03:42 |
| stevage | so swift will generate a pseudo directory listing HTML page? | 03:45 |
| stevage | hmm...actually it seems to generate XML | 03:45 |
| stevage | https://vault.melbourne.vicnode.org.au:8888/v1/AUTH_5efcf00a5431448586564b8341ba6a17/test_fromextdrive/# | 03:45 |
| mattoliverau | stevage: by default it will return just the list in test (try swift list CONTAINER). | 03:46 |
| stevage | so I will probably need swift-ui after all, unless there is something better. | 03:46 |
| mattoliverau | swift supports xml and json as well though: | 03:46 |
| stevage | well, I'm trying to get to a browsable HTML interface | 03:47 |
| mattoliverau | by appending ?format=xml or ?format=json | 03:47 |
| stevage | so re-enabling index.html (swift-ui), but I still get the login box. hmm | 03:47 |
| mattoliverau | I haven't used swift-ui :( | 03:52 |
| stevage | know any alternative ways to achieve something like this? | 03:53 |
| notmyname | hello again | 03:54 |
| notmyname | holy scrollback, batman | 03:54 |
| stevage | btw, just back on what you were saying about tempurls: the tempurl is per-file? or per container? and what key do I need to set? Temp-URL-Key on the container? | 03:55 |
| *** mitz_ has quit IRC | 03:56 | |
| notmyname | stevage: still looking to solve the upload problem? | 03:56 |
| *** mitz has joined #openstack-swift | 03:56 | |
| notmyname | stevage: (hopefully) https://swiftstack.com/blog/2015/01/29/swift-feature-highlight-tempurls/ should answer your questions about tempurls | 04:00 |
| mattoliverau | Yeah ^^ is a good article :) | 04:05 |
| mattoliverau | notmyname: aren't you suppose to be sleeping? | 04:05 |
| notmyname | it's only 8pm here. kids just went to bed | 04:05 |
| mattoliverau | notmyname: ahh, ok :) | 04:05 |
| stevage | notmyname: not sure. matt gave me a potential solution but I won't know for a while if it works | 04:05 |
| mattoliverau | stevage: so setting retries doesn't help? | 04:05 |
| stevage | I still don't have a solution to being able to present a list of all files in container to the world | 04:05 |
| notmyname | stevage: how many files are in the container? less than 10000? | 04:06 |
| mattoliverau | stevage: if you need finer grained control, you might need to script it (ie use sleep) | 04:06 |
| stevage | notmyname: not sure exactly, probably more than that. | 04:07 |
| mattoliverau | stevage: from the swift side .rlistings acl is who you give everyone access to the listing.. but I don't know what swift-ui does to this output | 04:07 |
| notmyname | stevage: you're looking for something like the apache auto-listings? or are you looking for something else? | 04:07 |
| mattoliverau | In a cafe, and need to move locations brb. | 04:08 |
| stevage | I started off thinking it would be easy to just do your standard HTTP directory browse | 04:09 |
| stevage | this is an open data project. I'm just dumping a bunch of files for the public to access | 04:09 |
| stevage | next I will do something fancier, but this was supposed to be an easy first step. | 04:09 |
| notmyname | stevage: ok. I think we should definitely be able to get that done (and pretty quickly | 04:10 |
| mattoliverau | stevage: what happens when you put the username and password into the login.. Does swift-ui display the objects? | 04:10 |
| stevage | there are at least 150,000 files | 04:10 |
| notmyname | stevage: what is swift-ui? looks like some javascript UI you've put into your container? | 04:10 |
| stevage | matt: no. I'm putting mytenant:myuserid as the user, and my keystone password as the password | 04:10 |
| openstackgerrit | Daisuke Morita proposed openstack/swift: Execute object daemons' tasks according to dir information https://review.openstack.org/141252 | 04:11 |
| stevage | btw, to be clear, there are 150k files *to be uploaded*, I just have a few dozen test files actually uploaded for now. | 04:12 |
| notmyname | stevage: ok. what's swift-ui. is that the javascript in your index.html? | 04:12 |
| notmyname | I think it's a red herring here | 04:13 |
| stevage | it's this: https://github.com/fanatic/swift-ui | 04:14 |
| stevage | an HTML/javascript file browser that doesn't seem to work for me. very happy to go with a different solution. | 04:15 |
| stevage | maybe building a static HTML dump would be simpler. | 04:15 |
| notmyname | interesting | 04:15 |
| notmyname | stevage: ok, I'd like to walk your through cleaning out some of the container metadata, setting it up properly, and seeing if that meets your needs | 04:15 |
| stevage | that would be awesome. any explanation along the way would be great, because I'm still a bit confused about some of the terms, the different kinds of passwords/ids/tokens etc. | 04:17 |
| notmyname | of course | 04:17 |
| stevage | I'm trying to write up a blog post as I go, to help other people doing something similar using VicNode | 04:17 |
| notmyname | stevage: first, let's clean out the container metadata. can you pastebin the results of `swift stat <your container name>` please? | 04:18 |
| stevage | http://pastebin.com/eH9S3YSc | 04:21 |
| notmyname | stevage: perfect, thanks | 04:21 |
| notmyname | from that, I see that you've got a world-readable container and you have an auto-index page turned on at index.html | 04:22 |
| notmyname | this means that anyone can go to the URL without an auth token, and they'll be redirected to index.html | 04:22 |
| stevage | yup - still not sure what the world-readable container gives me apart from the ability to query it and get XML/json back | 04:22 |
| notmyname | stevage: oh, you'll see :-) | 04:22 |
| stevage | (which isn't, I think, super useful for my current needs) | 04:22 |
| stevage | yep | 04:22 |
| *** jyoti-ranjan has joined #openstack-swift | 04:23 | |
| notmyname | stevage: ok, so first, just to rewind a little of your previous work, let's remove the web-index metadata | 04:23 |
| notmyname | `swift post -m "web-index:" test_fromextdrive | 04:23 |
| stevage | yep I know how to do that | 04:23 |
| notmyname | then verify that it's gone with a `swift stat test_fromextdrive` | 04:23 |
| stevage | done | 04:24 |
| notmyname | great | 04:24 |
| notmyname | stevage: now, to confirm, you should see 2 things | 04:25 |
| notmyname | first, `curl -i https://vault.melbourne.vicnode.org.au:8888/v1/AUTH_5efcf00a5431448586564b8341ba6a17/test_fromextdrive/index.html` will give you the same results | 04:25 |
| *** rdaly2 has joined #openstack-swift | 04:25 | |
| notmyname | as before | 04:25 |
| *** rdaly2 has quit IRC | 04:25 | |
| stevage | yep | 04:26 |
| notmyname | and taking off the "index.html" will .... hmm..return xml | 04:26 |
| *** rdaly2 has joined #openstack-swift | 04:26 | |
| notmyname | ok | 04:26 |
| notmyname | so now, there's one other piece of metadata to set | 04:26 |
| * notmyname is doing it first locally to verify | 04:28 | |
| stevage | hey, I jumped ahead a bit and got a kind of static directory listing working: https://vault.melbourne.vicnode.org.au:8888/v1/AUTH_5efcf00a5431448586564b8341ba6a17/treetest/tree.html | 04:28 |
| notmyname | oh that's good | 04:28 |
| stevage | yeah maybe generating static HTML is the easiest way here | 04:28 |
| notmyname | ummmm | 04:29 |
| notmyname | that's really .... odd | 04:29 |
| stevage | oh? | 04:29 |
| stevage | I did that like this: tree -H $STORAGEURL/$CONTAINER > tree.html && swift upload $CONTAINER tree.html && swift post --read-acl '.r:*,.rlistings' $CONTAINER | 04:29 |
| notmyname | so your test content is the contents of one of my github repos? | 04:29 |
| stevage | yours? heh. to be honest I don't remember what those files are | 04:30 |
| stevage | they were just the nearest thing to hand | 04:30 |
| notmyname | lol | 04:30 |
| notmyname | https://github.com/notmyname/python_scripts | 04:30 |
| stevage | what do they do? I must have downloaded it last time I was mucking around with this stuff a month ago | 04:30 |
| stevage | that's really .... odd | 04:31 |
| stevage | oh! | 04:31 |
| notmyname | super old one-off stuff I wrote | 04:31 |
| stevage | now I remember | 04:31 |
| stevage | you must be the author of the blog post I tried to follow, about uploading stuff to swift using a python script | 04:31 |
| notmyname | heh, yeah | 04:31 |
| stevage | http://programmerthoughts.com/programming/quickly-uploading-to-cloud-files-part-2/ | 04:31 |
| notmyname | http://programmerthoughts.com/programming/quickly-uploading-to-cloud-files-part-2/ | 04:32 |
| notmyname | yup | 04:32 |
| notmyname | that's me | 04:32 |
| stevage | I had a bit of a go, then abandoned it as I don' think it was solving a problem that I had | 04:32 |
| stevage | and it just didn't seem to work. etc. | 04:32 |
| stevage | sorry for the mild freakout? :) | 04:33 |
| notmyname | no worries. just not what I expected :-) | 04:33 |
| notmyname | ok, back to getting mine working locally to make sure I don't say the wrong thing | 04:33 |
| *** silor has joined #openstack-swift | 04:34 | |
| notmyname | ah, ok. that's it | 04:34 |
| notmyname | now, let's set one more piece of container metadata | 04:34 |
| notmyname | `swift post -m "web-listings:true" test_fromextdrive | 04:35 |
| stevage | are all these metadata flags documented somewhere? | 04:36 |
| notmyname | yes. I'll point you to it in just a bit (but I'll also admit they aren't the most easily findable--I need to fix that) | 04:37 |
| notmyname | yay! you set it (I see because I refreshed my browser) | 04:37 |
| notmyname | stevage: ok, now you have swift's auto-listings working | 04:38 |
| stevage | weird, I don't see it? | 04:38 |
| stevage | I just get "no data received" | 04:38 |
| notmyname | https://vault.melbourne.vicnode.org.au:8888/v1/AUTH_5efcf00a5431448586564b8341ba6a17/test_fromextdrive/ | 04:38 |
| notmyname | works for me | 04:38 |
| stevage | oh, https | 04:38 |
| stevage | ok | 04:38 |
| stevage | looks awful. | 04:38 |
| stevage | :) | 04:38 |
| notmyname | lol | 04:38 |
| notmyname | yes it does :-) | 04:38 |
| stevage | meanwhile my treetest is looking better than ever: proper recursive listings: https://vault.melbourne.vicnode.org.au:8888/v1/AUTH_5efcf00a5431448586564b8341ba6a17/treetest/tree.html | 04:39 |
| notmyname | but, the point is that swift's listings are working without you doing anything else | 04:39 |
| stevage | any way to style it? | 04:39 |
| notmyname | yup | 04:40 |
| notmyname | `swift post -m "web-listings-css:<your css object>" test_fromextdrive | 04:40 |
| notmyname | then upload your css and you're good to go | 04:40 |
| notmyname | stevage: here's where all that is documented http://docs.openstack.org/developer/swift/middleware.html#staticweb | 04:41 |
| zaitcev | What's the objective here? Frankly I'm kinda mad that Apache does not produce nice machine-readable directory listings by default. Or, actually, at all. So apps cannot read my directories (video player cannot get a list of videos in a directory). | 04:41 |
| notmyname | stevage: you can also set custom error pages too | 04:41 |
| notmyname | zaitcev: I think stevage is just looking for a way to have a listing of links, ideally in a way that looks sortof pretty | 04:42 |
| stevage | zaitcev: my ultimate objective is to produce a cool web interface for a bunch of data google-streetview-like data from VicRoads. the short term objective is to serve it all up through a kind of FTP-like interface. | 04:42 |
| notmyname | "the goal is to send a spaceship to jupiter. but short-term, I'm just driving to lunch" | 04:43 |
| zaitcev | Hmm. Well, that sounds possibly useful. Man, that "OS/2 support" footer thogh... looks quaint :-) | 04:43 |
| notmyname | stevage: ok, so you've got the auto listings working. you're playing with the tree view | 04:44 |
| notmyname | stevage: so it sounds like you've got the basics done | 04:44 |
| stevage | great, I css-ed it! | 04:45 |
| stevage | yeah, I'm moving forward - so much for your help, both of you | 04:46 |
| notmyname | stevage: but the next step is to verify what happens after you add the 10001st object into that container | 04:47 |
| stevage | cool, you can even use an external CSS file - check it out now | 04:48 |
| mattoliverau | notmyname is the man :) nice work guys | 04:48 |
| notmyname | nice | 04:48 |
| stevage | is 10000 a magic number?? | 04:48 |
| notmyname | stevage: since the nectar swift cluster has a container listing limit of 10000 objects (the default), you'll have to build your own index with a listing after that | 04:48 |
| notmyname | basically, the "page size" for a listing is 10000 | 04:49 |
| notmyname | and swift's staticweb feature (which you're using) does the auto-listings for the first page | 04:49 |
| stevage | oh, and because the directories are pseudo-directories, every file will count, no matter how deeply nested | 04:49 |
| notmyname | I don't remember | 04:49 |
| stevage | ok, so probably the tree method is what I'll need | 04:49 |
| stevage | that is, generating that statically | 04:50 |
| notmyname | right | 04:50 |
| notmyname | which just means you add your own index file and then add the web-index container metadata back | 04:50 |
| stevage | yup | 04:50 |
| notmyname | stevage: anything else for now? | 04:51 |
| stevage | no, that's great. I really appreciate the support - was really confused and frustrated before. | 04:54 |
| notmyname | sorry about that. I'm glad it's working for you now | 04:55 |
| stevage | there just seems to be such a step up in complexity when switching from block storage to object store - much more than I expected | 04:55 |
| stevage | I guess it's mostly the fact that so many standard | 04:55 |
| stevage | tools suddenly don't work - scp etc | 04:55 |
| notmyname | it's a different way to think about the storage. ya. the different API is a big change | 04:55 |
| stevage | I don't normally use any of the OpenStack APIs directly - even though I use NeCTAR a lot, I just use the dashboard | 04:56 |
| stevage | but there just aren't enough tools for Swift yet | 04:56 |
| notmyname | I'm working on that. but yeah, there's a long ways to go | 04:57 |
| stevage | oh, great. what are you current projcets? | 04:57 |
| notmyname | initially working with backup tools and storage managers to make sure swift is supported as a first-class citizen. like commvault, veeam, storage made easy, and a lot of others | 05:01 |
| *** jrichli has quit IRC | 05:02 | |
| stevage | ah yep. how many people work on swift full time, do you know? | 05:03 |
| *** oomichi has quit IRC | 05:04 | |
| notmyname | depends on what you mean by "work on", "full time", and "swift" :-) | 05:04 |
| notmyname | as in, ops running clusters, devs writing code, tools and/or products around swift, how many hours == full time :-) | 05:05 |
| notmyname | but in general, it's easier to say this: | 05:05 |
| stevage | yeah, fair point. I mean devs writing server side code and general purpose tools | 05:06 |
| notmyname | we've got about 200 people who have contributed to swift. and every six months there are about 50-60 individuals who contribute some amount of code | 05:06 |
| stevage | as opposed to companies hooking into swift for their own purposes or whatever | 05:06 |
| notmyname | we've got about 7 or 8 active core devs | 05:06 |
| zaitcev | My #1 request would be Duplicity. I probably need to get off my butt and contribute to it. | 05:07 |
| notmyname | and 5 or 6 companies who are super-active in the community (code and reviews) | 05:07 |
| *** ppai has joined #openstack-swift | 05:10 | |
| *** david-lyle_afk has quit IRC | 05:11 | |
| *** david-lyle_afk has joined #openstack-swift | 05:11 | |
| *** rmcall has quit IRC | 05:12 | |
| *** nellysmitt has joined #openstack-swift | 05:13 | |
| *** SkyRocknRoll has joined #openstack-swift | 05:16 | |
| *** nellysmitt has quit IRC | 05:17 | |
| *** stevage has quit IRC | 05:23 | |
| *** silor has quit IRC | 05:39 | |
| *** mtreinish has quit IRC | 05:40 | |
| *** silor has joined #openstack-swift | 05:41 | |
| *** nshaikh has joined #openstack-swift | 05:55 | |
| *** mtreinish has joined #openstack-swift | 05:56 | |
| openstackgerrit | John Dickinson proposed openstack/swift: added swift-ui browser to associated projects https://review.openstack.org/152799 | 06:12 |
| *** tellesnobrega_ has quit IRC | 06:32 | |
| *** tellesnobrega__ has joined #openstack-swift | 06:32 | |
| zaitcev | okay I need to sleep | 06:33 |
| zaitcev | It clearly is a ReST not Md | 06:34 |
| zaitcev | maybe install a rule of never review or propose patches after supper | 06:34 |
| *** zaitcev has quit IRC | 06:35 | |
| *** nshaikh has quit IRC | 06:50 | |
| *** nshaikh has joined #openstack-swift | 06:51 | |
| ppai | Is there any detailed doc about "application/directory" header ? Other than http://docs.openstack.org/api/openstack-object-storage/1.0/content/pseudo-hierarchical-folders-directories.html | 07:00 |
| *** nexusz99 has joined #openstack-swift | 07:04 | |
| *** chlong has joined #openstack-swift | 07:06 | |
| *** nellysmitt has joined #openstack-swift | 07:13 | |
| *** nexusz99 has quit IRC | 07:15 | |
| *** nexusz99 has joined #openstack-swift | 07:17 | |
| *** nellysmitt has quit IRC | 07:18 | |
| *** stevebennett has joined #openstack-swift | 07:20 | |
| ppai | found it here: http://loose-bits.com/2010/12/rackspace-cloud-files-and-pseudo.html | 07:22 |
| *** chlong has quit IRC | 07:28 | |
| *** stevebennett has quit IRC | 07:32 | |
| *** stevebennett has joined #openstack-swift | 07:34 | |
| *** nellysmitt has joined #openstack-swift | 07:42 | |
| *** stevebennett has quit IRC | 07:54 | |
| *** stevebennett has joined #openstack-swift | 07:56 | |
| *** stevebennett has quit IRC | 07:57 | |
| *** stevebennett has joined #openstack-swift | 08:01 | |
| *** stevebennett has quit IRC | 08:09 | |
| *** tdasilva has joined #openstack-swift | 08:10 | |
| tdasilva | eikke: hello, you there? | 08:17 |
| eikke | tdasilva: Not here. | 08:17 |
| *** acoles_away is now known as acoles | 08:22 | |
| *** geaaru has joined #openstack-swift | 08:25 | |
| *** jistr has joined #openstack-swift | 08:29 | |
| *** rledisez has joined #openstack-swift | 08:30 | |
| *** yuan has joined #openstack-swift | 08:34 | |
| acoles | tdasilva: good morning, i guess you are in brno? | 08:44 |
| tdasilva | acoles: hey! yes I am...red hat meetings here this week | 08:48 |
| acoles | tdasilva: your body clock is going to be very confused in sfo next week ;) | 08:50 |
| *** rdaly2 has quit IRC | 08:51 | |
| tdasilva | acoles: hehe, i imagine I will be drinking lots of coffee or maybe the EC reviews are going to be very exciting ;) | 08:51 |
| *** jordanP has joined #openstack-swift | 08:53 | |
| acoles | tdasilva: both i hope :) | 08:53 |
| tdasilva | acoles: when are you flying out? | 08:53 |
| acoles | sunday, arrive around 3.30pm | 08:53 |
| acoles | you? | 08:53 |
| tdasilva | same, should be there around 2:30 | 08:54 |
| tdasilva | if there's no snow storm in Boston ;) | 08:55 |
| tdasilva | they've been getting hammered for the past 10 days | 08:55 |
| acoles | tdasilva: so you escaped to the warmth of brno??? :) | 08:57 |
| tdasilva | acoles: lol, yes! it is a balmy 30F here, wearing tshirts and flip-flops | 08:58 |
| acoles | tdasilva: lol | 08:59 |
| *** jyoti-ranjan has quit IRC | 09:04 | |
| openstackgerrit | Merged openstack/swift: added swift-ui browser to associated projects https://review.openstack.org/152799 | 09:06 |
| *** silor1 has joined #openstack-swift | 09:10 | |
| *** silor has quit IRC | 09:11 | |
| *** jordanP has quit IRC | 09:18 | |
| *** jordanP has joined #openstack-swift | 09:25 | |
| openstackgerrit | Merged openstack/swift: Don't skip account acl functional tests https://review.openstack.org/152644 | 09:38 |
| *** ppai has quit IRC | 09:55 | |
| *** jordanP has joined #openstack-swift | 09:58 | |
| *** ppai has joined #openstack-swift | 10:08 | |
| *** ppai has quit IRC | 10:30 | |
| *** navid__ has joined #openstack-swift | 10:39 | |
| *** nshaikh has quit IRC | 10:40 | |
| *** jasondotstar has joined #openstack-swift | 10:46 | |
| eikke | tdasilva: now I am, sorry | 10:51 |
| tdasilva | hey! how's going? I was just wondering if you were going to be in SFO for the hackathon? | 10:51 |
| *** haomaiwang has quit IRC | 10:51 | |
| eikke | afraid not... it's not really closeby, and we didnt make any arrangements. not sure I could convince mgmt about going :) | 10:52 |
| eikke | just booked vancouver tickets though :) | 10:55 |
| *** jyoti-ranjan has joined #openstack-swift | 10:55 | |
| *** navid__ is now known as nshaikh | 11:03 | |
| tdasilva | eikke: alright...you had mentioned making some changes/cleanups to single_process work, feel free to send that whatever way you prefer | 11:06 |
| *** nexusz99 has quit IRC | 11:11 | |
| *** silor1 has quit IRC | 11:27 | |
| *** tellesnobrega__ has quit IRC | 11:43 | |
| *** stevebennett has joined #openstack-swift | 11:45 | |
| *** rdaly2 has joined #openstack-swift | 11:56 | |
| eikke | tdasilva: started dworking on that, will push to a git clone of swift and provide the link | 11:59 |
| tdasilva | eikke: great, thanks! | 11:59 |
| *** rdaly2 has quit IRC | 12:00 | |
| *** david-lyle_afk is now known as david-lyle | 12:01 | |
| *** Tyger_ has joined #openstack-swift | 12:13 | |
| *** Anticime1 has joined #openstack-swift | 12:15 | |
| *** jyoti-ranjan has quit IRC | 12:15 | |
| *** clayg_ has joined #openstack-swift | 12:17 | |
| *** ChanServ sets mode: +v clayg_ | 12:17 | |
| *** jasondotstar has quit IRC | 12:20 | |
| *** clayg has quit IRC | 12:20 | |
| *** Anticimex has quit IRC | 12:20 | |
| *** zul has quit IRC | 12:20 | |
| *** Tyger has quit IRC | 12:20 | |
| *** zackmdavis has quit IRC | 12:20 | |
| *** stevebennett has quit IRC | 12:21 | |
| *** stevebennett has joined #openstack-swift | 12:23 | |
| *** zackmdavis has joined #openstack-swift | 12:23 | |
| *** rmcall has joined #openstack-swift | 12:26 | |
| *** EmilienM|afk is now known as EmilienM | 12:27 | |
| *** zul has joined #openstack-swift | 12:28 | |
| *** dmorita has quit IRC | 12:28 | |
| *** david-lyle is now known as david-lyle_afk | 12:36 | |
| *** david-lyle_afk is now known as david-lyle | 12:36 | |
| *** silor has joined #openstack-swift | 12:41 | |
| *** david-lyle is now known as david-lyle_afk | 12:44 | |
| *** david-lyle_afk is now known as david-lyle | 12:45 | |
| *** silor has quit IRC | 12:48 | |
| *** jyoti-ranjan has joined #openstack-swift | 12:54 | |
| *** stevebennett has quit IRC | 12:57 | |
| *** esmute has quit IRC | 12:59 | |
| *** NobodyCam has quit IRC | 13:00 | |
| *** aix has joined #openstack-swift | 13:01 | |
| *** esmute has joined #openstack-swift | 13:03 | |
| *** NobodyCam has joined #openstack-swift | 13:07 | |
| *** dmsimard_away is now known as dmsimard | 13:09 | |
| *** dmsimard is now known as dmsimard_away | 13:13 | |
| *** rdaly2 has joined #openstack-swift | 13:19 | |
| *** nshaikh has quit IRC | 13:34 | |
| *** nshaikh has joined #openstack-swift | 13:35 | |
| *** nshaikh has quit IRC | 13:41 | |
| *** rdaly2 has quit IRC | 13:42 | |
| *** rdaly2 has joined #openstack-swift | 13:43 | |
| *** rdaly2 has quit IRC | 13:51 | |
| *** SkyRocknRoll has quit IRC | 13:55 | |
| *** jyoti-ranjan has quit IRC | 14:09 | |
| *** mahatic has joined #openstack-swift | 14:13 | |
| *** reed has joined #openstack-swift | 14:14 | |
| *** dmsimard_away is now known as dmsimard | 14:18 | |
| openstackgerrit | Joel Wright proposed openstack/python-swiftclient: Fix deleting SLO segments on overwrite https://review.openstack.org/152903 | 14:19 |
| *** erlon has quit IRC | 14:26 | |
| *** tdasilva has quit IRC | 14:29 | |
| *** rdaly2 has joined #openstack-swift | 14:31 | |
| *** zaitcev has joined #openstack-swift | 14:32 | |
| *** ChanServ sets mode: +v zaitcev | 14:32 | |
| *** bsdkurt has quit IRC | 14:48 | |
| openstackgerrit | Alistair Coles proposed openstack/swift-specs: Increasing ring partition power https://review.openstack.org/152935 | 14:55 |
| *** jwalcik has joined #openstack-swift | 14:57 | |
| *** bsdkurt has joined #openstack-swift | 15:01 | |
| *** EmilienM has quit IRC | 15:14 | |
| *** EmilienM has joined #openstack-swift | 15:16 | |
| *** EmilienM is now known as EmilienM|afk | 15:35 | |
| *** jrichli has joined #openstack-swift | 15:43 | |
| cschwede | acoles: nice spec! | 15:53 |
| briancline | indeed | 15:53 |
| openstackgerrit | Nicolas Trangez proposed openstack/swift: Add `swift-scality-backend` to associated projects https://review.openstack.org/152960 | 15:55 |
| *** erlon has joined #openstack-swift | 15:56 | |
| acoles | cschwede: hi! thanks. i'll be interested to hear people's thoughts. managing the transition procedure will be 'interesting' | 15:58 |
| acoles | are you heading to sfo next week? | 15:58 |
| cschwede | acoles: https://github.com/enovance/swift-ring-tool ;) | 15:59 |
| cschwede | acoles: yes, will be in SFO next week :) | 15:59 |
| ahale | oh thats a great spec at first glance | 16:01 |
| acoles | cschwede: yes, swift-ring-tool is referenced line 291 | 16:01 |
| cschwede | acoles: oh great, i didn’t read so far yet! | 16:02 |
| acoles | cschwede: but am i correct, swift-ring-tool requires services to be stopped? | 16:03 |
| cschwede | acoles: i run it on a dev cluster without stopping services, but the load due to replication is quite high | 16:04 |
| acoles | cschwede: yeah. the *key* idea in the spec is to avoid a replication storm by relinking to another filesystem branch <epoch>-objects etc. then flipping the switch once everything is double linked | 16:07 |
| acoles | as much as a switch can be flipped in a distributed system | 16:08 |
| *** rmcall has quit IRC | 16:08 | |
| notmyname | good morning | 16:10 |
| notmyname | cschwede: acoles: looks like you're doing crazy stuff there! changing ring part powers--I love it! | 16:11 |
| acoles | notmyname: oh my, is it that time already :o | 16:11 |
| cschwede | notmyname: morning! | 16:12 |
| cschwede | acoles: interesting stuff, i will read it in depth. looks promising | 16:12 |
| notmyname | for those of you in SFO next week: http://forecast.io/#/f/37.7771,-122.4196 | 16:13 |
| notmyname | (I don't know how to change that to non-USA degrees) | 16:13 |
| notmyname | just don't freak out about IT'S 53 DEGREES WE'RE GOING TO DIE | 16:14 |
| acoles | notmyname: 61 == 16 | 16:14 |
| notmyname | that's a good one to remember :-) | 16:14 |
| notmyname | one of the few I keep in my head is -40 == -40. isn't very commonly useful, though | 16:15 |
| acoles | notmyname: oh really, so my brother reported being somewhere at -40 last week and i wondered whether F or C, now I know! | 16:16 |
| zaitcev | I always subtract 32 and then divide by half, comes out close enough | 16:18 |
| *** EmilienM|afk is now known as EmilienM | 16:18 | |
| acoles | cschwede: thx & see you next week | 16:19 |
| cschwede | acoles: yep, see you in SFO! have a safe trip! | 16:20 |
| cschwede | notmyname: clicking on °C in the upper right corner? ;) | 16:21 |
| *** AbyssOne_ is now known as a1|away | 16:21 | |
| acoles | cschwede: lol | 16:21 |
| notmyname | cschwede: that's way up in the corner! I didn't see it :-) | 16:21 |
| cschwede | i think i need some t-shirts and shorts for next week. flip flops might come handy too | 16:21 |
| zaitcev | don't fall for it | 16:22 |
| zaitcev | It may be +15 during the day, but it's going to be windy and wet. Perfect for deceptive hypothermia. | 16:23 |
| notmyname | there's a local joke in SF about that. tourist come thinking "it's california, it will be hot". especially in summer (summer is the coldest part of the year in SF). then they get _really_ cold and go buy a giant fluffy "I <3 SF" sweatshirt/hoodie. so you see people walking around in shorts and giant sweatshirts/coats | 16:24 |
| acoles | zaitcev: sounds like every day of a british summer | 16:24 |
| mahatic | lol I just read this and was going to say the same | 16:24 |
| notmyname | lol | 16:25 |
| mahatic | I was like 12 C and shorts?! | 16:25 |
| zaitcev | People walk around in shorts when it's +3..7 around here when it's not windy. | 16:25 |
| mahatic | anything less than 20 here, i'm all layered up. sweaters and everything! | 16:25 |
| mahatic | wow | 16:25 |
| *** Anticime1 is now known as Anticimex | 16:29 | |
| openstackgerrit | Donagh McCabe proposed openstack/swift: Add multiple reseller prefixes and composite tokens https://review.openstack.org/137086 | 16:29 |
| cschwede | notmyname: good to know, i need some souvenirs for my kids. sweaters might be a good idea, and will be useful over here (it’s freezing here) :) | 16:31 |
| notmyname | anyone remember the DBConnectionError patch that landed a while back? I remember there was something, but I can't remember enough of it to find the commit now | 16:34 |
| notmyname | I had something unexpectedly come up at home, so today's team meeting will be max 30 minutes | 16:36 |
| notmyname | is donagh around? (acoles) | 16:39 |
| acoles | notmyname: don't see him in the channel, i'll nudge him | 16:40 |
| notmyname | I'm going through my emails from overnight | 16:41 |
| notmyname | he gave a great review on https://review.openstack.org/#/c/149930/. Just wanted to say that | 16:41 |
| acoles | notmyname: ok i'll relay that. yeah, we have been discussing it quite a bit, hope to discuss more with ho next week | 16:42 |
| notmyname | great | 16:43 |
| *** donagh has joined #openstack-swift | 16:44 | |
| acoles | donagh's here? | 16:44 |
| donagh | notmyname: acoles: just nudged me. Thanks for the comment. | 16:44 |
| notmyname | donagh: it's a great review. thanks for spending the time on it | 16:45 |
| notmyname | glange: that's funny https://twitter.com/greg_lange/status/563014418062376960 | 16:45 |
| *** gyee has joined #openstack-swift | 16:47 | |
| notmyname | if you're looking for an excuse to have a pint on the way home, look at https://review.openstack.org/#/c/141229/. It's been frustrating me more than it should this week | 16:48 |
| *** donagh has quit IRC | 16:48 | |
| *** annegent_ has joined #openstack-swift | 16:58 | |
| openstackgerrit | Mahati proposed openstack/swift: Tests for the base class of storage nodes https://review.openstack.org/151755 | 17:00 |
| *** atan8 has joined #openstack-swift | 17:04 | |
| *** rmcall has joined #openstack-swift | 17:06 | |
| *** mahatic has quit IRC | 17:09 | |
| acoles | notmyname: i see. imho /resourcetype/resource_id/metadata should link to /metadata/metadata_id which in turn may be extended /metadata/metadata_id/metadata | 17:10 |
| notmyname | acoles: naturally | 17:11 |
| notmyname | acoles: mostly the only reason I care is because I don't want to have an argument 6 months from now when we're asked "why doesn't swift implement this like everyone else?" | 17:12 |
| acoles | notmyname: yeah, i know, retrospective uniformity can be painful | 17:13 |
| notmyname | yes, I do support the goal of having as-close-as-possible APIs. I'd just prefer that the guidelines adopted aren't practically impossible for swift right at the start | 17:13 |
| notmyname | especially when I know that even if it starts as a guideline, that tends to change over time (see tempest, oslo.*, pbr, defcore, etc) | 17:14 |
| acoles | and who says metadata can't have metadata? | 17:14 |
| * acoles made that mistake in former life | 17:14 | |
| notmyname | it's metadata all the way down | 17:15 |
| acoles | indeed | 17:15 |
| notmyname | the contents of an object is really just an attribute of the canonical ring location, right? so that's metadata too!!! | 17:15 |
| *** gyee has quit IRC | 17:15 | |
| acoles | :) you're on my wavelength | 17:16 |
| notmyname | https://review.openstack.org/#/c/151755/ (from mahati) is a pretty short review) | 17:17 |
| notmyname | wait | 17:17 |
| acoles | notmyname: commit msg says its WIP | 17:18 |
| notmyname | ya, and I saw that it gives 100% coverage, but failed to see that it fails | 17:18 |
| notmyname | one small bug in it still | 17:18 |
| acoles | lol | 17:19 |
| *** mahatic has joined #openstack-swift | 17:20 | |
| notmyname | mahatic: we were just talking about you :-) | 17:20 |
| notmyname | or your patch | 17:20 |
| notmyname | one bug in it. I'm looking at it | 17:21 |
| *** atan8 has quit IRC | 17:22 | |
| mahatic | notmyname, ah okay. I typed up a msg about that here and just then my internet acted up. I just saw the logs and you didnt receive it | 17:22 |
| mahatic | my HP Pavilion 15-p045TX Notebook does not have and I can't find appropriate wifi drivers! I'm doing away with an internet dongle until I do some more research on those drivers | 17:24 |
| mahatic | data card* | 17:24 |
| *** rledisez has quit IRC | 17:25 | |
| *** jistr has quit IRC | 17:25 | |
| openstackgerrit | John Dickinson proposed openstack/swift: Tests for the base class of storage nodes https://review.openstack.org/151755 | 17:29 |
| notmyname | mahatic: how does that look? ^ | 17:29 |
| notmyname | mahatic: this is what I changed https://review.openstack.org/#/c/151755/2..3/test/unit/common/test_base_storage_server.py | 17:30 |
| *** gyee has joined #openstack-swift | 17:30 | |
| mahatic | notmyname, that looks cool. So, http://paste.openstack.org/show/166911/ specifying an empty list was throwing up the error? (You removed that) | 17:32 |
| notmyname | it wasn't needed. what I changed there (the important part) is that .allowed_methods is not a callable. it's a list (since that's what the @property returns) | 17:33 |
| notmyname | the assignment to an empty list didn't do anything since the variable was getting reassigned in the next line | 17:34 |
| mahatic | oh sorry! yes that's right | 17:34 |
| notmyname | so that's the important part about what I changed | 17:34 |
| mahatic | correct, because in my earlier test, I did that (declaring an empty list) and it was working | 17:34 |
| mahatic | yup | 17:34 |
| notmyname | the other stuff (s/Test/Fake/) is mostly style to be more similar with other test modules | 17:34 |
| mahatic | true. I was fascinated with my own names ;) | 17:35 |
| notmyname | I added the FakeANOTHER class just to be explicit about adding a method in a subclass. code had coverage anyway, but it made me feel better ;-) | 17:35 |
| mahatic | and the other test looks good. declaring FakeAnother | 17:35 |
| mahatic | :D | 17:35 |
| notmyname | oh and I removed the WIP comment in the commit message | 17:36 |
| notmyname | acoles: now you can look at that patch :-) | 17:36 |
| notmyname | mahatic: I think it would be good for you to start on the recon stuff as soon as possible. don't wait for this test patch to land | 17:36 |
| mahatic | notmyname, sure. Thanks much for the corrections :) | 17:36 |
| mahatic | yup | 17:36 |
| notmyname | I'm doing a remote meetup tonight. I need to work on my talk! | 17:38 |
| mahatic | ah | 17:38 |
| mahatic | can you quickly direct me on the recon stuff? Or do you want to drop an email on that sometime when you're free? | 17:38 |
| mahatic | notmyname, ^ | 17:39 |
| notmyname | mahatic: what do you know about recon so far? how far have you gotten in looking at it? | 17:39 |
| mahatic | notmyname, that I need to get info about the nodes on the ring and validate. What I'm not sure is what do I validate it with? | 17:42 |
| mahatic | how do I get to the ring and access the nodes on them? | 17:44 |
| notmyname | mahatic: you'll be adding functionality to swift/cli/recon.py | 17:46 |
| mahatic | correct | 17:46 |
| notmyname | that already knows how to grab a ring (and filter it) | 17:46 |
| notmyname | the "hosts" variable on line 909 | 17:46 |
| mahatic | oh cool. okay | 17:47 |
| mahatic | just checked that | 17:47 |
| notmyname | so, what you'll need to do is query each of those hosts (with an OPTIONS call) to see what kind of server they are | 17:47 |
| notmyname | and you need to compare it with self.server_type | 17:48 |
| notmyname | and report the results | 17:48 |
| mahatic | okay. For now, I need to only find out to see what kind of server that is and validate? | 17:49 |
| notmyname | the goal is so that I can do `swift-recon object --<some server checker command>` and find out if all of the hosts in my ring actually are object servers (or account or container, depending on the given server type) | 17:49 |
| notmyname | you'll have to add functionality to do an OPTIONS call (right now it only does GETs to the recon middleware--see Scout). and then you'll have to add code to aggregate the results in a nice way | 17:50 |
| mahatic | hmm okay. That should keep me rolling for now. I'll post back any questions. | 17:52 |
| notmyname | great | 17:53 |
| *** annegent_ has quit IRC | 17:53 | |
| mahatic | thanks! | 17:53 |
| notmyname | mahatic: I think you could have most of the OPTIONS call stuff done by friday. the reporting/aggregation and display probably will take longer, depending on how you choose to display it | 17:54 |
| mahatic | notmyname, oh okay. Sure, thanks for the cue. | 17:57 |
| *** rmcall has quit IRC | 17:58 | |
| *** rmcall has joined #openstack-swift | 18:01 | |
| openstackgerrit | Daniel Wakefield proposed openstack/python-swiftclient: Verify MD5 of uploaded objects. https://review.openstack.org/129254 | 18:02 |
| *** geaaru has quit IRC | 18:03 | |
| *** geaaru has joined #openstack-swift | 18:04 | |
| *** aix has quit IRC | 18:05 | |
| *** jasondotstar has joined #openstack-swift | 18:11 | |
| *** dkingshott has joined #openstack-swift | 18:12 | |
| acoles | notmyname: mahatic: review done! | 18:12 |
| *** jasondotstar has quit IRC | 18:14 | |
| *** acoles is now known as acoles_away | 18:15 | |
| notmyname | acoles_away: thanks | 18:15 |
| *** IRTermite has quit IRC | 18:30 | |
| *** jordanP has quit IRC | 18:41 | |
| *** clayg_ is now known as clayg | 18:41 | |
| *** IRTermite has joined #openstack-swift | 18:45 | |
| *** cutforth has joined #openstack-swift | 18:50 | |
| *** annegent_ has joined #openstack-swift | 18:54 | |
| *** kota_ has joined #openstack-swift | 18:55 | |
| notmyname | backblaze published their HDD data sets https://www.backblaze.com/hard-drive-test-data.html | 18:56 |
| *** annegent_ has quit IRC | 18:59 | |
| *** geaaru has quit IRC | 18:59 | |
| *** annegent_ has joined #openstack-swift | 19:00 | |
| notmyname | swift team meeting time in #openstack-meeting | 19:00 |
| *** acoles_away is now known as acoles | 19:00 | |
| openstackgerrit | Mahati proposed openstack/swift: Tests for the base class of storage nodes https://review.openstack.org/151755 | 19:02 |
| *** mahatic_ has joined #openstack-swift | 19:02 | |
| mahatic_ | acoles, thanks! changes made | 19:03 |
| acoles | mahatic: ok no problem | 19:03 |
| *** mahatic has quit IRC | 19:04 | |
| joeljwright | acoles: any chance you can look at this bugfix? https://review.openstack.org/#/c/152903 | 19:04 |
| joeljwright | acoles: I'm acutely aware that we're leaving segments lying around when overwriting SLOs at the moment | 19:05 |
| joeljwright | it's a swiftclient bug introduced with the service code | 19:05 |
| joeljwright | would be nice to get the fix in before another swiftclient release | 19:06 |
| *** bill_az_ has joined #openstack-swift | 19:07 | |
| acoles | joeljwright: ok i will try to look tomorrow. | 19:07 |
| joeljwright | acoles: thanks a lot | 19:08 |
| *** peluse_ is now known as peluse | 19:11 | |
| *** ChanServ sets mode: +v peluse | 19:11 | |
| acoles | joeljwright: i recall noticing that the manifest gets deleted before the segments, which struck me as vulnerable to failure after manifest is deleted. is this bug worse? | 19:12 |
| joeljwright | yes, this bug is much worse | 19:12 |
| joeljwright | whenever you overwrite an SLO at the moment its segments are ALWAYS left behind | 19:13 |
| joeljwright | swiftclient 2.3.0 and 2.3.1 both have this bug | 19:13 |
| acoles | joeljwright: oh :( right, so overwiting rather than deleting | 19:14 |
| joeljwright | yeah | 19:14 |
| joeljwright | acoles: upload creates the new segments and writes the new manifest, but it loses track of all the old segments | 19:15 |
| joeljwright | acoles: so they're never removed | 19:15 |
| acoles | joeljwright: ugh. ok, on the list for tomorrow! | 19:16 |
| joeljwright | acoles: yeah, this is why I thought it was worth bringing up | 19:16 |
| joeljwright | acoles: I've had a long day cleaning up orphaned segments :( | 19:17 |
| *** silor has joined #openstack-swift | 19:19 | |
| *** silor1 has joined #openstack-swift | 19:22 | |
| *** silor has quit IRC | 19:24 | |
| notmyname | joeljwright: thanks for bringing it up. I've got to go pick up my son from preschool. I'll try to take a look later today | 19:27 |
| joeljwright | notmyname: thanks, I'll get straight onto whatever changes you think are needed | 19:28 |
| *** gvernik has joined #openstack-swift | 19:31 | |
| *** jrichli has quit IRC | 19:37 | |
| *** jrichli has joined #openstack-swift | 19:38 | |
| *** kota_ has quit IRC | 19:51 | |
| *** jwalcik has quit IRC | 19:52 | |
| *** EmilienM is now known as EmilienM|afk | 19:53 | |
| *** acoles is now known as acoles_away | 19:54 | |
| *** dkingshott has quit IRC | 19:58 | |
| *** annegent_ has quit IRC | 19:58 | |
| *** jwalcik has joined #openstack-swift | 19:59 | |
| *** rmcall has quit IRC | 20:03 | |
| *** annegent_ has joined #openstack-swift | 20:04 | |
| *** rmcall has joined #openstack-swift | 20:05 | |
| *** gvernik has quit IRC | 20:05 | |
| *** dkingshott has joined #openstack-swift | 20:06 | |
| *** jwalcik has quit IRC | 20:16 | |
| *** lcurtis has joined #openstack-swift | 20:36 | |
| *** jwalcik has joined #openstack-swift | 20:37 | |
| *** nellysmitt has quit IRC | 20:56 | |
| *** cutforth has quit IRC | 21:00 | |
| *** mahatic_ has quit IRC | 21:02 | |
| *** zul has quit IRC | 21:08 | |
| *** zul has joined #openstack-swift | 21:08 | |
| *** annegent_ has quit IRC | 21:10 | |
| notmyname | hello again | 21:23 |
| zaitcev | why did we ever switch to Requests | 21:24 |
| notmyname | because ssl is hard? | 21:25 |
| zaitcev | But it's too hard even for developers of Requests, so what did we win | 21:25 |
| zaitcev | https://github.com/kennethreitz/requests/issues/2214 | 21:25 |
| *** EmilienM|afk is now known as EmilienM | 21:45 | |
| *** annegent_ has joined #openstack-swift | 22:04 | |
| *** annegent_ has quit IRC | 22:05 | |
| *** achhabra has joined #openstack-swift | 22:06 | |
| *** silor1 has quit IRC | 22:15 | |
| *** rdaly2 has quit IRC | 22:17 | |
| *** lcurtis has quit IRC | 22:20 | |
| dmsimard | Uh, this is pretty basic but it doesn't look like I can change a temp url key ? Once it's set through "swift post -m 'Temp-URL-Key:<key>'", either submitting a new one or a blank one doens't chant it. | 22:22 |
| dmsimard | change* | 22:23 |
| torgomatic | dmsimard: it should | 22:26 |
| *** stevebennett has joined #openstack-swift | 22:27 | |
| *** tellesnobrega_ has joined #openstack-swift | 22:27 | |
| dmsimard | torgomatic: Any way I'm not thinking of that would have a better chance at resetting it ? :p | 22:28 |
| dmsimard | Using 1.13 fwiw | 22:31 |
| torgomatic | dmsimard: works on my machine? http://paste.openstack.org/show/iSaPrBFQTNEAL0pLjG3S/ | 22:32 |
| torgomatic | that's against Swift master, but that hasn't changed in... roughly forever | 22:33 |
| torgomatic | it's just account metadata, and that's been basically untouched for years | 22:34 |
| dmsimard | I'm puzzled :( http://paste.openstack.org/show/167048/ | 22:35 |
| dmsimard | Doesn't look like there's any errors in the swift proxy logs | 22:35 |
| torgomatic | dmsimard: aha, someone did this: -m "X-Account-Meta-Temp-Url-Key:test" | 22:35 |
| torgomatic | so the HTTP header is X-Account-Meta-X-Account-Meta-Temp-Url-Key | 22:36 |
| torgomatic | which gets munged by the client into "Meta X-Account-Meta-Temp-Url-Key: test" | 22:36 |
| torgomatic | hm, wait | 22:36 |
| dmsimard | Well it's something I eventually tried - so there's both right now | 22:36 |
| torgomatic | there's also the fact that it didn't get set to "primero" | 22:36 |
| dmsimard | Temp-Url-Key *AND* Meta X-Account-Meta-Temp-Url-Key | 22:37 |
| torgomatic | huh... maybe nginx is screwing up the requests? can you talk directly to the proxy? | 22:37 |
| dmsimard | I'm just fooling/testing around right now but ideally I'd be able to remove both of these somehow | 22:37 |
| torgomatic | at least, I'm assuming nginx, since it says "Server: nginx" in the response | 22:37 |
| dmsimard | Probably, good idea. Let me check | 22:37 |
| * torgomatic has a meeting; back in a few | 22:39 | |
| dmsimard | There's something funny going on with nginx indeed.. Will investigate further | 22:48 |
| *** joeljwright has quit IRC | 22:53 | |
| *** nellysmitt has joined #openstack-swift | 22:57 | |
| *** stevebennett has quit IRC | 23:00 | |
| *** nellysmitt has quit IRC | 23:01 | |
| *** jwalcik has quit IRC | 23:01 | |
| *** echevemaster has joined #openstack-swift | 23:08 | |
| dmsimard | Well, not as bad as it could have been, looks like it's caching related | 23:08 |
| *** rdaly2 has joined #openstack-swift | 23:12 | |
| *** abhirc has joined #openstack-swift | 23:12 | |
| *** rdaly2 has quit IRC | 23:13 | |
| *** rdaly2 has joined #openstack-swift | 23:14 | |
| *** g4rg4m3|_ has quit IRC | 23:18 | |
| *** jasondotstar has joined #openstack-swift | 23:19 | |
| *** jrichli has quit IRC | 23:21 | |
| notmyname | peluse: yuan: https://software.intel.com/en-us/blogs/2015/02/03/using-multiple-backends-in-openstack-swift?utm_content=bufferb97f8&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer | 23:22 |
| *** reed has quit IRC | 23:24 | |
| torgomatic | dmsimard: are you aware that nginx is not recommended for use in front of Swift? | 23:30 |
| peluse | cool Yuan! | 23:31 |
| dmsimard | torgomatic: Yeah, that's why I'm using tengine :) | 23:31 |
| * torgomatic googles | 23:31 | |
| dmsimard | torgomatic: Google might land you on https://dmsimard.com/2014/06/21/a-use-case-of-tengine-a-drop-in-replacement-and-fork-of-nginx/ :P | 23:31 |
| torgomatic | Unbuffered upload, eh? That answers my one objection. :) | 23:32 |
| dmsimard | Essentially a fork of nginx started by Alibaba/Taobao with some extra features. | 23:32 |
| torgomatic | dmsimard: that's the fourth link for me; first is tengine.taobao.org | 23:32 |
| dmsimard | And that's the right link to be first :p | 23:32 |
| torgomatic | AFAIK, that's the only reason Swift and nginx don't get along. | 23:33 |
| dmsimard | It's been working great here, using it as a load balancer and ssl termination | 23:34 |
| dmsimard | Leveraging the caching to a certain extent but it's what's biting me right now | 23:34 |
| dmsimard | It shouldn't be caching the HEADs :) | 23:34 |
| *** jwalcik has joined #openstack-swift | 23:38 | |
| *** jwalcik has quit IRC | 23:38 | |
| *** rdaly2_ has joined #openstack-swift | 23:41 | |
| *** rdaly2 has quit IRC | 23:43 | |
| *** annegent_ has joined #openstack-swift | 23:55 | |
| *** rdaly2_ has quit IRC | 23:57 | |
| *** lcurtis has joined #openstack-swift | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!