Monday, 2014-12-08

mattoliveraunotmyname: gotta go to a meeting, be back in about 30 mins or so.00:01
notmynamemattoliverau: hmm...I don't see a way to give you access to that BP (yet another reason I don't like Launchpad)00:01
notmynamemattoliverau: the spec is more important than the BP00:01
mattoliveraunotmyname: OK, I'll just work on the Spec :)00:03
*** miurahr has quit IRC00:03
*** kopparam has quit IRC00:03
*** sandywalsh_ has quit IRC00:03
*** Masahiro has joined #openstack-swift00:07
*** Masahiro has quit IRC00:11
*** jasondotstar has joined #openstack-swift00:21
*** jasondotstar has quit IRC00:21
*** dmorita has joined #openstack-swift00:24
*** oomichi has joined #openstack-swift00:35
*** miurahr has joined #openstack-swift00:43
*** Masahiro has joined #openstack-swift00:44
*** miurahr has quit IRC01:01
*** kopparam has joined #openstack-swift01:02
*** kajinamit has quit IRC01:08
*** addnull has joined #openstack-swift01:11
*** infotection has joined #openstack-swift01:11
notmynameinteresting proposed http extension spec for reporting specifics of error messages http://tools.ietf.org/html/draft-ietf-appsawg-http-problem-0001:30
*** kopparam has quit IRC01:33
mattoliverauI'll have to add it to me reading list01:36
notmynameit's pretty short01:37
notmynameI think what's interesting to me is really just proposing a standard way to return a json doc about the error. defined content type and a few recommended keys and what they should have in them01:38
mattoliveraunotmyname: well, finally got my LCA flights, I'll be arriving on Saturday arvo so I can attend the ATC meet up :)01:56
*** Masahiro has quit IRC02:01
*** Masahiro has joined #openstack-swift02:04
*** fifieldt has joined #openstack-swift02:05
*** haomaiwang has joined #openstack-swift02:13
*** tkay has joined #openstack-swift02:25
*** kopparam has joined #openstack-swift02:29
*** kopparam has quit IRC02:34
*** Masahiro has quit IRC03:07
*** addnull has quit IRC03:07
*** SkyRocknRoll has joined #openstack-swift03:08
*** anticw_ has quit IRC03:23
*** anticw has joined #openstack-swift03:24
*** Masahiro has joined #openstack-swift03:30
*** david-lyle_afk has quit IRC03:31
*** bkopilov has quit IRC03:33
*** Masahiro has quit IRC03:35
*** Masahiro has joined #openstack-swift03:40
*** miqui_ has quit IRC03:40
*** david-lyle_afk has joined #openstack-swift03:43
*** Masahiro has quit IRC03:48
*** david-lyle_afk has quit IRC03:50
*** Masahiro has joined #openstack-swift03:52
*** david-lyle_afk has joined #openstack-swift04:02
*** david-lyle_afk has quit IRC04:02
*** Masahiro has quit IRC04:02
*** addnull has joined #openstack-swift04:04
*** Masahiro has joined #openstack-swift04:09
*** kopparam has joined #openstack-swift04:17
*** kopparam has quit IRC04:21
*** tkay has quit IRC04:49
*** david-lyle_afk has joined #openstack-swift05:01
*** bkopilov has joined #openstack-swift05:08
*** ipolyzos has quit IRC05:09
*** ipolyzos has joined #openstack-swift05:11
*** kopparam has joined #openstack-swift05:24
*** ppai has joined #openstack-swift05:26
*** SkyRocknRoll has quit IRC05:26
*** SkyRocknRoll has joined #openstack-swift05:27
*** nshaikh has joined #openstack-swift05:32
openstackgerritXiang Hui proposed openstack/swift: Fix getaddrinfo if dnspython is installed.  https://review.openstack.org/11661805:32
*** kopparam has quit IRC05:35
*** serverascode___ has quit IRC05:37
*** kopparam has joined #openstack-swift05:37
*** Masahiro has quit IRC05:38
*** Masahiro has joined #openstack-swift05:38
*** serverascode___ has joined #openstack-swift05:41
*** ipolyzos has quit IRC05:45
*** ipolyzos has joined #openstack-swift05:46
*** tkay has joined #openstack-swift05:54
*** SkyRocknRoll has quit IRC05:55
*** SkyRocknRoll has joined #openstack-swift05:55
*** tkay has quit IRC06:01
*** k4n0 has joined #openstack-swift06:01
*** addnull has quit IRC06:19
*** Masahiro has quit IRC06:29
*** kopparam has quit IRC06:30
*** Masahiro has joined #openstack-swift06:31
*** SkyRocknRoll has quit IRC06:33
xianghuinotmyname: hello, around?06:36
notmynamexianghui: hi06:37
xianghuiif you got a minute, can we talk a bit about https://review.openstack.org/#/c/116618/06:37
xianghuinotmyname: hello : )06:37
xianghuiI replied inline06:37
*** kopparam has joined #openstack-swift06:38
notmynamexianghui: ah, yes. I haven't spend the time I need to fully look at that patch. my comment was one of the first things that jumped out06:38
xianghuinotmyname: ah , understand06:38
notmynamexianghui: I get your point about circular imports. but refactoring to avoid a whole new dependency just to know if  and env var as "yes" is a True value is worth it. phrased another way, I don't want a whole new dependency to determine that. especially since we already have a function to do it in swift06:39
xianghuinotmyname: so do you mind I write a simple func same as utils.config_true_value() in __init__.py to avoid new dependency?06:40
notmynamexianghui: however, I want to be careful to say that's just an implementation detail on the patch. it's important, but thank you for working on this issue!06:40
notmynamexianghui: is there a way to refactor so you don't have to have the code in __init__.py?06:40
xianghuinotmyname: sorry, what you mean about 'refactor', I may not fully follow you06:41
xianghuiremove the change in __init__py instead of somewhere?06:41
notmynamexianghui: basically, yes. let me look at the patch a little more closely06:43
xianghuinotmyname: thank you very much! take you time :)06:44
notmynamexianghui: I'm not going to be able to give it a full review tonight (my local time), but the basic principle I'm saying is that adding a dependency is a Big Deal that we really want to avoid, if possible06:45
xianghuinotmyname: understand06:45
notmynamexianghui: so mostly I have a challenge to you: implement it using existing functionality instead of adding a dependency. I don't know exactly the best way to do that right now06:48
notmynamexianghui: but thank you for working on it. I love to see patches that help with ipv6 support!06:49
xianghuinotmyname: I would like to thank you for your efforts to review this patch today and future :)06:50
*** SkyRocknRoll has joined #openstack-swift06:51
*** sungju has quit IRC06:53
openstackgerritMatthew Oliver proposed openstack/swift-specs: Container sharding spec  https://review.openstack.org/13992106:56
notmynamexianghui: I'm curious. why is the import checking required in __init__.py rather than doing inside cname_lookup.py (ie where the current import checking is done)?06:59
xianghuinotmyname: they reason is the checking need to be done before anyplace import eventlet first07:02
xianghuiand for swift  services, most of them are importing swift/common /utils in the begginning07:02
openstackgerritMatthew Oliver proposed openstack/swift-specs: Container sharding spec  https://review.openstack.org/13992107:03
xianghuiso to support swift services could listening on ipv6 address, the greendns need to be disabled07:03
notmynameis EVENTLET_NO_GREENDNS an env var that eventlet respects?07:04
xianghuiso we added checking in __init__.py, reffering nova's implementation in nova/cmd/__init__.py07:05
mattoliverauI'm calling it a day, have a great night/day all!07:05
notmynameotherwise all I see is that cname_lookup now uses a ThreadPool07:05
mattoliveraunotmyname: get some sleep07:05
notmynamemattoliverau: ;-)07:05
notmynamemattoliverau: still gotta finish my preso07:05
notmyname(I'm about 2/3 done)07:06
mattoliveraunotmyname: you'll need extra coffee tomorrow!07:06
xianghuiyep, if setting EVENTLET_NO_GREENDNS this evironment variable, the greendns will be disabled, and ipv6 addr will be working07:06
xianghuimattoliverau: night07:06
notmynamexianghui: ah, interesting07:06
xianghuinotmyname: there are two changes: 1. set env EVENTLET_NO_GREENDNS=yes, if eventlet has been imported before and dnspython is used07:07
notmynamexianghui: ok but all you're doing is warning. and telling the user to set the env var before running the code07:07
xianghui2. the cname_lookup is used for using a ThreadPool to improve performance which down by disable greendns07:07
notmynameyes, you're doing the ThreadPool, but that's not disabling anything in eventlet07:08
xianghuinotmyname: yep, set the env var will be working , the evenlet patch  will lookup this env var to decide whether use greendns07:08
notmynamewhich brings me back to wondering why you can't raise the warning in cname_lookup instead of the __init__07:08
xianghuihmm, let me explain it07:09
notmynamewhich would allow easy removal of the dependency07:09
notmynamethanks. clearly I'm being slow tonight ;-)07:09
xianghui:) I am slow a bit today, still in weekend mode07:09
xianghui1. we can ignore the cname_lookup first07:10
xianghui2. what we wantis mainly in the __init__07:10
ahaleI hate that 2/3 point in pressos, where you have the cat memes in and need to write the actual content07:11
notmynameahale: lol. you've been looking over my shoulder?07:11
ahalehehe07:11
xianghui3. that is: check whether eventlet is imported and the env var is set no, if eventlet is already imported, then all we can do is just give a warnning07:11
xianghui4. but if eventlet is not imported yet, then we set the  env var to yes, to let evenlet not use greendns if it is imported later07:12
notmynamexianghui: ok. got it. it's #4 that I was missing07:12
xianghuiyep, so only then env var is set to yes, then canme_loolkup will be go to the added logistic07:13
notmynamexianghui: but let me propose something else. if a deployer is using cname_lookup, they are using eventlet. ie by the time the execution point gets to that middleware eventlet is already imported (in all likelihood). and the point is that this happens at startup, not when the first cname lookup request comes in07:14
notmynamexianghui: so at what point will we ever be using cname lookup and then later need to give a warning. it all happens at startup, which is effectively instant for the user. I'm thinking that maybe just the error (and resulting failure to start) is sufficient instead of needing to patch eventlet with an env var at runtime07:15
notmynameand as a follow-on question (after we get this one resolved), is it good to always use a thread pool for lookups? why not do that every time and remove the conditional?07:16
notmynamexianghui: the user story for you patch (IIRC) is that the user sets an env var to tell the process that IPv6 addrs should be expected. at that point it fails to start if eventlet is already imported. otherwise, it monkypatches eventlet to enable IPv6 DNS support07:17
xianghuinotmyname: yep07:18
notmynamexianghui: but what I'm learning now is that if the env var is set by the deployer explicitly, then it should Just Work (tm)07:18
xianghuinotmyname: what do you mean "[07:14] <  notmyname> | xianghui: but let me propose something else.."07:18
xianghuinotmyname: what you mean is that we remove the warnning in __init__, and maybe move it to cname_lookup?07:19
notmynameya, that's one option. let me type out what I think the options are07:19
xianghuiand hold thet env var set in __init__?07:19
notmyname1) check in __init__. this means that the deployers sets an openstack-specific env var to get the process to set an eventlet env var (assuming that swift.common is imported before eventlet)07:20
notmyname2) check at startup in cname_lookup. the openstack env var to say IPv6 support should be enabled. effectively, the deployer would need to set 3 env vars now. the openstack one and the eventlet one07:22
notmyname3) keep the checks in cname_lookup, and only check the IPv6 stuff if an IPv6 request is give. I don't know the detail here yet, but the goal is to return an error code (5xx?) to the client for IPv6 lookups instead of current behavior (crash?). also a log message could tell the deployer what to do07:23
notmynamethat's what I've got right now07:23
notmynamenumber 3 is weak, I'll admit07:24
notmynameoh. in number 2, the deployer has to set 2 env vars. not 307:24
notmynameso I'm wondering if we can keep the check in cname_lookup and keep it at one env var for the deployer to explicitly set (rather than swift setting it at runtime)07:25
xianghuithe background of chaning cname_lookup is that in the beginning we don't have the cname_lookup modification, that's to fix other review's concern about performance when enable IPv6 due to disable greendns07:25
xianghuiabout openstack-specific evn var , is that 'OS_ENABLE_IPV6' here?07:26
notmynameright07:26
xianghuisorry I am a bit confused about 2) and 3), what's the reason to check in  cname_lookup ?07:28
notmynamemattoliverau: still around?07:28
*** addnull has joined #openstack-swift07:30
*** stevage has joined #openstack-swift07:32
stevageg'day, I was venting about Swift on Twitter and @notmyname suggested I come here :)07:33
notmynamestevage: hi!07:33
stevageI'm trying to make a public repository of some open data, about a terabyte, with basically a simple HTML file browser07:34
notmynamexianghui: with 2, the idea would be to simply check at process startup (like in your patch) for the appropriate things. with 3 the idea is to only fail (ie log a warning) when an IPv6 request is handled.07:34
notmynamexianghui: but that's probably not possible at that point in the execution. (like I said, I'm fuzzy on that)07:34
notmynamestevage: cool07:34
stevageI'm not super familiar with the openstack techs (and especially their names), although I use the compute services a lot - on NeCTAR, in the Australian university sector07:34
*** addnull has quit IRC07:35
notmynamestevage: our resident Oz contributor just logged off for the night, but I'll be happy to help as I can until it gets too late for me07:35
stevageso first I tried uploading a lot of files with Cyberduck, then accessing them through Owncloud - but those two tools seem to treat directories differently. end result: no directories in Owncloud.07:35
xianghuinotmyname: thanks for your great advice, I will think about how to refactor it .:)07:35
notmynamestevage: yup, I'm familiar with NeCTAR. I know some of the people who manage it07:35
stevagethen I tried uploading files directly with Owncloud desktopclient, but that didn't work because for some reason it didn't support "hard links" from my external hard drive07:35
*** nellysmitt has joined #openstack-swift07:36
stevagecool, thanks. it's one of those things where I just don't really know what the right tools are and how they fit together.07:36
notmynamestevage: yup. I can understand your frustration with that.07:36
notmynamestevage: ok, I'll try to help07:36
stevagecurrently I can upload files with cyberduck, or with python swift client - so far so good.07:37
notmynamestevage: just so I know where to start, what have you tried so far? have you looked at any of the docs around openstack object storage (swift)?07:37
notmynameok07:37
stevagenow to make a website. I tried to follow the instructions here: http://docs.openstack.org/api/openstack-object-storage/1.0/content/static-website.html07:37
notmynameok07:37
stevageit shows the contents of some unidentified config file. what is it? where is it? etc.07:38
notmynamestevage: that config is for swift deployers. in your case, that's nectar07:38
stevageI don't know if I have the time to really learn Swift from the ground up - I'm trying to just solve this one use case. always a tough decision knowing how much time to invest learning it...07:38
*** nottrobin has quit IRC07:39
notmynamestevage: totally understand. if you've got about 30 minutes, I'll be we can get you up and running07:39
stevagehmm. so I jumped into trying to follow this: http://docs.openstack.org/api/openstack-object-storage/1.0/content/Examples_for_static_web-dle4025.html07:39
stevageI set the container world readable and that seems to work: https://vault.melbourne.vicnode.org.au:8888/v1/AUTH_5efcf00a5431448586564b8341ba6a17/newcontainer/index.html07:40
notmynamestevage: yup. that's an example of how to use the CLI to set metadata to do it07:40
stevagebut it doesn't seem to allow directory listing: https://vault.melbourne.vicnode.org.au:8888/v1/AUTH_5efcf00a5431448586564b8341ba6a17/newcontainer/07:40
notmynamestevage: `swift post -m 'web-listings: true' container`07:40
*** nottrobin_ has joined #openstack-swift07:40
stevagewhere I'm trying to get to is running something like this: https://github.com/fanatic/swift-ui - but preferably something better than "alpha quality" if there is such a thing.07:41
stevageyep, I did: $ swift post -m 'web:listings: true' newcontainer vagrant@vagrant-ubuntu-trusty-64:/vagrant/newcontainer$07:41
notmynameI just did this exact thing for my kid's soccer team a few weekends ago (upload a bunch of photos and server them as a static website)07:41
stevagethat doesn't seem to have enabled it.07:41
notmynameinteresting. I hadn't seen that swift browser before07:42
notmynamestevage: do you have access to curl?07:42
stevagehmm, seems to work in my browser with a trailing slash, but not without it.07:42
stevagehttp://vault.melbourne.vicnode.org.au:8888/v1/AUTH_5efcf00a5431448586564b8341ba6a17/newcontainer/07:42
stevagecurl? yep.07:42
stevagewhere I'm intending to end up is a small NeCTAR node running a simple (probably static) website that points to all the files on VicNode (Swift)07:43
notmynamestevage: ok, and do you know the auth token (ie can you stick it in an env var)?07:44
stevagewhat web UI did you use for your project?07:44
stevageyes, I think it is already - ie, I have downloaded the settings file and sourced it07:44
notmynamefifieldt: hi. I'm helping stevage with getting a static website set up on NeCTAR07:44
notmynamestevage: ( fifieldt is one of the original NeCTAR admins, although he's moved on since then)07:44
stevageoh yeah, I know Tom.07:45
stevage(like the rest of the openstack world I guess :)07:45
notmynameheh07:45
* fifieldt waves07:45
*** addnull has joined #openstack-swift07:46
stevageum, so this settings file sets OS_TENANT_ID and OS_PASSWORD, but I think that's not the same thing?07:46
notmynamestevage: I used a small python script to upload the files. and IIRC some curl to set the staticweb settings07:46
stevageah, but you built the static website manually? no templates even?07:47
notmynamestevage: `swift stat -v` should give you your token info (assuming your creds are set07:47
notmynamestevage: ya. another little script to generate the html. definitely nothing pretty ;-)07:48
stevagewhoa. auth token is massive. heh.07:48
notmynamestevage: heh. ok. throw that in an env var. $TOKEN maybe? that way I can give you some stuff to copy/paste07:48
stevageok, done - $TOKEN07:49
*** quack_quack_ has quit IRC07:49
notmynameto start with, `curl -I -H "X-Auth-Token: ${TOKEN}" http://vault.melbourne.vicnode.org.au:8888/v1/AUTH_5efcf00a5431448586564b8341ba6a17/newcontainer/`07:49
notmynamethat will do a HEAD request to that container and return the headers. you'll be able to see the metadata that is set07:49
stevageit said empty reply from server07:50
stevage$ curl -I -H "X-Auth-Token: ${TOKEN}" http://vault.melbourne.vicnode.org.au:8888/v1/AUTH_5efcf00a5431448586564b8341ba6a17/newcontainer/ curl: (52) Empty reply from server07:50
notmynamehmm...I used the URL you had above. maybe you need https?07:50
stevagesweet. HTTP/1.1 204 No Content X-Container-Meta-Web: listings: true07:51
*** nottrobin_ has quit IRC07:51
notmyname"-web: listings: true" that seems off07:51
*** miqui has quit IRC07:51
stevageso what are we accessing here - Swift's HTTP API? this does something the swift python client can't?07:51
*** quack_quack_ has joined #openstack-swift07:52
notmynamestevage: yes, this is Swift's API. doing with curl allows us to be more explicit. good for debugging07:52
stevageah, I think I typoed earlier07:52
stevageI set "web:listings" instead of "web-listings"07:52
notmynameah. that's it07:53
*** bkopilov has quit IRC07:53
notmynameok, here's the new thing to copy/paste07:53
stevageHTTP/1.1 204 No Content X-Container-Meta-Web-Listings: true X-Container-Meta-Web: listings: true07:53
stevagenow:07:53
notmynameah, ok :-)07:53
notmynameway ahead of me :-)07:53
stevagestill not browsable07:53
*** nottrobin_ has joined #openstack-swift07:54
stevage(yeah, I'm actually pretty comfortable with command line etc, run a lot of linux servers - I've just never really used openstack from cli)07:54
notmynamestevage: you have the x-container-meta-web-index value set?07:54
stevageI do now!07:55
notmynamestevage: ok, it's because you have that set to index.html and you have an index.html object in that container07:55
stevageok, so now it's browsable with trailing /07:55
notmynamestevage: yup. I see it07:56
stevageah ok, makes sense now.07:56
stevageI can either have my own index.html or allow the default browsing thing.07:56
notmynamestevage: right07:56
stevageso...the default directory listing is standard swift?07:56
stevagebut I can CSS-ify it?07:57
notmynamestevage: the auto-generated listing is the bare-bones copy of apache's index functionality07:57
notmynameya, everything you're using there is included in "standard swift" (ie it's all upstream functionality07:57
stevagecool. and it supports the pesudo-directory break-on-slash thing07:57
stevagesweet, got some basic CSS going.08:00
notmynamestevage: ya. swift supports breaking on arbitrary delimiter characters. and the "staticweb" functionality uses the / to break it up08:00
notmynameI'm curious about the need for the trailing / on the listing08:01
notmynameshoud return a 301 redirect in that case08:01
fifieldtcould be load balancer08:01
fifieldt?08:01
notmynameah. could also be my browser cache. curl says 301 there08:01
notmynamesee? curl is good for debugging :-)08:02
stevageshouldn't be a problem for me, I'll have an nginx in front of it I think08:02
notmynamestevage: so what else do you need? how can I help?08:02
notmynamestevage: for serving the content? or uploading?08:02
stevageactually you've solved my biggest blockers, so that's awesome08:02
notmynamegreat!08:02
stevageI'll like have nginx for serving content. There'll be some other static stuff that I'll just serve from a normal file store I think.08:03
notmynamehttp://adrift.org.au is pulling content directly from the NeCTAR Swift cluster. I'd love to see your site when it's put together (if it's a public thing). I'm always looking for end-user stories :-)08:03
stevageyeah, it's very public.08:03
stevageIt's an open data project for VicRoads, our road management authority08:04
stevagesimilar to Google StreetView, but a different tech08:04
stevagethey send a car out along the whole network every 2 years, taking 5 photos every 20 metres08:04
stevagethey've never had a way to share the images with anyone08:04
notmynameso all those photos are what you're putting in swift?08:04
stevageyeah. there's a lot of associated raw data, too.08:05
notmynamethat's so cool!08:05
stevagewell, metadata08:05
stevageso also I have to build a kind of map interface to it08:05
stevageI think I know how all the pieces fit together now08:05
stevagejust afraid it might be too big for me to get done08:06
notmynameplease let me know how I can help. you've obviously found me on twitter, and I'm in here all the time. we also have people in all kinds of timezones in here (although mostly the USA). mattoliverau is in sydney though08:06
notmynametoo big? ie just to upload it all?08:06
stevageno, project too big for me to code in the time I have08:07
stevagesize is not a huge problem, althoug for some reason upload is pretty slow08:07
stevageis swift support your day job, or...?08:08
notmynamestevage: very special case, but maybe https://github.com/notmyname/directory_uploader will help you. also, I have an older blog post on that topic http://programmerthoughts.com/programming/quickly-uploading-to-cloud-files-part-2/08:08
notmynamestevage: heh. seeing as it's midnight in san francisco, I guess it's my night job too ;-)08:08
notmynamestevage: I'm one of the swift contributors08:08
*** bkopilov has joined #openstack-swift08:08
* fifieldt nods at notmyname's modesty08:09
notmynamestevage: and I like to help people out to get the to use swift08:09
notmynamestevage: if you're going to LCA in auckland next month, I'll be there08:10
stevagesadly not08:10
stevageI'd never heard of it until a few months ago, but apparently everyone I know goes.08:10
stevageok, so your python script is faster than using "swift upload ..."?08:10
stevageand if so, how come?08:11
stevageCloud Files = Rackspace's Swift service?08:11
*** Masahiro has quit IRC08:11
notmynamestevage: the CLI does "helpful" things like a HEAD request for every object you want to upload. mine doesn't. the blog post talks about another way to make things a lot faster: use concurrency08:12
stevageoic, cloud files also has a CDN08:12
notmynamestevage: ya. swift was originally written at rackspace as the engine for the cloud files product. I used to work there.08:12
stevageoh, cool. so to use your script is it just the HTTP endpoint I have to change? everything else is the same b/w rackspace and nectar08:13
stevage?08:13
stevageHTTPS rather08:13
*** rledisez has joined #openstack-swift08:14
notmynamestevage: should be. no warranties and I've not tested that, etc etc. but yeah. rackspace cloud files == swift. and nectar == swift.08:14
notmynamestevage: yup. I just checked. nectar is running a recentish version of swift (`curl https://vault.melbourne.vicnode.org.au:8888/info | python -m json.tool`)08:15
*** jistr has joined #openstack-swift08:16
*** Masahiro has joined #openstack-swift08:16
notmynamecompare at rackpsace (`curl https://storage101.dfw1.clouddrive.com/info | python -m json.tool`)08:17
* notmyname needs to finish writing this presentation08:17
notmynamestevage: please let me know where else I can help point you in the right direction08:18
stevageseriously, you've been a huge help - was really frustrated, and now I'm on the way forward again08:18
stevage(and me too - only I need to *start* writing a presentation for tomorrow morning. ulp.)08:18
notmynameyes, but my presentation starts in just under 10 hours ;-)08:19
stevage15 here. :)08:21
stevagegood luck!08:22
notmynamethanks. you too08:24
*** addnull has quit IRC08:24
notmynamehas anyone seen or used https://github.com/wizzard/SwiftFS ? the author's contact link is mega.co.nz08:44
*** addnull has joined #openstack-swift08:45
*** jordanP has joined #openstack-swift08:46
*** nellysmitt has quit IRC08:49
*** exploreshaifali has joined #openstack-swift08:50
*** jordanP has quit IRC08:51
*** nellysmitt has joined #openstack-swift08:52
*** nellysmitt has quit IRC08:54
*** nellysmitt has joined #openstack-swift08:56
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements  https://review.openstack.org/8873608:59
*** acoles_away is now known as acoles08:59
*** jordanP has joined #openstack-swift09:03
*** Masahiro has quit IRC09:05
*** stevage has quit IRC09:06
*** Masahiro has joined #openstack-swift09:08
*** nellysmitt has quit IRC09:12
*** exploreshaifali has quit IRC09:12
*** nellysmitt has joined #openstack-swift09:18
notmynameok, presentation done. time to go to bed09:19
*** jordanP has quit IRC09:21
*** Roland has joined #openstack-swift09:21
*** oomichi has quit IRC09:22
*** jordanP has joined #openstack-swift09:36
*** kopparam has quit IRC09:37
*** addnull has quit IRC09:41
*** nellysmitt has quit IRC09:45
*** aix has joined #openstack-swift09:46
*** dmorita has quit IRC09:46
*** Masahiro has quit IRC09:56
*** Masahiro has joined #openstack-swift10:00
*** kopparam has joined #openstack-swift10:05
*** Masahiro has quit IRC10:09
*** jordanP has quit IRC10:17
*** nellysmitt has joined #openstack-swift10:28
*** jordanP has joined #openstack-swift10:29
*** bkopilov has quit IRC10:45
*** addnull has joined #openstack-swift10:46
*** jistr has quit IRC10:51
*** erlon has joined #openstack-swift11:02
*** kopparam_ has joined #openstack-swift11:02
*** kopparam has quit IRC11:03
*** jistr has joined #openstack-swift11:09
*** Masahiro has joined #openstack-swift11:10
*** ppai has quit IRC11:11
*** Masahiro has quit IRC11:14
*** ppai has joined #openstack-swift11:23
*** haomaiwang has quit IRC11:23
openstackgerritHisashi Osanai proposed openstack/swift: Allow hostnames for nodes in Rings  https://review.openstack.org/13315511:24
*** bkopilov has joined #openstack-swift11:27
*** addnull has quit IRC11:27
RolandHello I have a quick question but I don't know if this is the right place to do so but anyway here goes. Currently we are deploying a standalone swift cluster with keystone as the authentication service. My question is can I run to completely separte keystone servers? This meaning that even if the same project and username exists on both keystone servers it won't conflict on de swift cluster itself?11:33
*** yuanz has quit IRC11:42
*** yuanz has joined #openstack-swift11:43
acolesRoland: never tried it, I think you would need two instances of the authtoken middleware in the swift proxy server, one pointing to each keystone service, BUT I suspect that may not work because (looking at the code) the second authtoken will remove any ID attributes that the first may add to the request env11:53
acolesRoland: you could try to verify that in #openstack-keystone11:53
*** addnull has joined #openstack-swift12:02
*** ppai has quit IRC12:04
*** X019 has quit IRC12:08
RolandAcoles: Okay I will try it there as a side note I will use two different proxies each connecting to a different keystone server but using the same swift cluster. Do you think it will work?12:17
*** ppai has joined #openstack-swift12:18
*** cdelatte has joined #openstack-swift12:24
*** bkopilov has quit IRC12:27
*** jistr has quit IRC12:27
*** jistr has joined #openstack-swift12:28
*** tellesnobrega has joined #openstack-swift12:29
*** bkopilov has joined #openstack-swift12:31
*** silor has joined #openstack-swift12:32
acolesRoland: i guess that would work, but swift will not distinguish between users authenticated on the two keystones i.e. project:user X:Y in keystone A will gain access to the same swift account as project:user X:Y in keystone B. Even if you separate the account namespace prefixes in the service endpoint returned from each keystone service, that will not prevent users with same id/project accessing each others accounts i.e. its12:35
acolesnot secure.12:35
RolandAcoles: Okay thank you very much if have all the information I need right now!12:39
*** kopparam_ has quit IRC12:40
*** kopparam has joined #openstack-swift12:40
*** kopparam has quit IRC12:41
*** ppai has quit IRC12:41
*** miurahr has joined #openstack-swift12:43
*** Masahiro has joined #openstack-swift12:44
*** ppai has joined #openstack-swift12:46
*** Masahiro has quit IRC12:49
*** sandywalsh has joined #openstack-swift12:52
*** tellesnobrega has quit IRC12:53
*** exploreshaifali has joined #openstack-swift13:03
*** miurahr has quit IRC13:05
*** miurahr has joined #openstack-swift13:08
*** miurahr has quit IRC13:29
*** aerwin_ has joined #openstack-swift13:32
*** kopparam has joined #openstack-swift13:41
*** addnull has quit IRC13:42
*** bkopilov has quit IRC13:43
*** tdasilva has joined #openstack-swift13:43
*** kopparam has quit IRC13:48
*** kopparam has joined #openstack-swift13:53
*** addnull has joined #openstack-swift13:54
*** addnull has quit IRC13:54
*** SkyRocknRoll has quit IRC13:54
*** kopparam has quit IRC14:01
*** mahatic has joined #openstack-swift14:02
*** SkyRocknRoll has joined #openstack-swift14:06
*** ppai has quit IRC14:16
*** SkyRocknRoll has quit IRC14:25
*** nshaikh has quit IRC14:33
*** Masahiro has joined #openstack-swift14:33
*** Masahiro has quit IRC14:38
*** mahatic has quit IRC14:44
*** mahatic has joined #openstack-swift14:45
*** delattec has joined #openstack-swift14:59
*** cdelatte has quit IRC15:01
*** lpabon has joined #openstack-swift15:07
*** miurahr has joined #openstack-swift15:08
*** tellesnobrega has joined #openstack-swift15:12
*** nellysmitt has quit IRC15:12
*** miurahr has quit IRC15:13
*** exploreshaifali has quit IRC15:14
*** dmsimard_away is now known as dmsimard15:15
*** nellysmitt has joined #openstack-swift15:19
*** silor has quit IRC15:23
*** tellesnobrega has quit IRC15:28
*** kani has joined #openstack-swift15:30
ctennisportante: I wanted to talk a little more about this replicator sync stuff if you have a few minutes today15:33
*** k4n0 has quit IRC15:40
*** delatte has joined #openstack-swift15:41
*** acoles is now known as acoles_away15:42
*** delattec has quit IRC15:43
kaniHi... I am new to openstack-swift. I have configured swift as glance's banckend. glance and swift services are running in 2 different vms. While uploading an image to glance, it is uploaded to glance and then it is chunked and transferred to swift. Is there any configuration to upload directly to swift to avoid this 2 hops?15:45
*** sagar_nikam has joined #openstack-swift15:46
*** miurahr has joined #openstack-swift15:46
openstackgerritDaniel Wakefield proposed openstack/python-swiftclient: Fix misnamed dictionary key.  https://review.openstack.org/12957415:47
*** tellesnobrega has joined #openstack-swift15:49
openstackgerritDaniel Wakefield proposed openstack/python-swiftclient: Fix misplaced check for None in SwiftUploadObject.  https://review.openstack.org/13310715:51
*** aix has quit IRC15:55
dmsimardnotmyname: Haven't used SwiftFS but I've tried s3ql.16:00
*** david-lyle_afk is now known as david-lyle16:06
*** ashaw_ has quit IRC16:09
*** Masahiro has joined #openstack-swift16:22
*** exploreshaifali has joined #openstack-swift16:23
*** Masahiro has quit IRC16:26
*** lpabon has quit IRC16:32
*** jdaggett_ is now known as jdaggett16:37
*** Roland has quit IRC16:42
kaniI have configured swift as glance's banckend. glance and swift services are running in 2 different vms. While uploading an image to glance, it is uploaded to glance and then it is chunked and transferred to swift. Is there any configuration to upload directly to swift to avoid this 2 hops?16:45
kanican someone help me?16:46
*** exploreshaifali has quit IRC16:49
*** rdaly2 has joined #openstack-swift16:50
glange_kani: I don't know much about glance -- if you stick around, you might get an answer as people come onto channel and start their workday16:52
glange_kani: also, you might try asking on a nova channel, they might know the answer over there16:53
glange_#openstack-nova16:54
kani+glange_: Thank you16:57
*** sagar_nikam has quit IRC17:01
*** tkay has joined #openstack-swift17:05
*** kani has quit IRC17:07
notmynamegood morning17:12
*** tellesnobrega has quit IRC17:13
*** silor has joined #openstack-swift17:13
*** openstackgerrit has quit IRC17:19
*** openstackgerrit has joined #openstack-swift17:20
pelusegood morning17:33
*** bkopilov has joined #openstack-swift17:38
*** rledisez has quit IRC17:41
*** tab___ has joined #openstack-swift17:44
*** aerwin_ has quit IRC17:49
openstackgerritMahati proposed openstack/swift: OPTIONS verb implemented for object server. Note: Tests are not written yet. This is only an initial version for review.  https://review.openstack.org/14010318:00
mahaticnotmyname, good morning18:01
notmynamemahatic: good morning18:01
*** jwang__ has quit IRC18:02
mahaticnotmyname, just submitted an initial version of patch, let me know the review when you find time18:02
notmynamemahatic: will do18:03
*** jordanP has quit IRC18:09
*** Masahiro has joined #openstack-swift18:11
*** jistr has quit IRC18:11
*** jonb has joined #openstack-swift18:14
*** Masahiro has quit IRC18:15
*** abhirc has joined #openstack-swift18:24
*** shri has joined #openstack-swift18:33
tdasilvanotmyname, clayg: Hi, I've seen references in docs that a large object manifest file cannot be versioned, but while testing against master I'm able to version not only a segment but also the manifest file. Do you guys have any history on this? wondering if it's a bug that was introduced recently or if there's something else going on that is not obvious to me...18:37
* notmyname wonders if it was introduced when manifests were pulled to middleware18:38
tdasilvanotmyname: plus, I think the unit test for the case is broken: https://github.com/openstack/swift/blob/master/test/unit/proxy/test_server.py#L4252,L427318:47
*** nellysmitt has quit IRC18:52
*** lpabon has joined #openstack-swift19:02
notmynametdasilva: sorry. was in a meeting. out now19:10
tdasilvanotmyname: np19:10
tdasilvanotmyname: i'm still digging to see what I can find out...19:11
*** zaitcev has joined #openstack-swift19:15
*** ChanServ sets mode: +v zaitcev19:15
openstackgerritMahati proposed openstack/swift: OPTIONS verb implemented for object server. Many times new deployers get mysterious errors after first setting up their Swift clusters. Most of the time, the errors are because the values in the ring are incorrect (e.g. a bad port number). OPTIONS will be  https://review.openstack.org/14010319:19
*** aix has joined #openstack-swift19:19
*** zul has quit IRC19:25
notmynametdasilva: the issues IIRC is that you can't copy the manifest since that squashes it down to one object. so if a manifest refers to more than max_object_size, the versioning can't work. however, a manifest can refer to a versioned object. make sense?19:31
tdasilvanotmyname: oh...i understand why you wouldn't want to version the manifest file...just saying, i think it is at the moment....19:32
notmynametdasilva: thanks for looking.19:33
*** zul has joined #openstack-swift19:38
*** abhirc has quit IRC19:39
*** abhirc has joined #openstack-swift19:51
*** jwang has joined #openstack-swift19:54
*** Masahiro has joined #openstack-swift19:59
*** nellysmitt has joined #openstack-swift20:01
*** Masahiro has quit IRC20:04
*** gyee has joined #openstack-swift20:13
*** tdasilva has quit IRC20:36
*** nellysmitt has quit IRC20:37
portantectennis: I am available now20:47
*** silor has quit IRC20:51
*** tdasilva has joined #openstack-swift20:52
*** abhirc_ has joined #openstack-swift21:04
*** annegent_ has joined #openstack-swift21:04
*** abhirc has quit IRC21:06
*** annegent_ has quit IRC21:11
notmynamectennis: best. bug report. ever. thanks for including the builder files that actually exhibit the bug21:12
*** fifieldt_ has joined #openstack-swift21:15
mattoliverauMorning21:16
notmynamehi mattoliverau21:17
mattoliveraunotmyname: how'd the presentation go? And how much coffee have you had to drink to stay awake today :p21:18
notmynamemattoliverau: heh21:18
notmynameI hope it went well. haven't seen the recording yet21:18
*** fifieldt has quit IRC21:19
*** gyee has quit IRC21:32
*** tdasilva has quit IRC21:36
notmynamemattoliverau: FWIW https://www.youtube.com/watch?v=EUpuMTZOMVI&t=40m56s21:46
*** Masahiro has joined #openstack-swift21:48
mattoliverauNice, I'll give it watch :)21:49
*** Masahiro has quit IRC21:53
*** mahatic has quit IRC22:05
*** oomichi has joined #openstack-swift22:11
*** bill_az has joined #openstack-swift22:11
mattoliveraunotmyname: just finished it, you did well! Nice work22:15
bill_azquestion - looking at the Paris user survey.  Under compatibility API section, It shows 34% of production deployments using S3 compatibility API.  Does that mean that 34% of all swift deployments are using swift3 middleware?22:16
bill_azhttp://superuser.openstack.org/articles/openstack-user-survey-insights-november-201422:16
notmynamebill_az: I don't think you can draw that conclusion22:20
notmynamebill_az: "use s3" might mean some older nova and/or glance functionality. more of it is probably using swift3 with swift and using s3 clients22:21
bill_aznotmyname:  I was suspicious of such a large number.  Do you know if production deployments are using swift3 or is there another approach for this?22:21
notmynamebill_az: swift3 is the only compat thing I know of22:21
notmynamebill_az: on the other hand, I don't think the full set of swift deployments are represented in the survey22:22
bill_aznotmyname:  right, I'm sure it's not a very statistically correct sample.  But deployers/end users  that are using swift3 middleware find that it covers most of their requirements?  (it works for the most common use cases)22:23
notmynamebill_az: I think there's selection bias. the answer to the question you asked is "yes" because otherwise they wouldn't use it ;-)22:24
notmynamebill_az: but yeah, the more general question remains22:24
notmynamebill_az: swift 3 covers some basic functionality. but it's under active development to close some of the remaining gaps22:25
bill_azbill_az: yes, we did some testing of this and are working with Kota to update compatibilty matrix thats in Swift wiki, and hope to also assist with new features.22:26
notmynamenice22:26
bill_aznotmyname: but I wanted to see if you could help interpret the results in the fall survye22:27
notmynamebill_az: I don't have any more insights than you do22:27
notmynamebill_az: one thing I've speculated on in the past is that the use of swift3 might show that people are moving from s3 to swift22:28
bill_aznotmyname:  I would like to believe that too22:30
notmynamebill_az: one piece of evidence for this is that there are more s3 compat cluster reported in prod, even though there are the same number reported as POC. ie the fact that newer clusters seem to tend away from swift3 shows that s3 compat is an important bridge but not the key feature22:31
*** tellesnobrega has joined #openstack-swift22:33
bill_aznotmyname:  that surprised me, I would expect to see a larger POC  percentage, but what you say  makes sense22:33
bill_aznotmyname: And I would hope it's not the key feature; other features/properties of Swift provide the incentive to move22:34
notmynamebill_az: oh don't worry. that's just speculation layered on guesses, all mixed with hopes and dreams22:35
*** nellysmitt has joined #openstack-swift22:38
bill_aznotmyname:  bottom line, we're free to interpret the results in the way that makes us happy!  ;-) Thanks22:39
notmynamelol22:39
notmynameyes22:40
notmynamefifieldt_: ^^ you'll love that comment about the user survey ;-)22:40
*** nellysmitt has quit IRC22:43
ctennissorry portante I was in transit today, I'll hit you up tomorrow22:49
*** tellesnobrega has quit IRC22:49
*** sungju has joined #openstack-swift22:50
ctennisportante: sorry, I'm reading my history adn realize I meant to ping peluse originally!22:52
ctennisnotmyname: set my straight22:52
eikkenotmyname: I toook the liberty today to slightly edit the 'priority reviews' wiki page, add one a link to a minor review related to the splice patch22:57
eikkehope that's OK22:58
notmynameeikke: thanks for letting me know22:59
eikkey/w23:00
fifieldt_bill_az, it's important to read the start of the results before looking at the graphs and comparing them to last year23:00
fifieldt_in the most recent survey we basically only looked at stuff that had been updated since Feb or something23:01
fifieldt_I just woke up, but feel free to email me random things and I can look into them23:01
*** jonb has quit IRC23:04
bill_azfifieldt_:  ok, I have read the introduction, and I wasn't comparing with past surveys, but trying to understand compatibility api section, stating that 34% of production deployments  use s3 api.  That seems high for s3 & swift, but as nonmyname said, this number could also include glance and other projects23:09
fifieldt_ah,. sorry, my bad - early morning :)23:10
fifieldt_that number is primarily for swift23:10
fifieldt_I can try and do some correlation23:10
fifieldt_(ps the next version of the survey will have this directly correlated, I hope - as in it will only be shown for people who tick swift)23:11
bill_azfifieldt_: thanks, it would be really useful know how much of that is swift3 middleware23:12
fifieldt_so I did some very rough grepping, which is probably innacurate, but the lowest % I'd say for swift3 would be 20% of swift users23:15
fifieldt_but this was over a different dataset23:16
fifieldt_rather than the survey one23:16
fifieldt_a larger dataset23:16
fifieldt_but 20% seems like a decent low23:16
fifieldt_and 34 a decent high23:16
openstackgerritSamuel Merritt proposed openstack/swift: Improve object-replicator startup time.  https://review.openstack.org/14017823:26
*** annegent_ has joined #openstack-swift23:26
*** rdaly2 has quit IRC23:31
*** miurahr has quit IRC23:36
*** Masahiro has joined #openstack-swift23:37
*** Masahiro has quit IRC23:41
*** lpabon has quit IRC23:45
*** rebelshrug has joined #openstack-swift23:46
*** dmsimard is now known as dmsimard_away23:52
*** gyee has joined #openstack-swift23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!