| *** openstack has joined #openstack-api | 00:55 | |
| *** woodster_ has quit IRC | 01:31 | |
| *** Apoorva has quit IRC | 01:33 | |
| *** e0ne has joined #openstack-api | 06:25 | |
| *** e0ne has quit IRC | 06:31 | |
| *** fzdarsky has joined #openstack-api | 07:16 | |
| *** alex_klimov has joined #openstack-api | 07:45 | |
| *** cdent has joined #openstack-api | 08:16 | |
| *** lucasagomes has joined #openstack-api | 08:17 | |
| *** e0ne has joined #openstack-api | 08:18 | |
| *** e0ne has quit IRC | 08:25 | |
| *** e0ne has joined #openstack-api | 09:28 | |
| *** e0ne is now known as e0ne_ | 09:52 | |
| *** e0ne_ is now known as e0ne | 09:54 | |
| *** e0ne is now known as e0ne_ | 11:19 | |
| *** e0ne_ has quit IRC | 11:29 | |
| *** lucasagomes is now known as lucas-hungry | 11:32 | |
| *** e0ne has joined #openstack-api | 12:08 | |
| *** lucas-hungry is now known as lucasagomes | 12:35 | |
| *** woodster_ has joined #openstack-api | 12:40 | |
| openstackgerrit | Ryan Brown proposed openstack/api-wg: Add section clarifying PUT vs PATCH semantics https://review.openstack.org/183945 | 12:42 |
|---|---|---|
| *** HenryG has quit IRC | 13:05 | |
| *** HenryG has joined #openstack-api | 13:05 | |
| *** HenryG has quit IRC | 13:06 | |
| *** HenryG has joined #openstack-api | 13:07 | |
| *** _elmiko is now known as elmiko | 13:08 | |
| *** subscope has quit IRC | 13:56 | |
| elmiko | so, i'm looking at the comments on the 1xx review(thanks btw), and in rewriting this i'm looking at 2 main points. 1) talking about this from the api behavior level, which is to say that most of the time the implementors will not have to deal with 1xx codes as they will be handled by the framework. and 2) in the cases where an implementor needs to use 100 continue that they should take care in implementing it as there are details both from | 13:56 |
| elmiko | i'd like to capture those sentiments and then provide some example links for the challenges. | 13:56 |
| elmiko | sigmavirus24_awa, lifeless, notmyname, cdent, ^^ does this sound reasonable, and does anyone have suggestions for the links? | 13:57 |
| cdent | elmiko: makes sense but I'm afraid I don't really have any suggested links | 13:59 |
| cdent | I wasn't quite able to tease out the point lifeless was making in his question | 13:59 |
| elmiko | i think i get what lifeless is getting at, but i'm trying to figure out if i can provide something more concrete than just "be careful" | 14:00 |
| cdent | Can you rephrase it to me for the sake my poor bwains? | 14:02 |
| elmiko | here's my take: | 14:02 |
| elmiko | 1. in most cases, implementors should not be concerned with generating 1xx responses in their code. this will be handled by the framework used. | 14:03 |
| elmiko | 2. in the cases where implementors do need to use them, most likely for 100 continue, that they should take care in implementing these returns as the wsgi frameworks provide varying levels of coverage and the clients may have issues recieving them. | 14:04 |
| *** pballand has quit IRC | 14:04 | |
| elmiko | that's my understanding so far | 14:05 |
| cdent | yeah, I got that part too, what I didn't understand was: | 14:05 |
| cdent | "Is the WG issueing advice on *implementation*, or advice on *API behaviour*. | 14:05 |
| cdent | If the former, we should be tailoring to wsme, pecan, falcon, and potentially java and go stacks. " | 14:05 |
| elmiko | my take was that lifeless was looking for more resolution on our scope. | 14:06 |
| elmiko | and i'm solidly in the "we are advising api behavior" camp, i suppose | 14:07 |
| elmiko | i just means all the research is on the implementor in terms of figuring out *how* to return the necessary responses | 14:07 |
| elmiko | *it just... | 14:07 |
| dstanek | elmiko: ++ on that scope | 14:08 |
| cdent | ah okay. I think there's a considerable bit of "if you are trying to do X in your code then you are maybe trying to do the wrong thing" | 14:08 |
| cdent | that's part of the educational mission | 14:08 |
| cdent | e.g: if you find yourself wanting to "start_repsonse('100 ..." think | 14:09 |
| elmiko | agreed cdent, which i think we need to keep this 1xx language high level. but i recognize the we need to acknowledge there are legitimate uses for 1xx and those impls may be complicated. | 14:09 |
| cdent | yeah | 14:09 |
| elmiko | man.. too early for typing... /me sighs | 14:09 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 14:12 | |
| elmiko | sigmavirus24: would love your opinion on the convo up there ^^ | 14:31 |
| sigmavirus24 | So I'm also in agreement with the scope, but these are also guidelines for OpenStack, and I think we should at least have a footnote explaining that WSGI (and frameworks sitting atop it) all woefully lack proper support for 100 continue, although I need to see how swift does it | 14:35 |
| sigmavirus24 | lifeless: also was pointing out that, in his opinion, the 100 continue should be the responsibility of apache (I think that's the "framework" he was referring to) | 14:36 |
| elmiko | interesting... | 14:36 |
| elmiko | (about the apache part) | 14:36 |
| sigmavirus24 | The other thing that I think we should stress is that 100 continue is not a response | 14:36 |
| sigmavirus24 | elmiko: ah, lifeless called it the gateway | 14:37 |
| sigmavirus24 | not the framework | 14:37 |
| sigmavirus24 | "sigmavirus24: but thats a separate discussion: - the gateway in this case - e.g. apache2 - has to send it" | 14:37 |
| elmiko | hmm | 14:37 |
| elmiko | ok, i need to read the rfc about 1xx again. | 14:37 |
| sigmavirus24 | 1xx are not "responses", I forget the exact name that the RFCs have for it though | 14:37 |
| elmiko | right, interim codes or w/e | 14:38 |
| sigmavirus24 | right | 14:38 |
| sigmavirus24 | so let's please not confuse people by calling it a response | 14:38 |
| sigmavirus24 | Otherwise we might see someone try to return 2 responses, one with 100 continue and the other after the upload, which is probably wrong in every wsgi application | 14:38 |
| elmiko | ok, that just makes it more confusing from the "providing api behavior guidance" level | 14:39 |
| sigmavirus24 | elmiko: https://tools.ietf.org/html/rfc7231#section-6.2 calls it an "Interim Response" | 14:39 |
| sigmavirus24 | yeah | 14:39 |
| sigmavirus24 | I mean that's implementation details | 14:39 |
| elmiko | and that's the language i used in the guidance "interim response" | 14:39 |
| sigmavirus24 | Which we don't have to mention explicitly | 14:39 |
| sigmavirus24 | But I think these should be at least considerations. I think we should definitely encourage best practices, but we should also consider the reality of OpenStack's technical limitations | 14:40 |
| elmiko | so, is it accurate if we say something like; these interim responses *should not* be implemented in application code, unless there is a specific need for them (most usually 100 continue), and in those cases where they are needed *extreme caution* should be exercised by implementors. ? | 14:41 |
| elmiko | (i'm paraphrasing) | 14:41 |
| cdent | I think that's what we've been trying to say all along but haven't quite nailed it | 14:42 |
| elmiko | yea | 14:42 |
| elmiko | well, we didn't mention the "danger don't step on the tracks" or "sometimes these might be needed" parts | 14:42 |
| sigmavirus24 | Re-reading https://tools.ietf.org/html/rfc7231#section-5.1.1 for the first time in a while makes me thing requests is not spec compliant. I don't think we do any reads before we finish sending the body | 14:42 |
| elmiko | sigmavirus24: but we shouldn't have to worry about talking to that level in the guidance? | 14:43 |
| sigmavirus24 | elmiko: more like "Danger, the third rail is electrified" | 14:43 |
| sigmavirus24 | Yeah | 14:43 |
| elmiko | hehe | 14:43 |
| sigmavirus24 | This becomes the trick then | 14:44 |
| elmiko | i never realized how sticky this one would get lol, i'm learning a bunch though =) | 14:44 |
| sigmavirus24 | Because the 5xx guideline also talks about application code | 14:44 |
| sigmavirus24 | Which is an implementation detail | 14:44 |
| elmiko | yea | 14:44 |
| sigmavirus24 | Not an API behaviour | 14:44 |
| elmiko | i need to revisit that as well | 14:44 |
| sigmavirus24 | Which makes me wonder, do we need to split the guidance into implementation details and behaviour details? | 14:45 |
| elmiko | right, but i think the usage of broad language like "application code" shouldn't be too deep a dive into impl details | 14:45 |
| sigmavirus24 | I was one of the people firmly against talking about oslo.utils for the truthy/falsey values | 14:45 |
| sigmavirus24 | Yeah, I would hope it's not considered that way | 14:45 |
| elmiko | i'd be ok with creating impl details advice, but i agree it should entirely separate | 14:45 |
| sigmavirus24 | I hope people are okay with shades of grey around this | 14:45 |
| elmiko | me too | 14:46 |
| elmiko | even if we are just doing api behavior advice, we will need to use terminology that references the part of this that is not just high-level api stuff. which is why i chose the term "application code" | 14:46 |
| elmiko | oh well, i'll spin again and we'll see how it goes. | 14:47 |
| sigmavirus24 | The comments on http://grammarist.com/spelling/gray-grey/ are golden | 14:47 |
| sigmavirus24 | Because I always use "grey" and people tell me it's "gray" and I was curious what the difference was just now | 14:48 |
| cdent | I think there's a big difference between talking about oslo and talking about WSGI apps choosing to 100 or not (and advising that that in general they should not) | 14:49 |
| elmiko | cdent: yea, agreed | 14:50 |
| ryansb | when folks t a chance, please review https://review.openstack.org/186546 so we can update the guideline template | 14:52 |
| ryansb | s/t a/get a/ | 14:52 |
| elmiko | ryansb: done | 14:53 |
| ryansb | Hoping to bring it up for freeze next meeting | 14:56 |
| elmiko | cool, not sure if we need to freeze the temple, but add it to the agenda ryansb | 14:56 |
| elmiko | actually, i need to fix up the agenda wiki for next week. needs a little trimming | 14:57 |
| ryansb | yeah, I'll add when I get a few more reviews, so I'm sure it'll be set to go. | 14:58 |
| elmiko | cool, thanks! | 14:58 |
| ryansb | I think the template is important enough to freeze for, since it matters to all guidelines going forward | 14:59 |
| ryansb | but that's just IMO | 14:59 |
| elmiko | good point, and that makes sense to me | 14:59 |
| lifeless | sigmavirus24: FWIW i'm ok with shades of grey | 15:00 |
| lifeless | i asked about scope to tailor my reviews of the guidelines | 15:00 |
| sigmavirus24 | lifeless: yeah that's fair | 15:00 |
| sigmavirus24 | I just think we're all more heavily on the "Behaviour" side | 15:01 |
| lifeless | or gray | 15:01 |
| elmiko | lifeless: thanks for bringing it up, i think it helps me too as i can better understand how to approach these in a more complete manner | 15:01 |
| lifeless | or greay | 15:01 |
| lifeless | or graey | 15:01 |
| sigmavirus24 | But there are caveats where we might serve the readers best by dipping into some minor impl details | 15:01 |
| lifeless | exactly | 15:02 |
| elmiko | and that's where i could use some helpful nudging, i'm a little unsure as to how far into impl we need to dip | 15:02 |
| * ryansb senses a guideline for proper spelling of grey coming soon to a WG near you | 15:02 | |
| elmiko | haha! | 15:03 |
| cdent | elmiko: ItDepends™ | 15:03 |
| cdent | I don't think there's going to be a magical heuristic that tells us what to do | 15:03 |
| elmiko | agreed | 15:04 |
| elmiko | ok, so here's a question along those lines. should i be using the term "HTTP framework" or "WSGI framework"? | 15:05 |
| elmiko | (when referring to whence these codes should commence) | 15:05 |
| cdent | For sake of fleshing out the context: why would you ever want to say either? | 15:05 |
| cdent | my feeling is that these guidelines are >90% for people writing python code that happens to be able to return http response codes some of the time | 15:06 |
| elmiko | hmm, i don't think so, but i want to be sure that "WSGI framework" covers the extend of libraries we use for doing http stuff | 15:06 |
| cdent | I call that code "the app" | 15:06 |
| cdent | swift has special cases all over the map, but that's what swift does | 15:06 |
| elmiko | that's how i brek it down internally as well, app = code that you wrote which runs on top of a framework | 15:07 |
| cdent | but if you are sending and receiving JSON objects back and forth...? | 15:07 |
| elmiko | for me, framework = the library which is helping to negotiate all the HTTP factors involved | 15:07 |
| cdent | early on we decided that we were not going to try to police frameworks (e.g. pecan not being able to do 405 (it almost fixed it btw, but there are bugs)) | 15:07 |
| cdent | but there's no reason we can't police how frameworks are called | 15:08 |
| elmiko | ok, yea. that's kinda what i was thinking too | 15:08 |
| cdent | so if you have a situation where you want to say "don't explicitly do 500" | 15:08 |
| cdent | it doesn't matter if you are talking about the framework or app code or whatever | 15:09 |
| cdent | it's: in the place where you have control, don't do this bad thing | 15:09 |
| elmiko | right | 15:09 |
| elmiko | unless you absolutely need to, and understand the implications | 15:09 |
| elmiko | i feel like that should be a caveat for all these guidelines though... | 15:09 |
| elmiko | thanks, that helped. i gotta run out for a bit, back later | 15:10 |
| * cdent waves and bows | 15:10 | |
| elmiko | sigmavirus24, lifeless, thanks to you guys as well. i'm sure i'll have more questions later =) | 15:10 |
| sigmavirus24 | lifeless: and I should form an information team "RFC Wonks" | 15:11 |
| elmiko | ha! | 15:11 |
| lifeless | so yeah | 15:11 |
| lifeless | I'm going back to bed but | 15:11 |
| * elmiko wishes he had more time for rfc wonking | 15:11 | |
| lifeless | a) we have a wide spread of pure WSGI through to eventlet-extensions-of-WSGI depending on what tree and what framework | 15:11 |
| lifeless | for 100 specifically WSGI doesn't deliver useful 100 continue functionality. | 15:12 |
| lifeless | https://www.python.org/dev/peps/pep-3333/#http-1-1-expect-continue is the spec | 15:12 |
| lifeless | notice: the server or gateway must do that - thats the thing that actually is on the HTTP socket, not the WSGI app or WSGI middleware | 15:12 |
| lifeless | eventlet offers an extension, which swift uses to customise the 100 response sent, but in general it should be entirely automatic | 15:13 |
| lifeless | and - in WSGI land including the swift use of eventlet extensions, anyone doing yield ([('100', 'Continue'),.... is going to break things so badly its not funny | 15:13 |
| elmiko | ok, so neither the "app code" nor the "framework code" should be making these interim responses directly. they should deligating that to the gateway? | 15:14 |
| ryansb | cdent: do we have the "no explicit 500" rule in a guideline yet? | 15:14 |
| lifeless | but at the API level 100's are a MUST | 15:14 |
| ryansb | elmiko: lifeless ^^ | 15:14 |
| cdent | ryansb, yeah, lemme find it | 15:14 |
| ryansb | thanks | 15:14 |
| elmiko | ryansb: i'm working on this https://review.openstack.org/#/c/183698 | 15:14 |
| cdent | https://review.openstack.org/#/c/179365/ | 15:14 |
| lifeless | elmiko: I think a block diagram might be useful | 15:15 |
| cdent | I guess the "no explicit" part faded out of that | 15:15 |
| elmiko | lifeless: interesting, i might have to circle back around on that when you are up again. what timezone are you in? | 15:15 |
| lifeless | elmiko: client <here we care about the RFC's and 100's are a must> Server .... WSGI gateway/server [here we must not care about 100] Python code | 15:15 |
| lifeless | elmiko: UTC+12 | 15:16 |
| lifeless | elmiko: the API behaviour is speciying what folk see from the outside, but the impl hints are for the inside | 15:16 |
| lifeless | elmiko: and they are diametrically opposed for this case, for the impl-is-WSGI set | 15:16 |
| elmiko | lifeless: ok, thanks. i'm gonna need to let this settle in more. expect another revision of the 1xx stuff, but i imagine i will need more help refining it. | 15:17 |
| lifeless | cdent: why would we say 'no explicit 500's' ? | 15:17 |
| cdent | s/no/rare/ | 15:18 |
| cdent | because if you know an exception happened, such that you can trap it, you also can decode it to something more meaningful | 15:18 |
| cdent | people have a tendency to use 500 as a way of sort of throwing up their hands, rather than being properly defensive and informative | 15:19 |
| cdent | another way to put it: a lof of 500s could be 4xx with some effort | 15:19 |
| lifeless | I think thats a lot better way to put it | 15:20 |
| lifeless | otherwise we'll get noddy reviews where folk have to climb a high barrier to do the right thing | 15:21 |
| cdent | or sometimes 502 | 15:21 |
| lifeless | if we have a direct test, thats much better than saying 'X should be very rare' and making everything a special case | 15:21 |
| lifeless | jay's prose is entirely good to me | 15:21 |
| * cdent sighs | 15:22 | |
| cdent | I lost the network there for a bit | 15:22 |
| lifeless | oh, I thought that was on my comments ;) | 15:22 |
| lifeless | anyhow, I really must go back to sleep or I'll be a wreck in the daytime | 15:22 |
| lifeless | ciao | 15:22 |
| * cdent waves | 15:22 | |
| *** cdent_ has joined #openstack-api | 15:25 | |
| *** pballand has joined #openstack-api | 15:26 | |
| cdent_ | nnnnnggg | 15:26 |
| *** cdent has quit IRC | 15:29 | |
| *** cdent_ is now known as cdent | 15:29 | |
| *** alex_klimov has quit IRC | 15:56 | |
| *** e0ne has quit IRC | 16:10 | |
| *** Apoorva has joined #openstack-api | 16:54 | |
| *** lucasagomes is now known as lucas-beer | 17:01 | |
| *** e0ne has joined #openstack-api | 17:38 | |
| *** cdent has quit IRC | 17:45 | |
| elmiko | anyone know if i can have a change depend on 2 patches? | 18:40 |
| elmiko | basically, i want to rebase on top of 2 different commit to ensure that those advice is in the patch i'm creating | 18:40 |
| sigmavirus24 | elmiko: no | 18:45 |
| sigmavirus24 | The only commits that can have 2 parents are a merge commit | 18:45 |
| sigmavirus24 | afaik | 18:45 |
| sigmavirus24 | You're essentially saying "This commit has two parents" which I've never seen otherwise and I'm not certain if git gives you a way to do it | 18:45 |
| elmiko | sigmavirus24: ok, thanks. i thought it kinda had a bad smell to it | 18:46 |
| ryansb | yeah, has to be a merge (afaik) to have many parents | 18:46 |
| elmiko | i'm rewriting the 5xx advice thing, and i want to add a sentence about looking at the 500 and 501 specific advice | 18:46 |
| sigmavirus24 | elmiko: if it has to depend on two other changes, one of those changes has to have the other as a parent | 18:46 |
| elmiko | ack, makes perfect sense | 18:46 |
| elmiko | sigmavirus24: 502, 503, 504 and 505 should never be generated by application code? these should only come from some sort of gateway device or something? | 18:56 |
| sigmavirus24 | yeah | 18:57 |
| sigmavirus24 | that's my inkling at least | 18:57 |
| sigmavirus24 | let me look at what those codes typically are | 18:57 |
| elmiko | ok, i'm mentioning 500 and 501 specifically. i might as well call out those as not being generated by app. code | 18:57 |
| *** lucas-beer has quit IRC | 19:17 | |
| elmiko | sigmavirus24: hmm, i'm wondering about 503 (service unavailable) now. i could see a situation where someone might make a case for wanting to return this from application code. for example, if the server implements some sort of "maintenance mode" or something. what do you think? | 19:25 |
| sigmavirus24 | But that wouldn't be returned from a single endpoint, that'd be in middleware, no? | 19:25 |
| sigmavirus24 | I would expect the middleware, not the application, to be implementing that | 19:26 |
| sigmavirus24 | Although, I'm not sure how realistic that use case is | 19:26 |
| elmiko | i would expect it to be from middleware, or some specific app in place. maybe i'm overthinking this | 19:26 |
| stevelle | I would say it is from reverse proxies and gateways. | 19:29 |
| elmiko | thanks, that makes sense | 19:30 |
| stevelle | Nginx rate throttling unfortunately uses 503, for example | 19:30 |
| stevelle | Amazon's rate throttling gives 503s as well, again misuse. | 19:30 |
| stevelle | that may or may not be Nginx | 19:31 |
| openstackgerrit | Michael McCune proposed openstack/api-wg: Adding 5xx guidance https://review.openstack.org/183698 | 19:31 |
| elmiko | ok, lets see how that works | 19:31 |
| elmiko | dstanek: ^^ i'm curious if this is closer to what we've been talking about with respect to keystone | 19:32 |
| dstanek | elmiko: i think so, yes | 19:45 |
| elmiko | dstanek: awesome, thanks for helping me out =) | 19:45 |
| dstanek | elmiko: hey, you're doing all of the work, so thank you! | 19:46 |
| elmiko | hehe | 19:47 |
| sigmavirus24 | If-Modified-Since is a really confusing header | 20:04 |
| elmiko | how so, seems kinda straight forward in the rfc | 20:11 |
| elmiko | ? | 20:11 |
| *** e0ne has quit IRC | 20:11 | |
| sigmavirus24 | Most people don't read that | 20:26 |
| sigmavirus24 | see the mailing list | 20:26 |
| elmiko | k | 20:27 |
| notmyname | elmiko: I'm here now | 20:43 |
| elmiko | notmyname: hey | 20:44 |
| elmiko | so, i'm just curious to understand a little more about how swift uses the 100-continue from the application layer, i think i understand conceptually why it's needed | 20:44 |
| elmiko | to frame this, | 20:45 |
| elmiko | i'm trying to rewrite the guidance in a way that pays attention to the need for some application to generate the continue, but i'm a little confused after reading the rfc about where this code comes from | 20:45 |
| *** fzdarsky has quit IRC | 20:46 | |
| notmyname | there are some interesting things we're using for internally (ie between swift processes), but the basic user-facing use case is pretty simple | 20:46 |
| notmyname | if you send a "Expect: 100" header in a request, we'll send back a 100 Continue response line based on the contents of the headers (ie before we read any bytes from the body) | 20:47 |
| notmyname | so it's a way to fast-fail if eg you want to upload something bigger than the max object size and have a content-length header | 20:48 |
| notmyname | or otherwise bad headers/request info | 20:48 |
| notmyname | I can give you an example pretty easily. let me star t up my saio | 20:48 |
| elmiko | cool | 20:49 |
| notmyname | successful example: https://gist.github.com/notmyname/efa3ddca6e1fb3083d6f | 20:50 |
| * elmiko reading | 20:51 | |
| elmiko | ok, that makes sense | 20:51 |
| elmiko | and is the swift application code handling the logic about sending the 100 ? | 20:51 |
| notmyname | I set the max object size to 10MB and here's a failure example: https://gist.github.com/notmyname/f87d19ccca7800bfee9a | 20:52 |
| notmyname | in this case, the application is curl, and yes | 20:52 |
| elmiko | and on the server (swift) end, how does it handle returning the 100, followed by the 201? | 20:53 |
| notmyname | swift does that in with our HTTPConnection wrapper, but actually it's handled by eventlet | 20:53 |
| elmiko | interesting | 20:54 |
| notmyname | if the expect header was in the request, the 100 Continue is sent when we start reading the bytes off the body (ie the app_iter) | 20:54 |
| notmyname | unless we've already sent/set a response, in which case that's what you get on the client side | 20:54 |
| elmiko | how do you coordinate the error example with that? | 20:55 |
| elmiko | like, how does eventlet know to send the 413? | 20:55 |
| elmiko | er HTTPConnection wrapper | 20:55 |
| elmiko | or i guess, is it just "if Content-Length > max object size then return 413" | 20:57 |
| notmyname | let me trace that through to give you some links | 20:57 |
| elmiko | cool, i appreciate it =) | 20:57 |
| notmyname | ok, thanks for waiting. took me a bit to walk through it | 21:07 |
| elmiko | np | 21:07 |
| notmyname | it's been a while since I've looked at that part of the code :-) | 21:07 |
| elmiko | hehe | 21:07 |
| elmiko | i'm keeping you on your toes ;) | 21:07 |
| notmyname | https://github.com/openstack/swift/blob/master/swift/common/bufferedhttp.py | 21:09 |
| notmyname | oh wait :-) | 21:10 |
| notmyname | I may have found something else | 21:10 |
| notmyname | but that BufferedHTTPConnection class is kinda where swift consolidates it | 21:10 |
| notmyname | ah. there it is | 21:12 |
| notmyname | ok, 2 places. one in swift, one in eventlet | 21:12 |
| elmiko | k | 21:12 |
| lifeless | elmiko: so the WSGI spec is this: the WSGI gateway MUST be responsibe for it. | 21:12 |
| lifeless | elmiko: eventlet then offers an extension API that lets you explictly trigger it with some custom headers | 21:12 |
| notmyname | not quite | 21:12 |
| lifeless | notmyname: I climbed through the code last night, so I'm curious what I've missed | 21:13 |
| notmyname | so the eventlet.wsgi.Input class has the feature | 21:13 |
| lifeless | notmyname: https://www.python.org/dev/peps/pep-3333/#http-1-1-expect-continue is the WSGI spec bit | 21:13 |
| lifeless | http://eventlet.net/doc/modules/wsgi.html#continue-response-headers is the eventlet extension | 21:13 |
| notmyname | in Input()._do_read() as soon as you start reading the input, eventlet sends the 100-continue response | 21:13 |
| elmiko | lifeless: i did read that in the wsgi spec after you posted it yesterday | 21:13 |
| lifeless | notmyname: yes, thats conformant with WSGI (and isn't even an extension) | 21:14 |
| lifeless | notmyname: swift/obj/server.py is where I found the eventlet extension use | 21:15 |
| lifeless | lines 632 and 562 | 21:16 |
| notmyname | lifeless: what method? that doesn't seem to line up with current master? what's it look like? | 21:17 |
| notmyname | elmiko: because of that eventlet behavior, if you set a response before you read anything from wsgi.input, you get the 100-continue handling. swift's buffered connection wrappers (linked above) provide us some helpers to read those from the storage servers and then we can return the right thing to the client | 21:18 |
| lifeless | notmyname: oh, I'm a month out | 21:18 |
| notmyname | elmiko: but mostly, like with the examples I pasted above, that's free from event. ie we didn't write special code to handle that other than just setting a response _before_ reading anything from wsgi.input | 21:19 |
| lifeless | let me update my tree | 21:19 |
| notmyname | lifeless: are you talking about the multi-phase commit stuff? | 21:19 |
| elmiko | notmyname: ack, makes sense. you need to intercept the call to ensure that hte object is not too big before doing the read from eventlet or it will automatically return the 100, is that accurate? | 21:19 |
| lifeless | notmyname: 'set a response' - where are you doing that pattern? - middleware response objects aren't part of WSGI | 21:19 |
| notmyname | elmiko: yes. as soon as you call Request.environ['wsgi.input'].read(), the 100-continue handling is done in eventlet | 21:20 |
| lifeless | notmyname: line 562 and line 632 still in swift/obj/server.py | 21:20 |
| lifeless | notmyname: thats done for both multiphase *and* has-metadata-footer | 21:21 |
| elmiko | notmyname: got it. so would it fair to say that in this case the swift code is very much reaching through the boundary between "application code" and "wsgi framework code" to achieve the desired result? | 21:21 |
| lifeless | notmyname: if you can help me find this 'set a response' thing tat would be grand? | 21:21 |
| notmyname | lifeless: nah, no special extensions there. but we did patch upstream eventlet to support header values returned with a 100 continue response. that's what we're using there | 21:22 |
| notmyname | lifeless: both of those parts are part of the same "multiphase commit" thing | 21:22 |
| lifeless | notmyname: so the metadata_footer is only used in multiphase commit ? | 21:23 |
| notmyname | elmiko: I don't know that I'd agree with that phrasing. let me find a specific place we do the response before reading for you and lifeless. that might make it more clear | 21:23 |
| notmyname | lifeless: yes. that's actually one of the "phases" in multi-phase ;-) | 21:23 |
| lifeless | notmyname: ack | 21:23 |
| lifeless | so yeah, please do find the other place you do manual 100 stuff | 21:23 |
| elmiko | notmyname: ok, maybe a poor choice of words | 21:24 |
| notmyname | lifeless: that's our wacky internal protocol way to implement footers instead of headers. http says you can do footers, but _nothing_ supports that. this was easier, believe it or not :-) | 21:24 |
| notmyname | lifeless: well, we don't do manual 100 checking. that is, the only "manual" step is not to read wsgi.input | 21:24 |
| elmiko | notmyname: i feel like you are definitely working the border of the framework code though, this is some deep dive type stuff imo | 21:25 |
| notmyname | https://github.com/openstack/swift/blob/master/swift/proxy/controllers/obj.py#L883-L920 | 21:25 |
| notmyname | so look at that section | 21:26 |
| lifeless | notmyname: oh, I'm well aware about how spotty trailer support is | 21:26 |
| notmyname | first we do some basic sanity checking | 21:26 |
| * elmiko reads | 21:26 | |
| notmyname | then we check auth | 21:26 |
| lifeless | notmyname: you forget my heritage in HTTP :) | 21:26 |
| notmyname | then we check constraints (object length, metadata, etc) in check_object_creation() | 21:26 |
| notmyname | lifeless: or didn't/don't know it ;-) | 21:26 |
| notmyname | if anythign is amiss, we return the response object. | 21:27 |
| lifeless | notmyname: oh, squid core developer (inactive these days, but very active in early 2000's) | 21:27 |
| notmyname | note that we don't actually call the wsgi.read until the _store_object(). actually, that has some more checking too before we call read() | 21:28 |
| lifeless | notmyname: ok, but - this is just PEP-3333 expect handling, isn't it? | 21:28 |
| lifeless | notmyname: 'don't read from input unless you're willing to have 100 sent' | 21:29 |
| notmyname | the read() isn't called until in _transer_data() in line 782 | 21:29 |
| lifeless | notmyname: btw, are you interested in h2 support in swift? | 21:29 |
| notmyname | (note it's in an iter by that point) | 21:29 |
| elmiko | notmyname: just to continue my understanding, when you call the reader at https://github.com/openstack/swift/blob/master/swift/proxy/controllers/obj.py#L936 is when eventlet will automatically send the 100? | 21:30 |
| notmyname | elmiko: that just wraps it in an iter. doesn't call it yet :-) | 21:30 |
| elmiko | notmyname: wow, the rabbit hole goes deeper ;) | 21:30 |
| notmyname | elmiko: look at line 782 | 21:30 |
| elmiko | ok, got it | 21:31 |
| lifeless | notmyname: ok, so this isn't what I'd call 100 utilisation - in that you're not specifically generating or customising the 100 | 21:31 |
| elmiko | hmm, this gives me much food for thought. | 21:31 |
| lifeless | notmyname: what I tink you're saying is just that you wrote things carefully to ensure the gateway (eventlet here) doesn't send a 100 until you've actually done your pre-flight checks for auth etc | 21:32 |
| notmyname | lifeless: from the client point of view, we are using the functionality. but to implement it, we're "just" doing the right thing with eventlet | 21:32 |
| lifeless | right | 21:32 |
| notmyname | lifeless: correct | 21:32 |
| notmyname | and I think it's pretty important | 21:32 |
| lifeless | I agree | 21:32 |
| notmyname | now, internally, we're doing something more interesting | 21:32 |
| lifeless | my issue with the advice that was drafted was that it was nonconformant with the RFC :) | 21:32 |
| notmyname | that's the multiphase stuff for EC that uses http with 100 Continue as a request/response protocol ;-) | 21:32 |
| notmyname | (yes it's rather icky) | 21:33 |
| elmiko | this is very nuanced, but i think we need to explain some of it if only for the purpose of helping a new developer understand to not just return Response(100, ...) (for example) | 21:33 |
| lifeless | 100 is essential for 1.1 clients (except Python clients, because most of them are broken) | 21:33 |
| lifeless | elmiko: I don't think you do. | 21:33 |
| lifeless | elmiko: because that simply won't work. | 21:33 |
| elmiko | lifeless: right, but i think we want to say that the framework should be responsible for the 100, but that if you need to control behavior around it you better understand the rfc and understand how the framework handles it. does that seem fair? | 21:34 |
| elmiko | notmyname: ^^ | 21:37 |
| notmyname | or "don't do dumb things"? | 21:38 |
| elmiko | haha | 21:38 |
| elmiko | someone i don't think i'll get away with that ;) | 21:38 |
| notmyname | I don't see a lot of value in guidelines that basically say "follow the RFC and know what you're doing" | 21:39 |
| notmyname | and as an extension, I don't see what clarification to the RFC is needed for the API-WG | 21:39 |
| elmiko | agreed, i was actually going to call out the example of needing to validate an input for sizing or something to help drive the point home | 21:39 |
| elmiko | do you think we should just skip the 1xx advice then? | 21:40 |
| notmyname | ie if you want to offer 100-continue support in your openstack project, do that. however is appropriate for your chosen framework | 21:40 |
| elmiko | right, that makes sense | 21:40 |
| notmyname | yeah, I'd be totally fine with no API-WG voice on 1xx support | 21:40 |
| notmyname | (less is better) | 21:40 |
| elmiko | ok, i'll add this to the agenda for the next meeting | 21:40 |
| notmyname | I'm sure you're shocked (shocked! I tell you) to hear me of all people say that ;-) | 21:40 |
| elmiko | well, i've mainly been trying to implement some guidance for the sections we had listed. i was under the, false, impression that 1xx would be straight forward, but it's actually a little more nuanced than that. | 21:41 |
| elmiko | the other part of this is that the wg has decided that part of the goal of these documents should be informative, so i disagree with you a little about not having anything for 1xx. | 21:43 |
| notmyname | :-) | 21:43 |
| elmiko | regardless, many thanks for helping me understand this better. i do appreciate you taking the time to walk me through it. | 21:44 |
| notmyname | in that case, using swift as an example of how 100 continue can be effectively used is ok :-) | 21:44 |
| notmyname | my pleasure | 21:44 |
| notmyname | I like the chance to walk through dusty sections of the code | 21:44 |
| elmiko | notmyname: you don't happen to have an opinion on the use of 101? ;) | 21:44 |
| elmiko | hehe | 21:44 |
| sigmavirus24 | lol | 21:45 |
| notmyname | 101 is eg switching to http/2 right? | 21:45 |
| sigmavirus24 | I think Swift is the best use case for using 100 continue | 21:45 |
| notmyname | lifeless: oh you asked me about that | 21:45 |
| elmiko | sigmavirus24: total agreement | 21:45 |
| sigmavirus24 | Glance would be too | 21:45 |
| sigmavirus24 | But ... well ... glance is glance ;) | 21:45 |
| elmiko | i might file off the serial numbers though, i think the use case in general is very applicapable | 21:45 |
| sigmavirus24 | Maybe we can introduce it in artifacts | 21:45 |
| elmiko | notmyname: yea, 101 is switching protocols | 21:46 |
| sigmavirus24 | elmiko: for large request bodies being uploaded, yes | 21:46 |
| sigmavirus24 | How applicable that is to openstack, I'm not sure | 21:46 |
| elmiko | sigmavirus24: i just don't like mentioning projects by name in the guidelines | 21:46 |
| notmyname | I'd say there's more use to 100 than just swift/glance. mostly it's a question of if processing the body is cheaper than transferring the body. ie 100 is good when it's expensive to put the body of the request ont he wire | 21:47 |
| notmyname | lifeless: you asked me about http/2 right? | 21:47 |
| sigmavirus24 | we can't even pat projects on the back for doing the right thing? | 21:47 |
| elmiko | sigmavirus24: maybe we can have an accolades section ;) | 21:47 |
| notmyname | lifeless: sure, and I've talked with mnot briefly about it a while back. but it would be a lot of work, and so it's a matter of prioritization. compared to other stuff going on, there's not a lot of demand for http/2. | 21:48 |
| elmiko | sigmavirus24: seriously though, i think we should just tend not to because, imo, it introduces a level of brittleness into the docs. like, would we need to continue checking on those projects/implementation to ensure they have continued to "do the right thing". | 21:49 |
| sigmavirus24 | proposal, if a project breaks our docs | 21:49 |
| notmyname | there's only marginal benefit for swift supporting http/2. yes, some use cases would be great (eg content served to web browsers). but eg with storing backups or file-sync/share, it doesn't help a lot | 21:50 |
| sigmavirus24 | they're put in the wall of shame | 21:50 |
| elmiko | sigmavirus24: LOL | 21:50 |
| notmyname | lifeless: but, that's just, like, my opinion, man | 21:50 |
| elmiko | i think we need a new project for openstack, the Wall of Shame as a Service | 21:50 |
| sigmavirus24 | openstack wall of shame | 21:52 |
| notmyname | I'm sad that everyone hates on glance. there's so much potential there. I'd love to see glance as a general indexing service for other projects. it already tracks compute images. it would be cool eg for it to also track objects in swift (show me all the images that are blue that were uploaded before last week). | 21:52 |
| sigmavirus24 | notmyname: I'm a glance core/driver | 21:52 |
| sigmavirus24 | So I'm mostly making fun of myself at this point | 21:52 |
| sigmavirus24 | That said, we have a lot of split focus in glance | 21:52 |
| sigmavirus24 | A lot | 21:52 |
| elmiko | i must be too new to openstack, i seriously do not pickup on all this hate | 21:52 |
| sigmavirus24 | elmiko: did you see termie's talk at the summit? | 21:53 |
| notmyname | sigmavirus24: one thing we're working on in swift is metadata indexing. I think it would be great to store that in glance (or notify glance for glance to index it). instead, it seems I'm going to need to work with elasticsearch or something and add somethign new to the mix | 21:54 |
| elmiko | sigmavirus24: not sure, which one was that? | 21:54 |
| sigmavirus24 | elmiko: https://www.openstack.org/summit/vancouver-2015/summit-videos/presentation/openstack-is-doomed-and-it-is-your-fault | 21:54 |
| sigmavirus24 | notmyname: that's what OpenStack Searchlight is for | 21:54 |
| sigmavirus24 | Searchlight sits atop Elasticsearch and provides search/indexing | 21:54 |
| sigmavirus24 | Searchlight is being ripped out of Glance during Liberty | 21:54 |
| elmiko | sigmavirus24: no, i did not see this. gonna have to watch it. | 21:54 |
| sigmavirus24 | elmiko: I won't spoil the talk for you | 21:55 |
| * elmiko goes off to watch | 21:55 | |
| sigmavirus24 | It was enjoyable | 21:56 |
| elmiko | the description looked funny, can't remember why i missed it. probably talking too much lol | 21:56 |
| sigmavirus24 | yeah | 21:58 |
| *** pballand has quit IRC | 22:06 | |
| etoews | best quote from termie's prez "stop with the terrible apis already" | 22:12 |
| sigmavirus24 | yep | 22:18 |
| *** alex_klimov has joined #openstack-api | 22:23 | |
| elmiko | sigmavirus24: good talk, although a bit depressing for a friday afternoon | 22:39 |
| sigmavirus24 | yeah sorry | 22:40 |
| elmiko | lol, no you're not! ;P | 22:40 |
| sigmavirus24 | true | 22:41 |
| elmiko | HAHA | 22:41 |
| *** alex_klimov has quit IRC | 22:41 | |
| elmiko | honestly thought, really good feedback from termie. i worked in closed source for almost all of my career (20yrs), and this has been the most fun project i've worked on. | 22:45 |
| elmiko | (maybe some of my early days as a game dev come close) | 22:45 |
| *** terrylhowe has left #openstack-api | 23:00 | |
| openstackgerrit | Everett Toews proposed openstack/api-wg: Add reviewers to an API WG review https://review.openstack.org/193753 | 23:16 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!