| *** JimCheung has quit IRC | 00:01 | |
| *** alenavolk has joined #openstack-swift | 00:10 | |
| *** NM has quit IRC | 00:10 | |
| notmyname | kota_: I've got to go home now, but here is what I was wanting to talk to you about | 00:20 |
|---|---|---|
| *** alenavolk has quit IRC | 00:21 | |
| notmyname | kota_: https://review.openstack.org/#/c/448240/ is nearly done. we've talked about it as a community for a long time. we've had a couple of different implementations proposed. we've discussed the design. now let's land it | 00:21 |
| patchbot | patch 448240 - swift - Enable per policy proxy config options | 00:21 |
| *** gyee has quit IRC | 00:22 | |
| *** alenavolk has joined #openstack-swift | 00:22 | |
| notmyname | kota_: between you, tim, acoles, and clayg I'd love to see it landed on (my) wednesday. | 00:22 |
| notmyname | kota_: looking at timezones, here's how I see that working out | 00:22 |
| *** alenavolk has quit IRC | 00:22 | |
| notmyname | kota_: right now, it's the beginning of your tuesday. at the end of your tuesday, acoles will be getting online. and clayg and timburke will be starting our tuesday at the end of acoles's day | 00:23 |
| notmyname | kota_: if you could give the patch a review today, that will allow acoles to respond to anything you might find. then timburke and/or clayg can pick up as acoles is finishing his day | 00:24 |
| notmyname | I don't expect that there will be any major things found (based on the review and community work already spent on it) | 00:24 |
| notmyname | kota_: thanks for your help on this patch | 00:25 |
| *** tonanhngo has quit IRC | 00:36 | |
| *** tovin07_ has joined #openstack-swift | 00:37 | |
| *** itlinux has joined #openstack-swift | 00:40 | |
| *** itlinux has quit IRC | 00:41 | |
| *** itlinux has joined #openstack-swift | 00:42 | |
| *** tonanhngo has joined #openstack-swift | 00:43 | |
| openstackgerrit | Tim Burke proposed openstack/swift master: fixup! Enable per policy proxy config options https://review.openstack.org/466952 | 00:45 |
| openstackgerrit | Tim Burke proposed openstack/swift master: Remove tempauth docs from deployment guide https://review.openstack.org/466953 | 00:45 |
| *** tonanhngo_ has joined #openstack-swift | 00:47 | |
| *** tonanhngo has quit IRC | 00:48 | |
| *** tonanhngo_ has quit IRC | 00:51 | |
| *** JimCheung has joined #openstack-swift | 00:53 | |
| clayg | kota_: thanks | 01:00 |
| *** klrmn has quit IRC | 01:05 | |
| *** vint_bra has joined #openstack-swift | 01:11 | |
| *** esnyder has joined #openstack-swift | 01:22 | |
| *** itlinux has quit IRC | 01:28 | |
| *** catintheroof has joined #openstack-swift | 01:29 | |
| *** catintheroof has quit IRC | 01:30 | |
| esnyder | hey all, notmyname recommended i ask here (just tried #openstack-operators) | 01:32 |
| esnyder | https://www.irccloud.com/pastebin/rn6KpxY5/ | 01:32 |
| zaitcev | LB or proxy isn't going to keep the connection open forever | 01:34 |
| zaitcev | oh, wait | 01:34 |
| zaitcev | I was going to sit this one out | 01:34 |
| esnyder | the error I see is: | 01:35 |
| esnyder | May 22 17:40:06 emile-swifttest-2-3020 proxy-server: ERROR with Object server 10.9.212.22:6007/d2 re: Trying to write to /KEY_11d76633a93e4dc996de269bcfd16efd/puttest/putobj-00003: #012Traceback (most recent call last):#012 File "/usr/lib/pymodules/python2.7/swift/proxy/controllers/obj.py", line 1542, in _send_file#012 self.conn.send(to_send)#012 File | 01:35 |
| esnyder | "/usr/lib/python2.7/httplib.py", line 840, in send#012 self.sock.sendall(data)#012 File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 393, in sendall#012 tail += self.send(data[tail:], flags)#012 File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 384, in send#012 return self._send_loop(self.fd.send, data, | 01:35 |
| esnyder | flags)#012 File "/usr/lib/python2.7/dist-packages/eventlet/greenio/base.py", line 371, in _send_loop#012 return send_method(data, *args)#012error: [Errno 32] Broken pipe | 01:35 |
| zaitcev | yeah | 01:35 |
| esnyder | i get it won't hold the connection forever, but 1 second more than the default client_timeout triggers it | 01:36 |
| esnyder | what is the point of having client_timeout if i can't adjust it up? :) | 01:36 |
| zaitcev | Presumably you can if the server is under your administrative control, just just have to find which one trips it and tweak it. Start with the LB, and then Swift proxy https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample#L79 | 01:37 |
| esnyder | zaitcev: sorry, you said you wanted to sit this one out, so feel free to ignore me, i'm just going to keep talking | 01:39 |
| esnyder | so, *before* adjusting the client_timeout up, I get 'ERROR Client read timeout'. *after* bumping it up from 60 seconds, and running test where i insert delay of 61 seconds, I get that 'ERROR with Object server...' one | 01:40 |
| esnyder | what other dial should i be turning? | 01:40 |
| zaitcev | Oh, I thought you adjust the timeout on the client. | 01:40 |
| esnyder | sorry, nope, (naming :) ) i adjust the client_timeout in the /etc/swift/proxy-server.conf | 01:41 |
| zaitcev | the code says it's putter.spawn_sender_greenthread( | 01:43 |
| zaitcev | pool, self.app.put_queue_depth, self.app.node_timeout, | 01:43 |
| zaitcev | er | 01:43 |
| esnyder | so, the self.app.node_timeout turns into a ChunkWriterTimeout of that duration | 01:43 |
| esnyder | but i'm pretty sure that the Broken pipe means that this is the timeout inside the socket itself | 01:44 |
| zaitcev | Well, it's node timeout anyway. Apparently the _send_file receives only 1 timeout value, which ... apparently... governs both sides. Maybe I'm reading it wrong though. | 01:44 |
| esnyder | which it doesn't look like any of the code i can see modifies? | 01:44 |
| zaitcev | or, actually again... | 01:45 |
| esnyder | one more confusion i have re: node_timeout: it's default value is 10 | 01:46 |
| zaitcev | _send_file uses node_timeout appropriately, by using with: around conn.send(). BUT | 01:46 |
| *** JimCheung has quit IRC | 01:46 | |
| zaitcev | something triggers client timeout for it | 01:46 |
| zaitcev | An iterator, I suspect | 01:46 |
| zaitcev | hmm | 01:47 |
| zaitcev | Ah, yes | 01:47 |
| zaitcev | You have to bump this one too: | 01:48 |
| zaitcev | https://github.com/openstack/swift/blob/master/etc/object-server.conf-sample#L67 | 01:48 |
| esnyder | ah! | 01:48 |
| esnyder | (sheesh, can't believe i missed it.) thanks! | 01:48 |
| esnyder | tips to where in the code that is getting used for this case? (if you have it handy, i can go dig for myself too...) | 01:49 |
| zaitcev | Well, I thought, why is this a EPIPE? Clearly the object server closed the connection. Ergo, it's in the object server somewhere. | 01:50 |
| *** lan2 has joined #openstack-swift | 01:52 | |
| *** lan2 has quit IRC | 01:53 | |
| *** jamielennox is now known as jamielennox|away | 01:54 | |
| *** links has joined #openstack-swift | 01:58 | |
| esnyder | (fyi, just verified by updating my /etc/swift/object-server/*.conf files on my test cluster object nodes, retrying my test) | 02:31 |
| *** tovin07__ has joined #openstack-swift | 02:34 | |
| zaitcev | object-server needs a restart to pick up new config (unlike e.g. the ring) | 02:35 |
| *** tovin07_ has quit IRC | 02:37 | |
| *** klrmn has joined #openstack-swift | 02:52 | |
| *** mingyu has joined #openstack-swift | 03:00 | |
| *** alenavolk has joined #openstack-swift | 03:03 | |
| *** itlinux has joined #openstack-swift | 03:29 | |
| *** klrmn has quit IRC | 03:43 | |
| *** two_tired has joined #openstack-swift | 03:46 | |
| *** psachin has joined #openstack-swift | 03:49 | |
| *** kei_yama has quit IRC | 03:55 | |
| *** alenavolk has quit IRC | 03:55 | |
| *** alenavolk has joined #openstack-swift | 03:56 | |
| *** tovin07_ has joined #openstack-swift | 03:58 | |
| *** tovin07__ has quit IRC | 04:00 | |
| *** alenavolk has quit IRC | 04:00 | |
| *** kei_yama has joined #openstack-swift | 04:01 | |
| *** alenavolk has joined #openstack-swift | 04:04 | |
| *** alenavolk has quit IRC | 04:09 | |
| *** abhitechie has joined #openstack-swift | 04:12 | |
| *** mingyu has quit IRC | 04:16 | |
| *** jamielennox|away is now known as jamielennox | 04:23 | |
| *** itlinux has quit IRC | 04:25 | |
| *** gyee has joined #openstack-swift | 04:31 | |
| *** winggundamth has joined #openstack-swift | 04:35 | |
| *** kei_yama has quit IRC | 04:35 | |
| kota_ | hello world | 04:39 |
| kota_ | notmyname: ack, sorry late response. I had to take care of my kids in this morning | 04:40 |
| notmyname | kota_: no worries. :-) | 04:40 |
| mahatic | kota_: notmyname o/ | 04:42 |
| *** chsc has joined #openstack-swift | 04:42 | |
| kota_ | notmyname: alright. I'll take my time today for that patch with my priority | 04:42 |
| notmyname | kota_: thank you | 04:42 |
| notmyname | mahatic: good morning | 04:42 |
| kota_ | mahatic: o/ | 04:43 |
| notmyname | mahatic: any thoughts on a different meeting time? I looked at a few time slots | 04:46 |
| mahatic | notmyname: oh nice, which are? I haven't looked into it yet, I was going to ping you on that | 04:49 |
| notmyname | mahatic: it seems to me that 0700UTC on Wednesdays might be a good time. actually it's pretty good for everyone except USA times ;-) | 04:49 |
| kota_ | notmyname: it's good to me too | 04:50 |
| kota_ | that is 4 pm on my wednesday :) | 04:51 |
| notmyname | that's midnight wednesday morning in san francisco. 8am london, 12:30 pm India, 4pm tokyo | 04:51 |
| mahatic | notmyname: oic, that seems good. what time will that be in china? (I could look up I know ;) | 04:52 |
| notmyname | 4pm | 04:52 |
| notmyname | no, sorry. 3pm | 04:53 |
| notmyname | one hour earlier than kota | 04:53 |
| kota_ | correct | 04:53 |
| notmyname | how about lets announce it this week on the ML and in the normal meeting, then have the first one next week (May31) | 04:54 |
| mahatic | sounds good | 04:56 |
| *** vint_bra has quit IRC | 04:57 | |
| *** chsc has quit IRC | 04:58 | |
| *** kei_yama has joined #openstack-swift | 05:06 | |
| *** jamielennox is now known as jamielennox|away | 05:11 | |
| zaitcev | Good lord | 05:17 |
| zaitcev | liberasurecode detects the instruction set of the CPU at build time | 05:17 |
| zaitcev | So, when a distribution (such as Fedora) runs ./configure and then make on Intel, the result blows up with SIGILL on AMD. | 05:18 |
| zaitcev | But only if you download from a EC container. Reconstructors and uploads are fine. | 05:18 |
| *** jaosorior_away is now known as jaosorior | 05:24 | |
| *** gyee has quit IRC | 05:32 | |
| *** bkopilov__ has quit IRC | 05:34 | |
| *** mtreinish has quit IRC | 05:38 | |
| *** cshastri has joined #openstack-swift | 05:41 | |
| *** jamielennox|away is now known as jamielennox | 05:51 | |
| *** mingyu has joined #openstack-swift | 06:00 | |
| *** rcernin has quit IRC | 06:01 | |
| *** ChubYann has quit IRC | 06:15 | |
| *** mingyu has quit IRC | 06:16 | |
| *** rcernin has joined #openstack-swift | 06:18 | |
| *** skudlik has joined #openstack-swift | 06:21 | |
| *** jamielennox is now known as jamielennox|away | 06:23 | |
| *** mingyu has joined #openstack-swift | 06:27 | |
| *** mtreinish has joined #openstack-swift | 06:33 | |
| *** pcaruana has joined #openstack-swift | 06:40 | |
| *** hseipp has joined #openstack-swift | 06:44 | |
| *** baffle has quit IRC | 07:12 | |
| *** psachin has quit IRC | 07:12 | |
| *** baffle has joined #openstack-swift | 07:12 | |
| *** rcernin has quit IRC | 07:16 | |
| *** rcernin has joined #openstack-swift | 07:17 | |
| *** psachin has joined #openstack-swift | 07:29 | |
| *** kei_yama has quit IRC | 07:37 | |
| *** psachin has quit IRC | 07:43 | |
| *** psachin has joined #openstack-swift | 07:45 | |
| *** kei_yama has joined #openstack-swift | 07:49 | |
| *** geaaru has joined #openstack-swift | 07:53 | |
| *** cbartz has joined #openstack-swift | 07:57 | |
| *** openstackgerrit has quit IRC | 08:18 | |
| *** mvk has quit IRC | 08:19 | |
| *** joeljwright has joined #openstack-swift | 08:45 | |
| *** ChanServ sets mode: +v joeljwright | 08:45 | |
| *** thurloat[m] has quit IRC | 08:46 | |
| *** mattoliverau has quit IRC | 08:46 | |
| *** nikivi has quit IRC | 08:48 | |
| *** mahatic has quit IRC | 08:49 | |
| *** mahatic has joined #openstack-swift | 08:49 | |
| *** nikivi has joined #openstack-swift | 08:50 | |
| *** thurloat[m] has joined #openstack-swift | 08:51 | |
| *** jistr is now known as jistr|call | 08:56 | |
| *** mvk has joined #openstack-swift | 08:57 | |
| *** mingyu has quit IRC | 09:15 | |
| *** jistr|call is now known as jistr | 09:19 | |
| *** mingyu has joined #openstack-swift | 09:21 | |
| *** mattoliverau has joined #openstack-swift | 09:21 | |
| *** ChanServ sets mode: +v mattoliverau | 09:21 | |
| *** mvk has quit IRC | 09:32 | |
| acoles | good morning | 09:36 |
| joeljwright | good morning | 09:37 |
| *** oshritf has joined #openstack-swift | 09:38 | |
| *** mingyu has quit IRC | 09:39 | |
| *** esnyder has quit IRC | 09:41 | |
| kota_ | hello acoles and joeljwright | 09:41 |
| acoles | kota_: joeljwright o/ | 09:41 |
| joeljwright | acoles: kota_: how was the summit? | 09:43 |
| kota_ | it makes me good progress on my proposed work but i was feeling I'd like to get more *developer* conversation there. | 09:44 |
| kota_ | joeljwright: ^^ | 09:44 |
| kota_ | joeljwright: it was a few time slot for the developer work in official. | 09:45 |
| *** mvk has joined #openstack-swift | 09:45 | |
| kota_ | we have had non-scheduled conversation though. | 09:45 |
| joeljwright | :) | 09:45 |
| joeljwright | always going to happen :) | 09:45 |
| acoles | joeljwright: like Kota said - I guess it was as expected, we made more of it than the scheduled 3 sessions by hanging out in a room, but that was a bit hit and miss, not everyone knew where we were. | 09:47 |
| joeljwright | acoles: that's a bit of a shame, hopefully the Denver PTG will be better | 09:49 |
| acoles | joeljwright: the ops feedback session was good i.e. a good number of ops in the room | 09:49 |
| joeljwright | well that's good | 09:50 |
| acoles | joeljwright: I think PTG will be much better | 09:51 |
| acoles | we met PavelK face to face which was great | 09:52 |
| acoles | and a few folks came along to an onboarding session | 09:52 |
| acoles | kota_: did you have chance to look at patch 448240 ? | 09:54 |
| patchbot | https://review.openstack.org/#/c/448240/ - swift - Enable per policy proxy config options | 09:54 |
| acoles | I see timburke has left me some comments to respond to | 09:54 |
| kota_ | acoles: right now, I'm at there | 09:54 |
| acoles | kota_: super! thank you | 09:54 |
| *** tovin07_ has quit IRC | 09:55 | |
| kota_ | acoles: but now, I'm collecting my thought on that patch.. I'm wondering a couple of things. | 09:56 |
| *** adriant has quit IRC | 09:57 | |
| acoles | kota_: ok. let me know if you want to discuss anything. | 09:57 |
| kota_ | acoles: I'm now stopping my step at https://review.openstack.org/#/c/448240/11/swift/proxy/server.py | 09:57 |
| patchbot | patch 448240 - swift - Enable per policy proxy config options | 09:57 |
| kota_ | a couple of things... | 09:58 |
| kota_ | 1. the scope - in OverrideConf | 09:58 |
| kota_ | it's actually used only in the log message | 09:58 |
| kota_ | looking at the use case, it could be "(default)" so it seems necessary which config was invalid but | 10:00 |
| acoles | kota_: yes only used in logs | 10:01 |
| kota_ | ... | 10:01 |
| kota_ | sorry, I cannot make my idea solid but feeling weird | 10:02 |
| kota_ | so in my first impression | 10:02 |
| kota_ | OverrideConf is a sort of configuration dictionary | 10:03 |
| kota_ | but it's actually not | 10:03 |
| kota_ | it keeps some attributes which were as proxy app attributes. | 10:04 |
| acoles | no, it's not a dict because ... of that ^^ | 10:04 |
| kota_ | yes, I understood how it works already | 10:04 |
| acoles | so wherever the old code called app.<foo> it now calls app.get_conf().<foo> | 10:04 |
| kota_ | yes, that's my second question | 10:05 |
| kota_ | why not using just a default dict for the conf | 10:05 |
| kota_ | get_conf is actually doing to get defaultddict[key] | 10:06 |
| *** JimCheung has joined #openstack-swift | 10:06 | |
| kota_ | deafultdict | 10:06 |
| kota_ | oops | 10:06 |
| kota_ | my typing were crazy | 10:06 |
| acoles | hehe | 10:06 |
| acoles | but override_confs is a defaultdict | 10:06 |
| acoles | line 275 in proxy/server.py | 10:07 |
| kota_ | acoles: exactly, so... why do you want to get_conf method? directly self.app._override_confs[policy] was insufficient? | 10:08 |
| kota_ | or customize the variable name as more intuitive? | 10:08 |
| acoles | OIC. yes that would be sufficient. I just preferred to hide the implementation (dict) behind a method. That way the implementation could change without the callers being affected. | 10:10 |
| *** JimCheung has quit IRC | 10:10 | |
| kota_ | so... self.app.per_policy_config[policy].sort_method or brabrabrah | 10:10 |
| kota_ | i see | 10:10 |
| acoles | kota_: It's a small thing, but the get_conf() method means the detail of how the per-policy configs are managed is encapsulated in the proxy app. | 10:13 |
| kota_ | ok, that would be meaning reason | 10:15 |
| kota_ | circle back to the 1st one, probably I felt the *config* is a dict when we call the instance as "config" | 10:17 |
| kota_ | perhaps | 10:17 |
| kota_ | OverrideOptions? | 10:17 |
| kota_ | but it's actually from config values... | 10:17 |
| acoles | OIC. | 10:18 |
| acoles | OverrideAttrs ? | 10:18 |
| kota_ | acoles: sort of | 10:18 |
| kota_ | probably that is the reason I felt it's as inconsistent name | 10:19 |
| acoles | yeah I see | 10:19 |
| acoles | so change get_conf() -> get_options() ??? | 10:20 |
| kota_ | seems more intuitive to me | 10:20 |
| kota_ | get_policy_options(policy)??? | 10:21 |
| acoles | yeah it is a potentially confusing that at top of the Application we are parsing a conf dict, then we have 'conf' used in relation to class properties | 10:21 |
| acoles | kota_: TBH I was trying to avoid making the overrides and getter method specific to policies. In back of my mind was possibility for other future custom options. Which I know is crazy. | 10:24 |
| acoles | I also like avoiding line-wraps, so probably tried to choose a short method name :) | 10:24 |
| kota_ | ok | 10:25 |
| acoles | but its important that it is intuitive to everyone so it's not a big deal for me to change | 10:25 |
| acoles | And in a future world we can always have 'get_policy_options = get_options' ;) | 10:26 |
| kota_ | sure | 10:27 |
| acoles | I mean, if there *ever* is a reason toe have a more general get_options method | 10:27 |
| acoles | to* | 10:27 |
| kota_ | sounds nice to have (to me!) | 10:28 |
| *** foutatoro has quit IRC | 10:29 | |
| acoles | kota_: choosing the right names is always so hard : https://martinfowler.com/bliki/TwoHardThings.html | 10:29 |
| kota_ | it's true | 10:30 |
| acoles | http://hilton.org.uk/blog/why-naming-things-is-hard | 10:31 |
| kota_ | acoles: to the scope thing (far from get_options), i might get good idea | 10:31 |
| kota_ | acoles: so i guess we can avoid *scope* arg from OverrideConfig | 10:32 |
| kota_ | i think, what we need is just raise ValueError in the config | 10:33 |
| kota_ | and catch and log in _load_per_policy_config | 10:33 |
| kota_ | with policy info or default | 10:34 |
| acoles | kota_: ok, but again I kinda deliberately avoided passing policy to OverrideConf, which IIRC is why scope gets assigned first and passed as an arg | 10:34 |
| kota_ | yes, we don't have to pass the policy to OverrideConf | 10:34 |
| acoles | kota_: ah. could work. there's also the check_config method at line 309 which uses scope | 10:36 |
| kota_ | oops | 10:36 |
| kota_ | i missed that | 10:36 |
| acoles | kota_: TBH that's a little weird, IDK why that is called late after all the app has been instantiated | 10:36 |
| acoles | the check could be made while parsing the conf | 10:36 |
| acoles | right now it is called from app_factory | 10:37 |
| kota_ | like https://gist.github.com/bloodeagle40234/45648a4adbaf57473a724856cdece014 | 10:38 |
| kota_ | not changed everything yet | 10:38 |
| kota_ | let me check "check_config" | 10:39 |
| kota_ | app_factory! | 10:39 |
| kota_ | er - I'm getting to remember that I also was wondering why it exists in the outside of init. | 10:41 |
| kota_ | the validation should be in app initialization, shouldn't it? | 10:42 |
| acoles | kota_: IDK if it was deliberate choice to check from app_factory | 10:44 |
| kota_ | acoles: me too | 10:45 |
| acoles | well maybe we can change that, there's nothing on the original review to explain why it is that way https://review.openstack.org/#/c/65588 | 10:46 |
| patchbot | patch 65588 - swift - Warn if read_affinity is configured but not enabled (MERGED) | 10:46 |
| kota_ | oh, it's 3 years, 4 months ago | 10:48 |
| kota_ | acoles: and it was early addoption of global cluster, make sense. i agree with you we can change it. | 10:49 |
| kota_ | but could be ok in follow-up? | 10:49 |
| kota_ | ah | 10:49 |
| kota_ | if we could do it in follow up, we cannot something on this patch | 10:50 |
| kota_ | er - yes, scope thing | 10:50 |
| kota_ | hmm... :/ | 10:50 |
| kota_ | if we could have a time before landing it, change the check_config -> per_policy_configuration seems straight forward to me | 10:51 |
| kota_ | on the other hand, notmyname seems to want to land per_policy_configuration asap | 10:52 |
| *** kei_yama has quit IRC | 10:53 | |
| acoles | kota_: ok let me work on that today, and I'll get some other opinions on moving check_config (torgomatic was original author) | 10:53 |
| kota_ | acoles: thanks. basically I like the section style override, it's awsome work. | 10:54 |
| kota_ | acoles: and let me push some other small things I found while reviewing today. | 10:54 |
| acoles | kota_: thanks for reviewing! I should be able to get a new patchset done today. | 10:54 |
| *** abhinavtechie has joined #openstack-swift | 10:56 | |
| *** abhitechie has quit IRC | 10:56 | |
| kota_ | acoles: thx! done to push my comments | 10:57 |
| * kota_ is starting to prepare to get back home | 10:58 | |
| acoles | kota_: have a good evening | 10:58 |
| kota_ | acoles: have a nice day | 10:58 |
| *** two_tired has quit IRC | 11:00 | |
| *** hseipp has quit IRC | 11:05 | |
| *** hseipp has joined #openstack-swift | 11:05 | |
| *** abhinavtechie has quit IRC | 11:08 | |
| *** hseipp has quit IRC | 11:20 | |
| *** NM has joined #openstack-swift | 11:23 | |
| *** NM has quit IRC | 11:30 | |
| *** NM has joined #openstack-swift | 11:31 | |
| *** NM has quit IRC | 11:33 | |
| *** NM has joined #openstack-swift | 11:33 | |
| *** NM has quit IRC | 11:38 | |
| *** mingyu has joined #openstack-swift | 12:01 | |
| *** mattoliverau has quit IRC | 12:09 | |
| *** thurloat[m] has quit IRC | 12:09 | |
| *** Raymii has quit IRC | 12:11 | |
| *** sgundur| has quit IRC | 12:11 | |
| *** charz_ has quit IRC | 12:11 | |
| *** Raymii has joined #openstack-swift | 12:11 | |
| *** sgundur1 has joined #openstack-swift | 12:12 | |
| *** MVenesio has joined #openstack-swift | 12:13 | |
| *** charz has joined #openstack-swift | 12:14 | |
| *** thurloat[m] has joined #openstack-swift | 12:15 | |
| *** mattoliverau has joined #openstack-swift | 12:39 | |
| *** ChanServ sets mode: +v mattoliverau | 12:39 | |
| *** catintheroof has joined #openstack-swift | 12:42 | |
| *** jamielennox|away is now known as jamielennox | 12:49 | |
| *** jaosorior has quit IRC | 12:57 | |
| *** jaosorior has joined #openstack-swift | 12:57 | |
| *** winggundamth has quit IRC | 13:08 | |
| *** klamath has joined #openstack-swift | 13:13 | |
| *** klamath has quit IRC | 13:13 | |
| *** klamath has joined #openstack-swift | 13:14 | |
| *** lucasxu has joined #openstack-swift | 13:17 | |
| *** lucasxu has quit IRC | 13:17 | |
| *** lucasxu has joined #openstack-swift | 13:30 | |
| *** NM has joined #openstack-swift | 13:43 | |
| *** chlong has joined #openstack-swift | 13:55 | |
| *** links has quit IRC | 13:57 | |
| *** oshritf has quit IRC | 14:02 | |
| *** hseipp has joined #openstack-swift | 14:05 | |
| *** mingyu has quit IRC | 14:05 | |
| *** cshastri has quit IRC | 14:42 | |
| *** psachin has quit IRC | 14:50 | |
| *** vint_bra has joined #openstack-swift | 14:52 | |
| *** itlinux has joined #openstack-swift | 14:57 | |
| notmyname | good morning | 15:03 |
| *** mingyu has joined #openstack-swift | 15:06 | |
| notmyname | kota_: thanks for reviewing the per-policy config patch | 15:07 |
| mwheckmann | morning. | 15:07 |
| *** mingyu_ has joined #openstack-swift | 15:09 | |
| *** mingyu has quit IRC | 15:09 | |
| mwheckmann | notmyname: I managed to upgrade to 2.13. There were not any dependancy problems in the end. My replication time issues were mainly down to the Rsync temp file bug and pushing too many parallel rsync's to regions with less available throughput. I had to increase Rsync timeout as well. Things are under control now. | 15:10 |
| notmyname | nice! great to hear | 15:10 |
| mwheckmann | notmyname: which leads me to a piece of feedback: it would be awesome if we could specify write/read affinity per ring instead of globally | 15:11 |
| notmyname | lol | 15:11 |
| mwheckmann | funny? :) | 15:11 |
| notmyname | you mean like https://review.openstack.org/#/c/448240/ which should land today or tomorrow? | 15:11 |
| patchbot | patch 448240 - swift - Enable per policy proxy config options | 15:11 |
| acoles | mwheckmann: working on it right now | 15:11 |
| acoles | :) | 15:11 |
| notmyname | lol because I was just talking to acoles about it :-) | 15:11 |
| mwheckmann | Nice! | 15:12 |
| mwheckmann | looking forward to that :) | 15:12 |
| *** alenavolk has joined #openstack-swift | 15:12 | |
| mwheckmann | The other piece of feedback would be a way to limit replication concurrency per region. Right now the only way to control concurrency is max Rsyncd connections. That can make the logs a little noisy. | 15:13 |
| mwheckmann | Once again, due to more BW constrained regions | 15:13 |
| notmyname | there's a larger plan in the works to eliminate rsync entirely. that will definitely take a long time to write and merge, but you're not alone in your request | 15:15 |
| notmyname | here's an etherpad with some of the bigger picture discussions on that https://etherpad.openstack.org/p/swift-rebalance | 15:16 |
| *** klrmn has joined #openstack-swift | 15:16 | |
| mwheckmann | notmyname: thanks | 15:18 |
| *** alenavolk has quit IRC | 15:21 | |
| *** rcernin has quit IRC | 15:22 | |
| *** tovin07 has joined #openstack-swift | 15:23 | |
| *** alenavolk has joined #openstack-swift | 15:28 | |
| *** tovin07 has left #openstack-swift | 15:28 | |
| *** mingyu_ has quit IRC | 15:28 | |
| *** chlong has quit IRC | 15:31 | |
| *** gyee has joined #openstack-swift | 15:32 | |
| *** mingyu has joined #openstack-swift | 15:38 | |
| *** itlinux has quit IRC | 15:39 | |
| *** chlong has joined #openstack-swift | 15:42 | |
| *** skudlik has quit IRC | 15:44 | |
| *** lucasxu has quit IRC | 15:44 | |
| *** pcaruana has quit IRC | 15:47 | |
| *** lucasxu has joined #openstack-swift | 15:49 | |
| *** alenavolk has quit IRC | 15:53 | |
| *** alenavolk has joined #openstack-swift | 15:55 | |
| *** hseipp has quit IRC | 15:55 | |
| *** geaaru has quit IRC | 16:02 | |
| notmyname | all right! let's see how many toes I can step on. http://lists.openstack.org/pipermail/openstack-dev/2017-May/117304.html | 16:13 |
| *** lucasxu has quit IRC | 16:14 | |
| notmyname | timburke: I linked your py3 gist, but I hope it doesn't become a discussion forum | 16:16 |
| *** joeljwright has quit IRC | 16:21 | |
| *** JimCheung has joined #openstack-swift | 16:26 | |
| *** chlong has quit IRC | 16:30 | |
| timburke | good morning | 16:35 |
| tdasilva | what would cause swift to have a file name like this: -rw-r--r--. 1 swift swift 3932160 May 19 18:48 .....1494882753.31470.data.3rYccn.AnOLSa.tJipc8.EKzdZv.vz9KoD | 16:36 |
| notmyname | tdasilva: partial/incomplete rsync? | 16:36 |
| timburke | that's my first reaction, too | 16:36 |
| timburke | or more likely, several failed/interrupted rsyncs | 16:37 |
| notmyname | yeah | 16:37 |
| tdasilva | ah, interesting | 16:37 |
| timburke | 5 or so? | 16:37 |
| *** pxwang has joined #openstack-swift | 16:38 | |
| mwheckmann | tdasilva: what version are you running? | 16:40 |
| *** mvk has quit IRC | 16:43 | |
| timburke | tdasilva: swift should deal with it better following https://github.com/openstack/swift/commit/1d03803 and https://github.com/openstack/swift/commit/2bd8d050 (so starting around 2.7.0) | 16:44 |
| tdasilva | mwheckmann, timburke checking | 16:44 |
| *** cbartz has quit IRC | 16:54 | |
| *** jaosorior is now known as jaosorior_away | 17:17 | |
| *** itlinux has joined #openstack-swift | 17:24 | |
| *** aselius has joined #openstack-swift | 17:33 | |
| *** Anticimex has quit IRC | 17:38 | |
| *** Anticimex has joined #openstack-swift | 17:39 | |
| *** lucasxu has joined #openstack-swift | 17:43 | |
| *** chsc has joined #openstack-swift | 17:44 | |
| *** chsc has quit IRC | 17:44 | |
| *** chsc has joined #openstack-swift | 17:44 | |
| *** klrmn has quit IRC | 17:45 | |
| *** ChubYann has joined #openstack-swift | 17:46 | |
| *** pxwang has quit IRC | 18:02 | |
| *** itlinux has quit IRC | 18:02 | |
| *** klrmn has joined #openstack-swift | 18:09 | |
| *** itlinux has joined #openstack-swift | 18:12 | |
| *** oshritf has joined #openstack-swift | 18:29 | |
| *** oshritf has quit IRC | 18:30 | |
| *** mingyu has quit IRC | 18:33 | |
| clayg | cschwede_: tdasilva: I just saw patch 466255 - I'm confused and would like some additional details about *that* specific change - but more importantly I'm curious to brainstorm some ideas on how the upstream community can better support these contributions! Can I put something on tomorrow's meeting agenda? | 18:33 |
| patchbot | https://review.openstack.org/#/c/466255/ - swift - Make mount_check option usable in containerized en... | 18:33 |
| *** mingyu has joined #openstack-swift | 18:33 | |
| *** itlinux has quit IRC | 18:34 | |
| *** oshritf has joined #openstack-swift | 18:34 | |
| tdasilva | clayg: sure, i'm not sure cschwede_ will be around tomorrow, but I can try to answer any questions | 18:36 |
| tdasilva | clayg: just read your comments, and yeah, i agree, my initial feeling was similar, "this is a hack" and honestly I feel like cschwede_ feels the same | 18:38 |
| clayg | hrmmm... I wonder if we could do anything to get him around... specifically I'm smelling some redhat on the container train and I want to have an open an honest brainstorm about how to a) allow for fast rapid meaningful progress in a way that is rewarding to those contributing and b) doesn't leave us in a bind if the end of the experiment doesn't land in a | 18:38 |
| clayg | place that we all feel proud of | 18:38 |
| *** mingyu has quit IRC | 18:39 | |
| tdasilva | clayg: Christian described the problem well on the commit msg. The issue is that we want to bind mount the disk on the container, but you can do that for only one container ??? not multiple | 18:39 |
| tdasilva | so I think the plan was to mount the device on the host machine, and then directory /srv/node/sdb[n]1 on the containers | 18:40 |
| tdasilva | at least that's my understanding | 18:41 |
| *** Renich has joined #openstack-swift | 18:41 | |
| clayg | basically I'm thinking feature-branch - single +2 sorta thing - get some rapid progress/sprint going - make mistakes and go back to fix them - maybe try and get some new people for surrounding ecosystem or larger openstack community "interested" without all the "this needs docs - i don't know anything about containers but have you considered - blah balh | 18:41 |
| clayg | balh" stuff that comes with a 5 year old project that we depend on everyday in production and is going to be maintained for a long time even if awz eats the world and we all decide to retire and go fishing. | 18:41 |
| *** tonanhngo has joined #openstack-swift | 18:42 | |
| *** itlinux has joined #openstack-swift | 18:43 | |
| tdasilva | clayg: FWIW, for now, that's the only patch that I'm aware of in Swift....TripleO is already running swift containerized, without mount-check | 18:43 |
| clayg | tdasilva: sounds reasonable to me! My gut is "why doesn't mount_check = false work" - but I'm not really in a posistion to drop working on composite rings and try and get swift in a container to answer that question for myself? | 18:43 |
| timburke | clayg: i think it's worth noting that cschwede_ (i'm fairly certain) wants mount_check to still be useful in such an environment -- sure, `mount_check = false` "works", but you've now lost that check and when your container provisioning goes awry, you (presumably) get some very very tiny object nodes | 18:44 |
| tdasilva | right | 18:44 |
| tdasilva | so we have mount_check = false today, and this patch was to just enable that | 18:44 |
| clayg | oh, yeah - do we not have a bug for that yet? | 18:45 |
| clayg | basically we do some `mkdir -p` call when the part dir doesn't exist - but we should *never* create the device dir | 18:46 |
| clayg | it's sad because the whole mkdirs call thing was unrolled by portante back in the day - but we never followed through to prevent the creation of a device dir resulting in writing data onto the root filesystem - too bad | 18:46 |
| *** tonanhngo has quit IRC | 18:49 | |
| acoles | timburke: re per-policy deployment guide changes, do you know how to make a link to a table line in rst? I spent a while trying and failing. | 18:50 |
| timburke | acoles: yeah, me to. screw it, tooling doesn't seem to allow it. i think i still prefer a bulleted list over the code block, though | 18:51 |
| acoles | timburke: oh, yeah, you got your bullets | 18:51 |
| acoles | just no links | 18:51 |
| *** itlinux has quit IRC | 19:01 | |
| portante | clayg, yeah, I failed to make that complete | 19:01 |
| clayg | portante: the failure is ours - you are winner in my book | 19:02 |
| * clayg is working on the bug | 19:02 | |
| clayg | er.. bug *report* :P | 19:02 |
| *** lucasxu has quit IRC | 19:02 | |
| portante | clayg: thanks! | 19:03 |
| *** mvk has joined #openstack-swift | 19:06 | |
| *** JimCheung has quit IRC | 19:09 | |
| *** Renich_ has joined #openstack-swift | 19:14 | |
| *** lucasxu has joined #openstack-swift | 19:15 | |
| *** Renich has quit IRC | 19:18 | |
| *** openstackgerrit has joined #openstack-swift | 19:23 | |
| openstackgerrit | Alistair Coles proposed openstack/swift master: Enable per policy proxy config options https://review.openstack.org/448240 | 19:23 |
| acoles | kota_: ^^ | 19:23 |
| acoles | timburke: ^^ | 19:23 |
| *** JimCheung has joined #openstack-swift | 19:24 | |
| clayg | tdasilva: timburke: thanks for the insight on patch 466255 - I'm +2 I think | 19:27 |
| patchbot | https://review.openstack.org/#/c/466255/ - swift - Make mount_check option usable in containerized en... | 19:27 |
| tdasilva | clayg: thanks | 19:28 |
| timburke | i *think* that might even mean that we can meaningfully enable mount_check in the vsaio? | 19:28 |
| acoles | timburke: I wasn't sure which way to jump on this question - remove the check completely, or beef it up to include IOErrors, or narrow it to the __file__ key. | 19:28 |
| acoles | timburke: https://review.openstack.org/#/c/448240/11/swift/proxy/server.py@671 | 19:28 |
| patchbot | patch 448240 - swift - Enable per policy proxy config options | 19:28 |
| *** JimCheung has quit IRC | 19:29 | |
| timburke | acoles: yeah, i wasn't sure either. part of why i framed it as a question rather than a suggestion :-) | 19:29 |
| timburke | probably fine? not really worried about it | 19:30 |
| acoles | timburke: can we throw it on to the follow-up heap? | 19:30 |
| timburke | yeah, that seems fine. i think we can just scrap the check? probably | 19:31 |
| tdasilva | notmyname, clayg: did you see the message about tripleo hackathon? seems like we are not the only ones thinking about that: http://lists.openstack.org/pipermail/openstack-dev/2017-May/117263.html | 19:31 |
| tdasilva | still i'm not sure how well it would work or how different it would feel from just a regular video-conference call meeting | 19:32 |
| clayg | tdasilva: that's super cool - are you (or anyone you know) going to attend? Perhaps we can get some ideas? | 19:32 |
| tdasilva | clayg: I should probably log in at some point to at least get an idea of how it is going, see if we can learn anything from them | 19:33 |
| tdasilva | clayg: did you know tripleo actually relies heavily on swift for deployment? it's pretty neat, they have this concept of undercloud and overcloud. The undercloud is just one VM and it runs saio on it | 19:34 |
| clayg | tdasilva: I can't even *spell* OOO! | 19:35 |
| tdasilva | it's a single replica saio, but they make extensive use of tempurls | 19:35 |
| clayg | that's crazy!? | 19:35 |
| tdasilva | yeah, i think they store a bunch of artifacts that are then downloaded by the overcloud nodes during deployment | 19:36 |
| *** mingyu has joined #openstack-swift | 19:39 | |
| *** JimCheung has joined #openstack-swift | 19:41 | |
| *** mvk has quit IRC | 19:43 | |
| *** mingyu has quit IRC | 19:44 | |
| *** JimCheung has quit IRC | 19:45 | |
| *** oshritf has quit IRC | 19:46 | |
| *** itlinux has joined #openstack-swift | 19:48 | |
| *** JimCheung has joined #openstack-swift | 19:49 | |
| *** NM has quit IRC | 19:54 | |
| notmyname | crazy awesome! | 19:55 |
| *** NM has joined #openstack-swift | 19:56 | |
| *** vint_bra has quit IRC | 19:56 | |
| *** NM has quit IRC | 19:56 | |
| *** NM has joined #openstack-swift | 20:00 | |
| *** vint_bra has joined #openstack-swift | 20:02 | |
| *** Renich_ has quit IRC | 20:08 | |
| *** Renich has joined #openstack-swift | 20:10 | |
| *** Renich has quit IRC | 20:10 | |
| *** lucasxu has quit IRC | 20:17 | |
| *** lucasxu has joined #openstack-swift | 20:21 | |
| *** alenavolk has quit IRC | 20:23 | |
| *** rcernin has joined #openstack-swift | 20:24 | |
| *** alenavolk has joined #openstack-swift | 20:24 | |
| *** alenavolk has quit IRC | 20:43 | |
| *** oshritf has joined #openstack-swift | 20:52 | |
| *** chsc has quit IRC | 20:56 | |
| *** itlinux has quit IRC | 21:00 | |
| *** lucasxu has quit IRC | 21:03 | |
| *** chsc has joined #openstack-swift | 21:08 | |
| *** oshritf has quit IRC | 21:16 | |
| -openstackstatus- NOTICE: The logserver has filled up, so jobs are currently aborting with POST_FAILURE results; remediation is underway. | 21:23 | |
| *** ChanServ changes topic to "The logserver has filled up, so jobs are currently aborting with POST_FAILURE results; remediation is underway." | 21:23 | |
| *** NM has quit IRC | 21:29 | |
| *** vint_bra has quit IRC | 21:29 | |
| *** vint_bra has joined #openstack-swift | 21:32 | |
| *** catintheroof has quit IRC | 21:33 | |
| *** alenavolk has joined #openstack-swift | 21:36 | |
| *** alenavolk has quit IRC | 21:43 | |
| *** MVenesio has quit IRC | 21:46 | |
| *** jroll has quit IRC | 21:47 | |
| *** jroll has joined #openstack-swift | 21:47 | |
| *** alenavolk has joined #openstack-swift | 21:47 | |
| *** jroll has quit IRC | 21:49 | |
| *** alenavolk has quit IRC | 21:52 | |
| *** jroll has joined #openstack-swift | 21:53 | |
| *** stradling has quit IRC | 22:24 | |
| *** stradling has joined #openstack-swift | 22:24 | |
| *** klamath has quit IRC | 22:51 | |
| *** vint_bra has quit IRC | 22:59 | |
| *** adriant has joined #openstack-swift | 23:16 | |
| *** NM has joined #openstack-swift | 23:16 | |
| *** tristanC has quit IRC | 23:17 | |
| *** jungleboyj has quit IRC | 23:26 | |
| *** jungleboyj has joined #openstack-swift | 23:26 | |
| *** kei_yama has joined #openstack-swift | 23:31 | |
| *** chsc has quit IRC | 23:39 | |
| *** NM has quit IRC | 23:56 | |
| openstackgerrit | Pete Zaitcev proposed openstack/liberasurecode master: Stop using ceill() to compute padded data size https://review.openstack.org/467418 | 23:57 |
| openstackgerrit | Tim Burke proposed openstack/swift master: Remove tempauth docs from deployment guide https://review.openstack.org/466953 | 23:59 |
| openstackgerrit | Tim Burke proposed openstack/swift master: Follow-up for per-policy proxy configs https://review.openstack.org/466952 | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!