| *** tdasilva has joined #openstack-swift | 00:03 | |
| *** caiobrentano_ has joined #openstack-swift | 00:11 | |
| *** caiobrentano_ has quit IRC | 00:12 | |
| clayg | so it turns out file systems reuse inode numbers because of course they do | 00:19 |
|---|---|---|
| clayg | is it really so hard to ask has the data in this file changed since I read it earlier? | 00:20 |
| zaitcev | mtime | 00:38 |
| clayg | unless you're *really* quick | 00:38 |
| clayg | anyone know cindy? lp bug #1659147 | 00:41 |
| openstack | Launchpad bug 1659147 in OpenStack Object Storage (swift) "swift-get-nodes not working in "full" mitaka " [Undecided,New] https://launchpad.net/bugs/1659147 | 00:41 |
| patchbot | Error: No closing quotation | 00:41 |
| zaitcev | Yes, you have assume the worst if times match. | 00:41 |
| clayg | right - so if you expected times to mostly *not* match and sometimes match that would be great - but we expect times to mostly match - only even when the match - you might still be racing | 00:42 |
| clayg | so we took to mocking it in tests | 00:43 |
| clayg | then I thought I'd use inode because it's good enough for os.path.samefile | 00:43 |
| clayg | then I thought I'd use inode + mtime because ... probabilities? | 00:43 |
| clayg | then I thought - i hate computers - so I'm going to do this when I need to know if the content in the file is the same as when I read it earlier | 00:44 |
| clayg | *read the content and see if it's the same as what I read earlier* | 00:44 |
| clayg | crazy right? | 00:44 |
| clayg | it's not like this is 5GB of data - these are less than a block size - i could probably md5 instead of deserialize - maybe that's more attractive by some measure | 00:45 |
| zaitcev | you can compare hashes but yes, you have to read the current state. Unless you want to use locking. | 00:45 |
| *** vinsh has quit IRC | 01:06 | |
| *** ChubYann has quit IRC | 01:13 | |
| *** chosafine has joined #openstack-swift | 01:14 | |
| *** sams-gleb has joined #openstack-swift | 01:22 | |
| *** ChubYann has joined #openstack-swift | 01:25 | |
| *** sams-gleb has quit IRC | 01:28 | |
| *** tqtran has quit IRC | 01:37 | |
| *** vint_bra has quit IRC | 01:47 | |
| *** clu_ has quit IRC | 01:57 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Better optimistic lock in get_hashes https://review.openstack.org/419787 | 02:02 |
| *** vint_bra has joined #openstack-swift | 02:03 | |
| *** vint_bra has quit IRC | 02:10 | |
| clayg | ^ tdasilva onovy timburke jrichli mahatic mattoliverau notmyname acoles_ this change is my everything | 02:14 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Better optimistic lock in get_hashes https://review.openstack.org/419787 | 02:23 |
| *** two_tired has joined #openstack-swift | 02:29 | |
| *** winggundamth has joined #openstack-swift | 02:35 | |
| *** JimCheung has quit IRC | 02:39 | |
| *** ppai has joined #openstack-swift | 03:16 | |
| *** sams-gleb has joined #openstack-swift | 03:25 | |
| *** sams-gleb has quit IRC | 03:30 | |
| *** ppai has quit IRC | 03:33 | |
| *** JimCheung has joined #openstack-swift | 03:35 | |
| *** JimCheung has quit IRC | 03:39 | |
| *** chsc has joined #openstack-swift | 03:41 | |
| *** chsc has joined #openstack-swift | 03:41 | |
| *** vinsh has joined #openstack-swift | 03:42 | |
| *** vinsh has quit IRC | 03:43 | |
| *** chsc has quit IRC | 03:57 | |
| *** SkyRocknRoll has joined #openstack-swift | 04:00 | |
| *** kei_yama has joined #openstack-swift | 04:07 | |
| *** abhitechie has joined #openstack-swift | 04:34 | |
| *** ppai has joined #openstack-swift | 04:41 | |
| *** psachin has joined #openstack-swift | 04:42 | |
| *** mkrai_ has joined #openstack-swift | 04:42 | |
| *** chosafine has quit IRC | 05:02 | |
| *** jamielennox is now known as jamielennox|away | 05:06 | |
| *** klrmn has quit IRC | 05:16 | |
| *** sams-gleb has joined #openstack-swift | 05:28 | |
| *** abhitechie has quit IRC | 05:32 | |
| *** sams-gleb has quit IRC | 05:32 | |
| *** tqtran has joined #openstack-swift | 05:37 | |
| *** tqtran has quit IRC | 05:41 | |
| *** abhitechie has joined #openstack-swift | 05:42 | |
| *** dmorita has quit IRC | 05:48 | |
| *** two_tired has quit IRC | 05:49 | |
| *** dmorita has joined #openstack-swift | 05:49 | |
| *** dmorita has quit IRC | 05:54 | |
| *** dmorita has joined #openstack-swift | 06:36 | |
| *** chsc has joined #openstack-swift | 06:36 | |
| *** dmorita has quit IRC | 06:40 | |
| *** chsc has quit IRC | 06:42 | |
| *** ChubYann has quit IRC | 06:47 | |
| *** Jeffrey4l_ has quit IRC | 06:48 | |
| *** Jeffrey4l has joined #openstack-swift | 06:53 | |
| *** abhitechie has quit IRC | 07:06 | |
| *** SkyRocknRoll has quit IRC | 07:25 | |
| *** pcaruana has joined #openstack-swift | 07:28 | |
| *** sams-gleb has joined #openstack-swift | 07:30 | |
| *** tesseract has joined #openstack-swift | 07:34 | |
| *** sams-gleb has quit IRC | 07:35 | |
| *** jlwhite has quit IRC | 07:40 | |
| *** sams-gleb has joined #openstack-swift | 07:42 | |
| *** abhitechie has joined #openstack-swift | 07:43 | |
| *** abhitechie has quit IRC | 07:43 | |
| *** hseipp has joined #openstack-swift | 07:50 | |
| *** mingyu has joined #openstack-swift | 07:54 | |
| *** si1ver has quit IRC | 08:04 | |
| *** rledisez has joined #openstack-swift | 08:10 | |
| *** oshritf_ has joined #openstack-swift | 08:27 | |
| *** mkrai_ has quit IRC | 08:29 | |
| *** Jeffrey4l_ has joined #openstack-swift | 08:30 | |
| *** Jeffrey4l has quit IRC | 08:34 | |
| *** sams-gleb has quit IRC | 08:38 | |
| *** jlwhite has joined #openstack-swift | 08:39 | |
| *** geaaru has joined #openstack-swift | 08:42 | |
| *** si1ver has joined #openstack-swift | 08:56 | |
| *** cbartz has joined #openstack-swift | 09:03 | |
| *** oshritf_ has quit IRC | 09:07 | |
| *** jordanP has joined #openstack-swift | 09:16 | |
| *** oshritf_ has joined #openstack-swift | 09:16 | |
| *** tqtran has joined #openstack-swift | 09:24 | |
| *** geaaru has quit IRC | 09:28 | |
| *** tqtran has quit IRC | 09:28 | |
| *** geaaru has joined #openstack-swift | 09:30 | |
| *** winggundamth_ has joined #openstack-swift | 09:33 | |
| *** winggundamth has quit IRC | 09:35 | |
| *** JimCheung has joined #openstack-swift | 09:40 | |
| *** m_kazuhiro has joined #openstack-swift | 09:44 | |
| *** JimCheung has quit IRC | 09:45 | |
| m_kazuhiro | good morning. | 09:45 |
| openstackgerrit | Kazuhiro MIYAHARA proposed openstack/swift: WIP: Swift Automated Tiering https://review.openstack.org/287057 | 09:49 |
| *** m_kazuhiro has quit IRC | 09:52 | |
| *** m_kazuhiro has joined #openstack-swift | 09:53 | |
| *** manous has joined #openstack-swift | 10:05 | |
| *** manous has quit IRC | 10:09 | |
| *** openstackgerrit has quit IRC | 10:17 | |
| *** lespaul has quit IRC | 10:23 | |
| *** mvk has quit IRC | 10:23 | |
| *** kei_yama has quit IRC | 10:24 | |
| *** mingyu has quit IRC | 10:29 | |
| *** sams-gleb has joined #openstack-swift | 10:31 | |
| *** rledisez has quit IRC | 10:42 | |
| *** rledisez has joined #openstack-swift | 10:44 | |
| *** sams-gleb has quit IRC | 10:47 | |
| *** sams-gleb has joined #openstack-swift | 10:48 | |
| *** dmorita has joined #openstack-swift | 10:48 | |
| *** jordanP has quit IRC | 10:51 | |
| *** dmorita has quit IRC | 10:52 | |
| *** sams-gleb has quit IRC | 10:52 | |
| *** jordanP has joined #openstack-swift | 10:54 | |
| *** mvk has joined #openstack-swift | 10:56 | |
| *** vatlas has joined #openstack-swift | 10:57 | |
| *** dfflanders has quit IRC | 10:59 | |
| *** jordanP has quit IRC | 11:04 | |
| *** sams-gleb has joined #openstack-swift | 11:05 | |
| *** openstackgerrit has joined #openstack-swift | 11:10 | |
| openstackgerrit | Gábor Antal proposed openstack/swift: Use more specific asserts in test/unit/common/middleware https://review.openstack.org/342770 | 11:10 |
| *** ganders has joined #openstack-swift | 11:13 | |
| *** m_kazuhiro has quit IRC | 11:16 | |
| *** mkrai has joined #openstack-swift | 11:22 | |
| *** cdelatte has quit IRC | 11:23 | |
| *** mingyu has joined #openstack-swift | 11:29 | |
| *** vatlas has quit IRC | 11:33 | |
| *** mingyu has quit IRC | 11:34 | |
| openstackgerrit | Christian Schwede proposed openstack/swift: Add support to increase object ring partition power https://review.openstack.org/337297 | 11:57 |
| *** cdelatte has joined #openstack-swift | 12:10 | |
| *** d0ugal has quit IRC | 12:10 | |
| *** sams-gleb has quit IRC | 12:14 | |
| *** sams-gleb has joined #openstack-swift | 12:14 | |
| *** sams-gleb has quit IRC | 12:15 | |
| *** sams-gleb has joined #openstack-swift | 12:15 | |
| *** d0ugal has joined #openstack-swift | 12:19 | |
| *** catintheroof has joined #openstack-swift | 12:28 | |
| openstackgerrit | Jan Krcmar proposed openstack/python-swiftclient: Add kerberos http authentication https://review.openstack.org/424581 | 12:43 |
| -openstackstatus- NOTICE: Gerrit is going to be restarted due to slow performance | 12:47 | |
| -openstackstatus- NOTICE: Gerrit has been successfully restarted | 12:50 | |
| *** sams-gle_ has joined #openstack-swift | 13:15 | |
| *** sams-gleb has quit IRC | 13:15 | |
| *** ppai has quit IRC | 13:22 | |
| *** tqtran has joined #openstack-swift | 13:25 | |
| *** tongli has joined #openstack-swift | 13:27 | |
| *** NM has joined #openstack-swift | 13:28 | |
| *** tqtran has quit IRC | 13:30 | |
| *** mingyu has joined #openstack-swift | 13:31 | |
| *** mingyu has quit IRC | 13:36 | |
| *** jordanP has joined #openstack-swift | 13:39 | |
| *** caiobrentano_ has joined #openstack-swift | 13:43 | |
| *** acoles_ is now known as acoles | 13:44 | |
| *** sams-gleb has joined #openstack-swift | 13:51 | |
| *** sams-gle_ has quit IRC | 13:51 | |
| *** psachin has quit IRC | 14:22 | |
| *** winggundamth_ has quit IRC | 14:31 | |
| *** mingyu has joined #openstack-swift | 14:32 | |
| *** mingyu has quit IRC | 14:37 | |
| *** tongli has quit IRC | 14:42 | |
| *** mvk has quit IRC | 14:55 | |
| *** catinthe_ has joined #openstack-swift | 15:00 | |
| *** catintheroof has quit IRC | 15:02 | |
| *** vint_bra has joined #openstack-swift | 15:06 | |
| *** sams-gleb has quit IRC | 15:09 | |
| *** sams-gleb has joined #openstack-swift | 15:10 | |
| *** oshritf_ has quit IRC | 15:13 | |
| *** sams-gleb has quit IRC | 15:14 | |
| *** sams-gleb has joined #openstack-swift | 15:28 | |
| *** vinsh has joined #openstack-swift | 15:31 | |
| *** oshritf has joined #openstack-swift | 15:37 | |
| *** catintheroof has joined #openstack-swift | 15:43 | |
| *** catinthe_ has quit IRC | 15:46 | |
| *** oshritf has quit IRC | 15:54 | |
| *** klrmn has joined #openstack-swift | 16:12 | |
| *** chsc has joined #openstack-swift | 16:14 | |
| *** chsc has joined #openstack-swift | 16:14 | |
| openstackgerrit | Thiago da Silva proposed openstack/swift: remove func-fast-post from tox.ini https://review.openstack.org/425264 | 16:17 |
| *** _JZ_ has joined #openstack-swift | 16:19 | |
| *** chsc has quit IRC | 16:20 | |
| *** mvk has joined #openstack-swift | 16:25 | |
| *** chosafine has joined #openstack-swift | 16:25 | |
| *** chosafine has quit IRC | 16:26 | |
| *** zul has quit IRC | 16:28 | |
| *** zul has joined #openstack-swift | 16:29 | |
| *** mingyu has joined #openstack-swift | 16:33 | |
| *** mingyu has quit IRC | 16:38 | |
| tdasilva | rledisez: hi, was just reading your comments on patch 287057 and I have been thinking that maybe patch 212824 could be useful for the issue you brought up | 16:47 |
| patchbot | https://review.openstack.org/#/c/287057/ - swift - WIP: Swift Automated Tiering | 16:47 |
| patchbot | https://review.openstack.org/#/c/212824/ - swift - Let developers/operators add watchers to object audit | 16:47 |
| openstackgerrit | Christopher Bartz proposed openstack/python-swiftclient: ISO 8601 timestamps for tempurl https://review.openstack.org/423377 | 16:51 |
| rledisez | tdasilva: hi. i'm not big fan of disk scanning. i mean, i really hate that :) | 16:55 |
| rledisez | tdasilva: i sometime think i could stop the auditor because in the end, there is always opportunities to quarantine a corrupted object (customer access, replication during rebalance) | 16:55 |
| rledisez | tdasilva: i prefer the method to register a task in a container and distributing the load accross object servers | 16:55 |
| rledisez | tdasilva, but it's the same topic, avoiding code duplication | 16:55 |
| notmyname | good morning | 16:56 |
| *** chsc has joined #openstack-swift | 16:56 | |
| tdasilva | rledisez: I agree that scanning is not good and I think that was torgomatic intention with that patch, to try to limit how much scanning is done and try to do as much as possible in a single scan pass | 16:58 |
| rledisez | notmyname: hello. i was wondering about the etherpad for PTG, is topic list "open". i mean, can i ask a topic ? (small files) | 16:59 |
| *** dmorita has joined #openstack-swift | 16:59 | |
| notmyname | rledisez: yes! absolutely | 16:59 |
| tdasilva | rledisez: OTOH, there's scanning that reads the data (and take a long time) and there's scanning that don't read the data | 17:00 |
| *** jordanP has quit IRC | 17:00 | |
| tdasilva | rledisez: either way I think what m_kazuhiro is proposing is to register tasks in a container, but to do that, he needs to first scan the files to discover what needs to be moved, no? | 17:01 |
| rledisez | tdasilva: "and there's scanning that don't read the data" => does not mean it's fast, not for us at least, mainly because every directory access means disk access because inode are evicted from cache quickly | 17:04 |
| *** dmorita has quit IRC | 17:04 | |
| rledisez | tdasilva: in our experience, just scanning files on one disk is few days | 17:04 |
| tdasilva | rledisez: without reading data? | 17:05 |
| rledisez | tdasilva: yes ;) | 17:05 |
| rledisez | tdasilva: i'm wondering if it would be ok for some kind of task to only work on new uploads so that registration happens at upload (like object expirer) | 17:05 |
| rledisez | and providing a tool that scan objects once for ops that need to work on past object too | 17:06 |
| tdasilva | rledisez: i was thinking if it would make sense to scan container server? | 17:06 |
| *** Jeffrey4l_ has quit IRC | 17:06 | |
| tdasilva | to build up the list of what needs to be moved | 17:06 |
| rledisez | could be smart, it's easier to scale, and often on SSD so few impact on latencies. and still, it would be a "run once" job | 17:07 |
| tdasilva | and by scan, i mean traverse the container list, but | 17:07 |
| tdasilva | or a db query | 17:08 |
| notmyname | rledisez: oh! I wanted to say that I'm very interested in the small-file optimization work you wrote up on the wiki. it looks very very interesting, and I think your approach might be better than my first idea | 17:09 |
| notmyname | tdasilva: rledisez: on the topic of drive scans being slow... | 17:09 |
| *** joeljwright has joined #openstack-swift | 17:10 | |
| *** ChanServ sets mode: +v joeljwright | 17:10 | |
| notmyname | tdasilva: rledisez: not sure how much specifics I can share yet (currently in progress writing up somethign that will be public), but I've been working with Intel on their CAS software, and I've seen great results when using it to cache filesystem metadata (eg replication cycle time being 5-6x faster) | 17:10 |
| rledisez | notmyname: i have a colleague working on that for few weeks now. for now, we are very close of your original idea for one main reason: being compatible with the existing deployment. it would be insane to have to re-format everything in some kind of ZFS or else... | 17:12 |
| rledisez | but it could be an intermediary step | 17:13 |
| *** klrmn has quit IRC | 17:13 | |
| notmyname | rledisez: interesting. I really like your "put it in a DB" idea (especially after talking with an XFS dev last week) | 17:13 |
| rledisez | oh, yes, that we have it now. very efficient. about 20 bytes per object in memory :) | 17:14 |
| rledisez | it opens many possibilities | 17:14 |
| tdasilva | "that we have it now" ? | 17:16 |
| tdasilva | you mean in prod? | 17:16 |
| tdasilva | in swift? | 17:16 |
| rledisez | no, not in prod, not yet at least :D | 17:17 |
| rledisez | in the dev version | 17:17 |
| tdasilva | heh | 17:17 |
| notmyname | yeah, which part of https://wiki.openstack.org/wiki/Swift/ideas/small_files/experimentations do you have? | 17:17 |
| rledisez | it's actually very close to haystack in the idea. we were concerned about the compaction, so we tried to reduce it with the use of fallocate PUNCH HOLE | 17:18 |
| rledisez | so we free space without resizing the "volumes" | 17:18 |
| rledisez | and a key value database to locate data in the volumes/ ohash -> volume,offset | 17:19 |
| rledisez | the bigger problem for now is eventlet :/ | 17:20 |
| notmyname | rledisez: I'd expect to have some pretty tricky problems when trying to manage the gc/compaction of slab files. In fact, it might be nearly a whole filesystem to do that. So that's why the idea of "just store small files in a DB" is interesting. Turns out swift is pretty good at understanding and managing sqlite DBs | 17:21 |
| notmyname | what's the eventlet problem? | 17:21 |
| *** Jeffrey4l has joined #openstack-swift | 17:25 | |
| *** tqtran has joined #openstack-swift | 17:26 | |
| tdasilva | notmyname: cool info on the CAS software, looking forward to your future report | 17:29 |
| *** Jeffrey4l has quit IRC | 17:30 | |
| rledisez | notmyname: mainly process hanging without obvious reason. sometimes a thread closing the wring file descriptor. using a C module create some weird behavior too | 17:32 |
| notmyname | hmm..interesting | 17:32 |
| *** joeljwright has quit IRC | 17:32 | |
| rledisez | notmyname: there is a lot of bug reports about that kind of stuff | 17:32 |
| rledisez | monkey patching is not that magic :D | 17:32 |
| notmyname | rledisez: we're going to rewrite all that in golang, though, right? ;-) | 17:33 |
| rledisez | notmyname: yeah! | 17:33 |
| *** tesseract has quit IRC | 17:33 | |
| *** BlackDex has left #openstack-swift | 17:34 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Bring docs inline with changes to tox envs https://review.openstack.org/425311 | 17:34 |
| acoles | tdasilva: jrichli small follow up to recent tox changes ^^ | 17:35 |
| rledisez | notmyname: very interested too about CAS. we had some contact few weeks ago with intel R&D in poland about that, i think some colleague are about to test it on ceph | 17:36 |
| jrichli | acoles: looks good, thx | 17:36 |
| acoles | jrichli: tdasilva is doing the significant work :) | 17:37 |
| clayg | woooo! | 17:38 |
| *** JimCheung has joined #openstack-swift | 17:39 | |
| *** dmorita has joined #openstack-swift | 17:39 | |
| tdasilva | acoles: lol, timburke did all the work | 17:39 |
| acoles | clayg: I've been on other stuff today but I'll try to look at your "everything" patch soon, looks like you made a lot of progress | 17:39 |
| clayg | mahatic: that stupid comment was like a honey pot - I thought about leaving it out - but I was like "nah, *someone* is going to ask why it's there" | 17:40 |
| clayg | acoles: the inode thing didnt' work out :'( | 17:40 |
| clayg | but then sleeping on it I had this idea that maybe if I held an open file handle the whole time the file system couldn't reuse *my* inode - but idk, that's an even bigger refactor | 17:40 |
| acoles | clayg: and yeah sorry I should have squashed patch 421385 and patch 422076 rather than having the to-ing and fro-ing in the tests :/ | 17:41 |
| patchbot | https://review.openstack.org/#/c/421385/ - swift - Add more tests for diskfile consolidate_hashes | 17:41 |
| patchbot | https://review.openstack.org/#/c/422076/ - swift - Handle unreadable hashes.pkl same way as non-exist... | 17:41 |
| clayg | this could easily be adjusted to just checksum the contents instead of deserialization if we find that's significantly faster | 17:41 |
| openstackgerrit | Joel Wright proposed openstack/swift: Add Preamble and Postamble to SLO and SegmentedIterable https://review.openstack.org/365371 | 17:41 |
| acoles | tdasilva: true, sorry timburke ! | 17:41 |
| tdasilva | :) | 17:42 |
| clayg | acoles: that was my conclusion as well - I was happy to see the tests work out (in the good final state) after rebasing on patch 419787 | 17:42 |
| patchbot | https://review.openstack.org/#/c/419787/ - swift - Better optimistic lock in get_hashes | 17:42 |
| clayg | acoles: I really did rip off your "handle missing/bad/etc hashes all the same" | 17:42 |
| acoles | clayg: it simply reflected my workflow - one day I wrote some tests while reviewing your patch, next day I wrote a fix, next day I didn't go back and squash them | 17:43 |
| *** JimCheung has quit IRC | 17:43 | |
| clayg | but there may be even more better goodness if we keep working on it - but for now I think patch 419787 *could* be a "stopping" point | 17:43 |
| patchbot | https://review.openstack.org/#/c/419787/ - swift - Better optimistic lock in get_hashes | 17:43 |
| clayg | acoles: k, np - makes total sense - i work like that as well | 17:44 |
| acoles | clayg: yeah, I had a quick scan over your latest. we better land it before you think too much more ;) | 17:44 |
| *** JimCheung has joined #openstack-swift | 17:46 | |
| *** jcaron has quit IRC | 17:53 | |
| *** rledisez has quit IRC | 17:56 | |
| timburke | what's this? who's accusing me of doing what work? | 17:58 |
| *** hseipp has quit IRC | 17:59 | |
| clayg | timburke: it was tdasilva - he said it was all your fault (but we all know it's notmyname's fault) | 18:00 |
| notmyname | it normally is | 18:00 |
| tdasilva | yep | 18:00 |
| timburke | PTL-for-life: it's a blessing *and* a curse | 18:00 |
| clayg | for notmyname - a curse - for everyone not notmyname - a blessing | 18:01 |
| tdasilva | you mean, blessing to us and curse to notmyname, right? | 18:01 |
| clayg | tdasilva: jinx! | 18:01 |
| notmyname | lol | 18:01 |
| notmyname | cschwede: FYI https://www.heise.de/newsticker/meldung/Call-for-Papers-Deutsche-OpenStack-Tage-2017-3607542.html?wt_mc=rss.ho.beitrag.atom&utm_source=dlvr.it&utm_medium=twitter | 18:02 |
| acoles | cschwede: I understand the photo :) | 18:04 |
| clayg | google translate in chrome is getting *scary* good - like star trek good | 18:04 |
| clayg | acoles: oh, what is up with that swift-get-nodes bug report? | 18:06 |
| tdasilva | clayg: is that an extension? | 18:06 |
| clayg | tdasilva: my chrome just asked me if i wanted to transalte - when I clicked yes it was all like well spoken nice grammer english explain all about the OpenStack Days in Germany | 18:07 |
| clayg | maybe technical news is easier to translate than some other subjects - just seemed to do a better job than I normally expect form google translate | 18:07 |
| tdasilva | neat | 18:08 |
| acoles | clayg: IDK, I'm confused by reference to partions (dbs) and dbs in object dirs | 18:11 |
| acoles | clayg: I'll leave another reply | 18:11 |
| clayg | acoles: GL! | 18:11 |
| acoles | does anyone have a trick to filter bug emails so that ones you care about get your attention? I want to be able to "star" a bug and then have emails about that bug have some flag that lets me filter them | 18:13 |
| *** jcaron has joined #openstack-swift | 18:14 | |
| *** klrmn has joined #openstack-swift | 18:18 | |
| *** acoles is now known as acoles_ | 18:25 | |
| *** chsc has quit IRC | 18:25 | |
| *** cbartz has left #openstack-swift | 18:27 | |
| *** ChubYann has joined #openstack-swift | 18:28 | |
| *** catinthe_ has joined #openstack-swift | 18:33 | |
| *** mingyu has joined #openstack-swift | 18:35 | |
| *** catintheroof has quit IRC | 18:35 | |
| *** mingyu has quit IRC | 18:39 | |
| *** Jeffrey4l has joined #openstack-swift | 18:41 | |
| *** mvk has quit IRC | 18:42 | |
| *** Jeffrey4l has quit IRC | 18:46 | |
| *** pcaruana has quit IRC | 18:53 | |
| *** cebreidian has quit IRC | 19:05 | |
| *** chsc has joined #openstack-swift | 19:07 | |
| *** chsc has joined #openstack-swift | 19:07 | |
| *** Jeffrey4l has joined #openstack-swift | 19:10 | |
| *** cebreidian has joined #openstack-swift | 19:12 | |
| *** Jeffrey4l has quit IRC | 19:15 | |
| *** geaaru has quit IRC | 19:18 | |
| *** caiobrentano_ has quit IRC | 19:18 | |
| *** caiobrentano_ has joined #openstack-swift | 19:19 | |
| clayg | #easyfollowupforsomeone | 19:21 |
| *** tqtran has quit IRC | 19:26 | |
| *** ganders has quit IRC | 19:33 | |
| *** mingyu has joined #openstack-swift | 19:36 | |
| *** mingyu has quit IRC | 19:40 | |
| *** Jeffrey4l has joined #openstack-swift | 19:58 | |
| *** cebreidian has quit IRC | 19:59 | |
| *** Jeffrey4l has quit IRC | 20:04 | |
| *** tqtran has joined #openstack-swift | 20:05 | |
| *** cebreidian has joined #openstack-swift | 20:08 | |
| *** mvk has joined #openstack-swift | 20:13 | |
| clayg | ^ that's just an OH i thought was funny | 20:15 |
| clayg | is it meeting time!? | 20:15 |
| tdasilva | 45min | 20:15 |
| clayg | aacacahcphaphcap | 20:23 |
| clayg | let's have a *pre* meeting | 20:23 |
| *** catinthe_ has quit IRC | 20:27 | |
| clayg | torgomatic: so I saw lp bug #1554233 and was thinking "gah that's not merged yet!?" - then I saw I had a -1 on patch 289664 - so I was thinking about taking it over? | 20:31 |
| openstack | Launchpad bug 1554233 in OpenStack Object Storage (swift) "Servers-per-port can consume excessive OS threads" [High,In progress] https://launchpad.net/bugs/1554233 - Assigned to Samuel Merritt (torgomatic) | 20:31 |
| patchbot | https://review.openstack.org/#/c/289664/ - swift - Make eventlet.tpool's thread count configurable in... | 20:31 |
| *** lespaul has joined #openstack-swift | 20:32 | |
| *** NM has quit IRC | 20:35 | |
| clayg | I think patch 326967 is a quick easy win | 20:36 |
| patchbot | https://review.openstack.org/#/c/326967/ - swift - Rebalance with min_part_seconds_left > 0 | 20:36 |
| clayg | closes an open bug! | 20:36 |
| clayg | timburke: tdasilva: you gotta keep after patch 371150 - it's a really good one! | 20:37 |
| patchbot | https://review.openstack.org/#/c/371150/ - swift - Return 404 on a GET if tombstone is newer | 20:37 |
| *** caiobrentano_ has quit IRC | 20:37 | |
| *** Jeffrey4l has joined #openstack-swift | 20:45 | |
| *** jamielennox|away has quit IRC | 20:46 | |
| tdasilva | clayg: re patch 371150 I hit a snag trying to understand the EC code in the proxy controller | 20:46 |
| patchbot | https://review.openstack.org/#/c/371150/ - swift - Return 404 on a GET if tombstone is newer | 20:46 |
| tdasilva | so i need more time to dig in | 20:46 |
| tdasilva | there's some crazy stuff there with best buckets vs bad buckets and all that | 20:47 |
| notmyname | clayg: yeah https://bugs.launchpad.net/swift/+bug/1629410 is fix-released. I marked it as such | 20:48 |
| openstack | Launchpad bug 1629410 in OpenStack Object Storage (swift) "update pyeclib dependency to >=1.3.1" [Medium,Fix released] - Assigned to Nicolas Helgeson (nhelgeson) | 20:48 |
| notmyname | clayg: the hummingbird stuff was because the landed patch was included in a merge from master. so it triggered the scripts | 20:48 |
| clayg | and do the scripts maybe like un-mark it was fix released? | 20:52 |
| clayg | i found another old bug that seemed to be fixed and everyone was tagged there was comment in gerrit calling it reviewed (and the a bunch of other commnets about other swift feature branches) - and it was in progress? | 20:52 |
| clayg | if there's an automated tool out there going around re-opening bugs we've already closed i'ma be sorta disappointed | 20:53 |
| clayg | lp bug #1537811 (and esspecially all the bug reports it references is still a great read) | 20:53 |
| openstack | Launchpad bug 1537811 in OpenStack Object Storage (swift) "204 No Content responses have Content-Length specified" [Low,In progress] https://launchpad.net/bugs/1537811 - Assigned to Trevor McCasland (twm2016) | 20:53 |
| notmyname | no, I don't think it reopens things. I think it never closed it because it was tagged as "partial-fix" instead of "closes-bug" | 20:53 |
| clayg | esp. the varnish bug report clients be all like "umm... this has regressed" and varnish is like "no your backend is returning invalid - INVALID - respones" but then they were like "ok, fine, I guess some crappy web servers (e.g. apache with mod_php) are just *broken*" | 20:56 |
| *** jamielennox|away has joined #openstack-swift | 20:56 | |
| *** jamielennox|away is now known as jamielennox | 20:56 | |
| clayg | doing the right thing for clients always feels good - doing the right thing for documents on the internet (when it hurts clients) always makes you sound like a monster | 20:56 |
| *** acoles_ is now known as acoles | 20:58 | |
| clayg | timburke: I lost that one bug report for the reconstructor blowing up unless you move sleep | 20:58 |
| notmyname | ok, *now* it's time for the swift meeting | 20:59 |
| notmyname | in #openstack-meeting | 20:59 |
| *** m_kazuhiro has joined #openstack-swift | 20:59 | |
| mattoliverau | monring | 21:00 |
| clayg | timburke: nm, found it lp bug #1653174 | 21:02 |
| openstack | Launchpad bug 1653174 in OpenStack Object Storage (swift) "weird traceback/race in manual reconstructor: cannot switch to a different thread " [Undecided,Confirmed] https://launchpad.net/bugs/1653174 | 21:02 |
| *** caiobrentano_ has joined #openstack-swift | 21:12 | |
| *** caiobrentano_ has quit IRC | 21:13 | |
| *** JimCheung has quit IRC | 21:18 | |
| timburke | fyi on the mod_wsgi thing -- i think that might be the cause of https://bugs.launchpad.net/python-swiftclient/+bug/1586690 and https://bugs.launchpad.net/python-swiftclient/+bug/1621581 ? at any rate, one of those included some logging that saw a Server: Apache header but not Content-Length which seems... bad... | 21:19 |
| openstack | Launchpad bug 1586690 in python-swiftclient "Uploading empty(0 B) file fails" [Undecided,Incomplete] - Assigned to Uday Swami (swamius) | 21:19 |
| *** JimCheung has joined #openstack-swift | 21:19 | |
| openstack | Launchpad bug 1621581 in python-swiftclient "swiftclient returns response headers without 'Content-Length' param, thus causing upload object to fail" [Undecided,In progress] - Assigned to Arun Mani (arun-mani) | 21:19 |
| *** JimCheung has quit IRC | 21:23 | |
| *** vint_bra has quit IRC | 21:23 | |
| *** vint_bra has joined #openstack-swift | 21:24 | |
| *** vint_bra has quit IRC | 21:24 | |
| *** Jeffrey4l_ has joined #openstack-swift | 21:34 | |
| *** Jeffrey4l has quit IRC | 21:35 | |
| *** mingyu has joined #openstack-swift | 21:37 | |
| *** clu_ has joined #openstack-swift | 21:38 | |
| *** jidar has quit IRC | 21:39 | |
| *** jidar has joined #openstack-swift | 21:40 | |
| *** mingyu has quit IRC | 21:42 | |
| *** NM has joined #openstack-swift | 21:48 | |
| *** dmorita has quit IRC | 21:58 | |
| *** dmorita has joined #openstack-swift | 22:00 | |
| *** clu_ has quit IRC | 22:00 | |
| clayg | tdasilva: did you manage to *land* that refactoring? | 22:00 |
| *** clu_ has joined #openstack-swift | 22:01 | |
| *** dmorita has quit IRC | 22:01 | |
| *** dmorita has joined #openstack-swift | 22:01 | |
| clayg | tdasilva: because that'd acctually make me a lot more hopeful about this change | 22:01 |
| tdasilva | clayg: itś better now: https://github.com/openstack/swift/tree/master/test/functional | 22:01 |
| mattoliverau | tdasilva: I was only joking about volenteering you :) | 22:01 |
| tdasilva | mattoliverau: hehe, psachin's got it | 22:01 |
| mattoliverau | lol | 22:01 |
| jrichli | clayg: I am sometimes a little nervous on test refactoring because we dont test tests. so you have to know the same things are still getting tested for sure | 22:01 |
| clayg | tdasilva: can you dig up the series of reviews where you did that and suggest that in the review? | 22:01 |
| tdasilva | clayg: sure | 22:02 |
| acoles | when tdasilva broke up the func tests it really helped review that almost nothing changed other then lines of code moving between files | 22:02 |
| jrichli | yes, that was nice | 22:02 |
| clayg | acoles: i don't really have a good tool workflow for that - i've seen you do it in your ide before | 22:02 |
| clayg | jrichli: you have a trick for that too!? | 22:02 |
| acoles | clayg: yeah pycharm lets me copy N lines from one branch to clipboard and then compare that with another selection | 22:03 |
| jrichli | clayg: i wish i had tricks | 22:03 |
| timburke | i seem to remember going through two test files more-or-less line-by-line and checking that we still had comparable coverage... for versioned_writes, maybe? or copy? pulling one of those out as middleware, anyway | 22:03 |
| acoles | oh no did I just volunteer to review this monster !?!? ;) | 22:03 |
| clayg | acoles: but you don't know how to do that outside of your tool suite? | 22:03 |
| tdasilva | acoles: hehe | 22:03 |
| kota_ | timburk, clayg: could you continue the review for my global ec patch? Maybe i would be back to upstream in the next week | 22:03 |
| clayg | kota_: yes yes yes! i very very much want that in ocata acctually! | 22:04 |
| acoles | clayg: no, but I have never looked for another tool | 22:04 |
| mattoliverau | k, me is going to go Australia day. Hooroo <-- aussie slang for good bye. | 22:04 |
| clayg | mattoliverau: ttyl | 22:04 |
| jrichli | Hooroo! | 22:04 |
| timburke | kota_: yup, was coming up with a diff and a few comments before the meeting | 22:04 |
| acoles | mattoliverau: ttfn | 22:04 |
| kota_ | Thanks a lot! | 22:04 |
| acoles | clayg: tdasilva anyone, the tip being - don't try to fix or cleanup anything unless absolutely necessary, just move the lines of code | 22:05 |
| tdasilva | clayg: what about using a tool like meld, copy and paste?? it's manual work, but should work... | 22:05 |
| * kota_ should get well soon. Still on bed though | 22:05 | |
| timburke | kota_: i'm still a little nervous that we can get some pathologically bad distributions (both copied of frags 0, 1, and 2 in region 1, both copies of frags 3, 4, and 5 in region 2), but maybe it's better to land it and iterate from there? | 22:06 |
| acoles | kota_: stay there and get well! | 22:06 |
| clayg | tdasilva: I've never used meld - i would really like instructions (on the review) that tell me how to see "here's the lines that moved, here's the the lines that then changed" | 22:07 |
| tdasilva | notmyname: what's the next step with this: http://lists.openstack.org/pipermail/openstack-dev/2017-January/110467.html ? | 22:07 |
| *** jcaron has left #openstack-swift | 22:07 | |
| clayg | timburke: IME it just worked!? but then kota_ said I just got much lucky :'( | 22:07 |
| tdasilva | clayg: agreed, i will post a comment for psachin to try and help him out with that | 22:07 |
| *** JimCheung has joined #openstack-swift | 22:07 | |
| clayg | timburke: I think that composite rings help with that somehow eventually? | 22:08 |
| acoles | tdasilva: I also like your suggestion to move out one controller at a time. hard to say how big a deal it is without looking at the patch though | 22:08 |
| clayg | timburke: i'm not sure the value of landing it to iterate - in fact - w/o the other stuff maybe we should documnt it as experimental dragons or something? | 22:08 |
| timburke | clayg: yeah, and i see how it *should* Just Work most of the time, but there's nothing to *prevent* it, which means with lots of device additions/removals, it could certainly happen | 22:08 |
| tdasilva | acoles: yeah, i'm trying to bring it up now to take a closer look | 22:09 |
| *** m_kazuhiro has quit IRC | 22:09 | |
| clayg | timburke: I don't even know why it "just works" *some* of the time - i went looking because I was so sure it would be wrong - i was very surprised | 22:09 |
| acoles | good night | 22:09 |
| clayg | acoles: thanks! | 22:09 |
| clayg | tdasilva: good question on TC golang resolution! | 22:10 |
| *** acoles is now known as acoles_ | 22:10 | |
| *** NM has quit IRC | 22:19 | |
| notmyname | tdasilva: honestly, I don't know yet | 22:20 |
| *** sams-gleb has quit IRC | 22:22 | |
| *** sams-gleb has joined #openstack-swift | 22:23 | |
| portante | key notmyname, did you see the pinterest talk at linux.conf.au on debugging website latencies? https://www.youtube.com/watch?v=bectZn_yNwg | 22:26 |
| portante | seemed pretty good | 22:26 |
| openstackgerrit | Clay Gerrard proposed openstack/swift: Do not revert fragments to handoffs https://review.openstack.org/425441 | 22:27 |
| clayg | portante: !!! | 22:27 |
| portante | hi clayg | 22:27 |
| *** sams-gleb has quit IRC | 22:27 | |
| *** sams-gleb has joined #openstack-swift | 22:28 | |
| *** sams-gleb has quit IRC | 22:29 | |
| clayg | why do i know joe - i know he's been around openstack? infra? hp? redhat? rax? swift? | 22:29 |
| clayg | OMG if the punchline is golang I'm going to be so mad :D | 22:30 |
| tdasilva | lol | 22:30 |
| tdasilva | clayg: added the comment there, feel free to shoot it down if you don't like what i said | 22:31 |
| timburke | i was watching that just yesterday, seemed pretty cool. i'm curious what the statistical profiling looks like on a proxy server | 22:31 |
| tdasilva | notmyname: no worries, thanks | 22:31 |
| timburke | esp. replicated vs. ec, encryption vs. not... | 22:32 |
| tdasilva | gotta run, have a good night everyone | 22:32 |
| clayg | yeah!!! MOAR processes! | 22:36 |
| clayg | there was a pretty visualization - it wasn't helpful - pstats FTW | 22:41 |
| notmyname | portante: hey. yeah, it turns out I just watched that this morning. I didn't catch it in person | 22:50 |
| portante | cool | 22:50 |
| notmyname | portante: but mattoliverau saw it in person (and said he wanted to make some flame graphs for swift) | 22:51 |
| portante | yes, that would be cool | 22:51 |
| clayg | timburke: you want one of those flame graph things!? | 22:51 |
| portante | at 1.5 speed the talk seemed to be pretty to the point | 22:51 |
| timburke | mattoliverau: yes! do it! but finish sharding first ;-) | 22:51 |
| timburke | clayg: largely for the diffs | 22:52 |
| clayg | awww man and it's picking on deepcopy now | 22:53 |
| timburke | yup | 22:53 |
| clayg | protobufffffffffffff | 22:55 |
| clayg | awwww the punchline is "C" :'( | 22:55 |
| clayg | that's not as funny | 22:56 |
| clayg | this is really slowing down my rebasing :) | 22:56 |
| clayg | portante: yeah that was great - thanks | 22:58 |
| *** jordanP has joined #openstack-swift | 22:59 | |
| zaitcev | as in extern "C" { ... } ? | 22:59 |
| *** jordanP has quit IRC | 22:59 | |
| *** catintheroof has joined #openstack-swift | 23:00 | |
| *** NM has joined #openstack-swift | 23:03 | |
| *** cebreidian has quit IRC | 23:09 | |
| openstackgerrit | John Dickinson proposed openstack/python-swiftclient: 3.3.0 authors/changelog update https://review.openstack.org/425454 | 23:13 |
| notmyname | for your consideration for this week's client release ^ | 23:13 |
| *** cebreidian has joined #openstack-swift | 23:18 | |
| *** dmorita has quit IRC | 23:23 | |
| timburke | looks like i get to dig into some C code, too? yay! "*** Error in `python': free(): invalid size: 0x0000000001bd1790 *** Aborted" "Segmentation fault" | 23:24 |
| *** dmorita has joined #openstack-swift | 23:25 | |
| *** sams-gleb has joined #openstack-swift | 23:29 | |
| *** sams-gleb has quit IRC | 23:34 | |
| *** NM has quit IRC | 23:35 | |
| *** mingyu has joined #openstack-swift | 23:39 | |
| *** caiobrentano_ has joined #openstack-swift | 23:39 | |
| *** caiobrentano_ has quit IRC | 23:39 | |
| *** mingyu has quit IRC | 23:43 | |
| *** caiobrentano_ has joined #openstack-swift | 23:52 | |
| *** chsc has quit IRC | 23:54 | |
| *** kei_yama has joined #openstack-swift | 23:57 | |
| *** catintheroof has quit IRC | 23:58 | |
| *** catintheroof has joined #openstack-swift | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!