| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements https://review.openstack.org/88736 | 00:07 |
|---|---|---|
| *** jamielennox is now known as jamielennox|away | 00:24 | |
| *** jlwhite has quit IRC | 00:28 | |
| *** jlwhite has joined #openstack-swift | 00:28 | |
| *** catintheroof has joined #openstack-swift | 00:36 | |
| *** catintheroof has quit IRC | 00:37 | |
| *** jamielennox|away is now known as jamielennox | 00:50 | |
| *** vint_bra has joined #openstack-swift | 00:54 | |
| *** dmorita has joined #openstack-swift | 01:03 | |
| *** dmorita_ has joined #openstack-swift | 01:06 | |
| *** dmorita has quit IRC | 01:09 | |
| *** nikivi has quit IRC | 01:17 | |
| *** sileht has quit IRC | 01:19 | |
| *** sams-gleb has joined #openstack-swift | 01:19 | |
| *** nikivi has joined #openstack-swift | 01:19 | |
| *** dmorita_ has quit IRC | 01:19 | |
| *** amoralej|off has quit IRC | 01:19 | |
| *** amoralej has joined #openstack-swift | 01:22 | |
| *** sileht has joined #openstack-swift | 01:22 | |
| *** sams-gleb has quit IRC | 01:23 | |
| *** dmorita has joined #openstack-swift | 01:24 | |
| *** dmorita has quit IRC | 01:28 | |
| *** dmorita has joined #openstack-swift | 01:29 | |
| *** dmorita has quit IRC | 01:32 | |
| *** vint_bra has quit IRC | 01:33 | |
| *** vint_bra has joined #openstack-swift | 01:38 | |
| *** dmorita has joined #openstack-swift | 01:50 | |
| *** dmorita has quit IRC | 01:53 | |
| *** dmorita has joined #openstack-swift | 01:56 | |
| *** vint_bra has quit IRC | 01:56 | |
| *** dmorita has quit IRC | 01:58 | |
| *** dmorita has joined #openstack-swift | 01:59 | |
| *** dmorita has quit IRC | 02:00 | |
| *** dmorita has joined #openstack-swift | 02:01 | |
| *** dmorita has quit IRC | 02:02 | |
| *** dmorita has joined #openstack-swift | 02:03 | |
| *** dmorita_ has joined #openstack-swift | 02:16 | |
| *** dmorita has quit IRC | 02:19 | |
| *** ChatSharp has joined #openstack-swift | 02:20 | |
| *** ChatSharp has left #openstack-swift | 02:23 | |
| *** dmorita_ has quit IRC | 02:30 | |
| *** dmorita has joined #openstack-swift | 02:31 | |
| *** dmorita has quit IRC | 02:34 | |
| *** klrmn has quit IRC | 02:35 | |
| *** dmorita has joined #openstack-swift | 02:43 | |
| *** dmorita has quit IRC | 02:47 | |
| *** dmorita has joined #openstack-swift | 03:07 | |
| *** dmorita has quit IRC | 03:08 | |
| *** sams-gleb has joined #openstack-swift | 03:21 | |
| *** sams-gleb has quit IRC | 03:25 | |
| *** SkyRocknRoll has joined #openstack-swift | 03:48 | |
| *** dmorita has joined #openstack-swift | 03:55 | |
| *** dmorita has quit IRC | 04:01 | |
| *** dmorita has joined #openstack-swift | 04:03 | |
| *** dmorita has quit IRC | 04:09 | |
| *** psachin has joined #openstack-swift | 04:10 | |
| *** SkyRocknRoll has quit IRC | 04:11 | |
| *** openstackgerrit has quit IRC | 04:32 | |
| *** hoonetorg has quit IRC | 04:34 | |
| *** hoonetorg has joined #openstack-swift | 04:47 | |
| *** dja has quit IRC | 05:10 | |
| *** tonanhngo has joined #openstack-swift | 05:14 | |
| *** tonanhngo has quit IRC | 05:16 | |
| *** mtreinish has quit IRC | 05:20 | |
| *** sams-gleb has joined #openstack-swift | 05:23 | |
| *** sams-gleb has quit IRC | 05:27 | |
| *** mtreinish has joined #openstack-swift | 05:31 | |
| *** SkyRocknRoll has joined #openstack-swift | 05:54 | |
| *** bkopilov_ has quit IRC | 05:58 | |
| *** bkopilov_ has joined #openstack-swift | 05:59 | |
| *** Jeffrey4l__ has joined #openstack-swift | 06:01 | |
| *** bkopilov_ has quit IRC | 06:04 | |
| *** Jeffrey4l_ has quit IRC | 06:04 | |
| *** bkopilov_ has joined #openstack-swift | 06:04 | |
| *** chsc has joined #openstack-swift | 06:09 | |
| *** chsc has joined #openstack-swift | 06:09 | |
| *** chsc has quit IRC | 06:15 | |
| *** psachin has quit IRC | 06:33 | |
| *** geaaru has quit IRC | 06:40 | |
| *** tnovacik has joined #openstack-swift | 06:50 | |
| *** psachin has joined #openstack-swift | 06:50 | |
| *** psachin has quit IRC | 06:54 | |
| *** psachin has joined #openstack-swift | 06:54 | |
| *** seongsoocho has joined #openstack-swift | 07:09 | |
| *** rcernin has joined #openstack-swift | 07:15 | |
| seongsoocho | Hi All! . I want to debug swift line by line and see a variables. I already build swift on vm. Is there any idea how to debugging on remote? | 07:18 |
| *** sams-gleb has joined #openstack-swift | 07:25 | |
| *** tesseract has joined #openstack-swift | 07:28 | |
| *** sams-gleb has quit IRC | 07:29 | |
| *** tesseract is now known as tesseract-RH | 07:29 | |
| *** newmember has quit IRC | 07:30 | |
| *** newmember has joined #openstack-swift | 07:31 | |
| *** tnovacik has quit IRC | 07:32 | |
| *** sams-gleb has joined #openstack-swift | 07:44 | |
| *** pcaruana has joined #openstack-swift | 07:49 | |
| *** kei_yama has quit IRC | 08:24 | |
| *** rledisez has joined #openstack-swift | 08:28 | |
| *** furlongm has quit IRC | 08:30 | |
| *** furlongm has joined #openstack-swift | 08:31 | |
| *** geaaru has joined #openstack-swift | 08:36 | |
| *** kei_yama has joined #openstack-swift | 08:56 | |
| *** cbartz has joined #openstack-swift | 09:01 | |
| *** tqtran has joined #openstack-swift | 09:05 | |
| *** joeljwright has joined #openstack-swift | 09:08 | |
| *** ChanServ sets mode: +v joeljwright | 09:08 | |
| mattoliverau | seongsoocho: if you want a line by line debugger that can get hard, as swift has so many moving peices. On an SAIO on my desktop, I've got it working with pycharms. The paid version has remote debugging, but don't use. If you do a bit of scripting to start things I've gotten pudb working working through an SSH connection to debug things like the replicators debugged. Pycharms was easier but you have to start each | 09:09 |
| mattoliverau | service though the ide, or at least the services you want to debug at the time. | 09:09 |
| *** tqtran has quit IRC | 09:09 | |
| *** oshritf_ has joined #openstack-swift | 09:12 | |
| *** Shashikant86 has joined #openstack-swift | 09:22 | |
| *** Shashikant86 has quit IRC | 09:34 | |
| *** oshritf_ has quit IRC | 09:37 | |
| *** oshritf_ has joined #openstack-swift | 09:43 | |
| *** Shashikant86 has joined #openstack-swift | 09:44 | |
| *** xlucas has quit IRC | 09:47 | |
| *** jamielennox is now known as jamielennox|away | 09:48 | |
| *** Shashikant86 has quit IRC | 09:50 | |
| *** oshritf_ has quit IRC | 09:56 | |
| *** xlucas has joined #openstack-swift | 10:01 | |
| *** Shashikant86 has joined #openstack-swift | 10:02 | |
| *** tonanhngo has joined #openstack-swift | 10:03 | |
| *** tonanhngo has quit IRC | 10:05 | |
| *** Shashikant86 has quit IRC | 10:12 | |
| *** tqtran has joined #openstack-swift | 10:15 | |
| *** dims_ has quit IRC | 10:16 | |
| *** tqtran has quit IRC | 10:20 | |
| *** dims has joined #openstack-swift | 10:23 | |
| *** jamielennox|away is now known as jamielennox | 10:24 | |
| *** sams-gleb has quit IRC | 10:35 | |
| *** mvk has quit IRC | 10:40 | |
| seongsoocho | mattoliverau, Thanks. Use PyCharm in same host which is installed SAIO seems to be an approciate solution to debug line by line. Then.. How does contributors works? How they develop swift? | 10:50 |
| *** jamiec has quit IRC | 11:04 | |
| *** oshritf has joined #openstack-swift | 11:04 | |
| *** psachin has quit IRC | 11:08 | |
| *** mvk has joined #openstack-swift | 11:09 | |
| *** jamiec has joined #openstack-swift | 11:12 | |
| *** psachin has joined #openstack-swift | 11:25 | |
| *** oshritf has quit IRC | 11:26 | |
| *** sams-gleb has joined #openstack-swift | 11:27 | |
| *** oshritf has joined #openstack-swift | 11:28 | |
| *** psachin has quit IRC | 11:39 | |
| *** kei_yama has quit IRC | 11:42 | |
| *** ChatSharp has joined #openstack-swift | 11:52 | |
| *** ChatSharp has left #openstack-swift | 11:55 | |
| *** SkyRocknRoll has quit IRC | 11:56 | |
| *** oshritf has quit IRC | 12:06 | |
| *** oshritf has joined #openstack-swift | 12:11 | |
| *** psachin has joined #openstack-swift | 12:12 | |
| *** SkyRocknRoll has joined #openstack-swift | 12:13 | |
| *** oshritf has quit IRC | 12:14 | |
| *** tqtran has joined #openstack-swift | 12:17 | |
| *** oshritf has joined #openstack-swift | 12:19 | |
| *** oshritf has quit IRC | 12:22 | |
| *** tqtran has quit IRC | 12:22 | |
| *** dims has quit IRC | 12:23 | |
| *** dmorita has joined #openstack-swift | 12:26 | |
| *** oshritf has joined #openstack-swift | 12:26 | |
| *** oshritf has quit IRC | 12:38 | |
| *** oshritf has joined #openstack-swift | 12:38 | |
| *** catintheroof has joined #openstack-swift | 12:41 | |
| *** CowboyPride has joined #openstack-swift | 12:57 | |
| CowboyPride | Morning everyone, could someone look over the upgrade plan for a swift cluster I've been handed and see if there is anything I'm missing. http://pastebin.com/f5yJacuv | 12:58 |
| CowboyPride | Node 5 was in a bad way but I think I've got that fixed now, just have get everything up to latest stable release. | 13:00 |
| *** oshritf has quit IRC | 13:01 | |
| *** dmorita_ has joined #openstack-swift | 13:11 | |
| *** dmorita has quit IRC | 13:12 | |
| *** dmorita has joined #openstack-swift | 13:12 | |
| *** dmorita_ has quit IRC | 13:15 | |
| *** oshritf has joined #openstack-swift | 13:19 | |
| *** silor has joined #openstack-swift | 13:20 | |
| *** dims has joined #openstack-swift | 13:20 | |
| *** klamath has joined #openstack-swift | 13:24 | |
| *** klamath has quit IRC | 13:24 | |
| *** klamath has joined #openstack-swift | 13:25 | |
| *** silor1 has joined #openstack-swift | 13:26 | |
| *** oshritf has quit IRC | 13:27 | |
| *** silor has quit IRC | 13:27 | |
| *** silor1 is now known as silor | 13:27 | |
| *** vinsh has joined #openstack-swift | 13:30 | |
| *** oshritf has joined #openstack-swift | 13:32 | |
| *** dmorita has quit IRC | 13:33 | |
| *** pumaranikar has quit IRC | 13:34 | |
| *** pumaranikar has joined #openstack-swift | 13:38 | |
| *** dmorita has joined #openstack-swift | 13:41 | |
| *** dims_ has joined #openstack-swift | 13:43 | |
| *** dims has quit IRC | 13:45 | |
| *** dmorita has quit IRC | 13:56 | |
| *** mvk has quit IRC | 14:03 | |
| *** oshritf has quit IRC | 14:03 | |
| *** tonanhngo has joined #openstack-swift | 14:05 | |
| *** tonanhngo has quit IRC | 14:06 | |
| *** oshritf has joined #openstack-swift | 14:06 | |
| *** amoralej is now known as amoralej|lunch | 14:11 | |
| *** dmorita has joined #openstack-swift | 14:12 | |
| *** dmorita has quit IRC | 14:17 | |
| *** mvk has joined #openstack-swift | 14:18 | |
| *** cdelatte has joined #openstack-swift | 14:21 | |
| *** amoralej|lunch is now known as amoralej | 14:37 | |
| *** jamielennox is now known as jamielennox|away | 14:39 | |
| *** vinsh_ has joined #openstack-swift | 14:45 | |
| *** vinsh has quit IRC | 14:45 | |
| *** nikivi has quit IRC | 14:45 | |
| *** nikivi has joined #openstack-swift | 14:48 | |
| *** pcaruana has quit IRC | 14:50 | |
| *** oshritf has quit IRC | 14:53 | |
| *** sams-gleb has quit IRC | 14:55 | |
| *** pcaruana has joined #openstack-swift | 14:55 | |
| *** sams-gleb has joined #openstack-swift | 14:56 | |
| *** sams-gleb has quit IRC | 15:00 | |
| *** sams-gleb has joined #openstack-swift | 15:12 | |
| *** tqtran has joined #openstack-swift | 15:19 | |
| *** oshritf has joined #openstack-swift | 15:23 | |
| *** tqtran has quit IRC | 15:24 | |
| *** SkyRocknRoll has quit IRC | 15:30 | |
| *** bikmak has joined #openstack-swift | 15:31 | |
| *** _JZ_ has joined #openstack-swift | 15:44 | |
| *** Jeffrey4l__ is now known as Jeffrey4l | 15:44 | |
| *** psachin has quit IRC | 15:44 | |
| *** vinsh_ has quit IRC | 15:47 | |
| *** vinsh has joined #openstack-swift | 15:48 | |
| -openstackstatus- NOTICE: We are currently investigating an issue with our AFS mirrors which is causing some projects jobs to fail. We are working to correct the issue. | 15:50 | |
| *** vinsh has quit IRC | 15:52 | |
| *** mvk has quit IRC | 15:53 | |
| *** klrmn has joined #openstack-swift | 15:55 | |
| *** vinsh has joined #openstack-swift | 15:56 | |
| notmyname | good morning | 15:58 |
| notmyname | I'm going on a kid's field trip today with his class, so I'll be offline most of the day. | 15:58 |
| zaitcev | A day of productivity for the rest of us. | 15:59 |
| notmyname | CowboyPride: I'm not going to flippantly say "looks good" or anything like that. I'm looking at your pastebin now | 15:59 |
| notmyname | CowboyPride: you have a note about looking for the changelog. it's here https://github.com/openstack/swift/blob/master/CHANGELOG | 16:00 |
| notmyname | this week there should be a 2.13.0 release for Ocata | 16:00 |
| *** oshritf has quit IRC | 16:01 | |
| zaitcev | Somehow this reminds me about "take your daughter to work day" skit of Robot Chicken https://www.youtube.com/watch?v=Cizlx6ODhuE | 16:01 |
| *** oshritf has joined #openstack-swift | 16:02 | |
| notmyname | lol | 16:04 |
| jrichli | zaitcev: thanks, i needed that :-) | 16:04 |
| notmyname | there was one day I had to bring a kid to the office. he hated it | 16:04 |
| jrichli | *had* to? :-) | 16:04 |
| zaitcev | Yeah. It's not like you can work with them around either. | 16:04 |
| notmyname | yeah, actually I did have to. it's a more involved story that I want to get into on IRC though ;-) | 16:06 |
| jrichli | It was interesting to me to see some people without kids leave on bring your kid to work day - because they couldnt get work done. | 16:06 |
| notmyname | so... who's "OpenStack Clean" and how do I buy this bot a beer? https://review.openstack.org/#/c/414881/ | 16:06 |
| patchbot | patch 414881 - python-swiftclient - Removes unnecessary utf-8 coding | 16:06 |
| notmyname | jrichli: +100 | 16:06 |
| jrichli | omg - that comment is great - re: os clean | 16:09 |
| mahatic | notmyname: lol jrichli: it's an amazing comment! :D re: OS clean | 16:12 |
| notmyname | looks like it's on all the patches associated with that same topic | 16:12 |
| notmyname | so, a lot of them | 16:12 |
| jrichli | mahatic: o/ will you be at the PTG? | 16:13 |
| notmyname | -infra says it might not be a bot. and I think it's a surprise for everyone | 16:13 |
| mahatic | jrichli: nope :( | 16:13 |
| jrichli | mahatic: boo! :( | 16:13 |
| jrichli | notmyname: interesting! | 16:14 |
| notmyname | logistics for the week... | 16:15 |
| mahatic | not a bot and yet so comments on so many patches? | 16:15 |
| notmyname | I'll be finishing up the authors/changelog patch Real Soon Now | 16:15 |
| notmyname | so whatever's in the release needs to be landed ASAP | 16:16 |
| notmyname | (I'll likely spend some more time on that late today or tomorrow) | 16:16 |
| mahatic | Idk why I remembered the release date as 16th Feb | 16:16 |
| notmyname | and the PTG is next week, so in the second half of the week I'll be organizing the etherpad topics into a rough schedule | 16:16 |
| *** rcernin has quit IRC | 16:19 | |
| *** dims_ has quit IRC | 16:19 | |
| *** pcaruana has quit IRC | 16:20 | |
| *** dims has joined #openstack-swift | 16:24 | |
| *** tqtran has joined #openstack-swift | 16:25 | |
| notmyname | ok, maybe not a bot. there's quite a few different reviews https://review.openstack.org/#/q/reviewer:%22OpenStack+Clean+%253Copenstack.clean%2540gmail.com%253E%22 | 16:28 |
| notmyname | but it's likely that there was some scripting involved | 16:28 |
| notmyname | CowboyPride: FWIW, https://www.swiftstack.com/blog/2013/12/20/upgrade-openstack-swift-no-downtime/ has some good tips for a general upgrade process, too | 16:30 |
| *** pcaruana has joined #openstack-swift | 16:34 | |
| CowboyPride | Thanks! | 16:34 |
| *** oshritf has quit IRC | 16:43 | |
| notmyname | CowboyPride: in general, there's a few things that stick out to me on your pastebin | 16:50 |
| notmyname | CowboyPride: first is that you're doing a lot of stuff in one upgrade. OS upgrade and swift upgrade. that seems like a lot of moving pieces. however, take this (and other stuff) as from someone who's a dev and not a sysadmin | 16:51 |
| *** dims has quit IRC | 16:51 | |
| notmyname | CowboyPride: I wouldn't expect any of those upgrades to cause issues, but doing them all at once seems to have a higher likelihood of confusing troubleshooting if something goes wrong | 16:51 |
| notmyname | CowboyPride: second is that you should definitely do a canary-node style deployment. upgrade one server and put it in the cluster and see if anything breaks | 16:52 |
| notmyname | CowboyPride: swift is really really good at working around a down server, so even if worst case you end up having to wipe and restore that one server, the cluster should handle it just fine | 16:53 |
| *** dmorita has joined #openstack-swift | 16:53 | |
| *** dims has joined #openstack-swift | 16:55 | |
| *** tesseract-RH has quit IRC | 16:55 | |
| notmyname | CowboyPride: however, that being said, if there are any chnages that have changed the way data is written to disk, you'll have to be more careful. eg in your upgrade path you've got the crypto functionality. so you can't enable crypto on the new node without the new code being everywhere first (otherwise all the old code can't read it) | 16:55 |
| notmyname | CowboyPride: that's generally the way we write stuff in swift. clean upgrades with new features enabled by option after the new code is everywhere | 16:56 |
| notmyname | "generally" | 16:56 |
| *** pcaruana has quit IRC | 16:58 | |
| *** pcaruana has joined #openstack-swift | 16:59 | |
| CowboyPride | notmyname: Won't be doing upgrades all at once, will be doing one node at at time and making sure the node is funcational again before continueing on to next | 17:01 |
| *** klrmn has quit IRC | 17:01 | |
| notmyname | CowboyPride: great. as with all ops things, monitoring is key :-) | 17:01 |
| *** cbartz has quit IRC | 17:02 | |
| CowboyPride | Node5 had a undate that was done to it by previous person and it was an incomplete update. And I finally got it back to a good state. It went from icehouse to liberty. | 17:02 |
| CowboyPride | *update | 17:02 |
| notmyname | CowboyPride: keep an eye on response codes (5xx is bad), async pendings (huge spikes can indicate a problem), handoff vs primary partitions, req/sec, any tracebacks in the logs, etc (in addition to normal CPU, memory, drive stuff) | 17:03 |
| CowboyPride | but they forgot to start repliction and forgot to do a few other things as well, mostly because there was NO action plan. | 17:03 |
| notmyname | ah, yikes | 17:03 |
| *** dmorita has quit IRC | 17:03 | |
| *** dims_ has joined #openstack-swift | 17:06 | |
| *** dims has quit IRC | 17:07 | |
| notmyname | CowboyPride: please don't hesitate to drop in here and ask questions as they come up. we're *always* looking for ops feedback (and many people in here run swift clusters and can probably answer general questions) | 17:07 |
| notmyname | CowboyPride: what time zone are you in? | 17:07 |
| tdasilva | jrichli: have you had a chance to run probetest on master since those fixes merged? | 17:09 |
| notmyname | I'm out. I'll check in later this afternoon | 17:14 |
| *** rledisez has quit IRC | 17:15 | |
| *** tnovacik has joined #openstack-swift | 17:19 | |
| -openstackstatus- NOTICE: AFS replication issue has been addressed. Mirrors are currently re-syncing and coming back online. | 17:20 | |
| *** JimCheung has joined #openstack-swift | 17:27 | |
| *** openstackgerrit has joined #openstack-swift | 17:35 | |
| openstackgerrit | Clay Gerrard proposed openstack/swift master: Deprecate broken handoffs_first in favor of handoffs_only https://review.openstack.org/425493 | 17:35 |
| clayg | I still thinkg "Don't use this. You're so screwed. I'm sorry." would have been better | 17:37 |
| *** mvk has joined #openstack-swift | 17:38 | |
| *** chsc has joined #openstack-swift | 17:39 | |
| *** pcaruana has quit IRC | 17:44 | |
| *** klrmn has joined #openstack-swift | 18:00 | |
| *** david-lyle_ has joined #openstack-swift | 18:08 | |
| *** joeljwright has quit IRC | 18:09 | |
| *** david-lyle_ is now known as david-lyle | 18:12 | |
| jrichli | tdasilva: i dont think i have done that after the merges. i had moved on to digging into handoffs only patch. but ill do that now. thanks! | 18:16 |
| timburke | good morning | 18:24 |
| *** newmember has quit IRC | 18:25 | |
| *** newmember has joined #openstack-swift | 18:26 | |
| *** gatuus has joined #openstack-swift | 18:31 | |
| *** geaaru has quit IRC | 18:37 | |
| *** SkyRocknRoll has joined #openstack-swift | 18:39 | |
| jrichli | clayg: do you mind if i push a rebase up on patch 425493 since it changes the logic a little in _revert and it's good to see how it will all fit once merged | 18:48 |
| patchbot | https://review.openstack.org/#/c/425493/ - swift - Deprecate broken handoffs_first in favor of handof... | 18:48 |
| tdasilva | jrichli: ok, just checking, i ran into errors, but I think it was due to a faulty dev enviroment. I just started with a new VM and everything passed correctly | 18:50 |
| clayg | jrichli: a rebase is fine for m | 18:50 |
| clayg | *me | 18:50 |
| clayg | it only effects reviewers ability to see changes from base to base AIUI | 18:50 |
| clayg | jrichli: anyway - what changed? did something else get fixed in the reconstructor? | 18:51 |
| jrichli | looking at the review in gerrit, it still shows the adding of the handoff nodes to what is looped over in revert | 18:51 |
| jrichli | but yes, i guess then the changes would not be isolated, so maybe i wont do that | 18:52 |
| clayg | oic - i had made them orthogonal - but now that the other is merged it makes sense to me to squash things down | 18:52 |
| clayg | er... rebase anyway | 18:53 |
| clayg | you can do it locally if you just want to test it? | 18:53 |
| clayg | it'll come in a merge commit regardless | 18:53 |
| clayg | `git checkout master; git checkout -b review-test; git merge review/clay/thing` - that's what will acctually be on master after +A | 18:53 |
| jrichli | i did it locally, yes. | 18:53 |
| *** dmorita has joined #openstack-swift | 19:04 | |
| *** joeljwright has joined #openstack-swift | 19:04 | |
| *** ChanServ sets mode: +v joeljwright | 19:04 | |
| jrichli | clayg: I am currently focused on the incrementing of handoffs_remaining in the revert method. specifically, how the code determines when it should be incremented. | 19:06 |
| jrichli | I am having a hard time understanding why 'syncd_with < len(job['sync_to'])' necessarily implies that there is another handoff remaining. | 19:07 |
| jrichli | the handoff nodes are no longer in the set of nodes looped over. this must be bigger picture thing I dont know about | 19:08 |
| *** dmorita has quit IRC | 19:08 | |
| *** newmember has quit IRC | 19:08 | |
| *** tonanhngo has joined #openstack-swift | 19:09 | |
| *** tonanhngo_ has joined #openstack-swift | 19:12 | |
| *** tonanhngo has quit IRC | 19:13 | |
| jrichli | i see how that particular condition will be true if there are any failures with SYNC | 19:16 |
| *** tonanhngo_ has quit IRC | 19:16 | |
| *** tonanhngo has joined #openstack-swift | 19:17 | |
| clayg | i think the question is just "was the job successful" | 19:19 |
| clayg | the sync_to list I guess can be different lengths - so you need to success to at least that many nodes | 19:19 |
| clayg | iirc? | 19:19 |
| *** oshritf has joined #openstack-swift | 19:20 | |
| *** amoralej is now known as amoralej|off | 19:37 | |
| jrichli | clayg: ok, so I think I see. whether we are in handoffs mode or not, all jobs in job['sync_to'] in _revert are going to be REVERT jobs, so handoffs. I think the "node in job['sync_to']" had me forgetting each loop was for a revert job. | 19:38 |
| *** oshritf has quit IRC | 19:41 | |
| jrichli | i guess that is still have to wonder about how this looping goes if there are more than one REVERT jobs needed on the same node | 19:41 |
| *** klamath has quit IRC | 19:43 | |
| *** oshritf has joined #openstack-swift | 19:43 | |
| jrichli | is there currently a test somewhere that involves the case where more than one REVERT job is needed on the same node? walking through that whole flow with handoffs only will help me. | 19:46 |
| clayg | idk, timburke was trying to explain to me that at somepoint when we dropped handoff reversion the code was correct but convoluted | 19:47 |
| jrichli | clayg: hehe. yes, maybe that is what is making me doubt things. it seems like there is a one-to-one assumption here that is probably fine, but I am not familiar enough with this yet to know why it is. | 19:50 |
| jrichli | i am focusing on the tests. looking for a way to setup that case i mentioned (and maybe we already have that situation somewhere) | 19:51 |
| *** dmorita has joined #openstack-swift | 19:53 | |
| *** dmorita has quit IRC | 19:57 | |
| jrichli | tdasilva, clayg: it seems that I am still getting the intermittent failures ... from different replication probe tests. tdasilva - is that the type of test that had failed for you before the new | 19:58 |
| jrichli | dev env? | 19:58 |
| *** gk-1wm-su has joined #openstack-swift | 19:59 | |
| *** gk-1wm-su has left #openstack-swift | 19:59 | |
| *** SkyRocknRoll has quit IRC | 20:01 | |
| tdasilva | tdasilva: not sure, since I spawned a new VM the probe tests have run fine for me (i ran it 2x. one with rsync and another with ssync) | 20:04 |
| tdasilva | jrichli: ^^ | 20:04 |
| tdasilva | heh...i typically don't talk to myself | 20:05 |
| jrichli | lol. ok, i will keep testing to see. | 20:06 |
| tdasilva | jrichli: anyway....the probetests errors I saw in the old environment went beyong just repl tests, there were like 23 failures... | 20:06 |
| *** gk-1wm-su has joined #openstack-swift | 20:07 | |
| gk-1wm-su | .çôêå | 20:07 |
| jrichli | tdasilva: oh, wow. ok - that isn't anything like i am seeing. i will get one error only at a time. but its always with some kind of replication or handoff test - jsut different ones. | 20:09 |
| tdasilva | jrichli: yeah, that's why I mean, once i saw that I knew my dev. environment was just busted | 20:10 |
| *** gk-1wm-su has quit IRC | 20:10 | |
| jrichli | clayg: looking more at SSYNC, i think i am more clear on the revert now | 20:11 |
| *** amoralej|off is now known as amoralej | 20:11 | |
| jrichli | and as for the probe tests, i think i had just forgotten to rebase when i should have on a couple runs | 20:38 |
| *** klamath has joined #openstack-swift | 20:55 | |
| *** gatuus has quit IRC | 20:59 | |
| *** jamielennox|away is now known as jamielennox | 21:02 | |
| *** amoralej is now known as amoralej|off | 21:23 | |
| mattoliverau | morning | 21:25 |
| *** dmorita has joined #openstack-swift | 21:31 | |
| *** Jeffrey4l has quit IRC | 21:36 | |
| *** dmorita has quit IRC | 21:36 | |
| *** Jeffrey4l_ has joined #openstack-swift | 21:36 | |
| *** dmorita has joined #openstack-swift | 21:36 | |
| tdasilva | mattoliverau: o/ | 21:42 |
| timburke | :-( i just realized that https://github.com/openstack/liberasurecode/blob/master/Makefile.am#L63 doesn't do what we want at all -- using libtool means that test/liberasurecode_test is actually a shell script | 21:48 |
| timburke | ...which explains why our gate keeps saying there were no allocs http://logs.openstack.org/50/431850/3/check/gate-liberasurecode-unit/c613b91/console.html#_2017-02-12_22_44_38_327007 | 21:49 |
| *** dmorita_ has joined #openstack-swift | 21:50 | |
| *** dmorita has quit IRC | 21:53 | |
| clayg | timburke: but there *is* no handoffs_first :'( | 21:54 |
| clayg | timburke: i'm also trying to keep the words to a minimum - it's just an example config - more detailed prose would be beter suited in docs - but there's an standing bug that those are missing | 21:55 |
| timburke | clayg: there used to be, we included it in our sample configs (https://github.com/openstack/swift/blob/2.10.0/etc/object-server.conf-sample#L299), and now it's going away. it was terrible and didn't do what ops wanted, but it existed | 21:56 |
| *** silor has quit IRC | 21:57 | |
| timburke | i agree that it doesn't need to be as long as all i said, but i feel like there should be *something* there about this thing that used to be but soon won't be | 21:57 |
| *** gatuus has joined #openstack-swift | 21:59 | |
| *** dmorita_ has quit IRC | 22:03 | |
| *** dmorita has joined #openstack-swift | 22:06 | |
| *** dmorita has quit IRC | 22:10 | |
| *** catintheroof has quit IRC | 22:20 | |
| *** tnovacik has quit IRC | 22:25 | |
| *** dja has joined #openstack-swift | 22:39 | |
| *** sams-gleb has quit IRC | 22:50 | |
| *** sams-gleb has joined #openstack-swift | 22:50 | |
| *** sams-gleb has quit IRC | 22:51 | |
| *** sams-gleb has joined #openstack-swift | 22:51 | |
| *** sams-gleb has quit IRC | 22:52 | |
| *** oshritf has quit IRC | 23:07 | |
| *** dmorita has joined #openstack-swift | 23:13 | |
| *** oshritf has joined #openstack-swift | 23:14 | |
| *** jamielennox is now known as jamielennox|away | 23:15 | |
| *** dmorita has quit IRC | 23:17 | |
| *** dmorita has joined #openstack-swift | 23:18 | |
| *** jamielennox|away is now known as jamielennox | 23:20 | |
| *** joeljwright has quit IRC | 23:20 | |
| *** dmorita has quit IRC | 23:22 | |
| *** g-k--1-w-m- has joined #openstack-swift | 23:23 | |
| *** g-k--1-w-m- has left #openstack-swift | 23:23 | |
| *** kei_yama has joined #openstack-swift | 23:30 | |
| *** jtomasek has quit IRC | 23:31 | |
| *** chsc has quit IRC | 23:34 | |
| *** jtomasek has joined #openstack-swift | 23:35 | |
| *** tonanhngo has quit IRC | 23:48 | |
| *** sams-gleb has joined #openstack-swift | 23:52 | |
| *** catintheroof has joined #openstack-swift | 23:55 | |
| *** tonanhngo has joined #openstack-swift | 23:55 | |
| *** sams-gleb has quit IRC | 23:56 | |
| kota_ | good morning | 23:58 |
| *** tonanhngo has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!