| notmyname | mattoliverau: I wanted to ask you about container sharding. how's it going? | 00:02 |
|---|---|---|
| notmyname | mattoliverau: also, have you talked to alex about docs at all? I'm going to try to send an email to her and the hp docs person tomorrow | 00:03 |
| mattoliverau | notmyname: I have talked briefly to Alex, told her would introduce her and get her more information when they came to hand. She's keen | 00:07 |
| notmyname | ok | 00:07 |
| mattoliverau | notmyname: sharing is going well, I've come up with a new "find best candidate subtrie" that doesn't use much ram. Tested it on some really large containers I've made, and the algorithm seems to work. So spend most of Friday last week plumbing it into the POC, plan on testing it this morning. Once that's done, I'm going to start running some better bench marks and start working on shrinking. | 00:09 |
| mattoliverau | *sharding | 00:09 |
| notmyname | ok | 00:09 |
| notmyname | what can we do to get more eyes on it? | 00:09 |
| mattoliverau | Now that is a good question :) If the benchmarks go well, maybe get it out of my github and into something more visable. | 00:11 |
| swifterdarrell | mattoliverau: yay benchmarks! | 00:12 |
| notmyname | I added a line on it to the meeting agenda | 00:12 |
| notmyname | https://wiki.openstack.org/wiki/Meetings/Swift | 00:14 |
| *** asettle-afk is now known as asettl | 00:14 | |
| *** asettl is now known as asettle | 00:14 | |
| *** dmorita has joined #openstack-swift | 00:15 | |
| notmyname | mattoliverau: thanks for talking to alex | 00:15 |
| mattoliverau | notmyname: cools, I'll see what I can get by then :) I have a team member of jrichli helping me on sharinding now, cause they were interested, so that's awesome. | 00:16 |
| notmyname | nice | 00:16 |
| notmyname | who? | 00:16 |
| notmyname | in irc? | 00:16 |
| mattoliverau | His IRC nick is blmartin, getting him up to speed. He's new to swift, so given him some brain dumps etc, and got him working on some unit tests until he feels he is ready to bite into something more meaty ;) | 00:18 |
| notmyname | great | 00:18 |
| notmyname | mattoliverau: also, you were one of the last people to comment on https://bugs.launchpad.net/swift/+bug/897451. If you don't leave a comment on there by the meeting, I'm going to ask you about it in the meeting :-) | 00:18 |
| openstack | Launchpad bug 897451 in OpenStack Object Storage (swift) "memcached exception when trying to connect to offline node" [Medium,Confirmed] - Assigned to Matthew Oliver (matt-0) | 00:18 |
| *** ho has joined #openstack-swift | 00:19 | |
| notmyname | mattoliverau: also, same story with https://bugs.launchpad.net/swift/+bug/1040883 | 00:19 |
| openstack | Launchpad bug 1040883 in OpenStack Object Storage (swift) "object-server partital write prevention not tested" [Low,In progress] - Assigned to Matthew Oliver (matt-0) | 00:19 |
| notmyname | mattoliverau: I'm not trying to dump it all on you, but you were the last person on these (and you assigned them to yourself) | 00:20 |
| notmyname | see what happens when i start looking at launchpad? ;-) | 00:20 |
| mattoliverau | notmyname: lol | 00:23 |
| notmyname | and with that, I'm headed home | 00:23 |
| mattoliverau | notmyname: no probs, I'll go and reset my brain on them, sorry for letthing them drop out of my mind not prioritising them | 00:24 |
| notmyname | no worries at all | 00:24 |
| ho | good morning! | 00:28 |
| mattoliverau | ho: morning | 00:29 |
| *** wbhuber has joined #openstack-swift | 00:31 | |
| *** gyee_ has quit IRC | 00:32 | |
| *** kutija|away has quit IRC | 00:33 | |
| ho | mattoliverau: morning! | 00:33 |
| *** kutija has joined #openstack-swift | 01:16 | |
| *** lastops has quit IRC | 01:20 | |
| *** lastops has joined #openstack-swift | 01:21 | |
| *** Kudos2me2 has joined #openstack-swift | 01:30 | |
| *** jamielennox|away is now known as jamielennox | 01:32 | |
| *** Kudos2me2 has quit IRC | 01:33 | |
| *** haigang has joined #openstack-swift | 01:34 | |
| *** asettle is now known as asettle-afk | 01:35 | |
| wbhuber | notmyname: you mentioned that you had 6+4 scheme across 5 servers, meaning 5 storage nodes? 20 drives each node? i guess that cluster was configured to handle 2 each server (5 * 2 = 6 + 4 scheme). | 01:43 |
| *** kota_ has joined #openstack-swift | 01:49 | |
| *** ChanServ sets mode: +v kota_ | 01:49 | |
| wbhuber | notmyname: disregard my question. | 01:53 |
| *** jkugel has left #openstack-swift | 02:37 | |
| *** wbhuber has quit IRC | 02:57 | |
| *** asettle-afk is now known as asettle | 03:02 | |
| *** nottrobin has quit IRC | 03:06 | |
| *** briancurtin has quit IRC | 03:06 | |
| *** zhiyan has quit IRC | 03:06 | |
| *** serverascode has quit IRC | 03:06 | |
| *** jeblair has quit IRC | 03:08 | |
| *** cppforlife_ has quit IRC | 03:09 | |
| *** lastops has quit IRC | 03:09 | |
| *** harlowja has quit IRC | 03:22 | |
| *** harlowja_ has joined #openstack-swift | 03:22 | |
| *** blmartin has joined #openstack-swift | 03:35 | |
| *** ajiang has left #openstack-swift | 04:08 | |
| *** haigang has quit IRC | 04:30 | |
| *** zhill has joined #openstack-swift | 04:38 | |
| *** ppai has joined #openstack-swift | 04:51 | |
| *** blmartin has quit IRC | 04:54 | |
| *** ktsuyuzaki has joined #openstack-swift | 04:56 | |
| *** kota_ has quit IRC | 04:57 | |
| *** kota_ has joined #openstack-swift | 05:00 | |
| *** ChanServ sets mode: +v kota_ | 05:00 | |
| *** SkyRocknRoll has joined #openstack-swift | 05:01 | |
| *** ktsuyuzaki has quit IRC | 05:02 | |
| *** ktsuyuzaki has joined #openstack-swift | 05:05 | |
| *** Triveni has joined #openstack-swift | 05:06 | |
| *** kota_ has quit IRC | 05:07 | |
| *** proteusguy has joined #openstack-swift | 05:08 | |
| *** zhill_desktop has quit IRC | 05:10 | |
| *** zhill has quit IRC | 05:11 | |
| *** kota_ has joined #openstack-swift | 05:13 | |
| *** ChanServ sets mode: +v kota_ | 05:13 | |
| *** haigang has joined #openstack-swift | 05:15 | |
| *** ktsuyuzaki has quit IRC | 05:16 | |
| *** ktsuyuza_ has joined #openstack-swift | 05:16 | |
| *** kota_ has quit IRC | 05:18 | |
| *** kota_ has joined #openstack-swift | 05:19 | |
| *** ChanServ sets mode: +v kota_ | 05:19 | |
| *** ktsuyuza_ has quit IRC | 05:21 | |
| *** ktsuyuzaki has joined #openstack-swift | 05:21 | |
| *** SkyRocknRoll has quit IRC | 05:23 | |
| *** kota_ has quit IRC | 05:23 | |
| *** kota_ has joined #openstack-swift | 05:26 | |
| *** ChanServ sets mode: +v kota_ | 05:26 | |
| *** ktsuyuzaki has quit IRC | 05:28 | |
| *** ktsuyuzaki has joined #openstack-swift | 05:28 | |
| *** kota_ has quit IRC | 05:30 | |
| *** kota_ has joined #openstack-swift | 05:31 | |
| *** ChanServ sets mode: +v kota_ | 05:31 | |
| *** ktsuyuzaki has quit IRC | 05:33 | |
| *** ktsuyuzaki has joined #openstack-swift | 05:34 | |
| *** kota_ has quit IRC | 05:36 | |
| *** haomaiwa_ has quit IRC | 06:08 | |
| *** silor has joined #openstack-swift | 06:22 | |
| *** ktsuyuzaki has quit IRC | 06:27 | |
| *** nottrobin has joined #openstack-swift | 06:30 | |
| *** haomaiwang has joined #openstack-swift | 06:35 | |
| *** serverascode has joined #openstack-swift | 06:39 | |
| *** mmcardle has joined #openstack-swift | 06:41 | |
| *** zhiyan has joined #openstack-swift | 06:46 | |
| *** ahonda has joined #openstack-swift | 06:47 | |
| *** cppforlife_ has joined #openstack-swift | 06:53 | |
| *** briancurtin has joined #openstack-swift | 07:09 | |
| *** mfalatic has joined #openstack-swift | 07:25 | |
| mfalatic | For SAIO, how can I force object reclamation to start after adjusting the object-replicator values in the configs? Trying to get tombstones to be collected quickly (for the purpose of experimentation) | 07:26 |
| mattoliverau | shorten the reclaim age, and shorten the run_pause between replication jobs | 07:35 |
| mfalatic | Thanks! | 07:46 |
| *** geaaru has joined #openstack-swift | 07:49 | |
| mfalatic | run_pause is only 30 seconds by default... I've set reclaim_age to 60 and it's still not happening - could something else be triggering it? | 07:52 |
| *** haomaiwang has quit IRC | 07:55 | |
| *** haomaiwang has joined #openstack-swift | 07:55 | |
| *** SkyRocknRoll has joined #openstack-swift | 07:57 | |
| *** haomaiwang has quit IRC | 08:05 | |
| *** jordanP has joined #openstack-swift | 08:20 | |
| *** haomaiwang has joined #openstack-swift | 08:20 | |
| *** mandarine has joined #openstack-swift | 08:21 | |
| *** mfalatic has quit IRC | 08:22 | |
| *** mfalatic has joined #openstack-swift | 08:22 | |
| mandarine | Hello Swifties (sorry, this was too easy and I couldn't not do this) | 08:34 |
| mandarine | I appear to have two more questions :) | 08:34 |
| mandarine | 1) Do any of you know a way to change the zone of a device? | 08:35 |
| mandarine | 2) Is there an internal mechanism in swift to make empty container expire? | 08:35 |
| mandarine | If you have a lead on any of these two topics, I'll be eternally thankful (well, relative eternity is a thing, ain't it?) | 08:36 |
| *** leopoldj has joined #openstack-swift | 08:42 | |
| *** chlong has quit IRC | 08:52 | |
| *** joeljwright has joined #openstack-swift | 08:56 | |
| *** proteusguy has quit IRC | 08:57 | |
| *** aix has joined #openstack-swift | 09:05 | |
| openstackgerrit | Christian Schwede proposed openstack/swift: WIP - Empty test commit - do not merge https://review.openstack.org/189609 | 09:20 |
| ho | mandarine: (1) Use swift-ring-builder <builder_file> set_info (2) I think swift doesn't have container expire. | 09:27 |
| *** tellesnobrega has quit IRC | 09:28 | |
| *** dmorita has quit IRC | 09:34 | |
| *** silor has quit IRC | 09:41 | |
| *** Triveni has quit IRC | 09:44 | |
| *** aix has quit IRC | 09:53 | |
| *** aix has joined #openstack-swift | 10:05 | |
| *** km has quit IRC | 10:16 | |
| openstackgerrit | Jiri Suchomel proposed openstack/swift: Let swift-object-info print information about files in given directory, if directory is provided instead of data file together with --recursive option. https://review.openstack.org/189258 | 10:45 |
| *** silor has joined #openstack-swift | 11:20 | |
| *** distroguy has joined #openstack-swift | 11:28 | |
| *** csmart has quit IRC | 11:30 | |
| *** distroguy has quit IRC | 11:32 | |
| *** SkyRocknRoll has quit IRC | 11:55 | |
| *** kei_yama has quit IRC | 11:56 | |
| *** haigang has quit IRC | 12:02 | |
| *** joeljwright has quit IRC | 12:04 | |
| *** tellesnobrega has joined #openstack-swift | 12:11 | |
| *** XiOViC has joined #openstack-swift | 12:21 | |
| *** XiOViC has quit IRC | 12:22 | |
| *** ppai has quit IRC | 12:23 | |
| openstackgerrit | Jiri Suchomel proposed openstack/swift: Let swift-object-info print information about files in given directory, if directory is provided instead of data file together with --recursive option. https://review.openstack.org/189258 | 12:29 |
| *** thurloat_isgone is now known as thurloat | 12:33 | |
| *** chlong has joined #openstack-swift | 12:37 | |
| mandarine | ho: The set_info thingy doesn't seem to be okay if I modify zones | 12:39 |
| *** joeljwright has joined #openstack-swift | 12:40 | |
| *** acoles_away is now known as acoles | 12:44 | |
| acoles | notmyname: oh dear (lost pin), does that mean we will lose all our good communication and harmony ;) | 12:51 |
| dmsimard | Apple's Open sourcing Swift -- should definitely abandon the Go rewrite and go with Swift so you can say Swift is written in Swift /s | 12:57 |
| *** wbhuber has joined #openstack-swift | 13:00 | |
| *** wbhuber has quit IRC | 13:07 | |
| *** acampbell has joined #openstack-swift | 13:11 | |
| *** janonymous_ has joined #openstack-swift | 13:12 | |
| janonymous_ | Hi, I have a Question: Just like every package comes with uninstall procedure , it there any uninstall procedure for swift also ? | 13:14 |
| *** petertr7_away is now known as petertr7 | 13:14 | |
| janonymous_ | + **along with dependencies. | 13:16 |
| petertr7 | Hello! I was wondering if anyone was familiar with form post. In particular, my cluster has trouble handling large file uploads (without segmenting roughly 4-5gb) through formposts. Specifically I get a 500 Internal Error. | 13:19 |
| petertr7 | In addition, if I run a script to continuously upload files onto swift, formpost is unable to handle even 2gb files. | 13:19 |
| *** proteusguy has joined #openstack-swift | 13:19 | |
| petertr7 | I was reading through the formpost code and thinking perhaps the way the cluster handles form post and other uploads causes conflicts of resources | 13:20 |
| *** lastops has joined #openstack-swift | 13:21 | |
| *** jamielennox is now known as jamielennox|away | 13:23 | |
| cschwede | petertr7: the 2gb fompost limitation sounds familiar, which swift version are you running? | 13:27 |
| cschwede | petertr7: https://github.com/openstack/swift/commit/7056ec6a16fd8707564ec4b0a05cab461ee2a80d | 13:27 |
| petertr7 | Good morning/afternoon Christian! I'm running Kilo | 13:27 |
| cschwede | petertr7: Hello Peter! Well, that patch is included in Kilo, so there must be another reason. Do you have access to the proxy server logs? | 13:29 |
| petertr7 | I should. Let me check | 13:30 |
| petertr7 | They should be in /etct/swift/logs? | 13:30 |
| petertr7 | Sorry, my mistake, we're still on Juno | 13:32 |
| cschwede | More likely /var/logs/swift | 13:32 |
| petertr7 | Strangely my /var/logs/swift/proxy.log is empty and not logging any errors as I attempt to upload large files | 13:34 |
| *** janonymous__ has joined #openstack-swift | 13:36 | |
| *** janonymous_ has quit IRC | 13:36 | |
| jordanP | petertr7, do you have a webserver in front of the proxy ? like apache ? if so check its logs maybe | 13:36 |
| petertr7 | That's right, apache atm. I'm pulling up the logs now | 13:37 |
| petertr7 | My /var/log/apache2/error.log doesn't get updated on failed uploads either | 13:38 |
| *** acampbell has quit IRC | 13:39 | |
| *** acampbell has joined #openstack-swift | 13:40 | |
| *** ppai has joined #openstack-swift | 13:40 | |
| *** jeblair has joined #openstack-swift | 13:43 | |
| janonymous__ | http://docs.openstack.org/developer/swift/development_saio.html#setting-up-rsync-----> sudo apt-get install policycoreutils (is also needed which doesn't come in ubuntu packages by default, should be added in doc) for -- >setsebool-->sudo setsebool -P rsync_full_access 1 | 13:50 |
| *** ajiang has joined #openstack-swift | 13:53 | |
| *** breitz has quit IRC | 13:56 | |
| *** ptb has joined #openstack-swift | 13:59 | |
| *** wbhuber has joined #openstack-swift | 14:00 | |
| *** ptb has quit IRC | 14:01 | |
| *** breitz has joined #openstack-swift | 14:02 | |
| *** blmartin has joined #openstack-swift | 14:03 | |
| openstackgerrit | Charles Hsu proposed openstack/python-swiftclient: Add ability to download objects to particular folder. https://review.openstack.org/160283 | 14:06 |
| *** ppai has quit IRC | 14:07 | |
| *** proteusguy has quit IRC | 14:11 | |
| *** proteusguy has joined #openstack-swift | 14:12 | |
| *** jrichli has joined #openstack-swift | 14:16 | |
| *** ptb has joined #openstack-swift | 14:16 | |
| *** lpabon has joined #openstack-swift | 14:32 | |
| *** minwoob has joined #openstack-swift | 14:46 | |
| *** lpabon has quit IRC | 14:48 | |
| *** ptb has quit IRC | 14:49 | |
| *** leopoldj has quit IRC | 14:50 | |
| janonymous__ | Hi , | 14:52 |
| janonymous__ | Jun 9 20:22:01 janonymous- proxy-server: STDERR: (10485) accepted ('127.0.0.1', 48084) Jun 9 20:22:01 janonymous- account-server: STDERR: (10534) accepted ('127.0.0.1', 43394) | 14:53 |
| swifterdarrell | mfalatic: still unable to get tombstones reaped? | 14:53 |
| janonymous__ | could somebody point out the problem .. | 14:53 |
| mfalatic | Hey there | 14:54 |
| mfalatic | swifterdarrell - they do eventually get reaped (after a day or more it seems) but I had no luck getting them to reap in the 60 second timeframe by adjusting reclaim_age. | 14:56 |
| mfalatic | note that this is SAIO based on kilo stable, nothing unusual. | 14:57 |
| mfalatic | I added "reclaim_age 60" to object_expirer.conf as well as the four confs under object_server | 14:59 |
| mfalatic | "reclaim_age = 60" | 14:59 |
| swifterdarrell | mfalatic: perhaps double-check that the reclaim_age setting is under [object-replicator] for all object server confs | 14:59 |
| mfalatic | It is | 14:59 |
| swifterdarrell | mfalatic: and then restart them? or use the --run-one or whatever, if that deamon has that flag? | 15:00 |
| mfalatic | It's not urgent BTW, just peculiar. I actually did resetswift, remakerings and swift-init main stop followed by start. | 15:01 |
| mfalatic | Then I uploaded a test file, deleted it, and watched the /srv filesystem. The tombstones didn't go away for a many minutes I spent checking in on it periodically. | 15:02 |
| swifterdarrell | mfalatic: what kind of start? startmain or startall? | 15:02 |
| mfalatic | start-main. | 15:02 |
| mfalatic | sorry | 15:02 |
| mfalatic | start-init main | 15:02 |
| mfalatic | (which is what saio has you do) | 15:03 |
| swifterdarrell | mfalatic: object-replicator isn't part of main, I don't think; so you'll need to run the replicator by hand or startall | 15:03 |
| mfalatic | startmain = start-init main of course | 15:03 |
| mfalatic | hmm. Replicator is separate? Let me look. | 15:03 |
| swifterdarrell | mfalatic: looks like swift-object-replicator has this option: -o, --once only run one pass of daemon | 15:04 |
| swifterdarrell | mfalatic: ya, "main" is just the swift-*-server procs | 15:04 |
| mfalatic | I've rebooted the vm this is all in. | 15:05 |
| mfalatic | Not seeing any swift processes at this point (before manually starting things) | 15:06 |
| mfalatic | I've reset swift and remade the rings. I normally just run startmain at this point. | 15:08 |
| mfalatic | I see startrest there... but I've never actually run it. Wouldn't it be quite broken if replication isn't running at that point? | 15:09 |
| *** chlong is now known as chlong-zzz | 15:10 | |
| mfalatic | swifterdarrell: that's interesting... I've never run "rest" but clearly it kicks off many more processes. | 15:11 |
| mfalatic | yet up to now it seemed like things worked normally despite that, for what I've tested. | 15:12 |
| swifterdarrell | mfalatic: it's an SAIO... there's little definition of "broken" | 15:12 |
| swifterdarrell | mfalatic: if by "broken" you mean that the background consistency daemons (auditor, replicator, reapers, etc.) aren't running then yes, "startmain" leaves your SAIO "broken" | 15:12 |
| swifterdarrell | mfalatic: really, for lots of testing you only need the main daemons and all the daemons consume more RAM and whatnot, and often SAIO vms are small | 15:13 |
| swifterdarrell | mfalatic: do the SAIO docs not mention startrest and how background daemons run/get started and how they're separate from the "main" servers? | 15:13 |
| swifterdarrell | mfalatic: (I haven't looked at SAIO docs in a whlie) | 15:13 |
| swifterdarrell | mfalatic: while we're on the topic, clayg made a super-sweet Vagrant-based SAIO that I've been using for a bit now: https://github.com/swiftstack/vagrant-swift-all-in-one | 15:14 |
| mfalatic | Will make a note of that, thanks! | 15:16 |
| swifterdarrell | mfalatic: np | 15:16 |
| *** annegentle has joined #openstack-swift | 15:16 | |
| mfalatic | swifterdarrell: As a note, even with "rest" running, tombstones aren't getting removed any more quickly. My main question is, which config should have the reclaim_age set for this to work? | 15:17 |
| *** annegentle has quit IRC | 15:18 | |
| swifterdarrell | mfalatic: what'ever the swift-object-replicator is using; and inside that, under the [object-replicator] section | 15:19 |
| swifterdarrell | mfalatic: process command-line ought to indicate the config file used by swift-object-replicator | 15:19 |
| thurloat | has anyone had experience with the seagate HMR drives in a swift deployment? | 15:21 |
| thurloat | sorry, SMR | 15:21 |
| mfalatic | swifterdarrell: Yep - (It's clearer now that all the processes are running) it uses the object-server configs... which definitely have the reclaim_age set in them. | 15:21 |
| *** adutta has joined #openstack-swift | 15:22 | |
| adutta | The accoun auditor throws the following error for quite a few accounts: "The total object_count for the container (5) does not match the sum of object_count across policies (0)" . Probably it is thrown at this line https://github.com/openstack/swift/blob/master/swift/account/auditor.py#L123 . Any reason on why it may be happening. It is after upgrade from 1.12 to 2.2.2 | 15:25 |
| *** aix has quit IRC | 15:29 | |
| swifterdarrell | mfalatic: maybe there's something else affecting actual reclaiming in the code that we're forgetting; I don't have time to dig atm | 15:31 |
| mfalatic | That's ok, at least now I know to run "rest" (which with "main" == "all") | 15:33 |
| mfalatic | Thanks for the vagrant tip! | 15:33 |
| *** gyee_ has joined #openstack-swift | 15:34 | |
| *** lpabon has joined #openstack-swift | 15:38 | |
| *** lpabon has quit IRC | 15:40 | |
| *** aix has joined #openstack-swift | 15:41 | |
| *** Fin1te has joined #openstack-swift | 15:51 | |
| *** haomaiwang has quit IRC | 15:55 | |
| openstackgerrit | Alex Khivin proposed openstack/swift: Make umask customizable for Swift daemons https://review.openstack.org/189785 | 15:55 |
| *** haomaiwang has joined #openstack-swift | 15:55 | |
| openstackgerrit | Alex Khivin proposed openstack/swift: Make umask customizable for Swift daemons https://review.openstack.org/189789 | 16:00 |
| hugokuo | morning ! | 16:01 |
| mfalatic | Indeed it is | 16:03 |
| mfalatic | :-) | 16:03 |
| hugokuo | err.. actually it's midnight here :) | 16:04 |
| hugokuo | is there a way to disallow any access to an Swift account temporarily ? | 16:05 |
| hugokuo | account rate limit to 0 ? | 16:05 |
| *** petertr7 is now known as petertr7_away | 16:07 | |
| openstackgerrit | Alex Khivin proposed openstack/swift: Make umask customizable for Swift daemons https://review.openstack.org/189785 | 16:09 |
| *** gyee_ has quit IRC | 16:10 | |
| peluse | clayg, around? | 16:14 |
| openstackgerrit | Michael Barton proposed openstack/swift: go: ring self-reloading https://review.openstack.org/187030 | 16:18 |
| *** annegentle has joined #openstack-swift | 16:25 | |
| *** lcurtis has joined #openstack-swift | 16:37 | |
| *** jordanP has quit IRC | 16:38 | |
| *** aix has quit IRC | 16:39 | |
| *** Fin1te has quit IRC | 16:42 | |
| hurricanerix_ | hugokuo: I think the ratelimit middleware allows you to black/white list accounts: http://docs.openstack.org/developer/swift/ratelimit.html#black-white-listing | 16:45 |
| *** ptb has joined #openstack-swift | 16:45 | |
| *** ptb has quit IRC | 16:46 | |
| *** ptb has joined #openstack-swift | 16:47 | |
| *** ptb has quit IRC | 16:48 | |
| *** acampbell has quit IRC | 16:49 | |
| hugokuo | hurricanerix_: thx. This way may still allow GET request to the account ? | 16:50 |
| hurricanerix_ | hugokuo: no, if you blacklist an account, it should prevent all access. | 16:51 |
| hugokuo | hurricanerix_: nice. I'll check the code tmr. Thanks for this information. pretty useful. | 16:51 |
| hurricanerix_ | np | 16:51 |
| hugokuo | X-Account-Sysmeta-Global-Write-Ratelimit <--- make me a bit confused due to the string "Write" in it :) | 16:52 |
| hurricanerix_ | hugokuo: Yeah, I just tested it on my SAIO, and it did prevent GETs. | 16:58 |
| hugokuo | hurricanerix_: appreciate for a quick try . You rock | 16:59 |
| *** petertr7_away is now known as petertr7 | 17:00 | |
| *** Sripadwallabha has joined #openstack-swift | 17:00 | |
| Sripadwallabha | Hello @all | 17:00 |
| Sripadwallabha | I am trying to use python-swiftclient and trying to authenticate using user_id instead of user_name | 17:01 |
| Sripadwallabha | is it possible? | 17:01 |
| Sripadwallabha | I am getting following excption | 17:01 |
| Sripadwallabha | File "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 1209, in get_auth insecure=self.insecure) File "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 406, in get_auth auth_version=auth_version) File "/usr/lib/python2.7/dist-packages/swiftclient/client.py", line 341, in get_auth_keystone raise ClientException('Authorization Failure. %s' % err) | 17:01 |
| *** dstanek has left #openstack-swift | 17:02 | |
| Sripadwallabha | if I dont specify user_name but just mention user_id inside os_options dict | 17:02 |
| hurricanerix_ | hugokuo: So the first request after blacklisting was allowed, but after that is blocks access. http://pastebin.com/Rw9dE9su | 17:03 |
| hugokuo | hurricanerix_: perfect. I like it. I'm going to update and add more stuff into Swift community document. | 17:05 |
| hugokuo | ctennis: mlanner : Check this out http://pastebin.com/Rw9dE9su .... This could be the best choice to disallow access to the account :) | 17:06 |
| openstackgerrit | Louis Taylor proposed openstack/python-swiftclient: Rename `obj` to `name` in *_object wrappers https://review.openstack.org/189815 | 17:07 |
| *** kevinc_ has joined #openstack-swift | 17:07 | |
| *** acoles is now known as acoles_away | 17:10 | |
| notmyname | good morning | 17:11 |
| *** mmcardle has quit IRC | 17:12 | |
| *** remix_tj has quit IRC | 17:12 | |
| kevinc_ | If I have admin access on keystone, is there any way to do a head on every tenant with out having the admin account added to every tenant? | 17:13 |
| *** remix_tj has joined #openstack-swift | 17:14 | |
| notmyname | kevinc_: if that user has the "reseller admin" role/group, then it will work. IIRC there's some keystone config to map the swift reseller admin to a role in keystone | 17:23 |
| notmyname | kevinc_: but the general answer is that a "reseller admin" as full read/write access for a given auth system (ie root-like) | 17:23 |
| *** zhill_desktop has joined #openstack-swift | 17:29 | |
| *** SkyRocknRoll has joined #openstack-swift | 17:30 | |
| *** Sripadwallabha has left #openstack-swift | 17:44 | |
| *** janonymous__ is now known as new | 17:47 | |
| *** new is now known as zzzzz | 17:47 | |
| *** mmcardle has joined #openstack-swift | 17:47 | |
| *** otoolee has quit IRC | 17:47 | |
| *** zzzzz has left #openstack-swift | 17:48 | |
| *** zzzzz has joined #openstack-swift | 17:48 | |
| *** zzzzz is now known as janonymous_ | 17:48 | |
| kevinc_ | notmyname: perfect, thank you that solved my issue | 17:49 |
| *** torgomatic has quit IRC | 17:52 | |
| *** torgomatic has joined #openstack-swift | 17:53 | |
| *** ChanServ sets mode: +v torgomatic | 17:53 | |
| *** otoolee has joined #openstack-swift | 17:55 | |
| kevinc_ | One more question, when I add a tenant to Keystone when does the account/tenant get added to Swift? Does the account/tenant get added when the tenant is added to keystone, when the user first tries to access Swift or something else? | 17:56 |
| *** mmcardle has quit IRC | 18:00 | |
| *** gyee_ has joined #openstack-swift | 18:02 | |
| *** geaaru has quit IRC | 18:02 | |
| *** harlowja_ has quit IRC | 18:18 | |
| hugokuo | kevinc_: 1) If account_auto_create=true , the swift account for corresponding keystone tenant will be created while the user try to access it first time | 18:22 |
| hugokuo | kevinc_: 2) if auto create = false, the reseller admin must creates the swift account for the tenant as initialization | 18:23 |
| MooingLemur | gmornin | 18:24 |
| kevinc_ | hugokuo: thanks! | 18:24 |
| hugokuo | kevinc_: np | 18:24 |
| *** harlowja has joined #openstack-swift | 18:25 | |
| openstackgerrit | Jiri Suchomel proposed openstack/swift: Let swift-object-info print information about files in given directory, if directory is provided instead of data file together with --recursive option. https://review.openstack.org/189258 | 18:26 |
| *** annegentle has quit IRC | 18:30 | |
| *** si1v3r has joined #openstack-swift | 18:35 | |
| si1v3r | Hey all. We have 20ish object storage nodes and one is running a much higher load than it's peers with about 100 more running rsync processes. | 18:36 |
| si1v3r | Where would be a good place to find out why one node would be running so many more than peers? | 18:37 |
| *** blmartin has quit IRC | 18:39 | |
| ctennis | I'd look in the swift logs and see if replication is running, perhaps due to hardware issues | 18:39 |
| si1v3r | object-replicator logs show a similar number of events between the bad node and good ones. | 18:42 |
| *** acampbell has joined #openstack-swift | 18:42 | |
| *** petertr7 is now known as petertr7_away | 18:47 | |
| *** openstackgerrit has quit IRC | 18:48 | |
| *** openstackgerrit has joined #openstack-swift | 18:48 | |
| *** petertr7_away is now known as petertr7 | 18:49 | |
| *** silor1 has joined #openstack-swift | 18:52 | |
| *** silor has quit IRC | 18:52 | |
| si1v3r | Looks like the old rsyncs are hanging onto one specific disk, according to lsof | 18:53 |
| notmyname | maybe an old replicatin process is running and firing those off | 18:58 |
| si1v3r | There is this monster running: http://pastebin.com/NmzUqS5m | 19:01 |
| si1v3r | other rsync processes seem to be replicating objects from more than one disk at a time. | 19:02 |
| ctennis | did you change your storage? or do you have a failed disk elsewhere? | 19:02 |
| si1v3r | No changes have been made recently (all users say that...), the disk is working and has no errors in dmesh. | 19:03 |
| si1v3r | *dmesg | 19:03 |
| ctennis | replication wouldn't be running unless something is failed or something changed | 19:03 |
| ctennis | rsync replication that is | 19:03 |
| MooingLemur | peluse, clayg: any chance either of you might be available today to take a look at a reconstruction issue that's been happening? I've kinda left the cluster alone so that the issue will actually persist, so there might be a smoking gun pointing to an actual bug that can be found :) | 19:04 |
| ctennis | or things were out of whack for a while | 19:04 |
| *** acampbell has quit IRC | 19:04 | |
| *** annegentle has joined #openstack-swift | 19:06 | |
| si1v3r | ctennis: Is there a way to see what triggered it? | 19:07 |
| ctennis | you could find a partition that is replicating to another node and see where it's supposed to be using swift-get-nodes, and check the primary locations. if it just happened all of a sudden it smells like you have a bad disk somewhere in the system. | 19:09 |
| *** annegentle has quit IRC | 19:14 | |
| *** kevinc_ has quit IRC | 19:22 | |
| *** marcusvrn has joined #openstack-swift | 19:24 | |
| *** annegentle has joined #openstack-swift | 19:30 | |
| *** Syed_A has joined #openstack-swift | 19:34 | |
| Syed_A | Hello folks, I created a SwiftOperator role, added it in proxy-server.conf but when a user with SwiftOperator roles tries to use swift it gets 403(forbidden), however a user with admin role can successfully use the commands. | 19:36 |
| Syed_A | How do i make sure that SwiftOperator also has admin like privileges to use swift | 19:37 |
| *** gyee_ has quit IRC | 19:38 | |
| occupant | You may want a cyanide pill in case things go bad. | 19:38 |
| Syed_A | Do i need to make any change in keystone policy.json file | 19:42 |
| *** InAnimaTe has left #openstack-swift | 19:50 | |
| notmyname | Syed_A: today swift doesn't use the policy.json file. so I don't htink that would help anything | 19:52 |
| *** mmcardle has joined #openstack-swift | 19:54 | |
| *** silor1 has quit IRC | 19:57 | |
| Syed_A | notmyname: I am slightly confused as to why a user with operator_role i.e. SwiftOperator is forbidden from using swift | 19:59 |
| *** jamielennox|away is now known as jamielennox | 20:01 | |
| Syed_A | notmyname: operator_roles = Member, admin, SwiftOperator, ResellerAdmin, _member_ | 20:02 |
| notmyname | Syed_A: there are 2 config vars that can be set in swift (in proxy-server.conf in the [filter:keystoneauth] section) that relate to what you're doing | 20:04 |
| notmyname | operator_roles and reseller_admin_role | 20:04 |
| notmyname | the first is for what you would consider an "owner" of the swift account. eg you get an account on a swift cluster and you can do what you want with it, but you can't mess with stuff in other accounts | 20:05 |
| *** SkyRocknRoll has quit IRC | 20:06 | |
| notmyname | the second is what you'd consider the owner of every account that keystone manages in swift. ie it's the owner of the entire reseller (keystone in your case) and can do anything in any account for that auth system | 20:06 |
| notmyname | it's a bad idea to have end-users that have a reseller admin role. that role is intended for operators | 20:07 |
| notmyname | I'll admin that the first config value (operator_roles) is poorly named. I think that's because of early miscommunication between swift and keystone about what is what, and it's stuck because of compatibility reasons | 20:08 |
| notmyname | s/admin/admit/ | 20:08 |
| Syed_A | notmyname: Roger that. I did the first one. Created a user and then assigned it the SwiftOperator role. | 20:09 |
| notmyname | ok. I think that just means that the user will have access to their own account | 20:10 |
| Syed_A | notmyname: this user is able to get authenticated by keystone but somehow it is getting rejected by swift, thus getting a 403 | 20:10 |
| notmyname | ok | 20:10 |
| notmyname | Syed_A: in the [app:proxy-server] section, do you have account_autocreate set to true? | 20:11 |
| Syed_A | notmyname: Yes i do. | 20:12 |
| Syed_A | notmyname: I can pastebin the proxy-server.conf for you ? | 20:12 |
| notmyname | ok | 20:13 |
| *** mmcardle has quit IRC | 20:13 | |
| *** lpabon has joined #openstack-swift | 20:14 | |
| Syed_A | notmyname: Here it is: http://pastebin.com/z4Aedt22 | 20:15 |
| *** thurloat is now known as thurloat_isgone | 20:20 | |
| notmyname | Syed_A: looking | 20:24 |
| Syed_A | notmyname: keystone endpoint-list : http://pastebin.com/LAkpcjTw | 20:25 |
| *** zaitcev has joined #openstack-swift | 20:28 | |
| *** ChanServ sets mode: +v zaitcev | 20:28 | |
| notmyname | Syed_A: do you see anything in the logs? | 20:32 |
| notmyname | I see one place (only one) that can return a 403 and also has some extra logging | 20:33 |
| Syed_A | notmyname: Nope, i don't see anything unsual. | 20:36 |
| Syed_A | notmyname: What is it, please do tell :) | 20:36 |
| *** lastops has quit IRC | 20:37 | |
| Syed_A | notmyname: when i use --debug flag, i can see this : INFO:urllib3.connectionpool:Starting new HTTP connection (1): proxy.cloud.comDEBUG:urllib3.connectionpool:"HEAD /v1/AUTH_s HTTP/1.1" 403 0 | 20:46 |
| Syed_A | notmyname: http://paste.openstack.org/show/278851/ | 20:47 |
| Syed_A | notmyname: proxy-server logs http://paste.openstack.org/show/278853/ | 20:49 |
| openstackgerrit | Joel Wright proposed openstack/python-swiftclient: Reduce memory usage for download/delete and add --no-shuffle option to st_download https://review.openstack.org/130339 | 20:49 |
| zaitcev | Syed_A: You typoed something in the endpoint descriptor in Keystone. That AUTH_s is the remnant of what should be interpolated from AUTH_%(tenant_id)s. | 20:55 |
| *** petertr7 is now known as petertr7_away | 20:56 | |
| *** thurloat_isgone is now known as thurloat | 20:59 | |
| *** annegentle has quit IRC | 21:00 | |
| Syed_A | notmyname: Ahhh, i feel so stupid. | 21:04 |
| Syed_A | notmyname: Thanks a lot for pointing this out :) | 21:04 |
| Syed_A | notmyname: I was struggling from past 2 days | 21:04 |
| notmyname | Syed_A: was it what zaitcev said? | 21:04 |
| Syed_A | notmyname: Ahh yeas, looks like it. I missed it in the endpoint. I have to check it. | 21:06 |
| Syed_A | zaitcev: Thanks man :) | 21:06 |
| notmyname | thanks zaitcev! | 21:06 |
| *** annegentle has joined #openstack-swift | 21:10 | |
| *** wbhuber_ has joined #openstack-swift | 21:14 | |
| *** thurloat is now known as thurloat_isgone | 21:14 | |
| *** wbhuber has quit IRC | 21:17 | |
| *** csmart has joined #openstack-swift | 21:18 | |
| zaitcev | Syed_A: That syntax catches people all the time, I wish we had a simple special case... Like, AUTH_# and voila. | 21:20 |
| *** annegentle has quit IRC | 21:21 | |
| *** annegentle has joined #openstack-swift | 21:22 | |
| *** csmart has quit IRC | 21:22 | |
| *** lpabon has quit IRC | 21:26 | |
| *** evanjfraser has quit IRC | 21:27 | |
| *** evanjfraser has joined #openstack-swift | 21:27 | |
| Syed_A | zaitcev: Yes it does, i spent a lot of time trying to figure it out. Thanks to you now it is fixed. :) | 21:30 |
| *** csmart has joined #openstack-swift | 21:32 | |
| *** lpabon has joined #openstack-swift | 21:34 | |
| *** lpabon has quit IRC | 21:36 | |
| *** csmart has quit IRC | 21:36 | |
| *** Syed_A has quit IRC | 21:38 | |
| *** thurloat_isgone is now known as thurloat | 21:45 | |
| *** thurloat is now known as thurloat_isgone | 21:47 | |
| *** csmart has joined #openstack-swift | 21:53 | |
| *** joeljwright has quit IRC | 21:54 | |
| *** csmart has quit IRC | 21:57 | |
| *** csmart has joined #openstack-swift | 22:03 | |
| si1v3r | Ok, I cannot figure out the difference between this slot drive and the rest of the cluster. How can I make sure it's safe to reformat the thing? | 22:06 |
| *** wbhuber_ has quit IRC | 22:06 | |
| *** wbhuber_ has joined #openstack-swift | 22:06 | |
| *** wbhuber__ has joined #openstack-swift | 22:07 | |
| *** csmart has quit IRC | 22:08 | |
| *** annegentle has quit IRC | 22:08 | |
| *** jrichli has quit IRC | 22:09 | |
| *** wbhuber_ has quit IRC | 22:11 | |
| *** csmart has joined #openstack-swift | 22:12 | |
| *** minwoob has quit IRC | 22:13 | |
| si1v3r | What does "container-updater account is not mounted" in container.error.log mean? | 22:15 |
| *** csmart has quit IRC | 22:16 | |
| mattoliverau | Morning | 22:17 |
| *** csmart has joined #openstack-swift | 22:20 | |
| *** csmart has quit IRC | 22:24 | |
| *** wbhuber__ has quit IRC | 22:35 | |
| *** adutta has quit IRC | 22:37 | |
| *** csmart has joined #openstack-swift | 22:46 | |
| *** chlong-zzz has quit IRC | 22:58 | |
| zaitcev | si1v3r: The only similar message that I see is swift/container/updater.py: self.logger.warn(_('%s is not mounted'), device). Did you actually call one of your devices "account"? | 23:00 |
| *** kei_yama has joined #openstack-swift | 23:18 | |
| *** lcurtis has quit IRC | 23:19 | |
| *** zhill_desktop has quit IRC | 23:21 | |
| *** asettle has quit IRC | 23:23 | |
| notmyname | patch 189015 is rechecked. -infra says that the pyeclib/jerasure issue should be fixed | 23:29 |
| patchbot | notmyname: https://review.openstack.org/#/c/189015/ | 23:29 |
| notmyname | I'm watching zuul status | 23:30 |
| notmyname | if it passes, then torgomatic can you get out the plunger? | 23:30 |
| *** lpabon has joined #openstack-swift | 23:38 | |
| *** lpabon has quit IRC | 23:38 | |
| *** harlowja has quit IRC | 23:42 | |
| *** harlowja has joined #openstack-swift | 23:43 | |
| notmyname | seems to have passed | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!