MooingLemur | they are in the same container (cyberduck's default of putting them in .file-segments/... | 00:00 |
---|---|---|
notmyname | MooingLemur: can you read the segments normally? | 00:00 |
openstackgerrit | Samuel Merritt proposed a change to openstack/swift: Move all DLO functionality to middleware https://review.openstack.org/63326 | 00:01 |
*** mmcardle has quit IRC | 00:01 | |
*** ekarlso- has joined #openstack-swift | 00:04 | |
MooingLemur | notmyname: looks like it. I picked one of the segments, and I was able to get a 200 off of it. I can also read the manifest with ?multipart-manifest=get unauthenticated | 00:04 |
*** mlipchuk has quit IRC | 00:04 | |
*** matsuhashi has joined #openstack-swift | 00:12 | |
*** byeager has quit IRC | 00:23 | |
*** shri has joined #openstack-swift | 00:27 | |
*** nosnos has joined #openstack-swift | 00:33 | |
*** nosnos_ has joined #openstack-swift | 00:44 | |
*** nosnos has quit IRC | 00:47 | |
*** ekarlso has quit IRC | 00:50 | |
*** nosnos_ has quit IRC | 00:54 | |
*** nosnos has joined #openstack-swift | 00:54 | |
*** dmorita has joined #openstack-swift | 00:57 | |
*** mmcardle has joined #openstack-swift | 00:58 | |
*** gyee has quit IRC | 00:59 | |
*** mmcardle has quit IRC | 01:02 | |
openstackgerrit | Jon Snitow proposed a change to openstack/swift: Non-blocking but useful improvements to Account ACLs https://review.openstack.org/69321 | 01:09 |
*** shri has quit IRC | 01:49 | |
openstackgerrit | A change was merged to openstack/swift: Fix container quota MW for handling a bad source path https://review.openstack.org/58500 | 01:53 |
*** mmcardle has joined #openstack-swift | 01:58 | |
*** mmcardle has quit IRC | 02:03 | |
*** byeager has joined #openstack-swift | 02:18 | |
openstackgerrit | Samuel Merritt proposed a change to openstack/swift: Move all DLO functionality to middleware https://review.openstack.org/63326 | 02:54 |
*** byeager has quit IRC | 02:56 | |
*** byeager has joined #openstack-swift | 02:57 | |
*** mmcardle has joined #openstack-swift | 02:59 | |
*** mjseger has quit IRC | 03:00 | |
*** mmcardle1 has joined #openstack-swift | 03:01 | |
*** mmcardle has quit IRC | 03:01 | |
*** mmcardle1 has quit IRC | 03:06 | |
openstackgerrit | A change was merged to openstack/swift: Add some tests for bin/swift-recon https://review.openstack.org/62584 | 03:11 |
*** byeager_ has joined #openstack-swift | 03:50 | |
*** byeager has quit IRC | 03:53 | |
*** byeager_ has quit IRC | 03:59 | |
*** byeager has joined #openstack-swift | 04:02 | |
*** mmcardle has joined #openstack-swift | 04:02 | |
*** byeager has quit IRC | 04:06 | |
*** mmcardle has quit IRC | 04:06 | |
*** ctennis has quit IRC | 04:16 | |
*** joearnold has quit IRC | 04:16 | |
*** swifterdarrell has quit IRC | 04:16 | |
*** mlanner has quit IRC | 04:16 | |
*** minnear has quit IRC | 04:16 | |
*** otherjon has quit IRC | 04:16 | |
*** zackmdavis has quit IRC | 04:16 | |
*** amandap has quit IRC | 04:16 | |
*** hugokuo has quit IRC | 04:16 | |
*** acorwin has quit IRC | 04:16 | |
*** koolhead17 has quit IRC | 04:16 | |
*** anderstj has quit IRC | 04:16 | |
*** acorwin has joined #openstack-swift | 04:22 | |
*** minnear has joined #openstack-swift | 04:22 | |
*** ctennis has joined #openstack-swift | 04:22 | |
*** swifterdarrell has joined #openstack-swift | 04:23 | |
*** ChanServ sets mode: +v swifterdarrell | 04:23 | |
*** hugokuo has joined #openstack-swift | 04:24 | |
*** joearnold has joined #openstack-swift | 04:25 | |
*** mlanner has joined #openstack-swift | 04:26 | |
*** alpha_ori has joined #openstack-swift | 04:27 | |
*** otherjon has joined #openstack-swift | 04:28 | |
*** zackmdavis has joined #openstack-swift | 04:29 | |
*** amandap has joined #openstack-swift | 04:30 | |
*** anderstj has joined #openstack-swift | 04:30 | |
*** koolhead17 has joined #openstack-swift | 04:31 | |
*** mmcardle has joined #openstack-swift | 05:02 | |
*** mmcardle has quit IRC | 05:07 | |
*** haomaiwang has quit IRC | 05:12 | |
*** haomaiwang has joined #openstack-swift | 05:12 | |
*** nshaikh has joined #openstack-swift | 05:23 | |
*** mkollaro has joined #openstack-swift | 05:28 | |
*** ppai has joined #openstack-swift | 05:43 | |
*** nosnos_ has joined #openstack-swift | 06:00 | |
*** nosnos has quit IRC | 06:03 | |
*** mmcardle has joined #openstack-swift | 06:04 | |
*** mmcardle has quit IRC | 06:09 | |
*** zaitcev has quit IRC | 06:26 | |
*** matsuhashi has quit IRC | 06:33 | |
*** matsuhashi has joined #openstack-swift | 06:33 | |
*** glange___ has joined #openstack-swift | 06:34 | |
*** ChanServ sets mode: +v glange___ | 06:34 | |
openstackgerrit | A change was merged to openstack/swift: Add Requirement for Storage Policy Names https://review.openstack.org/70497 | 06:35 |
*** jd__` has joined #openstack-swift | 06:37 | |
*** nshaikh has quit IRC | 06:38 | |
*** alpha_ori has quit IRC | 06:38 | |
*** joearnold has quit IRC | 06:38 | |
*** acorwin has quit IRC | 06:38 | |
*** peluse has quit IRC | 06:38 | |
*** glange has quit IRC | 06:38 | |
*** zul has quit IRC | 06:38 | |
*** jd__ has quit IRC | 06:38 | |
*** krtaylor has quit IRC | 06:38 | |
*** mordred has quit IRC | 06:38 | |
*** jd__` is now known as jd__ | 06:38 | |
*** alpha_ori has joined #openstack-swift | 06:39 | |
*** acorwin has joined #openstack-swift | 06:40 | |
*** joearnold has joined #openstack-swift | 06:41 | |
*** zul has joined #openstack-swift | 06:45 | |
*** nshaikh has joined #openstack-swift | 06:47 | |
*** mordred has joined #openstack-swift | 06:54 | |
*** krtaylor has joined #openstack-swift | 06:59 | |
*** haomaiwa_ has joined #openstack-swift | 07:02 | |
*** haomaiwang has quit IRC | 07:04 | |
*** odyssey4me has joined #openstack-swift | 07:08 | |
*** saju_m has joined #openstack-swift | 07:12 | |
*** gvernik_ has joined #openstack-swift | 07:27 | |
openstackgerrit | Gil Vernik proposed a change to openstack/swift: On demand data migration for Swift https://review.openstack.org/64430 | 07:28 |
*** bvandenh has joined #openstack-swift | 08:04 | |
*** mmcardle has joined #openstack-swift | 08:06 | |
*** mfactor has joined #openstack-swift | 08:09 | |
*** mmcardle has quit IRC | 08:11 | |
*** xga has joined #openstack-swift | 08:11 | |
*** gvernik_ has quit IRC | 08:12 | |
*** mmcardle has joined #openstack-swift | 08:30 | |
*** fbo is now known as fbo_away | 08:32 | |
openstackgerrit | Sascha Peilicke proposed a change to openstack/swift-bench: Sync with global requirements https://review.openstack.org/67064 | 08:43 |
*** cschwede has joined #openstack-swift | 08:49 | |
openstackgerrit | Steve Kowalik proposed a change to openstack/swift: Drop nose, switching to testtools and testr https://review.openstack.org/65014 | 08:58 |
*** gvernik_ has joined #openstack-swift | 09:04 | |
*** ctennis has quit IRC | 09:07 | |
*** ctennis has joined #openstack-swift | 09:07 | |
*** xga has quit IRC | 09:07 | |
*** foexle has joined #openstack-swift | 09:09 | |
*** flwang has joined #openstack-swift | 09:11 | |
flwang | hi guys, a question about a commit of swift client, see https://github.com/openstack/python-swiftclient/commit/d34f12614820416aebe5e91eab9726b556d1f973 | 09:11 |
flwang | I'd like to confirm if it's only existed since version 1.6, thanks | 09:11 |
flwang | in glance, a recent patch is using the parameter 'headers' of get_object, but as you may know, the requirements is requiring version 1.5, the weird thing is we didn't run into any gate issue. Though we're running the issue in internal build server. so I'd like to confirm with you guys about my above question, thanks a lot. | 09:15 |
*** xga has joined #openstack-swift | 09:20 | |
nacim | Hi everybody, a question about swift s3 plug-in status, | 09:20 |
nacim | is there any plan for a new release ? last one has more than one year, and later commits fixes many serious bug (regarding url canonicalization for example) ? | 09:21 |
*** fbo_away is now known as fbo | 09:22 | |
flwang | anybody can help confirm my above question? thank you so much | 09:24 |
*** mlipchuk has joined #openstack-swift | 09:25 | |
*** gvernik_ has quit IRC | 09:25 | |
*** nshaikh has left #openstack-swift | 09:29 | |
*** xga has quit IRC | 10:08 | |
*** xga has joined #openstack-swift | 10:19 | |
*** xga_ has joined #openstack-swift | 10:23 | |
*** xga has quit IRC | 10:24 | |
*** mkollaro has quit IRC | 10:42 | |
openstackgerrit | A change was merged to openstack/swift: Documentation of the logs formats used in Swift https://review.openstack.org/64343 | 10:47 |
openstackgerrit | A change was merged to openstack/swift: Add missing sample config of object-replicator https://review.openstack.org/69301 | 10:47 |
*** russellb has joined #openstack-swift | 11:00 | |
*** mef has joined #openstack-swift | 11:01 | |
*** mfactor has quit IRC | 11:03 | |
*** madhuri has quit IRC | 11:23 | |
*** nshaikh has joined #openstack-swift | 11:24 | |
*** xga_ has quit IRC | 11:24 | |
*** nosnos_ has quit IRC | 11:30 | |
openstackgerrit | Fabien Boucher proposed a change to openstack/swift: Handle COPY verb in container quota middleware https://review.openstack.org/61181 | 11:31 |
*** dmorita has quit IRC | 11:52 | |
*** mfactor_ has joined #openstack-swift | 12:03 | |
*** mef has quit IRC | 12:07 | |
*** bada_ has joined #openstack-swift | 12:21 | |
*** bada has quit IRC | 12:25 | |
*** xga has joined #openstack-swift | 12:26 | |
*** mmcardle has quit IRC | 12:35 | |
*** matsuhashi has quit IRC | 12:36 | |
*** matsuhashi has joined #openstack-swift | 12:36 | |
*** matsuhashi has quit IRC | 12:41 | |
*** Trixboxer has joined #openstack-swift | 12:49 | |
*** byeager has joined #openstack-swift | 12:51 | |
*** mmcardle has joined #openstack-swift | 13:02 | |
openstackgerrit | Christian Schwede proposed a change to openstack/swift: Add containeralias middleware https://review.openstack.org/62494 | 13:06 |
*** byeager has quit IRC | 13:19 | |
openstackgerrit | Prashanth Pai proposed a change to openstack/swift: Move common expirer config params to swift.conf https://review.openstack.org/70978 | 13:22 |
*** annegentle has joined #openstack-swift | 13:27 | |
openstackgerrit | Christian Schwede proposed a change to openstack/python-swiftclient: Add option to skip downloading/uploading identical files https://review.openstack.org/65470 | 13:29 |
openstackgerrit | Christian Schwede proposed a change to openstack/python-swiftclient: Add option to skip downloading/uploading identical files https://review.openstack.org/65470 | 13:36 |
*** saju_m has quit IRC | 13:41 | |
*** nacim has quit IRC | 13:44 | |
*** ekarlso- has quit IRC | 13:45 | |
*** ekarlso has joined #openstack-swift | 13:45 | |
*** nacim has joined #openstack-swift | 13:50 | |
*** xga has quit IRC | 14:03 | |
*** xga has joined #openstack-swift | 14:05 | |
*** tongli has joined #openstack-swift | 14:07 | |
*** ppai has quit IRC | 14:11 | |
*** mkollaro has joined #openstack-swift | 14:18 | |
*** glange___ is now known as glange | 14:21 | |
chmouel | fyi about asyncio/python2 http://article.gmane.org/gmane.comp.cloud.openstack.devel/15807 /cc creiht | 14:41 |
*** flwang has left #openstack-swift | 14:49 | |
*** nshaikh has left #openstack-swift | 15:03 | |
*** jasondotstar has joined #openstack-swift | 15:04 | |
openstackgerrit | paul luse proposed a change to openstack/swift: Add storage policy support for the Replicator https://review.openstack.org/52194 | 15:06 |
cschwede | notmyname: swifterdarrell: torgomatic: chmouel: thanks to all of you for the swift-recon-test review! wanted to discuss this with you in tomorrows meeting, but you were faster :) i will write more tests for the bin-scripts using this way | 15:24 |
notmyname | cschwede: ya, swifterdarrell and torgomatic set me straight :-) | 15:25 |
cschwede | notmyname: i wasn't 100% sure about this, but looks like you all agree so I'm happy :) this makes it much easier for testing. I tried to do some tricks to import from bin, but that didn't feel very pythonic... | 15:27 |
notmyname | cschwede: ya. everyone wants the existing bin scripts to have better testing. the question was pretty much around where the boilerplate for creating the actual executable script is | 15:27 |
*** hurricanerix has joined #openstack-swift | 15:28 | |
notmyname | cschwede: I was arguing for using the same pattern that eg swift-object-server has: boilerplate in a file in bin. swifterdarrell and torgomatic (and your patch) were arguing for doing that in setup.cfg | 15:28 |
cschwede | notmyname: the only thing which might be worth for a discussion in the meeting tomorrow is if i should submit a patch for moving all bin/* to swift.cli at once, which is then very easy to review | 15:29 |
notmyname | cschwede: ya, that would be a good question to bring up | 15:29 |
cschwede | notmyname: the other way would be to just move each script when submitting a test for that script | 15:29 |
*** peluse has joined #openstack-swift | 15:30 | |
cschwede | notmyname: ah yes, boilerplating would have been the other way to do this. but both ways need to move most of the code from bin/ to swift.cli (or somewhere else) and i wasn't sure about that | 15:31 |
notmyname | right. none of us were arguing about that. the majority of the code (ie the full functionality) needs to be moved out of bin. the only question was how to do that | 15:33 |
notmyname | cschwede: and you can see from my transcript that torgomatic and swifterdarrell tested it out on a couple of different configurations (eg bare precise install, sdist, etc) to see how it worked | 15:34 |
cschwede | notmyname: yes, posting the irc logs was very interesting, thanks for that! | 15:35 |
openstackgerrit | paul luse proposed a change to openstack/swift: Add Storage Policy Support to ssync https://review.openstack.org/65347 | 15:35 |
notmyname | unfortunately al that happened in our private IRC channel (which is why you didn't see it in here). we were halfway through it before anyone realized we were having a good discussion that should probably be in here. so we kept it in there, and I just copied the transcript | 15:36 |
notmyname | cschwede: anyway, as chmouel said, it's probably good practice to include those sort of transcripts in gerrit when possible | 15:37 |
openstackgerrit | paul luse proposed a change to openstack/swift: Add Storage Policy Support to list_endpoints https://review.openstack.org/65517 | 15:38 |
chmouel | notmyname: or a link to the paste that include it :) since mister gerrit and madame formatting are not best buddies | 15:38 |
notmyname | heh | 15:38 |
cschwede | notmyname: ah, ok - unfortunately my irc bouncer just died yesterday evening, and i only noticed it this morning. couldn't see it even if the discussion would have been here ;-) | 15:39 |
notmyname | ah yes. seems that freenode itself was having problems yesterday | 15:39 |
notmyname | anyway, thanks for the patch. better testing == better swift | 15:40 |
notmyname | gotta step out for a bit. I'll be around later | 15:40 |
*** zanc has quit IRC | 15:43 | |
*** byeager has joined #openstack-swift | 15:47 | |
openstackgerrit | Eamonn O'Toole proposed a change to openstack/swift: Parallel object auditor https://review.openstack.org/59778 | 15:49 |
openstackgerrit | paul luse proposed a change to openstack/swift: Add Storage Policy Support for /Info https://review.openstack.org/69700 | 16:06 |
*** mfactor has joined #openstack-swift | 16:14 | |
*** mfactor_ has quit IRC | 16:15 | |
*** xga has quit IRC | 16:20 | |
*** byeager has quit IRC | 16:20 | |
openstackgerrit | paul luse proposed a change to openstack/swift: Replace Policy Index with Policy Name in Response Headers https://review.openstack.org/70265 | 16:22 |
*** zaitcev has joined #openstack-swift | 16:39 | |
*** ChanServ sets mode: +v zaitcev | 16:39 | |
*** tnewsome has quit IRC | 16:44 | |
*** mef has joined #openstack-swift | 17:00 | |
*** mfactor has quit IRC | 17:04 | |
*** bvandenh has quit IRC | 17:06 | |
*** gyee has joined #openstack-swift | 17:10 | |
*** gyee has quit IRC | 17:11 | |
*** mlipchuk has quit IRC | 17:12 | |
*** gyee has joined #openstack-swift | 17:13 | |
*** tnewsome has joined #openstack-swift | 17:20 | |
*** peluse has quit IRC | 17:22 | |
*** peluse has joined #openstack-swift | 17:23 | |
*** jasondotstar has quit IRC | 17:25 | |
*** jasondotstar has joined #openstack-swift | 17:44 | |
*** fbo has quit IRC | 17:46 | |
*** dfg has quit IRC | 17:46 | |
*** creiht has quit IRC | 17:46 | |
*** creiht has joined #openstack-swift | 17:47 | |
*** ChanServ sets mode: +v creiht | 17:47 | |
*** dfg has joined #openstack-swift | 17:48 | |
*** peluse has quit IRC | 17:54 | |
*** peluse has joined #openstack-swift | 17:55 | |
*** fbo has joined #openstack-swift | 17:55 | |
*** byeager has joined #openstack-swift | 17:58 | |
*** bsdkurt1 has quit IRC | 18:01 | |
*** mfactor_ has joined #openstack-swift | 18:03 | |
*** nacim has quit IRC | 18:06 | |
*** mef has quit IRC | 18:06 | |
*** shri has joined #openstack-swift | 18:09 | |
*** byeager has quit IRC | 18:12 | |
*** pberis has joined #openstack-swift | 18:18 | |
openstackgerrit | Pete Zaitcev proposed a change to openstack/swift: Fix numbering of steps in proxy section of install_multinode https://review.openstack.org/71051 | 18:22 |
*** mmcardle has quit IRC | 18:41 | |
*** byeager has joined #openstack-swift | 18:43 | |
*** mfactor_ has quit IRC | 18:47 | |
*** thebloggu has joined #openstack-swift | 18:48 | |
*** Diddi has joined #openstack-swift | 18:48 | |
thebloggu | i have done a careless reboot of all my servers and now my swift cluster is giving me errors like 404 when i try to create a new container. can someone guide me? | 18:51 |
*** byeager has quit IRC | 18:51 | |
*** pheadron has joined #openstack-swift | 18:51 | |
notmyname | thebloggu: I'm hoping you mounted your drives via drive id (or something else stable across reboots). it may be that the drives were auto-mounted at boot in a different order (thus sdb is not sde or something) | 18:52 |
thebloggu | notmyname: no, i believe the drives are fine | 18:52 |
thebloggu | notmyname: problem is, for example, i try to create a new container and fives an error saying it couldn't create it, but it does | 18:53 |
notmyname | thebloggu: hmm.. | 18:54 |
thebloggu | notmyname: and since i don't know even where to start troubleshooting, am asking here for help | 18:54 |
shri | thebloggu: You can look at syslog on the servers for hints | 18:54 |
notmyname | thebloggu: first look in the proxy logs to see what's happening there. you may see log messages about difficulty communicating with the backend servers. | 18:54 |
thebloggu | shri: i'm using fedora so i believe they'll be in /var/log/messages? | 18:55 |
shri | yeah | 18:57 |
thebloggu | i see a lot of openstack2 object-replicator @ERROR: failed to open lock file | 18:57 |
thebloggu | and openstack2 object-replicator Bad rsync return code: 5 | 18:57 |
zaitcev | thebloggu: yes, if you didn't edit rsyslog.conf, then everything localN ends in /var/log/messages. | 18:58 |
*** byeager has joined #openstack-swift | 19:02 | |
thebloggu | notmyname, shri, zaitcev: i see lot of Bad rsync return code: 5 | 19:02 |
*** hugokuo has quit IRC | 19:02 | |
*** swifterdarrell has quit IRC | 19:03 | |
zaitcev | thebloggu: just look into your /srv/node and verify that you mounted the volumes. I know you said "i believe the drives are fine" but they probably are not. | 19:03 |
*** amandap has quit IRC | 19:03 | |
zaitcev | thebloggu: I presume you already set SElinux to permissive in /etc/sysconfig/selinux. | 19:04 |
*** mlanner has quit IRC | 19:04 | |
thebloggu | zaitcev: not really. on the proxy server or all of them? | 19:04 |
zaitcev | thebloggu: wherever you get the error. Proxy does not typically have volumes of its own, unless you combine functions. | 19:05 |
*** mlanner has joined #openstack-swift | 19:05 | |
thebloggu | zaitcev: yes, the proxy node has a storage node too | 19:06 |
*** swifterdarrell has joined #openstack-swift | 19:06 | |
*** ChanServ sets mode: +v swifterdarrell | 19:06 | |
*** hugokuo has joined #openstack-swift | 19:06 | |
*** amandap has joined #openstack-swift | 19:07 | |
thebloggu | zaitcev: it seems it was the selinux | 19:09 |
thebloggu | zaitcev: set enforce 0 on all my machines got it to work | 19:10 |
thebloggu | zaitcev: and yes, i checked and the drives are mounted | 19:10 |
zaitcev | thebloggu: well, don't forget to make it permanent in /etc/sysconfig/selinux then | 19:10 |
*** jasondotstar has quit IRC | 19:27 | |
*** bsdkurt has joined #openstack-swift | 19:28 | |
*** Trixboxer has quit IRC | 19:32 | |
*** Trixboxer has joined #openstack-swift | 19:32 | |
thebloggu | zaitcev: already did, thank you very much for the reminder | 19:33 |
thebloggu | notmyname, shri, zaitcev: thank you for the help! | 19:33 |
*** thebloggu has quit IRC | 19:34 | |
*** pheadron has quit IRC | 19:44 | |
*** shri has quit IRC | 19:48 | |
*** fbo is now known as fbo_away | 19:52 | |
*** bsdkurt has quit IRC | 20:02 | |
*** jasondotstar has joined #openstack-swift | 20:09 | |
*** bsdkurt has joined #openstack-swift | 20:10 | |
*** mlipchuk has joined #openstack-swift | 20:10 | |
*** foexle has quit IRC | 20:13 | |
*** odyssey4me has quit IRC | 20:18 | |
*** kragniz has quit IRC | 20:24 | |
openstackgerrit | A change was merged to openstack/swift: Sanify handoff search depth with non-integer replica counts https://review.openstack.org/63298 | 20:25 |
openstackgerrit | A change was merged to openstack/swift: Fix up double mocking of swob.Request.__init__ https://review.openstack.org/70855 | 20:25 |
*** mlipchuk has quit IRC | 20:30 | |
openstackgerrit | A change was merged to openstack/swift: Fix numbering of steps in proxy section of install_multinode https://review.openstack.org/71051 | 21:01 |
*** kragniz has joined #openstack-swift | 21:02 | |
notmyname | chmouel: serious questions (because you're the non-native English speaker I know the best in here). are translated logs considered a good thing? seems that would make it terribly difficult to find things online when you are hunting down a problem | 21:16 |
openstackgerrit | Christian Schwede proposed a change to openstack/swift: WIP - Add tests for swift-recon-builder https://review.openstack.org/71106 | 21:30 |
chmouel | notmyname: i never configured any of my system in french including phone/tv or other stuff | 21:32 |
chmouel | most french people I know are against it | 21:32 |
chmouel | i think it's just some gov requirement | 21:32 |
notmyname | it seems to be important to some people on the mailing list and it was mentioned in the team meeting just now | 21:33 |
notmyname | and the proposed "you can't google for swahili error messages" is "we'll log it in both languages" | 21:34 |
notmyname | chmouel: thanks | 21:35 |
*** pscheie has joined #openstack-swift | 21:40 | |
pscheie | Trying to run 'swift list --insecure' but it's failing, saying 'OpenSSL.SSL.SysCallError: (9, 'Bad file descriptor')' | 21:44 |
pscheie | Apparently, someone else had a similar problem on Jan. 3: http://paste.openstack.org/show/59792/ | 21:45 |
pscheie | I'm running version 1.8.0 | 21:46 |
pscheie | Any ideas as to the cause? | 21:46 |
pscheie | My colleagues tell me it was working just the other day. | 21:46 |
*** gyee has quit IRC | 21:50 | |
torgomatic | zaitcev: looks like your stack trace there; you have any thoughts on this? ^^ | 21:56 |
zaitcev | torgomatic: looking... | 21:56 |
zaitcev | pscheie, torgomatic: That trace is not supposed to be happening in current swiftclient. I hit that trace while running tests for the "Verify SSL cert" thing which is now in development. But I suppose if you hit it on stock swiftclient it makes Tristan's burden lighter. | 21:59 |
*** jasondotstar has quit IRC | 22:05 | |
*** pheadron has joined #openstack-swift | 22:13 | |
*** shri has joined #openstack-swift | 22:36 | |
*** Trixboxer has quit IRC | 22:37 | |
*** pheadron has quit IRC | 22:46 | |
*** bsdkurt has quit IRC | 22:57 | |
*** mdonohoe has joined #openstack-swift | 23:25 | |
*** ChanServ sets mode: +v dfg | 23:26 | |
*** markd has quit IRC | 23:29 | |
*** tongli has quit IRC | 23:30 | |
*** byeager has quit IRC | 23:35 | |
*** byeager has joined #openstack-swift | 23:36 | |
*** byeager has quit IRC | 23:45 | |
*** themadcanudist has joined #openstack-swift | 23:51 | |
themadcanudist | guys, on a really busy swift server, why would keystone be pegged at 100%, running an strace i see most calls are on read/write.. (i'm using memcache as well) I would imagine this *shouldn't* be the case | 23:52 |
themadcanudist | swift cluster , not server. | 23:52 |
themadcanudist | 50.91 0.004397 0 19822 read | 23:53 |
themadcanudist | 28.28 0.002442 0 19416 write | 23:53 |
portante | themadcanudist: the keystone daemon process? | 23:53 |
notmyname | the keystone process is pegged? the swift processes are not? | 23:53 |
themadcanudist | yeah | 23:53 |
themadcanudist | they are *busy*, but keystone is a bottleneck | 23:53 |
portante | :) | 23:53 |
themadcanudist | turning logging off helped a LOT, but it's still a bottle neck | 23:53 |
themadcanudist | is that "normal"? | 23:53 |
portante | is it backed by mysql? | 23:53 |
notmyname | probably | 23:53 |
themadcanudist | portante: yes | 23:54 |
portante | what version, master, havana? | 23:54 |
portante | what does mytop report? | 23:54 |
portante | what is the traffic like to your memcache servers? | 23:54 |
themadcanudist | i have mysql query caching on… it's putting 0 load on server | 23:54 |
themadcanudist | mysql is not the problem | 23:55 |
portante | good | 23:55 |
themadcanudist | literally it's sitting idle mostly | 23:55 |
themadcanudist | sec re: version/memcache | 23:55 |
portante | k | 23:55 |
portante | and what OS version? | 23:55 |
*** hurricanerix has quit IRC | 23:55 | |
themadcanudist | linux kernel 2.6.32-279.14.1.el6.x86_64 / centos 6.3 | 23:56 |
themadcanudist | keystone 2012.2.1 | 23:56 |
themadcanudist | memcache is also mostly idle | 23:57 |
portante | so that would be folsom? | 23:57 |
themadcanudist | i'll get some stats | 23:57 |
portante | themadcanudist: k | 23:57 |
portante | can you strace the keystone process to see what it is doing? | 23:57 |
notmyname | hmm..folsom would be bad. it wasn't until grizzly that keystone could use swift's memcache ring | 23:57 |
portante | notmyname: yes, that is what I am hoping it is not | 23:57 |
themadcanudist | =( | 23:58 |
portante | =) | 23:58 |
themadcanudist | portante: i did strace -c -p to profile it, that's what i pasted… you want to know what it's read() write() 'ing? | 23:58 |
notmyname | themadcanudist: and do you know what version of swift you're using? | 23:58 |
portante | pasted? | 23:58 |
portante | ah, above I see | 23:58 |
* themadcanudist nods - swift 1.7.5 | 23:59 | |
themadcanudist | which memcache stat would you want to look at? | 23:59 |
themadcanudist | i have em all ;) | 23:59 |
themadcanudist | it looks sane | 23:59 |
portante | how many proxy nodes and where is this keystone server in relation to the proxy nodes? | 23:59 |
portante | that is folsom | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!