Friday, 2014-03-28

*** amandap has quit IRC00:00
*** saschpe has quit IRC00:00
*** amandap has joined #openstack-swift00:01
*** zaitcev has quit IRC00:01
*** saschpe has joined #openstack-swift00:06
*** d89 has joined #openstack-swift00:28
*** matsuhashi has joined #openstack-swift00:31
*** shri has quit IRC00:41
*** sunh has quit IRC00:46
*** _sEBAs_ has quit IRC00:59
*** judd7 has quit IRC01:03
*** RockKuo has joined #openstack-swift01:14
*** RockKuo_ has joined #openstack-swift01:16
*** RockKuo has quit IRC01:16
*** SpamapS has joined #openstack-swift01:19
SpamapSFYI, http://git.openstack.org/cgit/openstack/swift/commit/?id=4b075c107201e3064418d3e5dc41f0ec5ca02e7d has broken TripleO ..01:19
SpamapStempfile.NamedTemporaryFile creates files with a stricter umask01:19
SpamapSmaking the ring file 0600 instead of 064401:19
SpamapSdevstack is not affected because of https://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/swift#n285 .. which is a pretty big hammer.01:20
SpamapSWe'll adapt, but just FYI.. users may experience issues.01:20
*** piousbox has quit IRC01:29
*** nosnos has joined #openstack-swift01:37
*** csd_ has joined #openstack-swift01:38
gholtpandemicsyn: ^01:44
*** d89 has quit IRC01:44
*** tdasilva has left #openstack-swift01:45
*** godb has joined #openstack-swift01:56
*** d89 has joined #openstack-swift02:02
*** h6w has quit IRC02:04
*** csd_ has quit IRC02:05
hugokuogodb: HI02:16
*** saschpe has quit IRC02:24
*** saschpe has joined #openstack-swift02:24
*** RockKuo_ has quit IRC02:38
*** matsuhashi has quit IRC03:04
*** occup4nt has quit IRC03:08
*** matsuhashi has joined #openstack-swift03:10
*** nosnos has quit IRC03:17
*** changbl has joined #openstack-swift03:19
*** csd has joined #openstack-swift03:21
*** occupant has joined #openstack-swift03:29
*** matsuhashi has quit IRC03:31
*** RockKuo has joined #openstack-swift03:41
openstackgerritYuan Zhou proposed a change to openstack/swift: Update swift-object-info/swift-get-nodes to be storage policy aware  https://review.openstack.org/8273403:50
*** matsuhashi has joined #openstack-swift03:53
openstackgerritPeter Portante proposed a change to openstack/swift: Mock out time and sleep to avoid races  https://review.openstack.org/8327703:57
*** nosnos has joined #openstack-swift04:06
*** chandankumar_ has joined #openstack-swift04:23
*** jokke_ has quit IRC04:30
*** jokke_ has joined #openstack-swift04:30
*** ppai has joined #openstack-swift04:47
*** sungju has joined #openstack-swift04:57
*** sungju has quit IRC05:07
*** gyee has quit IRC05:33
*** sungju has joined #openstack-swift06:29
*** psharma has joined #openstack-swift06:30
ppaiHi, does anyone know how to figure out which branches were not covered in nosetests when "BrMiss" is not zero.06:32
*** saju_m has joined #openstack-swift06:38
*** csd has quit IRC06:52
*** bvandenh has joined #openstack-swift07:07
*** matsuhashi has quit IRC07:20
godbhugokuo : hi :)07:22
*** matsuhas_ has joined #openstack-swift07:24
godbhugokuo : if you here. please call me. i have some question about swift handoff07:24
hugokuogodb: am back07:25
godbhi :)07:26
hugokuohi07:26
hugokuoHad your .ts been removed ?07:26
godbtoday, i try to test about upload object / expired object / check .ts file and dir07:27
godbno, net yet :(07:27
godbmy swift suffer from many .ts file.. it makes much disk inode07:27
hugokuogodb: it's normal. unless the partition content changed.07:28
hugokuogodb: If the partition been reused, then the .ts will be removed.07:28
godbhow can i reuse partition??07:29
hugokuogodb: The inode consumption question is worth to discuss here.07:29
godbhum....07:30
hugokuogodb: once a new object been calculated to store on the same partition. That's swift's work... not yours07:30
hugokuogodb: But I recognize the inode consumption by .ts. I'd like to ask the logic with other. Maybe chmouel or notmyname.07:31
godbthe partition have many objects but replicator never delete .ts file07:31
hugokuogodb: hmm... that's fun. give it a test to delete the hashed.pkl in a test partition directory.07:32
godball replication files?07:32
*** haomaiw__ has quit IRC07:33
hugokuoAfter that, all replicas of the .ts will be flushed .07:33
* hugokuo I tested it yesterday.07:33
godbok. i'll try it.07:33
godbthat's good. but i will test on my hand :)07:34
hugokuogodb: so what can I help for the handoff question ?07:34
godband .. i have some question about handoff07:34
godbyeah:)07:34
godbupload object -> expire object -> delete .data file and rename to .ts file07:35
godband time is running out  (reclaim_age)07:36
godbthen, swift makes another directory07:37
hugokuoyes07:37
godbthe dir consist .ts file ;(07:37
godbbut .ts's timestamp is different07:38
godbi wonder why swift make new directory about expire object after reclaim_age07:38
hugokuothinking ... never deep dive the behavior of object-expire.07:39
godbis it object handoff? i don't know why do that. for what?07:39
godbobject-expirer demon delete .data file and change file name to timestamp.ts  but i didnt know why make another dir and another .ts file07:40
godbdo you know any doc about it ?07:41
hugokuoI guess the developer document or source code would be the best place.07:41
hugokuoFor this question, I'm learning from code now.07:42
godbyes.. so i read source file a little.. replicator.py. diskfile.py.07:42
godbin the ../swift/obj/diskfile.py has logic about delete .ts file07:43
godbi will read consistently07:43
godbconsistently07:45
godband when expired object sqlite3 object table do change 'delete=1'07:46
*** cheri has joined #openstack-swift07:47
godbbut in my think. it is needless row07:47
godbthis row will be delete? when? who?07:47
* hugokuo testing 07:49
hugokuogodb: I did not see "swift makes another directory" tho.  Can your explain more ?07:59
hugokuoit looks normal for me. The 1395993134.85122.data turn into 1395993354.43130.ts , the timestamp of .ts is the timing when the object been deleted at. But still in the same directory.08:02
hugokuoThere's no additional directory created here ...08:03
godbhugokuo :  what is "reclaim_age" in your object-server.conf08:23
godbdefalut is 1 week.08:23
hugokuoreclaim_age = 3008:23
godbokey. after file rename to .ts08:24
hugokuoI test with expirer after 30secs.08:24
hugokuok08:24
godbk08:24
hugokuoWhat's ur case08:24
*** Manish_ has joined #openstack-swift08:24
godbno more add dir??08:24
hugokuoshow me your's08:24
Manish_hugokuo: Hi..there08:25
hugokuoprolly paste it on pastebin08:25
hugokuoManish_: yes... but about to grab a coffee08:25
Manish_ok..:)08:25
openstackgerritVictor Stinner proposed a change to openstack/python-swiftclient: Python 3: Add six dependency  https://review.openstack.org/8368308:25
Manish_Just one quick query..08:25
godbyeah.. give me little time.. i will delete all partition and test it08:25
Manish_if u have 2 minutes before coffee...:)08:26
hugokuoManish_: k08:26
Manish_In case of Object versioning..if i have a object by Name "Manish"....08:28
godbhugokuo : sorry, please give a pastebin url?08:28
godbhere is not my desktop08:28
hugokuogodb: http://paste.openstack.org/08:28
Manish_if a new version comes, it will be ranemd to "006Manish/timestamp" and will be moved to Archive folder08:28
godbtk.08:28
Manish_right?08:28
Manish_Now can some one delete directly "006Manish/timestamp" file from archive folder?08:29
hugokuoManish_: yes... the versioned object can be deleted directly by pointing the DELETE against it.08:30
openstackgerritClay Gerrard proposed a change to openstack/swift: Add object-reconciler daemon  https://review.openstack.org/8073008:30
Manish_hugokuo: Thanks alot...08:31
openstackgerritPrashanth Pai proposed a change to openstack/swift: Read-Only Middleware  https://review.openstack.org/8368708:34
*** Longgeek_ has joined #openstack-swift08:38
*** haypo has joined #openstack-swift08:43
haypohi08:43
haypoi sent patches for swiftclient to port it to Python 3. chmouel just approved 4 patches. 2 remainings patches requires https://review.openstack.org/#/c/83683/ which adds six dependency08:47
hugokuogodb: I may understand what you mean now http://paste.openstack.org/show/74526/08:48
hayposix is already used by almost all OpenStack projects and it is already required in test-requirements.txt of swiftclient08:48
haypomy goal is to fix all Python 3 issues in swiftclient and then make the py33 gate voting to not regress08:50
*** matsuhas_ has quit IRC08:53
*** matsuhas_ has joined #openstack-swift08:57
*** psharma has quit IRC08:58
*** psharma has joined #openstack-swift08:58
*** saju_m has quit IRC09:05
*** mmcardle has joined #openstack-swift09:06
godbhugokuo : yes. buf after reclaim age time09:11
godbswift make another dir09:11
godbi will check now09:12
*** nacim has joined #openstack-swift09:12
hugokuohmm.. I tried the blueprint description of expirer for more details. But can't find it :(09:13
godbhugokuo :  i heve same test result09:16
godbin my think..09:16
godbnono,,09:16
godbi don'n understand why swift make .ts file and different "Ring Locations"09:17
godbanyway thx your help. i need more testing09:17
godbif i know this reason. tell you :)09:18
hugokuogodb: got it.09:19
godbhugokuo : good day. good week. i need some beer :(09:22
hugokuogodb: r u operator or developer ?09:24
godbi am developer09:24
godblive in south korea09:24
*** saju_m has joined #openstack-swift09:25
godbi develop c/c++ server side system mostly.09:25
godbdo you?09:25
godbin my memory you live in singapore right?09:26
hugokuocool.... Am much more works as an operator. Live in Taiwan but work in SwiftStack09:26
hugokuoThat's nice to have asian people around this timezone XD09:26
godboh. sorry. :) and that is very good luck to me09:27
godbbecause of  talking with you.09:27
godbno one asked around me.09:28
godband my english skills short ;09:28
hugokuoWell, it's good. I can have more experience with any kind of questions about Swift. Sometimes I know the behavior of a feature but not deep dive in the code. While someone has question about Swift, that means the question prolly asked by our customers in the future XD09:29
hugokuono problem.... I think my english is suck...09:30
hugokuohaha09:30
godbthx kindly.09:30
hugokuoas this case, I learned much more of how expirer working in the underlying.  So feel free to ping me on IRC. We can discuss thing here. Of course, I may not know everything in details.09:32
godbi considering make external script for delete .ts file and directory. but before that . i will more stduy and test about swift operation09:32
*** psharma has quit IRC09:33
godbsounds good .. i  if anyone give question if i knows .. as possible as answer it. if he understand my engilist :(09:34
godbenglish09:34
*** bada has joined #openstack-swift09:35
*** godb has quit IRC09:35
*** ahale has quit IRC09:39
hugokuoI have same confusion about .ts occupied inodes forever if the partition hasn't been used by new object.09:41
* hugokuo feel not make sense. Maybe there's some reason for such implementation. 09:42
*** jamie_h_ has joined #openstack-swift09:47
*** Trixboxer has joined #openstack-swift09:53
*** Longgeek_ has quit IRC09:54
openstackgerritA change was merged to openstack/python-swiftclient: Replace dict.iteritems() with dict.items()  https://review.openstack.org/8255709:55
*** Manish_ has quit IRC09:55
openstackgerritA change was merged to openstack/python-swiftclient: Python 3: Replace iter.next() with six.next(iter)  https://review.openstack.org/8255509:59
*** matsuhas_ has quit IRC10:00
*** Midnightmyth has joined #openstack-swift10:03
*** Longgeek_ has joined #openstack-swift10:12
openstackgerritA change was merged to openstack/python-swiftclient: Python 3: Fix tests using temporary text files  https://review.openstack.org/8255410:14
openstackgerritA change was merged to openstack/python-swiftclient: Python 3: cast map() result to list  https://review.openstack.org/8255310:14
*** nosnos has quit IRC10:16
*** Longgeek_ has quit IRC10:46
*** Longgeek_ has joined #openstack-swift10:57
*** Longgeek_ has joined #openstack-swift10:58
*** d89 has quit IRC10:59
*** RockKuo has quit IRC11:05
*** RockKuo_iPad has joined #openstack-swift11:05
*** Longgeek_ has quit IRC11:07
*** lpabon has joined #openstack-swift11:07
*** Longgeek_ has joined #openstack-swift11:11
*** RockKuo_iPad has quit IRC11:16
*** RockKuo_iPad has joined #openstack-swift11:16
*** RockKuo_iPad has quit IRC11:17
*** RockKuo_iPad has joined #openstack-swift11:17
*** RockKuo_iPad has quit IRC11:17
*** RockKuo_iPad has joined #openstack-swift11:18
*** RockKuo_iPad has quit IRC11:24
*** Longgeek_ has quit IRC11:27
*** RockKuo_iPad has joined #openstack-swift11:41
*** RockKuo_iPad has quit IRC11:45
*** saju_m has quit IRC11:50
*** madhuri_ has joined #openstack-swift11:53
*** RockKuo_iPad has joined #openstack-swift12:01
*** tanee-away is now known as tanee12:07
*** mkollaro has joined #openstack-swift12:07
*** JuanManuelOlle has joined #openstack-swift12:09
*** JuanManuelOlle1 has joined #openstack-swift12:28
*** JuanManuelOlle has quit IRC12:30
*** Longgeek_ has joined #openstack-swift12:31
*** madhuri_ has quit IRC12:39
*** Longgeek_ has quit IRC12:40
*** tdasilva has joined #openstack-swift12:41
*** tanee is now known as tanee-away12:41
*** erlon has joined #openstack-swift12:45
*** Longgeek_ has joined #openstack-swift12:47
*** chandan_kumar has quit IRC12:51
*** tanee-away is now known as tanee12:59
*** Midnightmyth has quit IRC13:00
*** JuanManuelOlle1 has quit IRC13:10
*** cheri has quit IRC13:11
*** ppai has quit IRC13:11
*** tanee is now known as tanee-away13:11
*** RockKuo_iPad has quit IRC13:11
*** zigo has quit IRC13:13
*** zigo has joined #openstack-swift13:14
*** mmcardle has quit IRC13:15
*** saju_m has joined #openstack-swift13:15
*** mrsnivvel has quit IRC13:17
*** JuanManuelOlle has joined #openstack-swift13:19
*** mmcardle has joined #openstack-swift13:23
*** JuanManuelOlle has quit IRC13:38
*** JuanManuelOlle has joined #openstack-swift13:47
*** zigo has quit IRC13:47
*** zigo has joined #openstack-swift13:47
*** RockKuo_iPad has joined #openstack-swift13:48
*** zigo has quit IRC13:51
*** zigo has joined #openstack-swift13:55
portanteRegarding eventlet and Python3, from Sergey Shepelev: "Hopefully we can have Eventlet working on Python3 in next few months"13:56
*** tanee-away is now known as tanee13:58
*** tanee is now known as tanee-away13:58
*** zigo has quit IRC13:59
*** RockKuo_iPad has quit IRC13:59
*** zigo has joined #openstack-swift13:59
*** nacim has quit IRC14:00
*** nacim has joined #openstack-swift14:01
*** Longgeek_ has quit IRC14:01
*** russellb is now known as rustlebee14:08
*** openstack has joined #openstack-swift14:21
*** Midnightmyth has joined #openstack-swift14:23
*** piyush1 has joined #openstack-swift14:27
*** Longgeek_ has joined #openstack-swift14:35
openstackgerritA change was merged to openstack/python-swiftclient: Python 3: Add six dependency  https://review.openstack.org/8368314:37
*** csd has joined #openstack-swift14:52
creihtportante: yeah I saw that14:54
haypoportante: nice14:57
creihtit is nice that they are making progress14:58
openstackgerritVictor Stinner proposed a change to openstack/python-swiftclient: Python 3: Fix module names in import  https://review.openstack.org/8255615:04
haypoeventlet working on python3 will reduce the pressure to port openstack to python315:05
chmouelyes that would be very nice15:06
chmouelhttps://github.com/eventlet/eventlet/issues/6#issuecomment-3891290415:07
chmouel@ewindisch please share it with OpenStack team that Eventlet has some momentum real progress into Python3 compatibility.15:07
*** JuanManuelOlle has quit IRC15:08
portantecreiht: it might be worth our time to see the details of that progress to get a sense of their trajectory15:23
*** saju_m has quit IRC15:35
openstackgerritVictor Stinner proposed a change to openstack/python-swiftclient: Python 3: Get compatible types from six  https://review.openstack.org/8255215:48
creihtfor some measure of our :)15:50
*** judd7 has joined #openstack-swift15:50
*** jogo is now known as flashgordon16:07
*** Longgeek_ has quit IRC16:08
*** changbl has quit IRC16:14
*** marcusvrn has joined #openstack-swift16:19
*** dencaval has joined #openstack-swift16:23
*** dencaval has quit IRC16:24
*** thebloggu has joined #openstack-swift16:33
*** gyee has joined #openstack-swift16:35
*** thebloggu has quit IRC16:45
*** thebloggu has joined #openstack-swift16:46
*** thebloggu has left #openstack-swift16:46
*** thebloggu has joined #openstack-swift16:46
thebloggucan someone help me understand why my requests are really slow (between 10 to 20 s) altough not failing? how can i find the problem? i have 1 proxy node and 1 storage node in the same real machine and 4 virtual machines running storage nodes) and all of them seem to be operating fine16:58
*** JuanManuelOlle has joined #openstack-swift17:02
*** therve has left #openstack-swift17:03
*** piousbox has joined #openstack-swift17:06
*** bvandenh has quit IRC17:13
*** saju_m has joined #openstack-swift17:19
*** zaitcev has joined #openstack-swift17:20
*** ChanServ sets mode: +v zaitcev17:20
*** changbl has joined #openstack-swift17:27
*** saju_m has quit IRC17:35
*** saju_m has joined #openstack-swift17:35
*** piyush1 has quit IRC17:36
openstackgerritpaul luse proposed a change to openstack/swift: Fix issue with account broker and pre-SPI databases  https://review.openstack.org/8384017:44
*** haypo has left #openstack-swift17:45
*** occupant has quit IRC17:54
*** occupant has joined #openstack-swift17:54
*** nacim has quit IRC17:54
*** krtaylor has quit IRC17:55
*** csd has quit IRC17:58
*** piyush1 has joined #openstack-swift18:01
*** mkerrin has quit IRC18:02
*** piyush2 has joined #openstack-swift18:03
*** piyush1 has quit IRC18:06
*** thebloggu has quit IRC18:12
*** thebloggu has joined #openstack-swift18:16
*** csd has joined #openstack-swift18:19
*** jamie_h_ has quit IRC18:35
*** openstackgerrit has quit IRC18:48
*** openstackgerrit has joined #openstack-swift18:48
piousboxhi guys18:49
piousboxis there v1.1 of Swift API?18:49
piousboxOr v2?18:49
notmynamepiousbox: no. just v118:56
piousboxthanx19:06
*** jamie_h_ has joined #openstack-swift19:06
notmynamepiousbox: we've added things to the API, but if you have a client written 4 years ago, it should still work with current tip of master today19:07
piousboxright on. We don't expect there to be v2 right now, but I was asked to make sure that in fact there isn't one ;)19:08
*** thebloggu has quit IRC19:10
*** mmcardle has quit IRC19:25
*** csd has quit IRC19:43
*** mkollaro has quit IRC19:49
*** lpabon has quit IRC19:53
*** krtaylor has joined #openstack-swift20:07
*** jamie_h_ has quit IRC20:09
*** csd has joined #openstack-swift20:17
*** dmsimard has joined #openstack-swift20:25
*** csd has quit IRC20:51
*** dmsimard1 has joined #openstack-swift20:53
*** dmsimard has quit IRC20:57
*** piyush2 has quit IRC21:06
*** jergerber has joined #openstack-swift21:08
*** mmcardle has joined #openstack-swift21:09
*** dmsimard1 has quit IRC21:10
*** JuanManuelOlle has quit IRC21:12
notmynamereally quiet in here today21:20
*** shri has joined #openstack-swift21:23
*** marcusvrn has quit IRC21:38
* portante just hears the crickets and croaking frogs, hoping Spring will come to New England soon21:43
*** mmcardle has quit IRC21:53
*** judd7 has quit IRC21:58
openstackgerritPeter Portante proposed a change to openstack/swift: Speed up obj auditor sorting hsh dirs by inode #  https://review.openstack.org/8159022:05
portantenotmyname: have two bug fix commits out and ready for review22:06
portantehttps://review.openstack.org/#/c/8327622:07
portantehttps://review.openstack.org/#/c/8327722:07
notmynameportante: ya, I was just going over what's outstanding when I got distracted by if or how much the proxy-logging bug is affecting our customers. doing some research on it22:07
portanteThey fix bug 129815422:08
portantenotmyname: so we think that proxy-logging fix covers both SLO and DLO, right?22:08
notmynameyes I think so. but there is still the SLO range request thing22:09
portanteI was not sure if there was a problem with DLO, had a hard time following the exchanges22:09
portantenotmyname: yes, but that will take some consideration because we don't know to log that request as a SLO manifest until we read the object itself22:09
notmynameportante: basically it comes down to filtering the subrequests (swift.source set) or not22:10
portantewe just don't know if an object is a SLO or DLO manifest until we get a hold of it22:10
portantereally, until the headers come back22:10
portanteso if SLO marked every GET that it thinks might be a SLO manifest, all GETs would end up getting logged twice at it stands today22:11
portanteif SLO is in the pipeline22:12
portantewell, that is obvious ...22:12
portanteso the best that we can do is have SLO enter a fake log request when it knows it is a manifest that is going to be fetched, at least that is what I can see so far22:13
portanteanyways, gotta go22:13
portanteback on line later22:13
notmynameportante: running my last test now. I'll paste my summary22:14
notmynameportante: looks like everything is correct in both SLO and DLO in master22:16
notmynameportante: the segments are logged with swift.source set, and there is a final proxy log line without swift.source set that has the full number of bytes22:16
notmynameswifterdarrell: 1.11.0.1 looks fine for both SLO and DLO. one log line, no subrequests, b/w is correct. in 1.13.0.1, each chunk has the bytes but the log line without swift.source set only has the bytes in the manifest22:18
*** Trixboxer has quit IRC22:19
openstackgerritJohn Dickinson proposed a change to openstack/python-swiftclient: set user-agent header  https://review.openstack.org/8391922:46
*** JuanManuelOlle has joined #openstack-swift22:53
*** jergerber has quit IRC22:57
notmynamethere are some tempest patches for swift tests referenced at the bottom of https://wiki.openstack.org/wiki/Swift/PriorityReviews now23:08
notmynamethe tempest team has asked that we take a look at them, please23:08
*** erlon has quit IRC23:22
*** gyee has quit IRC23:31
*** mmcardle has joined #openstack-swift23:35
*** mmcardle has quit IRC23:37
*** JuanManuelOlle has quit IRC23:47

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!