Monday, 2015-07-20

*** chlong has joined #openstack-swift00:19
*** hrou has joined #openstack-swift00:21
openstackgerritJanie Richling proposed openstack/swift: Encrypting/Decrypting account and container metadata  https://review.openstack.org/20345400:29
openstackgerritJanie Richling proposed openstack/swift: Encrypting/Decrypting account and container metadata  https://review.openstack.org/20345400:32
*** jrichli has quit IRC00:55
*** jrichli has joined #openstack-swift00:56
*** Kennan2 has joined #openstack-swift02:05
*** Kennan has quit IRC02:05
*** Kennan2 is now known as Kennan02:07
openstackgerritJanie Richling proposed openstack/swift: Encrypting/Decrypting account and container metadata  https://review.openstack.org/20345402:39
*** jrichli has quit IRC02:43
*** wbhuber has quit IRC03:32
*** sanchitmalhotra has joined #openstack-swift03:44
*** pbourke has quit IRC03:56
*** pbourke has joined #openstack-swift03:57
openstackgerritpaul luse proposed openstack/swift: Add ability for GET path to see/select alternate frag archs  https://review.openstack.org/20128304:06
*** cdelatte has joined #openstack-swift04:11
*** delattec has joined #openstack-swift04:11
*** sanchitmalhotra1 has joined #openstack-swift04:18
*** darrenc is now known as darrenc_afk04:20
*** sanchitmalhotra has quit IRC04:21
*** sanchitmalhotra has joined #openstack-swift04:29
*** sanchitmalhotra1 has quit IRC04:31
*** ppai has joined #openstack-swift04:34
*** darrenc_afk is now known as darrenc04:35
*** briancurtin has quit IRC04:58
*** briancurtin has joined #openstack-swift04:59
*** nottrobin has quit IRC05:02
*** nottrobin has joined #openstack-swift05:05
*** chlong has quit IRC05:27
*** Protux has quit IRC05:35
*** serverascode has quit IRC05:35
*** h00327910_ has quit IRC05:36
*** briancurtin has quit IRC05:36
*** chlong has joined #openstack-swift05:40
*** serverascode has joined #openstack-swift05:41
*** briancurtin has joined #openstack-swift05:44
*** h00327910_ has joined #openstack-swift05:48
*** Protux has joined #openstack-swift05:48
*** sanchitmalhotra1 has joined #openstack-swift06:00
*** sanchitmalhotra has quit IRC06:02
*** hrou has quit IRC06:03
*** chlong has quit IRC06:20
*** chlong has joined #openstack-swift06:36
*** sanchitmalhotra has joined #openstack-swift06:46
*** sanchitmalhotra1 has quit IRC06:48
*** jith_ has joined #openstack-swift06:59
*** ig0r_ has joined #openstack-swift07:01
*** rledisez has joined #openstack-swift07:05
*** hseipp has joined #openstack-swift07:20
*** hseipp has quit IRC07:20
*** hseipp has joined #openstack-swift07:20
*** rledisez has quit IRC07:30
*** nottrobin has quit IRC07:31
*** Protux has quit IRC07:31
*** zhiyan has quit IRC07:31
*** cppforlife_ has quit IRC07:31
*** serverascode has quit IRC07:31
*** chlong has quit IRC07:34
*** ntt has joined #openstack-swift07:41
*** pcaruana has quit IRC07:42
*** nottrobin has joined #openstack-swift07:44
*** zhiyan has joined #openstack-swift07:48
*** rledisez has joined #openstack-swift07:49
*** cppforlife_ has joined #openstack-swift07:53
*** jistr has joined #openstack-swift07:54
*** serverascode has joined #openstack-swift07:59
*** Protux has joined #openstack-swift08:02
*** silor has joined #openstack-swift08:16
*** jordanP has joined #openstack-swift08:26
*** silor has quit IRC08:29
openstackgerritOndrej Novy proposed openstack/swift: Rsync port in ring file.  https://review.openstack.org/19219808:29
openstackgerritOndrej Novy proposed openstack/swift: Time synchronization check in recon.  https://review.openstack.org/19256208:31
*** jordanP has quit IRC08:31
*** zhill has joined #openstack-swift08:38
*** pcaruana has joined #openstack-swift08:43
*** jordanP has joined #openstack-swift08:45
*** zhill has quit IRC08:54
*** ppai is now known as ppai|afk08:56
*** rledisez has quit IRC09:25
openstackgerritOndrej Novy proposed openstack/swift: Rsync port in ring file.  https://review.openstack.org/19219809:41
*** aix has quit IRC09:41
*** SkyRocknRoll has joined #openstack-swift09:42
jith_hi all.. what is swift_store_key in configuring swift for glance backend... is it password for swift user??09:44
*** SkyRocknRoll has quit IRC09:47
*** SkyRocknRoll has joined #openstack-swift10:01
*** lcurtis has joined #openstack-swift10:15
*** ppai|afk is now known as ppai10:19
*** ig0r_ has quit IRC10:24
*** lcurtis has quit IRC10:35
*** rvasilets__ has joined #openstack-swift10:36
*** lcurtis has joined #openstack-swift10:48
*** jasondotstar has joined #openstack-swift10:50
*** lcurtis has quit IRC10:54
*** ig0r_ has joined #openstack-swift11:08
*** ntt has quit IRC11:29
*** sanchitmalhotra has quit IRC11:37
*** aix has joined #openstack-swift12:00
*** ppai has quit IRC12:03
*** ig0r_ has quit IRC12:05
*** alexcampbell has joined #openstack-swift12:06
*** delatte has joined #openstack-swift12:14
*** ppai has joined #openstack-swift12:15
*** cdelatte has quit IRC12:15
*** delattec has quit IRC12:17
*** chlong has joined #openstack-swift12:34
*** hrou has joined #openstack-swift12:36
*** rvasilets__ has quit IRC12:44
*** aagrawal has joined #openstack-swift13:10
*** ppai has quit IRC13:16
*** aagrawal has quit IRC13:17
tdasilva-good morning13:25
*** annegentl_ has joined #openstack-swift13:30
*** alexcampbell has quit IRC13:33
*** janonymous has joined #openstack-swift13:34
*** killa_kyle has joined #openstack-swift13:41
*** killa_kyle has left #openstack-swift13:42
*** annegentl_ has quit IRC13:48
*** breitz has joined #openstack-swift14:05
*** annegentl_ has joined #openstack-swift14:11
*** wbhuber has joined #openstack-swift14:16
*** alexcampbell has joined #openstack-swift14:19
*** haomaiwang has quit IRC14:21
*** haomaiwang has joined #openstack-swift14:21
*** pgbridge has quit IRC14:25
*** jrichli has joined #openstack-swift14:31
*** wbhuber_ has joined #openstack-swift14:35
*** _hrou_ has joined #openstack-swift14:35
*** sc68cal_ has joined #openstack-swift14:36
*** mgriffin_ has joined #openstack-swift14:36
*** occup4nt has joined #openstack-swift14:36
*** sc68cal has quit IRC14:37
*** sc68cal_ is now known as sc68cal14:37
*** matt____ has joined #openstack-swift14:38
*** darrenc_ has joined #openstack-swift14:38
*** jordan_ has joined #openstack-swift14:38
openstackgerritHiroshi Miura proposed openstack/python-swiftclient: fix flake8 config  https://review.openstack.org/20290914:39
*** xnox_ has joined #openstack-swift14:41
*** dmsimard has quit IRC14:42
*** wasmum- has quit IRC14:42
*** sudorandom has quit IRC14:42
*** briancline has quit IRC14:42
*** jordanP has quit IRC14:42
*** xnox has quit IRC14:42
*** ujjain has quit IRC14:42
*** jamielennox has quit IRC14:42
*** mattoliverau has quit IRC14:42
*** CrackerJackMack has quit IRC14:42
*** klrmn has quit IRC14:42
*** wbhuber has quit IRC14:42
*** hrou has quit IRC14:42
*** occupant has quit IRC14:42
*** mgriffin has quit IRC14:42
*** darrenc has quit IRC14:42
*** wer has quit IRC14:42
*** sudorandom has joined #openstack-swift14:42
*** briancline has joined #openstack-swift14:42
*** CrackerJackMack has joined #openstack-swift14:42
*** wasmum has joined #openstack-swift14:42
*** xnox_ is now known as xnox14:42
*** jamielennox has joined #openstack-swift14:42
*** ujjain has joined #openstack-swift14:43
*** ujjain has joined #openstack-swift14:43
*** dmsimard has joined #openstack-swift14:43
*** wer has joined #openstack-swift14:43
*** klrmn has joined #openstack-swift14:45
*** jistr is now known as jistr|mtg15:13
*** minwoob has joined #openstack-swift15:14
*** janonymous_ has joined #openstack-swift15:16
*** gyee has joined #openstack-swift15:19
*** thebloggu has joined #openstack-swift15:22
*** jistr|mtg is now known as jistr15:24
*** proteusguy has quit IRC15:26
*** nadeem has joined #openstack-swift15:27
*** marzif has joined #openstack-swift15:35
*** tongli has joined #openstack-swift15:35
*** proteusguy has joined #openstack-swift15:39
thebloggudo I need special configurations to support server-side copy of objects with X-Copy-From in a PUT request or a COPY request? I'm getting an unauthorized 401 error everytime I try to do it (even though I can GET/POST other objects)15:42
*** mgarza_ has joined #openstack-swift15:51
thebloggu:q16:01
theblogguups, wrong window, sorry :)16:01
claygnothing in lastlog to clayg - so I think i'm just going to skip the scroll back and get the overview from notmyname16:02
*** gyee has quit IRC16:11
*** alexcampbell has quit IRC16:13
openstackgerritjanonymous proposed openstack/swift: Metaclass Python 3.x Compatibility  https://review.openstack.org/20371416:13
*** gyee has joined #openstack-swift16:15
*** jistr has quit IRC16:24
*** blmartin has joined #openstack-swift16:25
*** jordan_ has quit IRC16:32
*** SkyRocknRoll has quit IRC16:34
notmynamegood morning16:35
pelusemorning16:35
notmynameI had a good weekend at the community leadership summit. took lots of notes and now I need to parse them to see if they make any sense. but I got some inspiration, so that's good I think16:36
peluseclayg, https://review.openstack.org/#/c/201283/ is the GET alt frag index thing and the fix for the bug that was about EC overwrite (1457691).  Not done w/test code and may refactor a bit now that it works but ahve a gander when you get a chance.16:37
notmynameclayg is back!16:37
notmynameIIRC, acoles_away should be back this week too16:37
pelusealso there's some code in the ResumeGetter that I don't understand that I might have duplciated - need to talk to torgomatic16:37
peluseoh yeah, welcome back :)16:38
* peluse left glasses at home today. this should be interesting...16:39
notmynamepeluse: that's a really big deal for you.16:39
notmynamepeluse: I think you need to ask your boss for an even bigger monitor, then :-)16:39
pelusethey don't make a bigger one :)16:40
*** annegentl_ has quit IRC16:40
*** lyrrad has joined #openstack-swift16:41
openstackgerritAzhagu Selvan SP proposed openstack/swift: Minor change in development saio setup doc  https://review.openstack.org/20372616:42
claygdoes anyone know how to gerrit?  I just want to star patch 198429 so I can make sure to answer torgomatic's question today16:42
patchbotclayg: https://review.openstack.org/#/c/198429/16:42
claygpeluse: i saw that update to the bug report16:43
clayglol @ "they don't make a bigger one"16:43
peluseclayg, where did ya go anyway?16:43
claygtexas16:43
pelusearen't we going to TX in a few weeks?  :)16:43
openstackgerritAzhagu Selvan SP proposed openstack/swift: Minor change in development saio setup doc  https://review.openstack.org/20372616:48
claygwhoa - there's a lot of starred patches16:52
claygpeluse: yeah I should have just waited!16:52
notmynameclayg: I starred all the patches that are associated with an EC bug report16:54
*** occupant has joined #openstack-swift17:00
*** occupwnt has joined #openstack-swift17:01
pelusenotmyname, we're still "on track" (as much as we ever claim to be) for encryption middleware in Liberty right?  Enough to claim so in a public preso?17:01
*** occup4nt has quit IRC17:03
notmynamepeluse: that's a good question17:03
notmynameI briefly chatted with jrichli about that late last week17:03
notmynamethere's still a bunch to do17:03
notmynameI've got a list (from jrichli) of some of the big outstanding things, and I'll be going through that early this week to try to highlight some of the areas that need help17:04
* peluse bites his tongue (fingers) after almost typing the word "17:04
pelusebeta"17:04
notmynameie where people can work17:04
notmynamepeluse: my gut reaction is that I don't want to do that :-)17:05
*** occupant has quit IRC17:05
peluseindeed!17:05
peluseone beta 'something' per year should be enough :)17:07
notmynamemy top priority is getting the EC stuff finished up. I want to see the outstanding issues closed and do a swift release with them (with enough time for another release for Liberty combined release17:07
peluseagreed17:08
pelusethe patch I just posted took a really unacceptable amount of time to get working... working on slides today for all these danged upcoming conferences but will continue on that one after I'm done and then start hitting bugs...17:10
theblogguI'm getting 401 unauthorized errors when I try to copy a file server side with COPY/PUT. can someone help me?17:10
*** jordanP has joined #openstack-swift17:12
*** mahatic has joined #openstack-swift17:12
*** openstackgerrit has quit IRC17:16
*** openstackgerrit has joined #openstack-swift17:17
*** zhill has joined #openstack-swift17:21
*** zhill_desktop has joined #openstack-swift17:23
minwoobthebloggu: Are you trying to do this in the context of single-ranged read?17:30
theblogguminwoob, do you mean if the object is copied with a single range? if so, yes. I'm copying the whole object, I'm not even using range headers17:31
notmynamethebloggu: first thing to look at is if your token is still valid. if so, then you need to look at the ACLs on the source and destination containers17:32
minwoobthebloggu: Hmm then it should be fine. I was referring to the case with a single range header.17:32
thebloggunotmyname, the token is valid since I can make other GET and PUT requests. i'm copying inside the same container and i've got the default ACL's since I didn't change them (swift stat shows them empty so I believe I've got read and write permission right?)17:35
thebloggunotmyname, actually, sorry but I was wrong, I can seem to make a GET request to the object17:37
openstackgerritAzhagu Selvan SP proposed openstack/swift: Minor change in development saio setup doc  https://review.openstack.org/20372617:38
*** occupwnt is now known as occupant17:43
notmynamethebloggu: ok, so you can make a GET request to the source object. that's good. can you PUT a new object to the destination container?17:43
*** hseipp has quit IRC17:44
*** annegentl_ has joined #openstack-swift17:44
*** aix has quit IRC17:46
thebloggunotmyname, I can. just for context I'm using python-swiftclient so I believe if the token's not valid it requests another one, right?17:48
*** ccavanna has joined #openstack-swift17:55
*** annegentl_ has quit IRC17:56
*** annegentl_ has joined #openstack-swift17:56
*** pcaruana has quit IRC18:00
notmynameit should ;-)18:04
notmynamebut a new token doesn't solve ACLs issues. definitely try putting an object directly into the target container18:05
notmynameif that doesn't work, then that's likely your problem18:06
notmynameif it does work, then it's likely something more subtle. maybe related to your config18:06
ccavannaHi. I ran into a problem starting swift after executing tox. I hope anyone here knows what may be going on.18:08
ccavannaI executed tox earlier today and after that I am no longe able to start Swift.18:09
ccavannaI get "pkg_resources.DistributionNotFound: swift==2.3.1.dev33"18:09
ccavannaInterestingly, under swift/swift.egg-info/PKG-INFO I see:18:09
ccavannaVersion: 2.3.1.dev3418:10
ccavannaAnd I also noticed that /usr/local/bin/swift-init shows18:10
*** jasondotstar has quit IRC18:10
ccavanna'swift==2.3.1.dev33' as a requirement.18:10
ccavannaI noticed swift/swift.egg-info/PKG-INFO has a timestamp from today, even though I installed this a while ago.18:11
ccavannaSo something was updated, it seems.18:11
notmynameIIRC that's an issue with not having the latest python-setuptools (pip)18:12
thebloggunotmyname, it was indeed a more subtle error. I was accidentally printing a \n to a metadata header. thank you very much for your help :)18:15
notmynamethebloggu: ah. yeah, that's tricky. glad you found it!18:16
openstackgerritGreg Lange proposed openstack/swift: go:  Adds memcache ring  https://review.openstack.org/20375818:28
*** thebloggu has quit IRC18:31
*** mahatic has quit IRC18:35
claygtorgomatic: so no, patch 185679 addressed the empty-dir-post-reclaim a little differently than patch 18567918:37
patchbotclayg: https://review.openstack.org/#/c/185679/18:37
torgomaticclayg: ah, darn... I was hoping acoles_away would just tell me "yep, it's all taken care of"18:37
claygtorgomatic: honestly I had forgotten about patch 18567918:38
patchbotclayg: https://review.openstack.org/#/c/185679/18:38
claygI mean I *think* i looked at it once :\18:38
notmynamejrichli: around?18:40
jrichlinotmyname: hello18:40
jrichlireading now18:40
notmynamejrichli: I've reformatted the encryption todos email you sent me in an etherpad so we can all look at the stuff outstanding (from a higher level than trello)18:41
*** jlhinson has joined #openstack-swift18:42
jrichlinotmyname: great!  link?18:43
notmynamehttps://etherpad.openstack.org/p/swift_encryption_issues18:43
notmynamefor everyone, these are the big problems still outstanding for encryption ^^18:43
jrichlilooks great, thanks!  should I put a link to this on the encryption section of priority reviews page?18:45
tdasilva-clayg: hi! can you comment on the effect of versionining middleware to encryption?18:46
claygnotmyname: the look roughly ordered by priority to me - was that on purpose?18:46
notmynameclayg: nope18:46
notmynameI think they should be moved to a priority order, but it's roughly in the order jrichli gave them to me18:46
claygtdasilva-: versioning and copy middleware would help encryption *greatly*18:47
jrichliand I roughly gave them priority order - but not completely18:47
jrichlii put the 2 highest at the top18:47
claygtdasilva-: like as in merging version middlware moves encryption from "not going to be done by tokyo" to "probably not going to be *done, done* by tokyo"18:47
claygnotmyname: so you're saying you just do whatever jrichli tells you18:48
clayg... smart18:48
notmynamelol18:48
jrichlilol18:48
tdasilva-clayg: ok, I was reading this comment: "clay seemed to think that the versioning patch for moving to middleware would affect encryption.  I am not sure of the connection." and thought there was some issue with it...18:48
jrichlinotmyname: maybe some filtering was required ;-)18:49
jrichlibut I really do need to be educated on that18:50
claygtdasilva-: ok yeah... alright let me see if I can remember how versioned middlware + encryption works - i'm *entirely* sure that encyrption + in-proxy-versions is doomed18:50
claygyeah so in middleaware it shoudl be fine - but there's some questions with key management and how encryption is scoped - if the cluster manages keys and public objects work then encryption would be transparent to the versioned-objects-*middleware*18:51
notmynameI've put some of these topics down for this week's meeting https://wiki.openstack.org/wiki/Meetings/Swift18:51
jrichlithx!18:52
claygnotmyname: meeting looks good18:54
notmynamedon't let that stop discussing in here now :-)18:54
jrichliI am still absorbing what clayg just said.  On a different note: the copy as middleware would be really good to have - so we dont have to write a copy hook18:56
sweeperany idea why swift nodes would be reporting unmounted drives that no longer exist in the ringfile?18:57
redboReporting how?  If you mean from recon, it doesn't even look at the ring.18:58
sweeperyea recon18:58
sweeperso just commented lines in fstab will make that happen?18:59
redboIt just looks for directories under the root that aren't mounted.18:59
ccavannanotmyname: I updated python-pip with apt-get. I am assuming I will have to update my swift files as well (git checkout?)19:00
*** DericHorn-HP has joined #openstack-swift19:00
notmynameccavanna: no, you shouldn't19:01
ccavannaMMm. But there will still be this explicit dependency in swift-init (I'm still getting the same error).19:02
openstackgerritSamuel Merritt proposed openstack/swift: Rename WsgiStringIO -> WsgiBytesIO.  https://review.openstack.org/20377519:03
sweeperoic19:03
sweeperinteresting19:03
*** losingle has joined #openstack-swift19:05
*** robefran has joined #openstack-swift19:05
claygdepnding on how new your debian/ubuntu is you probably don't want the system python-pip19:06
*** janonymous_ has quit IRC19:10
ccavannaIt's part of the installation steps for SAIO, for Ubuntu 14.19:15
*** DericHorn-HP has quit IRC19:15
claygwell - then that's a doc bug - but only because pip releases a new version every other day19:17
wasmumswift-bench v2 compatable?  if so, anyone know the format for bench config file?19:18
claygwasmum: I would have swore that chmouel_ added v2 support to swift-bench19:20
wasmum+clayg: saw the issue in the tracker19:21
wasmumnot any docs on how or if the auth_version setting will allow anything other that "1"19:21
wasmumI'll keep digging19:21
claygi'm looking at ef3e55 - in the swift code base19:22
claygit ws moved into it's own repo some some of the history is in the swift tree19:22
claygyeah I see -V option?19:23
claygbut it's just a wrapper around swiftclient get_auth?19:23
wasmumah19:24
wasmumok, I'll wrap that up.  Just looking into a sall job for jenkins to run19:24
wasmumthanks, sir19:24
wasmum**small19:25
*** zaitcev has joined #openstack-swift19:25
*** ChanServ sets mode: +v zaitcev19:25
*** zhill has quit IRC19:25
claygah so client.get_auth does the ':'.split trick to tenant:username for auth v2/v319:26
*** nadeem has quit IRC19:26
claygyeah idk, that might work19:27
openstackgerritThiago da Silva proposed openstack/swift: Add container reverse listings  https://review.openstack.org/12070919:28
openstackgerritGreg Lange proposed openstack/swift: go: add ability to lock device/drive with file on file system  https://review.openstack.org/20378919:29
*** hemanthm has left #openstack-swift19:31
openstackgerritGreg Lange proposed openstack/swift: go: add ability to lock device/drive with file on file system  https://review.openstack.org/20378919:31
*** miand has joined #openstack-swift19:35
*** miand has quit IRC19:35
*** DericHorn-HP has joined #openstack-swift19:37
*** zhill has joined #openstack-swift19:38
openstackgerritJoel Wright proposed openstack/python-swiftclient: Reduce memory usage for download/delete and add --no-shuffle option to st_download  https://review.openstack.org/13033919:45
*** tdasilva- is now known as tdasilva19:53
*** DericHorn-HP has quit IRC19:59
openstackgerritClay Gerrard proposed openstack/swift: Handle removed suffix dirs the same as empty suffix dirs  https://review.openstack.org/18567920:11
claygtorgomatic: oh - it kept you as the owner - i should have put myself as second billing :P20:12
torgomaticclayg: authored, co-authored, written, produced, directed by, and starring... ME :)20:12
claygglange: I'm +1 just on the *title* of patch 20378920:12
patchbotclayg: https://review.openstack.org/#/c/203789/20:12
notmynametorgomatic: https://pbs.twimg.com/media/BhV6LA0CIAI120o.jpg20:13
claygglange: esspecially if you plan on using it to lock out PUTS on full drives ;)20:13
*** ChanServ sets mode: +v tdasilva20:14
glangewell, our ops_ guys asked for that -- sometimes they want to umount and do something with a drive but they have to catch it win it's not busy20:15
claygglange: yeah fer-real20:15
notmynameglange: clayg: did I miss the first part of that conversation?20:15
claygnotmyname: the title of patch 203789?20:16
patchbotclayg: https://review.openstack.org/#/c/203789/20:16
notmynameat the summit we had requests from ops people to be able to mark a drive as disabled20:16
notmynameah ok20:16
claygnotmyname: and it is now possible - by the power of go20:16
glangehaha20:16
glangesomebody should port that to python at the hackathon20:16
notmynamewell, all the cool stuff I've seen so far in hummingbird doesn't really have anything to do with go20:17
ccavannaNo luck yet, but I traced back what I did since I could successfully run swift for the last time. I did execute this git command, as part of my check in: "git rebase -i HEAD~2". Will that change 2.3.1.dev33 to 2.3.1.dev34 ?20:17
claygglange: nah - let's just hack on the the hummingbird branch of vagrant-swift-all-in-one20:17
claygand have a 'how the fuck do you guys develop on this shit' getting started session20:17
claygccavanna: if the bin scripts are throwing version errors it normally means you need to reinstall the source again - but that normally squares it - i'm sure i'm missing some interesting context20:19
claygccavanna: but just about any time you change the commits if you installed with 'pip install .' instead of 'pip install -e .' then you probably need to re-run 'pip install .'20:19
*** zhill has quit IRC20:20
Guest48719notmyname: ?20:20
jrichlilol, welcome back clayg.  I hope you had a good time in TX.  i have to admit, the channel was a little boring without you20:20
*** Guest48719 is now known as scotticus20:21
notmynamescotticus: yes?20:21
scotticuswhat cool things have you seen in hummingbird?20:21
notmynamescotticus: the per-drive locking we were just talking about and limiting the drives to a certain number of connections are the 2 that quickly spring to mind20:22
notmynameI feel like there was amother/more20:23
scotticusi think that was sort of made easier with Go...20:23
claygjrichli: L-O-L20:23
notmynameah, the async container update on an object put (for which there is a patch for the python object server)20:23
claygnotmyname: oh - yeah - is that patch ready?20:24
notmynameclayg: yup. been sitting there a while for ya :-)20:24
claygawww you were *saving* it for me!20:24
notmynamescotticus: could be. I think there's some really interesting stuff that y'all have done on the golang side, but most of what I've seen is better algorithms than a language-specific thing20:25
notmynameclayg: https://review.openstack.org/#/c/189080/20:25
claygnotmyname: found it on priority reviews - thanks!20:25
notmynamescotticus: what plans do you have for making and sharing performance comparisons?20:26
notmynamescotticus: of python vs golang20:26
notmynameI'd love to see something in Tokyo like that20:27
*** zhill has joined #openstack-swift20:29
ccavannaclayg: I'm sure I did not use pip install -e, since I was following the SAIO instructions.20:30
ccavannaAnd that's installing from a requirements file.20:31
ccavannaare you suggesting to just run "pip install" with no other options? What will that do?20:31
claygccavanna: I'm really sorry those instructions are out of date :\20:34
claygno - "pip install ." where "." means current directory because bash20:34
ccavannaOh, we all used those instructions here :/20:35
ccavannaI confirmed it shows dev34, pip list shows:20:36
ccavannaswift (2.3.1.dev34, /home/swift/swift)20:36
ccavannaI'm sorry for the 101 questions, I'm really new to all these possible problems.20:36
claygccavanna: so you used "python setup.py develop"20:41
claygccavanna: what's your `pip --version`20:42
ccavannayes, I used that command (again, from SAIO :) )20:42
ccavannapip 1.5.4 from /usr/lib/python2.7/dist-packages (python 2.7)20:42
claygok - so just run that command again20:43
ccavannaI executed that (from python-swiftclient, as I originally did). I noticed in the output that it says:20:48
ccavannaInstalling swift script to /usr/local/bin20:48
ccavannawhich is probably what I'm looking for. But the files there were not overwritten.20:48
clayg`sudo python setup.py develop` do it in all the git dirs - it's fine20:50
*** blmartin has quit IRC20:52
*** nadeem has joined #openstack-swift20:57
ccavannaGreat!20:58
ccavannait updated when I exectued from inside the swift directory.20:58
ccavanna10^6 thanks clay.20:59
*** nadeem has quit IRC21:01
*** nadeem has joined #openstack-swift21:01
openstackgerritRobert Francis proposed openstack/swift: add test_copy_to_same  https://review.openstack.org/20382021:02
notmynamerobefran: thanks for your first patch!21:09
robefrannotmyname: just a func test for copy_to_same, mostly to go through the dev/review process, let's see if it passes the auto-testing21:12
claygccavanna: maybe you could recommend (via a bug report, or gerrit change) something that should be updated on those docs that would have helped you out?21:12
notmynamerobefran: did it pass locally? if so, gate checks should be no problem21:12
robefrannotmyname: yes, it passed tox (py27 pep8) and .functests locally21:13
notmynamegreat21:13
claygrobefran: your first attempt to contribute to the project was a new functional test of existing behavior?21:13
* clayg hugs robefran 21:13
notmynameclayg: sounds like a pretty cool guy, that robefran ;-)21:13
claygnotmyname: wait - do I know him?21:13
claygrobefran: do I know you?21:13
notmynameI don't21:13
claygnice21:13
robefranclayg: yes, I know there is already a unit test for copy_to_same, but I thought a functest would be a small/safe addition21:13
claygrobefran: good call21:14
notmynamerobefran: I see from your contrib email that you're with IBM. are you with jrichli or wbhuber_ in Austin?21:14
claygtimburke: joel keeps trying to get patch 130339 merged - i apparently tried to +2 about six revisions ago - is it worth another look?21:15
patchbotclayg: https://review.openstack.org/#/c/130339/21:15
robefrannotmyname: yes, I work with Janie, but I'm in Hamdi's (hrou) and Carlos (ccavanna) team in Toronot21:15
robefranarrg, Toronto21:15
pelusetorgomatic, ping me when you're available.  Wanna chat a little about used_source_etag in ResummingGetter as it related to patch 20128321:15
patchbotpeluse: https://review.openstack.org/#/c/201283/21:15
notmynamerobefran: ah ok. (did we talk on the phone about SMR drives last week?)21:16
claygrobefran: cool!  does that mean you'll be in austin?21:16
wbhuber_notmyname: no, robefran is with IBM but in a different location.21:16
claygfwiw google auto corrects Toronot to Toronto21:16
ccavannaclayg: yes, I will do that. I need to think what would be the best change.21:16
robefrannotmyname: I think you were talking to Hamdi (hrou) about SMR21:16
notmynamerobefran: yup21:16
ccavannanotmyname: and me.21:17
notmynameah, ok!21:17
* peluse wonders what a toronot is21:17
claygccavanna: that would be so very helpful - people that have worked on the project a long time loose all perspective when it comes to encountering this stuff for the first time :'(21:17
timburkeclayg: yes. seems good as of last patchset; haven't looked at latest, but sounds like it's just resolving a merge conflict21:18
claygpeluse: maybe it's like a torus - but not?21:18
ccavannaclayg: it happens to everone after a while.21:18
claygpeluse: maybe we could design a new placement algorithm that's based on a toronot?21:18
jrichlipeluse: sounds like something from my topology class21:18
peluseheh21:19
dfg_are there some instructions somewhere about how to get you SAIO unit tests running?21:19
claygdfg_: do yours not work?21:20
dfg_no21:21
claygdfg_: maybe related to pyeclib if you haven't been following along?  you can try to pip a few things?21:21
clayg`sudo pip install -r requirements.txt ; sudo pip install -r test-requirements.txt` maybe?21:21
dfg_i think i tried that21:23
notmynameor if your SAIO is out of sync enough, might just be easier to set it up again21:23
dfg_notmyname: tried that too :)21:23
*** losingle has quit IRC21:23
claygdfg_: any particuarl group of tests failing for any particularlly coherent reason?21:23
dfg_like 30 failures 40 something errors or somehting?21:24
dfg_as far as people know if you follow the SAIO instructions on the docs they should pass?21:24
notmynameyup. running mine now21:25
claygdfg_: that might not be true for the latest versions of some stuff that pip drags in - esspecially on an old distro using system pip21:25
dfg_i created a brand new VM, precise, followed the instructions and got a ton of failures21:25
claygdfg_: but ccavanna went through this recently apparenlty - so he mgiht now?21:25
clayg*precise*21:25
claygdfg_: I'm guessing there's a few battles in there for you21:26
dfg_i don't want to fight these battles21:26
claygdfg_: I'd drop system setuptools and pip asap21:26
claygdfg_: then use vagrant-swift-all-in-one21:26
claygno one has time to dick with that crap21:26
notmynameRan 4051 tests in 121.474s (all OK, on precise)21:26
dfg_:/21:27
claygnotmyname: you have a working precise install!21:27
claygnotmyname: i'm floored21:27
dfg_so the instructions on how to get this working are to use the swiftstack vagrant saio image?21:28
notmynameI normally try to have the min version we say is required installed21:28
dfg_or use trusty?21:28
dfg_or what?21:28
claygdfg_: NO ONE IN THIS FUCKING CHANNEL HAS ANY IDEA WHAT ERRORS YOU ARE SEEING21:28
*** robefran has quit IRC21:28
claygdfg_: you want help tell me what the problem is21:29
notmynamedfg_: no, the instructions are the saio docs at swift.openstack.org like always. but if you don't want to mess with all the nasty details and just get it working, you can use the vagrant swift all in one21:29
claygyou don't wnat to say what's wrong try the vagrant image or trusty or something else21:29
clayggeez21:29
dfg_clayg: do you need a nap or something?21:29
notmynamelol21:29
claygdfg_: probably moar coffee ;)21:29
dfg_i'll try it again. thx21:29
*** pgbridge has joined #openstack-swift21:29
openstackgerritJanie Richling proposed openstack/swift: Encrypting/Decrypting account and container metadata  https://review.openstack.org/20345421:34
*** jordanP has quit IRC21:39
*** ccavanna has quit IRC21:57
*** thurloat_isgone is now known as thurloat21:58
*** peterbusque has joined #openstack-swift22:00
*** tongli has quit IRC22:00
openstackgerritJoel Wright proposed openstack/python-swiftclient: Log and report trace on service operation fails  https://review.openstack.org/17169222:13
*** robefran has joined #openstack-swift22:14
*** matt____ is now known as mattoliverau22:15
*** ChanServ sets mode: +v mattoliverau22:15
mattoliveraumorning everyone22:16
notmynamewbhuber_: does the comment from peluse make sense on https://review.openstack.org/#/c/201791/?22:16
notmynamemattoliverau: hullo22:16
mattoliveraunotmyname: have fun at the weekend conference?22:16
pelusedo my comments ever make sense?22:16
notmynamemattoliverau: yeah, it was good. I met https://twitter.com/dfflanders from your neighborhood22:18
*** mgriffin_ has quit IRC22:18
*** mgriffin_ has joined #openstack-swift22:18
*** mgriffin_ is now known as mgriffin22:18
notmynamemattoliverau: I got some good ideas to think about and some stuff to work on. so that was good22:19
*** zhill has quit IRC22:19
wbhuber_notmyname: yep.  his comments made sense.  i've to go close and abandon it.22:20
mattoliveraunotmyname: wow, he follows me, he looks familar so he must be one of the guys from the openstack meetup group here in melb.22:20
notmynamemattoliverau: yeah. he's at uni melb22:21
notmynameruns some of the nectar stuff there, I believe22:21
mattoliverauahh, yes I do know him then :)22:21
robefranmy patch https://review.openstack.org/#/c/203820/ failed in gate-swift-dsvm-functional, what is gate-swift-dsvm-functional? and how do I run it locally?22:21
notmynamerobefran: it's the `tox -e func` tests22:22
clayglooks like a build error tho22:22
notmynamerobefran: which are the `./.functests`22:22
*** jrichli has quit IRC22:22
notmynameright, not your code22:22
claygdistutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('pbr')22:22
mattoliveraunotmyname: so I have a beta version of shrinking in container sharding written, so had a productive Friday (and weekend) :)22:22
robefranI ran ./.functests directly, should I run tox -e func instead?22:22
notmynamemattoliverau: nice!!22:22
notmynamerobefran: no22:23
mattoliveraunow just need to test22:23
notmynamerobefran: note the last comment on the review now22:23
notmynamerobefran: the elastic recheck bot noticed the failure and corollated it with other failures that have been seen. there's already a bug for it22:23
robefrannotmyname: how to I trigger a recheck?22:24
notmynameso I added the recheck comment22:24
notmynamerobefran: I just did :-)22:24
notmynamesyntax is "recheck [bug number]"22:24
notmynamewbhuber_: thanks22:26
*** jlhinson has quit IRC22:27
wbhuber_peluse: the charts you sent is based for 3-way replica?22:28
wbhuber_Not sure about "tuned" vs "untuned"22:29
*** nadeem has quit IRC22:32
pelusewbhuber_, yes 3x only for now and 'tuned' just means where Bill was able to tweak settings for incremental performance22:34
*** zhill has joined #openstack-swift22:37
*** _hrou_ has quit IRC22:42
*** km has joined #openstack-swift22:51
*** chlong has quit IRC22:54
*** zhill has quit IRC22:59
claygnotmyname: crap - there's local trick that you have to do to keep the transaction id in the spanwned greenthread for the container update :'(23:00
notmynamebah!23:01
notmynamegot a pointer to it?23:01
claygnotmyname: looking23:01
claygjust noticed when reviewing and trying to figure out what's going on in the logs :'(23:02
torgomatic0x10794012023:02
claygthen I remember the thing23:02
claygtorgomatic has a pointer23:02
claygkudos23:02
torgomaticsomeone had to say it23:02
claygnotmyname: some proxy code is going to do it - there's probably a comment23:02
claygswifterdarrell: ran into it during the per-disk-object-server testing too I think - not sure why - his thing may have just been an existing bug?23:03
*** peterbusque has left #openstack-swift23:06
claygnotmyname: so this was swifterdarrell's bug -> https://bugs.launchpad.net/swift/+bug/140930223:10
openstackLaunchpad bug 1409302 in OpenStack Object Storage (swift) "Use of GreenAsyncPile can lose txn_id logging" [Undecided,New]23:10
*** hrou has joined #openstack-swift23:11
notmynameclayg: looks straightforward https://gist.github.com/notmyname/4121933a86aa3477b2f623:13
notmynameI'll write a test and resubmit it23:14
claygummm gists are different23:14
claygnotmyname: we've got to quit doing this -> https://gist.github.com/clayg/b36f6196156999f1671c23:15
notmynamelol23:15
notmynameah, yeah. there's one other call. so your way is better with the named param23:17
claygnotmyname: yeah i wasn't sure what tests would have to say about the new arg if I didn't make it optional :\23:17
claygnotmyname: I was sorta liking your way of making it required and forcing it23:18
notmynamedelete_at_update() is the only other caller23:18
claygnotmyname: oh it defaults to node_timeout :'(23:18
claygyeah the default should be "better"23:18
notmynameyup agreed. we had talked about that. I'll do it with this patchset23:19
notmynamenode timeout is 3. new default  = 1? my tests in the commit message used .7523:19
*** bitblt has joined #openstack-swift23:20
claygnotmyname: no no - saying there *should* be a better default is the easy part - acctually *picking* the better default gets into the realm of biksheadding - glwt23:20
notmynameok, it's 1.023:21
*** kei_yama has joined #openstack-swift23:22
claygbah - i tried putting the option in the container-server config :'(23:27
*** minwoob has quit IRC23:27
notmynameand?23:28
claygheh - oddly it did not effect the value of the object-server's container_update_timeout value!?23:29
*** janonymous has quit IRC23:31
*** mgarza_ has quit IRC23:47
pelusetorgomatic, got a sec to talk EC GET fun?23:47
torgomaticpeluse: sure23:47
claygpeluse: that *does* sound fun23:48
peluseCool... will try IRC first and if I make no sense resort to telephone.. patch is 20128323:48
torgomaticpatchbot: patch 20128323:48
patchbottorgomatic: https://review.openstack.org/#/c/201283/23:48
patchbottorgomatic: https://review.openstack.org/#/c/201283/23:48
peluseso... we fire off ndata GETs and if we get back <n where one is old (overwrite during node down) we'd basically puke23:49
pelusewhat I did was start paying attention to ETAGs of the original object and sort the incoming FAs into ETAG buckets, then find the largest one and use that23:49
torgomaticseems reasonable23:49
peluseand if it wasn't big enough, jump back to building anohter set of getters to go fetch from the parity nodes23:49
peluseworks pretty good.23:50
pelusequesiton is on ResumingGetter and an element in there that looks like it might have been trying to do something similar23:50
peluseone sec23:50
peluseused_source_etag - I think you added that one?  I tried to sort out what its goal in life is but can't seem to get it23:50
peluse(BTW this is just a part of the patch, will cover the rest here in a sec after we talk about this used_source_etag thing)23:51
pelusewhile you're reading code, my best guess was that it was trying to track etags on GET and if it found one that didn't match a previous find would 404 forcing the node_iter to pick up the next node.  However...23:52
torgomaticthat doesn't look familiar, but it appears to be enforcing that we don't ever mix two FAs with different source etags23:52
peluseeither I misunderstand otally (probably) or since there's one ResumeGetterper FA GET, it'd never find a 2nd23:53
openstackgerritMerged openstack/swift: Minor change in development saio setup doc  https://review.openstack.org/20372623:53
torgomaticthe only relevant bit is "if self.used_source_etag != src.headers.get('x-object-sysmeta-ec-etag', ...)"23:53
peluseOK, so I wasn't far off... but how, when there's one instance of the ResumeGetter per GET and it gets just one FA would there ever be a 2nd one to not match?23:53
torgomaticuhh23:54
pelusethis line confuses my small brain: "if not self.newest:  # one good source is enough"23:54
torgomaticgood question; let me throw out that whole if branch and see if anything fails23:55
pelusein _get_source_and_node()23:55
claygpeluse: torgomatic: I'm not sure x-newest makes sense at all in EC land23:55
peluseI think it might be dead code23:55
notmynameclayg: or in non-ec land, really ;-)23:55
claygit'd be one weird schema failure if you can rebuild multiple versions of an ec object23:55
torgomaticpeluse: ECObjectController always calls ResumingGetter(..., newest=False)23:56
torgomaticso you're right, x-newest + EC is nonsense23:56
claygnotmyname: why doesn't x-newest make sense in the replicated land - if you have multiple versions - why *wouldn't* you want the newest!23:56
pelusetorgomatic, ahh, I missed that false parm...23:56
*** nadeem has joined #openstack-swift23:56
notmynameclayg: yeah, but "newest" is defined by "latest timestamp found on primary nodes", not latest in the cluster. x-newest is ok, but it certainly isn't a guarantee23:57
claygnotmyname: x-newest always digs handoffs23:57
notmynamereally!?23:57
claygit doesn't check the *whole* cluster - but it's not terrible23:57
claygyeah post-as-copy hates your cluster23:57
notmynameI thought it was just the timestamps on primaries23:57
clayg:'(23:57
claygsorry bro23:57
notmynameok, so better for users, worse for the cluster, and apps still need to know about eventual consistency ;-)23:58
peluseOK, so seems like self.used_source_etag was clearly added for EC though (it checks the ec etag header) but still AFAICT its dead code.  Or at least with the changes I made on that patch its not needed I dont think23:58
*** darrenc_ is now known as darrenc23:58
pelusemaybe :)23:58
torgomaticpeluse: well, when I comment out that code block, I get a couple failing tests, so something's using that functionality23:59
torgomaticOH23:59
torgomaticright23:59
torgomaticso it's a ResumingGetter, right?23:59
peluseheh, test code23:59
peluseyeah23:59
torgomaticif shit breaks and it resumes, it's gotta resume from something with the same etag23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!