Thursday, 2017-07-06

*** hoonetorg has joined #openstack-swift00:03
openstackgerritTim Burke proposed openstack/liberasurecode master: Use zlib for CRC-32
openstackgerritTim Burke proposed openstack/liberasurecode master: Stop pretending to support SSE4-optimized CRC-32C
zaitcevwait, which one is it00:14
timburkeboth. i realized ripping out the never-been-used code might be more reasonable as a separate change00:15
*** JimCheung has quit IRC00:16
*** ukaynar has joined #openstack-swift00:18
claygthurloat: handoffs_first mode can help with that!00:23
*** JimCheung has joined #openstack-swift00:25
thurloatclayg oh that's neat, it priotizes shipping off files that would be otherwise missing?00:25
thurloatyou have to restart the object server to apply that though?00:25
claygit's an object-replicator option00:26
claygit prioritizes replication of partitions that are not in the right place - which could cause 404/missing00:26
thurloatcool beans00:27
thurloatif I apply that on the object server nodes where I know the data currently exists that I want to be found again asap, that could help speed up the process00:27
thurloatI could imagine a situation where the replicator is backed up with actual new data replication tasks that don't get shipped off in time00:28
thurloatcould that be an issue/00:28
*** vint_bra has joined #openstack-swift00:35
*** vint_bra has quit IRC00:59
openstackgerritTovin Seven proposed openstack/swift master: OSprofiler in OpenStack Swift
*** cshastri has joined #openstack-swift01:14
*** caiobrentano has joined #openstack-swift01:33
zaitcevI'm trying to rebuild liberasurecode on Fedora, it blows up because it links with the installed version and then checks fail.01:37
zaitcevUsing libtool is (almost) always a mistake these days. You can use LDFLAGS on everything, including OSX and Windows.01:38
*** gyee has quit IRC01:58
*** vint_bra has joined #openstack-swift02:03
kota_back to online02:07
kota_oh, increase part power get merged!? congrats02:07
*** vint_bra has quit IRC02:08
*** JimCheung has quit IRC02:11
*** bkopilov has quit IRC02:12
*** SkyRocknRoll has quit IRC02:27
*** vint_bra has joined #openstack-swift02:33
*** vint_bra has quit IRC02:48
*** d0ugal has quit IRC03:19
*** d0ugal_ has joined #openstack-swift03:19
*** winggundamth has joined #openstack-swift03:24
*** gkadam has joined #openstack-swift03:30
*** gkadam has quit IRC03:30
*** gkadam has joined #openstack-swift03:30
*** bkopilov has joined #openstack-swift03:32
*** abhitechie has joined #openstack-swift03:54
*** JimCheung has joined #openstack-swift03:54
openstackgerritPete Zaitcev proposed openstack/liberasurecode master: Undo parts of I0e79bed7755a1f286b746a70fcf56fdc972bfd5d
zaitcevSorry, kota_03:55
*** itlinux has joined #openstack-swift03:55
kota_zaitcev: no worries, it looks like you need that03:58
zaitcevbut it reproduces on my laptop too03:58
*** JimCheung has quit IRC03:59
*** abhitechie has quit IRC04:02
*** links has joined #openstack-swift04:05
*** abhitechie has joined #openstack-swift04:13
*** psachin has joined #openstack-swift04:14
*** JimCheung has joined #openstack-swift04:25
kota_zaitcev: perhaps, you may run the test via /build/BUILD/liberasurecode-1.5.0/test/liberasurecode_test ?04:26
kota_ah, no, it looks like you are calling just `make test`04:27
kota_AFAIK, test/liberasurecode_test is a script to run .libs/lt-liberasurecode_test with relink to ../src/.libs/liberasurecode.so04:29
*** JimCheung has quit IRC04:29
kota_hmm... interesting04:29
zaitcevThe point is, "make test" used to work in 1.4.0. I tried to figure out in detail what went wrong, but ended in the innards of libtool very quickly.04:30
kota_ok I was missing something, thanks for bringing up04:33
kota_zaitcev: ^04:33
zaitcevThings happen when software gets used.04:33
kota_the interesting point is `make test` works with 1.5.0 in my ubuntu box04:38
kota_that sounds from version diff for libtool (or automake?) ? idk04:39
zaitcevdo you have it installed by any chance too?04:39
zaitcevIt will link to one in /usr/lib or /usr/local/lib without telling you04:40
*** caiobrentano has quit IRC04:41
kota_lemme check04:43
kota_liberasurecode.1.5.0 located in /usr/local/lib but even i remove the library with `sudo rm /usr/local/lib/*`, `make test` runs successfully so that it looks to point out the correct local .so files in the repository04:45
zaitcevright... you can also verify with ldd04:46
kota_ldd shows like " => /opt/stack/liberasurecode/src/.libs/ "04:48
kota_ /opt/stack/liberasureode is a clone from github repo04:48
zaitcevThere's something fishy about it. If I capture the build directory and re-run configure in it, everything works too.04:49
zaitceveven if I just run ./config.status04:50
zaitcevSo, the only way to reproduce it is to let RPM to build it.04:51
*** SkyRocknRoll has joined #openstack-swift04:56
*** ukaynar has quit IRC05:01
*** ukaynar_ has joined #openstack-swift05:01
*** psachin has quit IRC05:01
*** psachin has joined #openstack-swift05:11
*** andymccr_ has joined #openstack-swift05:11
*** cargonza_ has joined #openstack-swift05:13
*** ujjain- has joined #openstack-swift05:14
*** ktsuyuzaki has joined #openstack-swift05:14
*** ChanServ sets mode: +v ktsuyuzaki05:14
*** ktsuyuzaki is now known as kota___05:15
kota___hmm... irc disconnected :/05:15
*** JimCheung has joined #openstack-swift05:18
*** clarkb1 has joined #openstack-swift05:19
*** joelinux_ has joined #openstack-swift05:20
*** dosaboy_ has joined #openstack-swift05:20
*** andymccr has quit IRC05:22
*** ujjain has quit IRC05:22
*** cargonza has quit IRC05:22
*** kota_ has quit IRC05:22
*** clarkb has quit IRC05:22
*** dosaboy has quit IRC05:22
*** joelinux73 has quit IRC05:22
*** cargonza_ is now known as cargonza05:22
*** JimCheung has quit IRC05:23
*** ukaynar_ has quit IRC05:47
*** ukaynar has joined #openstack-swift05:48
*** SkyRocknRoll has quit IRC05:51
*** winggundamth has quit IRC05:52
*** ukaynar has quit IRC05:52
*** itlinux has quit IRC05:58
*** rcernin has joined #openstack-swift05:59
*** nadeem has quit IRC06:01
*** ChubYann has quit IRC06:01
*** rcernin has quit IRC06:04
*** nadeem has joined #openstack-swift06:05
*** cschwede_ has joined #openstack-swift06:14
*** cschwede has quit IRC06:16
*** tovin07_ has joined #openstack-swift06:17
*** winggundamth has joined #openstack-swift06:53
*** skudlik has joined #openstack-swift07:03
*** tesseract has joined #openstack-swift07:04
*** tesseract-RH has joined #openstack-swift07:04
*** tesseract has quit IRC07:04
*** tesseract-RH has quit IRC07:04
*** hieulq has joined #openstack-swift07:05
*** tesseract has joined #openstack-swift07:05
*** abhitechie has quit IRC07:06
*** abhitechie has joined #openstack-swift07:09
*** onovy has quit IRC07:15
*** vladiskuz has quit IRC07:17
*** onovy has joined #openstack-swift07:17
*** skudlik has quit IRC07:24
*** d0ugal_ has quit IRC07:46
*** d0ugal has joined #openstack-swift07:46
*** d0ugal has quit IRC07:46
*** d0ugal has joined #openstack-swift07:46
*** cbartz has joined #openstack-swift07:51
*** kota___ is now known as kota_07:59
acolesgood morning08:04
openstackgerritDaniele Pizzolli proposed openstack/swift master: Add full working example of sharing a container with another user
*** gkadam has quit IRC08:58
*** gkadam has joined #openstack-swift09:06
*** andymccr_ is now known as andymccr09:11
*** abhitechie has quit IRC09:26
*** abhitechie has joined #openstack-swift09:27
*** gkadam has quit IRC09:47
*** gkadam has joined #openstack-swift09:48
*** kei_yama has quit IRC09:58
*** tovin07_ has quit IRC10:05
*** cshastri has quit IRC10:21
*** bkopilov has quit IRC10:45
*** abhitechie has quit IRC10:48
*** abhitechie has joined #openstack-swift10:49
*** abhitechie has quit IRC11:16
*** ukaynar has joined #openstack-swift11:17
*** abhitechie has joined #openstack-swift11:23
*** mat128 has joined #openstack-swift11:26
*** ukaynar has quit IRC11:32
*** JimCheung has joined #openstack-swift11:34
*** dancn has joined #openstack-swift11:37
*** JimCheung has quit IRC11:39
*** caiobrentano has joined #openstack-swift12:11
kota_acoles: hi12:13
acoleskota_: hi are you feeling better?12:13
kota_better than yesterday12:14
kota_acoles: i have a bad news for isa_l_rs_vand discussed in the irc meeting12:14
kota_acoles: at least, i found a bad combination with isa_l_rs_vand k=24, m=412:14
kota_that means m <= 4 could have bad parameters as discussed the issue board12:15
kota_I've been running gready check for the k x m combinations but it looks it needs more time to finish whole calculation12:16
acoleskota_: :( so that is 24 data, 4 parity?12:16
kota_I'll continue the calculation tommorow and summarize what is actually bad parameters12:17
kota_just notify you for current status12:17
kota_and, the better news is the newer pyeclib and liberasurecode can raise an error for the bad combination12:18
kota_so that it results in just an error, not making a new bad fragment.12:18
acolesso, in terms used on , that is (k=24, m-k=4) so, yeah, outside the bounds described there12:18
acoleskota_: thanks for sharing that result12:19
acoleskota_: so does pyeclib raise an error when failing to decode or when creating the encoder/decoder?12:19
kota_wait a bit12:20
acolesI mean, can swift use (24,4)? (I think so, I am just confirming your statement)12:20
kota_i think <- this commit saves us12:21
kota_yes, swift can use. current k + m constraint is just... k + m <=32 and  m !>= 5 for isa_l12:22
kota_for pyeclib raise an error, pyeclib can detect un-recoverable fragment combination when calling *decode*12:25
kota_and raise an error if it is not possible to decode12:25
kota_(or reconstruct)12:25
kota_so it still is able to store (PUT/encode) objects into the k=24, m=4 isa_l_rs_vand policy, IIRC.12:26
mattoliveraukong: just been re-testing patch 470158. Looks great. Works well, deletes and only uses the minimum extra .ts files as needed based on the partitions number of primary locals (affinity).12:26
patchbot - swift - Write-affinity aware object deletion12:26
mattoliveraukong: so given it the +2/+A. Nice work!12:27
mattoliverauand on that note. I'm going to bed :)12:27
mattoliveraunight all12:27
acolesmattoliverau: g'night, good work!12:27
acoleskota_: yep, that's what I thought12:27
kota_mattoliverau: good night!12:28
kota_acoles: kk, i'm willing to update the status more tommorow after calculating which parameter can be chosen and not good.12:29
*** caiobrentano has quit IRC12:30
acoleskota_: are you running tests on other schemes e.g. 22+4?12:30
kota_for all combinations, in k + m <= 3212:30
kota_greedy testing takes a long time12:30
acoleskota_: fantastic! thanks12:31
kota_i hope it finished today but it looks still in mid of whole testing yet :/12:32
*** chlong_ has joined #openstack-swift12:36
*** NM has joined #openstack-swift12:42
*** caiobrentano has joined #openstack-swift13:03
kongmattoliverau: thanks!13:04
kongseems I got a good news before i go to bed :-)13:05
*** saint_ has joined #openstack-swift13:05
*** caiobrentano has quit IRC13:07
*** caiobrentano has joined #openstack-swift13:14
*** ouchkernel has joined #openstack-swift13:18
*** lucasxu has joined #openstack-swift13:19
*** klamath has joined #openstack-swift13:21
*** klamath has joined #openstack-swift13:21
*** catintheroof has joined #openstack-swift13:25
*** vint_bra has joined #openstack-swift13:29
*** catintheroof has quit IRC13:36
*** gkadam has quit IRC13:36
*** catintheroof has joined #openstack-swift13:37
*** abhitechie has quit IRC13:38
*** vint_bra has quit IRC13:41
*** winggundamth has quit IRC13:52
openstackgerritMerged openstack/swift master: Write-affinity aware object deletion
openstackgerritOpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements
openstackgerritAlex Szarka proposed openstack/swift master: Use more specific asserts in test/unit/account tests
openstackgerritAlex Szarka proposed openstack/swift master: Use more specific asserts in tests
*** saint_ has quit IRC14:12
*** JimCheung has joined #openstack-swift14:13
*** JimCheung has quit IRC14:17
*** links has quit IRC14:18
thurloatclayg thanks again for the handoffs_first tip yesterday, worked out perfectly most objects are resolved again.14:19
*** ouchkernel has quit IRC14:19
*** SkyRocknRoll has joined #openstack-swift14:54
notmynamekota_: thanks for the info on EC14:55
notmynameremind me why we use vand instead of cauchy?14:55
*** skudlik has joined #openstack-swift14:57
*** catintheroof has quit IRC14:58
*** catintheroof has joined #openstack-swift15:02
*** psachin has quit IRC15:07
*** vint_bra has joined #openstack-swift15:08
*** sanchitmalhotra has quit IRC15:22
*** sanchitmalhotra has joined #openstack-swift15:23
openstackgerritChristopher Bartz proposed openstack/python-swiftclient master: Option to ignore mtime metadata entry.
*** gyee has joined #openstack-swift15:51
*** clarkb1 is now known as clarkb15:54
*** tesseract has quit IRC15:54
*** itlinux has joined #openstack-swift15:55
*** cbartz has left #openstack-swift16:06
*** chsc has joined #openstack-swift16:15
*** chsc has joined #openstack-swift16:15
*** chsc has quit IRC16:19
*** oshritf has quit IRC16:31
*** ChubYann has joined #openstack-swift16:36
*** JimCheung has joined #openstack-swift16:44
claygkong: wtg!  thanks for that fix!16:46
claygkong: what's next!?16:47
claygkota_: omg - thank you so much for doing that16:47
claygtimburke notmyname: if we change the "requirements" for isa-l rs_vand do you think we have to do a whole 'nother cycle of "warning this is bad" before we switch "newly" bad policies to proxies wont' start unless deprecated?16:48
*** JimCheung has quit IRC16:50
notmynameif requirements for isa-l change, we don't have to do anything in swift, I think16:54
notmyname(un)fortunately, we don't have any way to even enforce a new isa-l dependency. at least currently16:56
notmynamethat is, there isn't a hard dependency. we could write something in libec to check the version and enforce it, but still, does that mean anything for swift?16:57
notmynametertiary dependencies, no fun16:57
*** JimCheung has joined #openstack-swift17:09
openstackgerritTim Burke proposed openstack/python-swiftclient master: Option to ignore mtime metadata entry.
*** chsc has joined #openstack-swift17:31
*** chsc has joined #openstack-swift17:31
*** hieulq has quit IRC17:33
openstackgerritTim Burke proposed openstack/liberasurecode master: Use zlib for CRC-32
*** itlinux has quit IRC17:43
*** hieulq has joined #openstack-swift17:48
*** oshritf has joined #openstack-swift17:56
openstackgerritTim Burke proposed openstack/liberasurecode master: Un-inline get/set_metatdata_chksum
openstackgerritTim Burke proposed openstack/swift master: WIP: Move listing formatting out to proxy middleware
openstackgerritMerged openstack/swift master: Switch from oslosphinx to openstackdocstheme
*** ChubYann has quit IRC18:37
notmynamerledisez: if you've got more specific points for the LOSF, that would be great to add before the PTG. i'd love to walk into the conversation with a set of actual questions instead of just "what should we talk about with LOSF"18:47
*** tonanhngo has joined #openstack-swift18:49
*** ChubYann has joined #openstack-swift18:51
*** hieulq has quit IRC18:51
*** itlinux has joined #openstack-swift18:58
timburkethanks for the libec reviews zaitcev!19:06
zaitcevyou did all the work19:06
timburkeonly five months after kicking over that rock :P19:06
*** hieulq has joined #openstack-swift19:06
openstackgerritOpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements
*** pcaruana has quit IRC19:12
openstackgerritTim Burke proposed openstack/python-swiftclient master: Allow --meta on upload
openstackgerritJohn Dickinson proposed openstack/swift master: docs migration from openstack-manuals
*** hieulq has quit IRC20:21
tdasilvatimburke, kota_, clayg: can you confirm i'm following the correct approach for this bug:
openstackLaunchpad bug 1665242 in OpenStack Object Storage (swift) "Reconstruction with liberasurecode_rs_vand can get memory leak" [Undecided,New]20:23
tdasilvasince it has been fixed in libec, i want to set it to released in swift too???20:23
timburkei wonder if we should just remove swift from the bug or mark it invalid... we're not really "affected" except insofar as we may have noticed it originally in swift -- but if we noticed some memory leak as the result of cpython or eventlet, would we have a swift bug to track the upstream bug?20:26
tdasilvatimburke: yeah, i wasn't really sure why it was created like that, even for pyeclib, it's the same, right? I noticed the comment said: "Note that this leak caused at the liberasurecode layer so what both pyeclib and Swift users can do is just upgrading your *liberasurecode* after the fix get merged"20:28
claygyay, patch 481221 is going to be amazing!20:31
patchbot - swift - docs migration from openstack-manuals20:31
notmynameclayg: remains to be seen ;-)20:31
notmynameif this is the right way, then there's one more thing to do (move the existing install-guide to docs/source/install), but I think that's pretty much it20:32
claygtdasilva: timburke: does swift say it depends on a fixed version of libec?  Otherwise maybe that can be our bug?20:33
*** hieulq has joined #openstack-swift20:37
notmynametdasilva: is ugly now ;-)20:37
notmynameno, wait. "it's big because we're all old" now20:37
claygi think it's fine20:38
tdasilva that's just messed up20:38
notmynameyes, well you've already said you don't care about how it looks. I think you might be reading it in lynx or something20:38
tdasilvaclayg: we do not specify a version of liberasurecode in bindep :/20:44
claygok, so fix released then!  We could open a new bug that says "openstack should be able to specify versions for all (not-python) dependencies to downstream packagers" - but that may be a dupe of something else somewhere20:46
clarkbclayg: pretty sure that bindep does support that, problem is if you then try to use it on platform without that dep you will have a sad time20:47
timburkewe *do* have a >=1.3.1 on pyeclib...20:47
timburke...which in turn has a *warning* if it's using (was built against?) libec<1.3.1 ...20:48
clarkb"Version constraints are a comma separated list of constraints where each constraint is (== | < | <= | >= | > | !=) VERSION, and the constraints are ANDed together (the same as pip requirements version constraints)." from
*** lcurtis has joined #openstack-swift20:50
claygclarkb: yeah i was seeing that, none of the examples include version specifiers - not sure I've seen them used in practice - but that's probably what we need to do!?20:52
lcurtishello all - wondering if someone knows of any good openstack swift training? Need to bring a guy up to speed.20:52
clarkbclayg: ya if you want to signal that old versions are a no go that is what I would do20:53
claygcould be weird/strange during development cycle when newer versions aren't packaged "in the wild" yet20:53
clarkbclayg: ya that is likely where you will haev trouble20:53
clarkbclayg: perhaps leave a commetn about it in bindep for now?20:53
timburkelcurtis: out of curiosity, for more of a dev or op role?20:54
timburkenot that anything immediately springs to mind for either :-/20:54
tdasilvalcurtis: funny you ask that, i've been thinking if we should/could build a curated Wiki page with different reading/watching resources for learning swift, sort of start with swift 101, to a more in-depth topics list...20:55
tdasilvanot thinking about creating new content, but just listing what we know exists already...20:56
claygI think is still a thing - lots of swift concepts evolve rather than get replaced20:56
*** lucasxu has quit IRC20:57
*** cschwede_ has quit IRC20:58
tdasilvaclayg: yep, that's a very good book!20:59
*** skudlik has quit IRC21:24
*** itlinux has quit IRC21:40
*** itlinux has joined #openstack-swift21:41
lcurtistimburke - an op role21:45
lcurtisthank you guys, as always, for your input21:46
zaitcevI suppose he could buy Joe Arnold's book if he reads English.21:47
lcurtisyeah we have the book21:48
lcurtiskind of want to inject the knowledge asap into this guy21:48
lcurtisalong with what he has already been picking up21:48
claygobligatory joe arnold book joke:
claygmaybe ???21:49
*** hieulq has quit IRC21:53
lcurtisthats good link21:53
lcurtishave not seen that before21:53
*** catintheroof has quit IRC21:55
*** NM has quit IRC21:58
*** lcurtis has quit IRC21:58
*** hieulq has joined #openstack-swift22:08
*** tonanhngo has quit IRC22:19
*** tonanhngo has joined #openstack-swift22:32
*** tonanhngo has quit IRC22:37
*** klamath has quit IRC22:38
*** caiobrentano has quit IRC22:48
*** vint_bra has quit IRC22:54
*** JimCheung has quit IRC22:57
*** JimCheung has joined #openstack-swift22:58
*** mat128 has quit IRC23:01
*** hieulq has quit IRC23:24
*** oshritf has quit IRC23:29
*** NM has joined #openstack-swift23:29
*** kei_yama has joined #openstack-swift23:34
*** NM has quit IRC23:35
notmynameclarkb: timburke: what does it take to get a new version of liberasurecode in the gate? the warning causes problems23:37
clarkbnotmyname: we just install from the distro so you will need to get it into the distro, or enable UCA if UCA has what you need23:39
timburkei thought that was fixed in p 469314 ?23:39
patchbot - swift - Stop having Sphinx treat warnings as errors (MERGED)23:39
notmynameah, yeah, that sounds familiar23:39
clarkbenabling UCA in the templated jobs taht all projects share is probably something that we should talk about in a wider audiene though ( or maybe swift just has a one off until UCA isn't required)23:39
notmynametimburke: ah! that's where I was remembering that. I was sure we did the warning-as-error thing, then i couldn't find it. part of the docs migration instructions include "be sure to turn on warnings-as-error"23:40
*** hieulq has joined #openstack-swift23:40
*** chsc has quit IRC23:40
notmyname(which we don't actually have to do, since libec is the only warning we have. I mean, it's better to have it on, but we're at the mercy of ubuntu packagers, ithink23:40
*** Sukhdev has joined #openstack-swift23:40
*** itlinux has quit IRC23:43
openstackgerritJohn Dickinson proposed openstack/swift master: docs migration from openstack-manuals
-openstackstatus- NOTICE: has been cleaned up and rebooted, and should return to building rotation23:45
timburkenotmyname: bonus! now the gate will complete *way* faster23:48
notmynameah, right. since it's only docs jobs that will be rerun23:48
timburkethere will be some others, but none of the dsvm ones23:49

Generated by 2.15.3 by Marius Gedminas - find it at!