Tuesday, 2015-05-12

*** esker has joined #openstack-swift00:03
hogerrit doesn't work today?00:05
*** kota_ has joined #openstack-swift00:05
mattoliverauho: we are experiencing infra technical details00:08
kota_mattoliverau: gerrit is stopping?00:10
*** esker has quit IRC00:10
torgomatichttp://media.tumblr.com/1e1d8ecea26aa3ce4e442c713e1e6004/tumblr_inline_mok7jhHaaZ1qz4rgp.png00:10
*** esker has joined #openstack-swift00:10
hotorgomatic: lol00:10
kota_lol00:10
homattoliverau: thanks!00:12
mattoliverausomeone on my team is an infra core, and he's said he's missing our team meeting cause infra is exploding.. so can't be good, and now gerrit is down :P00:13
*** yuanz has quit IRC00:14
*** esker has quit IRC00:14
notmynamesounds like they got the "turn it off" part down. did they try turning it on again? ;-)00:15
mattoliveraunotmyname: lol, maybe they've tried but it isn't plugged in00:17
mattoliverauyou know it's bad when they have to attempt a downgrade00:18
*** vinsh has quit IRC00:36
*** dmorita has joined #openstack-swift00:48
*** ChanServ changes topic to "Review Dashboard: http://goo.gl/8IUcKl | Summary Dashboard: http://goo.gl/qHus5v | Summit scheduling: https://etherpad.openstack.org/p/liberty-swift-summit-topics | Logs: http://eavesdrop.openstack.org/irclogs/%23openstack-swift/"00:57
-openstackstatus- NOTICE: Gerrit has been downgraded to version 2.8 due to the issues observed today. Please report further problems in #openstack-infra.00:57
*** tobe has joined #openstack-swift00:58
*** km__ has joined #openstack-swift01:03
*** km has quit IRC01:03
*** nadeem has joined #openstack-swift01:10
*** gyee has quit IRC01:12
*** ndk has quit IRC01:26
*** ndk has joined #openstack-swift01:26
*** jkugel has joined #openstack-swift01:57
*** km has joined #openstack-swift02:15
*** km__ has quit IRC02:16
*** bkopilov has quit IRC02:31
*** vinsh has joined #openstack-swift02:39
*** nadeem has quit IRC02:44
*** vinsh has quit IRC03:07
*** tamizh_geek has joined #openstack-swift03:08
*** wbhuber has quit IRC03:14
*** clayg has quit IRC03:17
*** clayg has joined #openstack-swift03:18
*** ChanServ sets mode: +v clayg03:18
*** zaitcev has quit IRC03:24
*** tamizh_geek has quit IRC03:25
*** jrichli has quit IRC03:34
*** bill_az has quit IRC03:41
*** haomai___ has quit IRC04:03
*** bill_az has joined #openstack-swift04:05
*** bkopilov has joined #openstack-swift04:07
*** tamizh_geek has joined #openstack-swift04:19
*** asettle has quit IRC04:21
*** tobe has quit IRC04:52
*** 5EXAA60VI has joined #openstack-swift04:52
*** SkyRocknRoll has joined #openstack-swift05:46
*** SkyRocknRoll has joined #openstack-swift05:46
*** hugokuo has quit IRC06:07
openstackgerritKota Tsuyuzaki proposed openstack/swift: Remove confusable query string on post as copy  https://review.openstack.org/17892706:08
*** hugokuo has joined #openstack-swift06:09
*** zhill has joined #openstack-swift06:20
*** silor has joined #openstack-swift06:25
*** bill_az has quit IRC06:29
*** zhill has quit IRC06:54
*** ppai has joined #openstack-swift06:57
*** tobe43 has joined #openstack-swift07:00
*** kota_ has quit IRC07:02
*** remix_auei is now known as remix_tj07:07
*** geaaru has joined #openstack-swift07:27
*** jistr has joined #openstack-swift07:33
*** openstackstatus has quit IRC07:52
*** openstack has joined #openstack-swift07:54
*** jordanP has joined #openstack-swift08:10
*** val2k has joined #openstack-swift08:15
*** kota_ has joined #openstack-swift08:17
val2kHello, I would like to know if there is a way to log on Swift without the environment variables (OS_USERNAME, etc...)08:19
openstackgerritKota Tsuyuzaki proposed openstack/swift: Remove confusable query string on post as copy  https://review.openstack.org/17892708:23
kota_val2k: you mean you want to get the Swift log? or to put a log from something into Swift?08:26
hoval2k: kota_: I don't think there is a way to log on.08:27
kota_val2k: if you want to get the swift log, perhaps you could setup rsyslog like as SAIO setup08:27
kota_val2k: documentation is here, http://docs.openstack.org/developer/swift/development_saio.html08:28
kota_ah, something like 'log in' (i.e. making a session?)08:28
*** exploreshaifali has joined #openstack-swift08:31
hoval2k: kota_: sorry, i may have mis-understand.08:31
val2kkota_: ho: yes I meant "log in", sorry08:32
kota_ho: no worries, I'm not sure, too :P08:32
*** csmart has quit IRC08:33
kota_ho: you're correct!08:33
kota_val2k: if you allow GUI, I guess you can use some browser base tool like as "swift-django-browser"08:34
kota_val2k: link, https://github.com/cschwede/django-swiftbrowser08:35
val2kthank you kota_ I will check this!08:35
kota_val2k: there are some related products for swift08:36
kota_val2k: wait a while08:36
*** csmart has joined #openstack-swift08:36
kota_val2k: http://docs.openstack.org/developer/swift/associated_projects.html08:37
kota_val2k: I don't know which one is suitable (or no one) for you :\08:38
val2kkota_: swauth seems perfect !08:38
kota_val2k: ;)08:40
*** csmart has quit IRC08:41
*** acoles_away is now known as acoles08:45
tab___Hi there. Quick question, installing identity service and object storage swift (following instructions http://docs.openstack.org/juno/install-guide/install/yum/content/). Now i want also install dashboard Horizon - does Horizon need Messenger server to be installed? (http://docs.openstack.org/juno/install-guide/install/yum/content/ch_basic_environment.html#basics-messaging-server)08:47
*** csmart has joined #openstack-swift08:48
hoval2k: I see. You want to make a request without authentication (anonymous access)08:57
*** tellesnobrega has quit IRC08:59
val2kho: Exactly, I use Swift via Runabove and I would like not to use their shell file (used to set these env var) in my script09:02
val2kbecause these var are not persistent and I have to launch the .sh again and again09:02
*** kota_ has quit IRC09:07
hotab__: I'm not sure horizon uses messanger but keystone and swift don't need to install messanger server. messanger server useually uses for communication b/w controller node and controlled node of nova, cinder, glance and neutron.09:15
*** aix has joined #openstack-swift09:16
*** exploreshaifali has quit IRC09:23
*** tobe43 has quit IRC09:31
*** hunius has joined #openstack-swift09:41
*** kota_ has joined #openstack-swift09:54
*** kota_ has quit IRC09:54
tab___ho: ok09:55
*** hunius has quit IRC10:08
openstackgerritMerged openstack/swift: Properly re-raise exceptions in proxy_logging  https://review.openstack.org/18156610:35
hoval2k: sorry for late response. https://swiftstack.com/docs/cookbooks/swift_usage/container_acl.html may be useful.10:59
hogood night all!11:00
*** ho has quit IRC11:00
*** tamizh_geek has quit IRC11:00
*** links has joined #openstack-swift11:04
*** kota_ has joined #openstack-swift11:08
openstackgerritKota Tsuyuzaki proposed openstack/swift: Remove confusable query string on post as copy  https://review.openstack.org/17892711:12
*** val2k has quit IRC11:18
*** proteusguy has joined #openstack-swift11:23
acoleskota_: hi! what did you change ^^ ? i was just reviewing patchset 5 ;)11:28
*** km has quit IRC11:29
kota_acoles: hi,11:29
kota_acoles: just remove a inline comment to my memo.11:30
kota_acoles: sorry to confuse that :(11:30
acoleskota_: okay no problem11:31
*** tellesnobrega has joined #openstack-swift11:31
acoleskota_: oic the TODO - yes, the existing behaviour is correct11:32
kota_acoles: exactly, the reason I remove the TODO comment is what you know.11:34
acolesmakes sense :)11:35
kota_acoles: have you started to work about bug 1453807?11:41
openstackbug 1453807 in OpenStack Object Storage (swift) "Post (not as copy) to SLO manifest destroys its state as a manifest" [Undecided,New] https://launchpad.net/bugs/145380711:41
kota_acoles: if not yet, I'm willing to work it during this week.11:42
kota_acoles: is it okay for you?11:44
*** jistr is now known as jistr|class11:46
*** jistr|class is now known as jistr11:47
*** jistr is now known as jistr|class11:50
kota_well, finishing up today.11:54
kota_see you tommorow guys ;)11:55
*** kota_ has quit IRC11:55
*** jistr|class is now known as jistr11:57
*** fthiagogv has joined #openstack-swift12:05
*** zul has quit IRC12:06
acoleskota_: i am not working on it yet12:12
*** jkugel has quit IRC12:12
*** aix has quit IRC12:17
*** zul has joined #openstack-swift12:18
*** SkyRocknRoll has quit IRC12:20
*** aix has joined #openstack-swift12:28
*** bkopilov has quit IRC12:32
*** NM has joined #openstack-swift12:46
*** proteusguy has quit IRC12:55
*** zul has quit IRC12:59
*** zul has joined #openstack-swift13:03
*** jistr is now known as jistr|mtg13:04
*** jkugel has joined #openstack-swift13:07
*** jkugel1 has joined #openstack-swift13:09
*** jkugel has quit IRC13:11
*** wbhuber has joined #openstack-swift13:20
*** annegentle has joined #openstack-swift13:35
*** ppai has quit IRC13:41
*** links has quit IRC13:44
*** ppai has joined #openstack-swift13:55
*** StevenK has quit IRC14:12
*** bkopilov has joined #openstack-swift14:13
openstackgerritAlistair Coles proposed openstack/swift: Cleanup and extend end to end ssync tests  https://review.openstack.org/17937914:14
*** vinsh has joined #openstack-swift14:15
*** zul has quit IRC14:17
*** cdelatte has joined #openstack-swift14:19
*** delattec has joined #openstack-swift14:19
*** jrichli has joined #openstack-swift14:19
*** zul has joined #openstack-swift14:19
*** annegentle has quit IRC14:21
*** Gu_______ has joined #openstack-swift14:21
openstackgerritAlistair Coles proposed openstack/swift: Make SSYNC receiver return a reponse when initial checks fail  https://review.openstack.org/17783614:22
openstackgerritAlistair Coles proposed openstack/swift: Remove _ensure_flush() from SSYNC receiver  https://review.openstack.org/17783714:22
*** annegentle has joined #openstack-swift14:22
*** breitz has quit IRC14:28
*** breitz has joined #openstack-swift14:28
*** jistr|mtg is now known as jistr14:39
*** StevenK has joined #openstack-swift14:42
*** minwoob has joined #openstack-swift14:53
*** ppai has quit IRC14:54
*** NM has quit IRC14:55
*** bsdkurt1 has left #openstack-swift15:16
*** barra204 has joined #openstack-swift15:21
*** shakamunyi has joined #openstack-swift15:22
*** esker has joined #openstack-swift15:28
openstackgerritJanie Richling proposed openstack/swift: WIP - working on the encryption feature.  https://review.openstack.org/15790715:29
*** annegentle has quit IRC15:32
*** vjujjuri has joined #openstack-swift15:32
vjujjuriHi All, I am new to swift and trying to get my hands dirty with  SAIO cluster using instructions (http://docs.openstack.org/developer/swift/development_saio.html)15:33
vjujjuriI am running into a problem while bringing up proxy manger.15:33
vjujjuri sudo swift-init proxy restart15:34
vjujjuriSignal proxy-server  pid: 20985  signal: 1515:34
vjujjuriNo proxy-server running15:34
vjujjuriStarting proxy-server...(/etc/swift/proxy-server.conf)15:34
vjujjuriTraceback (most recent call last):15:34
vjujjuri  File "/usr/local/bin/swift-proxy-server", line 7, in <module>15:34
vjujjuri    execfile(__file__)15:34
vjujjuri...15:34
vjujjuri File "/home/vjujjuri/swift/swift/swift/common/memcached.py", line 55, in <module>15:34
vjujjuri    from eventlet.pools import Pool15:34
vjujjuriImportError: No module named pools15:34
vjujjuriI retried after pip install pool15:34
vjujjuristill same issue15:34
vjujjuriI see memcached running15:34
vjujjuriAny help will be greatly appreciated15:34
vjujjuri70% of my unit tests are passing but still I don't believe cluster is up and running15:35
vjujjurias I can't even do swift stat15:35
treydvjujjuri: does your system have the eventlet library installed (pip list installed | grep eventlet)?15:36
glangevjujjuri: did you get a swift all in one running first?  http://docs.openstack.org/developer/swift/development_saio.html  You'd learn a lot doing that.  When you are done, all your unit and functional tests should work.15:37
glangeoh, I read above and this is for a saio :)15:38
vjujjurivjujjuri@twinstan-wsl6:~$ sudo pip install eventlet15:39
glangeif you carefully follow the instrustions on the saio, it should work15:39
vjujjuri[dzdo] password for vjujjuri:15:39
vjujjuriRequirement already satisfied (use --upgrade to upgrade): eventlet in /usr/local/lib/python2.7/dist-packages15:39
vjujjuriRequirement already satisfied (use --upgrade to upgrade): greenlet>=0.3 in /usr/lib/python2.7/dist-packages (from eventlet)15:39
vjujjuriCleaning up...15:39
vjujjuriglange: I think I followed carefully. :)15:40
glangeok :)15:41
*** gyee has joined #openstack-swift15:41
glangewhen I first started, I had to rebuild my saio more than once before I got it right15:41
vjujjuriglange: ok. So basically I am able to bring up all other mangers15:42
vjujjuriexcept this proxy15:42
vjujjuritreyd15:42
vjujjurijujjuri@twinstan-wsl6:~$ sudo pip list installed15:43
vjujjuriUsage: pip COMMAND [OPTIONS]15:43
vjujjuripip: error: No command by the name pip list15:43
vjujjuri  (maybe you meant "pip install list")15:43
vjujjurivjujjuri@twinstan-wsl6:~$ sudo pip installed list15:43
vjujjuriUsage: pip COMMAND [OPTIONS]15:43
vjujjuripip: error: No command by the name pip installed15:43
vjujjuri  (maybe you meant "pip install list")15:43
*** zaitcev has joined #openstack-swift15:43
*** ChanServ sets mode: +v zaitcev15:43
openstackgerritAlistair Coles proposed openstack/swift: Don't ssync fragments that conflict with another fragment  https://review.openstack.org/18140715:43
openstackgerritAlistair Coles proposed openstack/swift: Add POST capability to ssync for .meta files  https://review.openstack.org/13849815:43
InAnimaTeanyone know a quick way to remove all buckets and their contents for a specific user?15:44
vjujjuriglange: so do you suggest me to repeat from the start?15:45
vjujjuriAlso glange , treyd  - If  I want to setup swift on real setup with 5 machiens, where are the best instructions? can you guys give me a link?15:46
vjujjuriAlso is there a way to disable keystone for testing purposes15:46
glangemaybe you have two versions of eventlet installed and python is pulling the wrong one?15:46
vjujjuritrying to get some perf numbers and don't really need keystone for now.15:46
vjujjuriThanks a lot for your help15:46
glangerun "python"15:47
glangeand do "import eventlet"15:47
glangeand then "eventlet.__version__"15:47
glangewhat number do you see?15:47
vjujjurihmm15:47
*** dencaval has joined #openstack-swift15:47
vjujjurivjujjuri@twinstan-wsl6:~$ python15:48
vjujjuriPython 2.7.3 (default, Dec 18 2014, 19:10:20)15:48
vjujjuri[GCC 4.6.3] on linux215:48
vjujjuriType "help", "copyright", "credits" or "license" for more information.15:48
vjujjuri>>> import eventlet15:48
vjujjuriTraceback (most recent call last):15:48
vjujjuri  File "<stdin>", line 1, in <module>15:48
vjujjuriImportError: No module named eventlet15:48
vjujjuri>>>15:48
vjujjuribut pip install shows it is good. as I pasted above.15:48
vjujjuriglange: ^15:48
*** links has joined #openstack-swift15:50
*** jistr has quit IRC15:50
treydvjujjuri: maybe pip is installing packages in the wrong location than what your python expects?15:50
glangeor pip is confused and it didn't install it at?15:51
glangeI'm at "apt-get" guy :)15:52
InAnimaTelooks like --all is what i needed15:52
vjujjurivjujjuri@twinstan-wsl6:~$ python --version15:52
vjujjuriPython 2.7.315:52
vjujjurivjujjuri@twinstan-wsl6:~$ sudo ls /usr/local/lib/python2.7/15:52
vjujjuridist-packages  site-packages15:52
vjujjurivjujjuri@twinstan-wsl6:~$ sudo pip install eventlet15:52
vjujjuri[dzdo] password for vjujjuri:15:52
vjujjuriRequirement already satisfied (use --upgrade to upgrade): eventlet in /usr/local/lib/python2.7/dist-packages15:52
vjujjuriRequirement already satisfied (use --upgrade to upgrade): greenlet>=0.3 in /usr/lib/python2.7/dist-packages (from eventlet)15:52
vjujjuriglange: I can try apt-get15:52
vjujjurilet me try that15:52
glangewell, I don't know if you should mix those two15:52
glangeI don't know if they play well together15:53
vjujjuriok;15:53
treydglange: vjujjuri: you can check your python path if you do an 'import sys' and then 'print sys.path' in a python interpreter15:53
glangethat's a good idea15:54
vjujjurivjujjuri@twinstan-wsl6:/usr/lib$ python15:56
vjujjuriPython 2.7.3 (default, Dec 18 2014, 19:10:20)15:56
vjujjuri[GCC 4.6.3] on linux215:56
vjujjuriType "help", "copyright", "credits" or "license" for more information.15:56
vjujjuri>>> import eventlet15:56
vjujjuriTraceback (most recent call last):15:56
vjujjuri  File "<stdin>", line 1, in <module>15:56
vjujjuriImportError: No module named eventlet15:56
vjujjuri>>> import sys15:56
vjujjuri>>> print sys.path15:56
vjujjuri['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PIL', '/usr/lib/python2.7/dist-packages/gst-0.10', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/python2.7/dist-packages/ubuntu-sso-client']15:56
treydyeah... /usr/local/lib vs /usr/lib15:57
vjujjuriright15:57
vjujjurihow do I add it to the path?15:57
*** mtreinish has quit IRC15:58
glangeor could you get pip to install it in a "right" place15:59
*** mtreinish has joined #openstack-swift16:00
vjujjuritreyd: glange no that did not helo16:00
vjujjurihelp16:00
vjujjurivjujjuri@twinstan-wsl6:/usr/lib$ export PYTHONPATH=$PYTHONPATH:/usr/local/lib/python2.7/dist-packages16:00
vjujjurivjujjuri@twinstan-wsl6:/usr/lib$ python16:00
vjujjuriPython 2.7.3 (default, Dec 18 2014, 19:10:20)16:00
vjujjuri[GCC 4.6.3] on linux216:00
vjujjuriType "help", "copyright", "credits" or "license" for more information.16:00
vjujjuri>>> import sys16:01
vjujjuri>>> print sys.path16:01
vjujjuri['', '/usr/lib', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PIL', '/usr/lib/python2.7/dist-packages/gst-0.10', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/python2.7/dist-pa16:01
vjujjurickages/ubuntu-sso-client']16:01
vjujjuri>>> import eventlet16:01
vjujjuriTraceback (most recent call last):16:01
vjujjuri  File "<stdin>", line 1, in <module>16:01
vjujjuriImportError: No module named eventlet16:01
treydvjujjuri: what OS are you using?  precise or trusty?16:03
vjujjurino precise116:04
vjujjuri12.0416:04
*** Gu_______ has quit IRC16:04
vjujjurisorry precise16:04
*** wbhuber has quit IRC16:05
zackmdavisvjujjuri, I think pip's list-installed-packages command is actually `pip freeze` (a contemptibly counterintuitive name), so the command to see if Pip thinks it has installed eventlet would actually be `pip freeze | grep eventlet`16:05
*** wbhuber has joined #openstack-swift16:05
*** wbhuber_ has joined #openstack-swift16:06
*** wbhuber_ has quit IRC16:06
vjujjurizackmdavis: that doesn't spit out any eventlet16:07
vjujjurivjujjuri@twinstan-wsl6:/usr/lib$ pip freeze | grep eventlet16:07
vjujjuriWarning: cannot find svn location for distribute==0.6.24dev-r016:07
vjujjuriso looks like it is not installed properly16:07
vjujjuribut pip install says everything is fine :(16:07
vjujjurivjujjuri@twinstan-wsl6:/usr/lib$ sudo pip uninstall eventlet16:09
vjujjuriUninstalling eventlet:16:09
vjujjuri  /usr/local/lib/python2.7/dist-packages/eventlet16:09
vjujjuri  /usr/local/lib/python2.7/dist-packages/eventlet-0.17.4.egg-info16:09
vjujjuriProceed (y/n)? y16:09
vjujjuri  Successfully uninstalled eventlet16:09
vjujjurivjujjuri@twinstan-wsl6:/usr/lib$16:09
vjujjurivjujjuri@twinstan-wsl6:/usr/lib$ sudo pip install eventlet -E /usr/bin/python16:09
vjujjuriDownloading/unpacking eventlet16:09
vjujjuri  Downloading eventlet-0.17.4.tar.gz (284Kb): 284Kb downloaded16:09
vjujjuri  Running setup.py egg_info for package eventlet16:09
vjujjuri16:09
vjujjuriRequirement already satisfied (use --upgrade to upgrade): greenlet>=0.3 in ./python2.7/dist-packages (from eventlet)16:09
vjujjuriInstalling collected packages: eventlet16:09
vjujjuri  Running setup.py install for eventlet16:09
vjujjuri16:09
vjujjuriSuccessfully installed eventlet16:09
vjujjuriCleaning up...16:09
vjujjurivjujjuri@twinstan-wsl6:/usr/lib$ pip freeze | grep eventlet16:09
vjujjuriWarning: cannot find svn location for distribute==0.6.24dev-r016:09
vjujjurivjujjuri@twinstan-wsl6:/usr/lib$16:09
zackmdavisvjujjuri, that is indeed puzzling16:10
*** wbhuber has quit IRC16:10
zackmdavisvjujjuri, (we notice that Pip said it uninstalled eventlet from /usr/local/lib/python2.7/dist-packages/, which was in the sys.path as expected earlier, suggesting that it's not a path issue)16:11
*** openstackgerrit_ has quit IRC16:11
vjujjurizackmdavis: it is puzzling16:13
zackmdavisvjujjuri, this StackOverflow question (found via a popular search engine) poses a similar problem http://stackoverflow.com/questions/8860731/pip-freeze-requirements-txt-error16:14
notmynamegood morning16:14
vjujjurizackmdavis: I am not worrying about the warning16:16
vjujjuriit does report other stuff16:16
vjujjuriso what I am seeing is -16:16
vjujjurifreeze is not reporting anything under local/bin16:16
charzvjujjuri: which version of pip you isntalled?16:18
vjujjurivjujjuri@twinstan-wsl6:/usr/lib$ pip --version16:19
vjujjuripip 1.0 from /usr/lib/python2.7/dist-packages (python 2.7)16:19
vjujjurivjujjuri@twinstan-wsl6:/usr/lib$16:19
*** annegentle has joined #openstack-swift16:36
*** vjujjuri has quit IRC16:37
*** mahatic has joined #openstack-swift16:38
*** annegentle has quit IRC16:41
*** NM has joined #openstack-swift16:57
*** nadeem has joined #openstack-swift16:59
*** nadeem has quit IRC17:00
*** mtreinish has quit IRC17:02
*** esker has quit IRC17:02
*** dmorita has quit IRC17:02
*** alpha_ori has quit IRC17:02
*** redbo has quit IRC17:02
*** tanee has quit IRC17:02
*** mtreinish_ has joined #openstack-swift17:02
*** redbo has joined #openstack-swift17:02
*** ChanServ sets mode: +v redbo17:02
*** mtreinish_ is now known as mtreinish17:02
*** bsdkurt has joined #openstack-swift17:03
*** esker has joined #openstack-swift17:03
*** dmorita has joined #openstack-swift17:03
*** nadeem has joined #openstack-swift17:04
*** tanee has joined #openstack-swift17:06
*** alpha_ori has joined #openstack-swift17:07
notmynametdasilva: are you going to the summit? my guess would be that you're staying home, but that's just a guess and why i'm asking :-)17:08
*** jordanP has quit IRC17:13
*** links has quit IRC17:14
*** acoles is now known as acoles_away17:17
minwoobHi. Out of curiosity, what kind of work is scheduled regarding the development of the EC libs that are available to Swift?17:19
minwoobSo far, I've learned a lot about the different EC libs that are available to Swift, from reading through some of the Trello cards dedicated to them.17:19
minwoobDoes anyone know if there is a specific Trello *page* (like the one for Swift/EC) dedicated for one or more of these libs? Perhaps not Trello, but something similar?17:19
notmynameminwoob: what do you mean? (very intentionally) swift doesn't include it's own EC implementation and uses 3rd party projects. so any development on them will be on their own terms and using tools of their own choosing17:20
minwoobnotmyname: One sec. Let me try to pull up one of those Trello tasks I was looking at.17:20
notmynameminwoob: so eg if you are looking at jerasure, you will need to coordinate with those maintainers. if you're using ISA-L, that will probably be pretty tightly controlled by intel17:21
minwoobnotmyname: It looked like some of the community members are involved with PyECLib, liberasure, etc.17:21
notmynamethat's true17:21
minwoobnotmyname: Pretty actively, too.17:21
*** harlowja has quit IRC17:22
minwoobnotmyname: And it seems that a lot of the EC backend work is going on in these libs.17:22
*** harlowja has joined #openstack-swift17:22
notmynameso swift uses pyeclib, and python binding for liberasurecode. liberasurecode is a plugable C library for EC algorithms17:22
minwoobhttps://trello.com/c/MPGkXJMC/76-pyeclib-move-the-core-ec-code-from-pyeclib-to-liberasurecode17:23
notmynameKevin is the maintainer of both jerasure and liberasurecode. Tushar (tsg in here) is involved with pyeclib and liberasurecode and ISA-L. then I know that eg NTT has their own proprietary one that also works with liberasurecode17:23
minwoobhttps://trello.com/c/O6BFDHUG/78-pyeclib-update-pyeclib-to-use-the-liberasurecode-api17:23
minwoobnotmyname: Then it seems that those backend plugins are mainly an independent effort, but a lot of active work seems to be going into PyECLib and liberasurecode, from Kevin, Tushar, and etc.17:24
notmynamethat's correct17:25
notmynameminwoob: without having your own proprietary version available, you will probably use either ISA-L or jerasure for EC in Swift. if you have something else available, then you can use it if it works with liberasurecode. AIUI that's a pretty small interface to implement17:25
*** vjujjuri has joined #openstack-swift17:25
notmynamethe small final detail is that liberasurecode also implements a very simple XOR erasure code that you could use without using any other library17:26
notmynameso that's the result of all of those trello cards you see17:26
minwoobnotmyname: I see. I just found it interesting that a lot of the major changes to PyECLib and liberasurecode seems to have happened during the EC implementation period of Swift development.17:33
minwoobnotmyname: Some that I would even characterize as structural changes.17:34
notmynameminwoob: pycelib and liberasurecode were developed for swift. they are new things themselves17:34
minwoobnotmyname: Ah, I see.17:35
*** annegentle has joined #openstack-swift17:37
*** tellesnobrega_ has joined #openstack-swift17:38
*** tellesnobrega_ has left #openstack-swift17:38
*** annegentle has quit IRC17:42
dmsimardRunning tempest tests against my Swift environment.. running into some sort of race condition, it'll get 401 unauthorized like one out of fifteen times for the tempurl tests. Turned on debug logging on the proxy logs but there's a lot to sift through.17:46
dmsimardHas anyone run into this kind of issue ?17:46
*** chlong has quit IRC17:46
dmsimardI see the HTTP 401 in the Swift proxy logs but it doesn't say why.17:47
dmsimardThis is one of the defcore required tests17:47
dmsimardSounds very race-condition-ish17:48
*** Gu_______ has joined #openstack-swift17:49
*** geaaru has quit IRC17:51
tdasilvanotmyname: hi, staying home this time :)17:54
*** Johncr1 has joined #openstack-swift17:56
Johncr1Hello17:56
Johncr1I have a question. Is item metadata case-sensitive in swift ?17:57
Johncr1and does it preserve the case when storing it  ?17:57
Johncr1Is item metadata case-sensitive in swift ? Does it preserve the case when storing it  ?17:58
*** chlong has joined #openstack-swift17:59
*** zhill has joined #openstack-swift18:02
*** fthiagogv has quit IRC18:04
hurricanerixJohncr1: The meta data key is not case sensitive when being sent to Swift. (I think they are modified with .Title() to ensure keys of differing case are treated as the same key).  As for the values, as far as I know, the values are untouched and saved with whatever case you send.18:06
*** Gu_______ has quit IRC18:08
*** Gu_______ has joined #openstack-swift18:09
*** jodah has joined #openstack-swift18:18
*** vjujjuri has quit IRC18:31
*** Johncr1 has quit IRC18:34
*** nadeem has quit IRC18:37
*** nadeem has joined #openstack-swift18:39
*** wbhuber has joined #openstack-swift18:45
*** proteusguy has joined #openstack-swift18:57
*** silor1 has joined #openstack-swift18:59
*** silor has quit IRC19:01
*** thurloat_isgone is now known as thurloat19:11
*** dencaval has quit IRC19:15
*** openstackgerrit_ has joined #openstack-swift19:16
*** aix has quit IRC19:18
*** silor1 has quit IRC19:22
*** openstackgerrit_ has quit IRC19:25
wbhubernotmyname: what are the current priorities for Erasure Coding for the next few months before the Liberty releases?  Two of the immediate ones are probably making it production ready and coming up with comprehensive performance results.19:28
wbhuber*release19:28
wbhuber(Those two are what I can think of right now.)19:28
*** annegentle has joined #openstack-swift19:31
*** shakamunyi has quit IRC19:35
*** barra204 has quit IRC19:36
*** bkopilov has quit IRC19:46
*** bkopilov has joined #openstack-swift19:49
*** shakamunyi has joined #openstack-swift19:49
*** barra204 has joined #openstack-swift19:49
pelusewbhuber, you got it!19:52
*** gyee has quit IRC19:52
notmynamewbhuber: yeah, that's a big part of it. For EC over the next few months, we want to do perf characterization and finish a few of the necessary refactorings (eg torgomatic and clayg have strong and conflicting opinions here). there are a few gaps to fill like the multi-range GET for EC20:10
notmynameand then fixing or documenting and understanding the results we get from various functional and performance tests20:10
*** thurloat is now known as thurloat_isgone20:17
*** Johncr1 has joined #openstack-swift20:20
*** logan2 has quit IRC20:33
pelusewbhuber, we've actually still got (I think) a Fri session that acoles setup to level set on those main refactoring/known efforts that notmyname just mentioned, right notmyname?20:36
notmynameboth a fishbowl and working session on thursday, plus any time needed on friday20:37
*** openstackgerrit has quit IRC20:37
*** openstackgerrit has joined #openstack-swift20:37
wbhuberpeluse: ok, I will ensure one of my guys attends those sessions to understand those efforts.20:38
peluseway cool20:38
pelusewbhuber, wrt the cluster details on the Intel side - will post those up here shortly20:39
wbhuberpeluse: sweet!20:39
peluse(not results yet, just config details)20:39
wbhuber(understood.)20:39
*** Johncr1 has quit IRC20:42
minwoobnotmyname: So has there been a rough sketch yet of what performance runs need to be done?20:57
minwoobnotmyname: For example, "run COSBench with x% distribution for reads, y% for writes, sizes ranging from 5KB to 3GB", etc.20:57
notmynameminwoob: that sounds great :-)20:58
notmynameactually that's very similar to what I've been looking at20:59
minwoobnotmyname: I see.20:59
notmynameminwoob: think of it from two perspectives: one is to functionally test all the areas. this would include doing reads/writes/updates/deletes. at different scales21:00
notmynametwo is to get understanding of production behavior21:00
notmynameso testing different sizes of objects and understanding what happens. finding the various bottlenecks (cpu? memory? netword? iops?)21:00
*** gyee has joined #openstack-swift21:01
notmynameseeing what happens when you have two nodes. seeing what happens when you have 200 nodes21:01
peluseminwoob, so we have a spreadsheet that we'll bring to the summit for discussion that has a ton of stuff in it like that (proposals for test scenarios, etc)21:01
notmynameoh cool21:01
notmynameI'm putting together a thing this coming sunday night for swift contributors and operators to get together. something small, casual. a low-key way to kick of the summit21:03
peluseyeah and once we all poke and prod at it, if it works out to make sense as sorta the 'production perf validation plan' then, well, that'd be great21:03
notmynameping me with your email if you want some details21:03
minwoobSo I see -- right now, the goal is to attempt to get as much performance data as possible over the upcoming weeks is what we've gathered :-)21:06
minwoobNothing too specific at the moment is what it sounds like.21:07
minwoobat this stage, at least.21:08
*** annegentle has quit IRC21:10
notmynameor rather, it's as specific as what you make it :-)21:11
wbhuberpeluse: that'd be great if you could post the spreadsheet for those who don't go to summit in person.21:11
*** vjujjuri has joined #openstack-swift21:12
peluseyup, will do21:13
*** gyee has quit IRC21:13
pelusewant to review it first so its not assumed to be 'community approved' - I suspect there will be lots of feedback but will make it available for sure21:13
peluseor we'll make available whatever the outcome is, maybe someone else brings somthing better...21:14
*** gyee has joined #openstack-swift21:15
*** thurloat_isgone is now known as thurloat21:24
notmynameI'm hoping to have all the details of the tests I've done so far in the community cluster, along with all the results. my goal is to have a public link to all of it21:24
*** chlong has quit IRC21:24
notmynamepeluse: is tsg going to the summit?21:25
pelusenope21:27
peluse:(21:27
notmynameack21:27
notmynamealso, :-(21:27
*** asettle has joined #openstack-swift21:30
*** vjujjuri has quit IRC21:39
MooingLemurSo it looks like it may not be easy to check the health of an arbitrary object inside of an EC storage policy, or is there an analog to swift-dispersion-report for EC policies?21:42
*** NM has quit IRC21:43
MooingLemurI haven't actually deployed an EC storage policy yet, just been reading21:43
*** annegentle has joined #openstack-swift21:50
*** vjujjuri has joined #openstack-swift21:56
peluseyeah, I don't remember dispersion-report coming up as a topic but it won't work "as is" for sure21:58
MooingLemurOn my prod cluster, I already do my own sort of emulation of dispersion-populate and dispersion-report21:59
pelusedo share :)22:00
MooingLemurit's written in perl, loads the ring structure into memory, feeds a container, then some objects, then queries all the objects that are expected to exist directly from the object nodes, and populates a db that is shown on a health report webpage22:01
*** jkugel1 has left #openstack-swift22:02
glangeyou lost us at perl :p22:02
MooingLemurhehe22:02
pelusecool, yeah was gonna ask why not use what's there?22:02
MooingLemurI use what I know22:02
MooingLemur:)22:02
MooingLemurI hadn't touched python code whatsoever until swift22:03
peluseahhh22:03
MooingLemurI mean, if I can HEAD or GET the EC fragments directly, from what I understand, I can probably determine if they're all there22:04
pelusesure, you can22:04
MooingLemureven if I don't bother to understand how to read the object... and I don't even care at that point22:04
peluseyup, was about to say - if you don't care about the orig obj its even easier22:04
MooingLemurif the object servers give me a 200, I gotta assume the fragments are good22:04
*** mahatic has quit IRC22:05
*** vinsh has quit IRC22:06
*** wasmum has quit IRC22:09
*** hurricanerix has quit IRC22:09
MooingLemuralso, I haven't seen this reported, but in 2.3.0, `swift-ring-builder object.builder set_weight d0 100 d1 100 d2 400 d3 400` does something very silly22:10
MooingLemurhttps://bpaste.net/show/41fadad1387222:10
MooingLemurI assume that's not supposed to do that :)22:10
*** hurricanerix_ has joined #openstack-swift22:11
*** wasmum has joined #openstack-swift22:16
*** wbhuber has quit IRC22:16
*** logan2 has joined #openstack-swift22:18
mattoliverauMorning22:21
notmynamehello22:21
openstackgerritpaul luse proposed openstack/swift: Remove 1 line of dead code from EC reconstructor  https://review.openstack.org/18247922:22
pelusemornin'22:22
MooingLemureh, submitted it as bug :322:24
peluseMooingLemur, cool22:25
*** esker has quit IRC22:29
mattoliverauOnly a few more days until lift off and then summit! :)22:30
*** nadeem has quit IRC22:33
peluselift off?  coming up in a rocketship?22:34
notmynamepeluse: you aren't?22:35
peluseI'm taking my invisible plane this time :)22:36
peluseborrowed it from wonder woman22:36
pelusenotmyname, do you remember anyone looking at the swift-dispersion stuff for policies in general?  Looks like (cursory look) it may just need to be made policy aware and it might work for EC22:38
notmynameIIRC dmorita was complaining about that not working for policies, and he might have been looking at it22:38
peluseOK, cool22:40
openstackgerritJanie Richling proposed openstack/swift: WIP - working on the encryption feature.  https://review.openstack.org/15790722:45
mattoliverauUs in the future (gmt+10) only take rockets.. Its the only way to fly.22:49
notmynamehmm...in addition to new swift tshirts, I'll have some extra "screaming frenzy" ones from paris. (assuming I can figure out how to get them into a suitcase)22:49
notmynamemattoliverau: how is tomorrow?22:49
*** Gu_______ has quit IRC22:49
mattoliverauIts good, bit of rain, getting colder.. I can give you the lotto numbers though :p22:50
notmynameheh. nice22:50
jrichliyou could just wear the shirts on the plane :-)  you would just be well padded22:51
jrichlinotmyname ^^22:51
notmynameme on the plane: https://www.youtube.com/watch?v=r6tlw-oPDBM22:52
jrichlilol!  ThatI guess most of them would have to be a large size in that case22:54
jrichlithat's great22:54
jrichliok, im out.  you all have fun22:54
*** jrichli has quit IRC22:54
peluseyeah, so couple of policy support hacks and dispersion stuff works with EC (well, quick and dirty basic test).  Guess I'll put something real together and throw it out there22:56
notmynamepeluse: IIRC the 2 patches I had to apply to the QA cluster to get it to work (well) were https://review.openstack.org/#/c/180795/ and https://review.openstack.org/#/c/175076/. would be good to see them land soon22:59
notmynameof course it looks like you've already commented at least on one of them with a question that hasn't been answered yet ;-)23:00
*** vjujjuri has quit IRC23:04
mattoliveraunotmyname: can't you use clayg's suit case? Re: tshirts23:04
*** km has joined #openstack-swift23:06
notmynameI'll probably just end up checking a bag for them23:06
*** chlong has joined #openstack-swift23:15
*** proteusguy has quit IRC23:15
*** esker has joined #openstack-swift23:16
*** annegentle has quit IRC23:17
*** esker has quit IRC23:22
*** kei_yama has joined #openstack-swift23:23
pelusebetter yet, make clayg wear them all on the plane23:29
pelusenotmyname, OK on my way out the door but I'll look at both of those early in the am.  I +2 the first one but need to look at changes since and would have +A the 2nd one but clayg has a #willfix comment in there so was waiting for hom to #dofix23:31
notmynamegot it. thanks23:31
peluselater...23:31
*** ho has joined #openstack-swift23:42
*** vjujjuri has joined #openstack-swift23:48
hogood morning!23:56

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!