| *** nadeem has quit IRC | 00:00 | |
| *** haomaiwa_ has quit IRC | 00:01 | |
| *** haomaiwang has joined #openstack-swift | 00:01 | |
| *** Jeffrey4l has joined #openstack-swift | 00:02 | |
| mattoliverau | notmyname: yeah not as hot as Perth :) its been a milder summer but with some hot days thrown in. But that's Melbourne ;) in fact the next week in Melbourne looks like its going to be especially mild. Looks like low to mid twenties. Tho weather in this city can change quickly and could be 40 before you know it. | 00:12 |
|---|---|---|
| *** zhill has quit IRC | 00:12 | |
| *** lcurtis_ has quit IRC | 00:12 | |
| *** ho_ has joined #openstack-swift | 00:15 | |
| ho_ | good morning! | 00:16 |
| *** arch-nemesis has quit IRC | 00:19 | |
| *** chlong has quit IRC | 00:20 | |
| *** chlong has joined #openstack-swift | 00:20 | |
| *** zhill has joined #openstack-swift | 00:21 | |
| *** zhill has quit IRC | 00:25 | |
| *** baojg has joined #openstack-swift | 00:25 | |
| *** diogogmt has quit IRC | 00:28 | |
| *** zhill has joined #openstack-swift | 00:31 | |
| *** mingdang1 has joined #openstack-swift | 00:32 | |
| *** awelleck has quit IRC | 00:37 | |
| *** chlong has quit IRC | 00:41 | |
| *** chlong has joined #openstack-swift | 00:41 | |
| *** baojg has quit IRC | 00:41 | |
| *** sgundur has left #openstack-swift | 00:42 | |
| openstackgerrit | Merged openstack/swift: Fixed docs: Title underline too short https://review.openstack.org/272189 | 00:42 |
| openstackgerrit | Merged openstack/swift: Ring Overview updates https://review.openstack.org/244412 | 00:42 |
| *** david-lyle has joined #openstack-swift | 00:42 | |
| notmyname | clayg: ring docs updated ^ ;-) | 00:43 |
| *** baojg has joined #openstack-swift | 00:46 | |
| kota_ | good morning | 00:46 |
| *** dmorita has quit IRC | 00:46 | |
| *** baojg has quit IRC | 00:46 | |
| *** chlong has quit IRC | 00:48 | |
| *** dmorita has joined #openstack-swift | 00:48 | |
| *** chlong has joined #openstack-swift | 00:48 | |
| *** m_kazuhiro has joined #openstack-swift | 00:49 | |
| *** chlong has quit IRC | 00:53 | |
| *** chlong has joined #openstack-swift | 00:53 | |
| *** pgbridge_ has joined #openstack-swift | 00:57 | |
| *** haomaiwang has quit IRC | 01:01 | |
| *** pgbridge has quit IRC | 01:01 | |
| *** haomaiwa_ has joined #openstack-swift | 01:01 | |
| *** garthb has quit IRC | 01:03 | |
| takashi | ho_, kota_: good morning | 01:03 |
| kota_ | ho_, takashi: o/ | 01:03 |
| *** mrmoje has joined #openstack-swift | 01:05 | |
| *** xiangxinyong_ has joined #openstack-swift | 01:07 | |
| ho_ | kota_: takashi: morning! | 01:08 |
| *** peterlisak has quit IRC | 01:10 | |
| *** kaleta has quit IRC | 01:11 | |
| *** kaleta has joined #openstack-swift | 01:11 | |
| *** onovy has quit IRC | 01:12 | |
| *** peterlisak has joined #openstack-swift | 01:12 | |
| *** onovy has joined #openstack-swift | 01:12 | |
| *** haomaiwa_ has quit IRC | 01:17 | |
| *** diogogmt has joined #openstack-swift | 01:24 | |
| *** mingdang1 has quit IRC | 01:25 | |
| *** kaleta has quit IRC | 01:38 | |
| *** kaleta has joined #openstack-swift | 01:38 | |
| *** peterlisak has quit IRC | 01:39 | |
| *** peterlisak has joined #openstack-swift | 01:39 | |
| *** xiangxinyong_ has quit IRC | 01:44 | |
| charz_ | Morning! kota_ | 01:52 |
| charz_ | Morning! ho_ and takashi | 01:53 |
| kota_ | charz: o/ | 01:53 |
| *** chlong has quit IRC | 01:55 | |
| *** chlong has joined #openstack-swift | 01:55 | |
| *** rminmin has joined #openstack-swift | 02:00 | |
| *** david-lyle has quit IRC | 02:05 | |
| *** baojg has joined #openstack-swift | 02:06 | |
| ho_ | charz_: morning! did you get big snowing? | 02:10 |
| charz_ | ho_: YEAH! That's REALLY big, we got snow in many places includes my parent' | 02:15 |
| *** klrmn has quit IRC | 02:15 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: CompositeRing Support https://review.openstack.org/271920 | 02:15 |
| *** mingdang1 has joined #openstack-swift | 02:16 | |
| charz_ | s/parent'/parents' house | 02:16 |
| charz_ | That's crazy. | 02:17 |
| ho_ | charz_: are you and your family safe? | 02:17 |
| charz_ | ho_: They're good! Thanks! | 02:17 |
| ho_ | charz_: good to hear it :-) | 02:18 |
| charz_ | ho_: How about weather in your places? still snowing? | 02:18 |
| ho_ | charz_: yeah, but i live in north part of japan so i have a lot of experience for snowing :-) | 02:21 |
| *** haomaiwang has joined #openstack-swift | 02:24 | |
| charz_ | ho_: cool! You can play snowboard/ski in the winter season. | 02:24 |
| *** diogogmt has quit IRC | 02:28 | |
| *** thumpba has joined #openstack-swift | 02:29 | |
| *** rminmin has quit IRC | 02:30 | |
| *** wanghua has joined #openstack-swift | 02:35 | |
| kota_ | ho_: still? | 02:47 |
| kota_ | ho_: i am starting to see patch 202411 and a bit confused about the patch. | 02:48 |
| patchbot | kota_: https://review.openstack.org/#/c/202411/ - swift - Add functional test for access control (RBAC) with... | 02:48 |
| *** chlong has quit IRC | 02:49 | |
| *** chlong has joined #openstack-swift | 02:49 | |
| kota_ | ho_: i wanna make sure you ware working on keystone's reseller admin on that patch, right? | 02:50 |
| *** baojg has quit IRC | 02:50 | |
| *** superflyy has joined #openstack-swift | 02:50 | |
| kota_ | ho_: Do we need another work for tempauth (or not)? | 02:50 |
| kota_ | ho_: test configuration in the patch told me as "Account must have a role for reseller_admin_role(keystoneauth)" but the concept of "reseller_admin" exists all over swift so I am wondering if we could have such kinds of functional test for every authentication. | 02:53 |
| kota_ | i might be missing something, though. | 02:53 |
| kota_ | and got failures with tempauth account6 which has reseller_admin :/ | 02:54 |
| kota_ | seems only keystone supported... | 02:55 |
| kota_ | ho_: what do you think of? | 02:55 |
| *** daemontool has quit IRC | 02:56 | |
| *** haomaiwang has quit IRC | 03:01 | |
| *** haomaiwang has joined #openstack-swift | 03:01 | |
| *** d0m has quit IRC | 03:03 | |
| *** dmorita has quit IRC | 03:04 | |
| clayg | notmyname: what was wrong with the ring doc the way it was!? | 03:13 |
| clayg | oh, you ment "updated" as it's now avialable at swift.openstack.org - nice | 03:14 |
| clayg | YES! http://docs.openstack.org/developer/swift/overview_ring.html#partition-replica-terminology | 03:14 |
| clayg | you people let me put that last sentence of the last paragraph in the official docs - suckers | 03:14 |
| clayg | hell you probably shouldn't have even let me merge any of it | 03:15 |
| clayg | too late now tho!!! ahahaha | 03:15 |
| *** bill_az has quit IRC | 03:16 | |
| *** diogogmt has joined #openstack-swift | 03:17 | |
| *** chlong has quit IRC | 03:17 | |
| *** chlong has joined #openstack-swift | 03:18 | |
| *** chlong has quit IRC | 03:19 | |
| *** chlong has joined #openstack-swift | 03:19 | |
| *** dmorita has joined #openstack-swift | 03:24 | |
| *** david-lyle has joined #openstack-swift | 03:25 | |
| *** baojg has joined #openstack-swift | 03:45 | |
| *** baojg has quit IRC | 03:59 | |
| *** SkyRocknRoll has joined #openstack-swift | 04:00 | |
| *** haomaiwang has quit IRC | 04:01 | |
| *** superflyy has quit IRC | 04:01 | |
| *** haomaiwa_ has joined #openstack-swift | 04:01 | |
| *** klrmn has joined #openstack-swift | 04:05 | |
| jrichli | timburke: looks like your +2 was lost to a rebase of https://review.openstack.org/#/c/259410/. it would be good to have it back - and to have another reviewer :-) | 04:06 |
| ho_ | kota_: i'm back. did you get failures for account6? i think this func test skip cases if there is no configuration for keystone (skip_if_not_v3). | 04:08 |
| kota_ | ho_: 1 sec, make sure my setup again... | 04:10 |
| *** dmorita has quit IRC | 04:13 | |
| *** sgundur has joined #openstack-swift | 04:14 | |
| kota_ | ho_: tests looks skiped but an error happened (looks like tearDown?) | 04:17 |
| *** silor has joined #openstack-swift | 04:18 | |
| notmyname | clayg: yeah. just meant that the patch had landed :-) | 04:18 |
| kota_ | i just copied test/sample.conf to /etc/swift/test.conf and remove comment out for account6 | 04:18 |
| kota_ | ho_: and then, run "nosetests test/functional/test_access_control.py" | 04:19 |
| ho_ | kota_: ./.functest also same result? | 04:20 |
| ho_ | kota_: i will create own env and check the behavior. | 04:20 |
| kota_ | wait, some server process seems not running | 04:20 |
| kota_ | probably my bad | 04:20 |
| kota_ | ho_: i tried again, finished successfuly, sorry | 04:21 |
| ho_ | kota_: nice! | 04:22 |
| kota_ | ho_: i mean all of rbac skipped | 04:22 |
| kota_ | ok, that's runnable. I need keystone setup for them, though. | 04:23 |
| ho_ | kota_: thanks! i answered your question. this test is only for keystone now. | 04:23 |
| ho_ | kota_: to run with tempauth, it is necessary to adapt the cases for it. | 04:24 |
| *** sgundur has left #openstack-swift | 04:24 | |
| kota_ | ho_: all right | 04:24 |
| notmyname | goal for tonight: finish my LCA presentation on erasure codes | 04:28 |
| ho_ | notmyname: can i read it after your presentation in LCA? | 04:30 |
| notmyname | ho_: it should be recorded (and mattoliverau said it will actually be live-streamed) | 04:30 |
| ho_ | notmyname: cool!!! | 04:30 |
| kota_ | ho_: perhaps, it requires keystone v3 too? | 04:36 |
| ho_ | kota_: yep | 04:38 |
| ho_ | kota_: ^^^ i don't understand how to write the underscore... | 04:38 |
| kota_ | oh, god. my script to setup for swift3 is for v2.0 | 04:39 |
| ho_ | kota_: v2 is not recommended from swift point of view. it's good change to shift v3 :-) | 04:40 |
| ho_ | kota_: s/change/chance/ | 04:40 |
| kota_ | ho_: hehe :/ | 04:41 |
| kota_ | ho_: and it's good to write down the rbac tests are for v3 specific as well as fourth test account :/ | 04:42 |
| ho_ | kota_: do you use vsaio? | 04:42 |
| kota_ | ho_: yes, but older one because i forked it to use my AWS setup. | 04:43 |
| ho_ | kta | 04:43 |
| *** mrmoje has quit IRC | 04:43 | |
| ho_ | kota_: i updated vsaio to vsaio with keystone and eranrom has a good script to setup keystone. https://gist.github.com/eranrom/7c4c398ca4b7bb196fed | 04:44 |
| kota_ | i know | 04:44 |
| ho_ | kota_: ok. just fyi :-) | 04:44 |
| kota_ | i have subsription of github | 04:44 |
| *** mrmoje has joined #openstack-swift | 04:45 | |
| kota_ | ho_: but, thanks | 04:46 |
| ho_ | kota_: no i have to say "thanks for the review" :-) | 04:47 |
| *** stevemar has quit IRC | 04:47 | |
| timburke | jrichli: good call. i've been meaning to circle back on that | 04:51 |
| *** diogogmt has quit IRC | 04:56 | |
| *** stevemar_znc has joined #openstack-swift | 04:57 | |
| *** haomaiwa_ has quit IRC | 05:01 | |
| *** haomaiwang has joined #openstack-swift | 05:01 | |
| *** dmorita has joined #openstack-swift | 05:02 | |
| timburke | kota_: would it be worth having two keystone functests? one using v2 on an old version of keystone, the other using v3 and tip of master. or maybe just most recent release? not sure i like the idea of depending on keystone's stability on master... | 05:06 |
| *** cebruns has quit IRC | 05:06 | |
| kota_ | timburke: +100! | 05:07 |
| *** dmorita has quit IRC | 05:07 | |
| *** cebruns has joined #openstack-swift | 05:08 | |
| timburke | i definitely like the idea of checking that both versions work | 05:12 |
| timburke | anyway, i should be paying attention to my wife. g'night! | 05:16 |
| ho_ | timburke: kota_: difference b/w v2/v3 is a domain concept support and this difference takes care of keystoneauth so if we make tests for v3, it also can cover swift (server side) behavior | 05:26 |
| ho_ | timburke: good night! | 05:26 |
| kota_ | timbirke: have a good night ;) | 05:27 |
| *** nadeem has joined #openstack-swift | 05:36 | |
| notmyname | ho_: here's my slides (just finished v1 of them) http://d.not.mn/lca2016_ec_in_swift.pdf | 05:38 |
| openstackgerrit | Tim Burke proposed openstack/swift: Add delete markers to versioned_writes middleware https://review.openstack.org/214922 | 05:38 |
| ho_ | notmyname: thanks! | 05:39 |
| notmyname | ho_: like most of my presentation decks, this is supporting material rather than primary material. there will be a lot of words said around all the pictures | 05:39 |
| notmyname | but you should be able to get the basic idea, I think | 05:40 |
| notmyname | (also, because you know swift very well ;-) | 05:40 |
| ho_ | notmyname: i know your style and yeah i would like to get basic idea. btw the cat is so cute :-) | 05:41 |
| notmyname | heh | 05:41 |
| notmyname | ho_: I've got a coworker who gets mad at me (jokingly) about my presentation style. he asks me, "you gave a presentation on this topic. can I see your slides, because I'm building something similar for someone". then I give him like one big, full-screen picture, and he's like, "what do i do with this?!" | 05:42 |
| timburke | notmyname: what's the -1 on the graphs? 1MB segment size? | 05:42 |
| *** trifon has joined #openstack-swift | 05:43 | |
| notmyname | timburke: the graphs? on the math-y part? | 05:43 |
| ho_ | notmyname: lol. | 05:43 |
| kota_ | ho_: it seems done to setup keystone v3 in my env, but still rbac skipped. | 05:43 |
| timburke | performance graphs | 05:43 |
| notmyname | oh | 05:43 |
| kota_ | ho_: any thought? | 05:43 |
| timburke | i suppose i should have said *-1 | 05:44 |
| timburke | or xxx-1 | 05:44 |
| notmyname | timburke: yes. those were all 1MB segment sizes. (these are exactly the same things presented in tokyo with peluse_ on EC) | 05:44 |
| *** nadeem has quit IRC | 05:45 | |
| notmyname | timburke: it's the info doug produced with tests in our office lab | 05:45 |
| timburke | cool. just checking. bunch of labels for different EC modes, wanted to confirm what they meant | 05:45 |
| notmyname | yeah. refer back to "supporting info. john will say a lot of words" | 05:45 |
| timburke | now i'm really going | 05:45 |
| notmyname | l-) | 05:45 |
| notmyname | me too. slides done enough that I need some wife time before I leave for 10 days ;-) | 05:45 |
| notmyname | good night, all | 05:45 |
| ho_ | kota_: did you comment out for all account in sample.conf? | 05:46 |
| kota_ | notmyname: good night :-) | 05:46 |
| kota_ | ho_: no? | 05:46 |
| kota_ | ho_: just account6 | 05:46 |
| kota_ | ho_: is it neccesarry to setup all accunts? | 05:47 |
| ho_ | kota_: yep. the condition is L1068/L1078 in https://review.openstack.org/#/c/202411/20/test/functional/test_access_control.py | 05:48 |
| ho_ | notmyname: good night! | 05:48 |
| kota_ | which line? | 05:49 |
| kota_ | ho_:^^ | 05:49 |
| ho_ | kota_: L1069 and L 1078 | 05:49 |
| kota_ | ho_: gotcha | 05:50 |
| ho_ | kota_: L75-89 is a list for users | 05:51 |
| kota_ | i see | 05:52 |
| kota_ | getting understand overview | 05:53 |
| *** klrmn has quit IRC | 05:54 | |
| kota_ | ho_: do you have a plan to add docs like "access control tests requires all accounts through account1-6"? | 05:55 |
| kota_ | ho_: otherwise, newbees seems to fall down same hole with me by just enabling account6 for access control test. | 05:56 |
| kota_ | ho_: exactly, if reading code, we can find the condition but... not all. | 05:57 |
| kota_ | not all developpers (or operaters, newbees) | 05:57 |
| kota_ | just imho :P | 05:59 |
| ho_ | kota_: i see. this explanation is only in test_access_control.py. it's better to have the explanation in sample.conf. | 06:00 |
| *** haomaiwang has quit IRC | 06:01 | |
| kota_ | ho_: +1 | 06:01 |
| *** haomaiwa_ has joined #openstack-swift | 06:01 | |
| ho_ | kota_: so far i don't have any plan for it but i will add it | 06:01 |
| ho_ | kota_: s/don't/didn't/ | 06:02 |
| kota_ | thx :-) | 06:02 |
| kota_ | now i'm setting up more users for testing =v= | 06:03 |
| *** zarric has joined #openstack-swift | 06:03 | |
| *** dmorita has joined #openstack-swift | 06:04 | |
| *** ChubYann has quit IRC | 06:05 | |
| zarric | Hi Stackers, can I ask your help? Is it possible for swift to have one openstack user read-only access for the container while another user can do read-write(both users under same tenant)? | 06:06 |
| kota_ | zarric: i think so | 06:08 |
| kota_ | zarric: docs is here, http://docs.openstack.org/developer/swift/overview_auth.html | 06:08 |
| *** dmorita has quit IRC | 06:08 | |
| *** trifon has quit IRC | 06:08 | |
| kota_ | but might depends on auth system. | 06:08 |
| *** trifon has joined #openstack-swift | 06:09 | |
| kota_ | The included TempAuth also has the concept of admin and non-admin users within an account. Admin users can do anything within the account. Non-admin users can only perform operations per container based on the container’s X-Container-Read and X-Container-Write ACLs. | 06:09 |
| kota_ | that is a line of the doc. | 06:09 |
| zarric | @kota, thanks for help. I ever tried with set the acl for the container to its owner tenant: self, the other user seems can still delete that container | 06:09 |
| kota_ | non-admin user cannot do anything if he is not allowed to operate on specific container. | 06:10 |
| zarric | yes, the auth system I am using is Keystone | 06:10 |
| zarric | so for non-admin users can view the container objects? | 06:10 |
| zarric | if non-admin and admin both in the same tenant.. sorry for the silly question, I am new to swift. | 06:11 |
| kota_ | i think, "cannot" but wait a minutes to make sure the code | 06:12 |
| kota_ | looks like it's impossible to see both object-list and objects themselvs for non-admin user. | 06:15 |
| kota_ | confirmed by tempauth, though. | 06:15 |
| kota_ | sorry i'm not so familier with keystone. | 06:16 |
| zarric | thanks. I will try the keystone | 06:16 |
| *** trifon has quit IRC | 06:18 | |
| kota_ | zarric: you could see around http://docs.openstack.org/developer/swift/overview_auth.html#access-control-using-keystoneauth | 06:19 |
| kota_ | By default the only users able to perform operations (e.g. create a container) on an account are those having a Keystone role for the corresponding Keystone project that matches one of the roles specified in the operator_roles option. | 06:19 |
| *** trifon has joined #openstack-swift | 06:19 | |
| kota_ | zarric: you could try some oparation with an user not in a projecto which have perator_roles. | 06:20 |
| zarric | looks the normal user can delete the container created by other user if they are under the same project | 06:21 |
| zarric | oh, let me check the roles | 06:21 |
| zarric | a question, if the user's role is not belonging to any operator role, is he expect to see everything except creation? | 06:23 |
| kota_ | not sure, but i didn't think so. | 06:24 |
| kota_ | instuitively, an user which doesn't have operator role means equals to non-admin user... | 06:25 |
| kota_ | ho_: thought? | 06:26 |
| ho_ | kota_: oh, keystone related questions! | 06:27 |
| kota_ | ho_: you know in detail rather than me :-) | 06:28 |
| ho_ | kota_: ok, i will check questions. | 06:28 |
| *** thumpba has quit IRC | 06:29 | |
| ho_ | zarric: "looks the normal user can delete the container created by other user if they are under the same project". i think it depends on roles which user has. | 06:30 |
| *** trifon has quit IRC | 06:30 | |
| zarric | ho_, thank you. The non-admin role not list in operator roles can view only, correct? | 06:31 |
| *** mrmoje has quit IRC | 06:31 | |
| ho_ | zarric: wait a minute. i'm in meeting :-) | 06:32 |
| *** mrmoje has joined #openstack-swift | 06:33 | |
| *** thumpba has joined #openstack-swift | 06:34 | |
| *** trifon has joined #openstack-swift | 06:34 | |
| zarric | tried with creation a new role (not list in operator), and user with that role seems would not able to view or create containers.. Will try grant the ACL | 06:35 |
| *** gyee has quit IRC | 06:37 | |
| ho_ | zarric: it's not possible to view info. i think. | 06:38 |
| zarric | ho_: even by certain configuration to allow to that tenant and user? | 06:40 |
| zarric | hmm...that seems to be not flexible, do you know any other approach that can make certain container read-only for the user under same account | 06:41 |
| ho_ | zarric: sorry, i would like to say "it's not possible to view ..." for " The non-admin role not list in operator roles can view only, correct?" | 06:41 |
| ho_ | zarric: if you use container acl, non-admin role can view container and objects. | 06:42 |
| zarric | ok, let me try. thank you | 06:44 |
| zarric | ho_: btw, do you know if there is some permission settings that can disallow the operator role to delete the container within the swift account | 06:45 |
| zarric | ho_: I tried to use acl swift post "bbb" -r "6e5361c5a89f4841aeaba7d19637fee3:eb16568e11424cb1880f2d304feb579a,.rlistings" in the operator role to grant the non-admin role ,but the non-admin role fail to get the container name "bbb" | 06:48 |
| kota_ | my openstack client doesn't support --domain option... | 06:52 |
| *** rcernin has joined #openstack-swift | 06:53 | |
| kota_ | oh my. the newest 2.0.0 still doesn't have. | 06:54 |
| ho_ | zarric: i'm not familier with swiftclient but it should be worked exectes swift post "bbb" -r "<project-id>:*,.rlistings" with operator_role then access the container/object with other users in same project | 06:55 |
| zarric | ho_: I tried that either, but seems not work. If they are in different project(account), that seems work | 06:59 |
| *** haomaiwa_ has quit IRC | 07:01 | |
| *** haomaiwang has joined #openstack-swift | 07:01 | |
| *** mrmoje has quit IRC | 07:03 | |
| ho_ | zarric: can i see your container setting (X-Container-Read after you set acl)? | 07:03 |
| kota_ | ho_: sorry during answering, do you know how to set domain to keystone? | 07:04 |
| *** mrmoje has joined #openstack-swift | 07:05 | |
| *** thumpba has quit IRC | 07:05 | |
| kota_ | ho_: openstack client doesn't look to support --domain option used in https://gist.github.com/eranrom/7c4c398ca4b7bb196fed | 07:05 |
| *** dmorita has joined #openstack-swift | 07:06 | |
| *** thumpba has joined #openstack-swift | 07:06 | |
| kota_ | so i stuck at account4 setting... | 07:06 |
| ho_ | kota_: i usually use cURL for it. few weeks ago I can not setup keystone with openstack-client so I switched to cURL | 07:07 |
| kota_ | ho_: meh, ok. thx | 07:07 |
| *** thumpba has quit IRC | 07:09 | |
| *** dmorita has quit IRC | 07:10 | |
| ho_ | kota_: you are welcome! | 07:11 |
| zarric | hi ho_, sorry for the delay. DEBUG:swiftclient:REQ: curl -i http://10.113.16.71:8080/v1/AUTH_6e5361c5a89f4841aeaba7d19637fee3/bbb -X POST -H "Content-Length: 0" -H "X-Auth-Token: 3a6e7da84f4b4ea386b3144145d5b0d8" -H "X-Container-Read: 6e5361c5a89f4841aeaba7d19637fee3:*,.rlistings" | 07:12 |
| zarric | DEBUG:swiftclient:RESP STATUS: 204 No Content | 07:12 |
| ho_ | zarric: thanks. can i have head for the container? | 07:13 |
| zarric | NFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): 10.113.16.71 | 07:14 |
| zarric | DEBUG:requests.packages.urllib3.connectionpool:"HEAD /v1/AUTH_6e5361c5a89f4841aeaba7d19637fee3/bbb HTTP/1.1" 204 0 | 07:14 |
| zarric | DEBUG:swiftclient:REQ: curl -i http://10.113.16.71:8080/v1/AUTH_6e5361c5a89f4841aeaba7d19637fee3/bbb -I -H "X-Auth-Token: a829c3717cf64bc69a571d3c5f3eab59" | 07:14 |
| zarric | DEBUG:swiftclient:RESP STATUS: 204 No Content | 07:14 |
| zarric | DEBUG:swiftclient:RESP HEADERS: [('Content-Length', '0'), ('X-Container-Object-Count', '0'), ('Accept-Ranges', 'bytes'), ('X-Container-Read', '6e5361c5a89f4841aeaba7d19637fee3:*,.rlistings'), ('X-Storage-Policy', 'Policy-0'), ('X-Container-Bytes-Used', '0'), ('X-Timestamp', '1453790251.80519'), ('X-Trans-Id', 'tx5c33018ad675460d8827f-0056a71cc6'), ('Date', 'Tue, 26 Jan 2016 07:14:15 GMT'),... | 07:14 |
| zarric | ...('Content-Type', 'text/plain; charset=utf-8')] | 07:14 |
| zarric | looks the format is a mess...sorry | 07:15 |
| *** cbartz has joined #openstack-swift | 07:15 | |
| ho_ | zarric: then you executed get a object in the container with user who belongs to same project. right? | 07:16 |
| ho_ | zarric: format is no problem. i could check ('X-Container-Read', '6e5361c5a89f4841aeaba7d19637fee3:*,.rlistings') | 07:17 |
| zarric | yes, the 403 is raise if I use another user in this project. | 07:18 |
| zarric | ClientException: Account GET failed: http://10.113.16.71:8080/v1/AUTH_6e5361c5a89f4841aeaba7d19637fee3?format=json 403 Forbidden [first 60 chars of response] <html><h1>Forbidden</h1><p>Access was denied to | 07:18 |
| zarric | we are using kilo code base, does it matter? | 07:19 |
| ho_ | zarric: the code is not change from kilo to master. it's not a problem. maybe problem is my understanding. let me check the code. | 07:23 |
| ho_ | zarric: sorry, did you execute get account or get object? | 07:29 |
| ho_ | zarric: i checked the code this case (get object) should work with the line (https://github.com/openstack/swift/blob/master/swift/common/middleware/keystoneauth.py#L389) | 07:29 |
| zarric | hi ho_, the file can be downloaded ... | 07:31 |
| ho_ | zarric: does it mean worked as expected? | 07:32 |
| zarric | oh, looks swift list will fail, but swift list bbb can be success | 07:33 |
| ho_ | zarric: :-) | 07:33 |
| zarric | thanks ho_, do you know if any way to restrict the operator role to delete some thing | 07:34 |
| *** dmorita has joined #openstack-swift | 07:35 | |
| ho_ | zarric: i think swift doesn't have it. but in the future you can control the behavior per methods. maybe it solves your problem. | 07:37 |
| zarric | ok, is there a BP in place? if convenient, maybe you can share to me? | 07:38 |
| *** dmorita has quit IRC | 07:39 | |
| ho_ | zarric: this patch is abandoned but i will restore soon: https://review.openstack.org/#/c/149930/ | 07:41 |
| *** zaitcev has quit IRC | 07:42 | |
| zarric | Thank you ho_. Wish it be accepted soon | 07:45 |
| ho_ | zarric: thanks! | 07:46 |
| *** arnox has joined #openstack-swift | 07:48 | |
| *** mrmoje has quit IRC | 07:53 | |
| *** mrmoje has joined #openstack-swift | 07:55 | |
| kota_ | gave up, couldn't set available domain for my keystone... | 07:55 |
| ho_ | oh, no.... | 07:56 |
| ho_ | kota_: this might help you (https://github.com/osanai-hisashi/vagrant-swift-all-in-one/blob/keystone-auth/scripts/register_keystone_data.sh) | 07:57 |
| *** zhill has quit IRC | 07:59 | |
| *** haomaiwang has quit IRC | 08:01 | |
| *** haomaiwang has joined #openstack-swift | 08:01 | |
| *** hseipp has joined #openstack-swift | 08:08 | |
| *** thumpba has joined #openstack-swift | 08:08 | |
| *** thumpba has quit IRC | 08:09 | |
| *** rledisez has joined #openstack-swift | 08:11 | |
| *** geaaru has joined #openstack-swift | 08:23 | |
| *** mrmoje has quit IRC | 08:24 | |
| *** mrmoje has joined #openstack-swift | 08:26 | |
| *** dmorita has joined #openstack-swift | 08:47 | |
| *** natarej_ has joined #openstack-swift | 08:48 | |
| *** natarej has quit IRC | 08:49 | |
| *** jmccarthy has quit IRC | 08:49 | |
| *** jmccarthy has joined #openstack-swift | 08:50 | |
| *** dmorita has quit IRC | 08:51 | |
| *** haomaiwang has quit IRC | 09:01 | |
| *** haomaiwa_ has joined #openstack-swift | 09:01 | |
| *** joeljwright has joined #openstack-swift | 09:07 | |
| *** ChanServ sets mode: +v joeljwright | 09:07 | |
| *** jistr has joined #openstack-swift | 09:19 | |
| *** trifon has quit IRC | 09:27 | |
| *** trifon has joined #openstack-swift | 09:29 | |
| *** trifon has quit IRC | 09:36 | |
| *** trifon has joined #openstack-swift | 09:40 | |
| *** jordanP has joined #openstack-swift | 09:42 | |
| *** wer has quit IRC | 09:49 | |
| *** wer has joined #openstack-swift | 09:49 | |
| *** dmorita has joined #openstack-swift | 09:59 | |
| *** haomaiwa_ has quit IRC | 10:01 | |
| *** haomaiwang has joined #openstack-swift | 10:01 | |
| *** dmorita has quit IRC | 10:03 | |
| *** silor has quit IRC | 10:04 | |
| *** jamielennox is now known as jamielennox|away | 10:15 | |
| *** jamielennox|away is now known as jamielennox | 10:25 | |
| *** aix_ has quit IRC | 10:32 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: CompositeRing Support https://review.openstack.org/271920 | 10:32 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: WIP: Support EC duplication factor at reconstructor https://review.openstack.org/237867 | 10:40 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: WIP: PUT/GET path for global ec cluster https://review.openstack.org/219165 | 10:40 |
| *** ho_ has quit IRC | 10:42 | |
| *** acoles_ is now known as acoles | 10:47 | |
| *** SkyRocknRoll has quit IRC | 10:49 | |
| *** dmorita has joined #openstack-swift | 10:51 | |
| *** dmorita has quit IRC | 10:55 | |
| *** haomaiwang has quit IRC | 11:01 | |
| *** haomaiwang has joined #openstack-swift | 11:01 | |
| *** chlong has quit IRC | 11:05 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Merge branch 'master' into feature/crypto https://review.openstack.org/272201 | 11:05 |
| *** chlong has joined #openstack-swift | 11:10 | |
| *** admin0 has joined #openstack-swift | 11:12 | |
| admin0 | any devs around :) | 11:12 |
| admin0 | i got a question | 11:12 |
| admin0 | if I add 1GB to a running swift cluster, does it mean shuffling around 1 gb to balance the 1gb added, or multiple GBs | 11:12 |
| *** aix_ has joined #openstack-swift | 11:13 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Fix handling of "Permission Denied" error from NamedTemporaryFile function https://review.openstack.org/270607 | 11:28 |
| *** km has quit IRC | 11:28 | |
| *** silor has joined #openstack-swift | 11:35 | |
| *** zmielna has joined #openstack-swift | 11:37 | |
| *** admin0 has quit IRC | 11:39 | |
| *** kei_yama has quit IRC | 11:41 | |
| *** m_kazuhiro has quit IRC | 11:41 | |
| *** admin0 has joined #openstack-swift | 11:49 | |
| *** haomaiwang has quit IRC | 12:01 | |
| *** haomaiwa_ has joined #openstack-swift | 12:01 | |
| *** dmorita has joined #openstack-swift | 12:03 | |
| *** mingdang1 has quit IRC | 12:07 | |
| *** dmorita has quit IRC | 12:07 | |
| *** daemontool has joined #openstack-swift | 12:14 | |
| *** admin0 has quit IRC | 12:19 | |
| *** admin0 has joined #openstack-swift | 12:24 | |
| admin0 | f I add 1GB to a running swift cluster, does it mean shuffling around 1 gb to balance the 1gb added, or multiple GBs | 12:26 |
| tdasilva | jrichli: around? | 12:32 |
| *** mrmoje has quit IRC | 12:34 | |
| admin0 | tdasilva: are you a dev ? do you know about rebalancing ? | 12:43 |
| admin0 | my question is if I add 1GB to a running swift cluster, does it mean shuffling around 1 gb to balance the 1gb added, or multiple GB | 12:43 |
| admin0 | and if I add 1 more gb , does the reshuffeling of the same objects take place or different | 12:44 |
| admin0 | is there a chain reaction ? | 12:44 |
| *** daemontool has quit IRC | 12:55 | |
| *** mrmoje has joined #openstack-swift | 12:58 | |
| *** mingdang1 has joined #openstack-swift | 13:00 | |
| *** haomaiwa_ has quit IRC | 13:01 | |
| *** haomaiwang has joined #openstack-swift | 13:01 | |
| *** bill_az has joined #openstack-swift | 13:02 | |
| *** admin0 has quit IRC | 13:04 | |
| *** admin0 has joined #openstack-swift | 13:06 | |
| tdasilva | admin0: the amount of data movement also depends on other factors as I understand it. for example, how much data in currently in the system. Generally, yeah, as you add more capacity, there will be data movement, but I don't think it will ever be more then the amount of data added, unless you screw up the ring configuration. | 13:09 |
| *** mrmoje has quit IRC | 13:09 | |
| tdasilva | admin0: here's some good reading on capacity mgmt: https://swiftstack.com/blog/2012/04/09/swift-capacity-management/ | 13:09 |
| tdasilva | admin0: what objects get moved is really dependent on the ring | 13:10 |
| tdasilva | admin0: http://docs.openstack.org/developer/swift/overview_ring.html | 13:10 |
| *** dmorita has joined #openstack-swift | 13:15 | |
| *** haomaiwang has quit IRC | 13:18 | |
| *** dmorita has quit IRC | 13:19 | |
| *** admin0 has quit IRC | 13:42 | |
| *** mingdang_ has joined #openstack-swift | 13:43 | |
| *** mingdang1 has quit IRC | 13:43 | |
| *** jordanP has quit IRC | 13:55 | |
| *** diogogmt has joined #openstack-swift | 13:57 | |
| *** aix_ has quit IRC | 14:00 | |
| *** aix has joined #openstack-swift | 14:02 | |
| *** admin0 has joined #openstack-swift | 14:03 | |
| *** proteusguy has quit IRC | 14:04 | |
| *** MVenesio has joined #openstack-swift | 14:07 | |
| *** haomaiwang has joined #openstack-swift | 14:10 | |
| *** jordanP has joined #openstack-swift | 14:11 | |
| openstackgerrit | Merged openstack/swift: Merge branch 'master' into feature/crypto https://review.openstack.org/272201 | 14:14 |
| *** proteusguy has joined #openstack-swift | 14:16 | |
| openstackgerrit | Merged openstack/swift: Fix handling of "Permission Denied" error from NamedTemporaryFile function https://review.openstack.org/270607 | 14:17 |
| *** diogogmt has quit IRC | 14:20 | |
| *** dslevin1 has quit IRC | 14:20 | |
| *** daemontool has joined #openstack-swift | 14:24 | |
| *** bill_az has quit IRC | 14:34 | |
| *** yarkot has quit IRC | 14:35 | |
| *** mingdang_ has quit IRC | 14:38 | |
| *** dmsimard is now known as dmsimard|afk | 14:39 | |
| *** daemontool has quit IRC | 14:43 | |
| *** coreycb` is now known as coreycb | 14:44 | |
| *** sgundur has joined #openstack-swift | 14:47 | |
| *** daemontool has joined #openstack-swift | 14:47 | |
| mahatic_ | acoles: hello, are you here | 14:55 |
| mahatic_ | ?* | 14:55 |
| acoles | mahatic_: hi | 14:57 |
| mahatic_ | acoles: I was looking at this card, https://trello.com/c/v5koQkms/70-protect-against-imcomplete-read, I was wondering if you have more information on that? | 14:58 |
| mahatic_ | acoles: Not sure if it helps, but I am encountering an IncompleteRead issue with encryption (while looking at some SLO test that was failing). But I can't figure why | 14:59 |
| *** haomaiwang has quit IRC | 15:01 | |
| *** haomaiwang has joined #openstack-swift | 15:01 | |
| acoles | mahatic_: looking | 15:01 |
| *** dmorita has joined #openstack-swift | 15:02 | |
| mahatic_ | acoles: ok, thanks | 15:02 |
| *** blmartin has joined #openstack-swift | 15:04 | |
| acoles | mahatic_: that comment is on a stale review but still applies to this line https://github.com/openstack/swift/blob/feature/crypto/swift/common/middleware/encrypter.py#L115-115 | 15:06 |
| *** dmorita has quit IRC | 15:06 | |
| acoles | mahatic_: the concern is that if readchunk never returns an empty chunk then the plaintext etag is never transformed to its hexdigest. i.e. if the caller for some reason does not call readchunk to consume all the body. | 15:07 |
| mahatic_ | acoles: yeah, I did go the said review and looked at the line. Did it mean that line is expected to throw an IncompleteRead at some point? | 15:07 |
| acoles | mahatic_: no, i don't believe so. if the scenario occurred it would result in an invalid plaintext etag | 15:08 |
| acoles | mahatic_: also remember this is in the PUT path - where is your test seeing an IncompleteRead? | 15:09 |
| mahatic_ | acoles: https://github.com/openstack/swift/blob/master/test/functional/tests.py#L2709 -> if I remove "seg_info['seg_c']" from there, it throws me an IncompleteRead | 15:12 |
| *** esker has quit IRC | 15:12 | |
| mahatic_ | acoles: also, if I do the same, without encryption in pipeline, it doesn't throw an error. It works fine | 15:12 |
| *** aix has quit IRC | 15:12 | |
| *** esker has joined #openstack-swift | 15:13 | |
| *** aix has joined #openstack-swift | 15:13 | |
| mahatic_ | acoles: I am not sure as to when/why an IncompleteRead would occur | 15:13 |
| *** esker has quit IRC | 15:14 | |
| *** zmielna has quit IRC | 15:14 | |
| *** esker has joined #openstack-swift | 15:14 | |
| acoles | mahatic_: what throws the error? which func test and where? that line is in common setup for a bunch of slo tests | 15:15 |
| mahatic_ | acoles: ah sorry, hold on | 15:15 |
| mahatic_ | acoles: https://github.com/openstack/swift/blob/master/test/functional/tests.py#L2804 | 15:16 |
| acoles | mahatic_: ok, but is the error raised in the test client, or in the decrypter middleware, or proxy? | 15:19 |
| acoles | is it in a log file or in the func test error ? | 15:19 |
| mahatic_ | acoles: test client. It is a func test error. Let me paste that | 15:20 |
| acoles | mahatic_: back in 5 mins | 15:20 |
| mahatic_ | acoles: sure, thanks for looking. Here it is: http://paste.openstack.org/show/485015/ | 15:21 |
| *** lcurtis has joined #openstack-swift | 15:22 | |
| *** esker has quit IRC | 15:22 | |
| pchng | mahatic_: I think the Swift server runs into the same or similar error I was getting when that test line is commented out. Let me post some server logs when I commented out that test line | 15:25 |
| pchng | This is what I see from the Swift logs when I comment out that line in tests.py to remove seg_info['seg_c'] from the ranged-submanifest: | 15:26 |
| pchng | http://paste.openstack.org/show/485018/ | 15:26 |
| mahatic_ | pchng: that happens without commenting out anything as well right? | 15:27 |
| pchng | I have not debugged yet, but it looks like the same error I was getting with that functest before, i.e. the multipart functions in utils.py have trouble parsing the multiparts/byterange encrypted response | 15:28 |
| pchng | mahatic_: Correct, I see the same error in Swift logs without commenting out the line in tests.py, but the functests still fail. Let me post the original functest error | 15:29 |
| pchng | mahatic_: Here are the functest and Swift logs for that test ( test_slo_get_ranged_submanifest) without the line (seg_c) commented out: http://paste.openstack.org/show/485019/ | 15:31 |
| mahatic_ | pchng: yes, but if you look at my latest patch (decrypt multipart mime), I do the 'read' in it and multiparts/byterange is parsed ok. but if you test this https://github.com/openstack/swift/blob/master/test/functional/tests.py#L1661 without that patch, you'd see the same error as you are now with the func test error | 15:31 |
| pchng | slightly different Swift error, but same problem, "content-range" header can't be extracted from a part of the multipart/byteranges document. | 15:32 |
| mahatic_ | pchng: but how come, I mean it is erroring out in test client itself, but logs show it's erroring in decrypter? | 15:33 |
| pchng | mahatic_: I have your latest patch I think | 15:33 |
| *** mragupat has joined #openstack-swift | 15:33 | |
| pchng | mahatic_: I think in both cases, the Swift server error causes the test to error, i.e. it doesn't even get to the point where it can assert/fail | 15:34 |
| pchng | When I debugged into this late last week (with your latest patch), it looked like utils._MultipartMimeFileLikeObject.read() was not returning the proper parts of the (encrypted) multipart document | 15:35 |
| acoles | pchng: are you pastebin's from swift master or feature/crypto? | 15:36 |
| pchng | I don't know whether utils._MultipartMimeFileLikeObject.read() has trouble finding the boundaries when there is encrypted (i.e. raw bytes) in there | 15:36 |
| pchng | acoles: From feature/crypto with mahatic_ 's patch 248377. But I haven't pulled the latest from feature/crypto that has the merge from master this morning. | 15:36 |
| patchbot | pchng: https://review.openstack.org/#/c/248377/ - swift (feature/crypto) - Decrypt multipart mime doc | 15:36 |
| acoles | mahatic_: idk for sure but your IncompleteRead might be due to an incorrect content-length header which causes the client to attempt to read more data than there is to read?? | 15:37 |
| acoles | mahatic_: pchng the other thing i notice is that func test uses a manifest that has one segment followed by sub-manifests, so the line you delete changes it to a manifest that only container sub-manifests | 15:38 |
| acoles | only has* | 15:39 |
| mahatic_ | acoles: hmm, why is it expecting more? It wasn't w/o encryption.. | 15:39 |
| acoles | bah, "the line you delete changes it to a manifest that only contains sub-manifests" | 15:39 |
| acoles | mahatic_: a bug in the patch? | 15:40 |
| pchng | acoles: That's correct. We were investigating this functest failure: test_slo_get_ranged_submanifest(), I believe Mahatic was removing that segment just to see if it was the issue. | 15:40 |
| pchng | acoles: It's kind of complicated, but that test (test_slo_get_ranged_submanifest) results in a multi-range GET that is quite peculiar | 15:40 |
| mahatic_ | acoles: :) content-length maybeed some probing | 15:41 |
| mahatic_ | may need* | 15:41 |
| pchng | acoles: And that multi-range get is the one that fails when I debugged in. Basically there is an object of length 1048576 and the multi-range get asks for "all bytes and the last byte again" | 15:41 |
| mahatic_ | acoles: it won't throw that error, if I remove anything else from that manifest except seg_info['seg_c'] , which is a bit confusing | 15:42 |
| pchng | I think the response to that ranged request (Range: bytes=0-1048575,1048575-), for some reason it isn't being parsed correctly by utils._MultipartMimeFileLikeObject.read() | 15:42 |
| pchng | I've put most of my notes on this here: https://trello.com/c/rPjrIefY/61-fix-functests-failures-in-slo-ranged-gets | 15:43 |
| *** awelleck has joined #openstack-swift | 15:43 | |
| * mahatic_ is stepping away for a bit | 15:50 | |
| *** nadeem has joined #openstack-swift | 15:50 | |
| acoles | pchng: thanks, nice notes | 15:50 |
| pchng | acoles, mahatic_ : My notes can be kind of verbose and wandering. Let me know if something's unclear and I'll clarify it. | 15:51 |
| acoles | mahatic_: pchng how about putting some logging of the multipart headers at line 253 here https://review.openstack.org/#/c/248377/10/swift/common/middleware/decrypter.py before the call to parse_content_range, since it looks like the content-range header value is bad? | 15:52 |
| acoles | or perhaps the header is missing? | 15:53 |
| pchng | acoles: It's missing, basically none of the part's headers get returned by that iterator | 15:54 |
| acoles | pchng: oic. and only when the segment content is encrypted? | 15:54 |
| pchng | Ultimately, utils._MultipartMimeFileLikeObject.read() is responsible for parsing out the "pieces" of the multipart document, and when I debugged in, it appeared that it was not returning anything for the second part of the document, which results in the calling functions returning empty headers/body | 15:55 |
| mahatic_ | acoles: pchng the value is none. But w/o encrption, it seems to take a range like this '0-0' well | 15:55 |
| pchng | acoles: It happens when encryption is enabled and this test passes without encryption I believe. | 15:55 |
| *** garthb has joined #openstack-swift | 15:55 | |
| mahatic_ | pchng: note that the same function is responsible without encryption as well: https://github.com/openstack/swift/blob/master/swift/common/request_helpers.py#L419 | 15:57 |
| mahatic_ | that is called in SLO | 15:57 |
| pchng | acoles: I am sort of wondering whether the code in utils._MultipartMimeFileLikeObject.read() has trouble finding the boundaries when there is encrypted content (i.e. "raw" bytes) in the response: https://github.com/openstack/swift/blob/feature/crypto/swift/common/utils.py#L3356 | 15:57 |
| mahatic_ | slo middleware* | 15:57 |
| mahatic_ | pchng: well, in patch 248377 , utils._MultipartMimeFileLikeObject.read() seems to parse encrypted content | 15:58 |
| patchbot | mahatic_: https://review.openstack.org/#/c/248377/ - swift (feature/crypto) - Decrypt multipart mime doc | 15:58 |
| pchng | acoles, mahatic_: Do you know if this code would have any trouble/weirdness if `input_buffer` had raw bytes in it? `boundary_pos = self.input_buffer.find(self.boundary)` | 15:58 |
| *** haomaiwang has quit IRC | 16:01 | |
| *** haomaiwa_ has joined #openstack-swift | 16:01 | |
| mahatic_ | in the submanifest, removing anything else other than seg_info['seg_c'], seems to pass the test. Which makes me wonder if there is any overlap? | 16:06 |
| *** nadeem has quit IRC | 16:10 | |
| *** dmorita has joined #openstack-swift | 16:14 | |
| mahatic_ | acoles: you were asking about checking headers, this is what happens when I tried. I tried to summarize in a comment here from line 8 - 12: http://paste.openstack.org/show/485037/ | 16:14 |
| *** dmorita has quit IRC | 16:19 | |
| *** SkyRocknRoll has joined #openstack-swift | 16:19 | |
| acoles | mahatic_: pchng so here's a hunch (well, I stared at the code a bit :)...how about if the decrypter does not read all the content of the first mime part? i.e. the first _MultipartMimeFileLikeObject is not fully read? the next instance of _MultipartMimeFileLikeObject to be yielded shares the same wsgi_input as the first, so will carry on yielding content from wherever the read got to on the first, which will not be where t | 16:20 |
| acoles | he next boundary is expected (we didn't read that far)... | 16:20 |
| acoles | so the second call to parse_mime_headers will try to parse some of the first part's body :( and presumably not find any headers?? | 16:20 |
| *** admin0 has quit IRC | 16:22 | |
| acoles | And then I see that in https://review.openstack.org/#/c/248377/10/swift/common/middleware/decrypter.py there is just one call to read() the mime part body and it only reads 65536 bytes, which doesn't seem right? | 16:22 |
| *** tongli has joined #openstack-swift | 16:22 | |
| acoles | line 257 in https://review.openstack.org/#/c/248377/10/swift/common/middleware/decrypter.py looks wrong - I'd expect to see repeated reads of the body until it is exhausted i.e. read returns '' | 16:24 |
| *** mrmoje has joined #openstack-swift | 16:24 | |
| *** cbartz has quit IRC | 16:25 | |
| *** diogogmt has joined #openstack-swift | 16:26 | |
| acoles | mahatic_: pchng its a complicated call path but at the bottom of this all there is a single wsgi input with a single read pointer and i think that when a _MultipartMimeFileLikeObject is instantiated there's a (reasonable) assumption that the read pointer has been left at the end of the last document part. | 16:26 |
| *** geaaru has quit IRC | 16:27 | |
| * acoles thinks torgomatic would know | 16:27 | |
| *** chlong is now known as chlong_zzz | 16:28 | |
| pchng | acoles, mahatic_ : i think that might be it, i.e. line 257 in https://review.openstack.org/#/c/248377/10/swift/common/middleware/decrypter.py, I think body.read() aka utils._MultipartMimeFileLikeObject.read() needs to be repeatedly called | 16:30 |
| pchng | until exhausted, otherwise the next utils._MultipartMimeFileLikeObject won't "start" at the right location | 16:31 |
| pchng | i.e. because of this: https://github.com/openstack/swift/blob/feature/crypto/swift/common/utils.py#L3432 | 16:32 |
| pchng | The next _MultipartMimeFileLikeObject is based on the current state of input_buffer, if the previous _MultipartMimeFileLikeObject.read() wasn't called enough and didn't advance all the way, I'm not sure the next _MultipartMimeFileLikeObject() starts off properly. | 16:33 |
| mahatic_ | sorry, I was away. Reading now | 16:34 |
| acoles | pchng: yeah. you said it more succinctly ;) | 16:34 |
| *** zaitcev has joined #openstack-swift | 16:35 | |
| *** ChanServ sets mode: +v zaitcev | 16:35 | |
| mahatic_ | acoles: pchng great! that sounds about (more than) correct | 16:37 |
| *** stevemar_znc is now known as stevemar | 16:38 | |
| *** arch-nemesis has joined #openstack-swift | 16:38 | |
| notmyname | good morning | 16:40 |
| mahatic_ | pchng: acoles but instead of "utils._MultipartMimeFileLikeObject.read() needs to be repeatedly called", I think it should be corrected in the way it is only reading 65536 bytes in line 257 in https://review.openstack.org/#/c/248377/10/swift/common/middleware/decrypter.py | 16:41 |
| mahatic_ | ? | 16:41 |
| pdardeau | :q | 16:42 |
| *** arch-nemesis has quit IRC | 16:43 | |
| *** arch-nemesis has joined #openstack-swift | 16:43 | |
| *** diazjf has joined #openstack-swift | 16:44 | |
| acoles | mahatic_: you will need to call it repeatedly with some reasonable chunk read size until there's no more to read. You don't want to call it once with the expected doc part length because that could be large and you don't want to read it all in one piece. | 16:46 |
| acoles | mahatic_: if you want to confirm my hunch then i guess you could change 65536 for some bigger number (> expected part length) to see if the test then passes. But that doesn't feel like the correct way to go. | 16:48 |
| mahatic_ | acoles: yeah. sure, I'll just do that to test it out. Of course, won't be the fix :) | 16:49 |
| pchng | acoles, mahatic_: Maybe make def decrypt_part_iter() yield the decryption of each chunk of body.read()? | 16:50 |
| pchng | good idea to test it first though by reading everything | 16:50 |
| *** mrmoje has quit IRC | 16:50 | |
| pchng | (i.e. put the body.read() call inside of decrypt_part_iter ) | 16:51 |
| acoles | pchng: +1 that method doesn't look like a generator right now, should probably take the doc body and yield decrypted chunks | 16:51 |
| mahatic_ | pchng: yup | 16:51 |
| mahatic_ | acoles: confirmed your hunch :) | 16:57 |
| *** trifon has quit IRC | 16:57 | |
| acoles | mahatic_: cool! | 16:58 |
| mahatic_ | pchng: ^ | 16:58 |
| pchng | mahatic_, acoles: Nice!!! | 16:58 |
| *** haomaiwa_ has quit IRC | 17:01 | |
| *** haomaiwang has joined #openstack-swift | 17:01 | |
| *** Jeffrey4l has quit IRC | 17:02 | |
| *** diazjf has quit IRC | 17:03 | |
| *** diazjf1 has joined #openstack-swift | 17:03 | |
| *** diogogmt_ has joined #openstack-swift | 17:04 | |
| *** garthb_ has joined #openstack-swift | 17:05 | |
| *** garthb has quit IRC | 17:07 | |
| *** stevemar has quit IRC | 17:07 | |
| *** diogogmt has quit IRC | 17:07 | |
| *** diogogmt_ is now known as diogogmt | 17:07 | |
| *** zaitcev has quit IRC | 17:07 | |
| *** zaitcev has joined #openstack-swift | 17:07 | |
| *** ChanServ sets mode: +v zaitcev | 17:07 | |
| *** stevemar_znc has joined #openstack-swift | 17:08 | |
| *** jordanP has quit IRC | 17:15 | |
| *** rledisez has quit IRC | 17:16 | |
| *** openstackgerrit has quit IRC | 17:17 | |
| *** openstackgerrit has joined #openstack-swift | 17:17 | |
| *** SkyRocknRoll has quit IRC | 17:23 | |
| *** jmccarthy has quit IRC | 17:24 | |
| *** jmccarthy has joined #openstack-swift | 17:24 | |
| *** stevemar_znc has quit IRC | 17:30 | |
| *** sw3 has quit IRC | 17:30 | |
| *** dmsimard|afk has quit IRC | 17:30 | |
| *** EmilienM has quit IRC | 17:30 | |
| *** aix has quit IRC | 17:30 | |
| timburke | good morning | 17:32 |
| *** dmsimard has joined #openstack-swift | 17:34 | |
| *** stevemar_znc has joined #openstack-swift | 17:34 | |
| *** EmilienM has joined #openstack-swift | 17:34 | |
| *** sw3 has joined #openstack-swift | 17:37 | |
| tdasilva | timburke: o/ | 17:37 |
| *** dmorita has joined #openstack-swift | 17:42 | |
| *** dmorita has quit IRC | 17:43 | |
| *** diogogmt has quit IRC | 17:44 | |
| *** klrmn has joined #openstack-swift | 17:51 | |
| openstackgerrit | Mahati Chamarthy proposed openstack/swift: Decrypt multipart mime doc https://review.openstack.org/248377 | 17:51 |
| mahatic_ | acoles: pchng ^^ | 17:51 |
| openstackgerrit | Merged openstack/swift: Port swift.common.utils.Timestamp to Python 3 https://review.openstack.org/237015 | 17:51 |
| *** diazjf1 has quit IRC | 17:52 | |
| mahatic_ | okay, I'm off to bed | 17:54 |
| *** nadeem has joined #openstack-swift | 17:55 | |
| *** nadeem has quit IRC | 17:56 | |
| *** dmorita has joined #openstack-swift | 17:56 | |
| *** nadeem has joined #openstack-swift | 17:57 | |
| *** diogogmt has joined #openstack-swift | 17:58 | |
| *** haomaiwang has quit IRC | 18:01 | |
| *** haomaiwang has joined #openstack-swift | 18:01 | |
| acoles | mahatic_: g'night | 18:04 |
| pchng | mahatic_: Thanks! have a good night! | 18:06 |
| *** jistr has quit IRC | 18:11 | |
| *** arnox has quit IRC | 18:15 | |
| *** mgarza has joined #openstack-swift | 18:19 | |
| *** stevemar_znc is now known as stevemar | 18:20 | |
| *** hseipp has quit IRC | 18:40 | |
| *** ChubYann has joined #openstack-swift | 18:43 | |
| *** mrmoje has joined #openstack-swift | 18:47 | |
| *** admin0 has joined #openstack-swift | 18:57 | |
| *** diazjf has joined #openstack-swift | 18:58 | |
| *** haomaiwang has quit IRC | 19:01 | |
| *** haomaiwang has joined #openstack-swift | 19:01 | |
| *** zhill has joined #openstack-swift | 19:02 | |
| sgundur | with the current repo of Swift , upon python setup.py develop getting the error - | 19:15 |
| sgundur | pkg_resources.RequirementParseError: Expected ',' or end-of-list in dnspython>=1.12.0;python_version<'3.0' at ;python_version<'3.0' | 19:15 |
| sgundur | http://paste.openstack.org/show/485062/ | 19:15 |
| sgundur | anyone come across similar issues? | 19:15 |
| admin0 | if i add 1gb storage to an existing openstack network, does it reshuffle just 1gb around or does it fire a chain reaction ? | 19:15 |
| admin0 | [7:58pm] | 19:15 |
| jrichli | sgundur: update pip pbr setuptools and make sure there is not a version owned by the OS that eclipses the more recent version | 19:19 |
| sgundur | jrichli: thanks, I ll try that | 19:20 |
| ahale | admin0: your 1GB would have a weight in proportion to other storage, and the weight would decide how many "partitions of ring" that 1GB gets assigned, then one replica of that many partitions would get moved to the new storage from across the rest of the drives in the cluster. | 19:21 |
| admin0 | ahale: if i paste you my ring privately, would you be able to guide ? | 19:21 |
| ahale | not really, its pretty late and im getting drunker by the moment! | 19:22 |
| admin0 | :) | 19:22 |
| admin0 | i need to drop 100TB to an existing ring .. and i am looking for the best options | 19:22 |
| *** MVenesio has quit IRC | 19:24 | |
| clayg | peluse_: boooooooooh | 19:24 |
| ahale | ah fun times :) hmm yeah the docs dont really talk much about best practices etc | 19:24 |
| clayg | ahale: stupid docs | 19:24 |
| ahale | i learnt recenty having unbalanced zone weights is pretty bad | 19:24 |
| ahale | clayg: yah indeed if only some ops guy would write some more | 19:25 |
| clayg | ahale: can we change the docs to "before you do anything *stupid* - you should check with ahale" | 19:25 |
| ahale | ! | 19:25 |
| * glange seconds | 19:25 | |
| ahale | i do plenty stupid :) see comment ^^ about unbalanced zones | 19:25 |
| admin0 | ahale: sent you a PM with the current weights | 19:26 |
| admin0 | yeah so my only worry is that if I drop 1 TB, it starts doing a chain reaction from everywhere | 19:26 |
| notmyname | when dealing with unbalanced zones, it's a good idea to use the latest versions of the code. clayg fixed/improved a lot of that recently | 19:27 |
| *** okdas has quit IRC | 19:27 | |
| *** okdas has joined #openstack-swift | 19:27 | |
| *** okdas has joined #openstack-swift | 19:27 | |
| notmyname | admin0: best practice is to add that gradually (in proportion to the rest of the cluster -- eg 1TB in a 10TB cluster is a big deal, not so much in a 50PB cluster) | 19:27 |
| ahale | notmyname: i experimented with master as of friday :) | 19:28 |
| notmyname | and it's awesomer right? ;-) | 19:29 |
| ahale | welll... | 19:29 |
| ahale | handoff is quite ugly :) | 19:29 |
| notmyname | but is it worse? | 19:30 |
| ahale | i think its the same | 19:30 |
| notmyname | winnig! | 19:30 |
| ahale | the maths matches my prod graphs | 19:30 |
| notmyname | ahale: you're coming to bristol, right? we need you to knock us in the head a bit ;-) | 19:31 |
| ahale | im not yet, i'd like to | 19:32 |
| *** trifon has joined #openstack-swift | 19:32 | |
| ahale | i wasnt sure if i would be in the way or anything, not being awesome at code - and then saw ireg was closed | 19:34 |
| *** diogogmt has quit IRC | 19:35 | |
| clayg | acoles: nice work on patch 270607 - a fixup and a +A that's how to get shit done | 19:37 |
| patchbot | clayg: https://review.openstack.org/#/c/270607/ - swift - Fix handling of "Permission Denied" error from Nam... (MERGED) | 19:37 |
| *** daemontool_ has joined #openstack-swift | 19:39 | |
| clayg | ahale: having unbalanced zone weights where # of zones == # of replicas is *terrible* | 19:40 |
| ahale | i would maybe go as far as TERRIBLE | 19:40 |
| clayg | notmyname: is right that the latest ring fixes were specifically going after problems that we saw as a result of unbalanced failure domains - and I think it acctually does the absolute very best possible thing - the replica_part_planner sort of enumerates what the "optimum" placement would be for a given topology even before we start putting parts out there | 19:41 |
| ahale | so my example - 5 zones. 4 have 2 nodes, z1 has 1 node. all boxes have same weight. node in z5 goes down. amount of handoff going to z1 is same as handoff to z2 despite half aggregate zone weight | 19:41 |
| clayg | so unless we change the definition of what's the best possible placement - the current code is correct (I'm not sure that can be said for any previous version of balancing) | 19:42 |
| ahale | ah good point , i should def rebalance with latest patch tho and retest | 19:42 |
| clayg | ahale: that's weird about the handoffs acctually :\ | 19:43 |
| *** daemontool has quit IRC | 19:43 | |
| notmyname | that's separate than balance | 19:43 |
| clayg | ahale: handoffs should be proportionate to the number of parts assigned in the that failure domain | 19:43 |
| notmyname | /undo | 19:44 |
| clayg | notmyname: well maybe torgomatic should enlighten us - he just looked at that code - but when he *explained* it to me it sounded like you sort of psudeo-randomly grab a part and see if the device assigned for that replica is "suitable" (i.e. in a zone you haven't used yet) | 19:46 |
| notmyname | yeah, I think that's true too | 19:46 |
| acoles | clayg: heh. once I see it my OCD makes me just have to fix it | 19:46 |
| clayg | acoles: i *love* me some acoles OCD! | 19:46 |
| notmyname | I was thinking that it's separate from balance, because maybe get_more_nodes is doing something different (eg not respecting weights or soemthing). but if it's "just" a walk around the ring, then yeah it's also what you said--proportional to the weight of that failure domain | 19:47 |
| notmyname | or maybe both things are true ;-) | 19:47 |
| acoles | clayg: btw had a 'murican colleague here today giving me 1:1 tutoring on usage of the word 'down' :P | 19:48 |
| *** diogogmt has joined #openstack-swift | 19:49 | |
| ahale | yah honestly not sure what its doing , or if all that is still valid - its not a super recent ring-builder generated ring | 19:49 |
| *** joeljwright has quit IRC | 19:50 | |
| notmyname | yeah it would different with a different ring (ie rebalance with new code should give better/different results) | 19:51 |
| clayg | acoles: I would have been down to here that, i'm feelings on using slang in irc have been fairly down lately | 19:52 |
| admin0 | trying to use swift-dispersion-populate and I get this: AttributeError: 'NoneType' object has no attribute 'rsplit' | 19:53 |
| clayg | admin0: stupid None's they *never* seem to have the attribute's you need | 19:55 |
| clayg | I could fix like 99% of of all python tracebacks if I could implement getattribute on None, sigh | 19:56 |
| notmyname | I suppose a very evil code obfuscation would be to replace None with a magicmock | 19:56 |
| clayg | notmyname: you get total hacker creds if you could make that happen __builtins__ maybe? | 19:57 |
| notmyname | yeah, the "is" might be tricky. but the result would be totally evil | 19:58 |
| notmyname | ie worth it ;-) | 19:58 |
| *** rcernin has quit IRC | 19:58 | |
| notmyname | it's nothing. it's everything. it's anythign you want. it would be very zen | 19:58 |
| admin0 | ERROR: swiftclient missing, only auth v1.0 supported | 19:59 |
| admin0 | boo | 19:59 |
| admin0 | hmm .. | 19:59 |
| *** pgbridge_ has quit IRC | 19:59 | |
| clayg | i would have swore someone fixed swift-dispersion to work with keystone (probably chmouel) | 20:00 |
| admin0 | i do not have v1.0 of the api | 20:00 |
| clayg | admin0: can you install swiftclient? | 20:00 |
| admin0 | i can try | 20:00 |
| *** haomaiwang has quit IRC | 20:01 | |
| admin0 | pip install openstack-swiftclient ? | 20:01 |
| admin0 | python:) | 20:01 |
| *** haomaiwang has joined #openstack-swift | 20:01 | |
| clayg | werd | 20:01 |
| admin0 | its installed | 20:02 |
| admin0 | now ? | 20:02 |
| admin0 | its installed in my laptop inside a virtualenv and I have auth using my credentials | 20:02 |
| admin0 | can i still do the disperson thing ? | 20:03 |
| sgundur | jrichli: thanks, that solved the issue and tests passed too! | 20:03 |
| clayg | sure give it a shot - you need your rings tho - i'm not sure 100% running from your laptop is the best plan - but who knows | 20:03 |
| clayg | admin0: you might try to do all of this in a swift-all-in-one first | 20:03 |
| clayg | admin0: i'm not 100% sure that swift-dispersion isn't an entirely flakey beast | 20:04 |
| admin0 | i installed python-swiftclient .. can’t find that command : | 20:04 |
| notmyname | sgundur: did you run into that based on the SAIO docs? if so, we should update them | 20:04 |
| admin0 | pip freeze: python-openstackclient==2.0.0 python-swiftclient==2.7.0 | 20:05 |
| *** dmorita has quit IRC | 20:05 | |
| clayg | sgundur: oh yeah we know the saio docs are broken - because I just had to do stuffto like pin pip and shit | 20:05 |
| clayg | ... on vsaio | 20:05 |
| notmyname | clayg: ah ok. | 20:06 |
| notmyname | that needs to be written down somehwere. I'm happy to do it (but I'm not going to right this moment) | 20:07 |
| openstackgerrit | Merged openstack/swift: Update AUTHORS https://review.openstack.org/271027 | 20:07 |
| admin0 | installed pip installed siwft as well .. dispersion tool still missing | 20:07 |
| admin0 | humm.. tryign hard | 20:07 |
| notmyname | worst patch ever | 20:07 |
| clayg | notmyname: yeah we talked about it in channel - maybe with pchng or zaitcev - not sure who or what happened | 20:08 |
| Zyric | Good morning | 20:08 |
| notmyname | hello Zyric | 20:08 |
| admin0 | figured out what to do .. isntalling :) | 20:09 |
| clayg | oh, or maybe the part that was busted was like swift-bench or somebody had some old version of pbr in it's requirements some how? | 20:10 |
| zaitcev | I continue to endeavour to package everything for users, so they should never need pip, so their /usr is never polluted and issues with requirements.txt do not matter. | 20:10 |
| clayg | swifterdarrell: looks like https://github.com/pypa/pip/issues/3384 has been fixed/closed testing w/ pip 8.0.2 now | 20:10 |
| clayg | zaitcev: you're doing the lords work my friend | 20:11 |
| sgundur | clayg: yeah, I had seen that pbr update soln here https://review.openstack.org/#/c/207790/ but till jrichli pointed , didn't occur | 20:11 |
| zaitcev | clayg: it may be an obsolete approach in the age of containers | 20:11 |
| *** diazjf has quit IRC | 20:11 | |
| clayg | zaitcev: oh gosh - you don't believe that | 20:11 |
| admin0 | guys, what module am I missing here ? http://pastebin.com/5iwF5LmU | 20:12 |
| clayg | zaitcev: you're just trying to be provocative | 20:12 |
| clayg | admin0: a module named internal_client obviously | 20:12 |
| zaitcev | clayg: Indeed I don't use containers for development when I hack on Swift, but users ask often about some kind of Docker images or other. Apparently it's convenient and I do not presume to know better. | 20:13 |
| clayg | admin0: so swift-dispersion-populate needs to have *swift* installed - which is a disator because you're going to have to get pyeclib installed too - which is going to need to have liberasure's .so - which on a laptop/mac you'll probably have to compile by hand | 20:13 |
| Zyric | swift | 20:13 |
| admin0 | i am going to fire up a vagrant ubuntu and try from there | 20:14 |
| clayg | admin0: vagrant you say!? http://docs.openstack.org/developer/swift/associated_projects.html#developer-tools | 20:15 |
| admin0 | :D | 20:15 |
| * clayg be big p1mpin' | 20:15 | |
| *** pgbridge_ has joined #openstack-swift | 20:16 | |
| clayg | tdasilva: you still working on the ansible swift stuff? I was starting up an unrelated project the other day and needed some provisioning foo... | 20:16 |
| clayg | tdasilva: I ended up going with chef (devil you know) but I was *really* tempted to dive into some ansible | 20:17 |
| clayg | tdasilva: I think you said you didn't have much experience with chef - but IME now one who uses it really *loves* it - which I'm not sure is the case for ansible users | 20:17 |
| ahale | clayg: if you want to live dangrously i believe openstack-ansible installs chef to isntall swift | 20:17 |
| clayg | ahale: of course it does | 20:18 |
| admin0 | slow download :) | 20:20 |
| ahale | oh no it actually looks a bit smarter now, thats good :) | 20:20 |
| *** diazjf has joined #openstack-swift | 20:31 | |
| *** dmorita has joined #openstack-swift | 20:32 | |
| admin0 | auth_user = test:tester — is this tenantID:username ? | 20:32 |
| *** rcernin has joined #openstack-swift | 20:32 | |
| onovy | hi, patch 258158 anyone? :) | 20:32 |
| patchbot | onovy: https://review.openstack.org/#/c/258158/ - swift - Keystone middleware deprecated option is_admin rem... | 20:32 |
| clayg | admin0: I think it mostly just works however swiftclient works | 20:33 |
| admin0 | http://pastebin.com/HRc5BnbP — new error | 20:34 |
| admin0 | sorry my mistake | 20:35 |
| admin0 | single space in front of the variables | 20:35 |
| admin0 | yay | 20:36 |
| clayg | lol | 20:36 |
| admin0 | the command works | 20:36 |
| admin0 | but .. is the dispersion-report immediate ? | 20:36 |
| clayg | wat? | 20:36 |
| admin0 | http://pastebin.com/1cbSTtwm | 20:37 |
| admin0 | the populate and report | 20:37 |
| admin0 | openstack contailer list — i can see that the dispersion_0_x containers are created | 20:39 |
| admin0 | how long does it take for the report to find it out ? | 20:39 |
| admin0 | oh | 20:40 |
| admin0 | oh | 20:40 |
| admin0 | i did a —debug | 20:40 |
| admin0 | its creating in public opestack, querying local disks for it | 20:41 |
| admin0 | doh! | 20:41 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Copy more headers when copying object https://review.openstack.org/272724 | 20:43 |
| clayg | MOAR | 20:43 |
| admin0 | how do I tell swift-dispursion-report in aio to not query locally, but read the /etc/swift/dispursion.conf file ? | 20:43 |
| admin0 | http://pastebin.com/JJwShLwV — the report is querying locally .. not reading the values in the conf | 20:44 |
| clayg | Usage: swift-dispersion-populate [options] [conf_file] | 20:45 |
| clayg | maybe? | 20:45 |
| admin0 | the populate worked .. it created in the right cluster | 20:45 |
| clayg | congradulations? | 20:45 |
| admin0 | not yet .. report is not working :) | 20:46 |
| admin0 | so yes, they are created .. | 20:46 |
| admin0 | but i cannot see the reports | 20:46 |
| acoles | tdasilva: this patch 272724 relates to what you are doing with versioning in patch 260179 | 20:46 |
| patchbot | acoles: https://review.openstack.org/#/c/272724/ - swift - Copy more headers when copying object | 20:46 |
| patchbot | acoles: https://review.openstack.org/#/c/260179/ - swift - decouple versioned writes from COPY | 20:46 |
| admin0 | because the report is not reading the config file and just querying locally | 20:46 |
| acoles | tdasilva: because your decoupled version copying should probably also copy more headers than master does right now | 20:47 |
| admin0 | clayg: swift-dispersion-populate did it work, but swift-dispersion-report polls locally only .. so 1/2 happy :) | 20:49 |
| *** acoles is now known as acoles_ | 20:51 | |
| *** zul has quit IRC | 20:52 | |
| admin0 | anyone can help me further .. on how to get dispersion-report on saio query/read from the conf file ? | 20:54 |
| *** diazjf has quit IRC | 20:54 | |
| clayg | did you try passing the config file to the script on the command line as an arg as described in the --help? | 20:56 |
| admin0 | clayg: yes | 20:58 |
| admin0 | but it still queries local | 20:58 |
| admin0 | http://pastebin.com/fZfX3LdD | 20:58 |
| clayg | admin0: what is the 'swift_dir' in the conf (defaults to /etc/swift) - and what is the output of swift-ring-builder <ring.builder> for the rings in that dir? | 21:00 |
| *** daemontool has joined #openstack-swift | 21:00 | |
| clayg | fwiw it's not dispersion-report that makes up those 127.0.0.1 address - those comes from rings - not sure if it's the rings you want to use or not | 21:00 |
| *** haomaiwang has quit IRC | 21:01 | |
| admin0 | clayg: this aio is inside my virtualbox .. i just passed around the URL | 21:01 |
| admin0 | oh | 21:01 |
| *** haomaiwang has joined #openstack-swift | 21:01 | |
| admin0 | so popupate works with just the 4 lines in dispersion.conf and populates remotely, but report needs access to the ring-builder | 21:02 |
| admin0 | does it also need ssh access to the machines ? | 21:02 |
| admin0 | or just the ring builder files ? | 21:02 |
| clayg | admin0: it needs to talk to the backend storage services - that's what it does - why are you using this tool? | 21:02 |
| clayg | admin0: not ssh - just the ip/ports of the storage servers - it would live in the same space as your proxies | 21:03 |
| clayg | well except probably not attached to the outward facing network | 21:03 |
| admin0 | clayg: i need to add 4 new storage-servers to a already 20+ node cluster .. so before adding, I want to use this tool to check the process | 21:03 |
| *** daemontool_ has quit IRC | 21:03 | |
| admin0 | “clayg: admin0: not ssh - just the ip/ports of the storage servers - it would live in the same space as your proxies” — this makes it much clear | 21:04 |
| clayg | ok, great - have you read http://docs.openstack.org/developer/swift/admin_guide.html?highlight=dispersion#cluster-health carefully? i feel like it gives a good overview and says "These tools need direct access to the entire cluster and to the ring files (installing them on a proxy server will probably do)." | 21:05 |
| *** diazjf has joined #openstack-swift | 21:06 | |
| mattoliverau | morning | 21:07 |
| *** daemontool_ has joined #openstack-swift | 21:07 | |
| pdardeau | good morning mattoliverau | 21:07 |
| *** esker has joined #openstack-swift | 21:08 | |
| *** silor has quit IRC | 21:10 | |
| *** daemontool has quit IRC | 21:10 | |
| admin0 | clayg: my proxy gives: ./swift-dispersion-report AttributeError: 'NoneType' object has no attribute 'rsplit' | 21:11 |
| admin0 | account = url.rsplit('/', 1)[1] | 21:13 |
| admin0 | so i was trying to use the remote keystone url as well | 21:16 |
| admin0 | i switched to localip:port where the proxy was listening to | 21:16 |
| admin0 | and now checking | 21:16 |
| admin0 | clayg: i have the error output and the ring output here: http://pastebin.com/GfbvfMU6 | 21:20 |
| *** bsdkurt has quit IRC | 21:22 | |
| admin0 | nvm .. i was doing it wrong | 21:24 |
| admin0 | the dispursion-populate was done on the saio, so the report was getting it wrong | 21:25 |
| admin0 | thank you guys .. this tool is finally working for me :) | 21:26 |
| clayg | well, the output from your -report is showing me that it's not getting account name from the auth_url correctly - i'm not sure exactly what your keystone endpoint is returning storage_url wise | 21:26 |
| clayg | it looks like maybe it's not robust to a storage_url that ends in a slash | 21:26 |
| clayg | admin0: yeah it's not terribly ergonomic - and in almost all cases whatever tracebacks you're seeing point to some error handling that needs to be improved | 21:27 |
| clayg | admin0: you could pay back the favor by filing bugs -> https://bugs.launchpad.net/swift/+filebug | 21:27 |
| *** rcernin has quit IRC | 21:28 | |
| clayg | admin0: also if there's any wording that could be strengthend in the doc (patches would be most appreciated) -> https://github.com/openstack/swift/blob/master/doc/source/admin_guide.rst#cluster-health | 21:29 |
| clayg | you can get started here -> http://docs.openstack.org/developer/swift/first_contribution_swift.html | 21:30 |
| clayg | admin0: glad you got it working | 21:30 |
| admin0 | not fully .. the command works | 21:30 |
| admin0 | report still shows the same error like i pasted, but now 5000x times | 21:31 |
| admin0 | becasue its now doing Created 5242 containers for dispersion reporting, 1m, 0 retries | 21:31 |
| clayg | admin0: if you do a `swift stat -v` does your StorageURL end in a / | 21:31 |
| admin0 | the swift tool is not installed in the proxy virtualenv : | 21:33 |
| clayg | admin0: then how are you hitting the auth v2.0 endpoint? | 21:33 |
| admin0 | in the dispersion.conf, i changed the url to point the local ip and port where swift-proxy was listening on | 21:34 |
| admin0 | coz i could not reach the v2 | 21:34 |
| clayg | admin0: I don't think that configuration is supported - it's currently expecting to get the account name from the auth details returned from the auth endpoint | 21:35 |
| admin0 | this cluster is not recent .. icehouse | 21:36 |
| clayg | admin0: I don't really know if that's relevant, the swift-dispersion-* tools are ancient | 21:38 |
| clayg | admin0: so did you find some auth creds that work against the proxy directly - or are you still authing against your keystone - i'm confused | 21:38 |
| admin0 | against the proxy directly | 21:39 |
| admin0 | but using the keystone user/pass/tenant setting | 21:39 |
| clayg | that doesn't make sense | 21:40 |
| admin0 | clayg: here is my config: http://pastebin.com/5PWGHvxV | 21:41 |
| clayg | there's no good reason for the /auth end point on your proxy to respond to anything if you're using keystone | 21:42 |
| clayg | admin0: do you have tempauth installed in the pipeline as well - or swauth or something? grep pipeline /etc/swift/proxy-server.conf | 21:42 |
| admin0 | pipeline = catch_errors healthcheck cache swift3 s3token basicauth authtoken keystone staticweb formpost tempurl proxy-logging proxy-server | 21:43 |
| clayg | admin0: so i'd guess basicauth is responding to the request with some sort of storage url - that's not stock code tho - you should debug with curl | 21:44 |
| admin0 | so for this to work correctly, i need to install swift client also | 21:44 |
| admin0 | to use v2 auth | 21:44 |
| clayg | admin0: the swift-dispersion-* tools don't really care what account you use - but you need either a valid auth v1.0 or v2.0 response | 21:45 |
| admin0 | ok | 21:45 |
| clayg | you're getting *something* from the v1.0 endpoint on your cluster (probably from whatever basicauth is) - you can probably adapt to use that | 21:45 |
| clayg | curl -i -H "X-Auth-User: <USERNAME>" -H "X-Auth-Key: <PASSWORD>" <AUTH_URL> <- fill in with what you have in your conf, see what comes back | 21:46 |
| admin0 | HTTP/1.1 401 Not Authorized Invalid credentials :D | 21:47 |
| clayg | i wonder if simpleclient's get_auth could be so stupid | 21:48 |
| admin0 | my mistake .. got somethign back: http://pastebin.com/5tRvNvmW | 21:49 |
| clayg | idk, urllib seems like it would raise a HTTPError for 401 | 21:53 |
| clayg | wtf, that's not auth v1.0 | 21:54 |
| admin0 | that is what i got back . | 21:54 |
| clayg | admin0: well go find a developer and hit them on the head - having something answer a v1.0 auth request with something other than a v1.0 auth response is an asshole move | 21:55 |
| clayg | 'https://xxx.public.url/'.rsplit('/', 1)[1] == '' | 21:55 |
| clayg | ^ explains the "missing" account | 21:55 |
| clayg | admin0: can you *use* that token it's giving you to head your account (the storage url returned from the keystone server) | 21:56 |
| admin0 | to head my account :D ? | 21:56 |
| admin0 | how does that translate to in curl :D ? | 21:56 |
| clayg | http://developer.openstack.org/api-ref-objectstorage-v1.html#showAccountMeta | 21:57 |
| admin0 | aha :) | 21:57 |
| clayg | you can get the $publicURL as the StorageURL returned from `swift stat -v` on a host that can hit your Keystone endpoint | 21:58 |
| clayg | admin0: you might could also ask whoever wrote basicauth if there's a way to get it to return a realistic storage url with an account prefix? | 21:58 |
| admin0 | i was able to do swift stat -v from my client | 22:00 |
| *** haomaiwang has quit IRC | 22:01 | |
| clayg | you might have this -> https://github.com/CloudVPS/swift-basicauth/blob/master/swift_basicauth/middleware.py ??? | 22:01 |
| clayg | maybe the storage_url_template config option is boo-berries | 22:01 |
| *** 21WAAWT67 has joined #openstack-swift | 22:01 | |
| clayg | hrmm... seems undocumnted - i don't really know what's going on with that middleware | 22:02 |
| admin0 | this is swift stat -v http://pastebin.com/q4eTYsWy | 22:03 |
| openstackgerrit | Janie Richling proposed openstack/swift: Support for http footers - Replication and EC https://review.openstack.org/237393 | 22:05 |
| clayg | ok, yeah that StorageURL is not compatible with the account parsing in the swift-dispersion-* tools | 22:06 |
| clayg | admin0: neat trick getting the account in the hostname tho | 22:06 |
| *** diazjf has quit IRC | 22:07 | |
| admin0 | hmm.. so is there any other way for me to use the swift-dispersion-tools ? | 22:07 |
| *** mrmoje has quit IRC | 22:08 | |
| clayg | admin0: you could try something like this patch to make the account parsing work with your crazy auth schema -> https://gist.github.com/clayg/61db02d7aa82d06c8f29 | 22:09 |
| clayg | I'm not even sure how to address that in the general sense :\ | 22:09 |
| *** diazjf has joined #openstack-swift | 22:10 | |
| admin0 | clayg: it helped in the sense it did not break .. result: http://pastebin.com/xmQseb12 | 22:13 |
| admin0 | how do I cleanup a populate ? | 22:14 |
| admin0 | beause i used the aio and this , i think there is this overlap | 22:14 |
| admin0 | nvm, using a loop to delete them | 22:17 |
| *** daemontool has joined #openstack-swift | 22:23 | |
| *** daemontool_ has quit IRC | 22:27 | |
| *** trifon has quit IRC | 22:29 | |
| admin0 | clayg and ahale : thank you sirs :) | 22:33 |
| clayg | notmyname: admin0's storage_url was configured to return just 'https://<account>.host.name/' - i can't even imagine how many places something has assumed storage_urls were always in the form https://storage.example.com/v1/<account> - but on the other hand there's probably just as many places we assume authenticated requests go to <storage_url>/<container>/<object> - i'm not sure which assumption is wrong - does v1.0 api requ | 22:39 |
| *** daemontool_ has joined #openstack-swift | 22:39 | |
| clayg | or does v1.0 api treat storage_url as a opaque endpoint that serves as the root of your account? | 22:40 |
| clayg | *surely* the /v1 part is part of the api spec? and it's just a happy coniencidence that most clients treat is an opaque root? | 22:41 |
| admin0 | clayg: question is for me or notmyname ? | 22:41 |
| *** daemontool has quit IRC | 22:42 | |
| notmyname | hmmm | 22:49 |
| admin0 | this might be interesting for you guys: http://pastebin.com/sSwxfiaS openstack object list container-name —debug | 22:49 |
| notmyname | there's a ton of places in the code where we split_path to get back version, account, container, obj | 22:49 |
| notmyname | and yeah. wasn't there a patch within the past year about making sure that part of the path is actually a valid string? | 22:49 |
| clayg | notmyname: oh good point - how is that even working - clients aside - swift should balk at that! | 22:50 |
| notmyname | /v1/ is very much part of the api, IMO | 22:50 |
| admin0 | i use the python-openstackclient .. just my auth .. do swift operations and it seem to work just fine with our url scheme .. i do not see the v1/v2 anywhere | 22:51 |
| admin0 | like i listed in the paste above ^^ | 22:51 |
| *** mragupat has quit IRC | 22:53 | |
| clayg | notmyname: myabe something with the cname resolver is doing path manipulation? | 22:53 |
| *** awelleck has quit IRC | 22:53 | |
| clayg | [21:57] admin0 | pipeline = catch_errors healthcheck cache swift3 s3token basicauth authtoken keystone staticweb formpost tempurl proxy-logging proxy-server | 22:55 |
| clayg | notmyname: ^ dunno pipeline looks sane except for whatever basicauth is doing? | 22:56 |
| clayg | notmyname: I found https://github.com/CloudVPS/swift-basicauth/blob/master/swift_basicauth/middleware.py but i'm not sure that's the same thing as admin0 is running | 22:56 |
| admin0 | i can look .. how/were do i verify that ? | 22:56 |
| clayg | anyway it doesn't seem to do anything with evn['PATH_INFO'] that would "fix" the path coming in w/o the /v1 | 22:57 |
| clayg | python -c 'import swift_basicauth; print swift_basicauth' maybe? | 22:57 |
| *** asettle has joined #openstack-swift | 22:58 | |
| admin0 | <module 'swift_basicauth' from '/usr/local/virtualenvs/swift-proxy/local/lib/python2.7/site-packages/swift_basicauth/__init__.pyc'> | 22:59 |
| admin0 | is the output | 22:59 |
| *** forcebanana has joined #openstack-swift | 23:00 | |
| clayg | admin0: maybe head /usr/local/virtualenvs/swift-proxy/local/lib/python2.7/site-packages/swift_basicauth/middleware.py - see if its about the same as https://github.com/CloudVPS/swift-basicauth/blob/master/swift_basicauth/middleware.py | 23:00 |
| *** 21WAAWT67 has quit IRC | 23:01 | |
| clayg | admin0: although like I said - that CloudVPS thing doesn't seem to doing everything that would be needed to make your storage_urls work the way they do - so it's sort of confusing | 23:01 |
| *** haomaiwang has joined #openstack-swift | 23:01 | |
| *** dmorita has quit IRC | 23:04 | |
| admin0 | md5sum matches :) | 23:04 |
| admin0 | its the exact | 23:04 |
| openstackgerrit | Darrell Bishop proposed openstack/swift: Allow IPv6 addresses/hostnames in StatsD target https://review.openstack.org/270991 | 23:04 |
| clayg | admin0: well i guess that answers that question - idk, maybe in icehouse it works somehow - very strange - you say the dispersion tools worked for you? You're good? | 23:05 |
| admin0 | the dispersion-populate worked | 23:05 |
| admin0 | the report did not | 23:06 |
| clayg | lol | 23:06 |
| *** dmorita_ has joined #openstack-swift | 23:06 | |
| admin0 | so yes, i have containers and objects all over the cluster .. but no clue how to check status when i add nodes , remove nodes etc :) | 23:06 |
| clayg | maybe my little diff/hack didn't quite cut the mustard | 23:06 |
| *** blmartin has quit IRC | 23:06 | |
| admin0 | it helped that it at least tried to work , but still does not find | 23:07 |
| clayg | what the delta was for report - wasn't it? | 23:07 |
| admin0 | http://pastebin.com/xmQseb12 | 23:07 |
| admin0 | yes.. it passed on and did not break looking for account | 23:07 |
| admin0 | but i think it still expects the storage path with someting /v1 etc which we do not have ? | 23:08 |
| *** km has joined #openstack-swift | 23:10 | |
| *** tongli has quit IRC | 23:11 | |
| clayg | admin0: yeah idk, direct_client path normally looks like HEAD /sdb2/996/AUTH_test/foo - maybe in your system it's different? you're probably too far off the rails | 23:12 |
| clayg | ... you could check your storage logs | 23:13 |
| torgomatic | pop quiz... when you see some stupid behavior in the code, do you (a) fix it, (b) fix it and write a test, or (c) write a test ensuring that the stupidity persists? | 23:13 |
| *** tongli has joined #openstack-swift | 23:13 | |
| torgomatic | TRICK QUESTION! I'm in here ranting, so you know the answer must have been C! | 23:13 |
| clayg | torgomatic: hehe, how stupid we talking about here? | 23:15 |
| torgomatic | this particular example is small, but obvious... you make a call to get_container_info(..., swift_source='MC') and then assert that the logged swift sources were ['GET_INFO', 'MC'] | 23:15 |
| torgomatic | and it's obvious that some plumbing is missing since that GET_INFO thing is just a default | 23:16 |
| torgomatic | it's more that I've been unhosing all the get_*_info functions and my brain is fried | 23:16 |
| clayg | torgomatic: I would have swore we had a reasonable fix somewhere? it all started with using the response directly to set the cache info in the env - instead of the cache data set in the req environment when handling the response - from there it should just be a mole-whack to remove the setting on the way out altogether until tests pass? | 23:18 |
| clayg | it was that one stupid bug something with auth - container tempurls maybe? | 23:19 |
| torgomatic | clayg: yeah, I resurrected the swift.infocache stuff out of launchpad, but that didn't move things out of GETorHEAD_base and into the get_*_info functions. I'm working on the latter now. | 23:21 |
| *** diazjf has quit IRC | 23:22 | |
| clayg | torgomatic: yeah i had a diff on there somewhere - do you have the lp bug # | 23:26 |
| torgomatic | clayg: one sec... | 23:26 |
| torgomatic | clayg: in bug 1501939 you reference bug 1449212 which has the patch in there somewhere | 23:27 |
| openstack | bug 1501939 in OpenStack Object Storage (swift) "get_info environ caching is leaky and brittle" [Medium,Confirmed] https://launchpad.net/bugs/1501939 | 23:27 |
| openstack | bug 1449212 in OpenStack Object Storage (swift) kilo "Container level temp URLs can unintentionally leak data." [Undecided,Fix committed] https://launchpad.net/bugs/1449212 | 23:27 |
| clayg | notmyname: I *love* me some wishlist bugs! I have no idea who this cat is or if we'll acctually get a patch out of the deal - but he is *on* it! lp bug #1537879 | 23:27 |
| openstack | Launchpad bug 1537879 in OpenStack Object Storage (swift) "proxy.controllers.base should use swob.HeaderKeyDict" [Wishlist,New] https://launchpad.net/bugs/1537879 - Assigned to Ben Keller (bjkeller) | 23:27 |
| admin0 | clayg: HEAD /sdb2/996/ seems to be missing .. example: HEAD http://objectstore.internal/v1/AUTH_3760c6c750a94a5ab9b869e933734481/xxx/yyy/4/2016/01/26/ZZZ.mp3 | 23:27 |
| notmyname | clayg: :-) | 23:28 |
| admin0 | that is from log on the storage server | 23:28 |
| torgomatic | also, 10 points to Gryffindor for making the openstack bot catch multiple bug numbers in one message | 23:28 |
| *** darrenc is now known as darrenc_afk | 23:28 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 23:29 |
| *** kei_yama has joined #openstack-swift | 23:30 | |
| * clayg comes back from know your meme | 23:30 | |
| admin0 | bye guys .. thanks for everything .. will come back tomorrow | 23:36 |
| *** darrenc_afk is now known as darrenc | 23:36 | |
| *** admin0 has quit IRC | 23:36 | |
| clayg | torgomatic: yeah I got scared fo the test fallout trying to remove it - but this worked for setting it in the right place IIRC -> https://gist.github.com/clayg/72f9ae5bf5d94fa6dd1f | 23:37 |
| *** ho_ has joined #openstack-swift | 23:51 | |
| *** lcurtis has quit IRC | 23:54 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!