| *** baojg has joined #openstack-swift | 00:00 | |
| *** baojg has quit IRC | 00:06 | |
| *** rmcall has quit IRC | 00:07 | |
| *** baojg has joined #openstack-swift | 00:07 | |
| *** baojg has quit IRC | 00:12 | |
| *** baojg has joined #openstack-swift | 00:13 | |
| kota_ | good morning | 00:15 |
|---|---|---|
| kota_ | mattoliverau, notmyname: currently I don't have any special topics for 0700 meeting | 00:16 |
| mattoliverau | kota_: morning | 00:16 |
| *** baojg has quit IRC | 00:17 | |
| *** baojg has joined #openstack-swift | 00:18 | |
| *** two_tired has joined #openstack-swift | 00:19 | |
| *** mbirru has joined #openstack-swift | 00:25 | |
| *** shuyingya has joined #openstack-swift | 00:25 | |
| mbirru | Hi | 00:26 |
| mbirru | Could somebody help me to figureout the issue I am seeing the "Account HEAD failed: http://A1-39978-controller:8080/v1/AUTH_380ec248a37a402baf9f529e5b936326 503 Service Unavailable" for my "swift stat" | 00:27 |
| mbirru | here my swift-proxy service is running on 8080 port | 00:28 |
| tdasilva | mattoliverau: sounds awesome! do you remember if the ring building part is working well? | 00:28 |
| mbirru | mattoliverau: First I would like thank you for kind response | 00:28 |
| mattoliverau | tdasilva: it works... but definitely could be better. it needs swift installed on the localhost. and so there is a bit of a chicken and egg prob. | 00:30 |
| mbirru | my ring building was success. here is the output https://pastebin.com/WFFJNTmr | 00:31 |
| mattoliverau | mbirru: your welcome, tho I must admit, I can't remember, its been a crazy last few months :) | 00:31 |
| *** baojg has quit IRC | 00:32 | |
| tdasilva | mattoliverau: oh yeah, i remember that | 00:32 |
| *** kiennt has joined #openstack-swift | 00:32 | |
| mattoliverau | mbirru: what does your proxy and account logs say? | 00:32 |
| tdasilva | yeah, sometimes i almost wish we would have a swift utils rpm | 00:32 |
| *** baojg has joined #openstack-swift | 00:32 | |
| *** baojg has quit IRC | 00:33 | |
| mattoliverau | mbirru: do you have account_autocreate enabled in the proxy config? | 00:34 |
| mbirru | mattoliverau: for my proxy-server I couldnt find logs. could you pint me where logs are writing by default. I am searching in /var/log | 00:35 |
| mattoliverau | mbirru: looking at your rings, it seems one of your devices in the account ring is pointing to /dev/sda6 | 00:36 |
| mbirru | mattoliverau: yes account_autocreate is true in proxy.conf | 00:36 |
| mbirru | mattoliverau: yes you are right .its pointing to /dev/sda6 | 00:37 |
| mattoliverau | mbirru: depends on how you set it up.. if you haven't given rsyslog rules to point to something like /var/log/swift/.. then they could be in /var/log/messages or /var/log/syslog depending on your distro. | 00:38 |
| mattoliverau | mbirru: you can't point them directly to the devices (the /dev) they need to point to mount points. | 00:38 |
| mattoliverau | tdasilva: what are you still doing up anyway? | 00:39 |
| tdasilva | mattoliverau: just put baby to sleep, so just stopped by to check things out here | 00:39 |
| mattoliverau | tdasilva: I can see why somepeople use a dedicated node to build rings.. if only there was a ringbuilder that could deploy rings :P | 00:40 |
| mattoliverau | tdasilva: ahh :) | 00:40 |
| mbirru | mattoliverau: I could see proxy-log in /var/log/syslog here are some error I see 00 | 00:42 |
| mbirru | 00 | 00:42 |
| mbirru | mattoliverau: here are errors https://pastebin.com/Rta2ewks | 00:42 |
| mattoliverau | mbirru: yeah, so the proxy is finding insufficant storage (507) and so error limiting the nodes and retuning a 503. | 00:44 |
| mattoliverau | what are your devices in your account, container and object servers, are they all pointing at /dev? or have you mounted the all the devices somewhere, and pointing to the mount point locations? because the latter is what you need. | 00:45 |
| mattoliverau | mbirru: ^ | 00:45 |
| mbirru | mattoliverau: I allocated 30 GB to /dev/sda6 and /dev/sda7. Could you please let me know how is recommended. just to test the basic storage is working fine. Here I am using only one harddisk where my linux is installed. But I created 3 partitions with each30 GB | 00:46 |
| mattoliverau | only asking because of the /dev/sda6 in the account ring | 00:46 |
| mattoliverau | mbirru: that should be fine | 00:46 |
| mattoliverau | mbirru: so you need to format sda6 and sda7 with XFS, then mount them somewhere, say under /srv.. eg: | 00:47 |
| mattoliverau | mkdir /srv/{sda6,sda7} | 00:47 |
| mattoliverau | then mount them there. | 00:48 |
| mattoliverau | then finally in your swift account, container and object configs point the devices to /srv/ | 00:48 |
| mattoliverau | then your rings will point to the mountpoints rather then the devices, which it needs. | 00:48 |
| mattoliverau | Obviously this is just to get it working, it would be better to setup fstab entries, so they automount. | 00:49 |
| mattoliverau | mbirru: ^ | 00:49 |
| mattoliverau | mbirru: does that make sense? | 00:49 |
| mbirru | mattoliverau: yes I did XFS and mounted to /srv/node | 00:49 |
| mbirru | https://pastebin.com/kM8N5uHw | 00:50 |
| mattoliverau | mbirru: cool | 00:51 |
| mbirru | mattoliverau: I followed exactly as you said from the https://docs.openstack.org/mitaka/install-guide-ubuntu/swift-storage-install.html | 00:51 |
| *** itlinux has joined #openstack-swift | 00:51 | |
| mattoliverau | mbirru: then you need to make sure your devices paramater in your configs (account, container and object) are /srv/node | 00:52 |
| mattoliverau | and your rings would need to point to sdb and sdc | 00:52 |
| mattoliverau | ie. point the rings to the mount points not the devices. | 00:52 |
| mattoliverau | mbirru: ^ | 00:53 |
| mbirru | mattaoliverau: here is config https://pastebin.com/1qtbJWsr | 00:54 |
| mbirru | mattoliverau: Yes I did | 00:54 |
| mattoliverau | mbirru: great | 00:54 |
| mattoliverau | mbirru: we should now just need to fix the rings :) | 00:55 |
| mattoliverau | just rebuild them and point to sdb and sdc | 00:55 |
| mattoliverau | instead of sda6 and asd7 | 00:56 |
| mbirru | mattoliverau: got it understood. thanks a lot for you are kind help | 00:56 |
| mbirru | mattoliverau: let change and update you | 00:56 |
| mattoliverau | mbirru: anytime :) you know where to find us if you need anymore help! | 00:56 |
| mattoliverau | mbirru: kk | 00:56 |
| mattoliverau | mbirru: and thanks for trying out swift :) | 00:57 |
| *** cshastri has joined #openstack-swift | 00:57 | |
| mbirru | mattoliverau: thank you | 00:57 |
| *** shuyingya has quit IRC | 01:01 | |
| *** itlinux has quit IRC | 01:04 | |
| *** tovin07_ has joined #openstack-swift | 01:25 | |
| *** shuyingya has joined #openstack-swift | 01:27 | |
| *** itlinux has joined #openstack-swift | 01:29 | |
| *** gyee has quit IRC | 01:29 | |
| *** itlinux has quit IRC | 01:35 | |
| *** itlinux has joined #openstack-swift | 01:43 | |
| *** bkopilov has quit IRC | 01:48 | |
| *** itlinux has quit IRC | 01:56 | |
| *** itlinux has joined #openstack-swift | 02:06 | |
| *** itlinux has quit IRC | 02:21 | |
| mbirru | mattoliverau: After rebuilding the rings.I am able to successfully run the swift-proxy server able to store the object | 02:25 |
| mbirru | mattoliverau: Thanks a lot for help | 02:25 |
| mattoliverau | mbirru: \o/ | 02:28 |
| mattoliverau | Nps | 02:28 |
| *** baojg has joined #openstack-swift | 02:35 | |
| *** baojg has quit IRC | 02:43 | |
| *** itlinux has joined #openstack-swift | 02:50 | |
| *** itlinux has quit IRC | 02:57 | |
| *** bkopilov has joined #openstack-swift | 03:04 | |
| *** itlinux has joined #openstack-swift | 03:12 | |
| *** kei_yama has quit IRC | 03:12 | |
| *** itlinux has quit IRC | 03:14 | |
| *** kei_yama has joined #openstack-swift | 03:17 | |
| *** itlinux has joined #openstack-swift | 03:17 | |
| *** kei_yama has quit IRC | 03:18 | |
| notmyname | kota_: if you or eran have a question on storlets for me (ie for the pike release), please let me know in here, and I'll get back to you | 03:21 |
| *** itlinux has quit IRC | 03:30 | |
| *** gkadam has joined #openstack-swift | 03:40 | |
| openstackgerrit | Tim Burke proposed openstack/swift master: Socket errors don't warrant tracebacks when talking to memcached https://review.openstack.org/496150 | 03:43 |
| *** links has joined #openstack-swift | 03:57 | |
| kota_ | notmyname: thanks | 04:01 |
| kota_ | notmyname: sorry, i'll leave for a while, so if something needed with eranrom, I'll ask you later. | 04:02 |
| openstackgerrit | Merged openstack/swift master: Drop reconstructor stats when worker has no devices https://review.openstack.org/495805 | 04:17 |
| *** two_tired has quit IRC | 04:30 | |
| *** itlinux has joined #openstack-swift | 04:30 | |
| *** mvk_ has joined #openstack-swift | 04:31 | |
| *** itlinux has quit IRC | 04:34 | |
| *** mkrcmari__ has joined #openstack-swift | 04:34 | |
| *** mvk_ has quit IRC | 04:35 | |
| *** rmcall has joined #openstack-swift | 04:38 | |
| *** chetna has joined #openstack-swift | 04:39 | |
| *** mvk has joined #openstack-swift | 04:43 | |
| mahatic | mattoliverau: skipping 0700 meeting makes sense, thanks for checking | 04:45 |
| mahatic | notmyname: ^ | 04:45 |
| mattoliverau | kk, we'll skip it | 04:45 |
| *** mkrcmari__ has quit IRC | 04:46 | |
| mattoliverau | acoles, rledisez and anyone else, the 0700 meeting has been canceled for today. | 04:46 |
| *** chetna has quit IRC | 04:48 | |
| *** mvk has quit IRC | 04:58 | |
| *** mrhillsman has quit IRC | 05:02 | |
| *** Jeffrey4l has quit IRC | 05:02 | |
| *** afazekas has quit IRC | 05:02 | |
| *** d34dh0r53 has quit IRC | 05:02 | |
| *** dhellmann has quit IRC | 05:02 | |
| *** kencjohnston_ has quit IRC | 05:02 | |
| *** mrhillsman has joined #openstack-swift | 05:02 | |
| *** afazekas has joined #openstack-swift | 05:02 | |
| *** dhellmann has joined #openstack-swift | 05:03 | |
| *** Jeffrey4l has joined #openstack-swift | 05:03 | |
| *** jcook has quit IRC | 05:04 | |
| *** d34dh0r53 has joined #openstack-swift | 05:04 | |
| *** kencjohnston has joined #openstack-swift | 05:04 | |
| *** jcook has joined #openstack-swift | 05:07 | |
| *** itlinux has joined #openstack-swift | 05:07 | |
| *** silor has joined #openstack-swift | 05:11 | |
| *** mvk has joined #openstack-swift | 05:12 | |
| *** itlinux has quit IRC | 05:19 | |
| *** mat128 has joined #openstack-swift | 05:24 | |
| *** mvk has quit IRC | 05:26 | |
| *** mvk has joined #openstack-swift | 05:27 | |
| *** mat128 has quit IRC | 05:28 | |
| *** mvk_ has joined #openstack-swift | 05:34 | |
| *** mvk has quit IRC | 05:37 | |
| openstackgerrit | Tim Burke proposed openstack/swift master: slo/dlo: Segment/ListingErrors don't need tracebacks https://review.openstack.org/496493 | 05:39 |
| zaitcev | LOL guys I knew it was PEP8 :-) https://review.openstack.org/#/c/496150/ | 05:41 |
| patchbot | patch 496150 - swift - Socket errors don't warrant tracebacks when talkin... | 05:41 |
| zaitcev | Gets you every time doesn't it | 05:41 |
| *** kiennt has quit IRC | 05:41 | |
| timburke | every time :-( | 05:49 |
| *** ChubYann has quit IRC | 05:51 | |
| *** hoonetorg has quit IRC | 05:54 | |
| *** hoonetorg has joined #openstack-swift | 06:07 | |
| *** kiennt has joined #openstack-swift | 06:09 | |
| *** rcernin has joined #openstack-swift | 06:16 | |
| *** skudlik has joined #openstack-swift | 06:17 | |
| *** skudlik has left #openstack-swift | 06:17 | |
| *** tovin07_ has quit IRC | 06:20 | |
| *** jlvillal has quit IRC | 06:21 | |
| *** jlvillal has joined #openstack-swift | 06:21 | |
| *** kiennt has quit IRC | 06:23 | |
| *** kiennt has joined #openstack-swift | 06:24 | |
| *** hseipp has joined #openstack-swift | 06:40 | |
| *** m_kazuhiro has joined #openstack-swift | 06:43 | |
| *** geaaru has joined #openstack-swift | 06:45 | |
| mathiasb | good morning | 06:52 |
| openstackgerrit | Tim Burke proposed openstack/swift master: container sync: reraise properly, don't log traceback for ClientExceptions https://review.openstack.org/496517 | 06:53 |
| *** cschwede_ has joined #openstack-swift | 06:55 | |
| *** tovin07_ has joined #openstack-swift | 06:55 | |
| acoles | mattoliverau: ack, thanks | 06:59 |
| *** m_kazuhiro has quit IRC | 07:01 | |
| *** pcaruana has joined #openstack-swift | 07:11 | |
| *** karmine has joined #openstack-swift | 07:14 | |
| *** tesseract-RH has joined #openstack-swift | 07:17 | |
| mattoliverau | mathiasb: o/ | 07:19 |
| mattoliverau | acoles: morning | 07:19 |
| acoles | mattoliverau: o/ | 07:19 |
| mahatic | acoles: mathiasb o/ | 07:22 |
| *** klrmn has quit IRC | 07:29 | |
| mattoliverau | tdasilva: I've pushed what I have into the master of my fork.. but might needed some more testing.. only really checked it on redhat/cenos, the debian variables are clearly wrong, but I want to get it to work on something first, and cause I'm using your vagrant redhat wins :) | 07:30 |
| mattoliverau | but will PR it to you when I'm more happy with it. (just an FYI). | 07:31 |
| openstackgerrit | Tim Burke proposed openstack/swift master: expirer: unexpected responses don't warrant tracebacks https://review.openstack.org/496534 | 07:35 |
| openstackgerrit | Tim Burke proposed openstack/swift master: Simplify testing for logging at error vs exception https://review.openstack.org/496535 | 07:35 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift master: Imported Translations from Zanata https://review.openstack.org/496137 | 07:39 |
| timburke | i'm not entirely sure i like the tactics of that last one, but getting there certainly shook out a bunch of cases where we seem to be logging tracebacks that aren't useful | 07:41 |
| *** oshritf has joined #openstack-swift | 07:55 | |
| *** openstackgerrit has quit IRC | 08:03 | |
| *** hseipp has quit IRC | 08:05 | |
| *** cbartz has joined #openstack-swift | 08:09 | |
| *** klrmn has joined #openstack-swift | 08:12 | |
| *** klrmn has quit IRC | 08:31 | |
| *** rmcall has quit IRC | 08:34 | |
| *** oshritf has quit IRC | 08:34 | |
| *** dosaboy has joined #openstack-swift | 08:38 | |
| *** spotz has quit IRC | 08:46 | |
| *** oshritf has joined #openstack-swift | 08:46 | |
| *** spotz has joined #openstack-swift | 08:49 | |
| *** mrhillsman has quit IRC | 08:51 | |
| *** mrhillsman has joined #openstack-swift | 08:52 | |
| *** oshritf has quit IRC | 09:13 | |
| *** bkopilov has quit IRC | 09:16 | |
| *** tovin07_ has quit IRC | 09:49 | |
| *** oshritf has joined #openstack-swift | 09:53 | |
| *** mkrcmari__ has joined #openstack-swift | 09:54 | |
| *** mvk_ has quit IRC | 09:58 | |
| *** kiennt has quit IRC | 10:03 | |
| *** oshritf has quit IRC | 10:11 | |
| *** oshritf has joined #openstack-swift | 10:12 | |
| *** mbirru has quit IRC | 10:18 | |
| *** cshastri has quit IRC | 10:42 | |
| *** hoonetorg has quit IRC | 10:43 | |
| *** hoonetorg has joined #openstack-swift | 10:43 | |
| *** oshritf has quit IRC | 11:30 | |
| *** mat128 has joined #openstack-swift | 11:37 | |
| *** hseipp has joined #openstack-swift | 11:40 | |
| *** iprokg has joined #openstack-swift | 11:51 | |
| *** iprokg has quit IRC | 11:52 | |
| *** iprokg has joined #openstack-swift | 11:52 | |
| *** iprokg has joined #openstack-swift | 11:52 | |
| *** iprokg has quit IRC | 11:54 | |
| *** kmARC has quit IRC | 12:11 | |
| *** kencjohnston has quit IRC | 12:12 | |
| *** kmARC has joined #openstack-swift | 12:14 | |
| *** kencjohnston has joined #openstack-swift | 12:14 | |
| *** mat128 has quit IRC | 12:19 | |
| *** openstackgerrit has joined #openstack-swift | 12:22 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements https://review.openstack.org/88736 | 12:22 |
| *** gkadam has quit IRC | 12:43 | |
| *** shuyingya has quit IRC | 12:56 | |
| *** lucasxu has joined #openstack-swift | 13:10 | |
| *** abhitechie has joined #openstack-swift | 13:15 | |
| *** shuyingya has joined #openstack-swift | 13:19 | |
| *** baojg has joined #openstack-swift | 13:21 | |
| *** shuyingya has quit IRC | 13:23 | |
| *** bkopilov has joined #openstack-swift | 13:26 | |
| openstackgerrit | Peter Lisák proposed openstack/swift master: Added swift version to recon cli https://review.openstack.org/413991 | 13:28 |
| *** baojg has quit IRC | 13:34 | |
| *** links has quit IRC | 13:38 | |
| *** shuyingya has joined #openstack-swift | 13:42 | |
| *** SkyRocknRoll has joined #openstack-swift | 13:43 | |
| *** baojg has joined #openstack-swift | 13:47 | |
| *** baojg has quit IRC | 13:47 | |
| *** baojg has joined #openstack-swift | 13:48 | |
| *** mat128 has joined #openstack-swift | 13:52 | |
| *** mat128 has quit IRC | 13:53 | |
| *** mat128 has joined #openstack-swift | 13:57 | |
| *** baojg has quit IRC | 14:03 | |
| jrichli | mathiasb: congrats on the merge! I was on vaca and just now saw that in scrollback this morning. | 14:04 |
| *** gabor_antal has joined #openstack-swift | 14:04 | |
| mathiasb | jrichli: thanks, and thanks for all your feedback and testing also! | 14:06 |
| *** baojg has joined #openstack-swift | 14:09 | |
| *** baojg has quit IRC | 14:15 | |
| *** bkopilov has quit IRC | 14:15 | |
| *** abhitechie has quit IRC | 14:20 | |
| *** abhitechie has joined #openstack-swift | 14:21 | |
| *** abhitechie has quit IRC | 14:21 | |
| *** vint_bra has joined #openstack-swift | 14:25 | |
| *** SkyRocknRoll has quit IRC | 14:26 | |
| *** itlinux has joined #openstack-swift | 14:27 | |
| *** abhitechie has joined #openstack-swift | 14:30 | |
| *** abhitechie has quit IRC | 14:31 | |
| *** abhitechie has joined #openstack-swift | 14:31 | |
| *** abhitechie has quit IRC | 14:32 | |
| *** abhitechie has joined #openstack-swift | 14:33 | |
| *** itlinux has quit IRC | 14:34 | |
| *** abhitechie has quit IRC | 14:35 | |
| *** abhitechie has joined #openstack-swift | 14:35 | |
| *** two_tired has joined #openstack-swift | 14:38 | |
| *** SkyRocknRoll has joined #openstack-swift | 14:38 | |
| *** abhitechie has quit IRC | 14:40 | |
| *** abhitechie has joined #openstack-swift | 14:40 | |
| *** srbgupta has joined #openstack-swift | 14:41 | |
| *** gyee has joined #openstack-swift | 14:41 | |
| *** mkrcmari__ has quit IRC | 14:47 | |
| *** abhitechie has quit IRC | 14:48 | |
| *** abhitechie has joined #openstack-swift | 14:48 | |
| *** mvk has joined #openstack-swift | 14:48 | |
| openstackgerrit | Merged openstack/swift master: Socket errors don't warrant tracebacks when talking to memcached https://review.openstack.org/496150 | 14:55 |
| *** shuyingya has quit IRC | 15:08 | |
| *** abhitechie has quit IRC | 15:11 | |
| *** abhitechie has joined #openstack-swift | 15:11 | |
| *** lucasxu has quit IRC | 15:13 | |
| *** abhitechie has quit IRC | 15:16 | |
| *** abhitechie has joined #openstack-swift | 15:16 | |
| *** klrmn has joined #openstack-swift | 15:19 | |
| *** abhitechie has quit IRC | 15:23 | |
| *** abhitechie has joined #openstack-swift | 15:23 | |
| *** abhitechie has quit IRC | 15:24 | |
| *** abhitechie has joined #openstack-swift | 15:25 | |
| *** catintheroof has joined #openstack-swift | 15:25 | |
| *** two_tired has quit IRC | 15:27 | |
| openstackgerrit | Gábor Antal proposed openstack/swift master: Use more specific asserts in test/unit/common https://review.openstack.org/342781 | 15:29 |
| openstackgerrit | Gábor Antal proposed openstack/swift master: Use more specific asserts in test/unit/account tests https://review.openstack.org/342766 | 15:29 |
| openstackgerrit | Gábor Antal proposed openstack/swift master: Use more specific asserts in test/unit/common/middleware https://review.openstack.org/342770 | 15:29 |
| openstackgerrit | Gábor Antal proposed openstack/swift master: Use more specific asserts in test/unit/container https://review.openstack.org/342808 | 15:31 |
| openstackgerrit | Gábor Antal proposed openstack/swift master: Use more specific asserts in test/unit/obj tests https://review.openstack.org/342830 | 15:31 |
| openstackgerrit | Gábor Antal proposed openstack/swift master: Update hacking and enable off-by-default checks https://review.openstack.org/486627 | 15:31 |
| *** itlinux has joined #openstack-swift | 15:47 | |
| *** lucasxu has joined #openstack-swift | 15:49 | |
| *** karmine has quit IRC | 15:51 | |
| timburke | good morning | 15:51 |
| *** abhitechie has quit IRC | 15:53 | |
| *** abhitechie has joined #openstack-swift | 15:53 | |
| *** klrmn has quit IRC | 16:09 | |
| notmyname | good morning | 16:11 |
| notmyname | joeljwright: around? do we have anything to talk about with *ambles today? | 16:21 |
| joeljwright | notmyname: I am | 16:21 |
| *** cbartz has quit IRC | 16:21 | |
| joeljwright | I don't think there's much to cover regarding *ambles | 16:22 |
| notmyname | ok | 16:22 |
| joeljwright | other than a nudge | 16:22 |
| joeljwright | and a promise that I now have a tlo WIP | 16:22 |
| notmyname | you were working on finding/showing some different formats, right? any progress on that? | 16:22 |
| joeljwright | that I will be sharing this week | 16:22 |
| notmyname | ah, right. the full TLO thing | 16:22 |
| joeljwright | TLO implemented as a manifest transformation | 16:23 |
| notmyname | ok, in that case, let's cancel today's 2100 meeting | 16:23 |
| joeljwright | sure, I'll have more ammo for next week's meeting :) | 16:23 |
| notmyname | acoles: mattoliverau: kota_: cschwede_: mahatic: zaitcev: clayg: timburke: tdasilva: jrichli: torgomatic: jungleboyj: no 2100 meeting today | 16:24 |
| *** rcernin has quit IRC | 16:29 | |
| *** mat128 has quit IRC | 16:35 | |
| *** chsc has joined #openstack-swift | 16:38 | |
| *** chsc has joined #openstack-swift | 16:38 | |
| cschwede_ | notmyname: alright, thx for the info! | 16:40 |
| notmyname | cschwede_: do you know what's the status on https://bugs.launchpad.net/swift/+bug/1580678 (which is likely related to https://bugs.launchpad.net/swift/+bug/1648082 which might be closed by https://review.openstack.org/#/c/446762/) | 16:42 |
| openstack | Launchpad bug 1580678 in OpenStack Object Storage (swift) "UnicodeDecodeError when rebalancing a ring" [Medium,In progress] - Assigned to Christian Schwede (cschwede) | 16:42 |
| patchbot | patch 446762 - oslo.i18n - add notes about skipping log translation setup (MERGED) | 16:42 |
| openstack | Launchpad bug 1648082 in OpenStack Object Storage (swift) "Yet UnicodeDecodeError: 'ascii' codec can't decode byte..." [Undecided,New] | 16:42 |
| *** itlinux has quit IRC | 16:44 | |
| timburke | notmyname: i'm skeptical of any claim that a commit to oslo.i18n "closes" a bug that reports a traceback in swift ;-) | 16:45 |
| notmyname | :-) | 16:45 |
| timburke | we still need to go through an purge any gettext calls for log messages. *then* i could consider those bugs closed | 16:46 |
| timburke | although, i'm not sure we *should* purge translations for ring rebalances -- operator CLI tool *might* actually be worth having translations? i suppose you still have the discoverability issues, coupled with the fact that it's hard to translate jargon, especially when *we make up things* like "replicanths" | 16:48 |
| cschwede_ | notmyname: no, but i can have a look again at it. and i agree with timburke to purge any gettext logging calls | 16:49 |
| notmyname | cschwede_: just was asking you since your name is on it ;-) | 16:49 |
| jungleboyj | notmyname: Thanks for letting me know. That helps my schedule for the day. :-) | 16:49 |
| notmyname | timburke: cschwede_: yeah, I agree about "closed when log translations dropped" | 16:50 |
| notmyname | my gut says to not translate errors or logs because of the problems mentioned. but also my gut said that in English and is not really qualified to say if it should be in non-English | 16:52 |
| *** itlinux has joined #openstack-swift | 16:55 | |
| timburke | maybe there's some additional context that ought to be considered? like, if we (1) expect this to be something that only an operator (being the people that actually control what locale a server uses) would see, either at a CLI *or* in logs, and (2) are providing fairly specific, actionable information (like, say, https://github.com/openstack/swift/blob/2.15.1/swift/obj/replicator.py#L112-L115) then maybe it *should* be translated? | 16:58 |
| timburke | i'm really not sure that "don't translate logs" is the right call, particularly since our CLI stuff sometimes uses logging to emit messages to the user, like in https://github.com/openstack/swift/blob/master/swift/cli/relinker.py | 17:02 |
| timburke | maybe i was wrong to push for that there? but it seemed silly to reinvent log levels for --debug | 17:03 |
| *** klrmn has joined #openstack-swift | 17:06 | |
| *** pcaruana has quit IRC | 17:08 | |
| *** hseipp has quit IRC | 17:12 | |
| *** tesseract-RH has quit IRC | 17:12 | |
| *** mvk_ has joined #openstack-swift | 17:30 | |
| *** mat128 has joined #openstack-swift | 17:31 | |
| *** itlinux has quit IRC | 17:32 | |
| *** mkrcmari__ has joined #openstack-swift | 17:33 | |
| *** mvk has quit IRC | 17:33 | |
| *** mat128 has quit IRC | 17:34 | |
| *** rcernin has joined #openstack-swift | 17:34 | |
| *** mvk_ has quit IRC | 17:35 | |
| *** rmcall has joined #openstack-swift | 17:49 | |
| *** itlinux has joined #openstack-swift | 17:50 | |
| *** SkyRocknRoll has quit IRC | 18:06 | |
| *** catintheroof has quit IRC | 18:07 | |
| *** mkrcmari__ has quit IRC | 18:28 | |
| notmyname | team photo (like at the last PTG) scheduled for 2:50 pm on the wednesday of the ptg | 18:30 |
| *** cschwede_ has quit IRC | 18:40 | |
| *** ChubYann has joined #openstack-swift | 18:43 | |
| tdasilva | notmyname: cool, also thanks for the meetings heads up | 18:44 |
| *** rcernin has quit IRC | 18:47 | |
| *** rmcall has quit IRC | 19:07 | |
| *** itlinux has quit IRC | 19:10 | |
| *** mat128 has joined #openstack-swift | 19:20 | |
| acoles | notmyname: ack no meeting | 19:33 |
| tdasilva | hi everyone! gnocchi is a time-series database (like influxdb) that is used by the openstack telemetry team (e.g., ceilometer) to store metrics. One of the cools things about it is that it is designed to be very scalable so it contains drivers to store its data directly into storage systems like swift and ceph | 19:39 |
| tdasilva | long story short: akrzos from red hat openstack performance team has been doing some cool work testing it with swift | 19:40 |
| tdasilva | and would like to talk about it during the PTG | 19:40 |
| tdasilva | so I encouraged him to add as a topic to the wiki | 19:40 |
| notmyname | tdasilva: nice | 19:42 |
| *** itlinux has joined #openstack-swift | 19:43 | |
| *** itlinux has quit IRC | 19:47 | |
| *** itlinux has joined #openstack-swift | 20:01 | |
| timburke | huh. do we not log ssync requests like we do other requests on the object server? i saw Unexpected response: ":ERROR: 500 'ERROR: With :UPDATES: 1 failures to 0 successes'" somewhere, went grepping for ' 500 ' on the other end and didn't turn up anything | 20:02 |
| timburke | eventually got "ssync subrequest failed with 409: PUT ..." but i had to go searching by the path | 20:02 |
| timburke | ...which, given the load i was driving (lots of overwrites/deletes of the same object) isn't surprising... but i wonder if we should maybe consider a 409 as successful in this context? the other end told me they have something newer; my job as relay is done | 20:05 |
| acoles | timburke: IIRC ssync receiver tolerates some subrequest failures before giving up | 20:05 |
| acoles | timburke: this may be relevant https://review.openstack.org/#/c/181407/ | 20:07 |
| patchbot | patch 181407 - swift - EC: Avoid conflicts when ssync'ing fragments | 20:07 |
| acoles | but it's so long ago I'd need to reload context | 20:07 |
| acoles | final sentence of commit msg says "The patch also results in newer fragments on the receiver always resulting in the sender not attempting to PUT data." | 20:07 |
| timburke | yeah, and that's fine... i guess my original complaint is that, when one end says they received a 500, i want to be able to search for ' 500 ' (with the spaces on either side to avoid partial matches on trans ids/content lengths) in my logs... | 20:08 |
| *** rmcall has joined #openstack-swift | 20:12 | |
| acoles | OIC | 20:13 |
| *** itlinux has quit IRC | 20:14 | |
| *** itlinux has joined #openstack-swift | 20:17 | |
| *** silor has quit IRC | 20:29 | |
| notmyname | kong: I just saw your questions about horizon in the meeting. if you can't get what you need from horizon, would other object storage clients work? | 20:32 |
| kong | notmyname: hey | 20:33 |
| kong | notmyname: what do you mean by 'other clients' | 20:33 |
| notmyname | kong: there's a lot more ways to talk to a swift cluster and see what's in it that just using horizon. I only mean using something else | 20:34 |
| notmyname | kong: however, if horizon is your single view for customers into the cloud, then that might not be possible | 20:35 |
| kong | notmyname: unfortunately, horizon is the only one dashboard we are using | 20:35 |
| notmyname | ok, I understand | 20:35 |
| *** rcernin has joined #openstack-swift | 20:35 | |
| robcresswell | fwiw, we're happy to accept improvements, it's just that "the horizon team" nowadays is something like 3 part-timers | 20:35 |
| notmyname | robcresswell: no offense intended :-) | 20:36 |
| robcresswell | We're not being deliberately obtuse about adding functionality :) | 20:36 |
| robcresswell | notmyname: None taken, I just wanted to clarify my stance on it | 20:36 |
| notmyname | got it :-) | 20:36 |
| notmyname | kong: it sounds like you've got some pretty active users on your swift cluster, too. that's really great to hear | 20:37 |
| notmyname | kong: have you kept up with the long-term work we've been doing in swift to support very large containers? (the "container sharding" work) | 20:38 |
| *** lucasxu has quit IRC | 20:38 | |
| notmyname | kong: will you be at the PTG next month? | 20:38 |
| *** lucasxu has joined #openstack-swift | 20:38 | |
| kong | notmyname: yeah, and many customers were considering transit to swift after we made the announcement last year | 20:39 |
| kong | notmyname: no :-( | 20:39 |
| notmyname | cool (using swift) and I'm sorry to hear it (you not being at the ptg) | 20:39 |
| mattoliverau | notmyname: ack | 20:39 |
| notmyname | kong: what was the announcement you made last year? | 20:40 |
| mattoliverau | tdasilva: cool | 20:40 |
| kong | notmyname: we provide Geo-replicated Object Storage Service | 20:40 |
| notmyname | nice! | 20:40 |
| notmyname | have you used the geo-replicated EC policies yet? | 20:40 |
| *** itlinux has quit IRC | 20:41 | |
| kong | notmyname: not yet, but i think it's a good addon if customers are asking for that | 20:41 |
| tdasilva | mattoliverau: o/ | 20:43 |
| notmyname | reminder: no 2100 swift team meeting today | 20:46 |
| *** lucasxu has quit IRC | 20:48 | |
| *** m_kazuhiro has joined #openstack-swift | 20:57 | |
| notmyname | peluse just did a code review?! (on a 2 year old patch) | 20:58 |
| kota_ | oh | 20:58 |
| kota_ | no 2100 meeting? | 20:58 |
| notmyname | kota_: no 2100 meeting today | 20:58 |
| notmyname | kota_: enjoy a slow breakfast :-) | 20:59 |
| kota_ | notmyname: thanks, being back to my bed and enjoy my birthday ;-) | 20:59 |
| notmyname | kota_: happy birthday! | 20:59 |
| kota_ | :P | 21:00 |
| openstackgerrit | Tim Burke proposed openstack/swift master: container sync: reraise properly, don't log traceback for ClientExceptions https://review.openstack.org/496517 | 21:09 |
| openstackgerrit | Tim Burke proposed openstack/swift master: expirer: unexpected responses don't warrant tracebacks https://review.openstack.org/496534 | 21:09 |
| openstackgerrit | Tim Burke proposed openstack/swift master: Simplify testing for logging at error vs exception https://review.openstack.org/496535 | 21:09 |
| timburke | notmyname: yeah, i saw that too :-) i think it's my fault for leaving a comment on it | 21:11 |
| notmyname | heh | 21:11 |
| *** eckesicle has joined #openstack-swift | 21:12 | |
| mattoliverau | kota_: happy birthday!! | 21:13 |
| eckesicle | hello. I have some data hosted with a cloud provider in openstack swift, that I would like to securely wipe. is there any way to do this in swift? | 21:13 |
| *** m_kazuhiro has quit IRC | 21:13 | |
| notmyname | eckesicle: no, there is not | 21:13 |
| eckesicle | if i overwrite an object, it creates a new object on a different disk node? | 21:14 |
| notmyname | assuming the cluster is in a steady state with no failures, it will write to the same disks, but it will be new files | 21:14 |
| notmyname | the existing on disk data would be unlinked, but there is no overwrite of existing data | 21:14 |
| eckesicle | and if I add encryption with a random key, it will create new files too? | 21:15 |
| notmyname | you mean if you PUT the same object but this time encrypted? | 21:15 |
| eckesicle | yes | 21:15 |
| notmyname | yeah, same story (swift just sees you send a blob of bytes. doesn't matter if you've encrypted them or not, swift treats them the same way) | 21:16 |
| eckesicle | that's a shame | 21:16 |
| notmyname | (for completeness, since you mentioned it, the cluster may also write the new object into different places than the existing object. just depends on what's going on in the cluster at the time) | 21:17 |
| notmyname | eckesicle: it sounds like you implied that you are not the cluster operator | 21:17 |
| eckesicle | im not | 21:20 |
| eckesicle | ive been asked by a client to securely wipe some data for them | 21:20 |
| notmyname | eckesicle: in my experience, overwrite requirements normally come from certain compliance or regulatory requirements. (otherwise, unlinking the data should be sufficient) | 21:20 |
| notmyname | ah | 21:20 |
| eckesicle | it is for compliance | 21:20 |
| eckesicle | i guess we'll have to contact the operator | 21:21 |
| notmyname | yeah, the best you can do is either send an overwrite or a delete | 21:21 |
| notmyname | it will also be very difficult for the operator | 21:21 |
| notmyname | depending on what the requirements are | 21:21 |
| notmyname | eckesicle: you've got a couple of paths forward | 21:22 |
| jrichli | secure deletion is something that must hit the hardware level, and would have to involve some hardware-specific element | 21:22 |
| *** mat128 has quit IRC | 21:23 | |
| notmyname | one is to send a PUT for the same object name to overwrite it. the existing data will be unlinked. or send a delete and the existing data will be unlinked. the data will not be overwritten on the drive | 21:23 |
| eckesicle | @notmyname with a bit of luck we are on our own cluster | 21:23 |
| notmyname | this is pretty much the best you'll be able to do (when considering time/cost/etc) | 21:24 |
| eckesicle | and they can just wipe the disks/send them to us | 21:24 |
| notmyname | well, sortof | 21:24 |
| notmyname | the question then is, "which disks" | 21:24 |
| notmyname | and that's the second thing you could do | 21:24 |
| notmyname | ask the cluster operator to scan every drive for the bad data and overwrite it if found | 21:24 |
| eckesicle | well if we're on our own cluster, then just all the disks, after we migrate the data we want to keep to a new cluster | 21:25 |
| *** itlinux has joined #openstack-swift | 21:25 | |
| *** mbirru has joined #openstack-swift | 21:25 | |
| notmyname | imagine this scenario: @T0, PUT bad.data. @T1, one of the primary drives fails. the cluster replicates the data to a different location. @t@ the operator repairs the failed drive and the cluster moves a replica back to there and delete (ie unlinks, not overwrites) the replica on the handoff | 21:26 |
| notmyname | in this case, you've got your replicas in the cluster in the right place, but there was a time when you had another copy of the data in the cluster that was deleted but not overwritten | 21:27 |
| notmyname | so the only way to be sure that the string of bytes that you need scrubbed are well and truly scrubbed from the cluster is to check every disk | 21:27 |
| torgomatic | you might also consider a wood chipper with its output aimed into an acid bath | 21:28 |
| jrichli | yes, it goes farther than that. again, it also depends on hardware - is there a way to guarantee on your hardware that your particular "overwrite" really used the same memory | 21:29 |
| notmyname | eckesicle: the at-rest encryption functionality that we have in swift is designed to allow drive to leave the cluster without leaking object data (but that's not exactly the same thing you're looking for) | 21:29 |
| notmyname | lol, we're all piling on :-) | 21:30 |
| jrichli | honesti] | 21:30 |
| eckesicle | yeah | 21:30 |
| jrichli | honestly, without us having code that was hardware dependent cases, i dont think real secure deletion is possible | 21:30 |
| eckesicle | exactly, which is why ideally id like to have the disks in my hand | 21:30 |
| eckesicle | and run them through a microwave | 21:30 |
| torgomatic | https://www.youtube.com/watch?v=aCbfMkh940Q | 21:31 |
| notmyname | eckesicle: so, I'm not trying to make light of your situation. but from a technical perspective, there's not really a way to securely overwrite data in a swift cluster | 21:31 |
| eckesicle | i was afraid of that | 21:31 |
| eckesicle | so ill break in to the data center | 21:31 |
| notmyname | lol | 21:31 |
| timburke | client-side crypto give you, the user, a lot more control. issue a delete (so you stop getting billed for it), securely wipe the key locally (which has the added benefit of being much smaller). but that requires that the plaintext *never* makes it to the cluster | 21:31 |
| notmyname | timburke: yes! +100 | 21:32 |
| eckesicle | yeah, but unfortunately there are a few really old files that made it in plaintext | 21:32 |
| eckesicle | those are the ones im trying to get rid of | 21:32 |
| timburke | yeah, i kinda figured. sorry. that's going to be problematic | 21:33 |
| eckesicle | that's ok. i can always blame my predecessor | 21:39 |
| notmyname | torgomatic: thanks for the comments on patch 365371 | 21:47 |
| patchbot | https://review.openstack.org/#/c/365371/ - swift - Add Preamble and Postamble to SLO and SegmentedIte... | 21:47 |
| torgomatic | notmyname: 👍 | 21:47 |
| *** srbgupta has quit IRC | 22:01 | |
| *** geaaru has quit IRC | 22:06 | |
| *** abhitechie has quit IRC | 22:19 | |
| *** abhitechie has joined #openstack-swift | 22:19 | |
| *** itlinux has quit IRC | 22:29 | |
| *** abhitechie has quit IRC | 22:53 | |
| *** vint_bra has quit IRC | 23:03 | |
| mattoliverau | moring (officially as I'm at my desk) :) | 23:05 |
| mattoliverau | *morning even | 23:05 |
| notmyname | mattoliverau: hello | 23:12 |
| notmyname | mattoliverau: hope the commute to work wasn't too rough on your this morning | 23:12 |
| kota_ | mattoliverau: thx! | 23:17 |
| mattoliverau | notmyname: it was tough, its slightly colder downstairs in the office. And not to mention having to deal with a gate on the stairs to protect the toddler. That add a few seconds :P | 23:18 |
| *** kei_yama has joined #openstack-swift | 23:25 | |
| *** chsc has quit IRC | 23:35 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements https://review.openstack.org/88736 | 23:36 |
| *** vint_bra has joined #openstack-swift | 23:41 | |
| *** vint_bra has quit IRC | 23:46 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!