| *** _JZ_ has quit IRC | 00:03 | |
| clayg | i think there's plenty of work to go around - getting ec disk engine support in the new-object-server that works with the existing proxy would be pretty great | 00:03 |
|---|---|---|
| zaitcev | But that does not need liberasurecode, right? Only reconstructor needs it, right? | 00:04 |
| clayg | ... or terrible if you assume we want to replace multipart chunked upload with something else that will support commit & fotters | 00:04 |
| clayg | *footers | 00:04 |
| zaitcev | HTTP/2 | 00:04 |
| zaitcev | channels! frames! | 00:04 |
| clayg | zaitcev: that's no longer true now that the ec object app iter does the ec fragment checking | 00:05 |
| zaitcev | oh | 00:05 |
| clayg | zaitcev: I could probably be talked into http/2 - but I think http/1 pipelined requests would be sufficient | 00:05 |
| *** catintheroof has quit IRC | 00:14 | |
| *** catintheroof has joined #openstack-swift | 00:15 | |
| *** catintheroof has quit IRC | 00:19 | |
| *** StraubTW has joined #openstack-swift | 00:32 | |
| *** vint_bra has joined #openstack-swift | 00:35 | |
| *** kei_yama has quit IRC | 00:49 | |
| *** chlong has joined #openstack-swift | 00:49 | |
| *** kei_yama has joined #openstack-swift | 00:49 | |
| *** vint_bra has quit IRC | 00:52 | |
| *** diogogmt has quit IRC | 00:52 | |
| *** hoonetorg has quit IRC | 01:11 | |
| openstackgerrit | Pete Zaitcev proposed openstack/swift: Patch the policy through proxy operations https://review.openstack.org/394685 | 01:14 |
| zaitcev | + /usr/bin/docker run --name hbt250 -e GERRIT_REFSPEC=refs/changes/85/394685/2 hummingbirdtest /root/tests | 01:15 |
| zaitcev | time="2016-11-09T01:14:53Z" level=fatal msg="Error response from daemon: Unable to open the database file: unable to open database file" | 01:15 |
| zaitcev | I'm amazed anything of this giant pile of kludges even works at all | 01:16 |
| kota_ | good morning | 01:23 |
| *** hoonetorg has joined #openstack-swift | 01:24 | |
| *** diogogmt has joined #openstack-swift | 01:39 | |
| *** david_c has quit IRC | 01:39 | |
| *** david_c has joined #openstack-swift | 01:42 | |
| *** arch-nemesis has joined #openstack-swift | 02:05 | |
| kota_ | notmyname: quick confirmation, to progress the review speed, can we strat to manage the review process that a core can add +A for a trivial change written by another core? | 02:14 |
| kota_ | notmyname: have it been to work already? | 02:15 |
| *** arch-nemesis has quit IRC | 02:18 | |
| *** abalfour has quit IRC | 02:18 | |
| *** abalfour has joined #openstack-swift | 02:18 | |
| *** david-lyle has joined #openstack-swift | 02:22 | |
| *** clu_ has quit IRC | 02:25 | |
| *** bkopilov has quit IRC | 02:58 | |
| *** chenhuayi has joined #openstack-swift | 03:02 | |
| *** dmorita has quit IRC | 03:03 | |
| chenhuayi | Seeking for help. If there have the processes to do Swift compatibility certification? If Someone have the information about the Swift compatibility certification? Waiting for help online. | 03:17 |
| notmyname | kota_: yes | 04:04 |
| kota_ | notmyname: thx | 04:04 |
| *** links has joined #openstack-swift | 04:38 | |
| *** bkopilov has joined #openstack-swift | 04:47 | |
| winggundamth | hi. Anyway that I can remove public directory listing from Swift? | 04:52 |
| winggundamth | I just want visitors to directly access to the file but not list all the object in container | 04:52 |
| openstackgerrit | Merged openstack/swift: Fix ZeroDivisionError in reconstructor.stats_line https://review.openstack.org/394714 | 04:54 |
| *** StraubTW has quit IRC | 04:56 | |
| winggundamth | found it. swift post -r .r:* test_container | 05:03 |
| *** dmorita has joined #openstack-swift | 05:04 | |
| *** dmorita has quit IRC | 05:08 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 05:20 |
| *** david-lyle has quit IRC | 05:22 | |
| *** david-lyle has joined #openstack-swift | 05:23 | |
| charz_ | Good morning! | 05:33 |
| charz_ | Just a simple question for x-newest header in HEAD/GET request, if I deleted the object(two replicas) in primary places, but still have one copy in handoff. It seems like I able to HEAD this object form client even I add x-newest header, but I can't see it in container listing. | 05:38 |
| charz_ | Should I get `404 not found` not a `200 ok`? | 05:40 |
| charz_ | okay, I found a bug in https://bugs.launchpad.net/swift/+bug/1560572 | 05:41 |
| openstack | Launchpad bug 1560572 in OpenStack Object Storage (swift) "X-Newest doesn't take tombstones into consideration" [Undecided,New] | 05:41 |
| *** garthb has joined #openstack-swift | 05:48 | |
| *** SkyRocknRoll has joined #openstack-swift | 06:08 | |
| *** links has quit IRC | 06:11 | |
| *** SkyRocknRoll has quit IRC | 06:12 | |
| *** SkyRocknRoll has joined #openstack-swift | 06:12 | |
| *** dmorita has joined #openstack-swift | 06:14 | |
| *** chenhuayi_ has joined #openstack-swift | 06:16 | |
| *** dmorita has quit IRC | 06:19 | |
| *** chenhuayi has quit IRC | 06:20 | |
| *** chenhuayi__ has joined #openstack-swift | 06:39 | |
| *** links has joined #openstack-swift | 06:39 | |
| *** qwertyco has joined #openstack-swift | 06:40 | |
| *** chenhuayi_ has quit IRC | 06:42 | |
| *** david_c has quit IRC | 06:42 | |
| *** rcernin has joined #openstack-swift | 06:54 | |
| *** StraubTW has joined #openstack-swift | 06:57 | |
| *** SkyRocknRoll has quit IRC | 06:59 | |
| *** StraubTW has quit IRC | 07:02 | |
| *** Jeffrey4l has joined #openstack-swift | 07:09 | |
| *** tesseract has joined #openstack-swift | 07:11 | |
| *** tesseract is now known as Guest39033 | 07:11 | |
| *** dikonoor has joined #openstack-swift | 07:30 | |
| *** sams-gleb has joined #openstack-swift | 07:34 | |
| *** sams-gleb has quit IRC | 07:49 | |
| *** sams-gleb has joined #openstack-swift | 07:49 | |
| *** sams-gleb has quit IRC | 07:54 | |
| kota_ | charz_: that seems to happen but it sounds like just an eventual consistency | 07:56 |
| kota_ | charz_: hello | 07:56 |
| kota_ | charz_: in the evnetual consitency world, we can see older data even if we overwrite the older data with a new one but it's ok. | 07:57 |
| charz_ | kota_: yeah! I think so, but I think we should take that when client try to issue a request with `x-newest` header. | 07:57 |
| kota_ | ah, it may be problem (?) the newest should be tombstone? | 07:57 |
| charz_ | kota_: it's quite easy to run into this issue, when you have a global cluster (WA enabled), and you upload a object and deleted it. | 07:59 |
| charz_ | before object replicator to clean/move it. You'll see 200 ok when you do a HEAD. | 08:00 |
| charz_ | You need to wait replicator to clean/move it and you might able to get 404 not found. | 08:01 |
| charz_ | It seems like a problem when you have a 3 party software, and they're doing some checks with HEAD operation. | 08:01 |
| charz_ | I think the `x-newest` header need to take care .ts file in this kind of case. | 08:04 |
| kota_ | charz_: the 3rd party sounds like swift3 :P | 08:07 |
| charz_ | kota_: ah, it could be. :) | 08:07 |
| charz_ | kota_: I think the client might not able to know how swift works, but we might need to provide a way to make sure they get correct data. | 08:09 |
| charz_ | s/client/users/ | 08:09 |
| *** silor has joined #openstack-swift | 08:10 | |
| *** oshritf has joined #openstack-swift | 08:18 | |
| *** silor1 has joined #openstack-swift | 08:19 | |
| *** silor has quit IRC | 08:21 | |
| *** silor1 is now known as silor | 08:21 | |
| *** geaaru has joined #openstack-swift | 08:23 | |
| *** rledisez has joined #openstack-swift | 08:25 | |
| *** oshritf has quit IRC | 08:27 | |
| *** oshritf has joined #openstack-swift | 08:35 | |
| *** klrmn has quit IRC | 08:40 | |
| kota_ | charz_: exactly | 08:40 |
| *** openstackgerrit has quit IRC | 08:48 | |
| *** openstackgerrit has joined #openstack-swift | 08:48 | |
| *** kei_yama has quit IRC | 08:49 | |
| *** jordanP has joined #openstack-swift | 08:53 | |
| *** admin6 has joined #openstack-swift | 09:34 | |
| *** qwertyco has quit IRC | 09:42 | |
| *** cbartz has joined #openstack-swift | 09:47 | |
| *** sams-gleb has joined #openstack-swift | 09:56 | |
| *** chenhuayi__ has quit IRC | 10:08 | |
| *** geaaru has quit IRC | 10:09 | |
| *** bkopilov has quit IRC | 10:14 | |
| *** geaaru has joined #openstack-swift | 10:22 | |
| *** mvk has quit IRC | 10:30 | |
| *** dikonoor has quit IRC | 10:53 | |
| *** asettle has joined #openstack-swift | 10:55 | |
| *** geaaru has quit IRC | 10:57 | |
| *** StraubTW has joined #openstack-swift | 10:59 | |
| *** mvk has joined #openstack-swift | 11:03 | |
| *** ChubYann has quit IRC | 11:03 | |
| *** StraubTW has quit IRC | 11:04 | |
| *** geaaru has joined #openstack-swift | 11:11 | |
| *** dikonoor has joined #openstack-swift | 11:14 | |
| *** dikonoor has quit IRC | 11:42 | |
| *** npf has joined #openstack-swift | 11:43 | |
| *** dikonoor has joined #openstack-swift | 11:49 | |
| *** Guess456787654 has joined #openstack-swift | 11:56 | |
| *** catintheroof has joined #openstack-swift | 12:16 | |
| *** catinthe_ has joined #openstack-swift | 12:20 | |
| *** catintheroof has quit IRC | 12:21 | |
| *** vint_bra has joined #openstack-swift | 12:39 | |
| *** bkopilov has joined #openstack-swift | 12:46 | |
| *** d0ugal has quit IRC | 12:51 | |
| *** d0ugal has joined #openstack-swift | 13:00 | |
| *** cbartz has quit IRC | 13:03 | |
| *** sams-gle_ has joined #openstack-swift | 13:06 | |
| *** sams-gleb has quit IRC | 13:06 | |
| openstackgerrit | Thomas Herve proposed openstack/swift: Handle deleted files with if-none-match https://review.openstack.org/395582 | 13:11 |
| *** chenhuayi has joined #openstack-swift | 13:18 | |
| *** jistr is now known as jistr|brb | 13:26 | |
| *** jistr|brb is now known as jistr | 13:46 | |
| *** catintheroof has joined #openstack-swift | 13:47 | |
| *** catinthe_ has quit IRC | 13:50 | |
| *** npf has quit IRC | 13:51 | |
| *** geaaru has quit IRC | 14:10 | |
| *** chenhuayi has quit IRC | 14:14 | |
| *** StraubTW has joined #openstack-swift | 14:14 | |
| openstackgerrit | Mahati Chamarthy proposed openstack/swift: Handle deleted files with if-none-match https://review.openstack.org/395582 | 14:16 |
| *** StraubTW_ has joined #openstack-swift | 14:18 | |
| *** StraubTW has quit IRC | 14:20 | |
| *** sgundur has joined #openstack-swift | 14:28 | |
| *** klamath has joined #openstack-swift | 14:30 | |
| *** dmorita has joined #openstack-swift | 14:31 | |
| *** klamath has joined #openstack-swift | 14:31 | |
| *** sgundur_ has joined #openstack-swift | 14:32 | |
| *** sgundur has quit IRC | 14:34 | |
| *** sgundur_ has quit IRC | 14:34 | |
| *** sgundur has joined #openstack-swift | 14:34 | |
| *** dmorita has quit IRC | 14:35 | |
| *** sgundur has quit IRC | 14:40 | |
| *** links has quit IRC | 14:49 | |
| *** cbartz has joined #openstack-swift | 15:10 | |
| *** diogogmt has quit IRC | 15:13 | |
| *** sams-gle_ has quit IRC | 15:15 | |
| *** sams-gleb has joined #openstack-swift | 15:15 | |
| *** sams-gleb has quit IRC | 15:20 | |
| *** sams-gleb has joined #openstack-swift | 15:34 | |
| *** tsg has joined #openstack-swift | 15:46 | |
| *** jerrygb has joined #openstack-swift | 15:57 | |
| *** _JZ_ has joined #openstack-swift | 16:00 | |
| *** vint_bra has quit IRC | 16:01 | |
| *** cebruns_ is now known as cebruns | 16:01 | |
| *** geaaru has joined #openstack-swift | 16:10 | |
| *** rcernin has quit IRC | 16:13 | |
| *** klrmn has joined #openstack-swift | 16:16 | |
| *** acoles_ is now known as acoles | 16:21 | |
| *** Guest39033 has quit IRC | 16:25 | |
| *** agarner has joined #openstack-swift | 16:26 | |
| *** dikonoor has quit IRC | 16:31 | |
| *** bkopilov has quit IRC | 16:39 | |
| *** david_c has joined #openstack-swift | 16:44 | |
| *** bkopilov has joined #openstack-swift | 16:46 | |
| *** sgundur has joined #openstack-swift | 16:50 | |
| *** sgundur_ has joined #openstack-swift | 16:57 | |
| *** sgundur has quit IRC | 16:57 | |
| *** admin6 has quit IRC | 16:57 | |
| notmyname | hello, world | 17:01 |
| *** chlong has quit IRC | 17:08 | |
| *** klrmn has quit IRC | 17:08 | |
| *** sgundur_ has quit IRC | 17:11 | |
| *** sgundur has joined #openstack-swift | 17:12 | |
| *** sgundur has quit IRC | 17:17 | |
| *** chlong has joined #openstack-swift | 17:17 | |
| *** sgundur has joined #openstack-swift | 17:17 | |
| *** sgundur has quit IRC | 17:21 | |
| *** sgundur has joined #openstack-swift | 17:22 | |
| *** sgundur has quit IRC | 17:23 | |
| *** asettle has quit IRC | 17:26 | |
| *** dmorita has joined #openstack-swift | 17:28 | |
| tdasilva | notmyname: o/ | 17:35 |
| *** bkopilov has quit IRC | 17:37 | |
| *** sgundur has joined #openstack-swift | 17:44 | |
| *** sgundur_ has joined #openstack-swift | 17:51 | |
| *** bkopilov has joined #openstack-swift | 17:54 | |
| openstackgerrit | Drew Balfour proposed openstack/swift: WIP: add byteorder information and logic to ring files https://review.openstack.org/395237 | 17:55 |
| *** cbartz has left #openstack-swift | 17:56 | |
| *** rledisez has quit IRC | 17:59 | |
| *** sgundur_ has quit IRC | 18:00 | |
| *** sgundur_ has joined #openstack-swift | 18:01 | |
| *** sgundur_ has quit IRC | 18:01 | |
| *** sgundur_ has joined #openstack-swift | 18:02 | |
| abalfour | So... I need some followup advice on that byteorder changeset. Currently it breaks the test_get_ring_md5_missing_ring_recovers_without_restart test in test/unit/common/middleware/test_recon.py, because it's trying to md5sum a generated ring file against a hard coded m5sum. With the new field in the ring file, the m5sum is different. The problem is that new field isn't static; if I run it on x64, I get one value (and one m5sum) whereas | 18:03 |
| *** mvk has quit IRC | 18:05 | |
| notmyname | abalfour: many of the ring tests are deliberately fragile like that so that it's super obvious when ring code changes. | 18:05 |
| notmyname | abalfour: ...since changing the ring is a Big Deal for deployed clusters | 18:06 |
| abalfour | sure, that makes sense, just wasn't sure the best way to fix it. | 18:06 |
| notmyname | abalfour: most likely, simply by changing the test to the newly correct md5 | 18:07 |
| abalfour | right, but the new md5sum isn't a constant anymore. it's endian dependent. | 18:08 |
| notmyname | neat | 18:08 |
| *** klrmn has joined #openstack-swift | 18:08 | |
| abalfour | my thought was to just have two sets of md5sums, and compare based on endianess. | 18:08 |
| notmyname | yeah, that was my first thought too | 18:09 |
| abalfour | ok, I shall make it so. | 18:09 |
| abalfour | thanks for the sanity check. | 18:09 |
| *** htruta has joined #openstack-swift | 18:09 | |
| timburke | or patch sys.byteorder? | 18:09 |
| abalfour | Sure, that'd work too but than it's intentionally avoiding a testcase? | 18:10 |
| *** oshritf has quit IRC | 18:11 | |
| *** jordanP has quit IRC | 18:19 | |
| *** asettle has joined #openstack-swift | 18:27 | |
| *** asettle has quit IRC | 18:31 | |
| *** david_c_ has joined #openstack-swift | 18:32 | |
| *** david_c has quit IRC | 18:33 | |
| *** david_c_ is now known as david_c | 18:33 | |
| *** david_c has quit IRC | 18:39 | |
| notmyname | remember the time change. the team meeting is at 2100UTC (2 hours and 20 minutes from now) | 18:39 |
| *** david_c has joined #openstack-swift | 18:40 | |
| *** clu_ has joined #openstack-swift | 18:44 | |
| *** acoles is now known as acoles_ | 18:49 | |
| *** mvk has joined #openstack-swift | 18:51 | |
| *** sgundur_ has quit IRC | 18:56 | |
| *** sgundur_ has joined #openstack-swift | 18:58 | |
| *** ChubYann has joined #openstack-swift | 18:59 | |
| *** rcernin has joined #openstack-swift | 18:59 | |
| *** diogogmt has joined #openstack-swift | 19:03 | |
| *** catinthe_ has joined #openstack-swift | 19:10 | |
| openstackgerrit | John Dickinson proposed openstack/python-swiftclient: 3.2.0 release notes https://review.openstack.org/395768 | 19:10 |
| *** catintheroof has quit IRC | 19:12 | |
| *** sgundur_ has quit IRC | 19:13 | |
| *** sgundur_ has joined #openstack-swift | 19:14 | |
| *** geaaru has quit IRC | 19:16 | |
| *** vint_bra has joined #openstack-swift | 19:18 | |
| openstackgerrit | John Dickinson proposed openstack/python-swiftclient: 3.2.0 release notes https://review.openstack.org/395768 | 19:25 |
| openstackgerrit | Nandini Tata proposed openstack/swift: Allow custom swift configuration directory https://review.openstack.org/393952 | 19:26 |
| *** rcernin has quit IRC | 19:55 | |
| *** sgundur_ has quit IRC | 20:00 | |
| *** sgundur_ has joined #openstack-swift | 20:06 | |
| *** sgundur_ has quit IRC | 20:12 | |
| clayg | timburke: I seem to be plateau'ing at 3 attempts to find `swift info --json | jq .swift.policies[].name` - but it feels *really* close - I think I'm going to break through from 3 -> first try any minute | 20:14 |
| *** silor has quit IRC | 20:14 | |
| *** Guess456787654 has quit IRC | 20:33 | |
| openstackgerrit | Christian Hugo proposed openstack/swift: Raise ValueError if a config section does not exist https://review.openstack.org/393388 | 20:52 |
| *** StraubTW_ has quit IRC | 20:53 | |
| *** tmoreira has quit IRC | 20:57 | |
| *** cschwede has quit IRC | 20:57 | |
| notmyname | swift team meeting time (in 2 minutes) | 20:58 |
| notmyname | I expect it to be relatively quick | 20:59 |
| kota_ | good morning | 20:59 |
| clayg | kota_: hello! Just put my +10000 on patch 393595 | 20:59 |
| patchbot | https://review.openstack.org/#/c/393595/ - liberasurecode - Fix error handling on gf_ivnert_matrix in isa-l ba... | 20:59 |
| *** joeljwright has joined #openstack-swift | 21:00 | |
| *** ChanServ sets mode: +v joeljwright | 21:00 | |
| notmyname | swift team meeting in #openstack-meeting | 21:00 |
| kota_ | clayg: thx! | 21:00 |
| *** admin6 has joined #openstack-swift | 21:01 | |
| openstackgerrit | Christian Hugo proposed openstack/swift: Raise ValueError if a config section does not exist https://review.openstack.org/393388 | 21:10 |
| jrichli | last but, if, we, fix, patchbot then I can't make it do this | 21:12 |
| patchbot | this | 21:12 |
| jrichli | doh! I thought it was gonna be worse. ill have to figure out what the pattern is | 21:12 |
| *** sgundur_ has joined #openstack-swift | 21:13 | |
| *** StraubTW has joined #openstack-swift | 21:15 | |
| *** sams-gleb has quit IRC | 21:17 | |
| *** sams-gleb has joined #openstack-swift | 21:17 | |
| *** sams-gleb has quit IRC | 21:22 | |
| *** admin6 has quit IRC | 21:25 | |
| *** asettle has joined #openstack-swift | 21:28 | |
| *** cschwede has joined #openstack-swift | 21:28 | |
| *** m_kazuhiro has joined #openstack-swift | 21:29 | |
| *** tmoreira has joined #openstack-swift | 21:30 | |
| *** asettle has quit IRC | 21:33 | |
| *** Jeffrey4l has quit IRC | 21:34 | |
| *** ntata_ has joined #openstack-swift | 21:35 | |
| *** ntata_ has quit IRC | 21:38 | |
| *** ntata_ has joined #openstack-swift | 21:48 | |
| *** ntata_ has quit IRC | 21:51 | |
| *** ChanServ sets mode: +v cschwede | 21:58 | |
| notmyname | cschwede: I think I had a question for you... let me see if I can remember it | 22:01 |
| cschwede | notmyname: well, that gives me the chance to talk to you :) | 22:01 |
| cschwede | notmyname: regarding the pyeclib discussions we had in Barcelona | 22:01 |
| cschwede | notmyname: the idea was to enable EC funtional testing in the gate, right? and probetests too? | 22:01 |
| notmyname | cschwede: that sounds like something I'd like :-) | 22:03 |
| notmyname | cschwede: but I don't remember the challenge of that. we've already got EC tests in the gate for swift... | 22:03 |
| kota_ | clayg: the libeasurecode fix for isa-l https://review.openstack.org/#/c/393595/3 depends on https://review.openstack.org/#/c/387879/2 and pach 3993595 is using a part of the patch 3993595 to enable to run non-args test function. | 22:03 |
| patchbot | https://review.openstack.org/#/c/3993595/ | 22:03 |
| patchbot | patch 393595 - liberasurecode - Fix error handling on gf_ivnert_matrix in isa-l ba... | 22:03 |
| patchbot | patch 387879 - liberasurecode - Fix liberasurecode skipping a bunch of invalid_arg... | 22:03 |
| kota_ | that's what I'd like to say in the end of irc meeting. | 22:04 |
| notmyname | kota_: thanks :-) | 22:04 |
| kota_ | :-) | 22:04 |
| kota_ | if we don't mind not running the unit tests, we can land patch 393595 first but i prefer running unit test always helps from regression | 22:05 |
| patchbot | https://review.openstack.org/#/c/393595/ - liberasurecode - Fix error handling on gf_ivnert_matrix in isa-l ba... | 22:05 |
| notmyname | cschwede: were we talking about adding them for the pyeclib/liberasurecode projects? | 22:06 |
| cschwede | notmyname: they already exist for the pyeclib project? | 22:06 |
| * kota_ is going to find breakfast | 22:06 | |
| clayg | kota_: sorry missed that on the liberasurecode test patch - i'm looking into that now - thanks | 22:09 |
| clayg | cschwede: nope, but I think maybe we have some non-dsvm functests with ec policy? acoles_ would know | 22:10 |
| notmyname | clayg: yes, we definitely have that | 22:11 |
| notmyname | I've been working on that part of the test infra for the xattr checksumming | 22:11 |
| *** joeljwright has quit IRC | 22:12 | |
| cschwede | clayg: looks to me like tests for pyeclib are properly running in the gate: http://logs.openstack.org/56/393656/1/check/gate-pyeclib-python27-ubuntu-xenial/321e5ef/console.html.gz or am i missing something here? | 22:13 |
| notmyname | cschwede: seems that it's still using an older liberasurecode | 22:14 |
| cschwede | notmyname: yes, that is likely | 22:15 |
| notmyname | this is coming back to the issue we've come to in many places, it seems | 22:15 |
| notmyname | in the openstack CI system, how do we pull in and test against compiled code? | 22:15 |
| notmyname | matters for the EC libs. matters for golang | 22:15 |
| cschwede | notmyname: well, i need to look into that - what’s the current way. i’ll check that out and get back to you | 22:16 |
| notmyname | cschwede: AFAICT the current way is to pull whatever's in the distro packages. eg ubuntu xenial | 22:16 |
| notmyname | point being, there's not a current way to pull sources for a dependency and build it and then test against *that* | 22:17 |
| notmyname | or there might be a way, but we're not doing it | 22:17 |
| notmyname | fungi: ^ this is one of our current big questions. relates to golang, but currently we're stuggling with it for a C-lang dependency | 22:17 |
| cschwede | notmyname: IIRC sometimes pip pulls in sources and even compiles C stuff then? | 22:18 |
| notmyname | cschwede: like in general it can do that or specifically with one of our dependencies now? | 22:19 |
| fungi | right, it's not that there's _no_ way, but that "the way" depends on the situation, so you may need to script itt | 22:19 |
| fungi | if your need is to compile c extensions for a python sdist, that is fairly straightforward to tackle in traditional python packaging ways | 22:20 |
| notmyname | fungi: current state of things is that we depend on pyeclib (python c bindings) which depends on liberasurecode. we'd like to pull in a newer version of liberasurecode and use that for testing swift (and pyeclib for that matter) | 22:20 |
| fungi | right, so you'd need to script that | 22:21 |
| notmyname | fungi: so a script in a builder (or called from a builder) that clones, builds, installs, then tests? something like that? | 22:21 |
| fungi | write a script to retrieve liberasurecode $somehow from $somewhere and then follow whatever its recommended build process is, including all that in some place your job | 22:21 |
| fungi | er, some place in your job | 22:22 |
| notmyname | right. ok | 22:22 |
| fungi | notmyname: yep | 22:22 |
| fungi | and if the pyeclib sdist needs to be rebuilt against that, you may need to do something to avoid using prebuilt pyeclib wheels for the platform you're running on | 22:23 |
| fungi | or make a separate step to rebuild pyeclib and install it separate from the requirements installation process | 22:23 |
| notmyname | fungi: ok. thanks | 22:25 |
| fungi | though that may depend on whether the symbols have changed in the rebuilt lib | 22:26 |
| *** m_kazuhiro has quit IRC | 22:32 | |
| *** admin6 has joined #openstack-swift | 22:41 | |
| *** admin6 has left #openstack-swift | 22:41 | |
| *** chlong has quit IRC | 22:47 | |
| openstackgerrit | Tim Burke proposed openstack/pyeclib: Use Py_BuildValue instead of manually building dicts https://review.openstack.org/395856 | 22:47 |
| clayg | zaitcev: you put a +2 on patch 387879 - but I'm not 100% what's going on with that test infrastructe - does my comment/question make sense? Should we land this and revist or should just land this and ignore my ignorant confusion? | 22:54 |
| patchbot | https://review.openstack.org/#/c/387879/ - liberasurecode - Fix liberasurecode skipping a bunch of invalid_arg... | 22:54 |
| clayg | timburke: I thought that Py_BuildValue stuff was related to the memory leak fix? | 22:55 |
| zaitcev | clayg: sec... | 22:55 |
| notmyname | tsg: yay! | 22:58 |
| notmyname | oh, and kevin looked too | 22:58 |
| tsg | notmyname: :^) still around, yes! | 22:58 |
| tsg | yep | 22:58 |
| tsg | even ran it by Mr ISA-L | 22:58 |
| notmyname | I appreciate it | 22:58 |
| tsg | he is going to look into a "fix" Kevin suggested | 22:58 |
| tsg | (to upstream ISA-L that is) | 22:59 |
| clayg | zomg really!? | 23:01 |
| timburke | clayg: kinda? that one looks like it was actually doing things right, but +7/-43 seems... nice. less to think about. compare against the diff at https://review.openstack.org/#/c/385185/1/src/c/pyeclib_c/pyeclib_c.c (that fixes the leak), and you'll see that one didn't have all the dangling PyObject references and (importantly) the calls to Py_XDECREF | 23:02 |
| patchbot | patch 385185 - pyeclib - Fix formatted get_metadata memory leak | 23:02 |
| clayg | timburke: yes! I'm very confused about https://review.openstack.org/#/c/385185/1/src/c/pyeclib_c/pyeclib_c.c on the surface? | 23:03 |
| patchbot | patch 385185 - pyeclib - Fix formatted get_metadata memory leak | 23:03 |
| kota_ | tsg: thanks for looking at! | 23:03 |
| clayg | it *seems* like ti removed a bunch of input validation? | 23:03 |
| *** sgundur_ has quit IRC | 23:03 | |
| *** catinthe_ has quit IRC | 23:04 | |
| clayg | zaitcev: it may not matter - now that tsg +2's the matrix validation change that *depends* on the test change I'm pretty inclined to say whatever followup if any on the test infrastructure stuff should indeed be a follow up | 23:04 |
| zaitcev | clayg: well I see you did +2 it, so I deduced your position | 23:05 |
| zaitcev | (I was trying to see if I'm missing something obvious, like maybe use your simple fix from the comments) | 23:05 |
| clayg | yeah i mean that diff *did* make it so running the test suite w/o kota's fixes on those few tests *failed* | 23:07 |
| clayg | so I think it was at least *running* the ones that got fixed | 23:07 |
| clayg | ... and kota's change is causing at least test_backend_available_invalid_args 's *name* to appear in my test output multiple times for each backend idx | 23:07 |
| clayg | so... i'm assuming it's running them more | 23:08 |
| openstackgerrit | Merged openstack/liberasurecode: Fix liberasurecode skipping a bunch of invalid_args tests https://review.openstack.org/387879 | 23:08 |
| timburke | clayg: on the leak -- nope, just skipped a bunch of checks for <0 returns from PyDict_SetItemString, since Py_BuildValue should be dealing with that. and before it didn't even check whether any of the Py*_From* calls failed (say, due to ENOMEM) | 23:08 |
| clayg | looking at the test runner dff it's like - "if this test calls for backends_max just run it" (instead of only run it if args does not return null) | 23:09 |
| kota_ | if something to ask on that, please ping me, i'm in parallel conversation at irc now but can be back qucicky here. | 23:09 |
| kota_ | clayg: the test fix is for enabling passing (approximately) 100 tests to run correctly | 23:10 |
| clayg | timburke: ahhhh! | 23:10 |
| clayg | timburke: i suck - i was thinking that < 0 was checking the ->blah_blah was > 0 | 23:10 |
| kota_ | probably you can see the diff in the test result (i.e. # of unit test running, w/o patch is around 200 and then w/ patch is around 300) | 23:10 |
| clayg | today I'm all like "math or c" -- "how do they work!?" | 23:11 |
| clayg | kota_: no rush - read my comment on the patch if you get a chance - it's +A'd for now | 23:11 |
| clayg | tdasilva: that means you're up soon! | 23:11 |
| timburke | clayg: yeah, https://docs.python.org/2/c-api/dict.html#c.PyDict_SetItem and https://docs.python.org/2/c-api/arg.html#c.Py_BuildValue were great resources as i was looking into it | 23:12 |
| clayg | tdasilva: do you have to... idk update a changelog or something? what's a release *really* besides some random sha | 23:12 |
| *** patchbot has quit IRC | 23:13 | |
| *** patchbot has joined #openstack-swift | 23:13 | |
| *** patchbot has quit IRC | 23:15 | |
| *** patchbot has joined #openstack-swift | 23:16 | |
| *** jerrygb has quit IRC | 23:16 | |
| *** asettle has joined #openstack-swift | 23:17 | |
| notmyname | clayg: yeah, looks like there's a ChangeLog file that should be updated | 23:17 |
| clayg | notmyname: I DON'T KNOW HOW TO DO THAT *freakout* | 23:19 |
| notmyname | clayg: I can do it, or I can point you in the right direction so you know how | 23:20 |
| *** jerrygb has joined #openstack-swift | 23:23 | |
| *** david_c has quit IRC | 23:24 | |
| clayg | notmyname: you should do it if you are able and tdasilva is not available - and also point me in the right direction if there's things I can read so that next time we're not in a crunch i'm better able to cover if we're not deep enough up there | 23:25 |
| timburke | notmyname: maybe we *do* need to do some release-shadowing | 23:25 |
| clayg | i have zero desire to know this stuff beyond bus-factor | 23:25 |
| clayg | ... or notmyname-is-on-a-plane-for-the-next-14-hours-factor | 23:26 |
| *** david_c has joined #openstack-swift | 23:26 | |
| notmyname | that like never happens. ;-) | 23:26 |
| notmyname | looks like the skipping tests patch made https://review.openstack.org/#/c/393595/ have a merge conflict | 23:27 |
| patchbot | patch 393595 - liberasurecode - Fix error handling on gf_ivnert_matrix in isa-l ba... | 23:27 |
| notmyname | or not? I don't know whcih one. point is, merge conflict | 23:27 |
| kota_ | notmyname: maybe the isa-l change depends on patch set 3 (or 2?) so | 23:28 |
| openstackgerrit | Clay Gerrard proposed openstack/liberasurecode: Fix error handling on gf_ivnert_matrix in isa-l backend https://review.openstack.org/393595 | 23:28 |
| kota_ | i think we can rebase easily to run git rebase to master and then skipping the test fix | 23:29 |
| kota_ | it looks clayg already did. | 23:29 |
| notmyname | yeah. looks like gerrit detected it as a trivial rebase, so kept the reviews | 23:30 |
| notmyname | I clicked +A | 23:30 |
| *** asettle has quit IRC | 23:30 | |
| kota_ | thx | 23:30 |
| clayg | nice! | 23:31 |
| *** Jeffrey4l has joined #openstack-swift | 23:31 | |
| zaitcev | superficially looks obvious to me too, but I see you guys already are on it | 23:31 |
| notmyname | kota_: clayg: what do you want the changelog summary to be? | 23:31 |
| clayg | yeah it's master -> testfix -> matrixfix now | 23:31 |
| zaitcev | there's a free() in out: that works | 23:31 |
| clayg | notmyname: `git log last-tag.. --oneline` | 23:32 |
| *** Jeffrey4l has quit IRC | 23:32 | |
| clayg | zaitcev: free() in-n-out!? | 23:32 |
| *** Jeffrey4l has joined #openstack-swift | 23:32 | |
| *** Jeffrey4l has quit IRC | 23:33 | |
| *** Jeffrey4l has joined #openstack-swift | 23:34 | |
| *** _JZ_ has quit IRC | 23:34 | |
| openstackgerrit | Merged openstack/liberasurecode: Fix error handling on gf_ivnert_matrix in isa-l backend https://review.openstack.org/393595 | 23:36 |
| tdasilva | clayg, notmyname: sorry, just taking off now, if it can wait until tomorrow, i can take care of it...but feel free to go ahead without me | 23:36 |
| *** kei_yama has joined #openstack-swift | 23:36 | |
| notmyname | tdasilva: I got it | 23:36 |
| openstackgerrit | John Dickinson proposed openstack/liberasurecode: 1.3.0 release https://review.openstack.org/395871 | 23:37 |
| notmyname | clayg: kota_: ^ look ok? | 23:37 |
| clayg | wfm | 23:39 |
| clayg | nice work | 23:39 |
| notmyname | I'm just pulling levers. you and kota_ did all the hard parts | 23:39 |
| kota_ | notmyname: +2ed | 23:40 |
| notmyname | ok, should be five whole minutes (or so) until that lands | 23:41 |
| kota_ | great | 23:41 |
| openstackgerrit | Merged openstack/liberasurecode: 1.3.0 release https://review.openstack.org/395871 | 23:44 |
| notmyname | bah! speed bump | 23:45 |
| notmyname | ! [remote rejected] 1.3.0 -> 1.3.0 (prohibited by Gerrit) | 23:45 |
| openstack | notmyname: Error: "remote" is not a valid command. | 23:45 |
| notmyname | so it's likely they've turned of pushing tags globally | 23:45 |
| notmyname | but let me check | 23:46 |
| notmyname | ah ha! | 23:48 |
| notmyname | done! (had to be a signed tag) | 23:48 |
| notmyname | ok. it's an official release and is done | 23:48 |
| notmyname | kota_: so now we need to make pyeclib depend on liberasurecode>=1.3.0 | 23:49 |
| timburke | huh. i wonder if openstack is also a supybot-powered bot... | 23:49 |
| notmyname | timburke: it is | 23:50 |
| notmyname | timburke: and there are those who have suggested that any new bot should be functionality added to openstack and managed by infra | 23:50 |
| notmyname | timburke: if you want to work on an openstack-bot project, I'll nominate you for PTL ;-) | 23:51 |
| kota_ | notmynam: k, will start to work on pyeclib | 23:52 |
| notmyname | kota_: are you just going to add a version specifier to bindep.txt or do you need to plumb some more stuff through (error checking?) | 23:52 |
| kota_ | notmname: thinking since today's irc meeting, i'm realizing using bindep for eunsuring liberasurecode >= 1.3.0 may not work well | 23:53 |
| notmyname | why's that? | 23:53 |
| kota_ | notmyname: i'm not entirly sure because the bindep is for packaging repository (apt/yum), right? | 23:53 |
| kota_ | notmyname: and no distros right now and we may be able to install liberasurecode from sorce | 23:54 |
| kota_ | source | 23:54 |
| notmyname | right. no distros have it yet. but that's ok (right?). I don't know if we need to do the build-from-source stuff that fungi was talking about first | 23:54 |
| kota_ | If i understand correctly, the procedure, 1. try to install liberasurecode 1.1.0 in xenial then 2. try to install from source due to incmpatibility | 23:55 |
| kota_ | then, how bindep works on that? | 23:55 |
| notmyname | I don't think there's any reason to do step 1 | 23:55 |
| kota_ | exactly | 23:55 |
| notmyname | no, bindep just gives a list of packages to install | 23:55 |
| kota_ | yes | 23:55 |
| kota_ | i'm telling how to the version constraint works on bindep | 23:56 |
| kota_ | s/how to/how/ | 23:56 |
| notmyname | ok, so we can't release new pyeclib until we can bump the dependency and we can't do the dependency bump until a distro updates to the new liberasurecode | 23:56 |
| kota_ | in my idea | 23:56 |
| notmyname | ah, and you're saying we set bindep to >=1.3.0 and then when that fails fall back to install from source? | 23:57 |
| kota_ | we can validate the liberasurecode version when loading shard object file (.so) for init. | 23:57 |
| kota_ | notmyname: yes | 23:57 |
| kota_ | my idea is instead of >=1.3.0 in bindep | 23:57 |
| *** dfflanders has joined #openstack-swift | 23:58 | |
| kota_ | if we can have time, i will work on that capability at least today in my time. | 23:58 |
| notmyname | makes me wonder why we have pyeclib and liberasurecode in 2 separate repos | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!