21:01:56 #startmeeting swift 21:01:57 Meeting started Wed May 29 21:01:56 2019 UTC and is due to finish in 60 minutes. The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:01:58 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:02:00 The meeting name has been set to 'swift' 21:02:06 o/ 21:02:12 who's here for the swift meeting? 21:02:13 o/ 21:02:15 o/ 21:03:00 meeting agenda: 21:03:03 #link https://wiki.openstack.org/wiki/Meetings/Swift 21:03:41 #topic migration to story board 21:04:02 diablo_rojo_phon and fungi have done a great job of getting our test migration squared away! 21:04:18 #link https://storyboard-dev.openstack.org/#!/project/openstack/swift 21:04:27 ...now has lots of stories 21:04:58 glad to hear it! 21:05:40 i hit some curious errors while playing around with it last week and trying to create a new story with multiple tasks 21:05:50 my chrome says `NET::ERR_CERT_AUTHORITY_INVALID` 21:06:04 yeah -- self-signed cert i'm afraid 21:06:19 oh, it uses that. 21:06:44 yeah, the -dev server doesn't have a purchased cert 21:06:45 uhhh... 21:06:53 oic - sorry 21:07:11 when we rename it to storyboard-dev.opendev.org we've got letsencrypt automation for that domain we can leverage 21:07:34 but openstack.org is hard due to shared dns jurisdiction with the osf staff 21:07:58 got it 21:08:27 so we (selectively) buy ssl certs for important production services in openstack.org 21:09:12 for example, the production storyboard.openstack.org has a purchased cert 21:09:33 anyway, i'd like everyone to try it out in the sandbox -- open and close some stories, add comments, patches -- start to get familiar with it and develop some opinions about it 21:10:02 +1 21:10:09 opinions welcome! 21:10:19 Let us know if you have any questions! 21:11:18 also be aware it's under continuous development/improvement and the devs are a friendly lot who are interested in your use cases (as long as they can fit them to some reasonably generalized model) 21:11:41 one (hopefully quick) question: it feels a little sluggish to me -- is that just because it's a sandbox area and so a bit under-provisioned? 21:12:24 it's more because the ssl queries are really inefficient and in need of optimization. we've got slow query logging turned on now to try and find some hotspots, and some changes in flight which ought to help 21:12:27 The queries need some optimization which we actually have an outreachy intern starting to work on now. 21:12:51 got it. cool! thanks guys 21:13:05 in the spirit of not prematurely optimizing, it was fast enough before we imported loads of historical data ;) 21:13:07 #topic libec release 21:13:27 heh, fair enough 21:13:34 the release went out! 21:13:49 🙌 21:13:50 nice 21:13:58 #link https://opendev.org/x/liberasurecode/releases 21:14:08 ...which now has a nice 1.6.1 21:15:00 side note, hopefully that will be openstack/liberasurecode soon enough, and we'll get the mirroring to GH going again 21:15:49 see also, https://review.opendev.org/#/c/657154/ https://review.opendev.org/#/c/657155/ and finally https://review.opendev.org/#/c/661779/ 21:16:21 though see my note in the governance change, its possible to transfer gh projects to a different org so redirects are added, and zuul can replicate git commits/branches/tags to any remote git hosting 21:16:33 *ahem* https://review.opendev.org/#/c/657154/ https://review.opendev.org/#/c/657155/ and finally https://review.opendev.org/#/c/661779/ 21:16:34 patch 657154 - governance - Add liberasurecode and pyeclib as Swift team deliv... - 3 patch sets 21:16:36 patch 657155 - project-config - Rename x/pyeclib and x/liberasurecode to openstack/* - 4 patch sets 21:16:37 patch 661779 - governance - Rename x/liberasurecode and x/pyeclib to openstack/* - 1 patch set 21:17:17 that's about all i had on that 21:17:25 #topic py3 21:18:56 diskfile's ported! but i think we hit a snag on tempurl -- in particular, we noticed that we're sticking wsgi-strings in memcached, which isn't going to be great when going to get tempurl keys :-( 21:19:41 :( 21:19:53 my initial hack to get func tests passing broke a bunch of unit tests, and i think we need to revisit how we handle responses in get_account_info and get_container_info instead 21:21:24 i'm currently trying to pull that apart, but found another python bug (to do with non-ascii header keys), so... bleh 21:22:07 still, there's a lot of good stuff to review and land; the priority reviews page should still be up-to-date 21:22:10 #link https://wiki.openstack.org/wiki/Swift/PriorityReviews 21:22:39 who's available to review some of the remaining patches? 21:23:04 we've got https://review.opendev.org/#/c/660542/ 21:23:05 patch 660542 - swift - py3: port ssync - 4 patch sets 21:23:09 I'll look at the ssync patch and try it with a py3 object server (sorry I'm late) 21:23:26 yay! thanks, alecuyer 21:23:58 1 or 2 would be... to me to try to take a look in this week. not so much. 21:24:42 there's also https://review.opendev.org/#/c/652819/ (though we might need more thought on how to test stats-logging) 21:24:43 patch 652819 - swift - py3: port obj/reconstructor - 2 patch sets 21:25:13 and the only remaining middlewares are https://review.opendev.org/#/c/654671/ and https://review.opendev.org/#/c/652928/ 21:25:13 patch 654671 - swift - py3: port symlink middleware - 3 patch sets 21:25:15 patch 652928 - swift - py3: Port the tempurl middleware - 3 patch sets 21:26:00 I'll try to get symlink 21:26:07 thanks kota_! 21:27:36 given that we're at the end of the month already, i'm inclined to prioritize getting the unit tests ported, then rely on some manual func testing (still running under py2) to get us the confidence to cut a tag declaring experimental support 21:28:32 timburke: where "manual func testing" is like - everything starts and mostly seems to work - upload and download, *LO, tempurl, etc - barring a few known issues and failing functests? 21:29:25 and that's of *python3* daemons - we expect when everything is running on python2 everything will continue to work with no observable differences or upgrade impact (as long as you stay on python2) 21:29:52 i.e. we ship a py3-rc that is experimental with some caveats for testing? 21:29:57 yes to all of that 21:30:26 timburke: Did you find a way to handle the BoundedSemaphore being passed to paste? 21:30:30 and call out that we'd like people to be testing with it and reporting any bugs found in the release notes 21:31:05 alecuyer, clayg had an idea to just move the call that shoves it into the config until after paste does its thing 21:31:46 i'll work on getting a patch up that does that 21:31:49 yeah that seemed to allow things to start (strangely?) but didn't verify that ssync can still count & reject connections 🤷‍♂️ 21:32:05 oh ok (that's one thing I hit when testing py3 obj server) 21:32:18 it'll be great 21:33:24 or we could just call off all this py3 maddness and move over to tauthon??? 21:33:59 if moving it doesn't allow us to correctly reject connections (ie, breaks py2) i think i'm fine with known-issuing the fact that replication_concurrency must be 0 on py3 21:34:03 ahah :D 21:35:14 clayg: does it introduce defer and go keywords? 21:35:42 next step for me is going to be reorganizing the reviews page to have a list of patches needed for a 2.22 release 21:36:47 (which i'll probably seed with mostly py3-related patches, but you all should feel free to update with other blockers as you see fit) 21:36:53 #topic lots of small files 21:37:09 rledisez, alecuyer, kota_: how's it going? 21:37:40 kota_ wrote some tests, https://review.opendev.org/#/c/659515/ 21:37:41 patch 659515 - swift (feature/losf) - Add small unit tests for vfile and related modules - 2 patch sets 21:38:33 I pushed another patch there, kota_ when you have time if you can take a look that would be great 21:38:44 and another patch is here https://review.opendev.org/#/c/660381/ 21:38:45 patch 660381 - swift (feature/losf) - Add tests for vfile_utils - 1 patch set 21:39:05 sure. mine is just an entry point so I'd love to get more unit tests to get more coverage. 21:39:30 so if the approach seems OK I will write more 21:40:05 i didn't get enough time to be online in the last week so I'd start to catch up the changes in this week. 21:40:12 and another thing, at the PTG we said LOSF is 'additive' , which is true in that it adds new files. But in inherits code from diskfile.py, so you can actually break from changes in diskfile.py 21:40:45 and I'd like your thoughts (maybe not now but think about it :) ) of that situation vs code duplication where LOSF would not inherit from diskfile.py 21:41:07 the new tests are great! we should probably ensure that they pass under py3 from the get-go 21:41:28 Yes, I'm sure I have some work there 21:41:37 yeah, i think there's a chance that i've already managed to break you once or twice with the py3 work... 21:43:15 i think as long as we have a plan to get it merged to master in some reasonable timeframe, it shouldn't be too bad, though, yeah? since at that point we'd have a losf gate job on master ensuring that we don't break things 21:44:13 To me it's fine, I was more wondering from other developer's point of view, what seems easiest for maintenance 21:45:33 hmmm 21:46:56 Just wanted to share this. We can come back to this another time 21:47:00 i don't know that *not* inheriting from BaseDiskFile has saved swiftonfile from any headaches... 21:47:24 i see, thanks for heading up. 21:48:07 i may look for a way to port *just tests* as experimental or non-voting to the master even if it's in under-development in the feature branch. 21:48:53 and conversely, i don't think that we've generally had many issues in having DiskFile and ECDiskFile sharing code 21:49:26 in my view, anyway, the biggest takeaway is that breakage will be minimized when all the code is in the same place: same repo, same branch 21:49:44 well, that's it for the agenda 21:49:49 #topic open discussion 21:49:59 o/ sorry I've pretty much missed the meeting, been dealing with flu season here, the whole family is down.. been fun. On the plus side the girls slept in today, my living alarm clocks :p 21:50:00 who has other business to bring up? 21:50:24 o/ mattoliverau! hope everyone feels better soon 21:50:56 Thanks, it's true what they say, when the baby is hit, the whole family goes down. 21:51:47 mattoliverau: sorry to hear that. take care of you all 21:53:03 all right, seems like a wrap then 21:54:12 i'll update the reviews wiki and hope for a release within the next week or two. i worry that if we push it out much further than that, we aren't going to have sufficient time to be confident in declaring full support by the end of the Train cycle 21:54:25 #endmeeting