Wednesday, 2020-12-16

*** yasufum has joined #openstack-meeting00:02
*** yasufum has quit IRC00:02
*** yasufum has joined #openstack-meeting00:02
*** hyunsikyang has quit IRC00:04
*** tosky has quit IRC00:04
*** hyunsikyang has joined #openstack-meeting00:04
*** yasufum has quit IRC00:07
*** yasufum has joined #openstack-meeting00:08
*** jmasud has joined #openstack-meeting00:50
*** jmasud has quit IRC00:55
*** macz_ has quit IRC00:58
*** rcernin_ has joined #openstack-meeting01:09
*** rcernin has quit IRC01:10
*** mlavalle has quit IRC01:30
*** _mlavalle_1 has joined #openstack-meeting01:30
*** kevinz has joined #openstack-meeting01:39
*** jmasud has joined #openstack-meeting01:58
*** jmasud has quit IRC02:04
*** yasufum has quit IRC02:05
*** jmasud has joined #openstack-meeting02:08
*** jmasud has quit IRC02:10
*** yasufum has joined #openstack-meeting02:16
*** dasp has quit IRC02:20
*** timburke has quit IRC02:21
*** dasp has joined #openstack-meeting02:21
*** yasufum_ has joined #openstack-meeting02:34
*** benj_- has joined #openstack-meeting02:35
*** yasufum has quit IRC02:35
*** yasufum_ is now known as yasufum02:35
*** benj_ has quit IRC02:35
*** benj_- is now known as benj_02:35
*** rcernin_ has quit IRC02:46
*** macz_ has joined #openstack-meeting02:55
*** macz_ has quit IRC03:00
*** rfolco has joined #openstack-meeting03:11
*** manpreet has joined #openstack-meeting03:18
*** jamesden_ has quit IRC03:29
*** rfolco has quit IRC03:32
*** gyee has quit IRC03:43
*** macz_ has joined #openstack-meeting03:46
*** macz_ has quit IRC03:50
*** yasufum has quit IRC04:09
*** ociuhandu has joined #openstack-meeting04:13
*** ociuhandu has quit IRC04:17
*** yasufum has joined #openstack-meeting04:53
*** vishalmanchanda has joined #openstack-meeting05:02
*** jmasud has joined #openstack-meeting05:08
*** dmacpher__ has joined #openstack-meeting05:25
*** dmacpher_ has quit IRC05:27
*** evrardjp has quit IRC05:33
*** evrardjp has joined #openstack-meeting05:33
*** yasufum has quit IRC05:39
*** ricolin has joined #openstack-meeting05:59
*** yamamoto has quit IRC06:48
*** yasufum has joined #openstack-meeting06:49
*** yasufum has quit IRC07:17
*** yasufum_ has joined #openstack-meeting07:17
*** yamamoto has joined #openstack-meeting07:29
*** yamamoto has quit IRC07:39
*** dklyle has quit IRC07:41
*** ociuhandu has joined #openstack-meeting07:50
*** ociuhandu has quit IRC07:54
*** slaweq has joined #openstack-meeting08:00
*** rpittau|afk is now known as rpittau08:18
*** ociuhandu has joined #openstack-meeting08:31
*** tosky has joined #openstack-meeting08:33
*** jmasud has quit IRC08:35
*** jawad_axd has joined #openstack-meeting08:43
*** ociuhandu has quit IRC08:51
*** e0ne has joined #openstack-meeting08:54
*** jmasud has joined #openstack-meeting09:27
*** jawad_axd has quit IRC09:30
*** jawad_axd has joined #openstack-meeting09:30
*** vishalmanchanda has quit IRC09:36
*** jmasud has quit IRC09:39
*** tosky_ has joined #openstack-meeting09:47
*** tosky is now known as Guest2437209:49
*** tosky_ is now known as tosky09:49
*** Guest24372 has quit IRC09:50
*** lpetrut has joined #openstack-meeting09:57
*** vishalmanchanda has joined #openstack-meeting10:07
*** yamamoto has joined #openstack-meeting10:12
*** baojg has quit IRC10:18
*** baojg has joined #openstack-meeting10:18
*** lbragstad has quit IRC10:31
*** lbragstad has joined #openstack-meeting10:31
*** masazumi-ota has quit IRC10:40
*** ociuhandu has joined #openstack-meeting10:41
*** yasufum_ has quit IRC10:41
*** yasufum has joined #openstack-meeting10:43
*** ociuhandu_ has joined #openstack-meeting10:44
*** ociuhandu_ has quit IRC10:46
*** ociuhandu_ has joined #openstack-meeting10:47
*** ociuhandu has quit IRC10:47
*** ociuhandu_ has quit IRC10:52
*** ociuhandu has joined #openstack-meeting10:52
*** jmasud has joined #openstack-meeting10:53
*** ociuhandu has quit IRC10:53
*** oneswig has joined #openstack-meeting11:00
oneswig#startmeeting scientific-sig11:00
openstackMeeting started Wed Dec 16 11:00:46 2020 UTC and is due to finish in 60 minutes.  The chair is oneswig. Information about MeetBot at
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.11:00
*** openstack changes topic to " (Meeting topic: scientific-sig)"11:00
openstackThe meeting name has been set to 'scientific_sig'11:00
oneswigWhat a week11:01
*** macz_ has joined #openstack-meeting11:18
*** macz_ has quit IRC11:23
*** openstack changes topic to "OpenStack Meetings ||"11:23
openstackMeeting ended Wed Dec 16 11:23:58 2020 UTC.  Information about MeetBot at . (v 0.1.4)11:23
openstackMinutes (text):
*** oneswig has quit IRC11:24
*** yamamoto has quit IRC11:32
*** yasufum has quit IRC11:44
*** raildo has joined #openstack-meeting11:53
*** ociuhandu has joined #openstack-meeting11:55
*** yamamoto has joined #openstack-meeting11:55
*** ociuhandu has quit IRC11:59
*** yamamoto has quit IRC12:00
*** rfolco has joined #openstack-meeting12:01
*** baojg has quit IRC12:09
*** baojg has joined #openstack-meeting12:09
*** yamamoto has joined #openstack-meeting12:20
*** baojg has quit IRC12:42
*** baojg has joined #openstack-meeting12:43
*** ociuhandu has joined #openstack-meeting12:58
*** ociuhandu has quit IRC13:12
*** ociuhandu has joined #openstack-meeting13:13
*** ociuhandu has quit IRC13:18
*** ociuhandu has joined #openstack-meeting13:21
*** ociuhandu has quit IRC13:24
*** ociuhandu has joined #openstack-meeting13:24
*** thgcorrea has joined #openstack-meeting13:48
*** jmasud has quit IRC13:51
*** ociuhandu has quit IRC13:54
*** lbragstad has quit IRC13:54
*** lbragstad has joined #openstack-meeting13:57
*** lajoskatona has joined #openstack-meeting13:57
*** lbragstad has quit IRC13:57
*** lbragstad has joined #openstack-meeting13:58
*** liuyulong has joined #openstack-meeting13:59
liuyulong#startmeeting neutron_l314:00
openstackMeeting started Wed Dec 16 14:00:46 2020 UTC and is due to finish in 60 minutes.  The chair is liuyulong. Information about MeetBot at
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
*** openstack changes topic to " (Meeting topic: neutron_l3)"14:00
openstackThe meeting name has been set to 'neutron_l3'14:00
liuyulongNo announcement from me, so let's go througth the bug directly.14:02
liuyulong#topic Bugs14:02
*** openstack changes topic to "Bugs (Meeting topic: neutron_l3)"14:02
*** lbragstad_ has joined #openstack-meeting14:03
liuyulongWe have 3 lists14:04
*** ociuhandu has joined #openstack-meeting14:04
*** ociuhandu has quit IRC14:05
*** ociuhandu has joined #openstack-meeting14:05
liuyulongFirst one:14:05
*** lbragstad has quit IRC14:05
openstackLaunchpad bug 1906375 in neutron "[L3] router HA port concurrently deleting" [Low,In progress] - Assigned to LIU Yulong (dragon889)14:06
liuyulongIt was reported by me.14:06
liuyulongDuring a local testing procedure, I found this exception.14:06
liuyulongBasically it has no harm to the neutron, but raised some noise LOGs during the test.14:06
liuyulongI've summitted the patch:
liuyulongslaweq, Hi14:07
slaweqI have another meeting in same time but I will be lurking here14:07
liuyulongThe patch just catch the port not exist related error, but left all other exceptions raised as it is.14:08
liuyulongIt is simple.14:08
*** ociuhandu has quit IRC14:08
slaweqliuyulong: what about oleg's comment?14:09
slaweqare You going to add debug message there?14:09
*** ociuhandu has joined #openstack-meeting14:09
liuyulongYes, I saw that. The code is same to the code in l3_hamode_db, there is no log.14:10
liuyulongSo if you guys insist, I will add it.14:10
slaweqfor me it's fine, if such log would be useful we can add it in both places in follow-up patch14:11
liuyulongOK, next one14:12
openstackLaunchpad bug 1907175 in neutron " intermittently ALL VM's floating IP connection is disconnected, and can be reconnected after 5-6 minutes " [Undecided,New]14:12
liuyulongI've looked at this bug these days.14:13
slaweqfor me this one looks more like some ovs bug maybe14:14
liuyulongThis is, IMO, not pretty sure, a kernel bug of the bond mode 4.14:14
slaweqas they said that it is fixed when they restart openvswitch14:14
slaweqor kernel bug14:14
slaweqbut not neutron issue really14:14
liuyulongI have a kernel patch which is trying to fix that, but it really similar to this. My case is bond mode 6.14:15
liuyulongWait a minute...14:15
liuyulongThere are some information about the topology in this kernel patch.14:18
liuyulongThe main issue is because the slave NIC is trying to send the IPv6 related traffic with run mac which make the physical world failed to find the way back.14:19
liuyulongThe LP bug reportor said they use the spine-leaf in their DC as well.14:21
liuyulongSo maybe they can try to find the issue in such way.14:21
liuyulongtcpdump the slave NIC to see if it will send some packets out, and monitor the MAC table in the switch to see if the learnt entry is refrshed to disturb the traffic.14:23
liuyulongThis is really a relly complicated issue, my suggestion for the user is to invite their switch manufacturer to work together to find out the real issue.14:25
liuyulongSometimes, the switch forwarding protocol may be different in implementation which will result some unexcepted behavior.14:26
liuyulongOK, no comments.14:29
liuyulongNext one14:29
openstackLaunchpad bug 1905295 in neutron "[RFE] Allow multiple external gateways on a router" [Wishlist,New] - Assigned to Bence Romsics (bence-romsics)14:29
*** ociuhandu has quit IRC14:29
liuyulongIn Last L3 meeting and last driver meeting, this had beed disscussed.14:30
liuyulongBence had replied a long comment. : )14:30
rubasovI hope I made some progress with the alternative we discussed14:33
liuyulongSo my response can be: if you do not need floating IP, but you want multiple external interface. You may do it like this.14:33
liuyulongChange your mind of that "external network" to "provider network" which should be a common use.14:34
liuyulongSo if you have multiple "provider network", then you can create VMs on it. And as many interface as you want.14:34
*** TrevorV has joined #openstack-meeting14:34
rubasovI guess I would have the provider networks as external, not putting VMs directly on them14:35
liuyulongThe external network are basically for L3 floating IP(NAT) which is for neutron to support some product in users' view, aka Elastic Public IP.14:36
rubasovthe change in neutron-dynamic-routing seems way simpler than making multiple external gw-s possible in neutron14:36
rubasovI think I will explore what exaclty that change in neutron-dynamic-routing would be14:37
liuyulongYes, you can do that, but external network is sometimes not visible to common user.14:37
rubasovbut overall I like this alternative14:38
rubasovlooks like it spares us from some significant amount of work14:38
lajoskatonaJust a question here: for me it seems strange (perhasp doc issue) that we can add external net to router as interface which is more "internal"14:39
liuyulongCool, it's glad to see that we do not need to involve changing all L3 related stuff.14:39
lajoskatonaSo perhaps it is just how I understand the API doc14:39
liuyulongBut can be done in some magic actions.14:39
rubasovif the neutron team agrees I'd definitely like to propose a change to the api-ref clearing this up14:40
liuyulonglajoskatona, you can consider the external network is a common one.14:40
*** ociuhandu has joined #openstack-meeting14:40
liuyulonglajoskatona, so it has subnets, then it can be attach to a router.14:40
lajoskatonaliuyulong: ok, it's really good14:41
liuyulonglajoskatona, but is is currently not available for common user, it is admin only action.14:41
lajoskatonaliuyulong: yeah, that's fair, as it is more an infrastructure thing14:41
rubasov(that api-ref piece blocked my thinking about this alternative for quite some time)14:41
liuyulongrubasov, my thougth is not to use the name "external network" in such case. Because the code has many "external_xxx", "external_yyy". My suggestion is to use "provider network".14:43
liuyulongAnd the deployment guide in for neutron, has a provider network only option.14:44
rubasovtechnically it's a decision done years ago I guess, the network already has an attribute called "external"14:44
rubasovthough I'm not sure if that's bit is used anywhere when attaching a subnet of the external=true network as a router interface14:46
liuyulongAnd there is a routed provider networks guide:14:46
liuyulongTotally no floating IP concept in these guide.14:47
liuyulongrubasov, my idea is not to attach a network's subnet with  external=true.14:47
liuyulongI remember there are some bugs related to such action...14:48
liuyulongSome routers added external gateway to the network. Some added the subnet.14:48
rubasovbut then what is the meaning of that external bit, if I shoud set it to False on a network that's actually external?14:49
liuyulongIMO, it really not a good option which may case the DVR, routing table wich some unexcepted hehaviors.14:49
liuyulongrubasov, remember the L3 toplogy14:50
liuyulongrubasov, user's network needs to go through the "Router" then to the external network.14:50
liuyulongrubasov, the external gateway of a router is the leg to the outsider world.14:51
*** ociuhandu has quit IRC14:51
rubasovthat's exactly my point in my LP comment, that the API today allows an external net's subnet as a router interface14:52
lajoskatonabut this is only a flag (for the user/admin) on the API or really makes something different on the backend?14:52
rubasovso can that be my router's north leg instead of the router gw?14:52
liuyulongrubasov, you can, but just routing, no NAT. And again, in order to make things clear, we should can such network as "provider network" to distinguish the concepts.14:55
rubasovlajoskatona: that's a very good question which is hard to very hard to answer by grepping through the code14:55
liuyulonglajoskatona, same answer. No NAT, but only routing in the backend.14:55
lajoskatonaok thanks14:56
lajoskatonathat's enough for me14:56
rubasovok, I think we are on the same page14:56
liuyulongOK, thanks then, no bugs from me now.14:58
liuyulongWe still have 2 mins.14:58
liuyulong#topic On demand agenda14:58
*** openstack changes topic to "On demand agenda (Meeting topic: neutron_l3)"14:58
liuyulongOne min now.14:58
liuyulongAny thing we need to take care?14:59
lajoskatonaliuyulong: this is (draft) spec for bfd support, if you could check it would be really helpful14:59
liuyulongGot it, I will review it recent days.14:59
lajoskatonaof course others' comments as well really helpful14:59
liuyulongOK time is up.15:00
*** openstack changes topic to "OpenStack Meetings ||"15:00
openstackMeeting ended Wed Dec 16 15:00:29 2020 UTC.  Information about MeetBot at . (v 0.1.4)15:00
openstackMinutes (text):
rubasovthank you15:00
liuyulongThank you guys, bye.15:00
*** belmoreira has joined #openstack-meeting15:01
*** ralonsoh has quit IRC15:15
*** ralonsoh has joined #openstack-meeting15:15
*** ricolin_ has joined #openstack-meeting15:15
*** lpetrut has quit IRC15:23
*** lajoskatona has left #openstack-meeting15:28
*** ociuhandu has joined #openstack-meeting15:41
*** liuyulong has quit IRC15:42
*** _mlavalle_1 has quit IRC15:46
*** ociuhandu has quit IRC15:53
*** macz_ has joined #openstack-meeting16:04
*** ociuhandu has joined #openstack-meeting16:08
*** mlavalle has joined #openstack-meeting16:16
*** jawad_axd has quit IRC16:23
*** ricolin_ has quit IRC16:31
*** ociuhandu_ has joined #openstack-meeting16:49
*** ociuhandu has quit IRC16:52
*** dklyle has joined #openstack-meeting16:52
*** ociuhandu has joined #openstack-meeting16:53
*** ociuhandu_ has quit IRC16:57
*** ralonsoh is now known as ralonsoh|afk17:00
*** ociuhandu_ has joined #openstack-meeting17:02
*** ociuhandu has quit IRC17:05
*** ociuhandu_ has quit IRC17:07
*** gyee has joined #openstack-meeting17:07
*** rpittau is now known as rpittau|afk17:11
*** ociuhandu has joined #openstack-meeting17:18
*** ociuhandu has quit IRC17:23
*** dklyle has quit IRC17:30
*** dklyle has joined #openstack-meeting17:30
*** belmoreira has quit IRC17:34
*** jmasud has joined #openstack-meeting17:59
*** jmasud has quit IRC18:56
*** jmasud has joined #openstack-meeting19:11
*** jmasud has quit IRC19:30
*** jmasud has joined #openstack-meeting19:32
*** armstrong has joined #openstack-meeting19:43
*** e0ne has quit IRC19:52
*** lbragstad_ is now known as lbragstad19:54
*** ociuhandu has joined #openstack-meeting20:35
*** ociuhandu has quit IRC20:40
*** timburke has joined #openstack-meeting20:41
*** jmasud has quit IRC20:43
*** timburke has quit IRC20:48
*** SpamapS has quit IRC20:53
*** rfolco has quit IRC20:56
*** timburke has joined #openstack-meeting20:56
*** vishalmanchanda has quit IRC20:56
*** acoles has joined #openstack-meeting20:57
timburke#startmeeting swift21:00
openstackMeeting started Wed Dec 16 21:00:16 2020 UTC and is due to finish in 60 minutes.  The chair is timburke. Information about MeetBot at
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.21:00
*** openstack changes topic to " (Meeting topic: swift)"21:00
openstackThe meeting name has been set to 'swift'21:00
timburkewho's here for the swift meeting?21:00
rledisezhi o/21:00
mattoliverauo/ (only for a short while)21:01
timburkethank you all for coming -- i may be a little in-and-out; handling some childcare duties again21:02
*** zaitcev has joined #openstack-meeting21:02
timburkeas usual, the agenda's at
timburkefirst up21:02
timburke#topic end-of-year meeting schedule21:02
*** openstack changes topic to "end-of-year meeting schedule (Meeting topic: swift)"21:02
clayghow about: this is it.  this is the last meeting of 2020.  see you later 2020.21:03
timburkeyeah, that :-)21:03
mattoliverauLol, damn you 2020 :p21:03
zaitcevRed Hat enters the Christmas shutdown until January.21:04
zaitcevBut we could have one last meeting if we wanted.21:04
timburkenah -- next meeting as Jan 6 seems perfectly reasonable21:04
timburkenext topic21:05
timburke#topic audit watchers21:05
*** openstack changes topic to "audit watchers (Meeting topic: swift)"21:05
timburkewe're so close!21:05
zaitcevWell, the df was the last principal problem, I think.21:06
zaitcevNow, even if we get back to independent processees in the future, we can.21:06
timburkesorry that i haven't done another pass since my review last week21:06
zaitcevSo, I'm honestly content with the final revision.21:07
mattoliverauI'll take another look this week, and hopefully add my +2 again21:07
timburkei am still a bit worried about the need to distinguish start/end for different workers when you've got more than one21:07
zaitcevmattoliverau: I added the doc that you asked. And it includes the Dark Data part after all. At first, I hoped to sweep it under the carpet and only use it in case of emergency at customer clusters.21:08
mattoliverautimburke: I thought it added their name to the logger21:09
zaitcevYes, you were right. Doing what I meant is how tribal memory is generated and its' wrong.21:09
zaitcevYes, logs have prefixes21:09
timburkei think the device_key/worker-id was my last major concern, and i think we could remedy that with a new arg to end (and maybe start? i'm not actually sure how important it is there...)21:09
zaitcevAnd in fact, using watcher_name is better because that comes from proxy-server.conf, and is not the name of the Python class.21:10
zaitcevIt's easy to add new arguments thanks to Sam's foresight. I was way more concerned about letting df stuck in there. But if you want to add some, easy to do in a follow-up.21:10
timburkemattoliverau, we get watcher prefixes, but we still don't have a way to distinguish between the same watcher spread across multiple workers21:11
zaitcevTrue, but why is it needed?21:11
timburkeif you've got, say, 24-bay chassis and 4 workers per node (so each worker is responsible for 6 disks), when you go to dump stats to recon (say), you don't want to have all four workers writing a quarter of the full stats to the same keys21:12
zaitcevWorkers are ephymeral, so... even if you know PID, all you can is kill the whole auditor and maybe restart it. Thinking as an operator here, the most important is to know which object triggered issues.21:12
zaitcevOh, that way.21:13
*** jmasud has joined #openstack-meeting21:13
timburkei was realizing it as i was thinking through (watchers: Add a policy-stat watcher)21:13
mattoliverauGood spot. So we need to append a worker I'd or something?21:14
timburkei think so. soemthing along the lines of the device_key from
*** baojg has quit IRC21:15
zaitcevRight... of course you can do os.getpid() now safely, but eh... My mental model was that you just increment all stats in some center place like memcached or Prometheus, and reset them at wall clock moment, like midnight on Mondays, rather than when auditor starts. That would give you comparable counts to watch trends.21:15
*** baojg has joined #openstack-meeting21:16
zaitcevIf you insist on recon specifically, than a key is needed.21:16
zaitcevBut I think you can add it in a follow-up.21:16
timburkegood thought on os.getpid() -- forgot about that... might be sufficient21:17
zaitcevos.getpid() changes when auditor restarts, so you'll have a ton of old recon files in /var21:17
zaitcevwell, if you reboot21:17
timburkei could also re-work it so that everything's always aggregated by device, and i write things to recon based on that. saves the same problem when worker count changes21:18
zaitcevHmm. We never have 2 workers crawling the same device?21:18
timburkeshouldn't; not for the same audit-type, anyway21:19
*** jmasud has quit IRC21:19
*** baojg has quit IRC21:19
timburke(seems like it'd make for more disk-thrashing)21:20
*** baojg has joined #openstack-meeting21:20
zaitcevOkay. I still think it's good for your and Matt's final review pass.21:21
timburkeoh, i also need to think about the resumability of auditors... if they get interrupted, they pick up again more or less where they left off, right? hmm...21:21
timburkeall right, i'll make sure to review it again within the next three weeks, and it sounds like mattoliverau will try to do the same21:21
zaitcevMore or less. They write that json thing checkpoint.21:21
timburke#topic py3 fixes21:22
*** openstack changes topic to "py3 fixes (Meeting topic: swift)"21:22
timburkei was noticing that we've got a few py3 fixes that i wanted to raise attention for21:23
acolestimburke: I'll volunteer to review if you like21:24
timburkethanks! it could use a test, but i know i've seen while running tests in my aio21:24
openstackLaunchpad bug 1900770 in OpenStack Object Storage (swift) "py3 comparison troubles" [High,In progress]21:24
*** larainema has quit IRC21:24
acolesyup, maybe I'll put a test together, will do me good to re-educate myself about bad buckets21:25
timburke has been observed in the wild:
openstackLaunchpad bug 1906289 in OpenStack Object Storage (swift) "Uploading a large object (SLO) in foreign language characters using S3 browser results in 400 BadRequest - Error in completing multipart upload" [High,Confirmed]21:25
* mattoliverau needs to take the car in for a service.21:26
mattoliverauGotta run, have a great one all o/21:26
zaitcevmattoliverau: later21:27
timburkeand is one that i'd mostly forgotten about, but can let bad utf-8-decoded-as-latin-1-encoded-as-utf-8 out to the client21:28
zaitcevright... are there any more besides these 321:28
timburkeprobably. those are the three i could remember ;-)21:29
timburkei *really* want to get to the point that i can feel confident in moving my prod clusters to py321:30
timburkemoving on21:31
timburke#topic finishing sharding21:31
*** openstack changes topic to "finishing sharding (Meeting topic: swift)"21:31
timburkei came in late last week, so i wanted to check if there was any more discussion needed here, or if we've got a pretty good idea of what would be involved21:32
zaitcevI don't, but I sent David to investigate and teach me :-)21:33
* zaitcev manages21:33
* zaitcev shuffles some more documents21:33
acolesmy summary was: 1. be able to recover from whatever could go wrong with auto-sharding (split brain) 2. do our best to prevent split-brain autosharding  3. get more confident about auto-shrinking21:34
acolesand suggested some current patches as a good starting place to get involved21:35
timburkesounds like a great plan :-)21:35
acolese.g. the chain starting with
timburkei won't worry then21:35
timburkeone last-minute topic21:35
timburke#topic stable gate21:35
*** openstack changes topic to "stable gate (Meeting topic: swift)"21:35
acolesBTW I updated priority reviews because I have squashed a couple of patches into
timburkecurrently, things are fairly broken. mostly to do with pip-on-py2 trying to drag in a version of bandit that's py3-only21:36
timburkethere are some patches to pin bandit, and at least some of them are mergeable, but it looks like there are some other requirements issues going on that complicate some branches21:38
acolesdidn't a bandit fix merge?21:38
timburkei'm going to keep working on getting those fixed, just wanted to keep people apprised21:38
acoles ?21:39
timburkeyeah, that at least got master moving. i might be able to do that for one or two of the more-recent branches, too21:40
acolesOIC there's a bunch of backport patches21:40
timburkesomeone proposed a fix back on pike through stein like
acolesok, is py2-constraints the right way though?21:41
timburkenot all branches have a py2-constraints. though maybe we could introduce that?21:42
timburkefwiw, a cap in test-requirements.txt hits failures like on some branches21:43
zaitcevno ;python_version=='2.7 though for that bandit, I wonder whyt21:43
*** TrevorV has quit IRC21:43
timburkeanother option would be to just drop bandit from test-requirements.txt on (some?) stable branches -- we don't backport *that* much, and i'm not sure how much value we get from running bandit checks on stable21:44
timburkethat's all i've got21:45
timburke#topic open discussion21:45
*** openstack changes topic to "open discussion (Meeting topic: swift)"21:45
timburkeanything else we should bring up this week?21:45
acolesthanks timburke for all your work on the gate issues, it's incredibly valuable21:45
timburkeanything i can do so you guys can focus on making swift great!21:46
zaitcevI don't understand how bandit even gets invoked. There's a [bandit] in tox.ini, but it's not in the list at the top or in any zuul jobs.21:46
zaitcevoh. maybe it's not no master branch.21:47
acolesits part of the pep8 tox env21:48
timburkeiirc it's a flake8 plugin -- just install it and it'll start getting run as well21:48
zaitcevOh, right.21:48
zaitcevOkay. I don't have anything else to discuss. Michelle managed to push through that patch for swift-init, but I have no idea if he's going to stick around.21:49
timburkeoh yeah! looking at the bug report (, i'm not actually sure that the title was really accurate...21:50
openstackLaunchpad bug 1079075 in OpenStack Object Storage (swift) "swift-init should check if binary exists before starting" [Low,In progress] - Assigned to Michele Valsecchi (mvalsecc)21:50
zaitcevhow so? He wanted not to have extra messages.21:51
zaitcevSo, there's no change in function.21:51
zaitcevBy "he" I mean the original reporter.21:51
timburkebut the reason processes didn't start up wasn't actually missing binaries (afaict)21:52
timburke"fails because some *configuration files* are not existent"21:52
zaitcevwell yeah21:53
zaitcevSomeone removed both configurations and binaries21:53
zaitcevYou know, I used to try that crap too. It was a mistake. But our RPM packages used to be very fine-grained like that.21:53
timburke*shrug* if it's still a problem, we'll get a new bug report ;-)21:54
zaitcevBut then we started to share a bunch of code across types of services. For example, GET on accounts and containers use a function that's not in common code, but in container IIRC. So, when someone installs just one type of service, it blows up.21:54
zaitcevI had to give up and create a common package that contains all of the code, no matter where it belongs.21:55
zaitcevSo the logic was, if swift-init starts checking for binaries, it would not attempt to run something that has no configuration.21:55
zaitcevSo, I think it was an appropriate patch and it was okay for us to include it.21:56
zaitcevWell, its value was very low. Only helps people who do this fine-grained installation.21:57
timburkecool. yeah, i'm not worried about the patch; i do think it makes swift better. just thinking about whether the bug should be closed or not21:57
zaitcevOh, Tim21:57
zaitcevYeah, of course close it.21:57
zaitcevOne question: when are we going to drop py2?21:57
timburkegreat question!21:58
timburkei don't know!21:58
* zaitcev backrolls in nagare kaiten21:58
timburkei feel like with train/ussuri we saw a decent number of new clusters stood up running py3-only21:59
timburkeand more recently in victoria/wallaby we're seeing clusters that were on py2 migrate to py321:59
zaitcevI'm sure projects other than bandit are going to put pressure on us. I think eventlet is the worst of them.21:59
timburkeyup -- it's a growing worry for me too -- see for their deprecation (i don't think they've dropped it yet, but it's just a matter of time)22:01
zaitcevRed Hat offers 7 years on some of the supported releases, but they have a controlled set of packages + backported patches. But in the trunk it's kind of a pain.22:01
timburkethinking mostly selfishly, i'll say "not until i've migrated off of py2 myself" ;-)22:02
zaitcevso, you have trunk on py2?22:02
timburkemy prod clusters run py2, yes22:03
zaitcevWhat's the OS? Some kind of old Ubuntu I presume.22:03
timburke(home cluster's py3 though!)22:03
timburkecentos7, mainly22:03
timburkei think we've got some legacy customers still on ubuntu22:03
zaitcevRight, that is py2.22:03
zaitcevOK thanks for the answer.22:03
timburkewe package our own python; system python is a pain22:04
*** dklyle has quit IRC22:04
timburkeall right, sorry, i let us go over time. thank you all for coming, and thank you for working on swift!22:04
*** openstack changes topic to "OpenStack Meetings ||"22:04
openstackMeeting ended Wed Dec 16 22:04:37 2020 UTC.  Information about MeetBot at . (v 0.1.4)22:04
openstackMinutes (text):
*** zaitcev has left #openstack-meeting22:04
*** acoles has left #openstack-meeting22:05
*** ralonsoh|afk has quit IRC22:09
*** timburke has quit IRC22:10
*** zaitcev has joined #openstack-meeting22:14
*** rcernin has joined #openstack-meeting22:17
*** dklyle has joined #openstack-meeting22:21
*** raildo has quit IRC22:30
*** slaweq has quit IRC22:42
*** armstrong has quit IRC23:02
*** haleyb is now known as haleyb|away23:18
*** baojg has quit IRC23:25
*** baojg has joined #openstack-meeting23:26

Generated by 2.17.2 by Marius Gedminas - find it at!