14:00:00 <rosmaita> #startmeeting cinder
14:00:00 <lseki> o/
14:00:01 <openstack> Meeting started Wed Mar 18 14:00:00 2020 UTC and is due to finish in 60 minutes.  The chair is rosmaita. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:03 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:05 <openstack> The meeting name has been set to 'cinder'
14:00:07 <e0ne> hi
14:00:09 <whoami-rajat> Hi
14:00:12 <rosmaita> #topic roll call
14:00:27 <rosmaita> i guess the roll call is kind of pointless at this point
14:00:30 <rosmaita> hello everyone
14:00:45 <rosmaita> #link https://etherpad.openstack.org/p/cinder-ussuri-meetings
14:01:00 <whoami-rajat> the roll call is over before the roll call :D
14:01:17 <jungleboyj> o/
14:01:22 <sfernand> hi
14:01:48 <rosmaita> #topic announcements
14:01:58 <walshh_> hi
14:02:01 <rosmaita> first up
14:02:01 <smcginnis> o/
14:02:39 <rosmaita> ttx is cleaning up the IRC channels and meetings, you may have seen his email
14:02:51 <rosmaita> this meeting room is going to be a casualty of that effort
14:03:04 <rosmaita> it will probably no longer exist next week
14:03:19 <rosmaita> so, we will be moving to meeting room #openstack-meeting-alt for next week
14:03:28 <rosmaita> the time WILL NOT CHANGE
14:03:32 <rosmaita> just the location
14:03:54 <rosmaita> i will send out a reminder and put a note in #openstack-cinder next week
14:04:04 <whoami-rajat> rosmaita, thanks for the heads up
14:04:15 <rosmaita> just be aware that if you try to connect to -meeting-4, it may not exist!
14:04:32 <rosmaita> next topic
14:04:44 <rosmaita> we had the second session of the virtual mid-cycle on monday
14:04:48 <rosmaita> thanks to all who attended
14:05:01 <rosmaita> jungleboyj posted the recording onto the cinder youtube channel
14:05:11 <jungleboyj> woot woot.
14:05:12 <rosmaita> #link https://youtu.be/cA_VfYnS77o
14:05:21 <rosmaita> and there's a summary available
14:05:30 <rosmaita> #link https://wiki.openstack.org/wiki/CinderUssuriMidCycleSummary#Session_Two:_16_March_2020
14:06:11 <rosmaita> one last announcement, i will be putting up release patches for stable/stein and stable/train later this week
14:06:32 <rosmaita> so please check if there's anything that you'd like to propose for backport
14:06:42 <jungleboyj> rosmaita:  Thanks for getting the summary and everything put together so quickly.
14:06:55 * rosmaita blushes
14:06:56 <jungleboyj> rosmaita:  Are there many patches waiting on approval?
14:07:00 <jungleboyj> :-)
14:07:13 <rosmaita> i don't think so, but i got distracted this morning and didn't look
14:07:21 <jungleboyj> Ok.  I will take a look.
14:07:32 <rosmaita> cool
14:07:54 <rosmaita> ok, that's all for announcements, other than i hope everyone is staying healthy in these stressful times
14:08:23 <rosmaita> #topic initial import of  devstack-plugin-open-cas
14:08:32 <rosmaita> LiangFang has been busy!
14:08:42 <rosmaita> #link https://review.opendev.org/#/c/713173/
14:08:47 <LiangFang> hi
14:08:58 <LiangFang> I have prepared the initial
14:09:20 <LiangFang> and I think I need to add tempest job
14:09:45 <LiangFang> I take a look of devstack-plugin-ceph
14:10:12 <LiangFang> seems I need to add tempest test cases in cinder-tempest-plugin
14:10:37 <LiangFang> and then TEMPEST_PLUGINS: '/opt/stack/cinder-tempest-plugin'
14:11:16 <rosmaita> you mean in .zuul.yaml ?
14:11:22 <LiangFang> yes
14:11:37 <LiangFang> not sure i'm correct or not
14:12:14 <rosmaita> i'm not sure either
14:12:30 <LiangFang> i will try to test this way
14:12:32 <rosmaita> but we don' t need to wait until we figure it out to get the repo merged
14:12:42 <rosmaita> you have some basic tests there of the plugin itself
14:12:56 <smcginnis> I haven't had a chance to actually stack with that plugin. I'm assuming you have and it all works LiangFang?
14:13:16 <LiangFang> yes, i use github repo
14:13:34 <LiangFang> and it can work when ./stach.sh
14:13:57 <LiangFang> it installed and uninstall open-cas normally
14:14:07 <smcginnis> Perfect, thanks.
14:14:12 <LiangFang> github repo is same with this repo in code
14:14:26 <rosmaita> great, that's a good start
14:15:00 <LiangFang> next, I need to add tempest test
14:15:46 <smcginnis> I think the tempest test can be a follow up. Just my opinion. Might be good to get the code in place so the repo is there, then enhance it with better testing.
14:15:54 <LiangFang> but I think i can add in next patch
14:16:10 <LiangFang> smcginnis: agreed, thanks
14:16:19 <e0ne> smcginnis: +1
14:16:23 <rosmaita> i agree with smcginnis, will be good to have this in place and then we can work with it
14:16:39 <LiangFang> +1
14:16:49 <rosmaita> plus, i think it can't actually break anything at this point
14:16:53 <rosmaita> :)
14:17:51 <LiangFang> no code referenced to this repo correctly I think
14:18:03 <rosmaita> right, that will be the next step
14:18:12 <rosmaita> thanks, LiangFang ... anything else?
14:18:35 <LiangFang> when I do test, I just add in local.conf
14:18:50 <LiangFang> enable_plugin devstack-plugin-open-cas https://...
14:19:29 <rosmaita> yes, i think that's the standard way to enable a plugin
14:19:30 <LiangFang> sorry I have not attend the virtual PTG meeting because my daught got fever
14:19:40 <rosmaita> hope she's OK
14:20:02 <rosmaita> my daughter also woke up with a fever this morning :(
14:20:06 <LiangFang> yes, she's OK now, thanks. that's all from me
14:20:09 <jungleboyj> rosmaita:  ++
14:20:19 <rosmaita> ok, great
14:20:27 <jungleboyj> rosmaita:  Hope your daughter is ok.
14:20:47 <rosmaita> yeah, just  a low-grade fever at this point, so normally noting to worry about
14:20:53 <rosmaita> *nothing
14:21:12 <rosmaita> #topic os-reset_status notifications not going to the standard publisher
14:21:23 <rosmaita> #link https://review.opendev.org/#/c/708563/
14:21:32 <rosmaita> hemna has ^^ up
14:21:37 <rosmaita> it looks ok to me
14:22:01 <rosmaita> but i just wanted to ask whether anyone knows of a reason why those notifications weren't going to the standard queues for cinder?
14:22:20 <eharney> do we have any general documentation on how these notifications work?
14:22:38 <smcginnis> ^
14:22:55 <smcginnis> I think we've had issues from time to time with notifications because not many people even are aware of them.
14:23:03 <eharney> it wasn't really obvious to me why, for instance, snapshot notifications should go to a different publisher than volumes
14:24:17 <rosmaita> i couldn't find anything
14:24:32 <tosky> it's a repository
14:24:59 <tosky> oh, sorry, ignore me
14:25:16 <rosmaita> tosky: was waiting for you to elaborate
14:25:19 <rosmaita> :)
14:25:47 <rosmaita> so yeah, i don't have an answer to eric's question
14:26:08 <rosmaita> i guess the thing to do now is to make these consistent with the others
14:26:12 <smcginnis> Well, we can use git blame to find the commits that added the notifications. I'm guessing will just find that whoever added them just didn't know how things are supposed to work.
14:26:35 <smcginnis> The tricky part will be that now there may be some consumers looking for these notifications in different places.
14:26:58 <smcginnis> So I guess if we change any of them, we need to at least be very clear in the release notes about it.
14:28:25 <rosmaita> sorry, was just looking at that patch
14:28:46 <rosmaita> yes, i agree about the release note being beefed up a bit
14:29:22 <rosmaita> i guess we could also send an email to [ops] requesting comments
14:29:33 <smcginnis> Not a bad idea.
14:29:43 <rosmaita> i'll do that
14:29:45 <jungleboyj> Yeah, I am having trouble following the release note.  Trying to suggest improvement there now.
14:30:17 <rosmaita> #action rosmaita email to [operators] about change to publishing queue for os-reset_status notifications
14:30:43 <rosmaita> ok, thanks ... if anyone has second thoughts or sudden recollections, please comment on the patch
14:31:04 <rosmaita> this next topic is exciting
14:31:11 <rosmaita> #topic new job to test Active/Active HA
14:31:17 <rosmaita> e0ne: that's you
14:31:28 <e0ne> :)
14:31:46 <jungleboyj> That is exciting.
14:31:51 <e0ne> to be clear, I'm not proposing adding any new tests now :)
14:32:17 <e0ne> everything I propose is to add a regular tempest tests with cinder plugin
14:32:42 <e0ne> so it will find such bugs as https://bugs.launchpad.net/cinder/+bug/1867906
14:32:46 <e0ne> #link https://bugs.launchpad.net/cinder/+bug/1867906
14:32:50 <openstack> Launchpad bug 1867906 in Cinder "group-create-from-src doesn't work in active/active mode" [Undecided,In progress] - Assigned to Ivan Kolodyazhny (e0ne)
14:33:27 <jungleboyj> Ah, sounds like a good idea.
14:33:54 <e0ne> I know that geguileo was going to create some active/active-specific test framework.. it's also great idea
14:34:24 <geguileo> e0ne: yeah, that's on hold and I don't expect to be able to work on that for a while... :-(
14:34:29 <e0ne> but with a limited capacity I would like to have tempest run to be sure that A/A doesn't break anything
14:34:50 <e0ne> geguileo: :(
14:34:58 <smcginnis> Any additional testing would definitely be an improvement.
14:35:22 <smcginnis> I'm still concerned that we have this feature in there with no real way of knowing it works right.
14:35:31 <smcginnis> Other than that geguileo has run some things locally. :)
14:35:31 <e0ne> smcginnis: +1
14:35:49 <e0ne> smcginnis: we're going to use it in our distro
14:36:00 <eharney> as are we
14:36:09 <smcginnis> We'll probably know for sure soon enough then. :D
14:36:19 <e0ne> :)
14:36:49 <jungleboyj> We use eharney 's distro, so ...  :-)
14:36:59 <rosmaita> this sounds like a good start
14:37:11 <jungleboyj> Testing in production is the best!
14:37:15 <jungleboyj> Adds spice to life.
14:37:21 <geguileo> rofl
14:37:24 <e0ne> jungleboyj: does it mean Lenovo will support  HA soon?
14:37:30 <rosmaita> well, it certainly adds urgency
14:37:40 <jungleboyj> geguileo:  Thought you could use a laugh.
14:37:49 <jungleboyj> e0ne:  The Lenovo driver?
14:38:06 <geguileo> jungleboyj: I do, thanks  :-)
14:38:07 <e0ne> yep
14:38:35 <e0ne> looks like we agree with my proposal... so I have one more question about our CI...
14:38:44 <jungleboyj> e0ne:  No, I don't think we will use it there but we are making use of the Ceph driver which has HA support.  Right?
14:39:11 <e0ne> jungleboyj: fair enough
14:39:41 <e0ne> does anybody remember where is our multinode job?
14:39:44 <jungleboyj> We will also be using NetApp's drivers.  I think they may have added support.
14:40:40 <sfernand> we have a patch enabling AA only for SF until now
14:41:27 <sfernand> you are saying for ONTAP right?
14:42:00 <jungleboyj> Yes, ONTAP.
14:42:03 <rosmaita> e0ne: i think the grenade jobs are multinode ?
14:42:35 <e0ne> rosmaita: I'm not sure
14:42:46 <rosmaita> me neither, tbh
14:42:54 <lseki> jungleboyj: we don't have plans to implement A/A on ONTAP yet
14:42:55 <e0ne> we've got lvm multibackend
14:43:09 <e0ne> but I don't see any multinode job now :(
14:43:16 <jungleboyj> lseki:  Ok.  Good to know.
14:43:23 <e0ne> I remember we'd got it in the past
14:43:36 <rosmaita> tosky: are you around?
14:43:47 <lseki> jungleboyj: but it's in progress for solidfire
14:43:56 <rosmaita> (Luigi is my go-to person for gate jobs)
14:44:05 <e0ne> :)
14:44:31 <tosky> rosmaita: yes
14:44:44 <eharney> the "mn" in "cinder-grenade-dsvm-mn-sub-volbak" implies it does something multi-node but i'm not sure what
14:45:08 <tosky> yes, it uses two nodes (and split the cinder services)
14:45:12 <rosmaita> tosky: maybe you can coordinate with e0ne -- he has some ideas for adding some tests, but needs a multinode job
14:45:40 <tosky> well, the grenade job is not the first choice as it is for upgrades, you probably want a normal multinode job
14:45:40 <whoami-rajat> the cinder-grenade-dsvm-mn-sub-volbak and one more job with -volsch in the experimental job were defined for multinode testing purpose
14:46:02 <e0ne> it
14:46:03 <tosky> all the tempest-based jobs are potentially multinode, you just need to change the nodeset
14:46:16 <e0ne> it's good that at lease grenade is multinode
14:46:36 <tosky> which kind of tests do you want to add?
14:46:43 <e0ne> tosky: sounds good. I'll try to add a new job
14:47:09 <rosmaita> tosky: do you know any examples of a "normal" (non-grenade) multinode job?
14:47:14 <e0ne> tosky: some A/A HA configuration
14:47:55 <rosmaita> i think e0ne just wants to get an A/A setup, and then run our normal tempest tests and see what breaks
14:48:28 <e0ne> rosmaita: thanks. it's exactly what I want to do
14:48:36 <tosky> rosmaita: tempest-multinode-full from tempest.git
14:48:52 <tosky> it's basically devstack-tempest with a different nodeset
14:49:06 <rosmaita> tosky: sounds like what we need
14:49:17 <e0ne> +1
14:49:24 <smcginnis> Different nodeset, then some config changes to set up the A/A bit.
14:49:55 <rosmaita> e0ne: see if you can get something started, and ask for help if you need it next week
14:50:10 <rosmaita> i think there's a lot of interest in getting something like this working
14:50:11 <e0ne> rosmaita: ok, will do it asap
14:50:47 <rosmaita> great
14:51:01 <rosmaita> so that actually was an exciting topic!
14:51:21 <rosmaita> #topic open discussion
14:51:41 <jungleboyj> :-)
14:52:03 <e0ne> :)
14:52:23 <smcginnis> rosmaita: Anything we need to do to start planning for the virtual PTG?
14:52:40 <rosmaita> i was going to wait until M-3
14:52:43 <jungleboyj> :-)
14:52:51 <rosmaita> smcginnis: is it virtual for sure?
14:52:59 <jungleboyj> Has it been officialy decided virtual?
14:53:02 <jungleboyj> Jinx.
14:53:13 <smcginnis> I thought so.
14:53:14 <e0ne> jungleboyj: afaik, not yet
14:53:15 <rosmaita> the foundation page hasn't been updated since 2 march
14:53:19 * smcginnis looks again
14:53:28 <rosmaita> https://www.openstack.org/events/covid-19-coronavirus-disease-updates
14:53:54 <jungleboyj> I think it is a safe bet that it will be.
14:53:58 <e0ne> https://www.openstack.org/events/covid-19-coronavirus-disease-updates
14:54:01 <jungleboyj> Unfortunately.
14:54:25 <tosky> the European Football Championship has been postponed to 2021 (and it was supposed to happen in June), I would be surprised if the PTG was going to happen
14:54:27 <e0ne> jungleboyj: +1
14:54:31 <smcginnis> It's probably safe to assume APril will be virtual.
14:54:51 <whoami-rajat> i think the KubeCon was postponded as well
14:55:06 <jungleboyj> May will probably be all virtual as well.
14:55:13 <rosmaita> i guess the only worry will be everyone using bluejeans at once may crash it!
14:55:23 <jungleboyj> They were talking about this going through the Summer in the US potentially but I really hope not.
14:55:38 <rosmaita> my thought would be, we'll hold the virtual PTG on the dates already reserved
14:55:46 <smcginnis> I've found this useful: https://stayinghome.club/
14:55:58 <rosmaita> i guess the only problem will be time zones
14:56:00 <jungleboyj> :-)  This is an amazing scaling test for VPN providers, streaming companies and video conferencing platforms.
14:56:25 <smcginnis> I've been impressed how little issues I've noticed so far.
14:56:33 <tosky> rosmaita: maybe  the foundation could provide a private jitsi instance on private server for the PTG
14:56:46 <rosmaita> i think we can do multiple 2 hour slots such that at least one slot is convenient for each TZ
14:56:55 <smcginnis> We do have our own asterisk server. Not sure how much that scales.
14:57:04 <jungleboyj> smcginnis:  ++  My internet is a little slower, but not bad.  Apparently some people in Lenovo were having VPN issues.  Not too bad though.
14:57:08 <rosmaita> tosky: i misread that as "private jet" and thought, "but where would we go?"
14:57:14 <smcginnis> :)
14:57:14 <jungleboyj> Rachel is having major issues with her VPN.
14:57:50 <rosmaita> ok, i guess the summary of that is we should start thinking about a virtual PTG
14:58:01 <jungleboyj> :-)
14:59:22 <rosmaita> ok, just about out of time
14:59:32 <rosmaita> we'll start planning at next meeting, i guess
14:59:47 <rosmaita> anyone have anything to say?
15:00:02 <jungleboyj> Stay safe everyone!
15:00:08 <e0ne> +1
15:00:12 <LiangFang> +1
15:00:13 <rosmaita> +1
15:00:24 <rosmaita> #endmeeting