Wednesday, 2022-05-04

*** dasm|ruck is now known as dasm|ruck|off05:13
*** dviroel|out is now known as dviroel11:20
*** dasm|ruck|off is now known as dasm|ruck13:43
whoami-rajat#startmeeting cinder14:00
opendevmeetMeeting started Wed May  4 14:00:07 2022 UTC and is due to finish in 60 minutes.  The chair is whoami-rajat. Information about MeetBot at http://wiki.debian.org/MeetBot.14:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
opendevmeetThe meeting name has been set to 'cinder'14:00
whoami-rajat#topic roll call14:00
rosmaitao/14:00
hemnayough14:00
fabiooliveirao/14:00
geguileohi! o/14:00
caiquemello[m]o/14:00
eharneyhi14:00
jungleboyjo/14:01
toskyo/14:01
enriquetasohi14:01
simondodsleyhi14:02
whoami-rajatgreat turnout14:02
whoami-rajatlet's get started14:02
whoami-rajat#topic announcements14:02
whoami-rajatopenstack-tox-docs failure14:03
whoami-rajatthe openstack-tox-docs job started breaking last week because of a change in oslo.policy14:03
whoami-rajat#link https://review.opendev.org/c/openstack/oslo.policy/+/83051414:03
whoami-rajatIt was fixed by14:03
whoami-rajat#link https://review.opendev.org/c/openstack/oslo.policy/+/83971114:03
whoami-rajatand released with this patch14:03
whoami-rajat#link https://review.opendev.org/c/openstack/releases/+/83977514:03
whoami-rajatthe reason cinder didn't see this failure will be discussed later (I've added a topic for it)14:04
rosmaitabecause cinder is the BEST!14:04
whoami-rajat:D there might be changes needed (if we agree to what I'm seeing in other projects) ...14:05
whoami-rajatbut let's discuss that later14:05
whoami-rajatanyone has anything else for announcements?14:05
whoami-rajatlooks like not, so we can get started with topics14:06
whoami-rajat#topic Ceph min client version set to Mimic: enable it by default14:06
whoami-rajatenriquetaso, that's you14:06
enriquetasoHello, argonauts, 14:06
enriquetasoAfter the problems we have testing RBD upstream, I think it's best to enable the Mimic client by default.14:06
* enriquetaso lost the link14:06
enriquetaso#link https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/782624/14:07
enriquetasoI know it is a safety mechanism to disable it by default, but the goal is to test Ceph in all our upstream CIs.14:07
enriquetasoFor backward compatibility, the patch includes a check line #49314:07
whoami-rajati think the initial time this change was done, it broke glance but glance is now fixed (with a workaround) so maybe this is safe now?14:08
toskydo we know if it can affect manila?14:09
enriquetasoI don't think our ci jobs affect manila 14:09
toskyenriquetaso: if we enable it by default, every user of devstack-plugin-ceph will be affected14:10
eharneyit could affect manila, i'm not sure if it does in any meaningful way or not14:10
toskyI mean, they should at least know14:11
toskyfor the record, we also need to ask Francesco and vkmc to also add the same feature in the new codepath which deploys with cephadm in any case, even more if it's true by default14:11
vkmctosky, seems it's set to false in the patch, so it's not enabled by default 14:11
vkmccan be toggled in ci as needed I guess14:11
toskyvkmc: the proposal is to turn it on by default14:11
vkmcah14:11
toskythat's what's being discussed14:11
whoami-rajatIIUC nova, glance, manila, cinder are the consumers of devstack-plugin-ceph so those projects needs to be consulted/considered14:12
eharneynova and glance are tested in the CI jobs on this patch, right?14:12
enriquetasoyes, it should be14:13
eharneyas is manila14:13
enriquetasoI think it would be of great benefit for us to activate it to start testing it properly.14:15
eharneyyes, this is the configuration we want to target, so we should have devstack configure it by default14:15
whoami-rajathave we tested that it breaks any job with the config option defaulted as True?14:17
eharneyit looks like that should be the next step14:17
whoami-rajatok14:17
eharneywell, i pushed for it to not be a True/False setting, but, conceptually anyway14:18
enriquetasoif nobody is against it, I can update the patch and we could continue the discussion on the patch 14:19
whoami-rajatsounds reasonable14:19
enriquetasocool 14:20
enriquetasothanks14:20
whoami-rajatyou can update the patch and see for any potential failures and from there we can continue this discussion14:20
enriquetasomakes sense14:20
whoami-rajatgreat14:20
whoami-rajatok, so let's move on to our next topic14:20
whoami-rajat#topic stable-cores: backport Zuul job playbook fix14:21
whoami-rajatrosmaita, that's you14:21
rosmaitahi14:21
rosmaitatosky fixed a problem in some of our zuul jobs14:21
rosmaitawe need to backport to all stable branches where appropriate14:21
jungleboyj\o/14:21
rosmaitabecause we are sometimes reporting success for failed jobs14:21
rosmaitaand nobody wants that14:22
whoami-rajat#link http://tiny.cc/cinder-legacyfacade14:22
rosmaitano, not that link14:22
toskyjust one job, the ceph one on the cinder.git repository14:22
whoami-rajatsorry wrong link14:22
whoami-rajat#link https://review.opendev.org/q/topic:fix-reporting14:22
rosmaitathat's it14:22
rosmaitaif you look at the patch to master, i left a comment there to show you that tosky's fix works14:22
rosmaita(in case you had any doubt)14:22
enriquetasotosky++14:22
rosmaitabut it would be good to get these backported as soon as possible before we let somthing slip past the goalie14:23
rosmaitathat's all14:23
whoami-rajatok, this looks important to get in, not good to get wrong reporting status to introduce more issues than we already have14:24
whoami-rajatso stable cores can take a look14:24
jungleboyjrosmaita: I can get these merged.14:24
rosmaitaworks for me!14:24
whoami-rajatjungleboyj++14:24
whoami-rajatand tosky++14:25
whoami-rajatok, let's move on to the next topic then14:25
whoami-rajat#topic remove legacy oslo.db enginefacade patches need more reviews!14:25
whoami-rajatrosmaita, that's you again!14:25
rosmaitanot much to say other than REVIEW! REVIEW! REVIEW!14:26
rosmaitaand you can use this groovy dashboard: http://tiny.cc/cinder-legacyfacade14:26
whoami-rajatI'm reviewing them and honestly the formatting is killing me, specially on the long patches14:26
rosmaitait's semi-black, so it's the wave of the future14:26
whoami-rajatI've left some comments on the last patch, i can move on with the review if those are addressed14:27
eharneythe pattern of bonus commas everywhere is quite painful and just feels wrong14:27
rosmaitaand the stacked parameter lists will help when we mypy those files14:27
whoami-rajatbut it would be great if others participate and get it done faster!14:27
whoami-rajatrosmaita, excluding the extra commas ...14:27
rosmaitai think in parameter lists the commas are wrong, but elsewhere i don't care that much14:28
whoami-rajati think there's also a patch to refactor whole cinder code to follow black's formatting but I saw quite a lot of negative reviews there (one is from me as well)14:28
rosmaitabut i don't think they are wrong enough to revise the patches and have to review all this ... stuff14:28
rosmaitaagain14:28
whoami-rajatyeah, rebasing that chain is not good for CI14:29
whoami-rajatok, so coming back to the main point, please review the patches proposed by Stephen14:30
rosmaitaanyway, please review so we can get the legacy enginefacade gone before we absolutely have to14:30
eharneydoes that mean we also won't think the extra commas in method calls aren't wrong enough to patch them away later?14:30
rosmaitai don't think so, you can fix it when you mypy those files  :D14:30
eharneycan do14:31
rosmaitaworks for me14:31
whoami-rajatgreat, so we can get rid of the unnecessary formatting as well, more incentive to review mypy14:31
whoami-rajatmoving on to the next topic14:32
whoami-rajat#topic Yoga "Known Issues" followup patches need reviews14:32
geguileoeharney: I would normally -1 a patch for those...14:32
eharneygeguileo: me too...14:32
whoami-rajatrosmaita, that's you again :D14:32
rosmaita#link https://review.opendev.org/c/openstack/cinder/+/83945114:33
rosmaitajust want to mention that it addresses a known issue in the Yoga release, so let's get it merged soon and backport it14:34
rosmaitageguileo is also working on the nvme-of known issues from Yoga14:34
rosmaitageguileo: do you have a link to your stuff that needs reviews?14:35
whoami-rajatgreat, good to get these issues resolved earlier so we need to do less backports14:35
geguileorosmaita: I'm changing the patches for the nvmeof stuff14:36
rosmaitaok, we can have a weekly "known issues" checkin14:36
geguileorosmaita: I went back and I'm redoing the big ones  :'-(14:36
rosmaitanot trying to rush you14:36
rosmaitaand now i don't feel bad about not reviewing them yet!14:36
rosmaitajbernard is working on the cgroup v1->v2 issue14:37
whoami-rajatsounds like a good idea if we've a lot of patches fixing known issues, i think fixing nvmeof issues is a long series itself14:37
rosmaitaand i think enriquetaso is working on a ceph related issue14:37
rosmaita#link https://docs.openstack.org/releasenotes/cinder/yoga.html#known-issues14:37
rosmaitaok, that's it from me14:38
rosmaitafor real this time, i don't think i have any more agenda items14:38
whoami-rajatthey were all pointing to important things that we should actively review, thanks rosmaita for keeping check on them14:39
hemnathe db reviews are quite a lot14:39
rosmaitayeah, tell me about it14:40
whoami-rajatguess there are too many items to review now, so core reviewers please take a look ^^14:41
hemnahttps://review.opendev.org/c/openstack/cinder/+/831247 this one too please :)14:41
whoami-rajatyep, we decided at the PTG that this one is important to get fixed as well ^14:42
whoami-rajatso moving on to the last topic14:42
hemnaat some point I'd like to discuss the idea of volume transaction tracking14:42
hemnaoh sorry, I thought we were done.14:42
hemnaignore more14:42
whoami-rajatwe can continue in open discussion and we've midcycle coming up on 1st june14:43
whoami-rajat#topic Move install_command constraints to deps14:43
whoami-rajatthat's me14:43
whoami-rajatso Cinder was not affected with the openstack-tox-docs failure because we take packages from upper constraints and not released versions14:43
rosmaitanot sure what that means?14:43
whoami-rajatwhich is good but we use install_command for that instead of the deps section of a particular job14:44
whoami-rajatrosmaita, so when oslo.policy 3.12.0 (or whatever the failing version was), it wasn't merged in upper-constraints file in requirements project14:44
rosmaitaright, so it's not a problem, cinder is doing the right thing14:45
whoami-rajatso projects installing from released versions saw this failing job but projects using upper-constraints file were unaffected14:45
rosmaitageguileo fixed this way back14:45
whoami-rajatyes14:45
whoami-rajatbut there is more to it14:45
whoami-rajatwe use the install_command which is generic for all jobs14:45
rosmaitait's not installing from released versions, it's installing from the development version14:45
whoami-rajatinstead of the deps section which is specific to every job14:45
rosmaitai think geguileo explains this on his commit message pretty clearly: https://opendev.org/openstack/cinder/commit/5ad7fe92690b29a05c6284aa52e01913b5b7cf7614:46
geguileoin my opinion we should always honor the upper constraints14:47
geguileothat's the whole point of having them14:47
rosmaitaexactly14:47
toskyI don't want to reopen old discussions, but iirc I remember a long global series of patches to remove the usage of install_command14:47
geguileoI'm ok if there's another way to do it14:48
geguileobut installing cinder with packages that don't honor upper constraints is without a doubt BAAAAAAAAAAAAAAAD14:48
geguileoand I'll -2 any patch that tries to do it14:48
jungleboyj++14:48
whoami-rajati think i should have worded the topic better as it created a lot of confusion here14:48
whoami-rajatwe are using upper constraints, which is correct, the point I'm trying to convey is to do it otherwise, i.e. not from install_commands but from deps14:49
rosmaitayes, except you can't do it by deps14:49
whoami-rajatsome of the reasons listed here on this patch seemed valid like passing multiple contraint files only takes the first one14:49
whoami-rajat#link https://review.opendev.org/c/openstack/heat/+/81880214:50
rosmaitai'm not sure that's correct14:50
rosmaitageguileo looked into this closely (while i watched)14:50
rosmaitawith usedevelop=True you get a two-phase install14:50
rosmaitathe way we have it defined, pip uses the same install command both times14:50
rosmaitaand both times it has the upper-constraints14:51
toskybut shouldn't other projects show the same issue?14:51
rosmaitathey do, that's why they broke14:51
rosmaitaor, they are ignoring it14:51
rosmaitait only showed up in a really weird case when we couldn't figure out why cinderlib was using an unconstrained library 14:52
geguileoyup, that drove me crazy14:53
rosmaitaa lot of times nothing will break ... but when it does, it's a real PITA to track down14:53
geguileo+1 to that14:53
rosmaitayeah, we wasted a lot of time on that, and i don't want to do it again14:53
rosmaitaand if they don't have usedevelop=True, then it's not a problem14:53
whoami-rajatok, i don't have any strong views on that, whichever way seems reasonable to the team works14:54
geguileojust to understand the issue, was it being suggested to do the same thing as that patch?14:55
whoami-rajatgeguileo, you mean the heat patch?14:55
toskylet me find a link14:55
geguileoyes14:55
geguileois that was we are discussing?14:55
rosmaitathat heat patch does not look correct to me14:56
whoami-rajatyeah, I've seen other projects move away from install_commands and the reasons on the heat one sounded most reasonable to me so took that example14:56
toskyI don't remember all the discussion, but there are some hints here: https://review.opendev.org/c/openstack/nova/+/684775/14:56
rosmaitayeah, that patch is not correct, you will get the problem geguileo tracked down14:57
geguileoif nova is not hitting the issue, good for them...14:58
geguileosince cinder will get bitten by this if we change it, we have to keep it14:58
rosmaitayes14:58
toskyok, but if there were other issues which lead to that recommendation, can't we at leat raise the question globally and have a general guideline?14:58
geguileomaybe we should add a comment to our tox.ini, since this may not be clear to other people reading the file14:59
whoami-rajattosky, yeah, this seems to be missing an openstack wide guideline14:59
rosmaitayeah, i guess you have to do a "blame" to see the reasoning14:59
whoami-rajatgeguileo, that would be good since it wasn't clear to me at least ...14:59
whoami-rajatok, we are out of time15:00
whoami-rajatthanks everyone for joining!15:00
whoami-rajat#endmeeting15:00
opendevmeetMeeting ended Wed May  4 15:00:47 2022 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:00
opendevmeetMinutes:        https://meetings.opendev.org/meetings/cinder/2022/cinder.2022-05-04-14.00.html15:00
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/cinder/2022/cinder.2022-05-04-14.00.txt15:00
opendevmeetLog:            https://meetings.opendev.org/meetings/cinder/2022/cinder.2022-05-04-14.00.log.html15:00
jungleboyjThank you!15:00
*** dviroel is now known as dviroel|lunch15:18
*** dviroel|lunch is now known as dviroel16:15
*** whoami-rajat__ is now known as whoami-rajat17:25
hemnaI have lots of these.  "| 179527  | Volume        | 08292ec1-d1ab-49fa-bdf8-247ce08f6f8d | creating | <null>     |"18:06
hemnaid, resource_type, resource_id, status, service_id18:06
hemnaservice_id is always null18:06
hemnaso I found another issue with tracking allocated_capacity19:29
hemnaif a volume is migrated the destination service doesn't update it's allocated_capacity_gb after the migration is complete19:29
hemnaonly the source side is updated even with https://review.opendev.org/c/openstack/cinder/+/836083 in place19:31
hemnathe RX service is never called to update it's allocated_capacity19:31
*** dviroel is now known as dviroel|out21:50

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!