Thursday, 2020-12-03

*** tosky has quit IRC00:18
*** mlavalle has quit IRC00:22
*** baojg has quit IRC00:26
*** baojg has joined #openstack-meeting-300:28
*** baojg has quit IRC01:50
*** baojg has joined #openstack-meeting-301:51
*** ricolin has quit IRC02:08
*** hemanth_n has joined #openstack-meeting-302:29
*** baojg has quit IRC02:58
*** baojg has joined #openstack-meeting-302:59
*** psachin has joined #openstack-meeting-303:59
*** imtiazc has quit IRC06:19
*** e0ne has joined #openstack-meeting-307:22
*** e0ne has quit IRC07:32
*** slaweq has joined #openstack-meeting-307:41
*** lpetrut has joined #openstack-meeting-307:46
*** belmoreira has joined #openstack-meeting-307:50
*** ricolin has joined #openstack-meeting-308:03
*** e0ne has joined #openstack-meeting-308:06
*** e0ne has quit IRC08:08
*** e0ne has joined #openstack-meeting-308:13
*** tosky has joined #openstack-meeting-308:34
*** baojg has quit IRC09:58
*** psachin has quit IRC11:45
*** artom has quit IRC12:44
*** baojg has joined #openstack-meeting-312:48
*** e0ne has quit IRC12:55
*** hemanth_n has quit IRC13:07
*** e0ne has joined #openstack-meeting-313:09
*** artom has joined #openstack-meeting-313:13
*** e0ne has quit IRC13:25
*** mugsie has quit IRC13:32
*** raildo has joined #openstack-meeting-313:32
*** baojg has quit IRC13:39
*** baojg has joined #openstack-meeting-313:40
*** baojg has quit IRC13:49
*** baojg has joined #openstack-meeting-313:50
*** e0ne has joined #openstack-meeting-314:00
*** psahoo has joined #openstack-meeting-314:47
*** lpetrut has quit IRC15:30
*** rafaelweingartne has joined #openstack-meeting-315:52
*** macz_ has joined #openstack-meeting-315:56
*** mlavalle has joined #openstack-meeting-315:57
*** elod has joined #openstack-meeting-315:58
gibi#startmeeting nova16:00
openstackMeeting started Thu Dec  3 16:00:33 2020 UTC and is due to finish in 60 minutes.  The chair is gibi. Information about MeetBot at http://wiki.debian.org/MeetBot.16:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.16:00
*** openstack changes topic to " (Meeting topic: nova)"16:00
openstackThe meeting name has been set to 'nova'16:00
gibio/16:02
gmanno/16:02
stephenfino/16:02
elodo/16:02
gibi#topic Bugs (stuck/critical)16:03
*** openstack changes topic to "Bugs (stuck/critical) (Meeting topic: nova)"16:03
gibiOne Critical bugs16:03
gibi    #link https://bugs.launchpad.net/nova/+bug/1906428 blocking the nova gate as nova-multi-cell job fails16:03
openstackLaunchpad bug 1906428 in OpenStack Compute (nova) "test_cold_migrate_unshelved_instance failing with cat: can't open '/mnt/timestamp': No such file or directory" [Critical,In progress]16:03
gibi    Patch is on the gate to skip the failing test until we find a solution #link https://review.opendev.org/c/openstack/nova/+/76514116:03
gibiI saw it bounced from the gate :/16:03
gmannah again failed.16:03
gmann134 run already in check pipeline I think it would not merge soon16:04
gibilyarwood promised to  continue looking into the actual problem next week16:04
gibigmann: yeah, gate feels slow these days16:05
bauzas\o16:05
gibi#link 14 new untriaged bugs (+0 since the last meeting): #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New16:05
gibiwe are hovering around this number during the whole week16:05
gibi#link 75 bugs are in INPROGRESS state without any tag (+0 since the last meeting): #link https://bugs.launchpad.net/nova/+bugs?field.tag=-*&field.status%3Alist=INPROGRESS16:06
gibithese are potentially un-triaged bugs. Check if they are still valid16:06
gibiIs there any bug we need to discuss here ?16:06
gibi#topic Gate status16:07
*** openstack changes topic to "Gate status (Meeting topic: nova)"16:07
gibiGate on master is blocked. Patch to unblock it is on the gate #link https://review.opendev.org/c/openstack/nova/+/76514116:07
gibiwe dicussed this already16:07
gibiGate on stable/victoria is blocked. Fix is on the gate #link https://review.opendev.org/c/openstack/nova/+/76443216:07
gibithis also bounced16:07
gibi:/16:07
gibiClassification rate 35% (+11 since the last meeting) #link http://status.openstack.org/elastic-recheck/data/integrated_gate.html16:08
gibiPlease look at the gate failures, file a bug, and add an elastic-recheck signature in the opendev/elastic-recheck repo (example: #link https://review.opendev.org/#/c/759967)16:08
gibiI don't know how relevant the classification rate as an absolute value16:08
gibias it is now show better classification than last week but the gate feels in worst shape16:08
gibimaybe what changed that we know why the gate fails but we didn't solved the failures yet16:09
gibianyhow I will keep reporting / tracking this number for a while to see if it is relevant16:09
gibiany other gate issue we need to talk about?16:09
gibi#topic Release Planning16:10
*** openstack changes topic to "Release Planning (Meeting topic: nova)"16:10
gibiWallaby Milestone 1 is today!16:10
gibiThe second spec review day was a success. We now have 11 blueprints approved to Wallaby. #link https://blueprints.launchpad.net/nova/wallaby16:11
gibiUntil Milestone 1 we finished 0 blueprint out of the 11 approved blueprints.16:11
gibiM2 is january 2216:11
gibiconsidering the holiday season there is not much time until M216:12
gibiM2 will be spec freeze so if you have an open spec please hurry up :)16:12
gibiany other release specific thing to disucss?16:12
gibi#topic Stable Branches16:14
*** openstack changes topic to "Stable Branches (Meeting topic: nova)"16:14
gibistable/victoria is blocked but patch to unblock is on the gate - https://review.opendev.org/76443216:14
gibiother stable branches seems to be OK, no outstanding issue16:14
gibiEOM16:14
elodsorry for repeating o:)16:14
gibino worry, thanks for consistently adding update to the agenda16:14
eloddid not see that it's already listed at gate status16:14
elodnp16:15
gibiany other stable thing to discuss? ( lyarwood is on PTO today)16:15
elodnothing that I'm aware of :)16:15
gibi#topic Sub/related team Highlights16:16
*** openstack changes topic to "Sub/related team Highlights (Meeting topic: nova)"16:16
gibiLibvirt (bauzas)16:16
bauzasnothing to say16:16
gibi#topic Open discussion16:16
*** openstack changes topic to "Open discussion (Meeting topic: nova)"16:16
gibithere are two on the agenda16:16
gibi(stephenfin): Stuck on what to do about invalid instance hostnames like 'ubuntu18.04'16:16
gibi#link https://review.opendev.org/c/openstack/nova/+/76448216:16
gibistephenfin: could you summarize where we are?16:17
stephenfinI've brought this up on the mailing list16:17
gibiI was only able to follow the ML thread partially16:17
stephenfintl;dr: people are using instance names that look like FQDNs16:17
stephenfinI haven't yet figured out if they're relying on these to be balid16:17
stephenfin*valid16:17
stephenfinIn any case, I'm not sure if we're going to be able to just replace all periods is the name16:18
stephenfinso I'm still thinking the "if it's an invalid FQDN, munge the name, otherwise don't" approach is best16:19
rafaelweingartneI would like to ask for guidance with a patch16:19
stephenfinbut I know sean-k-mooney at least disagrees16:19
rafaelweingartneI proposed this patch: https://review.opendev.org/c/openstack/nova/+/711113, but it has not received much reviews so far16:19
rafaelweingartneshould I open an RFE, and then a spec for it as well?16:19
gibirafaelweingartne: i will ping you after stephen's topic16:19
rafaelweingartneops, sorry, sure16:19
gibistephenfin: but sean is not here :)16:20
stephenfinquick - everyone review it while sean is distracted!16:21
stephenfin:)16:21
gibistephenfin: your proposed the split approach to support two separate use cases?16:21
gibiuse case a) server name is used as fqdn in the guest16:22
gibibut what is use case b)16:22
stephenfinuse case a) is more a FQDN is used as the server display name and therefore the server host name16:22
stephenfinwhile use case b) is a server display name with a period in it that is *not* a FQDN is used, so the server host name should be something else16:23
stephenfini.e. 'test.domain.com' is okay. 'test.01' will be converted to 'Server-{serverUUID}'16:23
stephenfinif that makes sense?16:24
gibiand in case b) what will be the hostname in the guest?16:24
stephenfin'Server-{serverUUID}'16:24
stephenfinwhich is the fallback today if you end up with an empty string after all non-alphanumeric characters are removed16:25
gibiI assum now test.01 causing a real failure somewhere down the line16:25
stephenfinif designate is deployed, you aren't able to boot an instance16:25
stephenfinbecause neutron will error out when creating/attaching a port16:25
gibiwith proper documentation I'm OK to have this split behavior. I guess you need a backportable solution16:26
gibihence not trying to disconnect the name and the hostname16:26
stephenfinyes, exactly16:26
stephenfinthe proper solution is 'openstack server create --hostname FOO ...'16:27
stephenfinbut that's not backportable (API change)16:27
gibiyeah16:27
gibidoes sean has a counter proposal that is also backportable?16:27
stephenfinNot backportable fwict, no16:27
gibiI see16:27
stephenfinIt's user error in his eyes16:28
gibithen I think we can say that do a backportable fix first then do a proper fix on master later16:28
*** sean-k-mooney has joined #openstack-meeting-316:28
sean-k-mooneyo/16:28
stephenfinand we should close as WONTFIX, which is user hostile16:28
gibisean-k-mooney: o/16:28
gibiwe are just discussing the server name test.01 issue16:28
sean-k-mooneyah ok16:29
bauzasmmmm16:29
* bauzas looks at the API docs to see what we tell about naming instances16:29
bauzas"The server name."16:29
bauzaswow16:29
sean-k-mooneybauzas: it tell you nothing16:30
gibisean-k-mooney: what is the reason you are against stephenfin's proposal to convert test.01 to server-{serverUUID} and not convert valid FQDNs16:30
bauzasdidn't see that coming16:30
sean-k-mooneyyep16:30
sean-k-mooneygibi: it would change the hostname seen in the guest for one16:30
sean-k-mooneythe precendiet is also based on a missunder standing that unicode was invalid in a hostname16:30
bauzasso, honestly, given we haven't told it's either the display name or the hostname, I think we are OK16:31
bauzasbecause the semantics can change16:31
gibisean-k-mooney: I gues we not just remove unicode charachters but other non hostname compatible charachters too16:31
gibilike /16:31
sean-k-mooneyso we should be allowing unicode hostnames16:31
sean-k-mooney but ath is a seperete fature16:32
bauzasdefinitelty ^16:32
sean-k-mooney*feature16:32
gibiagree ^^16:32
gibiso unicode aside16:32
bauzasasséééééé16:32
sean-k-mooneywe also are not transforming the hostnames acording to the relenvet RFEs16:32
sean-k-mooney*RFCs16:32
sean-k-mooneywe shoudl be substituiing all punctianto and other special symble with _16:32
sean-k-mooneysorry -16:32
bauzasor, just consider that if you provide a ".", then you knew you are providing a FQDN16:33
bauzasso, the hostname should only be the server name, not the TLD16:33
sean-k-mooneyso what we coudl do is in a new microversion add an fqdn filed and take only what is before the . for the instance.hostname16:34
bauzasie. if I wrote "bauzas.local", that meant to me that the name of my server is "bauzas"16:34
sean-k-mooneyyep16:34
sean-k-mooneywhich is what actully happens todya16:34
bauzasand I leave my DNS telling me my own TLD16:34
stephenfinan API microversion isn't backportable though16:34
gibibut as far as I understand we need a backportable solution first, then a proper solution on master16:34
sean-k-mooneybut as i pointed out in the email thread the metadat is totally wrong in that case16:34
stephenfinI totally agree that what we do is rubbish, but we do it and people rely on it to some degree16:35
sean-k-mooneyi dont belive we need a backporable solution16:35
sean-k-mooneyor at lease im not sold on it16:35
bauzasstephenfin: can't we consider to limit the server name to be "server" and not the whole FQDN ?16:35
sean-k-mooneybauzas: i woudl be ok backproting that although im uncofrotabel with the transformation in general16:36
bauzas(speaking of "server.domain")16:36
stephenfinif we do, that's a change in behavior for users that were doing e.g. 'openstack server create instance.domain.com'16:36
sean-k-mooneystephenfin: its not form a cloud init poitn of view16:36
bauzasstephenfin: that's why I said I'm cool with explaning this behavioural change16:36
sean-k-mooneythere hostname will be instance in both cases16:36
bauzasas we didn't promised anything with the servername16:36
sean-k-mooneye.g. with or without designate16:36
bauzaswe're not breaking the contract)16:37
stephenfinhmm, okay, so I'd assumed that would be rejected as non-backportable16:37
sean-k-mooneywhat that would change is the designate dns name16:37
bauzaswell, it says "The server name."16:37
bauzas"16:37
sean-k-mooneycurrently it appending the designate default domain to the full sever name16:37
sean-k-mooneynow it would do the sane thing and append the default domain tothe hostname16:37
bauzasyup16:37
sean-k-mooneywhich woudl acutlly be resolveable via dns16:38
bauzasyup16:38
* gibi lost 16:38
bauzasand we could keep the display name to be the FQDN16:38
stephenfinso if you create a server with 'instance.domain.com' and designate's default domain is 'domain.com', what happens?16:38
sean-k-mooneybauzas: sure16:38
bauzasgibi: trying to rephrase16:38
sean-k-mooneythe dispaly name could be that server name as it was passed in16:38
stephenfingibi: bauzas and sean-k-mooney are suggesting we drop everything after the first period, and suggesting it's backportable because we never made a guarantee about what the instance's hostname would be16:38
bauzasthis ^16:39
gibithanks16:39
stephenfinso 'test-instance.domain.com' would have a hostname of 'test-instance'16:39
gibiwould this change the hostname of existing instances?16:39
bauzas(with a big fat note explaining why we're so mean to the user)16:39
stephenfinand 'ubuntu18.04' would have a hostname of 'ubuntu18'16:39
sean-k-mooneygibi: no16:39
bauzasgibi: don't16:40
sean-k-mooneygibi: it would only change the hostname for new instances16:40
stephenfinit shouldn't - that information is only calculated once on initial boot and stored in instance.hostname16:40
sean-k-mooneyyep16:40
gibiok16:40
bauzasmustn't is the word :)16:40
sean-k-mooneydid peopel see http://lists.openstack.org/pipermail/openstack-discuss/2020-November/019137.html by the way16:40
stephenfinI don't think we recalculate it if you e.g. change the instance name via 'openstack server set --name NAME server'16:40
stephenfinassuming that is a command...16:40
sean-k-mooneywhere i wen ther how the info is actully prented to the gust16:40
* stephenfin knows you can set the name when rebuilding but isn't sure about otherwise16:41
gibithen I'm OK to do this change as a backportable fix with a fat note16:41
bauzassean-k-mooney: yup, I saw your email16:41
gibicould some of you please summarize it back to the ML to see if other will be against it?16:41
bauzassean-k-mooney: and that's why I think that people using periods in their server names are either foolish or smart enough16:41
gibisorry folks we have two other topics for today16:42
gibiso we should move on16:42
stephenfinyup16:42
* stephenfin will summarize16:42
gibithanks!16:42
bauzasI think we have a reasonable consensus here16:42
sean-k-mooneystephenfin++16:42
gibirafaelweingartne: your turn16:42
rafaelweingartneSure. I have proposed this patch (https://review.opendev.org/c/openstack/nova/+/711113), it has some conflicts, but before resolving them16:43
rafaelweingartneI would like to understand if we are missing something16:43
rafaelweingartnesuch as an RFE, or a spec16:43
gibirafaelweingartne: glancing at the patch and the commit message you plan to redefine what 'usage' currntly means in the os-simple-tenant-usage API16:44
*** e0ne has quit IRC16:44
rafaelweingartneyes, and no16:45
rafaelweingartnewe plan to externalise it. So, the default behaviour is maitained, and if somebody wants to redefine it, they could do so16:45
rafaelweingartneTo us, for instance, we were expecting something totally different from the data we get there (in the API) right now16:46
gibiextrenalize is with a config option I assume16:46
sean-k-mooneywell if you wanted to do it differntly you can do so alredy16:46
sean-k-mooneyvia consuming the instance notifocations16:46
rafaelweingartnegibi: exactly16:46
sean-k-mooneyand building a system to track the lifecycle of the servers as you see fit16:46
rafaelweingartnethat is what the API is doing16:46
gibiit feels like a config driver API16:46
rafaelweingartnesean: we have other systems in-place that do that16:46
gibidriven16:47
rafaelweingartnegibi: yes16:47
gibiwe try to avoid config driven APIs as it makes differnt public coulds behave differently16:47
rafaelweingartnewhen we saw that API, we just thought about using it to cross-check the data we already have in other monitoring and billing systems that we have in place16:47
gibiIs os-simple-tenant-usage admin only by default?16:48
sean-k-mooneyso so this is one of the apis that i dont really fit well in nova16:48
sean-k-mooneylong term i think it would live better in an external service16:48
rafaelweingartneprobably yes16:48
sean-k-mooneyits one of the larger performance hedaces for our custoemr16:48
sean-k-mooneythis is very slow to query and result in a slow horizion as it used in the defautl overview page16:49
rafaelweingartnebut the current docs gave us the idea of providing the usage for a VM, but as I explain in the patch, it consider usage the time between the instance was created up until now or when it was destroyed16:49
sean-k-mooneyso im concerned about adding more complexity to it16:49
rafaelweingartneI see16:49
rafaelweingartneRight now, the API does not provide usage data as it says16:50
rafaelweingartneat least, it is not the same understanding of usage as we have16:50
rafaelweingartnethat is why we proposed the patch16:50
gibirafaelweingartne: so it provides resource allocation usage but not runtime for the VM I guess16:50
rafaelweingartneexactly16:51
sean-k-mooneyrafaelweingartne: well it does provide usage info16:51
gibiI tend to agree with sean-k-mooney that this is not a good API for billing, and also rafaelweingartne you said that you have a different service anyhow for billing16:51
rafaelweingartnebut the documentation says usage, it does not differ between allocation and actual usage16:51
sean-k-mooneybut the defition of usage is differnt form what you are expecting16:51
rafaelweingartnetherefore, we tried to amend that16:52
gibiI don't really think we shoudl develop os-simple-tenant-usage further (hence the name simple) but fix the doc to be precies instead16:52
sean-k-mooneyso amending that woudl be an api change and require a spec not a bugfix16:52
rafaelweingartnewell, ok that would help as well then16:52
sean-k-mooneyhttps://github.com/openstack/nova/blob/0e7cd9d1a95a30455e3c91916ece590454235e0e/doc/source/contributor/policies.rst#metrics-gathering16:52
sean-k-mooneyits slightly tangental but we have delcare metrics gathering as out of scope before16:53
sean-k-mooneyi tought we had a similar statement for billing but i dont see one16:53
rafaelweingartneOk, so no sense in creating an RFE then16:53
rafaelweingartnewell, I will create a patch to make the docs more clear then16:53
gibirafaelweingartne: thank you!16:54
gibi(please file a doc bug for tracking)16:54
gibithere is one more topic from the agenda16:54
gibi         (gibi): do we want to merge the backports for the placement-audit command? https://review.opendev.org/q/topic:%22placement-audit-backport%2216:54
gibiIt was raised during the week on #openstack-nova16:54
stephenfinyes please16:55
gibidoes somebody remember what was the reason not to merge it?16:55
stephenfinartom: ^ ?16:55
stephenfinI think the concern was that it's kind of feature'y, but it's not user visible and is a huge win for operators (and us, diagnosing problems)16:55
artomOh, it was just super messy16:55
stephenfinoh, even simpler than that16:56
artomPast, like, 1 or 2 releases back16:56
bauzasyup16:56
bauzasthis was the concern16:56
stephenfinit was merged in stable/ussuri, right?16:56
artomNope, we didn't bother16:56
stephenfinno, I mean initially16:56
artomI used the upstrem DNM backports for CI, essentially16:56
artomBecause our RH CI is... well, it is.16:56
artomAh, you'd have to ask bauzas about the initial landing.16:57
bauzaswhen this was merged ?16:57
bauzaswell, I'm old16:57
bauzasussuri IIRC16:57
sean-k-mooneydansmith had an opion on it and i belive it was in favor of mergeing based on the operator win but i also dont recal16:57
gibimerged in ussuri16:57
bauzashttps://review.opendev.org/c/openstack/nova/+/670112 => ussuri16:57
bauzassean-k-mooney: I think his opinion was meh16:58
gibihow risky it is to backport the mess?16:58
sean-k-mooneybauzas: basically im rembering it was not a hell no16:58
bauzasbut honestly, audit is related to allocations recreate16:58
bauzasfrom mriedem16:58
gibiI assume the effor to create the backport was already spent so only future efforts on stable due to these patches in question16:59
bauzasone is deleting orphaned, the other is recreating them16:59
bauzasgibi: I'd say that the maintainance is low but the initial effort is worth it pre-Train16:59
bauzasTrain backport is easy16:59
gibibauzas: but the initial effort is already spent as we have the patches proposed17:00
bauzasbut then artom sweated a lot with older releases17:00
stephenfinbauzas: is or is not?17:00
bauzastechnically, we QE'd it on Queens17:00
gibiQE?17:00
artombauzas, did we tho?17:00
bauzasso the effort is already done and manually validated17:00
bauzasagainst Queens17:00
artomI'd have to double check the BZ17:00
gibiwe run out of time17:00
gibilets move this to #openstack-nova17:01
gibisorry17:01
gibi#endmeeting17:01
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"17:01
openstackMeeting ended Thu Dec  3 17:01:05 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)17:01
openstackMinutes:        http://eavesdrop.openstack.org/meetings/nova/2020/nova.2020-12-03-16.00.html17:01
bauzasgibi: sorry, I meant this was validated by one of our engineers17:01
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/nova/2020/nova.2020-12-03-16.00.txt17:01
openstackLog:            http://eavesdrop.openstack.org/meetings/nova/2020/nova.2020-12-03-16.00.log.html17:01
*** imtiazc has joined #openstack-meeting-317:11
*** psahoo has quit IRC17:25
*** tosky has quit IRC17:29
*** rafaelweingartne has quit IRC18:09
*** artom has quit IRC18:37
*** artom has joined #openstack-meeting-318:37
*** belmoreira has quit IRC19:11
*** tosky has joined #openstack-meeting-320:55
*** purplerbot has quit IRC21:24
*** purplerbot has joined #openstack-meeting-321:33
*** purplerbot has quit IRC21:35
*** purplerbot has joined #openstack-meeting-321:41
*** bnemec has quit IRC23:22
*** baojg has quit IRC23:43

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!