Monday, 2018-02-19

openstackgerritMatt Riedemann proposed openstack/nova master: Scheduler multiple workers support  https://review.openstack.org/15938200:11
*** mriedem has joined #openstack-nova00:11
*** acormier has quit IRC00:17
*** lbragstad has joined #openstack-nova00:20
*** acormier has joined #openstack-nova00:22
*** hiro-kobayashi has joined #openstack-nova00:25
*** lbragstad has quit IRC00:27
*** amodi has quit IRC00:36
openstackgerritTakashi NATSUME proposed openstack/nova master: Transform servergroup.addmember notification  https://review.openstack.org/54110100:41
openstackgerritTakashi NATSUME proposed openstack/nova master: Transform aggregate.update_metadata notification  https://review.openstack.org/46062500:41
openstackgerritTakashi NATSUME proposed openstack/nova master: [placement] Add functional tests for traits API  https://review.openstack.org/52409400:42
*** hshiina has joined #openstack-nova00:46
*** claudiub has quit IRC00:55
*** slaweq has joined #openstack-nova01:15
*** slaweq has quit IRC01:20
openstackgerritTetsuro Nakamura proposed openstack/nova-specs master: Support shared/dedicated vCPUs in one instance  https://review.openstack.org/54573401:29
openstackgerritTetsuro Nakamura proposed openstack/nova-specs master: Support shared and dedicated VMs in one host  https://review.openstack.org/54380501:34
*** hongbin has joined #openstack-nova01:36
*** amodi has joined #openstack-nova02:02
*** itlinux has quit IRC02:02
*** yamahata has joined #openstack-nova02:03
*** links has joined #openstack-nova02:11
*** lbragstad has joined #openstack-nova02:15
*** r-daneel has joined #openstack-nova02:24
*** amodi has quit IRC02:29
*** yangyapeng has quit IRC02:30
*** r-daneel has quit IRC02:32
openstackgerritNaichuan Sun proposed openstack/nova master: xenapi(N-R-P):Get vgpu info from `allocations`  https://review.openstack.org/52171702:35
*** amodi has joined #openstack-nova02:39
*** Dinesh_Bhor has joined #openstack-nova02:42
*** salv-orl_ has joined #openstack-nova02:45
*** salv-orlando has quit IRC02:48
*** yamamoto_ has joined #openstack-nova03:08
*** yamamoto has quit IRC03:12
*** pooja_jadhav has joined #openstack-nova03:18
*** lbragstad has quit IRC03:29
*** sree has joined #openstack-nova03:32
*** acormier has quit IRC03:34
*** r-daneel has joined #openstack-nova03:40
*** yangyapeng has joined #openstack-nova03:44
*** udesale has joined #openstack-nova03:59
*** rmcall has quit IRC04:02
*** rmcall has joined #openstack-nova04:12
*** moshele has joined #openstack-nova04:18
*** moshele has quit IRC04:23
*** sridharg has joined #openstack-nova04:28
*** abhishekk has joined #openstack-nova04:28
*** takashin_ has joined #openstack-nova04:31
*** takashin has quit IRC04:33
*** bhujay has joined #openstack-nova04:40
*** psachin has joined #openstack-nova04:47
*** trinaths has joined #openstack-nova04:47
*** dave-mccowan has quit IRC04:53
*** stakeda has quit IRC05:04
*** stakeda has joined #openstack-nova05:15
*** takashin_ has left #openstack-nova05:17
*** slaweq has joined #openstack-nova05:17
*** takashin has joined #openstack-nova05:17
*** slaweq has quit IRC05:22
*** claudiub has joined #openstack-nova05:27
*** lajoskatona has joined #openstack-nova05:29
*** amodi has quit IRC05:29
*** jafeha__ has joined #openstack-nova05:30
*** moshele has joined #openstack-nova05:33
*** jafeha has quit IRC05:33
*** yamamoto has joined #openstack-nova05:34
*** janki has joined #openstack-nova05:38
*** yamamoto_ has quit IRC05:38
*** mdnadeem has joined #openstack-nova05:47
*** amodi has joined #openstack-nova05:48
*** hongbin has quit IRC05:53
*** threestrands has quit IRC06:10
*** threestrands has joined #openstack-nova06:23
*** kholkina has joined #openstack-nova06:32
*** acormier has joined #openstack-nova06:35
*** acormier has quit IRC06:40
*** alexchadin has joined #openstack-nova06:42
*** amodi has quit IRC06:51
*** jafeha__ is now known as jafeha06:58
*** threestrands has quit IRC07:00
*** rcernin has quit IRC07:11
*** maciejjozefczyk has quit IRC07:23
hrwmorning07:26
*** andreas_s has joined #openstack-nova07:27
*** alexchadin has quit IRC07:27
*** hoonetorg has quit IRC07:30
*** yamamoto has quit IRC07:33
*** alexchadin has joined #openstack-nova07:39
*** yamamoto has joined #openstack-nova07:41
*** slaweq has joined #openstack-nova07:43
*** slaweq_ has joined #openstack-nova07:45
*** yamamoto has quit IRC07:46
*** hoonetorg has joined #openstack-nova07:46
*** pcaruana has joined #openstack-nova07:47
*** slaweq has quit IRC07:48
*** links has quit IRC07:55
*** AlexeyAbashkin has joined #openstack-nova07:56
*** slaweq has joined #openstack-nova08:03
*** links has joined #openstack-nova08:07
*** damien_r has joined #openstack-nova08:09
*** ccamacho has joined #openstack-nova08:09
*** slaweq has quit IRC08:10
*** ttsiouts has joined #openstack-nova08:10
*** tesseract has joined #openstack-nova08:15
*** ragiman has joined #openstack-nova08:22
*** sree_ has joined #openstack-nova08:29
*** sree_ is now known as Guest7922208:29
*** ralonsoh has joined #openstack-nova08:30
*** rmcall has quit IRC08:31
*** sree has quit IRC08:32
*** Eran_Kuris has joined #openstack-nova08:38
*** krtaylor has quit IRC08:38
*** yamamoto has joined #openstack-nova08:39
*** Eran_Kuris has quit IRC08:40
*** amoralej|off is now known as amoralej08:41
*** jpena|off is now known as jpena08:43
*** bauwser is now known as bauazs08:53
*** bauazs is now known as bauzas08:53
bauzasgood morning Nova08:54
*** Dinesh_Bhor has quit IRC08:56
hrwbauzas: morning08:56
hrwmy patch to control pcie ports is ready for reviews: https://review.openstack.org/545034 - there are tests for aarch64 and for x86-6408:57
hrwrelease note present, config entry commented08:57
*** sean-k-mooney has quit IRC08:59
*** Dinesh_Bhor has joined #openstack-nova09:01
*** tssurya has joined #openstack-nova09:04
*** lpetrut has joined #openstack-nova09:06
*** tetsuro has joined #openstack-nova09:07
*** elmaciej has joined #openstack-nova09:08
*** Dinesh_Bhor has quit IRC09:11
*** Dinesh_Bhor has joined #openstack-nova09:12
*** ameeda has joined #openstack-nova09:16
bauzashrw: I beg you for my ignorance about PCIe ports, but when you say "libvirt adds pcie ports as it needs" in https://review.openstack.org/#/c/545034/8//COMMIT_MSG, WDYM ?09:17
*** cdent has joined #openstack-nova09:17
bauzashrw: when you say "libvirt", it means libvirtd, right? not the nova libvirt driver09:18
ameedaHello when I remove pycrypto as this bug needed https://bugs.launchpad.net/openstack-requirements/+bug/1749574, I got this error https://pasteboard.co/H8l6IC4.png09:19
openstackLaunchpad bug 1749574 in OpenStack DBaaS (Trove) "[tracking] removal and migration of pycrypto" [Undecided,In progress] - Assigned to Ameed Ashour (ameeda)09:19
hrwbauzas: when I say libvirt I mean upstream libvirt so yes, libvirtd09:19
hrwameeda: migrating pycrypto -> cryphography?09:20
*** derekh has joined #openstack-nova09:21
ameedahrw: requirements.txt file already has cryphography09:22
*** mgoddard_ has joined #openstack-nova09:23
ameedahttps://github.com/openstack/barbican/blob/master/requirements.txt#L709:23
hrwsure09:24
*** lajoskatona has quit IRC09:25
*** r-daneel has quit IRC09:28
ameedahrw: as I saw, the error caused by test_dogtag, what do you think ?09:28
bauzashrw: had some comments09:28
bauzashrw: the main one is that I think you should create a blueprint for your feature09:28
bauzashrw: and engage with the community whether you need a spec09:29
bauzasit's better for tracking your feature09:29
*** salladi has joined #openstack-nova09:29
bauzasyou're also adding a new config option, that properly needs to be released09:29
bauzashrw: see https://docs.openstack.org/nova/latest/contributor/blueprints.html for the process-ish thingy09:30
bauzashrw: I'm personnally on the side that a blueprint is enough, we don't really need a spec09:30
bauzasbut devil could be in details, and somehow some people could argue about your implementation design that would mean a spec is needed09:31
*** takashin has left #openstack-nova09:32
*** links has quit IRC09:32
*** Dinesh_Bhor has quit IRC09:33
hrwbauzas: thanks. will create bp09:33
*** salladi has quit IRC09:34
bauzashrw: add your blueprint in the next nova meeting agenda then09:35
hrwok09:35
bauzasand be present at the nova meeting09:35
hrwsure09:35
bauzasso we could validate your blueprint or ask you for a spec09:35
bauzasyou're not in a rush, we're just beginning to review specs :)09:36
*** alexchadin has quit IRC09:36
hrwbauzas: would love to backport it to queens one day09:36
bauzasin case you're not familiar with the release cadence, we generally stop approving specs and blueprints past the first milestone09:36
bauzashrw: well, that's a feature, and we don't feature backport upstream09:37
hrwplease rephrase last sentence09:37
bauzashrw: feel free to do whatever you want in your downstream product, but there are no chances that a change including a features relnote could be backported09:37
hrwok.09:38
bauzasspeaking of the differences between the upstream nova project and any other distribution09:38
bauzashrw: I guess you're not familiar with the stable branches procedure, lemme find you some doc09:39
bauzashrw: https://docs.openstack.org/project-team-guide/stable-branches.html09:39
hrwbauzas: so far backported just simple changes09:39
*** masahisa has joined #openstack-nova09:40
bauzashrw: np, not all of us are familiar with all the procedures, I get that :p09:42
hrwyep09:42
*** links has joined #openstack-nova09:45
*** r-daneel has joined #openstack-nova09:46
*** Eran_Kuris has joined #openstack-nova09:54
*** Eran_Kuris has quit IRC09:55
*** Eran_Kuris has joined #openstack-nova09:55
hrwbauzas: replied09:58
hrwbauzas: and thanks for detailed review09:58
*** salv-orl_ has quit IRC10:02
*** salv-orlando has joined #openstack-nova10:03
hrwhttps://blueprints.launchpad.net/nova/+spec/configure-amount-of-pcie-ports10:05
*** r-daneel has quit IRC10:06
*** salv-orlando has quit IRC10:07
*** Guest79222 has quit IRC10:09
*** mdnadeem has quit IRC10:11
*** sree_ has joined #openstack-nova10:12
*** sree_ is now known as Guest8507510:12
*** zames2 has joined #openstack-nova10:14
*** kmalloc has joined #openstack-nova10:14
*** Guest85075 has quit IRC10:16
*** r-daneel has joined #openstack-nova10:22
*** mdnadeem has joined #openstack-nova10:27
*** mdnadeem_ has joined #openstack-nova10:30
*** mdnadeem has quit IRC10:32
*** lajoskatona has joined #openstack-nova10:33
*** moshele has quit IRC10:33
*** rmart04 has joined #openstack-nova10:34
*** cdent has quit IRC10:36
*** Eran_Kuris has quit IRC10:38
*** zames2 has quit IRC10:39
*** stakeda has quit IRC10:40
*** lajoskatona has quit IRC10:40
hrwbauzas: where in meeting agenda would you suggest to add spec link?10:41
bauzashrw: in the nova meeting agenda, lemme find it10:41
hrwhttps://wiki.openstack.org/wiki/Meetings/Nova10:41
hrwbauzas: I mean "which part of meeting"10:41
*** abhishekk has quit IRC10:48
*** zames2 has joined #openstack-nova10:48
*** hshiina has quit IRC10:50
bauzashrw: sorry got distracted10:51
hrwbauzas: no problem. work is work10:51
bauzashrw: so, in https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting (seems a bit stale, you should comment that it's for the 22th), in Open Discussion10:52
hrwthx10:52
bauzassaying "specless BP approval"10:52
hrwbauzas: added10:54
openstackgerritMarcin Juszkiewicz proposed openstack/nova master: Allow to configure amount of PCIe ports  https://review.openstack.org/54503410:56
*** alexchadin has joined #openstack-nova10:58
hrwcommit message partially rewritten + BP link. some suggested code changes10:58
*** cdent has joined #openstack-nova11:01
*** AlexeyAbashkin has quit IRC11:07
openstackgerritChris Dent proposed openstack/nova master: Fix PatternPropertiesTestCase for py 3.6  https://review.openstack.org/54579811:09
*** ragiman has quit IRC11:13
*** ragiman has joined #openstack-nova11:18
*** lucas-afk is now known as lucasagomes11:18
*** priteau has joined #openstack-nova11:19
*** krtaylor has joined #openstack-nova11:22
*** dtantsur|afk is now known as dtantsur11:23
*** salv-orlando has joined #openstack-nova11:25
*** ccamacho has quit IRC11:26
*** moshele has joined #openstack-nova11:26
*** sree has joined #openstack-nova11:27
*** ccamacho has joined #openstack-nova11:28
*** sree has quit IRC11:32
*** AlexeyAbashkin has joined #openstack-nova11:32
*** alex_ has joined #openstack-nova11:34
*** udesale_ has joined #openstack-nova11:35
*** udesale has quit IRC11:37
*** udesale_ has quit IRC11:39
*** alex_ is now known as fusmu11:40
*** bnemec is now known as bnemec-pto11:41
*** ccamacho is now known as ccamacho|lunch11:46
openstackgerritTetsuro Nakamura proposed openstack/nova-specs master: Support shared/dedicated vCPUs in one instance  https://review.openstack.org/54573411:47
*** cdent has quit IRC11:56
*** masber has joined #openstack-nova11:58
*** alexchadin has quit IRC11:59
*** sree has joined #openstack-nova12:03
*** slaweq has joined #openstack-nova12:07
*** zames2 has quit IRC12:08
*** slaweq has quit IRC12:12
*** dave-mccowan has joined #openstack-nova12:16
*** cdent has joined #openstack-nova12:16
*** belmoreira has joined #openstack-nova12:16
*** trinaths has quit IRC12:19
*** slagle has joined #openstack-nova12:20
*** yamamoto has quit IRC12:21
openstackgerritTetsuro Nakamura proposed openstack/nova-specs master: Support shared and dedicated VMs in one host  https://review.openstack.org/54380512:23
*** tetsuro has left #openstack-nova12:24
*** yamamoto has joined #openstack-nova12:25
*** sdague has joined #openstack-nova12:25
*** takashin has joined #openstack-nova12:27
openstackgerritTetiana Lashchova proposed openstack/python-novaclient master: Fix the docstring for the update method  https://review.openstack.org/54581912:32
*** acormier has joined #openstack-nova12:35
*** yasemin has quit IRC12:36
*** links has quit IRC12:37
*** jpena is now known as jpena|lunch12:39
*** acormier has quit IRC12:40
*** hiro-kobayashi has quit IRC12:41
openstackgerritSurya Seetharaman proposed openstack/nova master: Extending delete_cell --force to delete instance_mappings  https://review.openstack.org/54007312:42
*** sean-k-mooney has joined #openstack-nova12:46
sean-k-mooneystephenfin: enhancing your specs with images http://logs.openstack.org/90/541290/4/check/build-openstack-sphinx-docs/da8d825/html/specs/rocky/approved/numa-aware-vswitches.html thats almost cheating :P12:49
cdentI thought the same thing12:51
*** psachin has quit IRC12:51
*** artom__ has quit IRC12:52
sean-k-mooneycdent: you can tell being on the docs team is rubing off on him :) that said the fact that he uses images to help explain numa rather then 10 paragraps of text no one will follow might be a reason he is a good person to be on the docs team.12:55
cdentindeed12:56
*** masahisa has quit IRC12:56
*** tssurya has quit IRC13:03
*** edmondsw has joined #openstack-nova13:12
*** sree_ has joined #openstack-nova13:14
*** sree_ is now known as Guest8797213:15
*** sree has quit IRC13:17
*** mchlumsky has joined #openstack-nova13:17
*** tssurya has joined #openstack-nova13:18
*** ccamacho|lunch has quit IRC13:27
*** ccamacho has joined #openstack-nova13:27
*** jpena|lunch is now known as jpena13:31
*** mchlumsky has quit IRC13:35
*** mchlumsky has joined #openstack-nova13:38
*** mgoddard_ has quit IRC13:39
*** acormier has joined #openstack-nova13:40
openstackgerritJay Pipes proposed openstack/nova-specs master: Account for host agg allocation ratio in placement  https://review.openstack.org/54468313:41
*** pchavva has joined #openstack-nova13:42
*** pchavva1 has joined #openstack-nova13:43
*** acormier has quit IRC13:43
*** acormier has joined #openstack-nova13:44
*** belmorei_ has joined #openstack-nova13:46
*** belmoreira has quit IRC13:48
*** vladikr has joined #openstack-nova13:48
cdentsean-k-mooney: live migration is for virtualisers not cloud native people! true or false?13:48
*** liverpooler has joined #openstack-nova13:50
*** acormier has quit IRC13:51
kashyapFalse13:51
kashyapcdent: Because:13:51
cdentkashyap for context https://review.openstack.org/#/c/541290/4/specs/rocky/approved/numa-aware-vswitches.rst@241 (in other words I was mostly making a joke, but still want to know the because)13:52
kashyapPeople say Cloud is all about 'cattle' (sorry, can't think of a less trite analogy), but people _do_ care about live migratability in Cloud13:52
kashyapcdent: I realized you're joking with a straight face (also with the exclamation) :P13:53
* kashyap clicks on the review13:53
cdentmuch of that spec has trouble aligning with cattle-ness, but people do care about it13:54
*** masuberu has joined #openstack-nova13:54
edleafeScheduler subteam meeting in 5 minutes in #openstack-meeting-alt13:55
kashyapcdent: Saw asomething else on the spec, will comment there too13:55
*** masber has quit IRC13:58
*** yasemin has joined #openstack-nova13:59
sean-k-mooneycdent: cloud native people should be able to have there application live migrated too.13:59
cdentsean-k-mooney: for sake of getting your point of view, why? Shouldn't it just be restarted somewhere else?14:00
sean-k-mooneycdent: live migration is really to allow operators to do maintenace when they done really know what the tenant is running14:00
edmondswrestarting somewhere else means possible downtime14:01
sean-k-mooneycdent: live migration in openstack is admin only so its tenant initiated its operator driven14:01
*** ttsiouts_ has joined #openstack-nova14:01
*** artom has joined #openstack-nova14:01
*** acormier has joined #openstack-nova14:01
*** acormier has quit IRC14:01
sean-k-mooneyedmondsw: yes that said livemigration could still violate your sla with your tenant14:01
cdentsurely every service should have at least two instances of itself all the time :)14:02
*** leakypipes is now known as jaypipes14:02
*** acormier has joined #openstack-nova14:02
edmondswcdent ha! if only14:02
sean-k-mooneycdent: the best example counter example to that is a provider edge router14:02
edmondswsean-k-mooney that all depends14:02
edmondswhence your "could", I'm sure :)14:02
*** jackie-truong has joined #openstack-nova14:02
cdentshadow provider edge router14:03
sean-k-mooneycdent: you cant have ha by design as it is respocible for ratelimiting your broadband connection so all traffic from one endpoint most be handeled by 1 PE router14:03
*** zames2 has joined #openstack-nova14:03
*** zames2 has quit IRC14:03
*** zames2 has joined #openstack-nova14:04
sean-k-mooneycdent: you dont want to reboot in this case because it can thke 10+ minutes to rebuild your peering relationships with other PE routers14:04
*** takedakn has joined #openstack-nova14:05
*** acormier has quit IRC14:05
kashyapcdent: High availability, is it...14:06
kashyapYeah, sean-k-mooney explained it clearer -- about migratability14:06
*** amoralej is now known as amoralej|lunch14:06
cdentI hear and ack what you're saying.14:06
cdentBut will remain frustrated :)14:06
kashyapJust take a look at the number of config knobs we have for live migration14:07
kashyapA common scenario for it in OpenStack context is upgrades14:07
edmondswalso think about dev on the cloud... I'm still developing this service and haven't setup HA yet, but I don't want you restarting it while I'm working14:07
kashyapWhere you update Compute-B, migrate all instances from Compute-A to B14:07
kashyapAnd so forth14:07
sean-k-mooneycdent: well live migration cannot always be assumed to be available in a cloud and even if it is it my not be viable due to your perfromce constratints14:07
kashyapYeah, random restarts that catch tenant users off-guard are completely undesirable14:08
*** moshele has quit IRC14:09
kashyapsean-k-mooney: Yeah, that's also why, as you pointed out, tenant users don't get to migrate instances.  It is admins that drive it.14:09
cdentSounds like there could be a problem with "random restarts" that is orthogonal to everything else14:09
sean-k-mooneyi would generally say livemigration is more of an enterprise feature then telco as most nfv application cant tollerate the dataplane impact of a livemigration (latency,bandwith and packet loss)14:09
kashyapTrue14:10
edmondswkashyap tenant users don't get to migrate instances because migration assumes knowledge of the infrastructure. If I don't have that knowledge, why would I try to move it :)14:11
sean-k-mooneycdent: to your original question a cloud native application should not require livemigration to be avalible at all as it should be fault tolerant enough to cover a cold migration but most applications are not cloud native.14:11
kashyapedmondsw: Yeah, indeed.14:12
cdentI should probably have put some ™ in my original statement to make it clear that I was talking about over-idealized and over-hyped pictures.14:12
edmondswis there such a thing as over-idealized? ;)14:13
sean-k-mooneyedmondsw: that and allowing tennants to live migrate stuff is a great way to ddos you management network as that is what all the openstack installers configure to use by default.14:13
edmondsw++14:14
edleafeedmondsw: that's like being "more unique"14:14
edmondswbut if there's no use case to allow tenants to live migrate (because they lack the necessary infra knowledge) all other concerns become secondary14:14
* edmondsw nods at edleafe14:15
sean-k-mooneyedmondsw:  i would say the existence of apple product in the world would be evedence of over-idealized design above fuction.14:15
edmondswsean-k-mooney touche!14:15
jaypipesmriedem, dansmith, melwitt, bauzas, stephenfin: we've got pretty broad agreement on the approach in https://review.openstack.org/#/c/540111/. would be great to approve that and unblock the patch series implementing it.14:15
*** arvindn05 has joined #openstack-nova14:16
bauzasjaypipes: stephenfin is on PTO this week14:16
bauzasbut I can review it for sure14:16
sean-k-mooneyoh by the way is master open for rocky and i assume not is a good time to repopose the feature based schduling spec form last cycle?14:16
edmondswsean-k-mooney master is open14:17
sean-k-mooneyedmondsw: cool i have some patches to rebase that missed code freeze14:17
edmondswjoin the club!14:17
jaypipessean-k-mooney: feature-based scheduling?14:18
jaypipessean-k-mooney: oh, the nic feature one?14:18
sean-k-mooneyjaypipes: ya rodolfos work14:18
jaypipesgotcha14:18
sean-k-mooneyhe is full time on yardstick in opnfv now so im pickup all his patches at least to the end of Q1 /early Q214:19
*** esberglu has joined #openstack-nova14:20
*** belmorei_ has quit IRC14:21
*** jackie-truong has quit IRC14:21
openstackgerritMatt Riedemann proposed openstack/nova master: Provide a hint when performing a server action can't find the method  https://review.openstack.org/54538214:21
*** belmoreira has joined #openstack-nova14:21
mriedemthe bug fix series starting here needs review https://review.openstack.org/#/c/539758/ - +2 on all changes; fixes a problem going back to at least newton, if not since bfv was added forever ago14:23
*** lbragstad has joined #openstack-nova14:23
*** pchavva1 has quit IRC14:25
*** mgoddard_ has joined #openstack-nova14:26
*** jafeha__ has joined #openstack-nova14:26
mriedemjaypipes: i'll go through that spec later this morning if someone else hasn't gotten to it first14:28
mriedemi know dan has been through it before14:29
jaypipesmriedem: I'll go through that patch series for the bug.14:29
*** jafeha has quit IRC14:29
*** pchavva has quit IRC14:29
efriedkumbaya14:30
sean-k-mooneyjaypipes: mriedem  by the way the intel nfv ci should be operational again. can ye see any logs here http://http//52.27.155.124//portland/2018-02-19/540073/4/check/tempest-dsvm-intel-nfv-xenial/b045a8e14:30
*** takedakn has quit IRC14:30
sean-k-mooneyoh i see whats wrong they are not generating the url correctly.. ill let them know14:31
*** jistr is now known as jistr|mtg14:31
*** yasemin has quit IRC14:32
*** udesale has joined #openstack-nova14:38
*** rmart04 has quit IRC14:40
*** udesale has quit IRC14:40
*** udesale has joined #openstack-nova14:40
*** links has joined #openstack-nova14:41
*** sree has joined #openstack-nova14:41
*** eharney has joined #openstack-nova14:42
*** rmart04 has joined #openstack-nova14:43
*** pchavva has joined #openstack-nova14:44
*** Guest87972 has quit IRC14:45
*** jroll has quit IRC14:47
*** Placeed has joined #openstack-nova14:47
*** jroll has joined #openstack-nova14:48
*** tssurya has quit IRC14:49
*** links has quit IRC14:49
PlaceedHi all, I have a problem with ressource_tracker.py on my compute instance. This script return a disk usage of 90% to nova so I can't run new instances. The problem is on the calculation ... Actually I use Cinder with NFS backend on /var/lib/nova/mnt with few tera avaiable. I don't run ephemeral vm's. It sounds like the script sum all disk usage of vm running and check only the local disk space of the compute server.14:50
PlaceedSomeone can help me ?14:50
*** archit has joined #openstack-nova14:50
*** archit is now known as amodi14:51
*** arvindn05 has left #openstack-nova14:52
PlaceedAs you can see there : https://paste.ofcode.org/SGqTUpeuWMErjfUwmHwQN2 The node think he have only 270Go available but in reality he have few tera through NFS14:53
*** lbragstad has quit IRC14:53
efriedPlaceed: Is this a boot-from-volume setup?14:55
*** lbragstad has joined #openstack-nova14:55
Placeedefried : Yes, In user point of view, I create a new instance based on image and it will create a cinder volume for the root disk of the instance14:56
*** amoralej|lunch is now known as amoralej14:56
PlaceedBut it failed to find a host acceptable because of this "bug"14:57
*** sree has quit IRC14:57
*** awaugama has joined #openstack-nova14:58
*** kholkina has quit IRC14:58
kashyapbauzas: I'm not sure a spec is required in this case https://review.openstack.org/#/c/545034/ ("Allow to configure amount of PCIe ports")14:58
*** sree has joined #openstack-nova14:58
*** yamahata has quit IRC14:58
*** yamahata has joined #openstack-nova14:58
bauzaskashyap: sure, I discussed with hrw about the process14:58
bauzashe created a BP, and asked for a specless approval for the next nova meeting14:59
kashyapAh-ha14:59
kashyaphrw: The above change should also mention something about its usefulness in context of Nova coping with Q35 machine types14:59
kashyap(Recall the discussion from #virt, OFTC)14:59
kashyapAt some point distributions will default to the Q35, and Nova should be ready to gracefully handle it.14:59
kashyapSo your change helps with that; so it's worth mentioning.15:00
*** tidwellr has joined #openstack-nova15:00
kashyapI'll add a note in the review15:00
*** dtantsur is now known as dtantsur|brb15:01
*** links has joined #openstack-nova15:01
*** ttsiouts_ has quit IRC15:02
efriedPlaceed: I know mriedem has been looking at bfv stuff lately, though I don't know the details.  I think he has recently put up a series of fixes - but again, no idea if the bug is related to this.  Looking...15:02
*** jistr|mtg is now known as jistr15:02
efriedPlaceed: Here's the code series: https://review.openstack.org/#/c/539758/15:02
openstackgerritHamdy Khader proposed openstack/nova master: Adding NVMEoF for libvirt driver  https://review.openstack.org/48264015:02
*** acormier has joined #openstack-nova15:03
efriedPlaceed: Hmm, the bug doesn't sound like your issue: https://bugs.launchpad.net/nova/+bug/140486715:03
openstackLaunchpad bug 1404867 in OpenStack Compute (nova) "Volume remains in-use status, if instance booted from volume is deleted in error state" [Medium,In progress] - Assigned to melanie witt (melwitt)15:03
*** yasemin has joined #openstack-nova15:03
*** sree has quit IRC15:03
mriedemefried: it's not,15:05
mriedemit's likely https://bugs.launchpad.net/nova/+bug/146917915:05
openstackLaunchpad bug 1469179 in OpenStack Compute (nova) "instance.root_gb should be 0 for volume-backed instances" [Medium,In progress] - Assigned to melanie witt (melwitt)15:05
mriedemhttps://review.openstack.org/#/q/topic:fix-bfv-boot-resources+(status:open+OR+status:merged)15:05
*** fusmu has quit IRC15:05
*** zames2 has quit IRC15:05
*** tssurya has joined #openstack-nova15:05
mriedemwhich has been continually deferred until we have sharing disk_gb provider support, which continually gets deferred15:05
efriedmriedem: "sharing disk_gb provider support" as in placement?15:06
*** bhujay has quit IRC15:06
*** cdent_ has joined #openstack-nova15:06
*** cdent has quit IRC15:06
*** acormier has quit IRC15:06
mriedemyeah15:06
efriedokay.15:06
mriedemhaving the computes / RT be aware of shared storage pools15:06
Placeedmriedem : it really sounds like https://bugs.launchpad.net/nova/+bug/146917915:06
openstackLaunchpad bug 1469179 in OpenStack Compute (nova) "instance.root_gb should be 0 for volume-backed instances" [Medium,In progress] - Assigned to melanie witt (melwitt)15:06
mriedemPlaceed: yes it's an old old bug15:07
*** acormier has joined #openstack-nova15:07
*** cdent has joined #openstack-nova15:07
*** bhujay has joined #openstack-nova15:07
sean-k-mooneykashyap: i responded to your question regarding numa aware vswitches15:07
Placeedmriedem : It mean there is a fix actually ? I'm running SUSE Official Openstack, maybe they run an old version ?15:07
efriedPlaceed: There's not a fix yet.15:08
*** takashin has left #openstack-nova15:08
*** rmart04_ has joined #openstack-nova15:08
mriedemPlaceed: not unless suse patched it in their distro15:08
kashyapsean-k-mooney: Thanks; will read15:08
sean-k-mooneykashyap: the patch you linked is not quite what we need but i have added the details to the spec for what we would need to allow numa aware pcie virtualisation within the guest15:08
efriedPlaceed: The shared provider support mriedem is talking about is feature work that's been in progress for several releases now.15:08
*** rmart04 has quit IRC15:08
*** rmart04_ is now known as rmart0415:08
Placeedmriedem : Could you help me to identify which fix it is ? I can ask them15:08
mriedemPlaceed: there is no fix upstream yet15:08
kashyapsean-k-mooney: Ah; right.  For PCIe, we'd also mention the need for Q35 machine type — as that's mandatory for PCIe15:09
kashyap(I'd guess you already know that.)15:09
mriedemPlaceed: there is https://review.openstack.org/#/q/topic:fix-bfv-boot-resources+(status:open+OR+status:merged) but there is not agreement on merging those changes15:09
mriedemas they would introduce some technical debt15:09
sean-k-mooneykashyap: well i dont think it acutlly is15:09
mriedemPlaceed: you could take those, rebase them, and patch them into your env as a workaround15:09
sean-k-mooneykashyap: can you show me where that is stated?15:09
kashyapsean-k-mooney: Let me double-check on my comment w/ the Virt folks15:09
kashyapBecause, pretty sure I was told so by a QEMU dev15:09
efriedPlaceed: For the "real fix", a lot of the groundwork has been laid at this point, and it's just possible we'll put it on the slate for Rocky.  We'll be talking about that at the PTG next week.15:09
mriedemPlaceed: but if those patches never land, you're left with the fork15:09
Placeedmriedem : But it mean actually everybody who is using NFS / CInder have the same issue on openstack right ?15:10
PlaceedOr it's only me on my suse distribution15:10
mnasergood morning everyone, just going to drop https://review.openstack.org/#/q/status:open+topic:bug/1404867 here if someone feels like going through these patches, they're ready for final review (afaik)15:10
sean-k-mooneykashyap: based on https://github.com/qemu/qemu/blob/master/docs/pcie.txt it does seam to be related to Q35 but not sure its a hard dependecy15:10
*** cdent_ has quit IRC15:10
mriedemPlaceed: it's not just you15:11
hrwkashyap: thx15:11
sean-k-mooneykashyap: using the Q35 chipset is likely a good idea anyway15:11
Placeedmriedem : So another workaround would be to have a local disk as big as all my vm's root disk running on that server right ?15:11
hrwkashyap: BP mentions q35 but right, commit can describe it more too15:12
hrwkashyap: release note describes x86/q35 and aarch64/virt15:13
kashyapsean-k-mooney: Okay, I double-confirmed, and indeed Q35 is a hard dep for PCIe.15:14
kashyapsean-k-mooney: That doc you linked to should be updated; I'll update it15:14
mriedemPlaceed: i'm not sure about that15:14
kashyapsean-k-mooney: But, look here (the URL from the doc you mentioned), on slide-13: https://wiki.qemu.org/images/4/4e/Q35.pdf15:15
kashyapsean-k-mooney: It says: "Q35-only features: PCIe goodies [...]"15:15
hrwsean-k-mooney: q35 is hard dep for pcie.15:15
-openstackstatus- NOTICE: Zuul has been restarted to pick up latest memory fixes. Queues were saved however patches uploaded after 14:40UTC may have been missed. Please recheck if needed.15:15
Placeedmriedem : What could be a workaround while i wait the official patch ?15:15
sean-k-mooneykashyap: yes but the feature i was memtioning is not listed in that slide and qemu had limited pcie support before the uese of q3515:16
hrwsean-k-mooney: qemu has 3 x86 platforms: isa-pc (no one uses), i440fx (pci based), q35 (pcie based)15:16
*** sree has joined #openstack-nova15:16
mriedemPlaceed: i'm not intimately familiar with the bug nor the fix - melwitt has carried the patch for a long time, so she might be able to help once she's around15:16
sean-k-mooneyhrw: really i taught i440fx supported pcie15:16
hrwsean-k-mooney: i440fx is older than pcie ;D15:16
mriedemPlaceed: you could also ask about that bug in the #openstack-operators channel and ask if anyone has patched it out of tree and if so, how?15:17
kashyapsean-k-mooney: Yeah, first I'll read your full comment before talking further.  And yes, Q35 does have the limitation you note.15:17
mriedemPlaceed: i know SAP is probably doing something for it15:17
mriedemthey use NFS for everything, and are i think suse customers15:17
Placeedmriedem : Ok thank you very very much for your help. I will also check with SUSE Guys.15:18
kashyapsean-k-mooney: My second sentence in my previous message was misphrased.  Correct: Q35 _solves_ that limitation of older QEMUs PCIe support.15:18
hrwwould be nice to see some future qemu dropping <q35 ;D15:19
hrwbut that's not gonna happen15:19
bauzasstupid question but we get network information from the compute right ? https://bugs.launchpad.net/nova/+bug/174024115:19
openstackLaunchpad bug 1740241 in OpenStack Compute (nova) "Network info not always displayed for a created instance" [Undecided,New]15:19
bauzasat least until we do that from the conductor15:20
*** mlavalle has joined #openstack-nova15:20
*** mdnadeem_ has quit IRC15:20
*** rmart04 has quit IRC15:20
*** rmart04 has joined #openstack-nova15:21
*** sree has quit IRC15:21
mriedemefried: questoin here https://review.openstack.org/#/c/540111/5/specs/rocky/approved/update-provider-tree.rst@7315:21
bauzasnevermind, got the line15:21
*** elmaciej has quit IRC15:22
mriedemlyarwood: are you around this week? i want to flush some ocata patches since ocata eol is supposedly next week15:23
kashyaphrw: Yes, that won't happen.  Majority still use the 'i440fx', as you can guess.15:24
hrwyep15:25
kashyapmriedem: FWIW, I at least saw him on IRC briefly in the (CET) morning15:25
*** slaweq has joined #openstack-nova15:26
*** sree has joined #openstack-nova15:27
bauzasmriedem: I can help15:28
bauzasmriedem: lemme know which ones you'd like to see reviewed15:28
bauzasfor the moment, looking at LP15:28
*** cdent has quit IRC15:28
mriedembauzas: https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/ocata+label:Code-Review=215:30
mriedemsome of those might have things on pike yet15:31
mriedemlike https://review.openstack.org/#/c/539008/15:31
bauzasmriedem: ack, will look at them ASAIC15:31
mriedemthanks15:31
*** slaweq has quit IRC15:31
*** sridharg has quit IRC15:31
bauzas24 open bugs :p15:31
bauzaswoooot15:31
*** acormier_ has joined #openstack-nova15:32
*** burt has joined #openstack-nova15:34
*** lucasagomes is now known as lucas-hungry15:34
*** acormier has quit IRC15:36
*** sree has quit IRC15:37
sean-k-mooneyhrw: kashyap is there a spec to enable Q35 proposed. i assume it will be via an image metadata item or perhaps a config option in the libvirt section of the nova.conf15:45
kashyapsean-k-mooney: I updated the upstream QEMU doc to clearly spell out machine type requirements for PCIe features.15:46
hrwsean-k-mooney: you set hw_machine_type = q35 in image and done15:46
kashyapsean-k-mooney: Hmm, not that I know of; I have a TODO item to propose one, unless someone beats me to it15:46
*** awaugama has quit IRC15:46
kashyapBut yeah, currently what hrw tells above ^15:46
hrwsean-k-mooney: and then you review https://review.openstack.org/#/c/545034/ to have slots for hotplug ;d15:46
kashyapYou can either set it via the machine type Nova image metadat property15:46
kashyapOr via the global configuration option15:46
hrwsean-k-mooney: as AArch64 porter I would love to see more and more people moving to Q3515:47
kashyapsean-k-mooney: FWIW, I sent that (hopefully) clarifying patch -- https://lists.nongnu.org/archive/html/qemu-devel/2018-02/msg04905.html15:47
hrwsean-k-mooney: as then they will share similar issues as we have on aarch64 ;D15:50
*** sree_ has joined #openstack-nova15:50
*** sree_ is now known as Guest1819115:51
*** slaweq_ has quit IRC15:53
*** dtantsur|brb is now known as dtantsur15:54
*** slaweq has joined #openstack-nova15:54
*** Placeed has quit IRC15:55
* hrw off15:56
*** ccamacho has quit IRC15:56
kashyapsean-k-mooney: So these are the two possibilities with Nova:15:57
kashyap(1) Image metadata property:15:57
kashyap    $ openstack image set --property hw_machine_type=x86_64=pc-q35-2.10 Fedora27_Template15:57
kashyap(2) Nova conf:15:57
kashyap    [libvirt]15:57
kashyap    ...15:57
kashyap    hw_machine_type=x86_64=pc-q35-2.1015:57
*** ccamacho has joined #openstack-nova15:58
*** slaweq has quit IRC15:58
*** chyka has joined #openstack-nova15:58
kashyapsean-k-mooney: The remaining item in Nova is graceful handling of "distributions shipping 'q35' by default".15:58
*** salv-orlando has quit IRC15:59
*** Guest18191 has quit IRC16:00
openstackgerritJay Pipes proposed openstack/os-vif master: zuul: Enable functional tests in gate  https://review.openstack.org/53096116:03
*** pcaruana has quit IRC16:07
sean-k-mooneykashyap: yep that all makes sense to me. operator can set in in nova.conf if they want to force it by default or tenant can request it in image metadata if there image requires q35 features to work e.g. vIOMMU support16:07
sean-k-mooneykashyap: hrw it seams pretty strait forward to add Q35 support so a specless blueprint might be sufficent. will ye be at the ptg next week?16:09
*** lpetrut has quit IRC16:09
*** cdent has joined #openstack-nova16:09
kashyapsean-k-mooney: Yes, at least I'll be there16:10
kashyap(And nod to your previous comment, too)16:11
*** _pewp_ has quit IRC16:11
*** mdnadeem_ has joined #openstack-nova16:12
* kashyap goes to file a spec-less BP16:13
*** _pewp_ has joined #openstack-nova16:14
*** yamamoto has quit IRC16:16
mriedemdansmith: left a comment in https://review.openstack.org/#/c/544698/2/nova/tests/unit/db/test_sqlalchemy_migration.py - if you're meh on that, then i'll +2 anyway16:16
dansmithmriedem: I'm pretty meh16:19
*** masuberu has quit IRC16:21
*** udesale has quit IRC16:22
*** mdnadeem_ has quit IRC16:22
*** slaweq has joined #openstack-nova16:23
efriedjaypipes, cdent: edleafe: mriedem found a discrepancy in the update_provider_tree spec that I'd like to discuss and clear up.16:23
cdentwas just reading that16:23
efriedHere's the scenario: my compute node is aggregated with a sharing provider.  The sharing provider happens to be part of its own tree.  We want the sharing provider itself to be included in the ProviderTree that u_p_t gets - that's for sure.16:23
*** r-daneel has quit IRC16:23
efriedBut16:23
efriedDo we want to include the sharing provider's whole tree?  Or just the provider itself, as a "root" even if it isn't really a root in its own tree?16:24
efriedjaypipes: I want to say you and I discussed this briefly when I was writing the code and decided on the latter, but thinking through it again, I'm not convinced that's The Right Thing.16:25
edleafeefried: sounds like you have a huge recursion problem if you include trees of trees of trees of ...16:25
mriedemi assume the compute driver got an n-1 view of things it doesn't control16:25
mriedemedleafe: yes agree16:25
jaypipesmriedem: I don't understand what you mean by an "n-1 view"16:26
efriededleafe: Well, we have explicit boundaries.  We definitely don't go and spider out to the sharing provider's aggregate-associated providers.16:26
mriedem1 removed16:26
efried^ that16:26
mriedemso the virt driver gets the compute node + the sharing provider aggregate, but none of that things children16:26
edleafeefried: another way to look at it: would the compute node ever need to know about /interact with any of the shared provider's tree other than the SP itself?16:26
efriedArguably the unrelated bits of the sharing provider's tree are n-2 - but I'm not sure that's a horrible thing.16:26
*** acormier_ has quit IRC16:26
*** rmart04 has quit IRC16:26
cdentI think edleafe's question is the crux.16:27
efriededleafe: No.  But.  I don't feel great about the caveat that it would show up as a root even if it's not a root.  Rather have more information (that's not used) in the tree than have some of it be inaccurate?16:27
mriedemif the virt driver shouldn't mess with stuff outside of the compute node's tree, it should just be able to see that there is a relationship, right? but not the whole other thing's tree, lest the virt driver gets the idea it has access to update that other thing (which it shouldn't do)16:27
*** slaweq has quit IRC16:27
efriedYeah, I agree with that.16:27
*** slaweq has joined #openstack-nova16:28
mriedemi can't tell if it being a root or not matters16:28
efriedThat's the thing.16:28
efriedWe *do* want the virt to be able to mess with the sharing provider.16:28
mriedemif my virt driver shouldn't mess with the sharing provider, why do i care if it's a root or not?16:28
mriedemwhy?16:28
jaypipesmriedem: this goes back to the powervm virt driver structure.16:28
cdentefried: yes we do, but for those sharing providers that we want to mess with, do we want them to existing in a nested hieararchy? I would think/hope not.16:28
jaypipesmriedem: they want the ability to manage storage pools in their virt driver.16:28
efriedAs one example, yes.  I think VCenter also had an example.16:28
cdentvcenter's example is basically the same16:29
*** slaweq has quit IRC16:29
edleafeefried: if the shared provider is the one with the inventory, it would be the one we allocate against, no? Why do we need its root provider?16:29
* cdent points at edleafe 16:29
mriedemthe spec says, "Note,16:29
mriedem  however, that it may contain providers not directly owned/controlled by the16:29
mriedem  compute host.  Care must be taken not to remove or modify such providers16:29
mriedem  inadvertently."16:29
jaypipesedleafe: the virt driver doesn't allocate.16:29
dansmithmriedem: rc2 is tagged, yeah? are we good to land the compute rpc thing now?16:29
efriedmriedem: Yes, there's that too.16:29
sean-k-mooneyjaypipes: just regarding those 2 os-vif changes you +2'd i am happy to merge them too but i think there may be an alternitive way to adress https://review.openstack.org/#/c/531358/ that would also work on centos but it might be tricky to do in the gate so ill run it by ye in dublin.16:29
mriedemdansmith: sure16:29
mriedemdansmith: i haven't looked at it yet16:29
edleafejaypipes: but I thought we were talking placement16:30
*** lucas-hungry is now known as lucasagomes16:30
jaypipesedleafe: we're talking virt driver's use of the ProviderTree struct16:30
dansmithmriedem: okay, it's fairly straightforward and would like to land that soon before we break it of course16:30
efriedThat's for e.g. sean-k-mooney's cases when there's like a bandwidth provider that's owned by neutron, but is part of the compute node's tree structure.16:30
mriedemdansmith: i am now in update_provider_tree land16:30
jaypipessean-k-mooney: feel free to approve them then and we'll discuss in Dublin16:30
edleafejaypipes: sure, but we're also talking about shared providers linked to a compute node16:31
dansmithmriedem: good luck with that16:31
*** slunkad_ has quit IRC16:31
sean-k-mooneyjaypipes: cool. i need to talk to stephen about how to add custom jobs via zullv3. i think i understand but i would like to get a linux bridge job there also16:31
cdentcan we, at least temporarily, make guideance that shared providers don't nest? Does that get us further down the road without having to predict everything?16:32
mriedemefried: jaypipes: it doesn't seem like a good idea for the nova virt driver to be modifying things that aren't directly under nova's control, like compute resources - because if nova says it can mess with storage and network sharing providers, then cinder/neutron can also be mucking with those right? and we have a fun split brain problem.16:32
efriedcdent: It would, yes.16:32
kashyapsean-k-mooney: I need to run to my Dutch class now; will add the blueprint tomm or later when I get back.16:32
*** yamamoto has joined #openstack-nova16:32
cdentor rather, if they do nest, we don't care about it16:32
edleafe^^ that16:32
cdentbecause all we really need to know (for now) is that they are there16:32
efriedmriedem: Up to this point, virt.PowerVM has full control over the Shared Storage Pool.  We want to keep it that way in Placement-land.16:33
jaypipesmriedem: I'm not saying that it's my *preferred* architecture for a virt driver. but it is what it is..16:33
sean-k-mooneyefried: at least for the bandwidth case the provider would not be a shared provider but it was going to be associated to the neutron agent by an aggregate16:33
efriedsean-k-mooney: Okay, maybe it was neutron owning the NICs/PFs?16:34
mriedemefried: are those SSP's restricted to a single nova-compute service?16:34
sean-k-mooneyefried: in that case also not shared16:34
mriedemor can multiple virt drivers be monkeying with the same SSP?16:34
efriedmriedem: The latter.  We recognize and accept the caveats of co-management.16:34
sean-k-mooneyneutron would be creating RPs in the compute node tree in both cases however16:34
efried^16:34
sean-k-mooneymriedem: SSP?16:35
mriedemshared storage pool16:35
efriedsean-k-mooney: Shared Storage Pool, PowerVM style in this case.16:35
sean-k-mooneyoh ya ok16:35
mriedemok so the spec says the virt driver can see these other things but shouldn't modify them, but then you're saying you totally do want to modify them and that's the intent16:36
efriedmriedem: No, those are different things.16:36
mriedemin other words, update_provider_tree is a blank check to the virt driver16:36
mriedemas we've seen with the RT managing allocations,16:37
mriedemthis is probably goign to end badly16:37
*** yamamoto has quit IRC16:37
efriedThe networky examples in sean-k-mooney's camp are going to be providers in the tree that's rooted at the compute node RP.  Those are the ones virt should avoid mucking with.16:37
sean-k-mooneyefried: oh speaking of PowerVM  as an aside it support PCI devices corret but does not expose pci address am i rembering that correctly. i mentioned it on stephenfin's numa aware vswitch spec but was not sure if i remembered correctly16:37
*** acormier has joined #openstack-nova16:37
efriedsean-k-mooney: I haven't made it all the way through that spec yet, but will look out for that, thanks.16:38
mriedemso non-compute child resource providers - don't touch; non-compute shared aggregate providers, go nuts16:38
*** awaugama has joined #openstack-nova16:38
*** ttsiouts_ has joined #openstack-nova16:38
efriedmriedem: Blank check, sort of.  It's going to be important for the virt driver to understand what it's doing with its providers.16:38
mriedemdo you have an example of what a virt driver would need to do with a shared provider aggregate?16:39
efriedmriedem: The caveats are just caveats.  If a virt driver figures out some way it's going to co-manage the providers neutron creates... that sounds weird but doable.16:39
sean-k-mooneymriedem: well it depends for the neutron created resouce providers i had envisioned the nova conductor makeing the claims against them16:39
sean-k-mooneymriedem: that said nova could delegate that to neutron as part of the port bind also16:40
mriedemsean-k-mooney: are you talking about the bw based scheduling spec?16:40
mriedemwe want any kind of 'claim' happening before we hit a compute, which is where port binding happens16:40
sean-k-mooneymriedem: ya. that was one of the open questions should nova claim the bandwith or tell neutron to do it as part of port bining which we want to move to condutor16:41
mriedemport binding was never moving to conductor16:41
efriedmriedem: Sure: we're polling the SSP to get available capacity for inventory reporting.  If that changes (disk is added, disk goes offline, cluster node tanks, whatever), we want to modify the SSP's inventory accordingly.  In this example, whichever virt gets there first, wins.16:41
mriedemport creation was moving to conductor16:41
sean-k-mooneymriedem: it was as part of the multi port binding spec16:41
jaypipessean-k-mooney: port binding != resource allocation. I'd rather have the conductor do the resource allocation steps than the nova-compute during port-binding16:41
*** moshele has joined #openstack-nova16:42
mriedemjaypipes: agree, otherwise we have a late claim on the wrong host and we fail16:42
sean-k-mooneyjaypipes: today yes port-binding happens on compute but intent is for it to move to conductor before we call the compute node16:42
jaypipesmriedem: and we're back to square one again.16:42
*** jpena is now known as jpena|brb16:42
mriedemsean-k-mooney: i don't remember ever talking about moving port binding to conductor16:42
sean-k-mooneylet me see if i can get the relevent part of the spec16:42
*** slaweq has joined #openstack-nova16:42
mriedemin the long ago, johnthetubaguy was working on moving port *creation* to conductor16:42
efriedTo cdent's point, would y'all be on board with imposing a restriction in the code that MISC_SHARES_VIA_AGGREGATE and tree-ness are mutually exclusive?16:43
jaypipesmriedem: and that is resource allocation, not port binding...16:43
* johnthetubaguy nodes longingly16:43
johnthetubaguys/nodes/nods/16:43
efriedFreudian slip ^  :)16:43
johnthetubaguy:)16:43
jaypipesefried: I don't really see the need, frankly...16:43
cdentefried: I think it's probably okay to not enforce it. just don't got parent or child hunting in u_p_t16:43
mriedemjaypipes: port creation is resource allocation? in what way?16:44
mriedembesides quota16:44
efriedjaypipes: It's not a *need* per se; would just allow us to kick this can down the road.16:44
jaypipesefried: just don't ever search deeper than a single node level for any sharing providers16:44
johnthetubaguyit was nice for case where you pick the correct IP segment for routed networks16:44
*** ttsiouts_ has quit IRC16:44
johnthetubaguywhere placement deals with the IP resources, that may be limited in the case of public ones16:44
jaypipesmriedem: what johnthetubaguy said.16:44
jaypipesmriedem: but it's not important right now.16:45
jaypipesmriedem: I would be fine just mandating that sharing provider information gathering never exceeds the single level of the sharing provider.16:45
sean-k-mooneymriedem: jaypipes https://git.openstack.org/cgit/openstack/nova-specs/tree/specs/queens/approved/neutron-new-port-binding-api.rst#n13916:45
efriedjaypipes: Okay, but do we include the sharing provider's whole tree or not?16:45
mriedemefried: i'd say no16:45
cdentno16:46
efriedI'm worried about the future impact of spoofing a child as a root16:46
mriedemefried: i personally don't think the virt driver should be managing all of this stuff16:46
*** bhujay has quit IRC16:46
jaypipessean-k-mooney: oh, that's for live migration... yeah.16:46
cdentefried: just because you don't include the info, doesn't mean it isn't out there16:46
mriedemi mean, we've talked about not wanting nova to orchestrate everything in the cloud, and this is a giant step (it seems to me at least) in that direction of orchestratoin16:46
jaypipescdent: the truth is out there, Scully.16:46
sean-k-mooneyjaypipes: ya but the intent was to do the port binding always in the conductor form that point on. both binding and creation actully16:46
cdentit's all lies jaypipes16:46
efriedcdent: Well, that's kind of the issue, because right now, we have no way in ProviderTree of creating a root with a parent_uuid16:46
mriedemsean-k-mooney: https://git.openstack.org/cgit/openstack/nova-specs/tree/specs/queens/approved/neutron-new-port-binding-api.rst#n139 is not port creation16:47
jaypipesefried: no, we should not. the "whole tree" is clearly more than "the single level of the node".16:47
sean-k-mooneymriedem: right it happens after port creation16:47
mriedemsean-k-mooney: that spec has no reliance on johnthetubaguy's work to move port creation to the super-conductor16:47
mriedemas far as i remember anyway16:47
mriedemit's for live migration, where we already have the port created on a live vm16:48
cdentefried: all we want to do is represent a provider, in tree. you're saying that if a provider has a parent we can't _not_ include the parent in the provider tree?16:48
*** slaweq has quit IRC16:48
mriedemjohn's thing was for picking the correct host for routed networks16:48
efriedcdent: No, I'm saying if we do include it, we have to lie about its parentage.16:48
cdentno we don't. we just don't have it16:48
sean-k-mooneymriedem: that spec was written with live migration but for bandwitdh based sceduling we needed to move port/creation to the conductor too so we can read the bandwith request form the neutron port before hitting placement16:48
*** damien_r has quit IRC16:48
efriedI suppose that's one way of looking at it.16:49
mriedemsean-k-mooney: sure, the bw based scheduling spec is a whole other clusterfuck16:49
mriedemsean-k-mooney: the live migration one for port binding is pretty clear16:49
cdentThe ProviderTree is a representation of what we care about, now. It is not the whole truth?16:49
cdentI would hope that it is just whatever is useful.16:49
sean-k-mooneymriedem: maybe we did not explictly say it in either i guess i was jsut under the impressions we had discussed makeing that change for all code paths16:49
mriedemsean-k-mooney: the bw based scheduling spec has at least like 3 major dependencies16:50
efriedcdent: But please let's not make the argument that we shouldn't include the non-owned descendants of the compute host.16:50
mriedemand at this rate will get done in maybe the next openstack A release16:50
*** jackie-truong has joined #openstack-nova16:50
*** AlexeyAbashkin has quit IRC16:50
mriedemsean-k-mooney: the dep tree within just nova is already in LP https://blueprints.launchpad.net/nova/+spec/bandwidth-resource-provider16:51
efriedcdent: We have literally no way of knowing which ones those would be.16:51
mriedemthat doesn't include changes needed in neutron16:51
sean-k-mooneymriedem: well we could strip it donw a little but ya i know its currently state it need many things like nova neutron negociation via os-vif object which is very nice but strictly required16:51
mriedemyou mean but *not* strictly required?16:51
sean-k-mooneysorry yes16:52
cdentefried: my feeling is that the core meaning of "ProviderTree" is "this compute node and its children". That it includes auxilliary things like shared providers is bonus.16:52
mriedemdansmith: are the CI failures in https://review.openstack.org/#/c/543580/ all unrelated?16:53
sean-k-mooneythe os-vif object depency was there to not have yet another unversioned dictionary of stings going back and fort, it could be done without that chage however16:53
mriedemdansmith: all the grenade jobs failed there?16:53
efriedcdent: Acknowledging that the virt driver at least needs to be able to *see* (if not control) the sharing providers, but isn't allowed to talk to placement, I'm still asserting it's a requirement, not a bonus.16:53
mriedemi think seeing that they are there is OK16:54
mriedemlike, if i have a compute node tree and see that it's getting it's disk_gb from a shared provider, that's good16:54
mriedemthat's something the RT doesn't know about today, and is a problem16:54
cdentefried: it's a requirement for the interface which u_p_t wants to provide to be able to do that. That it is being done within the confines of the ProviderTree object is a matter of convenience (and a reasonably good choice).16:54
efriedptaytah ptahtah, cool.16:54
efriedmriedem: For those other two comments, do you want me to call out hypervisor_hostname and ironic explicitly?16:55
cdentefried: you can understand why "something that is not in this tree" feel like it doesn't fit in the definition of ProviderTree (strictly from a naming standpoint)?16:55
mriedemefried: no, just verifying my understanding16:55
efriedmriedem: And for the sharing tree issue, do you still want some ascii art?16:55
dansmithmriedem: I had only looked at the first one and it's been a while so figured a recheck, let me look at the others16:55
efriedcdent: Yeah.  But I ain't proposing a patch to rename it to ProviderCopse16:56
mriedemdansmith: http://logs.openstack.org/80/543580/1/check/neutron-grenade/f675ec2/logs/screen-n-api.txt.gz?level=TRACE#_Feb_12_22_21_47_79518016:56
mriedemthat's real16:56
cdentooooh. nice name! Then we can have free roaming agents doing copicing.16:56
dansmithah yup16:57
mriedemefried: well the prose is confusing16:57
efriedmriedem: I'm gonna reprose it.16:57
efriedGuess I'll do that and see if you think it's enough.16:57
mriedemefried: if we say that the provider tree just going to see the sharing provider and not if it's a root or none of its children (if it can even have children, but i don't see why it couldn't), then it's probably important16:57
efriedI didn't quiite parse that, but I think I get what you're saying.16:58
* efried edits...16:58
cdentbbl16:58
*** cdent has quit IRC16:58
*** slaweq has joined #openstack-nova17:00
efriedWill future ironic model each ironic node as a separate tree in its own right?  Or will they always be children/subtrees of the compute host?17:01
mriedemhttps://awwapp.com/b/u8ghqr9rr/17:01
mriedemefried: can you see ^?17:01
*** trinaths has joined #openstack-nova17:01
efriedmriedem: yes17:01
*** slaweq_ has joined #openstack-nova17:01
mriedemefried: in that diagram i'm asserting that the provider tree that the virt driver sees is in blue17:01
efriedmriedem: And yes, that's the idea.17:01
efriedYup.17:01
*** yamamoto has joined #openstack-nova17:02
efriedor even...17:02
mriedemcdent: jaypipes: edleafe: agree? ^17:02
sean-k-mooneyefried: i would assume future ironic might model each chassie as a resouce provierd of inventoies of custome_baremetal_node object but i done no about a RP per bermtal node17:02
*** chyka has quit IRC17:02
mriedemefried: there is no resource provider for the compute host, just the nodes17:03
dansmithmriedem: oh, I bet this is because we don't have grenade mappings for queens yet,17:03
mriedemso they'd be separate17:03
dansmithmriedem: so we're actually upgrading from pike to rocky here17:03
efriedmriedem: Okay.  We don't have a way to handle that yet FYI.17:03
dansmithsdague: can you remind me what I need to do for that?17:03
mriedemdansmith: i see a stable/queens branch for grenade now,17:03
mriedemso maybe it was just done after that CI run and your recheck will be happy17:04
dansmithmriedem: yeah, but logs show we have pike computes in there17:04
sean-k-mooneyefried: each ironic node bing its own provider tree?17:04
efriedmriedem: But once we do, it'll be a true statement that the ProviderTree can have multiple full actual trees.17:04
dansmithmriedem: well, maybe, but I thought we had to do something17:04
efriedsean-k-mooney: Yeah17:04
*** belmoreira has quit IRC17:04
sean-k-mooneyefried: that should just be a simple virt driver change no?17:04
efriedno17:04
dansmithmriedem: zuul backlog is 1249 items long, so it hasn't even started running yet :/17:04
efriedsean-k-mooney: rt will have no way to map the ironic node back to the compute host for purposes of scheduling.17:05
*** ragiman has quit IRC17:05
sean-k-mooneyefried: you could use an aggregate for that17:05
efrieds/rt/conductor/ I guess17:05
efriedsean-k-mooney: Could.  But like I say, we don't have that yet.17:05
efriedAnd that would be an interesting case for aggregated provider trees that aren't sharing providers.17:06
*** slaweq_ has quit IRC17:06
*** yamamoto has quit IRC17:06
efriedTBH, I don't know how ironic plans to model in u_p_t.  mgoddard around?17:07
jrollefried: we haven't made any, that I know of17:07
sean-k-mooneyefried: ironic may also want to use either nested resouce providers or aggregates to model node to chassis relationships for blade systems too17:07
* jroll hasn't been engaged at all, and not really watching the u_p_t stuff much17:07
jrolls/at all/on this topic/17:08
*** andreas_s has quit IRC17:08
*** moshele has quit IRC17:08
jrolljaypipes may also have some thoughts on it, though17:08
*** andreas_s has joined #openstack-nova17:09
*** links has quit IRC17:09
*** chyka has joined #openstack-nova17:09
sean-k-mooneyefried: i genally thing of aggregate just as arbitry groupings of RPs and shareing RPs as a special subset of that. in genneral i think aggregates are usfull for far mor thing that are not sharing related the sharing related uscases17:10
efriedsean-k-mooney: I agree that there are plenty of non-sharing use cases for aggregates.17:11
efriedsean-k-mooney: But as currently conceived (per recent refinement) we don't pull in non-sharing aggregated providers at all.17:11
mordredmriedem: is it possible to query nova as an end-user to find out what the default AZ is?17:11
jrollefried: so, in an ideal world, we just... don't care about mapping an ironic node to a compute host for scheduling purposes. because, any compute host can talk to ironic to do the build17:12
mriedemmordred: as in https://developer.openstack.org/api-ref/compute/#get-availability-zone-information doesn't tell you which is the default in config?17:12
efriedjroll: Coolcool.  But n-cond still needs to schedule a deploy to *some* n-cpu somewhere.17:13
*** jackie-truong has quit IRC17:14
efriedjroll: So if the ironic nodes are independent providers (independent of their compute host) how does it know where?17:14
*** andreas_s has quit IRC17:14
jrollefried: agree, and I'm not sure we want to special case it. but if we did: host = random(get_ironic_compute_hosts())17:14
mriedemmordred: doesn't look like it http://paste.openstack.org/show/677614/17:14
jrollefried: I realize now I don't know how that works with placement. in the old world, both the host and node are in the compute_nodes record17:15
mriedemmordred: well, i guess if there is only 1 'available' zone then you know what the default is :)17:15
efriedjroll: Oh, so all ironic nodes under an n-cond can be managed by any n-cpu under that same n-cond?17:15
jrollefried: in theory, yes, we don't do that in reality today17:15
jrollefried: but we do currently shuffle them between ironic n-cpu hosts that are up, as the hosts go up and down17:16
*** felipemonteiro has joined #openstack-nova17:18
efriedjroll: But don't ops also always deploy specific ironic nodes?17:18
jrollefried: as in "nova boot this-specific-machine"?17:19
efriedYeah; maybe I'm misremembering that.17:19
jrollnope, not at all17:19
efriedk17:19
jrollsome do - and they use things like scheduler hints17:19
efriedRightright, that's what I was thinking - that it's a requirement to be able to do so.17:20
jrollother installations are just another cloud, with the hypervisors missing17:20
jrollefried: the "requirement" part of that depends who you ask :)17:20
jrollI'm not sure it works upstream today17:20
efriedSo yeah, I have absolutely no idea how we're planning to model ironic in Placement-land.  But I suspect there will have to be some additional affordance outside of what we're doing in u_p_t.17:20
efriedBecause right now, if you tried to model each ironic node as its own root/tree, it wouldn't work.17:21
efried(by "right now" I mean "with NRP and u_p_t implemented as specced")17:21
*** itlinux has joined #openstack-nova17:22
*** jpena|brb is now known as jpena17:22
jrollyeah, I'm not up to speed on the nested things17:22
*** tssurya has quit IRC17:22
jrollI suspect jay has ideas here, dunno17:23
*** slaweq_ has joined #openstack-nova17:25
*** acormier has quit IRC17:25
*** acormier has joined #openstack-nova17:25
openstackgerritEric Fried proposed openstack/nova-specs master: Update Provider Tree  https://review.openstack.org/54011117:27
efriedmriedem: See how that grabs ya ^17:27
efriedjaypipes, edleafe, cdent ^17:27
*** cfriesen has joined #openstack-nova17:28
*** slaweq_ has quit IRC17:29
edleafeefried: stop grabbing me!17:32
efriedmriedem: Updated that diagram a little bit too https://awwapp.com/b/u8ghqr9rr/17:32
efriededleafe: Dangit, there goes my career 40 years from now.17:33
mriedemefried: then i think you definitely need a diagram in the spec to give an example17:34
jrollefried: I see nodename everywhere in this spec, that signals to me that one tree per ironic node is expected17:34
mriedemof what's in vs what's out when the driver gets the provider tree object17:34
*** rmart04 has joined #openstack-nova17:34
efriedjroll: That's probably just a lack of precision on my part more than anything.17:34
efriedjroll: I still don't understand the difference between nodename, hypervisor_hostname, and host.name.17:35
*** lpetrut has joined #openstack-nova17:35
efriedjroll: But I wrote the code first, and 'nodename' was the var that made sense to pass around in there, so that's probably how it ended up in the spec.17:35
jrollefried: nodename ~= hypervisor_hostname, I'm not sure what host.name refers to17:36
*** lpetrut has quit IRC17:36
*** lpetrut has joined #openstack-nova17:36
efriednodename == hypervisor_hostname, except for ironic where they're never equal; but hypervisor_hostname == host.name always ???  Or something ???17:36
jrollin the ironic world, nodename == hypervisor_hostname, for sure. compute_node.host is the different one (and the one that actually looks like a hostname)17:37
efriedjroll: When you say ~= is that "regular expression match" à la Perl, or "not equal" à la other things?17:37
efriedoh, okay.17:37
efriedSo that makes zero sense to me, that ironic node name should be the same as the hypervisor host's name.  So the latter is actually a total misnomer.17:38
jrollefried: equal, but I'm not 100% sure on weird drivers, so I'm going with approximately equal :)17:38
efriedgotcha17:38
jrollsure, it's a poorly named thing. originally it was for things like xen, where compute.host (how you reach n-cpu) might be separate from the actual hypervisor (how you reach xenapi)17:39
jrolls/hypervisor/hypervisor_hostname/17:39
mordredmriedem: :) ... and yah - the lack of info about default in config in the az list is, I think, the thing17:40
mordredmriedem: came up in some discussions around nodepool and azs - thanks for confirming17:41
mriedemmordred: from what i remember, there are actually >1 nova config options related to a 'default' az17:41
mordredmriedem: awesome. and of course there are :)17:41
openstackgerritsean mooney proposed openstack/nova master: Change 'InstancePCIRequest' spec field  https://review.openstack.org/44925717:41
openstackgerritsean mooney proposed openstack/nova master: Add Neutron port capabilities to devspec in request  https://review.openstack.org/45177717:41
openstackgerritsean mooney proposed openstack/nova master: Format NIC features using os-traits definitions  https://review.openstack.org/46605117:41
openstackgerritsean mooney proposed openstack/nova master: Read Neutron port 'binding_profile' during boot  https://review.openstack.org/50748117:41
mriedemdefault_availability_zone and default_schedule_zone17:42
*** tidwellr has quit IRC17:42
jrollefried: as far as the one vs many trees for ironic... _update_available_resource is called once per ironic node, not once per nova-compute host. (and thus the same for get_inventory and such). in old-nova terms, it's called once per compute_node record, of which many may exist per compute service. does that make sense?17:42
mordredmriedem: at root is that currently nodepool gets a list of azs and the balances across them by default - which has led to unexpected behavior before (citycloud had an az specifically for a type of flavor and were confused why were launching nodes there)17:42
jrollefried: I'll do a full review of the spec and drop some comments there, if that works better for you17:42
mordredmriedem: but on clouds with more than one az where they're all the same, if we DON'T balance across them we wind up with usage only in the default zone17:43
jrollefried: I guess I'm mostly curious if the "one tree" constraint you're thinking of is "one root per compute service" or "one root per resource provider"?17:43
efriedjroll: That would be great, though I don't know that we're going to get much satisfaction out of that.17:43
mriedemmordred: ok, and in the citycloud case, as a user, you don't have access to see the flavor linked to the AZ via host aggregate17:44
mriedembecause users can't see host aggregates17:44
efriedjroll: I explicitly state in the spec that update_provider_tree is being called once per ironic node, BUT that doesn't really affect how the result is used.17:44
mriedemmordred: in other words, if using this flavor, don't specify AZ17:44
mordredmriedem: yup17:45
mriedemin queens they can at least now put a description on the flavor to say that, but your tooling likely doesn't care about that until you've already tried debugging what's going wrong17:45
mordredmriedem: so there's a few potentially missing pieces of metadata .. however, clarkb has been advocating that we stop trying to balance across azs by default and instead require someone to configure az use explicitly17:46
mriedemand then you have to put in special logic for that flavor on that cloud17:46
*** slaweq_ has joined #openstack-nova17:46
mordredmriedem: exactly. when if there was some metadata, I could even validate the flavor/az combo in shade/openstacksdk17:46
mordredbefore we even bother making an API call17:46
mordred"dude, this combo won't work"17:46
jrollefried: hm, having trouble finding that note, but also seeing now that this code is mostly done, so maybe I need to back up and look at the bigger picture17:46
mriedemmordred: so do you end up getting a novalidhost in the citycloud case with that flavor and some other AZ?17:47
efriedjroll: I was not trying to (actually "trying not to") predict how ironic is going to model.  What I'm trying to say is, if ironic wants to model with a separate tree/root per ironic node, this spec is insufficient to handle it.17:47
*** yamamoto has joined #openstack-nova17:47
mordredmriedem: I don't think so - no - I think what happened was we got scheduled on hardware that was intended for something else (they weren't expecting people to request anything in that az unless someone told them to) so we got nodes that had messed up networking or something else17:48
jrollefried: sure, I'm trying to figure out what about this spec precludes doing such a thing17:48
efriedjroll: https://review.openstack.org/#/c/540111/5/specs/rocky/approved/update-provider-tree.rst L57-6117:48
efriedjroll: It's not anything about the spec.  It's the way the rest of Nova is shaped currently.17:48
jrollefried: ah, that block. I guess that reads to me as "one root per compute node", nothing that compute service != compute node. I guess I need to read the code.17:50
efriedjroll: Actually, the freshly-updated text in the note on L69-76 *does* preclude the multiple-trees-that-aren't-sharing thing.17:50
jrollhmm17:50
*** slaweq_ has quit IRC17:51
jrollok, I will dig, I need to learn more about NRP before I can say much else. thanks, efried17:51
efriedjroll: Let me know if you need pointers.  Much of the code for this is not yet merged.17:52
efriedjroll: Pending code is in series starting at https://review.openstack.org/#/c/537648/17:52
jrollefried: yep, I'm there17:52
*** yamamoto has quit IRC17:52
openstackgerritMerged openstack/nova stable/ocata: Add 'delete_host' command in 'nova-manage cell_v2'  https://review.openstack.org/51372117:53
openstackgerritMerged openstack/nova stable/ocata: Fix test_instance_get_all_by_host  https://review.openstack.org/51648617:53
*** felipemonteiro has quit IRC17:54
lyarwoodmriedem: back online, yeah I'm around this week, I'll take a look this evening if there's anything left.17:54
*** felipemonteiro has joined #openstack-nova17:54
efriedjaypipes: Do you have a couple minutes to help me understand the RT flow for ironic?17:59
*** felipemonteiro has quit IRC17:59
*** derekh has quit IRC18:00
jrollefried: looking through some of this, I'm too far behind on how some of this works to fully discuss why this is or isn't fundamentally broken for ironic today, but I hope to be able to later this week, or worst case in person next week18:00
jrollI can try to help you understand the current flow, if you have questions in mind18:00
efriedjroll: Yeah, if you don't mind.18:01
efriedI'm trying to understand how the current get_inventory is called.18:01
*** mgoddard_ has quit IRC18:02
efriedIn the ironic case, there's actually multiple ComputeNode s ?18:02
*** ralonsoh has quit IRC18:02
jrollcorrect - there is a ComputeNode per ironic node18:02
efriedAnd there's a loop over those, and RT does the update_compute_node stuff for each?18:02
jrollactually, there's an instance of the RT class created for each ComputeNode, IIRC18:03
efriedah, that would explain why I wasn't finding said loop.18:03
efriedBut ultimately what it means is that we are indeed creating a provider in placement for each ironic node, separately.18:03
jrollthat may have changed somewhere18:03
jrollcorrect18:03
efriedOkay, that helps a lot.  I need to go through this again and be more precise about using "host" vs "node", and probably reword the stuff in the aforementioned block about nodename.18:04
openstackgerritMerged openstack/os-vif master: Configure privsep binary  https://review.openstack.org/53135818:05
efriedThough TBH, I feel like we've reached the Pareto point with this spec...18:05
jrollefried: ah, it's still a singleton RT. but we call update_available_resource() for each node. this method is called per node: https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L724118:05
efriedjroll: Got it - here's the loop https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L7282 -- thanks.18:06
jrollefried: well, if each *node* may be a root, rather than each *host*, I think we'd be good to go (it would match what we're doing before nested RPs)18:06
jrollyep18:06
*** acormier_ has joined #openstack-nova18:06
efriedjroll: Yes, I didn't realize we were already handling separate providers per node.  So what I was saying earlier about things we don't handle was a lie (sean-k-mooney mriedem).18:07
*** slaweq_ has joined #openstack-nova18:07
*** hongbin has joined #openstack-nova18:07
jrollgotcha, cool. this seems workable then :)18:07
efriedAnd in fact without further hacking, ironic *gets* a separate root/tree per node - doesn't actually have a choice in the matter :)18:08
*** salv-orlando has joined #openstack-nova18:08
jrollyep!18:08
*** yamamoto has joined #openstack-nova18:08
*** acormier_ has quit IRC18:08
efriedBootstrap-wise, update_provider_tree will get just the root for the node on the initial call; it'll be up to the virt's impl of update_provider_tree if it wants to make child providers under that root or whatever.18:08
*** yamamoto has quit IRC18:08
jrollI think the code is good to go anyway - line 1405 here concerns me a bit https://review.openstack.org/#/c/533821/22/nova/scheduler/client/report.py18:09
*** acormier_ has joined #openstack-nova18:09
efriedYES18:10
*** acormier_ has joined #openstack-nova18:10
*** acormier has quit IRC18:10
efriedBecause I *think* that guy is actually going to contain *all* the trees for *all* the nodes.18:10
jrollyeah, either that or just the last tree operated on18:10
jrollold_tree = self._provider_trees[nodename] :)18:10
*** slaweq_ has quit IRC18:12
jrollbecause, we don't have an RP representing the compute host to be the root for all of those node RPs, they're all independent. if that makes esense.18:12
jrolls/esense/sense/18:12
efriedjroll: yeah.  I think that needs to be fixed in two or three places.18:13
jrollpossibly18:14
efriedI think it's a bigger problem18:14
*** acormier_ has quit IRC18:14
efriedor maybe, as you say, I do that filtering only here.18:14
efriedI'm going to have to look at this with fresh eyes, now that I have a better understanding of the flow.18:15
jrollawesome, thanks efried, this was helpful for my understanding too :)18:15
*** AlexeyAbashkin has joined #openstack-nova18:15
*** rmart04 has quit IRC18:16
efriedjroll: Your help is much appreciated too.  Let's have a Guinness next week.18:16
jroll++18:16
*** openstackgerrit has quit IRC18:18
*** jpena is now known as jpena|away18:19
*** AlexeyAbashkin has quit IRC18:20
*** openstackgerrit has joined #openstack-nova18:24
openstackgerritsean mooney proposed openstack/nova master: Add Neutron port capabilities to devspec in request  https://review.openstack.org/45177718:24
openstackgerritsean mooney proposed openstack/nova master: Format NIC features using os-traits definitions  https://review.openstack.org/46605118:24
openstackgerritsean mooney proposed openstack/nova master: Read Neutron port 'binding_profile' during boot  https://review.openstack.org/50748118:24
mriedemthis pretty simple bug fix has been hanging out with a +2 for a few months now https://review.openstack.org/#/c/519464/ - simply unquiesce an instance if we quiesced it but creating volume snapshots fails18:27
openstackgerritsean mooney proposed openstack/nova-specs master: Reintroduced nic feature based scheduling for rocky  https://review.openstack.org/54595118:30
*** tesseract has quit IRC18:30
*** r-daneel has joined #openstack-nova18:39
dansmithmriedem: got that and the two above it18:39
dansmithwhich seem like obvious should-be-doing-that items18:40
*** sdague has quit IRC18:40
openstackgerritMerged openstack/nova master: Imported Translations from Zanata  https://review.openstack.org/54156118:41
*** mgoddard_ has joined #openstack-nova18:44
*** moshele has joined #openstack-nova18:45
*** mvk_ has quit IRC18:48
*** cfriesen has quit IRC18:50
*** edmondsw has quit IRC18:55
*** edmondsw has joined #openstack-nova18:56
*** lucasagomes is now known as lucas-brb18:57
*** edmondsw has quit IRC19:00
*** felipemonteiro has joined #openstack-nova19:02
*** harlowja has joined #openstack-nova19:03
*** yamamoto has joined #openstack-nova19:09
*** dtantsur is now known as dtantsur|afk19:10
*** jafeha has joined #openstack-nova19:10
*** esberglu has quit IRC19:11
*** jafeha__ has quit IRC19:11
*** jpena|away is now known as jpena|off19:14
openstackgerritEric Fried proposed openstack/nova-specs master: Update Provider Tree  https://review.openstack.org/54011119:14
efriedmriedem, jaypipes, edleafe, cdent: I'm no stephenfin, but ^19:15
efriedjroll: ^19:15
* jroll looks19:15
*** slaweq_ has joined #openstack-nova19:16
edleafeefried: if only you included an ASCII diagram of CN2...19:18
efriededleafe: Eh?  Did I not?19:19
mriedemdansmith: thanks19:19
edleafeefried: I meant for the PT when UPT is invoked for CN219:20
efriedoh.  It's symmetrical.  Are you feing bunny?19:20
edleafeefried: just busting your chops, of course :)19:20
efriedphew19:20
efriedconsider my chops busted19:20
*** slaweq_ has quit IRC19:21
*** damien_r has joined #openstack-nova19:21
openstackgerritMerged openstack/nova stable/ocata: libvirt: bandwidth param should be set in guest migrate  https://review.openstack.org/51963519:24
*** alexchadin has joined #openstack-nova19:24
openstackgerritMerged openstack/python-novaclient master: Fix the docstring for the update method  https://review.openstack.org/54581919:24
*** mgoddard_ has quit IRC19:25
*** yamamoto has quit IRC19:26
*** alexchadin has quit IRC19:28
*** tssurya has joined #openstack-nova19:28
openstackgerritMatt Riedemann proposed openstack/nova master: Combine error handling blocks in _do_build_and_run_instance  https://review.openstack.org/54596019:29
*** esberglu has joined #openstack-nova19:29
*** damien_r has quit IRC19:31
openstackgerritMatt Riedemann proposed openstack/nova stable/queens: unquiesce instance on volume snapshot failure  https://review.openstack.org/54596119:31
*** alexchadin has joined #openstack-nova19:33
*** mriedem has quit IRC19:34
*** mriedem has joined #openstack-nova19:35
*** alexchadin has quit IRC19:36
*** moshele has quit IRC19:36
*** moshele has joined #openstack-nova19:38
*** moshele has quit IRC19:42
*** slaweq_ has joined #openstack-nova19:52
*** amodi has quit IRC19:52
*** mgoddard_ has joined #openstack-nova19:56
*** slaweq_ has quit IRC19:57
*** lucas-brb is now known as lucasagomes19:58
*** mvk_ has joined #openstack-nova19:59
*** hongbin_ has joined #openstack-nova20:01
*** hongbin has quit IRC20:03
openstackgerritMatt Riedemann proposed openstack/nova stable/pike: unquiesce instance on volume snapshot failure  https://review.openstack.org/54596620:03
*** alexchadin has joined #openstack-nova20:04
*** alexchadin has quit IRC20:04
*** amodi has joined #openstack-nova20:05
openstackgerritMerged openstack/nova stable/ocata: Use proper user and tenant in the owner section of libvirt.xml.  https://review.openstack.org/52599720:06
mriedemdansmith: want to get this simple fixture cleanup patch? https://review.openstack.org/#/c/539758/20:08
mriedemthat will unblock a few other approved changes20:08
mriedemthis is the series where bfv failing during scheduling always orphans your volumes20:08
mriedemwhich sucks20:08
mriedemFYI in case any cores want to shed 1300+ LOC https://review.openstack.org/#/c/544698/20:10
*** acormier has joined #openstack-nova20:11
*** dave-mccowan has quit IRC20:14
*** lucasagomes is now known as lucas-afk20:14
dansmithmriedem: ack20:16
*** moshele has joined #openstack-nova20:22
openstackgerritMerged openstack/os-vif master: zuul: Enable functional tests in gate  https://review.openstack.org/53096120:22
openstackgerritMerged openstack/nova master: unquiesce instance on volume snapshot failure  https://review.openstack.org/51946420:22
*** amoralej is now known as amoralej|off20:23
*** janki has quit IRC20:28
*** edmondsw has joined #openstack-nova20:30
openstackgerritMatt Riedemann proposed openstack/nova stable/ocata: unquiesce instance on volume snapshot failure  https://review.openstack.org/54597320:32
*** felipemonteiro_ has joined #openstack-nova20:32
*** awaugama has quit IRC20:33
*** felipemonteiro has quit IRC20:35
*** mgoddard has quit IRC20:35
*** itlinux has quit IRC20:36
*** moshele has quit IRC20:40
*** mgoddard has joined #openstack-nova20:42
*** tssurya has quit IRC20:43
*** tssurya has joined #openstack-nova20:47
*** tidwellr has joined #openstack-nova20:48
*** eharney has quit IRC20:49
*** acormier has quit IRC20:50
*** acormier has joined #openstack-nova20:51
*** mgoddard_ has quit IRC20:53
melwittdansmith, mriedem: on https://review.openstack.org/#/c/544698, "We dropped support for aggregates in newton" just means dropped support for the old "main db" aggregates, not dropped support for aggregates altogether, right?20:59
mriedemmelwitt: we just deleted the API20:59
mriedemit was an admin-only extension,20:59
mriedemwe said, fuck it20:59
dansmithmelwitt: heh, right, sorry. I meant we dropped support for this stuff I'm removing :D21:00
dansmithmelwitt: if you think it's important I can rev it21:00
melwittwhen I first read it I was like O.o21:00
melwittI'll just put a note to self on there21:01
mriedemdropped the migration compat code21:01
mriedemlike for flavors21:01
*** mvk_ has quit IRC21:01
melwittyeah. I see it in the patch, was just trying to connect the commit message in case there was something really major I was under a rock about21:01
*** slaweq_ has joined #openstack-nova21:07
*** ccamacho has quit IRC21:12
*** slaweq_ has quit IRC21:12
*** pchavva has quit IRC21:15
*** moshele has joined #openstack-nova21:18
*** awaugama has joined #openstack-nova21:21
mriedemdansmith: after reading the bug for https://review.openstack.org/#/c/543970/ can you make sure my comment is correct before I +W?21:23
*** andreas_s has joined #openstack-nova21:26
*** AlexeyAbashkin has joined #openstack-nova21:26
*** clayton has quit IRC21:28
*** hongbin has joined #openstack-nova21:29
*** moshele has quit IRC21:30
*** hongbin_ has quit IRC21:30
dansmithmriedem: replied for posterity, but in short: yep.21:30
*** AlexeyAbashkin has quit IRC21:31
*** andreas_s has quit IRC21:31
hrwsean-k-mooney: will be21:31
mriedemcool; i'll start the backport party21:31
*** tssurya has quit IRC21:34
openstackgerritMatt Riedemann proposed openstack/nova stable/queens: Lazy-load instance attributes with read_deleted=yes  https://review.openstack.org/54598721:35
*** rcernin has joined #openstack-nova21:36
*** threestrands has joined #openstack-nova21:37
openstackgerritMatt Riedemann proposed openstack/nova stable/pike: Lazy-load instance attributes with read_deleted=yes  https://review.openstack.org/54598821:37
*** vladikr has quit IRC21:37
openstackgerritMatt Riedemann proposed openstack/nova stable/ocata: Lazy-load instance attributes with read_deleted=yes  https://review.openstack.org/54598921:38
*** vladikr has joined #openstack-nova21:38
*** salv-orl_ has joined #openstack-nova21:42
hrwsean-k-mooney: added myself to nova etherpad21:43
*** salv-orlando has quit IRC21:44
*** felipemonteiro_ has quit IRC21:45
*** felipemonteiro_ has joined #openstack-nova21:45
mriedemnot sure why we'd even be lazy loading instance.system_metadata in that evacuate cleanup path, the db api should manually join it https://github.com/openstack/nova/blob/stable/pike/nova/db/sqlalchemy/api.py#L2141-L214321:45
*** tidwellr has quit IRC21:46
*** takashin has joined #openstack-nova21:48
openstackgerritMatt Riedemann proposed openstack/nova stable/ocata: libvirt: Don't VIR_MIGRATE_NON_SHARED_INC without migrate_disks  https://review.openstack.org/51963621:50
openstackgerritMerged openstack/nova master: Drop extra loop which modifies Cinder volume status  https://review.openstack.org/53975821:51
openstackgerritMerged openstack/nova master: Store block device mappings in cell0  https://review.openstack.org/54474821:51
openstackgerritMerged openstack/nova master: Add functional tests to ensure BDM removal on delete  https://review.openstack.org/54474721:52
dansmithmriedem: the compute drop passed grenade this time21:53
dansmithfailed something else, which I'm looking at but..21:53
*** felipemonteiro__ has joined #openstack-nova21:53
*** priteau has quit IRC21:53
dansmithhmm, rabbit crash maybe?21:54
dansmithhttp://logs.openstack.org/80/543580/1/check/legacy-tempest-dsvm-cells/e4a862c/logs/screen-n-cpu.txt.gz?level=TRACE#_Feb_19_20_06_46_94607421:54
*** priteau has joined #openstack-nova21:54
mriedemthere are crash reports in http://logs.openstack.org/80/543580/1/check/legacy-tempest-dsvm-cells/e4a862c/logs/rabbitmq/rabbit@ubuntu-xenial-inap-mtl01-0002617609-sasl.txt.gz21:56
dansmithah sweet21:56
*** felipemonteiro_ has quit IRC21:57
*** priteau has quit IRC21:59
mriedemmmedvede: can we get the pkvm CI to not vote at all on stable/ocata changes? it always fails immediately https://review.openstack.org/#/c/545973/22:05
*** lpetrut has quit IRC22:05
efriededleafe, melwitt, mriedem, jaypipes, cdent: Specless bp to add ?required=<trait list> to GET /resource_providers ?22:06
*** hamzy has quit IRC22:07
* efried forms full sentence...22:07
mriedemmmedvede: looks like the tests randomly timeout because it takes forever to spawn the actual guest22:07
mriedem2018-02-19 21:14:56.466 31787 INFO nova.compute.manager [req-21095306-b40d-4762-a083-8510184d386d tempest-ListServerFiltersTestJSON-391319940 tempest-ListServerFiltersTestJSON-391319940] [instance: 5b94a837-eb53-4275-b893-8e51b8d10518] Took 243.80 seconds to spawn the instance on the hypervisor.22:07
efriedWould "adding ?required=<trait list> to GET /resource_providers" be considered for a specless blueprint?22:07
edleafeefried: yeah, I think specless is ok. Just clarify in the BP that it will modeled on GET /allocation_candidates22:08
mriedemefried: no22:08
mriedemit's an api change22:08
melwittefried: I'd have expected an api change to be a spec, albeit a small one in that case22:08
efriedight22:08
edleafeoh geez, yeah, an API change. What they said.22:08
*** eharney has joined #openstack-nova22:09
*** slaweq_ has joined #openstack-nova22:13
jaypipesefried: I thought all API changes required a spec?22:14
efriedjaypipes: Apparently that's true; see above.22:14
* efried specs22:14
*** hamzy has joined #openstack-nova22:16
*** slaweq_ has quit IRC22:18
*** openstackgerrit has quit IRC22:18
*** priteau has joined #openstack-nova22:22
*** mvk_ has joined #openstack-nova22:22
*** masber has joined #openstack-nova22:23
*** elmaciej has joined #openstack-nova22:34
*** slaweq has quit IRC22:36
*** dave-mccowan has joined #openstack-nova22:37
*** acormier has quit IRC22:38
*** priteau has quit IRC22:38
*** acormier has joined #openstack-nova22:39
*** acormier_ has joined #openstack-nova22:41
*** openstackgerrit has joined #openstack-nova22:43
openstackgerritMerged openstack/nova master: Remove deprecated aggregate DB compatibility  https://review.openstack.org/54469822:43
*** acormier has quit IRC22:45
*** slaweq has joined #openstack-nova22:45
*** slaweq has quit IRC22:50
openstackgerritMerged openstack/nova master: Lazy-load instance attributes with read_deleted=yes  https://review.openstack.org/54397022:54
*** jafeha__ has joined #openstack-nova22:56
*** jafeha has quit IRC22:59
mriedemmdbooth: you should take a look at this https://review.openstack.org/#/c/542646/23:03
*** dave-mccowan has quit IRC23:05
*** AlexeyAbashkin has joined #openstack-nova23:06
*** liverpooler has quit IRC23:10
*** AlexeyAbashkin has quit IRC23:10
*** tetsuro has joined #openstack-nova23:13
tetsurois avolkov around?23:19
tetsuroI’d like you to answer the question in https://review.openstack.org/#/c/539865/. … or anyone who can help me.23:20
*** burt has quit IRC23:20
*** mlavalle has quit IRC23:24
*** felipemonteiro__ has quit IRC23:25
*** hongbin has quit IRC23:27
*** trinaths has quit IRC23:32
*** masber has quit IRC23:34
*** r-daneel has quit IRC23:36
*** masber has joined #openstack-nova23:37
*** masahisa has joined #openstack-nova23:39
mnaseris there any way i can do something to get output from nova-compute about memory usage23:47
mnaseri have a nova-compute process sitting on 13gb of ram.23:47
mnaser(before i restart it, just to get helpful information out)23:48
melwittmriedem: these comments aren't related to the patch, are they? just a suggestion on some better cleanup we could do in a separate patch? https://review.openstack.org/#/c/520158/3/nova/compute/api.py@282623:49
mnaserwow... actually almost every nova-compute process is using 13g of memory..23:50
*** amodi has quit IRC23:52
*** slaweq has joined #openstack-nova23:53
*** masber has quit IRC23:57
*** slaweq has quit IRC23:58
*** chyka has quit IRC23:58
*** chyka has joined #openstack-nova23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!