| *** tkajinam has quit IRC | 00:01 | |
| *** tkajinam has joined #openstack-nova | 00:02 | |
| *** LinPeiWen42 has joined #openstack-nova | 00:40 | |
| *** brinzhang has joined #openstack-nova | 00:47 | |
| openstackgerrit | melanie witt proposed openstack/nova master: WIP Enable unified limits in the nova-next job https://review.opendev.org/c/openstack/nova/+/789963 | 00:53 |
|---|---|---|
| *** martinkennelly has quit IRC | 01:01 | |
| *** whoami-rajat has quit IRC | 01:04 | |
| *** hemanth_n has joined #openstack-nova | 01:34 | |
| *** bbowen_ has quit IRC | 01:52 | |
| *** ebbex has quit IRC | 01:52 | |
| *** tamas_erdei has quit IRC | 01:52 | |
| *** mgoddard has quit IRC | 01:52 | |
| *** smcginnis has quit IRC | 01:52 | |
| *** purplerbot has quit IRC | 01:52 | |
| *** tristanC has quit IRC | 01:52 | |
| *** andreaf has quit IRC | 01:52 | |
| *** mloza has quit IRC | 01:52 | |
| *** Corwin has quit IRC | 01:53 | |
| *** bbowen_ has joined #openstack-nova | 01:54 | |
| *** ebbex has joined #openstack-nova | 01:54 | |
| *** tamas_erdei has joined #openstack-nova | 01:54 | |
| *** mgoddard has joined #openstack-nova | 01:54 | |
| *** smcginnis has joined #openstack-nova | 01:54 | |
| *** purplerbot has joined #openstack-nova | 01:54 | |
| *** tristanC has joined #openstack-nova | 01:54 | |
| *** andreaf has joined #openstack-nova | 01:54 | |
| *** mloza has joined #openstack-nova | 01:54 | |
| *** Corwin has joined #openstack-nova | 01:54 | |
| *** markmcclain has quit IRC | 01:55 | |
| *** markmcclain has joined #openstack-nova | 01:57 | |
| *** brinzhang_ has joined #openstack-nova | 02:01 | |
| *** brinzhang has quit IRC | 02:05 | |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Replace os-simple-tenant-usage with os-simple-project-usage https://review.opendev.org/c/openstack/nova/+/768852 | 02:34 |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Replace tenant_id with project_id in os-quota-sets path https://review.opendev.org/c/openstack/nova/+/768851 | 02:34 |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Replace tenant_id with project_id in Limits API https://review.opendev.org/c/openstack/nova/+/768862 | 02:34 |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Replace tenant* with project* in codes https://review.opendev.org/c/openstack/nova/+/769329 | 02:34 |
| *** sapd1_x has joined #openstack-nova | 02:42 | |
| *** macz_ has joined #openstack-nova | 02:43 | |
| openstackgerrit | David Vallee Delisle proposed openstack/os-vif master: Creating oslo.config.opts entry_points for plugins https://review.opendev.org/c/openstack/os-vif/+/789645 | 02:44 |
| *** sapd1_x has quit IRC | 02:46 | |
| *** macz_ has quit IRC | 02:47 | |
| *** sapd1 has joined #openstack-nova | 03:22 | |
| *** openstackgerrit has quit IRC | 03:31 | |
| *** psachin has joined #openstack-nova | 03:37 | |
| *** mkrai has joined #openstack-nova | 03:46 | |
| *** sorrison has joined #openstack-nova | 03:54 | |
| *** mkrai has quit IRC | 04:18 | |
| *** ratailor has joined #openstack-nova | 04:32 | |
| *** rcernin has quit IRC | 04:35 | |
| *** mkrai has joined #openstack-nova | 04:38 | |
| *** rcernin has joined #openstack-nova | 04:38 | |
| *** mkrai has quit IRC | 05:41 | |
| *** mkrai has joined #openstack-nova | 05:47 | |
| *** zzzeek has quit IRC | 05:52 | |
| *** sapd1 has quit IRC | 05:52 | |
| *** zzzeek has joined #openstack-nova | 05:54 | |
| *** vishalmanchanda has joined #openstack-nova | 05:57 | |
| *** openstackgerrit has joined #openstack-nova | 05:59 | |
| openstackgerrit | Balazs Gibizer proposed openstack/placement master: Adapt to SQLAlchemy 1.4 https://review.opendev.org/c/openstack/placement/+/789921 | 05:59 |
| openstackgerrit | Balazs Gibizer proposed openstack/placement master: DNM: test with SQLAlchemy 1.4 https://review.opendev.org/c/openstack/placement/+/789922 | 05:59 |
| *** slaweq has joined #openstack-nova | 06:15 | |
| *** dklyle has quit IRC | 06:18 | |
| *** ralonsoh has joined #openstack-nova | 06:31 | |
| *** whoami-rajat has joined #openstack-nova | 06:35 | |
| *** mkrai has quit IRC | 06:50 | |
| *** _mlavalle_1 has joined #openstack-nova | 07:12 | |
| *** andrewbonney has joined #openstack-nova | 07:13 | |
| *** rpittau|afk is now known as rpittau | 07:14 | |
| *** belmoreira has joined #openstack-nova | 07:15 | |
| *** mlavalle has quit IRC | 07:15 | |
| *** ociuhandu has joined #openstack-nova | 07:17 | |
| *** sapd1 has joined #openstack-nova | 07:22 | |
| *** rcernin has quit IRC | 07:36 | |
| openstackgerrit | Balazs Gibizer proposed openstack/placement master: Adapt to SQLAlchemy 1.4 https://review.opendev.org/c/openstack/placement/+/789921 | 07:36 |
| openstackgerrit | Balazs Gibizer proposed openstack/placement master: DNM: test with SQLAlchemy 1.4 https://review.opendev.org/c/openstack/placement/+/789922 | 07:37 |
| openstackgerrit | melanie witt proposed openstack/nova master: WIP Enable unified limits in the nova-next job https://review.opendev.org/c/openstack/nova/+/789963 | 07:43 |
| *** tosky has joined #openstack-nova | 07:47 | |
| openstackgerrit | melanie witt proposed openstack/nova master: WIP Enable unified limits in the nova-next job https://review.opendev.org/c/openstack/nova/+/789963 | 07:48 |
| *** ociuhandu has quit IRC | 07:50 | |
| openstackgerrit | melanie witt proposed openstack/nova master: WIP Enable unified limits in the nova-next job https://review.opendev.org/c/openstack/nova/+/789963 | 07:54 |
| *** dtantsur|afk is now known as dtantsur | 07:56 | |
| *** derekh has joined #openstack-nova | 08:00 | |
| *** sapd1 has quit IRC | 08:01 | |
| *** rcernin has joined #openstack-nova | 08:01 | |
| *** sapd1 has joined #openstack-nova | 08:04 | |
| *** rcernin has quit IRC | 08:07 | |
| *** lucasagomes has joined #openstack-nova | 08:09 | |
| *** hemanth_n has quit IRC | 08:10 | |
| *** rcernin has joined #openstack-nova | 08:20 | |
| *** nightmare_unreal has joined #openstack-nova | 08:21 | |
| *** ociuhandu has joined #openstack-nova | 08:21 | |
| *** rcernin has quit IRC | 08:26 | |
| *** ociuhandu has quit IRC | 08:29 | |
| *** rcernin has joined #openstack-nova | 08:34 | |
| *** mkrai has joined #openstack-nova | 08:37 | |
| *** martinkennelly has joined #openstack-nova | 08:37 | |
| *** ociuhandu has joined #openstack-nova | 08:58 | |
| *** ociuhandu_ has joined #openstack-nova | 09:01 | |
| openstackgerrit | Takashi Natsume proposed openstack/python-novaclient master: Refactor constructing request body https://review.opendev.org/c/openstack/python-novaclient/+/790017 | 09:01 |
| *** ociuhandu has quit IRC | 09:04 | |
| *** sapd1 has quit IRC | 09:23 | |
| *** k_mouza has joined #openstack-nova | 09:31 | |
| *** rcernin has quit IRC | 09:32 | |
| *** ociuhandu_ has quit IRC | 09:36 | |
| *** ociuhandu has joined #openstack-nova | 09:37 | |
| *** k_mouza has quit IRC | 10:00 | |
| *** k_mouza has joined #openstack-nova | 10:06 | |
| sean-k-mooney | gibi: do you remember this http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011044.html | 10:08 |
| gibi | sean-k-mooney: yes | 10:08 |
| sean-k-mooney | we might need to go impmente the api change and neutron code | 10:08 |
| gibi | I think we fixed it in neutron | 10:08 |
| sean-k-mooney | nope | 10:09 |
| sean-k-mooney | its still using socket.gethostname() | 10:09 |
| gibi | https://bugs.launchpad.net/neutron/+bug/1853840 | 10:09 |
| openstack | Launchpad bug 1853840 in neutron "Neutron fails to create bandwidth providers if CONF.host is set" [High,Fix released] - Assigned to Bence Romsics (bence-romsics) | 10:09 |
| gibi | there is configuration to override it | 10:09 |
| gibi | at least in neutron | 10:09 |
| sean-k-mooney | there is | 10:09 |
| sean-k-mooney | but you should not need to do that by default | 10:10 |
| sean-k-mooney | https://bugzilla.redhat.com/show_bug.cgi?id=1900500 and https://bugzilla.redhat.com/show_bug.cgi?id=1952073 | 10:11 |
| openstack | bugzilla.redhat.com bug 1900500 in openstack-neutron "Hostname in neutron agent's config don't match what is stored in placement by nova" [High,New] - Assigned to skaplons | 10:11 |
| openstack | bugzilla.redhat.com bug 1952073 in openstack-nova "[OSP16.1] Failed to schedule VMs with minimum bandwidth - SR-IOV bandwidth aware scheduling" [High,Closed: duplicate] - Assigned to nova-maint | 10:11 |
| sean-k-mooney | are basically caused by this | 10:11 |
| sean-k-mooney | they could use the config options | 10:11 |
| gibi | the fix was the config opiton | 10:11 |
| gibi | option | 10:11 |
| gibi | I don't get why they don't use it | 10:11 |
| gibi | why they cannot use it | 10:11 |
| sean-k-mooney | they could but really neutron should use the hypervior api no? | 10:12 |
| sean-k-mooney | gibi: they can totally workaround it using the config options | 10:13 |
| gibi | I assume this naming is static so it can be configured deployment time | 10:13 |
| sean-k-mooney | it could but the UX of that approch sucks | 10:13 |
| sean-k-mooney | it works but it would be nice if it just worked out of hte box | 10:14 |
| gibi | I don't get how this can be working without input from the deployer who know what are the matching host names | 10:14 |
| sean-k-mooney | by default socket.gethostname() will match the hostname returned by libvirt | 10:15 |
| gibi | if the neutron agent sees a different hostname than the nova-compute then how can the nova hypervisor API help in that | 10:15 |
| sean-k-mooney | gibi: we reqire the [DEFAULT]/host config option to be the same in both the nova and neutron config | 10:16 |
| sean-k-mooney | so you can find the compute service assocatied with the host | 10:16 |
| sean-k-mooney | then get its uuid | 10:17 |
| gibi | so the problematic case is when the hypervisor_hostname != hostname? But why that happen in case of libvirt? | 10:17 |
| sean-k-mooney | because of https://github.com/libvirt/libvirt/blob/master/src/util/virutil.c#L454-L473 | 10:17 |
| sean-k-mooney | if you have the fqdn as the canonical host name in /etc/hosts then libvirt will return an fqdn but socket.gethostname() will be the short hostname | 10:18 |
| *** mkrai has quit IRC | 10:18 | |
| gibi | and don't we require CONF.host to be set to fqdn is enough in this case? | 10:19 |
| gibi | sorry my english is breaking up :0 | 10:20 |
| gibi | so if in this case CONF.host is set to fqdn then everything is in synch isn't it? | 10:20 |
| sean-k-mooney | nope ooo already hardcodes that to the fqdn | 10:20 |
| gibi | don't we use that fqdn as the RP name ? | 10:21 |
| sean-k-mooney | no | 10:21 |
| sean-k-mooney | well somethimes | 10:21 |
| sean-k-mooney | we do not use the config value | 10:21 |
| sean-k-mooney | so we use what libvirt returns | 10:21 |
| sean-k-mooney | socket.gethostname will never be the fqdn | 10:22 |
| *** k_mouza has quit IRC | 10:22 | |
| sean-k-mooney | libvirt will return the fqdn if you have "172.17.1.17 compute-0.redhat.local compute-0" and short name if you have "172.17.1.17 compute-0 compute-0.redhat.locaL" | 10:22 |
| sean-k-mooney | gibi: one way to fix this is to try using the hostname and then the fqdn to lookup the rp | 10:23 |
| sean-k-mooney | gibi: we could also start using socket.gethostname in nova in the libvirt driver | 10:24 |
| sean-k-mooney | for hypervior hostname | 10:24 |
| *** rcernin has joined #openstack-nova | 10:24 | |
| *** rcernin has quit IRC | 10:25 | |
| *** rcernin has joined #openstack-nova | 10:25 | |
| sean-k-mooney | gibi: we can look at fixing this in ooo | 10:27 |
| sean-k-mooney | by having it generate the name that libvirt will return and put it in the neutorn config or we can do https://bugzilla.redhat.com/show_bug.cgi?id=1957363 | 10:27 |
| openstack | bugzilla.redhat.com bug 1957363 in tripleo-ansible "Canonical name should be a short name instead of FQDN" [High,New] - Assigned to rhos-maint | 10:27 |
| *** tesseract has joined #openstack-nova | 10:27 | |
| *** k_mouza has joined #openstack-nova | 10:28 | |
| gibi | I see that the name returned from libvirt is highly host configuration dependent. But I don't think we have to automate all the possible combinations to work, if there is at least one set of configuration that make the system work for every libvirt case | 10:28 |
| sean-k-mooney | well since this is not a problem with devstack as we see form the ci jobs it pretty clear that its actully caused by ooo's configuration of the host | 10:29 |
| sean-k-mooney | gibi: the defualt we have in code do work out of the box just not as deployed by ooo | 10:29 |
| gibi | does tripleo configures the hostname on the compute host ? | 10:30 |
| gibi | I mean the /etc/hosts and /etc/hostname | 10:30 |
| sean-k-mooney | yes | 10:30 |
| sean-k-mooney | both | 10:30 |
| sean-k-mooney | it templates out /etc/host with the fqdn first | 10:30 |
| sean-k-mooney | it alsso sets /etc/hostname | 10:30 |
| sean-k-mooney | one way to "fix" it woudl be to put the fqdn in /etc/hostname | 10:31 |
| sean-k-mooney | but that is kind of a hack | 10:31 |
| gibi | so then tripleo can make actually configures (indirectly) what hypervisor hostname libvirt returns | 10:31 |
| gibi | s/can make// | 10:31 |
| sean-k-mooney | yep | 10:31 |
| gibi | and also tripleo configures CONF.host for each services | 10:31 |
| sean-k-mooney | yep it does | 10:32 |
| gibi | so then tripleo needs to make these configrations consistent | 10:32 |
| gibi | at least in my eyes | 10:32 |
| sean-k-mooney | yep idealy | 10:32 |
| sean-k-mooney | but alos in a way that does not change hypervior hostname | 10:32 |
| sean-k-mooney | on upragde | 10:32 |
| sean-k-mooney | because that would break nova | 10:32 |
| sean-k-mooney | i think the current problem is i think that chnage may have already happened which i might need to check | 10:33 |
| gibi | so the problem is that the current config is not fully consistent and blocks the bandwidth feature, but making the config consistent would result in breaking nova as the we don't tolerate hostname changes | 10:33 |
| sean-k-mooney | well osp 13(queens) on rhel7 used the same config as as we used in osp 16(train) on rhel 8 | 10:35 |
| sean-k-mooney | so in queens we already had placement RPs but only nova used them | 10:35 |
| sean-k-mooney | i need to alk one of our downstream peopel what the hypervior hostname is on 13 and 16 and see if it has changed | 10:36 |
| sean-k-mooney | and proably compare the filed bettwen both | 10:36 |
| sean-k-mooney | gibi: at a minim ooo shoudl be setting the neutron hostname mapping automatically when using the bandwith feature | 10:37 |
| sean-k-mooney | gibi: there is a patch against netruon right now https://review.opendev.org/c/openstack/neutron/+/788893 | 10:38 |
| gibi | it is needed there as CONF.host is fqdn but libvirt returns short names? | 10:38 |
| sean-k-mooney | gibi: well i think that is actully the correct behaivor | 10:39 |
| sean-k-mooney | i.e. we have always talk about the hypervior_hostname as beign a hostname not an fqdn | 10:39 |
| gibi | If we all agree that hypervisor_hostname is always expected to be the shortname, and we require the CONF.host to always be fqdn, then neutron can simply get the shortname out from the fqdn to look up the RP | 10:41 |
| gibi | but if every config can be either short or fqdn in all combinations then we are lost | 10:41 |
| sean-k-mooney | well we do not retuire CONF.host to alwasy be the fqdn | 10:41 |
| sean-k-mooney | its normally the sort name too | 10:41 |
| gibi | I think we should write up the list of supported config combinations | 10:42 |
| gibi | and then fix what is broken | 10:42 |
| sean-k-mooney | right now technially both value can be short name or fqdn | 10:42 |
| sean-k-mooney | but hypervior_hostname was alwasy intended to be the sortname and host defualt to the short name | 10:42 |
| sean-k-mooney | we talked about this at the dublin ptg at some lenght as some operators wanted to use FQDNs for both | 10:43 |
| gibi | and libvirt uses fqdn is some cases already based on the comment you linked from the code | 10:43 |
| sean-k-mooney | gibi: yes only if you set the canonical hostname in /etc/hosts to an fqdn | 10:44 |
| gibi | so again, I think we first agree on what config fqdn/sortname combination OpenStack wants to supprot | 10:44 |
| gibi | then fix the system to work in these scenarios | 10:44 |
| *** k_mouza_ has joined #openstack-nova | 10:45 | |
| *** k_mouza has quit IRC | 10:45 | |
| sean-k-mooney | ya | 10:45 |
| sean-k-mooney | ya personally i would prefer to either onlyu use hostname or only use FQDNs | 10:47 |
| gibi | yepp, that would be clean and self consistent all the way | 10:48 |
| sean-k-mooney | i prefer hostnames but that is just a preference | 10:48 |
| sean-k-mooney | the delta in function ality is fqdns allows host-1.rack-1.mydomain and host-1.rack-2.mydomain to work | 10:48 |
| sean-k-mooney | where as hostnames only requires tehm all to be unique | 10:49 |
| gibi | yepp, I see the need for FQDNs in big deployments | 10:49 |
| gibi | or at least accept the need | 10:49 |
| sean-k-mooney | the only issue if we use FQDNs only is sometime we will still end up with hostnames | 10:50 |
| sean-k-mooney | i.e. if you dont have the FQDN in /etc/hosts | 10:50 |
| sean-k-mooney | well that and we told all the other service to use socket.gethostname() | 10:51 |
| *** sapd1 has joined #openstack-nova | 10:51 | |
| sean-k-mooney | i even changed cyborg to use gethostnaem form getfqdn https://github.com/openstack/cyborg/commit/ec0eab0ffb4fc18574dbd937f29545832d317e4c | 10:51 |
| sean-k-mooney | after that intial email thread | 10:51 |
| sean-k-mooney | gibi: part of me feels like we should add a fucntion to oslo.utils | 10:53 |
| sean-k-mooney | and then just have everyone use that | 10:53 |
| sean-k-mooney | and then in nova we can perhaps think about have a reshape or soemthing to normalise the RPs | 10:53 |
| gibi | I think you can rename without reshape | 10:54 |
| sean-k-mooney | the fact this currently depend on the virt driver you use is less then ideal. | 10:54 |
| sean-k-mooney | gibi: ya i think you are right | 10:54 |
| sean-k-mooney | the allocation dont need to move | 10:54 |
| gibi | yepp | 10:54 |
| sean-k-mooney | what we can do its encode the libvirt logic which is in the neutron patch in an oslo patch | 10:55 |
| sean-k-mooney | and then use that | 10:55 |
| sean-k-mooney | for libvirt at least it woudl not change behaivor and for other driver i suspect it wont either | 10:55 |
| gibi | I cannot comment on other drivers cases | 10:55 |
| sean-k-mooney | well ironic they use the ironic node uuid as the name right? | 10:56 |
| sean-k-mooney | not sure what hyperv or vmware does | 10:56 |
| gibi | yes. but I don't know if neutron ever needs that | 10:56 |
| gibi | that = ironic node name | 10:56 |
| sean-k-mooney | ya the neutron agent dont run on the ironic nodes | 10:56 |
| gibi | I have to jump to a call from the top of the hour | 10:57 |
| gibi | I linked this discussion to the neutron patch for reference | 10:57 |
| sean-k-mooney | no worries thanks for litening | 10:57 |
| sean-k-mooney | ill try an follow up downstream and see if we can come to can concreate conclutions | 10:58 |
| gibi | OK | 10:59 |
| gibi | sorry for being dense | 10:59 |
| sean-k-mooney | you werent being dense | 10:59 |
| sean-k-mooney | it helped | 10:59 |
| gibi | then I'm glad :) | 11:00 |
| *** ratailor_ has joined #openstack-nova | 11:14 | |
| *** ratailor has quit IRC | 11:16 | |
| *** sapd1 has quit IRC | 11:17 | |
| *** rcernin has quit IRC | 11:24 | |
| *** zzzeek has quit IRC | 11:39 | |
| *** zzzeek has joined #openstack-nova | 11:42 | |
| *** artom has quit IRC | 11:48 | |
| *** __ministry has quit IRC | 11:51 | |
| *** ociuhandu has quit IRC | 11:52 | |
| *** derekh has quit IRC | 12:17 | |
| *** ociuhandu has joined #openstack-nova | 12:18 | |
| sean-k-mooney | gibi: would adding a new weigher need a spec. or could that be a specless blueprint | 12:18 |
| gibi | if the config of the weigher is trivial then I'm OK with a specless bp | 12:18 |
| sean-k-mooney | i have 2 that i think might be useful. 1 is weighe based on the number of runing instnace and the other is a weigher that prefers the current host an instance is on | 12:19 |
| gibi | please note that today all the weigher is turned on by default so more weigher means a more complex behavior. | 12:20 |
| gibi | * all the weighers are | 12:21 |
| sean-k-mooney | yep | 12:21 |
| sean-k-mooney | we may or may not want that for these | 12:22 |
| sean-k-mooney | basically our perf team has a bunch of hosts with differnt size for ram disk and cpu | 12:22 |
| sean-k-mooney | and they are booting a bunch of small instance and they are all landing on 2 hosts because it has much more ram,cpu and disk then the others | 12:23 |
| gibi | I think it is a valid need to have weighers that are not turned on by default | 12:24 |
| sean-k-mooney | as in the have 46 hosts and booted about 520 instace at this point and they have 2 host with 237 and 247 instances | 12:24 |
| sean-k-mooney | so having a pretty dumb weither that jsut spread or packed by instances would help them. im goign to get them to set host_subset_size=46 so its just a random choice for now | 12:25 |
| sean-k-mooney | that will get the the even distibution they want but i think it coudl be useful | 12:26 |
| sean-k-mooney | the current host weigher would be so that people that want resize to same host to not change host could expess that preference if they really wanted too by truning on that weither | 12:27 |
| *** ociuhandu has quit IRC | 12:27 | |
| sean-k-mooney | gibi: im going to proably right a mini spec for moveing some of the fitlers to placmenet native approch so i could also add this there too | 12:27 |
| gibi | hm, I think if allow_resize_to_same_host=False then we add the scheduler will ignore the current host | 12:28 |
| sean-k-mooney | just make it a generic schduling enhancements spec. | 12:28 |
| gibi | sean-k-mooney: that also works for me | 12:28 |
| sean-k-mooney | gibi: it will the host manager will remove it | 12:28 |
| sean-k-mooney | but if you set it to ture there is no prefernce for stayingon the same host or not staying on the same host | 12:28 |
| gibi | ohh, I did not know that, then I see why you need the weigher | 12:29 |
| sean-k-mooney | well need is a stong word but it might be nice ot have | 12:29 |
| gibi | yeah, I agree | 12:30 |
| sean-k-mooney | we ocationally get customer confused by that, its not that common but it happens | 12:30 |
| *** ociuhandu has joined #openstack-nova | 12:41 | |
| *** rmart04 has joined #openstack-nova | 12:42 | |
| rmart04 | Hello, having some IRC issues, hopefully this works! | 12:42 |
| rmart04 | I've noticed that in VM's on Stein / host-passthrough cpu mode and dedicated core pinning policy, that the L2 and L3 cache values for the CPU do not match the physical CPU. Is this expected? Any way to configure? | 12:43 |
| gibi | belmoreira: thanks for the report https://bugs.launchpad.net/nova/+bug/1924123 it is an interesting finding. Unfortunately it is not easy to fix. | 12:47 |
| openstack | Launchpad bug 1924123 in OpenStack Compute (nova) "If source compute node is overcommitted instances can't be migrated" [Medium,Triaged] | 12:47 |
| *** ratailor_ has quit IRC | 12:47 | |
| sean-k-mooney | rmart04: yes | 12:50 |
| sean-k-mooney | you can also configure passthough of the host cache info but we dont do that in nova today | 12:50 |
| rmart04 | Ah, ok so possible in Libvirt/KVM but not exposed to nova? | 12:53 |
| sean-k-mooney | correct | 12:53 |
| sean-k-mooney | https://libvirt.org/formatdomain.html#cpu-model-and-topology | 12:54 |
| sean-k-mooney | you can add <cache mode='passthrough'/> | 12:54 |
| sean-k-mooney | to the cpu element | 12:54 |
| *** ociuhandu has quit IRC | 12:54 | |
| rmart04 | Interesting, is there anyway to add config like this that will persist? Or is it a case of modifying the XML and rebooting the VM in nova and when its trashed, start again? | 12:55 |
| sean-k-mooney | not without modifying the nova code to add it no | 12:55 |
| rmart04 | OK | 12:55 |
| rmart04 | thanks for the information!! | 12:55 |
| sean-k-mooney | we regenerate teh xml every time we move the vm or reboot it | 12:55 |
| rmart04 | spot on as always | 12:55 |
| sean-k-mooney | this is proably something we could add as a feature but its never really come up | 12:56 |
| *** ociuhandu has joined #openstack-nova | 13:04 | |
| *** dustinc has joined #openstack-nova | 13:07 | |
| *** tesseract has quit IRC | 13:13 | |
| *** rmart04 has quit IRC | 13:14 | |
| *** sapd1 has joined #openstack-nova | 13:16 | |
| *** dave-mccowan has joined #openstack-nova | 14:05 | |
| *** lemko is now known as lemko1 | 14:14 | |
| *** lemko1 is now known as lemko | 14:14 | |
| *** lemko is now known as lemko_1 | 14:15 | |
| *** lemko_1 is now known as lemko | 14:15 | |
| openstackgerrit | Tobias Urdin proposed openstack/nova master: When Nova-Api runs behind the load balancer or Reverse proxy, Loadbalancer IP is getting logged in nova_api.log instead of end user source ip by RequestLog https://review.opendev.org/c/openstack/nova/+/786766 | 14:19 |
| *** artom has joined #openstack-nova | 14:25 | |
| *** dklyle has joined #openstack-nova | 14:56 | |
| *** vishalmanchanda has quit IRC | 14:56 | |
| *** rcernin has joined #openstack-nova | 15:20 | |
| *** rcernin has quit IRC | 15:25 | |
| *** gyee has joined #openstack-nova | 15:26 | |
| *** psachin has quit IRC | 15:27 | |
| *** macz_ has joined #openstack-nova | 15:35 | |
| *** macz_ has quit IRC | 15:35 | |
| *** macz_ has joined #openstack-nova | 15:35 | |
| *** rpittau is now known as rpittau|afk | 15:42 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova-specs master: QoS minimum guaranteed packet rate https://review.opendev.org/c/openstack/nova-specs/+/785014 | 15:46 |
| *** ociuhandu_ has joined #openstack-nova | 15:48 | |
| *** ociuhandu has quit IRC | 15:51 | |
| *** ociuhandu_ has quit IRC | 15:52 | |
| gibi | nova meeting starts in 4 minutes in #openstack-meeting-3 | 15:56 |
| *** lucasagomes has quit IRC | 16:01 | |
| *** dtantsur is now known as dtantsur|afk | 16:02 | |
| *** ociuhandu has joined #openstack-nova | 16:08 | |
| *** ociuhandu has quit IRC | 16:12 | |
| *** mgoddard has quit IRC | 16:13 | |
| *** mgoddard has joined #openstack-nova | 16:15 | |
| *** k_mouza_ has quit IRC | 16:30 | |
| *** k_mouza has joined #openstack-nova | 16:30 | |
| sean-k-mooney | dansmith: artom how we are fixing this for them is having them change all the config value to lowercase to match what neturon is expecting. and then doing a db update of all the things | 16:51 |
| dansmith | yeah, I like the hardening of course, but in the interim we have to provide them a way to migrate and fix, since we don't control rabbit | 16:52 |
| dansmith | so either we say "don't change it once you've set it" (which seems reasonable to me) or we say we're going to change everybody, so get your homework done (also reasonable) | 16:53 |
| dansmith | if neutron is making some change that is going to require consistency then presumably the latter is the only option | 16:53 |
| *** ralonsoh has quit IRC | 16:58 | |
| bauzas | I don't have a particular opinion on how we should fix those RPC issues, but I'm frightened by us trying to change the queue names | 16:59 |
| bauzas | as it's a impactful and visible change | 17:00 |
| bauzas | anyhow, time to stop | 17:01 |
| artom | My brain has mostly checked out at this point (I know, it's only 13:00 here), and I need to run errands anyways | 17:01 |
| artom | I'll post a spec... I want to say next week? | 17:01 |
| sean-k-mooney | well we can have this just be a workaround config option and never enable it by default | 17:03 |
| openstackgerrit | Elod Illes proposed openstack/placement master: Add periodic-stable-jobs template https://review.opendev.org/c/openstack/placement/+/775384 | 17:12 |
| *** andrewbonney has quit IRC | 17:13 | |
| *** macz_ has quit IRC | 17:15 | |
| dansmith | sean-k-mooney: artom: We could probably lower() the hostname on service.create() and not service.update() in the short term | 17:15 |
| dansmith | that won't fix the cell mapping issue, but sounds like that's not the primary concern (or the biggest impact) | 17:15 |
| artom | So what's the call mapping issue again? | 17:18 |
| sean-k-mooney | dansmith: am maybe. that word normalise any new service yes | 17:21 |
| dansmith | sean-k-mooney: we could do the same for cell mappings actually | 17:21 |
| sean-k-mooney | dansmith: althougy we would have to ensure that compute agents used the host value form the service record not form the config when settin up rpc | 17:21 |
| dansmith | sean-k-mooney: yeah | 17:22 |
| sean-k-mooney | im not actully sure where that happens i was expecting to see it here but here https://github.com/openstack/nova/blob/master/nova/cmd/compute.py but i guess its elsewhere | 17:23 |
| sean-k-mooney | proably in the compute manager | 17:24 |
| sean-k-mooney | ah | 17:26 |
| dansmith | it's probably buried in rpc.py | 17:26 |
| sean-k-mooney | https://github.com/openstack/nova/blob/7953c0197d1a4466cb5b78070d47626c92f9db6e/nova/compute/rpcapi.py#L53 | 17:26 |
| sean-k-mooney | proably but that is where we get the name | 17:26 |
| sean-k-mooney | we are using instance.host | 17:26 |
| dansmith | yeah, which might also bring some problems updating those as well | 17:28 |
| dansmith | unless we use instance.host to look up service.hostname case insensitively and then use the result | 17:28 |
| sean-k-mooney | we are usign instace.host for the server filed in the client | 17:31 |
| sean-k-mooney | so if we look at the srever side of the rpc it has to match | 17:32 |
| sean-k-mooney | which is likly using the host value form the config | 17:32 |
| sean-k-mooney | https://github.com/openstack/nova/blob/7953c0197d1a4466cb5b78070d47626c92f9db6e/nova/service.py#L184-L194 | 17:33 |
| *** macz_ has joined #openstack-nova | 17:34 | |
| sean-k-mooney | so ya its usign self.host which shoudl be form the conf | 17:34 |
| dansmith | right, | 17:34 |
| dansmith | I'm saying that (a) instance.host has to be kept to match service.hostname, so we can't change one without the other | 17:34 |
| sean-k-mooney | oh yes | 17:35 |
| dansmith | and (b) we could make service.hostname lowercase on create and make sure that instance.host comes from service.hostname going forward | 17:35 |
| dansmith | but we sill have the problem of fixing current people or just telling them "don't do that" | 17:35 |
| sean-k-mooney | yes which i think it likely is | 17:35 |
| sean-k-mooney | ya i dont know if this is something we woudl want to do with a nova manage command or something | 17:36 |
| sean-k-mooney | like "normalise_host" | 17:36 |
| sean-k-mooney | with a nova status check | 17:37 |
| dansmith | that'll be expensive for cern-level people | 17:37 |
| sean-k-mooney | yep | 17:37 |
| sean-k-mooney | if the out come of this is just we wont fix it that is also ok i guess but im concened by our db scema being case inseitive today | 17:37 |
| *** hamalq has joined #openstack-nova | 17:38 | |
| sean-k-mooney | i dont know if we can normalise on loading form the db? | 17:38 |
| dansmith | well, we can't without requiring the cloud to be quiesced I think | 17:39 |
| *** hamalq has quit IRC | 17:39 | |
| dansmith | you'd have to make sure that they all start normalizing at the same time I think, | 17:39 |
| dansmith | or start doing a lot of extra db queries to check for consistency to decide which value to use | 17:39 |
| *** hamalq has joined #openstack-nova | 17:40 | |
| sean-k-mooney | well or a one time set this config value on all hosts now operation | 17:40 |
| sean-k-mooney | but ya its not a simple solution in anycase | 17:40 |
| sean-k-mooney | i wonder if we could have the service listen on 2 queues | 17:42 |
| sean-k-mooney | the lowercase one and whatever is in the file | 17:42 |
| sean-k-mooney | ,,, | 17:42 |
| *** vishalmanchanda has joined #openstack-nova | 17:43 | |
| sean-k-mooney | i was going to say then just start sendign to the lower case one if you trun it on in the contoler config | 17:43 |
| dansmith | that would be a super complicated and expensive alternative to "don't rename hosts" :) | 17:43 |
| sean-k-mooney | yep | 17:43 |
| melwitt | or "don't use UpPerCAse" | 17:43 |
| sean-k-mooney | ya se the thing is hostnames ans fqdns for that matter are technicaly case senseitive | 17:44 |
| sean-k-mooney | but often that is ignored | 17:44 |
| clarkb | I don't think dns treats them that way | 17:45 |
| clarkb | DNS record types, and the values of TXT and SPF records are case sensitive, but everything else is not | 17:46 |
| sean-k-mooney | good to know | 17:47 |
| sean-k-mooney | i may have been mixing that up with email since the case matters there or at least it used too | 17:47 |
| sean-k-mooney | although no one follows the spec when it comes to email | 17:48 |
| *** k_mouza has quit IRC | 17:51 | |
| sean-k-mooney | ok im going to call it a day o/ | 17:54 |
| dansmith | right, what clarkb said | 17:57 |
| *** belmoreira has quit IRC | 18:01 | |
| openstackgerrit | melanie witt proposed openstack/nova master: Add test coverage for requestlog X-Forwarded-For https://review.opendev.org/c/openstack/nova/+/790139 | 18:25 |
| *** zul_ has joined #openstack-nova | 18:56 | |
| *** rcernin has joined #openstack-nova | 19:21 | |
| *** rcernin has quit IRC | 19:25 | |
| *** sapd1 has quit IRC | 19:32 | |
| *** ociuhandu has joined #openstack-nova | 19:36 | |
| *** ociuhandu has quit IRC | 19:40 | |
| *** k_mouza has joined #openstack-nova | 19:53 | |
| *** k_mouza has quit IRC | 19:57 | |
| *** slaweq has quit IRC | 20:38 | |
| *** k_mouza has joined #openstack-nova | 20:49 | |
| *** k_mouza has quit IRC | 20:55 | |
| *** whoami-rajat has quit IRC | 21:04 | |
| *** hoonetorg has joined #openstack-nova | 21:05 | |
| *** hoonetorg has quit IRC | 21:05 | |
| *** artom has quit IRC | 21:07 | |
| *** vishalmanchanda has quit IRC | 21:12 | |
| *** k_mouza has joined #openstack-nova | 21:14 | |
| *** artom has joined #openstack-nova | 21:16 | |
| *** nightmare_unreal has quit IRC | 21:36 | |
| *** rcernin has joined #openstack-nova | 21:36 | |
| *** zul_ has quit IRC | 21:55 | |
| *** rcernin has quit IRC | 21:58 | |
| *** k_mouza has quit IRC | 22:42 | |
| *** damien_r has quit IRC | 22:43 | |
| *** damien_r has joined #openstack-nova | 22:46 | |
| openstackgerrit | melanie witt proposed openstack/nova master: WIP Enable unified limits in the nova-next job https://review.opendev.org/c/openstack/nova/+/789963 | 22:49 |
| *** rcernin has joined #openstack-nova | 23:02 | |
| *** macz_ has quit IRC | 23:14 | |
| *** tosky has quit IRC | 23:17 | |
| *** dustinc has quit IRC | 23:34 | |
| *** hamalq has quit IRC | 23:42 | |
| *** k_mouza has joined #openstack-nova | 23:59 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!