Monday, 2020-11-09

*** bbowen has joined #openstack-nova00:01
*** macz_ has joined #openstack-nova00:39
*** brinzhang has joined #openstack-nova00:41
*** tosky has quit IRC00:41
*** macz_ has quit IRC00:44
*** ociuhandu has joined #openstack-nova00:48
*** ociuhandu has quit IRC00:52
*** spatel has joined #openstack-nova00:54
*** spatel has quit IRC01:23
*** sapd1 has joined #openstack-nova01:28
*** macz_ has joined #openstack-nova01:30
*** Liang__ has joined #openstack-nova01:34
*** macz_ has quit IRC01:34
*** LinPeiWen has joined #openstack-nova01:38
*** deke997 has joined #openstack-nova01:58
deke997anyone here know the latest status of spice integration?02:05
*** sapd1 has quit IRC02:18
*** tbachman has joined #openstack-nova02:20
*** deke997 has quit IRC03:15
*** mtreinish has quit IRC03:15
*** k_mouza has joined #openstack-nova03:20
*** k_mouza has quit IRC03:24
*** dklyle has joined #openstack-nova03:24
*** psachin has joined #openstack-nova03:29
*** LinPeiWen has quit IRC03:38
*** slaweq has quit IRC03:40
*** benj_ has quit IRC03:41
*** slaweq has joined #openstack-nova03:42
*** benj_ has joined #openstack-nova03:45
*** xinranwang has joined #openstack-nova03:52
*** macz_ has joined #openstack-nova03:54
*** amotoki has quit IRC03:56
*** amotoki has joined #openstack-nova03:57
*** macz_ has quit IRC03:58
*** Liang__ has quit IRC03:59
*** dklyle has quit IRC04:00
*** Liang__ has joined #openstack-nova04:01
*** mkrai has joined #openstack-nova04:19
*** LinPeiWen has joined #openstack-nova04:26
*** LinPeiWen has quit IRC05:16
*** ratailor has joined #openstack-nova05:17
*** psachin has quit IRC05:23
*** vishalmanchanda has joined #openstack-nova05:28
*** evrardjp has quit IRC05:33
*** evrardjp has joined #openstack-nova05:33
xinranwanggibi: Hi gibi, thanks for your comments for smartnic spec.  For bandiwidth qos, can I assume that the co-existance  is not support at the first step.05:48
xinranwangmaybe we can add this restirction into document for the first phase05:51
xinranwangIs it acceptable?05:52
*** zzzeek has quit IRC05:58
*** zzzeek has joined #openstack-nova06:00
*** rcernin has quit IRC06:48
*** rcernin has joined #openstack-nova06:49
*** LinPeiWen has joined #openstack-nova06:50
*** whoami-rajat__ has joined #openstack-nova06:50
*** viks____ has joined #openstack-nova06:57
*** lpetrut has joined #openstack-nova07:11
*** LinPeiWen has quit IRC07:17
*** sapd1 has joined #openstack-nova07:19
*** rcernin has quit IRC07:38
*** ccstone has quit IRC07:40
*** eandersson has quit IRC07:40
*** ccstone has joined #openstack-nova07:41
*** eandersson has joined #openstack-nova07:41
gibixinranwang: yes, such limitation is acceptable for me07:50
*** iurygregory has quit IRC07:58
xinranwanggibi:  great, thanks. I will update the spec soon.:)07:59
gibiI guess you mean coexistens for the same physical device.08:00
*** LinPeiWen has joined #openstack-nova08:08
*** tesseract has joined #openstack-nova08:08
*** andrewbonney has joined #openstack-nova08:10
*** rpittau|afk is now known as rpittau08:30
*** iurygregory has joined #openstack-nova08:30
*** psachin has joined #openstack-nova08:32
*** sapd1 has quit IRC08:33
*** psachin has quit IRC08:37
*** vishalmanchanda has quit IRC08:37
*** Yumeng has joined #openstack-nova08:42
*** martinkennelly has joined #openstack-nova08:53
openstackgerritLee Yarwood proposed openstack/nova master: zuul: Reintroduce nova-dsvm-multinode-base
bauzasgood morning Nova09:05
*** ociuhandu has joined #openstack-nova09:08
gibibauzas: o/09:12
*** ociuhandu has quit IRC09:18
*** Liang__ has quit IRC09:24
*** vishalmanchanda has joined #openstack-nova09:25
*** Liang__ has joined #openstack-nova09:25
*** ociuhandu has joined #openstack-nova09:26
*** ociuhandu has quit IRC09:28
*** ociuhandu has joined #openstack-nova09:28
*** Yumeng has quit IRC09:29
*** ralonsoh has joined #openstack-nova09:33
*** iurygregory has quit IRC09:35
*** iurygregory has joined #openstack-nova09:36
*** derekh has joined #openstack-nova09:37
*** xek has joined #openstack-nova09:42
*** Liang__ has quit IRC09:44
*** tesseract has quit IRC09:54
*** tesseract has joined #openstack-nova09:57
*** xek has quit IRC09:58
*** xek has joined #openstack-nova09:58
*** psachin has joined #openstack-nova10:02
*** k_mouza has joined #openstack-nova10:12
openstackgerritLee Yarwood proposed openstack/nova master: rbd: Only log import failures when the RbdDriver is used
lyarwoodstephenfin: ^ hopefully fixed btw10:26
*** ociuhandu has quit IRC10:28
lyarwoodgibi / stephenfin / bauzas ; could use reviews today if you have time, for whatever reason zuul didn't raise an error with the original change, it just removes the job silently. I'll file a bug later for this.10:29
stephenfinalready done :)10:30
gibiI will look shortly10:30
*** iurygregory_ has joined #openstack-nova10:32
*** sapd1 has joined #openstack-nova10:34
*** iurygregory has quit IRC10:35
*** ociuhandu has joined #openstack-nova10:36
*** ociuhandu has quit IRC10:37
*** ociuhandu has joined #openstack-nova10:37
*** jangutter has joined #openstack-nova10:38
openstackgerritLee Yarwood proposed openstack/nova master: Add os-volume_attachments reference docs
lyarwoodstephenfin: ^ some basic ref docs for volume attachment btw if you're interested, came up as part of a spec.10:41
lyarwoodplan on adding lots of additional detail later in the cycle but the locking part is a good enough start at the moment10:41
stephenfinsure thing, I can take a look10:42
*** jangutter_ has joined #openstack-nova10:55
*** xek_ has joined #openstack-nova10:56
*** jangutter has quit IRC10:58
*** xek has quit IRC10:59
*** tosky has joined #openstack-nova11:09
*** sapd1 has quit IRC11:12
*** mtreinish has joined #openstack-nova11:14
*** ociuhandu has quit IRC11:17
*** ratailor_ has joined #openstack-nova11:20
*** ociuhandu has joined #openstack-nova11:21
*** ratailor_ has quit IRC11:22
*** ratailor has quit IRC11:22
*** ratailor_ has joined #openstack-nova11:23
*** sapd1 has joined #openstack-nova11:32
*** sapd1 has quit IRC11:46
*** ratailor_ has quit IRC11:50
*** Luzi has joined #openstack-nova11:56
*** xek__ has joined #openstack-nova11:59
*** xek_ has quit IRC12:02
*** ociuhandu has quit IRC12:04
*** xek_ has joined #openstack-nova12:06
*** xek__ has quit IRC12:08
*** JamesBenson has joined #openstack-nova12:10
*** ociuhandu has joined #openstack-nova12:13
*** mkrai has quit IRC12:18
*** ociuhandu has quit IRC12:18
*** ociuhandu has joined #openstack-nova12:25
*** raildo has joined #openstack-nova12:28
*** ociuhandu has quit IRC12:29
*** ociuhandu has joined #openstack-nova12:30
*** dtantsur|afk is now known as dtantsur12:32
*** ociuhandu has quit IRC12:35
openstackgerritMerged openstack/nova master: zuul: Reintroduce nova-dsvm-multinode-base
*** psachin has quit IRC12:53
*** spatel has joined #openstack-nova12:55
*** spatel has quit IRC12:59
*** iurygregory_ is now known as iurygregory13:01
*** priteau has joined #openstack-nova13:18
*** Luzi has quit IRC13:25
*** ociuhandu has joined #openstack-nova13:29
*** eharney_ has quit IRC13:45
*** brinzhang_ has joined #openstack-nova13:49
*** eharney has joined #openstack-nova13:50
*** eharney has quit IRC13:50
*** eharney has joined #openstack-nova13:51
openstackgerritBalazs Gibizer proposed openstack/nova stable/victoria: Warn when starting services with older than N-1 computes
openstackgerritBalazs Gibizer proposed openstack/nova stable/victoria: Add upgrade check about old computes
*** k_mouza has quit IRC14:06
*** ociuhandu has quit IRC14:06
*** nweinber has joined #openstack-nova14:07
*** brinzhang0 has joined #openstack-nova14:10
*** TheJulia has joined #openstack-nova14:10
*** brinzhang has quit IRC14:13
*** k_mouza has joined #openstack-nova14:13
stephenfinlyarwood: Thanks for the review on the 'openstack server evacuate' patch btw. I sent it on its way. Long overdue.14:18
stephenfinI need to go through the API with a fine comb but I _think_ we have the vast majority of the API covered at this point14:18
*** artom has joined #openstack-nova14:19
gibistephenfin: what is the equvivalent for nova instance-action-list ? I allways fall back to the nova client to do that14:20
lyarwoodgibi: openstack server event list14:20
lyarwoodI think14:21
gibi /o\14:21
stephenfinyeah, that14:21
gibiI need to build muscle memory for that14:21
lyarwoodstephenfin: I don't think we have a version of that for migrations in osc btw14:21
* lyarwood checks14:21
stephenfinlyarwood: we do now14:21
lyarwoodoh sweet14:22
lyarwoodoh wait do you mean you're going to write it?14:22
stephenfinnope, already done14:22
stephenfindon't know if its released yet, mind, but soon14:23
lyarwoodah right that might be why it's not in the docs yeat14:23
gibiI guess then we can check our docs and replace the nova CLI exemples with openstack CLI examples:
gibibahh that paste contains duplicates14:27
gibinah, much better now
stephenfinsounds reasonable; I could probably do that this week14:30
openstackgerritBalazs Gibizer proposed openstack/nova master: Use the non polling notification waiter in func test
openstackgerritBalazs Gibizer proposed openstack/nova master: Create a fixture around fake_notifier
openstackgerritBalazs Gibizer proposed openstack/nova master: Use NotificationFixture for legacy notifications too
openstackgerritBalazs Gibizer proposed openstack/nova master: Test the NotificationFixture
openstackgerritBalazs Gibizer proposed openstack/nova master: Move fake_notifier impl under NotificationFixture
gibistephenfin: cool. you can plug me in for review14:31
gibiI think the (only) agreement on the PTG about OSC was that we try to fix our docs to use OSC examples14:32
brinzhang_stephenfin: +1, good start14:33
sean-k-mooneygibi: once we have parity however i really would like to look at deprecating novaclient and requireing osc to be updated14:34
sean-k-mooneywe can defer that for a cycle or so but i would like that to be our medium term goal14:34
sean-k-mooneyeven if we dont do it this cycle14:34
stephenfinsean-k-mooney: that's happening already14:34
gibisean-k-mooney: agree14:35
sean-k-mooneynot quite14:35
stephenfingtema is doing a lot of work here14:35
sean-k-mooneystephenfin: wehn i say deprecate novaclint i mean stop accpeting any new features/commands being added14:35
sean-k-mooneyso its bugfix only and require that they are added to osc instead14:35
*** sapd1 has joined #openstack-nova14:35
stephenfinto the CLI, sure, but we should keep adding to the library code until we're sure openstacksdk is good enough14:36
sean-k-mooneypreferably doing one final realse of novaclinet when we freeze the features14:36
sean-k-mooneystephenfin: maybe i was not making that distintion really14:36
stephenfinI'd like to move pretty much all of OSC first, since we should have no issues doing that once openstacksdk is good enough14:36
sean-k-mooneysince you have put this effort in to close the gap i just dont want it to reopen14:37
*** ociuhandu has joined #openstack-nova14:42
openstackgerritsean mooney proposed openstack/nova master: libvirt: delegate ovs plug to os-vif
sean-k-mooneystephenfin: lyarwood can ye take a look at ^14:44
sean-k-mooneyfixed the pep8 issue otherwise its the same14:44
openstackgerritBalazs Gibizer proposed openstack/nova master: doc: require openstack client change for every new API microversion
*** ociuhandu has quit IRC14:47
gibistephenfin, sean-k-mooney: this ^^ is my contribution to the OSC topic14:47
*** brinzhang0 has quit IRC14:48
*** brinzhang0 has joined #openstack-nova14:49
gibi"Delay in Elastic Search: Indexing behind by 141 hours" :(14:51
sean-k-mooneythats only slightly longer then normally its normally 72 hours i think14:54
*** k_mouza has quit IRC14:55
sean-k-mooneyif you are fering to logstash/kibana upstream14:55
* sean-k-mooney that still looks wrong but less so14:55
*** k_mouza has joined #openstack-nova15:01
gibiit remember when it was close to 015:01
gibilast week it hanged around 100 hours15:01
*** lpetrut has quit IRC15:02
sean-k-mooneyit does tend to vary but i think the target was no more the 72 hours15:02
sean-k-mooneyit does catch up from time to time15:02
gibiI hope so15:03
*** amodi has joined #openstack-nova15:11
brinzhang_stephenfin: thanks fix that issue from openstack server migration list CLI, I am sure I was tested ti, but the strange thing is that this issue was not found15:13
openstackgerritBalazs Gibizer proposed openstack/nova master: Remove compute service level check for qos ops
gibistephenfin: I think you will like this code removal patch ^15:21
*** ociuhandu has joined #openstack-nova15:22
*** mkrai has joined #openstack-nova15:24
openstackgerritBalazs Gibizer proposed openstack/nova stable/victoria: Warn when starting services with older than N-1 computes
openstackgerritBalazs Gibizer proposed openstack/nova stable/victoria: Add upgrade check about old computes
*** iurygregory has quit IRC15:43
stephenfingibi: Done. I assume we're okay to merge things like that now that we've got the service version check?15:45
gibistephenfin: that is my idea too15:45
dansmiththe per-release service version check should catch anything that merged N-2 releases ago,15:45
dansmithso it should be fine to remove older ones and rely on the macro one yeah15:46
dansmith(assuming the qos one is old enough, I didn't look)15:46
*** nweinber has quit IRC15:46
gibiqos move ops are added in Ussuri15:47
gibiI mean that last one15:47
gibiso in Victoria we could have removed the service level check. But we did not for extra safety15:48
*** nweinber has joined #openstack-nova15:48
openstackgerritLee Yarwood proposed openstack/nova master: Migrate nova-grenade-multinode job to zuulv3 native
dansmithstephenfin: we're not really breaking RPC specifically here, because we're not changing any rpc versions or signatures or anything, but this is one of those "not covered by the rpc versions" behaviors.. it's breaking service-to-service interaction, but only for older computes (not even older RPC versions), but for which we've already said isn't supported15:52
*** rpittau is now known as rpittau|bbl15:52
stephenfinthat makes sense15:53
lyarwoodgmann: ^ I'd like to push ahead with this btw, we are currently hitting with the original bionic based job so I'd rather switch to Focal and add the ceph coverage later.15:53
openstackLaunchpad bug 1901739 in OpenStack Compute (nova) " libvirt.libvirtError: internal error: missing block job data for disk 'vda'" [High,Fix released] - Assigned to Lee Yarwood (lyarwood)15:53
dansmithstephenfin: I'll put this in a comment once I review, but just echoing here since you called me out :)15:53
stephenfinta :15:53
gibistephenfin, dansmith: thanks for the review btw15:53
*** brinzhang_ has quit IRC15:55
*** ociuhandu has quit IRC15:58
*** dklyle has joined #openstack-nova16:02
*** iurygregory has joined #openstack-nova16:03
openstackgerritMerged openstack/nova master: Remove six.moves
*** mlavalle has joined #openstack-nova16:06
gmannlyarwood: but we are going to loose ceph coverage right? or we can add ceph coverage as separate job using existing script and move them once ceph greande base job is ready16:10
lyarwoodgmann: we can try but I'd take a working gate over missing ceph coverage for a few weeks at the moment16:10
gmannlyarwood: existing zuulv2 grenade jobs is working right or it is failing?16:12
jgwentworthlyarwood: so what's the plan for adding ceph coverage back? seems like a risk to leave it uncovered for an extended period of time. do we have any idea how to do it for v3 jobs?16:12
*** jgwentworth is now known as melwitt16:12
lyarwoodgmann: it's failing pretty often at the moment due to a an issue with libvirt/QEMU on bionic16:13
gmannyou mean on victoria gate? on master gate, it should run on Focal16:13
lyarwoodmelwitt: - wire up a native zuulv3 multinode ceph job16:13
*** ircuser-1 has joined #openstack-nova16:14
lyarwoodgmann: master gate, multinode grenade still uses bionic16:14
lyarwoodgmann: and that's the problem here16:14
lyarwoodgmann: or we can move it to NV16:14
gmannlyarwood: oh we should move it to Focal16:14
gmannah legacy job16:15
melwittlyarwood: sorry, I don't understand what I'm looking at here that's related to ceph16:15
gmannlyarwood: i did not move base legacy job on bionic16:15
gmannlyarwood: I think we can move to zuulv3 using script for now like in PS1 - and once ceph grenade base is ready then remove the use of script ?16:17
lyarwoodgmann: we can try16:18
gmannok let me update it.16:18
lyarwoodmelwitt: sorry nova-grenade-multinode currently does some ceph stuff manually via the live migration hook - &
lyarwoodmelwitt: I wanted to break this out into a native zuulv3 job based on a multinode ceph job but that's taking a while to work out on the topic I shared above16:22
melwittlyarwood: ok, so the manual stuff could be made "native" somehow. I did not know that16:23
lyarwoodmelwitt: yeah instead of calling specific bash functions from the plugin I just wanted to have a generic job that would deploy multinode ceph that we'd run the LM tests on in Nova16:24
lyarwoodmelwitt: I made some progress a while ago with the key sharing etc just became stuck at the end with getting the subnode to actually connect to the ceph cluster on the main node16:25
openstackgerritStephen Finucane proposed openstack/nova master: functional: Add live migration tests for PCI, SR-IOV servers
openstackgerritStephen Finucane proposed openstack/nova master: functional: Expand SR-IOV live migration tests with NUMA
melwittlyarwood: I see, thanks, that helps. I hoped to be able to help in some way since I am concerned about the coverage loss but didn't know where to look or start16:27
lyarwoodgmann: so are you just going to change the base job and hope that works?16:32
lyarwoodto tempest-multinode-full-py316:32
openstackgerritMerged openstack/nova master: Remove six.iteritems/itervalues/iterkeys
openstackgerritMerged openstack/nova master: Remove six.byte2int/int2byte
gmannlyarwood: not tempest but grenade-multinode and running in post phase with disable to run smoke tests on new node (which run as part of grenade-multinode playbooks)16:35
lyarwoodgmann: right I'm asking which base job you're going to use16:37
*** slaweq has quit IRC16:37
gmannlyarwood: for grenade anyways we need to use grenade-multinode16:37
*** ociuhandu has joined #openstack-nova16:38
lyarwoodgmann: right sorry, that's zuulv3 based and would just use the scripts to avoid us losing coverage16:39
lyarwoodgot ya16:40
*** slaweq has joined #openstack-nova16:40
*** mkrai has quit IRC16:46
*** ociuhandu has quit IRC16:50
*** xinranwang has quit IRC16:57
*** ociuhandu has joined #openstack-nova16:58
*** ociuhandu has quit IRC17:03
*** ociuhandu has joined #openstack-nova17:08
openstackgerritGhanshyam Mann proposed openstack/nova master: Migrate nova-grenade-multinode job to zuulv3 native
*** hamalq has joined #openstack-nova17:11
openstackgerritLee Yarwood proposed openstack/nova master: Add os-volume_attachments reference docs
*** hamalq has quit IRC17:14
*** hamalq has joined #openstack-nova17:15
*** rpittau|bbl is now known as rpittau17:22
openstackgerritMerged openstack/nova stable/victoria: [doc]: Fix glance image_metadata link
*** viks____ has quit IRC17:25
openstackgerritDat Le proposed openstack/nova master: Fix unplugging VIF when migrate/resize VM
*** k_mouza has quit IRC17:32
*** ralonsoh has quit IRC17:34
*** ociuhandu_ has joined #openstack-nova17:43
openstackgerritsean mooney proposed openstack/nova master: libvirt: delegate ovs plug to os-vif
sean-k-mooneystephenfin: lyarwood  can you re +w
sean-k-mooneyit was rebased17:45
*** ociuhandu has quit IRC17:47
*** ociuhandu_ has quit IRC17:48
*** nweinber has quit IRC17:51
*** xek__ has joined #openstack-nova17:53
*** xek__ has quit IRC17:55
*** xek_ has quit IRC17:56
*** xek__ has joined #openstack-nova17:56
*** xek__ has quit IRC17:56
*** derekh has quit IRC18:00
openstackgerritGhanshyam Mann proposed openstack/nova master: DNM: Testing system scope in tempest
*** andrewbonney has quit IRC18:09
openstackgerritElod Illes proposed openstack/nova stable/ussuri: [doc]: Fix glance image_metadata link
mnasersean-k-mooney: have you done some investigation by any chance on any potential ways of speeding up nova startup time in environments with compute nodes that have a large port count18:23
mnaserin this case, it takes almost 5-6 minutes for the agent to go up in an env with ~150-160 ports18:23
mnaserusing osv18:23
mnaseri havent tried migrating to the native ovsdb driver, maybe that might help, but a whole lot of plugging happens18:23
*** dtantsur is now known as dtantsur|afk18:24
sean-k-mooneyam not really. we have seen that usign native does indeed help form your previous issues but in general we do need to ensure that all network interfacs are plugged for all vms when the compute agent starts18:26
sean-k-mooneyit also depend on the release i know we fixed some issue with ip adress checking on newer release18:27
mnasersean-k-mooney: wonder if it might make sense to make this happen in a threadpool or something, as it seems to be happening one-by-one18:27
sean-k-mooneynot really18:28
mnaserbecause right now in some of those bigger vms, nova-compute reports down as it goes through the restart that takes ~6m18:28
sean-k-mooneywe could but we have to wait for them all to complete18:28
sean-k-mooneyso if we dispatch them to a tread pool we will get more paralium but we will the fill the pool and have to wait for it to complete18:29
sean-k-mooneyplugging form a nova side shoudl not take that long in general18:29
sean-k-mooneywith the native driver its much much faster18:29
*** gyee has joined #openstack-nova18:30
mnaserok i guess thats' probably going to be my next step to see how that improves it18:30
sean-k-mooneyas its not spawnting multiple shells via privsep in that process18:30
mnasersean-k-mooney: seems valid, i'll check it out and report and let you know what sort of improvement i see :)18:31
sean-k-mooneyputting it in a tread pool would basically just increase the batch size at the cost of memroy during start up.18:31
sean-k-mooneycool there would be a speed up form a thread pool but not as much as using the native driver18:32
*** mlavalle has quit IRC18:43
*** mlavalle has joined #openstack-nova18:44
*** rpittau is now known as rpittau|afk18:45
openstackgerritMerged openstack/nova master: virt: Remove 'get_per_instance_usage' API
sean-k-mooneylyarwood: was that  for the removal ^18:59
sean-k-mooneyoh because something finally merged in the gate :P19:00
sean-k-mooneyboth are good19:00
*** whoami-rajat__ has quit IRC19:00
*** tesseract has quit IRC19:02
*** larainema has quit IRC19:12
*** jangutter has joined #openstack-nova19:26
*** jangutter_ has quit IRC19:30
dansmithhmm, on focal the ceph devstack plugin tells me I'm on an unsupported distro19:36
dansmithlyarwood: any idea about that?19:36
lyarwooddansmith: no idea, I thought it was there?19:37
lyarwood# git grep f3219:37
lyarwooddevstack/lib/ceph:    if [[ ! ${DISTRO} =~ (focal|bionic|xenial|f31|f32) ]]; then19:37
dansmithhmm, I nuked opt/stack but maybe I've got residue somewhere19:38
dansmithit's checking against a list that doesn't include focal19:38
dansmithheh, doh19:38
dansmithenable_plugin devstack-plugin-ceph /home/dan/devstack-plugin-ceph19:38
sean-k-mooneyoh your using your local copy19:59
sean-k-mooneygot an old version checked out19:59
dansmithyeah, I nuked /opt/stack and then it cloned it right back there for me :P19:59
sean-k-mooneyyep i do that intentionally for  the most part and put all my repos in /opt/repos20:00
sean-k-mooneyjust so it does not clone over itself20:00
sean-k-mooneygmann:is there a spec or something covering the use of token scopes with nova. i dont see anythin after the ussuri spec
sean-k-mooneythat does nto cover the domains20:05
sean-k-mooneythe 4 scopes in make sense to me20:06
sean-k-mooneybut i dont see a defeintion for system_member, project_admin, domain_(reader, member or admin)20:06
sean-k-mooneywe have some interset down stream in supporting a 3*3 matix (reader, memeber, admin) *( project, domain, system)20:07
sean-k-mooneybut i dont recall ever disscuing domains or proejct admin and system memebr in the context of nova upstream20:08
sean-k-mooney has entries for all 9 roles20:09
sean-k-mooneybut given we have not had any specs or cross project session on this im having  a hard time relateing that to how we would support this in nova20:10
sean-k-mooneygmann: i know there was a pop up team but any pointers on where this might have been discussed20:10
sean-k-mooneymelwitt: johnthetubaguy  any input on ^20:12
gmannsean-k-mooney: we have project_admin in nova for case like creating server with host specify but system_member is not there as not needed from current policy rules20:16
gmannalso domain scope (it's three pesona)20:16
sean-k-mooneydo you have docs for this20:16
sean-k-mooneythere were only 4 included in the spec20:17
sean-k-mooneyproject_admin was not one of them20:17
gmannyeah, I updated policy doc for that20:17
*** tosky has quit IRC20:17
sean-k-mooneyok because form a spec point of vew we only ever approved system_admin, project_memeber system_reader and project_reader20:18
gmannsean-k-mooney: this does not include what all combination nova support but give an idea on new policy -
sean-k-mooneyso since this is an api cahnge im kind of confused why we dont have a spec for supporting other scopes and ropels in nova20:18
melwittyeah, that ^ is the doc I was searching for wrt the support in nova20:19
gmannsean-k-mooney: project_admin came up during implementation only and that is for create server with specific request (like force host, zero disk flavor etc) on;y20:19
sean-k-mooneyok but are we actully using it in code20:19
sean-k-mooney is the only spec that exist for this so really without a spec to add it we should not be useing in the api right20:20
gmannas part of policy defaults yes but we do not change token in nova right? whatever token is used for API access will be validated against the policy values20:20
sean-k-mooneymy concern is really for domains im concerend that a domain member may not have a project20:21
gmannsean-k-mooney: I am not sure about supporting other scopes. if we want domain scope control in nova then we need to see what all nova API operations are domain level and system level.20:21
sean-k-mooneymaybe they do but if they dont then that will break our api assumtions20:22
gmanni never thought of domain in nova.20:22
sean-k-mooneyits come up downstream my responce is right now we dont support it20:22
gmannand they use domain member user to access nova API ?20:22
sean-k-mooneywell its not a thing yet20:22
sean-k-mooneywe have a request to supprot RBAC for osp 17 which will be based on wallaby20:23
melwittyeah, my understanding is that we now have the code needed to handle scoped tokens in nova, so the way that they get used is someone (a user) has to request scoped tokens and use them when they call nova and then policy can validate the token against the policy20:23
gmannok, in that case they can assign admin/member/reader role on required project20:23
sean-k-mooneybut since domain was never discussed upstream i wanted to push back to just the 4 roles we intended to support20:23
sean-k-mooneymelwitt: right but we need to agree what scopes and roles are required for each endpoint20:24
sean-k-mooneyand what each scope/roles allows you to do20:24
gmannsean-k-mooney:  these are combination we support currently -
gmannsean-k-mooney: i can add these in policy doc so that we do not need to point these to code20:25
sean-k-mooneyproject admin is the only one im not expecting since we nver approved that in a spect20:25
sean-k-mooneythe rest make sense to me20:25
sean-k-mooneyfor example we discussed in the ptg that prject admisn shoudl not be aware of the hosts vms run on20:26
gmannsean-k-mooney: also html policy doc mention the scope of each policy/API as part of oplicy documentation -
sean-k-mooneybut you mentioned that one of the usecase was allowing them to boot on a host20:26
sean-k-mooneyhave we released with project_admin yet20:27
gmannyou remember in PTG we discuss about it20:27
sean-k-mooneyi rememebr the one two weeks ago20:27
*** tosky has joined #openstack-nova20:28
gmannmy initial thought while adding PROJECT_ADMIN was that this is temp and TODO will remove this -
sean-k-mooneyi kind of which there was a spec for projecft admin because i really ame not sure that the current usaged in the api are valid20:29
sean-k-mooneygmann: we cant remove it20:29
gmannbut as we discussed in PTG,  I think it make sense to keep it and ask users to assign appropriate role to let users to boot on specific host20:29
sean-k-mooneywell not without a microverion20:30
gmannthis is policy default change so microversion not required if we change any of these20:30
sean-k-mooney making it sys_admin i think make sense20:30
sean-k-mooneyfrom an interoperablity point of view im not sure i agree20:31
gmannsean-k-mooney: that is the things we discussed in PTG. my proposal was sys_admin and take project_id as request param.20:31
sean-k-mooneyfor this case since i kind fo want to revert this out and make it system_admin im happy to agree20:31
sean-k-mooneythat is a different usecase20:32
gmannbut with sys_adm, project_id needs to be passed for which sys admin want to create server for20:32
sean-k-mooneyyes so that is the actuall cloud admin creating a server in a project in responce to a supprot request or something20:33
sean-k-mooneythat is differnet for giving a customer a project adminsitarto role and allowign them to do it in a self service manner20:33
sean-k-mooneythat is what project_admin role would mean20:33
sean-k-mooneygmann: im fine with the sysadmin doing what your proposing20:34
sean-k-mooneybut ithe proejct_admin persona im describing i dont think shoudl have any awareness of hosts20:34
sean-k-mooneygmann: anyway thanks ill review the docs you linked.20:35
gmannthat was my thought while proposing this but johnthetubaguy melwitt point was it is asking cloud admin to give extra role to user (project_admin or system_reader to know host info) to allow users to boot on specific host20:36
sean-k-mooneyso that is something i think we need to actully discuss if we are every to use those poices by default in code20:36
sean-k-mooneyvai a spec20:37
sean-k-mooneybecaue you are fundimentally change the way the api works allowing operation to work for a different set of users20:37
gmannwhich make sense to me too after PTG discussion. so from nova API side we say 'sustem_reader is role you need to know host info' and 'project_admin is role to boot instance on requested host' so you can assign those based on users req20:37
sean-k-mooneyya although i think there are more details to be worked out20:38
gmannsean-k-mooney: right, for migrating to new policy, cloud provider needs to change the existing users token20:38
*** whoami-rajat__ has joined #openstack-nova20:38
sean-k-mooneylike shold a project admin be allowed to do a live migration?20:39
sean-k-mooneyshoudl they see the host in the server show (based on the above no unles they have system_reader)20:39
gmannsean-k-mooney: as default policy no but live migration case we opened for system as well as project scope also. in case use case is to allow users to  perform live migration.20:40
sean-k-mooneygiving the proejct admin system reader also feels slightly wrong20:40
gmannsean-k-mooney:  let me propose doc update what we agreed in PTG and then we can see how it looks like. it is too much asking to providers or  ok.20:41
sean-k-mooneye.g. im not sure you should have readonly access to everything20:41
sean-k-mooneygmann: well im trying to gague the impact to us downstream from a support point of view20:41
gmannIf provider want to allow any users to boot on requested host then giving system_reader to them not bad so that they can access host info.20:43
sean-k-mooneygmann: tehy could also acess other info right20:43
sean-k-mooneye.g. list all servers20:43
sean-k-mooneyor all keypairs20:43
sean-k-mooneyfor all users20:44
gmannyeah that is one issue.20:44
sean-k-mooneythat makes it a non starter for me20:44
sean-k-mooneysystem reader is basically full admin but readonly right20:44
sean-k-mooneyso i could use it to find the fix ips of other tenants insntances20:45
sean-k-mooneyif it was not scoped to noava i could hten list the security groups to find open ports and try connecting to them20:45
gmannwell with override policy they can restrict, like list server for all is controlled with separate policy. list keypairs can be restricted with user_id20:45
sean-k-mooneybut im worried about the support matrix20:46
gmannbut i agree that is open things for them which was point in PTG also20:46
sean-k-mooneye.g.  use agreeing to support anythong other then the nova default by default with our custoemrs20:46
gmannmay be admin need to carefully select such users and trust them if they are allowed to boot on requested host  ?20:47
sean-k-mooneymaybe but that basically to me say we can never enabel this by default20:47
sean-k-mooneyno its the use fo system_reader that is too heavy here20:47
sean-k-mooneyi kind fo would liek if we could use the tenant isolation aggreate info20:48
sean-k-mooneyso that with project_admin you coudl list hosts and only see those hosts20:48
sean-k-mooneyso no system reader20:48
sean-k-mooneyneeded to boot to a host you are allowed to boot too20:48
sean-k-mooneyif you are not limited to an aggreate i guess that would list all hosts20:49
gmannbut host is not project level info right20:49
sean-k-mooneyit kind of is20:49
sean-k-mooneywe supprot assocating tenats/project to aggreates20:49
sean-k-mooneyvia either a placment prefilter or a schduler post filter20:50
gmannbut project_admin is our special case. means no where else it is being used20:50
sean-k-mooneythat the thing i see it potally being a thing in other services too20:50
*** tbachman has quit IRC20:50
gmannah yeah20:50
sean-k-mooneyi can totally see project admins being allowed to create users for a project in there project only via keystoen for example20:51
sean-k-mooneyor managing a subset of roles with in a porject20:51
sean-k-mooneye.g. givie a user project_reader, porject_admin or project member20:52
sean-k-mooneyfor that project20:52
gmannalso i think neutron might have when they do new policy like attr level policy rule etc20:52
sean-k-mooneyright i could see requiring proejct_admin to create shared networks for example20:52
sean-k-mooneyor better example20:53
gmannso both options (sys reader or project admin) have pros and cons.20:53
sean-k-mooneyadding a qos policy to a network20:53
*** tbachman has joined #openstack-nova20:53
melwittI'm not 100% following this convo but wanted to mention I think it's normal and expected that users will have to request and use appropriate tokens (and have appropriate roles) for individual APIs and that if we try to fit everything a user can possibly want to do to fall under one token scope/role then we're going back toward the "admin does everything" direction, trying to make everything fit into one box again20:53
sean-k-mooneyto make proejct_admin work we woudl need other code changes20:53
sean-k-mooneymelwitt: multiple tokens totally makes sense but i dont think system_reader is approcate for anyoen that you dont fully trust20:54
sean-k-mooneyso i can see system reader ever be appliable for a tenant of a vexhost cloud20:55
sean-k-mooneythat did not work at vexhost20:55
gmannmelwitt: yeah. project_admin was really a temp think with assumption that system scope users to allow creating server for projetcs20:55
sean-k-mooneythe main usecasue for system_reader is for audits right20:56
sean-k-mooneyso by default you would want system_reader to be able to read across multipel projects20:56
sean-k-mooneywhere as proejct_reader would be the same fucntion for a singel project20:57
sean-k-mooneyand domain is inbetween20:57
sean-k-mooneyread only access to all proejct in a domain20:57
sean-k-mooney*project resouces20:57
sean-k-mooneyif you require something more the system_reader for inter proejct server list then it kind of breakes its orginal usecause20:59
sean-k-mooneyanyway its late and im hungry so ill call it a night20:59
sean-k-mooneygmann: thanks for the info o/20:59
gmannbut if any users is allowed to boot on requested host then  it is special user right.21:00
gmannsean-k-mooney: sure. take rest. We can discuss tomorrow.21:00
*** sapd1 has quit IRC21:05
*** mgoddard has quit IRC21:25
*** mgoddard has joined #openstack-nova21:26
*** rcernin has joined #openstack-nova21:27
*** vishalmanchanda has quit IRC21:57
*** rcernin has quit IRC22:06
*** rcernin has joined #openstack-nova22:12
*** martinkennelly has quit IRC22:12
*** rcernin has quit IRC22:22
*** rcernin has joined #openstack-nova22:22
itsjgHello! Could someone help point me in the right direction, I can't get a machine to migrate off an old host due to the "No valid host found for cold migrate (HTTP 400)" error. I have tried setting every debug logging level to maximum but can't get any further details as to why nova-scheduler is not moving the VM to a different host. I have plenty of capacity across the whole cluster, checked all compute services across the whole cluster, etc. I guess f22:30
itsjgan easy way to see why the scheduler throws an error, is there some place to look that I'm missing?22:30
*** brinzhang_ has joined #openstack-nova22:30
*** brinzhang0 has quit IRC22:34
*** takamatsu has quit IRC22:35
*** slaweq has quit IRC22:39
*** iurygregory has quit IRC23:04
*** iurygregory has joined #openstack-nova23:16
*** takamatsu has joined #openstack-nova23:20
openstackgerritGhanshyam Mann proposed openstack/nova master: Improve policy doc for supported scope info
gmannsean-k-mooney: melwitt johnthetubaguy ^^ adding nova supported scope & roles into doc23:33
*** tosky has quit IRC23:33
openstackgerritMerged openstack/nova master: rbd: Only log import failures when the RbdDriver is used

Generated by 2.17.2 by Marius Gedminas - find it at!