Wednesday, 2018-02-14

*** claudiub has quit IRC00:00
*** tetsuro has joined #openstack-nova00:03
openstackgerritEric Berglund proposed openstack/nova master: WIP: PowerVM Driver: Snapshot
*** felipemonteiro has joined #openstack-nova00:12
*** hemna_ has joined #openstack-nova00:13
*** felipemonteiro_ has joined #openstack-nova00:13
*** itlinux has joined #openstack-nova00:16
*** liverpoo1er has quit IRC00:17
*** felipemonteiro has quit IRC00:17
*** liverpooler has quit IRC00:17
*** edmondsw has quit IRC00:18
*** liverpooler has joined #openstack-nova00:21
*** liverpoo1er has joined #openstack-nova00:21
*** lbragstad has quit IRC00:22
*** hshiina has joined #openstack-nova00:23
*** slaweq has joined #openstack-nova00:31
*** slaweq has quit IRC00:36
*** Dinesh_Bhor has joined #openstack-nova00:37
*** AlexeyAbashkin has joined #openstack-nova00:40
*** links has joined #openstack-nova00:40
*** mlavalle has quit IRC00:40
*** chyka has quit IRC00:42
*** chyka has joined #openstack-nova00:42
*** acormier has joined #openstack-nova00:44
*** AlexeyAbashkin has quit IRC00:44
*** acormier has quit IRC00:45
*** chyka has quit IRC00:47
*** mdnadeem has joined #openstack-nova00:50
*** sdague has joined #openstack-nova00:51
*** zhaochao has joined #openstack-nova00:55
*** sdague has quit IRC00:57
*** jichen has joined #openstack-nova00:59
gmann_mriedem_afk: gibi stephenfin as we faced the nova jobs irrelevant_files and branch restriction issues and have to fix on project-config side, i am thinking to move pipeline list jobs on project side.00:59
gmann_mriedem_afk: gibi stephenfin this will give nova a full control on those bits. i started a generic discussion on ML-
*** esberglu has quit IRC01:00
gmann_please check and if that looks ok/not ok from nova perspective.01:00
gmann_melwitt: ^^01:01
gmann_next issue is irrelevant_files which does not work in case of inheritance or pipeline list (current issue where lot of nova jobs run on doc/api-ref changes), i will push some debug patch and then debug with infra team.01:02
*** Dinesh_Bhor has quit IRC01:08
openstackgerritTakashi NATSUME proposed openstack/nova master: [placement] Add functional tests for traits API
*** Dinesh_Bhor has joined #openstack-nova01:10
openstackgerritRong Han proposed openstack/nova stable/pike: detach instance volumes when VM creation fails
*** moshele has joined #openstack-nova01:15
openstackgerritRong Han proposed openstack/nova stable/queens: detach instance volumes when VM creation fails
*** itlinux has quit IRC01:19
*** hongbin has joined #openstack-nova01:24
*** Dinesh_Bhor has quit IRC01:25
*** gyee has quit IRC01:25
*** Dinesh_Bhor has joined #openstack-nova01:26
*** Dinesh_Bhor has quit IRC01:31
*** felipemonteiro__ has joined #openstack-nova01:33
*** felipemonteiro_ has quit IRC01:33
tetsuroefried, dansmith, mriedem, jaypipes: Just finished reading back the dictussion about the update_provider_tree. Thank you for picking it up!01:34
*** hemna_ has quit IRC01:34
*** liverpooler has quit IRC01:35
*** Dinesh_Bhor has joined #openstack-nova01:36
*** Dinesh_Bhor has quit IRC01:41
*** itlinux has joined #openstack-nova01:44
*** acormier has joined #openstack-nova01:44
*** Dinesh_Bhor has joined #openstack-nova01:45
*** edmondsw has joined #openstack-nova01:46
*** felipemonteiro_ has joined #openstack-nova01:46
*** rmcall has joined #openstack-nova01:46
*** moshele has quit IRC01:47
*** felipemonteiro__ has quit IRC01:49
*** edmondsw has quit IRC01:51
*** esberglu has joined #openstack-nova01:57
*** takashin has quit IRC01:57
mriedem_afktakashin: i have a couple of comments in
mriedem_afkif you can address and then i'll +W tomorrow01:57
*** esberglu has quit IRC01:57
*** takashin has joined #openstack-nova01:57
*** felipemonteiro_ has quit IRC01:59
*** takashin has quit IRC01:59
*** takashin has joined #openstack-nova02:00
openstackgerritMatt Riedemann proposed openstack/nova master: Add admin guide doc on volume multiattach support
*** takashin has quit IRC02:05
*** takashin has joined #openstack-nova02:05
*** chyka has joined #openstack-nova02:09
*** takashin has quit IRC02:09
*** takashin has joined #openstack-nova02:09
*** zhaochao has quit IRC02:09
*** itlinux has quit IRC02:10
*** chyka has quit IRC02:13
openstackgerritMatt Riedemann proposed openstack/nova-specs master: Amend the migration paging spec for uuid in server migrations response
openstackgerritjichenjc proposed openstack/nova master: WIP: Add zvm admin doc
openstackgerritjichenjc proposed openstack/nova master: WIP: Add zvm CI information
*** zhaochao has joined #openstack-nova02:22
*** Dinesh_Bhor has quit IRC02:23
*** vivsoni has quit IRC02:23
openstackgerritMatt Riedemann proposed openstack/nova-specs master: Amend volume multi-attach spec
*** takashin has quit IRC02:28
*** takashin has joined #openstack-nova02:29
*** itlinux has joined #openstack-nova02:32
*** slaweq has joined #openstack-nova02:33
*** mriedem_afk has quit IRC02:34
*** lbragstad has joined #openstack-nova02:35
*** slaweq has quit IRC02:37
*** Tom-Tom has joined #openstack-nova02:40
*** mdnadeem has quit IRC02:42
*** yamahata has quit IRC02:42
*** liusheng has quit IRC02:43
*** liusheng has joined #openstack-nova02:43
*** itlinux has quit IRC02:44
*** salv-orl_ has joined #openstack-nova02:45
*** salv-orlando has quit IRC02:48
*** suresh12 has quit IRC02:53
*** suresh12 has joined #openstack-nova03:00
*** takashin has quit IRC03:00
*** takashin has joined #openstack-nova03:01
*** suresh12 has quit IRC03:04
*** takashin has quit IRC03:06
*** takashin has joined #openstack-nova03:06
*** Tom-Tom has quit IRC03:15
*** takashin has quit IRC03:15
*** takashin has joined #openstack-nova03:15
openstackgerritjichenjc proposed openstack/nova master: fix race condition of instance host
*** takashin has quit IRC03:19
*** takashin has joined #openstack-nova03:19
openstackgerritZhenyu Zheng proposed openstack/nova-specs master: WIP
*** r-daneel has joined #openstack-nova03:23
*** takashin has quit IRC03:24
*** acormier has quit IRC03:28
*** harlowja has quit IRC03:30
*** acormier has joined #openstack-nova03:32
*** edmondsw has joined #openstack-nova03:34
*** takashin has joined #openstack-nova03:38
*** edmondsw has quit IRC03:39
*** acormier has quit IRC03:43
*** yamamoto has joined #openstack-nova03:43
*** Tom-Tom has joined #openstack-nova03:46
*** links has quit IRC03:50
*** Tom-Tom has quit IRC03:50
*** felipemonteiro has joined #openstack-nova03:54
*** sree has joined #openstack-nova03:59
*** udesale has joined #openstack-nova03:59
*** lbragstad has quit IRC04:02
*** links has joined #openstack-nova04:07
*** dave-mccowan has quit IRC04:10
*** harlowja has joined #openstack-nova04:12
*** takashin has quit IRC04:12
*** takashin has joined #openstack-nova04:12
*** efried1 has joined #openstack-nova04:20
*** efried has quit IRC04:22
*** efried1 is now known as efried04:22
openstackgerritjichenjc proposed openstack/nova master: WIP: fix validator confusing info
*** takashin has quit IRC04:25
*** Tom-Tom has joined #openstack-nova04:29
*** harlowja has quit IRC04:30
*** slaweq has joined #openstack-nova04:34
*** suresh12 has joined #openstack-nova04:37
*** lpetrut has joined #openstack-nova04:38
*** slaweq has quit IRC04:38
*** suresh12 has quit IRC04:41
openstackgerritArvind Nadendla proposed openstack/nova-specs master: Support traits in Glance
*** felipemonteiro has quit IRC04:45
*** Tom-Tom has quit IRC04:45
*** abhishekk has joined #openstack-nova04:51
*** bhagyashri_s is now known as bhagyashris04:52
*** suresh12 has joined #openstack-nova04:54
*** hongbin has quit IRC04:55
*** psachin` has joined #openstack-nova05:04
*** psachin has joined #openstack-nova05:04
*** suresh12 has quit IRC05:04
*** dillaman has joined #openstack-nova05:08
*** jdillaman has quit IRC05:09
*** liverpoo1er has quit IRC05:09
*** lpetrut has quit IRC05:11
*** Tom-Tom has joined #openstack-nova05:12
*** takashin has joined #openstack-nova05:13
*** harlowja has joined #openstack-nova05:14
*** Tom-Tom has quit IRC05:17
*** moshele has joined #openstack-nova05:17
*** lpetrut has joined #openstack-nova05:19
*** Tom-Tom has joined #openstack-nova05:20
*** Tom-Tom has quit IRC05:21
*** claudiub has joined #openstack-nova05:21
*** Tom-Tom has joined #openstack-nova05:21
*** vladikr has quit IRC05:24
*** vladikr has joined #openstack-nova05:25
*** moshele has quit IRC05:27
*** lpetrut has quit IRC05:29
*** jichen has quit IRC05:32
*** tbachman has joined #openstack-nova05:33
*** jichen has joined #openstack-nova05:33
*** sridharg has joined #openstack-nova05:34
*** vladikr has quit IRC05:35
*** janki has joined #openstack-nova05:36
*** takashin has quit IRC05:39
*** takashin has joined #openstack-nova05:39
openstackgerritjichenjc proposed openstack/nova master: Avoid live migrate to same host
*** lbragstad has joined #openstack-nova05:42
*** bhujay has joined #openstack-nova05:42
*** acormier has joined #openstack-nova05:43
*** psachin has quit IRC05:45
*** psachin` has quit IRC05:45
*** hiro-kobayashi has joined #openstack-nova05:46
*** acormier has quit IRC05:48
*** psachin has joined #openstack-nova05:51
*** takashin has quit IRC05:54
*** takashin has joined #openstack-nova05:55
*** tbachman has quit IRC05:55
*** slaweq has joined #openstack-nova05:59
*** slaweq has quit IRC06:04
*** takashin has quit IRC06:14
*** takashin has joined #openstack-nova06:14
*** takashin has quit IRC06:16
*** takashin has joined #openstack-nova06:17
*** psachin has quit IRC06:20
*** takashin has quit IRC06:21
*** takashin has joined #openstack-nova06:22
*** masber has joined #openstack-nova06:22
*** tbachman has joined #openstack-nova06:22
*** harlowja has quit IRC06:24
*** psachin has joined #openstack-nova06:28
*** moshele has joined #openstack-nova06:30
openstackgerritjichenjc proposed openstack/nova master: Fix warning for user_id and project_id
*** suresh12 has joined #openstack-nova06:43
*** Eran_Kuris has joined #openstack-nova06:43
*** suresh12 has quit IRC06:47
openstackgerritTakashi NATSUME proposed openstack/nova-specs master: Fix missing blueprint URLs
openstackgerritTakashi NATSUME proposed openstack/nova-specs master: Fix missing blueprint URLs
openstackgerritTakashi NATSUME proposed openstack/nova-specs master: List/show all server migration types
*** belmoreira has joined #openstack-nova07:01
tonybstephenfin: You can remove your -W from when you're online :)07:04
*** jchhatbar has joined #openstack-nova07:05
*** janki has quit IRC07:09
*** edmondsw has joined #openstack-nova07:11
*** jchhatbar is now known as janki07:14
*** edmondsw has quit IRC07:15
*** AlexeyAbashkin has joined #openstack-nova07:16
*** lbragstad has quit IRC07:17
*** alexchadin has joined #openstack-nova07:17
*** AlexeyAbashkin has quit IRC07:21
*** AlexeyAbashkin has joined #openstack-nova07:26
*** takashin has left #openstack-nova07:30
*** mdnadeem has joined #openstack-nova07:30
openstackgerritMerged openstack/nova master: Bindep does not catch missing libpcre3-dev on Ubuntu
*** ragiman has joined #openstack-nova07:37
openstackgerritOpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata
*** lpetrut has joined #openstack-nova07:42
*** lajoskatona has joined #openstack-nova07:44
*** slaweq has joined #openstack-nova07:48
*** andreas_s has joined #openstack-nova07:48
*** slaweq has quit IRC07:53
*** slaweq has joined #openstack-nova07:54
*** pcaruana has joined #openstack-nova07:56
*** bhagyashri_s has joined #openstack-nova07:56
*** pooja-jadhav has joined #openstack-nova07:58
*** nsingh has joined #openstack-nova07:58
*** niraj_singh has quit IRC07:59
*** bhagyashris has quit IRC07:59
*** pooja_jadhav has quit IRC07:59
*** slaweq has quit IRC07:59
*** Tom-Tom has quit IRC08:04
*** masber has quit IRC08:05
*** slaweq has joined #openstack-nova08:05
*** slaweq has quit IRC08:10
*** damien_r has joined #openstack-nova08:13
*** jpena|off is now known as jpena08:16
*** amoralej|off is now known as amoralej08:20
*** yamamoto has quit IRC08:21
*** yamamoto has joined #openstack-nova08:22
*** tesseract has joined #openstack-nova08:28
*** bhagyashris_ has joined #openstack-nova08:28
*** bhagyashris_ has quit IRC08:28
*** pooja_jadhav has joined #openstack-nova08:30
*** niraj_singh has joined #openstack-nova08:31
*** pooja-jadhav has quit IRC08:31
*** bhagyashris has joined #openstack-nova08:31
*** stakeda has quit IRC08:31
*** nsingh has quit IRC08:32
*** bhagyashri_s has quit IRC08:32
*** rcernin has quit IRC08:33
*** slaweq has joined #openstack-nova08:40
*** mgoddard_ has joined #openstack-nova08:43
*** slaweq has quit IRC08:44
*** sshwarts has joined #openstack-nova08:47
*** tetsuro has left #openstack-nova08:48
openstackgerritClaudiu Belu proposed openstack/nova master: compute: Cleans up allocations after failed resize
*** ttx has joined #openstack-nova08:50
*** ralonsoh has joined #openstack-nova08:52
*** links has quit IRC08:53
*** hiro-kobayashi has quit IRC09:03
*** tssurya has joined #openstack-nova09:05
*** ociuhandu has joined #openstack-nova09:05
*** links has joined #openstack-nova09:06
*** yamahata has joined #openstack-nova09:07
*** ociuhandu has quit IRC09:10
*** jichen has quit IRC09:10
*** slaweq has joined #openstack-nova09:18
*** jangutter has quit IRC09:20
gibigmann_: thanks for the ML post, I put my 2 cents in a reply.09:21
*** janki has quit IRC09:22
*** slaweq has quit IRC09:23
*** slaweq has joined #openstack-nova09:24
*** yamahata has quit IRC09:24
gmann_gibi: thanks, i am not sure about moving from integrated-job but over time we can check those if creating issues09:27
gmann_gibi: and on irrelevant_files issue, i reply on that your ML thread with additional query. i think avoiding the irrelevant_files in base job/template solve the most of the issue.09:27
*** slaweq_ has joined #openstack-nova09:27
*** Tom-Tom has joined #openstack-nova09:28
*** slaweq has quit IRC09:28
*** yamahata has joined #openstack-nova09:31
gibigmann_: I'm not sure. The integrated-gate list jobs without irrelevant-files and that means those jobs will run on ever patch and we cannot override that later in project-config or in-tree09:37
gibigmann_: at least that how I understood the problem in
gmann_gibi: no, we can still define irrelevant-files  in-tree. if those job does not have  irrelevant-files  in base definition09:38
gibigmann_: then I'm still confused about the problem in :)09:39
*** derekh has joined #openstack-nova09:40
gibigmann_: what I see there is that we list tempest-full job twice, once in the integrated-gate template without irrelevant-files and once in the project-config with irrelevant-files09:40
gibigmann_: still the tempest-full is triggered on every patches as the above two listings are understood as two different possible triggers for the job09:41
gibigmann_: and the definition in integrated-gate matches to every patch (as no irrelevant-files define there)09:41
*** yikun_jiang has quit IRC09:45
*** umbSublime has quit IRC09:48
gmann_gibi: yea that seems the behavior.09:50
gmann_gibi: but that is opposite for 'branch' case, which get overridden from template to pipeline list definition  -
gmann_gibi: i got your point on
gmann_gibi: after your fix, i ll move those complete bits to nova-tree and there we control and do not need to use template if they create issues.09:56
*** alexchadin has quit IRC09:56
*** alexchadin has joined #openstack-nova09:57
gibigmann_: ohh, so branch behaves differently, I did not know that10:03
*** Tom-Tom has quit IRC10:04
stephenfintonyb: Done :)10:05
*** pcaruana has quit IRC10:05
openstackgerritLee Yarwood proposed openstack/nova stable/pike: libvirt: Block swap volume attempts with encrypted volumes
gmann_gibi: but issue is same i think, zuul found it in 2 place and run on stable branch also as it did not find any branch restriction on project pipeline list10:08
*** sree has quit IRC10:09
*** sree has joined #openstack-nova10:10
gibigmann_: I see. Then we cannot simply disallow defining branch and irrelevant-files for jobs in project-templates10:10
gibigmann_: we either have a job in the templates that has irrelevant-files and branch definition that is good for every porject using that template, or we cannot have jobs in the template10:11
openstackgerritMerged openstack/osc-placement stable/queens: Update UPPER_CONSTRAINTS_FILE for stable/queens
*** cdent has joined #openstack-nova10:12
gmann_gibi: +1. totally agree. as we cannot have generic irrelevant-files for each project i think we remove the integrated-job as you mentioned in ML10:12
*** sree has quit IRC10:15
*** slaweq_ is now known as slaweq10:16
gibigmann_: cool. As I mentioned I'm happy to help with the actual work as well if needed10:17
gmann_gibi: replied.10:17
gmann_gibi: thanks that will be helpful.10:17
gmann_gibi: ll coordinate with you once get final confirmation from on ML10:18
gibigmann_: thanks10:20
*** pcaruana has joined #openstack-nova10:20
* cdent waves good morning10:20
* gibi waves back10:21
*** zhaochao has quit IRC10:24
* bauzas forgot to wave earlier, but since morning greetings are subjective for everyone, then...10:24
bauzasgood morning Nova10:25
*** owalsh is now known as owalsh_afk10:28
cdentlately it feels like morning (pre-coffee) all the time10:28
*** lucas-pto is now known as lucasagomes10:30
bauzasI feel always undercaffeinated10:33
*** yamahata has quit IRC10:34
openstackgerritLee Yarwood proposed openstack/nova master: libvirt: Block swapping to an encrypted volume when using QEMU to decrypt
gibiI have some coffee beans covered in chocolate nearby all the time, that helps with my 'balanced' sugar and caffein intake10:36
openstackgerritHironori Shiina proposed openstack/nova-specs master: Ironic: Instance switchover
cdentgibi is wise10:37
bauzaswe have something called "Green chaud" here10:39
gmann_gibi: bauzas cdent enjoy your morning coffee, it's dinner time here :).10:39
gibigmann_: actually I'm also closer to lunch than morning coffee. Enjoy your dinner10:40
*** ttsiouts_ has joined #openstack-nova10:41
gibibauzas: that drink looks to me that somethin that helps to got to sleep at the end of a long day10:42
bauzasgibi: rather at the beginning of a skiing day10:42
gibibauzas: interesting10:42
cdenta) looks tasty, b) agree it looks sleep, not ski, inducing10:43
*** abhishekk has quit IRC10:44
*** hegemoOn has left #openstack-nova10:46
*** edmondsw has joined #openstack-nova10:47
*** sambetts|afk is now known as sambetts10:48
*** slaweq_ has joined #openstack-nova10:50
*** priteau has joined #openstack-nova10:52
*** edmondsw has quit IRC10:52
ameedaHi, just a question about gerrit review, I have 2 nova reviews waiting for reviewers (one of them got +2c, waiting for +2w) and another one doesn't have activities from reviewers from awhile , do you follow scheduler to check them or I have to ask you to check them ?10:55
*** slaweq_ has quit IRC10:55
cdentbauzas, gibi, stephenfin ^10:58
*** damien_r has quit IRC11:02
*** damien_r has joined #openstack-nova11:02
gibiameeda: it depends. For example priority feature work and high and critical bug fixes tend to get attention automatically the rest of the patches might need some encuragement11:04
*** AlexeyAbashkin has quit IRC11:05
gibiameeda: also I suggest to try to communicate why something is important / needs attention instead of just pinging people with reviews. It helps us reviewers to prioritize properly11:05
gibiameeda: when I have time I sometimes look at random reviews on gerrit having a clean CI run and some code review11:07
gibiameeda: but most of the time I have a list of things to make progress with11:07
openstackgerritAndrey Volkov proposed openstack/osc-placement master: RP list: member_of and resources parameters (v1.3, v1.4)
openstackgerritAndrey Volkov proposed openstack/osc-placement master: RP delete inventories (v1.5)
openstackgerritAndrey Volkov proposed openstack/osc-placement master: CLI for traits (v1.6)
openstackgerritAndrey Volkov proposed openstack/osc-placement master: Resource class set (v1.7)
openstackgerritAndrey Volkov proposed openstack/osc-placement master: Usages per project and user (v1.8, v1.9)
openstackgerritAndrey Volkov proposed openstack/osc-placement master: CLI allocation candidates (v1.10)
openstackgerritAndrey Volkov proposed openstack/osc-placement master: PUT dict format to /allocations (v1.11, v1.12)
openstackLaunchpad bug 1749410 in OpenStack Compute (nova) "placement api-ref unclear if capacity is meant to be total or current" [Wishlist,Confirmed]11:26
openstackgerritClaudiu Belu proposed openstack/nova master: compute: Cleans up allocations after failed resize
* cdent looks11:26
cdentbauzas, hmmm, yeah, not entirely clear what the best way to clarify that is. It might be useful for there to be separate glossary of placement terms, so that the tables in the api-ref don't get too clogged up11:28
bauzascdent: anyway, just FYI11:28
* cdent nods11:28
*** pcaruana has quit IRC11:30
mdboothHmm, instance snapshot is root disk only.11:32
*** udesale_ has joined #openstack-nova11:36
*** sree has joined #openstack-nova11:36
bauzascdent: yet again some good point
openstackLaunchpad bug 1749404 in OpenStack Compute (nova) "nova-compute resource tracker ignores 'reserved' while reporting 'max_unit'" [Undecided,New]11:37
*** alexchadin has quit IRC11:38
*** udesale has quit IRC11:38
*** AlexeyAbashkin has joined #openstack-nova11:38
*** sree has quit IRC11:40
cdentbauzas: yup, that is a good point. I suppose an argument could be made that "it depends on allocation ratio" but I think the intent is as stated in the bug.11:40
bauzascdent: I'll mark it as Wontfix11:40
bauzasit's a semantic point of view11:41
cdentno, I think it should be fixed11:41
*** udesale has joined #openstack-nova11:41
*** udesale_ has quit IRC11:41
bauzascdent: if so, reopen the bug with your comment please11:41
bauzasI'm fine with reopening it11:41
cdentif allocation_ratio is one and reserved is > 0 then it is wrong11:41
cdentwill do11:41
*** moshele has quit IRC11:42
*** pcaruana has joined #openstack-nova11:42
*** alexchadin has joined #openstack-nova11:47
*** udesale has quit IRC11:47
*** sree has joined #openstack-nova11:51
openstackgerritChen Hanxiao proposed openstack/nova master: libvirt: remove TODO on validation of scsi model
*** cdent has quit IRC11:56
*** sree has quit IRC11:56
*** dave-mccowan has joined #openstack-nova11:56
*** ttsiouts_ has quit IRC11:57
*** sree has joined #openstack-nova11:58
openstackgerritChen Hanxiao proposed openstack/nova master: libvirt: add Linux distribution guest only description for inject_xxx options
*** sree_ has joined #openstack-nova12:00
*** sree_ is now known as Guest1896812:01
*** sree has quit IRC12:03
*** elmaciej has joined #openstack-nova12:11
*** slaweq_ has joined #openstack-nova12:11
*** slaweq_ has quit IRC12:16
openstackgerritOpenStack Proposal Bot proposed openstack/nova master: Updated from global requirements
*** slaweq_ has joined #openstack-nova12:19
*** patriciadomin_ is now known as patriciadomin12:21
*** dtantsur|afk is now known as dtantsur12:21
*** slaweq_ has quit IRC12:23
*** yamamoto has quit IRC12:23
*** yamamoto has joined #openstack-nova12:24
*** cdent has joined #openstack-nova12:26
openstackgerritOpenStack Proposal Bot proposed openstack/python-novaclient master: Updated from global requirements
*** sdague has joined #openstack-nova12:30
*** moshele has joined #openstack-nova12:30
*** edmondsw has joined #openstack-nova12:36
*** moshele has quit IRC12:40
*** edmondsw has quit IRC12:40
*** moshele has joined #openstack-nova12:43
*** acormier has joined #openstack-nova12:46
*** edmondsw has joined #openstack-nova12:50
*** udesale has joined #openstack-nova12:51
*** jpena is now known as jpena|lunch12:51
*** r-daneel has quit IRC12:54
*** takedakn has joined #openstack-nova12:56
*** mvk_ has quit IRC12:58
*** weshay_PTO is now known as weshay13:01
*** takedakn has quit IRC13:01
*** r-daneel has joined #openstack-nova13:04
*** Guest18968 has quit IRC13:09
*** sree has joined #openstack-nova13:10
*** acormier has quit IRC13:10
*** acormier has joined #openstack-nova13:10
*** sree has quit IRC13:15
*** yamamoto has quit IRC13:17
*** yamamoto has joined #openstack-nova13:18
openstackgerritLajos Katona proposed openstack/osc-placement master: Random names for functional tests
*** yamamoto has quit IRC13:22
*** lbragstad has joined #openstack-nova13:23
*** vladikr has joined #openstack-nova13:23
*** mvk_ has joined #openstack-nova13:24
*** priteau has quit IRC13:27
*** amoralej is now known as amoralej|lunch13:30
*** awaugama has joined #openstack-nova13:38
* cdent has no brain, gives up and takes black panther break13:41
*** cdent has quit IRC13:42
*** rmcall has quit IRC13:43
*** acormier has quit IRC13:45
*** hrw has quit IRC13:45
*** pchavva has joined #openstack-nova13:45
*** hrw has joined #openstack-nova13:46
*** yamamoto has joined #openstack-nova13:48
*** liverpooler has joined #openstack-nova13:50
*** jpena|lunch is now known as jpena13:50
*** owalsh_afk is now known as owalsh13:52
*** mriedem has joined #openstack-nova13:57
mriedemhappy valentines day everyone <313:58
*** ttsiouts_ has joined #openstack-nova13:58
*** ttsiouts_ has quit IRC13:59
gibimriedem: same to you :)13:59
openstackgerritBence Romsics proposed openstack/nova master: Clarify 'capacity' in placement api-ref
gibimriedem: I left two questions in
*** andreas_s has quit IRC14:06
*** andreas_s has joined #openstack-nova14:06
*** andreas_s has quit IRC14:06
*** alexchadin has quit IRC14:08
*** alexchadin has joined #openstack-nova14:08
*** alexchadin has quit IRC14:08
*** alexchadin has joined #openstack-nova14:09
*** elmaciej has quit IRC14:09
*** hshiina is now known as hshiina|afk14:09
*** alexchadin has quit IRC14:09
*** alexchadin has joined #openstack-nova14:10
*** alexchadin has quit IRC14:10
*** alexchadin has joined #openstack-nova14:11
*** r-daneel has quit IRC14:11
*** Eran_Kuris has quit IRC14:11
*** alexchadin has quit IRC14:11
*** alexchadin has joined #openstack-nova14:11
*** alexchadin has quit IRC14:12
mriedemildikov: i realized yesterday that we never handled this
*** alexchadin has joined #openstack-nova14:12
mriedemildikov: but then realized, the only volume types that rely on that, don't support multiattach, so it's not really a problem we need to worry about right now14:12
*** alexchadin has quit IRC14:12
mriedemthings like NFS14:12
ildikovmriedem: good points, thanks for looking into it14:13
ildikovagreed on let's worry about it later14:14
ildikovdo you think it should be a PTG micro topic or we're good for now?14:14
mriedemunless someone really cares about making NFS support multiattach14:15
mriedemwhich i doubt14:15
mriedemwell, NFS + multiattach + snapshot14:15
ildikovyeah, let's not encourage people if don't have to :)14:16
openstackgerritMatt Riedemann proposed openstack/nova master: Test websocketproxy with TLS in the nova-next job
*** bhujay has quit IRC14:17
*** eharney has joined #openstack-nova14:19
*** lucasagomes is now known as lucas-hungry14:21
ildikovmriedem: I also didn't talk about shared_targets in the docs for now, wonder whether I should or not?14:22
*** alexchadin has joined #openstack-nova14:22
mriedemprobably not necessary; nova uses them for locking if the microversion is available14:23
mriedembut it's not required14:23
*** rmcall has joined #openstack-nova14:24
*** mdnadeem has quit IRC14:24
*** alexchadin has quit IRC14:26
ildikovok, fine, I'll leave it then14:28
*** udesale has quit IRC14:29
*** jafeha__ has joined #openstack-nova14:29
ildikovit's something more interesting for driver maintainers anyway so does not seem to be admin guide content, but wanted to check14:29
*** r-daneel has joined #openstack-nova14:29
*** udesale has joined #openstack-nova14:29
*** jafeha has quit IRC14:30
*** amoralej|lunch is now known as amoralej14:31
*** links has quit IRC14:33
*** elmaciej has joined #openstack-nova14:34
*** udesale has quit IRC14:36
*** READ10 has joined #openstack-nova14:38
*** hshiina|afk has quit IRC14:41
*** vladikr has quit IRC14:47
*** lbragsta_ has joined #openstack-nova14:49
*** bhagyashris has quit IRC14:50
*** nsingh has joined #openstack-nova14:50
*** nsingh has quit IRC14:51
*** lbragstad has quit IRC14:51
*** bhagyashris has joined #openstack-nova14:52
*** jobewan has joined #openstack-nova14:52
*** pooja-jadhav has joined #openstack-nova14:53
*** yamamoto has quit IRC14:53
*** yamahata has joined #openstack-nova14:54
*** yamamoto has joined #openstack-nova14:54
*** pooja_jadhav has quit IRC14:55
openstackgerritMerged openstack/nova-specs master: Fix missing blueprint URLs
*** niraj_singh has quit IRC14:55
mriedemdansmith: i won't be around for the cells v2 meeting today (if there was going to be one)14:56
*** yamamoto has quit IRC14:57
*** yamamoto has joined #openstack-nova14:57
openstackgerritMerged openstack/nova-specs master: List/show all server migration types
*** ragiman has quit IRC14:58
dansmithmriedem: ack14:59
dansmithI'm around for it, but I don't really have anything specific other than my reviews14:59
*** vladikr has joined #openstack-nova15:00
*** alexchadin has joined #openstack-nova15:00
*** moshele has quit IRC15:02
*** moshele has joined #openstack-nova15:03
*** alexchadin has quit IRC15:03
-openstackstatus- NOTICE: Due to a race in stable/queens branch creation and some job removals, Zuul has reported syntax errors for the past hour; if you saw a syntax error reported for "Job tripleo-ci-centos-7-ovb-containers-oooq not defined" you can safely recheck now15:05
belmoreirahave some placement related questions. dansmith, mriedem do you have some time?15:07
dansmithbelmoreira: I can try15:07
belmoreirathanks dansmith15:08
belmoreirawe are running local placements in ocata. Trying to consolidate them and get some performance numbers.15:08
belmoreiraBut, when nodes are retired (removed from the cloud) they continue to exist in placement. Are they eventually removed? not seeing how15:08
dansmithno they're not15:08
belmoreiraand they are included in the candidates?15:09
mriedemhmm, they probably are,15:09
dansmiththey would be from placement yeah, but scheduler would filter them out15:09
mriedemsimilarly, we don't have a direct hook to remove entries from the compute_nodes table15:09
belmoreirawe are seeing that placement continue to include them15:10
dansmiththis isn't really any different from our compute nodes befoe placement15:10
mriedembelmoreira: when you retire a node, do you manually remove it's entry from the nova.compute_nodes table?15:10
belmoreirathe difference is that the scheduling is now global15:10
belmoreiraremoving few hundred nodes could impact the performance of the entire cloud15:11
*** acormier has joined #openstack-nova15:11
mriedemwhat did you do when you removed them before this?15:11
belmoreiramriedem currently we remove entire cells, so we don't have this problem15:11
mriedemok so you drop the entire nova db15:12
mriedemfor that cell15:12
dansmithhmm, and compute nodes don't get mirrored up?15:12
*** moshele has quit IRC15:12
dansmithI can't remember how that works, but maybe they don't15:12
mriedemwe've talked before about adding a nova-manage CLI to remove a compute_nodes entry15:12
belmoreiradansmith what do you mean?15:12
mriedemthe host_mappings table would be pointing at hosts that no longer exist too15:13
dansmithbelmoreira: well, in cellsv1 a bunch of stuff gets mirrored up to the top db15:13
dansmithbelmoreira: I would have expected compute_nodes would too, but I don't really know I guess15:13
dansmithit doesn't matter though15:13
belmoreiradansmith compute nodes are only in the child cells15:13
dansmithgoing forward, pruning will have to happen when you remove stuff15:13
*** r-daneel has quit IRC15:14
belmoreirais there any work already done in that?15:14
dansmithso one way to do that, I think,15:14
mriedemwe likely could use a nova-manage CLI that removes the compute_nodes table entry in the cell, the host_mappings entry in the api db, and the resource_providers entry in placement15:14
*** ragiman has joined #openstack-nova15:14
dansmithwould be to go through host_mappings, find any mappings that don't have a cell, remove them and tell placement about it15:14
dansmithmriedem: yeah, but if you remove a cell first, you'll need a way to clean up the mess too15:14
tssuryamriedem : yes that would be a useful command15:15
mriedemcan we delete a cell mapping that has host mappings?15:15
mriedemdon't we have a fkey?15:15
mriedemcell_id = Column(Integer, ForeignKey(''),15:15
mriedem            nullable=False)15:15
mriedemand nullable=False15:16
mriedemyeah, so we have to remove host mappings if we remove a cell mapping15:16
dansmithyou can delete a cell with delete_cell and it will nuke all the host mappings15:16
dansmithso then you're really toast15:16
mriedemif you use --force yeah15:16
dansmithso we have delete_host15:17
*** AlexeyAbashkin has quit IRC15:17
dansmithwhich unmaps the compute node, and deletes the hostmapping15:17
dansmithso in both of those I guess we need to update placement15:18
mriedemi'm not sure why delete_host requires a --cell_uuid arg,15:18
mriedembut that's unrelated15:18
dansmithalthough nova-manage doesn't do any rpc or calls to other services right now, AFAIK, so we kinda need a --no-placement flag to each of those in case you're operating in an island15:18
*** lbragsta_ is now known as lbragstad15:19
mriedemnova-manage could use the scheduler report client and handle an http error response if it can't talk to placement15:19
dansmithyes, but you dont' want to proceed then,15:19
dansmithunless you tell it to15:19
dansmithbecause if we can't talk to placement, we can't proceed without corrupting the linkage (as we are now)15:20
dansmithso you need to fail in that case, unless instructed to ignore15:20
dansmithsomeone with placement down during an upgrade pruning a host wants to know that they're about to orphan some placement records15:20
mriedemcould just throw another --force on delete_host15:20
dansmithwe need it for both though15:20
dansmithdelete_cell and delete_host15:20
mriedemsure, we have --force on delete_cell already15:21
dansmithand delete_cell has --force already which means something else though15:21
dansmithit means "delete all the things, yeah", but this would be "and also, don't delete some stuff"15:21
mriedemit's just confusing ux15:21
*** cdent has joined #openstack-nova15:21
dansmithbut really, they're two meanings of force15:21
dansmithwhat we have now is really --recursive15:22
*** r-daneel has joined #openstack-nova15:22
*** lpetrut has quit IRC15:23
stephenfindansmith: This NUMA aware vSwitch spec is basically turning into a book. Be prepared!15:23
dansmithregardless, we need that thing and some appropriate flaggage to control intent15:23
dansmithstephenfin: noted15:23
openstackgerritMatt Riedemann proposed openstack/nova master: Remove old flavor_get_by_name db api method
*** tidwellr has joined #openstack-nova15:24
dansmithbelmoreira: tssurya: can one of you open a bug for this so I can point a patch at it?15:24
tssuryadansmith : sure15:24
belmoreiradansmith yes we can do it, we can also work on it if you prefer15:24
dansmithor that ;)15:25
mriedemdansmith: in the scenario you said, "someone with placement down during an upgrade pruning a host wants to know that they're about to orphan some placement records" - if i'm removing a compute host (node really here), are you just saying that you want to *not* delete the compute node if placement is down, right?15:25
*** mlavalle has joined #openstack-nova15:25
dansmithmriedem: right15:25
mriedemdefault behavior would be, we try to remove from placement and if that fails, we fail, unless instructed to ignore it15:25
dansmithmriedem: unless they say "I know what I'm doing" or something15:25
dansmithmriedem: yes15:26
mriedemthat still seems like --force should cover that case to me15:26
dansmithbut they may want --force to mean "delete all the hosts too",15:26
dansmithbut then they would be opting into "and orphan all the data in placement if it's down"15:26
dansmithso let's say you're deleting a cell while you're down for an upgrade, and that cell has 10 hosts15:27
mriedemwell, --force is a destructive option15:27
dansmithit is but it's not a "and corrupt my data"15:27
dansmithso you're down and deleting a cell with ten hosts15:27
dansmithyou run delete_cell --force, to delete all ten hosts and the cell record15:27
dansmithif placement is down,15:27
dansmithyou now just orphaned the placement records for those ten hosts,15:28
dansmithso when you come back up, ten providers in placement are no longer present, but will be returned by placement forever,15:28
dansmithand unless you know what the uuids were of those records you deleted,15:28
dansmithyou can't really clean that up15:28
openstackgerritMerged openstack/nova-specs master: Update Queens NRP spec to reflect reality
dansmithour current --force is poorly named.. it should be --recursive15:29
* cdent checks into getting dansmith an "astuteness merit badge"15:29
dansmithbecause --recursive means "and delete everything you have to to delete the cell"15:29
dansmithcdent: do you not get what I'm saying here?15:29
cdentNo, I totally get it. I'm not kidding. It is an astute observation.15:30
mriedemi get the scenario, i just, idk15:30
*** alexchadin has joined #openstack-nova15:30
cdentone way around the problem is "don't have placement down"15:30
cdentwhich is probably a good rule of thumb anyway15:30
*** lbragstad has quit IRC15:31
dansmithmriedem: in belmoreira's case, he would run --force to delete all the hosts, and then he would be in exactly the same position that started this conersation15:31
dansmithcdent: aye, it'd be better to not allow skipping the placement cleanup at all15:31
mriedemdansmith: did belmoreira say placement was down?15:31
*** lucas-hungry is now known as lucasagomes15:31
dansmithmriedem: no, but he'd want to use delete_cell --force to delete the cell and all the hosts,15:31
*** hongbin has joined #openstack-nova15:31
dansmithbut if placement was down when doing that (because maintenance), then we'd orphan all the records that he's currently saying we orphan, which is a problem15:32
dansmithi.e. you can't opt into the recursiveness without choosing the "and also corrupt the data if you need to"15:32
*** felipemonteiro has joined #openstack-nova15:33
*** dklyle has joined #openstack-nova15:33
mriedemso we could just say, you can't do that if placement is down, but sure i agree people will want a backdoor15:33
mriedemto bypass that15:33
*** lbragstad has joined #openstack-nova15:33
*** david-lyle has quit IRC15:33
*** slaweq has quit IRC15:33
dansmithright, which is why I'm saying we need another flag15:33
dansmithsince we already burned --force to mean --recursive, we need something else15:33
*** slaweq has joined #openstack-nova15:34
*** lajoskatona has quit IRC15:34
dansmithyeah, I said --no-placement above, but exactly.. something like that15:34
*** gibi has quit IRC15:34
mriedemapparently --force doesn't also remove the instances/mappings15:35
tssuryamriedem : I have a patch for that15:35
tssuryain gerrit15:36
dansmithyeah and I think I'm +2 on it already right?15:36
mriedemforce deleting instances from nova-manage is pretty scary15:36
mriedemyou're likely orphaning ports, volumes, etc15:36
tssuryamriedem : no --force does not delete instances15:36
mriedemjust the mappings?15:36
*** felipemonteiro_ has joined #openstack-nova15:36
tssuryamriedem : it only removes/cleans mappings for deleted instances15:36
tssuryajust mappings15:36
dansmithah, nm was thinking of the other one15:36
*** breton has joined #openstack-nova15:37
tssuryawe under no circumstance delete a cell if there are instances15:37
mriedemi'm saying delete_cell --force will fail today if there are instance mappings in the cell15:37
mriedemok this patch
tssuryamriedem : yes , my after merging my patch, delete_cell will fail if there any living instances15:38
*** slaweq has quit IRC15:38
mriedemok i see15:39
jaypipesstephenfin: <-- well, sheeet, let's just rewrite Nova in Golang (or Rust). :P15:39
*** links has joined #openstack-nova15:40
stephenfinjaypipes: If you can find me a tool to autoconvert one language to another, I'm all for it ;)15:40
*** felipemonteiro has quit IRC15:40
mriedemvish suggested java after joining oracle15:40
mriedemi know java, not go or rust15:41
jaypipesstephenfin: :)15:41
belmoreiraa concern that I also have with placement is that it doesn't know anything about cells, that is now an official think in nova15:41
mnaseri've been looking through the nova codebase throughout this morning -- we're running into an issue where nova fails to boot an instance "Invalid volume: Volume status must be available to reserve, but the status is attaching. (HTTP 400)"15:41
jaypipesI'd be up for a rewrite in C++.15:41
mriedembelmoreira: placement shouldn't know about cells15:41
belmoreirawe map projects to cells. Meaning that to handle a "small" VM the placement will return "all" nodes and then is the scheduler filtering to find the right cell15:41
stephenfinjaypipes: I do suggest checking out the sample patches I've thrown up through. An afternoons work and it really is minimally invasive15:41
belmoreirais there a way to get a more filtered answer from placement?15:42
jaypipesstephenfin: will do :)15:42
mnaserit seems like *somehow*, the cinder volume is in attaching state, and nova tries to reserve it, and so it fails after15:42
*** AlexeyAbashkin has joined #openstack-nova15:42
stephenfinjaypipes: and whatever you do, _don't_ listen to mdbooth! :D15:42
mriedemmnaser: this is pike 16.0.4?15:42
dansmithbelmoreira: aggregates15:42
dansmithbelmoreira: that's how you get a more filtered answer15:42
mdboothstephenfin: Generally good advice.15:42
* mdbooth wonders why specifically15:42
belmoreiradansmith placement aggregates?15:42
dansmithbelmoreira: aye15:42
mdboothstephenfin: Haha15:43
mnasermriedem: i believe the control plane is stable/pike, i think computes might be a bit older i'm not sure if they're 16.0.415:43
jaypipesdansmith: aggregate filtering isn't supported yet in allocation candidates...15:43
jaypipesdansmith: or at least, I don't think it is..15:43
*** felipemonteiro_ has quit IRC15:43
* jaypipes checks15:43
dansmithjaypipes: I'm saying that's the plan15:43
mnaserwe're running into a lot of weird conditions with heat15:43
belmoreiradansmith can they the used in Pike?15:43
dansmithjaypipes: we're not asking for it now anyway15:43
mdboothstephenfin: Spit shine that turd ;)15:43
dansmithbelmoreira: no15:43
*** felipemonteiro_ has joined #openstack-nova15:43
mnaserrather heavy heat usage of create volume + attach to instance15:43
mriedemmnaser: so this is boot from volume - is the volume provided to nova, or is nova creating the volume?15:43
mriedemi.e. does the failure happen in nova-api or nova-compute?15:43
mnasermriedem: boot from volume, volume created by heat and then given to bdm_v215:44
belmoreiradansmith are there any docs that describes how to use them? can find a lot of info about them15:44
mnaser <-- pretty much this heat template15:45
* mriedem has never used heat15:45
jaypipesI prefer a cold press for pain.15:45
dansmithbelmoreira: it's not going to solve your immediate problem, I'm just saying that aggregates, resources, and traits are the ways you can slice the result from placement (not cells)15:46
mriedemmnaser: ok likely getting here
mnasertechnically speaking heat is creating a volume and then giving it to nova (so boot from volume with provided volume id)15:46
jaypipesmnaser: well, that heat template isn't exactly complicated...15:46
dansmithbelmoreira: tbh, I'm not sure how to address your case of wanting to pre-filter or slice hosts by tenant15:46
openstackgerritAditya Vaja proposed openstack/nova master: remove IVS plug/unplug as they're moved to separate plugin
mnasermriedem: yeah that's as far as i got, but it seems that once it gets there, the volume is already in attaching state.. so something kicked off the attach somewhere15:47
belmoreiradansmith thanks, we will continue to dig into this15:47
mnaseri can see the volume stuck in 'attaching' state right now in cinder still15:47
mriedemmnaser: do you see anything in the cinder logs for that volume id before nova tries to reserve it?15:48
dansmithbelmoreira: let me think on it a bit15:48
mnasermriedem: good question, i should have thought of that15:48
belmoreiradansmith but for Pike the only option will be the have a "normal" aggregate per cell and let the filter scheduler do the job. concerned about the performance though15:48
mriedemmnaser: also this is likely something you want in pike
mriedemshould be in 16.1.015:49
dansmithbelmoreira: yeah, although you can have more schedulers and parallelize that load15:49
mnaseroh that's interesting, that might be related15:50
belmoreirayes, that's the plan. but in the worst case scenario each scheduler will need to go through 9k nodes15:50
*** itlinux has joined #openstack-nova15:50
belmoreirathat's why in ocata i decided to have local placements15:50
mriedemmnaser: yeah you might be hiting this
dansmithbelmoreira: aye, well, might need a hack for that case in pike until we think of something else15:51
mriedemso nova isn't actually trying to call cinder to reserve the volume15:51
openstackgerritEric Berglund proposed openstack/nova master: Add check for redundant import aliases
mriedemit's doing it's own conditional state check and failure15:51
mriedemalthough i don't know why the volume status would be 'attaching'15:51
*** tidwellr has quit IRC15:51
mnaserinterestingly enough i see a POST to /action15:52
*** esberglu has joined #openstack-nova15:52
dansmithbelmoreira: what's the reasoning for that split? was it for scheduling initially, or security or hardware ownership?15:52
*** tidwellr has joined #openstack-nova15:53
belmoreiradansmith manly specific hardware for a use-case15:53
dansmithbelmoreira: okay well that should really be a trait thing (although that's not done either :)15:54
mnaserok so i see a volume create in cinder-volume15:54
mriedemdansmith: well,15:54
belmoreiradansmith until now we isolate the hardware type per cell and allocate the right project to it15:54
mriedemwe do have traits-based scheduling,15:54
mriedemso you could have per-project flavors that have a trait for the hw you want for those projects15:55
dansmithmriedem: we're not passing it to placement yet right?15:55
mriedemwe are15:55
mriedemin queens15:55
dansmithin pike?15:55
dansmithright okay15:55
*** lajoskatona has joined #openstack-nova15:55
mriedemyeah not pike15:55
mriedemqueens solves all problems, i swear15:55
dansmithmaybe they could backport15:55
mriedemthe flavor plumbing stuff isn't versioned in anyway, the placement apis to use it are,15:55
cdentdansmith: traits in non-neste scenarios is working in queens last I checked15:55
mriedembut you could upgrade placement on the side, and backport the nova plumbing15:56
belmoreiradansmith i will have a look into that15:56
mriedemcdent: cern is on ocata15:56
mriedemqueens placement + ocata nova (+patches) should be ok15:56
mnaserok so15:56
mnaserit looks nova reserved the volume twice15:56
mriedem*should* == test in pre-prod please15:56
mnaserreserved it the first time succesfully, then shortly after it got reserved again15:57
belmoreiramriedem :)15:57
cdentmriedem: yeah, I know, it just sounded like dan was saying it's _still_ not done15:57
cdentand for some limited definitions of "done" it is, a bit15:57
dansmithcdent: well, tbh I didn't think we were even passing it in allocation_candidates :)15:58
dansmithI just hadn't been paying enough attention15:58
mriedemi know
mnaserso 12 seconds later, cinder got a second reserve request for the same volume and thats why it failed (i guess a volume goes to attaching state when its being reserved)15:58
* mnaser looks more15:58
mriedemmnaser: yeah it does,15:58
mriedemso i'm guessing the 2nd reserve request was made by nova-compute...15:59
mnaserim gonna investigate and see if that theory adds up15:59
mriedemthis is where compute starts doing the attach work
belmoreirathanks all16:00
mriedemmnaser: so what's probably happening is it's hitting an ocata compute16:00
mriedemwhich calls check_attach again, and fails16:00
mriedemmnaser: that's why the API has that min compute service version check, but that was busted with the multi-cell cells v2 stuff, unless you have melwitt's patch16:01
mriedemmnaser: so i think that's your issue16:01
mriedemcan you verify the compute it failed on is running ocata?16:01
mnasermriedem: that's what i'm double checing16:01
*** alexchadin has quit IRC16:02
mnasermriedem: i wonder if that was the wsgi registration bug we ran into a while back16:05
mnaser"MariaDB [nova]> select * from services where deleted_at is null order by version;" shows everything at 22 except for 3 control plane services16:05
mnaserwhich is at version 1616:05
mriedemmnaser: the problem is w/o that patch, the version comes back at 016:05
mriedemmnaser: your nova-api config is probably pointing [database] at cell0?16:06
mnasermriedem: oh you're right that's an old record, updated_at '2017-09-02 15:34:18'16:06
mriedemif your api config is pointing the [database] entry at nova_cell0, there are no nova-compute services in cell0, so it will return 0 as the version16:07
mriedemand hit that else block in the api code16:07
mriedemin other words, the api wasn't properly iterating non-cell0 cells for nova-compute service versions16:07
mnasermakes a lot of sense16:08
mriedemso that api patch will make your stuff to the right thing while you still have ocata computes16:09
mnaserwe shouldn't have ocata computes so im still tracking down that issue right now16:09
*** pcaruana has quit IRC16:13
*** yamamoto has quit IRC16:13
mnasermriedem: i think i might have another theory.. heat creates a volume, tries to create an instance, instance fails creation for some reason (but after reserving volume), it retries, and nova fails16:19
mnaserbecause im seeing a POST to /os-volumes_boot returning 40016:19
mnaserill have to dig up heat logs16:20
mriedemmnaser: yeah we apparently don't unreserve the volume in the api if something fails later in the api16:21
mriedemor in compute16:22
mnaserinteresting.. i see heat doing the create twice in logs 10 seconds apart, let me see if i can find an error there16:23
mnaserbut yeah, it's masking a failure16:23
mriedemdoes heat delete the failed instance (which reserved the volume) before trying to create the 2nd instance?16:23
mnaserdigging into that now16:23
mriedembecause that would likely be related to
*** lajoskatona has quit IRC16:24
mnaserok there was a scheduler error (not enough hosts), i'll dig into that16:24
*** suresh12 has joined #openstack-nova16:24
mnaser"Task pause timed out"16:25
mnaserand then it tried to create again16:25
mriedemi don't know what that is,16:25
mriedembut for the scheduler novalidhost,16:25
mriedemthe api would reserve the volume, rpc cast to conductor which rpc calls the scheduler for a host, that raises NoValidHost, and the instance is put into ERROR state, and the volume would still be reserved16:25
mnaserthe "Task pause timed out" is a heat thing so i'll look into why it didn't attempt to delete before retry16:25
mriedemso if heat deletes the failed instance, nova can clean up the volume, but that's probably where we need i think16:26
mnasermriedem: i think heat's normal behaviour is to just set the stack to failed16:27
mnaserand not retry16:27
*** yamamoto has joined #openstack-nova16:27
mnasergoogling "Task pause timed out" yields nothing so that's a nice start, but we're in heat problem land16:27
mriedemok, but even if heat did try to delete the instance before retrying, the 2nd attempt will likely fail because the volume is already reserved because of the bug that is trying to fix16:28
*** belmoreira has quit IRC16:28
*** psachin has quit IRC16:29
*** tidwellr has quit IRC16:29
cfriesendoes anyone know off the top of their head which operations would cause an instance with a qcow2-backed rootfs (that normally has the glance image file as a backing file) to get flattened?16:30
*** tidwellr has joined #openstack-nova16:30
mnasermriedem: yeah that's ideal, looks like melwitt was doing work on that, i can pick it up and work on whats missing if melwitt is busy with other things16:31
*** yamamoto has quit IRC16:31
mriedemmnaser: it's ready for review, just needs review16:32
mriedemthat patch has a long history16:32
mriedemof merging, breaking hte gate, getting reverted, repeat16:32
*** suresh12 has quit IRC16:32
mnasermriedem: there was a comment suggesting removing is None that's why i asked16:33
openstackgerritMatt Riedemann proposed openstack/nova master: Remove old flavor_access_get_by_flavor_id db api method
openstackgerritMatt Riedemann proposed openstack/nova master: Remove old flavor_destroy db api method
openstackgerritMatt Riedemann proposed openstack/nova master: Remove old flavor_get_by_flavor_id db api method
openstackgerritMatt Riedemann proposed openstack/nova master: Remove old flavor_get_by_name db api method
*** mriedem is now known as mriedem_afk16:34
*** slaweq has joined #openstack-nova16:34
*** slaweq_ has joined #openstack-nova16:36
*** imacdonn has quit IRC16:37
*** imacdonn has joined #openstack-nova16:37
*** tssurya has quit IRC16:39
*** slaweq has quit IRC16:39
*** alexchadin has joined #openstack-nova16:40
*** slaweq_ has quit IRC16:40
*** tssurya has joined #openstack-nova16:41
*** itlinux has quit IRC16:41
*** elmaciej has quit IRC16:42
*** krtaylor has joined #openstack-nova16:43
*** alexchadin has quit IRC16:44
cfriesenmriedem: do you know of any work in progress to deal with or ?16:47
openstackLaunchpad bug 1644248 in OpenStack Compute (nova) ocata "duplicate for #1583145 Nova incorrectly tracks live migration progress" [High,In progress] - Assigned to Matt Riedemann (mriedem)16:47
openstackLaunchpad bug 1644248 in OpenStack Compute (nova) ocata "Nova incorrectly tracks live migration progress" [High,In progress] - Assigned to Matt Riedemann (mriedem)16:47
*** hemna_ has joined #openstack-nova16:47
jaypipesmriedem_afk, dansmith: is pretty straightforward.16:50
*** READ10 has quit IRC16:51
*** tssurya has quit IRC16:51
*** itlinux has joined #openstack-nova16:52
*** slaweq has joined #openstack-nova16:56
*** ragiman has quit IRC16:56
*** rmcall has quit IRC16:58
dansmithmelwitt: mriedem_afk isn't around for the cells meeting, we talked to tssurya and belmiro earlier and neither are online now, so I assume we're canceling16:59
*** slaweq has quit IRC17:00
*** slaweq_ has joined #openstack-nova17:00
*** efried is now known as efried_rollin17:02
*** lpetrut has joined #openstack-nova17:03
*** rmcall has joined #openstack-nova17:03
*** READ10 has joined #openstack-nova17:04
*** slaweq_ has quit IRC17:05
*** jgriffith has joined #openstack-nova17:05
*** ameeda has quit IRC17:06
*** jgriffith has quit IRC17:07
*** AlexeyAbashkin has quit IRC17:07
*** rmcall has quit IRC17:08
*** beagles is now known as beagles_brb17:08
*** jgriffith has joined #openstack-nova17:08
*** dtantsur is now known as dtantsur|afk17:09
*** gyee has joined #openstack-nova17:11
*** chyka has joined #openstack-nova17:12
*** yamamoto has joined #openstack-nova17:12
cdentjaypipes: if you're chasing easy wins:
*** AlexeyAbashkin has joined #openstack-nova17:15
*** alexchadin has joined #openstack-nova17:16
*** pcaruana has joined #openstack-nova17:17
*** yamamoto has quit IRC17:17
*** AlexeyAbashkin has quit IRC17:20
*** links has quit IRC17:21
*** slaweq has joined #openstack-nova17:22
*** tssurya has joined #openstack-nova17:24
*** tssurya has quit IRC17:27
*** yamahata has quit IRC17:31
*** beagles_brb is now known as beagles17:33
*** mgoddard_ has quit IRC17:37
*** felipemonteiro__ has joined #openstack-nova17:45
*** jmlowe has quit IRC17:46
*** felipemonteiro_ has quit IRC17:48
*** lpetrut has quit IRC17:48
*** jmlowe has joined #openstack-nova17:49
*** alexchadin has quit IRC17:50
openstackgerritMerged openstack/python-novaclient master: Updated from global requirements
*** moshele has joined #openstack-nova17:52
*** arvindn05 has quit IRC17:52
*** sshwarts has quit IRC17:54
*** rmcall has joined #openstack-nova17:56
*** lpetrut has joined #openstack-nova17:56
*** sambetts is now known as sambetts|afk17:56
*** moshele has quit IRC17:58
*** jmlowe has quit IRC18:00
*** derekh has quit IRC18:03
*** hoonetorg has quit IRC18:05
*** yamahata has joined #openstack-nova18:06
*** mvk_ has quit IRC18:08
mnaserso it looks like this bug wasn't visible because we had old disabled but inactive pre-pike compute nodes18:09
mnaserremoving them made the minimum pike and landed us in this situation18:09
*** sridharg has quit IRC18:10
openstackgerritDan Smith proposed openstack/nova-specs master: Add placement-req-filter spec
*** mgoddard_ has joined #openstack-nova18:16
*** jackie-truong has joined #openstack-nova18:16
mnaser"The missing 'reserve_volume' call is added to the BFV flow. In case of build failure the volume will be locked in 'attaching' state until the instance in ERROR state is cleaned up." -- i guess the problem here is that it doesn't account for scheduling failures :x18:16
*** jpena is now known as jpena|off18:17
*** cdent has quit IRC18:21
*** hoonetorg has joined #openstack-nova18:22
*** suresh12 has joined #openstack-nova18:24
*** psachin has joined #openstack-nova18:26
*** jmlowe has joined #openstack-nova18:26
*** yamamoto has joined #openstack-nova18:27
*** suresh12 has quit IRC18:29
*** felipemonteiro__ has quit IRC18:29
openstackgerritMerged openstack/nova master: Updated from global requirements
*** alexchadin has joined #openstack-nova18:32
*** yamamoto has quit IRC18:32
openstackgerritDan Smith proposed openstack/nova-specs master: Add placement-req-filter spec
mriedem_afkmnaser: i want to say that "so it looks like this bug wasn't visible because we had old disabled but inactive pre-pike compute nodes" was also a bug fix18:32
*** mriedem_afk is now known as mriedem18:32
*** openstackgerrit has quit IRC18:33
mriedemmnaser: oh nvm we excluded forced_down services, but we do include disabled services in that min version check18:33
mriedemi was thinking of this so it looks like this bug wasn't visible because we had old disabled but inactive pre-pike compute nodes18:34
*** suresh12 has joined #openstack-nova18:34
mnasermriedem: yeah so right now i'm not too sure why this occurs because it looks like if it fails to schedule, it means when nova attempts to delete it, it does a local delete, which calls '_local_cleanup_bdm_volumes'18:35
mnaseror maybe im wrong?18:36
*** alexchadin has quit IRC18:36
mnaserbecause if it fails to schedule, its in cell0 so deletes will be a local delete based on what i think i understand from the code18:36
mriedemmnaser: presumably that's why melwitt has
mriedemi think the problem is that we do find the error'ed instance in cell0,18:38
mriedemand delete it in the api, but return early and don't cleanup bdms or ports18:38
mriedemand then we exit here
mnasermriedem: i dont know if logically this is clean but i was thinking maybe cleanup bdms in the _bury_in_cell0 codebase?18:42
mnaserbecause that will also release the volume right after failure, rather than until the volume is deleted18:42
mriedemyeah idk, we end up with a lot of cleanup volume logic strewn all over nova then18:43
mriedemb/c we already have some of this in nova-compute too18:43
mriedemand api already18:43
mnaseryeah thats what was making it a bit iffy for me is the fact that now multiple components are handling reserve/cleanup (which is probably how we ended up in this place)18:44
mnasermriedem: i think what you mentioned isn't the case, because that 'return' happens if there is an, and in this case, the error'd instance probably doesnt have one set if it failed to be scheduled18:47
mnaserbut thats what i think based on what little understanding of the code18:48
melwittif the instance fails to schedule, it shouldn't have done anything with the volumes yet, right?18:48
mnasermelwitt: the api layer calls cinder reserve volume when it gets the request18:49
melwittmnaser: oh :\18:49
mnasermelwitt: not sure if your patch fixes it (or if its technically another bug), but this was my write up -
*** jpena|off is now known as jpena18:50
mriedemwe only go down that route if the instance doesn't have a host yet18:51
mnasermriedem: my bad, the comment above it tricked me18:51
mriedemyeah, it's confusing, the "Normal delete should be attempted." means "skip this weird block"18:51
mriedemso i think that's likely the issue, is we get the instance from cell0, and delete it in this block of code and return w/o dealing with volumes/ports18:52
mriedemgranted we shouldn't have any ports if it's never been scheduled to a host18:52
mriedemas ports are created/bound in compute18:52
mriedembut volumes is definitely a gap here18:52
mnasermriedem: i think so too, because if i follow the rest of the code, it should technically do a local_delete which does clean up bdms18:52
*** yamamoto has joined #openstack-nova18:52
*** yamamoto has quit IRC18:52
*** tesseract has quit IRC18:54
mnasercatch-22... if compute reserves volume, it means api layer won't give the error right away.. if api reserves volume, get an error right away but have to get it cleaned up later18:55
*** lucasagomes is now known as lucas-afk18:56
*** eharney has quit IRC18:57
*** jpena is now known as jpena|off18:59
mnaserim not even sure if this issue exists in queens19:01
mnaserlooks like it creates an attachment rather than reserve a volume19:01
*** claudiub has quit IRC19:02
mriedemdansmith: so i think i get it, but
mriedemmnaser: it does exist in queens, because we don't delete the attachment, same thing as not unreserving19:03
mriedema volume with an empty attachment is in 'attaching' status just like reserve19:03
dansmithmriedem: in my playing around code, I have a note to self to go find that key and see if I can use it :)19:03
melwittmnaser: with my patch, I think you'd still see the volume stuck in 'attaching' until you delete the instance19:03
mriedemmelwitt: yes you would b/c we don't cleanup volumes in conductor19:04
melwittwithout the patch, even after you delete the instance, the volume would be stuck. with the patch, the volume should be released19:04
melwittafter deleting19:04
mnasermelwitt: so _has_been_scheduled(instance) would return true (because vm_state == ERROR), which means that we would enter that `if not has_been_scheduled:` conditional.  however, as mriedem pointed out, in your patch, line #1809 it returns so it doesn't follow the rest of the code19:06
mnaserwhich means it never makes it to the code under it which does clean ups etc (if i understand correctly)19:06
*** suresh12 has quit IRC19:06
mriedemit should be fairly easy to have a functional test to recreate that19:07
*** alexchadin has joined #openstack-nova19:07
melwittmnaser: if _has_been_scheduled is True, then it would not enter 'if not _has_been_scheduled'19:07
*** suresh12 has joined #openstack-nova19:07
mriedemthe functional tests that melwitt updated in that patch likely aren't doing boot from volume19:08
mnaserthese programming 101 mistakes19:08
* mnaser needs coffee19:08
*** lpetrut has quit IRC19:10
*** suresh12 has quit IRC19:10
melwittI have boot from volume unit tests, but need to check if any of the functional tests covering boot from volume could be enhanced to check this19:10
*** suresh12 has joined #openstack-nova19:10
melwittoriginally I hadn't updated any functional tests in the patch, something changed recently to where we needed to manually start the consoleauth service. not yet sure what caused that change19:11
*** tidwellr has quit IRC19:11
mriedemwe don't have many bfv functional tests, but a couple19:11
mriedemlocal delete will try to delete console auth entries19:11
*** tidwellr has joined #openstack-nova19:11
mriedemsince you weren't going down that path before your change, that's why those tests didn't need it19:12
mriedemhere is an easy example test
melwittyeah, I mean, in previous revisions I didn't need it19:12
mnaseri think `test_delete_error_state_with_no_host` covers this case, which makes sure that local_delete is actually being called (and local delete does clean up bdms)19:12
mriedemmnaser: that's a unit test19:13
mriedemwe really should have a functional regression test for this, given the various service interactions and cell0 stuff19:13
mnaseris there an example of functional regression tests involving other services (i.e. cinder)19:14
mnaseri dont mind trying to write one if that helps19:14
mnasermelwitt: if you have other things at hand, i can write a functional regression test for this right now19:16
*** suresh12_ has joined #openstack-nova19:17
melwittmnaser: sure, sounds good, thanks for the help19:17
mnaserok perfect, i'll work on it locally and then i can tack it on top of the change19:17
*** suresh12 has quit IRC19:20
mriedemideally the regression test patch comes before the fix,19:21
mriedemto show that the volume is still 'attaching' even after we delete the instance19:21
mriedemthen mel's patch goes on top of that and changes the test to show the volume is properly unreserved19:21
*** harlowja has joined #openstack-nova19:21
mriedemthe cinder fixture we have should be able to give you the volume state tracking and API support for figuring that out19:21
mnaserwell this is nice to have, heh19:25
*** openstackgerrit has joined #openstack-nova19:26
openstackgerritDan Smith proposed openstack/nova-specs master: Add placement-req-filter spec
*** READ10 has quit IRC19:31
mriedemmnaser: yeah you could probably build on that with a bfv scenario, and nix the shelve offload part19:33
*** jackie-truong has quit IRC19:33
*** dklyle has quit IRC19:33
*** mgoddard_ has quit IRC19:34
*** moshele has joined #openstack-nova19:34
openstackgerritDan Smith proposed openstack/nova-specs master: Add placement-req-filter spec
mriedemdansmith: couple small docs issues and then i'm +219:37
*** ralonsoh has quit IRC19:38
mriedemi really like the idea of using this for multiattach bfv19:38
mriedemassuming people don't hate the idea of storing the compute driver capabilities as traits19:39
mriedemwe can also use it for tagged bfv19:39
mriedemwhich artom should like19:39
*** alexchadin has quit IRC19:39
* artom would indeed19:40
artomI hope I wouldn't be the only one19:40
artomIt seems that sort of thing has come up for other problems19:40
dansmithmriedem: jaypipes was also saying this will let us clean up the pci request -> nets mapping thing, which sounds nice19:40
dansmithI have to go dig back into that stuff to see though19:40
openstackgerritDan Smith proposed openstack/nova-specs master: Add placement-req-filter spec
openstackgerritJackie Truong proposed openstack/nova master: Add trusted_certs to instance_extra
openstackgerritJackie Truong proposed openstack/nova master: Add trusted_certs object
openstackgerritJackie Truong proposed openstack/nova master: Implement certificate_utils
openstackgerritJackie Truong proposed openstack/nova master: Add trusted_image_certificates to REST API
*** suresh12_ has quit IRC19:47
*** moshele has quit IRC19:49
mriedemjackie should be in irc19:49
openstackgerritmelanie witt proposed openstack/nova-specs master: Re-propose convert consoles code to use objects framework
*** efried_rollin is now known as efried19:50
openstackgerritMatt Riedemann proposed openstack/nova master: Remove old flavor_get db api method
*** yamamoto has joined #openstack-nova19:53
dansmithmriedem: related to ^, I'll remove aggregate-in-cell-db compatibility before I add the methods to AggregateList that I need19:54
dansmithjust so you don't do it if you were looking for deprecations to remove19:54
mriedemdansmith: i had just been slowly working on this list
mriedemlast thing i removed was the flavor compat19:59
mriedemor maybe it was keypairs, don't remmeber19:59
dansmithmriedem: okay well, I've got the two aggregate ones in my sights19:59
mriedemok wfm,19:59
*** eharney has joined #openstack-nova19:59
mriedemwe can definitely remove these old newton compat things19:59
*** felipemonteiro has joined #openstack-nova20:02
openstackgerritmelanie witt proposed openstack/nova master: Add periodic task to clean expired console tokens
openstackgerritmelanie witt proposed openstack/nova master: Use ConsoleAuthToken object to generate authorizations
openstackgerritmelanie witt proposed openstack/nova master: Convert websocketproxy to use db for token validation
*** yamamoto has quit IRC20:02
*** tidwellr has quit IRC20:03
mriedemanyone want to send this home and the one below it? - the cinder admin guide patch for multiattach is dependent on this20:03
mriedemi think we should probably backport that to queens too20:04
*** amoralej is now known as amoralej|off20:07
*** tidwellr has joined #openstack-nova20:07
mnaserhey .. is this a bug in the cinder fixtures? .. shouldn't it modify volume['attach_status'] and not volume['status']?20:07
mnaserso it should be volume['status'] = 'in-use' if has_attachment else 'available'20:07
mnaservolume['attach_status'] = 'attached' if has_attachment else 'detached'20:08
mriedemjogo: can you remove your -2 on this?
mnasermy little functional test so far is failing because: HTTP exception thrown: Invalid volume: volume 'f8580fcb-b22d-4e50-b69a-d3d91f8f3a6b' status must be 'available'. Currently in 'detached'20:08
mriedemmnaser: yes20:08
mnaserok, ill fix that first20:08
mriedemmnaser: same problem here
melwittmnaser: is that what I proposed here?
mnasermelwitt: exactly that20:09
mnaserill rebasemy work on that, thank you20:09
mriedemlemme look, we could rebase quick20:09
melwittI ran into the same problem recently working on func tests20:09
mnasermelwitt: do we want to add in-use for volume status too?20:09
mnaserbut that might be a bit more work20:10
mnaserbecause im not sure what the status is when using multiattach (in-use or available?)20:10
melwittmnaser: we might. I didn't try to go further with changing the fixture20:10
mriedemi just left that comment20:10
mriedemif there are attachments, the status should be at least 'in-use'20:11
melwittonce it's attached, it should be 'in-use' from what I understand20:11
mriedemit might not be technically correct, but it's more correct than saying 'available'20:11
mnaseryeah, makes sense20:11
mriedemwith the new fixture, there can be 'empty' attachments, which puts the volume status in 'attaching' status, and the volume doesn't go to in-use until the attachment is 'completed'20:11
mriedemin the old fixture, completed means calling os-attach20:11
mriedembut we likely don't need to worry about those details in these fixtures20:12
melwittmnaser: you want to add 'in-use' and 'available' to that patch while you're rebasing?20:12
mnasermelwitt: sure, i can do that20:12
melwittcool, thanks20:12
*** david-lyle has joined #openstack-nova20:12
mriedemactually, the loop above deals with this already20:13
mriedemso likely don't need a change20:13
mriedemi don't even know why we have this block of code now20:14
mriedemthe loop above handles both of those status values20:14
openstackgerritPeter Hamilton proposed openstack/nova-specs master: Add support for certificate validation
openstackgerritJay Pipes proposed openstack/nova-specs master: Account for host agg allocation ratio in placement
*** alexchadin has joined #openstack-nova20:21
*** amodi has joined #openstack-nova20:24
*** alexchadin has quit IRC20:25
openstackgerritEric Berglund proposed openstack/nova master: WIP: PowerVM Driver: Snapshot
*** felipemonteiro_ has joined #openstack-nova20:27
*** felipemonteiro has quit IRC20:30
cfriesenmriedem: are you aware of any work going on as far as live migration monitoring?  (related to the broken progress monitor)20:30
mnasersorry for the questions, is there a way to turn on debug logging for functional tests?20:31
mriedemmnaser: run with OS_DEBUG=True20:32
mnasermriedem: thank you20:32
mriedemcfriesen: nope20:32
mriedemjaypipes: thanks for starting - a few thoughts inline20:40
mriedemwill definitely want to talk with dansmith about the new upcall20:41
*** suresh12 has joined #openstack-nova20:43
*** suresh12 has quit IRC20:48
mriedemefried: i moved back to queens and completed it20:49
mriedemlet us never speak of it again20:49
efriedSomeone still owes a NRP-in-alloc-cands spec for Rocky.20:49
efriedI think I'm still on the hook for that.20:50
mriedemyou just smelled it20:50
mriedemmelwitt: you gonna take the nova meeting baton tomorrow afternoon?20:52
*** mchlumsky_ has joined #openstack-nova20:52
openstackgerritMatt Riedemann proposed openstack/nova master: Remove old flavor_get_all db api method
dansmithgood gawd this aggregate de-compat is a mess20:54
*** mchlumsky has quit IRC20:54
*** suresh12 has joined #openstack-nova20:54
*** awaugama has quit IRC20:57
*** mgariepy has quit IRC21:00
*** suresh12 has quit IRC21:00
jaypipesmriedem: yup21:03
jogomriedem: done21:05
*** suresh12 has joined #openstack-nova21:07
*** tidwellr has quit IRC21:08
*** tidwellr has joined #openstack-nova21:08
openstackgerritJay Pipes proposed openstack/nova-specs master: Support member_of param for allocation candidates
*** esberglu has quit IRC21:08
*** esberglu has joined #openstack-nova21:09
*** liverpooler has quit IRC21:09
*** acormier has quit IRC21:11
*** acormier has joined #openstack-nova21:12
*** psachin has quit IRC21:12
*** esberglu has quit IRC21:13
*** suresh12 has quit IRC21:14
mriedemjogo: thanks, and o/21:15
*** acormier has quit IRC21:16
jogomriedem: o/ glad gerrit lets you drop a -2 even if you can't -2 new things21:17
*** suresh12 has joined #openstack-nova21:18
*** slaweq has quit IRC21:18
mriedemjogo: you can always come back21:20
*** felipemonteiro_ has quit IRC21:20
*** felipemonteiro_ has joined #openstack-nova21:20
jaypipesmriedem, dansmith, efried: don't forget about this other aggregate-related spec...
jaypipesdansmith: is ready for you to link to in your spec.21:21
efriedjaypipes: I can look again; maybe my opinion has changed.21:21
jaypipesdansmith: working on the spec for that one now.21:21
efried(the "opinion" that I don't understand most of it)21:21
dansmithjaypipes: thanks21:21
jaypipesefried: np. was just pointing out all my agg-related stuffs...21:21
*** esberglu has joined #openstack-nova21:22
efriedNo need to get all aggro21:22
dansmithjaypipes: the one I really need is the bp for adding aggregates to the allocation_candidates api21:22
dansmithah cool21:23
jaypipesdansmith: yup, pushed it up about 20 minutes ago21:25
dansmithjaypipes: yeah, found it thanks21:25
dansmithefried: yawna look at my replies before I push up this revision with the linkages?21:26
mriedemgod i see i need to read again21:26
efrieddansmith: You mean on ^ ?  Looking...21:27
*** pcaruana has quit IRC21:28
mnaserfunctional tests done (and failing), crossing fingers that putting the change on top fixes things21:29
jogomriedem: :)21:29
openstackgerritDan Smith proposed openstack/nova master: Remove deprecated aggregate DB compatibility
jaypipesaggregates are just the topic du jour I guess.21:31
efrieddansmith: Can you please clarify the traits thing for me?  What I suggested in L119 was yes, "bulk adding traits to large swaths of compute nodes" - but instead of getting the trait to the API call by pulling it from the flavor, you get it from the RequestSpec (or whatever) and fold it into any traits provided from the flavor.21:32
efrieddansmith: Is that what you're suggesting could be a solution for boot-from-volume in the future?21:32
dansmithefried: clarify how a trait in a flavor doesn't help?21:32
dansmithefried: ha21:32
dansmithefried: you could have flavors you allow BFV from but that aren't dedicated to bfv21:32
efrieddansmith: Getting the trait from not-flavor to avoid Skittles.21:32
*** suresh12 has quit IRC21:33
*** pchavva has quit IRC21:33
dansmithefried: so if you want to make sure that BFV requests using generic flavors don't get routed to any old host, you might want to slap a trait on those requests so that you can send them to the right place21:33
*** suresh12 has joined #openstack-nova21:33
mriedemmy idea was that the pre-placement filter thing dan's spec is adding, would shove a trait into the flavor embedded in the reqspec b/c that's currently how we get the traits from the user to the scheduler to placement21:33
efriedRight, so we're merging traits-from-flavor with traits-from-elsewhere21:34
dansmithwe could do that too21:34
mriedemso if we know we're doing bfv on a multiattach volume (or a tagged bdm), this code puts a CUSTOM_COMPUTE_SUPPORTS_MULTIATTACH on the flavor within the reqspec21:34
mriedemefried: yeah21:34
efriedmriedem: Not on the flavor - on the allocation_candidates request21:35
mriedemwe could always do something else like the pre-placement filters add RequestSpec.system_traits (like system_metadata) or something21:35
mriedemefried: but we have to get them to the allocation candidates request somehow21:35
dansmithefried: on the flavor in the reqspec if we want it to be honored for a resize or migration kinda thing21:35
efriedDo we make other "modifications" to flavor data between when we pull it from the flavor db and when we attach it to the instance?21:36
openstackgerritMatt Riedemann proposed openstack/nova master: Remove old flavor_create db api method
dansmithefried: we suggested doing this for zero-root-disk BFV things I think21:36
dansmithefried: we also probably need traits to be stored in the reqspec proper for when the user asked for them right/21:37
mriedem ?21:37
dansmithmriedem: no, mine below that21:37
efriedMy point is that it feels kinda weird for me to say 'boot an instance with this flavor, that has traits X,Y,Z' and then when I look at the flavor on the instance later, it also has traits A,B,C.  I'm like where tf did those come from?21:37
mriedemefried: you wouldn't see those21:38
efriedOkay, then I'm cool.21:38
efriedIf they're on the RequestSpec and not on the flavor per se.21:38
mriedemthe instance.flavor would show the public facing flavor you used to create the instace21:38
*** mvk_ has joined #openstack-nova21:38
dansmithinstance.flavor not reqspec.flavor21:38
mriedemright, they'd be on the flavor embedded within the request spec, which doesn't get exposed out of the api today21:38
mriedemGET /servers/{id}/detail shows the flavor embedded in the instance21:39
dansmithalthough we probably create instance.flavor from reqspec.flavor21:39
mriedemwe don't21:39
*** ansiwen has quit IRC21:39
*** mdbooth has quit IRC21:39
efriedSo yeah.  dansmith What I'm saying is that this is an alternative that should be at least documented in the Alternatives section; *and* some more explanation along these lines is needed for the reader to make sense of the statement on L47 that mentions traits for the first/last/only time.21:40
dansmithit doesn't really matter anyway, as that'd be a conversation for how we're going to do a thing with traits,21:40
dansmithunrelated to the general idea of augmenting the request to placement21:40
dansmithefried: I'm not sure I get what is an alternative21:40
dansmithefried: traits for the tenant isolation is not an option, IMHO21:41
dansmithif you mean it's an alternative to the "and also maybe later we could do a bfv thingy", then agree, but it's not really the subject of this spec21:41
mriedem"Although not covered by this spec directly, the mechanism added here21:41
mriedemwould enable other such request-limiting filters in the future. For21:41
mriedemthat seems pretty clear that this is a framework type thing21:42
mriedemthe spec is dealing with the specific tenant aggregate problem,21:42
dansmiththis could have been two specs, but I felt it was way better to not have a tiny spec for each21:42
mriedembut is generic enough that we can build on it for other problems21:42
mriedemother problems like bfv21:42
dansmithmaybe I'm missing what efried is saying is an alternative though21:42
*** jackie-truong has joined #openstack-nova21:43
*** esberglu has quit IRC21:43
mriedemi think he's saying, if i have A, B, C in aggregate X, rather than model it as an aggregate, the resource providers for A,B,C would have an X trait21:43
mriedemwhich is a PITA21:43
mriedemso i agree we don't want to do that21:43
dansmithfor tenant isolation for sure21:44
efrieddansmith: I agree that [traits for tenant isolation if you have to specify the traits in flavors] is not an option - but you describe it in Alternatives and explain why it's not an option (Skittles).  IMO, [traits for tenant isolation where we merge the traits from the RequestSpec that we got from conf just like you're talking about doing with aggregates] *may* also not be viable, but it doesn't suffer from your main objec21:44
efriedtherefore at least closer to being a viable option, and should therefore be described in the Alternatives section, and dismissed with an explanation as to why it's being dismissed if indeed there's a good reason for that.21:44
mnasermriedem: good call on suggesting functional test.  while the codebase seems to actually not stop there, it looks like it's not getting any bdms in _delete() .. BlockDeviceMappingList(objects=[])21:44
efriedAnd then you get the added bonus of being able to refer to that model from L46-7, because otherwise L46-7 doesn't make sense in the context of the rest of the spec.21:45
mnaseris it possible that bdms are created later? :<21:45
dansmithefried: ah okay, that's what I'm saying is also not an option IMHO, but for other reasons than flavor explosion21:45
mriedemmnaser: i bet i know why,21:45
mriedemmnaser: i bet the bdms aren't buried in cell021:45
efrieddansmith: Cool, please say that in the spec and I'm good.21:45
mnasershould they be buried in cell0?21:45
mriedemmnaser: just a guess, but that's my guess21:45
dansmithefried: because we have to label each compute individually with the trait and I think that's what aggregates are for, but yep, I'll put it in as a slightly different thing.. just for you :)21:46
efriedThanks dansmith.  Kisses.21:46
mriedemit's a valentines day miracle21:46
efrieddansmith: Having to label each compute individually with the trait - what I'm confused about on L46-7 is why that argument doesn't apply for BFV too.21:46
dansmithefried: well, the way I phrased that is silly, since that's traits all up and down, but let me say it in the spec and then you can argue about it, if it's clear21:47
mriedemefried: for one thing, nova-compute can report the BFV traits capabilities automatically, the operator / admin doesn't have to do that for every tenant21:47
mnaserselect * from block_device_mapping => Empty set21:48
mnaseri guess i'll get them added..21:48
efriedmriedem: It can do that for host aggs too, though, nah?21:48
dansmithmriedem: and compute can report aggregates automaticaly21:48
mriedemthat's an up-call for one thing21:48
mriedemnova-compute would have to get the aggregates that the host is in, find the tenant_id meta on those, and convert that to traits on the RP21:49
mriedemi agree we could do that,21:49
efriedmriedem, dansmith: Oh, so yeah, that was my point: we're talking about automatically mirroring host aggs as placement aggs; in the alternative I describe above, we would be instead automatically mirroring host aggs as traits on the compute node RP.21:49
mriedembut it's an up-call21:49
mriedemthe bfv trait is much more clear to me as a 'trait',21:49
dansmithefried: right, but then you create a tenant trait, apply to all computes. if you want to reassign a cell to a different tenant, you remove/reapply against all those computes21:50
mriedemthe tenant isolation thing is very much less a 'trait' in my head21:50
dansmithefried: compared with:21:50
efried...same but for agg associations21:50
dansmithefried: this cell is aggregate cell-foo, and add "tenant bar is allowed" to the cell-foo aggregate,21:50
mriedemmnaser: oh i know the problem,21:50
dansmithefried: then you can reassign a cell to a tenant in one place instead of mass add/remove of the other trait21:50
efriedSorry, I don't get the difference.21:51
dansmithefried: hangout?21:51
efriedIf the trait is CUSTOM_CELL_FOO21:51
mriedemmnaser: so in conductor schedule_and_build_instances, we call the scheduler for a host and if we get one, we look up what cell that host is in and create the instance and bdms in that cell,21:51
efrieddansmith: Sure21:51
mriedemmnaser: if scheduling fails, we create the instance in cell0,21:51
mriedembut we don't create the bdms in cell021:51
mnaserbut not the bdms21:51
mriedemmelwitt: ^21:51
dansmithefried: required, edleafe mriedem jaypipes optional21:51
mnaseri guess i'll look into making it create bdms in cell021:52
mriedempants required?21:52
dansmithmriedem: yes21:52
mriedemmnaser: that or we unreserve the volumes for the bdms right there in conductor...21:52
mriedemso that the api doesn't have to delete them locally21:52
mriedembut then we get the "how many different nova services does it take to detach a volume" problem21:52
mnaserthat means that the volume will become available right away21:53
mnaserbut yeah21:53
mriedemmnaser: melwitt: left notes on - thanks for the testing21:56
*** rcernin has joined #openstack-nova22:00
openstackgerritJackie Truong proposed openstack/python-novaclient master: Microversion 2.61 - Add trusted_image_certificates
mnasermriedem: melwitt i can do both (though storing bdms seems easier).. i'll leave the decision up to you22:01
*** slaweq has joined #openstack-nova22:04
*** rmcall has quit IRC22:05
*** slaweq has quit IRC22:11
mriedembefore all of this we'd have the bdms created in the api along with the instance (before cellsv2 happened), so might as well just go with that for starters and see how it goes22:11
mriedemnot api, the 'nova' db, but the nova-api service would create them in the nova db22:12
melwittmriedem: sure, I can take the meeting baton tomorrow. let me know if I need to update the agenda or if you already did22:13
mriedemi have not updated the agenda22:13
melwittk. I shall agenda then22:13
melwittand that sucks about the bdms thing.22:15
mriedemright so we'll likely need to stagger these patches for sanity22:15
openstackgerritDan Smith proposed openstack/nova-specs master: Add placement-req-filter spec
dansmithefried: ^22:15
mriedemregression test, do something about bdms in conductor on scheduling failure, then the local delete api patch22:16
efrieddansmith: ack22:16
melwittyeah, I was about to say ... should mnaser take that issue as a separate patch?22:16
mriedemmelwitt: i think so22:16
mriedemwe're going to want to backport all of this to ocata too22:16
melwittgood times22:16
mriedemthis is the most fun i've had all week22:16
mnaserI have 3 patches right now, fixing fixtures, functional tests and melwitt final patch on top. I’ll add the bdm creation in the stack.22:18
mnaserI’ll make sure to keep them22:18
mnaserUnder the same topic22:18
mriedemmnaser: thanks22:18
mnaser(But first supper)22:18
*** acormier has joined #openstack-nova22:19
*** suresh12 has quit IRC22:20
*** dave-mccowan has quit IRC22:20
mlavallemelwitt: I added some topics to the Nova PTG etherpad. With those topics, I figured 2 hours. Right now I have the Neutron / Nova session from 3 to 5 on Thursday. When you have a chance let me know what you think. No need to rush22:20
mlavallefro 3 to 5 in the Neutron etherpad^^^^22:21
mlavallebut I am keeping that afternoon open until you and I firm up the times22:21
melwittmlavalle: k, probably will be fine. just have to sync up with the ironic team. I'll let you know22:23
*** acormier has quit IRC22:24
*** tidwellr has quit IRC22:31
mlavallemelwitt: :-)22:31
*** threestrands has joined #openstack-nova22:31
*** suresh12 has joined #openstack-nova22:31
openstackgerritMatt Riedemann proposed openstack/nova master: Scheduler multiple workers support
*** claudiub has joined #openstack-nova22:43
*** claudiub has quit IRC22:43
*** artom_ has joined #openstack-nova22:48
*** abalutoiu_ has joined #openstack-nova22:49
*** felipemonteiro__ has joined #openstack-nova22:51
*** liverpooler has joined #openstack-nova22:51
*** abalutoiu has quit IRC22:51
*** itlinux has quit IRC22:53
openstackgerritMatt Riedemann proposed openstack/nova master: Check for leaked server resource allocations in post_test_hook
*** artom has quit IRC22:53
*** mriedem has quit IRC22:55
*** felipemonteiro_ has quit IRC22:55
*** jackie-truong has quit IRC22:59
openstackgerritDan Smith proposed openstack/nova master: Remove deprecated aggregate DB compatibility
openstackgerritDan Smith proposed openstack/nova master: Add AggregateList.get_by_metadata_value() query method
openstackgerritDan Smith proposed openstack/nova master: Add aggregates list to Destination object
openstackgerritDan Smith proposed openstack/nova master: WIP: Add request filter functionality and first filter
*** hongbin has quit IRC23:06
*** burt has quit IRC23:10
*** acormier has joined #openstack-nova23:11
*** liverpooler has quit IRC23:11
*** edmondsw has quit IRC23:12
melwittmlavalle: I've confirmed that 3-5 on thursday works for nova/neutron. thanks23:12
mlavallemelwitt: Thank you!23:12
*** acormier has quit IRC23:16
*** amodi has quit IRC23:19
*** mlavalle has quit IRC23:20
mnaseryay adding bdms to cell0 fixed it23:26
melwitttonyb: this says ocata enters phase 3 support on 2/26 but also says 2/26 is the EOL date does that make sense to you?23:35
*** mriedem has joined #openstack-nova23:38
*** hshiina|afk has joined #openstack-nova23:38
mriedemmelwitt: do you want/plan on doing a queens retrospective at the ptg?23:39
*** acormier has joined #openstack-nova23:40
melwittmriedem: I'm not opposed to having one but I hadn't thought about it because no one had mentioned it (whereas we have had obvious interest in retro in the past). do you think we should just plan for one?23:41
*** acormier has joined #openstack-nova23:42
melwittI also might have missed it if anyone has expressed interest in having a queens retro23:42
mriedemit hasn't come up, but i assume we should at least make it an option23:45
mriedemif no one has anything, it's a short thing23:46
melwittyeah, makes sense23:46
mnaseri have 4 patches right, is the following order ok or should i reorder: fix cinder fixtures -> add func regr. tests -> add bdms to cell0 -> patch to remove ports and bdms23:46
*** acormier has quit IRC23:46
melwittmriedem: maybe we could start off with that on wednesday since any takeaways could apply to later discussions23:46
mriedemmnaser: sounds correct23:47
mnaserok awesome, i have to fix a few unit tests because of the changes in _bury_in_cell0 which now create bdms and should be ready for review soon23:47
*** acormier has joined #openstack-nova23:48
*** acormier has quit IRC23:48
*** acormier has joined #openstack-nova23:48
*** acormier has quit IRC23:52
openstackgerritMurali Annamneni proposed openstack/nova master: Enables MySQL Cluster Support for Nova
*** acormier has joined #openstack-nova23:54
*** acormier has joined #openstack-nova23:54
*** felipemonteiro__ has quit IRC23:59

Generated by 2.15.3 by Marius Gedminas - find it at!