Thursday, 2017-10-19

*** smatzek has quit IRC00:01
*** gjayavelu has quit IRC00:01
*** markvoelker_ has quit IRC00:03
*** yamamoto has joined #openstack-nova00:05
*** markvoelker has joined #openstack-nova00:09
*** liverpooler has joined #openstack-nova00:10
*** crushil has quit IRC00:12
*** andreas_s has joined #openstack-nova00:13
*** markvoelker has quit IRC00:13
*** markvoelker has joined #openstack-nova00:18
*** gouthamr has joined #openstack-nova00:19
*** andreas_s has quit IRC00:21
*** yangyapeng has quit IRC00:21
*** andreas_s has joined #openstack-nova00:22
*** yangyapeng has joined #openstack-nova00:22
*** markvoelker has quit IRC00:22
*** ijw has joined #openstack-nova00:23
*** andreas_s has quit IRC00:26
*** markvoelker has joined #openstack-nova00:27
*** yangyapeng has quit IRC00:27
*** huanxie has joined #openstack-nova00:29
*** ijw has quit IRC00:31
*** thorst has joined #openstack-nova00:31
*** markvoelker has quit IRC00:32
*** andreas_s has joined #openstack-nova00:40
*** crushil has joined #openstack-nova00:43
*** salv-orlando has joined #openstack-nova00:43
*** salv-orlando has quit IRC00:47
*** mriedem has quit IRC00:48
*** andreas_s has quit IRC00:49
*** gjayavelu has joined #openstack-nova00:51
*** andreas_s has joined #openstack-nova00:54
*** markvoelker has joined #openstack-nova00:54
*** thorst has quit IRC00:57
*** chyka has joined #openstack-nova00:58
*** markvoelker has quit IRC00:59
*** mriedem has joined #openstack-nova00:59
*** masuberu has joined #openstack-nova01:00
*** Tom__ has joined #openstack-nova01:00
*** Apoorva_ has joined #openstack-nova01:01
*** Shunli has joined #openstack-nova01:02
*** chyka has quit IRC01:02
*** Apoorva has quit IRC01:03
*** masber has quit IRC01:04
*** Tom__ has quit IRC01:05
*** Apoorva_ has quit IRC01:05
*** andreas_s has quit IRC01:07
efriedmriedem I wouldn't mind listening in on the port binding debacle^Wpowwow01:09
*** gjayavelu has quit IRC01:10
mriedemok01:11
*** andreas_s has joined #openstack-nova01:12
*** markvoelker has joined #openstack-nova01:12
*** phuongnh has joined #openstack-nova01:12
*** gjayavelu has joined #openstack-nova01:13
openstackgerritNaichuan Sun proposed openstack/nova master: VGPU_support: add enabled white list  https://review.openstack.org/51258001:15
*** andreas_s has quit IRC01:16
openstackgerritMerged openstack/nova-specs master: Enable SR-IOV NIC offload feature discovery  https://review.openstack.org/50489501:18
*** yangyape_ has joined #openstack-nova01:21
*** thorst has joined #openstack-nova01:23
*** baoli has joined #openstack-nova01:25
*** ijw has joined #openstack-nova01:27
*** masayukig has quit IRC01:28
*** masayukig has joined #openstack-nova01:29
*** suresh12 has quit IRC01:30
*** mmehan has quit IRC01:30
openstackgerritTakashi NATSUME proposed openstack/nova master: Add 'delete_host' command in 'nova-manage cell_v2'  https://review.openstack.org/51032401:31
*** litao__ has joined #openstack-nova01:31
openstackgerritMatt Riedemann proposed openstack/nova-specs master: Spec for service create and destroy notification  https://review.openstack.org/44473101:32
*** ijw has quit IRC01:32
openstackgerritMatt Riedemann proposed openstack/nova-specs master: Spec for service create and destroy notification  https://review.openstack.org/44473101:32
*** liverpooler has quit IRC01:32
*** tojuvone has quit IRC01:35
*** thorst has quit IRC01:36
*** andreas_s has joined #openstack-nova01:39
*** salv-orlando has joined #openstack-nova01:44
*** masber has joined #openstack-nova01:44
*** Tom has joined #openstack-nova01:46
*** masuberu has quit IRC01:47
*** markvoelker has quit IRC01:47
*** salv-orlando has quit IRC01:49
*** vladikr has quit IRC01:51
*** vladikr has joined #openstack-nova01:51
*** coreywright has quit IRC01:53
*** hongbin has joined #openstack-nova01:54
*** gjayavelu has quit IRC01:55
*** baoli has quit IRC01:55
*** ijw has joined #openstack-nova01:56
*** josecastroleon has quit IRC01:59
openstackgerritjichenjc proposed openstack/nova master: Enhance doc for placement allocation list  https://review.openstack.org/51171501:59
*** markvoelker has joined #openstack-nova02:00
openstackgerritHuan Xie proposed openstack/nova master: VGPU: Define vgpu resource class  https://review.openstack.org/51118002:01
*** vladikr has quit IRC02:01
*** ijw has quit IRC02:01
*** josecastroleon has joined #openstack-nova02:01
*** vladikr has joined #openstack-nova02:01
*** ijw has joined #openstack-nova02:02
*** andreas_s has quit IRC02:02
*** yamahata has quit IRC02:02
*** baoli has joined #openstack-nova02:03
*** markvoelker has quit IRC02:05
*** ijw has quit IRC02:06
*** bkopilov has quit IRC02:07
openstackgerritjichenjc proposed openstack/nova master: Add create inventories doc for placement  https://review.openstack.org/51221502:08
openstackgerritjichenjc proposed openstack/nova master: check query param for server groups function  https://review.openstack.org/50034702:08
*** markvoelker has joined #openstack-nova02:10
*** coreywright has joined #openstack-nova02:11
*** andreas_s has joined #openstack-nova02:11
*** vladikr has quit IRC02:13
*** yikun has joined #openstack-nova02:13
*** vladikr has joined #openstack-nova02:13
*** vks1 has joined #openstack-nova02:14
*** markvoelker has quit IRC02:14
openstackgerritTakashi NATSUME proposed openstack/python-novaclient master: Microversion 2.54 - Enable cold migration with target host  https://review.openstack.org/40670702:14
*** vks1 has quit IRC02:14
BenderRodriguezanyone here02:15
BenderRodriguezI'm getting http 500 errors when attempting to run "openstack hypervisor list"02:15
BenderRodriguezhttp://controller:8774/v2.1/os-hypervisors/detail returned with HTTP 50002:15
BenderRodrigueznova is listed as a compute service in "openstack service list"02:15
BenderRodriguezhere's a debug output02:15
BenderRodriguezhttps://paste.debian.net/hidden/353b9966/02:15
*** andreas_s has quit IRC02:16
*** masber has quit IRC02:19
*** markvoelker has joined #openstack-nova02:19
openstackgerritjichenjc proposed openstack/nova master: Refactor placement version check  https://review.openstack.org/51249702:20
*** baoli has quit IRC02:20
*** gyee has quit IRC02:21
*** Tom has quit IRC02:23
*** Tom has joined #openstack-nova02:23
*** markvoelker has quit IRC02:23
*** andreas_s has joined #openstack-nova02:25
*** Swami has quit IRC02:25
*** Tom has quit IRC02:28
*** markvoelker has joined #openstack-nova02:28
*** andreas_s has quit IRC02:29
*** josecastroleon has quit IRC02:30
*** josecastroleon has joined #openstack-nova02:31
*** gjayavelu has joined #openstack-nova02:32
*** markvoelker has quit IRC02:33
*** ijw has joined #openstack-nova02:35
*** takashin has left #openstack-nova02:36
*** vladikr has quit IRC02:37
*** thorst has joined #openstack-nova02:37
mriedemBenderRodriguez: check the nova-api logs for the 50002:37
mriedemthe client trace isn't very helpful02:37
BenderRodriguez2017-10-18 21:18:13.480 27678 ERROR nova.api.openstack EmptyCatalog: The service catalog is empty.02:38
BenderRodriguezbut how02:38
BenderRodriguezopenstack service list shows a compute service02:38
mriedemwould need to see more of the trace from the nova-api log02:39
mriedemnot sure what's looking up a service catalog entry02:39
mriedemcompute isn't looking up itself02:39
*** ijw has quit IRC02:39
*** vladikr has joined #openstack-nova02:41
BenderRodriguezmriedem: http://termbin.com/p26m02:41
BenderRodriguez2017-10-18 21:13:33.467 27678 INFO nova.api.openstack [-] http://controller:8774/v2.1/os-hypervisors/detail returned with HTTP 50002:41
*** thorst has quit IRC02:42
*** andreas_s has joined #openstack-nova02:43
*** Tom has joined #openstack-nova02:44
*** salv-orlando has joined #openstack-nova02:45
*** Tom__ has joined #openstack-nova02:45
openstackgerritMerged openstack/nova-specs master: Websockify security proxy framework  https://review.openstack.org/49616002:45
*** markvoelker has joined #openstack-nova02:46
*** hshiina has joined #openstack-nova02:46
mriedemBenderRodriguez: looks like nova.conf is misconfigured for talking to keystone,02:46
mriedemit's trying to validate the token02:46
mriedemso double check your nova.conf for configuring auth stuff02:47
*** Tom___ has joined #openstack-nova02:47
*** karthiks has quit IRC02:48
*** Tom has quit IRC02:48
*** gmann_afk is now known as gmann02:49
*** Tom__ has quit IRC02:49
*** salv-orlando has quit IRC02:49
*** gongysh has joined #openstack-nova02:50
*** vladikr has quit IRC02:51
openstackgerritjichenjc proposed openstack/nova master: Remove glance usage inside compute  https://review.openstack.org/51139202:51
openstackgerritjichenjc proposed openstack/nova master: Remove usage of glance.generate_image_url  https://review.openstack.org/51139702:51
*** vladikr has joined #openstack-nova02:53
openstackgerritNaichuan Sun proposed openstack/nova master: VGPU_support: add enabled white list  https://review.openstack.org/51258002:55
openstackgerritZhenyu Zheng proposed openstack/nova master: nova-manage db archive_deleted_rows is not multi-cell aware  https://review.openstack.org/50748602:56
BenderRodriguezmriedem: well looking at the openstack hypervisor --debug list output it seems to be going through authentication fine02:56
BenderRodriguezhttp://termbin.com/tap8c02:56
*** andreas_s has quit IRC02:56
*** gouthamr has quit IRC02:57
mriedemare the auth values in your rc file for the CLI the same as in nova.conf for the [keystone_authtoken] section?02:57
BenderRodriguezmriedem: in the rc file the OS_USER/PASS values are for the admin user. In the keystone_authtoken section in the conf, the username/password is for the "nova" user02:59
openstackgerritmelanie witt proposed openstack/nova master: DNM: Test websocketproxy with TLS  https://review.openstack.org/51316002:59
*** TuanLA has joined #openstack-nova03:03
mriedemwhat does "openstack catalog list" show?03:03
BenderRodriguezmriedem: http://termbin.com/nalx03:04
BenderRodriguezall three hosts mentioned are pingable/reachable03:04
mriedemhuh, not sure03:05
mriedemi checked the error on ask.o.o and it's come up but no solutions03:05
mriedemi haven't seen this before03:05
BenderRodriguezoh well03:05
mriedemi'd think a config issue but not sure really03:05
BenderRodriguezI'll just nuke the set up and try again03:05
*** andreas_s has joined #openstack-nova03:05
BenderRodriguezat least for the controller03:06
mriedemnova user and project are all setup ok?03:06
BenderRodriguezhmm03:06
BenderRodriguezthere is a nova user defined03:08
*** sapcc-bot1 has joined #openstack-nova03:09
*** andreas_s has quit IRC03:10
openstackgerritMatt Riedemann proposed openstack/nova master: Add attachment_get to refresh_connection_info  https://review.openstack.org/51262603:11
*** sapcc-bot has quit IRC03:12
*** toabctl has quit IRC03:14
*** andreas_s has joined #openstack-nova03:15
*** nikhil has quit IRC03:20
*** markvoelker has quit IRC03:20
*** toabctl has joined #openstack-nova03:21
* alex_xu back to spec review instead of coding trait patch03:21
mriedemalex_xu: all of your coding dreams, and mine, can come true once the spec freeze is over03:21
alex_xumriedem: hehe, ok :)03:21
BenderRodriguezalex_xu: maybe fix this bug03:24
BenderRodriguezI found03:24
BenderRodriguezif it is a bug03:24
*** TuanLA has quit IRC03:25
*** hieulq has quit IRC03:25
*** trungnv has quit IRC03:25
*** phuongnh has quit IRC03:25
*** markvoelker has joined #openstack-nova03:25
alex_xuBenderRodriguez: today is spec freeze, so the spec review is top priority03:26
*** trungnv has joined #openstack-nova03:26
*** TuanLA has joined #openstack-nova03:26
*** phuongnh has joined #openstack-nova03:26
*** hieulq has joined #openstack-nova03:26
*** mriedem has quit IRC03:28
*** bkopilov has joined #openstack-nova03:29
*** markvoelker has quit IRC03:30
*** josecastroleon has quit IRC03:30
*** josecastroleon has joined #openstack-nova03:31
*** hshiina has quit IRC03:33
*** andreas_s has quit IRC03:33
openstackgerritjichenjc proposed openstack/nova master: Downgrade log for keystone verify client fail  https://review.openstack.org/51324303:34
*** markvoelker has joined #openstack-nova03:34
*** jgriffith has quit IRC03:34
openstackgerritHuan Xie proposed openstack/nova master: VGPU: Define vgpu resource class  https://review.openstack.org/51118003:36
openstackgerritMerged openstack/nova master: libvirt: add method to configure migration speed  https://review.openstack.org/49745603:36
*** peter-hamilton has quit IRC03:36
*** jgriffith has joined #openstack-nova03:38
*** thorst has joined #openstack-nova03:38
*** markvoelker has quit IRC03:39
*** josecastroleon has quit IRC03:40
*** amodi has joined #openstack-nova03:42
*** hongbin has quit IRC03:43
*** markvoelker has joined #openstack-nova03:43
*** josecastroleon has joined #openstack-nova03:44
*** jgriffith_ has joined #openstack-nova03:44
*** thorst has quit IRC03:46
*** salv-orlando has joined #openstack-nova03:46
*** jgriffith has quit IRC03:46
openstackgerritYikun Jiang proposed openstack/nova-specs master: Add pagination and changes since filter support for os-instance-action API  https://review.openstack.org/50776203:47
*** jgriffith_ is now known as jgriffith03:47
*** markvoelker has quit IRC03:48
*** yamahata has joined #openstack-nova03:48
*** salv-orlando has quit IRC03:50
*** markvoelker has joined #openstack-nova03:52
*** Tom___ has quit IRC03:53
*** Tom has joined #openstack-nova03:54
*** Tom__ has joined #openstack-nova03:56
*** markvoelker has quit IRC03:57
*** dave-mccowan has quit IRC03:57
*** Tahvok has quit IRC03:57
*** Tom has quit IRC03:58
*** Tom__ has quit IRC04:00
*** hshiina has joined #openstack-nova04:01
*** markvoelker has joined #openstack-nova04:02
*** Tahvok has joined #openstack-nova04:02
*** mdnadeem has joined #openstack-nova04:03
*** suresh12 has joined #openstack-nova04:03
*** markvoelker has quit IRC04:06
*** markvoelker has joined #openstack-nova04:10
*** gongysh has quit IRC04:12
*** markvoelker has quit IRC04:15
*** huanxie has quit IRC04:19
*** markvoelker has joined #openstack-nova04:20
*** TuanLA has quit IRC04:21
*** phuongnh has quit IRC04:21
*** trungnv has quit IRC04:21
*** TuanLA has joined #openstack-nova04:21
*** phuongnh has joined #openstack-nova04:21
*** trungnv has joined #openstack-nova04:21
*** psachin has joined #openstack-nova04:22
*** yikun has quit IRC04:23
*** ijw has joined #openstack-nova04:23
*** ijw has quit IRC04:28
*** amodi has quit IRC04:29
*** claudiub has joined #openstack-nova04:31
*** josecastroleon has quit IRC04:35
*** huanxie has joined #openstack-nova04:36
*** josecastroleon has joined #openstack-nova04:36
*** vladikr has quit IRC04:46
*** salv-orlando has joined #openstack-nova04:47
*** lajoskatona has joined #openstack-nova04:49
*** Tom has joined #openstack-nova04:50
*** salv-orlando has quit IRC04:51
*** markvoelker has quit IRC04:54
*** armax has quit IRC04:55
*** armax has joined #openstack-nova04:56
*** armax has quit IRC04:56
*** armax has joined #openstack-nova04:57
*** armax has quit IRC04:57
*** armax has joined #openstack-nova04:58
*** armax has quit IRC04:58
openstackgerritMerged openstack/nova master: Keep updating allocations for Ironic  https://review.openstack.org/51308504:58
openstackgerritMerged openstack/nova stable/pike: fix unstable shelve offload functional tests  https://review.openstack.org/50981804:58
*** armax has joined #openstack-nova04:58
*** armax has quit IRC04:59
*** armax has joined #openstack-nova04:59
*** armax has quit IRC04:59
*** sree has joined #openstack-nova05:00
*** armax has joined #openstack-nova05:00
*** armax has quit IRC05:00
*** armax has joined #openstack-nova05:01
*** armax has quit IRC05:01
*** armax has joined #openstack-nova05:02
*** armax has quit IRC05:02
*** armax has joined #openstack-nova05:02
*** armax has quit IRC05:03
*** armax has joined #openstack-nova05:03
*** salv-orlando has joined #openstack-nova05:03
*** armax has quit IRC05:03
*** armax has joined #openstack-nova05:04
*** armax has quit IRC05:04
*** vvargaszte has joined #openstack-nova05:07
*** markvoelker has joined #openstack-nova05:08
*** markvoelker has quit IRC05:12
*** abhi89 has joined #openstack-nova05:13
*** masber has joined #openstack-nova05:15
*** markvoelker has joined #openstack-nova05:17
*** masuberu has joined #openstack-nova05:17
*** masber has quit IRC05:19
*** vvargaszte has quit IRC05:21
*** Tom has quit IRC05:21
*** Tom has joined #openstack-nova05:21
*** markvoelker has quit IRC05:22
*** jdavis has joined #openstack-nova05:22
*** chyka has joined #openstack-nova05:22
openstackgerritjichenjc proposed openstack/nova master: Add more functional test for placement.usage  https://review.openstack.org/51326405:23
*** markvoelker has joined #openstack-nova05:26
*** chyka has quit IRC05:26
*** jdavis has quit IRC05:27
*** gongysh has joined #openstack-nova05:27
*** markvoelker has quit IRC05:29
*** markvoelker has joined #openstack-nova05:29
*** gongysh has quit IRC05:32
*** suresh12 has quit IRC05:34
*** thorst has joined #openstack-nova05:42
*** markvoelker has quit IRC05:42
*** markvoelker has joined #openstack-nova05:42
*** markvoelker has quit IRC05:42
*** thorst has quit IRC05:47
*** spectr has joined #openstack-nova05:48
*** ijw has joined #openstack-nova05:50
*** alex_xu has quit IRC05:54
*** alex_xu has joined #openstack-nova06:00
*** huanxie has quit IRC06:01
*** edand has joined #openstack-nova06:03
*** josecastroleon has quit IRC06:07
*** hoonetorg has quit IRC06:09
*** andreas_s has joined #openstack-nova06:11
*** Eran_Kuris has joined #openstack-nova06:17
*** Oku_OS-away is now known as Oku_OS06:18
*** sahid has joined #openstack-nova06:20
*** cshen has quit IRC06:21
openstackgerritMerged openstack/nova stable/newton: Use VIR_DOMAIN_BLOCK_REBASE_COPY_DEV when rebasing  https://review.openstack.org/47135606:24
*** hoonetorg has joined #openstack-nova06:25
*** salv-orlando has quit IRC06:26
*** salv-orlando has joined #openstack-nova06:26
*** sahid has quit IRC06:27
*** sahid has joined #openstack-nova06:28
*** salv-orlando has quit IRC06:30
*** huanxie has joined #openstack-nova06:33
*** sshwarts has joined #openstack-nova06:45
*** cshen has joined #openstack-nova06:45
*** ijw has quit IRC06:48
*** markvoelker has joined #openstack-nova06:49
*** vishwana_ has joined #openstack-nova06:54
*** vishwanathj has quit IRC06:54
*** pcaruana has joined #openstack-nova06:55
*** vishwanathj has joined #openstack-nova06:58
*** vishwana_ has quit IRC06:59
*** hshiina has quit IRC07:03
*** crushil has quit IRC07:19
*** avolkov`` has joined #openstack-nova07:19
*** ttsiouts has joined #openstack-nova07:19
*** tesseract has joined #openstack-nova07:21
*** avolkov`` has left #openstack-nova07:22
*** markus_z has joined #openstack-nova07:25
*** salv-orlando has joined #openstack-nova07:26
openstackgerritLajos Katona proposed openstack/nova master: Change live_migrate tests to use fakedriver  https://review.openstack.org/50520207:27
*** salv-orlando has quit IRC07:31
*** jpena|off is now known as jpena07:31
openstackgerritsahid proposed openstack/nova-specs master: update: introducing isolate emulthreads on host  https://review.openstack.org/51118807:35
*** salv-orlando has joined #openstack-nova07:42
*** thorst has joined #openstack-nova07:43
*** andreas_s has quit IRC07:45
*** thorst has quit IRC07:47
*** andreas_s has joined #openstack-nova07:54
*** gcb has quit IRC07:55
*** markvoelker has quit IRC08:01
*** ralonsoh has joined #openstack-nova08:03
openstackgerritsahid proposed openstack/nova-specs master: virt: allow instances to be booted with trusted VFs  https://review.openstack.org/48552208:06
*** ragiman has joined #openstack-nova08:11
*** lucas-afk is now known as lucasagomes08:16
*** stephenfin has left #openstack-nova08:17
*** stephenfin has joined #openstack-nova08:17
*** AlexeyAbashkin has joined #openstack-nova08:28
*** gcb has joined #openstack-nova08:28
*** tssurya has quit IRC08:29
*** Tom has quit IRC08:32
*** yamamoto has quit IRC08:32
*** dtantsur|afk is now known as dtantsur08:32
*** Tom has joined #openstack-nova08:32
*** gjayavelu has quit IRC08:35
*** Tom__ has joined #openstack-nova08:36
*** Tom has quit IRC08:37
*** Tom__ has quit IRC08:40
*** Tom has joined #openstack-nova08:40
*** tssurya has joined #openstack-nova08:41
*** derekh has joined #openstack-nova08:46
*** spectr has quit IRC08:47
*** Tom has quit IRC08:48
*** spectr has joined #openstack-nova08:48
*** spectr has quit IRC08:48
*** Tom has joined #openstack-nova08:48
*** spectr has joined #openstack-nova08:50
openstackgerritStephen Finucane proposed openstack/nova master: conf: Remove 'vendordata_driver' opt  https://review.openstack.org/39783508:55
openstackgerritStephen Finucane proposed openstack/nova master: conf: Validate '[api] vendordata_providers' options  https://review.openstack.org/51331808:55
*** cdent has joined #openstack-nova08:56
bauzasgibi: you did a bit of review on https://review.openstack.org/#/c/444731/8, fancy to vote it quickly so I could +W ?08:56
openstackgerritStephen Finucane proposed openstack/nova master: conf: Fix indentation of database options  https://review.openstack.org/44309708:57
openstackgerritStephen Finucane proposed openstack/nova master: conf: Resolve TODOs in 'database'  https://review.openstack.org/39369508:57
openstackgerritStephen Finucane proposed openstack/nova master: conf: Remove 'db_driver' config opt  https://review.openstack.org/50848708:57
*** chyka has joined #openstack-nova08:58
openstackgerritjichenjc proposed openstack/nova master: remove CONF.vendordata_driver  https://review.openstack.org/50151008:59
openstackgerritjichenjc proposed openstack/nova master: Add description for reousrce class creation  https://review.openstack.org/50808308:59
*** chyka has quit IRC09:03
*** sapd__ has joined #openstack-nova09:12
*** sapd_ has quit IRC09:12
*** yamamoto has joined #openstack-nova09:13
*** Tom has quit IRC09:16
*** TuanLA has quit IRC09:16
*** hieulq has quit IRC09:16
*** trungnv has quit IRC09:16
*** phuongnh has quit IRC09:16
*** TuanLA has joined #openstack-nova09:17
*** phuongnh has joined #openstack-nova09:17
*** trungnv has joined #openstack-nova09:17
*** hieulq has joined #openstack-nova09:17
openstackgerritMerged openstack/nova-specs master: Spec for service create and destroy notification  https://review.openstack.org/44473109:17
*** salv-orlando has quit IRC09:21
*** salv-orlando has joined #openstack-nova09:22
*** sambetts|afk is now known as sambetts09:24
*** salv-orlando has quit IRC09:27
*** lpetrut has joined #openstack-nova09:29
*** Shunli has quit IRC09:30
artomlegacy-grenade-dsvm-neutron-multinode legacy-grenade-dsvm-neutron-multinode : ERROR Project openstack/neutron does not have the default branch master09:34
artomo_O09:34
*** sree_ has joined #openstack-nova09:36
*** sree_ is now known as Guest4428909:36
*** sree has quit IRC09:37
*** sdague has joined #openstack-nova09:38
*** gszasz has joined #openstack-nova09:39
*** Tom has joined #openstack-nova09:40
*** thorst has joined #openstack-nova09:44
openstackgerritZhenyu Zheng proposed openstack/nova master: nova-manage db archive_deleted_rows is not multi-cell aware  https://review.openstack.org/50748609:45
*** Tom has quit IRC09:47
*** thorst has quit IRC09:49
*** yamamoto has quit IRC09:49
artomOh god we broke archive_deleted_rows_again09:49
*** cdent has quit IRC09:53
*** vivsoni has quit IRC09:53
*** vivsoni has joined #openstack-nova09:53
*** yassine has quit IRC09:56
*** andreas_s has quit IRC09:57
*** andreas_s has joined #openstack-nova09:58
*** TuanLA has quit IRC10:02
*** andreas_s has quit IRC10:02
*** andreas_s has joined #openstack-nova10:08
*** Tom has joined #openstack-nova10:11
*** rodolof has joined #openstack-nova10:13
*** andreas_s has quit IRC10:18
*** mvk has quit IRC10:19
*** salv-orlando has joined #openstack-nova10:21
*** andreas_s has joined #openstack-nova10:23
*** yamahata has quit IRC10:28
*** andreas_s has quit IRC10:30
*** andreas_s has joined #openstack-nova10:30
*** cdent has joined #openstack-nova10:31
openstackgerritStephen Finucane proposed openstack/nova-specs master: Add 'move-nova-cmds-to-cliff' spec  https://review.openstack.org/43360310:32
stephenfinbauzas: Addressed comments in ^10:32
stephenfintl;dr: Migration to cliff is only part of that spec. The larger aim is "cleanup nova-* commands". The spec title is misleading but I'm stuck with it now10:33
*** rodolof has quit IRC10:34
*** rodolof has joined #openstack-nova10:35
*** Guest44289 has quit IRC10:37
*** Tom has quit IRC10:37
*** sree has joined #openstack-nova10:37
openstackgerritShoham Peller proposed openstack/nova master: Handle spawning error on unshelving  https://review.openstack.org/37800910:40
*** rgerganov has quit IRC10:45
*** bkopilov has quit IRC10:47
*** mvk has joined #openstack-nova10:48
*** rgerganov has joined #openstack-nova10:48
*** huanxie has quit IRC10:49
*** yamamoto has joined #openstack-nova10:50
*** tbachman has quit IRC10:50
*** yamamoto has quit IRC10:53
*** yamamoto has joined #openstack-nova10:53
*** yamamoto has quit IRC10:53
*** jpena is now known as jpena|lunch10:59
openstackgerritChris Dent proposed openstack/nova master: [placement] Clean up TODOs in allocations.yaml gabbit  https://review.openstack.org/51305710:59
*** yangyape_ has quit IRC11:01
*** yamamoto has joined #openstack-nova11:02
*** phuongnh has quit IRC11:02
*** sree has quit IRC11:05
*** markus_z has quit IRC11:10
*** jdavis has joined #openstack-nova11:11
*** smatzek has joined #openstack-nova11:12
*** yassine has joined #openstack-nova11:17
*** vladikr has joined #openstack-nova11:22
*** lucasagomes is now known as lucas-hungry11:22
gibibauzas: sorry, I was pulled into internal meeting for the whole morning. But I agree with your judgement on https://review.openstack.org/#/c/444731/811:23
*** AlexeyAbashkin has quit IRC11:24
*** AlexeyAbashkin has joined #openstack-nova11:26
*** markvoelker has joined #openstack-nova11:27
sahidjaypipes: hola, i just updated the vf trusted specs, basically now it's just about to pass a tag in the binding and operator will have to isolate group of hosts11:30
sahidthey were a part where you asked me to use traits for the binding, can you have a look and perhaps suggest me what you want exactly?11:31
sahidjaypipes: here is the link https://review.openstack.org/#/c/485522/ so you don't even have to search for the pointer :)11:32
*** nicolasbock has joined #openstack-nova11:32
*** chyka has joined #openstack-nova11:32
*** smatzek has quit IRC11:33
*** itlinux has joined #openstack-nova11:34
*** chyka has quit IRC11:37
*** nicolasbock has quit IRC11:38
dtantsurjohnthetubaguy: mind checking https://review.openstack.org/#/c/511844/ please?11:41
*** nicolasbock has joined #openstack-nova11:50
*** AlexeyAbashkin has quit IRC11:51
*** edmondsw has joined #openstack-nova11:51
*** eharney has joined #openstack-nova11:52
*** salv-orlando has quit IRC11:54
*** mvk has quit IRC11:54
*** AlexeyAbashkin has joined #openstack-nova11:55
*** sree has joined #openstack-nova11:56
*** rcernin has joined #openstack-nova11:58
*** sree has quit IRC12:01
*** jpena|lunch is now known as jpena12:02
*** mvk has joined #openstack-nova12:07
bauzaswho knows IRC nick of https://www.openstack.org/community/members/profile/33046/zhenyu-zheng ?12:09
*** cdent has quit IRC12:10
*** litao__ has quit IRC12:10
*** cdent has joined #openstack-nova12:10
yikun_jiang@bauzas Kevin_Zheng12:12
*** dave-mccowan has joined #openstack-nova12:12
bauzasyikun_jiang: ty12:12
bauzasKevin_Zheng: around ?12:12
bauzasKevin_Zheng: just about https://review.openstack.org/#/c/509326/612:12
cdentbauzas: yikun_jiang beat me to it, but my usual trick is to search launchpad: https://launchpad.net/+search?field.text=zhenyu-zheng12:12
*** dillaman has joined #openstack-nova12:13
Kevin_Zhengbauzas what’s up?12:13
bauzasKevin_Zheng: if you could provide a follow-up change for all the nits, would be awesome :)12:13
*** thorst has joined #openstack-nova12:13
*** psachin has quit IRC12:13
Kevin_Zhengbauzas: sure I can do that12:13
bauzasthat's not urgent, but wanted to make sure you saw the points12:13
bauzasKevin_Zheng: thank you !12:13
Kevin_Zhengbauzas: np12:14
bauzascdent: I usually use https://www.openstack.org/community/members/ for that :)12:14
bauzasbtw, Kevin_Zheng it would be nice if you could add your nick there ^12:15
*** salv-orlando has joined #openstack-nova12:15
*** itlinux has quit IRC12:15
Kevin_ZhengI will try tomorrow:)12:15
cdentbauzas: sure, but a combo of both sometimes gets us there12:16
* cdent gets lunch12:17
*** cdent has quit IRC12:17
*** bkopilov has joined #openstack-nova12:18
*** rmart04 has joined #openstack-nova12:19
*** peter-hamilton has joined #openstack-nova12:25
*** dtantsur is now known as dtantsur|brb12:26
*** catintheroof has joined #openstack-nova12:27
*** yangyapeng has joined #openstack-nova12:29
*** tssurya has quit IRC12:30
*** lajoskatona has quit IRC12:30
*** lajoskatona has joined #openstack-nova12:30
*** jamesdenton has joined #openstack-nova12:30
*** tssurya has joined #openstack-nova12:30
openstackgerritMerged openstack/nova-specs master: Improve the performance of filtering instances by IP.  https://review.openstack.org/50932612:36
*** catintheroof has quit IRC12:42
*** jaosorior has joined #openstack-nova12:43
*** lucas-hungry is now known as lucasagomes12:43
*** bbbbzhao_ has joined #openstack-nova12:44
*** mriedem has joined #openstack-nova12:44
*** liverpooler has joined #openstack-nova12:47
*** yikun has joined #openstack-nova12:48
*** catintheroof has joined #openstack-nova12:51
johnthetubaguydtantsur|brb: will take a look at that ASAP12:52
*** catintheroof has quit IRC12:53
*** pchavva has joined #openstack-nova12:53
*** lyan has joined #openstack-nova12:54
*** hemna_ has joined #openstack-nova12:55
*** StevenK has quit IRC12:59
*** cdent has joined #openstack-nova13:02
*** jdavis has quit IRC13:06
*** yassine has quit IRC13:08
*** yassine has joined #openstack-nova13:09
*** tbachman has joined #openstack-nova13:09
*** baoli has joined #openstack-nova13:11
*** mdnadeem has quit IRC13:12
*** baoli has quit IRC13:12
*** sbezverk has joined #openstack-nova13:13
*** StevenK has joined #openstack-nova13:16
*** StevenK is now known as Guest2579213:16
openstackgerritsean mooney proposed openstack/nova-specs master: Use neutron's new port binding API  https://review.openstack.org/37558013:18
*** crushil has joined #openstack-nova13:18
*** Guest25792 is now known as StevenK13:18
bauzasmriedem: I think you made valid points on https://review.openstack.org/#/c/334732/65/specs/queens/approved/abort-cold-migration.rst but maybe we shouldn't block on those13:19
*** baoli has joined #openstack-nova13:19
*** rmcall has joined #openstack-nova13:20
sean-k-mooneymriedem: i dont know if you saw http://lists.openstack.org/pipermail/openstack-dev/2017-October/123776.html in your inbox. does that carify the desire workflow?13:21
*** rcernin has quit IRC13:21
bauzasmriedem: for the virt drivers support, maybe just making sure we update https://docs.openstack.org/nova/latest/user/support-matrix.html and https://docs.openstack.org/nova/latest/user/feature-classification.html could be a starting point, nope ?13:21
gibibauzas, mriedem: I added some comments on the cold migration patch regarding the notification impact but as a second thought we can do the notification impact in a later spec if we don't want to block on core of the feature13:23
openstackgerritRodolfo Alonso Hernandez proposed openstack/nova-specs master: Network bandwidth resource provider  https://review.openstack.org/50230613:25
*** itlinux has joined #openstack-nova13:27
*** awaugama has joined #openstack-nova13:28
*** edand has quit IRC13:35
*** rcernin has joined #openstack-nova13:35
*** rcernin has quit IRC13:40
*** andreas_s has quit IRC13:41
*** andreas_s has joined #openstack-nova13:42
*** felipemonteiro has joined #openstack-nova13:44
*** amodi has joined #openstack-nova13:45
*** felipemonteiro__ has joined #openstack-nova13:45
*** andreas_s has quit IRC13:46
*** mriedem1 has joined #openstack-nova13:47
*** andreas_s has joined #openstack-nova13:48
*** mriedem has quit IRC13:48
cdentjohnthetubaguy: yeah my allocations concern was definitely a long term thing. I don’t think it i something we need to do anything about now, but it feels like something we need in our brains. We get in the habit too often of dimissing the long term picture (because of all the usual constraints). So it was mostly just a reminder.13:48
mriedem1sean-k-mooney: i'm not sure that it does. your email is about the bw aware scheduling stuff. it also says you want nova to make allocations on the bw provider from conductor, but that's not really something we're wanting to do for anything else. we want allocations made in the scheduler if nova is going to make them. but i think neutron should manage the allocations for bw providers since the port is the consumer.13:48
*** mriedem1 is now known as mriedem13:48
mriedembauzas: i'll look again13:48
johnthetubaguycdent: yeah, +1 that. I was really just making sure its not a short term blocker.13:49
*** felipemonteiro has quit IRC13:49
cdentjohnthetubaguy: *nod* we do allocate for disk that is shared, by treating it as local…13:49
*** edand has joined #openstack-nova13:51
*** andreas_s has quit IRC13:53
*** xyang1 has joined #openstack-nova13:54
mriedembauzas: i'm interested in knowing what type of storage backend NTT is using and the size of these disks that's causing the cold migration to take so long that a user has to submit a support ticket to an admin to abort the cold migration13:55
bauzasthat's indeed a good point13:55
mriedemlike, how long does it take to cold migrate a 100GB disk vm backed by ceph?13:55
mriedemcburgess: ^?13:56
johnthetubaguycdent: "nova managed" is how I like to think of that, I wish we could get out of the disk and image business altogether, long term.13:56
*** lajoskatona has quit IRC13:56
*** takashin has joined #openstack-nova13:57
cdentjohnthetubaguy: yup, that would be nice. I wonder if that means we’re going to need to break away from the idea that only one thing should be writing all the allocations associated with a build? <- probably best to just store those worms back in the can for sometime later13:58
johnthetubaguycdent: get that lid back on :p13:59
* johnthetubaguy secretly nods13:59
*** andreas_s has joined #openstack-nova13:59
*** Tom__ has joined #openstack-nova14:00
mriedemmeeting time14:00
*** hongbin has joined #openstack-nova14:03
*** josecastroleon has joined #openstack-nova14:04
*** josecastroleon has quit IRC14:04
*** Tom__ has quit IRC14:04
*** lbragstad has joined #openstack-nova14:04
*** josecastroleon has joined #openstack-nova14:05
*** esberglu has joined #openstack-nova14:07
*** itlinux has quit IRC14:07
*** andreas_s has quit IRC14:08
*** baoli has quit IRC14:09
*** baoli has joined #openstack-nova14:09
*** baoli has quit IRC14:09
*** baoli has joined #openstack-nova14:11
*** andreas_s has joined #openstack-nova14:11
jaypipessahid: k. reviewing it again.14:12
jaypipessahid: see also: https://twitter.com/jaypipes/status/92073928060300902414:12
*** burt has joined #openstack-nova14:15
*** suresh12 has joined #openstack-nova14:15
*** itlinux has joined #openstack-nova14:17
jaypipessdague: around? hoping you could help me spot what the issue is with sahid's spec not passing the sphinx-docs job here: https://review.openstack.org/#/c/485522/14:19
jaypipessdague: can't seem to see anything in the logs..14:19
*** Swami has joined #openstack-nova14:20
*** gouthamr has joined #openstack-nova14:20
efriedjaypipes http://logs.openstack.org/22/485522/3/check/build-openstack-sphinx-docs/3402883/job-output.txt.gz#_2017-10-19_08_16_47_66698814:20
efriedwherezat log?14:21
efriedcause it looks like a sphinx bug masked the real problem.14:21
jaypipesefried: well, yeah, I saw THAT. :) I'm just saying that isn't useful14:21
efriedI'll build it locally and see if I can get at the real problem.14:22
jaypipesefried: ok, cool thank you sir.14:22
efriedjaypipes sahid: Warning, treated as error: /home/efried/Neo/nova-specs/doc/source/specs/queens/approved/sriov-trusted-vfs.rst:166:Footnote [5] is not referenced.14:24
efriedCouldn't repro that other thing, though.14:25
jaypipesefried: hey, thanks for doing that research. appreciated.14:25
efriedyahyoubetcha.14:25
*** gmann is now known as gmann_afk14:26
sdaguejaypipes: did efried figure the whole thing out?14:28
jaypipessdague: seems he did indeed.14:28
jaypipessdague: sorry for the bother.14:28
efriedsdague No, I haven't found where sphinx is missing a param to a log message.14:28
efriedsdague See http://logs.openstack.org/22/485522/3/check/build-openstack-sphinx-docs/3402883/job-output.txt.gz#_2017-10-19_08_16_47_66698814:29
efriedThat didn't hit when I built locally.14:29
jaypipesefried: but you identified the thing that's causing an error in the spec itself.14:29
efriedYeah.  Though interestingly, when I rebuilt with tox -r, it passed.14:29
efriedwhich is... weird.14:29
sdaguesunspots14:29
sean-k-mooneymriedem: hi well we could do the allocation in the scheduler also i just want them to happen before we call to the compute node.14:30
sean-k-mooneymriedem: if you want neutorn to do it the i would make that allocations be part of the bind port.14:31
efriedsean-k-mooney mriedem Have we considered doing the allocation from neutron at port binding time14:31
efriedyeah, THAT.14:31
mriedemyeah that's what i suggested in the spec14:32
efriedThen we close the window for races between [port binding succeeds] and [allocation fails]14:32
sean-k-mooneyefried:i was looking a at that code today14:32
mriedemthe port binding could fail as a result of the allocation failing, but then it's all managed within neutron14:32
*** tbachman has quit IRC14:32
cdenti also think neutron should do the allocation14:32
efriedCool.  And the `consumer` of the allocation is who?  The port or the instance?  (My vote is the port.)14:33
sean-k-mooneyefried: we can do it as an else to https://github.com/openstack/neutron/blob/1b3f982914d82494c63ac4fee405bf4972d2db32/neutron/plugins/ml2/managers.py#L74814:33
cdentthat’s what is currently suggested on the spc, no?14:33
cdent(that == port)14:33
efriedIt wasn't clear to me in the spec.14:33
dansmithallocation of what class?14:33
sean-k-mooneycdent: proably i have been in a meeting for the last little bit and have not looked at the lates version14:33
cdentsean-k-mooney: I dumped some thoughts on ps13 and then a new version happened which isn’t hugely different14:34
efrieddansmith Yeah.  At least the "bandwidth" - but that's a future thing.  The network interface resource, whatever that is.  E.g. a VF or OVS port.14:34
sean-k-mooneydansmith: allocation of bandwidth or vifs14:34
*** itlinux has quit IRC14:34
cdentefried: are we talking about the same spec (I’m talking about the network bandwidth rp one)14:34
efried^14:34
dansmithah, bandwidth14:35
efriedcdent We're mostly talking about the 'new neutron port binding api' spec14:35
dansmiththat means the scheduler can't pick a host with suitable bandwidth for the migration though right?14:35
efriedcdent but bleeding into the bw rp one.14:35
sean-k-mooneycdent: cool ill re read. if the neutron folks are oke with it ill update it shortly14:35
dansmithor for a new build for that matter14:36
cdentefried: that one is somewhere in the my “next” queue14:36
sean-k-mooneyefried: yes these two are tightly coupled in some respects14:36
efrieddansmith Can pick it, but wouldn't allocate it.14:36
*** READ10 has joined #openstack-nova14:36
*** armax has joined #openstack-nova14:36
dansmithefried: == resource race == fail14:36
sean-k-mooneydansmith: it can do a first pass without knowing the network backend and then we do the port bind in the conducto before casting to the compute14:36
mriedemgibi: this is very simple if you can take a look https://review.openstack.org/#/c/512626/14:36
efriedHow do we get around a race *somewhere*?14:37
gibimriedem: looking...14:37
sean-k-mooneydansmith: we might be able to do it in the schduler but have not look to see where yet14:37
dansmithsean-k-mooney: efried: that gets us a two-phase commit of allocations and we have to rollback from those both and then reschedule.. avoiding that is the whole point of doing this in the scheduler14:37
*** rmcall has quit IRC14:37
openstackgerritRodolfo Alonso Hernandez proposed openstack/os-vif master: Migrate from 'ip' commands to 'pyroute2'  https://review.openstack.org/48438614:37
efriedYeah, definitely don't disagree with that.  But we're also doing a "bind early, fail early" thing with the port.14:38
sean-k-mooneydansmith: well thats one of the reasons i want nova to do the claim instead of neutron14:38
efriedSo either we [bind early, allocate late] and have the race there, or [bind early & allocate network stuff early, allocate non-network stuff late] and have the race there.14:38
*** spectr has quit IRC14:38
efriedI guess the former is cleaner.14:39
dansmithum14:39
mriedemtbc, the 'use neutron port binding API' spec is really not the same thing here, has nothing to do with allocations - it's about using port binding during live migration14:39
mriedemwe should have probably renamed that spec title :)14:39
mriedemthe bw provider one is the thing talking about claims14:39
mriedemthe call i setup in 20 minutes is about the live migration spec, not the bw provider one14:39
dansmithyeah, I'm clearly missing some context from the bw one since I'm thinking about the live migration one14:39
mriedemyes people are blurring them14:39
efriedmriedem Regardless, there needs to be a stake in the ground about who's the consumer of the nw resource, who does the allocation, and when.14:40
dansmitheffing people14:40
mriedemefried: that doesn't have to be decided for https://review.openstack.org/#/c/375580/14:40
efriedUnless the vif isn't a resource....14:40
mriedemthe call in 20 minutes is for ^14:40
mriedembecause i can actually see that getting done in queens14:40
mriedemi can't see bw provider getting done in queens14:40
mriedemit's dependent on at least 3 other things and we're unclear on the design still14:41
*** efried is now known as efried|brb14:41
*** gyee has joined #openstack-nova14:42
sean-k-mooneymriedem: idealy though we would use the same workflow for spawning a new instance as we do for moving an instance14:43
sean-k-mooneymriedem: interms of when and where we claim resouces14:43
mriedemi don't disagree, but today we aren't claiming anything in placement for networking14:44
sean-k-mooneymriedem: how did you sent that invite by the way? it went straight into my calander and i dont see the email at all14:44
sean-k-mooneymriedem: well we are for routed networks correct?14:46
sean-k-mooneymriedem: or is that done by neutron?14:46
mriedemsean-k-mooney: nova doesn't do anything for routed networks yet14:46
sean-k-mooneyok that must be in nova so.14:47
mriedemthat was related to johnthetubaguy's prep for network-aware scheduling thing14:47
sean-k-mooney*neutron14:47
mriedemneutron is doing something yes14:47
johnthetubaguythe only bit nova does is not error when a port has no ip if its on a routed network14:48
dansmithheh14:49
dansmiththe only thing nova does is not explode14:49
sean-k-mooneyjust pinging mlavlle on the neutron channel to see if he is oke with doing the claim in the port bind. if so ill update the neutron and nova spec to reflect that14:49
dansmithsean-k-mooney: hold up dude,14:49
*** suresh12 has quit IRC14:49
dansmithwe should have a discussion, because I think we're missing something important if we do it that way14:49
*** nikhil has joined #openstack-nova14:50
sean-k-mooneydansmith: im not going to update anything untill after the call14:50
dansmithokay14:50
sean-k-mooneyi just want to get his initall reaction14:50
sean-k-mooneyif its a non starter from a neutron perspective that is good to know :)14:51
*** efried|brb is now known as efried14:51
*** yamamoto has quit IRC14:54
mriedemagain, the primary purpose of the call is about the live migration spec, not the bw provider spec,14:56
mriedemif there is time i'm sure we can get into that14:56
*** rodolof has quit IRC14:57
sean-k-mooneymriedem: right be we still need to decide on where to do the port bind and activate the binding for the live migration too so we can regenerate the xml if needed14:57
*** rodolof has joined #openstack-nova14:57
sean-k-mooneymriedem: for example vhost-user socket path changes or you migrate form linux bridge to ovs14:58
*** yikun has quit IRC14:58
mriedemjohnthetubaguy: were you going to join?15:02
*** jaosorior has quit IRC15:02
johnthetubaguymriedem: ah, I didn't get the invite / behind on mail15:02
mriedemhttps://hangouts.google.com/hangouts/_/calendar/bWF0dHJpZWQyMkBnbWFpbC5jb20.6jrqpj946jj56uo6dsuj4gnmjj15:02
*** sshwarts has quit IRC15:05
*** josecastroleon has quit IRC15:06
*** baoli has quit IRC15:07
openstackgerritBalazs Gibizer proposed openstack/nova master: cover migration cases with functional tests  https://review.openstack.org/49386515:07
*** baoli has joined #openstack-nova15:08
*** baoli has quit IRC15:08
*** dtantsur|brb is now known as dtantsur15:08
*** Oku_OS is now known as Oku_OS-away15:10
*** baoli has joined #openstack-nova15:11
*** markvoelker has quit IRC15:12
*** markvoelker has joined #openstack-nova15:13
*** acormier has quit IRC15:14
*** acormier has joined #openstack-nova15:14
*** penick has joined #openstack-nova15:15
*** edand has quit IRC15:16
*** nicolasbock has quit IRC15:17
*** markvoelker has quit IRC15:17
*** acormier has quit IRC15:18
*** nicolasbock has joined #openstack-nova15:18
openstackgerritMerged openstack/nova-specs master: Add serial numbers for local disks  https://review.openstack.org/51146615:20
openstackgerritTakashi NATSUME proposed openstack/nova-specs master: Abort Cold Migration  https://review.openstack.org/33473215:21
*** MVenesio has joined #openstack-nova15:21
openstackgerritTakashi NATSUME proposed openstack/nova-specs master: Abort Cold Migration  https://review.openstack.org/33473215:21
*** nicolasbock has quit IRC15:22
*** suresh12 has joined #openstack-nova15:23
*** suresh12 has quit IRC15:23
*** suresh12 has joined #openstack-nova15:23
*** penick has quit IRC15:23
stvnoyesHi nova cores, There's a stale bugfix review (https://review.openstack.org/#/c/389798)  that needs a core review.  It's a pretty simple change (in the nova compute api) so it should hopefully be quick(?). Thanks.15:24
*** penick has joined #openstack-nova15:26
*** takashin has left #openstack-nova15:27
*** andreas_s has quit IRC15:28
*** mvk has quit IRC15:30
*** rmart04 has quit IRC15:31
*** crushil has quit IRC15:32
*** andreas_s has joined #openstack-nova15:33
*** andreas_s has quit IRC15:37
*** zphir has joined #openstack-nova15:39
*** zphir has quit IRC15:40
bauzasgraaaah, the Gerrit flickering when you open a comment makes me mad15:40
*** chyka has joined #openstack-nova15:41
efriedbauzas If you can snap a video of that happening, send it to sdague15:42
*** andreas_s has joined #openstack-nova15:42
bauzasjaypipes: I officially hereby invoke your SQL skills and ask you if you know if SQLA can support an index of a tuple of multiple fields where one of them comes from a inherited mixin15:43
bauzasjaypipes: context being https://review.openstack.org/#/c/507762/1215:43
bauzasthat's where a POC would help a specs review :)15:44
dansmithbauzas: reports are that if you set render=slow it will be better about that15:44
sdagueI apparently have never seen this thing (or noticed it)15:46
jaypipesbauzas: you can include fields from a mixin into an index, sure.15:46
*** andreas_s has quit IRC15:47
jaypipesbauzas: added a comment (and link to an example) to the spec15:48
bauzasjaypipes: oh fsck, deleted of course15:49
openstackgerritEd Leafe proposed openstack/nova master: Remove the Pike migration code for flavor migration  https://review.openstack.org/49420615:49
* bauzas facepalms15:49
bauzasbut yeah the tuple is wrong then, since we support soft-deletions for instance actions...15:50
bauzasnot sure you can really *delete* an action tho15:50
bauzasdansmith: sorry, WDYM ?15:52
jaypipesbauzas: the gerrit flickering annoyance.15:52
efriedbauzas Go into settings => diff preferences and set Render => Slow15:53
dansmithbauzas: https://imgur.com/a/Oq5rw15:53
dansmithbauzas: gerrit, diff settings, render -> slow15:53
bauzasaaaah cool15:53
jaypipesbauzas: go to settings, then diff prefs, then render => slow.15:53
bauzasta !15:53
jaypipeslol, three of us :)15:53
efriedI win15:53
dansmithwhat? I was totally first15:53
dansmithby six seconds15:53
bauzasdansmith: +1 marmelade15:53
dansmithhehe15:53
bauzasdansmith: you should seriously consider then a checked-in luggage :p15:54
dansmithwilld o15:54
dansmither, will do15:54
*** yamamoto has joined #openstack-nova15:55
openstackgerritJohn Griffith proposed openstack/nova-specs master: Add multiattach support to Nova  https://review.openstack.org/49977715:57
*** jgriffith is now known as groot16:03
*** groot is now known as jgriffith16:04
*** jgriffith is now known as groot16:05
*** lpetrut has quit IRC16:07
*** ttsiouts has quit IRC16:08
*** andreas_s has joined #openstack-nova16:09
*** lucasagomes is now known as lucas-afk16:09
*** yamamoto has quit IRC16:11
*** salv-orl_ has joined #openstack-nova16:11
*** Swami has quit IRC16:12
*** Apoorva has joined #openstack-nova16:12
*** abhi89 has quit IRC16:14
*** andreas_s has quit IRC16:14
*** salv-orlando has quit IRC16:14
bauzasstephenfin: honestly, if all is about code refactoring for Queens, I don't think it needs a spec *unless two points*16:16
bauzas#1 we don't use a new framework16:16
bauzas#2 we don't change the consumed interface we have for all our customers16:16
stephenfinYou mean *assuming* those things?16:16
stephenfinAs opposed to unless16:17
bauzasin other words, if we still have a backwards-compatible behaviour16:17
stephenfinThe CLI won't change from a users perspective16:17
bauzasstephenfin: my point is that you said there are multiple issues16:17
stephenfinAll that will change is return code and docs16:17
*** penick has quit IRC16:17
stephenfinAnd the return code one could wait16:17
bauzasI'd love to see what could be done without cliff and what other things need it16:17
stephenfinWe would be changing the framework, though end-users won't see this16:17
*** pcaruana has quit IRC16:18
stephenfinI'm not dropping the cliff bit - it's still the main reason for this :)16:18
openstackgerritEric Fried proposed openstack/nova master: Service user token requested with no auth  https://review.openstack.org/51232916:18
stephenfinOtherwise is just deleting some dead code but no more. cliff delivers enough tangible benefits to justify its inclusion16:18
stephenfinand means nova is using the exact same tooling as everyone else in OpenStack16:18
stephenfin*CLI tooling16:18
bauzasstephenfin: what I'm saying is that we don't need a spec for something just pure internal refactoring, but I guess pulling a dependency like cliff is more than just internal refactoring16:19
bauzasstephenfin: how do you see the implementation path ?16:19
stephenfinYeah, I'm not sure if it is, hence the question :)16:19
stephenfinPretty much what I said in the spec (I think)16:19
bauzasstephenfin: fix first some issues, then use cliff for only some subcommands?16:19
stephenfinDump all the deprecated stuff16:19
bauzasor do you plan to just change all at once ?16:19
*** markvoelker has joined #openstack-nova16:20
bauzasstephenfin: the problem is that a cliff move would be like super low priority16:20
bauzasso I'm afraid that effort could be multi-cycle16:20
*** yassine has quit IRC16:20
bauzashence my question about how you plan to migrate all the existing commands16:21
stephenfinI plan to add the cliff stuff one by one with tests, but not register the 'console_scripts' entry points16:21
stephenfinthen, once it's done, we switch entry points and delete all the existing crud16:21
openstackgerritEric Fried proposed openstack/nova master: Service user token requested with no auth  https://review.openstack.org/51232916:21
*** MVenesio has quit IRC16:21
*** penick has joined #openstack-nova16:21
bauzas'one by one', one being a subcommand like Cellv2 ?16:21
*** Apoorva has quit IRC16:21
stephenfinYup16:21
*** Apoorva has joined #openstack-nova16:22
stephenfinYou wouldn't be able to run any of them via the command-line but they'd have unit tests16:22
stephenfinThere's only four or five sub-commands to migrate once the deprecated stuff is removed, in fairness16:22
bauzasif I'm using the existing CLI, would I go thru a different codepath ?16:23
stephenfinAlso, I know it's low priority. This is purely a tech debt reductionexercise16:23
stephenfinFor the CLI aspects of it, yes16:23
bauzasFWIW, I played with cliff but it lasts (4 or 5 years I don't remember exactly)16:23
stephenfinFor the actual underlying command, shared code path16:23
bauzask16:24
stephenfinnew UTs would only exercise the arg parsers plus any gaps I will uncover16:24
bauzasmriedem: do you think we would really a spec for the above ?16:24
*** markvoelker has quit IRC16:24
bauzasmriedem: honestly a specless BP seems good if we don't change the CLI16:24
bauzasthen we could defer the CLI change in a Rocky spec I guess16:25
*** andreas_s has joined #openstack-nova16:28
*** suresh12 has quit IRC16:29
*** crushil has joined #openstack-nova16:30
*** penick_ has joined #openstack-nova16:30
*** andreas_s has quit IRC16:32
*** ralonsoh has quit IRC16:33
*** penick has quit IRC16:33
stephenfinbauzas: Just to reiterate, the CLI won't change from a usage perspective (minus some help messages, return codes). How it's implemented under the hood would16:37
*** crushil has quit IRC16:38
*** yamamoto has joined #openstack-nova16:39
*** lpetrut has joined #openstack-nova16:43
*** bnemec has quit IRC16:45
*** bnemec has joined #openstack-nova16:46
*** AlexeyAbashkin has quit IRC16:46
*** Apoorva_ has joined #openstack-nova16:46
*** yamahata has joined #openstack-nova16:49
openstackgerritEric Fried proposed openstack/nova master: [Trivial] Fix up a docstring  https://review.openstack.org/51344716:50
efriedsdague ^ that's been on my backlog for a while :)16:50
*** Apoorva has quit IRC16:50
*** jpena is now known as jpena|off16:52
*** Apoorva_ has quit IRC16:52
sdague+A16:53
*** slaweq_ has joined #openstack-nova16:53
*** Apoorva has joined #openstack-nova16:53
*** yamamoto has quit IRC16:53
*** ijw has joined #openstack-nova16:54
*** chohoor has quit IRC16:55
*** lpetrut has quit IRC16:57
*** ijw has quit IRC17:00
*** derekh has quit IRC17:00
*** ijw has joined #openstack-nova17:02
*** slaweq_ has quit IRC17:02
*** amodi has quit IRC17:02
*** slaweq_ has joined #openstack-nova17:03
*** dtantsur is now known as dtantsur|afk17:04
*** sambetts is now known as sambetts|afk17:04
*** READ10 has quit IRC17:04
*** andreas_s has joined #openstack-nova17:04
*** tesseract has quit IRC17:04
*** felipemonteiro__ has quit IRC17:05
*** lpetrut has joined #openstack-nova17:10
*** slaweq_ has quit IRC17:12
*** bbbbzhao_ has quit IRC17:12
*** slaweq_ has joined #openstack-nova17:13
dansmithjaypipes: are you working on a rev to 415921?17:17
*** slaweq_ has quit IRC17:17
*** efried is now known as efried|afk17:18
*** smatzek has joined #openstack-nova17:19
*** suresh12 has joined #openstack-nova17:19
*** suresh12 has quit IRC17:20
*** suresh12_ has joined #openstack-nova17:20
*** fragatina has quit IRC17:23
*** salv-orlando has joined #openstack-nova17:25
*** salv-orl_ has quit IRC17:29
*** salv-orlando has quit IRC17:30
*** andreas_s has quit IRC17:31
*** smatzek has quit IRC17:32
mriedemildikov: ok notes are in https://review.openstack.org/#/c/499777/17:32
mriedemgroot: johnthetubaguy: ^17:32
*** gjayavelu has joined #openstack-nova17:33
*** lbragstad has quit IRC17:35
ildikovmriedem: thanks!17:35
*** groot is now known as jgriffith17:38
*** Swami has joined #openstack-nova17:39
*** lbragstad has joined #openstack-nova17:39
*** andreas_s has joined #openstack-nova17:41
*** mriedem has quit IRC17:41
*** mriedem has joined #openstack-nova17:43
*** andreas_s has quit IRC17:45
ildikovmriedem: johnthetubaguy: how much time do I have to upload the new version?17:47
*** amodi has joined #openstack-nova17:51
*** MVenesio has joined #openstack-nova17:58
*** baoli has quit IRC17:58
openstackgerritsahid proposed openstack/nova-specs master: virt: allow instances to be booted with trusted VFs  https://review.openstack.org/48552217:59
openstackgerritIldiko Vancsa proposed openstack/nova-specs master: Add multiattach support to Nova  https://review.openstack.org/49977717:59
melwittmriedem: I was talking to penick_ the other day and they've been testing ocata internally, and noticed delete is really slow (same as you encountered in your devstack fakedriver testing)18:00
ildikovmriedem: johnthetubaguy: jgriffith: new old version of the spec ^^18:00
melwittas in, the REST API call returns fast but it takes like a minute for the instance to disappear18:01
melwitthopefully they can investigate but I've made myself a TODO to look into it when I get a chance18:01
*** andreas_s has joined #openstack-nova18:03
*** tbachman has joined #openstack-nova18:04
*** felipemonteiro has joined #openstack-nova18:06
*** baoli has joined #openstack-nova18:07
*** felipemonteiro__ has joined #openstack-nova18:07
jaypipesdansmith: yes18:07
dansmithalright.18:09
mriedemildikov: 4 hours?18:11
*** felipemonteiro has quit IRC18:11
ildikovmriedem: uploaded a new version plz check :)18:11
mriedemmelwitt: cool, ok - i think what huawei did internally was run the volume and network cleanup in parallel like when you're creating the instance18:11
mriedemmelwitt: understanding if there is a regression would be nice too though18:12
jgriffithildikov couple questions/nits in review18:12
melwittmriedem: and that solved it? will they upstream it?18:12
mriedemmelwitt: not sure, i think they were still saying it's slow, but they are on mitaka18:12
melwittmriedem: yeah, it was a regression for sure compared with juno, so I'll dig into what happened there18:13
dansmithmriedem: you didn't have volumes in your test, right?18:13
mriedemnope18:13
mriedemnor networking18:13
dansmithyeah18:13
dansmithso the networking parallelism isn't going to buy you as much on delete I think18:13
melwittyahoo/oath doesn't have volumes either18:13
dansmiththe point there is to overlap some long-running things like image download with network provisioning,18:13
melwittand they're using nova-net via hack18:13
melwittso it's not related to neutron18:14
dansmithwhich wouldn't be as much of a thing on delete18:14
mriedemocata would mean cleaning up allocations in placement18:14
mriedembut not sure why that would take much longer18:14
openstackgerritsahid proposed openstack/nova-specs master: update: introducing isolate emulthreads on host  https://review.openstack.org/51118818:14
melwittbut huawei said mitaka is slow and that's pre-placement18:15
jaypipesdansmith: I don't see how cur_gen can be None in that patch.18:15
dansmithjaypipes: you're doing a .get() on it, that's how18:15
jaypipesdansmith: the comment I removed is this:18:16
jaypipes       # Update our generation immediately, if possible. Even if there18:16
jaypipes        # are no inventories we should always have a generation but let's18:16
jaypipes        # be careful.18:16
jaypipesdansmith: I should have just replaced the get() with a dict access.18:16
sahidjaypipes: about your tweet, i understand that it's a bit confusing, i just try to address an use-case18:16
dansmithjaypipes: that's what I'm saying18:16
jaypipesdansmith: because there simply isn't a way to get a None value there.18:16
sahidin any case i update the spec to fix the issue with ci I hope everytinhg is ok now :)18:16
dansmithjaypipes: either do that so we break, or check the result of the get() and raise something specific18:17
dansmithjaypipes: but don't just ignore it and pass None to placement18:17
jaypipesdansmith: will do the former.18:17
sahidoh... I just noticed your comments, let see18:17
dansmithjaypipes: thanks18:17
penick_melwitt: we "fixed" the delete timing this morning by setting reclaim_instance_interval to a non-zero number. Deletes still take a minute or two on the backend, but at least it'll seem fast for users.18:18
mriedemmelwitt: the new internal perf thing i got yesterday was they were saying claims were taking too long in the compute because of the lock18:18
mriedemthey are testing creating 70 instances on the same host at the same time18:18
*** AlexeyAbashkin has joined #openstack-nova18:19
melwittpenick_: you are now leveraging soft-delete18:19
penick_yep18:19
melwittmriedem: claims in mitaka, right? the old style claim18:20
mriedemyes18:20
*** gszasz has quit IRC18:20
mriedembut, we still have all that code if you're using the ram/core/disk filters18:20
mriedemand/or caching scheduler18:21
melwittso the COMPUTE_RESOURCE_SEMAPHORE18:21
mriedemanyway, there was no low level profiling done18:21
mriedemyeah18:21
dansmithand that's all been there for a long time18:21
mriedemi was going to see if there are things we hit in the db while we have the lock held that could be moved out18:21
openstackgerritsahid proposed openstack/nova-specs master: virt: allow instances to be booted with trusted VFs  https://review.openstack.org/48552218:21
*** andreas_s has quit IRC18:21
mriedemlike getting instance pci requests18:21
melwittI think that's what caused some stuff in ironic to be slow too right ... the slow initial query for ironic nodes18:21
dansmithmaybe penick_ wasn't running real conductor before but now is?18:22
mriedemi did notice this yesterday https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L147318:22
mriedemthat will call update_usage in the RT which has the lock18:22
mriedemand we do that for no apparent reason18:22
mriedemthe RT doesn't care about an instance setting up networking18:22
melwitthm18:23
*** AlexeyAbashkin has quit IRC18:23
mriedemi told them to comment that line out and see what happens18:23
mriedemand adjust their update_available_resource interval18:23
*** rodolof has quit IRC18:23
*** rodolof has joined #openstack-nova18:24
penick_dansmith: that's correct.. Our Ocata clusters use conductor, and our SE team is deploying the conductor service to our juno clusters today.18:24
penick_use the conductor service, i mean.18:24
*** lpetrut has quit IRC18:24
openstackgerritMerged openstack/nova stable/pike: Fix sending legacy instance.update notification  https://review.openstack.org/51095718:25
openstackgerritMerged openstack/nova stable/pike: Keep updating allocations for Ironic  https://review.openstack.org/51319818:25
dansmithpenick_: so I'd look to see if maybe you're pegging those whereas before you were doing a bunch of direct access with lower latency?18:25
dansmithmriedem's thing was also a single compute, so lots of contention for things in the claim process18:26
dansmithwhereas I assume penick_ has a plethora of computes18:26
dansmitha veritable cornucopia of bountiful resources18:26
* dansmith gets into the fall spirit18:26
mriedemoh yeah, i guess on _complete_deletion we are updating the rt,18:26
mriedemwhich grabs the lock18:26
penick_The ocata clusters are all dev and qe with only a 1-5 computes each, they're sitting pretty much idle. But i'll doublecheck to make sure we don't have something else cropping up gobbling up the resources on them18:26
mriedemand makes all of the deletes on a single node serialized18:26
*** rodolof has quit IRC18:28
ildikovjgriffith: added some answers18:28
mriedemso looking way back https://github.com/openstack/nova/blob/0ca98979da842f9accf0f8d8c4eb7d4f1f9eac7a/nova/compute/manager.py#L61218:29
mriedembefore versioned objects18:29
mriedemwe always updated an instance state using that method, which also updated the RT18:29
mriedemwhich grabs the lock18:29
mriedemso we have some vestigial use of that now, updating RT unnecessarily18:29
mriedemRT should only care if you're creating, moving or deleting an instance18:29
mriedemnot that you're setting up networking18:30
penick_The ocata api hosts running the conductor service are all idle18:31
dansmithpenick_: during a delete storm?18:31
penick_That I don't know yet, i'm going to set one up. I'll let you know what I come up with.18:32
dansmithat idle I'd expect them to be idle ;)18:32
penick_But first our -juno- conductors are all exploding as they're being deployed. Gotta help the team tune those first18:33
penick_I set myself up for that :|18:34
melwittexploding meaning they won't run? or they are overloaded and you need to adjust workers?18:35
*** baoli has quit IRC18:37
penick_Overloaded, we're only running 26 workers per node, on 3 nodes. Going to try adding more18:38
openstackgerritMatt Riedemann proposed openstack/nova master: Don't update RT in _allocate_network  https://review.openstack.org/51347318:40
*** coreywright has quit IRC18:43
*** tbachman has quit IRC18:45
*** tbachman has joined #openstack-nova18:45
*** slaweq_ has joined #openstack-nova18:46
*** baoli has joined #openstack-nova18:48
*** suresh12_ has quit IRC18:48
*** slaweq_ has quit IRC18:49
*** suresh12 has joined #openstack-nova18:50
openstackgerritsean mooney proposed openstack/nova-specs master: Use neutron's new port binding API for live migration  https://review.openstack.org/37558018:51
melwittpenick_: that seems like a lot to me but would have to consult dansmith. I don't remember what the sane starting point for number of workers per node is18:53
dansmithworkers per node is not a useful metric, it's workers per compute18:54
dansmith26*3 is a fair number of conductor workers, but if you have a thousand computes, then ... probably not insane18:54
dansmithit also depends on what is happening18:55
dansmithif everything is idle, that's probably more than enough18:55
melwittokay, sorry I think I was thinking of earlier when penick_ said 1-5 computes but if he's talking about juno then that would be 100s of computes18:55
melwittthe 1-5 computes is the ocata testing18:55
openstackgerritJay Pipes proposed openstack/nova master: placement: integrate ProviderTree to report client  https://review.openstack.org/41592118:55
openstackgerritJay Pipes proposed openstack/nova master: placement: add nested resource providers  https://review.openstack.org/37713818:55
openstackgerritJay Pipes proposed openstack/nova master: placement: allow filter providers in tree  https://review.openstack.org/37721518:55
openstackgerritJay Pipes proposed openstack/nova master: placement: adds REST API for nested providers  https://review.openstack.org/38480718:55
openstackgerritJay Pipes proposed openstack/nova master: placement: update client to set parent provider  https://review.openstack.org/38569318:55
jaypipesdansmith: ^18:55
* jaypipes needs to be afk for around 3 hours. catch y'all later this evening.18:56
*** coreywright has joined #openstack-nova18:56
*** suresh12 has quit IRC18:57
dansmithmelwitt: right18:57
*** suresh12 has joined #openstack-nova18:58
*** salv-orlando has joined #openstack-nova19:05
*** jdavis has joined #openstack-nova19:05
*** smatzek has joined #openstack-nova19:05
*** MVenesio has quit IRC19:07
*** smatzek has quit IRC19:07
*** smatzek has joined #openstack-nova19:08
*** abhi89 has joined #openstack-nova19:10
*** slaweq_ has joined #openstack-nova19:11
*** smatzek has quit IRC19:12
sahidjaypipes: ci is green https://review.openstack.org/#/c/485522/19:13
sahidbtw mriedem, did you take descison for this: https://review.openstack.org/#/c/511188/ ?19:14
*** suresh12 has quit IRC19:15
*** AlexeyAbashkin has joined #openstack-nova19:17
*** suresh12 has joined #openstack-nova19:18
*** nicolasbock has joined #openstack-nova19:18
*** AlexeyAbashkin has quit IRC19:21
mriedemsahid: no19:21
mriedemit's not on my list19:21
*** suresh12 has quit IRC19:23
sahidmriedem: thanks19:24
*** pchavva has quit IRC19:27
*** efried|afk is now known as efried19:28
*** suresh12 has joined #openstack-nova19:29
*** ijw has quit IRC19:29
penick_melwitt: this particular cluster is about 1500 nodes. But I think I found the issue. They had the hypervisors set to use the conductor service, but the API was still configured to use local conductor. Which i'm pretty sure will never work. I had them fix it and so far it appears things are working. They'll do some more tests and get back to me19:31
melwittaha, cool19:32
dansmithpenick_: APIs should use always go straight to the db for object access19:32
dansmithand in juno I'm not sure there was anything in conductor that apis needed to access, but local conductor for them should be fine19:32
dansmithpenick_: what was "fixed" .. the load on the conductors?19:33
*** awaugama has quit IRC19:34
efriedjaypipes This is old news, but I definitely saw a way to get None there.  The placement call fails, but report client uses raise_exc=False so the response just comes back, and the caller is doing soft gets everywhere, so populates the inventory with {} and no generation.  That percolates down to the code in question.19:34
melwittoh, oops. local conductor on API19:34
dansmithyeah: https://github.com/openstack/nova/blob/juno-eol/nova/cmd/compute.py#L6519:39
dansmithno such line there for api, which means it ignores the local conductor flag19:39
dansmithalso, github groaned at me when I selected the juno-eol tag19:39
dansmithso penick_ owes me a beverage19:39
penick_melwitt: this particular cluster is about 1500 nodes. But I think I found the issue. They had the hypervisors set to use the conductor service, but the API was still configured to use local conductor. Which i'm pretty sure will never work. I had them fix it and so far it appears things are working. They'll do some more tests and get back to me19:39
penick_dansmith sorry about that19:40
* penick_ is good for it19:40
dansmithheh19:40
melwittpenick_: yeah, as dansmith said, that shouldn't matter. the nova/cmd/api.py doesn't consider the local conductor conf setting19:41
dansmithactually I'm pretty sure penick_ has bought me beverages before19:41
dansmithso I'm probably in beverage debt19:41
melwittpenick_: and that API is supposed to access the database directly anyhow19:41
melwitt(it does so automatically)19:41
dansmithyeah,19:41
penick_that makes sense19:41
dansmithso maybe after updating the config a bunch of computes were storming the castle and it settled down near the time you flipped that flag,19:41
dansmiththinking it was causal?19:42
efriedWhat is "cold migration" aka?  E.g. in the support matrix https://docs.openstack.org/nova/latest/user/support-matrix.html19:42
dansmithefried: resize19:43
dansmithefried: or nova migrate19:43
efrieddansmith thx19:43
mriedemildikov: easy fixes in https://review.openstack.org/#/c/499777/ - you want to update quick? otherwise i can19:43
*** lpetrut has joined #openstack-nova19:44
efriedhow do you tell it to resize to another host?19:44
melwittresize automatically goes to another host19:44
dansmithwell19:45
mriedemefried: only way it can go back is the config option19:45
dansmithunless configured19:45
dansmithyeah19:45
mriedemallow_resize_to_same_host19:45
melwittunless you have configured "allow_resize_to_same_host" it won't consider the same host19:45
efriedwhoah, okay.19:45
melwittnote that means it CAN go to the same host if you config that, not that it will ONLY go there19:45
efriedAnd if you only have one host, presumably, you don't need to switch the setting on?19:45
melwittsome ppl get confused by it19:45
mriedemefried: remember that cold migrate is called 'resize' in the code,19:45
mriedemjust like evacuate is called rebuild19:45
mriedemtotally not confusing19:46
cdentefried: force resize to same has been discussed at various times and I’m sure patched in by vairous people19:46
mriedemand migrate != live migration19:46
efried"remember", sure.19:46
mriedembecause if it's not live, it's old19:46
mriedem*cold19:46
mriedemtakashi has a spec for specifying a host for cold migratoin19:46
mriedemit was only approved after i got him to remove the 'force' flag option19:46
mriedemspecifying a host is fine as long as the scheduler validates it first,19:46
*** salv-orl_ has joined #openstack-nova19:46
efriedOh, not the same spec as 'abort cold', the reviewing of which led me to this series of questions?19:46
mriedemforce flag bypasses the scheduler which gets all sorts of crazy19:46
mriedemabort cold is different19:47
dansmithefried: you still need the config option even if you only have one host19:47
mriedemdevstack defaults to set allow_resize_to_same_host=True19:47
mriedemb/c it's single node19:47
efrieddansmith otherwise it'll just fail?19:47
mriedemefried: yes19:47
efriedwhee19:47
mriedembecause cold migrating to yourself doesn't make sense,19:48
mriedemunless you're resizing the flavor19:48
mriedemand if you're resizing to the same host, we quintuple your cpu allocations during scheduling just to make you pay for it19:49
*** dave-mccowan has quit IRC19:49
*** salv-orlando has quit IRC19:49
efriedWell, to me "resize" means "change the size of".  Not "move to another host".  But what do I know.19:49
*** nicolasbock has quit IRC19:49
mriedemhence the 'remember cold migrate is called resize internally19:49
efriedperfect.19:50
efriedWell, also in the docs.19:50
mriedemand evacuate == rebuild19:50
efriedWhich is why /me was confused.19:50
penick_dansmith You're probably right about the thundering herd being the problem. I think the SE team missed that these machines have CPU and memory utilization spiking. The memory spiked enough to hit swap.  Also I see at least one panic in the log.19:50
dansmithefried: have you read my blog post?19:50
efrieddansmith Which?19:50
dansmithpenick_: ack19:51
dansmithefried: the secret decoder ring for all these move operations19:51
*** dave-mccowan has joined #openstack-nova19:51
efrieddansmith I hope it's not too long.19:51
efrieddansmith But send me the link and I'll see if I can plough my way through it.19:51
efried(In all seriousness, dansmith, I would love a decoder ring for the move ops)19:51
dansmithoh it's long, real long..19:52
dansmither, the post.. yeah, not too long19:52
dansmithefried: http://www.danplanet.com/blog/2016/03/03/evacuate-in-nova-one-command-to-confuse-us-all/19:52
efriedIBM forbids me even responding to that.19:52
*** ijw has joined #openstack-nova19:52
dansmithack19:53
dansmithmy corporate compliance training requirement is coming up soon19:53
dansmithI better re-read19:53
openstackgerritMerged openstack/nova master: Add 'delete_host' command in 'nova-manage cell_v2'  https://review.openstack.org/51032419:54
*** nicolasbock has joined #openstack-nova19:55
ildikovmriedem: not at the laptop now19:56
mriedemildikov: i'll update it19:56
*** ijw has quit IRC19:57
ildikovmriedem: thank you! I owe you, I guess way more than one!!!19:57
mriedemget me out of work meetings and customer dinners in sydney and the debt i spaid19:58
efrieddansmith Super helpful article, thank you.19:58
* dansmith bows19:59
*** nicolasbock has quit IRC20:00
penick_dansmith and melwitt thanks for the help, btw20:01
*** liverpooler has quit IRC20:01
dansmithpenick_: no problem. Now I have to go shower the juno off me20:01
penick_Sorry about that. On the upside you don't need to bathe in the juno-ness all day every day.20:02
*** penick_ is now known as penick20:02
*** dave-mccowan has quit IRC20:02
*** smatzek has joined #openstack-nova20:04
dansmithsdague: we kinda lost momentum on mikal's privsepapalooza: https://review.openstack.org/#/c/494423/3020:04
openstackgerritMerged openstack/nova master: Pick ironic nodes without VCPU set  https://review.openstack.org/51184420:08
openstackgerritMerged openstack/nova master: Add attachment_get to refresh_connection_info  https://review.openstack.org/51262620:08
openstackgerritMerged openstack/nova master: Update live migration to use v3 cinder api  https://review.openstack.org/46398720:08
*** nicolasbock has joined #openstack-nova20:09
mriedemstvnoyes: ^ woot20:12
openstackgerritMatt Riedemann proposed openstack/nova-specs master: Add multiattach support to Nova  https://review.openstack.org/49977720:14
mriedemjohnthetubaguy: ^ updated the multiattach spec - i'll proxy your +2 if you're not around to handle it tonight20:14
*** nicolasbock has quit IRC20:14
*** nicolasbock has joined #openstack-nova20:15
mriedemoh yeah today is the q-1 tag, suppose i should tag that20:16
*** sahid has quit IRC20:16
*** AlexeyAbashkin has joined #openstack-nova20:17
cdentmriedem, dansmith: since it is spec freezey day, and this spec is required by a spec already merged, you wanna drop some eyes on it? It was done as a seperate spec from the POST allocation (which is what requires it) because it’s a different microversion: https://review.openstack.org/#/c/508164/ please and thank you.20:20
cdentIt’s probably not strictly necessary that it merges as the work is going to be done anyway. It is mostly a bookkeeping thing I guess.20:20
*** AlexeyAbashkin has quit IRC20:21
mriedemprobably just gets a freeze exception given it's a dependency20:22
*** ijw has joined #openstack-nova20:23
cdentcool, thanks20:23
*** nicolasbock has quit IRC20:23
*** peter-hamilton has quit IRC20:26
*** ijw has quit IRC20:27
*** andreas_s has joined #openstack-nova20:30
*** andreas_s has quit IRC20:35
*** tbachman has quit IRC20:35
*** nicolasbock has joined #openstack-nova20:37
*** AlexeyAbashkin has joined #openstack-nova20:38
*** AlexeyAbashkin has quit IRC20:42
*** slaweq_ has quit IRC20:48
*** slaweq_ has joined #openstack-nova20:49
*** slaweq_ has quit IRC20:54
*** felipemonteiro__ has quit IRC20:54
*** swamireddy has joined #openstack-nova20:56
*** ijw has joined #openstack-nova20:56
openstackgerritMatt Riedemann proposed openstack/nova-specs master: Add support for certificate validation  https://review.openstack.org/48854120:58
*** AlexeyAbashkin has joined #openstack-nova20:59
*** Apoorva_ has joined #openstack-nova21:02
*** AlexeyAbashkin has quit IRC21:03
*** felipemonteiro has joined #openstack-nova21:04
*** Apoorva has quit IRC21:05
*** felipemonteiro has quit IRC21:09
*** abhi89 has quit IRC21:09
openstackgerritMerged openstack/nova-specs master: Add multiattach support to Nova  https://review.openstack.org/49977721:10
openstackgerritMerged openstack/nova-specs master: Add support for certificate validation  https://review.openstack.org/48854121:12
efriedmriedem I think you added me to https://review.openstack.org/#/c/334732/ to render opinion as a driver owner; did that.21:12
mriedemefried: ok, takashi said it was for NFS in his use case too, and it takes "several minutes" for the disk to migrate,21:13
mriedemwhich to me isn't a good use case for needing to add plumbing to abort the thing,21:13
efriedmriedem wha?  How could it take minutes to "migrate" NFS?21:13
mriedemif the admin is cold migrating the instance, it's for good reason21:13
*** markvoelker has joined #openstack-nova21:13
mriedemi think the live migrate cancel stuff was needed because live migrate could stall out and we needed a hook in to abort the job in the hypervisor21:14
efriedOkay, that's not what the spec says.21:15
mriedemthis is just like, whiny user says their cold migrating vm is down too long and tells the admin to stop doing that thing21:15
*** gouthamr has quit IRC21:15
mriedemright, this doesn't say anything about cold migrate stalling out and we need a way to kill it21:15
efriedThe spec implies that we're copying gigabytes of local (*actual* local) disk over the wire.21:15
efried...and want to be able to bail on that.21:15
mriedemthis is saying, "admin is cold migrating, user doesn't like that it's taking several minutes, and wants the admin to stop'21:15
mriedemidk maybe if there were actual numbers in here,21:16
mriedemlike, i've got a 200GB disk windows vm21:16
*** thorst has quit IRC21:16
mriedemand it's taking me 3 hours to cold migrate that thing21:16
mriedemthis is why i asked cburgess this morning b/c i know he uses ceph and i know he loves to migrate stuff21:17
*** rodolof has joined #openstack-nova21:17
cburgessOh sorry I missed something...21:17
cburgessDo what now?21:17
stvnoyesmridem. fantastic! :-) thanks for all the help from both you and johnthetubaguy!21:17
mriedemcburgess: do you think https://review.openstack.org/#/c/334732/ is useful?21:18
mriedemcburgess: do you have users with 100+ gb disks and if so, do you cold migrate those and if so, how long does that take?21:18
*** suresh12 has quit IRC21:18
cburgessmriedem Funny you should ask about 100+ GB disk... I'm just now starting at a 15TB  one.21:18
*** thorst has joined #openstack-nova21:18
*** suresh12 has joined #openstack-nova21:19
cburgessmriedem Give me a few to read this spec and I'll let you know.21:19
mriedemthanks21:19
*** Apoorva_ has quit IRC21:20
*** Apoorva has joined #openstack-nova21:20
ildikovstvnoyes: congrats!!! :)21:21
*** dave-mccowan has joined #openstack-nova21:23
*** thorst has quit IRC21:23
*** suresh12 has quit IRC21:25
*** suresh12 has joined #openstack-nova21:25
*** smatzek has quit IRC21:26
*** vladikr has quit IRC21:29
*** dave-mccowan has quit IRC21:29
*** penick has quit IRC21:30
*** dave-mccowan has joined #openstack-nova21:37
openstackgerritChris Dent proposed openstack/nova master: WIP: [placement] Enable limiting GET /allocation_candidates  https://review.openstack.org/51352621:37
openstackgerritMatt Riedemann proposed openstack/nova-specs master: Add pagination and changes since filter support for os-instance-action API  https://review.openstack.org/50776221:37
*** hongbin has quit IRC21:38
*** lpetrut has quit IRC21:39
*** baoli has quit IRC21:40
*** penick has joined #openstack-nova21:41
mriedemjaypipes: addressed your comments and sylvain's in https://review.openstack.org/#/c/507762/ - left a todo for alex's thing about sort order which i think we could hash out in review later21:41
cdentnight all21:42
*** cdent has quit IRC21:42
*** amodi has quit IRC21:43
*** dave-mccowan has quit IRC21:44
*** thorst has joined #openstack-nova21:47
*** xyang1 has quit IRC21:47
*** mvk has joined #openstack-nova21:48
*** markvoelker has quit IRC21:48
openstackgerritMerged openstack/nova master: [Trivial] Fix up a docstring  https://review.openstack.org/51344721:50
*** yamamoto has joined #openstack-nova21:50
*** penick has quit IRC21:51
*** thorst has quit IRC21:51
*** flanders_ has joined #openstack-nova21:55
*** jdurgin has quit IRC22:01
*** markvoelker has joined #openstack-nova22:02
cburgessmriedem So yeah thats useful. We tend to not recommend cold migrations with large amounts of data. Clients that use a lot of space we push towards a volume solution for the large storage and to either use BFV or to have small roots locally and put the bulk storage in volumes.22:03
cburgessBut its still a useful feature.22:03
mikalRandom fact for the day. Name the commit which added metadata to nova.22:03
*** burt has quit IRC22:03
cburgessmikal: Which part of metadata? The service?22:04
mikalThere has always been a web service22:04
mriedemcburgess: want to comment on the spec? operator input has weight on something like this.22:04
mikalSo, the commit which added any instance accessible metadata functionality22:04
cburgessmriedem Sure22:04
*** jdavis has quit IRC22:04
mriedem"copy what ec2 / aws does"?22:04
cfriesenmriedem: for what it's worth, the largest EC2 instances have 48TB of disk22:04
cburgessmikal Um... probably somewhere in in the first few commits if not the initial import.22:05
cfriesenmriedem: for the "abort cold migration" thing, what about the case where the dest node crashes mid-transfer?22:05
mriedemcfriesen: wouldn't that auto-kill the cold migrate?22:05
cburgesscfriesen I think thats covered already isn't it? Don't we detect that?22:05
cburgessmriedem cfriesen Oh maybe not now that we can't monitor disk transfer progress.22:06
cfriesenI'm thinking the connection could stay open until the TCP connection times out22:06
cfriesenin the scp case, at least22:06
cburgessOh right... libvirt will report the error back.22:06
cburgessAt some point22:06
cburgessIn theory22:06
*** markvoelker has quit IRC22:06
cburgessmikal So whats the answer?22:06
mikalcburgess: commit bf6e6e718cdc7488e2da87b21e258ccc065fe49922:07
mikalcburgess: which has the commit message "initial import"22:07
mikalcburgess: at that point, nova was 5,083 lines of code22:07
cburgessmikal Called it!22:07
mikalIn other words, its always been weird22:08
cburgesswow 5K lines of code. I have scripts to fix horked up volumes that are longer then that...22:08
cburgessOK thats a lie.. its only 1349 lines of code.22:09
*** markvoelker has joined #openstack-nova22:11
*** lyan has quit IRC22:13
*** hemna_ has quit IRC22:14
*** markvoelker has quit IRC22:16
cburgessmriedem Is the existing abort admin only? Or is that policy controlable?22:19
mriedemcburgess: anytihng related to live migration is admin-only by default22:19
mriedemand cold migration22:19
cburgessmriedem OK22:20
*** suresh12 has quit IRC22:20
mikalcburgess: I'm not sure volumes existed in the first commit, I'd have to check...22:20
*** markvoelker has joined #openstack-nova22:20
cburgessmikal What do volumes have to do with anything?22:21
mikalcburgess: well, your volume fix script...22:21
mikalcburgess: probably wasn't required back then22:21
cburgessmikal My volume fix script wasn't required until the hot mess that is/was os-brick circa liberty.22:21
*** suresh12 has joined #openstack-nova22:22
*** thorst has joined #openstack-nova22:23
*** fragatina has joined #openstack-nova22:23
*** markvoelker has quit IRC22:25
*** jdavis has joined #openstack-nova22:25
*** thorst_ has joined #openstack-nova22:26
*** thorst has quit IRC22:27
*** jdurgin has joined #openstack-nova22:28
openstackgerritMatt Riedemann proposed openstack/nova-specs master: Use neutron's new port binding API for live migration  https://review.openstack.org/37558022:28
*** markvoelker has joined #openstack-nova22:29
*** thorst_ has quit IRC22:30
*** sdague has quit IRC22:34
*** markvoelker has quit IRC22:34
mriedemdansmith: i updated ^ based on some review i had after sean updated it from the call this morning22:34
mriedemdansmith: one sticky thing in there yet which i know you won't like,22:35
mriedembut i think we can hammer in code review22:35
dansmithyeah I don't really think the argument of "avoid changing the object due to stuff in ml2" is sound22:36
dansmithbut as long as we can argue in the code I guess it's fine22:36
*** rodolof has quit IRC22:38
*** markvoelker has joined #openstack-nova22:39
*** markvoelker has quit IRC22:43
openstackgerritMerged openstack/nova-specs master: Use neutron's new port binding API for live migration  https://review.openstack.org/37558022:45
*** andreas_s has joined #openstack-nova22:47
*** markvoelker has joined #openstack-nova22:47
*** andreas_s has quit IRC22:51
*** dave-mccowan has joined #openstack-nova22:57
*** ijw has quit IRC22:58
*** suresh12 has quit IRC23:00
*** suresh12 has joined #openstack-nova23:02
*** Apoorva has quit IRC23:04
*** Apoorva has joined #openstack-nova23:05
*** baoli has joined #openstack-nova23:06
*** thorst has joined #openstack-nova23:07
*** Apoorva has quit IRC23:07
*** Apoorva has joined #openstack-nova23:08
*** ijw has joined #openstack-nova23:12
*** baoli has quit IRC23:12
*** ijw has quit IRC23:13
*** ijw has joined #openstack-nova23:13
*** andreas_s has joined #openstack-nova23:14
*** claudiub has quit IRC23:15
*** AlexeyAbashkin has joined #openstack-nova23:16
*** andreas_s has quit IRC23:18
*** AlexeyAbashkin has quit IRC23:21
*** markvoelker has quit IRC23:22
openstackgerritMerged openstack/nova-specs master: Add pagination and changes since filter support for os-instance-action API  https://review.openstack.org/50776223:26
*** jdavis has quit IRC23:26
*** smatzek has joined #openstack-nova23:27
*** chyka has quit IRC23:29
*** hemna_ has joined #openstack-nova23:31
*** smatzek has quit IRC23:31
*** hemna_ has quit IRC23:32
*** Swami has quit IRC23:34
*** markvoelker has joined #openstack-nova23:36
*** suresh12 has quit IRC23:36
*** tbachman has joined #openstack-nova23:37
*** gmann_afk is now known as gmann23:38
*** markvoelker has quit IRC23:40
*** dillaman has quit IRC23:44
*** markvoelker has joined #openstack-nova23:45
*** smatzek has joined #openstack-nova23:46
*** suresh12 has joined #openstack-nova23:49
*** markvoelker has quit IRC23:49
gmannbauzas: i replied on your comment on this spec - https://review.openstack.org/#/c/508101/723:51
*** andreas_s has joined #openstack-nova23:51
gmannbauzas: let me know if i need to make it clear about those 'Main policy' in spec. i can do that quickly23:52
*** markvoelker has joined #openstack-nova23:54
*** andreas_s has quit IRC23:55
*** suresh12 has quit IRC23:55
*** Apoorva_ has joined #openstack-nova23:58
*** AlexeyAbashkin has joined #openstack-nova23:58
*** suresh12 has joined #openstack-nova23:58
*** markvoelker has quit IRC23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!