Thursday, 2018-01-25

*** mvk has joined #openstack-nova00:02
*** jangutter has joined #openstack-nova00:03
openstackgerritEric Fried proposed openstack/nova master: New-style _set_inventory_for_provider
openstackgerritEric Fried proposed openstack/nova master: SchedulerReportClient.update_from_provider_tree
openstackgerritEric Fried proposed openstack/nova master: WIP: Use update_provider_tree from resource tracker
openstackgerritEric Fried proposed openstack/nova master: Fix nits in update_provider_tree series
openstackgerritEric Fried proposed openstack/nova master: Move refresh time from report client to prov tree
*** jangutter has quit IRC00:08
*** hoonetorg has quit IRC00:10
jaypipesefried: the client should just know it's an *atomically* increasing integer that, if it changed from the client's previous view of the generation, should trigger a re-read of the expected state.00:11
*** AlexeyAbashkin has joined #openstack-nova00:11
jaypipes(and now I have to leave, I'm sorry :(00:11
efriedjaypipes Have a good one.00:11
Spazmoticsee ya jay00:11
SpazmoticJust waking up and drinking my coffee and getting ready for hte gym.. whew00:11
*** AlexeyAbashkin has quit IRC00:15
*** edmondsw has joined #openstack-nova00:19
*** edmondsw has quit IRC00:23
*** acormier has joined #openstack-nova00:24
*** acormier has joined #openstack-nova00:25
*** hiro-kobayashi has joined #openstack-nova00:27
*** takedakn has joined #openstack-nova00:28
*** hoonetorg has joined #openstack-nova00:30
*** edmondsw has joined #openstack-nova00:31
openstackgerritTakashi NATSUME proposed openstack/nova master: Transform instance.resize_prep notification
*** edmondsw has quit IRC00:36
*** edmondsw has joined #openstack-nova00:37
*** chyka has quit IRC00:37
*** edmondsw has quit IRC00:41
*** SpazZz has joined #openstack-nova00:43
*** dave-mccowan has joined #openstack-nova00:43
*** edmondsw has joined #openstack-nova00:44
*** edmondsw has quit IRC00:45
*** edmondsw has joined #openstack-nova00:46
*** Spazmotic has quit IRC00:46
*** liuzz has joined #openstack-nova00:47
*** edmondsw has quit IRC00:50
*** gaoyan has joined #openstack-nova00:54
mnasermriedem: awesome stuff about osc-placement :D00:54
*** tiendc has joined #openstack-nova00:57
*** jangutter has joined #openstack-nova00:57
*** gaoyan has quit IRC01:00
*** jangutter has quit IRC01:02
*** acormier has quit IRC01:04
*** phuongnh has joined #openstack-nova01:04
*** zhurong has joined #openstack-nova01:05
*** zhaochao has joined #openstack-nova01:12
*** phuongnh has quit IRC01:16
*** hoonetorg has quit IRC01:17
*** mdnadeem has joined #openstack-nova01:18
*** phuongnh has joined #openstack-nova01:19
*** hoonetorg has joined #openstack-nova01:21
*** dave-mccowan has quit IRC01:22
*** yangyapeng has quit IRC01:23
*** yangyapeng has joined #openstack-nova01:24
*** takashin has left #openstack-nova01:25
*** oomichi has joined #openstack-nova01:27
*** crushil has joined #openstack-nova01:27
*** takashin has joined #openstack-nova01:28
*** jangutter has joined #openstack-nova01:33
*** jangutter has quit IRC01:37
*** tianhui has joined #openstack-nova01:41
*** acormier has joined #openstack-nova01:43
*** acormier has quit IRC01:43
*** acormier has joined #openstack-nova01:44
oomichitakashin: hi, still isn't favorite?01:46
*** acormier_ has joined #openstack-nova01:46
*** acormier_ has joined #openstack-nova01:47
*** yikun_ has joined #openstack-nova01:47
takashinoomichi: Hi. There are redirects. So the modification is not necessary.01:47
*** hoangcx has joined #openstack-nova01:47
oomichitakashin: do you see my comment?01:48
*** bjolo has quit IRC01:49
*** acormier has quit IRC01:49
*** bjolo has joined #openstack-nova01:49
takashinoomichi: Yes. but my patch was not just replacing http with https.01:49
Yikungmann,, hi, I saw your inline comments, so, should we add 2 config like project_network_cidr[1 & 2] to keep this? or some other idea?01:50
oomichitakashin: then, why did you change it even if there was a redirecting?01:50
*** acormier has joined #openstack-nova01:51
*** tojuvone has quit IRC01:51
*** dave-mccowan has joined #openstack-nova01:53
openstackgerritMatt Riedemann proposed openstack/nova master: Fix bug case by none token context
mriedemefried: can you tell me if the release note i threw in here sounds ok? ^01:54
*** acormier_ has quit IRC01:54
*** acormier has quit IRC01:54
takashinoomichi: My patch was changing 'pike' to 'queens'. Replaceing http with https was done at the same time, but it is not worth doing it separately.01:55
efriedmriedem LGTM, +101:56
oomichitakashin: I am not sure why it is not worth doing it *separately*01:58
efriedmriedem Course, you coulda addressed my nit, but meh.01:58
efriedtakashin Glad to catch you online.  I've been wanting to tell you how much I appreciate your super-thorough code reviews.01:58
efriedtakashin ありがとうございました ;)01:59
jose-phi_hi any idea02:00
takashinefied: You are welcome.02:00
jose-phi_of why when i set data-port=br-ext:eno1  this interface got dead "eno1"02:00
jose-phi_and this is my data network so i lost all connectivity to the compute node02:00
takashinoomichi: it is not worth doing it alone.02:01
*** fragatina has quit IRC02:01
takashinefried: You are welcome.02:02
openstackgerritEric Fried proposed openstack/nova master: set_{aggregates|traits}_for_provider: tolerate set
openstackgerritEric Fried proposed openstack/nova master: New-style _set_inventory_for_provider
openstackgerritEric Fried proposed openstack/nova master: SchedulerReportClient.update_from_provider_tree
openstackgerritEric Fried proposed openstack/nova master: WIP: Use update_provider_tree from resource tracker
openstackgerritEric Fried proposed openstack/nova master: Fix nits in update_provider_tree series
openstackgerritEric Fried proposed openstack/nova master: Move refresh time from report client to prov tree
gmannmriedem: let me rebase them, not all are required. i will do it today and let you know02:03
*** tojuvone has joined #openstack-nova02:03
*** salv-orl_ has joined #openstack-nova02:04
*** chyka has joined #openstack-nova02:05
oomichitakashin: I just don't want to see similar patches in the future even after dropping the patch02:05
openstackgerritEric Fried proposed openstack/nova master: ProviderTree.get_provider_uuids: Top-down ordering
openstackgerritEric Fried proposed openstack/nova master: set_{aggregates|traits}_for_provider: tolerate set
openstackgerritEric Fried proposed openstack/nova master: New-style _set_inventory_for_provider
openstackgerritEric Fried proposed openstack/nova master: SchedulerReportClient.update_from_provider_tree
openstackgerritEric Fried proposed openstack/nova master: WIP: Use update_provider_tree from resource tracker
openstackgerritEric Fried proposed openstack/nova master: Fix nits in update_provider_tree series
openstackgerritEric Fried proposed openstack/nova master: Move refresh time from report client to prov tree
takashinoomichi: okay. I agree with the change.02:06
*** salv-orlando has quit IRC02:07
*** gibi has quit IRC02:07
*** chyka has quit IRC02:09
*** tinwood has quit IRC02:10
takashinoomichi: I voted +1 for the patch.02:10
oomichitakashin: thanks :-)02:11
*** tinwood has joined #openstack-nova02:11
*** crushil has quit IRC02:12
*** zhurong has quit IRC02:14
*** fragatina has joined #openstack-nova02:14
*** gcb has joined #openstack-nova02:15
*** fragatina has quit IRC02:19
*** stakeda has joined #openstack-nova02:21
*** oomichi has quit IRC02:22
*** slaweq has joined #openstack-nova02:23
openstackgerritTakashi NATSUME proposed openstack/python-novaclient master: Add CLI to show instance usage audit logs
*** acormier has joined #openstack-nova02:26
*** hongbin has joined #openstack-nova02:27
*** jangutter has joined #openstack-nova02:27
*** slaweq has quit IRC02:27
*** acormier_ has joined #openstack-nova02:29
*** acormier_ has quit IRC02:29
*** acormier_ has joined #openstack-nova02:29
*** lyan has joined #openstack-nova02:30
*** jangutter has quit IRC02:31
*** acormier has quit IRC02:32
*** dave-mccowan has quit IRC02:33
*** acormier has joined #openstack-nova02:41
*** acormier_ has quit IRC02:45
*** suresh12 has quit IRC02:52
*** acormier has quit IRC02:53
*** acormier has joined #openstack-nova02:53
*** dave-mccowan has joined #openstack-nova02:54
*** acormier_ has joined #openstack-nova02:55
*** acormier has quit IRC02:59
*** hiro-kobayashi has quit IRC03:06
*** zhurong has joined #openstack-nova03:08
*** acormier_ has quit IRC03:09
alex_xumriedem: thanks, will cleanup in a follow up03:09
*** acormier has joined #openstack-nova03:09
openstackgerritGhanshyam Mann proposed openstack/nova master: Merge server create schema for availability zone extension
openstackgerritGhanshyam Mann proposed openstack/nova master: Merge server create for availability zone extension
openstackgerritJackie Truong proposed openstack/nova master: Reduce complexity of _from_db_object
alex_xumriedem: Kevin_Zheng, it would be great we add some testcase ensure the substring format can pass the regex format in the followup patch, since our API schema value the ip filter input with regex format.
alex_xumriedem: Kevin_Zheng and, I'm curious how neutron side handle the different of 'like' in the different db03:17
*** chyka has joined #openstack-nova03:19
Kevin_Zhengalex_xu, sure03:20
*** harlowja has quit IRC03:21
*** jangutter has joined #openstack-nova03:21
*** chyka has quit IRC03:23
*** acormier has quit IRC03:23
*** suresh12 has joined #openstack-nova03:23
*** armax has joined #openstack-nova03:24
*** jangutter has quit IRC03:25
*** zhurong_ has joined #openstack-nova03:27
*** takashin has quit IRC03:27
*** suresh12 has quit IRC03:28
Yikunalex_xu,, here is 'like'03:32
*** openstackgerrit has quit IRC03:33
*** openstackgerrit has joined #openstack-nova03:34
openstackgerritJackie Truong proposed openstack/nova master: Reduce complexity of _from_db_object
openstackgerritTakashi NATSUME proposed openstack/nova master: Fix missing marker functions
*** yamamoto has joined #openstack-nova03:37
*** lyan has quit IRC03:38
*** takashin has joined #openstack-nova03:38
*** tbachman has quit IRC03:40
*** armax_ has joined #openstack-nova03:43
*** armax has quit IRC03:43
*** armax_ is now known as armax03:43
*** jackie-truong has joined #openstack-nova03:44
*** abhishekk has joined #openstack-nova03:44
*** annp has joined #openstack-nova03:49
*** namnh has joined #openstack-nova03:57
*** Spazmotic has joined #openstack-nova04:04
*** janki has joined #openstack-nova04:09
*** links has joined #openstack-nova04:09
*** jangutter has joined #openstack-nova04:15
*** mdnadeem_ has joined #openstack-nova04:18
*** mdnadeem has quit IRC04:20
*** jangutter has quit IRC04:20
*** sree has joined #openstack-nova04:22
*** dave-mccowan has quit IRC04:24
*** bhujay has joined #openstack-nova04:28
*** harlowja has joined #openstack-nova04:28
openstackgerritTakashi NATSUME proposed openstack/nova master: api-ref: Parameter verification for
*** fragatina has joined #openstack-nova04:34
*** fragatina has quit IRC04:35
*** fragatina has joined #openstack-nova04:35
*** zhurong_ has quit IRC04:39
*** psachin has joined #openstack-nova04:42
openstackgerritJackie Truong proposed openstack/nova master: Add trusted_certs object
openstackgerritJackie Truong proposed openstack/nova master: Implement certificate_utils
*** claudiub has joined #openstack-nova04:48
openstackgerritJackie Truong proposed openstack/nova master: Add trusted_image_certificates to REST API
*** zhenguo has quit IRC04:54
*** bhujay has quit IRC04:54
*** bhujay has joined #openstack-nova04:55
openstackgerritBrooks Kaminski proposed openstack/nova master: XenAPI/Stops the migration of volume backed VHDS
*** crushil has joined #openstack-nova04:58
*** ratailor has joined #openstack-nova05:07
*** jangutter has joined #openstack-nova05:10
*** sree has quit IRC05:10
*** chyka has joined #openstack-nova05:10
*** bhujay has quit IRC05:11
*** sree has joined #openstack-nova05:12
*** bhujay has joined #openstack-nova05:12
*** jangutter has quit IRC05:14
*** chyka has quit IRC05:15
*** sree has quit IRC05:16
*** harlowja has quit IRC05:17
openstackgerritTakashi NATSUME proposed openstack/nova master: Fix missing 'if_notifications_enabled' decorator
*** fragatina has quit IRC05:20
*** jaosorior has joined #openstack-nova05:23
*** hongbin has quit IRC05:25
openstackgerritBrooks Kaminski proposed openstack/nova master: XenAPI/Stops the migration of volume backed VHDS
SpazmoticMy first rebase.. kind of scary05:27
Spazmoticthings are puckered05:27
*** salv-orlando has joined #openstack-nova05:31
*** salv-orl_ has quit IRC05:31
*** suresh12 has joined #openstack-nova05:32
*** sree has joined #openstack-nova05:33
*** sree has quit IRC05:37
*** gongysh has joined #openstack-nova05:41
*** yikun_ has quit IRC05:47
*** sridharg has joined #openstack-nova05:47
*** jackie-truong has quit IRC05:48
*** bhujay has quit IRC05:53
*** crushil has quit IRC05:54
alex_xuYikun: ah, thanks05:56
*** bhujay has joined #openstack-nova06:03
*** jangutter has joined #openstack-nova06:04
*** bhujay has quit IRC06:08
*** jangutter has quit IRC06:08
*** sree has joined #openstack-nova06:09
*** xinliang has quit IRC06:09
*** suresh12 has quit IRC06:11
*** zhurong has quit IRC06:13
*** sree has quit IRC06:13
*** rcernin has quit IRC06:14
*** sree has joined #openstack-nova06:16
*** sree has quit IRC06:20
*** AlexeyAbashkin has joined #openstack-nova06:22
*** xinliang has joined #openstack-nova06:22
*** zhurong has joined #openstack-nova06:23
*** lajoskatona has joined #openstack-nova06:25
*** Spazmotic has quit IRC06:25
*** Spazmotic has joined #openstack-nova06:28
*** bhujay has joined #openstack-nova06:29
*** Spazmotic has quit IRC06:29
*** AlexeyAbashkin has quit IRC06:30
*** Spazmotic has joined #openstack-nova06:31
*** Eran_Kuris has joined #openstack-nova06:31
*** Spaz-Work has joined #openstack-nova06:37
*** Spazmotic has quit IRC06:39
*** sree has joined #openstack-nova06:40
*** Eran_Kuris has quit IRC06:41
*** Spaz-Work has quit IRC06:41
*** Spazmotic has joined #openstack-nova06:41
*** janki has quit IRC06:42
*** janki has joined #openstack-nova06:43
*** Eran_Kuris has joined #openstack-nova06:44
*** armax has quit IRC06:45
*** armax has joined #openstack-nova06:46
*** armax has quit IRC06:46
*** armax has joined #openstack-nova06:46
*** armax has quit IRC06:47
*** sree has quit IRC06:47
*** armax has joined #openstack-nova06:47
*** armax has quit IRC06:47
*** armax has joined #openstack-nova06:48
*** armax has quit IRC06:48
openstackgerritMerged openstack/nova master: add whereto for testing redirect rules
*** armax has joined #openstack-nova06:49
*** armax has quit IRC06:49
*** armax has joined #openstack-nova06:49
*** armax has quit IRC06:50
*** sree has joined #openstack-nova06:51
*** markvoelker has quit IRC06:52
*** markvoelker has joined #openstack-nova06:53
*** vladikr has quit IRC06:54
*** vladikr has joined #openstack-nova06:54
*** sree has quit IRC06:55
*** bhujay has quit IRC06:57
*** markvoelker has quit IRC06:57
*** jangutter has joined #openstack-nova06:58
*** threestrands_ has quit IRC07:02
*** gouthamr has joined #openstack-nova07:02
*** jangutter has quit IRC07:02
*** bhujay has joined #openstack-nova07:03
*** tbachman has joined #openstack-nova07:03
*** tbachman has quit IRC07:05
*** vladikr has quit IRC07:06
*** vladikr has joined #openstack-nova07:06
*** tbachman has joined #openstack-nova07:09
*** avolkov has joined #openstack-nova07:10
*** takashin has left #openstack-nova07:11
*** phuongnh has quit IRC07:11
*** acormier has joined #openstack-nova07:14
*** acormier has joined #openstack-nova07:15
*** hoonetorg has quit IRC07:17
*** tbachman has quit IRC07:18
*** chyka has joined #openstack-nova07:21
*** edand__ has joined #openstack-nova07:22
*** chyka has quit IRC07:26
*** jose-phi_ has quit IRC07:26
*** slaweq has joined #openstack-nova07:28
*** hoonetorg has joined #openstack-nova07:29
*** acormier has quit IRC07:31
*** tianhui_ has joined #openstack-nova07:31
*** slaweq has quit IRC07:32
*** jangutter has joined #openstack-nova07:32
*** slaweq has joined #openstack-nova07:32
*** tianhui has quit IRC07:34
*** jangutter has quit IRC07:36
*** hshiina has joined #openstack-nova07:46
Spazmoticjianghuaw: addressed the nits by Bob and and rebased if you get a chance on Zuul confirms it :)07:46
Spazmoticonce* zuul07:46
*** stakeda has quit IRC07:47
jianghuawSpazmotic, thanks for the update. It looks good to me:-)07:49
SpazmoticThanks so much sir!07:49
Spazmotichunting around for my next project07:49
*** amorin has left #openstack-nova07:50
*** maciejjozefczyk_ is now known as maciejjozefczyk07:51
*** abhishekk is now known as abhishekk|away07:51
jianghuawSpazmotic, you may have interesting on this patch:
*** sree has joined #openstack-nova07:54
jianghuawHuan created it for getting advices.07:54
jianghuawit's to fix a migration issue with xenapi.07:55
Spazmoticsure will take a look07:55
*** pcaruana has joined #openstack-nova07:55
jianghuawSpazmotic, thanks.07:55
SpazmoticInteresting.  We do volume based LM quite a bit but we get around this with --block-migrate for iSCSI07:57
*** jangutter has joined #openstack-nova07:57
*** sree has quit IRC07:59
jianghuawSpazmotic, iirc the issue happens only in the XS7.0 and forwarding releases.07:59
SpazmoticThat's definately going to be a problem i'll need to test out08:00
jianghuawor maybe 7.1 => . I put 7.1 in the log.08:00
jianghuawBut anyway it will be issue for latest XS releases.08:01
jianghuawfeel free to put comments in it.08:01
jianghuawIf you interest, feel free to take it over:-)08:01
SpazmoticI'll certainly look at it heavily, appeals to me greatly08:02
*** blkart has quit IRC08:02
*** tianhui has joined #openstack-nova08:04
*** AlexeyAbashkin has joined #openstack-nova08:04
*** tianhui_ has quit IRC08:05
*** mvk has quit IRC08:08
*** tianhui_ has joined #openstack-nova08:08
SpazmoticDo you know if Huan has refactored it at all recently?  Mentions they wanted to tweak the vmops method around, but i'm sure time was tight for you guys08:08
*** tianhui has quit IRC08:09
jianghuawSpazmotic, not yet I'm sure.08:14
SpazmoticAlrighty cool.. I've got some 7.1s in an env, i'll run some tests over the next few days and get familiar with these maps, havn't had to use them before08:15
SpazmoticThanks for bringing to my attention is actually more important than you may know08:15
*** jose-phillips has joined #openstack-nova08:16
*** huanxie has joined #openstack-nova08:16
*** tesseract has joined #openstack-nova08:20
*** bhujay has quit IRC08:21
*** sahid has joined #openstack-nova08:21
*** vivsoni has quit IRC08:24
*** jose-phillips has quit IRC08:25
*** edand_ has joined #openstack-nova08:25
*** jose-phillips has joined #openstack-nova08:25
*** zhaochao has quit IRC08:26
*** gongysh has quit IRC08:26
*** gcb has quit IRC08:26
*** edand__ has quit IRC08:26
*** zhaochao has joined #openstack-nova08:27
*** gcb has joined #openstack-nova08:29
openstackgerritHironori Shiina proposed openstack/nova master: virt/ironic: Implement rescue and unrescue
*** gibi has joined #openstack-nova08:30
*** janki has quit IRC08:31
*** rha has joined #openstack-nova08:31
*** rha has quit IRC08:31
*** rha has joined #openstack-nova08:31
*** gongysh has joined #openstack-nova08:33
*** takedakn has quit IRC08:34
*** sahid has quit IRC08:34
*** sahid has joined #openstack-nova08:34
openstackgerritMaciej Jozefczyk proposed openstack/nova master: Do not normalize allocation_ratios
*** kwathore has quit IRC08:36
*** vivsoni has joined #openstack-nova08:38
*** gibi has quit IRC08:39
*** gibi has joined #openstack-nova08:41
*** bhujay has joined #openstack-nova08:42
*** damien_r has joined #openstack-nova08:43
*** jpena|off is now known as jpena08:44
*** mdnadeem_ has quit IRC08:45
*** mdnadeem has joined #openstack-nova08:45
*** sree has joined #openstack-nova08:45
*** abhishekk|away is now known as abhishekk08:47
*** janki has joined #openstack-nova08:47
*** jose-phillips has quit IRC08:47
* kashyap waves08:48
*** ralonsoh has joined #openstack-nova08:48
gmannmriedem: updated those. 2 are ready to go and 1 i need more work on test which i will tomorrow -
*** sree has quit IRC08:50
* Spazmotic waves08:50
gmannmriedem: there are other work also needed on this BP which we will continue on continued Rocky BP. if you want to move current patches to new one, it should be ok08:50
*** slaweq has quit IRC08:52
gmannmriedem: or we defer the queens one to Rocky ?08:52
*** slaweq has joined #openstack-nova08:54
*** markvoelker has joined #openstack-nova08:54
*** zhurong has quit IRC08:55
*** bhujay has quit IRC08:56
*** takedakn has joined #openstack-nova09:00
kashyapIt's middle of the night for Matt, so will take a few more hours for him to come online.09:05
*** sree has joined #openstack-nova09:09
*** chyka has joined #openstack-nova09:10
*** jafeha has joined #openstack-nova09:12
*** sree has quit IRC09:14
*** jafeha__ has quit IRC09:14
*** chyka has quit IRC09:15
*** jmlowe has quit IRC09:16
*** huanxie has quit IRC09:18
*** chyka has joined #openstack-nova09:24
SpazmoticAargh Zuul why you hate me09:24
*** zhurong has joined #openstack-nova09:26
*** huanxie has joined #openstack-nova09:26
Roamer`Spazmotic, if it's any help, I doubt it's personal - it's in Zuul's nature to be very, very paranoid and suspicious of everyone, you just have to catch one of its rare good moods... :)09:27
*** markvoelker has quit IRC09:27
*** chyka has quit IRC09:28
*** amoralej|off is now known as amoralej09:29
*** gibi_ has joined #openstack-nova09:29
*** hshiina has quit IRC09:29
*** derekh has joined #openstack-nova09:29
openstackgerritZengzhi Wang proposed openstack/nova stable/pike: Make eventlet hub use a monotonic clock
*** itlinux has joined #openstack-nova09:31
*** vivsoni has quit IRC09:34
*** vivsoni has joined #openstack-nova09:35
*** vivsoni_ has joined #openstack-nova09:43
*** vivsoni has quit IRC09:47
*** huanxie has quit IRC09:48
*** blkart has joined #openstack-nova09:50
*** zhurong_ has joined #openstack-nova09:57
*** hrw has joined #openstack-nova09:57
*** gszasz has joined #openstack-nova09:57
*** huanxie has joined #openstack-nova09:58
*** itlinux has quit IRC09:58
hrwI am looking at nova's code. it adds graphics if it is needed. but I did not see where it adds usb host for usbtablet09:58
hrwdoes it assume that usb host is already present in VM configuration?09:58
Roamer`OK, so the StorPool volume attachment driver finally passed the Zuul check pipeline (after the Cinder backup bug was fixed)... I'm pretty sure mriedem intends to +W it once he wakes up and sees it's passed the tests, but the gate being what it is, I'd be really obliged if somebody else could +W it earlier :)  Of course, "let's just wait for him" is a very reasonable option.09:59
Roamer`oof, and that would be of course10:00
*** kevinz has joined #openstack-nova10:01
*** itlinux has joined #openstack-nova10:01
*** bhujay has joined #openstack-nova10:03
kevinzalex_xu: Could you review if you have time? Stephen already give +2 :-)10:03
*** yamamoto has quit IRC10:04
*** bkopilov has quit IRC10:05
openstackgerritBalazs Gibizer proposed openstack/nova master: Transform instance.exists notification
*** bkopilov has joined #openstack-nova10:06
alex_xukevinz: got it10:08
*** itlinux has quit IRC10:09
*** hoonetorg has quit IRC10:09
*** jchhatbar has joined #openstack-nova10:10
hrwtracking how nova gets guest components suxx10:11
*** janki has quit IRC10:11
stephenfinmriedem, melwitt: Yeah, sorry about I was the approver and that's my bad. Won't happen again10:13
*** jchhatbar has quit IRC10:14
hrwok. nova assumes that usb host is present10:18
kashyapstephenfin: mriedem: As I've said in the past, I don't quite agree with that line of thinking.  Happy to also elaborate why in Dublin.10:19
mdboothmriedem: melwitt stephenfin I wonder if that policy of old might be up for review, especially in a case like this which is uncontroversial. IIRC the understanding came about after RH was accused of railroading a particular controversial patch. That's an entirely valid concern, but I don't think anybody is claiming it happened here.10:19
kashyap(It somehow seem to imply / assume "malice")10:20
* hrw -> bugtracker10:20
kashyap(But it _should_ be a concern, if it's a _pattern_ of two people from the same company ACKing changes in rapid succession without giving time for others to chime in.)10:20
kashyap(Or for some goddamned deadline's sake.)10:20
stephenfinmdbooth: Indeed, but the discussion should have happened _before_ /me did that :) EAFP probably doesn't apply where matters of trust are involved10:21
stephenfinA good PTG topic though, for sure10:21
kashyapSure.  But I do agree with Matt that, we shouldn't be too eager to push something for a deadline10:22
*** hoonetorg has joined #openstack-nova10:22
kashyapBecause, let's be honest --- most deadlines milestones don't matter.  It's all artificial lines in the sand.10:22
mdboothstephenfin: There was a spec, many rounds of review, at least 3 ML posts that I recall over a period of a few weeks, and additional thorough review from 2 core reviewers, including yourself. I resent the implication, tbh.10:23
*** markvoelker has joined #openstack-nova10:25
kashyapYeah, I recall the history / and even postmortem done by markmc.10:25
kashyap -- [openstack-dev] [nova] libvirt version_cap, a postmortem10:25
kashyapmdbooth: Ah, I thought you were talking about a previous case which caused this no two +2s from the same company thing.)10:25
mdboothThe previous case was also a misunderstanding, tbh, although there was more substance to it.10:27
kashyapYeah, absolutely it was a case: "too eager to misunderstand"10:27
hrw - what should I add to get it more complete?10:27
openstackLaunchpad bug 1745340 in OpenStack Compute (nova) "Nova assumes that USB Host is present" [Undecided,New]10:27
stephenfinMore complete?10:28
hrwstephenfin: more useful to anyone who would look at issue10:29
hrwif anyone at all ;D10:29
stephenfinI think you've pretty much got it10:30
*** liuzz has quit IRC10:30
stephenfinI assume the solution would be to check if we have a USB hub and either (a) add it if not or (b) refuse to start with a USB tablet?10:30
stephenfin*refuse to start with a USB tablet if the platform doesn't support the USB hub10:31
stephenfinhrw: I can point you to the code that needs modification, if you fancy the challenge? ;)10:31
hrwstephenfin: please do. in a bug comment10:31
hrwstephenfin: it may end on my or kevinz's plate or maybe we assign it to someone else at Linaro10:32
hrwstephenfin: imho the proper way would be to add usb-host device if not present10:32
*** acormier has joined #openstack-nova10:32
hrwas the goal is to have graphical console in Rocky (as it is too late for Queens)10:33
*** jpena is now known as jpena|off10:33
*** jpena|off is now known as jpena10:33
hrwduring ptg I have to track nova sticker for my laptop ;D to put it next to kolla one10:34
openstackLaunchpad bug 1745340 in OpenStack Compute (nova) "Nova assumes that USB Host is present" [Undecided,New]10:36
* stephenfin notices some cleanup that should have been done in Ocata there10:36
hrwstephenfin: exactly where I was looking at10:37
*** acormier has quit IRC10:37
*** lucas-afk is now known as lucasagomes10:37
hrwstephenfin: probably some code in nova.virt.libvirt.* too10:37
hrwto 'plug' host10:37
hrwwill later look into10:38
* hrw -> lunch10:38
*** abhishekk has quit IRC10:38
SpazmoticCome on tempest... you running slow but i have faith in you10:38
*** edand_ has quit IRC10:41
*** hrw has quit IRC10:43
*** edand_ has joined #openstack-nova10:44
*** annp has quit IRC10:46
*** mdnadeem has quit IRC10:48
*** jistr is now known as jistr|mtg10:49
*** gongysh has quit IRC10:54
*** kevinz has quit IRC10:55
*** sree has joined #openstack-nova10:56
*** markvoelker has quit IRC10:58
*** sree has quit IRC11:01
*** yamamoto has joined #openstack-nova11:05
*** bhujay has quit IRC11:07
*** sree has joined #openstack-nova11:08
*** hrw has joined #openstack-nova11:09
*** huanxie has quit IRC11:09
*** bhujay has joined #openstack-nova11:12
*** sree has quit IRC11:13
*** chyka has joined #openstack-nova11:13
*** rmart04 has joined #openstack-nova11:16
*** gcb has quit IRC11:17
*** yamamoto has quit IRC11:18
*** chyka has quit IRC11:18
*** gcb has joined #openstack-nova11:20
*** takedakn has quit IRC11:25
*** alexchadin has joined #openstack-nova11:31
*** yamamoto has joined #openstack-nova11:31
*** namnh has quit IRC11:31
*** jangutter has quit IRC11:32
*** tiendc has quit IRC11:35
*** zhurong has quit IRC11:35
*** gibi has left #openstack-nova11:36
*** sapd_ has quit IRC11:45
stephenfinsean-k-mooney: Think I've reviewed all of bp/enable-sriov-nic-features now :)11:46
stephenfinLemme know if you have questions11:46
*** gaoyan has joined #openstack-nova11:48
*** zhurong has joined #openstack-nova11:55
*** markvoelker has joined #openstack-nova11:55
Spazmotichey huanxie you still around11:55
Spazmoticor jianghuaw can you explain to me what XS expects of these VDI MAPS on a live migration?11:56
*** gaoyan has quit IRC11:57
*** AlexeyAbashkin has quit IRC11:58
*** AlexeyAbashkin has joined #openstack-nova11:58
*** sapd_ has joined #openstack-nova11:58
*** tetsuro has quit IRC12:01
*** smatzek has joined #openstack-nova12:02
*** jangutter has joined #openstack-nova12:03
*** jpena is now known as jpena|lunch12:03
jianghuawSpazmotic, I've talked with Huan. You can chat with her on tomorrow.12:06
SpazmoticSounds good, thanks, i'll stop by tomorrow :)12:06
lyarwoodmriedem: iirc live snapshot was only for local images and not volumes right? Even then we skip it for rbd?12:06
*** jangutter has quit IRC12:07
SpazmoticIBM PowerKVM CI failing constantly12:12
*** sree has joined #openstack-nova12:14
SpazmoticLooks like past 4 or 5 hours all failures on its tempest-dsvm-full-xenialchecks12:17
sean-k-mooneystephenfin: just in so ill take a look at them now12:17
*** janki has joined #openstack-nova12:19
*** gouthamr has quit IRC12:19
*** sree has quit IRC12:19
*** gouthamr has joined #openstack-nova12:20
*** tesseract has quit IRC12:24
owalshstephenfin: is vencrypt_ca_certs required when vencrypt is enabled? doesn't looks like it default to the system ca certs so I'm guessing yea12:27
*** janki has quit IRC12:28
*** markvoelker has quit IRC12:29
*** yamamoto has quit IRC12:31
*** yamamoto has joined #openstack-nova12:31
*** sambetts|afk is now known as sambetts12:35
*** alexchadin has quit IRC12:35
*** alexchadin has joined #openstack-nova12:36
*** hrw has quit IRC12:38
*** jistr|mtg is now known as jistr12:38
*** hrw has joined #openstack-nova12:38
openstackgerritBrianna Poulos proposed openstack/nova master: Reduce complexity of _from_db_object
openstackgerritGaudenz Steinlin proposed openstack/nova master: Ignore some updates from virt driver
*** jangutter has joined #openstack-nova12:39
*** tianhui has joined #openstack-nova12:45
*** tianhui_ has quit IRC12:48
*** takashin has joined #openstack-nova12:50
*** yamamoto has quit IRC12:52
*** alexchadin has quit IRC12:52
*** alexchadin has joined #openstack-nova12:53
*** ratailor has quit IRC12:54
*** alexchadin has quit IRC13:00
*** alexchadin has joined #openstack-nova13:00
*** lpetrut has joined #openstack-nova13:03
*** bhujay has quit IRC13:03
*** belmoreira has joined #openstack-nova13:04
*** dave-mccowan has joined #openstack-nova13:05
*** kevzha01 has joined #openstack-nova13:05
*** yamamoto has joined #openstack-nova13:07
*** kevzha01_ has joined #openstack-nova13:07
*** elod has quit IRC13:09
*** kevzha01 has quit IRC13:10
*** zhurong has quit IRC13:16
*** elod has joined #openstack-nova13:16
*** acormier has joined #openstack-nova13:16
*** alexchadin has quit IRC13:18
*** acormier has quit IRC13:19
ameedajaypipes: Sir are you around ?13:20
openstackgerritBrianna Poulos proposed openstack/nova master: Add trusted_certs object
openstackgerritBrianna Poulos proposed openstack/nova master: Implement certificate_utils
openstackgerritBrianna Poulos proposed openstack/nova master: Add trusted_image_certificates to REST API
openstackgerritBrianna Poulos proposed openstack/nova master: Add trusted_certs to instance_extra
*** pchavva has joined #openstack-nova13:22
*** abhishekk has joined #openstack-nova13:24
*** elod has quit IRC13:25
*** elod has joined #openstack-nova13:25
*** markvoelker has joined #openstack-nova13:26
*** tesseract has joined #openstack-nova13:27
*** liverpooler has joined #openstack-nova13:28
*** sree has joined #openstack-nova13:30
*** edmondsw_ has joined #openstack-nova13:30
*** liusheng has quit IRC13:32
*** sree has quit IRC13:39
stephenfinowalsh: I think so, yup. I should probably encode that13:41
*** yamamoto has quit IRC13:42
*** tesseract has quit IRC13:42
*** cleong has joined #openstack-nova13:42
*** tesseract has joined #openstack-nova13:42
*** tetsuro has joined #openstack-nova13:44
*** zhurong_ has quit IRC13:44
*** Yikun has quit IRC13:47
*** Yikun has joined #openstack-nova13:47
*** sree has joined #openstack-nova13:48
*** edmondsw_ is now known as edmondsw13:48
*** alexchadin has joined #openstack-nova13:49
*** eharney has quit IRC13:49
*** acormier has joined #openstack-nova13:51
SpazmoticIBM CI what is wrong with you :(
mmedvedeSpazmotic: it should be fixed now, at least the original problem that was causing earlier failures13:52
*** Tom-Tom has joined #openstack-nova13:52
SpazmoticAhhh thanks mmedvede13:52
SpazmoticI'll give it a new recheck hehe13:52
*** sree has quit IRC13:52
*** alexchadin has quit IRC13:52
mriedemRoamer`: i won't be +Wing your change because we require 2 +2s13:53
*** acormier has quit IRC13:54
*** esberglu has joined #openstack-nova13:55
*** bhujay has joined #openstack-nova13:56
mriedemgibi_: we should probably consider bumping the nova functional job timeouts as a change on top of your patch to move them into the nova tree13:58
*** gongysh has joined #openstack-nova13:58
*** weshay|rover is now known as weshay|ruck13:58
*** jmlowe has joined #openstack-nova13:59
*** markvoelker has quit IRC13:59
hrwstephenfin: turns out that one change to libvirt makes life easier.14:00
hrwstephenfin: will wait for libvirt devs to respond before testing it further14:00
mriedemnova meeting starting in #openstack-meeting14:00
gibi_mriedem: I can do that14:00
gibi_mriedem: is it just the functional or also the functional-py35 job that times out frequently?14:01
*** lyan has joined #openstack-nova14:01
*** david-lyle has quit IRC14:01
gibi_mriedem: OK14:01
*** psachin has quit IRC14:02
*** yamamoto has joined #openstack-nova14:02
openstackgerritStephen Finucane proposed openstack/nova master: Don't filter out sibling sets with one core
openstackgerritStephen Finucane proposed openstack/nova master: Ensure emulator threads are always calculated
openstackgerritStephen Finucane proposed openstack/nova master: Always pass 'NUMACell.siblings' to _pack_instance_onto_cores'
*** esberglu has quit IRC14:08
*** jose-phillips has joined #openstack-nova14:09
*** kevzha01__ has joined #openstack-nova14:09
*** tbachman has joined #openstack-nova14:09
*** kevzha01_ has quit IRC14:12
*** huanxie has joined #openstack-nova14:13
*** kevzha01_ has joined #openstack-nova14:13
stephenfinmriedem: Could you take a look at this after the meeting? It's blocking
stephenfinMaybe gibi_ too, if you have the time ^14:14
mriedemstephenfin: trade you
stephenfinSounds good14:16
*** kevzha01__ has quit IRC14:16
efriedholy crap, that one's >3 years old14:16
*** tesseract has quit IRC14:17
*** tesseract has joined #openstack-nova14:18
mriedemb/c the cinder driver for it was removed awhile back14:19
*** sahid has quit IRC14:19
Roamer`stephenfin, efried, we (StorPool) and, actually, also Canonical would be very, very happy for if this could go in14:20
*** sahid has joined #openstack-nova14:20
Roamer`I honestly hadn't realized that it needed a second +2 vote, my bad, my very bad :(14:20
Roamer`and yeah, there's history... we tried submitting a driver too late for Juno, then it went into Kilo, then we messed up and it was removed, and now we're trying to get it back14:21
Roamer`aaaaand I'll shut up now14:21
kashyapmriedem: Heya, so...I wrote this:  I agree with you on most / all topics.  Besides this.  Wonder what you think of my POV there.14:22
*** smatzek has quit IRC14:24
*** smatzek has joined #openstack-nova14:24
*** smatzek has quit IRC14:24
*** esberglu has joined #openstack-nova14:24
*** mgoddard_ has joined #openstack-nova14:24
*** huanxie has quit IRC14:25
*** alexchadin has joined #openstack-nova14:25
openstackgerritsahid proposed openstack/nova master: libvirt: disconnect volume from host during detach
-openstackstatus- NOTICE: We're currently experiencing issues with the server which will result in POST_FAILURE for jobs, please stand by and don't needlessly recheck jobs while we troubleshoot the problem.14:26
gibi_stephenfin: lgtm! +214:26
*** lucasagomes is now known as lucas-hungry14:27
*** Yikun has quit IRC14:27
*** psachin has joined #openstack-nova14:27
efriedgibi_ But don't +W it, cause the gate's broken.  Or whatever.14:27
*** Yikun has joined #openstack-nova14:27
gibi_efried: I was the first +214:28
*** burt has joined #openstack-nova14:29
*** jose-phillips has quit IRC14:29
*** gibi_ is now known as gibi14:29
*** elmaciej has joined #openstack-nova14:30
*** kevzha01 has joined #openstack-nova14:31
*** psachin has quit IRC14:32
*** yangyapeng has quit IRC14:32
*** amodi has joined #openstack-nova14:32
*** jaypipes has quit IRC14:32
*** psachin has joined #openstack-nova14:33
*** kevzha01_ has quit IRC14:33
*** markvoelker has joined #openstack-nova14:34
*** eharney has joined #openstack-nova14:34
*** jpena|lunch is now known as jpena14:35
gibistephenfin: could you look at these? to easy notification transformation before FF :)14:35
mriedemgibi: stephenfin: i'll look at again,14:35
mriedemsince i was -1 on it14:35
*** edand_ has quit IRC14:35
gibimriedem: thanks14:36
stephenfingibi: Sure, it's been on my list14:36
takashinmriedem: Thank you.14:36
stephenfinAlong with some of gmann's API stuff14:36
gibistephenfin: thanks14:36
*** jaypipes has joined #openstack-nova14:37
* dansmith yawns14:38
ameedahello, how to run pep8 for one class ?14:39
ameedatox -e pep8 -- -n nova/tests/unit/compute/ , doesn't work14:39
maciejjozefczykameeda: tox -e pep8 nova/virt/libvirt/ ?14:41
mriedemkashyap: replied14:41
stephenfin^ that14:41
ameedamaciejjozefczyk: Thanks, let me try14:42
SpazZzWhy is it still 40 degrees in south texas14:42
ameedamaciejjozefczyk: Thanks ^_^14:42
efriedameeda You can run flake8 directly from the pep8 venv.  source .tox/pep8/bin/activate; flake8 path/to/ path/to/ etc.py14:42
SpazZzOne more day of this and i'll die14:42
SpazZzthems the rules14:42
*** tetsuro has quit IRC14:42
efriedSpazZz ++14:42
ameedathanks all14:42
maciejjozefczykameeda: :)14:42
sean-k-mooneyameeda: if you have commited the chagne you can use tox -e fast8 to just run tox on the files you changed in the last commit14:42
maciejjozefczyksean-k-mooney: good to know14:43
SpazZzIf any cores got some cycles today for this would be much appreciated.  +1'd by two citrix folks, need mah +2 love now :)
efriedAnd to nerd out just a little bit more, to check the last e.g. 6 commits in a series, I do this from my pep8 venv: git diff --name-only HEAD~6 | grep '\.py$' | xargs flake814:43
mdboothsean-k-mooney: If that's the same as 'tox -e pep8 -- -HEAD' it will also include uncommitted changes in the working directory.14:44
SpazZzIgnore the IBM.. it'll go away eventually :P14:44
ameedasean-k-mooney: thank you, I appreciate all your help14:44
mdboothefried: Nice...14:44
sean-k-mooneymdbooth: oh am yes it is. i sometiems have line lenght issue in my commit message so i prefer to commit first out of habbit14:45
mriedemlyarwood: mdbooth: regarding - since it knowingly regresses swap volume with a luks-encrypted volume, is anyone working on a nova patch to workaround that and fallback to the old non-native routine?14:45
mdboothmriedem: I responded to that on the ML14:45
mriedemlyarwood: mdbooth: because it's getting pretty late to just merge this and put something in the release notes about oopsies14:45
mdboothI don't really regard it as a regression, tbh, as it was entirely broken until this Monday14:45
mdboothAnd there's already an upstream bug tracking it in libvirt14:46
jaypipesmriedem, efried: good morning fellas. I have a few hours before I need to go to the airport. so I'll try to focus on patch reviews during this time. do we have a priority list?14:46
jaypipesdansmith: you too..14:46
*** ralonsoh_ has joined #openstack-nova14:46
*** Adri2000 has left #openstack-nova14:46
mdboothmriedem: We'd be adding in workaround code for something which needs an underlying fix.14:46
mriedemmdbooth: then we need something more discoverable than a small note in a release note,14:47
*** kevzha01 has quit IRC14:47
mriedeme.g. maybe a docs update to cinder saying that you can't migrate/retype an encrypted volume for now14:47
efriedjaypipes The update_provider_tree series is ready up to update_from_provider_tree.  There's some possibly-controversial stuff towards the top.  Not sure how much chance of getting approved before FF, but at least worth skimming.14:47
mdboothmriedem: Well you never could...14:48
jaypipesmriedem, efried: I'm also continuing my patch series on the alloc candidates and n-r-p. been slow going because of $work the last two days14:48
mdboothmriedem: It's only a disappointment to anybody running master who updated on Monday14:48
*** ralonsoh has quit IRC14:48
jaypipesefried: what's the latest status on alex_xu's patches around traits in alloc candidates?14:48
efriedHeh.  $work meaning work you get a paycheck for?  :)14:48
jaypipesefried: yeah, that :)14:48
mriedemmdbooth: ok - my point is, i don't know if no one else in the world knows you never could14:48
efriedjaypipes I believe we're +W and in recheck hell, but that could just be the bottom patch.14:48
mriedemso, docs would be a good start14:48
mriedembecause users of openstack do'nt know, or shouldn't need to know, low level details about how the code works14:49
jaypipesefried: the REST API change one?14:49
alex_xujaypipes: yes, all of those patches get approved...then recheck and recheck...14:49
jaypipesalex_xu: gotcha, k, thx for the update14:49
*** rmcall has joined #openstack-nova14:49
stephenfinRoamer`, mriedem: That's done now. There's some cleanup/simplification that can be done, but that's a separate effort14:49
mdboothmriedem: Critically we're not regressing.14:50
jaypipesmriedem: depends. if you've an NFV user, of *course* you need to know all the low-level details of everything going on. ;)14:50
*** andreas_s has joined #openstack-nova14:50
Roamer`stephenfin, thanks a lot!  And, yes, good catch on the exception...14:51
mriedemjaypipes: yeah that's true14:52
mdboothmriedem: The way I see it is this implementation is much simpler and more robust for Nova, as I mentioned on the ML.14:52
mriedemjaypipes: well, except for the part where you just tell nova to put the instance on this host and this numa node and this set of cpus, totally bypassing the scheduler14:52
jaypipesefried: is the recheck hell due to timeouts in zuul jobs or is it due to a particular bug in a dependency (or nova etc)14:52
jaypipesmriedem: right. NFV.14:52
efriedjaypipes The latest and greatest is the log server went read-only.  We're supposed to not recheck anything until they've sent another broadcast saying it's fixed.14:53
mriedemjaypipes: there was a big regression on tuesday that caused most dsvm jobs to fail. the fix for that merged yesterday.14:53
mdboothWe can add complexity to work around a temporary limitation in the underlying library, but given that the feature in question never worked anyway I think it would be better just to note that it doesn't work.14:53
jaypipesmriedem: that's pretty much what I've been talking about at this $work thing for 2 days.14:53
mriedemhowever, still hitting job timeouts and post failures14:53
jaypipesefried: ack14:53
mdboothmriedem: Specifically, I don't want to add yet anothing config option for this: it should be transparent to the operator.14:53
efriedjaypipes I also noticed (not sure if there's actual cause->effect here) that rechecking jobs *later* in The Big Series caused the bottom patch to stall in the gate.14:54
efriedjaypipes So I think whenever stuff opens up again, I'm going to try just checking the bottom one.14:54
jaypipesnot sure I've personally seen that yet.14:54
efriedyeah, it was weird.  The bottom one was in the gate and blue worms creeping along, looking good.  Then I rechecked some above it.  And then the bottom one stopped having blue worms (but stayed in the gate queue).14:55
mriedemmdbooth: i'm not asking for a config option14:55
efried...where it has been for ~11.5h now14:55
*** zhaochao has quit IRC14:56
openstackgerritAmeed Ashour proposed openstack/nova master: detaching volumes when deleted VM is in error state
openstackgerritAmeed Ashour proposed openstack/nova master: detach instance volumes when VM creation fails
*** alexchadin has quit IRC14:57
mdboothmriedem: It's not a regression though, right? Specifically because it was always broken. There is nobody currently using this feature who will have their workflow broken by this change.14:57
*** Guest14094 has quit IRC14:58
mriedemmdbooth: if we're not going to put some kind of conditional fallback code in place, then i don't think the release note should mention it really - it's confusing and alarming if you didn't realize this iddn't work until recently; and on top of that, we likely need an api-ref update to cinder to mention that retype/migration of an attached encrypted volume won't work on the compute side because of some bug14:58
mriedems/of some bug/it's not yet supported/14:59
mdboothmriedem: Well we discovered that swap volume was broken in review, about a month ago. Lee opened a bug about it.14:59
*** yangyapeng has joined #openstack-nova14:59
* mdbooth finds the bug14:59
*** jackie-truong has joined #openstack-nova15:00
ameedaplease review my code for this two gerrit and , thanks in advance :)15:00
*** smatzek has joined #openstack-nova15:00
openstackLaunchpad bug 1739593 in OpenStack Security Advisory "Swapping encrypted volumes can lead to data loss and a possible compute host DOS attack" [Undecided,Incomplete]15:00
*** mlavalle has joined #openstack-nova15:00
mdboothYeah, that's the one15:00
mriedemok so that says fix released15:00
mdboothYep, last Monday15:00
*** smatzek has quit IRC15:00
*** smatzek has joined #openstack-nova15:01
mriedemso if i'm a new user to openstack and trying to swap an encrypted volume with libvirt, and it doesn't work, i might look for a bug and find this and it says it's fixed,15:01
dansmithbauzas: are you around today?15:01
mriedembut now my operator is running new enough libvirt/qemu for the native stuff, and it doesn't work15:01
*** takashin has left #openstack-nova15:01
mriedemwhich is a new bug/regression technically15:01
*** Guest14094 has joined #openstack-nova15:01
mriedemmdbooth: so what do you tell that user?15:01
mriedem"it never worked, until it did, and then it didn't again"15:01
mdboothI think we're splitting hairs here, as nobody at all will have used it in the intervening period, and it will be fixed upstream at some point anyway.15:02
*** r-daneel has joined #openstack-nova15:02
mdboothAnd as I said, this does simplify Nova.15:02
mriedemmdbooth: "fixed upstream at some point" in libvirt/qemu could be ages before someone actually picks up the versions of those packages that have those fixes, that's not using RHOSP15:02
mriedemmdbooth: if there is a bz for libvirt/qemu, then that's the known issue,15:03
mriedemand i think we should have something in the docs about this is a known limitation15:03
mdboothmriedem: It's linked in a code comment15:03
mriedemjesus christ15:03
mriedemend users shouldn't have to read code15:03
mriedemnevermind, i'll push the patch15:04
mdboothmriedem: Thanks.15:04
mriedemwhat's the bz?15:04
*** belmoreira has quit IRC15:05
*** tobasco has quit IRC15:05
mdboothmriedem: Incidentally, swap volume is also broken for multi attach.15:05
mdboothIt's generally a difficult feature.15:06
*** _pewp_ has quit IRC15:06
mdbooth bug 760547 in libvirt "[RFE] specifying the entire image chain as a qemu drive (blockdev-add) (libvirt)" [Medium,Assigned] - Assigned to pkrempa15:07
mriedemhow is swap volume broken for multiattach?15:07
*** _pewp_ has joined #openstack-nova15:07
mdboothmriedem: Do we not disable swap volume on multiattached volumes?15:07
kashyapThe above bugzilla in turn depends on libvirt wiring up the new QMP command 'blockdev-add'15:07
* mdbooth remembers pointing out in spec review that we needed to do that.15:07
kashyap(New from QEMU 2.9 onwards.)15:07
mdboothmriedem: So, swap volume does a block rebase in Nova, which is local to a single instance.15:08
*** pooja-jadhav has joined #openstack-nova15:08
mdboothIt makes a bunch of assumptions about things not changing under it, which are broken by multi attach.15:08
mdboothSo if you do a volume migration on a multi-attach volume, that would cause data corruption.15:08
mdboothIncidentally, I had some ideas about how you might make it work but they're crazy complex. Simplest just to disable it, which is what I assumed we'd done.15:10
*** tobasco has joined #openstack-nova15:10
kashyapmriedem: Swapping an ecrypted volume won't be solved soon -- as it requires libvirt doing some surgery as noted above.15:10
*** pooja_jadhav has quit IRC15:10
kashyapNova _swap_volume() method uses blockRebase() API.  And libvirt's current notion of encryption key is persistent only to the active layer.15:11
*** tidwellr has joined #openstack-nova15:11
kashyapWhere 'active layer' means, given  A.raw <-- B.qcow2 (live)15:11
mriedemkashyap: that's why i said, "meh, it'll eventually be fixed upstream so who cares" isn't a good response to a user15:11
kashyapB is the "active layer", where A is the backing file of B.15:11
openstackgerritBalazs Gibizer proposed openstack/nova master: Bumping functional test job timeouts
gibimriedem: ^^15:12
mriedemmdbooth: we don't currently block swap volume with a multiattach volume15:12
*** patriciadomin has joined #openstack-nova15:12
mriedemon the nova side15:12
mdboothmriedem: Cool, so we just prevent it? I didn't look specifically last time I reviewed that patch.15:13
mriedemi believe on the cinder side, you can't retype a multiattach volume that is attached to an instance15:13
mriedemjgriffith: right ^ ?15:13
mriedemmdbooth: i said we don't bloc kit15:13
mriedemmeaning we allow it15:13
mdboothmriedem: Ah...15:14
mriedemmdbooth: if the volume's secondary attachments are only read-only, it shouldn't be an issue right?15:14
mdboothmriedem: Well that's going to get a bit weird, but it won't cause data corruption.15:14
mdboothAlthough presumably they're also going to have to swap volume, because the old one goes away, right?15:15
*** andreas_s has quit IRC15:15
mriedemnot parsing that15:15
mriedemsee L50 here for what i'm talking about
*** andreas_s has joined #openstack-nova15:16
*** lajoskatona has quit IRC15:18
*** andreas_s_ has joined #openstack-nova15:18
mdboothThe way swap volume works is we create a local COW for the volume, which means we're no longer making changes to it. We (Nova) then copies all the data from the source to the dest. During this copy, all writes are written to the COW, which is local to a compute. When the copy is complete, we commit all the changes in the COW to the new copy, then flip to the new copy.15:18
*** janki has joined #openstack-nova15:18
openstackgerritJackie Truong proposed openstack/nova master: Add trusted_certs object
mriedem"Again NOTE we will NOT allow retype of multiattachment setting for an in-use volume."15:18
*** andreas_s has quit IRC15:18
mdboothmriedem: Right. If I'm understanding you correctly we missed that?15:18
mriedemit's supposed to be handled on the cinder side15:19
mriedemi don't know if that includes volume migration15:19
*** giodeco has joined #openstack-nova15:19
mriedemwhere it's not a retype15:19
mriedemjgriffith: smcginnis: ildikov: ^15:19
jgriffithmriedem: it's set in the policy15:19
*** gongysh has quit IRC15:19
mriedemjgriffith: i thought we said you simply couldn't retype a multiattach in-use volume, regardless of policy15:19
jgriffithmriedem: You can if policy permits, default policy being admin/owner15:19
jgriffithmriedem: Oh15:20
jgriffithyes you're correct15:20
mriedemok cool15:20
jgriffithsorry, I was confusing the bootable policy15:20
mriedemjgriffith: and does that apply to both retype and migration?15:20
jgriffithYou're correct, in-use should just return and error immediately15:20
*** pcaruana has quit IRC15:20
giodecoWhen trying to create a new instance on my Newton Openstack env,  I have an error in /var/log/nova-conductor.api of type "Corrupt image download. Checksum was d41d8cd98f00b204e9800998ecf8427e expected f8ab98ff5e73ebab884d80c9dc9c7290". I believe it is something related to network or rabbitmq. Any suggestion?15:20
mriedemjackie-truong: ^15:21
jgriffithmriedem: that's a great question :)15:21
mriedemjgriffith: heh :) basically anything that triggers a call to nova's swap volume api15:21
*** armax has joined #openstack-nova15:22
jackie-truongmriedem: which question are you pointing me to?15:22
mriedemjackie-truong: the image checksum one15:22
mriedemsounds like a john hopkins image signature validation question15:22
*** lucas-hungry is now known as lucasagomes15:22
jackie-truongit does15:23
mdboothmriedem: Bizarrely, we've also had users call it directly. However, I don't think that would cause a data corruption issue specifically.15:23
jackie-truonglemme look15:23
*** abhishekk has quit IRC15:23
mriedemmdbooth: it's an admin-only api in nova by default15:24
jgriffithmriedem: so currently anything that calls retype on a multiattach that's in use is going to return an exception15:24
mriedemso normal end users shouldn't be able to do that15:24
jgriffithmriedem: regardless of *how* it tries to do it15:24
mriedemjgriffith: does volume migration go through the retype flow even if it's not a retype?15:24
jgriffithmriedem: nope15:24
mriedemsort of like how we call cold migration a resize in nova15:24
jgriffithI don't know that swap ever goes through a cidner retype?15:24
jgriffithremember it's cinder's retype with migration that calls swap and get us in that weird case15:25
*** alexchadin has joined #openstack-nova15:25
openstackgerritBalazs Gibizer proposed openstack/nova master: DNM: test if tempest-full is triggered or not
mriedemjgriffith: looks like nova's swap volume is called from VolumeManager._migrate_volume_generic15:26
*** damien_r has quit IRC15:26
mdboothmriedem: Anyway, I only brought that up to point out that it's not without precedent. Ideally it would work, but given that it has never worked I think it's a simple enough message to users that: 'This doesn't work'. In the meantime we take the reduced complexity in Nova and wait for the upstream fix.15:26
mriedemcalled from migrate_volume15:26
jgriffithmriedem: right, the live-migration of volumes is the trigger15:26
jgriffitheither from the Cinder side or the Nova side; but not the swap call itself15:27
*** itlinux has joined #openstack-nova15:27
jgriffithmriedem: what I was getting at is that the swap process itself takes two volumes so that's not an issue; migrate is another story15:28
mriedemyeah ok i found in the volume manager code where retype calls migrate_volume15:29
mriedemand then there is just the migrate_volume called directly from the api15:29
jgriffithmriedem: my point is that I don't think there is ever a case where Nova calls cinders retype is there?15:29
mriedemi'm only worried about the case that cinder calls nova15:30
mriedemto swap with a multiattach volume15:30
jgriffithmriedem: yeah, but in that case Cinder should've already completed (or failed) to do the work of creating the destination volume etc15:30
mriedemand this is the retype check in the API
jgriffithso we should be *ok*15:30
*** dave-mccowan has quit IRC15:31
mriedemjgriffith: isn't ^ saying, that if i'm retyping from a multiattach=True to a multiattach=True type, we don't care if it's attached or not?15:31
jgriffithmriedem: right, no options there; it just won't let you do it with in-use15:31
mriedemwell, the in-use check is only if the src/dst multiattach values are changing15:31
jgriffithmriedem: yes15:32
jgriffithmriedem: yes to the if the multiattach setting is the same we don't care15:32
jgriffiththen we behave like any other retype command15:32
mriedemok so we can call swap_volume in nova for a retype/migrate on a multiattach volume15:33
mriedemmdbooth: ^15:33
mriedemwe == cinder15:33
SpazZzhehe when yo uassociate too closely to your code15:33
openstackgerritJackie Truong proposed openstack/nova master: Add trusted_certs to instance_extra
mdboothmriedem: Yeah, that would definitely cause a problem.15:34
mdboothSpazZz: hehe15:34
mdboothI think you're right though, that cinder is probably best placed to handle that.15:35
*** dave-mccowan has joined #openstack-nova15:35
mriedemwell i'll try to write up a cinder api-ref docs patch15:36
mriedemfor all the things15:36
mdboothjgriffith: Incidentally, this might be handled by context, but at first glance it looks like you've got a race there even if you fix the multiattach check.15:37
*** david-lyle has joined #openstack-nova15:38
mdboothWe're checking that volume.status is available, and if it is then we're proceeding. However, I don't see any assertion that volume.status continues to be available.15:38
mdboothIn Nova we typically do this by atomically changing task state.15:39
mriedemmdbooth: they do that here
mdboothSo we'd do: if instance.status == 'all good': atomic_update(instance.task_state = 'doing my thang', expected task_state = 'whatever')15:40
jgriffithmdbooth: We have a conditional on the volume object IIRC15:40
jgriffithmdbooth: after that yeah we don't lock or anything if that's what you mean15:41
openstackgerritJackie Truong proposed openstack/nova master: Implement certificate_utils
openstackgerritJackie Truong proposed openstack/nova master: Add trusted_image_certificates to REST API
SpazZzIBM CI looks like it's cleaning up.. can hopefully get the my +1 back and look sexier for the Core Reviewers15:41
SpazZzWalk by and be like, "girlll look at all these +1's over here.."15:42
openstackgerritJackie Truong proposed openstack/nova master: Add trusted_certs object
*** belmoreira has joined #openstack-nova15:42
mdboothjgriffith: Ah, yes. So in this case we'd want to modify 'expected' on line 1685 to contain only available.15:43
mdboothAssuming conditional_update is an atomic CAS, that would completely close the race window.15:43
mdboothBut yeah, it's currently very short.15:44
*** jangutter has quit IRC15:44
*** alexchadin has quit IRC15:45
*** elmaciej has quit IRC15:46
jgriffithmdbooth: yeah, IMO there's probably going to be some additions/fixes that are more critical to this before we get to things like closing that up.15:47
mdboothjgriffith: Easy to close, hard to spot...15:48
*** sree has joined #openstack-nova15:49
*** bpoulos has joined #openstack-nova15:50
*** damien_r has joined #openstack-nova15:50
*** rmart04 has quit IRC15:50
*** Guest14094 has quit IRC15:51
*** hongbin has joined #openstack-nova15:52
*** matrohon has joined #openstack-nova15:53
*** sree has quit IRC15:53
*** jackie-truong has quit IRC15:57
*** slaweq has quit IRC15:57
*** slaweq has joined #openstack-nova15:58
*** elmaciej has joined #openstack-nova16:00
*** belmoreira has quit IRC16:00
*** armax_ has joined #openstack-nova16:01
*** smatzek has left #openstack-nova16:01
-openstackstatus- NOTICE: is stabilized and there should no longer be *new* POST_FAILURE errors. Logs for jobs that ran in the past weeks until earlier today are currently unavailable pending FSCK completion. We're going to temporarily disable *successful* jobs from uploading their logs to reduce strain on our current limited capacity. Thanks for your patience !16:02
*** slaweq has quit IRC16:02
openstackgerritBrianna Poulos proposed openstack/nova master: Reduce complexity of _from_db_object
*** Tom-Tom_ has joined #openstack-nova16:03
*** armax has quit IRC16:03
*** armax_ is now known as armax16:03
*** Guest14094 has joined #openstack-nova16:04
*** Tom-Tom has quit IRC16:06
*** gszasz has quit IRC16:07
Tenguhello, anyone can tell me what the error "NovaException: Unsupported VIF type unbound convert '_nova_to_osvif_vif_unbound'" means? it apparently prevents launching a new instance in a specific project/tenant on openstack pike.16:11
*** yamahata has quit IRC16:11
mriedemTengu: probably means an unexpected vif type for the port and nova doesn't know how to handle it16:11
mriedemsean-k-mooney: ^16:11
Tengumriedem: hmm. weird thing is: this happens only in one project :/.16:12
bpoulosgiodeco: The checksum error you're seeing isn't related to image signature verification.  It looks like it's a Glance Exception raised if the MD5 checksum of the image isn't as expected (signature verification doesn't use MD5).  See
mriedemTengu: then i don't know16:12
Tenguand I didn't change anything in the configuration :/.16:12
Tengumight it be related to some kind of quota?16:12
mriedemnot on the nova side16:12
Tengushall I ping on neutron channel?16:13
*** Tom-Tom_ has quit IRC16:13
Tenguhmmm, nova-network is deprecated and not in use anymore in pike, is it?16:13
mriedemthis isn't nova-network16:14
mriedemthe libvirt driver in nova needs to know how to handle the vif type to setup the guest xml properly when attaching the network interface16:14
Tenguif it was for all project in the openstack deploy, I'd say "due to update"… but this isn't the case.16:15
mriedemwhat is the vif type? the error is saying the vif type is "unbound" for some reason16:15
mriedemwhich isn't correct16:15
sean-k-mooneymriedem: it means some how the fucntion with a neutron port that we have not done port binding on yet16:15
sean-k-mooneymriedem: e.g. vif_unbound is the state a port is in before it is added to a vm16:16
Tengumriedem: how may I check that?16:16
mriedemTengu: check the neutron logs for an error with port binding16:16
mriedemit's probably a port binding failure on the neutron side16:16
mriedemcausing a side effect issue in nova16:16
mriedemwhich could possibly be due to quota in neutron, idk16:16
Tengumriedem: hmm ok. I'll check on the neutron channel then.16:17
ildikovjgriffith: mriedem: smcginnis: jungleboyj: sorry, I admit I'm a bit more disorganized this week than I expected :/16:17
*** jafeha has quit IRC16:17
ildikovjgriffith: mriedem: smcginnis: jungleboyj: shall we have a short meeting?16:17
Tengumriedem: thank you for your support :).16:17
mriedemildikov: i'd like to skip this week if possible16:18
mriedemlots of stuff to deal with today16:18
mriedemTengu: np16:18
*** klindgren__ has quit IRC16:18
ildikovmriedem: all good, just wanted to check in whether there's anything we should talk about today or we can skip16:18
ildikovmriedem: tnx16:19
*** andreas_s_ has quit IRC16:23
sean-k-mooneymriedem: if it was a port binding failure in neutron it should be VIF_BINDING_FAILED not unbound16:23
*** andreas_s has joined #openstack-nova16:24
openstackgerritmelanie witt proposed openstack/nova master: Add release note for QEMU native LUKS decryption
*** andreas_s_ has joined #openstack-nova16:25
*** beagles has joined #openstack-nova16:26
openstackgerritBrianna Poulos proposed openstack/nova master: Add trusted_certs to instance_extra
openstackgerritBrianna Poulos proposed openstack/nova master: Add trusted_certs object
openstackgerritBrianna Poulos proposed openstack/nova master: Implement certificate_utils
openstackgerritBrianna Poulos proposed openstack/nova master: Add trusted_image_certificates to REST API
*** suresh12 has joined #openstack-nova16:27
mriedemgmann: here is the patch that adds the nova-multiattach job to tempest:
*** andreas_s has quit IRC16:28
*** matrohon has quit IRC16:29
*** andreas_s_ has quit IRC16:30
Tengusahid: hello! are you here?16:30
*** dtantsur|afk is now known as dtantsur16:31
efriedjaypipes I've now noticed that weird stalling behavior without having rechecked anything nearby, so it was probably just coincidence.  I still don't like it, though.16:32
Tengusean-k-mooney: hello! are you here?16:33
*** jangutter has joined #openstack-nova16:33
Tengumriedem: I was sent back here in order to get info from either sahid or sean-k-mooney - as they are the devs of the os-vif-core :). Pretty sure they will know a bit more about my issue.16:34
* mriedem hides16:34
*** chyka has joined #openstack-nova16:35
*** jangutter has quit IRC16:38
*** elmaciej has quit IRC16:40
*** tidwellr has quit IRC16:40
*** tidwellr has joined #openstack-nova16:41
*** bhujay has quit IRC16:41
*** openstackstatus has quit IRC16:41
*** hrw has quit IRC16:42
*** sahid has quit IRC16:42
*** linkmark has joined #openstack-nova16:43
*** openstackstatus has joined #openstack-nova16:43
*** ChanServ sets mode: +v openstackstatus16:43
*** hrw has joined #openstack-nova16:44
*** Eran_Kuris has quit IRC16:45
efriedgibi stephenfin If you're still around, care to push this 'un?
*** andreas_s has joined #openstack-nova16:45
*** jafeha has joined #openstack-nova16:46
efriedor mriedem dansmith if you don't mind jumping into the middle of this series.  The change itself is fairly small and straightforward, though understanding how and where it fits in is a bigger job.16:47
*** chyka has quit IRC16:48
*** hamzy has quit IRC16:49
*** chyka has joined #openstack-nova16:49
ameedagmann: can you please check this "" ?16:49
*** slaweq has joined #openstack-nova16:49
*** andreas_s has quit IRC16:50
stephenfinefried: Yeah, I can grab that16:50
efriedstephenfin Thanks!16:50
mriedemwhy was it using sets?16:51
gibiefried: done16:51
mriedemand are there problems if we're not ensuring uniqueness now?16:51
efriedgibi Thanks!  stephenfin ^16:51
gibistephenfin: sorry16:51
stephenfingibi: Nope, suits me :)16:51
efriedmriedem The only real reason it was using sets before was for O(1) operations.16:51
stephenfinbauzas has been bugging me about this vGPU thing anyway16:51
* stephenfin wonders if that one's also gone in already16:52
mriedemstephenfin: the vgpu stuff is all approved16:52
efriedmriedem And as super-double-insurance that we don't have cycles in the "tree".  But that's prevented elsewise in a number of ways.16:52
stephenfinI should work on bugs more often16:52
mriedemgod this provider tree stuff,16:52
stephenfinEveryone does my work for me16:52
mriedemall reminds me of some graph traversal stuff i wrote like 10 years ago at ibm16:52
efriedmriedem But the trees should be small enough that O(lognN) is no big deal, and is a rounding error anyway against all the placement API calls that feed it.16:52
mriedemback when i was young and smart16:52
efriedNot sure how that makes me feel, considering I'm older than you (pretty sure)16:53
mriedem<- 3616:53
mriedemjesus you're old16:53
*** slaweq has quit IRC16:53
efriedMan, when I was 36 I could do anything.16:53
efriedNow I sound like a bowl of rice krispies getting out of bed.16:54
efriedSo anyway, I think I now have 8 patches at the mercy of zuul.  Bottom one of that series has been thrashing around for three full days.16:55
mriedemooo nice osc-placement docs are published
*** links has quit IRC16:57
mriedemand releasenotes, although the version isn't what i expected
*** tesseract has quit IRC16:58
* gibi calls it a day. Happy FF folks!16:58
edmondswstephenfin if you're looking for something to review... :)
stephenfinedmondsw: Darn, I thought I'd got away with the powerVM stuff16:59
edmondswand they just reel ya back in...17:00
edmondswthat's the last one17:00
*** slaweq has joined #openstack-nova17:00
*** penick has joined #openstack-nova17:01
dansmithefried: looks like that set has a pretty healthy set of +2+W already, up to a patch with a -1 and no response17:01
*** giodeco has quit IRC17:01
*** jackie-truong has joined #openstack-nova17:02
*** slaweq has quit IRC17:02
*** fragatina has joined #openstack-nova17:02
openstackgerritStephen Finucane proposed openstack/osc-placement master: Resolve nits from I552688b9ee32b719a576a7a9ed5e4d5aa31d7b3f
*** fragatin_ has joined #openstack-nova17:03
stephenfinmriedem, melwitt: Fancy pushing that through? ^17:03
mriedemstephenfin: i know you love to `` everything17:04
mriedembut that's going to look weird in a shell17:04
*** janki has quit IRC17:05
stephenfinYeah, that's a good point actually17:05
stephenfincliff should probably strip those17:06
mriedemit's the best point i've ever made i think17:06
*** abhishekk has joined #openstack-nova17:06
*** Supun has joined #openstack-nova17:06
*** munimeha has joined #openstack-nova17:06
openstackgerritStephen Finucane proposed openstack/osc-placement master: Resolve nits from I552688b9ee32b719a576a7a9ed5e4d5aa31d7b3f
*** fragatina has quit IRC17:07
mriedemstephenfin: you didn't even have nits in :)17:10
stephenfinLet's call it muscle memory17:10
stephenfinOr me not waiting for the CI to build docs before approving them 🙈17:10
dansmither, I guess that one is not part of the main stack17:11
jlvillalstephenfin, dansmith mriedem: You may want to move your jobs out of the Zuul 'integrated' queue.17:11
jlvillalWe (Ironic) set our queue as 'ironic'17:12
*** sree has joined #openstack-nova17:12
jlvillalI notice the 'integrated' queue is enormous at the moment.17:12
dansmithjlvillal: um, what? doesn't that mean you don't co-gate with other projects?17:12
jlvillaldansmith, Yeah.17:12
mriedemkind of defeats the purpose of gating17:12
dansmithdoesn't that defeat the whole thing we're doing here?17:12
*** jistr is now known as jistr|conf17:13
jlvillalWell we gate with our patches.17:13
jlvillalBut yeah, not with cinder, glance, nova, etc...17:13
dansmithright, but it means you don't get tested against ours, and we don't get tested with yours17:13
jlvillalTrue. We did that after we migrated over.17:14
mriedemcan someone turn on the sdague signal17:14
jlvillalNot sure if the tradeoffs are worth it or not.17:14
*** dtantsur is now known as dtantsur|afk17:15
jlvillalToo bad they don't have a 'depend' on queue. Instead of everyone goes in the same bucket queue17:15
*** jaypipes has quit IRC17:16
*** sree has quit IRC17:16
dansmithjlvillal: depends-on between queues just makes it one queue no?17:17
jlvillaldansmith, Not a gerrit depends-on for one.17:18
* dansmith is oh-so-confused17:19
jlvillaldansmith, Now if 'ironic' depends on 'nova'. And 'glance' depends on 'nova' But ironic and glance don't depend on each other.17:19
jlvillaldansmith, So in the gate queue a failed glance wouldn't cause the ironic job to get re-queued.17:20
jlvillaldansmith, They do not have this idea of a 'depend' in zuul queues. There is just a queue.17:20
jlvillalIt was a random idea.17:20
dansmithI see what you mean, but I'm not sure the benefit is worth that complexity17:21
jlvillaldansmith, Yeah probably right. I just saw your nova jobs that have been 18 hours + in the gate queue. So that is why I mentioned it.17:21
*** abhishekk has quit IRC17:22
dansmithyeah, we're definitely feeling that pain, but..17:22
*** hrw has quit IRC17:25
*** jangutter has joined #openstack-nova17:28
*** jangutter has quit IRC17:31
*** jangutter has joined #openstack-nova17:32
*** jose-phillips has joined #openstack-nova17:32
*** jangutter has quit IRC17:33
*** jangutter has joined #openstack-nova17:33
mriedemgibi: once the two approved versioned notification transformation changes are merged i'm going to close that bp and open another one for rocky for anything left17:34
*** abhishekk has joined #openstack-nova17:37
mriedemso is bauzas out today?17:38
*** itlinux has quit IRC17:38
dansmithhe's not on our calendar as being out, but I've pinged him a few times yesterday and today already and haven't seen any response17:38
*** gouthamr has quit IRC17:38
*** penick has quit IRC17:38
dansmithhe was doing some conf thing earlier in the week, but I'm surprised he hasn't popped back up yet17:39
*** gouthamr has joined #openstack-nova17:40
*** yamamoto has quit IRC17:40
*** slaweq has joined #openstack-nova17:41
*** penick has joined #openstack-nova17:42
*** esberglu has quit IRC17:42
*** abhishekk has quit IRC17:42
*** yamamoto has joined #openstack-nova17:43
*** yamamoto has quit IRC17:44
*** yamamoto has joined #openstack-nova17:44
*** slaweq has quit IRC17:46
mriedemoh right snowbunnies17:48
*** abhishekk has joined #openstack-nova17:51
*** derekh has quit IRC17:54
*** slaweq has joined #openstack-nova17:54
jackie-truongdansmith: RE:
jackie-truongdansmith: I wasn't sure if we needed to reassign instance._context17:55
jackie-truongdansmith: The more I look at it, I'm thinking "no"17:55
jackie-truongdansmith: Should I remove?17:55
dansmithjackie-truong: it's done on L342 why would you have to do it again?17:56
jackie-truongdansmith: i was a little confused as to how things were getting passed around17:56
dansmithjackie-truong: you're calling that method :)17:56
*** Guest14094 has quit IRC17:57
dansmithjackie-truong: it's set on instance above, then you pass it to another method, so... it's still set17:57
jackie-truongdansmith: kk i'll remove17:57
*** abhishekk has quit IRC17:58
* dansmith notes jackie-truong has her gerrit reviews on 5-second refresh17:58
*** jangutter has quit IRC17:58
*** AlexeyAbashkin has quit IRC17:59
jackie-truongoutlook pop-ups ;-)17:59
* dansmith shudders17:59
dansmithI figured you had an army of interns, one patch per intern, tasked with rapid-fire refreshing until midnight17:59
jackie-truongnope, that was just me sitting on the couch, eating nachos18:00
dansmithhah nice18:00
*** Guest14094 has joined #openstack-nova18:00
*** slaweq has quit IRC18:02
*** ralonsoh_ has quit IRC18:02
openstackgerritJackie Truong proposed openstack/nova master: Reduce complexity of _from_db_object
*** sridharg has quit IRC18:05
dansmithjackie-truong: why do you keep pushing these up one by one? do you not have them in a proper stack?18:05
*** sapcc-bot1 has joined #openstack-nova18:06
jackie-truongdansmith: i don't think i know what i'm doing when it comes to tying the patches together for gerrit18:06
*** suresh12 has quit IRC18:07
*** brault_ has joined #openstack-nova18:07
dansmithjackie-truong: so you're doing what? rebasing the second one on the first, then the third on the second, etc?18:07
dansmithall in separate branches?18:07
mriedemgit rebase -i18:07
*** lyan has quit IRC18:07
*** sapcc-bot has quit IRC18:07
mriedemis your friend18:07
mriedemrebase the entire series at once,18:07
*** izombie has joined #openstack-nova18:07
mriedemmodify the patches you need18:07
*** lyan has joined #openstack-nova18:07
*** david-lyle has quit IRC18:08
jackie-truongoh. i was definitely doing "rebasing the second one on the first, then the third on the second, etc"...18:08
dansmiththen you'll hate us a little less for asking them to be split up :)18:08
mriedemstep 2 there can be 'git rebase -i master'18:08
mriedemin which case you get a menu to select the commits you want to edit18:08
dansmithjackie-truong: so do your rebase thing like normal this time, but before you submit, rebase the whole thing on master and then try "git rebase -i master" and see what it gives you18:09
izombieI'm trying to setup nova api manually and paste api seems to evasive. When I'm trying to list servers, I'm getting 404. Any suggestions on what could be the reason?18:09
dansmithi.e. git checkout master; git pull; git checkout last-patch-branch; git rebase -i master18:09
*** weshay|ruck is now known as weshay|ruck|brb18:09
*** penick has quit IRC18:09
*** mgoddard_ has quit IRC18:09
*** brault has quit IRC18:09
jackie-truongyeah, so i usually rebase on master (git rebase -i master), but that's just for one patch18:10
jackie-truongso you're saying checkout the api patch in this case, then rebase on master18:10
*** bpoulos has quit IRC18:11
*** jangutter has joined #openstack-nova18:11
dansmithjackie-truong: well, you have modified the bottom one now, so you need to do your normal rebase one by one thing18:11
dansmithjackie-truong: but then once that is done,18:11
dansmithonly use the api patch's branch from now on, and run git rebase -i master from that one18:12
jackie-truongdansmith: got it, thanks!18:12
dansmithjackie-truong: it'll show you all your patches from master on, and let you edit them without changing the stack18:12
jackie-truongso much time has been wasted on individual pushes...18:12
dansmithjackie-truong: then when you git review, it'll show you all the ones pending, and you'll see them all go up in one go here in channel18:12
dansmithjackie-truong: ....yeah :)18:13
*** gouthamr has quit IRC18:13
*** hamzy has joined #openstack-nova18:13
*** izombie has quit IRC18:13
*** jistr|conf is now known as jistr18:14
openstackgerritClaudiu Belu proposed openstack/nova master: tests: fixes mock autospec usage
*** lpetrut has quit IRC18:16
*** munimeha has quit IRC18:16
*** izombie_ has joined #openstack-nova18:16
mriedemgit review -y to push the entire stack18:16
*** fragatin_ has quit IRC18:16
mriedemit will prompt you though18:16
mriedemif you don't use -y18:16
mriedem-R if you don't want to rebase unchanged patches at the bottom the stack18:17
*** izombie_ is now known as izombie18:17
mriedembut maybe that's getting advanced18:17
dansmithI never use -y18:17
mriedemgit rebase -i HEAD~3 etc18:17
dansmithbecause if I have messed up the rebase, I might resubmit a year's worth of changes18:17
mriedemi use git rebase -R -y quite often18:17
*** hamzy has quit IRC18:18
*** damien_r has quit IRC18:20
*** rmcall has quit IRC18:20
*** gouthamr has joined #openstack-nova18:20
*** hamzy has joined #openstack-nova18:21
*** sambetts is now known as sambetts|afk18:22
*** READ10 has joined #openstack-nova18:23
*** imacdonn has quit IRC18:26
*** jackie-truong has quit IRC18:26
*** imacdonn has joined #openstack-nova18:26
*** naichuans_ has quit IRC18:27
*** jangutter has quit IRC18:27
*** weshay|ruck|brb is now known as weshay18:27
*** esberglu has joined #openstack-nova18:28
*** fragatina has joined #openstack-nova18:30
melwittmriedem: I'm good with this ironic "get IP address for volume connector" change if you wanted to revisit it
*** gouthamr has quit IRC18:35
*** goutham__ has joined #openstack-nova18:35
mriedemmelwitt: ok18:39
*** Spazmotic has quit IRC18:39
*** lyan has quit IRC18:39
*** Spazmotic has joined #openstack-nova18:41
*** yamamoto has quit IRC18:42
*** amoralej is now known as amoralej|off18:43
*** jpena is now known as jpena|off18:45
*** fragatina has quit IRC18:49
*** lucasagomes is now known as lucas-afk18:50
*** hrw has joined #openstack-nova18:52
hrwhello again18:52
hrwstephenfin: is initial version of 'lets add usb host' patch18:53
hrwstephenfin: I though it will be more complicated18:53
mriedemmelwitt: wording question in
mriedemedleafe: i don't think your alternate hosts change is ever going to get out of the check queue...18:56
*** yamahata has joined #openstack-nova18:56
edleafemriedem: maybe we need alternate CI18:57
*** lyan has joined #openstack-nova19:00
openstackgerritmelanie witt proposed openstack/nova master: Add release note for QEMU native LUKS decryption
*** psachin has quit IRC19:02
openstackgerritMarcin Juszkiewicz proposed openstack/nova master: [dnm] Make sure that we have USB Host controller for tablet
*** lyan has quit IRC19:06
hrwplease take a look as I am not sure is it proper way of doing that19:06
*** hoonetorg has quit IRC19:07
*** david-lyle has joined #openstack-nova19:09
*** david-lyle has quit IRC19:09
*** yamahata has quit IRC19:09
*** lyan has joined #openstack-nova19:10
openstackgerritMatt Riedemann proposed openstack/nova master: Ironic: Get IP address for volume connector
mriedemmelwitt: check my release note update on ^ and see if that's still OK, then i'll +W19:10
* melwitt looks19:11
*** chyka has quit IRC19:12
*** chyka has joined #openstack-nova19:13
melwittmriedem: looks good to me19:13
mriedemwant to put your +2 back on?19:14
melwittyep done19:14
*** lpetrut has joined #openstack-nova19:16
mriedemefried: so i should defer to rocky right?19:18
mriedemi think that was mentioned as a rocky thing at this point in the meeting today19:18
*** slaweq has joined #openstack-nova19:19
efriedmriedem It can't be finished in Q, because dependent on nrp alloc cands work, which probably also won't make Q.  But 95% of the code is done and merged (months ago).19:19
efriedmriedem So however you want to paperwork that...19:19
mriedemcan't functionality use it yet from the API right?19:19
mriedemok, then i'll defer19:20
*** chyka has quit IRC19:23
*** chyka has joined #openstack-nova19:24
*** AlexeyAbashkin has joined #openstack-nova19:24
mriedemmgoddard: so on the nova ironic traits stuff,19:24
mriedemit looks like the ironic API changes are done
mriedemand the ironicclient changes to support it are approved, but not yet released, and today is client release freeze19:25
mriedemand nova would need to bump the minimum required version of this in global-requirements to use the new client stuff, and i think the requirements team is going to freeze the requirements repo after today too19:25
mriedemjlvillal: are you waiting for these ironicclient changes to merge before you do the final ironicclient release for queens? ^19:26
*** david-lyle has joined #openstack-nova19:26
efriedmriedem The nova side stuff isn't all approved yet either FYI:
*** bpoulos has joined #openstack-nova19:27
mriedemefried: i know,19:27
efriedk :)19:27
mriedemi'm asking because i don't want to waste my time on the nova stuff if the deps won't get merged int time19:27
mriedemthe ironic client change failed in the gate with a post failure, so it's going to have at least one recheck yet19:27
mriedemplus a release, plus a g-r bump19:28
efriedNot happening today.19:28
efriedno chance19:28
*** AlexeyAbashkin has quit IRC19:28
jlvillalmriedem, I am not sure rloo and TheJulia know more the current status than me.19:28
mriedemok jroll is asking in -release too19:29
TheJuliaMoving to laptop with battery power19:29
*** rloo has joined #openstack-nova19:29
TheJuliamriedem: thanks for the recheck19:31
TheJuliamriedem: yes, waiting for the client changes we need to merge before cutting the release :(19:32
rloomriedem: the traits work will land for sure; just one client patch in gate. the rescue work has been approved but we need 2 ironic patches to land before client can land. and gate is super not-fast today as you probably know :-(19:33
mriedemthere hasn't been any review on the nova patches for rescue, so that's going to be deferred19:34
mriedemalso - that doesn't bump the minimum required ironic api version to 1.37 before trying to patch the node with traits19:35
mriedembut that's required isn't it?19:35
mriedemjroll: ^19:35
*** rmcall has joined #openstack-nova19:35
mriedemwe'll likely need someone to work the nova patches at this point since i'm guessing mgoddard and johnthetubaguy are eating boiled meat at this point19:35
rloomriedem: boiled meat. yummmy. which is better, doing that or reviewing? i'll take a look at the nova patches in a few min.19:36
jrollI can help too19:37
mriedemwell i'm assuming the mv needs to change to 1.37 since that's the ironic api version that added traits support19:37
jrollit doesn't19:37
mriedemi could be wrong, there are a lot of +1s on there19:37
jrollexplaining in gerrit now19:37
jrollcomment is there mriedem19:38
jrolland this is the one that needs the api version bump
jrollsince it's grabbing the traits from the node19:39
*** fragatina has joined #openstack-nova19:39
*** chyka has quit IRC19:40
*** chyka_ has joined #openstack-nova19:41
*** slaweq has quit IRC19:41
izombieTrying to list servers and I'm hitting 404 and I can't understand why19:41
izombieAny help?19:41
TheJuliaon a plus side jroll, that is the correct version :)19:42
*** yamamoto has joined #openstack-nova19:43
*** Guest14094 has quit IRC19:44
*** harlowja has joined #openstack-nova19:44
mriedemizombie: check the nova-api logs19:45
mriedemjroll: ah ok, thanks19:45
izombie:) I did. Even debugged! I'm lost the route is not matched. It reaches till route middleware and then throws HTTPNotFound19:46
*** Guest14094 has joined #openstack-nova19:48
artomizombie, technically I'm supposed to send you to #openstack (see /topic)19:48
melwittmriedem: any particular reviews I can help with today?19:49
artomizombie, but, what request are you making?19:49
izombieSimple nova list. And it breaks here
izombieartom: thanks for not sending :P19:50
artomizombie, well, I did, in a hint-y sort of way ;)19:50
artomizombie, can you run the client in --debug?19:50
artomAnd what are you running it against? devstack?19:50
izombieI will try my luck there, meanwhile I thought to try my luck here as I'm out of it now19:51
mriedemmelwitt: i'm just going through this ironic driver traits series19:51
*** jackie-truong has joined #openstack-nova19:51
*** yamamoto has quit IRC19:55
*** avolkov has quit IRC19:57
*** esberglu has quit IRC19:57
*** goutham__ has quit IRC19:58
*** Supun has quit IRC19:58
izombieguess, I won't fix it today.19:58
*** izombie has quit IRC19:58
artomI mean, I did ask them --debug output :/19:58
openstackgerritArtom Lifshitz proposed openstack/nova master: Refactor _build_device_metadata
openstackgerritArtom Lifshitz proposed openstack/nova master: Consider hostdev devices when building metadata
efriedmriedem The ironic traits thing, we're trying to get it in?20:00
efriedmriedem I can probably do any last-minute changes if you didn't get another volunteer.20:00
mriedemwe'll see20:01
mriedemi'm reviewing the changes now yes,20:01
jrollI'm happy to help too20:01
mriedemwhich requires looking at some of the provider tree stuff that already merged, like set_traits_for_provider20:01
efriedmriedem Yes, that was a prereq.  Let me know if you want the exec summary on that.20:01
mriedemi don't suppose there are any integration tests anywhere in ironic that run something that plumbs this all through20:03
mriedemlike, create a flavor with a required trait, decorate a node with it, pass the trait from the node to the resource provider in placement, etc20:03
*** chyka_ has quit IRC20:03
*** chyka has joined #openstack-nova20:04
jrollnot terribly hard to do other than the time crunch20:04
mriedemyou gonna tattoo a todo for yourself?20:05
jrollI'll think about it :P20:05
mriedemwe could also have a simple functional test in-tree that has a fake virt driver that returns a fake trait, and we make sure that the RT and such creates it in placement for us20:05
jrollI can add it to the existing job pretty easy20:05
mriedemrather than just these lowly unit test20:05
*** esberglu has joined #openstack-nova20:06
mriedemjroll: that test_get_traits_with_no_traits_attr test can also go away once we require 1.37 and depend on ironicclient to give us a node object that has a traits attribute20:12
mriedemjroll: you want to update then?20:13
mriedemi'll be +2 after removing the hasattr thing20:13
jrollmriedem: heh, we wrote a test for that? fun20:13
jrollI can do that but it'll fail hard until we release a new client20:14
mriedemalternative is,20:14
*** hoonetorg has joined #openstack-nova20:14
*** slaweq has joined #openstack-nova20:14
mriedemwe push a change on top that removes that, and has a dependency on whatever g-r bump we're going to have for the new ironicclient release version20:15
jrolloh, I'm fine with the former, just letting you know it won't be landable20:15
jroll(see also my -1 for the hasattr thing)20:15
mriedemi'm ok with it not being landable until the deps are done20:15
mriedemi did20:15
mriedemi'll FFE this20:16
jrollI'll hack out some sort of functional or integration test too20:16
mriedemalright, you might want to save any in-tree functional test patch for something that goes on top of the series20:16
*** Jeffrey4l has quit IRC20:19
*** chyka has quit IRC20:21
*** chyka has joined #openstack-nova20:21
*** Guest14094 has quit IRC20:22
*** flwang has quit IRC20:23
*** Jeffrey4l has joined #openstack-nova20:24
openstackgerritClaudiu Belu proposed openstack/nova master: tests: refactors and cleans up
openstackgerritJim Rollenhagen proposed openstack/nova master: Implement get_traits() for the ironic virt driver
jrollmriedem: efried: there you are20:25
* jroll attempts to find a good place for this functional test20:25
mriedemjroll: likely somewhere in nova.tests.functional.test_servers20:26
mriedemthere are lots of examples of tests using a specific fake virt driver,20:26
jrollmriedem: I'm liking nova/tests/functional/compute/test_resource_tracker.py20:26
mriedemand using a placement api client to assert things20:26
jrollhas a bunch of RT + ironic things already20:26
mriedemthis doesn't necessarily need to be an ironic virt driver functional test20:27
mriedemjust a fake driver (see nova/virt/ that returns a standard and custom trait20:27
jrolltrue20:27 is probably fine as a new test class20:27
mriedem*with a new test class20:27
jrollyeah, I'll poke around, thanks20:28
*** slaweq has quit IRC20:28
*** READ10 has quit IRC20:31
*** flwang has joined #openstack-nova20:36
*** slaweq has joined #openstack-nova20:37
*** Guest14094 has joined #openstack-nova20:40
openstackgerritClaudiu Belu proposed openstack/nova master: tests: fixes mock autospec usage
mriedemfyi, the cinder backup bug is still in the gate20:43
mriedemthere is another thing going on20:43
mriedemswiftclient is logging within the thread20:43
dansmithstill in the gate as in, unmerged?20:43
mriedemanother issue, same bug20:43
mriedemswiftclient is logging while their are in a thread and it switches20:44
mriedem*they are20:44
efriedshitballs, does this mean stuff that's been sitting in the gate for 18h is now going to fail??20:44
mriedemseeing if i can patch cinder-backup to not do debug logging for swiftclient if using that backup driver20:44
mriedemefried: it means it has a good chance of failing20:45
efried(That longish pause was me vocalizing my opinion of the situation)20:46
mriedemi'm pushing a cinder patch20:46
efriedmriedem, thanks for tracking it down.20:46
efriedSonofa... three of my patches were all green & blue, alllmost through the gate, and then stalled again.20:47
*** jackie-truong has quit IRC20:47
efriedat least they didn't outright fail, and have to start 18+h cycle over again20:51
* efried knocks wood20:51
mriedemefried: dansmith:
*** matrohon has joined #openstack-nova20:52
*** andreas_s has joined #openstack-nova20:52
efriedmriedem What happens if swiftclient=SOMETHING_ELSE was already in the string?  Does last win?20:53
mriedemdon't know20:54
mriedemi know it's not in the default list of log levels in oslo.log20:54
*** mvk has joined #openstack-nova20:55
efriedbut if it's explicitly in whatever conf (on customer site), we don't want to override it.20:56
*** andreas_s has quit IRC20:56
mriedemthat's why it's default_log_levels yeah?20:59
efriedCould be, could be.  I don't know.  I left comments, but +0 - smarter people can decide whether those things are really an issue.21:00
mriedemlook efried, do you want your 20 patches to merge or not21:01
efriedI have NO IDEA what's actually best for my karma at this point.  Walkin on eggshells, here.21:02
*** Jeffrey4l has quit IRC21:02
mriedemyeah, i'm about to just call it a day21:03
mriedemnot really much that can be done about anything21:03
efriedNo, you have to stick around to +2+W stuff so us peons can sit around rechecking them through the night.21:04
mriedemi was rechecking things until about 11:30 last night21:04
mriedemi've done my job21:05
*** tidwellr has quit IRC21:05
*** tidwellr has joined #openstack-nova21:06
efriedBTW, I was tooling around on stackalytics the other day - how did you manage to merge 18KLOC while pushing PTL paperwork 18 hours a day?21:06
efriedImpressive.  Most impressive.21:06
*** jackie-truong has joined #openstack-nova21:07
Roamer`hm, so here's a stupid question about stackalytics: if I put my name and e-mail address in default_data.json, will it pick up the patches that have already been merged, or does it just go forward?21:08
efriedRoamer` me no idea.  Someone in -infra might know, but they're frying great big fish at the moment.21:09
Roamer`efried, thanks, and yeah, I'm following the discussions there21:13
*** liverpooler has quit IRC21:13
*** Jeffrey4l has joined #openstack-nova21:14
openstackgerritBrianna Poulos proposed openstack/nova master: Reduce complexity of _from_db_object
openstackgerritBrianna Poulos proposed openstack/nova master: Add trusted_certs to instance_extra
openstackgerritBrianna Poulos proposed openstack/nova master: Add trusted_certs object
openstackgerritBrianna Poulos proposed openstack/nova master: Implement certificate_utils
openstackgerritBrianna Poulos proposed openstack/nova master: Add trusted_image_certificates to REST API
openstackgerritMerged openstack/nova master: Update VMWare vSphere link address
mriedemefried: don't know21:17
mriedemlikely api changes with lots of api samples21:18
efriedTell ya, wait til you're my age.  You'll never have the stamina.21:18
mriedemthat's why i take that thing that frank thomas is selling21:18
* efried googles...21:18
mriedem"stronger and leaner"21:19
efrieda powerful, man-boosting innovation21:19
mriedemthe guy has been like 300lb for 20 years21:19
mriedemman boosting testafen21:19
mriedemmy passion levels will be through the roof!21:19
edleafeIt's cute watching you young 'uns talking about being old.21:27
dansmithhush, dad's home!21:29
*** pramodrj07 has joined #openstack-nova21:29
melwitthah :D21:29
mriedemi'll grab the hearin' cone21:29
dansmiththat said, I'm quite sure edleafe remembers being 36 and feeling like life was over21:30
*** eharney has quit IRC21:31
*** cleong has quit IRC21:32
*** esberglu has quit IRC21:32
efriedAny time I think of 36...
*** pchavva has quit IRC21:33
mriedemumm, yikes21:33
melwittfor the native thread logging related gate issues, will there always be a message somewhere in the logs that says "cannot switch to a different thread"?21:38
melwittokay. I was wondering if it could be related to the stable branch cells v1 job 100% fails. libvirt is getting "connection broke" as though something is holding up the tpool and never letting it continue21:39
melwitt(I noticed all the libvirt calls go through eventlet.tpool)21:39
melwittbut the "cannot switch to a different thread" isn't in the logs anywhere21:40
*** jheroux has joined #openstack-nova21:40
melwittI've been investigating it but haven't figured out anything yet21:41
dansmiththe libvirt driver uses native threads properly21:41
dansmithso it should be fine unless something has regressed (or was never right)21:42
dansmiththe connection issue with libvirt almost always means libvirtd crashed,21:42
dansmithwhich happens a lot more than it should21:42
*** jheroux has quit IRC21:43
*** hamzy has quit IRC21:43
melwittyeah. in this case it hasn't crashed. it's as if something held it up so long that the connection dropped from not getting keepalive, or something like that21:43
Roamer`melwitt, what OS is this happening on?  Could it be the Ubuntu py27/eventlet problem?
melwittI was wondering if any other library also using eventlet could have made it so it never switches back to the libvirt driver21:44
dansmithmelwitt: why do you say it hasn't crashed?21:44
melwittdansmith: because from syslog and libvirtd I couldn't find any crashing. unless I totally don't know what I'm looking at, which is possible21:45
*** lpetrut has quit IRC21:45
dansmithmelwitt: link?21:45
melwittRoamer`: it's ubuntu xenial. and that's what I was wondering, but I'm still grasping at straws at this point21:45
melwittdansmith: here's an example from recently
dansmithhmm, empty journal on that one?21:50
dansmiththat's weird21:50
*** amodi has quit IRC21:50
melwittI feel like I've seen that before21:51
*** takashin has joined #openstack-nova21:52
dansmithyeah, libvirtd doesn't look like it was restarted across that connection failure boundary21:52
mriedeminterestingly, it's the first guest to be created it looks like
dansmiththis would be the first time I've ever seen that symptom not associated with a crash21:53
dansmithso I still feel a little doubtful, but it's certainly not as glaring as normal21:53
mriedemmakes me think of this
mriedemwhere guestfs on the first bring up runs in a tpool21:53
mriedemto get the guestfs capabilities21:53
dansmithso I wonder if it's something like qemu crashes and the libvirt thread we're talking to exits or something21:54
melwittyeah, that's kinda how I found that patch of yours mriedem21:54
dansmiththe journal should have dmesg in it to capture any segv21:54
*** bpoulos has quit IRC21:54
dansmithbut we don't have qemu logs otherwise21:55
melwittI wonder if the journal collection on the jobs has been busted this whole time21:55
melwittor is it not always empty?21:55
dansmithit wasn't busted a month or so ago21:55
melwittI haven't tried to look at it before21:55
melwittoh okay21:55
dansmithI expect it would be in syslog too, but it's clearer in the journal21:56
melwittI didn't see anything that looked like crashing in syslog
dansmith2018-01-25 19:06:13.215+0000: 3732: error : virKeepAliveTimerInternal:143 : internal error: connection closed due to keepalive timeout21:57
dansmithis right before you get it in nova21:57
melwittwhere do you see that? libvirtd.log?21:59
dansmithI think this is all in a separate thread in nova so it shouldn't be that nova is hung up and unable to do the keepalive bit, unless it's just system starvation of some sort21:59
dansmithyeah, libvirt21:59
dansmithand nova is doing stuff milliseconds before that, so it's not hung I don't think22:00
*** linkmark has quit IRC22:00
dansmithoh yeah in the nova log:22:00
dansmith2018-01-25 19:06:13.224 16016 INFO nova.virt.libvirt.driver [req-ea2c4c13-3bbe-4b35-a2b3-84976c11fc4b - -] Connection event '0' reason 'Connection to libvirt lost'22:00
dansmiththat's the real thread detecting the connection close22:01
dansmithoooh, but22:01
dansmith2018-01-25 19:06:09.554 16016 DEBUG oslo_messaging._drivers.amqpdriver [-] received reply msg_id: 9ebc51b2583b4f4da811e951e41068c9 __call__ /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/
dansmith2018-01-25 19:06:13.218 16016 DEBUG nova.virt.libvirt.firewall [req-733b7704-8aa3-45a5-8798-67eab246baf8 tempest-InstanceActionsV221TestJSON-176157016 tempest-InstanceActionsV221TestJSON-176157016] Cannot find UUID for filter 'nova-instance-instance-0000000d-fa163e269665': 'Cannot recv data: Connection reset by peer' _get_filter_uuid /opt/stack/new/nova/nova/virt/libvirt/
dansmiththat is five seconds apart with nothing in between22:02
*** yamahata has joined #openstack-nova22:02
*** matrohon has quit IRC22:02
openstackgerritJim Rollenhagen proposed openstack/nova master: Add functional tests for virt driver get_traits() method
dansmithmeh, I dunno, looking around that I don't think that's anything related to being stuck though22:03
dansmithbecause that's the exact instant libvirt logs its message22:03
*** slaweq has quit IRC22:04
*** slaweq has joined #openstack-nova22:05
*** yamamoto has joined #openstack-nova22:05
openstackgerritJim Rollenhagen proposed openstack/nova master: Add functional tests for virt driver get_traits() method
jackie-truongdansmit, mriedem: re: git commands22:08
jrollmriedem: ^ see my comment there, fairly certain some of this traits stack is somewhat broken. probably fine to land as drivers aren't using it yet, but ya know.22:08
jackie-truongi want to add a /new/ file ( to a patch in the middle of my patch stack22:09
jackie-truongchecking out the top of the patch stack, then adding and committing the file results in the file being added to the top patch22:09
mriedemjackie-truong: git rebase -i master22:10
dansmithjackie-truong: right, so git rebase -i master, edit the middle patch, git add path/to/file, git commit --amend -a22:10
mriedemjackie-truong: edit the commit you want to add the file to,22:10
mriedemcreate said file,22:10
mriedemgit add <file>22:10
mriedemgit rebase --continue22:10
jackie-truongah, i rarely actually edit commits when rebasing from master. got it, thanks again22:11
efriedmriedem VMX ain't a standard trait, or a valid CUSTOM_ one.22:11
efriedjroll ^22:11
mriedemefried: it's in os-traits22:11
mriedemoh wait22:12
mriedemi see22:12
mriedemjroll: he's right22:12
efriedHW_CPU_X86_VMX ?22:12
mriedemyeah, the traits are generated from their directory structure within os-traits22:12
jrolloh gdi22:12
* jroll tries22:12
*** slaweq_ has joined #openstack-nova22:13
*** slaweq_ has quit IRC22:15
jrollmriedem: efried: that did it, now I feel dumb, thanks :)22:15
mriedemdon't feel dumb22:16
*** slaweq_ has joined #openstack-nova22:16
efriedFor not understanding the bizarre architecture of os-traits?  Yeah, what mriedem said.22:16
mriedemironicclient change is 85 in the gate22:16
efriedI always just GET /traits from the API to find my testable traits.22:16
efriedBecause you'll also get tripped up by this one at some point in the future: Your compute's os-traits may not be the same as the placement server's os-traits.22:17
efriedwhich is why
*** slaweq_ has quit IRC22:18
*** slaweq has quit IRC22:19
*** slaweq_ has joined #openstack-nova22:19
*** slaweq has joined #openstack-nova22:19
openstackgerritJackie Truong proposed openstack/nova master: Reduce complexity of _from_db_object
openstackgerritJackie Truong proposed openstack/nova master: Add trusted_certs to instance_extra
openstackgerritJackie Truong proposed openstack/nova master: Add trusted_certs object
openstackgerritJackie Truong proposed openstack/nova master: Implement certificate_utils
openstackgerritJackie Truong proposed openstack/nova master: Add trusted_image_certificates to REST API
*** slaweq_ has quit IRC22:23
*** swamireddy has quit IRC22:25
dansmithI shall gaze upon thy patches upon my return from a brief respite22:25
mriedemmelwitt: this makes me think about the eventlet/python thing in the ML
melwittare those pom poms?22:26
melwittmriedem: interesting22:28
melwittseems like it could be related. nothing makes too much sense so far, only cells v1 job is getting constant libvirt keepalive timeouts, starting about 10 days ago, and only on stable22:29
melwittpike and ocata22:30
mriedemmelwitt: unrelated, but related to lyarwood's change to hard reboot and blow things away all the time
openstackLaunchpad bug 1744361 in neutron "test_server_security_groups failed to reboot with "Domain not found: no domain with matching uuid" because of missing vif event from linuxbridge agent" [Undecided,Confirmed]22:30
*** jackie-truong has quit IRC22:30
mriedemi thought there were some assertions in the review that we didn't need to wait for vif plugged events, or something along those lines22:31
melwittah, suck22:31
melwittwell, I had commented that we do need to wait for them since we are re-plugging vifs. but I didn't realize we'd be in a vm_state that would ignore them22:32
jrollefried: fun, thanks22:32
melwittthat is, I didn't expect the events to be ignored when they're being waited for22:33
mriedemmelwitt: it's not saying it's ignored b/c of the instance state22:36
mriedemit's saying the compute got the event but no callback was registered waiting for it22:36
mriedemJan 19 10:33:35.986648 ubuntu-xenial-rax-iad-0002010825 nova-compute[28907]: DEBUG nova.compute.manager [None req-4bcf94cf-2048-4e99-94e6-646519900469 service nova] [instance: dd2ec3b0-f909-4104-ae2a-9b878d936ed4] No waiting events found dispatching network-vif-plugged-567525a4-4b16-4004-91d5-545086d01227 {{(pid=28907) pop_instance_event /opt/stack/new/nova/nova/compute/}}22:36
melwittmriedem: okay, I was going by the last comment in the bug. okay, so maybe there's a bug in the _create_domain_and_network vifs_already_plugged logic somewhere22:37
*** itlinux has joined #openstack-nova22:37
melwittor maybe vifs_already_plugged doesn't mean what I thought it means22:38
*** lyan has quit IRC22:40
mriedemi wonder if we don't register the callback before the vif plug event has already happened22:40
mriedemno that doesn't make sense22:41
mriedemwe register the callback before calling plug_vifs22:41
melwitt"Timeout waiting for vif plugging callback for instance with vm_state active and task_state reboot_started_hard.: Timeout: 300 seconds" so that means it was waiting22:41
melwittso "Received unexpected event" happens when there weren't events in self.instance_events in compute manager. so how do they get there ...22:45
dansmithmriedem: yeah the whole design of that thing is so you can't trigger before you're registered22:45
*** amodi has joined #openstack-nova22:46
*** jmlowe has quit IRC22:48
*** tidwellr has quit IRC22:49
mriedemi don't know what to do about this bug
openstackLaunchpad bug 1740123 in OpenStack Compute (nova) "Can we create server group with empty policies?" [Undecided,New]22:56
mriedemit's not a bug per se22:56
melwittmriedem: it seems like the patch is related to the new bug. I think the only way this can happen is if _get_neutron_events(network_info) is returning no events, meaning there are no vifs in network_info. I'm looking to see where/how does network_info get reset after a reboot22:56
mriedemmelwitt: if it doesn't return any events, wouldn't we just not wait for anything?22:57
mriedemand then no timeout?22:57
melwittI would have thought network_info would remain as it was during a reboot (network_info came from when we originally allocated ports, right?)22:57
mriedemyes network_info should have whatever ports are attached to the instance, regardless of whether or not the vifs are actually plugged22:58
mriedemit's like bdms22:58
mriedemfor ports22:58
melwittgood point ... one would think. I'm not that familiar with how the events work. you pass a list of events to the "wait" context manager22:58
melwittyeah, looks like it would just pass by all the waiting if that list is empty. so the list isn't empty. yet it's not matching up the events22:59
*** armax has quit IRC23:00
*** mvk has quit IRC23:00
*** amodi has quit IRC23:01
mriedemwe should probably log the list of stuff we're registering when we register it23:03
mriedemfor debuggabilityamortiziation23:04
*** itlinux has quit IRC23:04
melwittyeah. cause so far this don't make no sense23:04
dansmithyou mean log at debug level I assume.. otherwise it's too chatty23:05
mriedemyes debug23:05
mriedemi can't be chattier than the virt topology shit we log 20 things for23:05
mriedemeven though we don't do any stuff with that in the gate23:05
*** itlinux has joined #openstack-nova23:06
*** slaweq has quit IRC23:06
openstackgerritJim Rollenhagen proposed openstack/nova master: Add functional tests for virt driver get_traits() method
jrollmriedem: ^ all yours, that worked out well23:07
*** yamahata has quit IRC23:08
openstackgerritJim Rollenhagen proposed openstack/nova master: Add functional tests for virt driver get_traits() method
jrolland pep8'd23:09
*** Pramod has joined #openstack-nova23:09
*** Aibot has joined #openstack-nova23:10
mriedemdamn it23:10
mriedemcomment in the last patchset23:10
mriedemseems like those could be one test,23:10
*** Aibot has quit IRC23:10
mriedembut idk23:10
*** Pramod has quit IRC23:10
*** pramodrj07 has quit IRC23:10
*** itlinux has quit IRC23:11
melwitthm, the 'No waiting events found dispatching' message means that the result from _pop_event was no_events_sentinel which has another log message in it 'Unexpected attempt to pop events during shutdown'23:11
*** pramodrj07 has joined #openstack-nova23:11
mriedemi think that's when the compute service is shutting down23:11
mriedemnot the guest23:11
melwittoh, there's another way to get no_events_sentinel, if no result for self._events.get(instance.uuid)23:11
melwittso it should be from that23:12
*** amodi has joined #openstack-nova23:12
mriedemrloo: TheJulia: jroll: is someone putting out a release?
rloomriedem: TheJulia is in charge of that!23:15
TheJuliawe have a reno in the gate that should only take like 10 minutes23:15
mriedemyeah i see23:16
mriedemso, you could start the release patch with a depends-on the reno patch23:16
openstackgerritJim Rollenhagen proposed openstack/nova master: Add functional tests for virt driver get_traits() method
mriedemthen you just have to mod the git hash once it merges23:16
mriedemi.e. we could start stacking up the dependencies for the release23:16
TheJuliadocs and pep8, even less time23:19
*** rmcall has quit IRC23:20
*** chyka has quit IRC23:23
openstackgerritOpenStack Release Bot proposed openstack/python-novaclient master: Update reno for stable/queens
*** chyka has joined #openstack-nova23:23
*** edmondsw has quit IRC23:27
*** dave-mccowan has quit IRC23:28
*** chyka has quit IRC23:28
mriedemTheJulia: ship it23:29
*** chyka has joined #openstack-nova23:30
TheJuliahad to just fix an excess space23:30
TheJuliabut yup23:31
* TheJulia goes and has a drink23:31
*** armax has joined #openstack-nova23:31
mriedemjroll: since you've got the series localm23:33
mriedemwant to change the depends-on in to
mriedemand add the requirements.txt change in nova23:33
*** chyka_ has joined #openstack-nova23:34
*** chyka has quit IRC23:35
mriedemthe gpu patches failed on the volume backup thing, so i'm going to wait to recheck those23:42
openstackgerritMatt Riedemann proposed openstack/nova master: Implement get_traits() for the ironic virt driver
openstackgerritMatt Riedemann proposed openstack/nova master: Add functional tests for virt driver get_traits() method
mriedemjroll: nvm i did it23:44
dansmithmriedem: cool, I was waiting for you to say it was good with that dep23:47
*** claudiub has quit IRC23:47
*** gongysh has joined #openstack-nova23:54
*** r-daneel has quit IRC23:57

Generated by 2.15.3 by Marius Gedminas - find it at!