Tuesday, 2017-01-31

*** baoli has quit IRC00:00
*** gatuus has joined #openstack-nova00:01
*** unicell has quit IRC00:07
*** wasmum has joined #openstack-nova00:07
*** diga has joined #openstack-nova00:08
*** takashin has joined #openstack-nova00:13
*** READ10 has joined #openstack-nova00:16
*** mriedem has joined #openstack-nova00:16
*** unicell has joined #openstack-nova00:19
*** hongbin has quit IRC00:19
*** Swami_ has joined #openstack-nova00:20
openstackgerritZhenyu Zheng proposed openstack/nova: Don't delete neutron port when attach failed  https://review.openstack.org/40356800:20
*** dtp has quit IRC00:22
*** tbachman has joined #openstack-nova00:29
*** mtanino_ has joined #openstack-nova00:29
*** unicell has quit IRC00:29
*** jose-phillips has quit IRC00:29
*** mtanino has quit IRC00:30
*** jose-phillips has joined #openstack-nova00:30
*** haplo37 has quit IRC00:32
*** haplo37 has joined #openstack-nova00:33
openstackgerritMatt Riedemann proposed openstack/nova: Fix tag attribute disappearing in 2.33 and 2.37  https://review.openstack.org/42475900:34
openstackgerritMatt Riedemann proposed openstack/nova: Fix tag attribute disappearing in 2.33 and 2.37  https://review.openstack.org/42475900:39
openstackgerritMatt Riedemann proposed openstack/nova: Pass APIVersionRequest to extensions  https://review.openstack.org/42587600:40
mriedemartom: i cleaned up the bottom change, +2 on that one now00:43
mriedemdansmith: ^00:43
*** jamesdenton has quit IRC00:47
*** gouthamr has joined #openstack-nova00:49
*** unicell has joined #openstack-nova00:52
mriedemjesus is this normal http://logs.openstack.org/06/425806/8/gate/gate-tempest-dsvm-cells-ubuntu-xenial/344170a/logs/screen-n-api.txt.gz?level=TRACE ?00:54
*** READ10 has quit IRC00:57
*** jamielennox is now known as jamielennox|away00:58
*** baoli has joined #openstack-nova01:02
*** catintheroof has quit IRC01:02
*** baoli has quit IRC01:02
*** jamesdenton has joined #openstack-nova01:03
*** unicell has quit IRC01:07
*** gatuus has quit IRC01:07
*** yamahata has quit IRC01:11
*** unicell has joined #openstack-nova01:11
*** chyka has quit IRC01:11
*** jamielennox|away is now known as jamielennox01:12
*** thorst_ has joined #openstack-nova01:16
*** jaosorior has quit IRC01:17
artommriedem, ah, much thanks!01:21
artomI got your review but was busy with offsprings01:21
mriedemartom: i'll have the tempest patch up shortly01:21
artomAll sleeping now, but you got ahead of me ^_^01:21
mriedemi'll have to deal with offspring in a bit01:21
mriedemdreaded reading homework, the bane of my existence01:22
*** ducttape_ has joined #openstack-nova01:22
artomFor what it's worth, I kept the single device_tags in parameters.yaml because, if we can't min/max/min on them, I saw no point in splitting them up01:23
mriedemthe point in splitting them up was to keep a bit more specific description for each parameter01:24
mriedemso we didn't have to cram either/or type wording into a single description01:25
mriedemhttps://review.openstack.org/#/c/426991/ is the tempest change01:25
artommriedem, hrmm, looks like I didn't completely grok it then01:26
*** tbachman has quit IRC01:27
artomOh, duh, I get the point of splitting them01:29
artommriedem, is there the same rc1 deadline on the novaclient bug?01:33
mriedemartom: novaclient is frozen for ocata01:33
mriedemwe can backport https://review.openstack.org/#/c/424745/ though01:33
mriedemlater01:33
artomAha, right01:34
*** dims has quit IRC01:35
*** dims_ has joined #openstack-nova01:35
gmannartom:  nova patch looks good01:35
*** mlavalle has quit IRC01:35
artomgmann, wootness, thanks for following up :)01:35
*** crushil has joined #openstack-nova01:36
gmannmriedem:  thanks for tempest tests, ll check today01:37
*** catintheroof has joined #openstack-nova01:37
*** catintheroof has quit IRC01:37
*** catintheroof has joined #openstack-nova01:37
* artom thinks he'll go to sleep early at this point, because offspring #1 went to sleep with a fever, and will probably wake him up during the night01:38
*** baoli has joined #openstack-nova01:38
gmannmriedem: on all_tenant policy  thing - https://review.openstack.org/#/q/topic:bp/add-whitelist-for-server-list-filter-sort-parameters01:40
gmannmriedem:  if we are deferring that to pike i can update doc patch of sort/ filter (removing the policy bits)01:41
gmannmriedem: we need to get release notes/docs for sort/filter in01:41
mriedemgmann: yes we're deferring the policy changes to pike01:41
mriedemso we can discuss at the PTG01:41
mriedemand yeah i have the sort/filter reno patch in our ocata rc1 todo etherpad01:42
*** rfolco has quit IRC01:42
gmannmriedem: cool. ll update doc one quickly01:42
*** tbachman has joined #openstack-nova01:42
*** dims_ has quit IRC01:42
*** baoli has quit IRC01:42
*** dims has joined #openstack-nova01:42
*** edmondsw has joined #openstack-nova01:43
*** edmondsw has quit IRC01:43
openstackgerritGhanshyam Mann proposed openstack/nova: Add release note and docs for filter/sort whitelist  https://review.openstack.org/42176001:43
*** edmondsw has joined #openstack-nova01:43
gmannmriedem: ^^01:43
*** jamesdenton has quit IRC01:46
mriedemgmann: thanks. i probably won't get to that until tomorrow at this rate.01:47
gmannmriedem:  sure.01:47
*** edmondsw has quit IRC01:47
mriedemmy wife has decided that simply because she has to get up at 4am, she wants to get some sleep and therefore i need to finish the parenting duties for the evening.01:47
gmannmriedem: cool. spend some good time with your kids01:48
gmanni have oomichi  in japan today. ll catch him too :)01:48
*** rfolco has joined #openstack-nova01:49
*** rfolco has quit IRC01:49
openstackgerritDiana Clarke proposed openstack/nova: nova-manage docs: cell_v2 map_cell_and_hosts  https://review.openstack.org/42142001:49
openstackgerritDiana Clarke proposed openstack/nova: nova-manage docs: cell_v2 verify_instance  https://review.openstack.org/42143601:49
openstackgerritDiana Clarke proposed openstack/nova: nova-manage docs: cell_v2 delete_cell  https://review.openstack.org/42145901:49
openstackgerritDiana Clarke proposed openstack/nova: nova-manage docs: cell_v2 discover_hosts  https://review.openstack.org/42144201:49
openstackgerritDiana Clarke proposed openstack/nova: nova-manage docs: cell_v2 list_cells  https://review.openstack.org/42145801:49
openstackgerritDiana Clarke proposed openstack/nova: nova-manage docs: cell_v2 create_cell  https://review.openstack.org/42144401:49
*** masber has quit IRC01:52
*** edmondsw has joined #openstack-nova01:52
*** tbachman has quit IRC01:52
diana_clarkeNow I need to switch to a GitHub workflow. I've grown to really like Gerrit.01:53
*** raunak has quit IRC01:53
artomdiana_clarke, gerrithub?01:55
diana_clarkeartom: is that a thing... off to google it01:56
melwittmriedem: I just looked at the cells v1 job "Failed to get metadata for IP 10.1.3.255: no metadata" in n-api logs and I think it's normal in that, the error means it couldn't lookup a FixedIP by address. and in cells v1 fixed_ips are populated only in the child cell database, so any reads from the api cell will yield not found01:59
*** browne has quit IRC01:59
*** edmondsw has quit IRC02:04
*** edmondsw has joined #openstack-nova02:05
mriedemmelwitt: should that be an ERROR though?02:06
*** ducttape_ has quit IRC02:06
mriedemplus, has that changed recently?02:06
mriedemi don't remember seeing that before02:06
melwittfor non cells v1, yes02:06
melwittand agreed I don't remember seeing it repeatedly. I thought it used to only show it once. so I'm not sure what changed02:06
*** ijw has quit IRC02:08
*** edmondsw has quit IRC02:09
openstackgerritChangBo Guo(gcb) proposed openstack/nova: rbd: Remove unnecessary 'encode' calls  https://review.openstack.org/41235602:10
*** diga has quit IRC02:15
*** mtanino has joined #openstack-nova02:17
*** mtanino_ has quit IRC02:19
*** owalsh is now known as owalsh-afk02:19
*** baoli has joined #openstack-nova02:20
*** armax has quit IRC02:22
*** unicell has quit IRC02:22
*** Apoorva has quit IRC02:26
*** Sukhdev has quit IRC02:27
*** ducttape_ has joined #openstack-nova02:33
*** edmondsw has joined #openstack-nova02:33
*** mtanino has quit IRC02:34
*** rfolco has joined #openstack-nova02:35
*** rfolco has quit IRC02:35
*** rfolco has joined #openstack-nova02:35
*** rfolco has quit IRC02:36
*** rfolco has joined #openstack-nova02:36
*** rfolco has quit IRC02:37
*** rfolco has joined #openstack-nova02:37
*** rfolco has quit IRC02:38
*** zerda3 has quit IRC02:39
*** thorst_ has quit IRC02:39
*** browne has joined #openstack-nova02:40
*** rfolco has joined #openstack-nova02:41
*** rfolco has quit IRC02:41
*** rfolco has joined #openstack-nova02:41
*** rfolco has quit IRC02:42
*** browne has quit IRC02:42
*** Apoorva has joined #openstack-nova02:42
*** jamesdenton has joined #openstack-nova02:43
*** marst has joined #openstack-nova02:47
*** masayukig has quit IRC02:47
*** zerda3 has joined #openstack-nova02:49
*** zerda3 has quit IRC02:50
*** Apoorva has quit IRC02:52
*** kaisers_ has joined #openstack-nova02:55
*** zz_dimtruck is now known as dimtruck02:55
*** hongbin has joined #openstack-nova02:55
*** Swami_ has quit IRC02:56
*** kaisers has quit IRC02:58
*** Swami_ has joined #openstack-nova02:59
*** ijw has joined #openstack-nova03:00
*** Swami__ has quit IRC03:03
*** Swami has quit IRC03:03
*** Swami_ has quit IRC03:03
*** nic has quit IRC03:03
*** ijw has quit IRC03:04
*** sudswas__ has joined #openstack-nova03:06
*** sudipto_ has joined #openstack-nova03:06
mriedemso, bauzas' scheduler change has failed in the gate twice in the cells v1 job on the volume swap test...03:06
mriedemand https://review.openstack.org/#/c/399735/ merged today03:07
mriedemand we don't run the cells v1 job on the check queue for devstack, which re-enabled the test03:08
mriedemhttps://review.openstack.org/#/c/374373/03:08
mriedembut i'm pretty sure that just introduced some kind of weird race failure03:08
mriedemlyarwood: ^03:08
*** catintheroof has quit IRC03:09
*** catintheroof has joined #openstack-nova03:11
mriedemmdbooth: ^03:11
mriedemlooks like it's also killing the ceph job03:12
*** zzzeek has quit IRC03:13
*** zzzeek has joined #openstack-nova03:15
*** catintheroof has quit IRC03:15
mriedemi'm assuming this is probably just something to do with cells v1 not supporting the swap volume API03:16
mriedemhttps://bugs.launchpad.net/nova/+bug/1660511/03:17
openstackLaunchpad bug 1660511 in OpenStack Compute (nova) "TestVolumeSwap.test_volume_swap fails in cells v1 job waiting for first volume to detach" [High,Confirmed]03:17
*** tbachman has joined #openstack-nova03:18
mriedemmelwitt: if cells v1 doesn't support the swap volume API, shouldn't it fail in some kind of spectacular way?03:18
melwittmriedem: cells v1 non support usually fails in subtle crappy ways, in my experience03:19
*** raunak has joined #openstack-nova03:20
mriedemyeah that seems to be the case here03:20
mriedemi was looking at the requests in a failed job earlier and at some point it looked like something just didn't happen,03:20
mriedemlike one of the requests just stopped03:20
melwittis this a new test, i.e. something that needs to be added to the blacklist?03:20
mriedemit's a newly enabled test as of today03:21
mriedembased on tempest config03:21
melwittah03:21
mriedemso i'm going to disable it for cells v1 in devstack03:21
mriedemwe could also blacklist it in nova,03:21
mriedemwhich would probably actually make this go faster since we can fast approve it and not wait for devstack cores03:21
mriedemmtreinish: are you around?03:21
melwittoh, disable the tempest conf option if cells v1 in devstack03:21
*** alexpilotti has joined #openstack-nova03:22
mriedemyeah03:22
mriedemi'm doing that now03:22
*** ducttape_ has quit IRC03:23
*** bkopilov has quit IRC03:24
mriedemhttps://review.openstack.org/42700903:24
mriedemif i don't get a quick ack from the cores i've pinged, i'll push up a blacklist patch in nova and just move forward that way03:26
*** alexpilotti has quit IRC03:26
melwittokay03:27
openstackgerritMatt Riedemann proposed openstack/nova: Blacklist test_volume_swap from cells v1 testing  https://review.openstack.org/42701303:30
*** amotoki has joined #openstack-nova03:30
*** ijw has joined #openstack-nova03:31
*** ijw has quit IRC03:31
*** ijw has joined #openstack-nova03:31
*** ijw has quit IRC03:32
*** tbachman has quit IRC03:34
*** mtanino has joined #openstack-nova03:35
*** vladikr has quit IRC03:40
*** Apoorva has joined #openstack-nova03:40
*** mdnadeem has joined #openstack-nova03:47
*** nicolasbock has quit IRC03:50
*** psachin has joined #openstack-nova03:55
*** owalsh-afk has quit IRC03:59
*** owalsh has joined #openstack-nova04:05
*** gouthamr has quit IRC04:07
*** hongbin has quit IRC04:08
*** ducttape_ has joined #openstack-nova04:13
*** mtanino has quit IRC04:19
*** jose-phillips has quit IRC04:19
*** edmondsw has quit IRC04:19
*** yamahata has joined #openstack-nova04:21
*** diana_clarke has quit IRC04:25
*** jamesdenton has quit IRC04:25
*** dave-mccowan has quit IRC04:26
*** raunak has quit IRC04:27
*** pbandark has joined #openstack-nova04:30
*** tlian has quit IRC04:31
*** ducttape_ has quit IRC04:33
*** edmondsw has joined #openstack-nova04:34
*** baoli has quit IRC04:36
*** ayogi has joined #openstack-nova04:37
*** adisky_ has joined #openstack-nova04:38
*** thorst_ has joined #openstack-nova04:40
*** thorst_ has quit IRC04:45
*** bkopilov has joined #openstack-nova04:47
*** dimtruck is now known as zz_dimtruck04:48
abhishekkis there any provision in nova to restrict users from addind certain metadata to server?04:48
*** udesale has joined #openstack-nova04:48
mriedemhere is the not-so-subtle error http://logs.openstack.org/06/425806/8/check/gate-tempest-dsvm-cells-ubuntu-xenial/ab0bba2/logs/screen-n-cell-child.txt.gz?level=TRACE#_2017-01-31_01_25_10_11504:49
*** Sukhdev has joined #openstack-nova04:50
*** ayogi has quit IRC04:50
*** ayogi has joined #openstack-nova04:51
*** masayukig has joined #openstack-nova04:57
*** ratailor has joined #openstack-nova05:01
*** prateek has joined #openstack-nova05:06
*** unicell has joined #openstack-nova05:10
*** dikonoor has joined #openstack-nova05:11
*** raunak has joined #openstack-nova05:14
*** raunak has quit IRC05:16
*** unicell has quit IRC05:21
*** unicell has joined #openstack-nova05:21
*** nkrinner_afk is now known as nkrinner05:31
*** ducttape_ has joined #openstack-nova05:35
*** zz_dimtruck is now known as dimtruck05:39
*** ducttape_ has quit IRC05:41
*** jose-phillips has joined #openstack-nova05:42
*** dimtruck is now known as zz_dimtruck05:49
*** zz_dimtruck is now known as dimtruck05:50
*** udesale__ has joined #openstack-nova05:51
*** udesale has quit IRC05:54
*** dimtruck is now known as zz_dimtruck06:00
*** raunak has joined #openstack-nova06:00
*** sree_ has joined #openstack-nova06:01
*** sree_ is now known as Guest2317506:01
*** satyar has joined #openstack-nova06:07
*** claudiub has joined #openstack-nova06:11
*** amotoki has quit IRC06:12
*** amotoki has joined #openstack-nova06:13
*** Jack_I has joined #openstack-nova06:13
*** diga has joined #openstack-nova06:19
*** vsaienko has joined #openstack-nova06:19
*** crushil has quit IRC06:28
*** udesale__ has quit IRC06:34
*** udesale has joined #openstack-nova06:34
*** lpetrut has joined #openstack-nova06:38
*** edmondsw has quit IRC06:39
*** vsaienko has quit IRC06:39
*** Guest23175 has quit IRC06:40
*** thorst_ has joined #openstack-nova06:41
*** sree_ has joined #openstack-nova06:42
*** sree_ is now known as Guest3371606:43
*** Guest33716 has quit IRC06:44
*** wsl has joined #openstack-nova06:44
*** wsl has left #openstack-nova06:45
*** thorst_ has quit IRC06:46
*** Sukhdev has quit IRC06:49
*** raunak has quit IRC06:50
*** zz_dimtruck is now known as dimtruck06:51
openstackgerritDinesh Bhor proposed openstack/nova: Remove unused exceptions from nova.exception  https://review.openstack.org/42704106:59
*** sridharg has joined #openstack-nova07:00
*** dimtruck is now known as zz_dimtruck07:01
*** ducttape_ has joined #openstack-nova07:06
*** ekuris__ has joined #openstack-nova07:09
*** ducttape_ has quit IRC07:11
*** rcernin has joined #openstack-nova07:12
*** zz_dimtruck is now known as dimtruck07:12
*** markus_z has joined #openstack-nova07:14
*** lpetrut has quit IRC07:20
*** Oku_OS-away is now known as Oku_OS07:22
*** dimtruck is now known as zz_dimtruck07:22
*** ayogi has quit IRC07:26
*** ayogi has joined #openstack-nova07:26
*** Apoorva has quit IRC07:29
*** lpetrut has joined #openstack-nova07:30
*** andreas_s has joined #openstack-nova07:36
*** BobBall has quit IRC07:44
*** abalutoiu has quit IRC07:47
*** edand has joined #openstack-nova07:50
*** yamahata has quit IRC07:50
*** tesseract has joined #openstack-nova07:54
*** unicell has quit IRC07:59
*** abalutoiu has joined #openstack-nova08:05
*** sixdoublefive321 has joined #openstack-nova08:05
*** udesale__ has joined #openstack-nova08:06
openstackgerritBalazs Gibizer proposed openstack/nova: reno for notification-transformation-ocata  https://review.openstack.org/42681008:06
*** mjura has joined #openstack-nova08:08
*** udesale has quit IRC08:08
*** zz_dimtruck is now known as dimtruck08:13
*** dimtruck is now known as zz_dimtruck08:23
*** ralonsoh has joined #openstack-nova08:23
*** aloga has quit IRC08:27
*** Apoorva has joined #openstack-nova08:30
*** vsaienko has joined #openstack-nova08:30
*** hfu has quit IRC08:36
*** jpena|off is now known as jpena08:36
*** hfu has joined #openstack-nova08:39
*** sree has joined #openstack-nova08:40
*** sixdoublefive321 has quit IRC08:40
*** thorst_ has joined #openstack-nova08:42
*** Apoorva has quit IRC08:43
*** pbandark has quit IRC08:44
*** Apoorva has joined #openstack-nova08:44
*** foexle has joined #openstack-nova08:47
*** thorst_ has quit IRC08:48
*** jpena is now known as jpena|off08:49
*** hfu has quit IRC08:53
*** karimb has joined #openstack-nova08:53
openstackgerritTakashi NATSUME proposed openstack/nova-specs: List/show all server migration types  https://review.openstack.org/40723708:54
*** hfu has joined #openstack-nova08:55
*** hfu has quit IRC08:55
*** hfu has joined #openstack-nova08:55
*** rmart04 has joined #openstack-nova08:56
*** hfu has quit IRC08:56
tdurakovpkoniszewski:hi,  left several comments on liv-mig claim patch, could discuss them on irc if you want08:56
*** hfu has joined #openstack-nova08:56
*** hfu has quit IRC08:57
*** Apoorva has quit IRC08:57
*** lpetrut has quit IRC08:57
*** jpena|off is now known as jpena08:57
*** Apoorva has joined #openstack-nova08:58
*** karimb has quit IRC08:59
*** zzzeek has quit IRC09:00
*** rmart04 has quit IRC09:01
*** rmart04_ has joined #openstack-nova09:01
*** zzzeek has joined #openstack-nova09:01
*** aloga has joined #openstack-nova09:04
*** romainh has joined #openstack-nova09:05
*** Apoorva has quit IRC09:06
*** Apoorva has joined #openstack-nova09:07
openstackgerritTakashi NATSUME proposed openstack/nova-specs: Fix formatting errors and warnings  https://review.openstack.org/40736409:08
*** vsaienko has quit IRC09:09
*** udesale has joined #openstack-nova09:09
*** vsaienko has joined #openstack-nova09:09
openstackgerritsahid proposed openstack/nova-specs: Libvirt driver emulator threads placement policy  https://review.openstack.org/42706609:10
*** udesale__ has quit IRC09:10
*** sudipto has joined #openstack-nova09:12
*** sudipto_ has quit IRC09:12
*** sudipto_ has joined #openstack-nova09:13
*** zz_dimtruck is now known as dimtruck09:14
*** sudswas__ has quit IRC09:14
*** Apoorva has quit IRC09:21
*** karimb has joined #openstack-nova09:21
*** romainh1 has joined #openstack-nova09:21
*** romainh has quit IRC09:22
*** dimtruck is now known as zz_dimtruck09:23
*** mvk has quit IRC09:24
*** Apoorva has joined #openstack-nova09:26
openstackgerritTakashi NATSUME proposed openstack/nova: Avoid forcing translation on logging calls  https://review.openstack.org/41387609:26
*** efoley_ has joined #openstack-nova09:30
*** sree has quit IRC09:32
*** efoley__ has joined #openstack-nova09:32
*** udesale__ has joined #openstack-nova09:32
*** hshiina has joined #openstack-nova09:32
*** BobBall has joined #openstack-nova09:33
*** vsaienko has quit IRC09:34
*** yassine has joined #openstack-nova09:34
*** udesale has quit IRC09:34
*** yassine is now known as Guest1536209:34
*** dikonoor has quit IRC09:34
*** efoley_ has quit IRC09:35
*** karimb has quit IRC09:37
*** rmart04_ has quit IRC09:37
*** Apoorva has quit IRC09:39
openstackgerritJordan Pittier proposed openstack/nova: Fix unspecified bahavior on GET /servers/detail?tenant_id=X as admin  https://review.openstack.org/42625909:42
*** lucas-afk is now known as lucasagomes09:44
*** Apoorva has joined #openstack-nova09:44
abhishekkhi all, is there any provision in nova to restrict users from adding certain metadata to server?09:45
*** udesale__ has quit IRC09:48
*** gcb has joined #openstack-nova09:52
*** karimb has joined #openstack-nova09:53
*** mvk has joined #openstack-nova09:55
*** Apoorva has quit IRC09:57
openstackgerritTakashi NATSUME proposed openstack/nova: Refactor a test method including 3 test cases  https://review.openstack.org/39473809:59
openstackgerritJohn Garbutt proposed openstack/nova: DNM: hack ironic with resource providers  https://review.openstack.org/42629610:01
*** lennyb has quit IRC10:02
*** Apoorva has joined #openstack-nova10:02
*** lennyb has joined #openstack-nova10:03
*** lpetrut has joined #openstack-nova10:04
*** derekh has joined #openstack-nova10:06
*** ducttape_ has joined #openstack-nova10:07
*** satyar has quit IRC10:12
*** ducttape_ has quit IRC10:12
*** sambetts|afk is now known as sambetts10:12
johnthetubaguybauzas: are you around by any chance, and do you have the context around the ironic changes?10:13
*** zz_dimtruck is now known as dimtruck10:14
*** efried has quit IRC10:15
bauzasjohnthetubaguy: I'm here yup10:21
johnthetubaguybauzas: I wondered if you could check my comments on this patch: https://review.openstack.org/#/c/40447210:22
johnthetubaguynot sure if I am talking nonsense or sense10:22
* bauzas looking10:22
andymccrmorning all - im having a few issues with the placement service, was hoping for a bit of guidance. The first is that it seems you can't define which endpoint interface compute connects to placement on. (I put a PR in for that which worked for me but I may be way off the mark: https://review.openstack.org/#/c/426163/ )10:23
openstackgerritAlex Szarka proposed openstack/nova: Transform instance.add_fixed_ip notification  https://review.openstack.org/33287610:23
*** dimtruck is now known as zz_dimtruck10:24
*** efried has joined #openstack-nova10:24
andymccrsecondly, i get a 409 on the first PUT from the compute host, 200 on all GETs but then 400's on subsequent PUTs. The errors are vague though - I'm not too sure what steps I can take to debug that further.10:25
*** Apoorva has quit IRC10:25
johnthetubaguyandymccr: hmm, interesting10:29
johnthetubaguyandymccr: the endpoint is picked using the keystone service catalog10:29
johnthetubaguy(or should be)10:30
*** Apoorva has joined #openstack-nova10:30
johnthetubaguyandymccr: oh, the interface10:30
andymccrjohnthetubaguy: yeah it is, but it forces "public" afaict because the keystoneauth endpoint_filter doesnt include an interface option when passed by nova10:30
johnthetubaguyshould we really default to private?10:30
andymccrmy pr defaults it to public - so in essence, no change if you don't set it10:31
andymccrinternal works for my usecase, but not sure it'd work for everybody - so figured its best to not change things like that midway :) rather make them changeable.10:31
andymccri imagine in things like role gates/devstack it makes no difference since the endpoints are probably all the same regardless of internal/public10:32
johnthetubaguyits an admin only API (at least by design) so its probably a mute point10:32
andymccrif its admin only perhaps it should default to admin instead?10:33
andymccrfrom my pov, i need it to not be public or else i have to do some munging when i create endpoints to ensure the public endpoint is created differently to all other public endpoints and uses the internal ip10:33
*** pbandark has joined #openstack-nova10:34
johnthetubaguyoh, possibly, yeah, default to admin makes more snese10:34
johnthetubaguyin theory non admin users would just get 403 errors I think, but its good to message that better10:35
johnthetubaguybauzas: cdent: do you have better context on the intended deployment stuff, for our OSA friends?10:36
bauzasjohnthetubaguy: sorry, I had an internal problem10:37
bauzasand I'm very busy, so could you please explain me the issue ?10:37
johnthetubaguybauzas: service catalog, we default to accessing the "public" URL for placement10:37
johnthetubaguythat seems odd, given its an admin only API10:38
andymccr(and you can't override it afaict)10:38
bauzaslemme verify something10:38
bauzasI think nova just doesn't use the public field10:38
johnthetubaguyit seems to, because of how keystone auth works10:39
johnthetubaguyinfact it only uses the public field, if I am remembering correctly10:39
*** Apoorva has quit IRC10:39
bauzas:/10:39
toabctlmriedem, hey. anything I can do to push https://review.openstack.org/#/c/402318/ ?10:41
bauzasjohnthetubaguy: AFAIK, we don't have yet policies10:41
johnthetubaguybauzas: I was wondering about that... uh, oh10:42
*** alexpilotti has joined #openstack-nova10:42
*** thorst_ has joined #openstack-nova10:44
*** alexpilotti has quit IRC10:44
*** alexpilotti has joined #openstack-nova10:45
*** efried has quit IRC10:45
*** hfu has joined #openstack-nova10:45
johnthetubaguyI think we are OK for policy, looking at this: https://github.com/openstack/nova/blob/a0100d61a9531a3e32f2c72eab54157c32114f5b/nova/api/openstack/placement/handler.py#L17510:47
bauzasagreed10:48
*** thorst_ has quit IRC10:49
gmannjohnthetubaguy: can you look into release notes patch of sort/filter things - https://review.openstack.org/#/c/421760/10:50
gmannjohnthetubaguy: i have removed the all_tenant policy bits from there10:50
bauzasjohnthetubaguy: when I wrote the PlacementFixture, I saw that it was returning 401s if I wasn't auth'd10:50
johnthetubaguygmann: ah, we trimmed it down, cool10:50
bauzasnow I remember that10:50
gmannyea10:50
johnthetubaguybauzas: cool10:50
bauzasjohnthetubaguy: but that's easily testable thanks to the fixture I wrote, I guess10:50
johnthetubaguybauzas: there is that other patch I was taking about before, the problem there I think is that some of the code added around allocations is unreachable, and doesn't fix things for instances created after we start reporting custom classes, but I could be reading it all wrong10:51
johnthetubaguyandymccr: sorry, I got fixated on the first issue10:51
johnthetubaguyandymccr: you had a second question10:51
bauzasjohnthetubaguy: I briefly looked at the code, but I need time for loading context in mind10:52
johnthetubaguylots of API errors10:52
johnthetubaguybauzas: ack10:52
bauzasand I'm awfully busy10:52
bauzas:(10:52
johnthetubaguybauzas: understood10:52
johnthetubaguygmann: I worry about advertising the alias10:57
andymccrjohnthetubaguy: yeah i'm trying to understand what a 409 would mean in terms of what is actually conflicting - the endpoint seems ok since it's accepting GET requests (responding with a 200), but any PUT requests are returning bad request 400, and the initial put request returns a 409 conflict10:58
*** efried has joined #openstack-nova10:58
andymccrmy understanding is the resource already exists, but im wondering where or what i can do to debug that a bit further10:59
johnthetubaguyandymccr: DB configuration is bad, maybe?10:59
andymccrjohnthetubaguy: yeah thats what im thinking - so i noticed the nova_placement db that is created isn't populated, perhaps it needs write permissions that i havn't set up properly10:59
johnthetubaguyandymccr: needs a new nova-manage cmd running11:00
andymccrfrom a nova perspective i just need to run the nova-manage api_db sync command?11:00
*** jpena has quit IRC11:00
*** ltomasbo has quit IRC11:00
*** dmellado has quit IRC11:00
johnthetubaguyandymccr: no11:01
gmannjohnthetubaguy:  i see your point, and agree that can avoid users using those alias. ll update those in respective  attribute description.11:01
johnthetubaguyandymccr: oh... well maybe, I can't remember now actually11:01
andymccrjohnthetubaguy: i think i must be missing something because it's not quite happening :)11:02
johnthetubaguygmann: awesome, thank you11:02
*** jpena has joined #openstack-nova11:02
johnthetubaguyandymccr: so do you have it configured the same as the API DB in the conf?11:02
openstackgerritBob Ball proposed openstack/nova: Fix live migrate with XenServer  https://review.openstack.org/42442811:02
*** takashin has quit IRC11:02
andymccrjohnthetubaguy: no there is a separate db11:03
andymccror rather we've setup a separate database11:03
johnthetubaguyandymccr: OK, that sounds good11:03
andymccrwith a separate user11:03
*** dmellado has joined #openstack-nova11:03
andymccrbut then the api_db sync doesn't populate the new db - so i wonder if its a permissions issue, although i imagine it would pull the connection string from nova.conf (for the placement db?)11:03
johnthetubaguyandymccr: oh, interesting. it should pull that from nova.conf, but I wonder if that works11:04
*** takashin has joined #openstack-nova11:05
*** ltomasbo has joined #openstack-nova11:07
johnthetubaguyandymccr: so that is what devstack does: https://github.com/openstack-dev/devstack/blob/master/lib/placement#L14311:07
*** Apoorva has joined #openstack-nova11:08
openstackgerritStephen Finucane proposed openstack/nova: Ignore IOError when creating 'console.log'  https://review.openstack.org/39264311:08
andymccrjohnthetubaguy: hmm yeah, so that pretty much lines up - i tested with --conf-file but let me test that again11:09
sfinucanartom, mdbooth: ^^^11:09
johnthetubaguyandymccr: this is our nova.conf in the gate: http://logs.openstack.org/63/426163/2/check/gate-tempest-dsvm-neutron-full-ubuntu-trusty/a5ece9b/logs/etc/nova/nova.conf.txt.gz11:09
*** mvk has quit IRC11:10
BobBallmriedem: See my updates to https://bugs.launchpad.net/nova/+bug/1658877 .  I'd very much hope we can still aim for a fix in Ocata as changes to the object are needed and I understand these are particularly difficult to backport?11:10
openstackLaunchpad bug 1658877 in OpenStack Compute (nova) "live migration failed with XenServer as hypervisor" [High,In progress] - Assigned to Bob Ball (bob-ball)11:10
mdboothsfinucan: You want to add a test for ignoring EPERM? ;)11:11
*** romainh1 has quit IRC11:11
johnthetubaguyandymccr: did you configure api_database with the placement database, and use the correct conf file when calling api_sync?11:11
sfinucanmdbooth: Damn - you're quick. Had just started fixing that :)11:11
* mdbooth floats like a butterfly...11:11
andymccrjohnthetubaguy: yes on the config file, but we have a the placement_database section defined.11:12
johnthetubaguyandymccr: something tells me we screwed up, and nova-manage doesn't read that11:12
andymccrjohnthetubaguy: interesting, let me test and remove that section11:13
andymccri assume it'll just default to the api_database then?11:13
johnthetubaguyandymccr: yeah, we will want to not do that, but it would be good to see that fix your issues11:13
andymccrjohnthetubaguy: although - i know the puppet team has dedicated configuration for nova_placement database.11:14
andymccrbut lets test11:14
andymccrim somewhat sure its db related11:14
johnthetubaguyandymccr: yeah, we should do a separate database, but it would be good to see if that fixes things11:14
*** zz_dimtruck is now known as dimtruck11:15
*** nicolasbock has joined #openstack-nova11:16
openstackgerritTakashi NATSUME proposed openstack/nova: Avoid forcing translation on logging calls  https://review.openstack.org/41387611:16
johnthetubaguybauzas: have you noticed how you can't have a separate placement database using the placement_database settings? seems in the gate we test with a single shared API database11:18
*** Apoorva has quit IRC11:21
*** mvk has joined #openstack-nova11:23
openstackgerritStephen Finucane proposed openstack/nova: Ignore IOError when creating 'console.log'  https://review.openstack.org/39264311:24
*** dimtruck is now known as zz_dimtruck11:25
*** hshiina has quit IRC11:25
*** Apoorva has joined #openstack-nova11:26
*** adisky_ has quit IRC11:29
openstackgerritAndy McCrae proposed openstack/nova: Allow placement endpoint interface to be set  https://review.openstack.org/42616311:31
*** sree has joined #openstack-nova11:32
*** sree has quit IRC11:34
*** sree has joined #openstack-nova11:34
*** Apoorva has quit IRC11:35
*** ducttape_ has joined #openstack-nova11:38
rpodolyakajohnthetubaguy: yeah, a separate placement db must only be possible with https://review.openstack.org/#/c/362766/11:38
*** Apoorva has joined #openstack-nova11:40
rpodolyakaI wonder if we left placement_database conf group intentionally in this revert - https://review.openstack.org/#/c/360638/ or by mistake11:40
*** timello has joined #openstack-nova11:40
*** stvnoyes has quit IRC11:41
*** hfu has quit IRC11:41
johnthetubaguyrpodolyaka: ah, right, I couldn't see that code when I went digging11:42
*** ducttape_ has quit IRC11:43
johnthetubaguyandymccr: so I think we could make the placement DB separate by having a placement.conf that is used for the nova-manage call and the placement API, where the api-database is configured as the placement database11:43
johnthetubaguyandymccr: seems we are missing some bits on our side for that to work... more sensibly11:44
openstackgerritAbhishek Kekane proposed openstack/nova: Use is_valid_cidr and is_valid_ipv6_cidr from oslo_utils  https://review.openstack.org/32037511:45
andymccrjohnthetubaguy: ahh ok - weird. i wonder how the puppet stuff works then - im running a few tests now wrt not using a set placement_db11:46
johnthetubaguyandymccr: +111:47
andymccri'll try speak to EmilienM a bit later - he put the pr in for that :)11:47
sfinucanlyarwood: Could you say if this is suitable bugfix to backport to Mitaka? https://review.openstack.org/42711911:48
sfinucanPer this bug report https://bugs.launchpad.net/nova/+bug/165779111:48
openstackLaunchpad bug 1578155 in OpenStack Compute (nova) newton "duplicate for #1657791 'hw:cpu_thread_policy=prefer' misbehaviour" [Medium,Fix committed] - Assigned to Stephen Finucane (stephenfinucane)11:48
*** Apoorva has quit IRC11:49
*** sree has quit IRC11:51
*** hfu has joined #openstack-nova11:52
*** gszasz has joined #openstack-nova11:57
tdurakovjohnthetubaguy: could you please take a look https://review.openstack.org/#/c/244489/58 especially interested in default rpc method params, see my comments on ps-58, is it really required? I would like to clarify that moment for myself11:57
*** sdague has joined #openstack-nova11:58
*** sdague_ has joined #openstack-nova12:00
johnthetubaguyandymccr: on a related note, not sure if you have the cell0 database setup yet, there are details in here: http://docs.openstack.org/developer/nova/cells.html#setup-of-cells-v212:00
*** ratailor has quit IRC12:01
*** udesale has joined #openstack-nova12:01
andymccrjohnthetubaguy: i was looking at that - is that a requirement? so you have to have a cell0 db (and a cell1 db)12:01
openstackgerritStephen Finucane proposed openstack/nova-specs: Add extra-specs-image-meta-conflicts spec  https://review.openstack.org/38238212:01
johnthetubaguyandymccr: yes, its all required now, I believe12:02
andymccrjohnthetubaguy: i wonder if that is the issue then.12:02
andymccrhmm12:02
andymccrok cool that gives me some more things to try at the lest12:02
andymccr*least12:02
openstackgerritStephen Finucane proposed openstack/nova-specs: Reserve NUMA nodes with PCI devices attached  https://review.openstack.org/36446812:03
*** Apoorva has joined #openstack-nova12:03
johnthetubaguyandymccr: you need the API db to know which cell a host is in, else we don't know which cell to send the build request, etc12:03
openstackgerritStephen Finucane proposed openstack/nova-specs: Add 'attributed-action-notifications' spec  https://review.openstack.org/38857312:04
lyarwoodsfinucan: hey sorry was on a call, no I don't think it is http://docs.openstack.org/project-team-guide/stable-branches.html#support-phases12:04
*** karimb has quit IRC12:05
*** dave-mccowan has joined #openstack-nova12:06
sfinucanlyarwood: I thought as much. I was hoping the clause might apply though :) " It’s nevertheless allowed to backport fixes for other bugs if their safety can be easily proved. "12:06
*** nicolasbock has quit IRC12:06
lyarwoodsfinucan: yeah we do make exceptions for simple fixes, I'm just going by the bug subject and Importance posted in the channel12:07
kashyapHmm, this has become a bit of blocker it seems -- https://bugs.launchpad.net/bugs/164391112:10
openstackLaunchpad bug 1643911 in OpenStack Compute (nova) "libvirt randomly crashes on xenial nodes with "*** Error in `/usr/sbin/libvirtd': malloc(): memory corruption:"" [Medium,Confirmed]12:10
kashyapAnd the upstream libvirt folks go: "the stack trace will be totally useless for malloc memory corruption"12:10
*** bkopilov has quit IRC12:11
kashyapBecause: https://bugs.launchpad.net/nova/+bug/1643911/comments/112:12
openstackLaunchpad bug 1643911 in OpenStack Compute (nova) "libvirt randomly crashes on xenial nodes with "*** Error in `/usr/sbin/libvirtd': malloc(): memory corruption:"" [Medium,Confirmed]12:12
kashyapIf you run libvirtd under valgrind in the Gate, then it might hide the problem12:12
kashyapAt least libvirt upstream admits: "I mean, we [libvirt] are clearly overwriting the memory somewhere; but stack trace is not helpful in those cases"12:13
*** vsaienko has joined #openstack-nova12:13
*** zz_dimtruck is now known as dimtruck12:16
*** Apoorva has quit IRC12:21
*** catintheroof has joined #openstack-nova12:22
*** nicolasbock has joined #openstack-nova12:23
*** rfolco has joined #openstack-nova12:25
*** rfolco has quit IRC12:26
*** dimtruck is now known as zz_dimtruck12:26
*** Apoorva has joined #openstack-nova12:26
*** rfolco has joined #openstack-nova12:26
*** ayogi has quit IRC12:28
*** ayogi has joined #openstack-nova12:29
openstackgerritGhanshyam Mann proposed openstack/nova: Add release note and docs for filter/sort whitelist  https://review.openstack.org/42176012:32
*** thorst_ has joined #openstack-nova12:35
*** vsaienko has quit IRC12:36
*** baoli has joined #openstack-nova12:37
*** bvanhav has joined #openstack-nova12:37
openstackgerritRodolfo Alonso Hernandez proposed openstack/nova: Optimize time during PCI alias parsing process  https://review.openstack.org/42714512:37
*** Apoorva has quit IRC12:40
*** sree has joined #openstack-nova12:40
*** baoli has quit IRC12:42
*** Guest15362 has quit IRC12:42
*** Apoorva has joined #openstack-nova12:44
*** lucasagomes is now known as lucas-hungry12:46
*** Guest15362 has joined #openstack-nova12:46
*** Jeffrey4l__ is now known as Jeffrey4l12:47
*** ayogi has quit IRC12:48
*** Hosam has joined #openstack-nova12:50
*** jaypipes has joined #openstack-nova12:50
*** takashin has left #openstack-nova12:52
*** edand has quit IRC12:54
*** romainh has joined #openstack-nova12:54
*** Apoorva has quit IRC12:58
*** cdent has joined #openstack-nova13:02
*** udesale has quit IRC13:02
*** udesale has joined #openstack-nova13:03
*** satyar has joined #openstack-nova13:03
*** ducttape_ has joined #openstack-nova13:06
*** kylek3h has quit IRC13:07
openstackgerritEmilien Macchi proposed openstack/nova: DNM - tripleo CI test  https://review.openstack.org/42694913:07
*** tblakes has joined #openstack-nova13:07
*** Apoorva has joined #openstack-nova13:08
*** tbachman has joined #openstack-nova13:09
*** mdnadeem has quit IRC13:10
*** edmondsw has joined #openstack-nova13:12
cdentjohnthetubaguy, andymccr: Did you guys work out what was going on a while back with the placement db or are things still weird?13:13
cdentandymccr: I'm pretty sure the original 409 that is happening is because the generation on the resource provider is run, but it corrects itself so the next writes out to work13:13
cdents/out/ought/13:13
cdent(they do for me when I've been testing things in devstack)13:14
cdentso your theory that it is a database problem is likely13:14
*** Hosam has quit IRC13:14
*** zz_dimtruck is now known as dimtruck13:17
*** Apoorva has quit IRC13:17
johnthetubaguycdent: I think the DB wasn't configured correctly13:17
johnthetubaguycdent: but I could be wrong, not seen andymccr latest results13:17
*** Apoorva has joined #openstack-nova13:17
johnthetubaguycdent: I forgot we didn't merge the stuff to support the separate placement_database13:18
*** liverpooler has joined #openstack-nova13:18
andymccrjohnthetubaguy: cdent - thanks for the info :) I'm testing some more things with database changes/perms etc - will update when i (hopefully) have something working!13:18
cdentyeah, it totally stalled out. I have a working change that I keep up to date for it, but people stated unspecific reservations about it, without specifics there was no way to move on...13:19
* cdent is looking at the endpoint adjustment patch now13:19
johnthetubaguycdent: jaypipes: I still had worries about the allocations in that ironic patch, I added comments/suggestions13:19
cdentjohnthetubaguy: that's next13:20
johnthetubaguycool, thanks13:20
cdentI skimmed your comments earlier today and my immediate thought was "moar tests"13:20
markus_zmriedem: Would you please remove/re-evaluate your -1 on https://review.openstack.org/#/c/406897/ ?13:21
*** diana_clarke has joined #openstack-nova13:22
*** Apoorva has quit IRC13:22
openstackgerritAndy McCrae proposed openstack/nova: Allow placement endpoint interface to be set  https://review.openstack.org/42616313:22
openstackgerritSylvain Bauza proposed openstack/nova: Add a status check for enabled filters  https://review.openstack.org/42720013:23
sfinucanKevin_Zheng: Could you take a look at https://bugs.launchpad.net/nova/+bug/1660317 ? fwict, the error message that they're receiving should be mitigated by a patch of yours13:24
openstackLaunchpad bug 1660317 in ironic (Ubuntu) "NotImplementedError for detach_interface in nova-compute during instance deletion" [Undecided,New]13:24
sfinucanNamely, this patch https://github.com/openstack/nova/commit/4ad414f3b1216393301ef268a64e61ca1a3d5be913:25
sfinucan...though I could be reading that incorrectly :)13:25
mriedemsfinucan: have you seen https://review.openstack.org/#/c/403925/ ?13:26
mriedemmarkus_z: removed the -113:26
mriedemuntil i can dig deeper13:26
*** dimtruck is now known as zz_dimtruck13:26
markus_zmriedem: Thanks13:26
*** Apoorva has joined #openstack-nova13:26
sfinucanmriedem: I had not. I can take a look now13:27
Kevin_Zhengsfinucan: sorry, I'm on vacation and didn't got my computer with me:(13:28
*** ducttape_ has quit IRC13:28
mriedemsfinucan: just wondering if it's related to what you're asking about13:28
bauzasmriedem: FYI https://review.openstack.org/42720013:28
*** jpena is now known as jpena|lunch13:28
sfinucanKevin_Zheng: No problem at all - it's not that important. Enjoy your vacation :)13:28
sfinucanmriedem: I'm looking but I don't think so13:29
*** jheroux has joined #openstack-nova13:29
Kevin_ZhengThanks13:29
openstackgerritOpenStack Proposal Bot proposed openstack/os-vif: Updated from global requirements  https://review.openstack.org/39534713:32
*** karimb has joined #openstack-nova13:33
*** Apoorva has quit IRC13:35
*** catinthe_ has joined #openstack-nova13:36
mriedemtoabctl: +2 on https://review.openstack.org/#/c/402318/ with some comments - the libvirt driver ignores the requested device name that tempest is sending in anyway13:37
mriedemso if the libvirt driver was always defaulting to vda i'm not sure how things worked with the xenproject ci13:37
mriedemdoesn't look like the libvirt+xen ci overrides that in tempest.conf either http://logs.openstack.xenproject.org/18/402318/1/check/dsvm-tempest-xen/6b9829a/logs/tempest_conf.txt.gz13:38
mriedemanthonyper: ^13:39
toabctlmriedem, is libvirt+xen tested?13:39
mriedemyes13:39
mriedemthat's the xenproject ci13:39
*** catintheroof has quit IRC13:40
*** kylek3h has joined #openstack-nova13:40
*** Apoorva has joined #openstack-nova13:40
mriedeme.g.13:41
mriedem2016-11-25 08:10:13.591 13563 WARNING nova.virt.libvirt.driver [req-98364dbb-0b80-49f3-9d59-ac476fdd3855 tempest-AttachVolumeShelveTestJSON-134397458 tempest-AttachVolumeShelveTestJSON-134397458] [instance: b0cf5c0f-ae34-4ca1-b441-08a3f4a85b71] Ignoring supplied device name: /dev/vdb13:41
*** lucas-hungry is now known as lucasagomes13:43
*** diga has quit IRC13:44
mriedemtoabctl: dropped to a +1 until someone that knows about the xenproject ci can explain why this wasn't a failure before13:45
mriedemmaybe it's just because that ci wasn't running the test in question, test_volume_boot_pattern13:46
mriedemanthonyper: is the xenproject ci whitelist/blacklist available somewhere? i'm wondering if i could enable test_volume_boot_pattern against https://review.openstack.org/#/c/402318/13:47
BobBallDEVSTACK_GATE_TEMPEST_REGEX='(?!.*\[.*\bslow\b.*\]|.*TestVolumeBootPattern\.test_(volume_boot_pattern|create <-- confirmed13:47
mriedemwhere is that defined though?13:48
*** vladikr has joined #openstack-nova13:48
BobBallDefined here http://xenbits.xen.org/gitweb/?p=openstack/ci-loop-config.git;a=blob;f=jenkins/jobs/jobs.yaml;h=a1a29f4b0fdaa4730c842736ee7e2ae4e49d06da;hb=HEAD13:48
BobBallwhich is not easily testable13:48
*** cburgess has quit IRC13:48
toabctlmriedem, yes. the test is not executed.13:48
toabctlbut then I should reenable the test with my changeset13:48
*** Apoorva has quit IRC13:49
openstackgerritAndy McCrae proposed openstack/nova: Allow placement endpoint interface to be set  https://review.openstack.org/42616313:49
mriedemBobBall: ah so i can't use a depends-on like the xenserver ci13:49
*** Apoorva has joined #openstack-nova13:49
toabctloh. it's on the external CI. so I can not simply change that13:49
*** cburgess has joined #openstack-nova13:49
mriedemright13:49
mriedemBobBall: we could move that blacklist into the nova tree...13:50
toabctlso I did test that manually on our internal CI...13:50
mriedemthen it's branch-specific13:50
BobBallSorry mriedem, it's not...13:50
*** baoli has joined #openstack-nova13:50
mriedemBobBall: like this https://github.com/openstack/nova/blob/master/devstack/tempest-dsvm-lxc-rc13:50
mriedemthen that job definition just needs to use the DEVSTACK_GATE_TEMPEST_REGEX variable13:51
BobBallThat's nice, but curious because clearly it's not *just* nova implicated with some of the tempest tests - for example there have historically been some tempest tests that only work with libvirt13:51
*** fnordahl has quit IRC13:51
*** fnordahl has joined #openstack-nova13:51
BobBalli.e. the tests make assumptions about the hypervisor e.g. that it can support qcow2 images made in the test themselves13:52
mriedemright so the tempest defaults are based on the nova defaults, which are libvirt13:53
mriedemif you're not using those defaults, you have to configure stufff13:53
mriedemthat doesn't mean we couldn't control the xenproject ci blacklist from within the nova tree13:53
BobBallIndeed - but the case I was thinking about wasn't configurable13:53
mriedembauzas: rechecked your scheduler stuff, the grenade patch merged before cells v1 broke, but the cells v1 thing is fixed now so all of the rechecks are running13:55
bauzasmriedem: cool, thanks for following that13:55
*** fnordahl has quit IRC13:55
*** fnordahl has joined #openstack-nova13:55
*** cleong has joined #openstack-nova13:56
*** gouthamr has joined #openstack-nova13:56
mriedemoomichi: sdague: johnthetubaguy: want to take a crack at this? i've got a +2 on it https://review.openstack.org/#/c/424759/13:57
mriedemand the tempest change is approved https://review.openstack.org/#/c/426991/13:57
johnthetubaguymriedem: talking a look now, would be good to get that in13:58
johnthetubaguymriedem: there was a spec to match it too, but thats more a documentation thing at this point I guess?13:59
tdurakovPaulMurray paul-carlton2 eliqiao shaohe_feng pkoniszewski yuntongjin mdbooth kashyap kmartin claudiub tobasco tdurakov abhishekk mrhillsman raj_singh wznoinsk siva_krishnan hi, live-migration subteam meeting in a minute on #openstack-meeting-313:59
*** Apoorva has quit IRC14:02
*** Apoorva has joined #openstack-nova14:03
*** sdague_ has quit IRC14:05
*** bvanhav_ has joined #openstack-nova14:06
*** bvanhav has quit IRC14:07
mriedemjohnthetubaguy: the spec was merged yesterday14:10
*** satyar has quit IRC14:10
mriedemjohnthetubaguy: the spec was a normal spec14:11
mriedemsince it's an api change and we require a spec for api changes14:11
johnthetubaguymriedem: yep, found that, all good14:11
*** pbandark has quit IRC14:12
edleafecdent: about to get sucked into 4 hours of meetings, but for "moar tests", you could start adding on to https://review.openstack.org/#/c/404472/28/nova/tests/functional/api/openstack/placement/gabbits/ironic-inventory.yaml14:14
*** bvanhav_ is now known as bvanhav14:14
edleafeIt's just the barest of tests now, but I started it with "moar" in mind14:14
cdentedleafe: reviewing that stuff now, will look14:14
*** Apoorva has quit IRC14:16
*** zz_dimtruck is now known as dimtruck14:17
cdentedleafe: I think we need functional tests on the resource tracker side, not the gabbi side?14:19
*** tbachman has quit IRC14:19
*** tbachman has joined #openstack-nova14:20
*** Apoorva has joined #openstack-nova14:21
*** mdrabe has joined #openstack-nova14:22
*** mriedem has quit IRC14:23
sean-k-mooneymriedem: regarding os-vif no i dont think there is anything that is outstanding that needs to merge before creating stable/ocata14:24
*** gcb has quit IRC14:24
edleafecdent: ah, ok. I just had time to briefly read the scrollback.14:24
cdentedleafe: I'll see if I can come up with something. At least think on it.14:25
*** Apoorva has quit IRC14:26
*** Apoorva has joined #openstack-nova14:27
*** ducttape_ has joined #openstack-nova14:27
*** dimtruck is now known as zz_dimtruck14:27
*** mlavalle has joined #openstack-nova14:29
*** Cristina_ has quit IRC14:29
*** Apoorva has quit IRC14:31
*** Apoorva has joined #openstack-nova14:31
*** ducttape_ has quit IRC14:31
*** Cristina_ has joined #openstack-nova14:32
*** breitz has quit IRC14:35
*** hfu has quit IRC14:35
*** Apoorva has quit IRC14:35
*** READ10 has joined #openstack-nova14:35
*** jpena|lunch is now known as jpena14:37
*** prateek has quit IRC14:38
dansmithhmm, the scheduler patches are still not through the gate?14:39
*** karimb has quit IRC14:41
*** karimb has joined #openstack-nova14:42
derekhcould somebody put a extra pair of eyes on https://review.openstack.org/#/c/417227/  , its holding back another patch I'm trying to fix14:45
*** mtanino has joined #openstack-nova14:45
*** READ10 has quit IRC14:46
*** hongbin has joined #openstack-nova14:49
*** jamesdenton has joined #openstack-nova14:50
*** READ10 has joined #openstack-nova14:50
*** zz_dimtruck is now known as dimtruck14:51
*** satyar has joined #openstack-nova14:55
*** gszasz has quit IRC14:56
*** gszasz has joined #openstack-nova14:56
*** eharney has joined #openstack-nova14:57
*** crushil has joined #openstack-nova14:58
johnthetubaguydansmith: I think its that nasty gate bug fix that hadn't merged, when I last checked14:58
dansmithwhich one? is it approved?14:59
*** tbachman has quit IRC15:01
*** tbachman has joined #openstack-nova15:02
*** prateek has joined #openstack-nova15:02
*** bkopilov has joined #openstack-nova15:03
sfinucanderekh: Hmm, much as I hate to say it, could you move that patch /after/ the bug patch?15:03
sfinucanI realize that might mean using Mox, but I'm not sure if we can merge a Mox-mock conversion patch first now...?15:04
*** dimtruck is now known as zz_dimtruck15:04
sfinucanLet's see what johnthetubaguy and/or dansmith think first though15:04
*** annegentle has joined #openstack-nova15:05
johnthetubaguysfinucan: I was eyeing that one up, I can try squash it15:05
johnthetubaguydansmith: one sec, checking15:05
*** burgerk has quit IRC15:06
*** mriedem has joined #openstack-nova15:06
johnthetubaguydansmith: https://review.openstack.org/#/c/427009/15:06
dansmithjohnthetubaguy: ah okay15:06
*** nkrinner is now known as nkrinner_afk15:07
mdboothdansmith: I've had 2 people call me on the N, N-1 upgrade thing in 2 days now: https://review.openstack.org/#/c/423392/15:09
mdboothdansmith: Can you confirm I am/am not over-assuming there?15:10
dansmithmdbooth: I think they want you to use versions to enforce the behavior and not just assume right?15:11
*** ayogi has joined #openstack-nova15:11
*** kaisers_ has quit IRC15:11
*** ayogi has quit IRC15:12
*** kaisers has joined #openstack-nova15:12
*** ayogi has joined #openstack-nova15:13
*** ayogi has quit IRC15:14
*** cdelatte has joined #openstack-nova15:14
*** ayogi has joined #openstack-nova15:14
*** tovin07 has joined #openstack-nova15:14
*** markus_z has quit IRC15:15
mriedemsean-k-mooney: jaypipes: sfinucan: i'm going to put up a releases patch to cut stable/ocata for os-vif today unless you are aware of anything we have to get merged before that happens15:17
sfinucanmriedem: Nothing on my end15:17
*** kaisers has quit IRC15:17
openstackgerritMaciej Szankin proposed openstack/nova: conf: Resolve TODOs in 'database'  https://review.openstack.org/39369515:18
*** jaosorior has joined #openstack-nova15:20
*** kaisers has joined #openstack-nova15:22
*** vsaienko has joined #openstack-nova15:22
sean-k-mooneymriedem: i dont think anything else needs to be added before the release15:23
jaypipesmriedem: go for it.15:23
*** burgerk has joined #openstack-nova15:26
*** kaisers has quit IRC15:26
*** tlian has joined #openstack-nova15:27
mriedemok, thanks. will do after morning meetings.15:27
*** owalsh is now known as owalsh-brb15:27
*** psachin has quit IRC15:27
*** gatuus has joined #openstack-nova15:27
mdboothmriedem: Do we have a concept of a minimum supported service version?15:28
*** owalsh has joined #openstack-nova15:30
*** ayogi has quit IRC15:31
*** owalsh-brb has quit IRC15:31
*** armax has joined #openstack-nova15:32
*** breitz has joined #openstack-nova15:33
*** udesale has quit IRC15:35
*** kaisers has joined #openstack-nova15:35
*** vsaienko has quit IRC15:36
*** mtanino_ has joined #openstack-nova15:39
*** mtanino has quit IRC15:40
mdboothjohnthetubaguy: Can we hold off on the libvirt+xen thing temporarily? Concerned about adding a quick hack for something that has been broken a long time. Quick hacks tend to last a long time, and are very expensive. Looking now.15:40
*** raunak has joined #openstack-nova15:40
mdboothIf this is causing a problem in the gate, could we disable the test again?15:41
johnthetubaguymdbooth: we could, just seems like a big gain not to have15:43
mdboothjohnthetubaguy: I'm not definitely saying we shouldn't take the patch, just that I'd like to look first at why we need it, especially as ftersin was also concerned about it.15:44
*** marst has quit IRC15:44
johnthetubaguymdbooth: you got the link again?15:44
mdboothProblem is it's adding a new foo(x) returns y normally, but z in this other bizarre situation which nobody will every think of.15:44
mdboothhttps://review.openstack.org/#/c/402318/1/nova/virt/libvirt/blockinfo.py15:44
johnthetubaguyI am just cruzing all bugs with a +2 already15:44
mdboothWe have lots of these, and they're just edge-case bug factories.15:45
*** archies_ has joined #openstack-nova15:45
*** catinthe_ has quit IRC15:45
mdboothjohnthetubaguy: My immediate thought, btw, is your suggestion to fix it during spawn15:47
*** kaisers has quit IRC15:48
*** marst has joined #openstack-nova15:51
*** marst_ has joined #openstack-nova15:52
*** raunak has quit IRC15:52
*** marst has quit IRC15:55
*** foexle has quit IRC16:00
*** jdurgin1 has joined #openstack-nova16:01
openstackgerritLee Yarwood proposed openstack/nova: libvirt: Limit destroying disks during cleanup to spawn  https://review.openstack.org/42726716:03
*** crushil has quit IRC16:03
*** ducttape_ has joined #openstack-nova16:03
*** READ10 has quit IRC16:04
*** crushil has joined #openstack-nova16:04
*** rcernin has quit IRC16:04
*** stvnoyes has joined #openstack-nova16:05
lyarwooddansmith, mdbooth ; ^ I'd appreciate your thoughts on https://review.openstack.org/427267 if you have time today16:05
mdboothlyarwood: It just set my spidey sense tingling :) Was about to look.16:05
lyarwoodmdbooth: thanks16:06
*** archies_ has quit IRC16:06
*** tbachman has quit IRC16:07
derekhsfinucan: sorry was on a call, so do I need to swap the patches around?16:08
sfinucanderekh: No, I think johnthetubaguy is just going to squash them16:09
derekhsfinucan: ok, cool16:09
mdboothlyarwood: Oof, so a failed reboot would delete the instance?16:10
*** ekuris__ has quit IRC16:10
lyarwoodmdbooth: well, failure to plug the vifs during a reboot would delete any local files, yeah16:10
mdboothEesh, that's baaaad16:11
mdboothlyarwood: Compare this change for context: https://review.openstack.org/#/c/389593/2/nova/virt/libvirt/driver.py16:13
dansmithlyarwood: oof, that's from me huh?16:13
dansmithalthough it's just because I'm calling cleanup where we weren't before16:13
mdboothI'm -1 on ^^^ because it superficially probably passes the cases we can think of right now, but it depends on the phase of the moon.16:14
mdboothIf we took your change and ^^^, rbd disks would be deleted anyway16:14
mdboothThat's just a comment on why I'm against that sort of thing in general, btw, and I want us to start fixing root causes.16:15
dansmithmdbooth: is this demonstrably worse than what we have today? because I don't think it is, and the bug is pretty bad...16:15
mdboothAt first glance, I think your patch effectively specifies the caller's intent.16:15
mdboothdansmith: I'm not being clear, I think the *other* patch is bad, and lyarwood's patch is an example of why.16:16
mdboothdansmith: Sorry, was internally also trying to justify being against https://review.openstack.org/#/c/402318/16:17
* dansmith is lost16:18
*** ducttape_ has quit IRC16:19
* lyarwood is back and also lost ;D16:19
lyarwooddansmith: yeah AFAICT we just didn't make it to cleanup before your patch16:19
*** ducttape_ has joined #openstack-nova16:19
dansmithso, to restate, I think we should merge lyarwood's patch16:20
lyarwoodcool thanks16:20
*** zz_dimtruck is now known as dimtruck16:20
mdboothdansmith: Yeah. Reviewing now.16:20
*** yamahata has joined #openstack-nova16:21
*** tovin07 has quit IRC16:22
*** Oku_OS is now known as Oku_OS-away16:22
*** tovin07 has joined #openstack-nova16:22
*** Guest52040 has joined #openstack-nova16:22
*** READ10 has joined #openstack-nova16:23
*** baoli has quit IRC16:26
*** _ducttape_ has joined #openstack-nova16:29
*** sridharg has quit IRC16:30
*** xinli has joined #openstack-nova16:31
*** tovin07 has left #openstack-nova16:31
*** ducttape_ has quit IRC16:32
mriedemsdague: did you see this? http://lists.openstack.org/pipermail/openstack-dev/2017-January/111137.html16:34
mriedemsdague: i was wondering if the new devstack-tools stuff somehow screwed up local.conf for multinode grenade?16:35
sdaguemriedem: it's possible, though I thought I had that covered16:35
mriedemwe know d-g is setting it http://logs.openstack.org/61/417961/40/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/8c363cd/console.html.gz#_2017-01-27_23_16_09_27178416:36
sdaguemriedem: where is that job definition16:36
mriedemecho 'export LIVE_MIGRATION_AVAILABLE=True'16:36
*** sudswas__ has joined #openstack-nova16:37
mriedemsdague: https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/devstack-gate.yaml#L167416:37
sdagueis that coming from project-config or d-g?16:37
mriedemecho 'export LIVE_MIGRATION_AVAILABLE=True' is from d-16:37
mriedem*d-g16:37
mriedemsdague: https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L54216:37
*** baoli has joined #openstack-nova16:38
mriedemsdague: the weird thing i noticed was,16:38
mriedemlooking at the local.conf http://logs.openstack.org/61/417961/40/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/8c363cd/logs/new/local.conf.txt.gz16:38
mriedemNOVA_ALLOW_MOVE_TO_SAME_HOST=False is in there16:38
mriedemfrom https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate.sh#L54116:38
*** chyka has joined #openstack-nova16:38
mriedembut export LIVE_MIGRATION_AVAILABLE=True isn't16:38
mriedemso i'm wondering if the export is screwing it up16:38
*** sudipto has quit IRC16:39
*** sudipto_ has quit IRC16:39
sdagueyeh, it's litterally using the export bit16:40
*** _ducttape_ has quit IRC16:40
*** tbachman has joined #openstack-nova16:41
sdaguemriedem: you want to do a depends on https://review.openstack.org/427282 and see if that addresses things?16:41
*** ducttape_ has joined #openstack-nova16:41
*** sudipto has joined #openstack-nova16:41
mriedemsure16:42
sdagueI took a strategy of doing a merge for valid looking lines, so you don't just have clobber clobber clobber going on, but that means understanding the kind of lines that can be brought over16:42
mdboothlyarwood: Done.16:43
*** sree has quit IRC16:43
*** sree has joined #openstack-nova16:44
sdaguemriedem: yeh - http://logs.openstack.org/61/417961/40/check/gate-grenade-dsvm-neutron-multinode-live-migration-nv/8c363cd/logs/grenade.sh.txt.gz#_2017-01-27_23_16_19_63716:44
*** _ducttape_ has joined #openstack-nova16:44
openstackgerritMatt Riedemann proposed openstack/nova: DNM: testing grenade multinode + live migration fix  https://review.openstack.org/42728516:45
*** artom_ has joined #openstack-nova16:45
*** artom has quit IRC16:45
mriedemskipped LIBS_FROM_GIT since there was no valid right?16:45
lyarwoodmdbooth: thanks, I'll sort that out now16:45
*** artom_ is now known as artom16:46
*** unicell has joined #openstack-nova16:47
*** ducttape_ has quit IRC16:47
*** bvanhav_ has joined #openstack-nova16:47
*** bvanhav has quit IRC16:48
sdaguemriedem: yeh16:48
sdaguebecause there was no value16:48
mriedemsdague: ok. pointed out OS_NO_CACHE in the review too16:48
mriedemotherwise lgtm16:48
*** sree has quit IRC16:48
mriedemgot the nova change up to test it16:49
sdaguemriedem: honestly, I'd leave the OS_NO_CACHE bit for now. I get it's probably also wrong, but it's not impacting anything right now.16:51
sdagueand, I'm not convinced it does anything useful anymore anyway16:51
mriedemyeah hmm, looks like that's all dead code http://git.openstack.org/cgit/openstack-dev/devstack/tree/openrc#n5616:52
mriedemlots of stuff setting it, but nothing using it http://codesearch.openstack.org/?q=OS_NO_CACHE&i=nope&files=&repos=16:53
*** mdrabe has quit IRC16:54
*** liangy has joined #openstack-nova16:55
*** kaisers has joined #openstack-nova16:56
*** tesseract has quit IRC16:56
*** efoley has joined #openstack-nova16:59
*** mdrabe has joined #openstack-nova16:59
sdaguemriedem: yeh16:59
sdaguethere were definitely a few things I noticed in d-g that were completely dead config settings17:00
*** sudswas__ has quit IRC17:03
*** sudipto has quit IRC17:03
*** archies has joined #openstack-nova17:04
*** sudipto_ has joined #openstack-nova17:09
*** sudipto has joined #openstack-nova17:09
*** raunak has joined #openstack-nova17:09
*** andreas_s has quit IRC17:10
openstackgerritLee Yarwood proposed openstack/nova: libvirt: Limit destroying disks during cleanup to spawn  https://review.openstack.org/42726717:12
*** browne has joined #openstack-nova17:13
openstackgerritChris Dent proposed openstack/nova: WIP: Explore functional test for the resource tracker  https://review.openstack.org/42730917:14
*** mgould has joined #openstack-nova17:15
mgouldhi everyone17:15
mgouldI'm getting "Host is not mapped to any cell" errors in nova  boot on a clean devstack on a freshly-installed baremetal  machine17:15
mgouldI've previously used the same local.conf on a VM without issues17:15
openstackgerritMihail proposed openstack/nova: proposed fix for bug 1646896 - System hangs when using NFS storage backend with loopback mounts, Closes-Bug: #1646896  https://review.openstack.org/42204117:15
openstackbug 1646896 in OpenStack Compute (nova) "System hangs when using NFS storage backend with loopback mounts" [Low,Confirmed] https://launchpad.net/bugs/164689617:15
mgouldany idea what I've done wrong?17:15
cdentjaypipes, edleafe, johnthetubaguy, mriedem : that review ^ about functional test for resource provider is trying to the basis for one that tests ironic. at the moment is just does non ironic17:16
*** sgordon has quit IRC17:16
cdentI needed to start there to get any traction17:16
*** kaisers has quit IRC17:17
jaypipescdent: hmm, I was right in the middle of adding a functional test to the Ironic resource classes patch.17:17
cdentah, good, now we'll have more17:17
cdent;)17:17
cdentthat one is very bland17:17
*** tbachman has quit IRC17:18
cdentjaypipes: perhaps we can compare and contrast later. I'm going to fix a couple bugs I found while making that.17:18
*** unicell has quit IRC17:21
jaypipescdent: k17:22
cdentif my version is for naught, was an excellent learning exercise17:22
*** tbachman has joined #openstack-nova17:23
*** unicell has joined #openstack-nova17:23
*** Swami has joined #openstack-nova17:25
*** Apoorva has joined #openstack-nova17:27
openstackgerritDiana Clarke proposed openstack/nova: Fix map_cell_and_hosts help  https://review.openstack.org/42217117:27
openstackgerritChris Dent proposed openstack/nova: Fresh resource provider in RT must have generation 0  https://review.openstack.org/42731917:28
*** crushil has quit IRC17:31
*** mgould has left #openstack-nova17:33
*** baoli has quit IRC17:35
*** efoley_ has joined #openstack-nova17:36
*** jdurgin1 has quit IRC17:36
*** efoley_ has quit IRC17:38
*** efoley_ has joined #openstack-nova17:38
*** efoley has quit IRC17:39
*** efoley__ has quit IRC17:39
openstackgerritChris Dent proposed openstack/nova: [placement] Use modern attributes of oslo_context  https://review.openstack.org/42733017:40
cdentmriedem: both of those ^ might be rc oriented17:41
mriedemi +2ed the first one17:41
mriedemhttps://review.openstack.org/#/c/427319/ isn't a regression in ocata though right? it was just always that way17:42
*** _ducttape_ has quit IRC17:42
*** sudipto_ has quit IRC17:43
*** sudipto has quit IRC17:43
*** liangy has quit IRC17:43
*** mdrabe has quit IRC17:46
*** efoley_ has quit IRC17:46
cdentmriedem: not a regression, no, but sending out something with noise on it seems untidy17:47
mriedemsure, but we're also 2 days from rc117:50
*** baoli has joined #openstack-nova17:50
mriedemso i can live with untidy17:50
mriedemsince we can backport fixes17:50
mriedemanyway, i'm +2 on it since it's a simple fix17:50
*** mdrabe has joined #openstack-nova17:50
cdentmriedem: that's cool. All this release schedule stuff is voodoo to me. I just smile and nod.17:50
dansmithjroll: sdague: so, we really need to be running discover_hosts after compute nodes are in the database, which for ironic happens pretty late compared to other virt drivers17:50
*** slaweq has quit IRC17:50
dansmithjroll: sdague: should I try to wedge in a discover_hosts call into the late stages of ironic devstack stuff?17:51
*** karimb has quit IRC17:51
*** gouthamr has quit IRC17:51
dansmithjroll: sdague: or maybe it's futile if people don't provide ironic nodes at the time of a devstack run? I'm not actually sure how that works17:52
*** jose-phillips has quit IRC17:52
*** unicell has quit IRC17:52
*** slaweq has joined #openstack-nova17:52
*** annegentle has quit IRC17:52
*** gatuus has quit IRC17:53
*** mvk has quit IRC17:54
*** mgould has joined #openstack-nova17:54
*** catintheroof has joined #openstack-nova17:55
mriedemcdent: it's pretty simple, just don't risk putting anything in right before rc1 unless it's super low risk, or super critical to get in17:55
* cdent smiles17:56
* cdent nods17:56
mriedemsince you don't want to release a major version with some kind of major bug which prevents upgrades or something17:56
mriedemha17:56
cdentI understand the mechanics.17:56
mriedemhaving said all that, i'd bet most deployments wait much longer to pick up anything,17:56
cdentthat17:56
mriedemso when they upgrade, they're going to pick up like the 3rd stable branch release anyway17:56
*** jamielennox is now known as jamielennox|away17:58
*** crushil has joined #openstack-nova18:00
* cdent has some dinner to fuel later debugging18:00
mriedemcdent: before you dine,18:01
cdentmriedem: yessir?18:01
mriedemdid either you or bauzas put anything up for the placement upgrade stuff in the devref? if not, i can start that18:01
mriedemit will be pretty simple18:01
mriedemjust upgrade from newton notes18:01
cdentI did not18:01
mriedembauzas: ?18:01
mriedemok i'll get started on that one18:01
cdentthanks18:02
*** lpetrut has quit IRC18:02
*** dimtruck is now known as zz_dimtruck18:04
*** romainh1 has joined #openstack-nova18:06
*** romainh has quit IRC18:06
*** gouthamr has joined #openstack-nova18:07
dansmithsdague: question for you here: https://review.openstack.org/#/c/42631018:10
sdaguedansmith: ok, question away18:12
dansmithsdague: it's in the review18:12
dansmithabout verify_noapi18:12
*** romainh1 has quit IRC18:12
sdaguehow can this run when the api is down?18:14
dansmithsdague: there are other ways to solve this, including some sort of post phase for the plugin, where we've already run the verify, if you'd prefer18:14
dansmithsdague: it's nova-manage18:14
sdagueit all depends on 'nova service-list'18:14
dansmithoh, did I do it with service-list? I did..18:14
dansmithI don't actually care that the api is down, of course, although I guess that means we won't have the primary node compute up either18:15
sdagueright, that too18:15
*** browne has quit IRC18:15
dansmithso, then, yeah, I need to run discover_hosts again after we've done the resource verify18:15
sdagueso, given the number of other things in flight, does this catch a failure that would have passed otherwise? Or is this just catching a failure earlier?18:17
dansmithsdague: so, according to the plugin docs, I could run discover from resource verify post-upgrade I guess18:17
sdaguedansmith: yeh, post-upgrade seems like a put to run the discover_hosts again if you need it18:17
dansmithsdague: this is a new thing we've introduced, and that log link proves that we're actually hitting it18:17
sdagueok, but the fix for that isn't this, right?18:18
sdaguethe fix for that is another discover_hosts call18:18
*** ijw has joined #openstack-nova18:18
dansmithsdague: no, this is the fix, otherwise the new discover_hosts call is still a race18:18
dansmithsdague: if both computes aren't up, the discover_hosts won't do anything18:18
dansmithsdague: am I missing why you don't like this service check?18:19
*** lucasagomes is now known as lucas-afk18:20
*** bvanhav__ has joined #openstack-nova18:22
sdaguedansmith: so where is the discover_hosts getting run?18:23
*** bvanhav_ has quit IRC18:23
dansmithsdague: right now it's only run in simple_cell_setup18:23
openstackgerritOpenStack Release Bot proposed openstack/os-vif: Update reno for stable/ocata  https://review.openstack.org/42734118:24
sdaguedansmith: and where does that get called during this flow that is failing?18:24
openstackgerritOpenStack Proposal Bot proposed openstack/os-vif: Updated from global requirements  https://review.openstack.org/39534718:25
dansmithsdague: I'm not sure what you mean.. it's not failing, it's just not seeing the computenode record for the subnode when it runs in simple_cell_setup, which means later we end up without the hostmapping, and thus the failed test in that log I linked18:25
openstackgerritOpenStack Release Bot proposed openstack/python-novaclient: Update reno for stable/ocata  https://review.openstack.org/42734418:25
sdaguedansmith: right... but the issue is that's a failure here - https://github.com/openstack-dev/grenade/blob/a1ea5cec068d59b9c2caa1c82d7ee2584d63aa08/grenade.sh#L27018:26
*** jlk has quit IRC18:26
sdaguebut all this code is executed here - https://github.com/openstack-dev/grenade/blob/a1ea5cec068d59b9c2caa1c82d7ee2584d63aa08/grenade.sh#L283-L28718:26
sdagueand I don't understand how code that runs after the failure in question is going to fix the issue18:27
*** unicell has joined #openstack-nova18:27
dansmithsdague: you mean it's on the old side?18:27
*** archies has quit IRC18:27
sdagueyes18:27
sdaguehttp://logs.openstack.org/79/396679/45/check/gate-grenade-dsvm-neutron-multinode-ubuntu-xenial/11607f2/logs/grenade.sh.txt.gz#_2017-01-30_09_49_58_10718:28
dansmithsdague: okay, I'm not sure how it's hitting that on the old side, since the thing that is failing merged on friday18:28
*** dharinic is now known as dharinic|lunch18:28
*** mvk has joined #openstack-nova18:28
sdaguedansmith: well that's where this failure is18:28
*** ducttape_ has joined #openstack-nova18:28
dansmithsdague: okay, so, I guess we'll continue to hold onto this fix for a condition that we know we've added18:28
*** archies has joined #openstack-nova18:28
dansmithsdague: if you could tell me why you have a problem with it, it'd sure save us time when this becomes a critical problem18:29
*** annegentle has joined #openstack-nova18:29
dansmith'cause I still am confused about that part18:29
sdaguedansmith: because I don't see what it's making better. This was framed with the logs in question as fixing that issue, but they don't, because that's a different and real issue about initial node racing on the way up18:30
*** mjura has quit IRC18:30
artomIs http://logs.openstack.org/59/424759/12/gate/gate-tempest-dsvm-cells-ubuntu-xenial/d7b1311/console.html#_2017-01-31_17_48_34_663215 something to do with ongoing cells work?18:30
dansmithsdague: you agree that race conditions exist that we know about, but just haven't hit yet right?18:30
*** vsaienko has joined #openstack-nova18:31
*** krtaylor has quit IRC18:32
sdaguedansmith: there are bunches of those though, and I'm fine fixing issues we're hitting. It just seems like this keeps getting pushed for issues we aren't yet hitting. And that are unlikely to hit based on the real timelines for additional stuff that happens before the second tempest run happens18:32
*** ducttape_ has quit IRC18:33
*** ralonsoh has quit IRC18:33
dansmithsdague: right, so paraphrasing, what you just said is: "This is a race we know about, but right now we think we'll never hit it, so we'll just wait until the context has fallen out of our brains and it becomes a gate blocker in six months"18:33
dansmithover ten lines of bash18:34
dansmithsdague: and fwiw, the reason they're hitting that (just found out) is that their patch set base to master18:34
dansmithhitting that in that way I mean18:34
*** lpetrut has joined #openstack-nova18:35
*** karimb has joined #openstack-nova18:38
*** prateek has quit IRC18:39
*** karimb has quit IRC18:40
melwittartom: not that I know of18:42
*** archies has quit IRC18:42
*** ducttape_ has joined #openstack-nova18:43
artommelwitt, hrmm, thanks :)18:43
* artom shall try a recheck18:43
*** xinli has quit IRC18:45
melwittartom: we recently merged a patch (on Jan 27) that might be related to this, so I'll do some digging through the logs to see what could be happening18:47
artommelwitt, 4 days ago? We'd have noticed it by now if it was causing consistent failure in the gate...18:48
mriedemi've seen that failure18:48
mriedemit's intermittent18:48
*** owalsh is now known as owalsh-brb18:49
*** ducttape_ has quit IRC18:49
* artom couldn't find any glaring traceback in the logs...18:49
*** ekhugen| has quit IRC18:50
melwittthis is the test that uses reset-state to set servers to error, then lists them. there must be some race here18:50
*** zz_dimtruck is now known as dimtruck18:51
*** ekhugen- has quit IRC18:51
*** liangy has joined #openstack-nova18:51
*** egarbade| has quit IRC18:51
*** jdurgin has quit IRC18:51
*** patriciadomin has quit IRC18:51
jaypipescdent: any ideas on this? (I've updated my tox venv to the latest wsgi-intercept release): http://paste.openstack.org/show/597078/18:53
*** nic has joined #openstack-nova18:53
* cdent looks18:53
*** owalsh-brb has quit IRC18:53
jaypipescdent: lol, nm18:53
*** avolkov has quit IRC18:54
cdentjaypipes: what was it?18:54
jaypipescdent: I'd forgotten to do:18:54
*** browne has joined #openstack-nova18:54
jaypipesself.rt.report_client = self.report_client18:54
jaypipeswhere self.report_client is the test_report_client.NoAuthKeystoneClient thing18:54
cdentah18:55
jaypipesboom.18:55
jaypipesperfecto.18:55
mriedemartom: melwitt: http://tinyurl.com/hkpuoel18:55
cdentjaypipes: cool, I was worried urllib3 had gone off and changed something (as they do) and made wsgi-intercept much more complicated (it's already a hairball)18:55
jrolldansmith: did you get your question answered? typical devstack registers nodes with ironic after starting ironic, and then we (re)start n-cpu. so yeah, thinking we should wedge it in here-ish https://github.com/openstack/ironic/blob/master/devstack/lib/ironic#L155618:55
jaypipescdent: nah18:55
artommriedem, hrmm, does keep popping up, doesn't it?18:56
artomAnd it predate's the change melwitt was talking about, since there are some on the 24th...18:57
*** baoli has quit IRC18:58
melwittyeah, looks like this has been happening for awhile, just not often18:58
*** Guest52040 has quit IRC18:58
*** owalsh-brb has joined #openstack-nova18:59
mriedemi'll open a bug19:00
*** ducttape_ has joined #openstack-nova19:00
openstackgerritPushkar Umaranikar proposed openstack/nova: Adopts keystoneauth with glance client.  https://review.openstack.org/41263419:01
*** ducttape_ has joined #openstack-nova19:01
*** tlian has quit IRC19:02
*** _ducttape_ has joined #openstack-nova19:02
*** patriciadomin has joined #openstack-nova19:03
artommriedem, melwitt, https://github.com/openstack/tempest/blob/master/tempest/api/compute/admin/test_servers.py#L5019:03
*** ekhugen has joined #openstack-nova19:03
artomThat seems... obvious? Surely you'd want to wait for the server to enter error state at that point...19:03
*** ekhugen_alt has joined #openstack-nova19:03
*** egarbade- has joined #openstack-nova19:03
*** baoli has joined #openstack-nova19:03
*** mgould is now known as mgould|afk19:04
*** mgould|afk has left #openstack-nova19:04
*** ducttape_ has quit IRC19:05
mriedemartom: well it could,19:06
artomI checked the api-ref, it returns 202, not 20019:06
mriedemit fails in cells v1, i assume, because the child cell is reporting the vm_state change back up to the parent api cell so the instances in both dbs are in sync, and when tempest makes the GET request, the api cell doesn't have the updated vm_state yet19:06
artomSo it's async and might not be done by the time the next requests lists the servers in error state...19:07
mriedemit's actually odd that it's a 20219:08
mriedembecause all that api does is literally change the vm_state in the db https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/admin_actions.py#L8219:08
*** tlian has joined #openstack-nova19:08
*** foexle has joined #openstack-nova19:10
mriedemthe upcall to the api cell is an rpc cast,19:10
mriedemi wonder if that's why it's a 202, for cells v119:10
*** jdurgin has joined #openstack-nova19:10
*** dharinic|lunch is now known as dharinic19:11
*** bvanhav__ has quit IRC19:12
artomWhere does the cells call happen in that?19:13
mriedemin nova.objects.Instance.save19:13
*** vsaienko has quit IRC19:14
*** _ducttape_ has quit IRC19:14
melwittI would think that would save it at the top and should be returned as 'error' as expected by the test. the save() would sync the vm_state change to the child. the get_all in compute/api is reading from child cells, so that's probably how it's happening19:14
*** krtaylor has joined #openstack-nova19:14
*** gszasz has quit IRC19:15
melwittwe've put in some conditionals "if cells v1, read from the top" but it looks like this was missed, for the get_all19:15
artomBah, vim ctags has failed me19:15
artomIt took me to agents.py save method for some reason19:15
mriedemhere is the bug btw https://bugs.launchpad.net/nova/+bug/166074719:16
openstackLaunchpad bug 1660747 in OpenStack Compute (nova) "test_list_servers_filter_by_error_status intermittently fails with MismatchError on no servers in response" [Medium,Confirmed]19:16
melwittthanks19:16
openstackgerritLee Yarwood proposed openstack/nova: WIP - libvirt: Remove redundant bdm serial mangling during swap_volume  https://review.openstack.org/42736419:16
*** enki_anunnaki has joined #openstack-nova19:16
mriedemsfinucan: do you know the ReST magic to find the root of a doc tree?19:17
mriedemfor a link19:17
*** mlavalle has quit IRC19:17
*** kaisers has joined #openstack-nova19:17
*** jaosorior has quit IRC19:18
artommelwitt, you wanna propose a fix to Tempest? Shall I?19:18
*** jaosorior has joined #openstack-nova19:18
dansmithmelwitt: I definitely had that in one of my patches at some point19:18
melwittartom: I'll propose a fix to nova, it's the get_all that needs a tweak I think19:19
dansmithmelwitt: for get_all, but I don't see it right now, so might've been the victim of a rebase, although I was failing the cells job regularly without it19:19
artommelwitt, aha!19:19
melwittdansmith: yeah, I think it must have gotten lost in the mix19:19
artomI still think Tempest should wait after a request that returns a 202, not just assume it's been completed...19:19
* artom shall propose that19:20
openstackgerritEric Fried proposed openstack/nova: Introduce base PowerVM driver  https://review.openstack.org/39128819:20
openstackgerritSarafraj Singh proposed openstack/nova-specs: Send service token along with user tokens  https://review.openstack.org/42693319:20
mriedemmelwitt: technically, we don't have to care about fixing latent bugs in cells v119:20
dansmithmriedem: this isn't latent19:21
melwittmriedem: it's not a latent bug in cells v1. it should have been working fine until we added cells v2 instance listing from all cells19:21
*** kaisers has quit IRC19:21
openstackgerritMatt Riedemann proposed openstack/nova: doc: add upgrade notes to the placement devref  https://review.openstack.org/42736719:22
mriedemcdent: bauzas: ^19:23
mriedemdansmith: melwitt: ok19:23
brownemriedem: do the RPC versions need to be updated for Ocata?  seemed so, so i put up this patch: https://review.openstack.org/#/c/425958/19:24
lyarwooddansmith: https://review.openstack.org/#/c/427267 should be ready for a review if you have time later today19:24
melwittartom: I think we've left the test alone since it helps catch our bugs, but technically you're right, a 202 means wait19:25
artommelwitt, well, is the problem that the server state update takes too long, or doesn't happen at all? Because the latter isn't affected by waiting19:26
mriedemit would have to be a race,19:26
mriedemelse this would be 100% failure19:26
melwittartom: the problem is our code is reading instances from the child cells instead of the parent cell, for cells v119:26
artomIt'll still catch the bugs, except it'll do it by timing out on the wait_for_server_status call19:26
melwittmost of the time, the sync down gets to the child before our read does, and in that case, it will pass the test19:27
melwittno, it wouldn't catch this bug I don't think19:27
melwittthis is like a split second difference19:27
artomAh, so the status ends up being synced in all cases19:28
melwittyes19:28
mriedemlyarwood: i threw the rc-potential tag on that one since it's a regression19:28
lyarwoodmriedem: ack thanks19:29
artommelwitt, wait, just to make this clear in my head: the write happens on the parent cell, the read from the child cells, so if you time your read really quickly after the write, you won't get the new state?19:29
*** abalutoiu has quit IRC19:29
artomAnd the fix you have in mind is to read from the parent cell?19:29
melwittyes. cells v1 normally reads from the parent cell. it has instances table in the parent and the childs and it syncs them19:30
*** bvanhav__ has joined #openstack-nova19:30
*** owalsh-brb is now known as owalsh19:31
artomHrmpf, this is indeed a case of a test being technically wrong but actually useful19:31
*** pcaruana has quit IRC19:31
dansmithartom: no19:31
dansmithartom: the test case can't tell where it's being read from19:31
melwitthe means the 202 return code implies async so it shouldn't just assume the state change is available after the call returns19:32
artommelwitt, words out of my mouth :)19:32
melwittwe know that it should be, from the code. but the API doesn't tell that19:32
dansmithis it not polling?19:32
*** crushil has quit IRC19:32
*** raunak has quit IRC19:32
artomdansmith, nope19:33
melwittno, it's just immediately reading after19:33
dansmithah, okay19:33
dansmiththen yeah19:33
jaypipesmriedem, cdent: yes, I'm still working on functional tests for the Ironic patch. Didn't want you to think I forgot or am slacking.19:33
* cdent gives jaypipes a sandwich19:33
mriedemi'm pretty far removed from that patch by now,19:34
mriedemonce john manhandled it19:34
mriedemlike, before i start reviewing it again, someone is probably going to have to give me a recap of what happened19:34
mriedembrowne: probably...19:35
mriedemi feel like russellb used to have a cheat sheet for this19:36
*** ociuhandu has joined #openstack-nova19:36
openstackgerritMatt Riedemann proposed openstack/nova: doc: add upgrade notes to the placement devref  https://review.openstack.org/42736719:42
*** annegentle has quit IRC19:45
mordredmriedem: I've got this error coming up on nodepool patches: http://logs.openstack.org/06/425706/7/check/gate-dsvm-nodepool-src/7004c5b/logs/screen-nodepool.txt.gz#_2017-01-31_19_02_00_17719:45
mordredmriedem: which is reporting a 500 error from nova and <class 'sqlalchemy.exc.ProgrammingError'>  as the error19:45
mriedemnice http://logs.openstack.org/06/425706/7/check/gate-dsvm-nodepool-src/7004c5b/logs/screen-n-api.txt.gz?level=TRACE#_2017-01-31_19_02_00_16919:46
mordredyah19:46
mordredisn't that pretty? (I was just about to paste that link next :) )19:46
mriedemTable 'nova_cell0.instance_extra' doesn't exist19:46
mriedemmordred: you don't have your cells v2 stuff setup19:47
mriedemin whatever that thing is19:47
mordredwell, it's nodepool's functional tests that run on top of a devstack node19:47
mordreddoes devstack-gate need cellsv2 now but not set it up by default? :(19:47
mriedemdevstack sets up cells v219:47
*** armax has quit IRC19:48
mriedemhttps://github.com/openstack-dev/devstack/blob/master/lib/nova#L68419:48
mordredneat. well, apparently something I'm doing there is not letting it I guess - Ill look fruther19:48
mriedem^ creates the cell0 db19:48
mordredwow. so it should just do it without any flags it looks like19:48
mriedemmordred: and this stuff at the end https://github.com/openstack-dev/devstack/blob/master/lib/nova#L94719:48
mriedemmordred: yes, cells v2 is required now in ocata19:48
*** raunak has joined #openstack-nova19:48
mriedemall ocata nova deployments will be at least a cell of one19:49
mordredmriedem: is there a cell service I need to enable?19:49
mriedemno19:49
mriedemthat's cells v119:49
mriedemcells v2 is baked in19:49
*** raunak has quit IRC19:50
mriedemhere is the nova_cell0 db create in your run http://logs.openstack.org/06/425706/7/check/gate-dsvm-nodepool-src/7004c5b/logs/devstacklog.txt.gz#_2017-01-31_18_58_29_04519:50
mriedemmap_cell0 http://logs.openstack.org/06/425706/7/check/gate-dsvm-nodepool-src/7004c5b/logs/devstacklog.txt.gz#_2017-01-31_19_01_45_58819:50
mriedemand simple_cell_setup http://logs.openstack.org/06/425706/7/check/gate-dsvm-nodepool-src/7004c5b/logs/devstacklog.txt.gz#_2017-01-31_19_01_48_33519:50
mordredWEIRD19:50
mriedemthat's an appropriate reaction19:51
mriedemhttp://docs.openstack.org/developer/nova/cells.html19:51
mriedemso when your thing blew up in n-api, cell0 would be mapped19:52
mriedemso i'm not sure why it's not finding the table19:52
mriedem*db19:52
mriedemor, maybe it found the db but the schema wasn't migrated?19:52
mriedemsimple_cell_setup will migrate the db schema on the cell0 db19:53
mordreddoes it do the migration asynchronously?19:53
mriedemyou can see it doing that here http://logs.openstack.org/06/425706/7/check/gate-dsvm-nodepool-src/7004c5b/logs/devstacklog.txt.gz#_2017-01-31_19_02_00_12819:53
mriedemno19:53
*** jose-phillips has joined #openstack-nova19:53
diana_clarkeperhaps this? "WARNING: cell0 mapping not found - not syncing cell0."19:53
*** annegentle has joined #openstack-nova19:53
mriedemWarning: (1831, u"Duplicate index 'block_device_mapping_instance_uuid_virtual_name_device_name_idx' defined on the table 'nova_cell0.block_device_mapping'19:53
*** vsaienko has joined #openstack-nova19:53
diana_clarkehttp://logs.openstack.org/06/425706/7/check/gate-dsvm-nodepool-src/7004c5b/logs/devstacklog.txt.gz#_2017-01-31_18_58_31_44919:53
mriedemdiana_clarke: that's expected19:54
mriedemwe've created the dbs, but haven't mapped cell0 yet, that comeslater19:54
mriedem*comes later19:54
*** foexle has quit IRC19:54
mriedemhttps://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L63819:54
mriedemdb sync attempts to fan out by default19:54
mriedemi'm not sure that should be a WARNING?!?!??!19:55
mriedemhttps://review.openstack.org/#/c/420973/ fixes the fanout thing19:55
*** ociuhandu has quit IRC19:56
mriedemwe see this https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L110019:56
*** ducttape_ has joined #openstack-nova19:56
mriedemand then it migrates the schema for the cell0 db here https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L110719:56
diana_clarkemriedem: but only in the non-cells v1 case, it looks to me like cells v1 is enabled here. (ignore me though, I'm probably wrong)19:57
openstackgerritEric Fried proposed openstack/nova: PowerVM Driver: power_on/off and reboot  https://review.openstack.org/42738019:58
diana_clarke(b/c is_service_enabled n-cell returns 1, which I assume means true)19:58
mriedemdiana_clarke: in the cells v1 case, cell0 db schema would be migrated here https://github.com/openstack-dev/devstack/blob/master/lib/nova#L95619:58
mriedembut no, cells v1 isn't enabled here19:59
mordredmriedem: I just pushed this up: remote:   https://review.openstack.org/427381 Update nodepool dsvm test to disable services19:59
*** slaweq has quit IRC19:59
mordredmriedem: just in case something is wonk with us whitelisting a set of services19:59
*** slaweq has joined #openstack-nova19:59
mordredmriedem: I don't see any other real differences between job configs19:59
*** vsaienko has quit IRC19:59
mriedemmordred: there is no extra service for cells so it wouldn't be a problem - but you probably need ^ for placement-api anyway20:00
mriedemsince nova requires placement-api in ocata to start computes20:00
mordredshouldn't that default to on in devstack then?20:00
mriedemor it will once this merges https://review.openstack.org/#/c/425806/20:00
mriedemmordred: it does20:00
mordredok. cool20:00
*** vsaienko has joined #openstack-nova20:00
mordredbut I need to not whitelist things20:00
mordredso that I get the placement-api things20:00
mriedemhttps://github.com/openstack-dev/devstack/blob/master/stackrc#L5820:00
mriedemindeed20:01
mriedemwhich makes me think, i should probably sound the alarm in the dev list quick20:01
*** ducttape_ has quit IRC20:01
mriedembefore the hordes show up saying their big tent CIs are now all failing because nova-compute won't start20:01
openstackgerritEric Fried proposed openstack/nova: PowerVM Driver: power_on/off and reboot  https://review.openstack.org/42738020:02
mordred++20:02
*** Sukhdev has joined #openstack-nova20:03
mriedemdone20:03
*** vsaienko has quit IRC20:04
mriedemso when this blows up, we're getting here https://github.com/openstack/nova/blob/master/nova/compute/api.py#L244020:04
mriedemwhich means map_cell0 worked, we have the cell0 mapping20:04
mriedemand from what i could tell in the devstack logs, the db schema was migrated for nova_cell020:05
*** abalutoiu has joined #openstack-nova20:06
mriedemshould be right here http://logs.openstack.org/06/425706/7/check/gate-dsvm-nodepool-src/7004c5b/logs/devstacklog.txt.gz#_2017-01-31_19_01_50_74320:07
mriedem2017-01-31 19:01:50.743 | 2017-01-31 19:01:50.743 4887 DEBUG migrate.versioning.script.base [req-067d40dc-316e-42b5-82e8-feaec4129b2b - -] Loading script /opt/stack/new/nova/nova/db/sqlalchemy/migrate_repo/versions/252_add_instance_extra_table.py... __init__ /usr/local/lib/python2.7/dist-packages/migrate/versioning/script/base.py:2720:07
mriedemactually more like http://logs.openstack.org/06/425706/7/check/gate-dsvm-nodepool-src/7004c5b/logs/devstacklog.txt.gz#_2017-01-31_19_02_22_72620:08
mriedemmordred: which is 22 seconds after your failure20:08
mriedemfails here 2017-01-31_19_02_00_17720:09
mordredmriedem: oh. I mean ...20:09
mriedemnova_cell0 db schema is being migrated here 2017-01-31_19_02_22_72620:09
mordredmriedem: it seems that devstack is still setting itself up when nodepool is running :(20:09
mordredjeblair: ^^ you may (or may not) find this scrollback interesting20:10
mordredjeblair: tl;dr - it's possible openstack is not yet ready when nodepool tries to start talking to it20:11
mordredsdague, ianw: ^^ you two may or may not also find it interesting20:13
jeblairmordred: that's weirdly parallel with our zookeeper conversation... but anyway, wouldn't any devstack test (eg tempest) have this issue?20:13
mriedemso...20:13
mriedemjeblair: d-g runs tempest after stack.sh is done20:13
mriedemso no20:13
mriedemi think we could move that map_cell0 call way further up in the stack run though,20:14
sdaguemordred: is there a log somewhere?20:14
mriedemso that the cell0 schema is migrated way early when setting up the dbs, not at the tail end20:14
mriedemdansmith: melwitt: ^20:14
mordredmriedem: ++20:14
mordredjeblair: OOOOHHHHHH - we install nodepool as a devstack plugin - we do not install it during the post_test_hook20:14
mriedemi'll push something up to test that out20:14
*** vsaienko has joined #openstack-nova20:15
jeblairmordred: aha, of course.  :)20:15
mriedemmordred: it's always good when it's the users fault :)20:15
mordredmriedem: :)20:15
mordredmriedem: although I vote for moving the cells call way further up too20:16
mriedemyeah i can still tinker on that20:16
sdaguemordred: yeh, the post_test_hook should give you a better guaruntee of where you want to be20:16
sdagueyou only need to move your start there, right?20:16
mordredsdague: ++20:16
sdagueall the install / configure can happen earlier I would think20:16
sdaguethe post_test_hook is kind of recent, because ironic needed to ensure they could do a thing after tempest was installed and configured20:17
mordredwell, we're already using post_test_hook too20:18
sdaguemordred: oh, interesting20:18
mordredso I think we just need to consolidate a bit20:18
mtreinishsdague: I thought post_test_hook was the d-g for running things after testing is done?20:18
sdaguemtreinish: oh, sorry, there is a new devstack plugin phase20:18
sdagueone sec20:18
mtreinishI thought I called the plugin phase test-config or something like that20:19
sdagueyeh, that20:19
mordredI may need/want that - because we're using devstack functions like crazy20:19
mordredso I make a test_config_nodepool function?20:19
mordredto use that phase?20:19
sdaguemordred: yeh, test-config20:19
sdaguemordred: so change https://github.com/openstack-infra/nodepool/blob/12d3caf560aa827ed47f320259cdb66a20f1e24b/devstack/plugin.sh#L465-L46920:20
sdagues/extra/test-config/20:20
mordredcool. thank you! that'll be super helpful20:20
sdaguethat executes here in stack.sh - https://github.com/openstack-dev/devstack/blob/e6f8dbb55fc8fccb18f4ccb5ed5723a2efc3b025/stack.sh#L140120:20
*** gouthamr has quit IRC20:20
*** ducttape_ has joined #openstack-nova20:21
mriedemmordred: sdague: here is the devstack change https://review.openstack.org/42739220:22
mordredmriedem: I tell you what - we can get some code written aroud here20:23
mriedemjust not reviewed or merged :)20:23
mriedemhi-o!20:23
*** Guest15362 has quit IRC20:23
mordred:)20:24
mordredI've been trying to land a shade patch since last tuesday20:24
*** vsaienko has quit IRC20:24
mordredI'm going to get it landed today if I have to write and land a patch to every repo in openstack20:24
*** mtanino_ has quit IRC20:25
*** vsaienko has joined #openstack-nova20:26
openstackgerritmelanie witt proposed openstack/nova: Read instances from API cell for cells v1  https://review.openstack.org/42739420:27
mtreinishmordred: don't you have ninja merge powers...20:27
mordredmtreinish: don't tempt me20:28
*** bvanhav__ is now known as bvanhav20:28
melwittdansmith: ^ see if that's what you expect for the cells v1 get_all20:29
*** gouthamr has joined #openstack-nova20:30
*** sree has joined #openstack-nova20:30
openstackgerritEric Fried proposed openstack/nova: Introduce base PowerVM driver (disk drv)  https://review.openstack.org/40940120:30
*** armax has joined #openstack-nova20:30
*** sree has quit IRC20:35
*** vsaienko has quit IRC20:35
*** xinli has joined #openstack-nova20:36
*** Guest15362 has joined #openstack-nova20:36
*** bvanhav has quit IRC20:36
*** mlavalle has joined #openstack-nova20:37
*** vsaienko has joined #openstack-nova20:39
*** READ10 has quit IRC20:41
*** rfolco has quit IRC20:45
mriedemmelwitt: lgtm for the most part - would be nice to have a comment in there about the conditional20:46
mriedemlike we have in _get_instance20:46
*** dave-mccowan has quit IRC20:46
cdentwe collided a bit on that20:46
melwittmriedem: yeah, adding now20:46
melwittI saw cdent's comment20:46
mriedemi had to pull it down to make sure the else condition was being hit in tests20:46
mriedemsince the compute api unit tests are confusing with the cells v1 and non-cells v1 stuff20:47
*** xyang1 has joined #openstack-nova20:47
dansmithmelwitt: effectively, yeah20:49
openstackgerritmelanie witt proposed openstack/nova: Read instances from API cell for cells v1  https://review.openstack.org/42739420:51
*** satyar has quit IRC20:51
*** cdelatte has quit IRC20:51
mriedem+220:53
openstackgerritAnusha Unnam proposed openstack/nova: Add post hook script to run tempest with old n-api  https://review.openstack.org/40139520:53
openstackgerritEric Fried proposed openstack/nova: Introduce base PowerVM driver  https://review.openstack.org/39128820:54
*** _ducttape_ has joined #openstack-nova20:54
openstackgerritEric Fried proposed openstack/nova: PowerVM Driver: power_on/off and reboot  https://review.openstack.org/42738020:55
mriedemoomichi: you were +2 before https://review.openstack.org/#/c/422171/ want to get that one now?20:56
openstackgerritEric Fried proposed openstack/nova: Introduce base PowerVM driver (disk drv)  https://review.openstack.org/40940120:56
*** ducttape_ has quit IRC20:57
*** raj_singh_ has joined #openstack-nova21:02
openstackgerritMatt Riedemann proposed openstack/nova: doc: add some detail to the map_cell0 man page  https://review.openstack.org/42745021:02
mriedemdiana_clarke: ^ addresses something i noted in your map_cell0 man page change after it was approved21:02
openstackgerritEric Fried proposed openstack/nova: PowerVM Driver: spawn/destroy  https://review.openstack.org/39128821:04
*** kentb1 has joined #openstack-nova21:04
*** raj_singh_ has quit IRC21:05
*** edmondsw has quit IRC21:05
openstackgerritEric Fried proposed openstack/nova: PowerVM Driver: power_on/off and reboot  https://review.openstack.org/42738021:06
openstackgerritEric Fried proposed openstack/nova: Introduce base PowerVM driver (disk drv)  https://review.openstack.org/40940121:07
openstackgerritScott DAngelo proposed openstack/nova: WIP Support cinder v3 API  https://review.openstack.org/42020121:16
mriedembrowne: so looking at when the newton version aliases were made, that happened in ocata https://review.openstack.org/#/c/386625/21:17
mriedembrowne: which leads me to believe we should wait to do https://review.openstack.org/#/c/425958/ in pike21:18
mriedemdansmith: is that right? or does it not matter? ^21:18
brownemriedem: ok, so maybe it can wait21:19
mriedemooo i found the checklist https://wiki.openstack.org/wiki/Nova/ReleaseChecklist21:20
openstackgerritChris Dent proposed openstack/nova: Optional separate database for placement API  https://review.openstack.org/36276621:21
*** slaweq has quit IRC21:21
cdentomg, "scheduler calling the placement api" just merged.21:21
bauzasindeed21:22
*** mtanino has joined #openstack-nova21:22
mriedemi thought it was getting bumped again21:22
* cdent wipes brow21:22
bauzasback from the cave21:22
bauzasmriedem: you okay if I'm just providing a new revision for the reno branch ?21:22
*** jaosorior has quit IRC21:23
mriedembauzas: yes21:23
bauzask21:23
dansmithmriedem: we need to add the aliases once know it's never going to change for a release, which usually means "after the release"21:23
browneah ok21:23
*** slaweq has joined #openstack-nova21:23
cdentmriedem: I'm done for the day, but can I assume that https://etherpad.openstack.org/p/nova-ocata-rc1-todos is still the place to go tomorrow morning for things, like, to do?21:25
mriedemcdent: yup21:25
mriedemgood progress today21:25
cdentcool21:25
cdentgoodnight everybody21:25
*** cdent has quit IRC21:26
mriedemdansmith: https://wiki.openstack.org/wiki/Nova/ReleaseChecklist#Pre-release_Checklist is a bit confusing since it says "Ideally add the RPC version alias just before opening the next release"21:28
mriedemi guess ideally is, well, ideal21:28
mriedembut optional21:28
kencjohnstonWhere would I find the name of Nova's default read-only role, if there is one?21:28
*** kfarr has joined #openstack-nova21:28
dansmithmriedem: right21:28
*** crushil has joined #openstack-nova21:32
*** vsaienko has quit IRC21:32
openstackgerritMatt Riedemann proposed openstack/nova: doc: refer to the cell_v2 man pages from the cells v2 doc  https://review.openstack.org/42745921:33
openstackgerritMatt Riedemann proposed openstack/nova: doc: update the man page entry for nova-manage db sync  https://review.openstack.org/42746021:33
*** Jeffrey4l_ has joined #openstack-nova21:34
*** READ10 has joined #openstack-nova21:35
*** Jeffrey4l has quit IRC21:35
*** catintheroof has quit IRC21:38
*** catintheroof has joined #openstack-nova21:38
*** catintheroof has quit IRC21:39
*** tblakes has quit IRC21:39
*** Jack_I has quit IRC21:39
*** rushil has joined #openstack-nova21:40
*** baoli has quit IRC21:43
*** cleong has quit IRC21:44
*** annegentle has quit IRC21:44
*** _ducttape_ has quit IRC21:46
*** ducttape_ has joined #openstack-nova21:46
*** rushil has quit IRC21:46
*** vsaienko has joined #openstack-nova21:46
openstackgerritSylvain Bauza proposed openstack/nova: Collected release notes for Ocata CellsV2  https://review.openstack.org/42473721:51
openstackgerritSylvain Bauza proposed openstack/nova: Add release note for nova-status upgrade check CLI  https://review.openstack.org/42691121:51
openstackgerritSylvain Bauza proposed openstack/nova: Add prelude section for Ocata  https://review.openstack.org/42686421:51
*** tblakes has joined #openstack-nova21:52
*** baoli has joined #openstack-nova21:54
*** jaosorior has joined #openstack-nova21:55
*** marst_ has quit IRC21:58
*** thorst_ has quit IRC22:00
*** liangy has quit IRC22:00
*** baoli has quit IRC22:01
*** edmondsw has joined #openstack-nova22:02
*** crushil has quit IRC22:04
*** catintheroof has joined #openstack-nova22:05
*** catintheroof has quit IRC22:05
*** catintheroof has joined #openstack-nova22:06
*** krtaylor has quit IRC22:07
*** catintheroof has quit IRC22:11
*** edmondsw has quit IRC22:13
*** ducttape_ has quit IRC22:14
*** jamielennox|away is now known as jamielennox22:15
*** kentb1 has quit IRC22:15
sfinucanmriedem: Bit late, but ':doc:/<path>' is relative to root22:15
sfinucanre: mriedem> sfinucan: do you know the ReST magic to find the root of a doc tree? for a link22:15
mriedemsfinucan: not too late yet22:16
mriedemthanks22:16
*** jeblair has quit IRC22:16
*** kaisers has joined #openstack-nova22:17
*** edmondsw has joined #openstack-nova22:17
openstackgerritEric Fried proposed openstack/nova: PowerVM Driver: localdisk  https://review.openstack.org/40940122:19
*** johndperkins has joined #openstack-nova22:19
*** thorst_ has joined #openstack-nova22:20
*** jheroux has quit IRC22:21
*** kaisers has quit IRC22:22
*** jamesdenton has quit IRC22:22
*** jeblair has joined #openstack-nova22:22
*** tblakes has quit IRC22:24
*** thorst_ has quit IRC22:25
*** baoli has joined #openstack-nova22:25
openstackgerritEric Fried proposed openstack/nova: PowerVM Driver: console  https://review.openstack.org/40940222:26
openstackgerritEric Fried proposed openstack/nova: Introduce base PowerVM driver (cfg drv)  https://review.openstack.org/40940422:26
*** edmondsw has quit IRC22:28
*** edmondsw has joined #openstack-nova22:28
*** edmondsw has quit IRC22:29
mriedemso another thought occurs to me,22:29
*** edmondsw has joined #openstack-nova22:29
*** READ10 has quit IRC22:29
mriedemnova-status upgrade check is checking for resource providers and compute nodes and fails if there are compute nodes but not RPs,22:29
mriedemi'm fixing that due to the fallback change in the filter scheduler,22:30
mriedembut i was just thinking, none of this matters unless you're using the filter scheduler - since the caching scheduler doesn't use placement at all in ocata, RPs or not22:30
*** ducttape_ has joined #openstack-nova22:31
*** kfarr has quit IRC22:33
openstackgerritSylvain Bauza proposed openstack/nova: Add a status check for enabled filters  https://review.openstack.org/42720022:34
*** edmondsw has quit IRC22:34
*** annegentle has joined #openstack-nova22:34
*** kylek3h has quit IRC22:35
*** lpetrut has quit IRC22:39
*** ansiwen has quit IRC22:40
*** ansiwen has joined #openstack-nova22:40
*** eharney has quit IRC22:42
*** amotoki has quit IRC22:44
*** thorst_ has joined #openstack-nova22:45
*** dimtruck is now known as zz_dimtruck22:49
*** burt has quit IRC22:52
*** krtaylor has joined #openstack-nova22:53
*** markvoelker_ has joined #openstack-nova22:58
*** markvoelker has quit IRC22:58
*** tlian2 has joined #openstack-nova22:59
*** _ducttape_ has joined #openstack-nova23:03
*** tlian has quit IRC23:03
*** xyang1 has quit IRC23:03
*** tlian2 has quit IRC23:04
*** mdrabe has quit IRC23:04
openstackgerritMatt Riedemann proposed openstack/nova: nova-status: relax the resource providers check  https://review.openstack.org/42749923:05
*** panda has joined #openstack-nova23:05
*** ducttape_ has quit IRC23:06
*** xinli has quit IRC23:06
pandadansmith: hi, did someone already ping you about https://bugs.launchpad.net/python-novaclient/+bug/1654183 today ?23:07
openstackLaunchpad bug 1654183 in tripleo "Token based authentication in Client class does not work" [High,Triaged]23:07
mriedemhopefully ^ gets nova-status working in grenade now https://review.openstack.org/#/c/426926/23:08
mriedempanda: that's fixed in novaclient 7.1.023:09
mriedemandreykurilin: ^23:09
pandamriedem: even using 7.1.0 we're still having problems when retrieving nova keypair using token authentication23:10
andreykurilinpanda: what project we are talking about?23:10
andreykurilinhi23:10
mriedemandreykurilin: tripleo23:10
mriedemandreykurilin: http://logs.openstack.org/31/426231/3/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/6015c34/logs/undercloud/var/log/undercloud_install.txt.gz#_2017-01-31_18_25_17_00023:10
*** Guest15362 has quit IRC23:11
andreykurilinlet me check23:11
mriedemhttp://logs.openstack.org/31/426231/3/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/6015c34/logs/undercloud/var/log/keystone/keystone.txt.gz#_2017-01-31_18_25_17_33923:11
mriedem2017-01-31 18:25:17.339 32100 DEBUG keystone.middleware.auth [req-74362ac9-6544-4b45-83f8-e8dc56b1d7e2 - - - - -] There is either no auth token in the request or the certificate issuer is not trusted. No auth context will be set. fill_context /usr/lib/python2.7/site-packages/keystone/middleware/auth.py:18823:12
andreykurilinok, it looks like I know the reason23:12
andreykurilinof failure23:12
*** trevormc has joined #openstack-nova23:13
*** sdague has quit IRC23:13
andreykurilinit is about that commit - https://github.com/openstack/python-novaclient/commit/9bbe5a87b7df62a7962debba5db7c96555da676123:13
pandaandreykurilin: anything we should check on our side ?23:13
andreykurilinpanda: fix should be simple23:14
andreykurilinjust +1;-123:14
andreykurilinthe problem is that projectid argument of novaclient had different meaning in different cases23:14
andreykurilinwe fix that23:15
andreykurilinand now projectid means Project ID in case of keystone termas23:15
mriedemandreykurilin: where do you see that from the logs?23:15
andreykurilinthere is nothing in logs:)23:15
mriedemheh23:15
andreykurilinI checked the code23:16
mriedemso it's your gut23:16
andreykurilinhttps://github.com/openstack/instack-undercloud/blob/master/instack_undercloud/undercloud.py#L1239-L124023:16
panda+1;-1 ok, and they said that math wasn't useful at school.23:16
andreykurilinhttps://github.com/openstack/instack-undercloud/blob/master/instack_undercloud/undercloud.py#L109923:16
andreykurilintenant is a tenant_name23:16
andreykurilinbut we expects tenant name23:17
andreykurilin*but we expect tenand id23:17
andreykurilin*tenant23:17
pandaso it must be fixed in instack-undercloud ?23:17
andreykurilinyes23:17
mriedemandreykurilin: will that also work with novaclient 6.0?23:18
mriedembecause 6.0 is the minimum in g-r right now23:18
openstackgerritJohn Griffith proposed openstack/nova: Implement new attach/detach Cinder flow  https://review.openstack.org/33028523:18
openstackgerritJohn Griffith proposed openstack/nova: Remove check_attach  https://review.openstack.org/33535823:18
pandaandreykurilin: ah, interesting23:20
andreykurilinmriedem: I think no, if we just transmit it `via project_name=tenant`. But I can assume that if we obtain OS_TENANT_ID at https://github.com/openstack/instack-undercloud/blob/master/instack_undercloud/undercloud.py#L1099 and use tenant_id keyword argument while novaclient initialization it should work with 6.0 and 7.123:21
*** baoli has quit IRC23:21
mriedemok23:21
mriedemwell, i've got to run, but thanks for the quick investigation23:21
mriedemi posted those notes into the bug report23:21
andreykurilinpanda: are you ok with that change^23:22
*** tlian has joined #openstack-nova23:22
andreykurilinI can quickly propose a patch23:22
andreykurilinmriedem: I'm glad I could help23:23
pandaandreykurilin: fine with me23:24
*** mriedem has quit IRC23:25
pandabnemec: ^23:27
openstackgerritEric Fried proposed openstack/nova: PowerVM Driver: spawn/destroy  https://review.openstack.org/39128823:28
openstackgerritEric Fried proposed openstack/nova: PowerVM Driver: power_on/off and reboot  https://review.openstack.org/42738023:28
openstackgerritEric Fried proposed openstack/nova: PowerVM Driver: localdisk  https://review.openstack.org/40940123:28
bnemecUmm, we don't generate OS_TENANT_ID, so it's not quite that simple.23:28
*** _ducttape_ has quit IRC23:29
*** ducttape_ has joined #openstack-nova23:30
openstackgerritEric Fried proposed openstack/nova: PowerVM Driver: console  https://review.openstack.org/40940223:30
*** trevormc has quit IRC23:31
jaypipes#success \0/ Finally got a good functional test created that stresses the Ironic and Nova integration and migration from Newton to Ocata.23:35
openstackstatusjaypipes: Added success to Success page23:35
*** ducttape_ has quit IRC23:35
*** gouthamr has quit IRC23:35
openstackgerritEric Fried proposed openstack/nova: PowerVM Driver: config drive  https://review.openstack.org/40940423:41
*** dave-mccowan has joined #openstack-nova23:43
*** takashin has joined #openstack-nova23:44
andreykurilinbnemec: hm23:45
andreykurilinbnemec: is it difficult to add that?23:46
*** tlian has quit IRC23:47
bnemecandreykurilin: I'm not sure how we would.  We'd need to look up the id in nova, which we can't do until we can create a client.23:47
bnemecandreykurilin: Can we just switch to using a project name or something?23:48
andreykurilinbnemec: yes, but it will not be compatible with min version in g-r. We can use simple check of novaclient version to support all versions. Are you ok about that solution?23:49
*** tlian has joined #openstack-nova23:50
openstackgerritScott DAngelo proposed openstack/nova: WIP Support cinder v3 API  https://review.openstack.org/42020123:50
bnemecandreykurilin: I'm not sure we're as concerned about the minimum.  Our repos are generated in such a way that I think if we find any way that works with the new novaclient we can start using that.23:50
andreykurilinoh23:50
andreykurilincool23:50
andreykurilinso it is just +2;-2 change (with tests)23:51
bnemecAnyone on the old novaclient would be using the old instack-undercloud, and when they update to the new novaclient they should also get the new instack-undercloud.23:51
*** crushil has joined #openstack-nova23:54
andreykurilinbnemec panda: https://review.openstack.org/42751523:55
andreykurilinit should fix an issue23:56
bnemecandreykurilin: Cool, thanks.23:56
pandanot until we put OS_TENANT_ID in stackrc ?23:57
andreykurilinpanda: that solution should work with OS_TENANT_NAME and with latest novaclient without changing stackrc23:57
pandaandreykurilin: nice, thanks23:59

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!