Thursday, 2016-05-12

*** jgriffith_ has joined #openstack-meeting-cp00:11
*** jgriffith has quit IRC00:19
*** lifeless has quit IRC00:46
*** lifeless has joined #openstack-meeting-cp00:47
*** gjayavelu has quit IRC00:48
*** markvoelker_ has joined #openstack-meeting-cp01:05
*** ebalduf has quit IRC01:48
*** markvoelker_ has quit IRC01:51
*** tyr_ has joined #openstack-meeting-cp01:53
*** tyr_ has quit IRC01:57
*** markvoelker has joined #openstack-meeting-cp01:59
*** markvoelker_ has joined #openstack-meeting-cp01:59
*** markvoelker has quit IRC02:03
*** gjayavelu has joined #openstack-meeting-cp02:06
*** markvoelker_ has quit IRC02:31
*** markvoelker_ has joined #openstack-meeting-cp02:38
*** tyr_ has joined #openstack-meeting-cp02:54
*** tyr_ has quit IRC03:00
*** markvoelker_ has quit IRC03:13
*** ebalduf has joined #openstack-meeting-cp03:36
*** tyr_ has joined #openstack-meeting-cp04:05
*** tyr_ has quit IRC04:10
*** sdake has quit IRC04:57
*** flwang1 has quit IRC05:06
*** evgenyf has joined #openstack-meeting-cp06:30
*** reed has quit IRC06:44
*** belmoreira has joined #openstack-meeting-cp06:44
*** belmoreira has quit IRC06:45
*** reed has joined #openstack-meeting-cp06:46
*** belmoreira has joined #openstack-meeting-cp06:51
*** evgenyf has quit IRC06:55
*** ebalduf has quit IRC07:21
*** gjayavelu has quit IRC08:30
*** gjayavelu has joined #openstack-meeting-cp08:31
*** gjayavelu has quit IRC08:54
*** evgenyf has joined #openstack-meeting-cp09:29
*** evgenyf has quit IRC09:52
*** sdague has joined #openstack-meeting-cp10:34
*** dansmith has quit IRC11:14
*** dansmith has joined #openstack-meeting-cp11:17
*** dansmith is now known as Guest5192611:17
*** vgridnev- has joined #openstack-meeting-cp11:39
*** vgridnev- is now known as vgridnev11:39
*** ttx has quit IRC11:56
*** ninag has joined #openstack-meeting-cp11:59
*** ninag has quit IRC12:05
*** ttx has joined #openstack-meeting-cp12:07
*** raildo-afk is now known as raildo12:10
*** ninag has joined #openstack-meeting-cp12:38
*** ninag has quit IRC12:38
*** ninag has joined #openstack-meeting-cp12:38
*** amrith is now known as _amrith_12:46
*** itisha has quit IRC13:09
*** xyang1 has joined #openstack-meeting-cp13:18
*** Guest51926 is now known as dansmith13:26
*** sigmavirus24_awa is now known as sigmavirus2413:28
*** belmoreira has quit IRC13:59
*** ebalduf has joined #openstack-meeting-cp13:59
*** sdake has joined #openstack-meeting-cp14:05
*** _amrith_ is now known as amrith14:09
*** flaper87 has quit IRC14:24
*** flaper87 has joined #openstack-meeting-cp14:27
*** flaper87 has quit IRC14:27
*** flaper87 has joined #openstack-meeting-cp14:27
*** sdake has quit IRC14:35
*** sdake has joined #openstack-meeting-cp15:17
*** hemnafk is now known as hemna15:21
thingeesdague: does the tng group still do standup - or are things on hold?15:21
sdaguethingee: things are still on hold right now, once the api-ref stuff is under control I'll kick it back up15:22
*** tyr_ has joined #openstack-meeting-cp15:59
*** ninag has quit IRC15:59
*** xinli has joined #openstack-meeting-cp15:59
*** sdake has quit IRC16:13
*** mriedem has joined #openstack-meeting-cp16:31
*** gjayavelu has joined #openstack-meeting-cp16:32
*** sdake has joined #openstack-meeting-cp16:49
*** gjayavelu has quit IRC16:51
*** mageshgv has joined #openstack-meeting-cp16:51
*** mageshgv has quit IRC16:51
*** gjayavelu has joined #openstack-meeting-cp16:57
ildikov#startmeeting cinder-nova-api-changes17:00
openstackMeeting started Thu May 12 17:00:23 2016 UTC and is due to finish in 60 minutes.  The chair is ildikov. Information about MeetBot at http://wiki.debian.org/MeetBot.17:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.17:00
*** openstack changes topic to " (Meeting topic: cinder-nova-api-changes)"17:00
smcginniso/17:00
openstackThe meeting name has been set to 'cinder_nova_api_changes'17:00
scottdahi17:00
mriedemo/17:00
ildikovscottda ildikov DuncanT ameade cFouts johnthetubaguy jaypipes takashin alaski e0ne jgriffith tbarron andrearosa hemna erlon mriedem gouthamr ebalduf patrickeast smcginnis diablo_rojo gsilvis17:00
ildikovhi17:00
*** aimeeu has joined #openstack-meeting-cp17:01
alaskio/17:01
thingeemriedem: I'm not sure why my question is being avoided after I asked twice. Is this because it's a priority problem in nova, or because it never will be for multiattach.17:01
smcginnisDo we have an agenda up somewhere?17:01
ildikovas far as I know jgriffith_ is out today, but we still have a few items to touch on17:01
cFoutso/17:01
mriedemthingee: later17:01
ildikovetherpad with info: #link https://etherpad.openstack.org/p/cinder-nova-api-changes17:02
*** ameade has joined #openstack-meeting-cp17:02
aimeeulurking and learning17:02
mriedemthere were some items from ildikov's meeting minutes from last week17:02
ildikovsmcginnis: I added the list of items we are targeting to get done to the etherpad17:02
mriedem" John Griffith will work on the above described solution, that target is to have patches up by next week."17:02
smcginnisildikov: Thanks!17:02
ildikovwe can go through those17:02
hemnaI have a question17:03
hemnaI'm working on a nova patch to not call check_attach at attach time17:03
ildikovI haven't seen patch(es) up from John yet17:03
hemnaand check_attach does 2 things.  1) it checks internal state of the volume and 2) checks the availability zone17:04
hemnadoes it make sense to add an optional AZ param to os-attach ?17:04
hemnaand have cinder check at os-reserve ?17:04
hemnaor just keep the check on the nova side only17:05
mriedemhttps://github.com/openstack/nova/blob/026468772672215d34a593e631d1e62d6a615aa4/nova/volume/cinder.py#L27917:05
hemnahttps://github.com/openstack/nova/blob/master/nova/volume/cinder.py#L289-L29917:05
mriedemso, the az stuff is a mess kind of17:05
hemnaI was just working on moving that code into a check_availability_zone() call in there instead17:05
hemnabut before I go forward, I'd like to hear opinions on it17:05
mriedemsee https://github.com/openstack/nova/blob/026468772672215d34a593e631d1e62d6a615aa4/nova/virt/block_device.py#L6017:05
mriedem^ is really for boot from volume where nova creates the volume17:06
hemnaI'd prefer to change nova's attach code to simply call os-reserve17:06
mriedembecause nova will create the volume in the same AZ that the instance is in, which might not exist in cinder17:06
hemnainstead of a volume get, then check, then os-reserve17:07
mriedemhemna: i think the az check in the api just needs to remain a separate thing17:07
mriedemsee my todo here https://github.com/openstack/nova/blob/026468772672215d34a593e631d1e62d6a615aa4/nova/virt/block_device.py#L7917:07
mriedemi've had a long-term wish of creating the volume in nova-api for boot from volume, and then attaching it later17:07
hemnahttps://github.com/openstack/nova/blob/master/nova/compute/api.py#L309517:08
mriedemso we do all of the az checking and stuff with cinder in the api rather than on the compute17:08
hemnathat thing17:08
scottdaBut for nova to do the AZ check, it will still need the volume.get, which defeats the point of what hemna is trying to do.17:08
hemnaI was hoping could simply be a call to self.volume_api.reserve_Volume()17:08
hemnascottda, +117:08
hemnayah17:08
hemnaso there is that.17:08
hemnathe get, then reserve means there is still a race17:09
mriedemso you'd have to pass the az to os-reserve17:09
hemnayah17:09
hemnaas an optional param17:09
hemnaif it's there, cinder tests it.17:09
mriedemre: https://github.com/openstack/nova/blob/master/nova/compute/api.py#L3095 ndipanov had a patch for a race in there also: https://review.openstack.org/#/c/290793/17:09
hemnaif it's not, it assumes it's open, re: no AZ17:09
mriedemyeah, and nova's logic for passing the az would be based on what we have in https://github.com/openstack/nova/blob/026468772672215d34a593e631d1e62d6a615aa4/nova/virt/block_device.py#L6017:10
mriedemfor bfv17:10
mriedemshall we take notes in https://etherpad.openstack.org/p/cinder-nova-api-changes ?17:11
hemnahttps://github.com/openstack/nova/blob/master/nova/volume/cinder.py#L28917:11
hemnaso right now, that's checked17:12
hemnakinda the same thing17:12
mriedemyeah, nova would just need to re-use some logic to determine if it needs to pass the az to os-reserve17:12
hemnasounds like the _get_volume_create_az_value() needs to be public17:12
mriedemif CONF.cinder.cross_az_attach, we'd pass None17:12
ildikovmriedem: I will add the decision points to the etherpad after the meeting17:12
hemnaI don't see any AZ check on the cinder side17:14
hemnaso I dunno17:14
scottdaI don't think there's any AZ checks enforced in Cinder17:14
mriedemyes there is17:14
mriedemwhen creating the volume17:14
mriedemnova can pass an az and if it doesn't exist cinder fails the volume create request17:15
mriedemUNLESS17:15
mriedemyou set a backdoor config option to ignore htat17:15
mriedem*that17:15
hemnathe create flow passes in an AZ17:15
smcginnisUnless a fallback is configured.17:15
mriedemsmcginnis: right, which was a hack because we didn't have the fix in nova17:15
mriedemwhich is https://github.com/openstack/nova/blob/026468772672215d34a593e631d1e62d6a615aa4/nova/virt/block_device.py#L6017:15
hemnableh17:15
mriedemhttps://github.com/openstack/nova/commit/f9a51b970f688b90baf0ae3ef31d79b3fec02ed117:15
hemnaok, so I don't want to make the AZ nightmare worse17:15
scottdahemna: You made is worse by mentioning it.17:16
mriedemwell, passing the az to os-reserve and cinder checking if it's provided, isn't really making it worse17:16
mriedemif nova doesn't provide it, it's a noop17:16
hemnascottda, :)17:16
mriedemif cinder microversion isn't new enough for nova to pass it, then nova still has to check like it is today17:16
hemnaI guess the real question is, should cinder care?17:16
hemnashould cinder be doing the check and fail if the AZ doesn't match ?17:17
mriedemso,17:17
mriedemwhen i was fixing this bug in nova, i had a thread in the ML about removing the nova cross_az_attach option17:17
scottdaThere are use cases where deployers had geographically distinct AZs, so this was needed.17:17
hemnaafaik AZ is a nova concept ?17:17
mriedemand there were operators saying they relied on it17:17
mriedemscottda: yes17:17
mriedemthat17:17
scottdaWe did it in our (now defunct) public cloud...17:17
mriedemsee http://lists.openstack.org/pipermail/openstack-operators/2015-September/008252.html17:18
mriedemfor some light bedtime reading17:18
hemna:)17:18
smcginnisThe backdoor config option work brought up the fact that AZs were never fully baked.17:18
mriedemstarts here http://lists.openstack.org/pipermail/openstack-operators/2015-September/008224.html17:18
scottdaYeah, the terminology is vague, and that's part of the problem....but we still have to live with it.17:18
hemnaand by 'fully baked' does that mean that nova should be passing the AZ in calls to Cinder ?17:18
hemnaso that they can both be on the same page?17:18
mriedemthis is the cinder workaround https://review.openstack.org/#/c/217857/17:18
smcginnishemna: I think to fully support and enforce AZs, yeah. :/17:19
mriedemthere are some decent details and background in the commit message of https://review.openstack.org/#/c/227564/17:19
hemnaso if a user creates a volume, is the AZ set?  and to what?  and how is that checked against attach calls from nova ?17:19
smcginnisBut maybe we should shelve this az discussion for now and get back to multiattach. AZs are an issue for single and multi attach.17:19
hemnableh17:19
* hemna cowers in defeat17:20
mriedemso ftr, to fully remove nova's check_attach, cinder's os-reserve would need to take an az17:20
mriedemto validate it17:20
hemnamriedem, yah17:20
mriedemat least to be consistent with how things are today17:20
hemnathat's why I brought it up17:20
mriedemlet it be written in the etherpad for all time!17:20
ildikovI guess we can make the 'check_attach' removal a two step process17:20
hemnaso, if I still do the AZ check on the nova side17:21
hemnathe race is smaller17:21
hemnaat least nova won't be checking volume state17:21
scottdaYeah, but that's a bit of code churn and review time for an incomplete fix...17:21
hemnaI think eventually, we do want to just pass the AZ to cinder and then nova can call reserve w/o a get.17:21
hemnaI won't change the functionality of check_attach for now.17:22
hemnabut I will refactor the AZ check out of there17:22
mriedemscottda: it's just a bug fix really17:22
hemnaand then simply call the new AZ check after the get.17:22
mriedemyeah i think hemna and i are on the same page17:22
scottdafair enough17:22
hemnathen reserve_volume will catch the state checks.17:22
ildikovis it only the BFV case?17:23
mriedemno17:23
scottdaildikov: no17:23
ildikovI mean when check_AZ will need to be called17:23
mriedemso in the remaining 7 minutes i have...17:23
hemnaok I'll forge ahead with this and push it up today then.17:23
mriedemhemna: you might want to look at https://review.openstack.org/#/c/290793/ too17:23
*** tyr_ has quit IRC17:24
ildikovcool, added a note to the etherpad about the AZ check17:24
hemnamriedem, ok will do17:25
ildikovmriedem: can you check the multiattach spec when you have some time?17:25
*** gjayavelu has left #openstack-meeting-cp17:25
mriedemildikov: is it any different from mitaka?17:25
ildikovmriedem: slightly updated, I added a link to the etherpad so that we would not need to add implementation details to the spec regarding how to sort out things in Cinder17:26
mriedembecause i was under the impression that the multiattach spec was going to be dependent on the POC that jgriffith_ was going to be doing17:26
ildikovdoes this mean we can talk about approving it, when that is ready?17:26
mriedemi'd prefer to not land a bunch of technical debt in nova just to get this in17:27
ildikovthe Cinder part is a dependency in the sepc, if these issues are not sorted out, than we're in trouble anyway17:28
ildikovit does not mean to sort it out in Nova instead in my view17:28
mriedemok i'll have to review the spec to see the changes then17:29
ildikovand the plan is to get them done :)17:29
mriedem#action mriedem to review multiattach nova spec17:29
ildikovtnx17:29
mriedem#action hemna to poke at cleaning up nova check_attach17:29
hemnacoolio17:29
mriedemwhat's the status on cinder migrate testing on the multinode job in the gate?17:29
ildikovif there's anything Nova specific that's missing I will add it17:29
scottdamriedem: We're starting with cinder migrate on a single node. We think we can get that working....17:30
scottdaBut it looks like Devstack support for multi-backend was removed. I'm trying to figure out why, and what alternative exists.17:30
mriedemscottda: as in resize?17:31
scottdaBut eventually want multi-node as well.17:31
hemnawait what?17:31
hemnacinder multi-backend removed from devstack ?17:31
scottdano, just have 2 LVM volume groups as separate backends, and migrate between them on a single node.17:31
thingeemriedem: no like like multi drivers17:31
scottdahemna: No, I've actually found a way to do it, the syntax has changed...17:32
scottdaand17:32
scottdaand Tempest multi-backend tests are failing for me. Not sure why.17:32
mriedemand that will still test swap volume?17:32
scottdayes, calling cinder migrate will call swap volume.17:33
mriedemok, do we want to talk about https://review.openstack.org/#/c/312773/ ?17:34
scottdaWhat do you think of that patch mriedem ?17:35
mriedemhonestly i haven't had the time to dig into it17:35
mriedemwould be nice to see the live migration job or multi node job passing on it17:36
mriedembut those are super flaky17:36
mriedemi can dig into the test failures for volume-backed live migration17:36
mriedemand see if they are related17:36
mriedemfinally, before i go,17:37
mriedemanyone talked to jgriffith_ on the os-initialize_connection changes?17:38
scottdano, I haven't17:38
ildikovmriedem: the job says for live migration that it passed, but I might missed smth in the logs...17:38
ildikovmriedem: I talked to him briefly, he's working on it, but we couldn't go into details17:38
mriedemildikov: yeah http://logs.openstack.org/73/312773/1/experimental/gate-tempest-dsvm-multinode-live-migration/c57f6b9/console.html#_2016-05-08_09_14_44_57217:39
hemnathe experimental jobs seem.....borked almost every time.  :(17:39
scottdaI think John said in IRC that he had unit tests for his patch mostly passing...17:39
ildikovhemna: it's weird a bit, it congratulates you and then marks the test failed...17:39
hemnahehe17:40
mriedemit is17:40
scottdalike a participation trophy.17:40
mriedemhttp://logs.openstack.org/73/312773/1/experimental/gate-tempest-dsvm-multinode-live-migration/c57f6b9/console.html#_2016-05-08_09_18_59_36017:40
mriedemsetting up ceph17:40
mriedemi've pinged tdurakov on that, he works on that job17:40
ildikovmriedem: scottda: I will try to catch him and add notes to the etherpad about that item this week or early next17:41
mriedemthat job sets up various storage backends in a single job17:41
mriedemand runs the same 4 tests17:41
mriedemlooks like it's not working for ceph atm17:41
mriedemildikov: ok17:41
mriedemalright, over by 11 minutes17:41
mriedemanything else?17:41
ildikovalso this time next week might be tricky for me17:41
*** markvoelker has joined #openstack-meeting-cp17:41
ildikovbut will try my best17:42
mriedemchange the time as needed17:42
scottdaLet's work on a new time. It'd be nice to have JohnGarbuttt here, and JohnG as well17:42
ildikovalso I know johnthetubaguy cannot make it at this slot, so if it's problematic to either of you in general please let me know and then we can find another one17:42
hemnaok17:42
mriedemalso, fyi, i'm out from 5/20-5/3017:42
hemnathanks for the help guys17:42
mriedemback on 5/3117:42
smcginnismriedem: Nice17:43
ildikovmriedem: ok, thanks for the info17:43
scottdaok, bye all.17:43
ildikovI will reach out to you regarding time slots17:43
ildikovthanks all!17:43
ildikov#endmeeting17:44
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings"17:44
openstackMeeting ended Thu May 12 17:44:15 2016 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)17:44
openstackMinutes:        http://eavesdrop.openstack.org/meetings/cinder_nova_api_changes/2016/cinder_nova_api_changes.2016-05-12-17.00.html17:44
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/cinder_nova_api_changes/2016/cinder_nova_api_changes.2016-05-12-17.00.txt17:44
openstackLog:            http://eavesdrop.openstack.org/meetings/cinder_nova_api_changes/2016/cinder_nova_api_changes.2016-05-12-17.00.log.html17:44
*** mriedem has left #openstack-meeting-cp17:45
*** ninag has joined #openstack-meeting-cp17:46
*** ninag has quit IRC17:46
*** ninag has joined #openstack-meeting-cp17:48
*** ninag_ has joined #openstack-meeting-cp17:50
*** ninag has quit IRC17:53
*** amrith has quit IRC17:53
*** _amrith_ has joined #openstack-meeting-cp18:08
*** _amrith_ is now known as amrith18:08
*** ninag_ has quit IRC18:13
*** flwang1 has joined #openstack-meeting-cp18:14
*** ninag has joined #openstack-meeting-cp18:15
*** markvoelker_ has joined #openstack-meeting-cp18:19
*** markvoelker has quit IRC18:22
*** belmoreira has joined #openstack-meeting-cp18:34
*** ninag has quit IRC18:35
*** ninag has joined #openstack-meeting-cp18:35
*** ninag has quit IRC18:35
*** ninag has joined #openstack-meeting-cp18:36
*** ninag has quit IRC18:39
*** sdake has quit IRC18:44
*** ninag has joined #openstack-meeting-cp18:46
*** ninag has quit IRC18:51
*** xinli has quit IRC19:10
*** ninag has joined #openstack-meeting-cp19:38
*** bswartz has quit IRC19:39
*** flwang1 has quit IRC19:40
*** ninag has quit IRC19:41
*** ninag has joined #openstack-meeting-cp19:42
*** ninag has quit IRC19:45
*** belmoreira has quit IRC19:54
*** xinli has joined #openstack-meeting-cp20:02
*** ebalduf has quit IRC20:09
*** amrith is now known as _amrith_20:12
*** tyr_ has joined #openstack-meeting-cp20:14
*** belmoreira has joined #openstack-meeting-cp20:16
*** belmoreira has quit IRC20:34
*** ninag has joined #openstack-meeting-cp20:36
*** raildo is now known as raildo-afk20:41
xinlixyang: the gate test failed in gate-tempest-dsvm-postgres-full, but in previous Jekins test it passed.20:52
*** sdake has joined #openstack-meeting-cp21:03
*** sdake_ has joined #openstack-meeting-cp21:24
*** sdake has quit IRC21:28
*** sdake has joined #openstack-meeting-cp21:31
*** flwang1 has joined #openstack-meeting-cp21:32
*** sdake_ has quit IRC21:32
*** bswartz has joined #openstack-meeting-cp21:47
*** ninag has quit IRC21:50
*** sigmavirus24 is now known as sigmavirus24_awa22:00
*** tyr_ has quit IRC22:02
*** markvoelker_ has quit IRC22:25
*** ninag has joined #openstack-meeting-cp22:52
*** ninag has quit IRC22:57
*** xinli has quit IRC22:57
*** sdague has quit IRC23:01
*** xyang1 has quit IRC23:06
*** dcwangmit01 has quit IRC23:17
*** dcwangmit01 has joined #openstack-meeting-cp23:17
*** dcwangmit01 has quit IRC23:21
*** dcwangmit01 has joined #openstack-meeting-cp23:21
*** dcwangmit01 has quit IRC23:23
*** dcwangmit01 has joined #openstack-meeting-cp23:23
*** markvoelker has joined #openstack-meeting-cp23:26
*** markvoelker has quit IRC23:31

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!