17:00:22 #startmeeting nova_cells 17:00:22 Meeting started Wed Aug 26 17:00:22 2015 UTC and is due to finish in 60 minutes. The chair is alaski. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:00:23 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:00:26 The meeting name has been set to 'nova_cells' 17:00:42 anyone around today? 17:00:48 o/ 17:00:48 o/ 17:00:50 o/ 17:00:57 o/ 17:01:00 awesome 17:01:07 #topic Tempest testing 17:01:22 it appears that elastic search isn't responding to me at the moment 17:01:29 but melwitt, any news? 17:01:52 I've noticed it's been like this the past few weeks at least, for the 7d choice. if I change it to 48h I can get results 17:02:07 ahh, that worked 17:02:59 things seem okay, yesterday there were 8 failures, one was a bad patch. the others are the instance not found problem in test_list_servers_by_admin_with_all_tenants 17:03:32 okay, great 17:03:50 mriedem was working on that one 17:03:58 oh, cool 17:04:16 https://review.openstack.org/#/c/215859/ I think 17:04:34 #topic Open Reviews 17:05:02 there are a few reviews that could always use feedback 17:05:11 I'll just link https://wiki.openstack.org/wiki/Meetings/NovaCellsv2 which has them all listed 17:05:53 alaski: about the flavor port one (http://review.openstack.org/201606) 17:05:55 if there are any missing feel free to add them there and to the priority etherpad 17:06:14 we dropped soft-delete for now because too many tempest and unit tests were failing 17:06:24 I mean we keep soft-delete option 17:06:44 dheeraj: tempest tests on that patch, or the following one? 17:06:52 on the following one 17:07:07 when we actually tried to write code that used new Flavor models 17:07:48 I commented on that in the other review 17:08:18 I think you should keep the object the same, i.e. don't remove deleted/deleted_at 17:08:19 alaski: yeah, so we figured it was simpler to keep deleted columns for now 17:08:27 about the dependent patch (https://review.openstack.org/#/c/213041/) 17:09:03 a lot of code looks at .id (the auto-generated id in the table) and when flavors will be moved, the id will obviously change 17:09:03 it's simpler now, but we'll need to deal with it later 17:09:12 alaski: agreed 17:09:16 +! 17:09:42 +1 17:10:10 dheeraj: okay, that sounds like another issue different from the deleted one 17:10:23 so id is a sticking point. Right now the WIP patch doesn't implement all the stuff (it is more focussed on get_*), but we would like to get more feedback about it 17:10:36 okay, I'll look at that and comment 17:10:56 alaski:: thanks 17:11:12 for soft-delete I think it can be handled reasonably well by having compatibility code in the db methods for the nova_api db 17:11:31 if the object passes in deleted/deleted_at just ignore it, and add those fields in to the db return 17:12:03 alaski: some db code also looks at deleted columns 17:12:41 dheeraj: yes. but any new db code shouldn't. so db code for nova db will, but nova_api db won't 17:12:56 as long as those don't get mixed it should be fine 17:13:29 I have some questions from working on https://review.openstack.org/161906 db connection switch 17:13:43 alaski: ok. we'll take a look 17:13:53 maybe have a dirty patch and see how tempest holds 17:14:12 dheeraj: okay. the preference would be to not have soft-delete for the new db, but if it's overly complex right now it's not a blocker 17:14:23 melwitt: go for it 17:14:30 alaski: cool 17:15:39 I've been trying (unsuccessfully) to write a functional test and ran into wondering, how is it supposed to work with for example, Instance object contains a SecurityGroupList field, when the security groups reside in a different database 17:16:50 like when FK will span across databases 17:17:04 melwitt: I think we need to assume it will be in the same db 17:17:11 unless it's something in the api and cell db 17:17:44 and in that case we may need to think about composing the objects differently 17:17:53 okay. I was using http://docs.openstack.org/developer/nova/cells.html#global-api-level-tables as a guide on how to think about it and currently we have security_groups in the api db, or maybe it will be both? 17:18:23 melwitt: do we write to security groups outside of the api, do you know? 17:19:17 alaski: in general I don't think so but I'd have to check to be sure. what happens now is during instance_create, it will create the default sec group if it doesn't exist 17:20:51 okay, I trying to think through cases here 17:20:55 *I'm 17:21:29 I think my first question is does changing a security group apply to all instances with that group? 17:21:54 I feel like we should treat it like a flavor, and write it to instance_extra with the instance in the cell db 17:22:09 but if changing the group is supposed to propagate that won't work 17:22:44 I'm not sure how it works but SecurityGroup has an orm relationship with SecurityGroupInstanceAssociation which has a FK of instance.uuid 17:23:50 *orm relationship Instance, SecurityGroupInstanceAssociation 17:24:04 I hate to say it, but we might need to look at breaking that 17:25:06 in general I'd like to move towards the instance picking up attributes as it moves through the system and storing those itself. like with flavor 17:25:36 but at this point I'm not sure I have a great answer 17:26:08 okay. there might be other things too but in my test I started with doing an instance.create() and saw it going to the sec groups so I stumbled over it 17:26:27 yeah, something to think about 17:26:43 for now it would be good to have a list of everything the instance object relies on having access to 17:27:18 we'll probably have to go case by case and come up with solutions 17:27:58 #topic Open discussion 17:28:10 okay, I'll go through and make a list. do you think we only have to be concerned with Instance? 17:28:34 melwitt: probably not 17:28:40 but for now it's the big concern 17:28:58 actually, it might just be instance 17:29:39 alaski: feature freeze is next week, so does that mean all code has to be merged before that or is there an exception procedure? 17:29:55 dheeraj: there is an exception process 17:30:07 but I'm not sure any of our stuff will qualify 17:30:23 mainly because it doesn't complete anything 17:30:36 alaski: yeah. In that case we should probably just roll it over to M 17:30:36 johnthetubaguy commented on the review also mentioning about, what will be the default database? I had been thinking "default" is whatever is local to the code querying, that is, each database will be configured for its local database. but then that wouldn't work to let it go to the default if it's say, querying for security groups. does that mean we'll need to actually have for example, get_api_session() for all security group related calls 17:30:37 inside the db api? 17:31:24 melwitt: my thinking has been that anything we know is in the api db shoudl use get_api_session 17:31:37 alaski: okay, that makes sense 17:31:41 and the context switching is primarily for things that could be in one of many cells 17:31:53 alaski: maybe compute cells will need to know about the api cell 17:32:09 dheeraj: that's seeming more and more likely 17:32:18 alaski: yeah, I think that would make it clearer to understand what's going on anyway too 17:32:24 dheeraj: but something I would like to avoid as much as possible 17:32:30 melwitt: cool 17:32:35 alaski: do we have to redo commits if we roll over to M ? 17:32:43 specs and code commits? 17:33:01 vineetmenon: for specs, maybe. that was unclear at this point. code shouldn't need to 17:33:42 vineetmenon: but if specs need to be redone it's a trivial approval process 17:33:49 I need to dig into how the sqlite test database stuff works because I find if I create two EngineFacade intended to point to the same database, it doesn't work. one of them errors saying no tables exist. (I'm trying to verify that different ones get used depending on context, throwing away the "connection" kwarg before letting it be created so they go to the same database really) 17:34:46 or should I instead be looking at actually making separate test databases and doing it that way? 17:35:22 (for the functional test under nova/tests/functional/db) 17:35:22 I vaguely recall running into something like that 17:35:47 from what I remember the sqlite db is created in memory so each connection ends up with it's own view 17:35:59 so you might need to run migrations on each 17:36:36 that's definitely what I'm seeing. I know from some doc or something I found somewhere that it's indeed an in-memory sqlite db 17:37:58 you might need to precreate the facades and run migrations on them, rather than letting them be lazily created in the tests 17:38:05 if I'm understanding correctly 17:38:12 what I don't understand is how "database" exists, migration.db_sync(database=self.database) where self.database is just "main" 17:38:34 from the Database fixture 17:38:47 melwitt: it runs API migrations if database="api" and nova DB mingrations if database="main" 17:38:59 is "main" indicating the EngineFacade or something else? 17:39:48 like how does it know from just the string "main" how to get to a thing to run migrations on 17:40:09 It is coded somewhere in fixtures 17:40:17 I think alaski committed that code 17:40:39 it's all in https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/migration.py#L90 17:41:37 okay, so it'll create its own new migrate repo if it doesn't exist 17:42:03 I think I understand enough to go that direction. thanks 17:42:14 it looks like only 'api' has meaning, and anything else just does the old default 17:42:42 alaski: yes 17:42:54 https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/migration.py#L1146 17:43:46 melwitt: sorry, I can't vouch for the prettiness of the code, only that I got it to work 17:43:48 I found that sync step actually creates the EngineFacade 17:43:56 ahh 17:44:12 oh, it calls get_engien 17:44:29 oh, right. that's what would do it 17:45:54 alaski: it's just new to me, didn't know what was happening :) 17:46:28 melwitt: heh, no worries. I remember it taking me some time to understand enough to do that migration piece 17:46:55 and it's mostly disconnected, except for that get_engine call 17:47:29 yeah, I was wondering how does the EngineFacade connect to the seemingly connectionless testing sqlite database 17:48:10 yeah, and the in-memory piece makes sqlite interesting compared to the other backends 17:48:22 anything else for today? 17:49:17 have a great day everyone 17:49:20 #endmeeting