22:00:42 <alaski> #startmeeting nova_cells
22:00:42 <openstack> Meeting started Wed Jan 21 22:00:42 2015 UTC and is due to finish in 60 minutes.  The chair is alaski. Information about MeetBot at http://wiki.debian.org/MeetBot.
22:00:43 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
22:00:46 <openstack> The meeting name has been set to 'nova_cells'
22:00:51 <bauzas> \o
22:01:04 <alaski> hi, who's around for the meeting?
22:01:06 <melwitt> o/
22:01:35 <belmoreira> hi
22:01:56 <alaski> I know dansmith is out in the wilderness
22:02:27 <alaski> I'm actually just going to go into open discussion this week
22:02:32 <alaski> I don't have a prepared agenda
22:02:39 <alaski> #topic open discussion
22:02:54 <bauzas> erm
22:03:15 <alaski> bauzas: do you have an update on test exclusions?
22:03:29 <bauzas> alaski: I was about providing a change
22:03:41 <bauzas> alaski: I will keep the template and do a conditional for it
22:03:57 <bauzas> alaski: I will CC you on the patch
22:04:10 <alaski> great
22:04:24 <alaski> I would like to get sdagues take on it as well
22:05:09 <bauzas> sure, to be clear, I did some templates for another stackforge project, but that's the first time I'm diretly working for that here
22:05:18 <bauzas> alaski: tbp https://github.com/openstack-infra/project-config/blob/f98caac794f6d1327cb7565c446c026cc951a873/jenkins/jobs/devstack-gate.yaml#L792-L837
22:05:33 <bauzas> alaski: I will provide a if conditional checking the pipeline
22:05:51 <bauzas> alaski: and if pipeline being 'check', it will add some exceptions
22:06:14 <edleafe> o/ (late)
22:06:25 <alaski> bauzas: okay
22:06:29 <bauzas> alaski: of course, I'll rename existing experimental jobs to exp-tempest-dvsm-cells
22:06:46 <alaski> I'm tempted to say that we should just run one job with all of the exceptions
22:07:01 <bauzas> alaski: then that's easier
22:07:20 <bauzas> alaski: but we won't have any way to check if we fix the regressions
22:07:26 <alaski> it won't entice anyone to work on fixing any tests, but that hasn't really been happening anyways
22:07:43 <bauzas> alaski: up to you actually
22:07:47 <melwitt> well... I have started running tempest with cells locally and taking down the cause for each failure in the same style as alaski had done, with the latest list of failures (I get 74 when running locally). then next I'll compare with the old failure list and also see if I can fix any of them
22:08:03 <alaski> melwitt: awesome
22:08:04 <bauzas> melwitt: orly ?
22:08:25 <bauzas> melwitt: interesting in getting your local list of failures
22:08:46 <melwitt> I'm updating https://etherpad.openstack.org/p/nova-cells-testing as I go along with my findings
22:09:31 <bauzas> melwitt: cool
22:09:50 <alaski> melwitt: very nice
22:10:02 <bauzas> alaski: ok, will then only update the current template by adding the 74 exceptions and will enable the test in non-voting check pipeline
22:10:33 <bauzas> if we need to have a CI reporting all the issues, we could add another conditional later
22:10:43 <alaski> bauzas: I think that might be good for now, we'll just have to reduce the exclusions in two steps then
22:10:56 <alaski> fix in nova, then remove from devstack regex
22:11:09 <bauzas> alaski: yup
22:11:25 <bauzas> ok, working it now
22:11:27 <alaski> melwitt: The flavor 42 could not be found failures are interesting, I thought that issue was fixed
22:11:31 <bauzas> that's really easy
22:12:05 <melwitt> alaski: yeah, I don't really understand all of this yet (not a cells expert) but it's the m1.nano flavor is in the top level instance_types table but not the nova_cell instance_types table
22:12:51 <alaski> bauzas: cool, thanks
22:13:17 <alaski> melwitt: oh.  that's a slightly different issue then
22:13:34 <alaski> when flavors are added via the api they aren't replicated to the cell db
22:14:06 <melwitt> alaski: okay. there's another one that puzzled me, it behaves as though region!child@1 is supposed to be the key value for an attr on compute node. but I don't find that encoded name anywhere in the db, even though from the code it has to have got it from there?
22:14:27 <alaski> melwitt: that issue is interesting
22:14:28 <melwitt> alaski: ah, okay. so if tempest is adding via api, they need to add to cell api too?
22:15:13 <alaski> melwitt: right, but there isn't a mechanism for doing that today
22:15:40 <alaski> we could fix the API calls so they route through cells properly, but it's not an insignificant effort
22:15:57 <melwitt> hm, okay
22:16:25 <alaski> those might be better skipped for now, but if someone wants to fix that it would be great
22:16:37 <alaski> for the other issue take a look at the code around https://review.openstack.org/#/c/145362/2/nova/compute/cells_api.py
22:17:20 <alaski> melwitt: essentially the cells api layer is adding routing information to the compute_node.id.  it's not actually in the db, just returned via the api
22:17:47 <melwitt> alaski: oh, thanks!
22:18:15 <alaski> melwitt: np.  I guess it actually happens a layer lower than that, but that's essentially it
22:19:00 <melwitt> yeah, helps a lot to know where to start looking either way
22:19:08 <alaski> so from my end I've been working on having nova-manage work with two databases
22:19:13 <alaski> melwitt: definitely
22:19:50 <alaski> I have a lot to learn about oslo.db and sqlalchemy to figure out the right way to work with multiple databases
22:20:42 <bauzas> alaski: is this really difficult ?
22:21:20 <alaski> bauzas: I don't think it will be difficult, there seems to be support in sqlalchemy for this.  It's mainly me understanding it
22:21:35 <alaski> and getting it setup properly in nova and oslo.db
22:22:16 <alaski> and not having as much time as I want :)
22:22:19 <belmoreira> alaski: it's great to know... we started looking as well and doesn't seems trivial
22:23:02 <bauzas> alaski: I'm saying that because there is a sql_connection opt to be done
22:23:08 <alaski> one thing I don't understand yet is whether we can dynamically add new databases to talk to, or if a restart will be required
22:23:50 <alaski> belmoreira: definitely not trivial, but I think it's mostly a matter of finding the right places to configure it
22:23:53 <bauzas> alaski: at least you need to setup DB
22:24:16 <alaski> bauzas: right
22:24:34 <bauzas> alaski: so if you're using a CONF object, then you need to restart
22:24:41 <alaski> so one question I have is what are peoples thoughts on naming of the dbs
22:24:51 <bauzas> because the CONF object is not live changing
22:25:09 <alaski> right now everything assumes one db and is named just sql_ something or db_ something
22:25:26 <alaski> I've been adding api_sql_ and api_db_ stuff
22:25:33 <bauzas> IIRC, you have a SQL connection string to provide
22:25:45 <bauzas> including the creds and the DB name
22:26:00 <alaski> bauzas: we decided to put that in a db right?
22:26:10 <alaski> the dynamic ones
22:26:15 <bauzas> alaski: right, just talking about how it can work
22:26:29 <bauzas> oslo.db is using a sql_connection opt IIRC
22:26:51 <alaski> right, I'm still detangling some of that
22:26:58 <bauzas> so, that sounds possible to dynamically change this
22:27:08 <belmoreira> alaski: yes... but in all_in_one deployments will we need at least 2 nova.conf
22:27:36 <bauzas> you just need to not use this opt, and provide the sql connection when instantiating the DB API implementation
22:27:40 <alaski> belmoreira: I think so, like we have today
22:27:59 <bauzas> belmoreira: that's already manageable per service
22:28:20 <alaski> bauzas: right, that's what I'd like to see.  But there's a lot of strangeness in how things are configured in oslo.db right now
22:28:24 <bauzas> belmoreira: I mean, oslo.config is allowing you to specify a conf file per service name
22:28:57 <bauzas> belmoreira: so assuming the cells service won't have the same name than the child service, it should be fine
22:29:05 <alaski> at one point I think a config option is being populated in the code passed on args passed in
22:29:29 <belmoreira> bauzas: great... I'm thinking how complex it can be to have a simple all_in_one devstack running
22:30:08 <bauzas> alaski: that depends if it's a cli_opt
22:30:41 <bauzas> depends on, (of course)
22:31:42 <alaski> bauzas: it's not even that, the way that the backend for migrations is looked up is done like that.  It reads it from a config option that isn't actually defined as an option to be read from cli/conf file that I can see
22:32:14 <alaski> it's just magically created at some point in the back and forth between nova and oslo.db
22:33:06 <alaski> but I'm working my way through it all for now
22:33:53 <bauzas> alaski: oh I see your question
22:34:16 <bauzas> alaski: that's an oslo.db option
22:34:29 <bauzas> IIRC of course
22:34:34 <bauzas> AFAIK
22:35:02 <bauzas> (well, I was using the oslo incubator a while ago, maybe things changed since that)
22:35:25 <alaski> I didn't find it in there.  But I think I just need to get in touch with an expert over there and ask some questions
22:35:49 <alaski> any other updates or things to discuss today?
22:36:20 <belmoreira> continuing DB discussion
22:36:59 <belmoreira> we will need to have two different nova.db.apis
22:37:20 <belmoreira> one for each DB
22:37:35 <alaski> right
22:38:06 <alaski> we may be able to work with objects so that we don't end up with another giant db.api type file
22:38:15 <bauzas> alleliuah
22:38:23 <alaski> but whatever we end up with will be separate from the current db.api
22:39:06 <bauzas> that said, using objects doesn't necessarly means we can get rid of a big giant DB API
22:39:24 <bauzas> it only allows us to hide the conductor methods right ?
22:39:48 <alaski> I'm not sure at this point
22:40:03 <alaski> I think Dan had a plan to move the db calls into the objects, or closer to them
22:40:15 <alaski> I'll ask him about it next week
22:40:19 <bauzas> alaski: yeah I remember that disussion
22:40:26 <bauzas> alaski: +1, will be at midcycle too
22:40:33 <alaski> great
22:41:11 <bauzas> alaski: IIRC the discussion was about the number of save() calls in the objects which was not efficient
22:41:12 <alaski> belmoreira: was there more you wanted to discuss?
22:41:54 <alaski> bauzas: yeah, that would be good to address
22:42:44 <belmoreira> alaski: we are looking into the same things that you mentioned... it will be good to share
22:43:18 <alaski> belmoreira: ok, good to know
22:43:48 <alaski> definitely don't want to duplicate work if we don't need to
22:45:01 <alaski> belmoreira: I'm out basically all next week for the mid-cycle and travel, but I'll get in touch when I get back
22:45:14 <alaski> anything else from anyone?
22:46:31 <alaski> alright, I'm going to call it then
22:46:44 <alaski> thanks everyone!
22:46:51 <bauzas> thanks
22:46:55 <alaski> #endmeeting