16:00:22 <priteau> #startmeeting blazar
16:00:23 <openstack> Meeting started Thu Nov 19 16:00:22 2020 UTC and is due to finish in 60 minutes.  The chair is priteau. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:25 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:27 <openstack> The meeting name has been set to 'blazar'
16:00:35 <priteau> #topic Roll call
16:05:55 <diurnalist> o/
16:06:19 <priteau> o/
16:06:22 <priteau> Hi diurnalist
16:06:31 <diurnalist> Welcome back!
16:07:53 <priteau> Thank you
16:08:39 <priteau> Sorry for skipping a couple of meetings due to being away
16:08:52 <priteau> We are now returning to our regular schedule
16:09:22 <priteau> I wanted to catch up with you, I saw a number of updated patches but I haven't had time to look at them yet
16:09:24 <diurnalist> sounds good
16:09:55 <diurnalist> yes, I haven't had much time for upstream contributions myself, but had a day where I tried to go through all of our patches and try to get them up
16:10:03 <diurnalist> rebasing on master etc
16:10:45 <diurnalist> jakecoll doesn't work on the project anymore, but before he left (end of last week) he helped me write up some of the missing docs for network reservations
16:11:14 <priteau> I saw an updated patch from him last Friday
16:11:27 <priteau> I was wondering if he was going to be involved again
16:12:38 <priteau> I've got limited time for reviews until end of last week most likely, are there some patches that you would suggest are good candidates to be merged?
16:12:45 <priteau> I see network reservation still needs work
16:14:20 <diurnalist> Yes, I think it needs pep fixes at minimum. We had also discussed a tempest test, should we still try that? I think the difficulty was that the network plugin needs to be enabled by default, likely
16:15:43 <diurnalist> These ones are pretty small: https://review.opendev.org/#/c/762006/ https://review.opendev.org/#/c/761994/ https://review.opendev.org/#/c/761990/ https://review.opendev.org/#/c/589899/
16:15:44 <priteau> I was asking for the tempest test mostly because my manual testing of the patch back in… April? uncovered various issues (even though unit tests were passing)
16:16:01 <priteau> Just left feedback about the first one
16:16:05 <diurnalist> I see your comment on the shuffling, I forgot we had discussed the toggle
16:16:45 <diurnalist> Once those are in I would move on to brushing up the enforcement stuff and submitting that
16:16:47 <priteau> I forgot about the traceback one, adding it to my list to look at later
16:17:41 <diurnalist> this one has been sitting for a while and is probably the most complicated (other than net res.) https://review.opendev.org/#/c/731586/
16:18:26 <priteau> Yeah it's a though one, need some proper review session to go through it
16:18:45 <priteau> I'll try to schedule one soon
16:19:54 <diurnalist> ok, appreciated-- I have also thought about setting aside some time to perform some other refactors. In my opinion much of the plugin logic could/should be pulled up one level into the base implementation, or some other component
16:20:05 <diurnalist> really there are 2 refactors I have been kicking around:
16:20:46 <diurnalist> 1.) update lease tear-down to gracefully handle individual reservation termination errors. I already have a patch for this and we're testing now. Currently a termination halts on the first failed reservation teardown failure.
16:21:24 <diurnalist> 2.) move resource filtering out of plugins and make it shared business logic
16:22:09 <priteau> Both would be really good
16:22:26 <diurnalist> what I would really like to do long-term is support out-of-tree resource plugins. i envision moving more stuff to core concerns will make it much easier to do this
16:22:38 <priteau> Another I thought about is common code for handling extra capabilities
16:22:44 <diurnalist> :nod:
16:22:54 <priteau> Currently there's lots of code duplication to do this across both hosts and networks
16:23:16 <diurnalist> yes, especially that extra capabilities are really just a key/value system that can apply to any resource type
16:23:30 <priteau> Do you see out of tree resource plugins being loaded like scheduler filters in nova/
16:24:02 <diurnalist> I'm not 100% how nova does it. I am familiar with the Ironic way, where it's via entry_points
16:24:11 <diurnalist> which I think is fine, and pairs well with stevedore
16:25:32 <priteau> ironic's way might be more modern since it was written more recently
16:25:46 <diurnalist> the way that would work is you'd define an entry point namespace like "blazar.resource_type" and then your python module would export an entry point like "my_type = module.path.to:Thing"
16:25:47 <priteau> I tend to see more old patterns in nova than in the rest of OpenStack projects
16:26:11 <diurnalist> and then blazar conf would have "enabled_plugins = physical:host,my_type" or similar
16:26:31 <diurnalist> as long as the module is installed in the same python  path as blazar, everything's good
16:27:52 <priteau> I am not too familiar with this area but I guess it should work
16:28:13 <priteau> We mostly need to make sure there are no conflicts between plugins
16:29:02 <diurnalist> yes, and I think the trickiest thing is going to be how to handle DB dependencies. the DB schema might need to change, which is going to be painful
16:29:22 <diurnalist> OSC also uses entry points btw, this describes it a bit: https://docs.openstack.org/python-openstackclient/latest/contributor/plugins.html
16:30:08 <priteau> Separate DBs maybe?
16:30:58 <diurnalist> I guess Neutron is the other project to look at here
16:30:58 <priteau> That way you could have an alembic serial number by plugin
16:31:37 <diurnalist> one question to think about is: can all resource plugins share a db. is there anything that is not possible w/ this
16:32:48 <diurnalist> tricky because some resource properties are more important than others, like hypervisor_hostname for the physical host plugin
16:32:51 <priteau> I thought alembic migrations would conflict, but maybe not
16:33:02 <priteau> I can see in neutron-lbaas it uses alembic as well
16:33:08 <priteau> and AFAIK it was using the neutron db
16:33:22 <diurnalist> i think it has its own tables that it targets though, could be wrong
16:33:29 <diurnalist> vpnaas is similar
16:34:43 <priteau> Yes, separate tables but same db
16:35:02 <priteau> Could we keep resource plugins constrained to their own separate tables
16:35:29 <priteau> They would still be able to manipulate existing tables, but through the existing models (no schema changes)
16:35:44 <diurnalist> yes, i think that's workable. the extra capabilities functionality is a bit tricky to express this way, maybe there is a way
16:36:18 <diurnalist> plugins should not be responsible for implementing that IMO
16:38:10 <priteau> Maybe with an additional layer in the DB, a resources table that would describe which plugin is responsible for each resource
16:38:24 <priteau> And extra caps would reference this table, rather than the custom table from the plugin
16:38:42 <priteau> A bit like how there is a split between the reservations table and computehost_reservations / instance_reservations
16:41:22 <diurnalist> i'll have to look again to see how that works
16:41:49 <priteau> There might be some details that I've not considered but it's worth trying
16:43:12 <diurnalist> :nod:
16:43:33 <diurnalist> will keep kicking it around, at some point there will need to be a spec. it's more of a general direction i think things should move
16:44:17 <priteau> I am all for it if it makes your life easier
16:45:38 <priteau> Anything else to cover?
16:46:32 <diurnalist> one other thing: has anybody worked on making blazarclient osc-compatible?
16:46:46 <priteau> Good question
16:47:22 <diurnalist> i am wondering if it might be a good project for a new hire as a way of getting acquainted w/ the process. a bit hard for us to argue for its utility, but it's always bugged me :)
16:47:28 <priteau> There was a start a while ago from someone in the community, but I don't think the patches were submitted upstream
16:48:19 <priteau> I'll see if I can get the patches, but it might be difficult
16:49:01 <priteau> It could be a good project to get used to the client and the API. It's not really useful for learning about the internals of the service of course.
16:49:31 <diurnalist> i think if the patches are very old it may be worth it to just start over
16:50:01 <diurnalist> ok, i'll let you know if it gets scheduled
16:50:36 <diurnalist> otherwise nothing else from me, I don't think. there is a general question of how to handle upstream contributions given that it seems like you are the only active reviewer these days
16:51:31 <priteau> Maybe you should become core reviewer ;-) Although that doesn't really help with your own patches
16:53:10 <priteau> But if there are contributions from the new hire in your group, we could merge them with +2 from you and I
16:53:52 <diurnalist> that may be helpful
16:54:18 <priteau> I'll propose something :)
16:54:31 <diurnalist> in principle i worry about taking a bit too much extra responsibility, but in practice we are the main ones proposing things these days, and i'm reviewing that code anyways
16:55:26 <priteau> Yes I don't think it would add much workload for you
16:56:20 <priteau> Let's wrap up
16:56:25 <priteau> Thanks a lot for joining!
16:56:27 <priteau> #endmeeting