08:00:18 <ttx> #startmeeting large_scale_sig
08:00:19 <openstack> Meeting started Wed Jun 10 08:00:18 2020 UTC and is due to finish in 60 minutes.  The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot.
08:00:20 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
08:00:22 <openstack> The meeting name has been set to 'large_scale_sig'
08:00:24 <ttx> #topic Rollcall
08:00:30 <belmoreira> o/
08:00:39 <ttx> belmoreira: hi!
08:01:02 <belmoreira> hi ttx
08:01:14 <ttx> amorin mdelavergne ping
08:01:28 <mdelavergne> hi !
08:01:49 <amorin> hello!
08:02:00 <ttx> I don't see masahito connected yet
08:02:14 <ttx> Our agenda for today is at:
08:02:16 <ttx> #link https://etherpad.openstack.org/p/large-scale-sig-meeting
08:02:26 <ttx> #topic Information from PTG sessions that are relevant to the SIG
08:02:36 <ttx> Did anyone attend any PTG session relevant to this SIG?
08:02:46 <ttx> I did attend the Oslo track and gave a sTatus report on progress toward oslo.metrics
08:03:03 <ttx> We agreed that once the initial code is posted, it will be reviewed for suggestions on making it more Oslo-like, if needed
08:03:18 <ttx> Warned oslo-core people to expect it in the coming month
08:03:46 <ttx> Anyone else with news from the PTG to share ?
08:04:04 <ttx> belmoreira: were you able to attend the scientifig SIG session?
08:04:08 <mdelavergne> I only had time to go to the Edge PTG
08:04:24 <ttx> amorin: did you find time for ops meetup?
08:05:14 <amorin> nop, I wasnt able to follow it
08:05:21 <belmoreira> yes, I attended the scientific sig
08:05:41 <ttx> anything related to large scale to share?
08:05:53 <belmoreira> nothing special... trying to find the etherpad
08:06:22 <ttx> https://etherpad.opendev.org/p/victoria-ptg-scientific-sig
08:06:24 <ttx> ?
08:06:24 <belmoreira> https://etherpad.opendev.org/p/victoria-ptg-scientific-sig
08:06:28 <ttx> I win!
08:06:48 <belmoreira> :)
08:07:03 <ttx> Looks like it was set up more as Q&A
08:07:04 <belmoreira> one interesting thing that came up, was billing
08:07:48 <belmoreira> but inside openstack we don't have anything native to solve this question
08:08:13 <belmoreira> I'm sure that at large scale we all have a different way to do it
08:08:30 <ttx> yeah, the state of metrics gathering / billing is not optimal
08:09:08 <ttx> That was raised in TC discussions too
08:09:21 <ttx> Anything else on PTG?
08:09:41 <belmoreira> maybe that is something that this group can give some feedback
08:10:18 <ttx> belmoreira: yes I could see how we could help amplify that concern from the scientific SIG
08:10:55 <ttx> waiting to see how/if the TC picks up that ball
08:10:56 <belmoreira> not only that, but how we are workaround the problem
08:12:24 <ttx> belmoreira: how about we add a section to our documentation etherpad asking our members how they solve it (custom code, ceilometer, monasca, something else...)
08:12:48 <ttx> see if trends emerge
08:13:35 <belmoreira> I think that would be good. Because we all must be doing a different thing
08:13:47 <ttx> ok so I added line 33 on https://etherpad.opendev.org/p/large-scale-sig-documentation
08:14:10 <ttx> I'll log an action for us to document briefly how we do it (if that's something that can be publicly communicated)
08:14:45 <ttx> #action all to describe briefly how you solved metrics/billing in your deployment on https://etherpad.opendev.org/p/large-scale-sig-documentation
08:15:14 <ttx> OK since we already talk about it, moving to next topic
08:15:19 <ttx> #topic Progress on "Documenting large scale operations" goal
08:15:24 <ttx> #link https://etherpad.openstack.org/p/large-scale-sig-documentation
08:15:31 <ttx> The patch against Nova doc was posted:
08:15:34 <ttx> #link https://review.opendev.org/#/c/729190/
08:15:42 <ttx> The initial reaction is a bit negative, mostly due to the linked page containing so little information
08:15:52 <ttx> I was wondering if we should improve the content on the wiki page a bit, before pushing again
08:16:00 <amorin> yes, I agree with that
08:16:04 <ttx> I think we can make a good case that it is a first step, and that something is better than nothing
08:16:12 <ttx> But maybe we should improve the examples on the wiki page before we actually do that...
08:16:37 <amorin> I can try to improve the wiki page
08:16:49 <amorin> I have some content to push there, based on what we have in the etherpad
08:17:03 <amorin> it's mostly a matter of doing it
08:17:05 <ttx> ok, once you are done let me know and I'll help push it on the review
08:17:46 <ttx> #action amorin to add some meat to the wiki page before we push the Nova doc patch further
08:18:16 <ttx> Is there any other parameter we could document ? With two the benefit would be more obvious
08:18:43 <amorin> plenty of params
08:18:47 <ttx> (personally I think baby steps are good, but I'm not nova-core)
08:18:56 <amorin> for exmaple the one related to db connection
08:19:25 <amorin> or some others related to workers
08:19:33 <amorin> rpc_workers, api_workers, etc.
08:20:06 <ttx> yes those would be good. Also if we have a rule of thumb to apply to dimension those correctly, could be useful
08:20:17 <amorin> yup
08:20:25 <amorin> that's the hard part :p
08:20:56 <ttx> OK. So I think the approach is still valid, it just needs to bring more information to be included
08:21:24 <ttx> OK, anything else on this topic?
08:21:54 <amorin> nop
08:22:00 <ttx> I hope Opendev will allow us to collect more data and get more help to push this forward
08:22:06 <ttx> #topic Progress on "Scaling within one cluster" goal
08:22:10 <ttx> #link https://etherpad.openstack.org/p/large-scale-sig-cluster-scaling
08:22:30 <ttx> masahito is not around, and the code was not dropped yet
08:22:33 * ttx rechecks
08:23:12 <ttx> oh actually
08:23:26 <ttx> https://review.opendev.org/#/c/730753/
08:23:46 <masahito_> o/
08:23:50 <masahito_> sorry for the late.
08:23:54 <ttx> I'll solve the merge conflict there
08:24:08 <ttx> It's my fault as I put example content in
08:24:42 <ttx> masahito_: I will get it in and notify oslo-core people
08:25:16 <masahito_> Thanks. Sorry for the slow sharing.
08:25:27 <ttx> #action ttx to solve merge conflict and push initial oslo.metrics code in from https://review.opendev.org/#/c/730753/
08:26:07 <amorin> I will review it and maybe test it in our lab
08:26:47 <ttx> I think adding tests will be the first thing to make it more openstacky
08:27:21 <ttx> #action all to review https://review.opendev.org/#/c/730753/
08:27:38 <ttx> Great to see progress there! Thanks masahito
08:27:49 <ttx> anything else on this topic?
08:28:39 <ttx> masahito_: I'll add you as oslo.metrics core so that you can help with +2 on the code reviews
08:29:11 <ttx> as I expect  some patches from oslo-core as they look into the code
08:29:54 <ttx> and we should be in good shape to mention it during the opendev event
08:29:58 <ttx> #topic Opendev event on Large scale operations, June 29 - July 1
08:30:02 <ttx> #link https://etherpad.openstack.org/p/LargeScaleOps_OpenDev
08:30:06 <ttx> Registration is at:
08:30:28 <ttx> https://opendev_largescale.eventbrite.com/
08:30:59 <ttx> Don't forget to register :) It's free
08:31:30 <ttx> Schedule is up at https://www.openstack.org/events/opendev-2020/opendev-schedule
08:32:16 <ttx> Questions? News?
08:32:41 <belmoreira> some meeting ago I mentioned a blog regarding ironic scalability
08:33:00 <ttx> yes I remember
08:33:12 <belmoreira> we publish it like 2 weeks ago. for reference: https://techblog.web.cern.ch/techblog/post/conductor-groups/
08:33:32 <ttx> #link https://techblog.web.cern.ch/techblog/post/conductor-groups/
08:33:41 <amorin> nice
08:34:10 <ttx> amorin: you are using custom code for bare metal instances, right?
08:34:34 <amorin> yes,
08:34:48 <amorin> we are using ironic in labs and plan to open it
08:34:53 <amorin> but that's not yet ready
08:35:17 <ttx> oh, great news! Will that ultimately replace your own system, or just be in addition to it?
08:35:27 <amorin> anyway it's nice to see some scaled ironic deployment
08:35:40 <amorin> I dont think it will replace it for now
08:35:45 <amorin> but more be an addition, yes
08:35:56 <ttx> we never replace, only add :)
08:36:06 <ttx> #topic Next meeting
08:36:18 <ttx> Next meeting in two weeks... That's the week just before the Opendev event, so good timing for last-minute preparation
08:36:24 <ttx> #info next meeting: Jun 24, 8:00UTC
08:36:42 <ttx> Anything else before we close?
08:37:05 <ttx> alright! Thanks everyone
08:37:08 <belmoreira> thanks all
08:37:08 <ttx> #endmeeting