08:00:18 #startmeeting large_scale_sig 08:00:19 Meeting started Wed Jun 10 08:00:18 2020 UTC and is due to finish in 60 minutes. The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot. 08:00:20 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 08:00:22 The meeting name has been set to 'large_scale_sig' 08:00:24 #topic Rollcall 08:00:30 o/ 08:00:39 belmoreira: hi! 08:01:02 hi ttx 08:01:14 amorin mdelavergne ping 08:01:28 hi ! 08:01:49 hello! 08:02:00 I don't see masahito connected yet 08:02:14 Our agenda for today is at: 08:02:16 #link https://etherpad.openstack.org/p/large-scale-sig-meeting 08:02:26 #topic Information from PTG sessions that are relevant to the SIG 08:02:36 Did anyone attend any PTG session relevant to this SIG? 08:02:46 I did attend the Oslo track and gave a sTatus report on progress toward oslo.metrics 08:03:03 We agreed that once the initial code is posted, it will be reviewed for suggestions on making it more Oslo-like, if needed 08:03:18 Warned oslo-core people to expect it in the coming month 08:03:46 Anyone else with news from the PTG to share ? 08:04:04 belmoreira: were you able to attend the scientifig SIG session? 08:04:08 I only had time to go to the Edge PTG 08:04:24 amorin: did you find time for ops meetup? 08:05:14 nop, I wasnt able to follow it 08:05:21 yes, I attended the scientific sig 08:05:41 anything related to large scale to share? 08:05:53 nothing special... trying to find the etherpad 08:06:22 https://etherpad.opendev.org/p/victoria-ptg-scientific-sig 08:06:24 ? 08:06:24 https://etherpad.opendev.org/p/victoria-ptg-scientific-sig 08:06:28 I win! 08:06:48 :) 08:07:03 Looks like it was set up more as Q&A 08:07:04 one interesting thing that came up, was billing 08:07:48 but inside openstack we don't have anything native to solve this question 08:08:13 I'm sure that at large scale we all have a different way to do it 08:08:30 yeah, the state of metrics gathering / billing is not optimal 08:09:08 That was raised in TC discussions too 08:09:21 Anything else on PTG? 08:09:41 maybe that is something that this group can give some feedback 08:10:18 belmoreira: yes I could see how we could help amplify that concern from the scientific SIG 08:10:55 waiting to see how/if the TC picks up that ball 08:10:56 not only that, but how we are workaround the problem 08:12:24 belmoreira: how about we add a section to our documentation etherpad asking our members how they solve it (custom code, ceilometer, monasca, something else...) 08:12:48 see if trends emerge 08:13:35 I think that would be good. Because we all must be doing a different thing 08:13:47 ok so I added line 33 on https://etherpad.opendev.org/p/large-scale-sig-documentation 08:14:10 I'll log an action for us to document briefly how we do it (if that's something that can be publicly communicated) 08:14:45 #action all to describe briefly how you solved metrics/billing in your deployment on https://etherpad.opendev.org/p/large-scale-sig-documentation 08:15:14 OK since we already talk about it, moving to next topic 08:15:19 #topic Progress on "Documenting large scale operations" goal 08:15:24 #link https://etherpad.openstack.org/p/large-scale-sig-documentation 08:15:31 The patch against Nova doc was posted: 08:15:34 #link https://review.opendev.org/#/c/729190/ 08:15:42 The initial reaction is a bit negative, mostly due to the linked page containing so little information 08:15:52 I was wondering if we should improve the content on the wiki page a bit, before pushing again 08:16:00 yes, I agree with that 08:16:04 I think we can make a good case that it is a first step, and that something is better than nothing 08:16:12 But maybe we should improve the examples on the wiki page before we actually do that... 08:16:37 I can try to improve the wiki page 08:16:49 I have some content to push there, based on what we have in the etherpad 08:17:03 it's mostly a matter of doing it 08:17:05 ok, once you are done let me know and I'll help push it on the review 08:17:46 #action amorin to add some meat to the wiki page before we push the Nova doc patch further 08:18:16 Is there any other parameter we could document ? With two the benefit would be more obvious 08:18:43 plenty of params 08:18:47 (personally I think baby steps are good, but I'm not nova-core) 08:18:56 for exmaple the one related to db connection 08:19:25 or some others related to workers 08:19:33 rpc_workers, api_workers, etc. 08:20:06 yes those would be good. Also if we have a rule of thumb to apply to dimension those correctly, could be useful 08:20:17 yup 08:20:25 that's the hard part :p 08:20:56 OK. So I think the approach is still valid, it just needs to bring more information to be included 08:21:24 OK, anything else on this topic? 08:21:54 nop 08:22:00 I hope Opendev will allow us to collect more data and get more help to push this forward 08:22:06 #topic Progress on "Scaling within one cluster" goal 08:22:10 #link https://etherpad.openstack.org/p/large-scale-sig-cluster-scaling 08:22:30 masahito is not around, and the code was not dropped yet 08:22:33 * ttx rechecks 08:23:12 oh actually 08:23:26 https://review.opendev.org/#/c/730753/ 08:23:46 o/ 08:23:50 sorry for the late. 08:23:54 I'll solve the merge conflict there 08:24:08 It's my fault as I put example content in 08:24:42 masahito_: I will get it in and notify oslo-core people 08:25:16 Thanks. Sorry for the slow sharing. 08:25:27 #action ttx to solve merge conflict and push initial oslo.metrics code in from https://review.opendev.org/#/c/730753/ 08:26:07 I will review it and maybe test it in our lab 08:26:47 I think adding tests will be the first thing to make it more openstacky 08:27:21 #action all to review https://review.opendev.org/#/c/730753/ 08:27:38 Great to see progress there! Thanks masahito 08:27:49 anything else on this topic? 08:28:39 masahito_: I'll add you as oslo.metrics core so that you can help with +2 on the code reviews 08:29:11 as I expect some patches from oslo-core as they look into the code 08:29:54 and we should be in good shape to mention it during the opendev event 08:29:58 #topic Opendev event on Large scale operations, June 29 - July 1 08:30:02 #link https://etherpad.openstack.org/p/LargeScaleOps_OpenDev 08:30:06 Registration is at: 08:30:28 https://opendev_largescale.eventbrite.com/ 08:30:59 Don't forget to register :) It's free 08:31:30 Schedule is up at https://www.openstack.org/events/opendev-2020/opendev-schedule 08:32:16 Questions? News? 08:32:41 some meeting ago I mentioned a blog regarding ironic scalability 08:33:00 yes I remember 08:33:12 we publish it like 2 weeks ago. for reference: https://techblog.web.cern.ch/techblog/post/conductor-groups/ 08:33:32 #link https://techblog.web.cern.ch/techblog/post/conductor-groups/ 08:33:41 nice 08:34:10 amorin: you are using custom code for bare metal instances, right? 08:34:34 yes, 08:34:48 we are using ironic in labs and plan to open it 08:34:53 but that's not yet ready 08:35:17 oh, great news! Will that ultimately replace your own system, or just be in addition to it? 08:35:27 anyway it's nice to see some scaled ironic deployment 08:35:40 I dont think it will replace it for now 08:35:45 but more be an addition, yes 08:35:56 we never replace, only add :) 08:36:06 #topic Next meeting 08:36:18 Next meeting in two weeks... That's the week just before the Opendev event, so good timing for last-minute preparation 08:36:24 #info next meeting: Jun 24, 8:00UTC 08:36:42 Anything else before we close? 08:37:05 alright! Thanks everyone 08:37:08 thanks all 08:37:08 #endmeeting