Wednesday, 2020-11-18

*** artom has quit IRC00:21
*** artom has joined #openstack-meeting-300:22
*** artom has quit IRC01:04
*** mlavalle has quit IRC01:35
*** bcafarel has quit IRC02:37
*** bcafarel has joined #openstack-meeting-303:33
*** yamamoto has quit IRC04:14
*** yamamoto has joined #openstack-meeting-304:14
*** yamamoto has quit IRC04:19
*** yamamoto has joined #openstack-meeting-304:19
*** yamamoto has quit IRC04:30
*** yamamoto has joined #openstack-meeting-304:43
*** yamamoto has quit IRC04:59
*** yamamoto has joined #openstack-meeting-305:03
*** yamamoto has quit IRC05:03
*** yamamoto has joined #openstack-meeting-305:03
*** yamamoto has quit IRC05:08
*** yamamoto has joined #openstack-meeting-307:07
*** yamamoto has quit IRC07:18
*** yamamoto has joined #openstack-meeting-307:21
*** yamamoto has quit IRC07:22
*** ralonsoh has joined #openstack-meeting-307:31
*** links has joined #openstack-meeting-307:42
*** slaweq has joined #openstack-meeting-307:43
*** yamamoto has joined #openstack-meeting-307:58
*** yamamoto has quit IRC08:12
*** links has left #openstack-meeting-308:20
*** tosky has joined #openstack-meeting-308:45
*** slaweq has quit IRC09:17
*** slaweq has joined #openstack-meeting-309:19
*** e0ne has joined #openstack-meeting-309:38
*** belmoreira has joined #openstack-meeting-310:46
*** hemanth_n has quit IRC11:04
*** lpetrut has joined #openstack-meeting-311:10
*** baojg has quit IRC11:30
*** baojg has joined #openstack-meeting-311:31
*** slaweq has quit IRC11:39
*** yamamoto has joined #openstack-meeting-311:39
*** e0ne has quit IRC11:44
*** slaweq has joined #openstack-meeting-311:45
*** e0ne has joined #openstack-meeting-312:18
*** yamamoto has quit IRC12:43
*** baojg has quit IRC12:48
*** baojg has joined #openstack-meeting-312:50
*** gibi is now known as gibi_away12:52
*** yamamoto has joined #openstack-meeting-313:05
*** yamamoto has quit IRC13:05
*** yamamoto has joined #openstack-meeting-313:05
*** yamamoto has quit IRC13:10
*** artom has joined #openstack-meeting-313:17
*** yamamoto has joined #openstack-meeting-313:26
*** yamamoto has quit IRC13:26
*** yamamoto has joined #openstack-meeting-313:27
*** yamamoto has quit IRC13:31
*** genekuo has joined #openstack-meeting-313:47
*** e0ne has quit IRC13:51
*** mdelavergne has joined #openstack-meeting-313:51
*** e0ne has joined #openstack-meeting-314:01
ttxo/14:01
genekuoo/14:01
ttx#startmeeting large_scale_sig14:01
openstackMeeting started Wed Nov 18 14:01:26 2020 UTC and is due to finish in 60 minutes.  The chair is ttx. Information about MeetBot at http://wiki.debian.org/MeetBot.14:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:01
*** openstack changes topic to " (Meeting topic: large_scale_sig)"14:01
openstackThe meeting name has been set to 'large_scale_sig'14:01
ttx#topic Rollcall14:01
*** openstack changes topic to "Rollcall (Meeting topic: large_scale_sig)"14:01
ttxWho is here for the Large Scale SIG meeting ?14:01
belmoreirao/14:01
genekuoo/14:01
ttxamorin: ?14:01
ttxmdelavergne: ?14:01
mdelavergneHi!14:02
amorinhey!14:02
amorinI am here!14:02
amorinthanks for ping :)14:02
ttxperfect, let's get started14:02
ttxOur agenda for today is at:14:02
ttx#link https://etherpad.openstack.org/p/large-scale-sig-meeting14:02
ttx#topic PTG/Summit postmortem14:02
*** openstack changes topic to "PTG/Summit postmortem (Meeting topic: large_scale_sig)"14:02
ttxA few weeks ago we had our Forum and PTG activities, and I'd like to discuss how successful that was14:03
ttxMy view on it is that the Forum session was a success with new people sharing experience14:03
ttxThe two PTG meetings were not as successful with only a couple of people showing up14:03
ttxWhat do people think?14:03
ttxPersonally, I think that means we need to separate SIG activities (the inner circle) from the sharing of experience (outer circle)14:04
genekuoYeah, I think summit have more operators attending14:04
ttxas it's a lot easier to get people to show up when you ask a specific question, than askign people to join a SIG meeting14:04
*** lajoskatona has joined #openstack-meeting-314:04
genekuoagree14:05
*** lajoskatona has left #openstack-meeting-314:05
ttxFor reference here is the link to the Forum and PTG etherpads:14:05
ttx#link https://etherpad.opendev.org/p/vSummit2020_OpenStackScalingStory14:05
ttx#link https://etherpad.opendev.org/p/wallaby-ptg-largescale-sig14:05
belmoreirattx if I remember well I didn't attend the PTG sessions because the timeslot14:05
ttxamorin, mdelavergne what did you think14:05
ttxbelmoreira: our hope was that some of the people from the week before would show up14:05
amorinI really enjoy the summit part, with a lot of operator giving feedbacks14:06
ttxbut i think that was misguided14:06
amorinunfortunately I was not avaible for PTG, so I miss that14:06
mdelavergnenothing much, sorry I was really occupied these last few weeks, I missed a lot :(14:06
amorinanyway, I have the feeling that operators have things to say14:06
amorinI dont know how we can imply them more in our meetings14:07
ttxI've been giving it some thought14:07
amorinyou want to split the group in 2 parts?14:07
ttxand we discussed that with genekuo at the PTG sessions14:07
belmoreirattx, true. But for some reason we see europeans participating more in these meetings and if I recall correctly it was at the beginning and end of the day. For example for me was impossible to attend14:07
genekuoI think we've discussed with collaborating with OpsMeetup during PTG14:07
ttxamorin: I think expecting people to join the SIG meeting and share their experience is not going to happen14:07
ttxWe need to use other types of exceptional events to raise the questions we careabout and get the feedback we need14:08
*** macz_ has joined #openstack-meeting-314:08
ttx But that will be the topic for the next meeting14:08
ttxas we have lots to cover today14:08
amorinack14:08
ttxbut yes, homework before next meeting will be to think about how we can more efficiently collect feedback14:08
belmoreirattx I'm just trying to understand why, maybe, the PTG sessions didn't have a wider participation14:08
ttxbelmoreira: yes that explains why the regulars did not show up, but not why the expected new people did not show up either :)14:09
ttx#topic Wallaby setup14:09
*** openstack changes topic to "Wallaby setup (Meeting topic: large_scale_sig)"14:09
ttxSo as we start a new cycle I'd like to discuss three things:14:09
ttxmeeting frequency, update co-chairs, and organization of workstreams14:10
ttxRegarding meeting frequency, during the PTG we discussed moving back to a single biweekly meeting14:10
ttxIt seems our experience with rotating meeting times resulted in spreading out the team without getting regular new members14:10
ttxSo I ran a poll and we arrived at that 14utc slot14:10
ttxWe might push it back one hour to 15utc to be more friendly to imtiaz (15utc was a popular time in the survey too)14:10
amorinok for me14:11
ttx(he wanted to join today but said 6am was a little early)14:11
belmoreiraI'm OK with both slots14:11
genekuoI'm ok, even a bit later at 16 utc14:11
amorinsame here, 16utc is my last slot14:11
belmoreirauh... 16 UTC can be more difficult for me14:11
ttxLet's try 15utc. I don;t want to over-adapt around someone that has not attended any meeting yet14:11
genekuono problem14:12
mdelavergne15 or 16 is fine by me14:12
ttxRegarding co-chairs, I'd like to update the list since we have not seen Pengju Jiao recently14:12
ttxAnyone interested? the role is about helping keeping track of workstream status and (when I'm not available) help chairing meetings14:12
ttxbelmoreira: interested in continuing?14:12
ttxanyone else ?14:12
belmoreirasure14:12
genekuoI'm ok14:12
*** macz_ has quit IRC14:12
genekuoto help out14:13
ttxgenekuo: ok to be chair or ok not to be chair? :)14:13
ttxok to be chair, that works14:13
genekuook to be co-chiar14:13
ttxthe more the merrier14:13
genekuo*chair14:13
ttxI'll update and replace Pengju Jiao with Gene Kuo14:13
ttxNow... Regarding organization of workstreams14:13
ttx#topic Reorganize work into a scaling journey14:14
*** openstack changes topic to "Reorganize work into a scaling journey (Meeting topic: large_scale_sig)"14:14
ttxDuring PTG with genekuo we discussed reorganizing our goals into more of a scaling journey14:14
ttxLike as you scale up, you:14:14
ttx1. configure/optimize your single cluster to handle additional load14:14
ttx2. monitor your cluster to detect strain/limits14:14
ttx3. scale up until you reach limits14:14
ttx4. scale out to multiple clusters/regions/cells once you reach limits14:14
ttxAnd our work is essentially about helping people throughout that journey14:15
ttxSo for (1) we work on documenting configuration options for large scale14:15
ttxFor (2) we work on oslo.metrics and other meaningful monitoring solutions14:15
ttxFor (3) we identify and push back low-hanging fruit limits scaling within one cluster14:15
ttx(That can include documenting how to properly shard a RabbitMQ install, for example)14:15
ttxand finally for (4) we explain the various options and how to do it14:16
ttxI feel like that makes our purpose a lot clearer. It's just about helping people at various stages of this journey.14:16
ttxAnd it does not have to be complete or anything. Every bit helps.14:16
ttxDoes that make sense to you?14:16
amorinyes14:16
belmoreirayes, I like the idea14:16
genekuoyep14:16
ttxit's a lot less abstract than setting artifical "goals" imho, and also less intimidating14:17
ttxIn terms of output format, I was thinking we could use a FAQ format. That way we can list common questions, and provide/improve answers when we have them14:17
ttxFAQs have two interesting properties:14:17
ttx- Listing questions is as important as listing answers14:17
ttx- It's OK if the FAQ is constantly work in progress14:18
ttxso again, less intimidating than a doc or a white paper that has to be "finished"14:18
ttxHow does that sound?14:18
mdelavergneyep, good idea14:18
amorinfaq on wiki for example?14:18
amorinor directly in documentation?14:18
genekuoYes, it's easier to answer questions than to write a document from scratch14:18
belmoreira+114:18
ttxyes, I was thinking refactoring the wiki to describe the journey14:18
amorinI'd like to have it in doc directly14:18
genekuoAgree with doc14:19
ttxit can still point to doc artifacts14:19
amorinusually, operators are looking at config options, deployments etc14:19
genekuoI rarely refer to wiki for OpenStack stuffs14:19
amorinif we have a page over there, it could be nice IMHO14:19
amoringenekuo same here14:19
ttxLike obviously, oslo.metrics will not live in a FAQ, it will be a proper software library14:19
ttxsame for documenting options14:19
amorinyup14:19
ttxBut the main artifact of our group should be this set of wiki pages (at least for now)14:20
ttxthat describe the journey and answer questions14:20
mdelavergnewherever this is put, we can still link it from the other anyway ?14:20
ttxpointing to other materials14:20
amorinok for me14:20
genekuosounds good14:20
ttxLike one question could be "my database is getting quite large with stale entries, what should I do about it" and point to OSarchiver14:21
amorin:)14:21
ttxpersonally I have a lot more questions than answers, so I like a format that lets me ask them, even if for now we don;t have all the answers14:21
ttxOK if that sounds good, I can work on refactoring our wiki docs14:22
genekuogood starting point :)14:22
ttx#action ttx to refactor wiki and other etherpads into that "journey" FAQ view14:22
mdelavergneyep, at some point somebody will have the answer and add it, hopefuly14:22
ttxI'll replace all our etherpads with a single one that tracks what the SIG is actively working on14:23
ttx(rather than one per stream)14:23
genekuoagreed, we have a lot of scattered pages, better reorganize or link them in one page14:23
ttxAny other suggestion on how we should reorganize how we work?14:23
genekuo:)14:24
ttxok moving on to next topic14:24
ttx#topic The road to oslo.metrics 1.014:24
*** openstack changes topic to "The road to oslo.metrics 1.0 (Meeting topic: large_scale_sig)"14:24
ttxSo one objective we have for the Wallaby cycle (so between now and April 2021) is to get to a proper oslo.metrics release14:24
ttxBefore our first 0.1 release we should have:14:25
ttx- Basic tests (https://review.opendev.org/#/c/755069/)14:25
ttx- Latest metrics code (?)14:25
ttxThen before the end of the Wallaby cycle we should have:14:25
ttx- oslo-messaging metrics code (https://review.opendev.org/#/c/761848/)14:25
ttx- Enable bandit (issue to fix with predictable path for metrics socket ?)14:25
ttx- Improve tests to get closer to 100% coverage14:25
ttxgenekuo: anything I missed?14:25
genekuottx: I basically have the latest main code merged in the last commit14:26
genekuoTHe rest of them are mostly additional metrics which I would like to add after 0.1 release14:26
ttxgenekuo: with the socket patch?14:26
genekuoYep14:26
ttxah ok14:26
ttxSo we could merge the basic tests14:26
ttxand then do a 0.1 release that you can start using in oslo.messaging14:27
genekuoAh, I should have another patch handling sigterm, but it shouldn't affect the test14:27
genekuoLet me update it later this week14:27
genekuoYes, thanks!14:27
ttxgenekuo: genekuo I'll add you to oslo-metrics-core so you can +2 my patch14:28
genekuoOK14:28
ttxok you should be able to +2a now14:28
ttxif you reload the page14:28
ttxthen I'll check what it takes to do a 0.1 release14:28
genekuothank you14:29
ttx#action genekuo to review/approve https://review.opendev.org/#/c/755069/14:29
ttx#action ttx to set up release jobs and request a 0.1 release14:29
ttxThat should keep us busy between now and next meeting14:30
ttx#topic Next meeting14:30
*** openstack changes topic to "Next meeting (Meeting topic: large_scale_sig)"14:30
ttxSo our next meeting should be December 2.14:30
ttxWe'll do 15utc, one hour later compared to today, in this channel if it's available14:30
ttx#action ttx to reschedule meeting to be biweekly on Wednesdays, 15utc14:30
amorinthere is a chance that I will not be available14:31
ttxI'd rather keep that meeting date that way we can do the next one on Dec 1614:31
ttxthen skip for holidays14:31
ttxand be back on January 1314:32
genekuoI'm ok with the timeline14:32
ttxis there anythig else we can make progress on between now and next meeting?14:33
ttxamorin: was there any progress on the OSops/OSarchiver side?14:33
amorinnop, was not able to find time for this14:34
ttxI suspect recent weeks at OVHCloud have been kind of busy14:34
amorinyes :)14:34
ttxOK anything else, anyone?14:35
ttxoh, I should write an action item for the homework14:35
ttx#action all to think about how to improve how we collect feedback14:36
ttxsince that was not a stellar success during the Victoria cycle...14:36
ttxSince we have a few more minutes, I'll talk about what we discussed in the PTG sessions14:36
ttxThe suggestion there was to make those feedback sessions more "exceptional" than a stabnding etherpad open for comments14:37
ttxLike we've had those etherpads like the scaling-stories etherpad open, with calls on the mailing-list for people to add experience to it14:37
ttxthat's just not working14:37
ttxBut when we create a session about "sharing your scaling story" at an ops meetup or a Forum, people come14:38
ttxso basically, making it an event, and focusing it on one question14:38
genekuoagree14:39
mdelavergneshould it be written or audio ?14:39
ttxCould be just piggybacking on existing events, every time they happen... or maybe create our own "event"14:39
ttxlike a monthly "question to operators" session on Zoom or whatever14:39
*** imtiazc has joined #openstack-meeting-314:40
amorinaudio/video is really nice in my opinion14:40
ttxI think IRC is very efficient for work group meeting as it's minutes are recorded and you can easily take #actions and such14:40
ttxBut not for ops sharing experiences14:40
ttxfor that audio/video seems to be a plus, or in-person14:41
ttxMore work for us to extract learnings from that, but that's better than no feedback at all14:41
belmoreiraif we keep this session (maybe more than one slot) in the forum it's a good start14:41
belmoreiraalso, a ops event... (not sure if the ops team is organizing something virtual)14:42
ttxYes, obviously we should make sure we leverage future forums. But once every 6 month might not be enough :)14:42
ttxSo that's the general idea I had, but you can noodle on it for two weeks and we'll dsicuss it again on Dec 214:42
genekuoI also vote for collaborating with ops event14:43
ttxAlso, make the SIG group more comfortable for existing attendees, rather than jump through hoops for hypothetical new members14:43
ttxthag means obviously creating an inner circle and an outer circle, but that's what we have anyway, the regulars and the others14:43
ttxYes we should track what the ops meetups group is giong to do next14:44
ttxAlright, that is all I had for today. Anything else we should be doing ?14:44
genekuottx: do ops meetup have regular irc meeting?14:45
ttxlet me see14:45
genekuowould like to join if possible14:45
imtiazcSorry for joining a bit late despite my best intentions to wake up earlier. Do we have a Zoom session as well for this meeting?14:45
ttxhttp://eavesdrop.openstack.org/#OpenStack_Ops_Meetup_Team14:45
ttximtiazc: no only IRC, makes it easier to write minutes :)14:45
genekuothanks ttx14:46
ttximtiazc: thanks for joining, we agreed to move one hour later for the next meeting14:46
ttxSo December 2, 15utc14:46
imtiazcThank you @ttx14:46
ttxI'll post the summary and logs for this meeting14:46
ttximtiazc: we have a few minutes left, maybe you can introduce yourself?14:46
ttxThen we will14:46
genekuo:)14:47
ttxor maybe we can all introduce ourselves at the same time14:48
imtiazcSure, this is Imtiaz Chowdhury, Cloud Architect from Workday. At Workday, we have been deploying OpenStack based private clouds for 7 years.14:48
genekuoHi!14:48
ttxI'm Thierry Carrez, VP Engineering at the now Open Infrastructure Foundation. I'm helping drive this group because I have an interest in getting large users to contribute their experience running openstack, and receive lots of questions from users that worry about the scaling journey and would very much like that we have great answers to that14:50
ttxSo even if I don;t have any answers, I help greasing the wheels of the group14:50
genekuoI'm Gene Kuo, working at LINE as Infrastructure engineer. Our team have been developing and operating OpenStack based private clouds to run our services.14:50
amorinI am Arnaud Morin, from OVHCloud, we are running OpenStack to provide a public cloud14:50
amorinI am involved in the team which deploy and manage the infrastructure14:51
mdelavergneHi, I'm Marie Delavergne, phd student for Inria and I work on scaling OpenStack in different ways :)14:51
belmoreiraI'm Belmiro Moreira. I work at CERN as Cloud Engineer14:51
ttximtiazc: I invite you to read the logs of this meeting, should give you a lot of insights on how we plan to reorganize work of the group for the Wallaby cycle14:53
ttxand we'll talk again in two weeks14:53
ttxAnything else before we close, anyone?14:53
genekuoNope :)14:54
ttxAlright then... thanks!14:54
ttx#endmeeting14:54
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"14:54
openstackMeeting ended Wed Nov 18 14:54:15 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:54
openstackMinutes:        http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-11-18-14.01.html14:54
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-11-18-14.01.txt14:54
openstackLog:            http://eavesdrop.openstack.org/meetings/large_scale_sig/2020/large_scale_sig.2020-11-18-14.01.log.html14:54
mdelavergnethanks everyone, see you!14:54
ttximtiazc: link to log ^14:54
ttxI'll send a summary to the list as always14:54
genekuoThanks!14:55
imtiazcThanks! Nice to meet everyone!14:55
*** bcafarel has quit IRC14:59
*** bcafarel has joined #openstack-meeting-315:04
*** artom has quit IRC15:19
*** lpetrut has quit IRC15:21
*** artom has joined #openstack-meeting-315:22
*** mlavalle has joined #openstack-meeting-315:25
*** jpward has joined #openstack-meeting-315:55
*** mdelavergne has quit IRC15:58
*** macz_ has joined #openstack-meeting-316:13
*** belmoreira has quit IRC16:27
*** e0ne has quit IRC17:21
*** e0ne has joined #openstack-meeting-317:22
*** e0ne has quit IRC17:33
*** yamamoto has joined #openstack-meeting-317:33
*** yamamoto has quit IRC17:37
*** baojg has quit IRC17:44
*** baojg has joined #openstack-meeting-317:45
*** e0ne has joined #openstack-meeting-317:57
*** ralonsoh has quit IRC18:31
*** e0ne has quit IRC18:36
*** baojg has quit IRC19:46
*** baojg has joined #openstack-meeting-319:48
*** ricolin has quit IRC20:40
*** slaweq has quit IRC20:58
*** slaweq has joined #openstack-meeting-321:31
*** yamamoto has joined #openstack-meeting-322:01
*** baojg has quit IRC22:03
*** baojg has joined #openstack-meeting-322:03
*** yamamoto has quit IRC22:10
*** yamamoto has joined #openstack-meeting-322:11
*** slaweq has quit IRC22:16
*** baojg has quit IRC23:08
*** baojg has joined #openstack-meeting-323:09
*** yamamoto has quit IRC23:32
*** raildo_ has quit IRC23:42

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!