| *** dkehn2 has quit IRC | 00:00 | |
| *** mattw4 has quit IRC | 00:03 | |
| *** mriedem has quit IRC | 00:09 | |
| *** ijw has quit IRC | 00:30 | |
| *** ijw has joined #openstack-meeting | 00:30 | |
| *** yamamoto has joined #openstack-meeting | 00:34 | |
| *** ijw has quit IRC | 00:36 | |
| *** larainema has quit IRC | 00:42 | |
| *** Adri2000 has quit IRC | 00:46 | |
| *** Lucas_Gray has quit IRC | 00:46 | |
| *** larainema has joined #openstack-meeting | 00:49 | |
| *** Adri2000 has joined #openstack-meeting | 00:51 | |
| *** whoami-rajat has joined #openstack-meeting | 01:16 | |
| *** diablo_rojo has joined #openstack-meeting | 01:17 | |
| *** jamesmcarthur has joined #openstack-meeting | 01:25 | |
| *** yamamoto has quit IRC | 01:33 | |
| *** rubasov has quit IRC | 01:34 | |
| *** rubasov has joined #openstack-meeting | 01:37 | |
| *** baojg has joined #openstack-meeting | 01:37 | |
| *** jamesmcarthur has quit IRC | 01:43 | |
| *** Swami has quit IRC | 01:44 | |
| *** bbowen_ has quit IRC | 02:04 | |
| *** jamesmcarthur has joined #openstack-meeting | 02:14 | |
| *** hongbin has joined #openstack-meeting | 02:19 | |
| *** jamesmcarthur has quit IRC | 02:30 | |
| *** jamesmcarthur has joined #openstack-meeting | 02:43 | |
| *** rcernin has quit IRC | 02:43 | |
| *** samueldmq has quit IRC | 02:46 | |
| *** boxiang has quit IRC | 02:46 | |
| *** iyamahat__ has joined #openstack-meeting | 02:47 | |
| *** iyamahat_ has quit IRC | 02:49 | |
| *** yamahata has quit IRC | 02:50 | |
| *** boxiang has joined #openstack-meeting | 02:50 | |
| *** iyamahat__ has quit IRC | 02:53 | |
| *** lbragstad has quit IRC | 02:54 | |
| *** jamesmcarthur has quit IRC | 03:04 | |
| *** jamesmcarthur has joined #openstack-meeting | 03:06 | |
| *** yamamoto has joined #openstack-meeting | 03:20 | |
| *** jamesmcarthur has quit IRC | 03:26 | |
| *** rcernin has joined #openstack-meeting | 03:29 | |
| *** tpatil has joined #openstack-meeting | 03:43 | |
| *** jamesmcarthur has joined #openstack-meeting | 03:56 | |
| *** imsurit_ofc has joined #openstack-meeting | 04:00 | |
| *** shilpasd has joined #openstack-meeting | 04:00 | |
| *** tashiromt has joined #openstack-meeting | 04:01 | |
| tpatil | #startmeeting Masakari | 04:02 |
|---|---|---|
| openstack | Meeting started Tue May 7 04:02:05 2019 UTC and is due to finish in 60 minutes. The chair is tpatil. Information about MeetBot at http://wiki.debian.org/MeetBot. | 04:02 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 04:02 |
| *** openstack changes topic to " (Meeting topic: Masakari)" | 04:02 | |
| openstack | The meeting name has been set to 'masakari' | 04:02 |
| tpatil | Hi All | 04:02 |
| tashiromt | Hi | 04:02 |
| tpatil | I don't see Sampath online, so I have started today's meeting | 04:02 |
| tpatil | tashiromt: Hi | 04:02 |
| tpatil | Let's wait for Sampath to join this meeting | 04:03 |
| tpatil | FYI, Last week PTG was held in Denver but this time there were no sessions planned during PTG | 04:05 |
| *** e0ne has joined #openstack-meeting | 04:06 | |
| tpatil | Ok, Let's start the meeting. This meeting I don't have any agenda from my side but let's go through the usual stuff one by one | 04:07 |
| tpatil | #topic Bugs | 04:07 |
| *** openstack changes topic to "Bugs (Meeting topic: Masakari)" | 04:07 | |
| tpatil | Any bugs you want to discuss here? | 04:07 |
| tpatil | I don't see any new bugs reported in the last two weeks | 04:08 |
| tpatil | #link https://bugs.launchpad.net/masakari/+bug/1815657 | 04:08 |
| openstack | Launchpad bug 1815657 in masakari "python 3 failures - /usr/bin/masakari-wsgi and unit tests" [High,Fix committed] - Assigned to Shilpa Devharakar (shilpasd) | 04:08 |
| tashiromt | I don't have any bugs today | 04:08 |
| tpatil | This bug was fixed in Stein release so it should be marked as Fix Released | 04:09 |
| tpatil | #action Mark #bug 1815657 as fix release | 04:09 |
| tpatil | tashiromt: Ok, Thank you | 04:10 |
| tpatil | #topic Train Work items | 04:10 |
| *** openstack changes topic to "Train Work items (Meeting topic: Masakari)" | 04:10 | |
| *** e0ne has quit IRC | 04:12 | |
| tpatil | Some of the blueprints in LP are still open even though they are implemented, I will discuss about it with Sampath and take action to mark it as implemented | 04:12 |
| tpatil | #action Mark completed blueprints as implemented (samP/tpatil) | 04:12 |
| tpatil | Real time rendering of notifications in masakari-dashboard (slow progress) | 04:13 |
| tpatil | Here, we would like to show the graph of host failure workflow in masakari-dashobard in same way as done for Network topology | 04:14 |
| tpatil | Second work item we are working on is : Add support to install host monitor in devstack | 04:15 |
| *** diablo_rojo has quit IRC | 04:16 | |
| tpatil | Work in going on, planning to complete it in milestone-1 | 04:16 |
| tpatil | Anyone who wants to talk about any work item you have planned to contribute in Train cycle? | 04:17 |
| tpatil | Seems like no one at this moment | 04:19 |
| tashiromt | Today I dont have a work item for Train | 04:20 |
| tpatil | tashiromt: Ok | 04:20 |
| tpatil | If you have no other topics for discussion, let's end this meeting early | 04:20 |
| *** jamesmcarthur has quit IRC | 04:22 | |
| tpatil | Thank you all for attending this meeting, I will end this meeting now. | 04:25 |
| tpatil | #endmeeting | 04:25 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 04:25 | |
| openstack | Meeting ended Tue May 7 04:25:35 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 04:25 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-05-07-04.02.html | 04:25 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-05-07-04.02.txt | 04:25 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/masakari/2019/masakari.2019-05-07-04.02.log.html | 04:25 |
| *** ricolin has joined #openstack-meeting | 04:26 | |
| *** hongbin has quit IRC | 04:33 | |
| *** tpatil has quit IRC | 04:36 | |
| *** janki has joined #openstack-meeting | 04:37 | |
| *** yamahata has joined #openstack-meeting | 04:42 | |
| *** tashiromt has quit IRC | 05:01 | |
| *** e0ne has joined #openstack-meeting | 05:22 | |
| *** ricolin has quit IRC | 05:24 | |
| *** ijw has joined #openstack-meeting | 05:29 | |
| *** ijw_ has joined #openstack-meeting | 05:31 | |
| *** ijw_ has quit IRC | 05:33 | |
| *** ijw has quit IRC | 05:34 | |
| *** ijw has joined #openstack-meeting | 05:35 | |
| *** dkushwaha has joined #openstack-meeting | 05:37 | |
| *** radeks_ has joined #openstack-meeting | 05:43 | |
| *** ijw_ has joined #openstack-meeting | 05:53 | |
| *** e0ne has quit IRC | 05:56 | |
| *** ijw has quit IRC | 05:56 | |
| *** kopecmartin|off is now known as kopecmartin | 06:01 | |
| *** lpetrut has joined #openstack-meeting | 06:07 | |
| *** vishalmanchanda has joined #openstack-meeting | 06:14 | |
| *** radeks_ has quit IRC | 06:20 | |
| *** pcaruana has joined #openstack-meeting | 06:21 | |
| *** radeks_ has joined #openstack-meeting | 06:22 | |
| *** jbadiapa has joined #openstack-meeting | 06:33 | |
| *** slaweq has joined #openstack-meeting | 06:36 | |
| *** e0ne has joined #openstack-meeting | 06:46 | |
| *** dkushwaha has left #openstack-meeting | 06:48 | |
| *** e0ne has quit IRC | 06:53 | |
| *** armax has quit IRC | 07:03 | |
| *** tesseract has joined #openstack-meeting | 07:26 | |
| *** rcernin has quit IRC | 07:27 | |
| *** dkushwaha has joined #openstack-meeting | 07:43 | |
| *** joxyuki has joined #openstack-meeting | 07:50 | |
| *** trident has quit IRC | 08:01 | |
| *** trident has joined #openstack-meeting | 08:02 | |
| *** bhagyashris has joined #openstack-meeting | 08:03 | |
| dkushwaha | #startmeeting tacker | 08:04 |
| openstack | Meeting started Tue May 7 08:04:45 2019 UTC and is due to finish in 60 minutes. The chair is dkushwaha. Information about MeetBot at http://wiki.debian.org/MeetBot. | 08:04 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 08:04 |
| *** openstack changes topic to " (Meeting topic: tacker)" | 08:04 | |
| openstack | The meeting name has been set to 'tacker' | 08:04 |
| dkushwaha | #topic Roll Call | 08:04 |
| *** openstack changes topic to "Roll Call (Meeting topic: tacker)" | 08:04 | |
| dkushwaha | who is here for Tacker weekly meeting? | 08:05 |
| joxyuki | hi | 08:05 |
| *** rossella_s has joined #openstack-meeting | 08:06 | |
| *** takahashi-tsc has joined #openstack-meeting | 08:06 | |
| hyunsikyang | Hi | 08:06 |
| hyunsikyang | Hi jo and dkushwaha | 08:06 |
| dkushwaha | hello joxyuki hyunsikyang | 08:07 |
| joxyuki | hi, hyunsikyang | 08:07 |
| dkushwaha | How was the summit? | 08:07 |
| *** jaewook_oh has joined #openstack-meeting | 08:07 | |
| hyunsikyang | not many people, compare to last summit. | 08:07 |
| bhagyashris | O/ | 08:08 |
| joxyuki | snow at Tuesday :( | 08:08 |
| dkushwaha | hi bhagyashris | 08:08 |
| hyunsikyang | and in the tacker session, around 20~ people attended. | 08:08 |
| dkushwaha | hyunsikyang, ~20 is also good :) | 08:09 |
| dkushwaha | hyunsikyang, how was your talk? | 08:09 |
| takahashi-tsc | Hello all, this is my first participation in Tacker meeting. | 08:09 |
| hyunsikyang | Hi takahashi | 08:09 |
| JangwonLee_ | Hi | 08:09 |
| joxyuki | hi | 08:09 |
| jaewook_oh | Hello | 08:09 |
| dkushwaha | hello takahashi-tsc, welcome in Tacker Team | 08:09 |
| hyunsikyang | Um.. I only have 10 min so I brifely explain it | 08:09 |
| *** hokeeeeun has joined #openstack-meeting | 08:10 | |
| dkushwaha | hyunsikyang, yea, 10 minute is really too short | 08:10 |
| dkushwaha | hello jaewook_oh JangwonLee_ | 08:10 |
| hyunsikyang | but, some people doesn't know about tacker. so That session was good for introducing Tacker. | 08:10 |
| hyunsikyang | I get namecard from some teleco company. | 08:11 |
| dkushwaha | joxyuki, hyunsikyang , do you want to share some input(if any) from summit | 08:12 |
| hyunsikyang | I talked with fenix PTL and Fenix also consider ETSI spec to adapt it to fenix. So we also good chance to consider ETSI spec as a MANO | 08:13 |
| hyunsikyang | And | 08:13 |
| hyunsikyang | More and More , Many people focus on the container, so whatif, tacker also focus on the container related features, it also good. | 08:15 |
| hyunsikyang | and we also consider how we provides some feautures for Edge cloud collaboration kinds of things | 08:15 |
| dkushwaha | hyunsikyang, yea, we needs to prioritize C-VNF related activities | 08:15 |
| joxyuki | yeah, generally edge computing session increased. | 08:16 |
| joxyuki | it seems affected by 5G growing. | 08:17 |
| dkushwaha | joxyuki, hyunsikyang yea, 5g growing rapidly now a days | 08:18 |
| hyunsikyang | That is, We should think about features as a perspective of telco view. | 08:18 |
| joxyuki | for example, I think, Multi-VIM support might be required. | 08:19 |
| hyunsikyang | Maintenance,Edge, lightweight things.. | 08:19 |
| dkushwaha | have you got chance to talk some telco operators and there views/inputes for tacker? | 08:19 |
| joxyuki | sorry nothing from me | 08:20 |
| dkushwaha | joxyuki, what it mean by Multi-VIM(as we already supporting it)?? | 08:20 |
| joxyuki | yeah, we already support it but is it feasible for edge? | 08:20 |
| hyunsikyang | I talked with one guy who come from middle east and he asked me why tacker is not well known.. | 08:21 |
| joxyuki | I mean, for edge, such as StarlingX will be supported as VIM | 08:21 |
| hyunsikyang | And he said, these feautres is good for telco provider. but, he want to know how they adapt tacker to real environment. | 08:21 |
| dkushwaha | joxyuki, I see, need to check us more for Edge cloud things | 08:22 |
| hyunsikyang | yes. Our team also consider it how we adapt it to tacker. | 08:22 |
| hyunsikyang | I met Jo and we also talked about etsi spec. | 08:22 |
| dkushwaha | nice :) | 08:23 |
| hyunsikyang | It also good features:) | 08:23 |
| dkushwaha | Thanks joxyuki hyunsikyang for good inputs. | 08:24 |
| dkushwaha | moving next.. | 08:24 |
| hyunsikyang | you're welcome. | 08:24 |
| dkushwaha | #topic BP | 08:24 |
| *** openstack changes topic to "BP (Meeting topic: tacker)" | 08:24 | |
| dkushwaha | #topic VNF Package support BP | 08:25 |
| *** openstack changes topic to "VNF Package support BP (Meeting topic: tacker)" | 08:25 | |
| dkushwaha | #link https://review.opendev.org/#/c/582930 | 08:25 |
| joxyuki | I wrote my memo on https://etherpad.openstack.org/p/tacker-train-grooming | 08:25 |
| dkushwaha | joxyuki, I gone though the spec, and overall looks good to me as of now | 08:26 |
| dkushwaha | I have few questions | 08:26 |
| joxyuki | thanks, but biggest problem is that we need to support ETSI NFV-SOL001. | 08:27 |
| dkushwaha | joxyuki, Any approach on that ? | 08:27 |
| joxyuki | yes, my team are also plannning to address it but it will be U cycle. | 08:28 |
| joxyuki | then, at Train, just creating VNF package will be availble but not able to create VNF from it. | 08:29 |
| dkushwaha | joxyuki, I see it will be a big task | 08:29 |
| dkushwaha | joxyuki, I hope that should be ok, to move step by step. | 08:30 |
| *** hadai has joined #openstack-meeting | 08:30 | |
| dkushwaha | joxyuki, regarding VNF package do you have some plan to how deal with extracted CSAR resources ? | 08:31 |
| joxyuki | ok, I will keep to make effort | 08:31 |
| joxyuki | dkushwaha, current plan is storing VNF packages to local file system | 08:32 |
| joxyuki | such as /var/lib/tacker but it is configurable | 08:32 |
| joxyuki | and VM images shall be registerd to Glance if any. | 08:33 |
| dkushwaha | joxyuki, agree to that approach. one more approach can be... | 08:33 |
| dkushwaha | create a temp VM(may be we can call it as resource/package manager), and use it to extract CSAR bundle, upload all images/binaries/meat etc files there. And once deployment done, clean that VM | 08:35 |
| dkushwaha | It will help to keep all csar resources separate and secure. | 08:36 |
| dkushwaha | It is one of my point, but yes I wanted all your opinions on that | 08:36 |
| dkushwaha | %s/meat/meta | 08:37 |
| joxyuki | dkushwaha, thanks for your idea :) | 08:37 |
| joxyuki | I will examine it | 08:39 |
| dkushwaha | joxyuki, cool, we can discuss more in vPTG | 08:39 |
| dkushwaha | moving next.. | 08:40 |
| dkushwaha | #topic visually design TOSCA templates | 08:40 |
| *** openstack changes topic to "visually design TOSCA templates (Meeting topic: tacker)" | 08:40 | |
| dkushwaha | #link https://review.opendev.org/#/c/553684 | 08:40 |
| dkushwaha | Looking for some volunteer who can pick it up | 08:41 |
| hyunsikyang | what you mean? Trinh Nguyen want to get off this work? | 08:43 |
| dkushwaha | visually design TOSCA templates support can help to make Tacker user friendly | 08:43 |
| dkushwaha | hyunsikyang, As Trinh is busy in some other activities, so it will be good if some other can lead it | 08:44 |
| *** priteau has joined #openstack-meeting | 08:44 | |
| hyunsikyang | Ah, OK:) | 08:45 |
| dkushwaha | moving next.. | 08:45 |
| dkushwaha | #topic #Tacker vPTG | 08:46 |
| *** openstack changes topic to "#Tacker vPTG (Meeting topic: tacker)" | 08:46 | |
| dkushwaha | #link https://etherpad.openstack.org/p/Tacker-PTG-Train | 08:46 |
| dkushwaha | I am thinking to have one day vPTG on 14th May | 08:47 |
| dkushwaha | Let me know if it ok, otherwise please suggest as per your availability | 08:48 |
| *** ralonsoh has joined #openstack-meeting | 08:48 | |
| hyunsikyang | It's good to me. | 08:48 |
| dkushwaha | and timing: it will be 4 hours virtual meetup 08:00 UTC to 12:00 UTC | 08:50 |
| dkushwaha | joxyuki, bhagyashris jaewook_oh JangwonLee_ takahashi-tsc ^^ | 08:51 |
| takahashi-tsc | It's good to me, and I'd like to join vPTG. I'm interested in support ETSI. | 08:51 |
| joxyuki | almost OK, but until 11a.m. UTC | 08:51 |
| joxyuki | let me talk my topic first | 08:52 |
| dkushwaha | joxyuki, ok, sure | 08:52 |
| jaewook_oh | ok | 08:53 |
| joxyuki | very thanks team :) | 08:53 |
| bhagyashris | Agree with joxyuki | 08:53 |
| jaewook_oh | i'll attend at that time | 08:53 |
| bhagyashris | Thank you :) | 08:53 |
| dkushwaha | thanks all | 08:54 |
| dkushwaha | I will share meeting channel later | 08:54 |
| dkushwaha | #topic OpenDiscussion | 08:55 |
| *** openstack changes topic to "OpenDiscussion (Meeting topic: tacker)" | 08:55 | |
| dkushwaha | I have noting to discuss as of now. | 08:55 |
| dkushwaha | couple of more minutes left. | 08:56 |
| hyunsikyang | I will present my blueprint in the vPTG. but, before that could you check my blueprint? | 08:56 |
| hyunsikyang | https://review.opendev.org/#/c/643242/ and https://review.opendev.org/#/c/636272/ | 08:57 |
| hyunsikyang | one is related fenix. another one is for multi interface for C-VNF. | 08:57 |
| dkushwaha | hyunsikyang, sure, it is in my list, will review it | 08:57 |
| hyunsikyang | are you gonna make a etherpad for vPTG? | 08:58 |
| dkushwaha | hyunsikyang, it is https://etherpad.openstack.org/p/Tacker-PTG-Train | 08:59 |
| dkushwaha | we use this one | 08:59 |
| hyunsikyang | OK | 08:59 |
| hyunsikyang | Thanks | 08:59 |
| dkushwaha | ok | 08:59 |
| dkushwaha | Thanks Folks for joining this meeting. | 08:59 |
| dkushwaha | closing it now. | 08:59 |
| joxyuki | :) | 08:59 |
| dkushwaha | #endmeeting | 09:00 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 09:00 | |
| openstack | Meeting ended Tue May 7 09:00:02 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 09:00 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/tacker/2019/tacker.2019-05-07-08.04.html | 09:00 |
| hyunsikyang | thanks all | 09:00 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/tacker/2019/tacker.2019-05-07-08.04.txt | 09:00 |
| *** hokeeeeun has quit IRC | 09:00 | |
| openstack | Log: http://eavesdrop.openstack.org/meetings/tacker/2019/tacker.2019-05-07-08.04.log.html | 09:00 |
| *** joxyuki has quit IRC | 09:08 | |
| *** takahashi-tsc has quit IRC | 09:09 | |
| *** electrofelix has joined #openstack-meeting | 09:21 | |
| *** hadai has quit IRC | 09:24 | |
| *** bhagyashris has quit IRC | 09:25 | |
| *** jaewook_oh has quit IRC | 09:30 | |
| *** jaewook_oh has joined #openstack-meeting | 09:30 | |
| *** cheng1 has quit IRC | 09:38 | |
| *** cheng1 has joined #openstack-meeting | 09:40 | |
| *** raildo has joined #openstack-meeting | 09:54 | |
| *** zbr has joined #openstack-meeting | 10:04 | |
| *** ijw_ has quit IRC | 10:17 | |
| *** ijw has joined #openstack-meeting | 10:17 | |
| *** pcaruana has quit IRC | 10:19 | |
| *** ijw has quit IRC | 10:22 | |
| *** dkushwaha has quit IRC | 10:36 | |
| *** bbowen_ has joined #openstack-meeting | 10:41 | |
| *** bbowen_ has quit IRC | 10:50 | |
| *** pcaruana has joined #openstack-meeting | 10:55 | |
| *** ijw has joined #openstack-meeting | 10:57 | |
| *** ijw has quit IRC | 11:05 | |
| *** priteau has quit IRC | 11:11 | |
| *** carloss has joined #openstack-meeting | 11:15 | |
| *** yamamoto has quit IRC | 11:18 | |
| *** Lucas_Gray has joined #openstack-meeting | 11:22 | |
| *** imsurit_ofc has quit IRC | 11:44 | |
| *** baojg has quit IRC | 11:44 | |
| *** yamamoto has joined #openstack-meeting | 11:49 | |
| *** bbowen has joined #openstack-meeting | 11:52 | |
| *** bbowen_ has joined #openstack-meeting | 11:53 | |
| *** yamamoto has quit IRC | 11:53 | |
| *** yamamoto has joined #openstack-meeting | 11:53 | |
| *** yamamoto has quit IRC | 11:54 | |
| *** yamamoto has joined #openstack-meeting | 11:54 | |
| *** yamamoto has quit IRC | 11:54 | |
| *** bbowen has quit IRC | 11:56 | |
| *** yamamoto has joined #openstack-meeting | 11:56 | |
| *** jamesmcarthur has joined #openstack-meeting | 12:02 | |
| *** mriedem has joined #openstack-meeting | 12:05 | |
| *** larainema has quit IRC | 12:12 | |
| *** jamesmcarthur has quit IRC | 12:28 | |
| *** rfolco|ruck is now known as rfolco|dentist | 12:29 | |
| *** _hemna has quit IRC | 12:47 | |
| *** hemna has joined #openstack-meeting | 12:48 | |
| *** yamamoto has quit IRC | 12:57 | |
| *** jaewook_oh has quit IRC | 12:57 | |
| *** yamamoto has joined #openstack-meeting | 12:58 | |
| *** yamamoto has quit IRC | 13:03 | |
| *** mmethot has joined #openstack-meeting | 13:04 | |
| *** tssurya has joined #openstack-meeting | 13:05 | |
| *** shilpasd has quit IRC | 13:09 | |
| *** lseki has joined #openstack-meeting | 13:09 | |
| *** ijw has joined #openstack-meeting | 13:18 | |
| *** lbragstad has joined #openstack-meeting | 13:20 | |
| *** ijw has quit IRC | 13:23 | |
| *** bobh has joined #openstack-meeting | 13:25 | |
| *** bobh has quit IRC | 13:25 | |
| *** hemna has quit IRC | 13:25 | |
| *** hemna has joined #openstack-meeting | 13:26 | |
| *** yamamoto has joined #openstack-meeting | 13:32 | |
| *** enriquetaso has joined #openstack-meeting | 13:33 | |
| *** e0ne has joined #openstack-meeting | 13:36 | |
| *** yamamoto has quit IRC | 13:37 | |
| *** e0ne has quit IRC | 13:39 | |
| *** lpetrut has quit IRC | 13:40 | |
| *** artom has quit IRC | 13:45 | |
| *** ayoung has quit IRC | 13:49 | |
| *** ijw has joined #openstack-meeting | 13:53 | |
| *** zaneb has quit IRC | 13:53 | |
| *** shintaro has joined #openstack-meeting | 14:01 | |
| *** boxiang has quit IRC | 14:02 | |
| *** boxiang has joined #openstack-meeting | 14:03 | |
| *** Yipei has joined #openstack-meeting | 14:06 | |
| *** artom has joined #openstack-meeting | 14:12 | |
| *** janki has quit IRC | 14:14 | |
| *** artom has quit IRC | 14:18 | |
| *** pcaruana has quit IRC | 14:19 | |
| *** rfolco|dentist is now known as rfolco|ruck | 14:20 | |
| *** Yipei has quit IRC | 14:21 | |
| *** apetrich has quit IRC | 14:29 | |
| *** armax has joined #openstack-meeting | 14:37 | |
| *** pcaruana has joined #openstack-meeting | 14:38 | |
| *** apetrich has joined #openstack-meeting | 14:38 | |
| *** efried is now known as efried_pto | 14:39 | |
| *** Yipei has joined #openstack-meeting | 14:47 | |
| *** Yipei has left #openstack-meeting | 14:48 | |
| *** Yipei has joined #openstack-meeting | 14:49 | |
| *** Yipei has quit IRC | 14:49 | |
| *** cheng1__ has joined #openstack-meeting | 14:59 | |
| ralonsoh | #startmeeting neutron_qos | 15:00 |
| openstack | Meeting started Tue May 7 15:00:11 2019 UTC and is due to finish in 60 minutes. The chair is ralonsoh. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| *** openstack changes topic to " (Meeting topic: neutron_qos)" | 15:00 | |
| openstack | The meeting name has been set to 'neutron_qos' | 15:00 |
| ralonsoh | Hello | 15:00 |
| *** shintaro has quit IRC | 15:01 | |
| ralonsoh | hmmm people is still suffering the hangover of the PTG | 15:01 |
| ralonsoh | anyway, for the records | 15:02 |
| ralonsoh | #topic RFEs | 15:02 |
| *** openstack changes topic to "RFEs (Meeting topic: neutron_qos)" | 15:02 | |
| ralonsoh | #link https://bugs.launchpad.net/neutron/+bug/1578989 | 15:02 |
| openstack | Launchpad bug 1578989 in neutron "[RFE] Strict minimum bandwidth support (egress)" [Wishlist,Fix released] - Assigned to Bence Romsics (bence-romsics) | 15:02 |
| ralonsoh | Both tempest and fullstack tests are still under development | 15:02 |
| ralonsoh | #link https://review.openstack.org/#/c/629253/ | 15:02 |
| ralonsoh | #link https://review.openstack.org/#/c/642527/ | 15:02 |
| ralonsoh | Docs are merged | 15:02 |
| ralonsoh | Nova API must be polished this release | 15:03 |
| ralonsoh | But great job, again | 15:03 |
| ralonsoh | next one | 15:03 |
| ralonsoh | #link https://bugs.launchpad.net/neutron/+bug/1560963 | 15:03 |
| openstack | Launchpad bug 1560963 in neutron "[RFE] Minimum bandwidth support (egress)" [Wishlist,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez) | 15:03 |
| ralonsoh | I didn't have time to finish it during the PTG | 15:03 |
| ralonsoh | I have some patches upstream | 15:03 |
| ralonsoh | https://review.opendev.org/#/q/status:open+project:openstack/neutron+branch:master+topic:bug/1560963 | 15:04 |
| ralonsoh | But the final patch to use VXLAN filtering in QoS is still under development | 15:04 |
| ralonsoh | next one | 15:04 |
| ralonsoh | #link https://bugs.launchpad.net/neutron/+bug/1818479 | 15:04 |
| openstack | Launchpad bug 1818479 in neutron "RFE Decouple placement reporting service plugin from ML2" [Wishlist,Incomplete] - Assigned to Bence Romsics (bence-romsics) | 15:04 |
| ralonsoh | This one is going to be discussed and followed in the drivers meeting | 15:04 |
| ralonsoh | but because is QoS related, we'll track it here | 15:05 |
| ralonsoh | #topic Bugs | 15:05 |
| *** openstack changes topic to "Bugs (Meeting topic: neutron_qos)" | 15:05 | |
| ralonsoh | #link https://bugs.launchpad.net/neutron/+bug/1826695 | 15:05 |
| openstack | Launchpad bug 1826695 in neutron "[L3][QoS] cache does not removed when router is down or deleted" [Medium,In progress] - Assigned to LIU Yulong (dragon889) | 15:05 |
| ralonsoh | There is an ongoing patch | 15:05 |
| ralonsoh | https://review.opendev.org/#/c/656105/ | 15:05 |
| ralonsoh | #link https://review.opendev.org/#/c/656105/ | 15:05 |
| ralonsoh | #topic Open Discussion | 15:06 |
| *** openstack changes topic to "Open Discussion (Meeting topic: neutron_qos)" | 15:06 | |
| ralonsoh | #link https://bugs.launchpad.net/neutron/+bug/1476527 | 15:06 |
| openstack | Launchpad bug 1476527 in neutron "[RFE] Add common classifier resource" [Wishlist,Triaged] - Assigned to Igor D.C. (igordcard) | 15:06 |
| ralonsoh | I'm still reviewing https://review.opendev.org/#/c/636333 | 15:06 |
| ralonsoh | I don't know how to decouple this feature from Neutron, making it totally independent from the Neutron code | 15:06 |
| ralonsoh | doing something like a service pluging | 15:07 |
| ralonsoh | the problem | 15:07 |
| ralonsoh | 1) is this feature depends on QoS | 15:07 |
| ralonsoh | 2) requires a DB modification | 15:07 |
| *** rfolco|ruck is now known as rfolco|lunch | 15:08 | |
| ralonsoh | 3) the agent QoS extension drivers should be aware of this new pluging | 15:08 |
| ralonsoh | --> I don't see a method of making this feature independent from Neutron | 15:09 |
| ralonsoh | and that's all for today | 15:10 |
| ralonsoh | see you in two weeks, bye! | 15:10 |
| ralonsoh | #endmeeting | 15:10 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 15:10 | |
| openstack | Meeting ended Tue May 7 15:10:50 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:10 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/neutron_qos/2019/neutron_qos.2019-05-07-15.00.html | 15:10 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/neutron_qos/2019/neutron_qos.2019-05-07-15.00.txt | 15:10 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/neutron_qos/2019/neutron_qos.2019-05-07-15.00.log.html | 15:10 |
| *** pcaruana has quit IRC | 15:16 | |
| *** ygbo has joined #openstack-meeting | 15:19 | |
| *** rfolco|lunch is now known as rfolco|ruck|free | 15:22 | |
| *** rfolco|ruck|free is now known as rfolco|ruck | 15:22 | |
| *** jamesmcarthur has joined #openstack-meeting | 15:24 | |
| *** armstrong has joined #openstack-meeting | 15:24 | |
| *** artom has joined #openstack-meeting | 15:27 | |
| *** ijw has quit IRC | 15:28 | |
| *** cheng1__ has quit IRC | 15:40 | |
| *** bobh has joined #openstack-meeting | 15:45 | |
| *** Lucas_Gray has quit IRC | 15:49 | |
| *** gyee has joined #openstack-meeting | 15:55 | |
| slaweq | #startmeeting neutron_ci | 16:00 |
| openstack | Meeting started Tue May 7 16:00:19 2019 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| *** openstack changes topic to " (Meeting topic: neutron_ci)" | 16:00 | |
| openstack | The meeting name has been set to 'neutron_ci' | 16:00 |
| *** mlavalle has joined #openstack-meeting | 16:00 | |
| slaweq | hi | 16:00 |
| mlavalle | o/ | 16:00 |
| haleyb | hi | 16:00 |
| slaweq | lets wait couple more minutes for njohnston_ ralonsoh and others, maybe they will join | 16:01 |
| *** ygbo has quit IRC | 16:01 | |
| *** e0ne has joined #openstack-meeting | 16:02 | |
| slaweq | ok, lets start | 16:03 |
| slaweq | first thing: | 16:03 |
| slaweq | #link http://grafana.openstack.org/d/Hj5IHcSmz/neutron-failure-rate?orgId=1 | 16:03 |
| slaweq | please open now so it will be ready later :) | 16:03 |
| slaweq | #topic Actions from previous meetings | 16:03 |
| *** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)" | 16:03 | |
| mlavalle | ok | 16:03 |
| slaweq | first action from 2 weeks ago was | 16:03 |
| slaweq | mlavalle to continue debuging reasons of neutron-tempest-plugin-dvr-multinode-scenario failures | 16:04 |
| mlavalle | I didn't make progress on this one | 16:04 |
| mlavalle | due to Summit / PTG | 16:04 |
| slaweq | sure, I know :) Can I assign it to You for next week? | 16:04 |
| mlavalle | yes please | 16:04 |
| slaweq | #action mlavalle to continue debuging reasons of neutron-tempest-plugin-dvr-multinode-scenario failures | 16:04 |
| slaweq | thx | 16:04 |
| slaweq | mlavalle to recheck tcpdump patch and analyze output from ci jobs | 16:05 |
| slaweq | that is next one ^^ | 16:05 |
| slaweq | any update? | 16:05 |
| mlavalle | I spent time this morning looking at that | 16:05 |
| mlavalle | I think my tcpdump command is too broad: http://logs.openstack.org/21/653021/2/check/neutron-tempest-plugin-dvr-multinode-scenario/0a24d77/controller/logs/screen-q-l3.txt.gz#_Apr_23_00_47_57_853357 | 16:06 |
| mlavalle | getting weird output as you can see | 16:06 |
| mlavalle | bad checksums and that kind of stuff | 16:06 |
| mlavalle | so I am going to focus a little bit more | 16:06 |
| mlavalle | I am going to trace qr and qg interfaces | 16:07 |
| mlavalle | with tcp and port 22 | 16:07 |
| mlavalle | makes sense? | 16:07 |
| slaweq | and please add "-n" option to not resolv IPs to hostnames | 16:07 |
| slaweq | IMHO it will be easier to look | 16:07 |
| mlavalle | yes, you are right | 16:08 |
| mlavalle | I will probably also focus on singe node jobs first | 16:08 |
| haleyb | and maybe -l to not buffer | 16:08 |
| slaweq | yes, and also "-e" to print mac addresses | 16:08 |
| mlavalle | thanks for the recommendations. I will follow them | 16:09 |
| mlavalle | I have a stupid question | 16:09 |
| mlavalle | can I ask it? | 16:09 |
| slaweq | sure :) | 16:09 |
| slaweq | there are no stupid questions ;) | 16:09 |
| mlavalle | in looking at what job to focus in kibana, I noticed this: http://logs.openstack.org/66/641866/10/check/neutron-tempest-dvr-ha-multinode-full/ec1105b/job-output.txt#_2019-05-07_14_03_22_454517 | 16:10 |
| mlavalle | we are not using password ssh in any case, right? | 16:10 |
| slaweq | no, ssh key is used always I think | 16:11 |
| mlavalle | yes, that's what I think also | 16:11 |
| mlavalle | but it was worth asking the dumb question | 16:11 |
| *** beagles has quit IRC | 16:11 | |
| slaweq | this is exactly example of this second "type" of errors with SSH connectivity | 16:12 |
| slaweq | look at http://logs.openstack.org/66/641866/10/check/neutron-tempest-dvr-ha-multinode-full/ec1105b/job-output.txt#_2019-05-07_14_03_22_550727 | 16:12 |
| slaweq | instance-id was received properly from metadata server | 16:12 |
| slaweq | but then 2 lines below, failed to get public-key | 16:12 |
| slaweq | it is exactly what I was testing last week | 16:13 |
| mlavalle | do you have a feel of what is the ratio between type 1 and type 2 failures? | 16:13 |
| *** beagles has joined #openstack-meeting | 16:13 | |
| slaweq | I don't know exactly but I would say 50:50 | 16:13 |
| mlavalle | I think the tcpdump testing I'm doing should help with type 1 | 16:14 |
| mlavalle | so I will focus on those | 16:14 |
| mlavalle | in type 2, we know we have connectivity | 16:14 |
| slaweq | yes | 16:14 |
| mlavalle | because we fail athenticating | 16:14 |
| *** jamesmcarthur has quit IRC | 16:15 | |
| slaweq | in this case problem is with slow answer for metadata requests | 16:15 |
| slaweq | http://logs.openstack.org/66/641866/10/check/neutron-tempest-dvr-ha-multinode-full/ec1105b/compute2/logs/screen-q-meta.txt.gz#_May_07_13_15_02_833919 | 16:17 |
| slaweq | here is this failed request in neutron metadata agent's logs | 16:18 |
| slaweq | and http://logs.openstack.org/66/641866/10/check/neutron-tempest-dvr-ha-multinode-full/ec1105b/controller/logs/screen-n-api-meta.txt.gz#_May_07_13_15_04_543966 -- that how it looks in nova | 16:18 |
| slaweq | 10 seconds gap in logs there | 16:18 |
| mlavalle | yeap | 16:18 |
| slaweq | mlavalle: maybe You can try to talk with someone from nova team to look into those issues | 16:19 |
| mlavalle | ok | 16:19 |
| slaweq | thx | 16:19 |
| slaweq | can I add it as an action for You also? | 16:20 |
| mlavalle | yes | 16:20 |
| slaweq | #action mlavalle to talk with nova folks about slow responses for metadata requests | 16:20 |
| mlavalle | please | 16:20 |
| slaweq | thx | 16:20 |
| slaweq | ok, next one was | 16:20 |
| slaweq | njohnston move wsgi jobs to check queue nonvoting | 16:20 |
| slaweq | I know it's done, we have wsgi jobs running in check queue currently | 16:21 |
| slaweq | and tempest job is kinda broken now | 16:21 |
| *** manjeets__ is now known as manjeets | 16:21 | |
| slaweq | so we will have to investigate it also | 16:21 |
| slaweq | but that isn't very urgent for now | 16:21 |
| slaweq | next one then | 16:22 |
| slaweq | ralonsoh to debug issue with neutron_tempest_plugin.api.admin.test_network_segment_range test | 16:22 |
| slaweq | I don't know if ralonsoh did anything with it | 16:22 |
| ralonsoh | sorry | 16:22 |
| ralonsoh | I didn't have time for it | 16:22 |
| slaweq | sure, no problem :) | 16:22 |
| slaweq | can I assign it to You for this week? | 16:23 |
| ralonsoh | sure | 16:23 |
| slaweq | #action ralonsoh to debug issue with neutron_tempest_plugin.api.admin.test_network_segment_range test | 16:23 |
| slaweq | thx ralonsoh :) | 16:23 |
| slaweq | and the last one was: | 16:23 |
| slaweq | slaweq to cancel next week meeting | 16:23 |
| slaweq | done - that was easy :P | 16:23 |
| slaweq | ok, any questions/comments? | 16:23 |
| slaweq | ok, I will take this silence as no :) | 16:24 |
| slaweq | so lets move on then | 16:24 |
| mlavalle | +1 | 16:24 |
| slaweq | #topic Stadium projects | 16:24 |
| *** openstack changes topic to "Stadium projects (Meeting topic: neutron_ci)" | 16:24 | |
| *** zaneb has joined #openstack-meeting | 16:24 | |
| slaweq | first "Python 3 migration" | 16:25 |
| slaweq | etherpad: https://etherpad.openstack.org/p/neutron_stadium_python3_status | 16:25 |
| *** whoami-rajat has quit IRC | 16:25 | |
| slaweq | I know that tidwellr started doing something with neutron-dynamic-routing repo | 16:25 |
| slaweq | his patch https://review.opendev.org/#/c/657409/ | 16:26 |
| slaweq | I just checked that for neutron-lib we are actually good | 16:27 |
| *** vishalmanchanda has quit IRC | 16:28 | |
| slaweq | I will try to go through those projects in next weeks | 16:28 |
| slaweq | anyone wants to add something in this topic? | 16:29 |
| mlavalle | nope | 16:29 |
| slaweq | ok, lets move on | 16:29 |
| slaweq | tempest-plugins migration | 16:30 |
| slaweq | Etherpad: https://etherpad.openstack.org/p/neutron_stadium_move_to_tempest_plugin_repo | 16:30 |
| slaweq | and I have a question here | 16:30 |
| slaweq | I was recently struggling with error on jobs run on rocky and queens repos for networking-bgpvpn | 16:30 |
| slaweq | but at the airport yesterday I realized that we probably don't need to run those jobs for stable branches yet | 16:31 |
| slaweq | as we will not remove tests from stable branches from stadium projects' repos, right? | 16:31 |
| slaweq | so we should only have this jobs in neutron-tempest-plugin repo for master branch for now and add stable branches jobs starting from Train release | 16:32 |
| slaweq | is that correct? | 16:32 |
| mlavalle | I think so | 16:32 |
| slaweq | ok, so that will make at least my patch easier :) | 16:33 |
| slaweq | I will remove jobs for stable branches from it and it will be ready for review than | 16:33 |
| slaweq | any other questions/updates? | 16:34 |
| mlavalle | not from me | 16:35 |
| slaweq | ok, so lets move on then | 16:35 |
| slaweq | #topic Grafana | 16:35 |
| *** openstack changes topic to "Grafana (Meeting topic: neutron_ci)" | 16:35 | |
| slaweq | http://grafana.openstack.org/dashboard/db/neutron-failure-rate - just a reminder :) | 16:35 |
| slaweq | there isn't anything very bad there - all looks pretty same as usual | 16:36 |
| *** njohnsto_ has joined #openstack-meeting | 16:37 | |
| mlavalle | yeap | 16:37 |
| mlavalle | I think so | 16:37 |
| slaweq | do You so anything what You want to talk about? | 16:37 |
| haleyb | is there a bug for the slow job failure? looks like a volume issue? | 16:38 |
| slaweq | haleyb: issues with volume (or volume backup) happens in various tempest jobs quite often | 16:38 |
| slaweq | haleyb: do You have link to example? | 16:38 |
| haleyb | http://logs.openstack.org/57/656357/1/check/tempest-slow-py3/900d859/testr_results.html.gz | 16:39 |
| haleyb | second failure | 16:39 |
| slaweq | yes, such errors happens from time to time | 16:39 |
| slaweq | I'm not sure if exactly this one was reported to cinder but I was reporting some similar errors already | 16:40 |
| *** njohnsto_ has quit IRC | 16:40 | |
| slaweq | and we also discussed about it in QA session on PTG | 16:40 |
| haleyb | just seemed like it picked up recently in the gate, at 20% now | 16:40 |
| slaweq | I hope You all saw recent email from gmann about it | 16:40 |
| *** whoami-rajat has joined #openstack-meeting | 16:43 | |
| slaweq | haleyb: I'm not sure if those 20% are only because of this issue | 16:43 |
| slaweq | often there are also problems with ssh to instances | 16:43 |
| slaweq | ok, lets move on to the next topic | 16:44 |
| haleyb | slaweq: yes, that was in the other test failure, it's just at 2x the other jobs for failures | 16:44 |
| slaweq | haleyb: where tempest-slow is 2x the other jobs failures? I don't see it to be so high | 16:44 |
| *** e0ne has quit IRC | 16:45 | |
| haleyb | slaweq: argh, it's number of jobs run, i was looking at the right side... | 16:45 |
| slaweq | ahh :) | 16:45 |
| slaweq | but that is kinda strange that this job was so many times :) | 16:46 |
| haleyb | right, they should all be the same | 16:46 |
| slaweq | yes, I will check if this graph is properly defined in grafana | 16:46 |
| slaweq | it is not | 16:47 |
| slaweq | it counts jobs from check queue instead of gate queue | 16:47 |
| slaweq | I will fix that | 16:47 |
| slaweq | #action slaweq to fix number of tempest-slow-py3 jobs in grafana | 16:48 |
| slaweq | thx haleyb for pointing this :) | 16:48 |
| haleyb | :) | 16:48 |
| slaweq | ok, lets move on then | 16:48 |
| slaweq | #topic fullstack/functional | 16:48 |
| *** openstack changes topic to "fullstack/functional (Meeting topic: neutron_ci)" | 16:49 | |
| slaweq | we still have quite high failure rates for those jobs :/ | 16:49 |
| slaweq | for functional tests quite often we still hits bug https://bugs.launchpad.net/neutron/+bug/1823038 | 16:49 |
| openstack | Launchpad bug 1823038 in neutron "Neutron-keepalived-state-change fails to check initial router state" [High,Confirmed] | 16:49 |
| slaweq | like e.g. in * http://logs.openstack.org/64/656164/1/gate/neutron-functional/c59dd7c/testr_results.html.gz | 16:49 |
| *** yamahata has quit IRC | 16:50 | |
| slaweq | and I have a question to You about that | 16:50 |
| slaweq | some time ago I did https://github.com/openstack/neutron/commit/8fec1ffc833eba9b3fc5f812bf881f44b4beba0c | 16:51 |
| slaweq | to address this race condition between keepalived and neutron-keepalived-state-change | 16:51 |
| slaweq | and it works fine for me locally | 16:51 |
| slaweq | but in the gate for some (unknown for me) reason, this initial check of status is failing with error like http://logs.openstack.org/64/656164/1/gate/neutron-functional/c59dd7c/controller/logs/journal_log.txt.gz#_May_07_11_21_05 | 16:52 |
| slaweq | I have no idea why it is like that | 16:52 |
| slaweq | maybe You can take a look into that and help me with it | 16:53 |
| haleyb | slaweq: sorry, tuned out for a second, will look | 16:53 |
| slaweq | I send today some DNM patch https://review.opendev.org/#/c/657565/ to check if this binary is really in .tox/dsvm-functional/bin directory | 16:53 |
| slaweq | and it is there | 16:54 |
| mlavalle | thanks haleyb. | 16:54 |
| slaweq | thanks haleyb | 16:55 |
| haleyb | slaweq: hmm, privsep-helper not found? that's odd | 16:55 |
| mlavalle | fwiw, I've been noting in the devstack I run in my mac (1 controller / network, 1 compute, DVR) that my HA routers sometimes has 2 masters | 16:55 |
| haleyb | slaweq: project-config fix @ https://review.opendev.org/657646 :) | 16:56 |
| mlavalle | that happens after I restart the deployment | 16:56 |
| slaweq | mlavalle: that is probably different issue | 16:56 |
| slaweq | when this "my race" happend there were 2 standby routers instead of masters | 16:57 |
| slaweq | haleyb: thx | 16:57 |
| mlavalle | ok | 16:58 |
| slaweq | haleyb: but in my DNM patch I tried to install oslo.privep simply | 16:58 |
| mlavalle | I'll try to debug it then | 16:58 |
| slaweq | and it looks that for python2.7 job it can find it now | 16:58 |
| slaweq | but there is another error there still :/ | 16:58 |
| slaweq | I will have to look into it | 16:58 |
| *** radeks_ has quit IRC | 16:59 | |
| slaweq | and this is odd because it works fine for me locally, and I also don't think there are such errors in e.g. tempest jobs | 16:59 |
| slaweq | so this issue is now strictly related to functional jobs IMO | 16:59 |
| slaweq | ok, I think we are running out of time now | 16:59 |
| slaweq | thx for attending and see You next week | 17:00 |
| slaweq | #endmeeting | 17:00 |
| ralonsoh | bye | 17:00 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 17:00 | |
| openstack | Meeting ended Tue May 7 17:00:08 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 17:00 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-05-07-16.00.html | 17:00 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-05-07-16.00.txt | 17:00 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-05-07-16.00.log.html | 17:00 |
| *** njohnston_ is now known as njohnston | 17:01 | |
| *** armax has quit IRC | 17:02 | |
| *** iyamahat has joined #openstack-meeting | 17:03 | |
| *** iyamahat_ has joined #openstack-meeting | 17:05 | |
| *** iyamahat has quit IRC | 17:08 | |
| *** mlavalle has left #openstack-meeting | 17:08 | |
| *** bobh has quit IRC | 17:14 | |
| *** armax has joined #openstack-meeting | 17:18 | |
| *** ttsiouts has joined #openstack-meeting | 17:22 | |
| *** yamahata has joined #openstack-meeting | 17:24 | |
| *** bobh has joined #openstack-meeting | 17:42 | |
| *** kopecmartin is now known as kopecmartin|off | 17:46 | |
| *** ttsiouts has quit IRC | 17:47 | |
| *** electrofelix has quit IRC | 17:47 | |
| *** ttsiouts has joined #openstack-meeting | 17:48 | |
| *** armax has quit IRC | 17:51 | |
| *** lbragstad has quit IRC | 17:59 | |
| *** lbragstad has joined #openstack-meeting | 17:59 | |
| *** ralonsoh has quit IRC | 18:00 | |
| *** mattw4 has joined #openstack-meeting | 18:06 | |
| *** ijw has joined #openstack-meeting | 18:08 | |
| *** armax has joined #openstack-meeting | 18:12 | |
| *** ttsiouts has quit IRC | 18:14 | |
| *** ijw has quit IRC | 18:14 | |
| *** ijw has joined #openstack-meeting | 18:14 | |
| *** ijw has quit IRC | 18:15 | |
| *** ijw has joined #openstack-meeting | 18:16 | |
| *** ijw has quit IRC | 18:17 | |
| *** ijw has joined #openstack-meeting | 18:17 | |
| *** armax has quit IRC | 18:17 | |
| *** armstrong has quit IRC | 18:18 | |
| *** ijw has quit IRC | 18:19 | |
| *** ijw has joined #openstack-meeting | 18:20 | |
| *** ttsiouts has joined #openstack-meeting | 18:29 | |
| *** ttsiouts has quit IRC | 18:35 | |
| *** ayoung has joined #openstack-meeting | 18:36 | |
| *** Shrews has joined #openstack-meeting | 18:40 | |
| *** zbr is now known as zbr|pto | 18:45 | |
| *** _hemna has joined #openstack-meeting | 18:51 | |
| *** ayoung has quit IRC | 18:54 | |
| *** bobh has quit IRC | 18:55 | |
| *** ayoung has joined #openstack-meeting | 18:56 | |
| clarkb | Infra meeting time | 19:00 |
| clarkb | anyone else here for that? | 19:00 |
| Shrews | o/ | 19:00 |
| corvus | o/ | 19:00 |
| dtroyer | o/ | 19:00 |
| clarkb | #startmeeting infra | 19:01 |
| openstack | Meeting started Tue May 7 19:01:09 2019 UTC and is due to finish in 60 minutes. The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot. | 19:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 19:01 |
| *** openstack changes topic to " (Meeting topic: infra)" | 19:01 | |
| openstack | The meeting name has been set to 'infra' | 19:01 |
| ianw | o/ | 19:01 |
| clarkb | #link http://lists.openstack.org/pipermail/openstack-infra/2019-May/006360.html | 19:01 |
| clarkb | #topic Announcements | 19:01 |
| *** openstack changes topic to "Announcements (Meeting topic: infra)" | 19:01 | |
| *** ttsiouts has joined #openstack-meeting | 19:02 | |
| clarkb | I've got a really weird week coming up between recovering from being away from home for 9 days, my wife is having minor surgery on Wednesday, and somewhere in there I'm visiting the dentist too. Mostly just a heads up that I may not be around during normal hours | 19:02 |
| clarkb | I get a feeling that for many of us that were in denver this is also true | 19:03 |
| corvus | this is true for me, though the particulars are different | 19:03 |
| fungi | yeah | 19:04 |
| fungi | best to assume it's a fairly weird week for everyone | 19:04 |
| fungi | and i'll be entirely afk thursday through monday | 19:04 |
| fungi | for vacation | 19:04 |
| clarkb | oh also I've got a shower that leaks | 19:05 |
| clarkb | though I think I'm just gonna have ap lumber deal with that so I can do the other stuff | 19:05 |
| clarkb | #topic Actions from last meeting | 19:05 |
| *** openstack changes topic to "Actions from last meeting (Meeting topic: infra)" | 19:05 | |
| clarkb | #link http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-04-23-19.01.txt minutes from last meeting | 19:05 |
| clarkb | It has been a while since we had a meeting and we were all recovering from post opendev migration in that meeting | 19:05 |
| clarkb | May as well jump ahead straight into the fun stuff I think | 19:06 |
| clarkb | #topic Priority Efforts | 19:06 |
| *** openstack changes topic to "Priority Efforts (Meeting topic: infra)" | 19:06 | |
| clarkb | #topic OpenDev | 19:06 |
| *** openstack changes topic to "OpenDev (Meeting topic: infra)" | 19:06 | |
| *** ttsiouts has quit IRC | 19:06 | |
| clarkb | At the PTG fungi, corvus, mordred, ttx and myself were able to have some discussions around the high level needs for OpenDev | 19:07 |
| clarkb | #link https://etherpad.openstack.org/p/2019-denver-ptg-infra-planning PTG Discussion notes | 19:07 |
| mordred | it was good | 19:07 |
| clarkb | That has some notes on the thoughts for this. Basically we would like to have namespace owners and delegate responsibility for what goes into a namespace to them | 19:07 |
| corvus | clarkb: lines 59-69? | 19:08 |
| clarkb | yup | 19:08 |
| clarkb | We were thinking that we would create a meta config repo per namespace to track and manage that | 19:08 |
| corvus | can folks not at the discuss follow lines 59-69? or do we need to write that up in more words (as a ML post or spec?) | 19:08 |
| corvus | not at the *discussion* | 19:08 |
| clarkb | I half expect we'll need to write that up as a spec | 19:09 |
| clarkb | but I agree, would be good to get input from folks that weren't able to be at the PTG. | 19:10 |
| mordred | yeah. I imagine it will be basically what's in those lines with more conjunctions and stuff | 19:10 |
| clarkb | I'm happy to write up a recap email using more words too (which may help me remember things better as well) | 19:10 |
| mordred | maybe more pronouns and adverbs thrown in | 19:10 |
| *** tesseract has quit IRC | 19:10 | |
| *** pabelanger has joined #openstack-meeting | 19:11 | |
| clarkb | Other opendev related items: We need to fix our gitea deployment so that we can upgrade gitea to 1.8.0 + corvus' patch | 19:11 |
| clarkb | Deleting the old cgit servers is on my todo list as well | 19:11 |
| mordred | ++ | 19:11 |
| mordred | the update to the test jobs is getting *closer8 | 19:12 |
| clarkb | ya thats the next topic of discussion maybe we should jump ahead and I'll take on the todo of writing a recap email in the near future (to add more words to our notes on the therpad) | 19:12 |
| corvus | the hangup on gitea is https://review.opendev.org/655522 | 19:12 |
| corvus | tests for that change caught a legit behavior change which needs to be investigated | 19:13 |
| clarkb | #link https://review.opendev.org/655522 Upgrades gitea to 1.8.0 but our tests discovered a legit behavior change that needs investigating | 19:13 |
| *** yamamoto has joined #openstack-meeting | 19:13 | |
| mordred | yeah. we're also making updates to that test job in https://review.opendev.org/#/c/656871/ - which is orthogonal to the behavior change | 19:13 |
| clarkb | #action clarkb Write PTG (and summit) recap email to capture our discussions there | 19:13 |
| clarkb | #topic Update Configuration Management | 19:14 |
| *** openstack changes topic to "Update Configuration Management (Meeting topic: infra)" | 19:14 | |
| clarkb | Another major topic of discussion was how to get from where we are today with cron + ansible to zuul + ansible for deploying our systems | 19:14 |
| corvus | lines 46-57 on the etherpad | 19:14 |
| clarkb | #link https://review.opendev.org/#/c/656871/ Split base playbook into many playbooks | 19:15 |
| corvus | 656871 implements lines 47-50 | 19:15 |
| mordred | it is currently unhappy making the root account in the gitea install for reasons I don't grok yet | 19:15 |
| clarkb | This change is the result of those discussions. The general idea being that if we have finer grained playbooks we can have zuul run those when appropriate without needing to run the whole thing every time | 19:15 |
| mordred | ++ | 19:15 |
| clarkb | The step after that chnage will be to add zuul jobs to run those playbooks | 19:15 |
| corvus | that'll be important for infra-core folks to know about, since once we land that, we'll want to keep going in that direction and no longer overload the base playbook. | 19:16 |
| clarkb | ++ | 19:16 |
| mordred | it's very exciting progress | 19:16 |
| clarkb | I think we expect to have a cron of some sort (perhaps zuul periodic pipeline driven) to ensure base level things like firewalls are up to date across the board | 19:17 |
| clarkb | but have most service level stuff driven by specific zuul updates | 19:17 |
| clarkb | Also on the config management side of the house we are ready to do our last set of puppet 4 updates. So I may bug people for reviews on those once I am properly back to being able to watch that | 19:18 |
| clarkb | Once that is done we can cleanup a bunch of our old puppet 3 jobs | 19:18 |
| clarkb | which should hopefully make testing quicker and more reliable for our system changes | 19:18 |
| fungi | also implications that we'll be able to (eventually) drop the puppet apply and beaker jobs | 19:18 |
| fungi | once we're exercising them all via the ansible run jobs | 19:19 |
| mordred | yeah. because then puppet becomes an impl detail of the playbook for the service | 19:19 |
| clarkb | And finally, I think our docker image jobs are pretty robust at this point but we still do sometimes have jobs fail because they try to authenticate to dockerhub as if they are our local registry. I've got an agenda item for this towards the end of the meeting | 19:19 |
| clarkb | #topic Storyboard | 19:20 |
| *** openstack changes topic to "Storyboard (Meeting topic: infra)" | 19:20 | |
| clarkb | we discovered that we had improperly created some storyboard groups at the summit | 19:20 |
| clarkb | fungi got those sorted out | 19:20 |
| clarkb | my email inbox went crazy with storyboard story updates during last week as well | 19:21 |
| clarkb | I think SotK persia and diablo_rojo_phon were busy | 19:21 |
| clarkb | was anyone better than me at following along on what changes were being made? | 19:21 |
| fungi | yes, they had a storysquash at the ptg | 19:21 |
| fungi | basically triaging and combining/splitting/closing stories for storyboard and storyboard-webclient | 19:22 |
| clarkb | There was also a constructive feedback session at the forum | 19:23 |
| * clarkb finds a link to the etherpad for that | 19:23 | |
| clarkb | #link https://etherpad.openstack.org/p/storyboard-pain-points | 19:23 |
| clarkb | #topic General Topics | 19:24 |
| *** openstack changes topic to "General Topics (Meeting topic: infra)" | 19:24 | |
| clarkb | #link https://etherpad.openstack.org/2019-denver-ptg-infra-planning Denver PTG 2019 notes | 19:24 |
| clarkb | in case anyone missed it ^ is where the notes were taken during the PTG | 19:25 |
| clarkb | I've also just taken an action to write up a recap email so keep an eye out on the -infra list for that | 19:25 |
| clarkb | Is there anything we haven't already touched on that people want more details on? questions? concerns? | 19:25 |
| ianw | does splitting up the base playbook like this mean the bridge mysql/ara work is basically not necessary? | 19:26 |
| ianw | we'll just essentially have the logs from post job runs instead? | 19:27 |
| corvus | ianw: we're still expecting the playbooks to run via bridge's ansible, so that could still be useful | 19:27 |
| corvus | (i think that ways always the plan) | 19:27 |
| clarkb | ianw: we will have those logs, but the way the current CD jobs are set up is to execute ansible-playbook on bridge | 19:27 |
| clarkb | (so we won't get the data in the zuul ara) | 19:27 |
| *** jamesmcarthur has joined #openstack-meeting | 19:27 | |
| ianw | ahh, right, that was your root login stuff | 19:27 |
| clarkb | given that I think having ara for the bridge run ansible-playbook is still useful | 19:28 |
| fungi | which is probably good for avoiding information leaks anyway | 19:28 |
| mordred | ++ | 19:28 |
| ianw | cool, ok thanks for clarifying | 19:28 |
| *** tssurya has quit IRC | 19:29 | |
| mordred | ianw: oh - speaking of - I pinged you in scrollback about the letsencrypt roles - but it turns out we weren't running one of them, so nothing to worry about :) | 19:29 |
| clarkb | That is a good transition to the next topic :) | 19:29 |
| clarkb | we have a buildset registry and a buildset proxy registry in our docker image build jobs | 19:29 |
| clarkb | sometimes that buildset proxy registry fails (seems to happen more on ipv6 clouds that NAT to dockerhub) so my idea to fix that is to use the in region docker proxy | 19:30 |
| mordred | ++ | 19:30 |
| clarkb | those in region mirror nodes have a floating IP so don't go through the many to one nat shared by the test nodes | 19:30 |
| clarkb | However corvus pointed out that since we'll be publishing the resulting docker images we should get tls set up on those first | 19:30 |
| clarkb | my plan is to roughly add all our in region mirrors to opendev.org DNS, add ssl vhosts to them then run the LE stuff to get certs for the opdnev.org names | 19:31 |
| clarkb | Then with SNI I don't even need to change ports | 19:31 |
| fungi | that'll be nice | 19:32 |
| corvus | ++ | 19:32 |
| clarkb | As part of this we probably want to transition most/many things to https (and not just the dockerhub proxy) | 19:32 |
| clarkb | I expect apt to present the biggest pain there but otherwise should be a win all around | 19:33 |
| clarkb | ianw: ^ any concerns with doing that given your LE work? | 19:33 |
| clarkb | like should https://review.opendev.org/#/c/652801/ get in first? | 19:33 |
| clarkb | #link https://review.opendev.org/#/c/652801/ Add handlers for LE certs updates for integration with existing services | 19:34 |
| clarkb | infra-root ^ if you have time to review that it would be much appreciated | 19:34 |
| *** jamesmcarthur has quit IRC | 19:34 | |
| *** _hemna has quit IRC | 19:34 | |
| *** jamesmcarthur has joined #openstack-meeting | 19:34 | |
| ianw | clarkb: yeah, i think it could be a bit asynchronous. we can deploy the keys first and make sure they're all there ok, then update the puppet to insert them into apache config | 19:35 |
| ianw | then, within at least 90 days i guess, we need something to figure out restarting apache when then renew :) | 19:35 |
| ianw | but probably having that before we start is a better idea | 19:35 |
| clarkb | ya, would be good to have that sorted out before we have too many to retrofit | 19:35 |
| *** jamesmcarthur has quit IRC | 19:35 | |
| *** jamesmcarthur has joined #openstack-meeting | 19:36 | |
| ianw | my idea, with 652801 would be that ansible would drop a flag file, which puppet would notice, remove and restart apache | 19:36 |
| ianw | as we, someday, move the hosts to more ansible control, we should be integrate more with listeners etc | 19:37 |
| clarkb | yup | 19:37 |
| clarkb | ianw: have you had a chance to see my comments on that change yet? | 19:37 |
| clarkb | curious to see what you think about them (but overall I was happy with the chagne) | 19:37 |
| corvus | why have puppet do it and not ansible directly? | 19:37 |
| clarkb | corvus: that was one of my questions :) | 19:37 |
| corvus | ah i see that now :) | 19:38 |
| ianw | i haven't cycled, back, but will today. i'm happy to help setup the initial mirror certs | 19:38 |
| clarkb | great thanks | 19:38 |
| corvus | (it doesn't bother me that ansible would be restarting a service it didn't set up) | 19:38 |
| clarkb | The last item I had on the agenda this week was to bring up the trusty upgrades/cleanups | 19:38 |
| clarkb | #link https://etherpad.openstack.org/p/201808-infra-server-upgrades-and-cleanup | 19:38 |
| ianw | will respond, but the idea was that ansible *can* do it, if we want -- but there might be things where it's more complex than service foo restart so i wanted to make sure it could be flexible | 19:39 |
| clarkb | I'd like to delete the groups servers this week if I can find time to sit down and make sure I back them up and delete the right instances | 19:39 |
| clarkb | If we can finish cleaning up this list we'll be able to remove old nodepool images and test jobs which is nice to have on top of the EOL problem | 19:39 |
| clarkb | fungi: ^ re groups ashlee had thoguht you had already done a db backup and/or instance snapshot | 19:39 |
| clarkb | fungi: do you know if any of that has been done yet? | 19:40 |
| fungi | yes, i did | 19:40 |
| clarkb | cool so mostly just a matter of deleting the instances now? | 19:40 |
| fungi | there is a db snapshot and a server snapshot | 19:40 |
| clarkb | I can add them to the list with the old cgit cluster | 19:40 |
| fungi | yes, should be safe to proceed | 19:40 |
| clarkb | For those of you that have names on updating some of those remaining services let me know if I can do anything to help | 19:41 |
| clarkb | And with that I think we've hit the end of our agenda | 19:42 |
| clarkb | #topic Open Discussion | 19:42 |
| *** openstack changes topic to "Open Discussion (Meeting topic: infra)" | 19:42 | |
| *** jamesmcarthur has quit IRC | 19:42 | |
| clarkb | We should set a date for straggler opendev renames to happen | 19:42 |
| clarkb | then encourage everyone to update the rename request list for that to happen if they want a rename | 19:42 |
| *** ttsiouts has joined #openstack-meeting | 19:43 | |
| clarkb | maybe the end of the month (gives people time to get back from summit and realize they want a rename) | 19:43 |
| *** ayoung has quit IRC | 19:43 | |
| fungi | sounds good to me, i should be around after this weekend | 19:43 |
| ianw | one thing to update on is testinfra which i've spent a bit of time with. it's ansible 2.8 compatible now ... but not using ansible-runner unfortunately | 19:44 |
| corvus | my schedule returns to normal after may 22 | 19:44 |
| clarkb | ianw: I suppose as long as it works with newer ansible our use case is met | 19:44 |
| clarkb | corvus: fungi rgr seems like the 31st might be a good day to pencil in then (thinking others are also in recovery mdoe and may not even notice yet that a name needs changing) | 19:44 |
| ianw | clarkb: sort of, i've sent in a fix for ipv6 addresses that broke us | 19:45 |
| pabelanger | I actually had one, but didn't add it to meeting agenda. Where do we stand on getting nodepool / k8s integration? ~5months ago we landed: https://review.opendev.org/620755 for kube config for nodepool, but I kinda lost track where things are. Given what we know about namespaces in k8s / nodepool / zuul, is that something someone could pick up again and try to say get tox job on k8s? Or do we need | 19:45 |
| pabelanger | new features in zuul / nodepool? | 19:45 |
| clarkb | ianw: fun | 19:45 |
| ianw | mordred: could you look at https://review.opendev.org/656908 which lets us install testinfra in tox sibilings correctly | 19:45 |
| clarkb | pabelanger: I believe everything is in place for that to work now. Just need to add jobs for it (but I'm sure that if you add jobs for it we'll discover other fixes are necessary_ | 19:45 |
| mordred | pabelanger: it's a thing I think we're definitely still poking at - but I don't think we're in a rush to run tox jobs in k8s because it'll ultimately be more expensive to do so | 19:45 |
| clarkb | mordred: pabelanger right I think the idea was to start with jobs that exercise nodepool' | 19:46 |
| clarkb | s support of k8s | 19:46 |
| mordred | yeah | 19:46 |
| clarkb | not to shift large workload to it | 19:46 |
| corvus | yeah, if someone wants to poke at that i say go for it | 19:46 |
| mordred | ++ | 19:46 |
| pabelanger | okay, that is fine, I mostly wanted to say I am interestd in it, and think I can offer some time to enable it. I just used tox as example. | 19:46 |
| mordred | totes - and I think it would be great to have you poke at it | 19:47 |
| corvus | there is a chance that i screwed up the credentials for that and we might need to re-export them | 19:47 |
| pabelanger | so, expect some questions in coming days on what we'd like to do :) | 19:47 |
| mordred | \o/ | 19:47 |
| *** yamamoto has quit IRC | 19:47 | |
| pabelanger | cool, thanks! | 19:47 |
| * mordred hands pabelanger a turkey | 19:48 | |
| clarkb | Alright, anything else before we call it a meeting? | 19:50 |
| clarkb | Sounds like no. Everyone enjoy their weird weeks and thank you! See you here next week. | 19:52 |
| clarkb | #endmeeting | 19:52 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 19:52 | |
| openstack | Meeting ended Tue May 7 19:52:28 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 19:52 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-05-07-19.01.html | 19:52 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-05-07-19.01.txt | 19:52 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/infra/2019/infra.2019-05-07-19.01.log.html | 19:52 |
| fungi | thanks clarkb! | 19:52 |
| *** bbowen_ has quit IRC | 20:12 | |
| *** ttsiouts has quit IRC | 20:16 | |
| *** artom has quit IRC | 20:31 | |
| *** diablo_rojo has joined #openstack-meeting | 20:32 | |
| *** slaweq has quit IRC | 20:36 | |
| *** slaweq has joined #openstack-meeting | 20:42 | |
| *** tssurya has joined #openstack-meeting | 20:52 | |
| *** slaweq has quit IRC | 20:55 | |
| *** raildo has quit IRC | 21:10 | |
| *** slaweq has joined #openstack-meeting | 21:12 | |
| *** diablo_rojo has quit IRC | 21:15 | |
| *** ttsiouts has joined #openstack-meeting | 21:18 | |
| *** ttsiouts has quit IRC | 21:23 | |
| *** slaweq has quit IRC | 21:25 | |
| *** bbowen_ has joined #openstack-meeting | 21:30 | |
| *** hongbin has joined #openstack-meeting | 21:35 | |
| *** mriedem has quit IRC | 21:36 | |
| *** mattw4 has quit IRC | 21:46 | |
| *** mattw4 has joined #openstack-meeting | 21:50 | |
| *** ijw has quit IRC | 21:57 | |
| *** ijw has joined #openstack-meeting | 21:57 | |
| *** slaweq has joined #openstack-meeting | 22:11 | |
| *** njohnston has quit IRC | 22:15 | |
| *** slaweq has quit IRC | 22:24 | |
| *** jamesmcarthur has joined #openstack-meeting | 22:57 | |
| *** bobh has joined #openstack-meeting | 23:03 | |
| *** ayoung has joined #openstack-meeting | 23:04 | |
| *** rcernin has joined #openstack-meeting | 23:06 | |
| *** slaweq has joined #openstack-meeting | 23:11 | |
| *** yamamoto has joined #openstack-meeting | 23:17 | |
| *** artom has joined #openstack-meeting | 23:21 | |
| *** bobh has quit IRC | 23:21 | |
| *** slaweq has quit IRC | 23:24 | |
| *** jamesmcarthur has quit IRC | 23:27 | |
| *** ttsiouts has joined #openstack-meeting | 23:35 | |
| *** lseki has quit IRC | 23:40 | |
| *** hongbin has quit IRC | 23:41 | |
| *** bbowen_ has quit IRC | 23:47 | |
| *** bbowen_ has joined #openstack-meeting | 23:47 | |
| *** whoami-rajat has quit IRC | 23:55 | |
| *** mattw4 has quit IRC | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!