| *** brinzhang has joined #openstack-meeting | 00:08 | |
| *** brinzhang_ has quit IRC | 00:12 | |
| *** jamesmcarthur has quit IRC | 00:12 | |
| *** jamesmcarthur has joined #openstack-meeting | 00:12 | |
| *** jamesmcarthur has quit IRC | 00:18 | |
| *** njohnston_ has quit IRC | 00:23 | |
| *** jamesmcarthur has joined #openstack-meeting | 00:24 | |
| *** igordc has quit IRC | 00:25 | |
| *** igordc has joined #openstack-meeting | 00:25 | |
| *** brinzhang has quit IRC | 00:39 | |
| *** igordc has quit IRC | 00:49 | |
| *** brinzhang has joined #openstack-meeting | 00:52 | |
| *** brinzhang has quit IRC | 00:54 | |
| *** ayoung has quit IRC | 00:54 | |
| *** ricolin has joined #openstack-meeting | 00:55 | |
| *** ayoung has joined #openstack-meeting | 00:57 | |
| *** artom has quit IRC | 01:00 | |
| *** jamesmcarthur has quit IRC | 01:04 | |
| *** jamesmcarthur has joined #openstack-meeting | 01:04 | |
| *** jamesmcarthur has quit IRC | 01:06 | |
| *** jamesmcarthur has joined #openstack-meeting | 01:06 | |
| *** masahito has joined #openstack-meeting | 01:07 | |
| *** masahito has quit IRC | 01:07 | |
| *** yamamoto has joined #openstack-meeting | 01:15 | |
| *** lseki has quit IRC | 01:40 | |
| *** rfolco has quit IRC | 01:41 | |
| *** Lucas_Gray has quit IRC | 01:49 | |
| *** jamesmcarthur has quit IRC | 01:56 | |
| *** jamesmcarthur has joined #openstack-meeting | 01:57 | |
| *** jamesmcarthur has quit IRC | 02:02 | |
| *** ircuser-1 has joined #openstack-meeting | 02:02 | |
| *** ykatabam has quit IRC | 02:09 | |
| *** ykatabam has joined #openstack-meeting | 02:11 | |
| *** mhen has quit IRC | 02:12 | |
| *** macz has joined #openstack-meeting | 02:15 | |
| *** yamamoto has quit IRC | 02:19 | |
| *** macz has quit IRC | 02:20 | |
| *** jamesmcarthur has joined #openstack-meeting | 02:24 | |
| *** ociuhandu has joined #openstack-meeting | 02:47 | |
| *** ociuhandu has quit IRC | 02:51 | |
| *** jamesmcarthur has quit IRC | 03:04 | |
| *** jamesmcarthur has joined #openstack-meeting | 03:05 | |
| *** dklyle has joined #openstack-meeting | 03:07 | |
| *** apetrich has quit IRC | 03:10 | |
| *** jamesmcarthur has quit IRC | 03:11 | |
| *** yamamoto has joined #openstack-meeting | 03:14 | |
| *** jamesmcarthur has joined #openstack-meeting | 03:30 | |
| *** jamesmcarthur has quit IRC | 03:33 | |
| *** tpatil has joined #openstack-meeting | 03:56 | |
| *** tpatil has quit IRC | 03:59 | |
| *** hongbin has joined #openstack-meeting | 04:03 | |
| *** tetsuro has quit IRC | 04:18 | |
| *** tetsuro has joined #openstack-meeting | 04:19 | |
| *** tetsuro has quit IRC | 04:23 | |
| *** hongbin has quit IRC | 04:39 | |
| *** jamesmcarthur has joined #openstack-meeting | 04:42 | |
| *** links has joined #openstack-meeting | 04:49 | |
| *** tetsuro has joined #openstack-meeting | 05:03 | |
| *** sridharg has joined #openstack-meeting | 05:09 | |
| *** macz has joined #openstack-meeting | 05:20 | |
| *** macz has quit IRC | 05:22 | |
| *** jamesmcarthur has quit IRC | 05:35 | |
| *** jamesmcarthur has joined #openstack-meeting | 05:37 | |
| *** jamesmcarthur has quit IRC | 05:43 | |
| *** ykatabam has quit IRC | 05:58 | |
| *** jamesmcarthur has joined #openstack-meeting | 06:06 | |
| *** epoojad1 has joined #openstack-meeting | 06:08 | |
| *** lpetrut has joined #openstack-meeting | 06:08 | |
| *** lpetrut has quit IRC | 06:09 | |
| *** lpetrut has joined #openstack-meeting | 06:10 | |
| *** jamesmcarthur has quit IRC | 06:13 | |
| *** epoojad1 has quit IRC | 06:18 | |
| *** ykatabam has joined #openstack-meeting | 06:23 | |
| *** lpetrut has quit IRC | 06:49 | |
| *** maohongbo has joined #openstack-meeting | 07:03 | |
| *** jamesmcarthur has joined #openstack-meeting | 07:09 | |
| *** jamesmcarthur has quit IRC | 07:14 | |
| *** keiko-k has joined #openstack-meeting | 07:24 | |
| *** ociuhandu has joined #openstack-meeting | 07:30 | |
| *** ociuhandu has quit IRC | 07:35 | |
| *** cschwede has joined #openstack-meeting | 07:37 | |
| *** lpetrut has joined #openstack-meeting | 07:38 | |
| *** mahatic has joined #openstack-meeting | 07:38 | |
| *** slaweq has joined #openstack-meeting | 08:00 | |
| *** joxyuki has joined #openstack-meeting | 08:02 | |
| joxyuki | #startmeeting tacker | 08:03 |
|---|---|---|
| openstack | Meeting started Tue Jan 21 08:03:45 2020 UTC and is due to finish in 60 minutes. The chair is joxyuki. Information about MeetBot at http://wiki.debian.org/MeetBot. | 08:03 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 08:03 |
| *** openstack changes topic to " (Meeting topic: tacker)" | 08:03 | |
| openstack | The meeting name has been set to 'tacker' | 08:03 |
| *** takahashi-tsc has joined #openstack-meeting | 08:04 | |
| joxyuki | #topic RollCall | 08:04 |
| *** openstack changes topic to "RollCall (Meeting topic: tacker)" | 08:04 | |
| hyunsikyang | Hello All. | 08:04 |
| joxyuki | hi hyunsikyang | 08:05 |
| takahashi-tsc | Hello | 08:05 |
| keiko-k | Hello | 08:05 |
| joxyuki | ok, let's get started... | 08:06 |
| *** bnemec has joined #openstack-meeting | 08:06 | |
| joxyuki | #topic spec: event alarm | 08:06 |
| *** openstack changes topic to "spec: event alarm (Meeting topic: tacker)" | 08:06 | |
| joxyuki | I posted some comments on the spec https://review.opendev.org/#/c/700167/ | 08:07 |
| joxyuki | takahashi-tsc, do you have any query or discussion? | 08:07 |
| takahashi-tsc | Not here, but there are something to consider. Basically I consider. | 08:09 |
| *** jamesmcarthur has joined #openstack-meeting | 08:10 | |
| takahashi-tsc | I replied comment, so I would be grateful if you have additional comments. | 08:10 |
| hyunsikyang | I didn't check yet. I will check it too:) thanks | 08:11 |
| joxyuki | yes, I saw your replay just now. Let's keep discussion on gerrit or IRC> | 08:11 |
| takahashi-tsc | OK, thanks. | 08:11 |
| joxyuki | #topic multi-interface for CNF | 08:12 |
| *** openstack changes topic to "multi-interface for CNF (Meeting topic: tacker)" | 08:12 | |
| joxyuki | thanks for the patch, hyunsikyang | 08:12 |
| joxyuki | do you have any discussion about it? | 08:12 |
| hyunsikyang | you're welcome. last time, we checked the code and got +1. | 08:12 |
| hyunsikyang | So now, If you have a time, please review test code. | 08:12 |
| hyunsikyang | and then we can finish it for next one.:) | 08:13 |
| *** jamesmcarthur has quit IRC | 08:14 | |
| joxyuki | ok | 08:15 |
| joxyuki | I found functional test failed. | 08:15 |
| hyunsikyang | in my code? | 08:15 |
| hyunsikyang | testcode? | 08:15 |
| joxyuki | I don't think your code. | 08:16 |
| hyunsikyang | Ah I thought you mentioned my code. | 08:16 |
| joxyuki | oh, it's due to your code. You deleted tools/test-setup-default-vim.sh so devstack installation failed in functional test. | 08:18 |
| joxyuki | and also tacker/test/etc/sample/local-vim.yaml. | 08:19 |
| joxyuki | Do you have any reason to delete these files? | 08:19 |
| hyunsikyang | Ah AM I delete it? I thought I delete it in my patch | 08:19 |
| hyunsikyang | sorry. | 08:19 |
| hyunsikyang | Because, last time when I upload code, it include without my intend,, so I delete it. | 08:20 |
| *** tesseract has joined #openstack-meeting | 08:20 | |
| hyunsikyang | So how can I fix it? | 08:21 |
| joxyuki | NP, can you revive these file on your patch? | 08:21 |
| hyunsikyang | OK | 08:21 |
| hyunsikyang | I will add it in the next patch! | 08:21 |
| hyunsikyang | TOday. | 08:21 |
| joxyuki | you should checkout these file from master branch and submit next patchset. | 08:22 |
| joxyuki | good. | 08:22 |
| joxyuki | moving next... | 08:22 |
| joxyuki | keiko-k, do you have your topic today? | 08:23 |
| keiko-k | nothing | 08:23 |
| joxyuki | #topic open-discussion | 08:24 |
| *** openstack changes topic to "open-discussion (Meeting topic: tacker)" | 08:24 | |
| joxyuki | we should suport python3 and drop python2 support. | 08:25 |
| joxyuki | I recommend you all to make your devstack on python3. | 08:26 |
| hyunsikyang | ok. | 08:27 |
| joxyuki | Notetaht when you do it, you need https://review.opendev.org/#/c/696227/. | 08:27 |
| joxyuki | You can get the patch by doing git pull on master branch as it was already merged. | 08:28 |
| *** priteau has joined #openstack-meeting | 08:29 | |
| joxyuki | okey, does anyone have another topic? if not, I will close the meeting. | 08:29 |
| *** ralonsoh has joined #openstack-meeting | 08:30 | |
| hyunsikyang | OK. | 08:30 |
| hyunsikyang | thanks | 08:30 |
| joxyuki | seems nothig. Thanks folks. | 08:31 |
| joxyuki | #endmeeting | 08:31 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 08:31 | |
| openstack | Meeting ended Tue Jan 21 08:31:11 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 08:31 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/tacker/2020/tacker.2020-01-21-08.03.html | 08:31 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/tacker/2020/tacker.2020-01-21-08.03.txt | 08:31 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/tacker/2020/tacker.2020-01-21-08.03.log.html | 08:31 |
| *** joxyuki has left #openstack-meeting | 08:33 | |
| *** links has quit IRC | 08:47 | |
| *** keiko-k has quit IRC | 08:48 | |
| *** rpittau|afk is now known as rpittau | 08:48 | |
| *** links has joined #openstack-meeting | 08:53 | |
| *** yamamoto has quit IRC | 08:53 | |
| *** yamamoto has joined #openstack-meeting | 09:14 | |
| *** brinzhang has joined #openstack-meeting | 09:16 | |
| *** yamamoto has quit IRC | 09:18 | |
| *** apetrich has joined #openstack-meeting | 09:18 | |
| *** brinzhang has quit IRC | 09:22 | |
| *** brinzhang has joined #openstack-meeting | 09:23 | |
| *** tetsuro has quit IRC | 09:56 | |
| *** links has quit IRC | 10:04 | |
| *** links has joined #openstack-meeting | 10:07 | |
| *** e0ne has joined #openstack-meeting | 10:19 | |
| *** priteau has quit IRC | 10:41 | |
| *** takahashi-tsc has quit IRC | 10:45 | |
| *** raildo has joined #openstack-meeting | 11:06 | |
| *** raildo_ has joined #openstack-meeting | 11:14 | |
| *** rpittau is now known as rpittau|bbl | 11:15 | |
| *** raildo has quit IRC | 11:17 | |
| *** dviroel has joined #openstack-meeting | 11:25 | |
| *** Lucas_Gray has joined #openstack-meeting | 11:40 | |
| *** rfolco has joined #openstack-meeting | 12:00 | |
| *** njohnston has joined #openstack-meeting | 12:00 | |
| *** priteau has joined #openstack-meeting | 12:07 | |
| *** ociuhandu has joined #openstack-meeting | 12:12 | |
| *** ociuhandu has quit IRC | 12:13 | |
| *** Lucas_Gray has quit IRC | 12:13 | |
| *** Lucas_Gray has joined #openstack-meeting | 12:17 | |
| *** hyunsikyang__ has joined #openstack-meeting | 12:18 | |
| *** hyunsikyang has quit IRC | 12:22 | |
| *** artom has joined #openstack-meeting | 12:24 | |
| *** rcernin has quit IRC | 12:28 | |
| *** Luzi has joined #openstack-meeting | 12:28 | |
| *** Liang__ has joined #openstack-meeting | 12:46 | |
| *** Liang__ is now known as LiangFang | 12:46 | |
| *** nicolasbock has joined #openstack-meeting | 12:49 | |
| *** ociuhandu has joined #openstack-meeting | 12:52 | |
| *** lseki has joined #openstack-meeting | 12:57 | |
| *** sfernand has joined #openstack-meeting | 12:57 | |
| *** ociuhandu has quit IRC | 13:01 | |
| *** priteau has quit IRC | 13:05 | |
| *** rpittau|bbl is now known as rpittau | 13:05 | |
| *** jamesmcarthur has joined #openstack-meeting | 13:18 | |
| *** zbr|drover has quit IRC | 13:18 | |
| *** zbr has joined #openstack-meeting | 13:19 | |
| ricolin | self-healing and auto-scaling SIG meeting starts in 30 mins on #openstack-meeting feel free to join | 13:27 |
| *** whoami-rajat_ has joined #openstack-meeting | 13:28 | |
| *** enriquetaso has joined #openstack-meeting | 13:29 | |
| *** jamesmcarthur has quit IRC | 13:33 | |
| *** Liang__ has joined #openstack-meeting | 13:37 | |
| *** LiangFang has quit IRC | 13:38 | |
| *** ykatabam has quit IRC | 13:41 | |
| *** Lucas_Gray has quit IRC | 13:42 | |
| *** jamesmcarthur has joined #openstack-meeting | 13:45 | |
| *** ociuhandu has joined #openstack-meeting | 13:47 | |
| *** dougsz has joined #openstack-meeting | 13:48 | |
| *** ociuhandu has quit IRC | 13:53 | |
| *** witek has joined #openstack-meeting | 13:58 | |
| ricolin | o/ | 14:00 |
| witek | hi | 14:00 |
| ricolin | #startmeeting self_healing | 14:01 |
| openstack | Meeting started Tue Jan 21 14:01:02 2020 UTC and is due to finish in 60 minutes. The chair is ricolin. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:01 |
| *** openstack changes topic to " (Meeting topic: self_healing)" | 14:01 | |
| openstack | The meeting name has been set to 'self_healing' | 14:01 |
| ricolin | witek, o/ | 14:01 |
| witek | hi ricolin | 14:01 |
| ricolin | got any suggest topics?:) | 14:02 |
| witek | I have tried to look into CI problems with Monasca auto-scaling, but didn't have too much time :) | 14:03 |
| witek | :( | 14:03 |
| ricolin | #link https://review.opendev.org/#/c/700989/ | 14:03 |
| witek | thanks for debugging information provided in review | 14:03 |
| ricolin | the way I put in https://review.opendev.org/#/c/700989/ is the same code we have for heat | 14:04 |
| ricolin | but if we can have the actual reason will be perfect | 14:05 |
| ricolin | I mean the root cause | 14:05 |
| *** cschwede has quit IRC | 14:06 | |
| ricolin | We also need to take action to update the SIG guideline for detail monasca+heat | 14:06 |
| ricolin | I assume most of info we need already in your repo (the one for workshop) | 14:07 |
| witek | but https://review.opendev.org/#/c/700989/ alone doesn't help, as we can see in https://review.opendev.org/700886 | 14:07 |
| witek | what kind of information would you like to see and where? | 14:07 |
| ricolin | witek, yeah, according to the log, codes from python-monasca didn't even shows up | 14:08 |
| ricolin | in some reason, the dependency python-monascaclient patch is ignored | 14:09 |
| ricolin | For guideline, I only have this story set up https://storyboard.openstack.org/#!/story/2007046 | 14:11 |
| ricolin | the detail of guideline is what we can discuss | 14:11 |
| ricolin | As long as people can read it and do it their own, will be a successful one:) | 14:12 |
| witek | :) is there some document for Telemetry available? | 14:13 |
| witek | we could reuse and copy some of the material we used in the workshop | 14:15 |
| ricolin | nope | 14:15 |
| witek | I think till now we have just linked the reference to the GitHub repo | 14:15 |
| ricolin | +1 on reuse | 14:16 |
| ricolin | I think most people confused when there's to many detail but no docs around | 14:16 |
| ricolin | *too | 14:16 |
| witek | yes, let's keep it as simple as possible | 14:17 |
| ricolin | witek, right, maybe like a workshop doc for a scenario? | 14:18 |
| witek | what do you mean? | 14:18 |
| ricolin | We have collected some scenarios like heat+monasca, maybe we can do the guideline like heat+monasca with cpu loading detect | 14:20 |
| witek | OK, get it | 14:20 |
| ricolin | otherwise separate by project might be another option | 14:21 |
| *** yamamoto has joined #openstack-meeting | 14:21 | |
| witek | I'll try to write up something | 14:22 |
| ricolin | that's great:) | 14:22 |
| ricolin | I will consult others and see if there's anyway to proper depends on python client patch | 14:23 |
| witek | thanks for helping with that! | 14:24 |
| *** links has quit IRC | 14:24 | |
| ricolin | the doc and the CI job sounds like all we need at this stage | 14:24 |
| ricolin | if that python client issue resolved, I can send out the CI job in very short time | 14:25 |
| witek | cool | 14:25 |
| ricolin | I assume that's all we got to works on | 14:26 |
| ricolin | If nothing else, let's close the meeting:) | 14:26 |
| witek | nothing from me | 14:27 |
| ricolin | Okay, thanks for join meeting witek:) | 14:27 |
| ricolin | let's keep the action rolling:) | 14:27 |
| witek | thanks rico | 14:27 |
| ricolin | #endmeeting | 14:27 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 14:27 | |
| openstack | Meeting ended Tue Jan 21 14:27:51 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:27 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/self_healing/2020/self_healing.2020-01-21-14.01.html | 14:27 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/self_healing/2020/self_healing.2020-01-21-14.01.txt | 14:27 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/self_healing/2020/self_healing.2020-01-21-14.01.log.html | 14:27 |
| *** ayoung has quit IRC | 14:30 | |
| *** yamamoto has quit IRC | 14:36 | |
| ricolin | Multi-Arch SIG meeting in 23 mins. feel free to join us:) https://etherpad.openstack.org/p/Multi-Arch-agenda | 14:37 |
| *** dtroyer has joined #openstack-meeting | 14:38 | |
| *** Luzi has quit IRC | 14:46 | |
| *** sridharg has quit IRC | 14:47 | |
| *** jeremyfreudberg has joined #openstack-meeting | 14:51 | |
| *** yamamoto has joined #openstack-meeting | 14:59 | |
| *** yamamoto has quit IRC | 14:59 | |
| *** yamamoto has joined #openstack-meeting | 14:59 | |
| ricolin | #startmeeting multi_arch | 15:00 |
| openstack | Meeting started Tue Jan 21 15:00:21 2020 UTC and is due to finish in 60 minutes. The chair is ricolin. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| *** openstack changes topic to " (Meeting topic: multi_arch)" | 15:00 | |
| openstack | The meeting name has been set to 'multi_arch' | 15:00 |
| ricolin | #topic roll call | 15:00 |
| *** openstack changes topic to "roll call (Meeting topic: multi_arch)" | 15:00 | |
| ricolin | o/ | 15:00 |
| jeremyfreudberg | o/ | 15:00 |
| ricolin | jeremyfreudberg, o/ | 15:00 |
| ricolin | jeremyfreudberg, how's going! | 15:01 |
| *** shintaro has joined #openstack-meeting | 15:01 | |
| jeremyfreudberg | ricolin: all good, a little busy | 15:01 |
| jeremyfreudberg | but busy can be good | 15:01 |
| *** rh-jelabarre has joined #openstack-meeting | 15:02 | |
| ricolin | true | 15:02 |
| jeremyfreudberg | ricolin: do we expect anyone else at this meeting time? or is everybody attending earlier? | 15:02 |
| ricolin | ianw, kevinz, and mrda joined earlier | 15:03 |
| rh-jelabarre | what would the earlier time be in EST? I think 3AM? | 15:03 |
| ricolin | I assume tonyb might be join this one, but not sure | 15:03 |
| jeremyfreudberg | rh-jelabarre: yes, i think 3am | 15:03 |
| jeremyfreudberg | ricolin: i did not hear anything specific of tony, but maybe he will join now | 15:04 |
| *** yamamoto has quit IRC | 15:04 | |
| ricolin | rh-jelabarre, Tuesday at 0800 UTC is all I know:) | 15:04 |
| jeremyfreudberg | ricolin: rh-jelabarre is in my time zone (and also work with me and tony) | 15:04 |
| tonyb | Oh! | 15:04 |
| ricolin | appear he is traveling | 15:04 |
| jeremyfreudberg | hi tonyb | 15:04 |
| ricolin | there you're! | 15:04 |
| tonyb | So it look slike the calendar is wrong as this is set to biweekly | 15:04 |
| jeremyfreudberg | tonyb: it is biweekely, so what's the problem? | 15:05 |
| tonyb | Oh nothing I'm just dense | 15:05 |
| ricolin | :) | 15:05 |
| ricolin | #chair ricolin tonyb | 15:06 |
| openstack | Current chairs: ricolin tonyb | 15:06 |
| ricolin | #topic adding items to agenda | 15:06 |
| *** openstack changes topic to "adding items to agenda (Meeting topic: multi_arch)" | 15:06 | |
| ricolin | #link https://etherpad.openstack.org/p/Multi-Arch-agenda | 15:06 |
| rh-jelabarre | tonyb: I resemble that remark... | 15:06 |
| ricolin | Also here's logs from earlier meeting http://eavesdrop.openstack.org/meetings/multi_arch/2020/multi_arch.2020-01-21-08.00.html | 15:07 |
| ricolin | #topic Follow on Repo and StoryBoard setup | 15:08 |
| *** openstack changes topic to "Follow on Repo and StoryBoard setup (Meeting topic: multi_arch)" | 15:09 | |
| ricolin | some quick paste info | 15:09 |
| ricolin | Our Repo is there, but I haven't add the doc and zuul structure yet. Will do it this week. | 15:09 |
| *** Liang__ is now known as LiangFang | 15:09 | |
| ricolin | #link https://opendev.org/openstack/multi-arch-sig | 15:09 |
| ricolin | According to the plan we got from last meeting, documents will first to put in. I think that can started now, and send out once the repo structure is ready. | 15:09 |
| ricolin | Also we have core reviewer list | 15:09 |
| ricolin | #link https://review.opendev.org/#/admin/groups/2079,members | 15:09 |
| ricolin | And I would like to suggest we add volunteers as core reviewers. | 15:10 |
| ricolin | Anyone volunteer? rh-jelabarre jeremyfreudberg :) | 15:10 |
| tonyb | ricolin: Works for me | 15:10 |
| jeremyfreudberg | ricolin: feel free to add me as core | 15:10 |
| ricolin | jeremyfreudberg, will add you later, or tonyb can add you too as he is also in the list | 15:11 |
| rh-jelabarre | not sure what a core reviewer would be doing | 15:11 |
| tonyb | jeremyfreudberg: done | 15:11 |
| ricolin | tonyb, that's quick | 15:11 |
| tonyb | rh-jelabarre: probably not a good fit then | 15:11 |
| tonyb | rh-jelabarre: We can help you learn "the openstack way" | 15:11 |
| ricolin | rh-jelabarre, we can definitely help you onboard:) | 15:12 |
| ricolin | Our StoryBoard is created, | 15:12 |
| tonyb | ricolin: you've been busy! | 15:13 |
| ricolin | #link https://storyboard.openstack.org/#!/project/openstack/multi-arch-sig | 15:13 |
| ricolin | tonyb, yep!:) | 15:13 |
| ricolin | in the good way:) | 15:13 |
| tonyb | #thanks ricolin For his awesome and rapid setup of the multi-arch-SIG | 15:14 |
| ricolin | We can start add tasks in Storyboard now | 15:14 |
| ricolin | tonyb, :) | 15:14 |
| ricolin | So if all members can put in their current work in progress task in this story will make others easier to track. We already have some collected WIP patch recorded in https://etherpad.openstack.org/p/Multi-arch which will make sense to move to StoryBoard too. | 15:15 |
| ricolin | jeremyfreudberg, also we need to add the document plan as task in Storyboard | 15:15 |
| jeremyfreudberg | ricolin: yes | 15:16 |
| jeremyfreudberg | i'll try to find some time | 15:16 |
| *** electrofelix has joined #openstack-meeting | 15:16 | |
| ricolin | can I put an action to have you add in document tasks in storyboard? | 15:17 |
| jeremyfreudberg | yup | 15:17 |
| ricolin | #action jeremyfreudberg create docs plan as tasks in Storyboard | 15:17 |
| ricolin | awesome:) | 15:17 |
| *** LiangFang has quit IRC | 15:18 | |
| ricolin | tonyb, I already added an action to ask everyone to put their current tasks (anything that related to Multi-arch) in storyboard if possible | 15:19 |
| tonyb | ricolin: Sounds good | 15:19 |
| ricolin | tonyb, cool! | 15:19 |
| tonyb | ricolin: It'll be good to get that written down aso we can find ways forward | 15:20 |
| *** dougsz has left #openstack-meeting | 15:20 | |
| ricolin | tonyb, I currently only aware of tasks that list in https://etherpad.openstack.org/p/Multi-arch | 15:20 |
| ricolin | tonyb, you mean we can add those in? | 15:21 |
| tonyb | ricolin: There are some ppc64le things I'm working on some of which will have arm64 work also so getting that written down and then looking for ways forward | 15:22 |
| ricolin | tonyb, can you help to put those in Storyboard | 15:23 |
| *** witek has left #openstack-meeting | 15:23 | |
| tonyb | ricolin: For sure | 15:23 |
| *** shintaro has quit IRC | 15:24 | |
| ricolin | Is ppc64le generally supported in OpenStack services? | 15:24 |
| ricolin | I mean core components | 15:25 |
| tonyb | ricolin: Yes a lot of the work is in deployment tools | 15:25 |
| *** jamesmcarthur has quit IRC | 15:25 | |
| tonyb | ricolin: It's "reasaonably" easy to build an all in one OpenStack (keystone, glance, swift, nova) | 15:26 |
| ricolin | I learn from ianw that devstack, *might* still not working with multi-arch, that's where I wonder ppc64le will need to check as well | 15:26 |
| tonyb | ricolin: I'm like 95% certain the same can be said for arm64 | 15:26 |
| ricolin | tonyb, good to learn that:) | 15:27 |
| tonyb | ricolin: Hmm I'll check but it *used* to work, I admit I don't use it (devstack) alot | 15:27 |
| tonyb | ricolin: I | 15:27 |
| tonyb | ll add it do the Action Items list | 15:27 |
| ricolin | ianw also take action to create a general job so we can test with it | 15:27 |
| ricolin | tonyb, great to have you can join that action:) | 15:28 |
| tonyb | Cool. | 15:28 |
| ricolin | you can make the action your self:) | 15:28 |
| tonyb | ricolin: Will do | 15:28 |
| ricolin | move on:) | 15:29 |
| ricolin | I think you all aware of the new irc channel :) | 15:29 |
| tonyb | ++ | 15:30 |
| ricolin | #openstack-multi-arch | 15:30 |
| *** ociuhandu has joined #openstack-meeting | 15:30 | |
| ricolin | for who don't know, please join that channel:) | 15:30 |
| rh-jelabarre | I am now... | 15:30 |
| ricolin | rh-jelabarre, join!:) | 15:31 |
| *** electrofelix has quit IRC | 15:31 | |
| ricolin | That's all I got for SIG infra setup | 15:31 |
| tonyb | ricolin: Thanks again! | 15:32 |
| ricolin | tonyb rh-jelabarre jeremyfreudberg , any other infra you think we need to add? | 15:32 |
| tonyb | ricolin: I want to see where we're at with arm and zuul etc but I don't know that's "setup" | 15:33 |
| ricolin | That's something we need to proper tracking by storyboard task I assume | 15:34 |
| ricolin | yeah, it's setup for sure:) | 15:34 |
| ricolin | I will working on puting init docs and docs test in repo this week | 15:35 |
| *** jamesmcarthur has joined #openstack-meeting | 15:35 | |
| *** electrofelix has joined #openstack-meeting | 15:36 | |
| ricolin | if we plan for some new zuul job, it can defined under our repo too | 15:36 |
| *** electrofelix has quit IRC | 15:36 | |
| *** electrofelix has joined #openstack-meeting | 15:36 | |
| tonyb | ricolin: Okay cool. | 15:38 |
| ricolin | tonyb, besides adding to storyboard, do you think it's helpful to have an repo on current arm status? | 15:38 |
| ricolin | s/repo/report/ | 15:38 |
| tonyb | ricolin: Yeah I think a report would be really good | 15:39 |
| tonyb | ricolin: we can probably make status.rst or sumilar in the repo to collect that information | 15:39 |
| ricolin | do you have anything in you mind that the status.rst might looks like? | 15:40 |
| tonyb | ricolin: Just a high level summary of what we have, what's known to work, what's known to break | 15:41 |
| tonyb | ricolin: but I don't have a strong idea | 15:42 |
| *** Lucas_Gray has joined #openstack-meeting | 15:42 | |
| ricolin | tonyb, got it, let's record it with an action so we don't forgot about that idea:) | 15:42 |
| tonyb | #action tonyb to create a placeholder "status.rst" for discussion | 15:43 |
| ricolin | :) | 15:43 |
| ricolin | thanks | 15:43 |
| ricolin | move on | 15:44 |
| ricolin | #topic Follow on CI Resources | 15:44 |
| *** openstack changes topic to "Follow on CI Resources (Meeting topic: multi_arch)" | 15:44 | |
| ricolin | For what I know mnaser mentioned that there might be some ARM resource Vexxhost can donate to community. | 15:44 |
| ricolin | Also Huawei mentioned that they can donate ARM resources too. I will keep tracking with them and see if anything scheduled. | 15:44 |
| ricolin | ianw mentioned from eariler meeting that the second group of node from Linaro is ready https://review.opendev.org/#/c/703535 | 15:45 |
| tonyb | Cool | 15:45 |
| ricolin | so we now have two groups of node | 15:45 |
| *** lpetrut has quit IRC | 15:46 | |
| ricolin | maybe some other provider can help to donates too | 15:46 |
| tonyb | ricolin: ideally yes | 15:47 |
| ricolin | will try to contact more provider to see if they're interested | 15:48 |
| ricolin | #topic Follow on documentation | 15:48 |
| *** openstack changes topic to "Follow on documentation (Meeting topic: multi_arch)" | 15:48 | |
| ricolin | jeremyfreudberg, do you have anything to update on the docs? | 15:49 |
| jeremyfreudberg | ricolin: i do not have much update today. as you suggested i will make some stories for what kind of docs we want. | 15:50 |
| * ricolin didn't have any to update for now except will try to build the init doc infra in repo | 15:50 | |
| ricolin | jeremyfreudberg, thanks! | 15:50 |
| ricolin | #topic open discussion | 15:50 |
| *** openstack changes topic to "open discussion (Meeting topic: multi_arch)" | 15:50 | |
| ricolin | tonyb, jeremyfreudberg rh-jelabarre anything we should discuss today? | 15:51 |
| *** ociuhandu has quit IRC | 15:51 | |
| jeremyfreudberg | not from me | 15:52 |
| rh-jelabarre | not at present | 15:52 |
| tonyb | ricolin: I don't have anything | 15:52 |
| ricolin | cool | 15:52 |
| ricolin | we already have a very productive starts:) | 15:53 |
| ricolin | let's see what we can gather on Storyboard:) | 15:53 |
| tonyb | ricolin: Cool beans | 15:54 |
| ricolin | let's end this meeting, thanks all for join:) | 15:55 |
| ricolin | #endmeeting | 15:55 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 15:55 | |
| openstack | Meeting ended Tue Jan 21 15:55:05 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:55 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/multi_arch/2020/multi_arch.2020-01-21-15.00.html | 15:55 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/multi_arch/2020/multi_arch.2020-01-21-15.00.txt | 15:55 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/multi_arch/2020/multi_arch.2020-01-21-15.00.log.html | 15:55 |
| *** jeremyfreudberg has left #openstack-meeting | 15:55 | |
| slaweq | #startmeeting neutron_ci | 16:00 |
| openstack | Meeting started Tue Jan 21 16:00:16 2020 UTC and is due to finish in 60 minutes. The chair is slaweq. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| slaweq | hi | 16:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| *** openstack changes topic to " (Meeting topic: neutron_ci)" | 16:00 | |
| openstack | The meeting name has been set to 'neutron_ci' | 16:00 |
| njohnston | o/ | 16:01 |
| ralonsoh | hi | 16:01 |
| bcafarel | hi | 16:01 |
| slaweq | I think we have our usuall attendees so lets start | 16:02 |
| slaweq | #topic Actions from previous meetings | 16:02 |
| *** openstack changes topic to "Actions from previous meetings (Meeting topic: neutron_ci)" | 16:02 | |
| slaweq | first one: | 16:02 |
| slaweq | slaweq to remove networking-midonet and tripleo based jobs from Neutron check queue | 16:02 |
| slaweq | I did patches: | 16:02 |
| slaweq | Midonet: https://review.opendev.org/#/c/703282/1 | 16:02 |
| slaweq | TripleO: https://review.opendev.org/#/c/703283/ | 16:03 |
| slaweq | I sent separate patches for each job because it will be easier to revert if we will want to bring those jobs back | 16:03 |
| ralonsoh | +2 to both | 16:03 |
| slaweq | thx ralonsoh | 16:03 |
| slaweq | and the next one: | 16:04 |
| slaweq | bcafarel to send cherry-pick of https://review.opendev.org/#/c/680001/ to stable/stein to fix functional tests failure | 16:04 |
| bcafarel | sent and merged! https://review.opendev.org/#/c/702603/ | 16:05 |
| slaweq | thx bcafarel | 16:05 |
| njohnston | For midonet, I am not sure it is still broken on py3... look at this midonet change, to drop py27, the py3 jobs all work well https://review.opendev.org/#/c/701210/ | 16:05 |
| slaweq | njohnston: yes, but there is no any scenario job there | 16:05 |
| ralonsoh | njohnston, good catch | 16:05 |
| slaweq | only UT | 16:05 |
| slaweq | problem is that midonet's CI is using centos and it's not really working on python 3 IIRC | 16:06 |
| njohnston | OK, I'll look into it and probably +2+W | 16:07 |
| slaweq | thx njohnston | 16:07 |
| slaweq | ok, I think we can move on | 16:08 |
| njohnston | may need to wait a little, recentl failures fail with the "pip._internal.distributions.source import SourceDistribution\nImportError: cannot import name SourceDistribution\n" error that is now getting cleared up | 16:08 |
| slaweq | njohnston: is that failure from midonet job or from where? | 16:08 |
| njohnston | I saw that in networking-midonet-tempest-aio-ml2-centos-7 results, yes | 16:09 |
| slaweq | ahh, ok | 16:10 |
| njohnston | FOr those who have not seen, it was discussed on openstack-discuss | 16:10 |
| njohnston | #link http://lists.openstack.org/pipermail/openstack-discuss/2020-January/012117.html | 16:10 |
| slaweq | if You have some time to play with it to fix it, that would be great, if not - I would still be for removing it for now until it will be fixed | 16:10 |
| njohnston | slaweq: Yeah, I'll take a quick look, if it gets outside the timebox then I'll approve the change and we can move on :-) | 16:11 |
| slaweq | njohnston: thx a lot | 16:11 |
| slaweq | ok, next topic than? | 16:11 |
| njohnston | +1 | 16:12 |
| slaweq | #topic Stadium projects | 16:13 |
| *** openstack changes topic to "Stadium projects (Meeting topic: neutron_ci)" | 16:13 | |
| slaweq | njohnston: any updates about dropping py2? | 16:14 |
| slaweq | You wasn't on yesterday's team meeting so maybe You have something new for today :) | 16:14 |
| njohnston | so midonet merged it's change | 16:14 |
| njohnston | which I think means the only one left for dropping py27 is neutron-fwaas | 16:15 |
| bcafarel | yeah open reviews list is quite short now | 16:15 |
| njohnston | #link https://review.opendev.org/#/c/688278/ | 16:15 |
| njohnston | if neutron-fwaas is going to be retired then I believe there's no sense in doing the work of fixing it | 16:16 |
| njohnston | so in short: mission complete! | 16:16 |
| ralonsoh | cool! | 16:16 |
| slaweq | njohnston: I don't think it's going to be retired this cycle | 16:16 |
| bcafarel | \o/ | 16:16 |
| slaweq | maybe in next one | 16:16 |
| slaweq | so IMHO we should merge this patch, but it shouldn't be too much work IMO | 16:17 |
| bcafarel | neutron-fwaas change was in already good shape last time I checked, should be able to merge it soon | 16:17 |
| njohnston | ok, I'll start pushing it | 16:17 |
| slaweq | thx njohnston | 16:17 |
| slaweq | yes, amotoki found some small issues in this patch, except that IMO it's good to go | 16:18 |
| njohnston | I'll push the fixes after the meeting | 16:18 |
| slaweq | and tempest job failure wasn't related to dropping py2 for sure | 16:18 |
| slaweq | thx njohnston | 16:18 |
| slaweq | ok, I think that's all about stadium projects | 16:19 |
| slaweq | so we can move on | 16:19 |
| slaweq | #topic Grafana | 16:19 |
| *** openstack changes topic to "Grafana (Meeting topic: neutron_ci)" | 16:19 | |
| slaweq | #link http://grafana.openstack.org/dashboard/db/neutron-failure-rate | 16:19 |
| njohnston | looking pretty good | 16:21 |
| ralonsoh | but during the last week, the number of rechecks has increased | 16:22 |
| ralonsoh | the gate is almost "closed" | 16:22 |
| ralonsoh | FT is failing a lot because ovsdb timeouts | 16:22 |
| *** ociuhandu has joined #openstack-meeting | 16:23 | |
| slaweq | yes, grafana looks pretty good but I also have impression that it's not working very well for some of the jobs | 16:23 |
| bcafarel | yup, same feeling here | 16:23 |
| slaweq | also my script shown me that last week we had 1 recheck in average to merge patch | 16:24 |
| slaweq | but I think it's because we didn't merge many patches and some of them were with e.g. docs only, so less jobs are run on them | 16:24 |
| ralonsoh | and, for example, the gate now is dropping all jobs | 16:25 |
| slaweq | ok, so lets maybe talk about some specific jobs and issues from last week | 16:25 |
| slaweq | ralonsoh: why? | 16:25 |
| slaweq | any specific reason? | 16:25 |
| slaweq | or just random failures? | 16:25 |
| ralonsoh | still reviewing it | 16:26 |
| slaweq | ok, thx | 16:26 |
| bcafarel | the recent pip issue? | 16:26 |
| slaweq | ahh, yes I saw some email about it today | 16:26 |
| ralonsoh | seems that the problem is fixed and released | 16:27 |
| ralonsoh | pip 20.0.1 | 16:27 |
| *** ociuhandu has quit IRC | 16:28 | |
| slaweq | yes, this should be better now | 16:28 |
| slaweq | ok, lets move on to the specific jobs now | 16:29 |
| slaweq | #topic fullstack/functional | 16:29 |
| *** openstack changes topic to "fullstack/functional (Meeting topic: neutron_ci)" | 16:29 | |
| slaweq | I have couple of issues for today | 16:29 |
| slaweq | first one is, already mentioned by ralonsoh ovsdbapp timeouts, like e.g.: | 16:29 |
| slaweq | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_559/703299/1/gate/neutron-functional/559c0ed/testr_results.html | 16:29 |
| slaweq | (fullstack) https://1a798a15650a97d81a82-17406e3478c64e603d8ff3ea0aac16c8.ssl.cf1.rackcdn.com/703366/1/check/neutron-fullstack/59e1877/testr_results.html | 16:29 |
| slaweq | https://0bad6b662fac8347dc41-be430d2f919a8698d2e96141ed3ac146.ssl.cf5.rackcdn.com/687922/15/gate/neutron-functional/b6769d6/testr_results.html | 16:29 |
| slaweq | I think there was old bug reported for that even | 16:30 |
| ralonsoh | #link https://bugs.launchpad.net/neutron/+bug/1802640 | 16:30 |
| openstack | Launchpad bug 1802640 in neutron "TimeoutException: Commands [<ovsdbapp.schema.open_vswitch.commands.SetFailModeCommand in q-agt gate failure" [Medium,Confirmed] | 16:30 |
| slaweq | yes, this one | 16:30 |
| slaweq | thx ralonsoh | 16:30 |
| ralonsoh | I would like to submit a patch to increase the ovsdbapp log | 16:30 |
| ralonsoh | and then, maybe, talk to Terry Wilson to do something there, in the command commit context | 16:31 |
| slaweq | if that may help to debug this issue, I'm all for this :) | 16:31 |
| *** macz has joined #openstack-meeting | 16:31 | |
| bcafarel | +1 if that gives a better image of what it is actually spending time on | 16:31 |
| bcafarel | I did not check the specific failure, but functional has been grumpier recently on stein/trein too - may be similar issue | 16:32 |
| bcafarel | (where rechecks are ping pong between functional and grenade timeouts) | 16:32 |
| slaweq | ok, so ralonsoh will increase log level for ovsdbapp in those jobs and we will see then | 16:33 |
| njohnston | +1 | 16:34 |
| slaweq | #action ralonsoh to increase log level for ovsdbapp in fullstack/functional jobs | 16:34 |
| slaweq | thx ralonsoh for taking care of it | 16:34 |
| ralonsoh | yw | 16:34 |
| slaweq | ok, next one which I saw this week was Issue with EVENT OVSNeutronAgentOSKenApp->ofctl_service GetDatapathRequest | 16:35 |
| *** eharney has quit IRC | 16:35 | |
| slaweq | like e.g. https://f26da45659020db1220c-76fc92e5e7c4e5a091c792a95503ad1d.ssl.cf5.rackcdn.com/701565/5/check/neutron-functional/4db8d3f/controller/logs/dsvm-functional-logs/neutron.tests.functional.agent.l2.extensions.test_ovs_agent_qos_extension.TestOVSAgentQosExtension.test_policy_rule_delete_egress_.txt | 16:35 |
| slaweq | and I remember that I already saw such issue in the past few times | 16:35 |
| *** ociuhandu has joined #openstack-meeting | 16:36 | |
| slaweq | and it failed on setup_physical_bridges because it couldn't get dp_id | 16:37 |
| slaweq | is there anyone who wants to check that? | 16:37 |
| ralonsoh | I can take a look at it | 16:38 |
| ralonsoh | is there a bug reported? | 16:38 |
| slaweq | ralonsoh: nope, but I will open one for it today | 16:38 |
| ralonsoh | perfect | 16:38 |
| slaweq | #action slaweq to open bug for issue with get_dp_id in os_ken | 16:38 |
| slaweq | and I will send it to You | 16:39 |
| *** gyee has joined #openstack-meeting | 16:39 | |
| slaweq | ok, lets move on to fullstack job than | 16:39 |
| slaweq | first issue there: | 16:39 |
| slaweq | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_315/601336/42/check/neutron-fullstack/3156265/testr_results.html | 16:39 |
| slaweq | I saw it few weeks ago also | 16:39 |
| slaweq | and according to http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22AssertionError%3A%20QoS%20register%20not%20found%20with%20queue-num%5C%22 it is happening from time to time recently | 16:40 |
| ralonsoh | yes, it's very "delicate" the min-qos feature | 16:40 |
| ralonsoh | if you have only one agent, ewverything is ok | 16:40 |
| ralonsoh | but when we are testing, we are introducing several qos/queue configurations | 16:41 |
| ralonsoh | the agent extension is not ready for this | 16:41 |
| ralonsoh | but we fake this in the tests | 16:41 |
| *** mattw4 has joined #openstack-meeting | 16:41 | |
| ralonsoh | but of course, we can have these kind of situations... | 16:41 |
| ralonsoh | but, since the last patch, is much more stable | 16:42 |
| ralonsoh | and, btw, this is still pending | 16:42 |
| ralonsoh | https://review.opendev.org/#/c/687922/ | 16:42 |
| ralonsoh | (re-re-re-rechecking) | 16:42 |
| slaweq | ahh, ok | 16:42 |
| slaweq | so this patch should helps with this issue, right? | 16:43 |
| ralonsoh | I hope so | 16:43 |
| slaweq | ok | 16:43 |
| ralonsoh | (patch you are "my last hope") | 16:43 |
| slaweq | LOL | 16:43 |
| slaweq | ok, next issue with fullstack is | 16:43 |
| slaweq | Failure on cleanup phase: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_564/702250/6/gate/neutron-fullstack/564244f/testr_results.html | 16:43 |
| slaweq | Do we really need cleanup in fullstack tests? | 16:43 |
| slaweq | I started thinking today that we maybe don't need cleanup phase in fullstack tests | 16:44 |
| slaweq | for each test we are spawning new neutron-server, agents and everything else | 16:44 |
| slaweq | and it's always only for this one test | 16:44 |
| slaweq | so what's the point to delete routers, ports, networks, etc. at the end? | 16:44 |
| slaweq | IMO it's just waste of time | 16:45 |
| slaweq | or am I missing something here? | 16:45 |
| njohnston | to make sure the delete functionality doesn't have bugs, yes? | 16:45 |
| ralonsoh | exactly | 16:45 |
| slaweq | njohnston: sure, but we are testing delete functionality in scenario and api tests also, no? | 16:45 |
| slaweq | fullstack is more for kind of whitebox testing where You can check if things are actually configured on host properly | 16:46 |
| *** bnemec has quit IRC | 16:48 | |
| njohnston | OK, you could make that argument. We could spin a change to remove those cleanups and make sure nothing bag happens. | 16:48 |
| slaweq | :) | 16:48 |
| njohnston | *bad | 16:48 |
| njohnston | SMH | 16:48 |
| slaweq | so I will send such patch to also see how much time we can safe on that job if we will remove cleanups from it | 16:48 |
| slaweq | if it's not too much, maybe we can drop this patch and stay with it as it is now | 16:49 |
| ralonsoh | my only concern are the L1 elements in the host (taps, devices, bridges, etc) | 16:49 |
| slaweq | ok for You? | 16:49 |
| slaweq | ralonsoh: sure, I'm not talking about things like that | 16:49 |
| ralonsoh | we can try it | 16:49 |
| ralonsoh | ok | 16:49 |
| slaweq | I just want to skip cleaning neutron resources | 16:49 |
| bcafarel | if you run fullstack in a loop and do no start seeing OOM errors it is probably good ;) | 16:50 |
| slaweq | bcafarel: good test indeed :) | 16:50 |
| slaweq | #action slaweq to try to skip cleaning up neutron resources in fullstack job | 16:51 |
| slaweq | ok, lets move on quickly to the next topic as we are running out of time | 16:51 |
| slaweq | #topic Tempest/Scenario | 16:51 |
| *** openstack changes topic to "Tempest/Scenario (Meeting topic: neutron_ci)" | 16:51 | |
| njohnston | just make sure that we also accomodate developers running fullstack on their personal laptops | 16:51 |
| slaweq | njohnston: sure, I will check if this will be fine in such case | 16:52 |
| slaweq | ok, speaking about tempest jobs I saw again this week issue with paramiko: | 16:52 |
| slaweq | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_ca7/653883/11/check/neutron-tempest-plugin-scenario-linuxbridge/ca7d140/testr_results.html | 16:53 |
| slaweq | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_820/611605/22/gate/neutron-tempest-plugin-scenario-linuxbridge/82023b2/testr_results.html | 16:53 |
| ralonsoh | #link https://review.opendev.org/#/c/702903/ | 16:53 |
| slaweq | basically there is error like "paramiko.ssh_exception.SSHException: No existing session" | 16:53 |
| ralonsoh | that's why I pushed this | 16:53 |
| ralonsoh | ahhh no, this is another error | 16:54 |
| ralonsoh | but that was solved last week | 16:54 |
| slaweq | I don't think it was solved last week | 16:54 |
| ralonsoh | https://review.opendev.org/#/c/701018/ | 16:55 |
| slaweq | there was another, similar issue but with error like "NoneType object has no attribute session" or something like that | 16:55 |
| ralonsoh | pfffff ok then | 16:55 |
| slaweq | there is bug reported for this new issue, I need to find it | 16:55 |
| slaweq | paramiko.ssh_exception.SSHException: No existing session | 16:56 |
| slaweq | sorry | 16:56 |
| slaweq | https://bugs.launchpad.net/neutron/+bug/1858642 | 16:56 |
| openstack | Launchpad bug 1858642 in neutron "paramiko.ssh_exception.NoValidConnectionsError error cause dvr scenario jobs failing" [High,Confirmed] | 16:56 |
| slaweq | if anyone whould have some time to look into it, that would be great | 16:56 |
| slaweq | and it's not only in dvr jobs | 16:56 |
| slaweq | and that's all from me for today | 16:57 |
| ralonsoh | I can take a look, but this problem seems to be in the zuul.test.base | 16:57 |
| slaweq | zuul.test.base? what is it? | 16:57 |
| ralonsoh | nothing, wrong search | 16:58 |
| slaweq | ahh, ok | 16:58 |
| slaweq | ok, lets move on quickly | 16:58 |
| slaweq | 2 more things | 16:58 |
| slaweq | 1. thx ralonsoh for fix mariadb periodic job - it's working fine now | 16:59 |
| bcafarel | thanks ralonsoh! | 16:59 |
| slaweq | 2. I would like to ask You about change time of this meeting | 16:59 |
| slaweq | what do You say about moving it to wednesday for 2pm utc, just after L3 meeting? | 16:59 |
| njohnston | that works fine for me | 17:00 |
| slaweq | as today at same time there is tc call in redhat every 3 weeks and I would like to attend it sometimes | 17:00 |
| ralonsoh | you mean 3pm UTC | 17:00 |
| slaweq | sorry, yes | 17:00 |
| slaweq | 3pm | 17:00 |
| slaweq | 2pm is L3 meeting | 17:00 |
| ralonsoh | ok for me | 17:00 |
| slaweq | thx | 17:00 |
| slaweq | I will propose patch and add all of You as reviewers than | 17:00 |
| slaweq | ok, that's all for today | 17:01 |
| bcafarel | and mail maybe also for interested folks | 17:01 |
| slaweq | thx for attending | 17:01 |
| ralonsoh | bye | 17:01 |
| slaweq | #endmeeting | 17:01 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 17:01 | |
| openstack | Meeting ended Tue Jan 21 17:01:12 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 17:01 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-01-21-16.00.html | 17:01 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-01-21-16.00.txt | 17:01 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/neutron_ci/2020/neutron_ci.2020-01-21-16.00.log.html | 17:01 |
| slaweq | bcafarel: sure, I will send email also | 17:01 |
| njohnston | o/ | 17:01 |
| *** tesseract has quit IRC | 17:01 | |
| *** rpittau is now known as rpittau|afk | 17:04 | |
| *** ricolin has quit IRC | 17:04 | |
| *** ricolin has joined #openstack-meeting | 17:05 | |
| *** raildo_ has quit IRC | 17:12 | |
| *** rfolco is now known as rfolco|brb | 17:16 | |
| *** e0ne has quit IRC | 17:24 | |
| *** eharney has joined #openstack-meeting | 17:46 | |
| *** jamesmcarthur has quit IRC | 18:00 | |
| *** ayoung has joined #openstack-meeting | 18:02 | |
| *** macz has quit IRC | 18:03 | |
| *** ociuhandu_ has joined #openstack-meeting | 18:13 | |
| *** Lucas_Gray has quit IRC | 18:15 | |
| *** ociuhandu has quit IRC | 18:17 | |
| *** ociuhandu_ has quit IRC | 18:18 | |
| *** slaweq_ has joined #openstack-meeting | 18:26 | |
| *** slaweq has quit IRC | 18:27 | |
| *** ralonsoh has quit IRC | 18:31 | |
| *** electrofelix has quit IRC | 18:34 | |
| *** enriquetaso has quit IRC | 18:36 | |
| clarkb | we'll start the infra meeting shortly | 18:59 |
| fungi | excellent | 19:00 |
| corvus | o/ | 19:00 |
| clarkb | #startmeeting infra | 19:01 |
| openstack | Meeting started Tue Jan 21 19:01:12 2020 UTC and is due to finish in 60 minutes. The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot. | 19:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 19:01 |
| *** openstack changes topic to " (Meeting topic: infra)" | 19:01 | |
| openstack | The meeting name has been set to 'infra' | 19:01 |
| clarkb | #link http://lists.openstack.org/pipermail/openstack-infra/2020-January/006577.html Our Agenda | 19:01 |
| zbr | o/ | 19:01 |
| clarkb | #topic Announcements | 19:01 |
| *** openstack changes topic to "Announcements (Meeting topic: infra)" | 19:01 | |
| ianw | o/ | 19:01 |
| clarkb | I did not have any announcements | 19:01 |
| clarkb | #topic Actions from last meeting | 19:02 |
| *** openstack changes topic to "Actions from last meeting (Meeting topic: infra)" | 19:02 | |
| clarkb | #link http://eavesdrop.openstack.org/meetings/infra/2020/infra.2020-01-14-19.01.txt minutes from last meeting | 19:02 |
| clarkb | I had an action item to start making the opendev governance changes official. This has been done, but lets talk about it during our OpenDev topic | 19:03 |
| clarkb | #topic Priority Efforts | 19:03 |
| *** openstack changes topic to "Priority Efforts (Meeting topic: infra)" | 19:03 | |
| clarkb | #topic OpenDev | 19:03 |
| *** openstack changes topic to "OpenDev (Meeting topic: infra)" | 19:03 | |
| clarkb | we can jump into that immediatly :) | 19:03 |
| clarkb | There are two related changes to make the OpenDev governance changes official. The first removes OpenDev from OpenStack Governance. The second creates a new self bootstrapped governance system for OpenDev | 19:04 |
| clarkb | #link https://review.opendev.org/#/c/703134/ Split OpenDev out of OpenStack Governance | 19:04 |
| clarkb | #link https://review.opendev.org/#/c/703488/ Update OpenDev docs with new Governance | 19:04 |
| clarkb | Both have received a fair amount of review, thank you allfor that. If you haven't yet had a chance to review them your feedback is very welcome | 19:04 |
| clarkb | Are there any questions or concerns at this point that I (or others) can help with? | 19:05 |
| fungi | seems like a fairly straightforward evolution for me | 19:06 |
| clarkb | One thing to point out is that I think the first change needs to merge before the second | 19:06 |
| clarkb | otherwise there is a governance conflict | 19:06 |
| fungi | i guess the main question i have is whether there will also be an openstack-infra governance defined after the opendev governance gets published in the place that used to live | 19:06 |
| clarkb | fungi: ajaeger brought that up too, and I don't htink it needs anything beyond the default openstack system | 19:06 |
| fungi | wfm | 19:07 |
| fungi | also we could consider openstack-infra becoming a sig, but one step at a time | 19:07 |
| clarkb | fungi: the repos/tools/systems that remain in openstack are fairly self contained and won't require subteam organization I don't think | 19:07 |
| corvus | yeah, if it end up needing more formal organization, maybe a sig | 19:08 |
| fungi | well, a sig would be *less* formal organization really | 19:09 |
| corvus | i mean more formal than nothing :) | 19:09 |
| fungi | (less organization than an openstack project team anyway) | 19:09 |
| fungi | oh, sure ;) | 19:09 |
| corvus | oh, yes. | 19:09 |
| corvus | on the question of sig vs subteam, sig sounds right, due to the value of cross-subteam collaboration. | 19:10 |
| corvus | (and relative dearth of release deliverables) | 19:10 |
| *** Shrews has joined #openstack-meeting | 19:11 | |
| clarkb | On the technical side of things jrosser noticed that some jobs running between 1400 and 1500 UTC today had git clone errors. I've suggested that the job instead rely on zuul managed repos (as they come from caches and allow for cross repo testing). I also discovered that about half the total connections to gitea04 (which was the unhappy node at the time) came from huawei IPs | 19:12 |
| clarkb | those IPs were all in the same /22 subnet | 19:13 |
| clarkb | it looks like a CI system in that way | 19:13 |
| *** dustinc|PTO is now known as dustinc | 19:13 | |
| clarkb | I think we have ~3 options. A) bigger gitea nodes with more memory to avoid swapping and OOMs B) work with huawei to not DDoS us 3) try least connections load balance method again (this is unlikely to be stable) | 19:13 |
| fungi | we also had a massive load spike on lists.o.o which looked like they could be a pipermail python process pileup | 19:14 |
| fungi | but i didn't find a smoking gun | 19:14 |
| clarkb | at this point I think its motsly a monitor situation. If it continues and or gets worse then we will need to sort out a plan | 19:14 |
| clarkb | but maybe this is a one off and we'll be ok | 19:14 |
| corvus | clarkb: ++ let's get 2 or 3 data points | 19:14 |
| fungi | and... same recommendation on the lists.o.o front for now | 19:15 |
| clarkb | Anything else opendev related? | 19:15 |
| fungi | nothing springs to mind | 19:16 |
| clarkb | #topic Update Config Management | 19:16 |
| *** openstack changes topic to "Update Config Management (Meeting topic: infra)" | 19:16 | |
| clarkb | review-dev is running without puppet at this point \o/ | 19:16 |
| clarkb | we did discover a couple gaps that need to be addressed though. The first is in disabling the old init script for gerrit once converted to podman. The other is configuring a webserver to front gerrit | 19:17 |
| clarkb | I think for the first one we ma yjust make that part of our migration document but the second will need updated to the ansible/container tooling | 19:17 |
| fungi | yeah, extra ansible to delete an initscript seems overkill | 19:18 |
| clarkb | I don't think the webserver change(s) have been written yet and I don't think we have mordred here today | 19:18 |
| fungi | or possibly he's just waking up | 19:18 |
| clarkb | I guess keep an eye out for those changes as they are the next step in getting gerrit switched over | 19:19 |
| clarkb | are there any other config management related changes to call out? | 19:20 |
| *** jamesmcarthur has joined #openstack-meeting | 19:20 | |
| clarkb | Sounds like no. Let's move on | 19:21 |
| clarkb | #topic General Topics | 19:21 |
| *** openstack changes topic to "General Topics (Meeting topic: infra)" | 19:21 | |
| clarkb | ianw: do you want to start with static.opendev.org or arm64 cloud? | 19:21 |
| ianw | i haven't made too much progress on static.o.o | 19:22 |
| clarkb | looks like the changes have landed at least | 19:22 |
| ianw | yep, i can loop back on that and i think starting a new server is probably the next step | 19:23 |
| clarkb | cool | 19:23 |
| ianw | then governance and some others should be publishing to afs and we can move them to be served from there | 19:23 |
| clarkb | related to this topic, we threw out last week that maybe having a day or two to focus on getting rid of the last trusty servers would be worthwhile | 19:23 |
| clarkb | suggestion is to use the 27th and 28th for that (28th and 29th australia days) | 19:24 |
| clarkb | I'm still on board for that and will plan to focus on this work on those days | 19:24 |
| clarkb | everyone else is more than welcome to join in :) | 19:24 |
| clarkb | ianw: we must be getting close to needing to set up dns records as well | 19:25 |
| clarkb | ianw: does that need to happen before the server starts (so that LE can configure things properly) | 19:25 |
| ianw | clarkb: umm, yes probably, i will loop back on that | 19:26 |
| fungi | i can try to set aside those two days to make progress on wiki.o.o configuration management in hopes we can migrate to a replaceable xenial server with actual puppet-applied configuration | 19:26 |
| fungi | (i think wiki-dev is close-ish) | 19:26 |
| clarkb | sounds good, thanks | 19:26 |
| clarkb | Next up is a new arm64 cloud | 19:27 |
| clarkb | #link https://review.opendev.org/#/q/topic:linaro-us+status:open Changes to deploy new arm64 cloud | 19:27 |
| clarkb | ianw: I left a minor -1 on the clouds.yaml changes | 19:27 |
| corvus | ooh neat -- we'll have 2 now? | 19:27 |
| clarkb | corvus: yup. This one looks to be quite a bit larger than the existing one as well | 19:28 |
| ianw | clarkb: yep, will do comments on that small stack asap | 19:28 |
| clarkb | 44 test instances (if using 8vcpu per instance) | 19:28 |
| ianw | if we can get that in, i can look at bringing up a mirror, and then ... it should work? :) | 19:28 |
| clarkb | ianw: ya I think we are less concerned with performance given it is arm and simply different | 19:29 |
| clarkb | people running jobs thee will have to accept it may be slower | 19:29 |
| ianw | kevinz did mention it was ipv6 only which is new territory for arm64 cloud so may be wrinkles there | 19:29 |
| clarkb | that is a good point. | 19:30 |
| fungi | well, in #openstack-infra he just said there were a lot fewer ipv4 addresses than quota for test nodes, but yes in practice that means ipv6 global with v4 egress via overload nat | 19:30 |
| fungi | (port address translation) | 19:31 |
| clarkb | and dedicated ipv4 for the mirror | 19:31 |
| frickler | this likely also means that we should setup some special routing for the v4 mirror like FN has | 19:32 |
| frickler | iirc | 19:32 |
| clarkb | frickler: thats a good point ,we should double check the traceroute for test nodes to the mirror | 19:32 |
| clarkb | in case the ipv6 setup is similar and bottlenecks through the router | 19:32 |
| fungi | yeah, or rather we end up overrunning the nat's port mapping pool with local connections to the mirror server | 19:33 |
| fungi | (remembering that the tcp/udp port range is a mere 65k, so that's not a lot of parallel connections in reality) | 19:34 |
| clarkb | Anything else on the topic of a new arm64 cloud? | 19:35 |
| ianw | we did have a mutli-arch meeting | 19:36 |
| ianw | #link https://storyboard.openstack.org/#!/project/openstack/multi-arch-sig | 19:36 |
| ianw | plan is to put interesting things to track @ ^ | 19:36 |
| fungi | awesome! | 19:38 |
| fungi | i hope that goes somewhere | 19:38 |
| clarkb | I guess that sig will be taking advantage of the new set of resources to do more testing of openstack on more than just x86? | 19:38 |
| fungi | it's apparently in tonyb's interest to have better support for powerpc now | 19:39 |
| fungi | so maybe even more than just x86 and aarch64 | 19:39 |
| ianw | :) yep, i think we start with a build-it-and-they-will-come approach | 19:40 |
| fungi | indeed | 19:40 |
| fungi | it's worked so far | 19:40 |
| clarkb | Alright anything else? | 19:41 |
| ianw | not from me | 19:42 |
| clarkb | #topic Open Discussion | 19:44 |
| *** openstack changes topic to "Open Discussion (Meeting topic: infra)" | 19:44 | |
| clarkb | I'll give it a few minutes for other topics | 19:44 |
| *** jamesmcarthur has quit IRC | 19:46 | |
| clarkb | not hearing any other business. I'll go ahead and call the meeting | 19:47 |
| clarkb | Thank you everyone! | 19:47 |
| clarkb | #endmeeting | 19:47 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 19:47 | |
| openstack | Meeting ended Tue Jan 21 19:47:31 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 19:47 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/infra/2020/infra.2020-01-21-19.01.html | 19:47 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/infra/2020/infra.2020-01-21-19.01.txt | 19:47 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/infra/2020/infra.2020-01-21-19.01.log.html | 19:47 |
| corvus | thanks clarkb! | 19:47 |
| *** jamesmcarthur has joined #openstack-meeting | 19:50 | |
| *** nicolasbock has quit IRC | 19:57 | |
| *** jamesmcarthur has quit IRC | 20:01 | |
| *** jamesmcarthur has joined #openstack-meeting | 20:01 | |
| *** jamesmcarthur has quit IRC | 20:08 | |
| *** yamamoto has joined #openstack-meeting | 20:11 | |
| *** jamesmcarthur has joined #openstack-meeting | 20:14 | |
| *** yamamoto has quit IRC | 20:16 | |
| *** Lucas_Gray has joined #openstack-meeting | 20:26 | |
| *** eharney has quit IRC | 20:27 | |
| *** ykatabam has joined #openstack-meeting | 20:41 | |
| *** rcernin has joined #openstack-meeting | 20:42 | |
| *** rbudden has joined #openstack-meeting | 20:50 | |
| *** oneswig has joined #openstack-meeting | 20:58 | |
| *** whoami-rajat_ has quit IRC | 20:58 | |
| *** Shrews has left #openstack-meeting | 20:58 | |
| *** martial has joined #openstack-meeting | 20:58 | |
| *** jamesmcarthur has quit IRC | 20:58 | |
| *** trandles has joined #openstack-meeting | 20:59 | |
| *** jmlowe has joined #openstack-meeting | 21:00 | |
| oneswig | #startmeeting scientific-sig | 21:00 |
| openstack | Meeting started Tue Jan 21 21:00:18 2020 UTC and is due to finish in 60 minutes. The chair is oneswig. Information about MeetBot at http://wiki.debian.org/MeetBot. | 21:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 21:00 |
| *** openstack changes topic to " (Meeting topic: scientific-sig)" | 21:00 | |
| openstack | The meeting name has been set to 'scientific_sig' | 21:00 |
| oneswig | up up and away! | 21:00 |
| rbudden | hello | 21:00 |
| oneswig | #topic who's here | 21:00 |
| *** openstack changes topic to "who's here (Meeting topic: scientific-sig)" | 21:00 | |
| trandles | Hi all | 21:00 |
| oneswig | hi all | 21:00 |
| jmlowe | hey! | 21:00 |
| martial | that's a topic! | 21:01 |
| oneswig | The new informal SIG. I'm not even wearing a tie tonight :-) | 21:01 |
| oneswig | OK, I forgot to post an agenda on the wiki, apologies. | 21:02 |
| oneswig | #chair martial | 21:02 |
| openstack | Current chairs: martial oneswig | 21:02 |
| oneswig | #topic Conferences and CFPs for 2020 | 21:03 |
| *** openstack changes topic to "Conferences and CFPs for 2020 (Meeting topic: scientific-sig)" | 21:03 | |
| oneswig | So what's coming up? | 21:03 |
| oneswig | If you're in London in early March, this free 1-day conference at the Francis Crick Institute is excellent: https://cloud.ac.uk/ | 21:04 |
| martial | GTC in March | 21:04 |
| martial | OpenStack Summit in June (CFP coming out soon I was told) | 21:05 |
| martial | PEARC ? | 21:05 |
| oneswig | martial: where and when for GTC? You going? | 21:05 |
| martial | GTC, LA, March | 21:05 |
| martial | planning to | 21:05 |
| jmlowe | ISC, SuperCompCloud, working on the CFP and program committee (nominations taken) https://sites.google.com/view/supercompcloud | 21:06 |
| oneswig | Once you're done with GTC, have some downtime lakeside in Ticino, Switzerland: https://www.hpcadvisorycouncil.com/events/2020/swiss-workshop/ | 21:06 |
| jmlowe | PEARC in Portland at the end of July | 21:06 |
| martial | SC20 in Atlanta ... November? | 21:07 |
| jmlowe | yep | 21:07 |
| oneswig | jmlowe: you planning a presentation to submit for PEARC this year? | 21:07 |
| jmlowe | Possibly, definitely if JS2 gets funded | 21:07 |
| oneswig | I haven't checked yet when the deadline is for OpenStack Vancouver - anyone know? | 21:10 |
| trandles | CFP hasn't even opened has it? | 21:11 |
| martial | don't think it is out yet | 21:11 |
| martial | Ildiko mentioned it should be soon | 21:11 |
| trandles | There's nothing but dates and location on the normal summit page. | 21:12 |
| oneswig | It's pretty minimal right now - https://www.openstack.org/summit/vancouver-2020/ | 21:12 |
| trandles | Oh, looks like an eventbrite registration page is open now. Totally new format by the look of it? | 21:13 |
| trandles | https://www.eventbrite.com/e/opendev-ptg-vancouver-2020-tickets-88923270897?aff=opendevptg2020&_ga=2.64326319.100008772.1579641137-574136464.1557423154 | 21:13 |
| oneswig | It actually looks great - quite a grassroots feel to the way it is presented. | 21:15 |
| martial | looks very different to what we are used to indeed | 21:16 |
| trandles | Yeah, much less like a conference and more like a big integrated PTG? | 21:16 |
| martial | sure does | 21:17 |
| oneswig | This plenary opener - "Short kickoff with all attendees to set the goals for the day or discuss the outcomes of the previous day." - sounds a lot more "intimate", for sure. | 21:17 |
| rbudden | indeed, it looks like the format has drastically changed | 21:18 |
| martial | I think there will still be an upstream academy | 21:18 |
| trandles | Wonder how the various SIGs fit into the structure? Will they still support SIG meetings as part of the program? | 21:19 |
| trandles | Or maybe we'll have to self-organize | 21:19 |
| oneswig | trandles: hopefully in a different way than in Shanghai, where it didn't work out at all | 21:19 |
| martial | not sure yet, once the call for participation is open we can ask | 21:20 |
| martial | yes that sound like it was less what was expected at the time | 21:20 |
| trandles | "PTG: Afternoon working sessions for project teams and SIGs to continue the morning’s discussions." | 21:20 |
| martial | wait it only one day? | 21:20 |
| trandles | 4 days | 21:21 |
| martial | okay 8-11 | 21:21 |
| martial | makes more sense | 21:21 |
| jmlowe | They were saying it would be different, I wasn't sure I believed them until now | 21:21 |
| trandles | It's 4 days and they call out 4 focus areas, I wonder if each day will be dedicated to each area | 21:22 |
| oneswig | I like it. I think we can get stuff done here. | 21:22 |
| trandles | gah - "if each day will be dedicated to one area" | 21:22 |
| *** armax has joined #openstack-meeting | 21:22 | |
| trandles | It's certainly a lot cheaper than past summits! | 21:22 |
| martial | well I look forward to learning new things | 21:23 |
| martial | although the question remains about "is it a long PTG" | 21:23 |
| martial | anything else? | 21:24 |
| oneswig | definitely scope for an evening social! | 21:24 |
| oneswig | OK, move on? | 21:24 |
| martial | go | 21:24 |
| jmlowe | I appreciate social events closer to sea level | 21:25 |
| martial | :) | 21:25 |
| oneswig | #topic large-scale SIG data | 21:25 |
| *** openstack changes topic to "large-scale SIG data (Meeting topic: scientific-sig)" | 21:25 | |
| oneswig | I've been taking part in the large-scale SIG meetings | 21:25 |
| martial | expand on the topic please? | 21:26 |
| oneswig | They are looking for operator data on pain points | 21:26 |
| martial | oh right, that is a lot of our SIG users :) | 21:26 |
| oneswig | #link mailing list post http://lists.openstack.org/pipermail/openstack-discuss/2020-January/011820.html | 21:26 |
| oneswig | #link etherpad for information on scaling https://etherpad.openstack.org/p/large-scale-sig-documentation | 21:27 |
| oneswig | Could actually become a handy resource that one. | 21:27 |
| jmlowe | tldr; did they define large-scale? | 21:27 |
| *** diablo_rojo has quit IRC | 21:27 | |
| *** diablo_rojo has joined #openstack-meeting | 21:28 | |
| oneswig | Not precisely. But approaching 1000 hypervisors is definitely in it. 500+ will be showing a lot of the early signs of scaling trouble | 21:28 |
| *** eharney has joined #openstack-meeting | 21:28 | |
| oneswig | Our contribution so far has been around "golden signals" for scaling problems - rates, latencies, errors, and all that. | 21:30 |
| oneswig | to which end, the most useful thing I've seen to date is telemetry from HAProxy | 21:30 |
| trandles | HAProxy at that scale? The OSA documents explicitly suggest you don't want to use haproxy in production at all. | 21:31 |
| oneswig | It will give response latencies by endpoint for example. Very cool. And a graph of 500 response codes. Highly useful. | 21:32 |
| oneswig | trandles: that's surprising. We build upon it as standard. Multiple in a deploy, sometimes. | 21:32 |
| oneswig | As someone who appreciates a tight piece of C code, what's the objection, and indeed the alternative? | 21:33 |
| jmlowe | um what!? no HAProxy, everything I have would fall over without it | 21:33 |
| trandles | Looking for the reference now, but IIRC, it says something like "HAProxy is appropriate for testing but you should use a real hardware director in production" | 21:33 |
| martial | what are the alternatives? | 21:34 |
| jmlowe | sure, I've got a spare couple of million for a enterprise load balancer | 21:34 |
| oneswig | trandles: did you heed this guidance? :-) | 21:35 |
| *** jamesmcarthur has joined #openstack-meeting | 21:35 | |
| jmlowe | I certainly didn't | 21:35 |
| trandles | oneswig, I can't even get a deployment to finish. I spend most of my time googling to figure out what configuration variable I need to set that is left out of the OSA guide. | 21:36 |
| oneswig | I don't think I've seen our OpenStack APIs generate enough traffic or connections to be a concern for it. Famous last words perhaps. | 21:36 |
| trandles | Maybe Vancouver should have one topic: decent documentation | 21:36 |
| oneswig | trandles: feel your pain. My own notes are the same. I'm like, "thanks, former self"... | 21:36 |
| trandles | oneswig, I have started a new notebook of nothing but "gotchas and solutions" to deploying OpenStack | 21:37 |
| oneswig | Thats awesome. Next time you do it, it'll sail through! | 21:37 |
| rbudden | +1 on docs. I feel like the HA guides are lacking, and there’s an overall sense in the community that if you are doing production it’s assumed you are doing OSA or something similar which doesn’t really fit all deployment models | 21:39 |
| martial | I feel that there is a book in the making here | 21:39 |
| trandles | # load balancer | 21:39 |
| trandles | # Ideally the load balancer should not use the Infrastructure hosts. | 21:39 |
| trandles | # Dedicated hardware is best for improved performance and security. | 21:39 |
| jmlowe | you're doing something seriously wrong if the openstack api's knock over haproxy for anything other than storing glance image | 21:39 |
| trandles | In the OSA config file | 21:40 |
| oneswig | Anyway, for haproxy users, I highly recommend the telemetry. There's a prometheus endpoint, which we scrape directly or poll and store in Monasca. | 21:40 |
| trandles | But that's not exactly the comment I'm remembering | 21:40 |
| oneswig | trandles: I might read that comment as "dedicated hardware for running haproxy" | 21:40 |
| trandles | oneswig, I agree | 21:40 |
| rbudden | indeed. running haproxy on control plane nodes isn’t really a great idea | 21:41 |
| rbudden | @trandles i’d be curious to chat sometime about your experiences with OSA | 21:41 |
| martial | +1 on the dedicated hardware ... but that was a long time ago when we did our first deployment | 21:41 |
| trandles | oneswig, you said prometheus...did you replace gnocchi with it? | 21:42 |
| trandles | rbudden, any time | 21:42 |
| jmlowe | just from a security stand point, you greatly reduce your attack surface by only letting haproxy touch the outside and keeping your controllers inside | 21:43 |
| oneswig | I saw it in two configs today - one with Prometheus (and for metric storage), the other with Monasca scraping the same endpoint (ending up in influx). | 21:43 |
| rbudden | jmlowe: +1 this is rather important for our setups | 21:44 |
| trandles | rbudden, the reason I'm asking about prometheus/gnocchi is because gnocchi wouldn't install via OSA without serious fiddling and someone in OSA project pointed me to a third-person's ansible for prometheus installation | 21:44 |
| oneswig | Interesting aside, the Red Hat monitoring framework SAF apparently standardises on collectd instead. | 21:44 |
| rbudden | the future of Gnocchni and/or it’s replacement is concerning | 21:45 |
| rbudden | being knee deep in a large scale Train deploy at the moment i’m curious if/when there will be a concensus in the community as to the direction of Telemetry | 21:46 |
| trandles | rbudden, I'm deploying train via OSA, we definitely need to have a call very soon | 21:46 |
| rbudden | sounds like a plan | 21:47 |
| oneswig | prometheus is getting wide adoption but the back-end storage has limitations, such as control over retention period. We've done a lot with influx (but never paid for the enterprise HA version) | 21:47 |
| rbudden | we currently have a custom hybrid Puppet/Ansible model with the intent on moving it entirely to Ansible as time allows | 21:47 |
| oneswig | Does anyone have good pointers for telemetry that they monitor? | 21:48 |
| martial | for us it is heavy Prometheus and heavy Ansible (but we do only have six racks) | 21:49 |
| rbudden | my understanding was Promethus with Ceilometer was not appropriate for billing data | 21:50 |
| trandles | 10 minute warning btw - in case we're in the weeds ;) | 21:50 |
| oneswig | As a peddlar of scalar telemetry, Prometheus doesn't get notifications, which can be useful for fine-grained timing. | 21:51 |
| martial | I think the next topic is AOB ... so "in the weeds" works :) | 21:51 |
| oneswig | #topic AOB | 21:51 |
| *** openstack changes topic to "AOB (Meeting topic: scientific-sig)" | 21:51 | |
| oneswig | seamless transition | 21:51 |
| rbudden | :) | 21:52 |
| oneswig | Wanted to know if anyone has been using WekaIO? | 21:52 |
| martial | I want my emojis to high give Stig :) | 21:52 |
| martial | -g+f | 21:52 |
| trandles | oneswig, took me a few months to get WekaIO to stop cold calling me. :P | 21:52 |
| oneswig | trandles: surely after the first call they knew who you were? :-) | 21:53 |
| trandles | Artful Dodger kept answering my phone | 21:53 |
| oneswig | I've been working on getting it well integrated with OpenStack clients, in a mutually beneficial way. | 21:54 |
| oneswig | With help we got a client mounting Weka in "fast mode", ie SR-IOV and DPDK | 21:55 |
| oneswig | Don't have benchmark data yet, got held up building images with very specific versions (CentOS 7.6, MLNX-OFED 4.6). It can be hard to go back a few versions on cloud software :-) | 21:57 |
| trandles | I did see this today: https://www.hpcwire.com/off-the-wire/openstack-software-adds-native-upstream-support-for-hdr-200-gigabit-infiniband/ | 21:58 |
| martial | oneswig: seems like a pain to be honest | 21:58 |
| oneswig | Ah, saw that too. What's the big deal, surely one version of IB is like another? | 21:58 |
| martial | trandles: that is cool | 21:58 |
| trandles | oneswig, no idea | 21:59 |
| oneswig | janders would probably know, but I think he's been doing this for ages anyway! | 22:00 |
| trandles | I don't know enough about HDR 200 to know if it has to do with virtual functions, verbs-type stuff...?? | 22:00 |
| oneswig | Don't know, will keep an eye out. | 22:00 |
| oneswig | We are on the hour, time to close | 22:00 |
| oneswig | thanks all | 22:00 |
| oneswig | #endmeeting | 22:01 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 22:01 | |
| openstack | Meeting ended Tue Jan 21 22:01:02 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 22:01 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/scientific_sig/2020/scientific_sig.2020-01-21-21.00.html | 22:01 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/scientific_sig/2020/scientific_sig.2020-01-21-21.00.txt | 22:01 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/scientific_sig/2020/scientific_sig.2020-01-21-21.00.log.html | 22:01 |
| rbudden | catch ya’ll later | 22:01 |
| martial | talk soon :) | 22:02 |
| trandles | By all | 22:02 |
| trandles | *bye even | 22:02 |
| *** oneswig has quit IRC | 22:02 | |
| *** mattw4 has quit IRC | 22:06 | |
| *** eharney has quit IRC | 22:06 | |
| *** mattw4 has joined #openstack-meeting | 22:06 | |
| *** jmlowe has quit IRC | 22:09 | |
| *** mattw4 has quit IRC | 22:14 | |
| *** mattw4 has joined #openstack-meeting | 22:15 | |
| *** slaweq_ has quit IRC | 22:16 | |
| *** trandles has left #openstack-meeting | 22:16 | |
| *** enriquetaso has joined #openstack-meeting | 22:30 | |
| *** masahito has joined #openstack-meeting | 22:42 | |
| *** jamesmcarthur has quit IRC | 22:42 | |
| *** hyunsikyang has joined #openstack-meeting | 22:50 | |
| *** hyunsikyang__ has quit IRC | 22:54 | |
| *** e0ne has joined #openstack-meeting | 23:00 | |
| *** Lucas_Gray has quit IRC | 23:08 | |
| *** masahito has quit IRC | 23:09 | |
| *** Lucas_Gray has joined #openstack-meeting | 23:09 | |
| *** rbudden has quit IRC | 23:11 | |
| *** Liang__ has joined #openstack-meeting | 23:20 | |
| *** armax has quit IRC | 23:41 | |
| *** rfolco|brb is now known as rfolco | 23:41 | |
| *** e0ne has quit IRC | 23:52 | |
| *** dviroel has quit IRC | 23:57 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!